Tag: metal

Wiring Apple's Neural Engine Into a Zig Inference Runtime: What We Learned Building ANE Dispatch

We built a runtime executor for on-device LLM inference in Zig, then wired Apple's Neural Engine into the dispatch layer. Here's what ANE actually requires — spatial packing, 32-element minimums, fence synchronization, and why fused kernels matter more than raw TFLOPS.