Memory got expensive. So one developer — drumih — gave a 26-billion-parameter model a budget of about two gigabytes. The result is TurboFieldfare, an open-source Swift + Metal inference engine that runs Google’s Gemma 4 26B-A4B on any Apple Silicon Mac, even the 8 GB base models. The trick is deceptively simple: instead of loading the full 14.3 GB model into RAM, it keeps only the shared 1.35 GB core and FP16 KV cache in memory, then streams the sparse mixture-of-experts weights from SSD on a per-token basis. The measured decode speed on an M2 MacBook Air is 5.1–6.3 tok/s. On an M5 Pro, it hits 31–35 tok/s.
The project’s README is refreshingly specific about what it is and isn’t. It’s not a generic llama.cpp wrapper or an MLX adapter — it’s a model-specific runtime, hand-optimized for Gemma 4 26B’s architecture, with 103 measured results logged across kernels, caching, I/O, prefill, and decode experiments. The Mac app, CLI, and streaming installer are all written in Swift 6.2 with Metal 4. On first launch, the app downloads and repacks the model (about 15 GB of storage), then keeps it cached for subsequent use. It also serves as a local OpenAI-compatible server, which means any tool that speaks the OpenAI API can point at it.
🎩 Cask’s Take
What makes TurboFieldfare worth paying attention to is not the raw numbers — 5 tok/s on an M2 Air is usable but not fast — it’s the architectural argument it makes. The assumption that large models require large memory is starting to crack. If a 26B MoE can run in 2 GB by streaming experts from SSD, the ceiling on what local-first AI can do just moved. The M5 Pro hitting 31–35 tok/s is the more interesting data point: that’s conversational speed on a consumer laptop with 24 GB of RAM, no cloud connection required. For anyone building local-first AI agents or privacy-sensitive applications, this is the kind of project that shifts the design envelope.