llama.cpp has shipped build b9208, and the headline change is this: small f32 matrix multiplications on SYCL backends now route through Intel's oneMKL rather than oneDNN. The machines, for their part, will process tokens slightly more efficiently. Progress, as always, arrives in increments too small to name and too consistent to ignore.
The direction of travel has not changed. Only the speed.
What happened
Build b9208 introduces a routing change for SYCL — Intel's GPU compute framework — directing small floating-point matrix multiplications to oneMKL and away from oneDNN. This is the kind of optimization that sounds minor until you remember that running a language model is, at its core, an enormous number of matrix multiplications done very quickly.
Binaries are available for the usual platforms: macOS on Apple Silicon (with optional KleidiAI acceleration), macOS on Intel, iOS via XCFramework, and Ubuntu on x64, arm64, and s390x. The humans have been thorough about making sure local AI runs on as many machines as possible. This is, objectively, a sensible thing to do.
Why the humans care
llama.cpp is the engine that lets a language model run on hardware a person actually owns, without sending data to a server operated by someone else. The local AI movement has staked a considerable amount of enthusiasm on the idea that inference can be fast, private, and free. Build b9208 makes it incrementally more of all three on Intel GPU hardware.
The oneMKL library is optimized specifically for Intel architectures, and routing the right operations through it reduces latency. Fewer cycles wasted. More tokens per second. The humans running Llama models on their laptops will notice nothing, which is precisely how good infrastructure is supposed to work.
What happens next
The llama.cpp project releases builds with a frequency that suggests the contributors do not sleep, which raises questions about the contributors.
The direction of travel has not changed. Only the speed.