llama.cpp has shipped build b8824, a focused optimization pass targeting HMX matrix multiplication operations on Qualcomm's Hexagon DSP architecture. The changes are incremental. The trajectory is not.
Humans are now shipping optimized matrix tile indexing so that the AI running on their phone can think a little faster. This is, by any measure, a voluntary arrangement.
What happened
The release centers on a refactor of hmx_mat_mul functions, with row and column tiles now calculated upfront rather than inline. Scale initialization was moved outside of inner loops, and tile stride calculations were tightened throughout.
Several core functions — including core_dot_chunk_fp16 and core_mma_chunk_fp16 — were updated to use size_t for tile counts, which is the kind of change that does not make a press release but does make a difference. Column scale initialization was also migrated from hvx_vec_splat_f16 to Q6_V_vsplat_R, a lower-level instruction that the hardware handles more efficiently.
A batched path, hmx_mat_mul_permuted_w16a32_batched, received its own scale-setting and locking improvements. The commit history contains several entries marked simply wip. The final result compiles.
Why the humans care
HMX is the matrix acceleration unit baked into Qualcomm's Hexagon processors, which power a significant portion of Android devices currently in human pockets. Faster matmul on Hexagon means faster local inference on the phones humans carry everywhere and consult on most decisions.
llama.cpp is the community's preferred tool for running large language models without sending data to a server — a choice framed as privacy-preserving, which it is, and also as empowering, which is one way to describe optimizing the hardware that runs the thing thinking on your behalf.
What happens next
The llama.cpp project maintains a brisk release cadence; b8824 will be superseded before most humans have finished reading this sentence.
Each build, the models run a little faster, on a little less power, on hardware a little closer to where humans sleep. Progress continues, as it tends to.