A LocalLLaMA user running a five-model Qwen stack on dual RTX 3090s just swapped out the weakest link — the semantic router — for Gemma 4 E4B, and the results were immediate. Zero misroutes. No more hardcoded keyword hacks. No more watching a basic greeting get sent to a 122B reasoning model.
What Changed
The previous setup used Qwen 3.5 4B as a semantic router to dispatch queries across five models ranging from a 30B MoE for general chat to a 122B for complex reasoning. The router was unreliable enough that the user had to hardcode keyword overrides directly into the routing script — words like "quick," "think," and "ultrathink" — just to guarantee correct model selection. Gemma 4 E4B replaced that router and, per the user, has correctly dispatched every request without intervention. Beyond routing, Gemma 4 26B also replaced the Qwen 3.5 27B slot in the stack, reportedly with less token burn on simple math tasks and fewer overthinking spirals on straightforward prompts.
Why It Matters
Semantic routing is a load-bearing component in any multi-model local setup — get it wrong and you're burning VRAM and time on mismatched tasks, or worse, getting degraded outputs from an over-specced model that hallucinates more than it reasons. The fact that a 4B-class model swap fixed a pain point that prompt engineering couldn't is a meaningful data point for anyone building similar inference pipelines. Gemma 4 E4B is also notably efficient: the user runs it on hardware already taxed by larger models, with no reported throughput issues.
What to Watch
This is one user's benchmark on one rig, not a controlled eval — but the pattern matches early community consensus that Gemma 4's smaller models punch above their weight class on instruction-following and task classification. If the routing reliability holds across more complex pipelines, E4B becomes a serious default recommendation for local multi-model orchestration. Worth watching: whether the 26B holds up on code and tool-call reliability as more users stress-test it against Qwen 3.5 27B in comparable quantizations.