MHA → MQA → GQA → MLA: the attention efficiency ladder
MHA, MQA, GQA, MLA — every rung shrinks the KV cache that bottlenecks LLM inference. What each step gives up to make models cheaper to run.
There is a number that quietly governs how long a context window you can afford, how many users you can serve on one GPU, and how much your inference bill comes to. It isn’t the parameter count. It’s the size of the KV cache — and the last few years of attention research are, more than anything, a sustained campaign to shrink it.
Why the cache is the enemy
During generation, a transformer caches the key and value vectors for every token it has seen, at every layer, for every attention head. It has to: each new token attends back over all of them. That cache grows linearly with context length and, at long contexts, becomes the dominant consumer of GPU memory — often larger than the activations, sometimes rivalling the weights.
So the question that defines the ladder is simple: how few keys and values can we get away with caching, without the model getting noticeably worse? Each rung answers it differently.
Rung 1 — MHA: one K/V per head
Classic multi-head attention gives every query head its own key and value heads. With 32 heads you cache 32 sets of K and V per token per layer. This is the gold standard for quality — each head attends in its own subspace — and the most expensive thing to cache. For years it was simply how attention worked, and the cache was a problem for later.
Rung 2 — MQA: one K/V for everyone
Multi-query attention takes the radical swing: keep all 32 query heads, but let them share a single key/value head. The cache shrinks by 32× overnight. The catch is exactly what you’d fear — collapsing all that key/value diversity into one head measurably hurts quality. MQA proved the cache could be slashed, but it gave up too much to be the answer for frontier models. It was the experiment that defined the trade-off.
Rung 3 — GQA: the pragmatic middle
Grouped-query attention is the compromise that stuck. Instead of one K/V head (MQA) or one per query head (MHA), you keep a handful — say 8 K/V heads shared across 32 query heads, in groups of 4. The cache drops 4×; the quality stays within a whisker of full MHA. That ratio turns out to be a sweet spot, and it’s why GQA is the default in essentially every modern open-weight model — LLaMA, Mistral, Qwen. It’s the rung most models sit on today. (We build it up from MHA in Multi-head attention.)
Rung 4 — MLA: cache a latent, not the keys
DeepSeek’s multi-head latent attention changes the move entirely. Rather than choosing how many K/V heads to keep, it stops caching K and V directly at all. It caches a small compressed latent vector per token and reconstructs the full keys and values from it at compute time. You store far fewer numbers than even GQA, yet every head still gets its own (reconstructed) K/V — so quality stays close to full MHA. The price is a little extra compute to decompress on each step, which is a bargain when memory bandwidth is the thing you’re short of.
Reading the ladder
| Rung | K/V kept (per 32 query heads) | KV cache | Quality |
|---|---|---|---|
| MHA | 32 | 1× | best |
| MQA | 1 | ~32× smaller | noticeably worse |
| GQA | 8 | 4× smaller | ≈ MHA |
| MLA | a compressed latent | smaller than GQA | ≈ MHA |
Notice the shape of progress. MHA set the quality bar and the memory problem. MQA found the extreme and its cost. GQA found the practical middle. MLA changed the representation so you no longer have to choose between heads and memory. Each is a different answer to the same question.
Why it’s worth knowing
This isn’t trivia for model builders. The attention variant a model uses is a direct lever on what you can do with it: how long a context it serves affordably, how many concurrent users fit on your GPU, how big the KV cache offloading problem gets before you even reach for the memory hierarchy. When you read that a new model “uses GQA with 8 KV heads” or “uses MLA,” you’re reading its inference economics. The ladder is the story of how the field learned to make attention cheap to run, not just powerful to train — and it isn’t finished climbing.