LLMs
15 articles in this topic.
activation checkpointing makes GPU memory a scheduling decision
Large-model training is constrained by more than weights. Activation checkpointing changes which forward tensors survive, trading recomputation for a smaller peak-memory footprint.
MHA, MQA, GQA, and MLA: attention efficiency trade-offs
MHA, MQA, GQA, and MLA offer different ways to shrink or restructure the KV cache for LLM inference. What each design trades in quality, compute, memory, and implementation complexity.
Running an LLM on your laptop: GGUF, llama.cpp, and the Q-soup
A 7B model now fits on a laptop. GGUF, llama.cpp, and quantization tiers like Q4_K_M — decoded, so running LLMs locally stops being intimidating.
Attention is O(n²) — and Mamba's linear escape
Attention costs O(n²), so long context gets expensive fast. State-space models like Mamba do linear sequence work at fixed dimensions, which can make some very long streaming workloads more tractable. Some long-context workloads may favor hybrid designs, but quality, latency, memory, and cost must be measured.
Beyond next-token: world models and the next paradigm
World models predict the next state of an environment, not merely the next token. See how action-conditioned prediction supports planning, and where visual and latent approaches fail.
Diffusion language models: when AI writes text all at once
Masked discrete diffusion language models often start from a masked sequence and refine many positions in parallel over several sequential denoising passes.
Reading a model's mind: sparse autoencoders explained
Sparse autoencoders pull human-readable features out of an LLM's tangled activations — the breakthrough tool of mechanistic interpretability.
How reasoning distillation gives small models stronger math and code behavior locally
A practical guide to teacher signals, verified reasoning traces, supervised fine-tuning, evaluation, and serving small reasoning models on local hardware.
DPO vs PPO-based RLHF: when offline preference tuning wins
RLHF aligned chat models with a reward model and a fragile RL loop. DPO drops both, learning the same preferences directly from chosen-vs-rejected pairs.
Small models for narrow tasks: when they beat larger models
When a small language model can match or outperform a larger one on a stable, measurable task slice—and when that claim fails.
Why a deployed model does not learn from a chat—and how fine-tuning can cause forgetting
A deployed checkpoint normally does not update its weights from a conversation unless an explicit memory or update loop exists. Fine-tuning on sequential data can cause catastrophic forgetting, meaning degraded old-task performance, but does not necessarily do so. Why continual learning remains an open problem in 2026.
Test-time compute: why thinking longer beats thinking bigger
Test-time compute lets a model think before answering, so a small reasoning model can beat a far larger one. How inference-time scaling works.
Temperature, top-p, top-k: the three knobs on an LLM
Every LLM call hides a small probability lottery behind its confident prose — temperature, top-k, and top-p are the rules of that lottery, and most practitioners tune them without understanding what they are reshaping.
Fine-tuning vs RAG: the settled debate of 2026
Three years of Twitter brawling about whether to fine-tune or retrieve has ended in a boring, useful answer: RAG for knowledge, fine-tuning for behaviour, and both when you actually need both. Here's the decision table production teams actually use.
Structured outputs engineering: JSON mode, function calling, constrained decoding
Three families of techniques get LLMs to return parseable data. One of them is a guarantee. The other two are negotiations. Here's when each one earns its keep — and the production regression that nobody warns you about.