LLMs
19articles in this topic.
Hill-climbing your evals: how AI teams actually improve, and how they fool themselves
Every serious AI team improves its system the same way: change one thing, run the eval, keep it if the number goes up. That loop is called hill-climbing, it is how frontier labs and two-person startups alike make progress — and if your eval set is small, it reliably manufactures improvements that do not exist.
Do Chinese AI models steal your data? A threat model, not a vibe
Many companies ban DeepSeek, Qwen and GLM on the grounds that they steal data. Some of that worry is exactly right and some of it is physically impossible. Separating the two — hosted apps, local weights, loader code, model behaviour and procurement rules — lets you write a policy that actually protects you.
Evolution with a language model inside: how AlphaEvolve, FunSearch and GEPA search for better programs
Some of the most surprising AI results of the last three years — a faster way to multiply 4×4 complex matrices, a scheduling heuristic recovering 0.7% of Google's compute, prompts that beat reinforcement learning — came from the same simple machine: a language model proposing changes, an automatic evaluator scoring them, and a population that remembers what worked.
Distillation quietly flipped: stop training on the teacher's answers
The standard recipe — have a big model write a corpus, fine-tune a small one on it — supervises the small model everywhere except where it actually operates. Sampling from the student and grading with the teacher fixes that, and it explains why so many distilled models are fine for a sentence and then fall apart.
activation checkpointing makes GPU memory a scheduling decision
Large-model training is constrained by more than weights. Activation checkpointing changes which forward tensors survive, trading recomputation for a smaller peak-memory footprint.
MHA, MQA, GQA, and MLA: attention efficiency trade-offs
MHA, MQA, GQA, and MLA offer different ways to shrink or restructure the KV cache for LLM inference. What each design trades in quality, compute, memory, and implementation complexity.
Running an LLM on your laptop: GGUF, llama.cpp, and the Q-soup
A 7B model now fits on a laptop. GGUF, llama.cpp, and quantization tiers like Q4_K_M — decoded, so running LLMs locally stops being intimidating.
Attention is O(n²) — and Mamba's linear escape
Attention costs O(n²), so long context gets expensive fast. State-space models like Mamba do linear sequence work at fixed dimensions, which can make some very long streaming workloads more tractable. Some long-context workloads may favor hybrid designs, but quality, latency, memory, and cost must be measured.
Beyond next-token: world models and the next paradigm
World models predict the next state of an environment, not merely the next token. See how action-conditioned prediction supports planning, and where visual and latent approaches fail.
Diffusion language models: when AI writes text all at once
Masked discrete diffusion language models often start from a masked sequence and refine many positions in parallel over several sequential denoising passes.
Reading a model's mind: sparse autoencoders explained
Sparse autoencoders pull human-readable features out of an LLM's tangled activations — the breakthrough tool of mechanistic interpretability.
How reasoning distillation gives small models stronger math and code behavior locally
A practical guide to teacher signals, verified reasoning traces, supervised fine-tuning, evaluation, and serving small reasoning models on local hardware.
DPO vs PPO-based RLHF: when offline preference tuning wins
RLHF aligned chat models with a reward model and a fragile RL loop. DPO drops both, learning the same preferences directly from chosen-vs-rejected pairs.
Small models for narrow tasks: when they beat larger models
When a small language model can match or outperform a larger one on a stable, measurable task slice—and when that claim fails.
Why a deployed model does not learn from a chat—and how fine-tuning can cause forgetting
A deployed checkpoint normally does not update its weights from a conversation unless an explicit memory or update loop exists. Fine-tuning on sequential data can cause catastrophic forgetting, meaning degraded old-task performance, but does not necessarily do so. Why continual learning remains an open problem in 2026.
Test-time compute: why thinking longer beats thinking bigger
Test-time compute lets a model think before answering, so a small reasoning model can beat a far larger one. How inference-time scaling works.
Temperature, top-p, top-k: the three knobs on an LLM
Every LLM call hides a small probability lottery behind its confident prose — temperature, top-k, and top-p are the rules of that lottery, and most practitioners tune them without understanding what they are reshaping.
Fine-tuning vs RAG: the settled debate of 2026
Three years of Twitter brawling about whether to fine-tune or retrieve has ended in a boring, useful answer: RAG for knowledge, fine-tuning for behaviour, and both when you actually need both. Here's the decision table production teams actually use.
Structured outputs engineering: JSON mode, function calling, constrained decoding
Three families of techniques get LLMs to return parseable data. One of them is a guarantee. The other two are negotiations. Here's when each one earns its keep — and the production regression that nobody warns you about.