Skip to content
datarekha
Blog Production AI · GATE DA · how concepts actually work

From theory to the systems that demand real understanding.

Long-form pieces on how production-AI teams orchestrate agents, serve models, and build the stack, plus GATE DA essays that turn probability, linear algebra, DBMS, ML, and AI into durable concepts.

281 posts · 20 categories
Latest Agents · Aug 28, 2026

Agent memory is a product decision, not a database choice

The hard part of agent memory is deciding what deserves to persist, who can use it, and how a person can take it back.

11 min read
Read story
Agents Aug 28, 2026

Computer-use agents need permission ladders

A browser session should not hand an agent every power its logged-in human happens to possess.

11 min read Read
Agents Aug 28, 2026

Guardrails are runtime checks, not prompt instructions

A prompt can ask an agent to behave, but only a runtime check can stop an unsafe action.

11 min read Read
Agents Aug 28, 2026

Human-in-the-loop is a control plane, not a confirmation dialog

Useful human oversight decides when to interrupt an agent, what the reviewer needs to see, and how the decision changes the system.

11 min read Read
Agents Aug 28, 2026

MCP security after the hype cycle

MCP made agent integrations portable, which means the hard security question is now what each connection is allowed to do.

11 min read Read
Agents Aug 28, 2026

The injection you're not filtering: tool output

Tool output is data to your application, but it can look like instructions to the model, so protect the action boundary rather than only the user message.

11 min read Read
Python Aug 15, 2026

apply() is a for loop in disguise

Rewriting a Python loop as .apply() makes the code shorter and almost never makes it faster, because .apply() is the same loop with a nicer costume. Here is the ladder that does make it faster.

11 min read Read
Agents Aug 15, 2026

MCP went stateless — what server authors have to change

The 2026-07-28 revision deleted the initialize handshake and the session header, replaced server-initiated requests with a retry pattern, and put Roots, Sampling and Logging on a twelve-month clock.

12 min read Read
Statistics Aug 15, 2026

Simpson's paradox is already in your dashboard

The new checkout converts better on desktop and better on mobile, and worse overall. Both statements are arithmetically correct, and one of them is about to be presented to your leadership team.

10 min read Read
Data Engineering Aug 15, 2026

The small files problem eats your read throughput

A 400 GB table that takes forty minutes to scan is not a 400 GB problem. It is a 1.2-million-file problem, and every one of those files charges the same fixed toll.

11 min read Read
Data Engineering Aug 15, 2026

The shuffle is the bill

Two Spark jobs read the same data and write the same answer. One costs six times more. The difference is almost always a wide transformation nobody noticed.

12 min read Read
SQL Aug 15, 2026

Why your index isn't being used

You built the index. The planner ignored it. Five query shapes silently disable an index, and four of them are things you wrote without thinking.

11 min read Read
LLMs Jun 18, 2026

activation checkpointing makes GPU memory a scheduling decision

Large-model training is constrained by more than weights. Activation checkpointing changes which forward tensors survive, trading recomputation for a smaller peak-memory footprint.

9 min read Read
RAG Jun 18, 2026

RAG poisoning is an evidence-integrity problem

A retrieval system can reason perfectly from corrupted evidence. Defending RAG means governing what enters the corpus, preserving provenance, isolating tenants, and treating retrieved text as untrusted data.

10 min read Read
Infrastructure Jun 18, 2026

Six system-design boundaries that prevent category mistakes

Stateless vs stateful, Lambda vs ECS, database vs cache, queue vs stream, retrieval vs reranking, and monitoring vs tracing—explained as operational contracts.

11 min read Read
Agents Jun 18, 2026

Token theft in AI agents is an architecture failure

Prompt injection gets the attention, but credentials turn a confused model into an authenticated attacker. The fix is to keep tokens out of model context and authorize every action at runtime.

11 min read Read
Infrastructure Jun 12, 2026

How vLLM actually serves a 7B model

Follow one request through vLLM — the scheduler, the KV-cache blocks, prefill vs decode, and what happens when 90,000 tokens of cache no longer fit.

13 min read Read
Infrastructure Jun 12, 2026

The vector-search memory wall: why HNSW eats RAM, and how quantization cuts the bill 32×

HNSW is fast because the whole graph lives in RAM — which is exactly why it gets expensive. At 100M vectors you're paying for ~600GB of memory before you serve a single query. Here's the math, and how binary quantization plus reranking is rewriting the cost model in 2026.

11 min read Read
LLMs Jun 11, 2026

MHA, MQA, GQA, and MLA: attention efficiency trade-offs

MHA, MQA, GQA, and MLA offer different ways to shrink or restructure the KV cache for LLM inference. What each design trades in quality, compute, memory, and implementation complexity.

10 min read Read
ML Jun 10, 2026

AutoML raised the floor, not the ceiling

AutoML can make a strong tabular baseline cheap. AutoGluon's 2025 release notes report a benchmark result under a five-minute budget, but that vendor report is not evidence of a current 2026 leader. The ceiling still depends on trustworthy data, domain features, and problem framing.

9 min read Read
Agents Jun 10, 2026

Context engineering: why accumulated context can degrade your agent

For multi-step agents, context engineering complements prompt engineering by curating the smallest high-signal working set at each step. Accumulated or irrelevant context can degrade reliability even when it fits the window.

11 min read Read
Infrastructure Jun 10, 2026

Don't auto-ship retrained models: collapse, feedback, and the challenger gate

Retraining can produce a worse model—from bad data, a pipeline bug, or learning from outcomes collected under its own serving decisions. Champion-challenger automates retraining while limiting the blast radius of regressions.

9 min read Read
ML Jun 10, 2026

Feature engineering still matters on many tabular problems

On many medium-sized tabular problems, engineered, point-in-time features can matter more than switching among strong algorithms. This is an engineering heuristic, not a universal law; TabPFN's Nature benchmark result makes the raw-versus-engineered comparison worth testing rather than settling it in advance.

9 min read Read
Infrastructure Jun 10, 2026

GPU compute isn't always the bottleneck: why decode can become a memory problem

During decode, LLM serving can depend on both HBM capacity and bandwidth: every step reads model weights and the relevant KV state, with KV traffic becoming especially important for long contexts and larger batches. Paged allocation reduces wasted capacity, and continuous batching can improve throughput and utilization, depending on the workload and its compute, bandwidth, communication, and latency limits.

13 min read Read