Skip to content
datarekha
Blog Production AI · GATE DA · how concepts actually work · Page 11 of 13

From theory to the systems that demand real understanding.

Long-form pieces on how production-AI teams orchestrate agents, serve models, and build the stack, plus GATE DA essays that turn probability, linear algebra, DBMS, ML, and AI into durable concepts.

295 posts · 20 categories
Agents May 18, 2026

Voice agents at scale: the Vapi, Retell, Bland.ai engineering

Voice agents are the unsexy success story of the agent era. Underneath the marketing they're all the same five-box pipeline — STT, LLM, TTS, turn-taker, telephony — fighting for the same 500ms latency budget. Here's how the three biggest platforms actually build it, what each one optimizes for, and where the real cost goes.

13 min read Read
Agents May 17, 2026

Devin's architecture, anatomised

Cognition's launch demo promised a 'fully autonomous software engineer.' The product that actually ships is more interesting — a planner-executor split running in a sandboxed VM, with explicit memory windows, hard pruning rules, and a shockingly disciplined view of where the model is allowed to be creative.

14 min read Read
Frameworks May 17, 2026

We built the same agent three times: MAF, LangGraph, and ADK

Three frameworks, one problem: an invoice-processing agent that ingests PDFs, validates against a database, asks a human when uncertain, and writes back to an ERP. What each framework made easy. What each one made painful.

14 min read Read
Agents May 17, 2026

Sierra's customer-service playbook

Bret Taylor and Clay Bavor built the highest-profile agent company by making three opinionated bets — every brand needs its own AI, agents must run inside the brand's workflows, and pricing should track outcomes. Two years on, the bets are paying off in ways the industry is still copying.

13 min read Read
Agents May 16, 2026

AutoGPT to 2026: what survived

Three years after AutoGPT briefly broke GitHub's star counter, almost nothing in its original form has shipped. The interesting question is what did survive — and it's a sharper, more honest answer than the discourse of 2023 ever got close to.

12 min read Read
RAG May 16, 2026

Self-RAG and Corrective RAG, the loops that actually help

Two papers from 2023 argued RAG should be reflective: retrieve, judge, re-query if the evidence is bad. By 2026 the practitioner verdict is in — Self-RAG and CRAG actually do help, but only on the queries that need them, and the latency cost is real.

12 min read Read
Cloud May 15, 2026

Bedrock, Vertex, Foundry: pick the one whose spirit matches yours

Every comparison of the three hyperscaler AI platforms is a feature matrix that nobody reads twice. The actual decision is about which company's spirit your team can live with for the next five years — and that's a different question.

12 min read Read
Infrastructure May 15, 2026

Continuous batching: the one trick that made LLM serving 10x cheaper

Static batching wastes GPUs by padding to the longest sequence and waiting for stragglers. Continuous batching reschedules every iteration, evicts finished requests, and admits new ones mid-stream. Here's why that single shift bought a 10-23x throughput win.

11 min read Read
Agents May 15, 2026

How Cursor's Composer actually works

Multi-file edits feel atomic in Cursor not because the model got smarter, but because the team built a stack of careful workarounds — speculative diffs, a separate Apply Model, and an indexer that stays one step ahead of you. Here's the engineering.

13 min read Read
Infrastructure May 14, 2026

Computer-use latency engineering: getting browser agents under a second

A naive browser-use loop is 4-8 seconds per step. Production systems run at 600ms. The gap is closed by half a dozen techniques — differential screenshots, prompt caching, batched actions, vision-model routing — each of which sounds boring until you measure the difference.

13 min read Read
Frameworks May 14, 2026

Google's Agent Development Kit, and when to reach for it

ADK is Google's bet on agent infrastructure: a thin SDK that takes Gemini's strengths — tool calling, grounding, multimodality — and makes them deployable to Vertex AI Agent Engine in one command. It is less framework than LangGraph, more framework than calling the model directly, and uniquely useful inside Google Cloud.

11 min read Read
RAG May 14, 2026

Late interaction, or why ColBERT keeps coming back

Single-vector dense retrieval crushes one nuance per chunk into one point in space. Late-interaction models like ColBERT keep a vector per token and decide similarity at query time. The math is beautiful. The storage cost is brutal. Here's when it's worth it.

11 min read Read
Agents May 13, 2026

Sub-agents, handoffs, supervisors — pick exactly one

Three topologies dominate multi-agent systems in 2026, and teams keep mixing them in ways that break debuggability. Sub-agents are parallel scoped work that returns to a parent. Handoffs are baton-passing with no return. Supervisors are explicit central routers. They are not interchangeable. Here's the framework for picking, and the failure modes when you don't.

13 min read Read
Infrastructure May 12, 2026

Prompt caching: the 90% cost cut explained

Anthropic, OpenAI, DeepSeek and Google all ship prompt caching now, but the pricing models diverge sharply. For a tool-using agent with a long system prompt, getting the cache pattern right is the difference between a viable product and a P&L disaster.

12 min read Read
Cloud May 12, 2026

Cloud Run is the most underrated platform for AI agents

Everyone deploys agents on GKE or EC2. The teams quietly shipping the fastest, cheapest production agents have moved to Cloud Run + Gemini + Firestore — and the reasons are not what you'd guess from the marketing pages.

11 min read Read
RAG May 12, 2026

Hybrid search, when one retriever isn't enough

Pure vector search misses exact identifiers. Pure BM25 misses paraphrases. Every serious production RAG stack — Notion, Perplexity, Vespa, Glean — runs both and fuses the results. Here's the math, the latency budget, and the production knobs that matter.

12 min read Read
Infrastructure May 12, 2026

vLLM vs TGI vs SGLang: choosing your inference server in 2026

Three open-source serving stacks, three different bets. vLLM optimises raw throughput. TGI optimises ease and HuggingFace integration. SGLang optimises structured generation and prefix-cache reuse. Here's how to pick.

13 min read Read
Agents May 11, 2026

Long-horizon agent failure modes: context rot, drift, looping

An agent that handles ten turns gracefully will likely collapse at fifty. The reasons aren't bugs — they're emergent properties of how transformers handle growing context. Here's a field guide to how agents disintegrate, with the public failure stories that mapped the territory.

13 min read Read
Frameworks May 11, 2026

Microsoft Agent Framework, six months in

Microsoft consolidated Semantic Kernel and AutoGen into a single Agent Framework in late 2025. Half a year later, the picture is clear: MAF is the Azure-native option for shops that already speak C# and .NET, and its workflow/agent split is the most pragmatic in the industry.

11 min read Read
Agents May 10, 2026

Coding agents in 2026: Cursor, Devin, Sweep, Aider, Claude Code compared

Two years after Devin launched and froze the term 'AI software engineer' in the popular imagination, here's where everyone actually landed. The dominant tools didn't converge — they specialized, and the ones that won did so by picking a workflow lane and dominating it.

14 min read Read
RAG May 10, 2026

Contextual Retrieval, two years on

Anthropic's September 2024 trick — prepend a sentence of context to each chunk before embedding — quietly became the new RAG default. Here's why a 49% retrieval-error reduction held up, what the numbers actually say, and when you still shouldn't bother.

11 min read Read
Agents May 9, 2026

Deep research agents explained: Perplexity, GPT Deep Research, Gemini Deep Research

Deep Research was OpenAI's first agent product to actually ship, and it landed in a market where Google had already been live for months and Perplexity had built the search-citation pipeline the others were chasing. The three converged on similar capabilities but radically different architectures.

13 min read Read
Infrastructure May 9, 2026

How Anthropic serves a hundred million tokens a second

The frontier labs and their serving partners — Anthropic, OpenAI, Together AI, Fireworks — hide an arsenal of inference optimisations behind the simple-looking chat endpoint. Here's the hierarchy, what each layer bought, and the public numbers behind it.

13 min read Read
Agents May 8, 2026

Agentic data analysis: PandasAI, Hex Magic, Julius — and why it's harder than it looks

'Just talk to your data' has been the cleanest agentic AI demo for two years. The shipped reality is much narrower than the demos suggest, because the gap between answering a question and answering it cheaply, safely, and correctly is enormous. Here's where the production line lives.

13 min read Read