You are deploying a Google ADK agent with bursty traffic, long-running tasks, private network access, and a requirement for operational control. How would you choose among Agent Engine, Cloud Run, and GKE, and what would you do about state, scaling, and observability?
I would usually start with Cloud Run for the stateless ADK API, a durable queue for long-running work, and external durable stores for sessions and job state. I would choose Agent Engine for the least operational burden, or GKE when private networking, custom workers, or node-level control are hard requirements.
How to think about it
The answer
Given all four constraints, I would put the synchronous ADK API on Cloud Run and send long-running work through a durable queue; I would choose GKE instead when private networking or worker-level control is a hard requirement, and Agent Engine when managed ADK operations outweigh those controls. In every option, I would keep agent instances stateless, put sessions and job state in durable stores, and instrument model calls, tool calls, retries, and queue work—not just HTTP latency.
That answer is less about picking a fashionable product than matching the failure boundary to the workload.
What each platform is buying you
Agent Engine is the managed choice. Google operates the serving runtime, scaling behavior, and much of the surrounding infrastructure for a deployed ADK agent. It is attractive when the agent follows a conventional request-and-session pattern and the team wants to spend its time on prompts, tools, evaluations, and safety rather than container operations.
The price is control. You have fewer knobs over the base runtime, process layout, worker topology, and network path. Agent Engine has Google Cloud integrations and managed session capabilities for supported patterns, but I would verify the exact networking and regional support for the deployment. “It runs in Google Cloud” does not automatically mean it can open arbitrary connections to every private IP in your VPC.
Cloud Run is the practical middle ground. You package the ADK application as a container, and Cloud Run starts more instances as requests arrive, including scaling to zero when idle. It supports bursty HTTP traffic without asking you to manage nodes. You can control revisions, IAM, CPU and memory, request concurrency, minimum and maximum instances, timeouts, and network egress.
Cloud Run services are request-oriented. The maximum HTTP request timeout is 60 minutes. That is long enough for some agent calls, but it is not a durable workflow engine. A browser connection held open while an agent works for two hours is not a production architecture; it is a hostage situation with a loading spinner.
For private resources, Cloud Run can use Direct VPC egress or a Serverless VPC Access connector, with the usual VPC firewall, routing, DNS, and NAT decisions. That makes it a strong default when the agent needs to call a private service but does not need a custom operating system or a resident worker process.
GKE is the control choice. It makes sense when you need private-cluster networking, custom sidecars, specialized scheduling, persistent worker pools, custom autoscaling signals, or tight control over how agent servers and workers are deployed. You can separate the public API, queue consumers, tool proxies, and batch jobs into different workloads.
The bill is operational complexity. You now own cluster upgrades, node pools, pod placement, capacity planning, disruption handling, ingress, and more failure modes. GKE can scale, but a burst that arrives before new nodes are ready may wait or fail. For a small team, “maximum control” can quietly become “the platform engineer is on call forever.”
A useful decision table is:
| Requirement | Agent Engine | Cloud Run | GKE |
|---|---|---|---|
| Least platform work | Best | Good | Weakest |
| Bursty HTTP traffic | Good | Excellent | Good, but capacity needs planning |
| Private VPC access | Verify supported path | Good | Strongest |
| Long-running workers | Use an external workflow pattern | Use jobs or queue workers | Strongest |
| Node and process control | Limited | Container-level | Maximum |
| Fastest path to a managed ADK deployment | Best | Good | Slowest |
State is not the agent process
An agent instance is disposable. Autoscaling, a deploy, a crash, or a node eviction can remove it. Anything important that exists only in Python memory disappears with it.
Separate three kinds of state:
- Conversation state: the user, session identifier, messages, summaries, and approved facts needed for the next turn.
- Workflow state: the task status, current step, retry count, lease, idempotency key, and final result.
- Artifacts: uploaded documents, generated files, tool responses, and large intermediate data.
Conversation and workflow state belong in durable services, not module globals or local disk. Use the session mechanism appropriate to the platform, or a database-backed ADK session service for a self-managed deployment. Store large artifacts in object storage and keep references in the session or job record.
The workflow record should be explicit. For example:
queued → running → waiting_for_tool → completed
It should also tolerate retries. Cloud Tasks, Pub/Sub, and similar systems can deliver work more than once. If a payment-refund tool runs twice because the first response was lost, “the queue retried” is not a comforting postmortem. Give each business operation an idempotency key and make the tool or downstream service reject duplicate application.
Do not treat a conversation transcript as the system of record for permissions, orders, or financial decisions. An agent may summarize a fact incorrectly, and a user may ask for a different answer later. Authoritative business state stays in the relevant database.
Scaling the real bottleneck
For an agent, CPU is often not the bottleneck. A request may spend 10 seconds waiting on a model and another 8 seconds waiting on a private tool. That means an instance can look “idle” to CPU-based autoscaling while its request slots are full.
On Cloud Run, set concurrency from load tests rather than accepting a convenient default. Low concurrency may waste instances; high concurrency can cause memory pressure, model-call contention, or a thundering herd against the private service. Set a maximum instance count to protect downstream quotas, and use minimum instances when cold-start latency matters.
On GKE, CPU-based Horizontal Pod Autoscaling is often an incomplete signal for an I/O-heavy agent. Scale workers using queue depth, oldest-message age, or active work, while the cluster autoscaler adds nodes. Keep the API deployment and long-running worker deployment separate: they have different latency and scaling goals.
Agent Engine removes much of this tuning, which is its main appeal. It also means you accept its scaling model and limits. That is fine when the managed behavior fits. It is not fine when you need to reserve capacity for a private dependency or guarantee a particular worker topology.
A concrete deployment
Imagine an internal claims agent. At 9:00 a.m., 100 employees submit document-review requests in 30 seconds. Each request calls a private claims API and usually takes eight minutes because the agent extracts facts, asks a model to classify them, and waits on two internal tools.
I would expose a small Cloud Run service that authenticates the caller, creates a job record, stores the document in object storage, and returns a job ID quickly. It would publish work to a durable queue. A worker would claim the job, checkpoint after each tool call, and write progress and the final answer to the database. The client would poll a status endpoint or receive a notification.
That design absorbs the burst instead of holding 100 fragile HTTP connections open. It also gives operations a useful control: pause consumers if the private claims API is unhealthy, while still accepting and recording new jobs.
If the claims API required unusual network appliances, a fixed egress path, custom sidecars, or workers that routinely ran for many hours, I would move the worker tier to GKE. The API could remain on Cloud Run. Choosing different runtimes for different parts of the agent is often better than forcing the whole system onto one platform.
If the agent were a mostly conversational assistant with ordinary request durations and no difficult private-network requirement, Agent Engine would be my first choice. The managed runtime would remove a great deal of undifferentiated work.
Observability that catches the 3 a.m. problem
I want one trace correlated by request ID, session ID, and job ID. Under it, I want spans for each model call, tool call, retry, queue wait, and database operation. The dashboard should show:
- request and job success rates;
- p50, p95, and p99 latency;
- queue age and active workers;
- model and tool latency, errors, and retry counts;
- token usage and estimated cost;
- cold starts, instance counts, memory pressure, and throttling;
- session-store and private-service failures.
Cloud Run gives useful request logs and service metrics. GKE gives pod, node, and cluster telemetry. Agent Engine provides managed Google Cloud observability integrations, but I would still add application-level metrics for tool outcomes, workflow states, and token cost.
Never log full prompts, private documents, or unrestricted tool output by default. Redact sensitive fields, sample verbose payloads, and retain hashes or identifiers where correlation is enough.
A common failure appears as healthy HTTP traffic: the endpoint returns status 200, but the agent quietly catches a tool exception and replies, “I could not complete that request.” If you measure only HTTP status, production looks green while users are getting polished failures.
The senior-level nuance
The textbook answer is not “always use the most managed service.” The correct answer depends on where the complexity lives. Agent Engine minimizes platform work; Cloud Run minimizes work while preserving container and network control; GKE maximizes control at the cost of operating a distributed system.
I would prototype the agent on Agent Engine if its networking and execution model fit, but I would not let that choice dictate the workflow architecture. Durable state, explicit jobs, idempotent tools, and end-to-end traces should survive a later move to Cloud Run or GKE.
What they’ll ask next
Why not keep the eight-minute job inside the original Cloud Run request?
It may fit under the 60-minute timeout, but a client disconnect, deploy, retry, or transient failure can lose the work. Returning a job ID and checkpointing makes completion independent of one HTTP connection.
How would you prevent a traffic burst from exhausting the model or private API quota?
Bound Cloud Run instances or GKE worker replicas, limit queue consumption, tune concurrency, and add backpressure. Autoscaling without downstream limits simply moves the outage.
What would you migrate to GKE for?
A hard requirement for custom private networking, specialized sidecars, long-lived or multi-hour workers, custom queue-based autoscaling, or node-level operational controls. I would not migrate merely because GKE offers more knobs.
One line to say in the room:
“I’d default to Cloud Run with a durable queue and external state, use Agent Engine when managed ADK operations are the priority, and reserve GKE for requirements that genuinely need cluster-level control.”