Agentic data analysis: PandasAI, Hex Magic, Julius — and why it's harder than it looks

The “talk to your data” demo is two years old now and remains the cleanest version of the agentic AI pitch. Open a chat box, type “what was our churn rate in Q1 by customer cohort,” watch the model write SQL, execute it, plot the result, and explain what it means. Magic.

In demos, this works. In production, it routinely catches fire. The reasons are well-cataloged at this point — ambiguous questions, sensitive PII, expensive table scans, governance — but the gap between “the model can write SQL” and “the model can be trusted to write SQL against the production warehouse” turns out to be a much harder engineering problem than the early demos suggested.

The products that actually shipped in this space — Hex Magic / Notebook Agent, PandasAI, Julius, Snowflake Cortex Analyst, and the various MetricFlow-backed agents — share a small set of design choices that make them work where the generic “chat with your warehouse” attempts didn’t. This post is about those choices.

Why generic warehouse-chat fails

Ambiguity. “How many active users do we have?” sounds simple. In a typical SaaS company, “active” means three different things to product (logged in last 7 days), billing (paid in last 30 days), and engineering (made an API call in last hour). The LLM, given the raw schema, picks one essentially at random. A senior analyst’s first hour with a new dataset is spent learning these conventions; the LLM has no equivalent affordance.

Cost. A naive LLM-written SQL query against a 10TB BigQuery table can cost $50-$100 per execution. If the user iterates (“actually, group by month instead”), each iteration is another full scan. Without partitioning awareness, dry-runs, or cost guardrails, the bill at the end of a curious afternoon can be eye-watering. Real teams have written postmortems about a single user running up a four-figure scan bill in an hour.

PII and governance. The LLM, given full warehouse access, can see PII it has no business seeing. Even if you trust the model with the data, the audit trail — who asked what, what got returned — needs to satisfy SOC 2, HIPAA, or whatever compliance regime applies. Generic chat interfaces don’t ship with this.

Wrong answer. The hardest failure to detect. The model writes plausible SQL with a subtle JOIN mistake or a missing filter. The result is a number that looks reasonable but is wrong. The user reports it to their boss. Nobody catches it for a week. This is the failure mode that turns “fun demo” into “compliance incident.”

The products that work in production address some subset of these explicitly. Let’s walk through how.

Hex Magic — scoped tools inside the analyst’s environment

The most-deployed agentic analytics product, and the one whose design choices have most influenced the rest of the space. Hex was already the leading collaborative data notebook before AI; their Notebook Agent (the evolution of what was originally branded “Hex Magic”) inherits the existing notebook context — connected warehouses, defined cells, the project DAG.

The architectural choice that distinguishes Hex from generic “chat with your warehouse” tools: the agent lives inside a notebook with explicit cell context, not on top of a raw warehouse. The agent sees:

• The notebook’s connected data sources (warehouse + tables already in use).
• The existing cells — their SQL, Python, and chart definitions.
• The DAG between cells (which cell’s output feeds which).
• The user’s scoped permissions to data.

This context dramatically narrows what the agent can do. It’s not writing a query against everything; it’s extending a specific notebook with knowledge of what the analyst is already doing. As Hex’s product team puts it: “Because it understands your project’s DAG and cell context, it knows exactly what’s in use, what’s in your published app, and what’s redundant.”

The other Hex-distinctive move: the user scopes context deliberately. You can tell the agent “use only these tables, edit only these cells” — making the agent’s permissions a first-class concern, not an implicit consequence of warehouse credentials. For analytics teams operating under any kind of governance regime, this matters.

Where Hex wins: it serves the analyst, not the executive. The user is already in a notebook, already thinking analytically, already familiar with the schema. The agent extends the human’s capability rather than replacing it. The 2025-2026 Hex traction is mostly in mid-to-large data teams where the analyst is the buyer.

Where it loses: the executive who wants to chat with the data without learning a notebook UI. Hex’s approach is unapologetically analyst-first.

Julius — natural language with code as artifact

Julius takes a different angle. Marketed as “a browser-based AI data analyst built for non-technical teams”, it’s designed for users who don’t want to know SQL or Python. You upload a CSV (or connect a database), ask a question in plain English, Julius writes the Python in the background, executes it, returns a chart or table, and also shows you the code it used.

Two design decisions matter:

Code is the artifact, not the output. Julius doesn’t return only the chart — it returns the Python (or R, or SQL) that produced it, in a reproducible notebook. This is critical for trust: the answer can be audited, re-run, and modified. The “agentic loop with eval queries” framing fits here — every answer carries the means of its own verification.

Sandbox execution. The generated code runs in an isolated environment with the connected data. Julius doesn’t return a raw SQL query for the user to run against their warehouse; it runs the code itself, returning only the result. This bounds what the LLM can do — there’s no path from a malformed query to a full table scan against a production data warehouse.

The data source story is broad: CSV, Excel, Google Sheets, Postgres, BigQuery, Snowflake. Julius also integrates statistical methods (regression, ANOVA, correlation), forecasting, and chart-generation in a single interface. The pitch is “your data, your analysis, no code required” — and for the non-technical user this works well.

Where Julius wins: the non-analyst use case. PMs, marketers, finance teams who want to ask data questions without involving the analytics team. Faster than asking a human, slower than a real-time dashboard, but covers a large space of “I just need this number now.”

Where it loses: depth of analysis. Julius is good at the standard analytical operations; it’s less good at the kind of bespoke multi-step investigation that an experienced analyst would do in a notebook. For that, Hex’s environment is richer.

The Hex product positioning

There’s a strategic move in Hex’s positioning worth highlighting. Hex didn’t try to compete with general-purpose “chat with your warehouse” products on the consumer-style experience. They positioned for analysts specifically — the people whose job involves spending real time in a notebook environment — and built the AI features as augmentations to that workflow.

This is the right move for the data tools market, because the analyst is the buyer (or at least the primary advocate) for most data tooling at non-trivial companies. PMs and executives can be users, but they rarely have the budget or the political capital to mandate tool choice for the data team. Building for the analyst means building for the buyer; building for the executive means building for someone who can be impressed but not converted.

The product surface reflects this. The Notebook Agent feels like an extension of how analysts already work, not a replacement. The “Magic” branding from the earlier era was deliberately downplayed because most analysts roll their eyes at AI marketing; what they want is a tool that makes their existing work faster. Hex delivered that.

PandasAI — the open-source agent for Python users

PandasAI takes yet another angle, this one aimed squarely at Python developers and analysts. The library wraps Pandas DataFrames with a natural-language interface: load your data into a SmartDataframe, ask questions in English, get answers (and the generated Pandas code) back.

What makes PandasAI interesting in 2026 is the SemanticAgent — an evolution that adds a semantic layer between the natural language query and the underlying execution. Instead of asking the LLM to generate Python directly, the SemanticAgent produces an intermediate JSON query that’s then compiled to Python or SQL. This separation of concerns lets the system:

• Validate the intermediate query before execution.
• Apply governance constraints at the semantic-layer level.
• Generate either Python or SQL from the same semantic representation.

The data source story has expanded substantially — CSV, XLSX, PostgreSQL, MySQL, BigQuery, Databricks, Snowflake — making PandasAI a real option for production agentic analytics with a Python-first team.

Where PandasAI wins: programmability. If you want to build an internal Slack-bot that answers data questions, PandasAI is one of the cleanest libraries to embed. The open-source nature also matters for teams that can’t ship data to a SaaS product.

Where it loses: it’s a library, not a product. The wrapping UI, governance, and audit-trail work is on you. For most enterprise teams this is a significant lift.

The Julius failure modes

Julius is good but not magic. The most common production failure pattern: the user uploads a CSV with subtly malformed data (mixed date formats, encoded categoricals, units-of-measure mismatches across rows). Julius’s Python generation usually doesn’t catch these — it writes code that runs successfully against the parsed DataFrame but produces incorrect aggregations because the input had latent inconsistencies.

The non-technical user has no way to spot the issue. The chart looks reasonable; the trend is plausible; the answer is wrong. This is the same “wrong answer that looks right” failure mode that plagues all natural-language data tools, just expressed differently.

Julius’s mitigations: code is always visible, so a more technical reviewer can audit it. The product supports re-prompting with corrections. But the failure mode of “non-technical user accepts a plausible-looking wrong answer” is intrinsic to the product framing and not fully solvable.

Semantic-layer agents — Snowflake Cortex Analyst, dbt MetricFlow

The most architecturally interesting category, and probably the future of safe production-grade agentic analytics. Snowflake Cortex Analyst is the canonical example.

The Cortex Analyst architecture, in one paragraph: the customer defines a semantic model in YAML — business concepts, metrics, dimensions, relationships between tables. The semantic model is the bridge between business language (“monthly recurring revenue”, “churned customers”, “active subscribers”) and the underlying schema. The LLM (Snowflake-hosted Mistral and Meta models by default) gets the semantic model as context, not the raw schema. Questions are answered against the semantic model first; the SQL is generated against the underlying tables only after the semantic layer has constrained what’s possible.

The wins this architecture delivers:

• Ambiguity disappears. “Active users” maps to exactly one metric defined in the semantic model. Different definitions, if they exist, get different metric names that the LLM must choose between explicitly.
• Cost is bounded. The semantic model constrains what tables can be joined and how. The LLM cannot accidentally write a 10-way cross-join against a fact table.
• Governance is centralized. Permissions live on the semantic model; the LLM operates within them.
• No data leaves Snowflake’s boundary. Cortex Analyst uses Snowflake-hosted models, so for compliance-sensitive customers, the entire chain runs inside the customer’s governance perimeter.

The downside: someone has to write the semantic model. This is real work, often dozens of person-weeks for a non-trivial enterprise. The model also has to be maintained as the underlying schema evolves. The cost of getting the semantic layer right is the cost of doing the analytics-team-and-business-language alignment that should have happened anyway — a one-time cost, but a substantial one.

dbt’s MetricFlow is the same architecture from the dbt side, with the semantic layer defined in dbt’s existing YAML. A wave of agentic-analytics products in 2025-2026 (Cube AI, Lightdash, AtScale’s AI features) all use this same pattern: define a semantic layer first, let the agent operate inside it.

Why semantic models work where raw schema doesn’t

A deeper observation about the semantic-layer pattern: it works because it converts an open-ended generation problem into a constrained selection problem. When the LLM has the raw schema, it has to generate SQL — and SQL generation is high-dimensional, prone to subtle errors, and impossible to fully constrain. When the LLM has a semantic model, it generates a structured query against pre-defined metrics and dimensions — much closer to selection from a menu than free-form composition.

This is the same insight that made retrieval-augmented generation work better than free-form generation for question-answering: constrain what the model has to invent, and the failure modes shrink. The semantic layer is RAG for SQL — a curated context that bounds the LLM’s outputs to the known-good operations.

The cost is the curation work. Defining the semantic model requires aligning the analytics team and the business stakeholders on what concepts mean, which is the same work that should happen for good analytics anyway. Companies that find semantic-layer agents work well usually have already done much of this work for their BI tool. Companies that haven’t are now being forced to.

PII as a hard problem

One discipline that the semantic-layer pattern handles well is PII control. The semantic model declares which columns are sensitive; the layer can mask, hash, or deny access at the layer rather than relying on the LLM to be careful. The model never sees the raw PII; it sees the masked or aggregated version that the semantic layer exposes.

This matters because the LLM cannot reliably be trusted to handle PII appropriately. A model with full warehouse access can — and will — surface a customer’s email or address in an answer to a casual question. The semantic layer makes that structurally impossible by never exposing the raw column to the model in the first place.

The same goes for row-level security. Snowflake’s semantic views can be combined with row-access policies so that user A asking about “all customers” gets a different scoped result than user B asking the same question. The LLM doesn’t have to reason about access control; the underlying layer enforces it.

For any deployment touching regulated data — healthcare, finance, EU GDPR-scope user data — semantic-layer constraints aren’t optional. They’re the difference between a deployment that ships and one that gets blocked by the compliance team.

Where the labs sit in this market

A note on what the foundation-model labs are doing in this space. OpenAI, Anthropic, and Google all have data-analysis features in their flagship products (ChatGPT’s Code Interpreter, Claude’s analysis tool, Gemini’s Data Insights) but none has shipped a dedicated agentic-analytics product that competes head-on with Hex or Cortex Analyst.

The split makes sense. The labs are good at the model; they’re less good at the surrounding workflow (warehouse connectors, semantic-layer integration, governance, audit trails). The products that win in this space are heavy on the workflow side and use the LLM as a component. That’s not the labs’ core competency.

The likely 2027 pattern: the labs continue to provide the model layer; the vertical products (Hex, Cortex, Cube, dbt’s emerging AI offerings) provide the workflow. The semantic-layer companies are the architectural moat — once a company has invested in a Cube or dbt semantic model, the agent-on-top is largely commoditized. Snowflake’s strategy of bundling Cortex Analyst with Snowflake itself is the most aggressive bid for vertical integration; whether it works depends on how much customers value choice vs. tight integration.

A war story — the $4,000 afternoon

In early 2026, a mid-sized e-commerce company’s analytics team integrated a popular open-source “chat with your warehouse” tool against their BigQuery instance. The tool worked. Analysts could ask questions in English; the LLM would write SQL and run it. The team rolled it out to product managers as a self-serve analytics tool.

Within a week, the BigQuery bill spiked by an order of magnitude. A junior PM, exploring the tool, had asked a series of progressively broader questions about user behavior across the company’s largest event table (~40TB partitioned by day, but the LLM was generating queries that scanned the entire table). Each question cost $80-$120 in scan cost. The PM ran 40 of them in an afternoon.

The fix involved three things:

• Hard limits on bytes_billed. BigQuery’s maximum_bytes_billed job-level setting was set to 100GB per query. Queries exceeding the limit fail with an error the LLM saw and could correct against.
• Mandatory --dry-run. Every generated query was dry-run first; if the estimated scan exceeded a per-user quota, the agent had to refuse or refine.
• Restrict the agent’s table catalog. Instead of the whole warehouse, the agent was given access to a curated semantic-model view that fronted the event table with pre-aggregated daily summaries.

Total saved bill in the following month: ~$30K. The lesson: do not point an LLM at a warehouse without cost ceilings. The model has no notion of dollar cost; the orchestrator must enforce it.

The discipline patterns

Across all the products that work, a small set of disciplines recur:

• Dry-run before execute. Run an EXPLAIN or a --dry-run before any query. For BigQuery and Snowflake, the dry-run returns the estimated bytes scanned (and hence cost) without executing. The agent should refuse or escalate any query above a threshold.
• Pin to a semantic layer where one exists. If you have dbt MetricFlow, Cube, Looker, or Cortex semantic models in your stack, route the agent through them. The reduction in failure modes is enormous.
• Show the code. Every successful product in this space (Hex, Julius, PandasAI, Cortex Analyst) returns the generated query alongside the result. Users who can verify the code catch wrong answers. Users who only see the answer don’t.
• Scope, scope, scope. Don’t give the agent the whole warehouse. Give it the notebook context, the connected tables, the cell-level scope. The narrower the surface, the smaller the failure space.
• Cache results. Many “analytical questions” the user asks have been asked before. A result cache against the semantic-model-level query (not the raw SQL) is a 10x productivity gain for question-answering interfaces.

The query cache pattern

A related discipline: cache results aggressively at the semantic-model level, not the raw-SQL level. Two different natural-language questions can compile to the same semantic query (“how much revenue did we book last quarter?” and “Q1 revenue total?” should hit the same cache). The semantic layer makes this possible because the canonical form of the question is a structured query, not a string.

Production deployments report 40-60% cache hit rates on semantic-layer queries — meaning roughly half the questions users ask have already been asked in a slightly different form recently. Each cache hit saves a warehouse query (and a few seconds of latency). For high-volume agentic analytics products, this is a meaningful unit-economics improvement.

The cache invalidation story is also more tractable at the semantic level. When the underlying data changes (a daily ETL job runs), the cache invalidates against the affected metrics, not against all queries that happened to touch the affected tables. The semantic layer’s metadata makes the invalidation precise.

The Cortex Analyst tradeoff

A specific note on Cortex Analyst since it’s the most aggressive bet on the semantic-layer pattern. Snowflake’s choice to require a YAML semantic model is, in 2026, both a strength and a limitation.

The strength: the semantic model is a forcing function for analytics teams to align on definitions. Companies that have rolled out Cortex Analyst describe the YAML-authoring exercise as surprisingly valuable independent of the LLM benefit — the act of writing down “what active user means” and “how MRR is calculated” surfaces the kind of business-language ambiguity that’s been latent in the warehouse for years.

The limitation: it requires the analytics team to maintain the semantic model as the warehouse evolves. New tables, new metrics, schema changes all need YAML updates. For organizations with high analytics velocity, this maintenance burden is non-trivial. Some companies have written CI-style automation to detect schema drift and prompt for semantic-model updates; others have accepted that the semantic model lags the warehouse by weeks and the LLM degrades accordingly.

The right framing: the semantic-layer pattern shifts work from “fixing wrong LLM answers” to “maintaining definitional clarity.” The latter is more sustainable but isn’t free.

Eval queries — the verification pattern

A pattern from the PandasAI ecosystem that’s quietly becoming standard: eval queries. The idea is to ask the LLM to also generate a verification query — usually a simpler version of the main analysis that should produce a related result — and to flag the answer as suspect if the two disagree.

A concrete example. The user asks “what was our revenue last quarter by region?” The agent generates the analytical query (a SUM(amount) GROUP BY region against a large fact table). The agent also generates an eval query — a coarser check like “what was our total revenue last quarter?” — that should sum to the same total. If the regional sum doesn’t equal the total, something is wrong (maybe an outer-join introduced NULLs, maybe a filter is mis-applied), and the agent surfaces the discrepancy rather than presenting the result confidently.

This pattern works because most analytical errors leave fingerprints: a JOIN that drops rows, a filter that double-counts, a misaligned date window. An eval query that checks the result against a known invariant catches a meaningful fraction of these errors automatically.

The cost is one extra query per analysis (typically much cheaper than the main one). The benefit is a measurable reduction in wrong-but-plausible answers reaching the user. For any production agentic analytics deployment, this should be considered table stakes.

The role of the human reviewer

Across every successful agentic analytics deployment we’ve seen, there’s a human reviewer somewhere in the loop — analyst, data scientist, or domain expert. The agent produces; the human verifies, refines, signs off. The deployment patterns that work assume this rather than fight it.

The shift relative to traditional BI work isn’t that the human is removed; it’s that the human’s first draft is now the model’s draft, and the human’s job moves up to verification, refinement, and judgment. This is a real productivity shift — a senior analyst who used to write the query themselves now reviews the agent’s query and edits the result — but it’s not the “no humans needed” promise of the early demos.

The successful products lean into this. Julius shows the code so the analyst can verify it. Hex’s Notebook Agent runs inside the analyst’s notebook so they’re already there to review. Cortex Analyst’s outputs include the generated SQL and the semantic model interpretation. The pattern in all cases is “agent generates; human reviews”; the products that hide the generation from the reviewer (or assume no reviewer is needed) ship more wrong answers.

Where the line is, in 2026

The takeaways for anyone building or deploying agentic data analysis:

• The successful products narrow the agent’s surface dramatically. Hex narrows to a notebook. Julius narrows to a sandbox. Cortex Analyst narrows to a semantic model. Generic “chat with your warehouse” does not work in production.
• Semantic layers are the unlock. If you’re building agentic analytics into your product, define the semantic layer first. The LLM is the easy part.
• Cost guardrails are mandatory. Dry-runs, EXPLAIN-first, per-user quotas. Without them, a curious analyst can run up a four-figure bill in an hour. Treat unexpected cost as a Sev-2.
• Show the work. Code, query, semantic-model path. Always. The product where the user can’t verify what happened is the product that eventually ships a wrong number that matters.

The “just talk to your data” pitch was always more aspiration than spec. The 2026 reality is that the agent is genuinely useful — but only inside a tightly-scoped context, with a semantic layer to constrain it, with cost guardrails to bound it, and with verifiable artifacts so the user can audit it. Get those four right and you have a real product; skip any of them and you have a recurring incident report.

The interesting thing is how this mirrors the broader agent story: the smart wrapper around the model is what makes the thing reliable. The model proposes; the orchestration, the semantic constraints, and the cost ceilings dispose. The teams that internalized this two years ago are shipping; the teams still hoping the next model release will solve their warehouse-chat reliability problem are not.

The likely 2027 evolution is more semantic-layer infrastructure, not less. The Snowflake/dbt/Cube/Looker semantic-layer push is becoming the standard substrate for agentic analytics, and the products built on top of it (whether Snowflake’s own Cortex Analyst or third-party tools that hook into the same models) will increasingly look similar at the layer where the LLM operates. The differentiation will be in the user experience — Hex’s analyst-first UI, Julius’s executive-first chat — not in the underlying constraint mechanics.

For builders entering this space, the path is well-trodden enough now that the “what to build” question has clearer answers than it did 18 months ago. Pick a workflow lane (analyst, executive, developer, Slack-user). Lean on an existing semantic-layer (dbt, Cube, Cortex, Looker). Wrap with cost ceilings, eval queries, code-visible artifacts. Sell to the buyer for that workflow lane. The unsuccessful “build it all from scratch with raw warehouse access” pattern has been thoroughly demolished by two years of failed deployments; the winning pattern is to stack on top of existing data infrastructure and add LLM where it helps.

---

Further reading: Hex’s Notebook Agent launch, the PandasAI repository, Julius AI’s product overview, Snowflake Cortex Analyst docs, and dbt Labs’ MetricFlow documentation. For a deeper look at the semantic-layer pattern, see ClearPeaks’ walkthrough of Snowflake semantic views. For the underlying patterns that make agentic products like these reliable, our five patterns post covers the orchestration shapes that recur.