Agents query nothing like dashboards do. They arrive unannounced, read narrow slices, chain several requests to resolve one question, and increasingly expect to write conclusions back. Any data lakehouse for agentic AI has to answer all four of those behaviours, and most lakehouses currently answer one.
Storage is seldom where this goes wrong. Parquet on object storage, an open table format and a query engine will carry agent traffic perfectly well. Difficulty concentrates one layer up, in the catalog that cannot say what a column means, the policy engine that only recognises human principals, and a write path built on the assumption that every change has a person behind it.
Evidence for the gap is not thin. Harvard Business Review Analytic Services, working with Cloudera, reported in March 2026 that only 7% of enterprises considered their data completely ready for AI. An IBM survey of chief data officers from November 2025 put 73% of them in difficulty over AI data preparation, with 79% still working out how their agents' datasets would be governed at all.
What follows is the architecture underneath those numbers: four behaviours that break existing assumptions, five capabilities worth building, a reference read and write path, the table-format properties that begin to matter under agent load, and the two cost lines that decide whether a pilot survives real volume.
Warehouse versus lake, one copy or two, which table format — those debates resolved largely on the economics of analytical scans, and that arithmetic still holds. It simply stopped being what constrains you.
Someone wrote the dashboard query. Someone reviewed it. It runs at 06:00 against a partition layout shaped for exactly that access, and the planner has optimised it a thousand times.
Give an agent a goal instead and it decomposes the request at runtime, picks its own sources, issues a narrow read, revises on what came back, then issues more. Nobody wrote those queries.
Fan-out tracks how ambiguous the original request was, so it runs unbounded until you bound it on purpose.
Large row groups and coarse partitions pay off when you scan half a table. Ask for eleven rows by business key and the same layout turns a point lookup into a partition scan. Users feel the result directly as latency, rather than as a batch window nobody watches.
Rarely does this call for a new platform. A keyed serving projection of the handful of entities agents resolve against, sitting in front of the analytical tables, usually settles it. Version that projection and rebuild it from the governed copy, or it quietly becomes a second source of truth.
Agent traffic follows human attention, so it spikes without notice. Concurrency limits sized around known scheduled jobs tend to fail first, and they fail as queue depth rather than as errors. Teams then spend a fortnight investigating model latency when admission control was the culprit.
Establish two numbers before cutover: how many concurrent resolutions the platform absorbs without queueing, and what happens past that ceiling. Shedding load with a clear response is recoverable. Queueing silently behind a forty-second timeout is not.
Blast radius is widest here, and reference architectures skip it most often. As soon as an agent proves useful, somebody asks it to act — update the record, close the case, revise the forecast. Your lakehouse has just acquired a write path whose author has no face.
Analytics-era governance has no ready answer, because the question never arose. Which leads to the fourth behaviour.
Row and column policy in most warehouses resolves against a user or a group. An agent qualifies as neither: it runs as a service identity on behalf of a human who carries their own entitlements. Correct authorisation is the intersection of what the agent may do and what its principal may see.
Miss that and you land in one of two failure modes. Grant the agent a broad service account and any user who can prompt it inherits everything it can read, the classic confused deputy.
Give it only the user's context and you forfeit the ability to constrain what it does with that access. Both conditions have to be evaluated together, on every call.
Five capabilities, ordered roughly by how often their absence is what actually stalls a programme.
Show a human analyst a column named rev_net_adj and they go and ask somebody. An agent picks whichever column looks plausible and returns a confident wrong answer with no signal that it guessed.
Practically, that means a machine-readable semantic layer: canonical metric definitions with formulas, dimensional grain, valid filters, and — the part usually absent — what each metric must never be used for. A revenue figure that excludes intercompany eliminations needs that stated as a readable constraint, not footnoted in a wiki.
Run one test before anything else. Hand an agent your catalog and three questions phrased the way a business user would phrase them. If a human has to intervene to settle which metric applies, no amount of retrieval engineering downstream will rescue the answer.
Table-level grants stop being sufficient the moment one agent serves users with different entitlements. Row filters and column masks need to evaluate against the effective principal described above, and the query engine has to enforce them. Application code does not count as enforcement.
Purpose is the dimension teams overlook. One row can be lawfully readable to service a customer and unlawfully readable to market to them. Most privacy regimes treat that distinction as binding. Encode purpose as an attribute of the request and filter on it, and you have an answer when a regulator asks why a record was touched.
Humans reading dashboards carry an intuition about staleness. Agents carry none, and will state a stale figure as current without hedging.
Give every table an agent reads a declared contract: maximum acceptable lag, behaviour when lag exceeds it, and whether answering outside the contract is permitted with a caveat or forbidden.
Declaring is the straightforward half. Enforcement requires the agent to read both the contract and the current lag at request time, which makes it a catalog capability rather than a pipeline one.
Three conditions apply once an agent can change state. Changes arrive as proposals that clear policy before they commit, never as direct mutations. Commits are keyed and idempotent, so a retried tool call cannot double-apply. Resulting rows carry provenance: which agent, acting for which principal, on which input, from which source rows, under which model and prompt version.
Table format stops being a religious argument at exactly this point, and we return to it below.
Analytical spend gets measured per query or per terabyte. Neither number helps a decision here, since one user request may fan out to a dozen retrievals and several model calls. Business cases need cost and latency per resolved request at projected concurrency.
All three converged for analytical work, then diverged again over this workload. Their positions against the five capabilities above:
|
Capability an agent needs |
Cloud warehouse |
Data lake |
Lakehouse |
|
Semantics resolvable without a human |
Strong. Typed schema and a metrics layer usually already exist. |
Weak. Column meaning lives in notebooks and tribal knowledge. |
Depends on whether a catalog was built, or just a table format added. |
|
Row, column and purpose policy at the engine |
Strong. Row filters and column masks are native. |
Weak. File-level at best, and usually enforced in application code. |
Strong where the catalog enforces it. Absent in lakes that only gained a table format. |
|
Unstructured content under the same policy |
Poor. Documents end up in a second store outside the policy model. |
Strong. Already where the files are. |
Strong. The only one of the three holding both halves under one policy engine. |
|
Governed write path with provenance |
Good for structured writes. No lineage for document-derived output. |
Absent. No snapshots, no row identity, nothing to replay. |
Strong. Snapshot isolation, time travel and row-level lineage in current specs. |
|
Cost shape under many narrow reads |
Predictable, though point lookups bill like queries. |
Cheap to scan, poor at point lookup. |
Tunable, and normally needs a keyed serving projection in front. |
|
Where it leaves you |
Viable for agents that only ever touch structured data. |
Fastest route to a demo, hardest to audit afterwards. |
The default recommendation, and usually a retrofit rather than a rebuild. |
A cloud warehouse hands you the strongest semantic and policy story out of the box, which is no small advantage: existing row policy, column masking and a metrics layer already cover much of capabilities one and two.
Documents defeat it. Unstructured content makes up the majority of most estates and the part agents most want, so warehouse-only answers typically end with a second, ungoverned copy of that content living elsewhere, undoing the governance the warehouse was bought for.
A data lake stores everything and governs little. Documents already live there, and retrieval over them comes cheap. Write paths and policy granularity defeat it: absent a transactional table format and a real catalog, there is no snapshot to reason about, no engine-level row filter, and no credible provenance record. Agent programmes reach a working demo fastest on a lake and stall hardest there.
A lakehouse can hold both halves of the corpus under one policy model with a transactional write path, making it the default recommendation — and also explaining why the recommendation so often disappoints.
Many existing lakehouses began as lakes and acquired a table format. Substrate, yes; catalog, policy engine and governed write path, no. Retrofit works on something largely correct, then, rather than a rebuild.
Jumping straight from intent to vector search is the most frequent architectural mistake here. Settle semantics first: which entity, which metric, which grain, which window, under whose definition.
Resolution caches well, costs almost nothing once warm, and prevents the agent inventing a definition further downstream.
Retrieval for agents is not one activity. Three indexes typically run in parallel, and picking the wrong one for a given question is the usual failure.
One router choosing between the three, able to call more than one, beats any single index tuned harder.
Retrieval produces candidates; assembly decides what the model sees. Without a ceiling, retrieval quality decays into retrieval quantity — more passages, higher spend, and worse answers, since the relevant span now competes with noise.
Rank, deduplicate, truncate to budget, and log what got dropped. The log is what makes a bad answer diagnosable a week later.
The agent emits a structured intent to change something and holds no write credentials on the target table. A separate, unglamorous, well-tested component validates against policy and applies the change. Of every decision on the write path, this separation returns the most, costs least to build early, and hurts most to retrofit.
Each proposal carries a deterministic key derived from its content and originating request; the commit path deduplicates on it. Retries then become safe, and retries are ordinary operations on any network you actually have rather than an edge case.
Provenance kept in a separate log beside the data drifts from it. Provenance carried by the row survives compaction, replication and time travel. Hence the table-format question mattering here specifically.
Semantic resolution prevents invented definitions. Retrieval scope prevents reads beyond the effective principal. A human gate prevents irreversible actions clearing on model confidence alone. Provenance prevents answers nobody can reconstruct.
Drop any one and the system still demos beautifully. First audit, or first incident replay, is where it comes apart.
Iceberg, Delta and Hudi all qualify. Differentiators that dominate BI evaluations — file layout, compaction strategy, engine coverage — matter less here than three properties that seldom come up analytically.
An agent reasoning across several reads must see one consistent view, or it assembles answers from two different states of the world. Snapshot isolation guarantees the view. Time travel lets you reconstruct months later exactly what the agent saw when it decided, separating an explainable system from an anecdote.
Recently this moved from theory to shipping infrastructure. Row lineage entered the Apache Iceberg specification at v3: tables track per-row identity and modification metadata, making an individual record's history queryable rather than inferred.
Google's open-source team summarised the v3 additions in August 2025, and AWS shipped support for v3 deletion vectors and row lineage in November 2025 across Glue, EMR and S3 Tables.
For an agentic write path, no feature matters more. Row lineage converts "the agent updated this record" from an assertion into something substantiable at row granularity, without a parallel audit system.
Many small agent-authored writes against one table is precisely the pattern optimistic concurrency handles worst. Two unglamorous mitigations work: batch proposals into windowed commits rather than committing per interaction, and isolate agent-written data in its own tables that join back to the analytical ones, so agent write volume never contends with analytical compaction.
Deletion vectors help materially, since storing row-level deletes as compact bitmaps rather than separate delete files cuts the merge and compaction cost that agent-rate updates would otherwise generate.
Whichever format wins, agents talk to the catalog, and the catalog should be reachable over an open interface rather than through one engine's SQL dialect alone. Tool interfaces are standardising along the same lines: the Model Context Protocol, released by Anthropic in November 2024 and donated to the Agentic AI Foundation under the Linux Foundation in December 2025, has become how agents discover and call tools, data tools included. Exposing governed data access as an MCP tool with policy enforced behind it will outlast credentials embedded in an agent framework.
Agents need first-class identities: registered, owned by a named human, scoped to specific tools and datasets, credentials that rotate, an expiry date. Share one service account across several agents and every later audit question becomes unanswerable, because no access can be attributed to an actor.
Two ideas borrowed from privacy law that turn out to be sound engineering irrespective of jurisdiction. Under purpose limitation, every request declares why it reads, and policy filters on the declaration.
Under minimum necessary, the agent receives the narrowest slice that answers the question rather than the widest slice it is entitled to. Minimum necessary also happens to be the single largest lever on context cost.
Assume a demand to reconstruct one decision eighteen months on, from a person holding either subpoena power or an audit charter. The record needs the request and its declared purpose, the effective principal, resolved semantics, the exact rows and documents retrieved with their table snapshots, model and prompt versions, the output, and whether a human approved.
Generate all of that as a byproduct of the read and write paths. Teams planning to assemble it afterwards discover that retrieval never logged which snapshot it read, and the snapshot has since expired.
Corpus shape and volume set retrieval and embedding cost. Fan-out per request multiplies everything downstream, and responds most sharply to how well semantics are defined.
Freshness requirements matter because sub-minute lag is a different pipeline from hourly. Evidence depth matters because regulated environments log more, retain longer, and validate ahead of cutover rather than after it.
Picture a support-resolution agent handling 60,000 requests monthly. Average fan-out runs to eight retrievals, and assembly passes 12,000 context tokens per model call because nobody sets a cap.
Trim retrieval to the six passages that genuinely answer the question and context falls to roughly 4,500 tokens: a two-thirds reduction on the largest line, usually with better answers rather than worse.
Add a tool failing 4% of the time with two automatic retries. Roughly 12% gets added to every affected call, invisible on any dashboard reporting averages.
Neither figure claims anything about your environment. Ranking is the transferable part: context volume and retry amplification dominate, and architecture governs both, not vendor pricing.
Sequenced so each stage produces something usable if the next one slips.
Read-only agents can reach production after stage three. Anything altering state waits for stage four, less as a governance preference than because a write path without provenance is unauditable by construction.
Four or more answers of "we would have to check" means the honest starting point is stage one.
Sequence changes with the first use case; substance does not. Retrieval over documents, agents running against production data in a customer-facing workflow, or a data platform modernization that has to leave room for agents afterwards all land on the same six questions.