Most of the stalled agent programs we get called into share one root cause, and it rarely sits in the model. Take a made-up but typical case we'll follow through this guide: an industrial distributor with about $900 million in revenue that sells maintenance and repair parts to factories in the US and Germany. Its quote-assist agent demoed well in March. The trouble started when it was pointed at real data. The same customer turned up under three IDs in SAP, Salesforce and the e-commerce site, the margin table hadn't had an owner since the last reorg, and who could see what was written down in a spreadsheet rather than enforced anywhere. Security stopped the pilot in June, and by September the CFO was asking why a competitor already had agents in production.
We've been building on AWS since 2008, much of it data and AI systems for banks, healthcare payers, chipmakers and consumer platforms that had to get through security and audit. What we've learned in that time is that the order of work matters more than the tooling. The teams that put the data foundation in first ship agents sooner and spend less doing it. We've rarely seen a team that started with agents avoid rebuilding the data layer; it just happens later, under deadline.
The distributor's CFO wasn't imagining the competition, either. McKinsey's 2026 State of AI survey asked about AI agents, and among respondents at companies over $1 billion in revenue, the share scaling them "increased from 27 percent to 40 percent". Gartner's prediction from early 2025 is the other half of the picture: through 2026, it expected organizations to abandon 60% of AI projects that weren't supported by AI-ready data, and the quote-assist pilot came close to being one of them.
If you're the one signing off on that order of work, whether as CEO, CFO, CTO or CIO or as the data and engineering lead who has to carry it out, this guide is written for you. We'll keep coming back to the distributor as we go through the readiness scorecard, the AWS reference architecture, the US and EU compliance picture, what it all costs, a gated plan and the questions to ask a partner.
It's the part of your data estate that's been made safe for software that reasons and acts by itself. Before the distributor's quote-assist agent could answer anything, somebody had to vouch for the data underneath it. Was the price it pulled current? Did "margin" mean what the agent assumed? Was the salesperson asking allowed to see that customer, and could anyone show where the number came from?
A dashboard survives messy data because there's a person between the number and the decision. An agent can't. Had the distributor's returns agent been live, it would have read a stale credit limit, trusted it, and issued the credit memo before anyone looked. By the time finance noticed, the customer would already have had the credit.
There are seven, and the figure lists what each gives the agent, what breaks without it and the AWS services we'd typically use.
Every one of them costs less to design in than to bolt on. Had the distributor launched the quote-assist agent first, adding lineage later would have meant re-instrumenting every pipeline the agent touched, and adding access control late would most likely have meant learning in an incident review which catalog files the index had already exposed.
"Agents fail at the data layer before they fail at the model layer." Models get swapped every few quarters, while the data foundation keeps paying off with each new agent you put on it.
By scoring it. We'd run an AI data readiness assessment on just the domains your first agents will touch, since whole-enterprise maturity models take months and rarely change what gets built next. We run the scorecard below with a client's data and engineering leads in one working session, scoring each of eight dimensions 0 (red flag), 1 (partial) or 2 (ready).
We ran it for the distributor's two most important domains. Customer data came out at 5 out of 16, which is the stop range: nobody owned it, sales, finance and e-commerce each defined an active customer their own way, there was no lineage, and access was still managed in that spreadsheet. The product and catalog domain did better at 10. Its PDFs were inventoried and had quality checks, and the one real problem was the SharePoint copy, which had lost its permissions. That put it in the 7-to-11 band, where most mid-market and lower-enterprise teams land and where we'd close the red flags on that one domain and start building. Twelve or more would have meant going straight to the agent build, with the effort shifting to evaluation and observability.
Not all four of the distributor's agents needed the same foundation, because agentic AI data requirements climb with every permission you grant. Lined up against the levels in the figure, they looked like this:
|
The distributor's agent |
Level |
What it needed before going live |
|---|---|---|
|
A product assistant answering inside-sales questions from catalog PDFs and spec sheets |
1, read-only |
A document inventory, classification and retrieval that respects SharePoint permissions |
|
A margin copilot for the pricing analysts |
2, analytics |
One definition of margin in the semantic layer, quality checks and row-level security by region |
|
Credit pre-screening for new accounts |
3, decision support |
Lineage on every input and a credit analyst reviewing each recommendation |
|
A returns agent that could issue credits up to $500 |
4, transactional |
Tool-level permissions, a spend limit, rollback and a full audit trail |
We'd ship the product assistant first. It could go live within weeks on the catalog alone, while the credit and returns agents were the ones where a weak foundation would turn into an audit finding or a very public mistake.
Six layers with governance running through all of them, as in the figure. When we sketched that stack for the distributor, the layers were the easy part to agree on. SAP, Salesforce and the e-commerce database would feed Iceberg tables through change data capture, the catalog PDFs would be indexed next to them, and the agents would sit at the bottom. The argument was about governance, which the IT team wanted as a review step at the end. We put it in as a plane that every layer reports to instead, so the access, lineage and quality rules are written once and the fourth agent inherits them exactly as the first did.
Ammar Nizami's piece on the data lakehouse for agentic AI explains why the storage layer carries so much weight here: "Agents query nothing like dashboards do."
The five in the table. We'd want all of them settled in the first two weeks, since getting one wrong is the usual reason a foundation needs rebuilding 18 months later, right when the second or third agent is due.
|
Decision |
What we'd lock in at the distributor |
Why it's hard to change later |
|---|---|---|
|
Table format |
Apache Iceberg on Amazon S3, so analytics, ML and agents read one governed copy |
Every engine and pipeline gets written against it |
|
Where entitlements live |
Row and column policies tied to identity in AWS Lake Formation |
Filtering results after retrieval leaks the first time an edge case shows up |
|
Who defines meaning |
One business glossary in Amazon SageMaker Catalog, signed by the sales and finance owners |
Otherwise five agents invent five definitions of revenue |
|
How fresh each source is |
Change data capture for order and inventory status, nightly batch for catalog content |
Real-time feeds cost money every hour they run |
|
Where tool permissions sit |
Amazon Bedrock AgentCore Gateway and Identity, outside the prompt |
A system prompt is an instruction, and instructions get ignored |
The table format is the call we'd least like to revisit. Ammar's guide to Apache Iceberg on AWS puts the stakes plainly: "Picking a table format is really picking who controls your data."
Gautham B, a forward-deployed engineer in our data engineering practice, describes the payoff in an analysis of enterprise AI agents in production: "When lineage, classification and access rules already live in the data layer, retrieval can just inherit them." That inheritance is the economic case for foundation first. Governance gets solved once, and every new agent rides on it.
At the distributor, the auditors asked the same three questions whether they were looking at SOX controls on pricing and credit or at GDPR for the German customers and staff: which data did the system use, who could reach it, and could the company rebuild a decision afterward? We'd answer them in the data layer itself, so each new agent starts out compliant instead of waiting for its own review.
Europe moved its timeline this year. The Digital Omnibus on AI, published in the Official Journal on July 24, 2026, pushed the AI Act's high-risk obligations for Annex III systems to December 2, 2027, and those tied to products under EU sector legislation to August 2, 2028. We'd treat that as build time, because the data governance duties in Article 10 take longer to stand up than most legal teams expect.
On access control, mostly. In IBM's 2026 Cost of a Data Breach research, roughly one in five organizations reported an AI-related breach, and 92% of those lacked proper AI access controls. That matches what we find in readiness assessments: retrieval indexes built from shared drives, with no record of who was allowed to read the source files. The distributor's quote-assist pilot was one of them.
Erasure is the quieter risk. If a customer in Munich asked the distributor to delete their data under GDPR, the record would have to come out of the warehouse and also out of the snapshots, the feature store and the vector index, and in our experience the last three are the ones teams forget.
We'd check six things for any agent going live at the distributor:
When those controls live in the platform, adding an agent stops being a compliance project. One regulated financial services client got exactly that result: "new analytical capabilities do not require separate compliance reviews."
We won't put a number on it before seeing your sources, and we'd be wary of anyone who does. The price moves with how many data domains the first agents touch, how fresh that data has to be and how much legacy gets retired along the way. What we can show is where the money goes at a company like the distributor, which is enough to pressure-test any estimate that lands on your desk.
|
Cost line |
What drives it at the distributor |
How we'd keep it in check |
|---|---|---|
|
Engineering hours |
Mapping the SAP, Salesforce and e-commerce schemas, then building and validating pipelines |
Let agents do discovery and test generation, and spend senior hours on design calls |
|
Storage and compute |
The legacy warehouse plus extracts copied into three sandboxes |
One open-format copy, and a dated plan to retire the old warehouse |
|
Data movement |
Six sources, two of them needing near-real-time order status |
Real-time only where an agent acts on live state |
|
Governance tooling |
Cataloging and classifying the customer and product domains, including every PII field |
We'd start with what AWS already offers and buy a third-party tool only for a gap |
|
Retrieval |
Tens of thousands of catalog PDFs, with suppliers sending updated ones every month |
We'd index what people are allowed to see and actually ask about, and skip the rest |
|
Agent inference |
Steps per task, model choice and context size |
Clean grounding shortens reasoning chains, and simple steps go to smaller models |
|
Rework |
Fixing data after agents are live |
Gate the agent build on the readiness score |
Over any horizon longer than one pilot, yes. The distributor's first quote-assist build looked cheaper until security review sent it back, and each later agent would have had to solve access, quality and lineage on its own. The figure compares the two sequences once you're past the demo.
For most CFOs, it's already in the IT run rate: legacy database licenses, a warehouse renewal nobody wants to sign again, and an integration tier that bills hours to keep itself alive. Our series on funding AI from your existing budget makes the point in one line: "The AI budget is trapped, not missing."
In practice that means timing the foundation work to renewal dates. The distributor's warehouse contract renews next September, so a move to an open lakehouse that cuts over before then would fund a large share of the foundation. Miss that date and it pays for another term while building in parallel. We'd put the renewal dates on the same slide as the agent roadmap, and plan the data platform modernization on AWS around them.
One data domain at a time, with a signed gate closing each phase, and the first agent in production on that domain before the scope widens. For a single well-scoped domain, the plan below usually runs about 12 weeks from kickoff to a production agent, which is also the median time to production across our engagements.
Two details in that plan do most of the risk reduction. The hard part goes early, because, as our how we work page puts it, "If a plan is going to break, it should break in week three on a proof rather than in month five on a deadline."
And the parallel run compares old and new systems field by field on real data, so nobody has to take the new numbers on faith. The phases follow the same gated structure we use on every modernization engagement, with a customer signature closing each one.
The sequence holds outside regulated work, too. In Hibu's conversational analytics build, the transcript platform and search connector came first and the assistant went on top of them. The result, in the case study's words: "Answers that waited 2 to 3 days in an analyst queue now arrive in under 5 minutes."
The distributor made most of them before it called us. Its data team spent the first quarter comparing platforms before anyone had named an agent use case, and the first plan tried to cover every domain at once, which would have turned a quarter's work into three years. Someone copied the SharePoint catalog into a vector store for the quote-assist demo, which looked good right up to the security review. The definition of an active customer was left to engineers, when only a sales or finance owner could sign it, and nobody had written an end date for the old warehouse.
We'd have fixed all five the same way, by starting from one agent, working backward to the one domain it needed and writing every end date into the plan.
Usually it comes down to who carries the delivery risk and how soon you need production. The distributor's own engineers were good, but they were also the people keeping SAP and the e-commerce site running, and the last modernization the company had tried with a big integrator ran well over budget. That's a common story among the mid-market and lower-enterprise teams we talk to.
|
The distributor's question |
In-house team |
Traditional integrator |
Agent-native firm |
|---|---|---|---|
|
"When does the first agent reach production?" |
Whenever competing priorities allow |
After a long discovery phase |
In weeks, because agents run discovery, mapping and test generation |
|
"How are we billed?" |
Internal headcount |
Time and materials, by the hour |
Fixed date, fixed scope, fixed fee |
|
"Who pays if it slips?" |
We do |
We do |
The firm, for delays inside its control |
|
"Who's actually doing the work?" |
Our own engineers, when they can get away |
Mostly junior consultants, managed in a pyramid |
A few senior engineers who own the architecture and the cutover |
|
"Where does audit evidence come from?" |
Built by hand, often late |
Delivered as documents |
Generated by the validation harness and lineage |
|
"When does it fit?" |
Spare platform capacity and no deadline |
When a big-brand signature is a procurement requirement |
When production has to land on a date the CFO can budget against |
Whichever route the distributor took, we'd tell it to get the ship date and the acceptance numbers written into the contract before anyone signs. It's the same standard our delivery commitment holds us to: "A single date, not a range and not a target quarter revisited at each steering committee."