Enterprise Generative AI: From Pilot to Production on AWS

Key takeaways

• Most backlogs hold one or two real candidates. The insurer in this guide has 12 pilots, and the one it should ship scores 15 out of 18 on the readiness scorecard. The board's favorite scores 9.
• The demo is rarely what's missing. The board's favorite, a claims summarizer, impressed everyone in the room and still couldn't go live, because nobody in claims owned it and its data came from a single export.
• Authorization is the layer pilots skip. At the insurer, the intake agent's IAM role lets it create a submission and nothing else. Binding a policy or sending a decline is out of its reach, whatever a prompt says.
• Our median is 12 weeks per workflow. The plan runs 11 to 20, and submission intake would sit near the short end, since its data and its baseline already exist.
• Compliance moves faster when it starts first. KlearTrust's claims platform went live in twelve weeks and passed HIPAA and internal audit on first review.

We've been building on AWS since 2008, and these days most of that work is generative AI for companies where an auditor reviews the system before it goes live. We wrote this for the CIO, CTO, or chief data officer who has a dozen pilots, a board asking which of them pay off, and no good answer yet.

Enterprise generative AI means large language models doing part of someone's actual job. Think of a model that reads a broker's email and files the submission, working on company data, under the same security rules as every other system, with a number in finance that shows whether it's paying off. Most programs get the first part working. The rest is where they stall.

It isn't for lack of trying. In McKinsey's 2026 State of AI survey, 44% of respondents said AI is now scaling across their enterprise, up from 38% the year before. The return hasn't kept pace, though: only 37% attributed at least some EBIT impact to it, about where that number sat a year earlier. Deloitte's 2026 State of AI in the Enterprise report found the same gap from another angle, with only 25% of leaders saying they'd moved 40% or more of their AI pilots into production.

To keep it concrete, we'll follow a made-up but typical US property and casualty insurer. It writes about $2.5 billion in premium a year, and its core systems run on AWS. Over 18 months, its innovation lab has built 12 generative AI pilots. The claims-notes summarizer wowed the board last quarter.

The underwriting team, meanwhile, built a submission-intake prototype that reads broker emails and ACORD applications. There's also a billing chatbot, a subrogation letter drafter, a fraud-referral triage tool, a coverage Q&A assistant for the call center, a marketing copy generator, an IT helpdesk bot, vendor contract review, reserve recommendations, and an HR policy bot. None of the 12 runs in production, and the CFO can't tie any of them to a line on the P&L.

Why Do Enterprise Generative AI Pilots Stall Before Production?

Mostly at the handoff, when a demo built on clean sample data runs into live systems, real permissions, and an auditor. In the stalled programs we've been asked to rescue, the model was almost never the problem. The data pipelines, the permissions, and the operating model around it were.

Suraj Chaudhary, made the same point in his analysis of the agentic AI production path: "The attrition between piloting and production is not a model capability problem — it is a distributed systems problem wearing a machine learning label."

Here's what changes when the insurer's claims summarizer has to survive a Monday morning in production:

If your backlog is stuck in AI pilot purgatory, the gaps in the middle of that figure are usually why. Here's how the claims summarizer ran into all five. The innovation lab built it, and nobody in claims operations signed up to run it, fund it, or answer for its numbers. Its knowledge index came from that one March export, so by summer its summaries were missing anything that had happened since. What it was allowed to see was written into the prompt rather than enforced, and a tester got it to summarize a claim from another region just by asking. Nobody had worked out what a summary would cost at 40,000 claims a month, either. Security saw the design for the first time in week 11, found Social Security numbers in the prompt logs, and sent it back.

There's a people side too. In IBM's 2026 CEO Study, CEOs estimated that only 25% of their workforce uses AI regularly as part of their job. Adjusters won't open a separate browser tab to read a summary, so the summarizer would have to show up inside the claims screen they already work in.

 

Which Pilots in Your Generative AI Backlog Should Reach Production First?

At the insurer, it's submission intake, even though the board would pick the claims summarizer. The pilots worth shipping first score well on business value and on production readiness at the same time, and that's rarely the board's favorite. Most leadership teams try to scale generative AI by nudging all twelve pilots forward a little, which spreads the best engineers thin and gets nothing over the line. We'd ship what lands top right and stop funding the rest.

We score every backlog against the same six criteria before anyone writes production code. Each one scores 0 to 3, and anything under 10 out of 18 isn't ready, however good the demo looked. Here's how the insurer's two most talked-about pilots would come out.

The claims summarizer loses points almost everywhere except volume. Only the innovation lab stands behind it, so ownership is a 1. Ask about the baseline and the answer is "it'll save adjusters time," another 1. It reads nightly exports from a 20-year-old claims system, which is a 1 for data, and bodily injury files carry medical details, a 1 for risk. Its authority is narrow, summarize and nothing else, so that's a 2, and at about 40,000 claims a month, volume earns a 3. It totals 9.

Submission intake totals 15. Its owner is the VP of commercial underwriting, and her team already knows the baseline: 25 minutes a submission, 1,500 a month. Those are two 3s. The agent can reach the submissions inbox and the policy admin API, a 2. Everything it's allowed to do is written down, extract, check appetite, and route, while declines stay with underwriters, so authority gets a 3. Broker and insured PII is in scope but card data isn't, a 2, and volume is a 2.

The scores put each pilot in a quadrant, and each quadrant gets a different decision.

At the insurer, submission intake lands top right on its own. The claims summarizer, the board's favorite, goes to "fix the foundation": fund a claims data feed and an owner in claims operations, then rescore it next quarter. Marketing copy and the IT helpdesk bot are cheap to run and worth little as custom builds, so an off-the-shelf tool covers them. The HR policy bot gets retired. That's a normal result. In most backlogs we review, one or two pilots are ready to ship, and that's where the budget and the strongest engineers should go this quarter.

Submission intake has the same shape as the patent automation system we built on Amazon Bedrock for Sterne Kessler Goldstein & Fox: high volume, document-heavy, with a hard baseline. Attorneys there spent 8-12 hours on each Office Action response. After a four-week assessment, the multi-agent system brought that to 2-4 hours per response, with human review at every decision point. The case study puts it this way: "The attorneys still hold the pen. The agents just stop them from reading the same prior art for the third time this week."

What Does a Production Architecture for Enterprise Generative AI Look Like on AWS?

Seven layers, and the model is just one of them. Pilots usually build two of the seven well, the model call and a thin retrieval step, and stub out the rest. The stubs are what break in production.

Here's what the layers would do for the insurer's submission-intake agent. A broker email lands with an ACORD application and loss runs attached. The agent runs on Amazon Bedrock AgentCore. It takes the insured's name, class of business, and requested limits off the forms and checks them against the underwriting guidelines, which live in a knowledge base. If the risk is in appetite, it creates the submission in the policy admin system and routes it to the right underwriter. If it's out of appetite, the agent drafts a decline and an underwriter decides whether to send it, since turning business away is a call with consequences. Each step goes into a log along with the version of the guidelines it used.

Authorization is the layer most teams get wrong. A guardrail can stop a model from saying something it shouldn't, but it can't stop an agent from doing something it shouldn't. Gautham B, one of our forward-deployed engineers, put it in six words in his breakdown of Amazon Bedrock agents in production: "Guardrails inspect language. Authorisation governs consequence." For the insurer, that means the agent's IAM role can create submissions and nothing else. It can't bind a policy or send a decline, however the prompt is worded.

Two AWS changes this year affect new builds. Amazon Bedrock Agents became Agents Classic and closed to new customers on July 30, 2026, so new agents go on AgentCore. The second one matters for the insurer's call center, whose coverage Q&A reads policy forms out of SharePoint. Since September 30, 2026, customer-managed knowledge bases can't add new Confluence, SharePoint, Salesforce, or Web Crawler connectors. So the call center's Q&A would go on the Bedrock Managed Knowledge Base, which is where AWS now sends new builds.

Does the workflow need RAG, fine-tuning, or an agent?

The simplest one that meets the requirement. Each step up adds cost, evaluation work, and audit surface.

Retrieval is only as good as the data under it, and the insurer's claims notes sit in a 20-year-old system that only exports nightly. That's the whole reason the claims summarizer scored a 1 on data access. For a case like that, we'd start with a targeted slice of data platform modernization, scoped to the sources the first workflow needs, so the knowledge base reads current, governed data rather than last night's export.

How Do You Keep Enterprise Generative AI Secure and Compliant in Production?

We'd map every control to a piece of evidence before the build starts, and walk the auditor through the architecture early in design. Compliance bolted on at the end is the most common reason a promising pilot gets sent back to square one.

Waiting has a cost too. While the insurer's approved tools sit in pilot, some adjusters and underwriters are probably pasting claim notes into whatever chatbot they have open. IBM's 2026 Cost of a Data Breach report found that incidents involving employees' shadow AI more than doubled, to 43%, and it put the global average breach at a record $4.99 million.

For the insurer, the list starts with its state regulators. The NAIC's model bulletin on insurers' use of AI, adopted in December 2023, has now been adopted in 29 jurisdictions, and it expects a written AI governance program that covers third-party systems as well. PCI DSS comes in through the billing chatbot. Most enterprises run into some mix of the rows below. For each one, we've put down the question an examiner or auditor would actually ask.

Which frameworks apply, and what evidence do they expect?

Three of those rows changed recently. For banks, SR 26-2 replaced SR 11-7 in April 2026. It leaves generative and agentic AI models out of its scope, so a bank's examiners will judge those against its broader risk management and governance instead. California added rules on automated decision-making technology that took effect on January 1, 2026. Companies that use it for significant decisions, such as lending or hiring, get until January 1, 2027, to comply. If the insurer ever sold policies in Europe, its billing chatbot would already have to tell customers they're talking to AI, since the AI Act's transparency duties have applied there since August 2, 2026.

The Digital Omnibus on AI pushed most high-risk obligations back, to December 2, 2027, for Annex III systems and August 2, 2028, for AI in regulated products. Annex III covers pricing and risk assessment in life and health insurance, not P&C, so the insurer's underwriting agent wouldn't land there. We still wouldn't read the delay as a reason to wait, since the Act's documentation and logs are mostly what NIST AI RMF and internal audit already ask for, and the insurer can build them once for both sides of the Atlantic.

What controls belong in every production deployment?

We put the same five controls into every production build, and they're easiest to see on a single submission. Say a broker in Ohio sends in a small roofing contractor, and the loss runs include the owner's driver's license number. The email and its attachments stay in US Regions. Amazon Bedrock doesn't use the prompt or the output to train models, which is usually the first thing legal asks about. The license number is masked before anything reaches the logs or the vector store. Roofing is outside the insurer's appetite. Declining is an underwriter's call, so the agent only drafts it. An auditor who asks about that submission next year will find all of it in the decision trace, down to the guideline version the agent checked and the model version it ran on.

This is how a regulated workflow gets through review on the first pass. KlearTrust had two prior automation pilots stall at the PHI and audit handoff, because PHI handling and audit-defensibility added scope its staffing model couldn't absorb. The HIPAA-compliant agentic claims platform we shipped replaced a six-month manual claims cycle in twelve weeks, and it passed HIPAA and internal audit on first review. Its auditor, according to the case study, "signed off on the architecture, not on a separate test."

Banks and insurers usually wait longest on model risk and data lineage. We'd build lineage and validation into the first release, since retrofitting them is what drags out remediation, and the decision trace doubles as validation evidence when an examiner asks how a model reached its output. Our financial services practice takes that approach across FFIEC- and SEC-regulated environments.

How Much Does Enterprise Generative AI Cost to Run in Production?

Price it per resolved task, and put that number next to the fully loaded cost of the process it replaces. Token prices and license fees on a rate card don't tell your CFO much on their own.

Most pilots get priced as if they were chatbots, where each question costs one model call and the bill is easy to forecast. The intake agent doesn't work that way. On one broker email with three attachments, it might call a model a dozen times before the submission is filed.

Finance teams have started to notice, too. McKinsey found the same: about a fifth of its respondents said AI operating costs, tokens included, were already limiting how much they used AI.

Cost per resolved task = (inference + retrieval + orchestration + observability + human review + amortized build) ÷ tasks resolved without rework

The bottom of that fraction is where the math goes wrong. If an underwriter has to redo a submission the agent got wrong, it doesn't count as resolved, but everything it cost stays on top. Leave that out and the dashboard will look better than the P&L does.

What would the business case look like for submission intake?

Here's a planning scenario built on the submission-intake workflow, so swap in your own numbers. The insurer's underwriting desk handles 1,500 submissions a month, each one takes 25 minutes, and an hour of that work costs about $45 fully loaded.

Over a year, that comes to $337,500. Say the agent resolves 80% of them within its authority. The other 20% still land on an underwriter's desk, and that's $67,500 of the old cost that doesn't go away. What's left, $270,000 a year, is the room the system has to fit in.

Then we'd check the other side. If a year of inference, retrieval, orchestration, and observability, plus the build spread over the system's useful life, doesn't fit inside that $270,000 with room to spare, the case doesn't work. Underwriter time on the other 20% stays out of that total, since the gray bar already counts it. At a run cost of $3 per resolved submission, the 14,400 submissions the agent resolves in a year would cost about $43,000 to run, which leaves plenty of room for the build. When the numbers don't fit, we'd narrow the scope or send the pilot back to "fix the foundation." We use the same kind of scenario when scoping generative AI consulting engagements, with the client's own baseline in place of these numbers.

Where does the budget come from?

In most enterprises, a good part of it is already in the IT budget. Look at the legacy database license coming up for renewal, the data warehouse nobody builds on anymore, and the integration tier that bills hours just to keep itself running. Each of those lines can be reallocated. That's how most of the companies we work with end up funding AI from their existing budget, and it makes generative AI a capital allocation decision the board can follow, rather than an R&D line that gets cut in the next downturn.

What Is the Fastest Path From Generative AI Pilot to Production?

One workflow at a time, in four phases, with a signed gate at the end of each. For submission intake, we'd expect 11 to 20 weeks from the scoping call to handoff. Platform-first programs tend to spend their first year building infrastructure nobody uses yet.

Every exit gate is a signature from your side. At the insurer, the VP of commercial underwriting signs off on production, because she owns the number the agent is supposed to move. Her signature is how ownership leaves the innovation lab and lands in operations.

Two habits make the biggest difference. The first is building the golden set before anyone tunes a prompt. For submission intake, that's a few hundred real submissions with known correct routing, and every release has to beat it. Without one, quality debates turn into opinion contests. The second is expecting the real requirements to show up after go-live. Brokers will send things in week one that no workshop predicted, so budget engineering time for the four weeks after cutover and keep the people who built the system on the hook for them.

It's the same gated sequence behind how we ship to production, with a median of 12 weeks to production across our engagements.

Should You Build In-House or Bring in a Generative AI Consulting Partner?

We'd look at who ends up carrying the delivery risk. A company with a mature AI platform team and a long runway can build in-house and learn as it goes. The insurer isn't in that spot. Its data team is strong, but nobody there has put an agent into production, so for submission intake it would bring in a partner whose contract carries part of the risk.

Most buyers haven't worked with forward-deployed engineers before. They embed with your team, write the production code, and stay accountable through cutover. Gautham B, who does the job himself, describes it in his piece on the forward-deployed engineer model: "That's the detail most descriptions of the role get wrong: an FDE's output is a running system, not a recommendation."

What should you ask any partner before you sign?

Say the insurer puts three firms through that test for its generative AI implementation. The first can't say when submission intake would be in production, only that a twelve-week strategy phase comes first. The second puts a date in its proposal, but it bills time and materials, which means a slip costs the insurer money and costs the firm nothing. The third writes the date into the statement of work and says who pays if it slips for reasons the firm controls. It also shows how much of the work its agents would handle and how much its analysts would. It's also the only one that can describe the audit evidence pack on day one and list what the insurer owns at handoff: code, evaluation sets, runbooks, and on-call. The insurer's last question would matter most: can it speak to a named customer the firm has put into production in insurance?