Healthcare Analytics on AWS: From Data Lake to Agentic AI

We've been building on AWS since 2008, and much of that work has been data platforms for regulated companies. In healthcare, the pressure on those platforms has changed in the last two years. Claims teams are asked to move faster while denials draw more scrutiny: insurers on HealthCare.gov denied 19% of in-network claims in 2024, according to KFF, and fewer than 1% of those denials were appealed. Providers carry the other side of it. In the AMA's latest survey, physicians said they complete an average of 40 prior authorizations a week, which takes about 13 hours of physician and staff time.

This guide is for the data, analytics and engineering leaders at payers, providers and health-tech companies who are deciding where their next AWS dollar goes. It follows one made-up but typical company from a stalled claims lake to a governed platform, and from there to agent-assisted claims work. The company is a regional health plan with about 450,000 members across commercial, Medicare Advantage and Medicaid managed care lines, processing about 9 million claim lines a month. It built its claims lake on Amazon S3 in 2021. Every night, 837 claim files from two clearinghouses land there, along with extracts from a core admin system on SQL Server and ADT messages from three hospital partners, while the 835 remittance files the plan sends to providers arrive on their own schedule. Last year it tried a prior authorization triage pilot for Medicare Advantage imaging requests, and security stopped it before it reached a reviewer.

Why do healthcare data lakes stall before they reach claims automation?

Mostly because they were built to hold data cheaply, not to say what it means. The health plan's lake did the first job well, keeping seven years of claims at a low price. What it couldn't do was tell anyone which version of a claim was current, which records belonged to the same member, or who was allowed to see what, and claims automation needs all three at once.

Claims data arrives late and loses its meaning

Each night the lake flattened the 837s into wide tables and loaded remittances later, on their own schedule. When a claim was adjusted, reversed or resubmitted, the change arrived as a new row with no reliable link to the original, so analysts rebuilt claim history inside every query, and rarely the same way twice. In the second quarter, finance and medical economics reported paid amounts that differed by about $2.3 million. Both teams had followed the data correctly. Neither could say which number an automated system should act on.

Member and provider identity never resolves

The same member showed up under different IDs in eligibility, claims and the hospital ADT feeds, and one orthopedic group appeared under its NPI, its tax ID and two internal contract numbers. Joins across those sources dropped rows without warning. When the data team finally counted, about 6% of claim lines fell out of the member-level views, which meant any model trained on those views had been learning from a partial picture.

Governance runs as a ticket queue

PHI access was granted per S3 bucket or per IAM role, through a ticket. A new analyst waited about three weeks, so teams copied extracts into sandboxes to move faster, and the audit trail split with every copy. When the prior authorization pilot came up for security review, the question that sank it was simple: which fields would the model see? Nobody could answer from the catalog, because the catalog didn't know.

None of those are storage problems. They're problems of modeling, identity and governance, which is why more compute in the lake wouldn't have fixed any of them. The figure shows what changes at each stage.

What does modern healthcare analytics on AWS look like?

It keeps the lake's economics and adds structure: transactional tables, one governed catalog, and curated layers built around claims, members and providers. Our HIPAA-aligned healthcare data platform work starts from the same principle, which the team puts as "Compliance engineered into the architecture, not audited after the fact." Every AWS service named in this guide is on AWS's list of HIPAA eligible services. That's necessary but not enough, since eligibility still depends on a signed business associate agreement and on how each service is configured.

How should EDI, HL7 v2 and FHIR data come in?

Side by side, with the raw form kept. At the health plan, we'd parse the 837, 835, 270/271 and 278 transactions with AWS B2B Data Interchange, which handles HIPAA X12 5010 transactions, and land both the raw file and the parsed output, because a provider dispute or an audit will eventually ask for the original. The hospitals' HL7 v2 ADT and ORU messages would stream through Amazon Kinesis or Amazon MSK and sit next to the claims data once they're normalized. For FHIR, AWS HealthLake stores resources in the FHIR R4 format, and we'd export them to the lake for analytics rather than query the FHIR store directly, since the FHIR store serves applications and APIs while the analytical tables serve everyone else. AWS DMS would read changes out of the SQL Server core admin databases as they happen, instead of waiting for nightly extracts.

Why use Apache Iceberg for claims tables?

Because claims change after they're paid. Iceberg tables on Amazon S3, self-managed or through Amazon S3 Tables, which run compaction and snapshot maintenance for you, take row-level updates for adjustments and reversals. They also keep snapshots you can query, so the plan can answer the question auditors and provider relations both ask: what did we know when we paid this claim? Ammar Nizami, a senior forward-deployed engineer in our data engineering practice, puts the longer-term stakes in his piece on Apache Iceberg on AWS: "Picking a table format is really picking who controls your data."

We'd organize the tables in three layers. The raw layer holds source files and messages exactly as they arrived, encrypted and closed to almost everyone. The conformed layer holds parsed, typed and deduplicated records with resolved member and provider keys, which AWS Entity Resolution can generate with rule-based or machine learning matching. The curated layer holds what people actually use: claim-line facts, episode views, provider performance and eligibility snapshots.

How does PHI access become a policy instead of a ticket?

Every table gets registered in the AWS Glue Data Catalog, and AWS Lake Formation enforces access with tag-based access control. At the health plan, we'd tag columns by sensitivity, separating direct identifiers, quasi-identifiers, clinical codes and financial amounts, and grant access by tag rather than by table. A new analyst on the provider network team would get the provider and financial tags on day one and never see a member's name. That one decision would have answered the question that stopped the pilot.

Which AWS service handles which claims job?

Once the platform looks like this, the agent layer gets much easier, because the governed tables have already done the hard work on meaning, identity and access.

How do a legacy lake, a modern platform and an agentic layer compare?

The table uses the planning ranges we'd apply to a plan the size of the example, anywhere from 5 to 50 million claim lines a month. They're for planning, not quotes, and the fourth row is the one to read twice.

The last column's rework figure is the trap. A plan that goes straight from a legacy lake to agents tends to pay twice, once to ship the pilot and again to rebuild it when security and compliance catch up, which is roughly what happened to the health plan's prior authorization pilot.

Where does agentic AI fit in claims operations?

Wherever a person already gathers evidence before making a call. Dashboards tell a claims team what happened. An agent reads a claim, its history and the relevant medical policy, uses tools to collect the evidence, and proposes an action that a person accepts or rejects. Ammar's piece on the data lakehouse for agentic AI explains why that changes the platform's job: "Agents query nothing like dashboards do." They make many small, specific reads, and each answer has to trace back to a row.

Agent-assisted adjudication

Take a pended claim at the health plan, say a spine surgery held over a contract rate question. An agent would pull the claim lines, the member's eligibility, the provider's contract terms and the applicable medical policy, then draft a recommendation with a citation for each source. The adjuster reads one structured summary instead of opening six screens, and accepts, edits or rejects it. The agent never finalizes payment.

Denial prediction and pre-submission review

On the provider side, a model scores claims for denial risk before they go out, and an agent explains the likely reason, whether that's a missing modifier, an authorization mismatch or a diagnosis that doesn't support the procedure, then suggests a fix for a coder to review. The health plan would run the mirror image, flagging denials likely to be appealed so a reviewer looks again before the letter goes out. That matters more than the appeal rate suggests, because with fewer than 1% of denials appealed in the KFF data, most wrong denials are never corrected by appeals at all.

Prior authorization triage

This is where the regulatory clock runs. Under the CMS Interoperability and Prior Authorization Final Rule (CMS-0057-F), Medicare Advantage organizations and Medicaid and CHIP programs, including their managed care plans, have had to send decisions within 72 hours for expedited requests and seven calendar days for standard ones since January 1, 2026, with a specific reason for every denial. They also have to run a Prior Authorization API from January 1, 2027. At the health plan, an agent would classify each incoming request, check whether the documentation is complete, match it against coverage criteria and route clear approvals to a fast-track reviewer. Denials would stay with a clinician.

Member and provider servicing

Service agents can answer claim status questions, explain an explanation of benefits or a deductible accumulator, and draft provider correspondence. At the health plan they'd read from curated tables rather than search free text across raw files, which keeps their answers consistent with what finance reports.

Where shouldn't agents decide yet?

Three places, at the health plan and at most others. Final denials on medical necessity stay with clinicians, and physicians have reason to be wary, since six in 10 told the AMA they worry AI may push denial rates higher. Anything that writes back to the core claims system needs a human approval step. And any workflow whose data isn't governed and reconciled waits until it is.

Those limits are how we build AI agents for regulated work, with narrow tools, explicit approvals and a trace for every decision. They're also how KlearTrust got to production. The HIPAA-compliant agentic claims platform we shipped in 12 weeks replaced a six-month manual claims-review cycle, clears about 80% of claims inside the agent's authority, and passed HIPAA and internal audit on first review. As the case study puts it, "The auditor signed off on the architecture, not on a separate test."

How do you build it safely with PHI controls and human review?

Layer by layer, with the controls decided before the first pipeline runs. A clinical AI program can't treat audit as the last step, and our healthcare practice puts the stakes plainly: "A clinical AI initiative that improves throughput and creates an audit incident is a strategic loss."

What does the reference architecture look like?

If you started an agent project in 2024 or 2025 on Bedrock Agents, note that AWS renamed it Bedrock Agents Classic and closed it to new customers on July 30, 2026. Existing agents keep running, but new builds go on AgentCore.

If the core admin system still runs on an aging relational database, application and database modernization is often what makes timely change data capture possible. The same work can give agent tools clean APIs to call, so they don't have to scrape legacy claims screens.

Which PHI and PII controls does each layer need?

Five, mapped to the HIPAA Security Rule's safeguards, and the figure shows where each one sits. At the health plan, every data domain would get its own customer-managed KMS key, with TLS everywhere and key policies that keep the raw zone apart from curated access. Direct identifiers would be tokenized in the conformed layer, and only roles that need a member's name could detokenize it. Lake Formation tags would sit on columns rather than tables, with separate roles for analysts, model training and the agent runtime. Agent tools would return only the fields a task needs, so a member's name or Social Security number never enters a prompt unless the task requires it. CloudTrail and Lake Formation logs, plus a trace of every agent invocation with its inputs, tool calls, outputs and the reviewer's decision, would be kept as long as the plan's records policy says.

The bar is rising, too. In the Security Rule update HHS proposed in December 2024, encryption of ePHI at rest and in transit and multifactor authentication would become required, with limited exceptions. It hasn't been finalized and the current rule still applies, but we'd design to the proposal now, since retrofitting encryption later costs far more.

How do guardrails and human review work together?

By action type. Reading and summarizing need no gate, a recommendation needs an adjuster or clinician to accept it, and a write-back to the claims system needs dual approval or isn't allowed at all. At the health plan, Amazon Bedrock Guardrails would filter sensitive information, block denied topics and run contextual grounding checks, and AgentCore Policy, generally available since March 2026, would stop a tool call before it runs if it falls outside what the agent may do. Any recommendation without a citation to a source row or policy passage would be rejected before a reviewer saw it. Low-confidence cases and policy conflicts would go straight to a person, with no agent draft to anchor them.

What does it take to run the agents day to day?

Mostly discipline. We'd tag agent workloads separately and track cost per claim reviewed, with budget alarms on model spend. Responses have to fit the adjuster's rhythm, which means seconds, and a timeout should drop the case back into the manual queue. Each workflow gets a kill switch that the team practices using before go-live. Prompts, tools and policies are versioned together, and evaluations re-run whenever the medical policy content changes, which at a health plan happens often.

What team and rollout plan does a claims agent program need?

Who should be on the team?

A small pod, not a large program. At the health plan, we'd staff it with a data platform lead who owns tables, identity and lineage, one or two agent engineers who own tools, orchestration and evaluations, and a claims operations expert who owns the workflow, the acceptance criteria and reviewer training. Security and compliance would sit with the pod from the first week rather than review it at the end.

Why run data and AI governance in one forum?

Because the gaps sit between committees. The group that approves a new column tag at the health plan would also approve which agent tools may read it. Separate data and AI committees leave room for a tool to read a column nobody tagged, and auditors tend to find that room quickly.

How should the rollout be sequenced?

In three steps, each with a gate. The pilot runs for six to eight weeks on one claim type and one queue, in shadow mode: the agent drafts, the humans decide, and the team measures how often they agree. Limited production follows for six to ten weeks, with a subset of adjusters working in assisted mode while throughput, accuracy and override rates are tracked. After that, claim types are added one at a time, each behind an evaluation gate. How we work follows the same logic: "If a plan is going to break, it should break in week three on a proof rather than in month five on a deadline."

What should be ready before a claims agent goes into shadow mode?

We'd want five things in place at the health plan before the first shadow run:

• A curated claim-line table with resolved member and provider keys
• Column-level PHI tags enforced through Lake Formation
• A few hundred labeled historical decisions per claim type, for evaluation
• A named reviewer group and an override process
• An agent trace retention policy that compliance has approved