Most enterprise agents are grounded in documents today, and many of them still give wrong business answers. The answer cites a source and passes a groundedness check, yet finance doesn't recognize the number, or the agent shares data a policy says it shouldn't, because the document it retrieved didn't carry the definition or the rule that decides the answer.
Mactores has built data platforms and AI agents on AWS since 2008. This guide draws on our context ontology work, the evaluation method we use to release agents, and how we run the open-source Context Ontology Accelerator. The research and AWS documentation it cites were checked on November 18, 2026.
AI grounding is the practice of connecting a model's output to a verifiable source of truth outside the model, such as documents, live system data, or governed definitions and rules, so each answer can be traced to that source and checked against it. A grounded answer says where it came from; an ungrounded one comes from what the model learned in training.
In practice the term usually means retrieval. Retrieval-augmented generation finds the passages most relevant to a question and asks the model to answer from them, and a grounding check then tests whether the answer stayed within what was supplied. AWS describes its contextual grounding check this way: it "checks if the model response is factually accurate based on the source and is grounded in the source," and any new information in the response counts as ungrounded.
That definition holds the answer to the source. It says nothing about whether the source was the right one, current, or agreed by the people who own the decision, and for business questions those are usually where the error sits.
|
Grounding source |
What it gives the agent |
What it can't settle on its own |
|---|---|---|
|
Documents (RAG) |
Policies, contracts, procedures and past answers, as passages to cite |
Which of several documents is authoritative, current or agreed |
|
Live system data |
Current records through tools, APIs or SQL |
What a field means, and which system wins when two disagree |
|
Governed metrics |
One approved formula per metric, with an owner |
How entities relate, and which rules or permissions apply |
|
Definitions and rules (ontology) |
What each term means, how things relate, which rules and permissions apply |
The text of a contract or a procedure, which retrieval still supplies |
Grounding in documents ties an answer to what someone wrote down. Grounding in business rules ties it to what the business has agreed is true: one definition per term, signed by an owner, and rules evaluated as checks that give the same result every time.
The difference shows up as soon as a question depends on a definition. Software companies report ARR from CRM bookings, billing run-rate and product analytics, and churn changes with whether a downgrade counts, so three well-written documents can support three different numbers. Retrieval picks whichever passage ranks highest, and the grounding check passes because the answer matches that passage.
|
Grounded in documents |
Grounded in business rules |
|
|---|---|---|
|
Source of truth |
Retrieved passages from wikis, decks, contracts and policies |
Governed definitions, relationships and rules in an ontology, with owners |
|
How it's checked |
Groundedness: does the answer match the passage? |
Known-answer tests: does the answer match what the owner agreed? |
|
What it handles well |
Explaining a policy, quoting a clause, summarizing a procedure |
Metrics, eligibility, thresholds, approvals and who may see what |
|
Typical failure |
Faithfully repeats an outdated or unofficial document |
Can't explain a clause no rule captures yet |
|
When it changes |
When someone edits or adds a document |
When an owner signs a new version of a definition or rule |
|
Who owns it |
Whoever wrote the document |
A named business steward for each definition and rule |
Agents need documents to explain and rules to decide. A rule can tell an agent that a request needs approval, but the contract clause it cites still comes from retrieval, and a retrieved policy can describe a limit, but only a rule check applies it the same way every time.
The gap is common. VentureBeat's VB Pulse research, published in October 2026 and based on 130 enterprises surveyed in August, found that 64% had traced at least one confidently wrong agent answer to missing or inconsistent business context in the past six months. Retrieval over documents was the primary context source for 32%, direct queries to live systems for 21% and a governed semantic layer or ontology for 13%.
Analysts point the same way. Gartner's May 2026 press release predicts that by 2027, organizations that prioritize semantics in AI-ready data will increase their agentic AI accuracy by up to 80%, and its analyst Rita Sallam warned that without an understanding of the organization's data relationships and rules, agents "are far more likely to hallucinate, introduce bias and produce unreliable results."
Here is one answer grounded both ways. The question has two parts: "What was net revenue retention last quarter, and can I send the per-customer breakdown to our reseller?" No company or figure is attached; run it against your own stack and compare.
|
Step |
Agent grounded in documents |
Agent grounded in business rules |
|---|---|---|
|
Understand "net revenue retention" |
Retrieves a finance wiki page, a board deck and a sales enablement guide, which count downgrades differently |
Resolves the term to one concept with one formula, owned by finance and versioned |
|
Get the number |
Quotes the figure from the highest-ranked passage, which may be last quarter's deck |
Runs the governed formula on current billing data, with no model call |
|
Check it |
Grounding check passes, because the answer matches the passage |
Known-answer test passes, because the result matches finance's last reported figure |
|
"Can I send the breakdown?" |
Retrieves a data-sharing policy that permits sharing "under contract" and says yes |
Per-customer revenue is classified, and sharing it needs an approved agreement and purpose, so it declines and names the owner |
|
What the user gets |
A cited answer that may be stale or the wrong team's definition, and an unsafe yes |
The figure with its definition version, plus a refusal that explains why and who can approve |
Net revenue retention is the revenue from a starting customer cohort a year later, including expansion and net of contraction and churn, as a share of where it began. That definition sits in one place in the rule-grounded agent, which is why its answer doesn't depend on which document ranked first.
Grounding an agent in an ontology means giving it a governed model of your business to consult before it answers: the things that exist, how they relate, what each term means and which rules and permissions apply. The agent reads the model through tools, and the rules run outside the model, so the same question gets the same treatment every time.
That model is what ontology in AI refers to. It's also the core of a context layer, the rules, definitions and consequences behind your data that tell an agent what a fact means for the decision in front of it.
|
# |
Step |
What it involves |
|---|---|---|
|
1 |
Start from questions, not a data model |
Pick 10 to 20 questions your agents get wrong, give each an owner and an agreed answer, and let those set the scope |
|
2 |
Resolve identity |
Decide which records in which systems describe the same customer, product or account before anything else |
|
3 |
Agree definitions and rules |
Write one definition per term and each rule as a check, signed by the business owner and versioned |
|
4 |
Model them in open standards |
OWL for classes and relationships, SHACL for rules, R2RML to map them to the tables that hold the data, queried in place through a virtual knowledge graph |
|
5 |
Serve them to agents |
Expose the ontology over MCP, so agents call governed definitions and rule checks as tools |
|
6 |
Enforce access and actions outside the model |
Apply permissions at query time and check each tool call against policy before it runs |
|
7 |
Gate every release |
Run the agreed questions, rule cases and access probes on every change, and block the release if they fail |
On the open-source Context Ontology Accelerator, an agent's question falls through three tiers. A governed metric answers directly with no model call; if the question carries qualifiers it can't apply, a structured query runs over the virtual knowledge graph from the governed formula and is checked by a SQL firewall; only questions whose answers live in documents go to retrieval and synthesis. Tier 1 deliberately declines a question with a filter, time window or aggregate it can't apply, so a governed number is never stretched to fit.
Rule enforcement outside the model is now standard on AWS too. AWS made Policy in Amazon Bedrock AgentCore generally available on March 3, 2026: policies written in natural language convert to Cedar, attach to a gateway that intercepts agent-tool traffic, and evaluate each request before it proceeds, outside the agent's code.
Keep what already works. Our approach to context ontology for software leaves existing dbt models and metrics layers in place, and the ontology reads those definitions instead of duplicating them, then adds what a metrics layer never held, such as which customer owns which tenant and which data an agent may touch.
Measure grounding in documents and grounding in rules separately, because they fail in different ways. A groundedness score tells you the answer stayed within what was retrieved; known-answer tests, rule cases and access probes tell you the answer matches what the business agreed.
Groundedness is the degree to which an agent's answer is supported by the source documents it retrieved. AWS's check scores grounding and relevance between 0 and 0.99, needs the grounding source, the query and the response, and suggests 0.7 as a reasonable starting threshold for its Standard tier. For Amazon Bedrock in production, Guardrails apply contextual grounding checks to model inputs and outputs independently of the model in use, and we treat them as one of six evaluation layers that gate a release rather than the whole gate.
|
What to measure |
What it proves |
When it fails, look at |
|---|---|---|
|
Retrieval recall |
The right passages reached the model |
Chunking, indexing and access filters |
|
Groundedness |
The answer stayed within what was retrieved |
The prompt, or a model answering from training |
|
Known-answer accuracy |
The answer matches what the owner agreed |
Definitions, identity resolution or a stale source |
|
Rule-case pass rate |
Rules change the outcome when they should |
Rules that live in documents instead of checks |
|
Access probes refused |
Restricted data stays out of answers |
Permissions applied to results instead of queries |
|
Trace completeness |
Each answer records sources, definition versions and policy decisions |
Logging at the agent instead of at each tool |
Set a threshold on each score that can hold a release. For a refund agent, the AI agents in production method we'd use pulls 200 to 500 past tickets, including the awkward ones, and scores each run for correctness, groundedness, policy compliance and whether it picked the right tools. After launch, someone reads 1 to 5% of live tickets, and any case an incident turns up gets added to the set, so it gets harder to pass over time.
For context ontology releases, the same rule applies at the definition level. Each phase of our context ontology services ends with a signed exit, and nothing goes live until golden questions pass, access policies are tested and every definition has a named steward. A first release built on 10 to 20 business questions usually takes 8 to 12 weeks.
If your agents are grounded in documents but still give wrong business answers, bring one question and three answers your agent gave, and walk through them with a forward-deployed engineer in a single working session. Within a week, you receive a diagnosis, a draft first-release scope and a target production date.