How to measure certainty before your agent acts
Confidence scoring assigns a numeric certainty estimate to every agent decision before execution — creating the risk gradient your governance model needs without eliminating autonomy.
How to measure certainty before your agent acts — and why it matters at production scale
The hardest problem in production agentic AI is not getting the agent to work — it's knowing when to trust it. An agent that performs perfectly 97% of the time is impressive in a demo. In a live financial workflow processing 10,000 decisions a day, that 3% failure rate is 300 wrong actions per day, many of them irreversible.
The solution is not to reduce autonomy across the board. That defeats the purpose of building agents at all. The solution is confidence scoring — a runtime measurement of how certain the agent is before each action, applied against a governance threshold that determines whether the action executes, escalates to human review, or blocks entirely.
This post covers the four-signal confidence model used in AgentTrust OS, how thresholds translate to governance policy, and why confidence scoring is the foundational primitive for any enterprise agent deployment.
A confidence score is only as reliable as the signals it aggregates. Four independent measurements feed the composite score — each targeting a different failure mode.
Before any action, the agent's proposed output is validated against a declared schema for that action type. A well-formed, complete, type-correct response scores high. Missing required fields, unexpected values, or malformed structures reduce the score proportionally. This catches hallucinated parameters and type errors before they reach downstream systems.
Not all tools are equally reliable or equally risky. A read-only retrieval tool carries different risk than a write-to-database or send-email call. The tool-trust score assigns each tool a pre-defined trust level, and reduces the composite confidence score for actions that invoke lower-trust tools — requiring higher overall conformance to clear the execution threshold.
Every agent operates under a policy contract that defines permitted actions, required approvals, and prohibited behaviors. The policy alignment score measures how cleanly the proposed action matches the declared contract. Actions that fall within the approved scope score at 1.0; actions that approach policy boundaries score lower; actions that cross them are blocked regardless of other signal scores.
For decisions above a configurable risk threshold, the harness re-samples the model independently and compares outputs. High agreement across samples indicates stable, repeatable reasoning. High variance — different action choices, different parameter values, or contradictory conclusions — indicates low model confidence in the underlying decision and reduces the composite score accordingly.
The composite confidence score is computed as a weighted average of the four signals. The weights reflect the risk profile of the deployment — a financial workflow weights policy alignment heavily; a customer service workflow may weight schema conformance and consistency more evenly.
| Composite Score | Default Routing | Use Case |
|---|---|---|
| 0.90 – 1.00 | Autonomous execution | Well-scoped, pre-approved action types |
| 0.75 – 0.89 | Execute with audit trail | Routine actions requiring post-hoc review |
| 0.60 – 0.74 | Escalate to human review | Ambiguous actions or novel tool combinations |
| Below 0.60 | Block and alert | High-risk, policy-adjacent, or malformed actions |
Thresholds are per-workflow, not global. A data enrichment agent and a payment-processing agent should not share the same execution threshold. Configure thresholds to match the irreversibility and business impact of the actions in scope.
Trust Runtime computes the four-signal score on every agent action and routes accordingly. The score and all four component values are attached to the action record in Trust Audit — so compliance reviews can trace exactly why an action executed autonomously, escalated, or was blocked.
import { AgentTrustRuntime } from "@agenttrust/sdk";
const runtime = new AgentTrustRuntime({
thresholds: {
autonomous: 0.90, // execute without review
supervised: 0.75, // execute + audit trail
escalate: 0.60, // route to human review
// below 0.60 → block + alert
},
weights: {
schemaConformance: 0.25,
toolTrust: 0.30,
policyAlignment: 0.30,
outputConsistency: 0.15,
},
});
// Every agent action is scored before execution
const decision = await runtime.evaluate({
action: proposedAction,
context: agentContext,
policy: workflowPolicy,
});
// decision.score, decision.route, decision.signals
// are all available for logging, alerting, and auditTrust Runtime computes per-action confidence scores at runtime. Start free, no credit card required.
See Pricing & Start Free →