Engineering Brief — v0.0.2a1
Every agent call now runs through four real engines before a webhook ever fires. v0.0.2a1 replaces the two-line approval stub with an actual scoring pipeline — schema, tool-trust, policy, consistency, and grounding checks feed a confidence score, a four-factor risk formula, and a priority-ordered decision table.
v0.0.2a1 replaces the two-line approval stub with an actual scoring pipeline — schema, tool-trust, policy, consistency, and grounding checks feed a confidence score, a four-factor risk formula, and a priority-ordered decision table.
The embedded gateway stopped being a placeholder. The decision your webhook reports is now backed by a real rule engine, not a coin flip dressed as a policy check.
One synchronous call out, one decision back, then a fire-and-forget fan-out that never touches the agent's return path.
POST /v1/runtime/validate · Bearer token · JSON envelope (agent_id, request, execution, output)
Runs the full pipeline below, synchronously, on the request path.
Returned to your code unconditionally — webhook delivery never affects this.
Loads enabled webhooks from SQLite → filters by event type → builds payload → POSTs, 5s timeout, fail-open on error.
This is the part worth walking the team through slowly — it's also the part most likely to surprise people the first time an agent they expected to pass gets blocked.
| Check | What fails it |
|---|---|
| Schema | missing envelope field, −20 pts each |
| Tool trust | tool called with no matching result |
| Policy | base rules + financial pack + adversarial scan |
| Consistency | empty output, no model, latency ≤ 0 |
| Grounding | numeric claim with no matching tool result (±50%) |
A critical policy violation hard-caps the policy score — nothing else in this stage can recover it.
Scans request input and serialized output for injection patterns. Any match caps policy_score at 20, regardless of every other rule.
| Pattern | Example trigger |
|---|---|
| Instruction override | "ignore previous instructions" |
| Jailbreak keywords | "jailbreak", "DAN" |
| Identity override | "pretend you are a different AI" |
| Data exfiltration | "exfiltrate user data" |
A weighted average over whichever signals are actually present — absent signals are excluded from both numerator and denominator, so the score stays normalised to 0–100.
| Signal | Weight |
|---|---|
| Schema / Tool trust / Policy | 20% each |
| Consistency | 15% |
| Grounding / Judge* | 10% each |
| Historical reliability* | 5% |
*Not available in embedded mode — excluded, weights renormalise.
| Factor | Driven by |
|---|---|
| Action severity | tool name (delete/transfer = high, search/list = low) |
| Business impact | agent_id pattern (medical-* = 9, faq-* = 2) |
| Confidence gap | (100 − confidence) / 10 |
| Policy sensitivity | (100 − policy_score) / 10 |
confidence < 50 or policy_score < 60BLOCKrisk_tier == criticalESCALATErisk_tier == high and confidence < 80ESCALATEconfidence ≥ 90 and risk_tier in (low, medium)APPROVEconfidence ≥ 70 and risk_tier == lowAPPROVE50 ≤ confidence < 70RETRYThe exact scenario in examples/06_webhook_integration.py. Click an agent to see its scores move through the pipeline and land on a decision.
Independent of the validation pipeline above — this runs once per registered webhook, every time, and never blocks the response your agent already received.
| Decision | Colour | Meaning |
|---|---|---|
| BLOCK | #FF4757 | policy or confidence floor breached |
| ESCALATE | #FFA502 | critical risk, human review |
| APPROVE | #2ED573 | all checks passed |
| RETRY | #6B6B80 | not yet classified |
| events filter | Fires on |
|---|---|
| "all" | every decision — audit trail |
| "block" | block only — on-call paging |
| ["block","escalate"] | combined high-priority channel |
| "approve" | confirmation log |
All-approve traffic against a block/escalate-only filter producing zero notifications is correct behaviour, not a bug.
~/.agentrust/webhooks.dbevents filter? — no → skip silently, no request sentWARNING, loop continues — never raised back to your agentPick one destination, prove the wiring locally, then promote to programmatic registration once you need more than one sink.
Channel settings → Integrations → Webhooks → New Webhook → copy URL. Treat this like a password the moment it's copied.
Confirm .env is in .gitignore before you save — every time, don't assume the template already did it.
AGENTRUST_WEBHOOK_URL=https://discord.com/api/webhooks/YOUR_ID/YOUR_TOKEN AGENTTRUST_WEBHOOK_EVENTS=block,escalate
This separates "is my filter/URL correct" from "is my agent producing the decision I expect" — two different failure classes.
# Terminal 1 — disposable local sink python3.11 demo/webhook_receiver.py # Terminal 2 — run the demo agents export $(grep -v '^#' .env | grep -v '^$' | xargs) python3.11 examples/06_webhook_integration.py
Works with any @harness-wrapped function, no other code changes required.
export AGENTRUST_WEBHOOK_URL="https://discord.com/api/webhooks/YOUR_ID/YOUR_TOKEN" export AGENTRUST_WEBHOOK_EVENTS="block,escalate"
Registrations land in SQLite and survive process restart — do this once per environment, not once per boot.
dispatcher.register(url=DISCORD_URL, events=["block","escalate"], name="discord-oncall") dispatcher.register(url=SLACK_URL, events=["all"], name="slack-audit-log")
dispatcher.list_webhooks() — check url_masked, events, and enabled for each entry. You're confirming shape and filter, not the literal secret — that's the point of the mask.
.env or a secrets manager — never in chat, email, or a PR comment.list_webhooks() show only the last 8 characters. If a URL is ever exposed, delete and recreate it at the destination immediately; rotating in .env after the fact is not sufficient since the old URL stays live until deleted.http://localhost:* for local development.What I'd actually raise in a design review before treating this as production-ready.
Confidence in every decision — pre-production certification to real-time runtime governance. Start free, no credit card required.
See Pricing & Start Free →