τ-BENCH-STYLE EVAL — TOOL RULE-FOLLOWING
measuring live…
Suite design after τ-bench arXiv:2406.12045 · AgentBench arXiv:2308.03688. pass^1 = task passes iff EVERY domain rule holds against the real produced trajectory.
CALIBRATION — ECE / BRIER + AUTO-RESPONSE GATE
measuring live…
ECE (equal-width bins) + Brier per arXiv:2505.15437. Gate: ECE < 0.05 required for any automated (no-human) response; fails closed on unmeasured calibration.
CONFORMAL PREDICTION — SETS, NOT BARE %
computing…
Split conformal per arXiv:2305.18404 · Angelopoulos & Bates arXiv:2107.07511. Shared helper szl_conformal — Dev D imports the SAME module for threat classification.
POLICY — FILE-BACKED COLANG ROE (INDEPENDENTLY AUDITABLE)
loading policy files…
NeMo Guardrails Colang syntax — github.com/NVIDIA-NeMo/Guardrails. Policy lives in version-controlled .co files (single source of truth), shown here with per-file sha256. No prompt-only policy.
IETF COMPLIANCE-RECEIPT PROFILE (DSSE ENVELOPE INTACT)
signing a live decision…
Compliance VIEW aligned to
draft-marques-asqav-compliance-receipts-05. The cross-app DSSE envelope (ECDSA-P256-SHA256) is reproduced UNCHANGED — verify against
/cosign.pub.
LEAN4AGENT — WORKFLOW INVARIANTS ROADMAP
…
Formalization after Lean4Agent arXiv:2606.06523. Proved-in-isolation invariants are real; full-pipeline + determinism theorems carry sorry and are NOT machine-checked yet.