AI Governance Evidence Pack for Audit Committees: A Quarterly Template
The Wrong Question Audit Committees Keep Asking
“Do we have AI expertise on the board?” is the wrong starting point — it leads to a training session, a consultant’s slide deck, and zero change in what management actually reports. The right question is simpler and harder: what evidence lands on our table every quarter that would let us catch an AI failure before the regulator, the press, or a plaintiff does?
If the answer is a vendor’s SOC report and a CIO’s assurance that “the models are monitored,” the committee is flying blind. What follows is the pack that fixes that: five sections, exception-based, designed to be read in twenty minutes and to surface exactly three things — what changed, what failed, and what nobody has looked at.
This complements Board of Directors duties, the Audit Committee Handbook, and our EU AI Act Compliance Guide — read this for the oversight mechanism, that for the regulatory substance.
Section 1: Scope and Accountability (One Page, Updated Quarterly)
Every pack opens with the inventory. No inventory, no oversight — you cannot govern systems you cannot name.
| Field per system | Example |
|---|---|
| System + use case | Credit-decision support; CV screening; fraud scoring |
| Materiality tier | High / medium / low, with the criterion (revenue, customers, regulatory exposure) |
| Owner (management) | Named individual, not a department |
| Accountable executive | Single C-suite name — see below |
| Regulatory tag | EU high-risk (Annex III, effective Dec 2027), GPAI downstream use, none |
Two rules make this section work. First, one accountable executive per material system — diffusion across “the business,” IT, and data science means nobody owns a failure. Second, the inventory must be reconciled, not self-reported: procurement records, cloud bills, and model registries cross-checked quarterly, because shadow AI (a team fine-tuning a vendor model on customer data) is the rule, not the exception. Untracked systems go straight to Section 4 as exceptions.
Section 2: Material Changes Since Last Meeting
AI systems change continuously; quarterly governance that re-reads the same dashboard is theater. This section lists only deltas:
- New deployments or material retraining (new data sources, expanded scope, version changes) with the validation performed before release.
- Vendor and dependency changes — new model providers, API changes, deprecated endpoints — tied to Vendor Due Diligence review dates.
- Threshold or policy changes (approval limits, auto-decision boundaries) with who approved them and the risk assessment behind the change.
- Regulatory developments affecting the inventory — e.g., Omnibus-driven timeline shifts, new GPAI guidance — with the compliance impact assessed, not just noted.
A quarter with “no material changes” is a finding, not a pass, unless the monitoring in Section 3 evidences continued fitness. Drift does not announce itself.
Section 3: Validation and Controls (Testing With Dates, Not Adjectives)
“Models are regularly tested” is the AI equivalent of “reconciliations are performed” — meaningless without dates, thresholds, and results. Demand:
- Test inventory: what was tested (accuracy, bias, robustness, adversarial), when, against what benchmark, and the pass/fail threshold set before testing.
- Human-review metrics: what share of decisions a human actually reviewed, override rates, and — critically — whether overrides skew in one direction (rubber-stamping). A 0.3% override rate on flagged items means the human is decoration, not control.
- Production monitoring: drift metrics, error rates, and complaint volumes trended quarter over quarter, with pre-agreed escalation thresholds.
- Data controls: lineage documentation, consent/licensing basis for training data, retention and deletion execution — the data layer where most regulatory exposure actually sits.
Each item needs the trio audit committees already demand everywhere else: who tested, when, and who reviewed the result. Tie the cadence to Month-End Close discipline — evidence produced on schedule or not at all.
Section 4: Exceptions and Incidents (The Section That Matters Most)
A pack with zero exceptions across four quarters signals a broken reporting channel, not perfect AI. This section must contain, at minimum:
- Failed tests and control exceptions — what failed, business impact, root cause, remediation owner, due date.
- Overdue remediation carried from prior quarters, aged like audit findings. Anything over 90 days past due gets named with the accountable executive present.
- Incidents and near-misses, including vendor-model incidents that affected operations and customer complaints alleging unfair or wrong automated decisions.
- Whistleblower and ethics reports touching AI use — routed through the existing Whistleblower Program channels, summarized here without identities.
Escalation rule worth adopting verbatim: any exception affecting a high-tier system, any overdue remediation past 90 days, and any incident with regulatory or reputational exposure goes to the full board, not just the committee. Write it into the charter — see Audit Committee Charter conventions.
Section 5: Assurance Coverage (Who Looked, and What Nobody Looked At)
The closing section maps assurance the way audit plans map coverage:
| Line | This quarter | Gaps |
|---|---|---|
| First line (model owners, validation teams) | Testing completed per Section 3 | Self-assessed; note bias risk |
| Second line (risk + compliance) | Policy reviews, threshold approvals | Coverage of new deployments? |
| Third line (internal audit) | Audits completed / in flight | Systems never audited — name them |
| External | Auditor inquiries, regulatory exams | Reliance limits stated explicitly |
The most valuable line in the whole pack is the last column’s bottom row: systems with zero independent coverage, listed by name. That list is the next year’s audit plan. Coordinate it with Internal Audit Framework planning and SOX 404 scoping where AI touches financial reporting controls.
The Four Risk Families (Map Everything to These)
| Family | Core question | Pack home | Typical owner |
|---|---|---|---|
| Model risk | Is the model right, fair, and stable? | Sections 2–3: validation results, drift, bias tests | Model owner + validation |
| Operational risk | Does it fail safely in production? | Sections 2, 4: incidents, overrides, fallbacks | Operations + IT |
| Data risk | Is the data lawful, good, and traceable? | Sections 1, 3: lineage, consent, retention | Data + privacy |
| Governance risk | Is anyone actually accountable? | Sections 1, 5: ownership, coverage gaps | Board + risk — see ERM |
If a committee discussion can’t place the issue in one of these four rows within a minute, the pack is missing a section. Use the families as the agenda structure and the pack writes its own meeting.
Why Traditional Audit Trails Fail Probabilistic Systems
Committees raised on deterministic controls need this mental-model shift, stated plainly in the pack’s appendix:
- Sampling proves less. A clean sample of 25 outputs does not bound behavior on input 26 — the distribution, not the instance, is the control target. Demand population-level monitoring alongside samples.
- Reproducibility is partial. Same model plus changed data, context, or vendor version equals different behavior. Version-freeze models, data snapshots, and prompts with the rigor of a close binder — see Record Retention discipline applied to model artifacts.
- Drift is silent. Performance degrades without code changes as the world moves. Point-in-time testing without continuous thresholds is assurance theater.
- Explainability has limits. For high-stakes decisions, require either interpretable models or compensating controls (human review with real authority, appeal paths, outcome monitoring) — and say which, per system, in Section 1.
- Human review degrades. Automation bias turns reviewers into rubber stamps within months. Measure override rates and rotate reviewers; a control nobody exercises is a control that does not exist.
The Pack Template (Copy Into Your Board Portal)
AI Governance Evidence Pack — Q[X] 20[XX]
- Scope & accountability: material-system inventory with owners, accountable executive, regulatory tags; reconciliation statement (sources checked, shadow-AI sweep result).
- Material changes: deployments, retrains, vendor/dependency changes, threshold changes, regulatory developments — each with validation evidence or an explicit gap flag.
- Validation & controls: test inventory with dates/thresholds/results; human-review and override metrics; production monitoring trends; data-control status.
- Exceptions & incidents: failed tests, control exceptions, 90-day+ overdue remediation (named), incidents/near-misses, whistleblower items.
- Assurance coverage: first/second/third-line activity this quarter; external auditor and regulator touchpoints; named list of systems with zero independent coverage.
- Appendix: risk-family map, EU AI Act applicability tracker (transparency live; Annex III Dec 2027), prior-quarter carryovers closed with evidence.
FAQ
How is AI oversight different from cyber oversight for audit committees?
Cyber oversight asks whether defenses hold; AI oversight asks whether decisions are right, fair, and accountable — including when the system works exactly as designed. The evidence pack differs accordingly: validation results and override metrics replace vulnerability counts.
Should the full board or just the audit committee own AI oversight?
The audit committee typically owns evidence review and control assurance, with material exceptions escalated to the full board per a written rule. Risk-committee overlap should be mapped once, not relitigated quarterly.
What is the single most important page in the pack?
The named list of material AI systems with zero independent coverage (Section 5). Everything else describes knowns; that list prices the unknowns.
Next Steps
- Adopt the five-section template at the next audit committee meeting and assign pack ownership (controller or chief risk officer) with a quarterly due date.
- Run the first inventory reconciliation — procurement, cloud bills, model registry — and publish the zero-coverage list even if it embarrasses management.
- Map EU AI Act applicability per system now (transparency duties live; Annex III high-risk from December 2027) so the December 2027 deadline arrives as a project update, not a surprise.
