Evidence governance and attestation at scale for portfolios

Most teams don't fail evidence governance because they lack controls. They fail because their controls live in people's heads, spreadsheets nobody opens, and quarterly rituals that get skipped the one quarter an auditor actually shows up. Attestation works fine when you have three products and one compliance owner who remembers everything. The whole thing bends the moment you cross into portfolio territory — ten, twenty, fifty products, each with its own release cadence, its own owners, and its own interpretation of what "evidence" even means.

This piece is about the system layer: how policy, cadence, validators, and exception handling connect into something that survives scale instead of collapsing under it. Not a checklist of controls — those exist everywhere. The harder question is how the parts coordinate when you're not personally in the room to enforce them.

Why portfolio attestation breaks differently than single-team attestation

At the single-team level, attestation is basically a memory problem. Someone needs to remember to sign off on the security review, confirm the accessibility pass happened, attach the right artifacts before release. One diligent lead can carry that load manually for a long time.

Across a portfolio, it stops being a memory problem and becomes a consistency and coordination problem. Three things start fighting each other:

1. 1
   Interpretation drift. Team A thinks "evidence of a passing security scan" means a screenshot. Team B attaches a full SARIF export. Team C links to a Jira ticket that references a scan that ran four months ago. None of them are technically wrong, which is exactly why it's a mess.
2. 2
   Cadence collision. Your weekly-release teams and your quarterly-release teams can't live under the same attestation rhythm. Force one cadence and half your teams drown in ceremony while the other half go stale between checkpoints.
3. 3
   Ownership ambiguity. When a shared platform service underpins six products, who attests to its controls? In practice, everyone assumes someone else did.

The breakdown rarely announces itself. There's no dramatic failure. Instead you get slow entropy — attestations that technically exist but point to artifacts that moved, expired, or never matched the actual released build. The audit finds it. You didn't.

Start with a policy DSL, not a policy document

The first structural shift that makes portfolio attestation work is treating policy as something machine-readable, not prose. A 14-page governance PDF is unenforceable at scale because every team reads it slightly differently and nothing validates against it.

`` policy "security-scan-evidence" { appliesto = products.where(tier in ["critical","high"]) requires = artifact(type: "sast-report", maxage: "30d") attestedby = role("security-owner") cadence = onevent("release.major") or every("90d") evidenceref = mustresolve // link must point to a live artifact } ``

The point isn't the exact syntax. Once policy is expressed this way, you get three things prose can never give you:

1. 1
   Unambiguous scope — the rule knows which products it applies to.
2. 2
   Validatable conditions — "max_age

   30d" is checkable; "recent scan" is not.

3. 3
   A single source of truth that both humans and automation read the same way.

A common mistake: teams write the DSL too tightly on day one, encoding every edge case, and the policy becomes so rigid that half the portfolio needs an exception by week two. Start coarse. Encode the five controls that actually matter for audit and let the exception system absorb the edge cases instead of bloating the policy.

Attestation cadence: stop running everything on the calendar

The cadence question is where most governance models quietly over-engineer. There's a strong instinct to put everything on a fixed quarterly review. It feels orderly. It also means a product that shipped a major architectural change in week two of the quarter carries unvalidated evidence for eleven weeks.

Better cadence design is event-driven with a time-based safety net. You attest when something meaningful happens, and you re-attest if too much time passes with nothing triggering. The combination catches both the fast-movers and the stale-sitters.

Here's how different cadence models actually behave at scale:

The event-plus-safety-net model is the one that holds up for most portfolios. The catch: it only works if something is actually watching the events and the clock. A human can't track "has product 23 had a major release trigger an un-attested control change" across fifty products. This is the first place where lightweight automation earns its keep — not as a flashy feature, but as the thing that watches the boring timers and fires the right attestation request to the right owner without anyone having to remember to.

Automated validators: the difference between "attested" and "actually true"

An attestation with no validator behind it is just a checkbox. The dangerous version of evidence governance is the one that looks complete — every box ticked, every signoff present — while the underlying artifacts are broken, expired, or disconnected from the released build.

Validators are the enforcement layer that checks whether the claim holds. They run on two levels:

Structural validators — does the evidence exist, resolve, and meet basic shape requirements? Does the link point to a live artifact? Is the scan report the right format? Is it within the age limit the policy declared?

Semantic validators — does the evidence actually correspond to what was released? A scan from 30 days ago passing the age check means nothing if it ran against a commit that's six releases behind. Good validators tie the evidence back to the specific build or requirement version it's supposed to cover.

A failure pattern worth knowing about: a mid-sized product org had clean attestation dashboards — green across the board. An audit pulled a sample of ten released features and asked to trace the evidence. Four of the ten linked to artifacts that had been regenerated and overwritten, so the "evidence" now reflected a later state than what actually shipped. The attestations were technically present and technically passing a structural check. They failed the moment anyone checked whether the evidence matched the actual release. That gap — present-but-meaningless evidence — is what semantic validators exist to close.

If you're still building the underlying traceability that validators depend on, the groundwork in a lightweight compliance-ready requirements traceability model is worth getting right first. Validators can only check links that exist.

Exception primitives: design for the "no" cases on purpose

Every attestation system meets reality within days: a product that legitimately can't meet a control this cycle, a vendor artifact that's delayed, a control that doesn't apply to a particular architecture. If the system has no first-class way to handle these, people route around it — they fake the attestation, skip it, or file a side agreement in Slack that nobody can find later.

Exceptions need to be primitives, not afterthoughts. A proper exception has structure:

1. 1
   Scope — exactly which policy, which product, which release it applies to.
2. 2
   Justification — why the control can't be met, in a form the auditor will accept.
3. 3
   Expiry — a hard date. Exceptions without expiry become permanent, and permanent exceptions are just undocumented policy gaps.
4. 4
   Approver — someone with the authority to accept the risk, recorded by role.
5. 5
   Compensating control — what you're doing instead, if anything.

The part most teams miss: an exception is itself a piece of evidence. It should be governed with the same rigor as a passing attestation. An approved, time-boxed, justified exception is defensible in an audit. An undocumented gap is not — even if the actual risk is identical.

Watch for exception sprawl. When the same exception gets requested by fifteen products, that's not fifteen exceptions — that's a signal your policy is wrong. Roll it back into the DSL with a proper scope condition instead of letting it accumulate as repeated one-offs.

CI validators: enforce evidence governance where work actually happens

Push the validators into CI. That's the structural move that separates governance that scales from governance that nags.

If evidence checks only run at a quarterly review, you've built a system that discovers problems 89 days after they were introduced. If the validators run in the pipeline — on every PR, every merge, every release build — the feedback arrives while the context is still warm and the fix is cheap. The attestation requirement becomes part of the definition of "can this ship," not a separate ceremony bolted on afterward.

A practical CI validator flow:

1. 1
   On PR open — check whether the change touches anything governed by a policy (a security-relevant module, an accessibility-affecting component, a regulated data path).
2. 2
   Flag the relevant policies — surface which attestations this change will require before merge, so the author isn't surprised at release time.
3. 3
   On merge to release branch — run structural validators

   do the required artifacts exist and resolve?

4. 4
   On release candidate — run semantic validators

   do the artifacts match this build?

5. 5
   Block or warn — hard-block on critical-tier policies, warn-and-track on lower tiers.
6. 6
   Record the attestation state as part of the build metadata, so the evidence travels with the release instead of living in a separate system.

The warn-vs-block decision matters more than it looks. Block everything and teams start disabling the checks. Warn on everything and nothing gets enforced. Tie the enforcement level to the product tier your policy DSL already declared — critical products hard-block, exploratory products warn. The policy language and the CI behavior stay in sync because they read from the same source.

A diagram like this makes it clear which validator runs at each CI stage.

“

Start by hard-blocking only critical-tier policies in CI to avoid teams disabling checks.

This connects directly to how investment and release gates get wired in portfolio requirements governance that ties investment gates to requirement evidence — CI validators are where those gates stop being policy statements and start being something a pipeline actually enforces.

A runbook for when an attestation fails in CI

Automation surfaces the problem. A runbook decides what happens next. Without one, a failed validator just becomes a blocked pipeline and an annoyed engineer pinging whoever's online.

When a structural validator fails (artifact missing or expired):

1. 1
   Author gets a direct notification with the specific policy and what's missing.
2. 2
   Clock starts

   48 hours to remediate for a standard release, 4 hours for a hotfix.

3. 3
   If the artifact genuinely can't be produced in time, the path is an exception request — not a bypass.

When a semantic validator fails (evidence doesn't match the build):

1. 1
   This is treated as higher severity, because it usually means the evidence was stale or mislinked.
2. 2
   Release owner is pulled in, not just the author.
3. 3
   The fix is regenerating the evidence against the current build, then re-running the validator.

When an exception is requested:

1. 1
   Routes to the role named in the policy, not a free-for-all.
2. 2
   Requires expiry and justification before it can be approved — the system shouldn't accept an exception missing those fields.
3. 3
   Logged as evidence in its own right.

SLA rules that keep the system honest

1. 1
   Critical-tier attestation requests acknowledged within 1 business day, resolved within 3.
2. 2
   Exception reviews decided within 2 business days — a pending exception is an unblocked risk sitting in limbo.
3. 3
   Expired-evidence re-attestation triggered automatically, with a 7-day window before the product's governance status flips to non-compliant on the portfolio dashboard.
4. 4
   Stale exception cleanup — anything within 14 days of expiry gets flagged to its approver automatically.

The portfolio-level dashboard is what makes these SLAs mean anything. If nobody can see which products are green, which are warning, and which are sitting on expired exceptions, the SLAs are just numbers in a doc. The retention and audit-readiness side of this — how long you keep all of it and how you prove it later — is worth structuring deliberately, and the model in portfolio evidence retention policy and audit-readiness model covers that layer.

A real scenario: fintech portfolio, 18 products

A fintech company running around 18 products hit the wall most portfolios hit. Each product had decent individual compliance habits, but there was no shared evidence model. When their first serious audit came, prep took close to six weeks — a small team manually chasing down artifacts, reconstructing which scan ran against which release, and discovering that roughly a third of their "evidence" was either stale or pointed at overwritten artifacts.

1. 1
   Encoded their five highest-impact controls into a policy DSL instead of a wiki page.
2. 2
   Moved from a flat quarterly cadence to event-driven attestation with a 90-day safety net.
3. 3
   Put structural validators in CI and semantic validators at the release-candidate stage.
4. 4
   Made exceptions a tracked primitive with mandatory expiry.

The next audit prep took a little over a week. Not because they worked harder — because the evidence was already assembled, already validated against the right builds, and the exceptions were already documented with justifications. The stale-evidence rate dropped to near-zero since CI caught mismatches at merge instead of a human catching them months later. The qualitative shift mattered more than the metrics: audits went from a scramble to a report pull.

When this level of rigor makes sense — and when it doesn't

This makes sense when:

1. 1
   You're running enough products that no single person can track evidence state across them.
2. 2
   You operate in a regulated space where audit failures carry real cost.
3. 3
   Your release cadences vary widely across teams, so a single manual rhythm can't cover everyone.

This is overkill when:

1. 1
   You have three or four products and one owner who genuinely has visibility across all of them. Policy DSL and CI validators are overhead you don't need yet.
2. 2
   Your releases are infrequent and low-risk enough that a quarterly manual review covers you.

Who should NOT do this yet:

1. 1
   Teams that don't have basic requirement-to-evidence links in place. Validators check links that exist; if the traceability isn't there, you're automating a hole. Build the traceability foundation first, then layer attestation on top.

The honest failure mode to avoid: building the whole DSL-plus-validators-plus-exception machine before anyone on the teams actually trusts or uses the attestations manually. Automation amplifies whatever process you point it at. If the underlying process is ignored, you'll just produce ignored results faster.

Where automation earns its place in all this

Nothing above requires exotic tooling to understand, but it does require something to watch the timers, resolve the links, and route the requests — because at portfolio scale, those tasks exceed what any person can hold.

The useful role for AI-assisted operational tooling here is narrow and boring in the best way: it watches cadence clocks across every product at once, re-validates evidence links before they go stale, flags when the same exception keeps recurring (your signal to fix the policy), and surfaces semantic mismatches that a human scanning a dashboard would never catch.

It's not the interesting part of the system — the policy design and the runbook decisions are the interesting parts. It's the part that makes sure the interesting parts actually run when you're not watching.

The teams that get portfolio attestation right aren't the ones with the most controls. They're the ones whose controls connect — policy that validators can read, cadence that automation can track, exceptions that carry their own evidence, and CI that enforces all of it at the moment work happens. That connected system is what survives the audit nobody warned you about, and more importantly, it's what lets you add the twentieth product without the whole governance model falling over.