Original lab · Secure delivery

Does a release gate detect the new finding without re-counting the baseline?

This benchmark fixes one synthetic error in the baseline, repeats it with the same partial fingerprint, then adds a second fingerprint in the blocking candidate. It tests identity and policy reasoning—not scanner quality or software security.

Worked sample, not a live result.The table is derived from three checked-in SARIF files. It is not the output of a current workbench run, repository scan, GitHub check, or deployment gate. Run the canonical tool to create a fresh delta, verdict, artifacts, and receipt.

Run SARIF Finding Delta & Release Gate Read the workflow guide Verify receipt + artifact bytes

Benchmark design

Baseline

One synthetic error result with rule FT001 and fingerprint fixture-ft001-a.

Pass candidate

The same result, location, level, and fingerprint. Under a zero-new-error policy, the fixture contributes no new error.

Block candidate

The baseline result plus synthetic rule FT002 with fingerprint fixture-ft002-b.

Policy

Block when the candidate contains one or more new unsuppressed error results. This is a fixture policy, not a universal security standard.

Method

Validate that each document declares SARIF version 2.1.0 and contains a supported self-contained run.
Prefer declared result fingerprints for identity. Treat fallback location/message matching as lower-confidence evidence.
Classify baseline-only results as fixed, shared identities as unchanged or updated, and candidate-only identities as new.
Apply the selected policy to the delta, while keeping accepted suppressions visible.
Report INDETERMINATE rather than PASS when input validation or identity evidence is insufficient.

Fixture bytes and hashes

shasum -a 256 app/fixtures/flagship-editorial-20260713/sarif/*.sarif

baseline.sariffbc18d49e1f35752fce06eadeab57d7af9458db849f68a78a24bfb06f52c7c22
candidate-pass.sariffbc18d49e1f35752fce06eadeab57d7af9458db849f68a78a24bfb06f52c7c22
candidate-block.sarifb1c23c9d0b42814927795cb5e7c87c1d7a766c54cd31c00479441bf4c91bff09

All locations, messages, and fingerprints are invented for this fixture. They do not represent a real codebase or vulnerability.

Deterministic observations

Comparison	Baseline results	Candidate results	New fingerprints	Fixture policy outcome
Baseline → candidate-pass	1	1	0	PASS candidate
Baseline → candidate-block	1	2	1 (`fixture-ft002-b`)	BLOCK candidate

These counts can be reproduced by reading the JSON arrays and fingerprint values. The words PASS and BLOCK describe the stated fixture policy only. They do not certify that the application is safe, that scanners found everything, or that a release was actually prevented.

Limits and privacy

Limits

No code scan, external SARIF property resolution, repository checkout, or deployment action.
Fingerprint stability depends on the producing analyzer. Fallback matching can misclassify moved or rewritten findings.
A PASS says the supplied delta fits the supplied policy; it says nothing about findings omitted by tools.

Privacy boundary

SARIF can expose source paths, snippets, rule metadata, and security details. Use approved, redacted, or synthetic files on a trusted device. The canonical browser tool does not fetch external property files, but the user remains responsible for local access and handling.

Official sources

Corrections

FastTool is responsible for this page. Send a fixture mismatch, source update, or methodological correction to [email protected] with evidence and a proposed change. Material corrections appear in the changelog. Reviewed 2026-07-13.