Evidence format, version 1
This is what osfi-gate writes, stated so that it can be checked without osfi-gate. Two
independent verifiers are implemented from this document alone, in verifier/: one in Python
using only its standard library and the openssl command, one in JavaScript using only Node's
built-in modules. If this document and either of them disagree, one of the three is wrong, and
the conformance tests in tests/test_standalone_verifiers.py exist to find out which.
It is licensed separately from the rest of the repository, under Apache-2.0, so that an institution can keep, run and reimplement it whatever becomes of this project.
1. Where things are
An evidence store is a directory (or a git branch, or an object-store prefix laid out the same way):
ledger/<slug>.jsonl one ledger per repository
bundles/<slug>/<bundle_id>/ a bundle, as a directory
bundles/<slug>/<bundle_id>.tar.gz or the same bundle, archived
<slug> is the repository name with every / replaced by __ (northwind/adjudication-service
→ northwind__adjudication-service). Where both a directory and an archive exist for a bundle,
the directory is read.
2. Canonical JSON
Used for exactly one thing: computing a ledger entry's entry_hash (§4). Manifests and
statements are hashed as the bytes on disk, never re-serialised.
- Objects: keys sorted by Unicode code point, recursively.
- Separators
,and:, with no whitespace anywhere. - Strings as UTF-8, not escaped to ASCII:
éis the two bytesc3 a9, noté. The characters JSON requires escaped (",\, and U+0000–U+001F) are escaped, using\",\\,\b,\f,\n,\r,\t, and\u00XX(lowercase hex) for the rest. - Values: objects, arrays, strings,
true,false,null, and integers between −(2⁵³−1) and 2⁵³−1, written without a fraction or an exponent. No other numbers. A ledger entry containing any other number —1.0and1e3included, though they are whole — is malformed, and a verifier must reject it rather than guess how it was formatted. (Languages disagree about all three: how to print a fraction, whether1.0is an integer, and what happens past 2⁵³.)
3. A bundle
| File | What it is |
|---|---|
manifest.json |
See below. |
statement.json |
An in-toto Statement (below). |
statement.sigstore.json |
The signature over statement.json, when the bundle is signed. |
| everything else | Evidence: context.json, gate-input.json, decision.json, summary.md, raw/*. |
manifest.json, statement.json and statement.sigstore.json are the control files, and
only at the top level of the bundle. A file with one of those names anywhere deeper is evidence
like any other.
The manifest is a JSON object with at least:
schema "osfi-gate/manifest/v1"
bundle_id "<seq, six digits>-<first 12 hex of the commit>"
repo the repository
seq integer, this bundle's position in the ledger
prev_entry_hash the entry_hash of the previous ledger entry (§4)
files { "<path relative to the bundle, / separated>": "<sha256 hex of its bytes>" }
files must list every evidence file in the bundle — every file that is not a control file —
and nothing else.
The statement is an in-toto Statement v1 (_type https://in-toto.io/Statement/v1) whose
subject array contains {"name": "manifest.json", "digest": {"sha256": "<sha256 hex of the
manifest's bytes>"}}, and may contain a git+<repo> subject with the commit. Its predicate
summarises the decision; it is informative, and the evidence files are authoritative.
4. The ledger
One JSON object per line, in order, UTF-8. Lines end with LF (\n) and only LF: U+2028,
U+0085 and their kind may appear raw inside a string, and a reader that also treats them as line
breaks (Python's str.splitlines, for one) splits an entry in half. Each entry has at least:
seq 1, 2, 3, ... with no gaps
bundle_id as in the manifest
repo
result "pass" | "fail"
manifest_sha256 sha256 hex of the bundle's manifest.json bytes
statement_sha256 sha256 hex of the bundle's statement.json bytes
signing { "signed": bool, "mode": "key" | "keyless" | "none", "key_id": hex | null, ... }
recorded_at "YYYY-MM-DDTHH:MM:SSZ", UTC
prev_entry_hash the previous entry's entry_hash; 64 zeros for the first
entry_hash sha256 hex of the canonical JSON (§2) of this entry without entry_hash
entry_hash covers every other field, including prev_entry_hash, so the hash at entry N
commits to entries 1 through N. That is why a single observation of it, held somewhere the
evidence's owner cannot write, is enough to detect a rewritten history (§7).
5. What verifying a bundle means
For a ledger entry and its bundle:
sha256(manifest.json)equals the entry'smanifest_sha256.- The manifest's
seqandprev_entry_hashequal the entry's. - Every file the manifest lists exists and hashes to its listed digest; no evidence file exists that the manifest does not list.
sha256(statement.json)equals the entry'sstatement_sha256, and the statement's subjects include the manifest's digest as in §3.- If the entry says the bundle is signed, the signature verifies (§6).
And for the ledger as a whole: seq runs 1, 2, 3 without gaps; each prev_entry_hash is the
previous entry's entry_hash; each entry_hash recomputes.
6. Signatures
Only one form is defined in version 1: a key-signed bundle, signing.mode "key", whose
statement.sigstore.json is
{ "base64Signature": "<base64 of a DER-encoded ECDSA signature>" }
The signature is ECDSA over NIST P-256 with SHA-256, computed over the exact bytes of
statement.json. It verifies against the institution's public key, a PEM PUBLIC KEY.
signing.key_id, when present, names the key: the SHA-256 hex of the PEM body — the base64
between the BEGIN and END lines with all whitespace removed — as text, before decoding.
A verifier given a public key whose id differs from the one the entry records should say so:
the key it holds is not the key that signed.
A statement.sigstore.json of any other shape, and every keyless ("keyless", Sigstore
Fulcio certificate) signature, is outside this version. A verifier implementing only this
document reports them not checked — never passed, never failed.
7. Anchors
An anchor is a JSON object observed somewhere the evidence's owner cannot rewrite:
{ "repo": "<repository>", "seq": <integer>, "entry_hash": "<hex>", "observed_at": "...Z", ... }
It holds if the ledger has an entry at seq and that entry's entry_hash equals the anchor's.
An anchor whose seq is beyond the end of the ledger means the ledger was truncated; one
whose hash differs means it was rewritten. A directory of anchors holds one file per
observation, named <slug>-<seq, eight digits>.json.
8. Audit samples
osfi-gate sample draws from ledgers by a rule stated in docs/audit-trail.md ("Sampling"),
version osfi-gate/sample/v1. Its inputs are the ledger entries above and nothing else.
9. What none of this proves
- That a decision follows from its policy. Re-running the policy over
gate-input.jsonis a separate check (osfi-gate verify --policy-dir); it needs the policy engine, and the standalone verifiers do not do it. - That the scanners saw everything. The evidence records what they reported.
- That the evidence is authentic, from the chain alone. Someone who can rewrite the store and re-sign it can produce a ledger that passes every check in §5. Signatures with a key the writer does not hold, and anchors (§7) held where the writer cannot reach, are the defences.
Machine-readable results
Each verifier's --json output names every finding by a stable code, so that two
implementations can be compared regardless of wording:
| Code | Meaning |
|---|---|
ledger.malformed |
A line is not a JSON object, or a field has the wrong type (including a non-integer number). |
chain.seq |
seq is not the next integer. |
chain.prev_link |
prev_entry_hash is not the previous entry's entry_hash. |
chain.entry_hash |
entry_hash does not recompute. |
bundle.missing |
No directory or archive for the bundle. |
bundle.unsafe_archive |
The archive holds a link, an absolute path, or a path containing ... |
manifest.digest |
§5.1 |
manifest.position |
§5.2 |
file.missing, file.changed, file.extra |
§5.3 |
statement.digest, statement.coverage |
§5.4 |
signature.invalid |
The signature does not verify. |
signature.no_key |
The bundle is key-signed and no public key was given. |
signature.key_mismatch |
The given key's id is not the key_id the entry records (a note when the signature still verifies). |
signature.unsupported |
Keyless, or a signature file this version does not define: not checked. |
signature.no_tool |
The verifier has no way to check an ECDSA signature here (the Python verifier needs the openssl command): not checked. |
anchor.truncated, anchor.rewritten |
§7 |