A log line can tell me that a VM started.
It usually cannot tell me who allowed it, why they allowed it, which policy they relied on, or what later actions depended on that decision.
That gap becomes obvious once a system starts making security-relevant decisions on behalf of people.
MVM already had a chain-signed audit log. Events were ordered, signed, and tamper-evident. The missing part was not integrity.
The missing part was meaning.
Three different jobs
I separate logs, audit evidence, and provenance now because they answer different questions.
Logs help operate the system.
They answer questions like:
- why did this process crash
- which backend was selected
- how long did the connection take
- what error did the guest return
Audit evidence records security-relevant events in a durable, verifiable sequence.
It answers:
- was this plan admitted
- which image digest launched
- which capability was invoked
- was an egress request allowed or refused
- has the record been altered
Decision provenance connects the event to an actor, rationale, and causal history.
It answers:
- who authorized the admission
- why the exception was approved
- which ticket or incident justified it
- which earlier decision influenced this one
- which later decisions should be reviewed if this one was wrong
These layers overlap. They are not interchangeable.
A signed event can still be semantically thin
Consider an audit entry:
{
"event": "plan.admitted",
"plan_id": "sha256:...",
"timestamp": "..."
}If it is signed and chained, I can verify that MVM recorded it and that the history has not been silently rewritten.
I still do not know:
- whether a human or service authorized it
- why it was admitted
- whether it was tied to a change request
- whether it was an emergency exception
- which decision allowed the network grant
- whether a later checkpoint relied on it
Cryptographic integrity does not create semantic context.
It preserves whatever context you chose to record.
Decision records
MVM’s decision-provenance layer adds a typed record for the missing context.
A simplified version looks like this:
struct DecisionRecord {
decision_id: DecisionId,
category: DecisionCategory,
actor: ActorRef,
scenario: DecisionScenario,
reasoning: String,
outcome: DecisionOutcome,
causal_links: Vec<CausalLink>,
metadata: DecisionMetadata,
attestation: AttestationBinding,
}The record is content-addressed. Its identifier is derived from a canonical representation of the decision body:
That gives the decision a stable identity independent of where a derived index stores it.
The chain-signed audit log remains the source of truth.
The decision store is an index that can be rebuilt.
Do not create a second authority
This was the most important design constraint.
A provenance system is attractive because it gathers structured context. That makes it tempting to let the provenance database become the place that decides what is allowed.
MVM does not do that.
The signed execution plan, capability grants, admission policy, and host enforcement remain authoritative. The provenance layer records why those authorities were exercised.
flowchart TB
P[Signed plan and policy] --> E[Enforcement]
E --> A[Chain-signed audit event]
A --> D[Decision record]
D --> I[Rebuildable decision index]
D -. records .-> P
D -. does not authorize .-> E
If the decision index disappears, MVM can rebuild it from the chain.
If the audit chain disappears, the index cannot become a substitute truth.
That direction matters.
Causal links are the useful part
A flat list of decisions is better than a flat list of logs, but it still leaves the investigator doing archaeology.
Security decisions form a graph.
An admission causes a launch. A launch establishes a workload identity. That identity requests egress. An approval changes the plan. A checkpoint captures the result. A restore creates a child. A later refusal may invalidate the assumptions behind all of them.
flowchart LR
A[Admission] --> L[Launch]
L --> E[Egress approval]
L --> C[Checkpoint]
C --> R[Restore as child]
E --> O[External operation]
X[Policy change] -. invalidates .-> E
X -. review impact .-> O
Once the links are explicit, the system can answer better questions:
- trace backward from a decision to its causes
- walk forward to find dependent decisions
- find similar decisions with the same scenario or artifact set
- identify which runs were influenced by a revoked exception
That is much closer to how an operator thinks during an incident.
The actor needs to be real
“Approved by admin” is not useful provenance.
The actor should name the principal that exercised authority:
- human identity
- on-call role
- automated service
- signing key identifier
- control-plane role
This does not mean putting personal details everywhere. It means recording enough to distinguish one authority from another.
A service acting under a promoter key is not the same actor as an operator responding to an incident.
A durable decision should make that distinction visible.
Reasons are dangerous data
Adding a free-form rationale field sounds harmless.
It is not.
People paste secrets into text boxes. They paste customer names, internal URLs, credentials, incident details, and entire chat transcripts.
Decision provenance has to treat rationale as potentially sensitive:
- validate known secret patterns
- avoid copying raw policy payloads
- keep the field short enough to review
- prefer ticket references over duplicated incident content
- define retention
- consider redaction or encryption where required
“Record why” cannot become “record everything.”
The audit chain is durable by design. Bad data inserted there is hard to remove.
Records and receipts are not controls
MVM also has execution receipts. They are useful summaries of what happened.
A receipt is a record. It should not be the durability boundary that decides whether a workload may run.
The chain-signed audit append is stronger because it is the authoritative ordered evidence. If a convenience receipt fails to render after admission, the launch does not suddenly become unauthorized.
That distinction prevents a reporting mechanism from becoming an accidental control plane.
I use the same test elsewhere:
If this storage layer is unavailable, should the security decision change?
For provenance indexes and receipts, the answer is no.
For signed admission and required audit durability, the answer may be yes.
Export is not the architecture
Once decisions are structured, standards become useful.
MVM can export provenance as PROV-O/RDF and as a simpler TIBET-shaped JSON representation. Other formats may make sense for artifacts: SPDX for SBOMs, in-toto for supply-chain steps, C2PA for media authenticity.
The mistake would be designing the runtime around whichever export format is fashionable.
The internal model should describe MVM’s actual decisions. Exporters translate that model for external consumers.
Standards are interfaces, not the source of truth.
Provenance changes reviews
A typed decision record is not only for auditors.
It improves engineering review.
When a pull request adds a new security decision, reviewers can ask:
- which actor is recorded
- what rationale is required
- which causal link connects it to admission
- what artifact digests bind the decision
- whether the event is chain-signed
- whether the derived index can be rebuilt
- whether secrets can enter the metadata
Those are concrete questions.
Without a decision model, the same review tends to end at “add a log.”
What I want to be able to answer
For any MVM workload, I want a future operator to reconstruct the story without guessing:
- Which immutable artifacts ran?
- Which signed plan admitted them?
- Who exercised the authorizing key?
- Why was the action allowed?
- Which policy and ticket were relevant?
- What network and service decisions followed?
- Which checkpoint or child execution came from it?
- Which later decision invalidated or superseded it?
A pile of text logs can contain all of those facts and still fail to answer the questions.
Structure is what makes the evidence usable.
The line I keep
Logging is for seeing the system.
Audit is for proving what the system recorded.
Provenance is for explaining the decision chain.
MVM needs all three.
It also needs them to stay in their lanes.
The provenance layer should make authority understandable. It should never quietly become authority itself.