The seductive part of a snapshot is how much work it appears to erase. Boot once, do the expensive setup, and suddenly three agents can start from the same useful place.

Boot the machine once. Install the dependencies. Index the repository. Warm the runtime. Capture that state, then branch it three ways and let three agents explore different paths. From a performance perspective, it feels almost magical.

The uncomfortable question comes a moment later: what else did we just copy?

Memory is obvious. The process tree, page cache, open descriptors, random state, and half-finished computation are all part of the appeal. Less obvious are session keys, network leases, secret bindings, sequence counters, audit identity, and every other piece of authority that happened to be live when the snapshot was taken.

That led to the rule behind MVM’s fork and restore work:

A child may inherit computation. It does not inherit trust.

Why branching is worth doing

Agent workloads naturally repeat an expensive prefix.

A code agent installs a large dependency set, scans a repository, and builds an index before it can try a repair. A browser agent reaches a useful test state and then needs to compare several possible actions. A controller prepares a toolchain once and wants multiple workers to begin from that same point.

Starting from zero for every branch wastes time and memory.

flowchart LR
    B[Boot] --> S[Set up environment]
    S --> W[Warm useful state]
    W --> C[Checkpoint]
    C --> A[Branch A]
    C --> D[Branch B]
    C --> E[Branch C]

The performance story is straightforward: pay for the common prefix once, then use copy-on-write state for the divergent work.

The security story begins where the diagram ends.

A snapshot quietly contains a security contract

Suppose the parent is running under a signed execution plan. That plan binds an image, resource limits, network policy, service grants, secrets, host shares, an audit identity, a validity window, and replay protection.

If a child simply resumes the saved bytes, it can also resume assumptions that were only valid for the parent:

  • two machines may believe they have the same identity
  • both may reuse one authenticated host session
  • a secret issued to the parent may remain live in the child
  • sequence numbers and replay state may diverge
  • an expired plan may continue through restored memory
  • two audit streams may become indistinguishable
  • one permitted child may turn into an arbitrary number of descendants

The snapshot implementation can be flawless while the authorization model is completely broken.

That was the shift for me: the checkpoint is not a paused permission. It is data that may be used as the starting point for a new admission.

Treat the checkpoint as an input

The restore path in MVM routes the child back through the same admit-and-boot boundary used for a new machine.

sequenceDiagram
    participant P as Parent VM
    participant H as Host runtime
    participant C as Checkpoint store
    participant A as Admission
    participant N as Child VM

    P->>H: checkpoint request
    H->>C: save memory and machine state
    H->>P: resume parent
    H->>A: request fresh child plan
    A->>A: verify image, limits, grants, lineage
    A-->>H: signed child admission
    H->>N: restore checkpoint with fresh identity
    H->>N: deliver new channels and secrets
    N-->>H: child ready

The checkpoint provides the computational starting point. Admission provides permission for the new execution to exist.

That means a child receives a new boot identity, fresh per-instance secrets, new host-channel state, and its own audit stream. The parent and child may share immutable artifacts and memory pages, but they do not share the live authority attached to the parent’s session.

Blind cloning would be faster. It would also create a machine whose identity had split in two.

The boot belongs in the identity

A machine name is not enough to authorize anything.

The same named VM can stop and start. A checkpoint can be restored more than once. A branch may eventually move to another node. Transport coordinates such as a vsock CID can be reused.

MVM therefore binds authorization to a per-boot identity:

VmInstanceIdentity {
  node_id,
  vm_id,
  boot_id,
  plan_digest,
}

The boot_id keeps a stale channel from becoming valid again after restart. The plan_digest binds the instance to the exact contract that admitted it. The node and VM identifiers locate the execution without pretending that a socket address is a durable principal.

The tuple matters more than any one field. A lease belongs to this particular boot of this machine under this exact plan.

Restore changes that tuple, so every live channel has to change with it.

The child cannot keep the parent’s channels

A vsock connection in MVM is more than a byte stream. It carries authenticated session state, authorization context, sequence numbers, flow registries, and service bindings.

Forking that state creates two senders who can both produce frames that appear to come from the same session. The cryptography may still validate while the identity behind it has become ambiguous.

The restore boundary therefore invalidates the old session and creates new:

  • session keys
  • sequence counters
  • network and writer leases
  • flow registries
  • service bindings
  • audit context

This is one of the places where the fastest possible snapshot path and the safe path disagree. The fast path wants the guest to resume immediately. The safe path pauses long enough to re-key and re-bind the machine before useful work continues.

That pause is part of the product, not overhead we should hide.

Warm parents and real checkpoints are different things

I found it useful to separate two forms of reuse that are often called “snapshots.”

A clean warm parent is prepared specifically for reuse. It can contain the kernel, initramfs, immutable root layers, verified runtime overlay, guest-agent protocol, CPU and memory shape, device topology, and a known warmup profile.

It must not contain tenant authority: no raw secrets, no tenant grants, no mutable workload state, no live host connection, and no host path into somebody’s checkout.

A dirty checkpoint captures a real workload in progress. It may contain valuable tenant state and belongs to a specific execution lineage. Restoring or branching it should require explicit authorization.

The storage format may be similar. The trust class is not.

Treating every checkpoint as a reusable pool template is how secrets and mutable state move from one workload into another while everyone is admiring the startup time.

Delegation has to shrink

Forking becomes even more interesting when one VM may request children of its own.

A controller can reasonably create workers. A worker may create a smaller helper. But the capability relation must be monotonic:

Cchild⊆CparentC_{\text{child}} \subseteq C_{\text{parent}}

The same is true of the budget:

∑iBchildi≤Bparent\sum_i B_{\text{child}_i} \leq B_{\text{parent}}

Otherwise, “may spawn a worker” becomes “may manufacture authority.”

In practice, every child request is checked as a subset of the parent plan: fewer repositories, a shorter lifetime, a smaller network allow-list, lower CPU and memory, fewer descendants, and narrower service grants.

Snapshots make branching cheap. They cannot be allowed to make permission amplification cheap too.

Keep ancestry, not shared identity

Fresh authority does not mean losing the history of where the child came from.

The audit graph should still record which parent created the checkpoint, which plan authorized the fork, which immutable artifacts were shared, and where the executions diverged.

flowchart LR
    P[Parent execution] --> K[Checkpoint]
    K --> C1[Child 1 execution]
    K --> C2[Child 2 execution]
    P -. authorized .-> C1
    P -. authorized .-> C2

That is lineage, not identity reuse.

During an investigation, “these machines began from the same state” and “these machines had the same authority” are very different statements. The system should be able to prove the first without accidentally creating the second.

The review question I use now

When I look at a fork or restore design, I ask what remains identical after the operation.

It is fine for these to match:

  • immutable artifact digests
  • kernel version
  • root filesystem lower layers
  • saved memory pages
  • checkpoint lineage

These should not:

  • boot identity
  • plan nonce
  • session keys
  • network lease
  • secret material
  • audit stream identity
  • child budget allocation

Try a restore on paper

Take one checkpoint and draw two columns: may be copied and must be renewed. Put memory pages, artifact digests, session keys, network leases, audit identity, and child budgets somewhere.

If an item is difficult to place, that is not a documentation problem. It is a missing restore rule.

That pair of lists is surprisingly effective. If the implementation cannot explain why an item is on one side or the other, the restore boundary is not finished.

I still want MVM to make speculative agent work cheap: checkpoint a useful state, branch it, try several paths, keep the winner, and discard the rest.

The part I no longer describe as reusable is trust. A child has to be admitted, named, budgeted, connected, and audited as a new machine. The snapshot may save the computation; it does not get to save the security decision.

Next: 200 Milliseconds of What, Exactly?