DISTRIBUTED RUNTIME FOR AGENT WORKFLOWS

Every run is a snowflake.

Safe harness engineering for self-modifying, distributed agent workflows — on your own machines.

Quickstart

Shell

npm install -g @baochunli/flakes
flakes skill install
flakes up
flakes run --adapter pi "Work in this repository. Run the needed checks."

npm install -g @baochunli/flakesflakes skill installflakes upflakes run --adapter pi "Work in this repository. Run the needed checks."

MEASURED, NOT ASSUMED

Safety and recovery under real faults.

Each result names the condition it was measured under.

10,000
faults injected into the real store over 250 trials
0
invariant violations across all eight invariants
0.69 s
median store resume after a terminated waiting harness
9.49×
more tokens ingested by cold than warm rounds in one eight-round trace
288 / 826 / 1,073
authored TypeScript lines — Flakes / LangGraph / Temporal

Median resume breakdown

840 token units

one round lost after a mid-round termination, in every trial and never more

0.24 s

detect

0.26 s

reassign

0.19 s

deliver the saved resume event

WHY A RUNTIME

Agent workflows outgrew one machine.

Across hosts and retries, a lost reply or stale process can turn one intention into zero or two durable effects. Flakes makes coordination a versioned operation recorded independently of the sandbox that executes it.

Retries duplicate effects: A lost reply leaves the caller unable to tell whether work committed, so a retry can apply it twice.
Tagged, exactly-once admission: A globally unique tag binds the operation and its recorded outcome in one transaction.

Stale work mutates new state: A delayed process can wake after reassignment and try to change work it no longer owns.
Assignment fencing: Every state-changing request is fenced to the task’s current assignment.

A crash loses the wait: An in-memory coordinator can disappear while children are still working.
Durable yield and deterministic replay: Each await becomes a durable yield; deterministic replay resumes from committed state.

ARCHITECTURE

Three tiers, one operation contract.

Orchestration proposes work. A trusted control plane admits and remembers it. User-owned hosts execute only their current assignments.

Flakes three-tier architecturePropose → admit → assign

An orchestration frontend sends a tagged and versioned operation to the durable control plane, which assigns admitted work to user-owned hosts.

ProposeOrchestration frontendA static workflow, an orchestrating agent, or a TypeScript harness proposes work as the graph grows.AdmitDurable control planeA transactional store admits, versions, schedules, and recovers each proposed change.ExecuteUser-owned hostsPhysical machines, virtual machines, and GPU servers advertise capacity and run assignment-scoped sandboxes.

Selected tier

Orchestration frontend

A static workflow, an orchestrating agent, or a TypeScript harness proposes work as the graph grows.

CLI · HTTP · Harness

Selected tier

Durable control plane

A transactional store admits, versions, schedules, and recovers each proposed change.

Control plane · PostgreSQL

Selected tier

User-owned hosts

Physical machines, virtual machines, and GPU servers advertise capacity and run assignment-scoped sandboxes.

Hosts · Pools · Sandboxes

STATE-CHANGING OPERATION

Admitted once, remembered forever.

Globally unique tag: tag
Observed workflow version: version
Semantic request: payload

The assigning transaction verifies ownership, authority, policy, budget, capacity, and placement legality before all durable effects commit.

An identical retry receives the recorded outcome instead of committing a second effect.

HARNESS SDK

Coordination as code.

Harnesses are deterministic TypeScript programs. Model intelligence stays with the agents; exact coordination stays with code that can suspend, replay, and recover.

A complete write–review–revise harness

TypeScript

write-review-revise.ts

export default defineHarness<PaperArgs>(async (ctx) => {
  const { brief, rubric, maxRounds } = ctx.args;
  const writer = ctx.agent({ title: "Write", live: true,
    prompt: brief, output: ctx.output.json() });
  const reviewer = ctx.agent({ title: "Review", live: true,
    prompt: rubric,
    output: ctx.output.json({ schema: Verdict }) });
  const rounds = Math.min(maxRounds,  // replay-stable
    (ctx.budget.total("tokens") ?? Infinity) / PER_ROUND);
  let draft = await writer.send(brief);  // durable yield
  for (let r = 1; r <= rounds; r += 1) {
    const review = await reviewer.send(draft.text());
    if (review.json().approved) break;
    draft = await writer.send(review.text());
  }
  await writer.finish(); await reviewer.finish();
  return ctx.succeeded({ data: { draft: draft.json() } });
});

export default defineHarness<PaperArgs>(async (ctx) => {  const { brief, rubric, maxRounds } = ctx.args;  const writer = ctx.agent({ title: "Write", live: true,    prompt: brief, output: ctx.output.json() });  const reviewer = ctx.agent({ title: "Review", live: true,    prompt: rubric,    output: ctx.output.json({ schema: Verdict }) });  const rounds = Math.min(maxRounds,  // replay-stable    (ctx.budget.total("tokens") ?? Infinity) / PER_ROUND);  let draft = await writer.send(brief);  // durable yield  for (let r = 1; r <= rounds; r += 1) {    const review = await reviewer.send(draft.text());    if (review.json().approved) break;    draft = await writer.send(review.text());  }  await writer.finish(); await reviewer.finish();  return ctx.succeeded({ data: { draft: draft.json() } });});

Live children3–7live: true keeps the writer and reviewer warm across rounds.
Budget-derived bound8–9The maximum round count comes from the same token pool admission enforces, so it is stable on replay.
Durable yields10–14Each awaited send records the wait, releases the sandbox, and resumes from committed state.

A harness simply moves the intelligence to where it is irreplaceable, and leaves coordination to a substrate that can make it durable, exact, and free.
Flakes research paper

Replay stays exact.

With the journal on, recorded responses make cooperative resume linear. If edited source derives a different operation, replay halts at the first divergent call, names it, and commits no side effects.

5n+1

durable reads across n cooperative resumes with the journal on

161

durable reads at n=32 with the journal on

n²+3n+1

durable reads across n cooperative resumes with the journal off

1,121

durable reads at n=32 with the journal off

200

seeded source-drift runs halted at precisely the altered position

THE COST OF ITERATION

Live task rounds stay warm.

A live child keeps its provider thread across iterations and sends only the new increment. Respawning rebuilds the conversation each round.

One preregistered eight-round trace.

Measured once per arm; warm reused a live child and cold rebuilt the conversation each round.

Measured at round eight

One preregistered eight-round trace. Measured once per arm; warm reused a live child and cold rebuilt the conversation each round.
Warm and cold cumulative token ingestion	Cumulative token ingestion	median time to first token
Warm live rounds · linear	3,482 tokens ingested by the warm arm	4.1 s median time to first token, warm
Cold respawns · quadratic	33,038 tokens ingested by the cold pattern	8.3 s median time to first token, cold

Cold cost grows quadratically with depth. Warm cost grows linearly. Eight rounds are where the separation starts, not where it ends.

8rounds in the preregistered trace9.49×cold-to-warm cumulative token ingestion1.1%difference between the cost model and the cold endpoint

Model predictiondifference between the cost model and the cold endpoint

Warm live rounds · linear: 3,482tokens ingested by the warm arm; 4.1 smedian time to first token, warm
Cold respawns · quadratic: 33,038tokens ingested by the cold pattern; 8.3 smedian time to first token, cold

OWNER-OPERATED CAPACITY

Your machines, your rules.

Bring physical machines, virtual machines, or GPU servers. Hosts advertise what they can run; the durable control plane decides where admitted work may execute.

Host

An owner-operated machine connected to the control plane.

Pool

A versioned offer describing an adapter, provider, resource bounds, capabilities, and locality.

Sandbox

Replaceable capacity that executes only its current assignment.

Choose the boundary for the work.

Worktree

Default

Runs directly as your user with your environment, filesystem, and network access. Use it with an agent you already trust on your own machine.

No isolation

MatchLock

Hardened

Runs inside a microVM with network allowlisting and resource bounds.

MicroVM isolation

LEGALITY BEFORE OPTIMIZATION

Placement is an admission decision.

The control plane evaluates admissibility inside the assigning transaction, before any optimizer sees the task–sandbox pair.

Same owner
Required adapter and capabilities present
Policy and capacity satisfied
Every input available under the run’s data-movement policy

Credentials narrow with the assignment.

A running task holds only a temporary, assignment-scoped capability — never the user’s credential.

A policy denial remains durable.

In the residency experiment, a run pinned to one region was admitted after its region string changed, but never scheduled because no compliant capacity remained.

POLICY_DENIED

Admitted · not scheduled

Move bytes through named paths.

Bytes are not implicitly visible across sandboxes: Flakes moves them through explicit, named paths where ownership, policy, location, and bounds can be checked, and a saved location does not promise they remain reachable. Placement requires every input to be available under the run’s data-movement policy, evaluated inside the assigning transaction.

Named paths · Policy-gated placement · Durable denial

DURABLE EVIDENCE

Observable by default.

The console and Herdr show the same durable work from different surfaces. Processes can disappear; task state and bounded transcript evidence remain.

Illustrative Flakes run in the console and HerdrIllustrative interface

Console

RUNNING

Write · review · revise

Runs
Tasks
Transcript
Artifacts

Review the current draft

Live transcript

WriterDraft ready for review.
ReviewerOne blocking finding: state the token trade-off.
WriterRevision submitted.

Follow upSteer

Workpad

Current plan, progress, findings, and handoff notes.

Artifacts

Metadata can outlive payload availability; downloads use a supported named location.

Herdr

Live task panes in your terminal

reviewWAITING

Waiting for the next durable round.

Run state is a derived label, not a flag.

Flakes derives the current label from required tasks after every graph change.

RUNNING over WAITING over QUEUED

RUNNING

At least one required task is reserved for a sandbox.

WAITING

At least one task yielded on a durable wait and is parked in the store.

QUEUED

Work remains, but nothing is live or parked on a durable wait.

Transcripts outlive the sandbox

Bounded transcript sessions upload while a task runs and remain readable after the source sandbox disconnects.

Workpads reject stale edits

A version check keeps an older note from silently overwriting newer work.

Artifacts keep explicit locations

Metadata can outlive payload availability; downloads use a supported named location.

Follow up or steer

Start a next round for live-idle work, or redirect a turn already in progress.

AN EXISTENCE PROOF

Built in itself.

A substantial fraction of Flakes was planned, written, and reviewed by harnesses running on Flakes itself.

A system that stays safe while its own authors are agents inside it is evidence that durable admission can support the way agent software is now built.

372,000+

lines of Flakes code in the reported codebase

THE FULL CONTRACT

An honest comparison.

The same write–review–revise loop was implemented natively and as a faithful port for each baseline.

Flakes, LangGraph, and Temporal on the same revision loop
Contract or measured result	Flakes	LangGraph port	Temporal port
Exactly-once effects	Tagged operation admitted once in the durable store	Not evaluated under Flakes’ exactly-once contract	At-least-once activities; external effects can repeat
Placement enforcement	Enforced before scheduling	No input carries the constraint	Task-queue naming; no built-in enforcement
Durable waits and recovery	Yield releases the sandbox; the store resumes the task	Checkpoints bound loss; the application must re-invoke	Workflow replay with cold activity retries
Deterministic resume and source drift	Journaled replay; first divergent call halts visibly	Checkpoint continues without a source-drift check	History replay detects workflow nondeterminism
Authored TypeScript	288 LOC	826 LOC	1,073 LOC
Median time to first visible token	4.1 s · warm child	19.4 s · cold prompt	21.9 s · cold prompt
Uncached tokens	634k	331k	362k

Fairness: each baseline was held to its own contract, not to Flakes’ stronger contract.

The trade-off: live agents consume more uncached tokens.

Flakes runs full agents that bootstrap, inspect mounted workspaces, and report rounds durably. In this suite, a median of one round left little time for warm threads to amortize that overhead.

634k

uncached tokens consumed by Flakes

331–362k

uncached tokens consumed by the ports

1.8–1.9×

Flakes uncached-token premium over the ports

The premium buys real live agents, enforced placement, durable rounds, and the fastest interaction in the measured suite.

START ON YOUR MACHINE

From a repository to a durable run.

Install the CLI once, converge the repository’s host and pools, then submit work to a detected adapter.

Before you start

A git repository with at least one commit
An authenticated Pi or Codex CLI
The shared agent skill directory

Install and run Flakes

npm install -g @baochunli/flakes
flakes skill install
flakes up
flakes run --adapter pi "Work in this repository. Run the needed checks."

npm install -g @baochunli/flakes
flakes skill install
flakes up
flakes run --adapter pi "Work in this repository. Run the needed checks."

The default worktree provider needs git on your PATH. MatchLock is optional.

DURABLE WORK, REPLACEABLE MACHINES

Let agents do the work. Keep coordination exact.

Use the console to inspect the durable record, or open the docs for hosts, harnesses, and placement.