What survives when a multi-agent system architecture loses the app mid-run

By the Cerevisor team

A harness is the layer that launches AI agents, hands them work, and reports what happened. Ours had a bug that lived in that last part for months, and the comment we eventually wrote above the fix describes it better than any bug report did: a thirteen-second sibling run flipped a seventeen-minute run’s status to completed, and every running visual on the canvas went dark mid-run.

Nothing had actually stopped. The long run was still going, still spending, still producing files. The screen just stopped believing in it, because two runs of the same workflow were writing to the same status slot and the short one finished first. Not a crash. A projection problem, which is the shape most run-reporting bugs in multi-agent tooling actually take.

We shipped the fix, along with restart survival for run state, in Cerevisor 2.3.0 today. The interesting part is not that runs now survive a restart. It is what we decided a survived run is allowed to claim.

Where a multi-agent workflow actually keeps its run state

A run is one execution of a workflow, from launch to whatever ends it. While it is live, everything interesting about it sits in memory in the app window: which agents are active, how many turns each has taken, what it has cost so far, what it produced. Close the app and that is gone, which is fine for a thirty-second run and infuriating for one that ran overnight.

There were two obvious places to put it. Inside the project file, so the run travels with the work. Or in a single settings blob, which is where small app state usually ends up.

We used neither. Run records live in their own store outside the project file, one file per finished run, with a small index alongside them. Two reasons, both boring in the way that load-bearing decisions usually are.

The first is cost. A few hundred heavy run records inside one settings blob means rewriting the whole blob every time a run ends, which is the exact cost the app’s cached storage layer already exists to avoid.

The second is honesty about portability. A project file stays a description of the work, not a log of every time somebody executed it. Move it to another machine and no run history follows. That is a real limitation, and better said plainly than implied away.

The store is bounded, oldest evicted first, with one exception that matters more than the cap: the newest record for each workflow is pinned. A workspace with forty workflows never loses the last run of any of them just because one busy workflow generated a lot of history. And a record too large to fit is dropped rather than written half-complete, since a partial view is worse than none.

The ownership rule that stopped one run from overwriting another

Now the thirteen-second sibling.

Every launch mints a fresh identifier for that run. Before 2.3, the interface kept one status per workflow, which is reasonable right up until the same workflow is running twice at once. Then both runs write to the same place, and whichever finishes first gets to describe both.

The fix has two halves. Each live run now keeps its own copy of its state: its own paused agents, its own cost, its own turn count, its own output paths. The shared per-workflow status became a projection of the newest live run rather than a shared bucket. A sibling can no longer carry off another run’s numbers.

Then the rule the whole thing exists for. A completion event may only write the final state when no other run of that workflow is still live. If one is, the display re-projects onto the survivor, and the finishing run is not allowed to overwrite it. The dying run still records its own history correctly. It just does not get to speak for the workflow.

That is a small amount of code and a genuinely different stance from “last write wins”, which is the default almost everywhere because it stays invisible until the day somebody runs two things at once.

Concurrency bugs in agent tooling rarely show up as a crash. They show up as a status that is confidently wrong, which is much harder to notice and much more expensive to believe.

Restored is not resumed, and that is the whole design

Here is the part that reads as either obviously right or as a missed opportunity, depending on what durability is expected to buy.

When the app reopens a workspace, it reattaches the last known run status to every workflow, so the operator can see which one was being worked on without opening each in turn. Open a workflow and its full run detail restores. If the app crashed or was force-closed during a run, a recovery prompt appears on the next launch showing the workflow name, when the run started, how many turns and how much cost were preserved, and which agents were interrupted.

And then nothing happens.

A restored run can never be live, so any agent that was mid-flight comes back as stopped, and so does the run itself. The recovery prompt is explicit: interrupted agents will not spend or act until a person resumes one, and resume is per agent, not per run.

We went back and forth on automatic resume and landed on the conservative side, for a reason that has nothing to do with technical difficulty. A half-finished multi-agent run has already had effects in the world. Files written, commits made, tickets moved, money spent. Replaying an agent that already did half its job is not neutral, and the harness genuinely does not know which half landed.

A restored run is a record, not a resurrection. The cost that comes back is what was actually spent, and the agents that come back are visibly finished rather than pretending to still be working.

The rest of the field is moving the other way, for defensible reasons. Cursor’s changelog on August 19 described agents that hold a goal until it is met and stay on course through long sessions, quietly moving the unit of work from the session to the objective. Anthropic shipped a shutdown option for self-hosted runners on August 20 that keeps serving attached sessions for a set number of minutes before parking the rest, which is the same instinct expressed as draining rather than killing. Both are right when the run lives on a server nobody is watching. Cerevisor runs on a laptop, in front of a person, spending that person’s money, and that changes the default.

Nobody has this fully solved. Google’s Antigravity changelog on August 20 lists a fix for conversations that got stuck showing as still running and never returned to idle, most often after somebody pressed Stop.

Remembering a failed sign-in once instead of forty times

The same principle produced the other change worth describing, and it fixes a failure anyone running a large workflow on a subscription plan has probably watched happen.

A provider sign-in expires. The first agent tries to start, fails on authentication, and errors. Then the second. Then the third. In a workflow with dozens of agents, the same dead credential gets exercised dozens of times, and the operator watches a long, slow cascade of one identical message.

That cascade is not a cosmetic annoyance, and there was a good illustration of why this week. The Register wrote on August 19 about GitHub’s postmortem of an outage two days earlier, and the detail that stopped me was not the provider side of it.

The client made the outage worse. A retry loop inside the editor turned a slow endpoint into a traffic storm and then slowed its own recovery. That is the same shape as forty agents hammering one dead credential, just with more zeroes on it. Retry behaviour is a harness design decision, and improvising it is expensive.

So now the first unambiguous authentication failure is remembered for the life of the app process, keyed by the shape of the credential rather than its name. Every remaining agent that would have used it fails immediately, with a message saying the failure was remembered from an earlier attempt, and the underlying tool is never launched at all. The memory clears the moment that credential is saved again in provider settings, or when a credential test passes.

Only strong authentication signals are allowed to set that memory. A vague error surfacing in a model’s own output text does not get to convince the app that an entire provider is dead, which is the failure mode that makes this kind of caching dangerous when it is done carelessly.

Three questions to ask before trusting a harness with an overnight run

Durability claims are hard to compare because everybody uses the same word for different guarantees. These three checks separate them quickly, on any tool, not only this one.

1. Run the same workflow twice at once, then let the short one finish.

   Watch what the interface then says about the long one. If it goes quiet, goes green, or goes dark, the tool has one status slot and two writers.

2. Force-quit during a run and reopen the app.

   Ask what came back: nothing, a record, or actually restarted agents. All three are defensible positions. Only one of them is what most tools imply.

3. Expire a credential mid-run and count the errors.

   One clear failure is a harness that learned something. Forty identical ones is a harness retrying a fact it already had.

If the answer to the second check is “actually restarted agents”, that is not wrong, but it earns a follow-up: what stops an agent from redoing work it already finished, and paying for it twice.

The full list of what changed is in the 2.3.0 release notes, and the exact behaviour of the recovery prompt, including what Discard does and what stays in analytics afterwards, is documented in the popup reference.

What I keep coming back to is that durability turned out to be mostly an honesty problem wearing an engineering costume. The hard question was never how to write the file. It was deciding what the restored thing is permitted to say about itself.