How much work survives a worker crash?
Move the crash across a real job timeline. See which steps remain durable, which steps replay, and which external writes need reconciliation.
Choose the durable boundary
A trace records what happened. Recovery also needs a durable state record. In LangGraph, a checkpointer saves graph state across execution steps. Identify the step whose result the next worker can load, then decide what evidence proves the corresponding external operation finished. A checkpoint before a payment attempt and a receipt after settlement answer different questions.
Draw one line for elapsed time and another for durable progress. If the worker disappears between two checkpoints, the next worker starts from the earlier durable boundary. Replayed CPU time is usually cheap. A duplicated external write can be expensive even when it took milliseconds.
Persistence - Docs by LangChainTest the write-before-acknowledgement gap
Give a test job a stable operation identifier. Use a harmless stub for its outbound write. Stop the worker after the stub records the identifier and before the worker records completion. Restart from the preceding checkpoint and inspect the stub. A second request with the same identifier should retrieve or confirm the first result under an appropriate idempotency contract.
Use trace spans around the outbound request and checkpoint write to reconstruct their ordering. The simulator flags a write in an interrupted step as uncertain because elapsed time alone cannot establish which side committed it. Verify the receipt before another worker repeats the operation.
Traces | OpenTelemetryPick what the next worker can read
Bedrock AgentCore Runtime can mount persistent storage at a configured path. If you use that path for recovery, put the job record there deliberately and test it after replacing the worker. Include the input revision, the last completed step, and the location of each output the next step consumes. Keep credentials out of that record and give the worker read or write access according to its actual role.
Reserve the workspace for disposable material such as dependency caches and temporary files. Success means a replacement worker can identify the exact input and read a completed step's output before deciding what to rerun. If the record survives but points to a missing file, move the output to durable storage before recording completion.
File system configurations for AgentCore Runtime - Amazon Bedrock AgentCore