mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 17:51:58 +01:00
fix(runtime): recover task creation after a lost acquire reply (#18692)
## Thinking Path > - Paperclip manages agent execution and the resources that execution owns. > - A retained environment task can need a replacement before the next turn. > - Core records the creation attempt before it asks the provider to acquire it. > - A lost reply can leave that journal without the resource ID needed for cleanup. > - This change lets an opted-in provider recover the identity from that recorded attempt. > - Core still requires the existing exact-task termination proof before it releases ownership. ## Linked Issues or Issue Description Refs #18405 and #18658. This is the separate lost-reply follow-up from their recovery audit. **Draft status:** Both prior review findings are fixed in `a66520c8d4e06e0eec6ccb5446190ace0ca3003f`. Fresh standard Greptile is 5/5 with no unresolved threads. Current-head CI is green (53 successful checks/statuses including Snyk, three expected skips). Apex completed 5/5 on this exact head at 2026-11-03T00:54:59Z (request 6592407130), with no unresolved threads. The review cycle is complete; the PR remains draft for human merge review. **What happened?** If a replacement task's acquire reply is never received, the creation journal can reach manual reconciliation without a resource ID. Startup cleanup cannot complete, and ordinary Stop cannot release the run owner. **Expected behavior** When the selected provider supports creation inspection, Core should retain enough binding information to inspect the recorded attempt and continue verified cleanup. An inspected identity must not count as termination proof. **Steps to reproduce** 1. Use a retained task whose provider supports creation inspection. 2. Lose the replacement acquire reply before Core records its resource ID. 3. Restart cleanup and inspect the existing creation journal. 4. Confirm that the exact task is stopped and its termination receipt is saved before cancellation releases the owner. **Paperclip version or commit** Current head `a66520c8d4e06e0eec6ccb5446190ace0ca3003f`, based on master `3a7bb4dac62dbd7a34a55deb665c55df310bdf64`. Tested with isolated PostgreSQL fixtures. **Deployment mode** Local source tests with controlled provider responses. No deployment or live model request. ## What Changed - Save the selected inspection capability and prior opaque binding in the existing creation journal. Acquire through that same worker so a worker replacement cannot change the recorded contract. - Make failed opted-in creations due through the existing cleanup notification and saved retry deadline. - Add optional SDK observation field `creationLease`. Use it only to recover the requested creation's identity and binding, then follow the existing Stop, status, completion and receipt checks. - Keep old entries and providers without the recorded opt-in under their existing reconciliation policy. - Add actual PostgreSQL recovery, cancellation and worker-replacement regressions. Document the provider contract. ## Verification - Current correction: 53 distinct targeted cases pass (20 real-worker IPC, 12 task-recovery PostgreSQL and 21 generic-sandbox cases), with server typechecking. The IPC selection was rerun after the final late-observation ordering refinement; the other tested sources were unchanged. - An assembled live-child IPC and PostgreSQL case proves cleanup and cancellation release the owner despite the original worker never replying and remaining alive. Accepted host work, unrelated acquisitions and late observations remain accounted for. Removing the recovery handoff and restoring the old drain snapshot produce distinct assertion failures. - The unsupported-host control skips all 12 PostgreSQL cases without setup; removing the guard fails the controlled setup. These tests do not constitute live VM/provider qualification. - Initial head `5db6882` passed an exact reference-tree comparison against master plus the reviewed follow-up patch, 97 targeted cases and server typechecking. Those checks predate the final inspection correction. - Previous head `59f87fa` passed 32 targeted cases (11 task-recovery and 21 generic-sandbox cases) and server typechecking. Both `pending` and `needs_reconciliation` observations without `creationLease` preserve ownership and get a future retry deadline. Restoring the previous guard makes both new cases fail. - The reproduced failure was a wrong generic cleanup shape that could resolve its journal while leaving the owner held—not a demonstrated false ownership release. The correction reuses the existing missing-identity guard before the generic decoder and leaves generic sandbox behavior unchanged. - Earlier affected selection: 72 cases passed on the original follow-up base. This overlaps the final selection and is not an additional count of unique final-head tests. - Two isolated negative controls restore the missing task inspection and redirected-worker behaviors. Both fail at the expected recovery and dispatch assertions without changing production source files. - The PostgreSQL cases verify one cleanup lease, both late-result orders, recovery after the old journal deadline, ordinary legacy behavior, retained ownership on missing or mismatched identity, and cancellation acknowledgement after exact termination proof. - The tests use actual database and run-control services with controlled worker callbacks. They do not prove live IPC or production recovery timing. - A compatible provider is needed for the new recovery path. No normal successful-startup database or network call is added. No ordinary TTFT improvement is claimed; the repair removes an indefinite ownership hold for supported lost replies. No full local CI suite was run; GitHub runs the full regression gates. ## Risks The provider must recover the identity for the recorded creation operation and preserve its selected binding. `creationLease` does not authorize a new turn or prove that a process stopped. Older journal entries do not infer a new identity after an upgrade. Unavailable original machine bindings or credentials retain the existing uncertainty behavior; this change does not repair machine or credential loss. No schema migration, deployment or ROM qualification is included. ## Model Used OpenAI Codex, GPT-6 family. Exact model ID and context window are not exposed in this session. Tool-assisted implementation, targeted tests and independent subagent review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
40ea7a9b89
commit
6d72243d8a
9 files changed
+417
-21
No files matched your search
@@ -709,6 +709,7 @@ export interface PluginEnvironmentInspectLeaseParams extends PluginEnvironmentDr
|
||||
}
|
||||
|
||||
export interface PluginEnvironmentLeaseObservation {
|
||||
creationLease?: PluginEnvironmentLease;
|
||||
creationCleanup?: import("./environment-creation-cleanup.js").PluginEnvironmentCreationCleanup;
|
||||
providerLeaseId: string;
|
||||
state: "active" | "stopped" | "destroyed" | "unknown";
|
||||
|
||||
Reference in new issue
Block a user