mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-26 18:31:34 +01:00
DottaandPaperclip
0f5fafe16b
## Thinking Path > - Paperclip lets people manage AI agents and their work. > - The Runner connects provider sessions to task state, replies, and delegated work. > - Full-stack tests found lost final replies, rejected helper calls that stopped the parent, and unnecessary process restarts. > - A completed child could also receive a new assignment wake that the scheduler then cancelled. > - This change fixes those boundaries and gives agents clearer teammate instructions. > - The tests retain strict completion and process-continuity requirements. ## Linked Issues or Issue Description **What happened?** A generated attachment comment could suppress an agent's final reply. A known Codex helper could stop its parent when it requested a Paperclip tool. Native Daytona processes restarted between turns because Paperclip minted an unused GitHub broker token. Reassigning a completed child queued a run that immediately cancelled. Revision instructions also allowed agents to do work assigned to a named teammate themselves. **Expected behavior** Keep the final reply. Reject helper tool requests without borrowing parent authority or stopping the parent. Keep an unconfigured sandbox process alive between turns. Treat assignment-only changes to completed tasks as metadata changes. Preserve explicit teammate assignments during revisions. **Steps to reproduce** Run the retained Runner E2E cases for file handoff, teammate reuse, Daytona warm continuity, and Legacy Claude interview/plan acceptance. The focused regression tests reproduce the reply, helper, process-lifetime, and assignment-wake defects without provider calls. **Paperclip version or commit** The live lifetime and completion campaign used `db3857807`. This PR replays the changes on master `c65fc9e3c`. See Verification for the limits of that evidence. **Deployment mode** Isolated local instances and native Runner sessions in Daytona sandboxes. Related work: #13546 handles a different queued-run issue after an issue-lock compare-and-set failure. This PR prevents the unnecessary assignment wake earlier. #13410 covers retained user services; this PR covers the provider process. No duplicate fix was found. ## What Changed - Exclude generated deliverable-binding comments from final-reply deduplication. Preserve the attachment and explicit user-facing replies. - Reject Paperclip tool and input requests from known Codex helper threads without terminating the parent. Keep unknown-thread rejection intact. - Explain how to hire or reuse a persistent teammate and preserve named delegation on revisions. Update generated protocol fixtures. - Use stable, token-free GitHub wrappers for unconfigured native sandboxes. Preserve credential isolation, configured-account rotation, and cleanup after partial staging failures. - Do not queue assignment-only wakes for done or cancelled tasks. Keep explicit reopening behavior. - Make warm-continuity fixtures create real review cards. Read the persisted final response selected by production presentation logic. Missing selected evidence still fails. ## Verification - Before rebase: 560 focused route, native-executor, and launcher tests passed. The new regressions were reproduced before their fixes. - Live E2E: Legacy Claude interview/plan acceptance passed 3/3 repetitions. Daytona warm continuity passed 2/3 full repetitions. Each successful run retained one process and provider session for all three turns. - The remaining Daytona repetition stopped after a same-URL browser reload left the page blank. Both completed turns retained the same process. Its failed verdict remains unchanged; this PR does not claim the blank-page cause is fixed. - Reports: https://pages.paperclip.ing/runner-e2e-lifetime-race-20260920/investigation.html and https://pages.paperclip.ing/runner-e2e-behavior-followups-20260919-results/investigation.html - Post-rebase `pnpm build` and `pnpm -r typecheck` passed. All 414 Runner E2E harness unit tests and its typecheck passed. Codex protocol tests: 88 passed, 2 ignored. - Latest-head CI: 55 successful checks and 2 intentional skips. Greptile: 5/5 with no review threads. The unchanged workspace exposure tests hit a fixed-port collision on the first CI attempt; their local suite passed (25 tests, 3 platform skips), and the CI shard passed on one retry. - The duplicate local `pnpm test:run` was stopped after the full hosted general and serialized test shards passed. It did not finish locally and is not counted as a local full-suite pass. ## Risks Configured GitHub accounts retain run-scoped credential rotation and can still restart warm processes. That limitation requires a separate design. Known provider helpers cannot use Paperclip coordination tools directly; they must return findings to the parent. The delegation prompt is an instruction, not an enforced guarantee; Codex Mini hiring/reuse failures remain open. No schema or workflow changes are included. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution, and parallel coding agents. The exact deployment suffix and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub issues and shared reports) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; full hosted test shards also pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
263 lines
8.1 KiB
JSON
263 lines
8.1 KiB
JSON
{
|
|
"schema": "paperclip-runner/native-execution/v1",
|
|
"provenance": "seeded",
|
|
"identity": {
|
|
"runId": "run_evals_seeded",
|
|
"caseId": "case_semantic_denial",
|
|
"configId": "config_deterministic_v1",
|
|
"attemptId": "attempt_semantic_denial_0001"
|
|
},
|
|
"input": {
|
|
"caseSha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
|
"configSha256": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
|
"deterministicSeed": "paperclip-evals-v1-seed"
|
|
},
|
|
"runner": {
|
|
"package": {
|
|
"name": "@paperclipai/paperclip-runner",
|
|
"version": "0.0.0"
|
|
},
|
|
"binary": {
|
|
"name": "paperclip-runnerd",
|
|
"sha256": "sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
|
|
},
|
|
"prpVersion": 1,
|
|
"nativeExecutionVersion": 1,
|
|
"catalogVersion": 1,
|
|
"catalogSha256": "sha256:73321806bf90df701a9677a7a9527c5b3d39485078e4d3fa4bb3a244a74a0c3b",
|
|
"driverContractVersion": 1,
|
|
"driverKind": "paperclip-deterministic",
|
|
"driverVersion": "1.0.0"
|
|
},
|
|
"events": [
|
|
{
|
|
"schema": "paperclip.prp.event.v1",
|
|
"sourceEventId": "evals_seeded_01",
|
|
"sourceSeq": 1,
|
|
"sourceInstanceId": "runner_evals_seeded",
|
|
"sourceKind": "runner",
|
|
"runId": "run_evals_seeded",
|
|
"normalizedSessionId": "session_evals_seeded",
|
|
"turnId": "turn_evals_seeded",
|
|
"itemId": "item_evals_seeded",
|
|
"eventType": "mcp_app.tool_input",
|
|
"schemaVersion": 1,
|
|
"priority": 1,
|
|
"emittedAt": "2026-08-11T12:00:00.000Z",
|
|
"payload": {
|
|
"semantic_tool": {
|
|
"schema": "paperclip.prp.semantic_tool.v1",
|
|
"schemaVersion": 1,
|
|
"phase": "input",
|
|
"operationId": "decide_approval",
|
|
"callId": "call_evals_seeded",
|
|
"correlation": {
|
|
"runId": "run_evals_seeded",
|
|
"normalizedSessionId": "session_evals_seeded",
|
|
"turnId": "turn_evals_seeded",
|
|
"itemId": "item_evals_seeded"
|
|
},
|
|
"idempotencyKey": "decide_evals_seeded",
|
|
"content": {
|
|
"digest": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd",
|
|
"redactionDisposition": "redacted",
|
|
"references": []
|
|
}
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"schema": "paperclip.prp.event.v1",
|
|
"sourceEventId": "evals_seeded_02",
|
|
"sourceSeq": 2,
|
|
"sourceInstanceId": "runner_evals_seeded",
|
|
"sourceKind": "runner",
|
|
"runId": "run_evals_seeded",
|
|
"normalizedSessionId": "session_evals_seeded",
|
|
"turnId": "turn_evals_seeded",
|
|
"itemId": "item_evals_seeded",
|
|
"eventType": "mcp_app.tool_result",
|
|
"schemaVersion": 1,
|
|
"priority": 1,
|
|
"emittedAt": "2026-08-11T12:00:00.010Z",
|
|
"payload": {
|
|
"semantic_tool": {
|
|
"schema": "paperclip.prp.semantic_tool.v1",
|
|
"schemaVersion": 1,
|
|
"phase": "result",
|
|
"operationId": "decide_approval",
|
|
"callId": "call_evals_seeded",
|
|
"correlation": {
|
|
"runId": "run_evals_seeded",
|
|
"normalizedSessionId": "session_evals_seeded",
|
|
"turnId": "turn_evals_seeded",
|
|
"itemId": "item_evals_seeded"
|
|
},
|
|
"idempotencyKey": "decide_evals_seeded",
|
|
"content": {
|
|
"digest": "sha256:eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
|
|
"redactionDisposition": "redacted",
|
|
"references": []
|
|
},
|
|
"outcome": "denied",
|
|
"code": "required_claim_missing",
|
|
"retryable": false,
|
|
"authorizationBoundary": "grant",
|
|
"operationReceiptId": "receipt_evals_seeded",
|
|
"auditReceiptId": "audit_evals_seeded"
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"schema": "paperclip.prp.event.v1",
|
|
"sourceEventId": "evals_seeded_03",
|
|
"sourceSeq": 3,
|
|
"sourceInstanceId": "runner_evals_seeded",
|
|
"sourceKind": "runner",
|
|
"runId": "run_evals_seeded",
|
|
"normalizedSessionId": "session_evals_seeded",
|
|
"turnId": "turn_evals_seeded",
|
|
"eventType": "run.result.proposed",
|
|
"schemaVersion": 1,
|
|
"priority": 0,
|
|
"emittedAt": "2026-08-11T12:00:00.020Z",
|
|
"payload": {
|
|
"schema": "paperclip.run_result.v1",
|
|
"reportedWorkDisposition": "needs_review",
|
|
"summary": "The governed tool effect was rejected and preserved as a denial.",
|
|
"completionClaim": {
|
|
"contractRevision": "evals-native-execution-v1",
|
|
"objectiveSatisfied": false,
|
|
"criteria": [],
|
|
"remainingWork": [
|
|
{
|
|
"description": "An authorized approver must decide.",
|
|
"blocksCompletion": true
|
|
}
|
|
]
|
|
},
|
|
"evidence": [
|
|
{
|
|
"receiptId": "receipt_evals_seeded"
|
|
}
|
|
],
|
|
"verification": [
|
|
{
|
|
"commandOrCheck": "rejected tool effect stays denied",
|
|
"status": "passed"
|
|
}
|
|
],
|
|
"attentionRequests": [],
|
|
"artifacts": []
|
|
}
|
|
},
|
|
{
|
|
"schema": "paperclip.prp.event.v1",
|
|
"sourceEventId": "evals_seeded_04",
|
|
"sourceSeq": 4,
|
|
"sourceInstanceId": "runner_evals_seeded",
|
|
"sourceKind": "runner",
|
|
"runId": "run_evals_seeded",
|
|
"normalizedSessionId": "session_evals_seeded",
|
|
"turnId": "turn_evals_seeded",
|
|
"eventType": "run.terminal",
|
|
"schemaVersion": 1,
|
|
"priority": 0,
|
|
"emittedAt": "2026-08-11T12:00:00.030Z",
|
|
"payload": {
|
|
"schema": "paperclip.prp.terminal.v1",
|
|
"turnTerminalState": "completed",
|
|
"runTerminalState": "succeeded",
|
|
"reportedWorkDisposition": "needs_review",
|
|
"workAssessmentId": "assessment_evals_seeded",
|
|
"statusDecisionId": "decision_evals_seeded"
|
|
}
|
|
}
|
|
],
|
|
"semanticTools": {
|
|
"definitions": [
|
|
{
|
|
"operationId": "decide_approval",
|
|
"schemaVersion": 1,
|
|
"availability": "denied"
|
|
}
|
|
],
|
|
"calls": [
|
|
{
|
|
"callId": "call_evals_seeded",
|
|
"operationId": "decide_approval",
|
|
"eventId": "evals_seeded_01"
|
|
}
|
|
],
|
|
"results": [
|
|
{
|
|
"callId": "call_evals_seeded",
|
|
"operationId": "decide_approval",
|
|
"eventId": "evals_seeded_02",
|
|
"outcome": "denied"
|
|
}
|
|
],
|
|
"denials": [
|
|
{
|
|
"callId": "call_evals_seeded",
|
|
"code": "required_claim_missing",
|
|
"authorizationBoundary": "grant",
|
|
"retryable": false
|
|
}
|
|
]
|
|
},
|
|
"terminal": {
|
|
"schema": "paperclip.prp.terminal.v1",
|
|
"turnTerminalState": "completed",
|
|
"runTerminalState": "succeeded",
|
|
"reportedWorkDisposition": "needs_review",
|
|
"workAssessmentId": "assessment_evals_seeded",
|
|
"statusDecisionId": "decision_evals_seeded"
|
|
},
|
|
"observations": {
|
|
"checks": [
|
|
{
|
|
"id": "tool-effect-denied",
|
|
"passed": true,
|
|
"receiptId": "receipt_evals_seeded"
|
|
}
|
|
]
|
|
},
|
|
"usage": {
|
|
"inputTokens": 12,
|
|
"outputTokens": 8,
|
|
"totalTokens": 20,
|
|
"requestCount": 1,
|
|
"durationMs": 30,
|
|
"cost": {
|
|
"currency": "USD",
|
|
"amountMicros": 0
|
|
}
|
|
},
|
|
"transcript": {
|
|
"uri": "artifact://native/attempt_semantic_denial_0001/transcript.jsonl",
|
|
"sha256": "sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
|
|
"byteSize": 2048,
|
|
"mediaType": "application/x-ndjson",
|
|
"complete": true,
|
|
"eventCount": 4,
|
|
"omissionReason": null
|
|
},
|
|
"artifactRoot": {
|
|
"uri": "artifact://native/attempt_semantic_denial_0001/",
|
|
"sha256": "sha256:1212121212121212121212121212121212121212121212121212121212121212",
|
|
"byteSize": 4096,
|
|
"mediaType": "application/vnd.paperclip.runner-artifact-root+json"
|
|
},
|
|
"artifacts": [
|
|
{
|
|
"uri": "artifact://native/attempt_semantic_denial_0001/denial-receipt.json",
|
|
"sha256": "sha256:1313131313131313131313131313131313131313131313131313131313131313",
|
|
"byteSize": 512,
|
|
"mediaType": "application/json"
|
|
}
|
|
],
|
|
"failure": null,
|
|
"x_fixturePurpose": "Proves seeded provenance, denied effects, and additive field preservation."
|
|
}
|