mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 12:59:13 +01:00
docs(runner): record corrected plan and current qualification evidence
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
5d4b3ac6c4
commit
8d8e57a58f
3 files changed
+23
-9
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Copilot 1.0.88 rich ACP capability audit
|
||||
|
||||
Current checkpoint (2026-09-30): **Copilot profile v7 remains unqualified**. Paid controller/Product harness source is `40064d28522de25fea85c1297f35b41bb8a8897a`; runtime source is `5b8e4454ef0bf12d0bb068c2e41d8c9df9356a1c`. All three target runtime builds are complete. Local attached-async attempt `40064-copilot-async-01` passed all eight matchers on `gpt-5.6-luna`, including exact one-command identity, trusted child exit and independent completion observations before terminal settlement, one run and one exact final marker. Done was visible, and full end integrity, owned process/semaphore retirement and temporary-root cleanup passed. The exact local native-permission-deny-write case `40064-copilot-deny-01` also passed all ten checks, with no target side effect through process retirement, correlated cancellation and successful cleanup/end integrity. These two local cases pass; the provider remains unqualified. Native per-run USD is unknown; included credits stayed 16→16 after async and reached 17 after denial, with possible dashboard lag, and extra usage remained disabled at $0. Result SHA-256: `d1d074bc9df4644794dfa8b020fdeba8a9b0de4618e1f55d1845f7726a2ece33`; reconciliation SHA-256: `8e9b9233ed987e9e522b17a271922fd409d311e04ee5c3422a4b5bd1fa2fce68`. The prior exact head passed CI and Greptile for [draft #14676](https://github.com/paperclipai/paperclip/pull/14676). The earlier v6 denial pass and attached-command failures remain historical, including the doubled final marker and missing settlement observations; they do not qualify v7. See the [comparative capability report](runner-rich-acp-capabilities.md) for exact source identities and remaining local/Daytona, billing and cleanup gates.
|
||||
Current checkpoint (2026-09-30): **Copilot profile v7 remains unqualified**. Paid controller/Product source is `40064d28522de25fea85c1297f35b41bb8a8897a`; native runtime is `5b8e4454ef0bf12d0bb068c2e41d8c9df9356a1c`. Local attached async passed eight checks and local native denial passed ten, with exact command/permission correlation, expected task states, owned cleanup and full end integrity. Daytona hello passed six checks on the current Linux image; one owned sandbox was removed and absence observed through the cleanup window. The local three-turn warm case failed its stable-process invariant despite nine passing behavioral checks. The new controller repair retains the exact agent-files directory with the warm native owner and protects collection authority; it has deterministic and database coverage, but no new paid warm proof yet. The first Daytona denial launch failed before Product dispatch because two executable-digest fields were missing; it created no sandbox and remains a failed attempt. The source `3d21d375d` PR head has green CI and Greptile 5/5. Native per-run USD is unknown; included credits reached 20 with additional usage disabled at $0. The cloud hello infrastructure bound is $0.086931, with provisional analytics. These passes qualify only the named cases. See the [comparative capability report](runner-rich-acp-capabilities.md) for exact evidence, original failures and remaining gates.
|
||||
|
||||
Message identity implementation (2026-09-30, **deterministic evidence and one paid async case; full qualification pending**): the pinned 1.0.88
|
||||
ACP mapper discards `assistant.message_start` and removes native `messageId`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Cursor ACP capability inventory
|
||||
|
||||
Current checkpoint (2026-09-30): **Cursor profile v7 remains unqualified**. Paid controller/Product harness source is `40064d28522de25fea85c1297f35b41bb8a8897a`; runtime source is `5b8e4454ef0bf12d0bb068c2e41d8c9df9356a1c`. All three target runtime builds are complete. Local native-plan attempt `40064-cursor-plan-01` passed all 21 bridge checks and retained empty workspace snapshots (`{}`), but failed passive accepted-plan settlement. Owned cleanup and the full end integrity audit passed; the overall case remains failed. Controller fix `c58a8881f` repairs a body-only hash comparison to use the production schema/policy/contract envelope; 67 focused tests, ten production-contract database tests and server typecheck pass. The native runtime/profile stays unchanged; corrected-controller paid proof remains pending. Native AskQuestion availability remains unverified. The prior exact head passed CI and Greptile for [draft #14669](https://github.com/paperclipai/paperclip/pull/14669); they do not qualify paid behavior. See the [comparative capability report](runner-rich-acp-capabilities.md) for current source identities and remaining gates. The dated observations below retain their original source/profile identities.
|
||||
Current checkpoint (2026-09-30): **Cursor profile v7 remains unqualified**. The corrected local native-plan case on controller/Product source `3d21d375de2b6249d9f5322bf001ce0ce0e052aa` and native runtime `5b8e4454ef0bf12d0bb068c2e41d8c9df9356a1c` passed all 26 checks. Reject, feedback, revision-bound acceptance, browser reconnect, exact delivery and unchanged workspace bytes passed. The single planning run succeeded while the task remained In Progress with an explicit next-message wait; no implementation or task completion is claimed. Cleanup and full end integrity passed. Result SHA-256: `685609f01291db84f40d890ef562473b95065e86907bd964b656ad732f6dd97a`. The earlier controller40064 attempt remains failed. Controller fix `c58a8881f` uses the production schema/policy/contract hash envelope; native runtime/profile bytes are unchanged. That exact Cursor PR head has green CI and Greptile 5/5. All three platform builds exist, but the remaining current-profile local/Daytona cases are still required. Native AskQuestion availability remains unverified. Cursor on-demand usage remains $0 under the approved fixed $25 account cap; native per-run USD is unknown. See the [comparative capability report](runner-rich-acp-capabilities.md) for exact source roles, evidence and remaining gates.
|
||||
|
||||
Evidence updated: 2026-09-29. Candidate: `2026.09.26-dd393fe`. This profile is **not live-qualified**. One authorized task-context prompt on the initial free account returned an upgrade requirement, invoked no semantic tools, and supplied no usage receipt. Its cost is unknown; the account dashboard remained unchanged at coarse precision. The user then selected another account, whose exact-model local Runner Eval passed task-context and history tools with clean terminal settlement. Its dashboard attributed 56K rounded tokens to included usage and no incremental charge; ACP still supplied no token or dollar receipt. The local Product hello subsequently passed through the real browser, server, database, native runner, and authenticated completion tool. A second local Product case passed file creation, editing, command validation, an independent exact-byte matcher, and visible workspace artifact presentation. The merged-source local Product question and revision-bound Plan approval also passed browser response and semantic-tool continuation. A serialized follow-up passed pending semantic-question recovery across a server restart. A verified native ACP probe also denied a shell write before any observed side effect. Native Cursor questions/plans and their recovery, durable permission presentation, interruption, and Daytona execution remain required qualification gates.
|
||||
|
||||
|
||||
@@ -1,17 +1,31 @@
|
||||
# Rich ACP integration and qualification report
|
||||
|
||||
Updated: 2026-09-30 UTC. **Cursor v7, Copilot v7 and Pi v9 remain unqualified.** Current paid controller and Product harness source is `40064d28522de25fea85c1297f35b41bb8a8897a`; runtime source is `5b8e4454ef0bf12d0bb068c2e41d8c9df9356a1c`. Runtime builds for macOS ARM64, macOS x64 and Linux x64 are complete. Builds and credential-free admission do not qualify authenticated behavior. Reviewed PR head `3f1933db551103e6b9e3e5fa1b3bc905becb6e71` differs from controller40064 only by the OpenCode test fixture's explicit Node-copy dereference option; it does not relabel the frozen execution source.
|
||||
Updated: 2026-09-30 UTC. **Cursor v7, Copilot v7 and Pi v9 remain unqualified.** Paid controller and Product harness source is recorded per case: `40064d28522de25fea85c1297f35b41bb8a8897a` or `3d21d375de2b6249d9f5322bf001ce0ce0e052aa`. Native runtime source remains `5b8e4454ef0bf12d0bb068c2e41d8c9df9356a1c`. Runtime builds for macOS ARM64, macOS x64 and Linux x64 are complete. The corrected 3d21 controller uses unchanged compiled UI/package outputs built on 40064; it executes the controller TypeScript from 3d21. These distinct source roles do not relabel the runtime or older failures.
|
||||
|
||||
Before the controller hash fix, exact-head CI passed for Cursor `b74ca5eb976a48e35027be6eb8ea1fb09d181b82` (53 successes, two skips) and Copilot `3f1933db551103e6b9e3e5fa1b3bc905becb6e71` (54 successes, four skips; two superseded duplicate checks cancelled). Those two reviewed heads had no failed or pending checks; Greptile reported 5/5 for both. The subsequent controller fix needs its own CI review. Foundation [#14430](https://github.com/paperclipai/paperclip/pull/14430) and Runner Eval definitions [#33](https://github.com/paperclipai/paperclip-evals/pull/33) are merged. The v7/v7/v9 definition update [#34](https://github.com/paperclipai/paperclip-evals/pull/34) merged as `52f6e897c08d236776a21723257e12c909d71137`, with 136 deterministic tests and 21 validated cells. These definitions are not paid proof. Provider PRs remain drafts.
|
||||
Exact-head CI and Greptile now pass for Cursor `c58a8881f2cb771b35e19e1bf56e08cb89397e83` (53 successful checks, two skips, Greptile 5/5) and Copilot `3d21d375de2b6249d9f5322bf001ce0ce0e052aa` (54 successes, four skips, Greptile 5/5). Copilot's first server-shard run timed out in an unchanged legacy Cursor test; an isolated reproduction and one failed-job rerun passed. Superseded cancelled checks are not failures. The new warm-directory fix needs its own full checks and live proof. Foundation [#14430](https://github.com/paperclipai/paperclip/pull/14430) and Runner Eval definitions [#33](https://github.com/paperclipai/paperclip-evals/pull/33) are merged. The v7/v7/v9 definition update [#34](https://github.com/paperclipai/paperclip-evals/pull/34) merged as `52f6e897c08d236776a21723257e12c909d71137`, with 136 deterministic tests and 21 validated cells. These definitions are not paid proof. Provider PRs remain drafts.
|
||||
|
||||
Current Cursor v7 local attempt `40064-cursor-plan-01` failed passive accepted-plan settlement. All 21 native bridge checks passed; the retained workspace snapshots were `{}`. Owned cleanup and the full end integrity audit passed. The failed case remains failed. Controller fix `c58a8881f2cb771b35e19e1bf56e08cb89397e83` uses the same schema/policy/contract hash envelope as production contract creation; the previous predicate incorrectly hashed the body alone. The production creation/reuse database regression failed before the fix; 67 focused tests, ten database tests and server typecheck pass after it. Runtime bytes and profile identities are unchanged; paid requalification is pending. Copilot v7 local attempt `40064-copilot-async-01` passed all eight matchers: exact one-command correlation, trusted child exit, native-client and independent-marker observations before terminal settlement, attached settlement, one run and one exact final marker. The screenshot shows Done; full end integrity, owned-process/semaphore cleanup and temporary-root cleanup passed. This qualifies this exact local case only. Local `40064-copilot-deny-01` also passed all ten checks, including exact browser denial, absence of the target through process retirement, no alternative native operation and correlated cancellation. Its final screenshot shows the cancelled run and unfinished task as required. Cleanup and the full end integrity audit passed. Denial result SHA-256: `c1fd910ecc4bf9d6df7645f3eb90b45d9cc895cfea9e2f93b4cfdf8a2caf8625`; reconciliation SHA-256: `796954baee57f6ff1f6f05fcfcd8216455e317d462bd68d7b272eabaa640b41b`. Result SHA-256: `d1d074bc9df4644794dfa8b020fdeba8a9b0de4618e1f55d1845f7726a2ece33`; root reconciliation SHA-256: `8e9b9233ed987e9e522b17a271922fd409d311e04ee5c3422a4b5bd1fa2fce68`. Native per-run USD is unknown; the included-credit counter stayed 16→16 after async and reached 17 after denial. Dashboard lag prevents exact per-run attribution. Extra usage remained disabled with a $0 budget; unchanged counters do not prove free inference. Pi v9 has no paid pass and still requires a verifiable OpenRouter spending cap. The optional transport-budget candidate remains frozen and unintegrated.
|
||||
| Current-profile paid case | Exact outcome and evidence limit |
|
||||
| --- | --- |
|
||||
| Cursor local native-plan reject/revise/accept, controller40064 | Failed passive settlement despite 21 passing bridge checks. Workspace snapshots stayed empty; cleanup and end integrity passed. Original failure is retained. |
|
||||
| Cursor local native-plan reject/revise/accept, controller3d21 | All 26 checks passed after the canonical contract-hash repair. Rejection feedback reached the revised plan; the browser accepted that exact revision. The single planning run succeeded and the task stayed In Progress with an explicit wait for the next user message. No implementation or task completion is claimed. Workspace bytes remained unchanged through cleanup. |
|
||||
| Copilot local attached async, controller40064 | Eight checks passed: one exact command, trusted child exit, native-client and independent-marker observations before terminal settlement, one run and one final marker. Done was visible. |
|
||||
| Copilot local native denial, controller40064 | Ten checks passed: exact browser denial, no target side effect through process retirement, no alternate native operation, and correlated cancellation. The task remained unfinished as required. |
|
||||
| Copilot local three-turn warm continuity, controller40064 | Nine behavioral checks passed, but the case failed because all three turns used different runner processes. Session-ID reuse alone is not warm process continuity. |
|
||||
| Copilot Daytona hello, controller40064 | Six checks passed on profile7 and the current immutable Linux image. One owned sandbox was removed, the full late-create observation passed, and all owned local processes retired. This qualifies only this case. |
|
||||
| Copilot Daytona denial, controller40064, first attempt | Failed before Product dispatch because the private launcher omitted the required image Node and runnerd digest environment fields. Account/analytics reads occurred, but the launch path never reached a provider or sandbox creation. The original failure remains failed; a corrected helper needs a new attempt. |
|
||||
|
||||
All passing cases above completed their owned process/semaphore cleanup, temporary-root cleanup and full end integrity audit. Cursor plan result SHA-256: `685609f01291db84f40d890ef562473b95065e86907bd964b656ad732f6dd97a`; reconciliation: `09a17d5aabc9d33bd7cfe083ad4875ed4da2c3d7b72438e00584237fc71190bc`. Copilot Daytona hello result: `960c9de5a78f4073918daed8f6f6ddfe6f90d1164bbb4cf8f2b4e3d20d5234a5`; reconciliation: `fd043e08b853e0a49901070593940a1aca68f8bd488c194578e6a3f65963d69e`. Copilot warm failure result: `1583d442c5bf70ea520fd63b3a6dcc03ba045a8af2af5e2f68dcf220bb4e8815`. Local async and denial result digests remain `d1d074bc9df4644794dfa8b020fdeba8a9b0de4618e1f55d1845f7726a2ece33` and `c1fd910ecc4bf9d6df7645f3eb90b45d9cc895cfea9e2f93b4cfdf8a2caf8625`.
|
||||
|
||||
The warm-directory repair retains the exact registered `AGENT_HOME` with the native owner, claims it before composing the next turn, and transfers collection authority with database compare-and-swap guards. Changes or a final configuration mismatch require verified retirement before collection. Missing stop proof preserves unresolved ownership. Stop-proved remote recovery records an explicit unavailable/no-save outcome when bytes cannot be recovered. Independent review found and fixed that recovery transition. Validation passed 527 native/probe tests, 63 working-copy database tests, four final composed lifecycle cases, five follow-up recovery cases, and direct server typecheck. The paid warm failure remains failed until a new run proves this change. See [agent-file lifecycle rules](../agent-files.md).
|
||||
|
||||
Native Cursor/Copilot per-run USD remains unknown. Cursor's fixed account-wide on-demand cap is $25, with $0 observed after the corrected plan case. Copilot additional usage remains disabled at $0; the included-credit counter reached 20 after Daytona hello and stayed 20 after the failed denial launch. Counters may lag. The hello sandbox's conservative cost bound is $0.086931; early analytics are provisional. The first 53 reservations are closed, retaining failed attempts. Pi v9 still needs a verifiable OpenRouter spending cap; its optional transport-budget candidate remains frozen and unintegrated. The $100 combined campaign ceiling remains in force.
|
||||
|
||||
The [prior evidence checkpoint](runner-rich-acp-validation-2026-09-30-current.json) retains controller/harness `ca7026182c2b861e3badbcb7e5b733445a6ee403` and runtime `e822b614fc368043f4b3d5e34a8d9bbda644e055`; despite its filename, it is historical for the current source. The [earlier September 30 checkpoint](runner-rich-acp-validation-2026-09-30.json) is also historical. Original failures, unknown costs and exact source/profile identities remain unchanged.
|
||||
|
||||
| Current production gate | Required evidence or decision |
|
||||
| --- | --- |
|
||||
| Cursor v7 | Resolve the failed accepted-plan settlement and qualify the exact corrected bytes. Native AskQuestion availability remains unverified; semantic questions cannot substitute for its callback. Complete the required local/Daytona native input, permission, recovery, isolation and warm-continuity cases. |
|
||||
| Copilot v7 | The exact local attached-async and native-denial cases passed; broader provider qualification remains pending. Complete the remaining local/Daytona denial, recovery, isolation and warm-continuity cases. Explicit detached work remains unsupported. Native external-tool/sampling/limits callbacks require proof that admitted tools cannot leave an unresolved request, or a qualified responder. |
|
||||
| Cursor v7 | The exact corrected local native-plan case passed; complete the remaining required cases. Native AskQuestion availability remains unverified; semantic questions cannot substitute for its callback. Complete the required local/Daytona native input, permission, recovery, isolation and warm-continuity cases. |
|
||||
| Copilot v7 | Local attached-async and native-denial plus Daytona hello passed. Repair and requalify failed warm process continuity; broader provider qualification remains pending. Complete the remaining local/Daytona denial, recovery, isolation and warm-continuity cases. Explicit detached work remains unsupported. Native external-tool/sampling/limits callbacks require proof that admitted tools cannot leave an unresolved request, or a qualified responder. |
|
||||
| Pi v9 | Obtain an enforced provider spending bound, then run the unchanged required local/Daytona cases against the exact model and final runtime. Historical controls, question and hello passes do not qualify v9. |
|
||||
| All providers | Bind every required case to exact controller/runtime/profile and platform evidence, retain failures, complete source/dependency/pack end audits and owned cleanup, and reconcile actual spend within the $100 ceiling. CI and packaging do not substitute for this paid evidence; native Cursor/Copilot per-run USD remains unknown. |
|
||||
|
||||
@@ -32,11 +46,11 @@ Exact source415 CI completed with 53 successful checks/statuses and two skips; G
|
||||
|
||||
At parent source `6b1e96af0`, CI reported 49 successes, two skips and four failures, including the E2E aggregate. Commit `f401b831f` moves cold route imports into fixture setup and dismisses the actual announcement UI in the attachment browser test. All 26 route tests passed. The attachment test also passed with a forced real announcement and a successful dismissal response. Parent source `aec34c693` then completed CI with 53 successes and two skips. The separate preview-runtime CI stall passed unchanged under local instrumentation; its cause is unconfirmed, and no speculative runtime fix is included.
|
||||
|
||||
Cursor v7 source `770dc88a1` preserves the native plan's parent tool identity through TypeScript, the sidecar and Rust. The controller requires one successful lifecycle for that exact tool in the same session and turn, and includes its events in the committed proof. It rejects unrelated work and altered proof. Historical Cursor v6 committed waits remain valid. All 228 focused tests, the Runner TypeScript build and server typecheck pass. Native distribution bytes are unchanged; current runtime builds are complete, while the v7 paid attempt above remains failed.
|
||||
Cursor v7 source `770dc88a1` preserves the native plan's parent tool identity through TypeScript, the sidecar and Rust. The controller requires one successful lifecycle for that exact tool in the same session and turn, and includes its events in the committed proof. It rejects unrelated work and altered proof. Historical Cursor v6 committed waits remain valid. All 228 focused tests, the Runner TypeScript build and server typecheck pass. Native distribution bytes are unchanged; current runtime builds are complete, and the corrected controller3d21 paid native-plan case above passed.
|
||||
|
||||
Copilot v7 source `70b186b3c` preserves real native message identity through the pinned mapper and separates intermediate messages from the final answer. The native executable is unchanged; an owned, verified inner distribution carries the patch. Review fixes add owned-output rollback after failed materialization and actual ACPX history replay coverage before fresh warm turns. All 49 focused JavaScript/TypeScript checks, Runner typecheck and the exact Rust admission check pass. The earlier source passed 212 focused tests and verified complete closures for all three platforms; only ARM64 executed its native loopback proof. That asset evidence remains separate from the current source checks. Current runtime builds and exact-head CI are recorded above; fresh remote image/admission evidence and the remaining paid qualification are separate gates. The earlier v6 async failure stays failed.
|
||||
|
||||
Cursor native AskQuestion remains unverified for the inspected exact models/modes. Pi v9 needs a verifiable OpenRouter spending bound before further paid runs. Its optional transport-budget candidate remains frozen: production allocation, launch scope, warm rotation, packaging and Daytona transport are incomplete. The provider-capped key remains the immediate path. Native Cursor/Copilot per-run USD remains unknown. Included usage and a disabled overage budget are billing coverage, not a native price receipt. The first 49 attempt records are closed, including the failed Cursor plan and passed Copilot async and denial cases. Later attempts remain separate evidence. The $100 campaign authorization remains in force, including failed attempts and infrastructure.
|
||||
Cursor native AskQuestion remains unverified for the inspected exact models/modes. Pi v9 needs a verifiable OpenRouter spending bound before further paid runs. Its optional transport-budget candidate remains frozen: production allocation, launch scope, warm rotation, packaging and Daytona transport are incomplete. The provider-capped key remains the immediate path. Native Cursor/Copilot per-run USD remains unknown. Included usage and a disabled overage budget are billing coverage, not a native price receipt. The first 53 attempt records are closed. The table above separates corrected Cursor plan and Copilot hello passes from retained failed plan, warm and launcher attempts. The $100 campaign authorization remains in force, including failed attempts and infrastructure.
|
||||
|
||||
Current profile identities bind shared runtime contract `paperclip.acpx-runtime-contract.v1`. All ACPX warm owners include that revision in their configuration fingerprint; incompatible owners retire without approval replay. Other transports retain their previous configuration digest. Legacy Cursor and Pi adapters remain unchanged.
|
||||
|
||||
|
||||
Reference in new issue
Block a user