mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-20 15:09:57 +02:00
Record current Pi paid proof and remaining qualification gates
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
3efe9f52d8
commit
fdcba3c877
1 file changed
+16
-9
@@ -31,8 +31,9 @@ The basic eval definitions and current profile pin follow-up are merged in
|
||||
`paperclip-evals` PRs [#29](https://github.com/paperclipai/paperclip-evals/pull/29)
|
||||
and [#30](https://github.com/paperclipai/paperclip-evals/pull/30). The latter passed
|
||||
134 unit tests, CI and Greptile 5/5 at `4a15550d16ced37c6edbafa91a11783a64e4e649`;
|
||||
its merge commit is `0ad9c5a4275fbfbed3b32f261d3e603d9761a07c`. A further pin
|
||||
update is required for Cursor/Copilot v4. These definitions
|
||||
its merge commit is `0ad9c5a4275fbfbed3b32f261d3e603d9761a07c`. The Cursor/Copilot v4 pin
|
||||
update is in [PR #31](https://github.com/paperclipai/paperclip-evals/pull/31), with
|
||||
134 deterministic tests and all three explicit lanes passing. These definitions
|
||||
are not paid qualification results. Versioned config IDs separate current
|
||||
candidate scorecards from historical runtime policies.
|
||||
|
||||
@@ -40,8 +41,8 @@ candidate scorecards from historical runtime policies.
|
||||
| --- | --- | --- |
|
||||
| Cursor v4, source `62e642590` | All three closures materialize and verify. Actual patched ACPX tests enforce instruction acknowledgement on new/load and automatic reload. Exact native source methods preserve the composed text through request context. Independent review has no remaining findings | Final combined pack and daemon, authenticated current-profile Product tests, remaining platform and Daytona proof; paid tests await the account spend-limit decision |
|
||||
| Copilot v4, source `95dfde7c5` | Fresh/load model-request capture proves native personal instructions reach the pinned executable. Refresh holds lifetime ownership through cancellation; rejected contenders cannot overwrite it. Both guards and current terminal diagnostics pass merged tests and independent review | Final pack and daemon, then repeat the v3 question case that failed restoration and complete current-profile qualification |
|
||||
| Pi v6, source `f09813bae` | Bounded snapshot copying and hashing repair the reproducible startup delay. A complete credential-free PRP startup finishes in 10.7 seconds on the observed host, with the unchanged 30-second deadline. Actual SDK fresh/load model requests receive current instructions | Final combined runtime build and fresh paid hello, native-question, agent-files, permission, planning and remote qualification. Startup timing depends on host load/cache |
|
||||
| All three, Linux/Daytona | The stored Daytona key and analytics endpoint work. Docker recovered without a restart; no task sandboxes have been created | Rebuild the immutable Linux image from final integrated source, then retain sandbox-specific usage and cleanup evidence |
|
||||
| Pi v6, combined source `97217c531` | Clean combined pack and daemon verified; credential-free PRP startup settles in 13.5 seconds. Authenticated Product hello passes all six matchers with one provider run and complete process cleanup | Native-question, agent-files, permission, planning, restart and remote qualification. The hello launcher configured one automatic retry despite a zero-retry declaration; none occurred. Corrected launchers explicitly enforce zero retries |
|
||||
| All three, Linux/Daytona | The stored Daytona key and analytics endpoint work. The first clean Linux install exhausted Docker memory. Build-only pnpm concurrency is now bounded; no task sandboxes have been created | Complete and verify the immutable Linux image from final integrated source, then retain sandbox-specific usage and cleanup evidence |
|
||||
|
||||
Cursor and Copilot ignore generic ACP `_meta.systemPrompt`; setting `AGENT_HOME`
|
||||
in the process environment does not make it part of model instructions. The v4
|
||||
@@ -62,11 +63,17 @@ a passing full run. Final-source full checks and current-head CI remain required
|
||||
|
||||
The Copilot attempt increased the visible included-credit counter from 5 to 6;
|
||||
additional usage remained disabled at a $0 budget. Per-run USD is unknown.
|
||||
Both current Pi failures have zero exclusive-key billing delta in delayed
|
||||
observations and no inference receipt. Failed attempts and owned-process cleanup
|
||||
remain recorded; there were no automatic paid retries. The $100 shared budget
|
||||
and provider allocations remain in force. These current failures supersede any
|
||||
historical local success as evidence for production readiness.
|
||||
The first two Pi attempts failed before inference and have zero exclusive-key
|
||||
billing delta in delayed observations. Pi hello attempt 03 passed on the combined
|
||||
`97217c531` runtime: 15,340 input, 332 output and 15,360 cached tokens, with a
|
||||
delayed exclusive-key delta of $0.00065884. The native USD receipt remains
|
||||
unpriced. All 48 observed owned process identities were retired. Its retained
|
||||
invocation policy allowed one automatic retry even though the supervisor declared
|
||||
zero; exactly one Product attempt ran. This configuration gap is retained in
|
||||
evidence, and subsequent supervisors explicitly pass `--max-automatic-retries 0`.
|
||||
Failed attempts, missing cost coverage and cleanup remain recorded. The $100
|
||||
shared budget and provider allocations remain in force; one successful hello
|
||||
does not establish production qualification.
|
||||
|
||||
## Registered persistent agent files
|
||||
|
||||
|
||||
Reference in new issue
Block a user