docs(hermes): record paid acceptance and qualification limits

Co-Authored-By: Paperclip <noreply@paperclip.ing>

doc/plans/2026-10-06-hermes-native-runner.md

@@ -1,7 +1,8 @@
 # Hermes native runner implementation
 
 Status (updated 2026-10-07): implementation candidate; **not qualified**.
-Branch: `codex/hermes-native-runner`.
+Current branch: `codex/hermes-qualification`; stacked on
+`codex/hermes-native-runner` and `codex/hermes-routines`.
 
 ## Accepted outcome
 
@@ -21,7 +22,8 @@ Keep the existing Hermes local/gateway adapters compatible.
 - [x] Focused TS/Python tests and production-path macOS execution with a deterministic model server.
 - [x] Complete final Rust/regression checks and resolve or classify failures.
 - [ ] Browser acceptance, Linux/Daytona execution and connection-method qualification.
-- [ ] Required repository checks and reviewable PR.
+- [x] Reviewable draft PR stack, green CI and fresh Greptile 5/5 on the implementation heads.
+- [ ] Complete the final local aggregate test invocation and release qualification.
 
 ## Qualification evidence
 
@@ -58,7 +60,8 @@ because fixtures pass.
   Native cron and gateway messaging are disabled.
 - Pending Hermes choice in existing runner configuration; existing connection,
   model, permission and transcript components. Ten Product E2E candidate cells
-  (five local, five Daytona) are registered but have not been run live.
+  (five local, five Daytona) are registered. Four local cases have passing paid
+  attempts; the complete release matrix has not passed.
 
 ## Evidence and outstanding release gates
 
@@ -74,7 +77,7 @@ Hermes uses the new shared ACP profile manifest. The shared extension fix also
 updates Cursor's ACPX patch attestation and profile identity to revision 15;
 the Cursor usage, delegation and model-selection package contracts pass.
 
-Post-rebase checks:
+Historical post-rebase checks (superseded where newer results appear below):
 
 | Check | Result |
 | --- | --- |
@@ -106,7 +109,7 @@ installed package (not workspace imports). The candidate provider-pack
 materializer also verifies the copied runtime. Both native execution targets
 are still pending real model and product qualification.
 
-Reproduced runtime closure SHA-256:
+Historical pre-review runtime closure SHA-256:
 
 | Target | Closure digest |
 | --- | --- |
@@ -125,22 +128,25 @@ Release blockers remain explicit:
    including refresh/revocation/concurrent ownership, permission prompts,
    questions across reconnect, steering/queue/stop, remote recovery,
    cross-task learned skills, routine firing and cost attribution.
-4. Obtain a green aggregate CI run and review the implementation before
-   promoting the candidate. Default production selection remains disabled.
+4. Complete the full live qualification before promoting the candidate.
+   Implementation CI and review are green; they do not replace live proof.
+   Default production selection remains disabled.
 
 ## Review handoff
 
-The routine service binding is a separate eight-file commit on
+The routine service binding is a separate review on
 `codex/hermes-routines`. The native integration is stacked on it on
 `codex/hermes-native-runner`, within the 100-file review limit. Changes are
 committed. The generated root lockfile is excluded as required by the
-repository; the Daytona Dockerfile pins the twice-reproduced resolved lock.
+repository. The Daytona Dockerfile requires the caller to provide the SHA-256
+of its resolved lock and verifies the copied lock before installing dependencies.
 
 The user subsequently authorized release qualification and PR verification.
 The routine PR is [#15434](https://github.com/paperclipai/paperclip/pull/15434).
 The native integration is stacked in
 [#15435](https://github.com/paperclipai/paperclip/pull/15435). Both are drafts;
-review and CI are running. Qualification fixes use `codex/hermes-qualification`
+their current implementation heads have green CI and fresh Greptile 5/5.
+Qualification fixes use `codex/hermes-qualification`
 to retain the under-100-file limit for each review.
 
 ## Paid qualification, 2026-10-07
@@ -211,15 +217,128 @@ paths and re-signs the changed macOS library with a deterministic ad-hoc
 signature. Independently provisioned interpreter paths produce identical
 closures; this is distribution proof, not live Linux sandbox qualification.
 
-| Final target | Closure digest |
+| Historical review target | Closure digest |
 | --- | --- |
 | macOS arm64 | `4c89b24335e1869a9faba6996a4e82979337ee5f850d792ab41a838e79afdce3` |
 | Linux amd64 | `f9919bd2e812e86e81ecd964d6e1961bb68f96e2154f816c554f31c4f1d78211` |
 
 The qualification stack is
 [#15436](https://github.com/paperclipai/paperclip/pull/15436). Runtime
-qualification fixtures and this implementation/evidence record belong to that
-PR to keep the native integration review below 100 files. The new shared ACPX
+qualification workflows, follow-up fixes and this implementation/evidence record
+belong to that PR. The native runtime fixtures belong to #15435. Each review
+remains below 100 files. The new shared ACPX
 patch has an explicit Cursor profile revision 16, preserving historical
 revision decoding. The Docker lock digest is twice reproduced; the root lock
 file remains owned by the repository's lock bot.
+
+### Current qualification record, 2026-10-07
+
+Hermes remains **pending qualification**. All three draft implementation PRs
+have passing CI and fresh Greptile 5/5 at these
+heads: routines `3745f3c5a46bda7778ee132682d1b7ae23b088c1`, native
+`6a7a006b0738558a4abb1c030f2b7b11f5afea2c`, and qualification
+`09195bb8c8bc564eaa3f5061a7d6b5d685e708a4`. A later native review also found a
+standalone image command without the required resolved-lock digest. Both image
+guides were corrected in `0dea682031f8e35631faee7a05519ed4dce80d07`, their shell
+syntax and checksum ordering were checked, and the addressed thread was
+resolved. The qualification commits were rebased onto that fix. Subsequent
+documentation heads require fresh checks and review before handoff.
+
+Paid planning exposed two integration defects. Assigned plan-document and
+task-title tools were rejected by the native read-only guard before the
+controller could apply its task-mode authority. Structured MCP results also
+appeared as `null` in the transcript. Assigned workflow tools now reach the
+existing controller authorization and configured permission check; native
+commands and file writes remain denied in planning mode. Tool results retain
+their actual output. A pre-dispatch denial synthesizes a failed call using
+Hermes's authoritative call ID exactly once, including overlapping calls.
+
+Managed Hermes restoration now loads the validated native history without
+emitting it again as new ACP transcript output. Paperclip owns the persisted
+transcript. Standalone, non-negotiated ACP clients retain native history replay.
+Missing or unreadable native history still fails restoration.
+
+Current runtime closure SHA-256 (fresh provisioning reproduced both):
+
+| Target | Closure digest |
+| --- | --- |
+| macOS arm64 | `970f0c48b905d17e616a3b75ef28d91a0e28afba8dbb3218628cd02b3b1e709c` |
+| Linux amd64 | `15fc9631318d50a2aafa9c566410b4d486265fb3e58a7981fd0c2525a5fb8f7a` |
+
+Current focused verification: 46 TypeScript permission/sandbox/configuration/
+installation tests, 20 pinned Python bridge tests, and both native production
+fixtures pass. The ACPX fixture additionally checks planning round trips,
+visible write denials, strict cancellation recovery and missing-history
+rejection. These fixtures use a deterministic model endpoint. The full Rust
+workspace ran 652 passing test executions with two ignored. Repository
+typecheck, build, protocol generation checks and UI token gates pass.
+
+The local aggregate test invocation finished with 16,084 passing tests, one
+failed test, 216 skipped tests and two failed suite setups. Both suite setups
+failed during embedded PostgreSQL bootstrap before their assertions ran. The
+real 40,000-file Git streaming test reached its existing five-minute deadline.
+These failures remain failures of that invocation; focused reruns and green
+sharded CI do not rewrite it as a pass.
+The isolated rerun passed both database suites (eight tests). The Git fixture
+still reached its five-minute deadline. Its test and shared Git implementation
+are unchanged from the source baseline; Linux PR CI passes that coverage. The
+local Git timeout remains an explicit verification limitation.
+
+Paid browser attempts used the managed OpenRouter account and exact model
+`deepseek/deepseek-v4-flash-0731`, on macOS arm64, through Chromium, the isolated
+Paperclip server/database, Rust Runnerd, ACPX and native Hermes:
+
+| Case | Passing campaign | Source provenance | Duration | Cleanup |
+| --- | --- | --- | --- | --- |
+| Hello/completion | `hermes-local-paid-20261007-first` | Report source null; observed checkout `2796b80a9` with image inputs in progress | 47.63 s | Pass |
+| Question/resume | `hermes-local-paid-20261007-question-replay-fix-retry2` | Recorded `74b692bbd8c98bbeda4c39cf8327680245ac2cbf` | 106.90 s | Pass |
+| File edit/validation | `hermes-local-paid-20261007-remaining-continuity` | Recorded `0ba0d75511cf9fdf1fa4b21d9e900503078f3620` | 122.28 s | Pass |
+| Plan/approve/complete | `hermes-local-paid-20261007-plan-policy-fix` | Recorded `09195bb8c8bc564eaa3f5061a7d6b5d685e708a4` | 134.19 s | Pass |
+| Structured question/controller restart/resume | `hermes-local-paid-20261007-restart-resource-retry` | Report source null: invocation used the wrong source-variable names; observed checkout `09195bb8c8bc564eaa3f5061a7d6b5d685e708a4` | 118.72 s | Pass |
+
+All five registered local cases have passing attempts across multiple heads.
+This is not a complete final-head campaign or the full requested release
+matrix. The restart case proves persistence of the pending question across a
+controller restart, submission of its answer, native session reuse, and task
+completion. Its resumed run reported 64,166 input and 281 output tokens. The
+planning completion run reported 62,268 input, 478 output and 37,888 cached
+input tokens. Their interrupted first runs have incomplete usage receipts.
+Cost remains unpriced; missing cost is not a zero-cost execution.
+
+The preceding restart attempt failed during test-database bootstrap, before
+any model request. Clearing only four confirmed user-owned, unattached,
+56-byte shared-memory segments with dead creators allowed the unchanged test
+to run. Earlier session-open timeout and transcript failures remain retained
+in their own campaign results. No deadline or oracle was weakened.
+
+Private numeric-only budget receipts show the qualification key's $5 hard
+limit still has $4.559226202 remaining after these attempts. The shared-key
+usage change is an aggregate ceiling, not exact attribution to individual
+runs. Sanitized results remain under `tests/runner-e2e/results/`; private native
+history, credentials and hidden reasoning are not published as artifacts.
+
+The clean installed-package root/evals/testing conformance check passed on
+`09195bb8c8bc564eaa3f5061a7d6b5d685e708a4`. It uses offline packed runtime
+dependencies, including the reviewed ACPX patch, rather than unmodified
+registry dependencies. The installed package's shipped provisioner reproduced
+the current macOS closure. Both native fixtures passed from installed compiled
+code, using the published release Runnerd artifact: 29.03 seconds for the Rust
+path and 146.75 seconds for the ACPX path. Only fixture import locations and the
+explicit Runnerd artifact path were adapted; assertions and deadlines stayed
+unchanged. This is clean-package transport proof with a simulated model.
+
+The protected paid workflow now provisions Python, bubblewrap, and the verified
+Hermes closure only for an explicit Hermes selection, before credentials enter
+the paid step. It must land on `master` before its trusted dispatch can run
+paid Linux/Daytona campaigns. No paid remote campaign has been dispatched.
+An available development host permits bubblewrap but is Linux arm64; it does
+not satisfy the requested Linux amd64 target. The emulated local Linux amd64
+container provides provisioning proof and still fails the namespace gate.
+
+Outstanding release proof includes the other API providers, subscriptions,
+custom protocols, Bedrock, real vision input, credential refresh/revocation and
+concurrent ownership, live steering/queue/stop, lower permission modes,
+cross-task memory/learned skills, routine firing and deduplication, and actual
+Linux amd64/Daytona execution and restoration. Exact approved Connection names
+and a compatible remote execution environment are still needed. The existing
+standalone Hermes local/gateway adapters retain their contracts.