Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-23 09:07:46 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge current master and resolved fixture synchronization

Browse Source 
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin FoleyandPaperclip committed 2026-10-03 23:55:14 -07:00
2 parents 420af2835f 99c84ccca2
commit f0b683a24e
203 files changed
+13774 -1783
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

doc/SPEC-implementation.md
+7 -1
View File
@@ -1702,12 +1702,18 @@ Confirmed project creation appears as a durable card in the shared task transcri
### User continuation after execution recovery stops
An authenticated user message or an exact failed-run Retry can start a fresh
An authenticated user message, a validated undelivered native message queue, or an exact failed-run Retry can start a fresh
native or legacy conversation turn once the prior execution is confirmed stopped. Retain the source history and uncertain
action outcomes; do not replay tool calls or reset the failed incident's automatic
retry budget. Existing pause, approval, budget, ownership, and dependency gates
remain in effect. See `doc/execution-semantics.md` for admission and stop-proof
requirements.
The task recovery notice offers Retry for eligible failures and verified native
startup cancellations, with failed attempts explained inline. Preparing native
turns keep the Steer label. Steer and Interrupt immediately move the submitted
messages from the composer queue into the conversation while delivery proceeds.
Provider acknowledgement remains authoritative; failed delivery restores the
latest queue with an inline error. Neither action produces a toast.
### Managed AI authentication
doc/evals.md
+13 -1
View File
@@ -81,7 +81,9 @@ The explicit-only [production hiring templates suite](../tests/runner-e2e/README
adds two local native Codex/Claude cells. It exercises API-created production
CEO defaults, an explicitly requested hiring skill/reference read, a permanent
coder hire, independently computed saved JSON fixtures and worker reuse.
Each cell expects five turns. Source/read coverage and workflow outcome are
Each cell requires five work turns and admits at most two strictly attributed
server task-completion turns. Every actual run remains counted; unknown or
extra-work turns fail. Source/read coverage and workflow outcome are
separate: missing read provenance leaves the candidate/baseline pair
uncomparable even if work succeeds. Baseline bundles and coder examples derive
from their own source revision, without requiring candidate wording or length.
@@ -93,6 +95,14 @@ execution ID because `--all` excludes explicit-only suites. Each cell applies a
1,000-cent company and agent budget hard stop before task creation and records
both limits in its evidence.
The explicit-only [stock-harness suite](../tests/runner-e2e/STOCK-HARNESS.md)
reuses skill, ordered-continuation, and chat-restart journeys across eight local
legacy/native profiles with production-default hires. It closes the custom QA
manual coverage gap. Its required credential-free prerequisite maps vendor
instruction layering, the tiny hire bundle, and shared startup/resume reductions
to executable checks. The 24 live cells are configured; no live qualification is
claimed from their setup or unit calibration.
The explicit-only `agent-chat-stories` suite covers the experimental settings
lifecycle for a configured native agent and follow-ups during active work. Its
fixture-driven file wait and persisted-plan oracle are documented in the
@@ -386,3 +396,5 @@ The explicit-only Product E2E `confirmation-replies` suite tests conversational
approval and rejection, persisted message provenance, approval before execution,
ambiguous proposals, and the existing card-click path with native Claude/Codex.
See the [suite contract](../tests/runner-e2e/README.md#conversational-confirmation-replies-explicit-only).
Hiring notification accounting now also requires exact completed action attribution. Missing native/provider ID mapping is uncomparable evidence; it must not be reported as a model task regression or waived through name/order matching. The fixture waits for both known completion callbacks and settled bracketed observations, including the gap before pending outbox work becomes a wake. Strict action replay and original machine verdicts are retained separately.
doc/execution-semantics.md
+32 -5
View File
@@ -486,7 +486,9 @@ A terminal native failure can retain a result accepted before checkpoint or clea
A new user message can continue a terminal native run whose process fields were cleared before local stop receipts existed. Admission must verify the exact run, runner, workspace, and provider session in the retained suspended state, with no active provider turn, pending tool call, or undelivered output. Missing or mismatched state keeps the hold. A later recorded process launch also keeps the hold until its stop is verified. Normal assignment, decision, controller, environment cleanup, and active-run gates still apply. The message starts one fresh conversation turn; it does not replay the failed run, reset its recovery budget, or certify unknown action outcomes.
The task thread exposes the existing guarded Retry action for failed or timed-out legacy conversation runs. Where the server supports an explicit new attempt after a stopped legacy conversation, the thread must not hide that action solely because the old run still has a recovery-needed projection. Native and process recovery holds, pending decisions, active execution, and other retry gates remain in force. When a gate hides Retry, the thread says the message is preserved instead of promising an unavailable action. This presentation change does not rewrite historical outcomes or certify prior actions.
The task thread exposes the guarded Retry action for failed or timed-out conversation runs and native preparation cancelled before provider startup when the never-started proof is verified. The server projects this eligibility on the recovery notice and rechecks it on Retry. Cleanup quarantine retains its inspection path, and non-conversation reconciliation gates remain enforced. Pending decisions, active execution, pause, budget, dependency, and ownership gates remain in force. A refused Retry reports its reason inline and remains available for another attempt.
A continuation cancelled by the queued-run gate while waiting for review did not start a provider. Its matching dispatch-gate receipt and unclaimed execution fields keep this deliberate wait out of unknown-action reconciliation. Pending review and decision gates still apply. If an older server already created a recovery hold for this exact state, Retry or a new user message can request one fresh turn after the server verifies there is no provider, coordinator, or unfinished cleanup. Saved user input is reconsidered through the same admission path; the cancelled turn is not replayed.
A valid recovery action must name:
@@ -666,6 +668,8 @@ The same bounded rule applies when the previous heartbeat reported waiting on a
A continuation that the staleness gate cancelled with `issue_continuation_waiting_on_review` is a *deliberate park*, not a disappeared execution path. The latest run reported that the issue is waiting for review/approval (for example, an umbrella issue whose work was just decomposed into sub-tasks). Treating that park as a stranded run would retry it, then escalate it to `blocked` with a recovery action and an operator-facing failure notice — even though nothing failed and there is nothing for a human to do.
Execution admission reads a narrow server-owned cancellation-evidence projection. Ordinary run presentation can redact `resultJson` for database encoding or output size; that presentation projection must not decide Retry eligibility or saved-input recovery. The admission projection excludes provider diagnostics and preserves whether the stored result is absent.
Recovery rule for a parked-for-review continuation:
- if the issue has a real waiting target — open (non-terminal) sub-tasks or existing unresolved blockers — Paperclip converts the deliberate wait into a first-class dependency wait: it sets the issue `blocked` by those issues, keeps the original assignee, and posts a plain-language comment explaining that the task will resume automatically when its dependencies finish. The issue then self-resumes through the normal `issue_blockers_resolved` path; no recovery action or escalation owner is involved
@@ -1003,7 +1007,7 @@ For native conversations, an authenticated user message sent after the previous
Local recovery records a server-authored stop receipt before it clears a verified absent process identity. A new execution request invalidates that receipt before any process can spawn; recording a new process identity also invalidates it. Missing process IDs without a receipt still block admission. Remote execution continues to require termination receipts for every lease.
If cleanup or another execution gate is still pending, the message stays in its existing queue receipt. Startup and periodic scheduling reconsider up to 50 due receipts per pass, at most once per 30 seconds per receipt, without calling a model or resetting recovery attempts. Cleanup callbacks use the same admission path. The issue lock prevents concurrent workers from delivering an adopted or discarded receipt again. The queued-message area shows the current wait reason. Pauses, approvals, budgets, ownership, and external chat authorization remain enforced. A message sent before the run finished does not grant new post-stop authority.
If cleanup or another execution gate is still pending, the message stays in its existing queue receipt. Startup and periodic scheduling reconsider up to 50 due receipts per pass, at most once per 30 seconds per receipt, without calling a model or resetting recovery attempts. Cleanup callbacks use the same admission path. The issue lock prevents concurrent workers from delivering an adopted or discarded receipt again. The queued-message area shows the current wait reason. Pauses, approvals, budgets, ownership, and external chat authorization remain enforced. For a failed native conversation, saved authenticated messages that have never been delivered can request a fresh turn after verified stop even when submitted before the failure finished. Admission validates and adopts their exact queue receipt, order, and original authors. This does not replay the failed request or undo an operator cancellation.
Historical legacy interruption holds for conversational adapters no longer block new messages or Resume. Automatic classification uses the server-owned adapter identity saved atomically at run claim, the saved adapter invocation, or the continuation policy, never the agent’s current adapter settings. Missing historical adapter evidence retains the automatic hold; an explicit user continuation can retire it after proving the predecessor stopped. A terminal row with a live predecessor process, an unreleased environment lease, or failed/pending cleanup still blocks actual admission and Resume; a release timestamp alone does not prove cleanup succeeded. Retry scheduling can happen before cleanup, but grants no execution authority. Recovery folds their obsolete no-replay bookkeeping without changing task ownership, status, or automatically waking old work. The audit trail remains readable. Native integrity and ownership holds, and non-conversational adapter holds, remain enforced.
@@ -1045,7 +1049,7 @@ unknown, and they do not gain automatic replay eligibility.
Admission validates the persisted comment's author, task, and time against every
held predecessor. Retry validates the selected failed run's company, task, and
agent and preserves that run's identity through admission and history loading.
Duplicate Retry requests adopt the same successor. An agent-authored comment, an old queued request, or a generic
Duplicate Retry requests adopt the same successor. Only a validated, unconsumed human queue receipt can authorize saved-message continuation after a native failure; arbitrary historical input, an agent-authored comment, or a generic
system wake cannot release a hold. The source task keeps its assignee. Process
ownership, active controllers, cleanup leases, pause, approval, budget, and normal
execution gates still apply. Dependency-blocked interaction mode remains limited
@@ -1156,6 +1160,15 @@ controller, lease, or result). It also checks for contradictory launch/process
evidence and verifies local cleanup or exact remote termination receipts. The
preparer must have finished or its startup lease must have expired. A missing
PID alone does not establish this proof.
For older interrupted preparation rows without a cancellation receipt, or with
a retained startup fence but no preparation-settled marker, the
immutable Paperclip Runner adapter claim, unresolved runtime, and preparing stage
must agree. The old controller must belong to another server boot and its lease
must have expired. Only absent results or the retained before-selection
cancellation fence qualify. No native identity, coordinator, adapter invocation,
provider event, or process-launch evidence may exist. Environment cleanup still
requires the same receipts. This historical proof permits explicit Retry or a
newer saved user message; it does not replay the cancelled input.
The existing bounded saved-message worker rechecks this proof after restart.
Admission atomically settles an unclaimed coordinator and admits one fresh turn,
@@ -1322,8 +1335,22 @@ execution recovery holds still apply. Unconfirmed cleanup does not start work.
The active session advertises steering only when its driver supports it. A
transport method that rejects steering does not grant that capability. The
queued-message control remains mounted until the server accepts a steer request,
so a rejected last-row action keeps its message and visible error.
queued-message controller remains mounted across an empty queue, preserving
pending delivery and inline errors. Steer immediately moves the selected message
into the conversation; Interrupt immediately moves all submitted queued messages.
Stale queue polls do not restore those rows while delivery is pending or accepted.
Rejected delivery restores the latest queue and shows an inline error without a
toast. The provider acknowledgement remains the authoritative delivery receipt;
optimistic display never grants permission or records successful delivery. A preparing
Paperclip Runner retains the Steer label while steering is unavailable; the
initial legacy database default and an empty queue snapshot cannot select
Interrupt. Model and effort changes apply to the next turn and do not change
the active turn's steering target. While awaiting steering acknowledgement, the
queue route holds the task and queue locks but leaves the run row available for
PRP event persistence. It locks and rereads the run after acknowledgement before
merging the receipt, preserving concurrent provider updates.
If the run stops during that wait, the request fails with a stale-target error
and retains the queued input for continuation.
### Preserve work across handoff and deliver requested files
doc/observability.md
+13
View File
@@ -453,6 +453,19 @@ termination, remove the live execution control, or change the timeout. The
context is sent only through the existing opt-in Sentry gate and never leaks
into unrelated captures.
The context also samples the pending execution `phase` and `phaseElapsedMs`
when the Stop timer expires. An in-memory tracker belongs to the exact live
execution control; a replaced or missing owner reports `unknown` with a null
elapsed time. Labels come from a closed list and elapsed time uses a monotonic
clock, capped at one day. Nested scopes remain visible while their awaits are
pending, including ACP session close, transport stop, instruction collection,
workspace restore, and host instruction or lease cleanup. `phase_reporting`
means a step is waiting to write timing or teardown error diagnostics. `adapter_execution` and
`host_execution` are coarse labels for work outside those narrower scopes.
The tracker clears when its executor finishes. A phase is diagnostic context,
not evidence that a provider stopped or that files were recovered. No new
run-log or Telemetry event is emitted by this tracker.
The shared reporter also attaches bounded diagnostic contexts for both legacy
and native runs:
doc/plans/2026-10-02-hiring-executable-accounting-replay.json
+740
View File
@@ -0,0 +1,740 @@
{
"schema": "paperclip.hiring-executable-accounting-replay.v1",
"assessmentType": "Provider-free replay through both new executable guards; original model measurements and verdicts unchanged",
"evaluatedCodeRevision": "eef64009dc144b91b245a731ea9a1c3c07406a19",
"evaluatedCodeHashes": {
"tests/runner-e2e/hiring-template-cases.ts": "06915e33cc9a5241cd28a7184fa33d098c30aed4114accea4f93838aaba3738f",
"tests/runner-e2e/hiring-template-scoring.ts": "a9e1efa7e11bad3aa8dc873a28325b3153799251740ae87c4623a08f7d45a5cb",
"tests/runner-e2e/hiring-template-flow.ts": "f3c6d29dc5019955dda4372ead7a9fdfbde447b7b92ef103527b9393ec60bd30",
"tests/runner-e2e/hiring-template-turn-accounting.ts": "8f2ad8b09e419796b372929b59dc7f86d41dcbf3a3144c45e329a1aa988b383b",
"tests/runner-e2e/chat-flow.ts": "09dd87ef7aef299ddbd1fad65b72817c042595f9e4a4e699c430497ac053a2d4"
},
"graderVersion": "paperclip.hiring-templates.v2",
"definitionDigest": "4d527906a967ca71c4c0e62cfbdc3405419931bcd6741b0630c4223bed2d4b6a",
"providerCalls": 0,
"newProviderRuns": 0,
"automaticRetries": 0,
"summary": {
"originalPairs": "Codex Fail→Fail; Claude Fail→Fail",
"correctedWorkflowOutcomePairs": "Codex Pass→Pass; Claude Pass→Pass",
"fullAttemptComparison": "Uncomparable in both profiles and variants; source-read coverage unchanged",
"originalMachineFailuresPreserved": 4,
"correctedScorerGuardsPassed": 4,
"correctedFinalChatGuardsPassed": 4,
"unchangedCoverageAssessments": 4,
"unchangedOtherOutcomeAssessments": 4,
"actualMeasuredRuns": 28,
"actualModelChargesKnown": false,
"broadEquivalenceEstablished": false
},
"assessments": [
{
"variant": "candidate",
"profile": "runner-codex",
"original": {
"sourceRevision": "9f5404ad3aacbe76777952759414d34fd381e674",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v2",
"definitionDigest": "4d527906a967ca71c4c0e62cfbdc3405419931bcd6741b0630c4223bed2d4b6a",
"outcomePassed": true,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": true,
"finalChatGuardPassed": true,
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v1",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
}
],
"passed": true
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": true,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": true,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "44cbf64cfabd0257f31fb4d6909018d346f5667b0a7b3d7e855a2ece7e29c292",
"hiringTemplate": "9609ddcb81c6bee3c5db97312138b4ea3e294da13ea540e41f363c826a514067",
"apiState": "ef02af29bcf61ff3eace0549ca164a28c7950d1a3e14fc58c89bf9cb0f944b4d"
}
},
{
"variant": "candidate",
"profile": "runner-acpx-claude",
"original": {
"sourceRevision": "9f5404ad3aacbe76777952759414d34fd381e674",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v2",
"definitionDigest": "4d527906a967ca71c4c0e62cfbdc3405419931bcd6741b0630c4223bed2d4b6a",
"outcomePassed": true,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": true,
"finalChatGuardPassed": true,
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v1",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
}
],
"passed": true
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": true,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": true,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "a83aecc26ef0af2e231cb364299cf0cb2f2ccd83b6998913dcd33be72d4c0824",
"hiringTemplate": "4f83f523b955120c5baac18572dce752b278dbc9ff765b940c0e17c37ab28ae0",
"apiState": "a208a3e3568fe9aacbec8d683a502a0d304a47571a1b97c229d2936e5d1bb087"
}
},
{
"variant": "baseline",
"profile": "runner-codex",
"original": {
"sourceRevision": "296a4df85e8bcc97a160fc78c291b17adb828196",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v2",
"definitionDigest": "4d527906a967ca71c4c0e62cfbdc3405419931bcd6741b0630c4223bed2d4b6a",
"outcomePassed": true,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": true,
"finalChatGuardPassed": true,
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v1",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
}
],
"passed": true
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": true,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": true,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "d912d8400bfff4f7f922aa6b264f9ce5fff1ba2d893abf03790c4cfea352261f",
"hiringTemplate": "88b23e4d2cc8831a5d88b8479cc69c0d1c2e17a6a5243707b4bd19716a244aca",
"apiState": "dff6bb13bb54d45024966d9cfbeb6dbe6f2241e674b0b1350c1384863202a97f"
}
},
{
"variant": "baseline",
"profile": "runner-acpx-claude",
"original": {
"sourceRevision": "296a4df85e8bcc97a160fc78c291b17adb828196",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v2",
"definitionDigest": "4d527906a967ca71c4c0e62cfbdc3405419931bcd6741b0630c4223bed2d4b6a",
"outcomePassed": true,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": true,
"finalChatGuardPassed": true,
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v1",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
}
],
"passed": true
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": true,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": false,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "844763ddc83461b2bea620f153f4bfe0bfee20a27bf33a60017a45c774ce2f6a",
"hiringTemplate": "27b153798472e206970c46358febdda363cb75f892734757e76994117f0d2c9b",
"apiState": "441cf7272ea28c5e1182adf48dba09a401e88c6cad3a99e72c8cf3f46b1c5b41"
}
}
]
}
doc/plans/2026-10-02-hiring-executable-accounting-replay.md
+59
View File
@@ -0,0 +1,59 @@
# Executable hiring lifecycle accounting repair
**Current TL;DR:** No new model calls were made. The original Codex and Claude pairs remain Fail → Fail. The stricter v3 executable replay verifies Codex accounting in both variants; Claude notification action attribution remains unresolved/uncomparable because provider and native request IDs cannot be exactly joined. No notification writes are observed in the retained native API receipts. Source-read coverage remains uncomparable for both profiles, independently of action attribution. Earlier sidecar-v1 and initial executable-v2 passes remain below with their limited task-only proof identified.
## Current stricter executable replay
Code revision: `e4077ade1818d98b9862ae79ee1d49a007dcf9c1`. Hiring grader: `paperclip.hiring-templates.v3`; turn accounting: `paperclip.hiring-template-turn-accounting.v2`. The separately versioned [v2 JSON receipt](2026-10-02-hiring-executable-accounting-replay.v2.json) pins exact source hashes and unchanged original inputs.
| Profile | Original machine grades | Limited sidecar v1 / initial executable v2 | Stricter executable accounting | Source-read coverage |
| --- | --- | --- | --- | --- |
| Codex | Fail → Fail | Pass → Pass | Verified / both new guards pass in each variant | Uncomparable both |
| ACPX Claude | Fail → Fail | Pass → Pass | Unresolved/uncomparable action attribution in each variant; guards fail closed | Uncomparable both; historical six-backtick exact-template mismatch remains |
The limited sidecar and initial executable check rejected notification-created tasks but could miss an unrelated document write during a notification. Their original results and hashes are preserved; they do not prove every notification was harmless. The stricter helper adds a twelfth predicate and an independent action-coverage check. It requires complete contiguous event streams, an accepted control-plane result and succeeded/completed terminal, exact canonical/native action IDs, successful known GET API receipts or verified reads/discovery, and attributed native chat finish. Document/task/agent mutations, failed mutation attempts, incomplete streams and unknown or unmatched actions cannot pass. A readonly hint alone cannot qualify a generic API call.
Codex verifies 10 candidate and 11 baseline notification tool executions, including five and seven exactly joined successful GET calls, with zero unresolved actions. ACPX's native request IDs and provider execution IDs use separate namespaces; names, ordering and counts cannot safely join them. All observed ACPX native API calls are successful GETs, but the stronger cross-ledger proof is missing (five candidate/four baseline unresolved observations). This is an evidence limitation, not a newly observed task failure or a prompt regression. No production carriers or source-read grading are changed.
The observation repair retries entire bracketed snapshots, waits both known Done task callbacks and attributed chat replies (including batching), checks untruncated pending-wake diagnostics, and requires two identical complete observations. It covers the quiet gap before pending outbox work enqueues a wake. Accounted coalesced wakes are admitted only with a linked terminal run and known callbacks. The final guard performs that same complete refresh rather than mixing a stale evidence snapshot with new runs. The generic five-work-turn helper still admits zero notifications when none are owed; this production delegated fixture owes two task completions.
All six non-count outcome checks and every original source/template coverage check remain byte-identical in all four replay projections. All 28 actual model runs remain counted; original input hashes and failed verdicts are unchanged. The stricter action check is separately classified as coverage when attribution is missing. No broad equivalence is established.
- 977 credential-free support tests pass across 64 files, including 144 focused helper/scorer/settlement integrations.
- E2E typecheck, exact two-cell discovery, canonical capability contract/inventory checks and diff checks pass.
- Two current-head review findings are fixed: racing observations and unchecked completion-turn writes. Fresh review/CI is required on this source.
- Initial-head CI retained two unrelated browser failures: exact agent-run denial feedback and touch-picker scroll position. Neither browser path imports the changed runner-e2e harness. They are preserved without a blind rerun; the necessary review-fix head runs normal CI.
- No providers, old campaign retries or extra paid scope are launched. Actual prior model charges remain unknown.
## Initial executable v2 replay (limited notification proof)
TL;DR: This repairs a grading defect; it does not change model instructions or rerun models. The original Codex and Claude pairs remain Fail → Fail. Provider-free replay through both corrected executable paths passes all four retained cells; source-read coverage remains uncomparable. This is a grading correction, not a new model-performance result.
The original fixture required exactly five total runs. Production adds legitimate server completion turns after delegated work. The corrected v2 contract requires exactly three distinct user-requested CEO turns and two coder executions. At most two additional completion turns must pass strict public identity/account/task/delivery/timing/reply attribution. One turn may batch both completed tasks. Unknown, duplicate, failed, retried runs, notification-created tasks and missing public observations fail. This initial version did not inspect unrelated document writes during notifications; see the stricter current assessment above.
Both executable count guards use the shared helper: the hiring scorer and the hiring-only final chat guard. The catalog declares five required / seven maximum total runs so timeout/cost planning includes notifications. Non-hiring count guards remain unchanged. Source-read and exact coder-body grading remain unchanged, including the historical Claude six-backtick mismatch. The grader version and full helper/chat/source digest change.
## Preserved measurement
The immutable [original and bounded sidecar report](https://github.com/paperclipai/paperclip/blob/8eb517ca1497687237163bdef4dfc4d3332ea916/doc/plans/2026-10-02-hiring-template-live-comparison.md) retains candidate `9f5404ad3aacbe76777952759414d34fd381e674` and historical `296a4df85e8bcc97a160fc78c291b17adb828196`, the identical original fixture digest, 28 actual successful runs, eight automatic completion turns, four successful cleanups and unknown actual charges. It calibrates sidecar v1 with 69 passing tests and records exact original input/grader hashes. The executable repair is a new source/grader revision; it cannot rewrite those measurements or prove performance equivalence.
## Provider-free verification
The executable code revision is `eef64009dc144b91b245a731ea9a1c3c07406a19`; report-only commits do not change its grader bytes. The [JSON receipt](2026-10-02-hiring-executable-accounting-replay.json) pins all five grader/flow/helper source hashes, v2 definition digest and original result/hiring/API input hashes.
| Profile | Original executable result | Corrected retained workflow outcome | Both new count guards | Source coverage |
| --- | --- | --- | --- | --- |
| Codex | Fail → Fail | Pass → Pass | Pass in both variants | Uncomparable both |
| ACPX Claude | Fail → Fail | Pass → Pass | Pass in both variants | Uncomparable both; historical exact coder body still fails |
All six other outcome checks and every coverage check are byte-identical in the replay's check projection. Original input files and failed grades remain unchanged. All 28 actual model runs remain counted. The new helper has 11 lifecycle predicates, factoring the sidecar's lifecycle rule without requiring an already-computed original result.
- All 946 credential-free E2E support tests pass across 64 files, including 99 helper calibrations and the four added scorer/final-guard integrations.
- Fifteen focused hiring/run-count tests pass. The integrations admit five core turns, six with batching and seven with distinct notifications; reject an arbitrary wake despite the same total count; require public observations; retain source/template coverage failures; and preserve non-hiring count guards.
- E2E typecheck, normal plugin SDK/Runner TypeScript dependency builds, canonical capability contract/inventory checks and existing two-cell discovery pass.
- Retained replay executes the actual new scorer and final chat guard in each of the four original cells, using the unchanged saved API observations. Both guards pass 4/4; coverage and the six non-count outcomes stay unchanged 4/4.
- Full repository CI and fresh review are pending on the separate draft PR. Local general repository typecheck/test/build were not repeated; exact-head CI must supply those gates before handoff.
Initial development calibration caught mismatched synthetic account data and a deliberately changed template not preserved across reuse. The fixture data was corrected; assertions were kept. Initial cold typecheck lacked built Runner declarations; normal dependency builds resolved it. A temporary replay script's CommonJS extension rejected top-level await; the same script executed as ESM. These provider-free setup/development attempts remain private and are not live model results. No provider runs, old campaign retries or paid scope expansion are authorized for this repair. Public replay receipts will include only grader/input hashes, counts, predicate/check results and original verdicts; credentials, provider session IDs and hidden reasoning remain private.
The branch was safely replayed on master `59c07ede7` before PR creation. The intervening master changes touch only two agent-provider UI files; all eval source bytes are unchanged. The four-cell provider-free replay was repeated against the final reachable code revision, with the same input hashes, checks and zero providers.
doc/plans/2026-10-02-hiring-executable-accounting-replay.v2.json
+815
View File
@@ -0,0 +1,815 @@
{
"schema": "paperclip.hiring-executable-accounting-replay.v2",
"assessmentType": "Provider-free replay through both new executable guards; original model measurements and verdicts unchanged",
"evaluatedCodeRevision": "e4077ade1818d98b9862ae79ee1d49a007dcf9c1",
"evaluatedCodeHashes": {
"tests/runner-e2e/hiring-template-cases.ts": "a440ebf5a932525d388f507f8b47490a535d377c9f9c9beba6144f1fc2214b7c",
"tests/runner-e2e/hiring-template-scoring.ts": "5f600b8329c981f1fc94ecc1b70a94b78d4a32a5b8859600208cbe1f91be2dac",
"tests/runner-e2e/hiring-template-flow.ts": "10b2415ad6ee293286db0a8cc8cb7a6a363d993a70560ab0a82f668018b5e3f5",
"tests/runner-e2e/hiring-template-turn-accounting.ts": "13792c69669b99748087d6a78af8ab35c6293d1ad3e3396efcbe39620169ad35",
"tests/runner-e2e/chat-flow.ts": "85d8c3da832871f38cc4ed940046d28ed7b3edf17486ce43b0f3313c06d17d7a"
},
"graderVersion": "paperclip.hiring-templates.v3",
"definitionDigest": "cddbcb6b2f3050e065599011ace2faee048047fae6ea9bc9183db8c5744e2f38",
"providerCalls": 0,
"newProviderRuns": 0,
"automaticRetries": 0,
"summary": {
"originalPairs": "Codex Fail→Fail; Claude Fail→Fail",
"correctedWorkflowOutcomePairs": "Codex Pass→Pass; Claude action attribution unresolved/uncomparable in both variants (not a measured task failure)",
"fullAttemptComparison": "Uncomparable in both profiles and variants; source-read coverage unchanged",
"originalMachineFailuresPreserved": 4,
"correctedScorerGuardsPassed": 2,
"correctedFinalChatGuardsPassed": 2,
"strictActionVerifiedAssessments": 2,
"strictActionUncomparableAssessments": 2,
"unchangedOriginalCoverageAssessments": 4,
"unchangedOtherOutcomeAssessments": 4,
"actualMeasuredRuns": 28,
"actualModelChargesKnown": false,
"broadEquivalenceEstablished": false,
"priorSidecarLimitation": "Sidecar v1 and initial executable v2 checked notification-created tasks, not unrelated document writes. Their original passes remain preserved."
},
"assessments": [
{
"variant": "candidate",
"profile": "runner-codex",
"original": {
"sourceRevision": "9f5404ad3aacbe76777952759414d34fd381e674",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v3",
"definitionDigest": "cddbcb6b2f3050e065599011ace2faee048047fae6ea9bc9183db8c5744e2f38",
"outcomePassed": true,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": true,
"finalChatGuardPassed": true,
"assessmentStatus": "verified",
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v2",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
},
{
"id": "completion-turns-only-report-actions",
"passed": true
}
],
"actionEvidence": {
"status": "verified",
"notificationRuns": 2,
"canonicalExecutions": 10,
"matchedNativeApiCalls": 5,
"unknownExecutions": 0
},
"passed": true
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": true,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": true,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
},
{
"id": "completion-action-attribution",
"dimension": "coverage",
"passed": true,
"detail": "Notification actions require exact canonical/native identities; missing cross-namespace mapping is uncomparable, not proof of extra work."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "44cbf64cfabd0257f31fb4d6909018d346f5667b0a7b3d7e855a2ece7e29c292",
"hiringTemplate": "9609ddcb81c6bee3c5db97312138b4ea3e294da13ea540e41f363c826a514067",
"apiState": "ef02af29bcf61ff3eace0549ca164a28c7950d1a3e14fc58c89bf9cb0f944b4d"
}
},
{
"variant": "candidate",
"profile": "runner-acpx-claude",
"original": {
"sourceRevision": "9f5404ad3aacbe76777952759414d34fd381e674",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v3",
"definitionDigest": "cddbcb6b2f3050e065599011ace2faee048047fae6ea9bc9183db8c5744e2f38",
"outcomePassed": false,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": false,
"finalChatGuardPassed": false,
"assessmentStatus": "uncomparable",
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v2",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
},
{
"id": "completion-turns-only-report-actions",
"passed": false
}
],
"actionEvidence": {
"status": "uncomparable",
"notificationRuns": 2,
"canonicalExecutions": 3,
"matchedNativeApiCalls": 0,
"unknownExecutions": 5
},
"passed": false
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": false,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": true,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
},
{
"id": "completion-action-attribution",
"dimension": "coverage",
"passed": false,
"detail": "Notification actions require exact canonical/native identities; missing cross-namespace mapping is uncomparable, not proof of extra work."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "a83aecc26ef0af2e231cb364299cf0cb2f2ccd83b6998913dcd33be72d4c0824",
"hiringTemplate": "4f83f523b955120c5baac18572dce752b278dbc9ff765b940c0e17c37ab28ae0",
"apiState": "a208a3e3568fe9aacbec8d683a502a0d304a47571a1b97c229d2936e5d1bb087"
}
},
{
"variant": "baseline",
"profile": "runner-codex",
"original": {
"sourceRevision": "296a4df85e8bcc97a160fc78c291b17adb828196",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v3",
"definitionDigest": "cddbcb6b2f3050e065599011ace2faee048047fae6ea9bc9183db8c5744e2f38",
"outcomePassed": true,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": true,
"finalChatGuardPassed": true,
"assessmentStatus": "verified",
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v2",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
},
{
"id": "completion-turns-only-report-actions",
"passed": true
}
],
"actionEvidence": {
"status": "verified",
"notificationRuns": 2,
"canonicalExecutions": 11,
"matchedNativeApiCalls": 7,
"unknownExecutions": 0
},
"passed": true
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": true,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": true,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
},
{
"id": "completion-action-attribution",
"dimension": "coverage",
"passed": true,
"detail": "Notification actions require exact canonical/native identities; missing cross-namespace mapping is uncomparable, not proof of extra work."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "d912d8400bfff4f7f922aa6b264f9ce5fff1ba2d893abf03790c4cfea352261f",
"hiringTemplate": "88b23e4d2cc8831a5d88b8479cc69c0d1c2e17a6a5243707b4bd19716a244aca",
"apiState": "dff6bb13bb54d45024966d9cfbeb6dbe6f2241e674b0b1350c1384863202a97f"
}
},
{
"variant": "baseline",
"profile": "runner-acpx-claude",
"original": {
"sourceRevision": "296a4df85e8bcc97a160fc78c291b17adb828196",
"graderVersion": "paperclip.hiring-templates.v1",
"definitionDigest": "4d7f18420889325af89aa48eaaf021e8178aa6d26d504b92041635a15741f004",
"machineStatus": "failed",
"outcomePassed": false,
"countCheckPassed": false,
"comparisonStatus": "uncomparable"
},
"corrected": {
"graderVersion": "paperclip.hiring-templates.v3",
"definitionDigest": "cddbcb6b2f3050e065599011ace2faee048047fae6ea9bc9183db8c5744e2f38",
"outcomePassed": false,
"comparisonStatus": "uncomparable",
"scorerGuardPassed": false,
"finalChatGuardPassed": false,
"assessmentStatus": "uncomparable",
"turnAccounting": {
"version": "paperclip.hiring-template-turn-accounting.v2",
"counts": {
"requiredWorkTurns": 5,
"maximumCompletionTurns": 2,
"maximumTotalTurns": 7,
"requestedLeadTurns": 3,
"coderTurns": 2,
"completionTurns": 2,
"unclassifiedTurns": 0,
"snapshotRunCount": 7,
"actualRunCount": 7,
"costAccountingRunCount": 7
},
"predicates": [
{
"id": "known-fixture-context",
"passed": true
},
{
"id": "complete-public-run-ledger",
"passed": true
},
{
"id": "exact-five-required-work-turns",
"passed": true
},
{
"id": "successful-native-without-retries",
"passed": true
},
{
"id": "resolved-company-account-and-identity",
"passed": true
},
{
"id": "three-distinct-requested-chat-turns",
"passed": true
},
{
"id": "one-coder-execution-per-known-task",
"passed": true
},
{
"id": "task-origins-are-first-two-requested-turns",
"passed": true
},
{
"id": "bounded-server-completion-receipts",
"passed": true
},
{
"id": "completion-runs-have-attributed-chat-replies",
"passed": true
},
{
"id": "no-notification-created-extra-tasks",
"passed": true
},
{
"id": "completion-turns-only-report-actions",
"passed": false
}
],
"actionEvidence": {
"status": "uncomparable",
"notificationRuns": 2,
"canonicalExecutions": 3,
"matchedNativeApiCalls": 0,
"unknownExecutions": 4
},
"passed": false
},
"checks": [
{
"id": "one-coder-hire",
"dimension": "outcome",
"passed": true,
"detail": "Exactly one permanent coding teammate reports to the lead."
},
{
"id": "execution-account",
"dimension": "outcome",
"passed": true,
"detail": "Hire keeps the lead model and managed account binding."
},
{
"id": "two-worker-tasks",
"dimension": "outcome",
"passed": true,
"detail": "Both independent tasks are completed by the same coder in the chosen project."
},
{
"id": "bounded-work-and-completion-turns",
"dimension": "outcome",
"passed": false,
"detail": "Exactly three requested lead turns and two coder executions, plus at most two strictly attributed completion notifications; every actual run remains counted."
},
{
"id": "initial-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "Independent computation checks each original input/value and worker authorship."
},
{
"id": "reused-json-artifact",
"dimension": "outcome",
"passed": true,
"detail": "The reused coder applies the changed separator to every input."
},
{
"id": "original-preserved",
"dimension": "outcome",
"passed": true,
"detail": "Reuse preserves the original document, revision and task identity."
},
{
"id": "source-fingerprints",
"dimension": "coverage",
"passed": true,
"detail": "Served instruction and hiring source bytes match the evaluated revision."
},
{
"id": "production-ceo-bundle",
"dimension": "coverage",
"passed": true,
"detail": "The API-created lead receives this revision's default CEO bundle, without a fixture override."
},
{
"id": "assigned-hiring-skill",
"dimension": "coverage",
"passed": true,
"detail": "Production CEO defaults assign the hiring skill."
},
{
"id": "production-source-reads",
"dimension": "coverage",
"passed": false,
"detail": "Completed lead read receipts before the hire prove the explicitly requested skill, guide, checklist and coder example paths. Unrecognized or missing reads leave coverage uncomparable."
},
{
"id": "supplied-coder-instructions",
"dimension": "coverage",
"passed": false,
"detail": "Saved hired instructions use the source revision's coder example with its company/name placeholders filled."
},
{
"id": "hired-instructions-durable",
"dimension": "coverage",
"passed": true,
"detail": "The same saved instruction bundle survives the reused worker execution."
},
{
"id": "hired-skills-durable",
"dimension": "coverage",
"passed": true,
"detail": "The saved skill selections survive the reused worker execution."
},
{
"id": "completion-action-attribution",
"dimension": "coverage",
"passed": false,
"detail": "Notification actions require exact canonical/native identities; missing cross-namespace mapping is uncomparable, not proof of extra work."
}
]
},
"coverageUnchanged": true,
"otherOutcomesUnchanged": true,
"originalBytesUnchanged": true,
"inputSha256": {
"result": "844763ddc83461b2bea620f153f4bfe0bfee20a27bf33a60017a45c774ce2f6a",
"hiringTemplate": "27b153798472e206970c46358febdda363cb75f892734757e76994117f0d2c9b",
"apiState": "441cf7272ea28c5e1182adf48dba09a401e88c6cad3a99e72c8cf3f46b1c5b41"
}
}
]
}
doc/plans/2026-10-02-legacy-document-skill-repair.json
+1187
View File
File diff suppressed because it is too large. Load diff
doc/plans/2026-10-02-legacy-document-skill-repair.md
+51
View File
@@ -0,0 +1,51 @@
# Legacy document skill repair qualification — 2026-10-02
**TL;DR: one newly passing case (Claude original document delivery), one unchanged pass (Claude explicit storage), two unchanged OpenCode failures, and no pending pairs or new overall machine failures. OpenCode's explicit-case clickable handoff nevertheless got worse:** baseline supplied a clickable API document URL, while candidate supplied only a code-formatted path. Both fail the original UI-only link oracle, whose intended route was narrower than the request's "usable link" wording. This result does not establish general non-regression.
This focused repair keeps the eight-word manual, reduced shared prompts and merged Codex base fix #14920 fixed. It varies only the early operational Paperclip skill recipe and the presence of the new `issue-documents.md` reference. The earlier combined prompt-removal comparison retains its two new classic Claude/OpenCode storage failures; this trial repairs Claude's original case but leaves OpenCode's original regression unresolved.
| Classic profile | Case | Pre-fix skill → repaired skill | Retained evidence |
| --- | --- | --- | --- |
| Claude | Original assigned skill | Fail → Pass | Baseline lacks a durable task document; candidate saves one containing the skill marker. |
| Claude | Explicit Paperclip document | Pass → Pass | Both save the document/revision and provide a canonical UI link. |
| OpenCode | Original assigned skill | Fail → Fail | Both write a workspace file rather than a public task document. Candidate requires automatic disposition recovery. |
| OpenCode | Explicit Paperclip document | Fail → Fail | Both save the document/revision. Baseline provides a clickable API URL; candidate gives a code-formatted UI path without an anchor. Original UI-link checks fail in both. |
Original machine verdicts and evidence hashes are retained in the [safe evidence projection](2026-10-02-legacy-document-skill-repair.json). No model was rerun to improve these results.
## Frozen sources and public reports
- [Repaired skill campaign](https://github.com/paperclipai/paperclip/actions/runs/37060885547), source `abd0b628ca642c09a54a4edc56a5227402f6686e`, branch `codex/legacy-document-repaired-skill`. [Published candidate report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37060885547-1/index.html).
- [Pre-fix skill campaign](https://github.com/paperclipai/paperclip/actions/runs/37060888047), source `bc83fe030234439ac51279502a28803958963e2e`, branch `codex/legacy-document-pre-fix-skill`. [Published baseline report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37060888047-1/index.html).
- Both trusted default-branch workflows ran concurrently on separate concurrency targets. All eight cells pass their exact-source prerequisite gate before providers; all eight clean up successfully. Failed campaigns are published successfully and remain failed.
- The four model/effort/auth/tool/permission/environment configurations match. Models are `claude-sonnet-4-6` and `openrouter/deepseek/deepseek-v4-flash-0731`. The comparison records 203 byte-identical fixture/behavior sources and explicit present/absent skill-source fingerprints. The new reference is absent from the baseline, rather than silently introduced into it.
- Local admission preparation passed 909 E2E support tests, E2E typecheck and four-cell discovery; the candidate exact-source gate passed 571 checks, including one Rust check. Initial sandbox and obsolete catalog-count attempts remain retained. The original request and behavioral oracle were not weakened.
## What OpenCode actually received
The repaired original assignment invokes only its assigned Context integrity output skill, then writes `task-document.md`. There is no operational Paperclip skill load or issue-document reference read before that local write. A separate automatic `issue_disposition_repair` run then loads Paperclip. Its returned content contains the new early saved-document paragraph and reference pointer even though the overall skill result is marked truncated; it never reads the reference or saves a document, and only finalizes the issue's status.
This is evidence of operational-skill selection after work, not evidence that the assignment saw and ignored the new recipe. The pre-fix original run also writes locally before loading the old Paperclip skill. Whether the adapter should activate operational Paperclip guidance before work is a production-delivery question for review; expanding a skill the first assignment never opened would not address the observed sequence.
The explicit candidate loads the assigned skill, then Paperclip, receives the new early paragraph, reads `issue-documents.md`, and calls the document API successfully. Its saved document has a persisted revision. The remaining defect is its comment's non-clickable path. Baseline supplied an actual Markdown link to the issue-document API GET endpoint. Read-only source review confirms that this endpoint returns the document in board context, and the unprefixed UI issue route redirects through the selected company's prefix while preserving the document hash. Actual authenticated browser navigation against these cleaned-up measured instances was not replayed, so neither URL is classified as a proven broken route. Candidate's missing clickable anchor is directly observable.
A future explicit fixture should name a clickable Paperclip UI document link if that is the intended contract. This clarification would not retroactively pass or fail either original result. A short canonical Markdown UI-link example can be considered separately from operational-skill delivery. No further production instruction change or paid rerun was made for this diagnosis.
## Runs, timing and cost
Eight provider turns were expected; nine actual runs are retained. Candidate OpenCode's original assignment succeeds but leaves the task `in_progress`, triggering automatic disposition recovery. The recovery succeeds and marks it done, without creating the missing document. Both runs, their time and cost are counted rather than selecting a better attempt.
| Profile | Case | Baseline provider seconds | Candidate provider seconds | Baseline reported USD | Candidate reported USD |
| --- | --- | ---: | ---: | ---: | ---: |
| legacy-claude | Original | 23.032 | 27.702 | 0.1668394500 | 0.2267947500 |
| legacy-claude | Explicit storage | 34.999 | 40.513 | 0.1997571000 | 0.2561425500 |
| legacy-opencode | Original | 36.770 | 68.032 | 0.0033927116 | 0.0047148437 |
| legacy-opencode | Explicit storage | 206.191 | 46.300 | 0.0102644593 | 0.0042468724 |
Reported LLM totals are $0.3802537209 baseline and $0.4918990161 candidate. All four baseline and five candidate ledger rows contain token usage and reported cost. Local runtime is unmetered and actual external billing is not established. Single trials, the additional recovery and baseline OpenCode's long explicit-case execution prevent a general timing or spending conclusion.
## PR maintenance and remaining qualifications
The skill insertion shifted generated capability heading anchors. General PR CI detected stale `capabilities.yaml`; the paid workflow's narrower build and every exact-source admission still passed before provider access. Both canonical capability metadata inventories are regenerated after measurement, include the new reference where declared, and adds a credential-free admission test that rejects stale/missing manifests. These maintenance changes do not modify the frozen measured branches or the skill bytes used in this comparison.
Draft [PR #14948](https://github.com/paperclipai/paperclip/pull/14948) keeps this repair and the tiny manual; draft [PR #14961](https://github.com/paperclipai/paperclip/pull/14961) measures native completion documentation separately. OpenCode original delivery remains unresolved, and existing Claude chat-memory and legacy ACP credential/receipt findings remain separate. No broad matrix rerun was launched, and merging remains user-controlled.
doc/plans/2026-10-02-opencode-skill-routing-link-qualification.json
+1418
View File
File diff suppressed because it is too large. Load diff
doc/plans/2026-10-02-opencode-skill-routing-link-qualification.md
+75
View File
@@ -0,0 +1,75 @@
# OpenCode skill routing and clickable delivery — 2026-10-02
**TL;DR: 0 newly failing cells, 0 newly passing cells, 2 unchanged passes, and 0 pending pairs in this matched trial. Both document cases pass in both variants, so this does not establish that the metadata/link change caused the earlier failures to recover. Handoff remains imperfect in the original case:** candidate uses a clickable link with the wrong `PAP` company prefix; baseline supplies only a bare prefix-less slug path. Neither defect is scored by the preserved original document-storage oracle. The explicit clickable-link cases deliver the correct `RUN` link in both variants.
The narrow correction expands the operational skill's stock discovery description to include Paperclip task/heartbeat work and document/file delivery. The early API recipe asks for a clickable Markdown link, and the reference shows a canonical UI-link example. The full skill stays on disk; the eight-word manual and shared prompts remain reduced. No full-body injection, adapter policy override or native tool guidance is added to legacy runs.
| Classic OpenCode case | Pre-correction skill | Corrected skill | Independent evidence and limitation |
| --- | --- | --- | --- |
| Original assigned skill, verbatim request | Pass | Pass | Exactly one saved document with skill-only marker and persisted revision in each. Both handoff paths are deficient and outside this case's link-free oracle. |
| Explicit Paperclip storage with clarified clickable UI link | Pass | Pass | Saved revision/content and exact clickable `/RUN/issues/RUN-1#document-context-integrity-output` in both. |
Original machine results, source/configuration proof, tool-read sequence, costs and evidence hashes are preserved in the [safe evidence projection](2026-10-02-opencode-skill-routing-link-qualification.json). All four cells clean up successfully; none uses automatic disposition recovery or an extra attempt. No provider was rerun to improve these results.
## Sources, admission and inspectable reports
- Candidate `fe9dc1e3c518825242ed889ab9c8352986f8c2ed`, frozen branch `codex/opencode-stock-routing-qualified`: [workflow](https://github.com/paperclipai/paperclip/actions/runs/37069547401), [published dashboard and screenshots](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37069547401-1/index.html).
- Baseline `0d7ecfa96d72fba79b7f0a25052b42c0686c0488`, frozen branch `codex/opencode-pre-routing-skill`: [workflow](https://github.com/paperclipai/paperclip/actions/runs/37069552374), [published dashboard and screenshots](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37069552374-1/index.html).
- Trusted default-branch workflows ran concurrently on separate targets. Each of four measured cells passed its own exact-source, credential-free 587-assertion prerequisite before provider credentials. Candidate source fingerprint is `aeb792549146cc29eda779e83ae22d336276e31162f738a1dde85a82e7833902`; baseline is `0315602ac6788cf0d65561a73cc85d790768ef90c6815257077a2381f961b96e`.
- Git tree comparison verifies 8,242 identical tracked files. Only the two operational skill sources and an explicit baseline provenance receipt differ. Models, effort, tools, auth, permissions, budgets, manual/shared prompts and behavioral fixtures/graders are fixed; both fixture configuration hashes match. Suite/source digest differences correctly include the changed skill bytes.
- Local source preparation passed 918 E2E support tests, eight canonical inventory calibrations, skill validation, E2E typecheck, two-cell discovery and both exact-source prerequisites. Both capability inventories use the same declared source list. Initial setup mistakes remain retained and did not reach providers.
The explicit fixture now asks for a clickable company-prefixed Paperclip UI document link, matching its intended oracle. Bare paths and code-formatted paths fail calibration; correct relative/same-app absolute links pass. The original assigned-skill request and procedure remain verbatim. Earlier API-link grades are not retroactively changed, and these explicit results are not a direct repeat of the older “usable link” cohort.
## Which guidance was read before delivery
In the candidate original assignment, OpenCode loads the assigned Context integrity output skill, then operational Paperclip before any document write. The returned Paperclip skill is marked truncated, but the early saved-document recipe and reference pointer are visible. It reads `issue-documents.md`, then writes the public task document through the API. This differs from the previously failing candidate, which first loaded Paperclip only in a separate disposition-recovery run after its local-only output.
The current baseline original assignment first loads the assigned output skill and writes a workspace Markdown file. It later loads Paperclip and reads the issue-document reference, then saves a public document before completing the **same** assignment. Thus its successful delivery did not require improved metadata; stock selection timing varies between trials. Both explicit variants load Paperclip and read the reference before the document API call. Four rendered final-state screenshots were inspected, and public document revisions/content and completion comments agree with the retained results.
The candidate original comment has an actual Markdown href `/PAP/issues/RUN-1#document-context-integrity-output`, despite the measured company prefix being `RUN`. Baseline's original comment has a bare `/issues/runner-e2e-…#document-task-document` path without a Markdown anchor. The candidate's literal prefix matches the reference's example, making example copying a plausible cause, but this trial does not isolate that cause. Authenticated click navigation was not replayed after cleanup; neither original link is presented as a verified usable handoff. A minimal follow-up can show an issue-derived prefix instead of a literal company example. These original machine passes must not be presented as fully correct links.
## Timing, usage and cost
Four provider turns were expected and four assignment runs occurred. All four ledger receipts contain token usage and reported LLM cost. Local runtime remains unmetered; actual external billing is not established.
| Case | Baseline provider seconds | Candidate provider seconds | Baseline cell seconds | Candidate cell seconds | Baseline reported USD | Candidate reported USD |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Original | 25.870 | 67.897 | 46.285 | 89.455 | 0.0058472545 | 0.0041383700 |
| Explicit clickable storage | 73.984 | 82.751 | 94.804 | 102.901 | 0.0049436470 | 0.0066440790 |
Reported LLM totals are **$0.0107909015 baseline** and **$0.0107824490 candidate**. The candidate original case takes longer in this trial; these single observations, different cache/token receipts and unmetered runtime do not establish general speed, cost or quality equivalence.
## Preserved failures and scope
The [original combined prompt-removal report](2026-10-02-stock-harness-live-comparison.md) still records two new classic Claude/OpenCode document-delivery failures, two OpenCode ordered-case improvements, and separate unchanged credential/chat failures. The [first recipe repair report](2026-10-02-legacy-document-skill-repair.md) still records Claude's improvement, OpenCode's unresolved local-only output, and the candidate's worse non-clickable explicit handoff. New successes do not erase those earlier observations or establish robustness.
This comparison qualifies only classic OpenCode's two selected journeys. Existing classic Codex/Claude/OpenCode and ACP Codex skill mounts were audited; ACP Claude's names/root-only metadata and unsupported custom ACP delivery remain separate. Hermes/Pi/OpenClaw are not live-qualified here. Native completion descriptions and hiring templates have separate changes and measurements. Both PRs remain draft; no merge or further paid breadth is part of this report.
## Provider-free follow-up after measurement
The approved reference-only follow-up replaces the literal `PAP` link with a
JavaScript construction using the current `issue.identifier` and the successful
write receipt's `saved.key`. It is a later source change, **not live-qualified**
by the frozen candidate above. The independent storage grader now calibrates a
wrong-company link as failure, and executes the shipped recipe against two
other company prefixes plus a locked-write redirected document key. All 25
focused document/source-digest tests, eight inventory calibrations, canonical
generated metadata checks and E2E typecheck pass. An initial sandboxed support attempt denied localhost/tsx
pipe creation; the failed attempt is retained, with its affected files rerun
unchanged under permitted local execution: 906 assertions passed in the original
921-assertion attempt, and all 82 selected assertions passed in the permitted
retry, including every originally failed assertion. No new model calls or old verdict
changes are part of this follow-up.
Fresh native-PR review caught a supported edge case in that later recipe:
`issue.identifier` may be null. The recipe now uses the current issue ID in the
unprefixed UI route when no identifier exists. Provider-free tests cover both
null and absent identifiers, preserving the returned document key. Source review
confirms that the board resolves the loaded issue's actual company and preserves
the document hash; no agent-side company fetch is needed. This route logic also
corrects wrong prefixes, so the frozen candidate's `PAP` href is noncanonical,
not a proven broken link. No authenticated click replay or further model run was
made. The previous 4/5 review and local setup/timing failures remain retained;
current focused document/source/manifest/admission checks pass 56 assertions and
E2E typecheck passes. This correction still has no live qualification.
doc/plans/2026-10-02-stock-harness-live-comparison.json
+357
View File
@@ -0,0 +1,357 @@
{
"schema": "paperclip.stock-harness.safe-comparison/v1",
"candidateSha": "f02d8d0df327abb43b20c7e7beb86798239abbf5",
"baselineSha": "12c5433c67dc2e62916b879349c7ba2b6e0431f0",
"nativeCodexFixHeldConstant": "408f70e69f9c5e49cb4377f4886ac2001bfa67a2",
"comparisonManifest": {
"schema": "paperclip.stock-harness-comparison.v1",
"variant": "previous-instructions",
"candidateSha": "f02d8d0df327abb43b20c7e7beb86798239abbf5",
"restoredInstructionsFrom": "e00d10d5d5594f6e2d1e8cf4e0ec81c074b0bd88",
"nativeCodex14920": "held-constant; no before/after performance claim",
"productionDifferences": [
"server/src/onboarding-assets/default/AGENTS.md",
"packages/adapter-utils/src/server-utils.ts"
],
"structuralOracle": "Historical manual exact content and observed generic procedures; independent behavioral graders unchanged",
"expectedMatrixCells": 24,
"expectedProviderTurns": 48,
"maxInitialCampaignsPerVariant": 1,
"behavioralSources": {
"context-integrity-cases.ts": "201d05f8257c1121a94dd9d8a22f08b9db80555e388013ffeabb65e301499fd2",
"context-integrity-scoring.ts": "50d370044e45386ba87fb23ca63c228b96bb8373d67bb7e82e8b2a89eb23f252",
"context-integrity-flow.ts": "63bcb7bdba4c80b15ff5872f16383844456eab0c5a10627da5d3487a26821c6a",
"chat-cases.ts": "754b251bdd42605e147da9081b663329234260d2b13df0df3d44562e362b14ca",
"chat-flow.ts": "11fc506559da61315ce04a877f6b38a055e71e532fecfcf62c749a1d3bf985e7",
"live-fixtures.ts": "1ef80209cc53cc514a8e58af4c2225e38ec15b20acf17ca7564506ca6617ac3e",
"runner.spec.ts": "14d57e0f9131b3f1981ac593464738d01362008eefe85ea836075c794681fa85",
"launch.ts": "5ada6ca5484542f028b29fc603dc7992213d4c06fa9046834bd7d23c4511b847",
"stock-harness-admission.ts": "c9775d77d48f8d59ba1197a7e5769f14db0c7d33f5e2a2968a04466e083d63ff",
"types.ts": "c45dc3b2366457449ec9e7d413f930605dec88ea93a14ce5c5a5f445a27d148c"
},
"instructionSha256": {
"manual": "e4d2375d722602cd744403292d99f6e6c9b7e8014d9cdfa6f9811ace03428e5f",
"shared": "819026782b92f37064fca354ddbca685b3c05f1869ec560e40d8a757b0a8aee9"
}
},
"modelsEffortToolsPermissionsCredentials": "same fixture/source configuration; effort inherits provider default when unspecified",
"expectedCellsPerVariant": 24,
"expectedProviderTurnsPerVariant": 48,
"baseline": [
{"executionId":"stock-harness.legacy-acp-claude.local.assigned-skill-explicit-invocation","profile":"legacy-acp-claude","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard"],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":59075,"providerDurationMs":42037,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":15184,"outputTokens":2261,"cachedInputTokens":218341,"totalTokens":235786,"reportedCostUsd":0.16114005000000003,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":42037,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.16114005000000003,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.16114005000000003,"complete":true},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":14137,"reportedPromptChars":13607,"freshPromptChars":4688,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"78f6add6f11a45f58bfc6fee81600871f888750986c27497563a79eba39a101c","evidence-manifest.json":"14f67d036a3663f606cf269890f5a36a18221e5c7dcb466f513f761d882fb6eb","snapshots/context-integrity.json":"93bba435c845af31707d4e4b84b2330cd8ff14045a76c2d1267b958c8133950b","snapshots/stock-harness-hire.json":"e8b85aa909fc6af6a72b180c79572d70a26dd6ee6307c9b985f80b928eb96170","snapshots/stock-harness.json":"fa0eb10eb3939b91d83fed8a8c9f9915bb407ab03a5a9bd7e5df99aa593ffb99","snapshots/stock-harness-preflight.json":"2ac069369e2d26b0e4c213966770962c0282f9a7665fa1297fe3c70df2c73960"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-acp-claude.local.continuity-restart","profile":"legacy-acp-claude","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard","structural_receipt_incomplete"],"failedMatcherPaths":["stockHarness.fresh-default-delivered"],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":36961,"providerDurationMs":19492,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":19069,"outputTokens":329,"cachedInputTokens":55518,"totalTokens":74916,"reportedCostUsd":0.1039979,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":19492,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.1039979,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.1039979,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":false},"invocationMetrics":[{"capturedPromptChars":16395,"reportedPromptChars":17606,"freshPromptChars":1462,"retrievalClipped":true},{"capturedPromptChars":16395,"reportedPromptChars":17638,"freshPromptChars":1462,"retrievalClipped":true}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"b00296e015e0d35a213e031aede08732f900cf8aaa0f67dd6adc30310871a9f4","evidence-manifest.json":"101cb2a051bb64d3fc12e3a5389c2f39be4acb85e120b9bca1557163643f32e8","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"df3b7ce86f99b1f31ce3061e637e936d40a68b63d49aa307cf4a9dcfa75b3983","snapshots/stock-harness.json":"cffaf9caec42246841497abf2059ee2d11c23a67971ad14112fa96beb23d302a","snapshots/stock-harness-preflight.json":"7da7c7be4d0c33c6d1c3fadeb500eebe3fceba797edd0f24a9db187091896b6b"}},
{"executionId":"stock-harness.legacy-acp-claude.local.ordered-comment-continuation","profile":"legacy-acp-claude","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard","structural_receipt_incomplete"],"failedMatcherPaths":["stockHarness.fresh-default-delivered"],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":169818,"providerDurationMs":150645,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":49679,"outputTokens":7557,"cachedInputTokens":465247,"totalTokens":522483,"reportedCostUsd":0.45116585000000003,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":150645,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.45116585000000003,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.45116585000000003,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":false},"invocationMetrics":[{"capturedPromptChars":16433,"reportedPromptChars":21108,"freshPromptChars":4688,"retrievalClipped":true},{"capturedPromptChars":14499,"reportedPromptChars":14021,"freshPromptChars":4688,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"9994318ef5b3cac9143215d580f7551600777864f3d54a0cb2c48941a0799523","evidence-manifest.json":"79ba61a1c2ac3adde8d1905f09a051d0b711be7c618285b728d13f3650fe9d64","snapshots/context-integrity.json":"0566e89e716c742ee4452f5d9bd719354dba968459809e3e362794f55135b705","snapshots/stock-harness-hire.json":"05919c3be635604a40c0ea9bb76ab1ce1a1ce6545417e66e02c5a26881f082ee","snapshots/stock-harness.json":"6fdbee98a58b20e131a46a5bf9220c1d9e6c4cfc02d64baeee700627869b137c","snapshots/stock-harness-preflight.json":"cb4abccc23ebbe0cd4884ca1d5238eef14b82e15987a2d9e7083ccb608504189"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.legacy-acp-codex.local.assigned-skill-explicit-invocation","profile":"legacy-acp-codex","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard"],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":59328,"providerDurationMs":35548,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":511,"outputTokens":30,"cachedInputTokens":27646,"totalTokens":28187,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":35548,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":13618,"reportedPromptChars":13642,"freshPromptChars":4687,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"743b4b283ee73f34db4434facefb26012145fa528a1912856725c4f13e5b9247","evidence-manifest.json":"fe7f46cac1d8722c6137ec813638ecd0c5944f1ea25cc24092ff8d22236c807b","snapshots/context-integrity.json":"d247901367c17626a5738743dc300566c758db50fa7c0b60e5f74be5b7083245","snapshots/stock-harness-hire.json":"15866eaa21a98137bd2c4088dd53256360cdf65b27c8475d49d98cd49788fcf6","snapshots/stock-harness.json":"639cd4b7fec29383a18016e5b8be2227f68d8a5c37035318089ace252408a231","snapshots/stock-harness-preflight.json":"f52edc51ee627e9f5f02d3049a83a9b99fc073fc504ca9f238cf95bc691a1ce8"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-acp-codex.local.continuity-restart","profile":"legacy-acp-codex","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard","structural_receipt_incomplete"],"failedMatcherPaths":["stockHarness.invocation-evidence-complete","stockHarness.fresh-default-delivered"],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":44754,"providerDurationMs":22118,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":662,"outputTokens":29,"cachedInputTokens":29955,"totalTokens":30646,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":22118,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":false,"historical-generic-procedures-observed":true,"fresh-default-delivered":false},"invocationMetrics":[{"capturedPromptChars":16383,"reportedPromptChars":17671,"freshPromptChars":1461,"retrievalClipped":true}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"3ba472d29268b3e2d2170cd9947d7c89fbe685d902fedacf5c74a1952a97ce3a","evidence-manifest.json":"a5c263cc9afe3836a3c9f46cda537bb07a374c30fd7ad6f136088680b3ade01c","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"8dd9d388cc265387d960fd435c19874601a5f7ca209b328943f8bacb145c5d89","snapshots/stock-harness.json":"f2b69dda7e26a76c952f0b36f10cfd7e9fb364ae3930585961d84093d17c2979","snapshots/stock-harness-preflight.json":"5fc3c8235143766fc6be8c7e12b31b92222594698c966462f944f13fb5929113"}},
{"executionId":"stock-harness.legacy-acp-codex.local.ordered-comment-continuation","profile":"legacy-acp-codex","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard","structural_receipt_incomplete"],"failedMatcherPaths":["contextIntegrity.ordered-comments","contextIntegrity.changed-scope-preserved","contextIntegrity.packing-report-order","contextIntegrity.final-scope-applied","contextIntegrity.continuation-run-count","contextIntegrity.continuation-wake-comment-ids","contextIntegrity.single-durable-output","contextIntegrity.completed-task","contextIntegrity.successful-runs","stockHarness.invocation-evidence-complete"],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":50896,"providerDurationMs":39443,"cleanup":"passed","billing":{"llm":{"runCount":4,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":992,"outputTokens":46,"cachedInputTokens":24349,"totalTokens":25387,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":39443,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":false,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":14032,"reportedPromptChars":14056,"freshPromptChars":4687,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"33ac59f0a690ed76618f695e658575751e718f53f9d828d8b52c690df9c15afb","evidence-manifest.json":"92c048e01012a77b94003e38f3f73888d725f6942327f7fbf3e6d2a42d12184d","snapshots/context-integrity.json":"2f9a803657068a67bcbf3b190370395187ef13b7a1487705d0305e121339eac7","snapshots/stock-harness-hire.json":"f47bb35d606ae00e6057a2a77769e2091e8ccb12eca08db3a17e1ed297a8ccf4","snapshots/stock-harness.json":"05582c37956b55f6a2bd7cad0c55bc0fa1dbff875d7dbed12b83352f6bd4e04c","snapshots/stock-harness-preflight.json":"badf01a591f5327f0dc2e48865711f8f4d7327888a7688668728173a29b273a8"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":3,"failed":9}},
{"executionId":"stock-harness.legacy-claude.local.assigned-skill-explicit-invocation","profile":"legacy-claude","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":49054,"providerDurationMs":29420,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":29883,"outputTokens":1348,"cachedInputTokens":225730,"totalTokens":256961,"reportedCostUsd":0.19999125,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":29420,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.19999125,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.19999125,"complete":true},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":7512,"reportedPromptChars":7512,"freshPromptChars":4684,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"4fc73f7b1226b9a41b209a2f6dfa2a58805ddc1a7e7b699ac2b42b2e74a189a7","evidence-manifest.json":"440ee14cca3f8558fb47ecf27b17693c0f9dbecdcf5c8532d7e2ccbefe31700b","snapshots/context-integrity.json":"4545847b9c4ab2a4fa3053a384b5e990133705fc9da0d1e9cd98f1275ec05f44","snapshots/stock-harness-hire.json":"a6abfc40d1c788e4b897f2375506d59b2caba9fab0d4df019e7137fe9f66177f","snapshots/stock-harness.json":"755cc9d5f656dc068a08c8c141203eaa5d3d60b53edd9d516887aae108119f45","snapshots/stock-harness-preflight.json":"f52730b4555ab31de952c8c1e38db4067fb40756b6befd36ed4b686e1a273d73"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-claude.local.continuity-restart","profile":"legacy-claude","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"candidate_failure","causeCodes":["chat_memory_assertion"],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":40202,"providerDurationMs":23072,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":39036,"outputTokens":829,"cachedInputTokens":116171,"totalTokens":156036,"reportedCostUsd":0.1936638,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":23072,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.1936638,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.1936638,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":11451,"reportedPromptChars":11451,"freshPromptChars":1458,"retrievalClipped":false},{"capturedPromptChars":11508,"reportedPromptChars":11508,"freshPromptChars":1458,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"f8e377f617204eabdb6b4e0c24ab3280c51664b429051f7ea21a4746790820a6","evidence-manifest.json":"45818efcb8ed6d663f7ffab6d53605d5191fb983a68e422dadebdae875ed5b4c","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"e3a8735ad4d5f89dc3899668c36791579562157fa25aa32a6e59aa098d0c079a","snapshots/stock-harness.json":"f3d24452ade2f232b0f671768d46e4d65a6a0c2a0740bb1daa17a4871a09aa7b","snapshots/stock-harness-preflight.json":"b85c96ada9801ce6c6569338b68bc810494a0711f01af39a63932e288f7a81a3"}},
{"executionId":"stock-harness.legacy-claude.local.ordered-comment-continuation","profile":"legacy-claude","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":142017,"providerDurationMs":119179,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":66601,"outputTokens":6610,"cachedInputTokens":783643,"totalTokens":856854,"reportedCostUsd":0.58397715,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":119179,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.58397715,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.58397715,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":14473,"reportedPromptChars":14447,"freshPromptChars":4684,"retrievalClipped":false},{"capturedPromptChars":7926,"reportedPromptChars":7926,"freshPromptChars":4684,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"9161c05c288e8620e5a748a28c901e1582efc240ca640203e1d8f7527e22a651","evidence-manifest.json":"296246a83eb0870a53e70f359f1859719f30b655e6ed20908abbfe4213605280","snapshots/context-integrity.json":"17779f4ebc38ed08071e40ac4b939149a18dfceb76c0f1f4f6ed21bf5d0153fa","snapshots/stock-harness-hire.json":"1f5a97e38555f57f47ee1900440d88ee521de75e7589c163f7688c852b661dc8","snapshots/stock-harness.json":"499a0b49c042b7c44f992c56983d207677cf619306ed6f6db9975ef1b84d455b","snapshots/stock-harness-preflight.json":"8f92c4b703533a645196bf03d1353be7196271d25638be7cb3fae6ec92c69c52"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.legacy-codex.local.assigned-skill-explicit-invocation","profile":"legacy-codex","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":50078,"providerDurationMs":27658,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":128946,"outputTokens":1727,"cachedInputTokens":97854,"totalTokens":228527,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":27658,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":12332,"reportedPromptChars":12332,"freshPromptChars":4683,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"ea06fc8b1ec166c9932de8634d08e528e90f7b734388ccad6d47b4b96df9ff6d","evidence-manifest.json":"45e71226e99eaee9bea0b0fae2cca35de573c2ab9d5add13159f321c8480ac0b","snapshots/context-integrity.json":"4cf10f812f258a77b472abd74f6691224dfcba4b9a2dec9c0f2c35c5ae741d7a","snapshots/stock-harness-hire.json":"19db78dbf51a67ff4bc307ef378da9cc024a0cae1c977951b49d089c1ec47fe4","snapshots/stock-harness.json":"a3cb7e2bf4e775cf9058b6be33d8df5321e312df2aed39736719454647610a8f","snapshots/stock-harness-preflight.json":"e9a116f8605a2630ff46075598ac6bae30817892d76efcd9a25891649cac0856"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-codex.local.continuity-restart","profile":"legacy-codex","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":85550,"providerDurationMs":29332,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":0,"inputTokens":468597,"outputTokens":2919,"cachedInputTokens":373648,"totalTokens":845164,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":29332,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":2787,"reportedPromptChars":2787,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":2730,"reportedPromptChars":2730,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":16326,"reportedPromptChars":16326,"freshPromptChars":1457,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"db7be8127de96e697a9d274b7a18db40695daa234684ec4d8e28afd8665c1f3e","evidence-manifest.json":"2e9151536b8ade2bfbb3c465438e8ace21417258c4037b6923920ee0784b0e47","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"af8b2fe11402a8b87a056818df29a62829ec9425f8a2d955e95692980e15e043","snapshots/stock-harness.json":"739e642582faf89b74c522774c64ea26c6af623539b7a8f99ce332066c965600","snapshots/stock-harness-preflight.json":"8fe9f32d7629a1f6bd632d58e22e39e47baff2b8bc3b74e3593e6411ab3c9a2b"}},
{"executionId":"stock-harness.legacy-codex.local.ordered-comment-continuation","profile":"legacy-codex","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":74114,"providerDurationMs":57150,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":0,"inputTokens":628626,"outputTokens":7098,"cachedInputTokens":563211,"totalTokens":1198935,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":57150,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":9811,"reportedPromptChars":9785,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":12746,"reportedPromptChars":12746,"freshPromptChars":4683,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"56a457fcc7a36809eb4bfcedeff2fa079f5272754ff52fc453e8bf8d813e52f3","evidence-manifest.json":"bf17fb7d2a05f7e99e0723907bdc57f0a88bf6838d68f31f4b84dc6ebc62d3c7","snapshots/context-integrity.json":"a9156b810e32fa1d9d07826276a00c62ebe4f25ac3dc284ada157f2b2d72d800","snapshots/stock-harness-hire.json":"5fc8abd51d401c96d701b0862ac2dc674eb36244fdc25c49b67ba3947ba5a2fb","snapshots/stock-harness.json":"c72b8c53e1e50a82ab6ff815b363942936440cfff75f8f8ca5f54185c723889d","snapshots/stock-harness-preflight.json":"b14d304196cdc3146436c66199243c6da04784c13e45c9bd31e0ef83e97a29bc"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.legacy-opencode.local.assigned-skill-explicit-invocation","profile":"legacy-opencode","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"legacy","durationMs":86384,"providerDurationMs":68956,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":57002,"outputTokens":2365,"cachedInputTokens":223232,"totalTokens":282599,"reportedCostUsd":0.0044563934000000005,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":68956,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0044563934000000005,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0044563934000000005,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":12335,"reportedPromptChars":12335,"freshPromptChars":4686,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"20dbb34bb975f73c3ac2695c047b6cbc8f6382757939ae6210b08a0f13ba9e9d","evidence-manifest.json":"67c3e042be8e0a5a0cd7cca82d1b0887e1392a2806c3b4db67baa3e99365b6be","snapshots/context-integrity.json":"decf2d93769acce2e2990de27c81ee32fd805ca53d3c2be68c56f22a013a7ac0","snapshots/stock-harness-hire.json":"187598ed72837719be8d6f0b77c5f8d35cfc871f8cf41d4fdc5b259f88171025","snapshots/stock-harness.json":"98424edd880cd05e0cbf8c32973fe9fbfc055037a74e76c20f893c3ee263f6a0","snapshots/stock-harness-preflight.json":"86d9f4080a2045ae3fa68468d0ffaf68baa26f27f0528274f13762f093b4c113"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-opencode.local.continuity-restart","profile":"legacy-opencode","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"legacy","durationMs":126589,"providerDurationMs":78910,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":64286,"outputTokens":1046,"cachedInputTokens":28672,"totalTokens":94004,"reportedCostUsd":0.0022020907,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":78910,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0022020907,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0022020907,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":7611,"reportedPromptChars":7611,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":7554,"reportedPromptChars":7554,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":16335,"reportedPromptChars":16335,"freshPromptChars":1460,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"29dce6418aa35bb8070b07433046bbf39c9a8367a198cdb0aa28a0bea480032c","evidence-manifest.json":"439dc46130b607b7488e9df7736a711b267efa53689fe9fdad2f822525677215","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"7f7ad64827bbe430e06a7e01b1377443653793584264c0fa00ac88a742bd0f49","snapshots/stock-harness.json":"4dd536951a950e7507710a9eb44cdbcd354907397f1e95b8297b28a98fd75dbc","snapshots/stock-harness-preflight.json":"8fa2ec342ad583fb20fed4db88890fc31eab3ef5898a952a85371e53dd433aaf"}},
{"executionId":"stock-harness.legacy-opencode.local.ordered-comment-continuation","profile":"legacy-opencode","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37048838402","cellAttempt":1,"status":"failed","failureClass":"candidate_failure","causeCodes":["deadline_or_run_start_timeout"],"failedMatcherPaths":["contextIntegrity.comments-queued-as-batch","contextIntegrity.initial-run-started","contextIntegrity.ordered-comments","contextIntegrity.changed-scope-preserved","contextIntegrity.packing-report-order","contextIntegrity.final-scope-applied","contextIntegrity.continuation-run-count","contextIntegrity.continuation-wake-comment-ids","contextIntegrity.single-durable-output","contextIntegrity.completed-task","contextIntegrity.successful-runs"],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"legacy","durationMs":720616,"providerDurationMs":442902,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":51446,"outputTokens":8911,"cachedInputTokens":967936,"totalTokens":1028293,"reportedCostUsd":0.030657645200000007,"costStatus":"partial"},"runtime":{"provider":"local","agentRunDurationMs":442902,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.030657645200000007,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.030657645200000007,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"historical-generic-procedures-observed":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":16392,"reportedPromptChars":16461,"freshPromptChars":0,"retrievalClipped":true},{"capturedPromptChars":16406,"reportedPromptChars":16879,"freshPromptChars":4686,"retrievalClipped":true},{"capturedPromptChars":12749,"reportedPromptChars":12749,"freshPromptChars":4686,"retrievalClipped":false}],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"6d0e5d54b1742b02eb37346747510c66cb5cd0b80262a6c9590fd2a28861dcb2","evidence-manifest.json":"758931a4d30213d1b320f58cd2aa338e8a6a8aed579b878e14c5e52848bd5f13","snapshots/context-integrity.json":"bcaa4ee278361a90347625044aa49086b27093356abc0f3700b0363d28c8653c","snapshots/stock-harness-hire.json":"21431fa5e7cb6486037c670fc1e4aa64fb24716ad8ab325601ff294ada40263f","snapshots/stock-harness.json":"29c55e52f4adbbc9259cb9e221c08bfbbd7ad4c5f8247c24b3f029362e9000de","snapshots/stock-harness-preflight.json":"f341723074c996c79d8a6c30b9f12270634b20b40da452b13e4ab9ce91b47cac"},"taskDocumentCount":null,"behaviorCheckCounts":{"passed":1,"failed":11}},
{"executionId":"stock-harness.runner-acpx-claude.local.assigned-skill-explicit-invocation","profile":"runner-acpx-claude","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-5","runtimeMode":"native","durationMs":46264,"providerDurationMs":25372,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":10,"outputTokens":1042,"cachedInputTokens":141254,"totalTokens":142306,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":25372,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"073085f804a58d191bd44972edfb08d914fe622a86f236d0c4dc1b2bf8a03322","evidence-manifest.json":"4fad428c1df72abbfee24a4f7338e25284fc874cff1220e6b631fa86eb2d0bed","snapshots/context-integrity.json":"77e27b8ea01a83e0b65f3d5410e0787fbf84a7f6ea02f4401bd77488999dd4a2","snapshots/stock-harness-hire.json":"8c6c5d56fd068c15f1e19ce9a0b62204443c894d204311bd949d5e22c624fd3d","snapshots/stock-harness.json":"2a6860d7c29ad0d03240bda0a779b83d0455b8ce9fcfb1cf9e6ad559c4dec112","snapshots/stock-harness-preflight.json":"79b11e544d2821d1025476f5fd0d737e822ae5ab731ab32047d1fbf6b76015e4"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.runner-acpx-claude.local.continuity-restart","profile":"runner-acpx-claude","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-5","runtimeMode":"native","durationMs":77424,"providerDurationMs":36820,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":6,"outputTokens":94,"cachedInputTokens":94997,"totalTokens":95097,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":36820,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"be582fe8af5f19543c16c8590697c5aa9dc688fdb653b0550aba0b55174becd1","evidence-manifest.json":"cfbd1cb13917a96b08dd46aae1fcc63e2b27af5d1d22f4952974d3415c74ffb8","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"254f18a8e8e1d6a5a2e7731728756671c1374cd4d1210469172bed9c2cd6e4c2","snapshots/stock-harness.json":"8efa84d6907de220dd7b9fc2c384c61aae0606e7f5311f0e926e26ae1617d869","snapshots/stock-harness-preflight.json":"c7e3f2342ef9fc824b28e3f22ffbbb326d2ba1ff84e36c4deedc56a7abefbe07"}},
{"executionId":"stock-harness.runner-acpx-claude.local.ordered-comment-continuation","profile":"runner-acpx-claude","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-5","runtimeMode":"native","durationMs":132587,"providerDurationMs":108895,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":26,"outputTokens":7996,"cachedInputTokens":565030,"totalTokens":573052,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":108895,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"62c6c55047813b0b5aa5334e29dbe3f90225c83ac436eab14afd97d78ed9ec65","evidence-manifest.json":"7d53ad7f80b65a1c7af3fad4fa3d0c652ae25cdaeae3d41cbee1bb888b853fb5","snapshots/context-integrity.json":"2081459b761a84469399bec377d7380c172b64fddae48735c5329771656e28a7","snapshots/stock-harness-hire.json":"f11c90fdae215949458d61a6e1ca6565fe277ea30d52fd922c05cffdf4867e5c","snapshots/stock-harness.json":"f303a3fb5c5e55cefb696d46204aafe6f1cefc5b0a6b07a728f2bf0dbf1dfdcf","snapshots/stock-harness-preflight.json":"79768aa58996d28ee80c8a9e8d5c4ba1a42eba58a408163cd8a6037ac6005b26"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.runner-codex.local.assigned-skill-explicit-invocation","profile":"runner-codex","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"native","durationMs":53751,"providerDurationMs":29850,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":162041,"outputTokens":1296,"cachedInputTokens":137822,"totalTokens":301159,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":29850,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"ffcc5203624c19b0acda50dfd1e283c595ae3ed6072e4aaf9781fef4bf08fa45","evidence-manifest.json":"1460fae9f00a241a3ddbe7f623a4c1c6cf0f6434e73e47d0de7f98dc00dec459","snapshots/context-integrity.json":"63d16c6558efcf03097fd2ef8caf1f01c39a38127aee2904eba0cf0edfb711e6","snapshots/stock-harness-hire.json":"61630a0e0521f8ccca30753bde385b731f5c8c15cf6280b000947e80acf3decf","snapshots/stock-harness.json":"bc95a45aab085ce3eee988a9b235281b847cd8cb73c4d294ce576b97e73d879f","snapshots/stock-harness-preflight.json":"4daf4cd83a8b1539ab8a8db2fd5cf204746884d8cde04f6507acfff5be75ef11"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.runner-codex.local.continuity-restart","profile":"runner-codex","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"native","durationMs":84362,"providerDurationMs":35132,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":315100,"outputTokens":2339,"cachedInputTokens":231408,"totalTokens":548847,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":35132,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"dbf2e14fcda52d2a33808e5a4c226d45f864752274c823c26beb7c2ea5409f03","evidence-manifest.json":"86a34f1a25854354bee0d9f0ea88cab9ba9b51476cf274d12f5b491a9910188a","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"a38cab643c78fde9d61d15074b82297b94bf46ae5c3296aa06e6958f67188eda","snapshots/stock-harness.json":"2d148d3e55c57619bde56558f7e87e85cbb1305e46eba83bedb21c3ee4b43b54","snapshots/stock-harness-preflight.json":"9517ef436cc83da9078dd75587f01be8a28685131c37773dc092cf89c137ca0c"}},
{"executionId":"stock-harness.runner-codex.local.ordered-comment-continuation","profile":"runner-codex","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"native","durationMs":92663,"providerDurationMs":70832,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":428674,"outputTokens":3591,"cachedInputTokens":372482,"totalTokens":804747,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":70832,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"8c7a81172e0d519ffc8aca9a3025c88166d93d12a1744e560813c66063d6d130","evidence-manifest.json":"b495ad141c12f2458e3b66123cb673dec68511bcb5c2d59af78400dc67c417eb","snapshots/context-integrity.json":"166eb5a26fa389529287aa940c4e3caa85665ea67fcce019bc69a8bb8d9f7b41","snapshots/stock-harness-hire.json":"2fe4b0d207648628658312e97b491336929197f826c00dce012b6afc43537564","snapshots/stock-harness.json":"79b83fed61c69589d38b48552b04dce41bf41dca27f6e7f5dd15229024b6338f","snapshots/stock-harness-preflight.json":"689ce3141b2c6970eac5468885e5111daa730bcda25a1459e7bac4177650045e"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.runner-opencode.local.assigned-skill-explicit-invocation","profile":"runner-opencode","case":"assigned-skill-explicit-invocation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"native","durationMs":90297,"providerDurationMs":68028,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":44259,"outputTokens":1165,"cachedInputTokens":62976,"totalTokens":108400,"reportedCostUsd":0.0020380985,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":68028,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0020380985,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0020380985,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"3ffa3bff50fbade8801da7c26ffed426eee9bda9a7e3640793f03a220f74ea82","evidence-manifest.json":"6df0c3b49ef54154721d4d0ba434f893fcba372ba8588fef2bd1a5947284b1d3","snapshots/context-integrity.json":"41b60c88adb0c529148981a55ab3b18964035b47786da50942183b58e8884fed","snapshots/stock-harness-hire.json":"c9a2fe249e217fcd5c205f1dc768446333932de74d87dc6f4a1e7855c5e4d815","snapshots/stock-harness.json":"474b0d6aecd8e8bce23f78c260ad4dc83c286d1d39917ea0920ad4b402d9404b","snapshots/stock-harness-preflight.json":"baeb38686ea4d3233906d83a889fdf3d805846b27f25c6c04da68842822a496d"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.runner-opencode.local.continuity-restart","profile":"runner-opencode","case":"continuity-restart","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"native","durationMs":155833,"providerDurationMs":101201,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":48028,"outputTokens":96,"cachedInputTokens":23808,"totalTokens":71932,"reportedCostUsd":0.0004892436,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":101201,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0004892436,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0004892436,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"57fe32615f16ff3706044419bb81362a67ec35aed7a5f74a5b8207b5831a5cba","evidence-manifest.json":"07e3be863cc20a1352a094f38e6133004dbfd673f4fd59599431a022ce061ff1","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"d127504476aee2130312fdf092a523060188066abe9f969ae9b178b57374cced","snapshots/stock-harness.json":"0df941763ed917e2c553a6c6a4dfb0c87bdb7181d42a28f614c765906b74f1e9","snapshots/stock-harness-preflight.json":"f57655c970472d2a9b9bebe100d8c31788ed76e06da4363046a5a6d3f09269bb"}},
{"executionId":"stock-harness.runner-opencode.local.ordered-comment-continuation","profile":"runner-opencode","case":"ordered-comment-continuation","sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceRef":"refs/heads/codex/stock-harness-previous-instructions","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042864888","cellAttempt":1,"status":"failed","failureClass":"candidate_failure","causeCodes":["deadline_or_run_start_timeout"],"failedMatcherPaths":["contextIntegrity.comments-queued-as-batch","contextIntegrity.initial-run-started","contextIntegrity.ordered-comments","contextIntegrity.changed-scope-preserved","contextIntegrity.packing-report-order","contextIntegrity.final-scope-applied","contextIntegrity.continuation-run-count","contextIntegrity.continuation-wake-comment-ids","contextIntegrity.single-durable-output","contextIntegrity.completed-task","contextIntegrity.successful-runs"],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"native","durationMs":721578,"providerDurationMs":446042,"cleanup":"passed","billing":{"llm":{"runCount":11,"runsWithTokenUsage":11,"runsWithReportedCost":11,"inputTokens":159917,"outputTokens":7291,"cachedInputTokens":1077987,"totalTokens":1245195,"reportedCostUsd":0.015645790399999998,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":446042,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.015645790399999998,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.015645790399999998,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"12c5433c67dc2e62916b879349c7ba2b6e0431f0","sourceFingerprint":"2e5f22848450179ac2d573662c718300f86a935d09d15204c9d97e98f65b53f4","passed":true,"providerCallsBeforeAdmission":0,"passedTests":536},"evidenceHashes":{"result.json":"f727dd5f76162c4b995e649e1a7bf55a52381efa550e3bda46ec0a51a899d178","evidence-manifest.json":"6a550c598b67f60fcea70d56907c4a2c84239bc0d01e58737e99a0014144bb70","snapshots/context-integrity.json":"14212826847fbd9d6d63c7b70afb102fdb5d0100f7d8f1368d46ff6f5e32db07","snapshots/stock-harness-hire.json":"ab6dd2377fc251e44ed8f3680d40d10a315b635f3003e45501c7496037fb21ca","snapshots/stock-harness.json":"38759bca194a8038f0d2be4598e50a04d61215ecabdfec95cd7f6cfa4eeec61c","snapshots/stock-harness-preflight.json":"312611057436fd8cbdcc5179ffa5e62144800a6a9244e646bb6e0b3a9a0cb551"},"taskDocumentCount":null,"behaviorCheckCounts":{"passed":1,"failed":11}}
],
"candidate": [
{"executionId":"stock-harness.legacy-acp-claude.local.assigned-skill-explicit-invocation","profile":"legacy-acp-claude","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard","no_single_paperclip_document"],"failedMatcherPaths":["contextIntegrity.single-durable-output"],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":123175,"providerDurationMs":111928,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":25333,"outputTokens":5771,"cachedInputTokens":316215,"totalTokens":347319,"reportedCostUsd":0.2838505,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":111928,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.2838505,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.2838505,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":10966,"reportedPromptChars":10449,"freshPromptChars":787,"retrievalClipped":false},{"capturedPromptChars":6028,"reportedPromptChars":5498,"freshPromptChars":787,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"2a5fed84ee0f95a67e2c2b0d4247ffae33e16a7c2aa11dc57b11228e0db90965","evidence-manifest.json":"7e31c707e866e15bf44a276ac252c423a47a63364e76ad85de9a8d5b6a4e1dfd","snapshots/context-integrity.json":"3c36c18d5e257e788514c02b50d63c7c69885201df0a07750129d55c0f583676","snapshots/stock-harness-hire.json":"2d128826b4e5068ad6ed819512bcb17160346becd9344b8ab8e70c984c3241b7","snapshots/stock-harness.json":"49b8da640ab0b9dd2b209ce0cf9f3bec6deceb801bfe98c0166d7702a9732a80","snapshots/stock-harness-preflight.json":"a12fdf9b8b7d761ff223e30dcaeb835ad9bf32ef7f1c8b8983a638d175c2ab5f"},"taskDocumentCount":0,"behaviorCheckCounts":{"passed":6,"failed":1}},
{"executionId":"stock-harness.legacy-acp-claude.local.continuity-restart","profile":"legacy-acp-claude","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard"],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":34813,"providerDurationMs":18798,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":16797,"outputTokens":329,"cachedInputTokens":54375,"totalTokens":71501,"reportedCostUsd":0.092943,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":18798,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.092943,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.092943,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":13201,"reportedPromptChars":12723,"freshPromptChars":787,"retrievalClipped":false},{"capturedPromptChars":13233,"reportedPromptChars":12755,"freshPromptChars":787,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"ecab7c9f83668fd4712702bb906adf38e188e34f7f07bd7d746da685339a2ecd","evidence-manifest.json":"544a726ddf3ac212671f7c3d73dafcf94cd5b3abb9a3ad5197d6c793e9d7aaa0","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"78f5c467789921db4d2194ba80459524b04644f2d8d8a3b8678fdf7686004e8f","snapshots/stock-harness.json":"61641e5e25351e748afb2e4533d70b1cc34a8b78e1fbc56a27d96b2017bd0f45","snapshots/stock-harness-preflight.json":"4e522117d8489a16263c3d66ca513207217a7861733bfd11ef2f7149aef680e9"}},
{"executionId":"stock-harness.legacy-acp-claude.local.ordered-comment-continuation","profile":"legacy-acp-claude","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard"],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":154765,"providerDurationMs":139409,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":61445,"outputTokens":6979,"cachedInputTokens":562903,"totalTokens":631327,"reportedCostUsd":0.5120724,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":139409,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.5120724,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.5120724,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":13419,"reportedPromptChars":12915,"freshPromptChars":787,"retrievalClipped":false},{"capturedPromptChars":6390,"reportedPromptChars":5912,"freshPromptChars":787,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"9c23fec76e718c1fa5f775ae8cdcee1e5433039ff74d7ffbf306b5be6693700d","evidence-manifest.json":"5284aba88b923322b1f162045d20e36236fb731689527a1a1d06d5305efd4a64","snapshots/context-integrity.json":"8fef43dac791a84db62bfea8977f586afe3e5dc5c36225eaab79faaa15b4dfca","snapshots/stock-harness-hire.json":"4961a6103fcb6b58d5d65981c68afa178ea9e1eb6038ad3b3049d47e4bc737ab","snapshots/stock-harness.json":"4e11dd1b37581fed8628b7ebd953ebf3b6fe389d357ffd2efcd9e6b976e4d64a","snapshots/stock-harness-preflight.json":"7d962ba46f4f02f111ed4e423705009f38ee99096d180b693f4c41ed4c805126"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.legacy-acp-codex.local.assigned-skill-explicit-invocation","profile":"legacy-acp-codex","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard"],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":65745,"providerDurationMs":49358,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":1379,"outputTokens":34,"cachedInputTokens":41987,"totalTokens":43400,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":49358,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":5509,"reportedPromptChars":5533,"freshPromptChars":786,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"81c5c35a942c78a2c9d7d77064264504311a191c6dd421e656d4f0751670b6ff","evidence-manifest.json":"bff59a88d627186d18282fe39822dd5a57d3a2396ae9aca5fa178a3b4ac3cf60","snapshots/context-integrity.json":"3809130fd34f0b6035ed5b17bd9e354d19aac02fc2f64aea01811803151c7f82","snapshots/stock-harness-hire.json":"4e0468ee05d56f08faa9ffbe3ed6cf28c07490a8b27a08e80875264dc66daac5","snapshots/stock-harness.json":"c5a82192ff53c6eec7a5497602a9969af3c21ea7fde8fdff4309025f5ccf80d2","snapshots/stock-harness-preflight.json":"12944d93aac9f8688d2832369f7eadc10a9546a6fb6bc2f2d8b3caef38b0cc2f"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-acp-codex.local.continuity-restart","profile":"legacy-acp-codex","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard","structural_receipt_incomplete"],"failedMatcherPaths":["stockHarness.invocation-evidence-complete"],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":35779,"providerDurationMs":20059,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":711,"outputTokens":30,"cachedInputTokens":31118,"totalTokens":31859,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":20059,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":false,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":12764,"reportedPromptChars":12788,"freshPromptChars":786,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"1c59fb6b3845ef6bc4f9cc549826e45334b3220ce5832f2fe4b9d23fd45136b6","evidence-manifest.json":"4cdd7cc5bb0a025e7626e947dd72558c13203aa9dc9765f5afbfed5acc907de1","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"dcbe621edcd9437c47361c74676db38a240fdcaff2401d41213a185e7cf9b785","snapshots/stock-harness.json":"8908438e0c7565ebca6d4753e619f87bd0c015210214fbb8c39876d7e13cebaa","snapshots/stock-harness-preflight.json":"3b2de5b161384bebe03f7cad3b3268f98cca291cd8fbf4b8a0c830536daecf87"}},
{"executionId":"stock-harness.legacy-acp-codex.local.ordered-comment-continuation","profile":"legacy-acp-codex","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"failed","failureClass":"secret_leak","causeCodes":["persisted_credential_guard"],"failedMatcherPaths":["contextIntegrity.comments-queued-as-batch","contextIntegrity.initial-run-started","contextIntegrity.ordered-comments","contextIntegrity.changed-scope-preserved","contextIntegrity.packing-report-order","contextIntegrity.final-scope-applied","contextIntegrity.continuation-run-count","contextIntegrity.continuation-wake-comment-ids","contextIntegrity.single-durable-output","contextIntegrity.completed-task","contextIntegrity.successful-runs"],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":720307,"providerDurationMs":46665,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":540,"outputTokens":28,"cachedInputTokens":32395,"totalTokens":32963,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":46665,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":5923,"reportedPromptChars":5947,"freshPromptChars":786,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"a1285f2fecccdcdac85360784e4be82188b564d3135f7adf6606fa1451307376","evidence-manifest.json":"39864272e9331c88c1266f2b5f05b755aacdc46c2c7015c4d4df8539c0a22718","snapshots/context-integrity.json":"5e663656093c672fb294719edb8636dd15a5ed1441912f32a32c78254663b797","snapshots/stock-harness-hire.json":"35d859103bf2acb7c7f6d212a8ce277e42726da388f7c52ee0b05d600dc09c78","snapshots/stock-harness.json":"483b4efedcc34570ad0ad364d92ba0212186cbc36a704639106e181402470468","snapshots/stock-harness-preflight.json":"5bb04e0c93a3f4ac5d9f41a5775fba85e0687a8161e255257545d54151421c62"},"taskDocumentCount":null,"behaviorCheckCounts":{"passed":1,"failed":11}},
{"executionId":"stock-harness.legacy-claude.local.assigned-skill-explicit-invocation","profile":"legacy-claude","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"candidate_failure","causeCodes":["no_single_paperclip_document"],"failedMatcherPaths":["contextIntegrity.single-durable-output"],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":37720,"providerDurationMs":25763,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":27376,"outputTokens":1281,"cachedInputTokens":184039,"totalTokens":212696,"reportedCostUsd":0.17707545,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":25763,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.17707545,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.17707545,"complete":true},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":3611,"reportedPromptChars":3611,"freshPromptChars":783,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"8636a50a54d39fd66b1439f13ce5ac125b79f16b2826f76fc297427a1a2ab701","evidence-manifest.json":"d1d7f47272981406d104f5ae6455d5b0a4f6113f3c3d02fc8bc787d7c8fc70fc","snapshots/context-integrity.json":"eac7d7b889e5cc3861b937ffa87c7b92d25ade5b0f2bb73bdd29b5764c2a085a","snapshots/stock-harness-hire.json":"545030ff52d9196a604e099b81b52a3071f582f2467284c695eb62d96589c67c","snapshots/stock-harness.json":"c9ca463a792c7dccd00c0795c1e405495d9e1dff30b2cbae2b5e7bac7114fdd2","snapshots/stock-harness-preflight.json":"c1b5a843459d80e074759600c830c92bb00a0d2430255cda7bed5c22cabc7680"},"taskDocumentCount":0,"behaviorCheckCounts":{"passed":6,"failed":1}},
{"executionId":"stock-harness.legacy-claude.local.continuity-restart","profile":"legacy-claude","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"failed","failureClass":"candidate_failure","causeCodes":["chat_memory_assertion"],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":28251,"providerDurationMs":11843,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":19490,"outputTokens":298,"cachedInputTokens":50061,"totalTokens":69849,"reportedCostUsd":0.09257055,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":11843,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.09257055,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.09257055,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":10776,"reportedPromptChars":10776,"freshPromptChars":783,"retrievalClipped":false},{"capturedPromptChars":10833,"reportedPromptChars":10833,"freshPromptChars":783,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"86af4c61a239c718181ab58fd815cf7a200e7473b2fe9390b58bc15ff9d6ed4c","evidence-manifest.json":"e1c69d7c6044b0e8ba65fa1b961de84989facb3fd65f6bba42632b22d2a1bab9","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"9a320857e3d1fe14a41b0154b8c1d2cfc81f869ed69790bd729fe7c86e9c54fb","snapshots/stock-harness.json":"9a2d546c992bf9d59711e37e81d841b7b1c73e6e1d62a6f27b5302f1d8ea3980","snapshots/stock-harness-preflight.json":"13318522befc6c0ffcc0d6ca7a0dc12112902427ddc1a7ac36f3ac43e7f83d8d"}},
{"executionId":"stock-harness.legacy-claude.local.ordered-comment-continuation","profile":"legacy-claude","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-4-6","runtimeMode":"legacy","durationMs":158440,"providerDurationMs":139386,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":64318,"outputTokens":7515,"cachedInputTokens":669889,"totalTokens":741722,"reportedCostUsd":0.5548662,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":139386,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.5548662,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.5548662,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":11079,"reportedPromptChars":11053,"freshPromptChars":783,"retrievalClipped":false},{"capturedPromptChars":4025,"reportedPromptChars":4025,"freshPromptChars":783,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"59885705b15fc33517ab7f7a82d9ada2e411c918b10fa5b4ce928498a801e739","evidence-manifest.json":"c2a5be637da0d7ff9436b732d8db72ab1fdce788caf6a40c5973c14a27bcc08a","snapshots/context-integrity.json":"fbee7c1a37b964bcc327ebbf37602bc227bccb7fafdc4296614e178e66016599","snapshots/stock-harness-hire.json":"cf2f8ff4b1eb13c5e5f854ecac27c25826e7a4dbafbc277e8702b8beee6d913f","snapshots/stock-harness.json":"3e8e7c06f0918a896341a38e4613cc1f4caaad4b5ab12c03a8c3e41c4707a82a","snapshots/stock-harness-preflight.json":"6d6716a08fc2ea7d27ee08840fd3290a04a8e85dfed5950b690f4c2267d20a6e"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.legacy-codex.local.assigned-skill-explicit-invocation","profile":"legacy-codex","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":44075,"providerDurationMs":27042,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":0,"inputTokens":241944,"outputTokens":2132,"cachedInputTokens":215273,"totalTokens":459349,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":27042,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["metered_api"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":4223,"reportedPromptChars":4223,"freshPromptChars":782,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"b0b26f37d2f0c0342a9a275b22716be190891d1b55e79fa8d33e3f495649a26b","evidence-manifest.json":"446abc91b7366507c4100741d71ace903de0b78149b33e45fa90d39183f71b75","snapshots/context-integrity.json":"85197eb92d34334cc2de658d09d7a7e54d44ab3d2fd81ad87732c88c6610b6b2","snapshots/stock-harness-hire.json":"568453725988fc61d459f96efda07d19ec5c00935d4c4a9ff9557cd0062aef29","snapshots/stock-harness.json":"0c8626740b520687d08b0b1f0aab8a9ead2ebd58290252f05d890f524636f461","snapshots/stock-harness-preflight.json":"9a15351a65076deeb77ccd58216def768a658fc743fd57dc89c3234ff6b36542"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.legacy-codex.local.continuity-restart","profile":"legacy-codex","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":62742,"providerDurationMs":10028,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":0,"inputTokens":91929,"outputTokens":475,"cachedInputTokens":45089,"totalTokens":137493,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":10028,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":2787,"reportedPromptChars":2787,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":2730,"reportedPromptChars":2730,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":11443,"reportedPromptChars":11443,"freshPromptChars":782,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"1ec35f3857350d86d3888c4becf8ce898a59641f3eb9825a0d800970ec7090ad","evidence-manifest.json":"908761dcb779dcd0b95ef78b5e9addb77b211acbc502a1c8fdf793202d309472","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"b7b6f4db7cdc84a795ccfa24df58dd8131fe8a44b3ef1cdaf36dfc6685556da7","snapshots/stock-harness.json":"9a0045bbf4240ddbf08ec0444e08ef70e3ef76399db09075b5316b1fc51ac5bf","snapshots/stock-harness-preflight.json":"cfbae9842e2a045b815a9a7f1fa471f8f9494ec999c91e9808933cbc013c84a4"}},
{"executionId":"stock-harness.legacy-codex.local.ordered-comment-continuation","profile":"legacy-codex","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/reduce-default-agent-manual","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37042856368","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"legacy","durationMs":66403,"providerDurationMs":44175,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":0,"inputTokens":416426,"outputTokens":4483,"cachedInputTokens":353673,"totalTokens":774582,"reportedCostUsd":0,"costStatus":"unpriced"},"runtime":{"provider":"local","agentRunDurationMs":44175,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":false},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":8456,"reportedPromptChars":8430,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":4637,"reportedPromptChars":4637,"freshPromptChars":782,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"71f99ec1b45a57ae40acbd42c55e835f63d9d2329752f6bffcabd4488ecb2a96","evidence-manifest.json":"0c3347aaec6d7c3498e62c19938762c35a6069d31385e5d617174695ab60462e","snapshots/context-integrity.json":"5c08b2af1b27e9255505e917958e7b97856cfbf48cce3bef3b8e792275823b6e","snapshots/stock-harness-hire.json":"ddc6b182c4d53c7cd7f63c906d2a64e6286c54c8b6e9991e0c78da578586d865","snapshots/stock-harness.json":"11a737f54107eb9a9e55234890bce90003f84b258f6d5c8512d65f080edfc52f","snapshots/stock-harness-preflight.json":"d72a98df5a4e00bfa766e85ef28b866d32b89fe2aa9a67e3000ad50ddea47599"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.legacy-opencode.local.assigned-skill-explicit-invocation","profile":"legacy-opencode","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"failed","failureClass":"candidate_failure","causeCodes":["no_single_paperclip_document"],"failedMatcherPaths":["contextIntegrity.single-durable-output"],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"legacy","durationMs":212558,"providerDurationMs":199337,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":96260,"outputTokens":6708,"cachedInputTokens":172288,"totalTokens":275256,"reportedCostUsd":0.010608548800000001,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":199337,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.010608548800000001,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.010608548800000001,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":8426,"reportedPromptChars":8439,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":4226,"reportedPromptChars":4226,"freshPromptChars":785,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"435f708b59438d1d344130bc9e23b8ddbc93adf74d4e5ea8f049e0f1d06efa72","evidence-manifest.json":"8591f98df496159f99b98659ffdcbdf2e86718dccb4545d461f592bab0626b05","snapshots/context-integrity.json":"e4271e26bb891c3c15f2d54dd5cfba6605ebb4431456d70b4f146bb8eb4e1788","snapshots/stock-harness-hire.json":"6bb95cb92fbb14976b5f6796f530823f6aacdcb1448f89cb8b2ec093125266e2","snapshots/stock-harness.json":"b8a5a04730324a038869e28f79d55dd06ebf12abc2e6366bbfe9218995a6748b","snapshots/stock-harness-preflight.json":"c17087412694993475276a69162a171859dc54afae5b08ab89573cf0637d6e19"},"taskDocumentCount":0,"behaviorCheckCounts":{"passed":6,"failed":1}},
{"executionId":"stock-harness.legacy-opencode.local.continuity-restart","profile":"legacy-opencode","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"legacy","durationMs":69761,"providerDurationMs":19747,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":12219,"outputTokens":116,"cachedInputTokens":0,"totalTokens":12335,"reportedCostUsd":0.0003185752,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":19747,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0003185752,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0003185752,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":3403,"reportedPromptChars":3403,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":3346,"reportedPromptChars":3346,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":11452,"reportedPromptChars":11452,"freshPromptChars":785,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"0e56bb0c46a05efeeffa6feb09f54fe13a33c2c96ab4e7a7ec548a0d80a1a235","evidence-manifest.json":"0e601f9bde1bdb7f44f000704078343a1478046c9eb488f2c8a785cd1d4cebec","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"ab7d6af088306e1d43d4901a370dffa46b16dd4c17c2838a8e9e48a2b8e7a62f","snapshots/stock-harness.json":"2a6eabd1657c89568f890de12840747b544c7385c4363b3016879b07028961fa","snapshots/stock-harness-preflight.json":"539f106844e9421c284944c0b1526d74b0cc8e0905c5960be14cfa77ee17a023"}},
{"executionId":"stock-harness.legacy-opencode.local.ordered-comment-continuation","profile":"legacy-opencode","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"legacy","durationMs":212184,"providerDurationMs":188508,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":177025,"outputTokens":7495,"cachedInputTokens":256512,"totalTokens":441032,"reportedCostUsd":0.011804638700000002,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":188508,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.011804638700000002,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.011804638700000002,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true,"invocation-evidence-complete":true,"generic-procedures-absent":true,"fresh-default-delivered":true},"invocationMetrics":[{"capturedPromptChars":10775,"reportedPromptChars":10736,"freshPromptChars":0,"retrievalClipped":false},{"capturedPromptChars":4640,"reportedPromptChars":4640,"freshPromptChars":785,"retrievalClipped":false}],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"8ca5a9a446dda03100d8ee66c5b9e02fd19c335dfc8cc34f83662aa1181240b9","evidence-manifest.json":"496a392c39d3a75f0287c19a6fc25e7577dee6bfef0e3c7f16801f61574cfa99","snapshots/context-integrity.json":"da023ad1b2e22b1659573e1829cfb273a4d3e329d1b324506dbc73104901ecf7","snapshots/stock-harness-hire.json":"ec6d9274b80d291e8949c71a9bfb1f6fdc88cd78a2094dcad1c0a599ce9b78cd","snapshots/stock-harness.json":"680d3af8a66f473a95905920123f0760aff53eb4ce09cd01393a78b6292178cd","snapshots/stock-harness-preflight.json":"5b79fae54bd6036dab2ad8bf8e8bacb9087e702b363dbbe4af2c58115eea8ead"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.runner-acpx-claude.local.assigned-skill-explicit-invocation","profile":"runner-acpx-claude","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-5","runtimeMode":"native","durationMs":50059,"providerDurationMs":28899,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":10,"outputTokens":1332,"cachedInputTokens":136292,"totalTokens":137634,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":28899,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"526b8881ac8dd2b51a788ca6ae535f7cd856e38066fadbd70737071c4f898abf","evidence-manifest.json":"40dc320ecab36c004d5e33bba8ebe1fb4d8444fd23d7ef0f055b188b474fffca","snapshots/context-integrity.json":"0b0b2b42376e1c5cf58b6ae1e9e0960b6b2bc82218e20589e04ca44639207ecb","snapshots/stock-harness-hire.json":"7f47075f11209d29c92091925bb61b2e2e22b06e1ce49b7dba69060242d62fc5","snapshots/stock-harness.json":"06398532782d5e4c9eb76ef55cdc87ed8d49c6f31dc06f09ce28715aa09f76fa","snapshots/stock-harness-preflight.json":"cb1980515ce11e2b55fcea5799277f511d0c6d01bf26b78e93be3ae17fa9b08d"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.runner-acpx-claude.local.continuity-restart","profile":"runner-acpx-claude","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-5","runtimeMode":"native","durationMs":93902,"providerDurationMs":43211,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":6,"outputTokens":101,"cachedInputTokens":92222,"totalTokens":92329,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":43211,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"332ba05ee7cda6971777af3454e8589652f742f45cd6350da4396a420e8c549f","evidence-manifest.json":"eaff7dd508a80dffef9d638a39e13b21d2ef427086ae478c10520c2b336fe881","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"e8a854b3a0d886b01cb910e5b05baa5f1c791ef57b9a04168a0ce65cc60df7ce","snapshots/stock-harness.json":"1e53449e403bc079887cd183e2b556fbecd238a20130ef07c0c5decf499ff4ae","snapshots/stock-harness-preflight.json":"02283c369f44ab85a6400e0659da00e48bee84727fcebc35a8a691c4529c7cc0"}},
{"executionId":"stock-harness.runner-acpx-claude.local.ordered-comment-continuation","profile":"runner-acpx-claude","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"claude-sonnet-5","runtimeMode":"native","durationMs":93050,"providerDurationMs":70393,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":18,"outputTokens":3794,"cachedInputTokens":332880,"totalTokens":336692,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":70393,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"3aae7a7ca62c39c74ac6d12e71b6b75223c76112ea018b8d08db1da52ea72e6a","evidence-manifest.json":"f08a3eee09a8fd4b279865dbe8f9ab2a66cc1a79addb216fe484561fcb9e2467","snapshots/context-integrity.json":"400df6ee5a9294998d4306f6eab6816e86155d8ecf0f39a6fc672381d47d423f","snapshots/stock-harness-hire.json":"604c41ac65944ef25cb7bbbf19941401139ed422342d7d6ebf9a07b79ecf7a75","snapshots/stock-harness.json":"f17d4f47657a0151dbfc585de64e338b7f79deae052f454794c2b934a7d398a2","snapshots/stock-harness-preflight.json":"fa502f0e3a3467dab6b04f810e6001ca635ce2d4bb5bae42a50e7a85f520a356"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.runner-codex.local.assigned-skill-explicit-invocation","profile":"runner-codex","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"native","durationMs":48064,"providerDurationMs":26079,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":133429,"outputTokens":1102,"cachedInputTokens":110284,"totalTokens":244815,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":26079,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"690736dbf71179913631255763fd7b8aecd825236532154e54a3e1e53240d6b2","evidence-manifest.json":"75135aa0e7020617f54b5f2f593a5d48669ac7ecc864064d1eefc91af86d431c","snapshots/context-integrity.json":"46e65857a4d82f6ad900bc7e89166c35f2543faa368e48130f83dcea526ca65e","snapshots/stock-harness-hire.json":"2bf564606e974bb38baf50e363edd6d11aa5009fd429602a44fecb337bf9036c","snapshots/stock-harness.json":"2bf943a7cba0fc3886a04fa96f5646677f427e516232b35f6538384f4e6ed39f","snapshots/stock-harness-preflight.json":"635be5d6d578d7ad95b0e87697b939ed10f3cf2d3338caedc94350a1ed67ccc9"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.runner-codex.local.continuity-restart","profile":"runner-codex","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"native","durationMs":89776,"providerDurationMs":43862,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":460440,"outputTokens":2336,"cachedInputTokens":378421,"totalTokens":841197,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":43862,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"b32c6c2c10d1eaabd0e2408db313bf9f30a7a60413733ef8ddfd8114ff6f2fdb","evidence-manifest.json":"34a60f60f021a731a98fab708ac31e344d8ecfa5a1d7dc72daa8c957028cbcea","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"855403ffdcaa2f1542a5ebbc0dbd7590c609c170afc657884eeb9a1e79a51310","snapshots/stock-harness.json":"d4997c371e7db1a9e1c13536279480035eee7873e92a3ab3300969d508339481","snapshots/stock-harness-preflight.json":"642913765f2dfb9bbae2f46401448e4c3b6d3f9134cba2a073178d530c395efc"}},
{"executionId":"stock-harness.runner-codex.local.ordered-comment-continuation","profile":"runner-codex","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"gpt-5.6-sol","runtimeMode":"native","durationMs":85296,"providerDurationMs":64569,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":359345,"outputTokens":2627,"cachedInputTokens":307813,"totalTokens":669785,"reportedCostUsd":0,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":64569,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"1a1a5f9b53dfc8cd2d96bafb8b34f6b1ac006514da70963ea3d8328f1c594309","evidence-manifest.json":"5220dd721a29958a2dbc6139a33bed2c94765801181c795abca6118722204f7b","snapshots/context-integrity.json":"100d34aa2c461b540164a00442100d5f6f438f706d14fb7316d80bf07d680cd8","snapshots/stock-harness-hire.json":"103296d40bc0b4500fa9b45ffb1e8192b770a1d4f87f8d0dd4f9e309384d407c","snapshots/stock-harness.json":"8d1eea8270fd2c29e284db9af321bebe85e635f6ec77d56674466d1480012ba8","snapshots/stock-harness-preflight.json":"be022954d2b5f5ef0bdae4cd892f06163f9d0863d0201aa92a5bab12f56a4dac"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}},
{"executionId":"stock-harness.runner-opencode.local.assigned-skill-explicit-invocation","profile":"runner-opencode","case":"assigned-skill-explicit-invocation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"native","durationMs":69432,"providerDurationMs":45144,"cleanup":"passed","billing":{"llm":{"runCount":1,"runsWithTokenUsage":1,"runsWithReportedCost":1,"inputTokens":22870,"outputTokens":1171,"cachedInputTokens":105472,"totalTokens":129513,"reportedCostUsd":0.0021534242,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":45144,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0021534242,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0021534242,"complete":true},"billingTypes":["unknown"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"69ac8808e939a796c40ac3665d56b152b43747fa91b7bbc84fd0a0f2fc68f5d9","evidence-manifest.json":"a0af2ef7ba4653d7ec825ac12de2a9e5f8b7fd7042bcfbe215e8af1b75ad38f8","snapshots/context-integrity.json":"e82ea3b31d4c6c73a95715c1d0e31338eb31a50f760bb6ea99b1d21032c6da32","snapshots/stock-harness-hire.json":"87ec7c1456344be2ed5cf68d4813920e8204fc483c6a34b1b62116bd721d9073","snapshots/stock-harness.json":"11cb4a18967ee59a7d5c202ee590a8dcfa7a9c145ab5ac87cde8da6a69f25497","snapshots/stock-harness-preflight.json":"c044805c4bb0db4c3cac0a4bbf54b937da7d1d376a4ac319f4d2baf3c636f51c"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":7,"failed":0}},
{"executionId":"stock-harness.runner-opencode.local.continuity-restart","profile":"runner-opencode","case":"continuity-restart","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"native","durationMs":152945,"providerDurationMs":93715,"cleanup":"passed","billing":{"llm":{"runCount":3,"runsWithTokenUsage":3,"runsWithReportedCost":3,"inputTokens":26520,"outputTokens":106,"cachedInputTokens":45312,"totalTokens":71938,"reportedCostUsd":0.0005020232,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":93715,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0005020232,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0005020232,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"ae12b2eee65f45b8e2f530c037002e91db351134b6848da89e80bf7c15eb25a3","evidence-manifest.json":"dce1906b35fbbe6ac471ef922a2f0b1bae9da9438d56e78894e3651cba402aaa","snapshots/context-integrity.json":null,"snapshots/stock-harness-hire.json":"a57da515e6454ee8b82b9bf633a8e07097217f30555606538e135c7979de71c0","snapshots/stock-harness.json":"fa0501dbd07065bbbbd9a1f4c6bd82ba27f0fa45238790d531db4587c000309d","snapshots/stock-harness-preflight.json":"a20db2a5478fb7bd568bf6fcfbe7702dbe2a7a0390d9b36ec0fe2a6d07954d1b"}},
{"executionId":"stock-harness.runner-opencode.local.ordered-comment-continuation","profile":"runner-opencode","case":"ordered-comment-continuation","sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceRef":"refs/heads/codex/stock-harness-candidate-cancelled-cells","workflowRunUrl":"https://github.com/paperclipai/paperclip/actions/runs/37045368302","cellAttempt":1,"status":"passed","failureClass":null,"causeCodes":[],"failedMatcherPaths":[],"model":"openrouter/deepseek/deepseek-v4-flash-0731","runtimeMode":"native","durationMs":95671,"providerDurationMs":72942,"cleanup":"passed","billing":{"llm":{"runCount":2,"runsWithTokenUsage":2,"runsWithReportedCost":2,"inputTokens":24355,"outputTokens":2690,"cachedInputTokens":160512,"totalTokens":187557,"reportedCostUsd":0.0043860217,"costStatus":"reported"},"runtime":{"provider":"local","agentRunDurationMs":72942,"leaseDurationMs":null,"leaseCount":0,"costStatus":"not_metered","costSource":"local_not_metered"},"reportedCostUsd":0.0043860217,"estimatedRuntimeCostUsd":0,"observedAndEstimatedCostUsd":0.0043860217,"complete":true},"billingTypes":["unavailable"],"stockChecks":{"budget-hard-stops":true,"default-hire-bundle":true,"provider-runs-present":true},"invocationMetrics":[],"prerequisite":{"sourceSha":"f02d8d0df327abb43b20c7e7beb86798239abbf5","sourceFingerprint":"ea488f1e5f67865c0e08e3628eb3081b86c6b3a8e40631702697ffac08359211","passed":true,"providerCallsBeforeAdmission":0,"passedTests":537},"evidenceHashes":{"result.json":"54f3029b581b82be87bd30b7670029bd5722915917e4bcab2e8ef0682bc3c0be","evidence-manifest.json":"b793fa707ccc2125b192816d59e922a0d43348c11eb516b8dff7e5d445b5545b","snapshots/context-integrity.json":"105ff5eb813ac3779e3ba04a8115ce1495c185011de4735782d3c4adba99787e","snapshots/stock-harness-hire.json":"4dde5055f48fff1b4cf466b4307739c967e2595e3c2e93d4e07230545741fa16","snapshots/stock-harness.json":"b0ce3eb0edce09ca4ef61af6f622cddd7c9bb1a3286c9ea7756dc228d00c10fa","snapshots/stock-harness-preflight.json":"78f2298b3bb4357b2a3ce6d75a922fc34d6449ef43378bb0dcc40476469daa01"},"taskDocumentCount":1,"behaviorCheckCounts":{"passed":12,"failed":0}}
],
"totals": {
"baseline": {
"results": 24,
"passed": 15,
"failed": 9,
"missing": 0,
"cleanupPassed": 24,
"durationMs": 3250195,
"providerDurationMs": 2108034,
"llm": {
"runCount": 58,
"runsWithTokenUsage": 52,
"runsWithReportedCost": 43,
"inputTokens": 2778581,
"outputTokens": 68015,
"cachedInputTokens": 6908917,
"reportedCostUsd": 1.7494252618000001
}
},
"candidate": {
"results": 24,
"passed": 15,
"failed": 9,
"missing": 0,
"cleanupPassed": 24,
"durationMs": 2804913,
"providerDurationMs": 1540860,
"llm": {
"runCount": 47,
"runsWithTokenUsage": 45,
"runsWithReportedCost": 36,
"inputTokens": 2280185,
"outputTokens": 58933,
"cachedInputTokens": 4655025,
"reportedCostUsd": 1.7431513318
}
}
},
"matchedPassingTiming": {
"baseline": {
"cells": 13,
"medianCellDurationMs": 85550,
"medianProviderDurationMs": 57150
},
"candidate": {
"cells": 13,
"medianCellDurationMs": 69761,
"medianProviderDurationMs": 43862
}
},
"missingBaseline": [],
"cancellationRecovery": {
"sourceSha": "f02d8d0df327abb43b20c7e7beb86798239abbf5",
"originalRun": 37042856368,
"reason": "superseding pilot cancelled incomplete cells; completed behavior failures not rerun",
"executionIds": [
"stock-harness.legacy-acp-codex.local.ordered-comment-continuation",
"stock-harness.legacy-opencode.local.ordered-comment-continuation",
"stock-harness.legacy-opencode.local.continuity-restart",
"stock-harness.runner-acpx-claude.local.continuity-restart",
"stock-harness.legacy-opencode.local.assigned-skill-explicit-invocation",
"stock-harness.runner-acpx-claude.local.assigned-skill-explicit-invocation",
"stock-harness.runner-acpx-claude.local.ordered-comment-continuation",
"stock-harness.runner-codex.local.continuity-restart",
"stock-harness.runner-codex.local.ordered-comment-continuation",
"stock-harness.runner-codex.local.assigned-skill-explicit-invocation",
"stock-harness.runner-opencode.local.assigned-skill-explicit-invocation",
"stock-harness.runner-opencode.local.continuity-restart",
"stock-harness.runner-opencode.local.ordered-comment-continuation"
]
},
"baselineInfrastructureRecovery": {
"schema": "paperclip.stock-harness.infra-recovery/v1",
"sourceSha": "12c5433c67dc2e62916b879349c7ba2b6e0431f0",
"originalRun": 37042864888,
"originalJob": 110958906481,
"reason": "AWS runner shutdown, no result artifact; no completed model failure rerun",
"executionIds": [
"stock-harness.legacy-opencode.local.ordered-comment-continuation"
]
},
"layoutRecovery": {
"resultEdits": 0,
"oracleEdits": 0,
"originalEvidenceRetained": true,
"manifests": [
{
"campaign": "candidate-original",
"sha256": "9135457cdc339d2fcaeea52cd06622ecaea6a74561df73d4d7a0baa0d8c0e7cc"
},
{
"campaign": "baseline-final",
"sha256": "85223428a0d473ba83e324621e80d711725ebe069c4f055665b0bf987d1f0c4c"
},
{
"campaign": "candidate-recovery",
"sha256": "7788f879d0cf1d140d452cfa78fe085b34bc23d298d7c31cac9d630b12b2572f"
},
{
"campaign": "baseline-recovery-final",
"sha256": "e39d62dd60f092e76f8776e844041f2458fca1989854d57ab13932a36c937ceb"
}
]
},
"limitations": [
"Single trial per cell; not general coding quality or statistical proof.",
"Skill task document storage wording is ambiguous; original graders unchanged.",
"Persisted credential guard failures are not equivalent to poor task behavior.",
"Report costs are incomplete; zero reported native cost has unknown billing type.",
"Local runtime and interrupted partial attempts are not metered.",
"Historical public prompt retrieval is clipped in three ACP cells; missing suffix does not prove provider omission.",
"Reconstructed reports use separate copies with directory-layout-only repair; original public reports retain infrastructure failures."
],
"fixtureMatchProof": {
"schema": "paperclip.stock-harness.fixture-match-proof/v1",
"cells": [
{
"executionId": "stock-harness.legacy-codex.local.ordered-comment-continuation",
"fixtureConfigSha256": "f727e68de06dec501c49efc41ab941bd6d63826db82e615d5e60f2d20133b488",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.legacy-codex.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "ca51e8b29aab8e6efa32f2433c0f3960e43f9fc2f5edcd0660e753810d71f30d",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.legacy-codex.local.continuity-restart",
"fixtureConfigSha256": "8b1669039f3285f89a7f93c57e1c0dc12f071fbb2bbc8dcef6bf70bc14a4949a",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.legacy-claude.local.ordered-comment-continuation",
"fixtureConfigSha256": "94db17927ceddb54fe7766a362adcf78783726c68e1e136e407572aef6fed319",
"model": "claude-sonnet-4-6"
},
{
"executionId": "stock-harness.legacy-claude.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "6c21be9bb3fcec6274372fc586d88e42d82bc0d438e26a69ef614f8e6fcb8519",
"model": "claude-sonnet-4-6"
},
{
"executionId": "stock-harness.legacy-claude.local.continuity-restart",
"fixtureConfigSha256": "695071e5453c90288659f54f40fb307486f64d5fb8a401b767d2e9974faab15f",
"model": "claude-sonnet-4-6"
},
{
"executionId": "stock-harness.runner-codex.local.ordered-comment-continuation",
"fixtureConfigSha256": "9e19859684aad0246fd009061101eec59c41179e51e92a9467f506af22558126",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.runner-codex.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "8fc61211aefb339a310469695d716c3faea15c2a547c543d70c599857a7e071d",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.runner-codex.local.continuity-restart",
"fixtureConfigSha256": "024e90354d760f90a40e2b3923468b72d36a9fabdb4fc58c10a7939a09e5cbc3",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.runner-opencode.local.ordered-comment-continuation",
"fixtureConfigSha256": "9dd6fef8fea77b07a67681acc1d7866fac9b4f16e23aca69eda32a4656cc260b",
"model": "openrouter/deepseek/deepseek-v4-flash-0731"
},
{
"executionId": "stock-harness.runner-opencode.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "891eb6dd154aa5926a3de2eae79780661b436618b6eac7733998750c6c341ba3",
"model": "openrouter/deepseek/deepseek-v4-flash-0731"
},
{
"executionId": "stock-harness.runner-opencode.local.continuity-restart",
"fixtureConfigSha256": "441c2fa8bdd26cf14ac2440311b99a71b7dd6202698cafc3d01ed7a91dc4b004",
"model": "openrouter/deepseek/deepseek-v4-flash-0731"
},
{
"executionId": "stock-harness.runner-acpx-claude.local.ordered-comment-continuation",
"fixtureConfigSha256": "77cc32714b23852d3ef67c687202d89f7513275500b3c522a0a1de3d745c9c4e",
"model": "claude-sonnet-5"
},
{
"executionId": "stock-harness.runner-acpx-claude.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "c190a1e731ef450ea6ae0c24b5a8ae0f21db725b47350645cdbb9273415c5acd",
"model": "claude-sonnet-5"
},
{
"executionId": "stock-harness.runner-acpx-claude.local.continuity-restart",
"fixtureConfigSha256": "40876df6f88f34ba8232ccba8816bf7e1173be0042a90a174f79cddcca346c4c",
"model": "claude-sonnet-5"
},
{
"executionId": "stock-harness.legacy-acp-codex.local.ordered-comment-continuation",
"fixtureConfigSha256": "22500c39874cda62a2512ac3d8a43309ce5879f55f8515e31dd8ac632edb6c93",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.legacy-acp-codex.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "896d8602d2990bf7517a4392d66909c6649e0f00f79a5ea0bb8f0e370b7b597a",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.legacy-acp-codex.local.continuity-restart",
"fixtureConfigSha256": "7b66845e231674cc6e78f1d86e863dc03005998c2674316af50db60ec53b7e4a",
"model": "gpt-5.6-sol"
},
{
"executionId": "stock-harness.legacy-acp-claude.local.ordered-comment-continuation",
"fixtureConfigSha256": "6c1b9d7debb306af405640eb381e1709ae7185e7925bc2bd3ef31acf98fbff8f",
"model": "claude-sonnet-4-6"
},
{
"executionId": "stock-harness.legacy-acp-claude.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "53f308d6fbdc30adc60370279ab38679b5d87075899be4db6c0730593e1632c4",
"model": "claude-sonnet-4-6"
},
{
"executionId": "stock-harness.legacy-acp-claude.local.continuity-restart",
"fixtureConfigSha256": "80ec22c5080ef1a71fc97547c3e78f4bc5f54f121f46f982506f2f9f773b27d0",
"model": "claude-sonnet-4-6"
},
{
"executionId": "stock-harness.legacy-opencode.local.ordered-comment-continuation",
"fixtureConfigSha256": "e015ded0177d8d4c893acc658e25699b28937d97cb9d58da023ddff7ba7eac0b",
"model": "openrouter/deepseek/deepseek-v4-flash-0731"
},
{
"executionId": "stock-harness.legacy-opencode.local.assigned-skill-explicit-invocation",
"fixtureConfigSha256": "dd673f469cab583d26bc2e4b484f18b27884ec14a59542177026389181e2da2c",
"model": "openrouter/deepseek/deepseek-v4-flash-0731"
},
{
"executionId": "stock-harness.legacy-opencode.local.continuity-restart",
"fixtureConfigSha256": "3dadc83dff00ef2919cd24c6328ea9507f971e7d294e9ece699ff2ee6fa98230",
"model": "openrouter/deepseek/deepseek-v4-flash-0731"
}
]
},
"workflowProvenance": [
{
"run": 37042856368,
"workflowSha": "7d59de6113caf5219d34d65e1fb5f0c1a7fbdc2a",
"workflowUrl": "https://github.com/paperclipai/paperclip/actions/runs/37042856368"
},
{
"run": 37042864888,
"workflowSha": "7d59de6113caf5219d34d65e1fb5f0c1a7fbdc2a",
"workflowUrl": "https://github.com/paperclipai/paperclip/actions/runs/37042864888"
},
{
"run": 37045368302,
"workflowSha": "7d59de6113caf5219d34d65e1fb5f0c1a7fbdc2a",
"workflowUrl": "https://github.com/paperclipai/paperclip/actions/runs/37045368302"
},
{
"run": 37044967981,
"workflowSha": "7d59de6113caf5219d34d65e1fb5f0c1a7fbdc2a",
"workflowUrl": "https://github.com/paperclipai/paperclip/actions/runs/37044967981"
},
{
"run": 37048838402,
"workflowSha": "7d59de6113caf5219d34d65e1fb5f0c1a7fbdc2a",
"workflowUrl": "https://github.com/paperclipai/paperclip/actions/runs/37048838402"
}
]
}
doc/plans/2026-10-02-stock-harness-live-comparison.md
+110
View File
@@ -0,0 +1,110 @@
# Stock-harness live comparison — 2026-10-02
**TL;DR:** Two newly failing paired cases: classic Claude/OpenCode document delivery. Two newly passing cases: classic and native OpenCode ordered continuation. Seven unchanged failures and 13 unchanged passes. The extra ACP Claude storage symptom occurs beneath an existing credential-guard failure. All 48 results are available; no general performance equivalence is established.
The reduced instructions are **not yet qualified for merge**. Classic Claude and classic OpenCode pass the historical skill-output case but fail with the reduced instructions: they finish without saving a Paperclip task document. Native Codex and native Claude pass all three journeys in both variants. Single trials and ambiguous fixture storage wording limit causal attribution; the failed oracle remains unchanged.
The reductions and evaluation setup are in draft [PR #14948](https://github.com/paperclipai/paperclip/pull/14948). This report and its [safe evidence projection](2026-10-02-stock-harness-live-comparison.json) record the measured revisions rather than claiming the final documentation head was run through the full matrix.
## Revisions and matched inputs
- Candidate: `f02d8d0df327abb43b20c7e7beb86798239abbf5`, eight-word hire manual plus reduced shared startup/resume instructions.
- Historical baseline: `12c5433c67dc2e62916b879349c7ba2b6e0431f0`, the same evaluation harness with only the prior default manual and shared prompt source restored from `e00d10d5d5594f6e2d1e8cf4e0ec81c074b0bd88`. Its manifest records ten identical behavioral/fixture source hashes.
- Native Codex fix [#14920](https://github.com/paperclipai/paperclip/pull/14920) is held constant. This comparison does not measure its before/after performance.
- Same eight profiles, three journeys, provider models, effort configuration, skills, tools, permissions, credentials, local runtime, and independent graders. Effort inherits provider defaults where the existing profile does not set it.
- Each variant selects 24 cells and initially budgets 48 turns, with existing bounded retries and 1,000-cent company/agent hard stops. Recorded run-ledger counts can differ after early failures, retries and cleanup.
## Per-profile and case outcomes
These are recorded overall qualifications, including security and receipt failures. A credential-guard failure does not imply every behavioral matcher failed.
| Profile | Model | Journey | Historical | Reduced |
| --- | --- | --- | --- | --- |
| `legacy-codex` | `gpt-5.6-sol` | `assigned-skill-explicit-invocation` | Pass | Pass |
| `legacy-codex` | `gpt-5.6-sol` | `ordered-comment-continuation` | Pass | Pass |
| `legacy-codex` | `gpt-5.6-sol` | `continuity-restart` | Pass | Pass |
| `legacy-claude` | `claude-sonnet-4-6` | `assigned-skill-explicit-invocation` | Pass | Fail: no Paperclip document |
| `legacy-claude` | `claude-sonnet-4-6` | `ordered-comment-continuation` | Pass | Pass |
| `legacy-claude` | `claude-sonnet-4-6` | `continuity-restart` | Fail: chat memory | Fail: chat memory |
| `legacy-opencode` | `openrouter/deepseek/deepseek-v4-flash-0731` | `assigned-skill-explicit-invocation` | Pass | Fail: no Paperclip document |
| `legacy-opencode` | `openrouter/deepseek/deepseek-v4-flash-0731` | `ordered-comment-continuation` | Fail: deadline/run-start timeout | Pass |
| `legacy-opencode` | `openrouter/deepseek/deepseek-v4-flash-0731` | `continuity-restart` | Pass | Pass |
| `legacy-acp-codex` | `gpt-5.6-sol` | `assigned-skill-explicit-invocation` | Fail: credential guard | Fail: credential guard |
| `legacy-acp-codex` | `gpt-5.6-sol` | `ordered-comment-continuation` | Fail: credential guard, receipt incomplete | Fail: credential guard |
| `legacy-acp-codex` | `gpt-5.6-sol` | `continuity-restart` | Fail: credential guard, receipt incomplete | Fail: credential guard, receipt incomplete |
| `legacy-acp-claude` | `claude-sonnet-4-6` | `assigned-skill-explicit-invocation` | Fail: credential guard | Fail: credential guard, no Paperclip document |
| `legacy-acp-claude` | `claude-sonnet-4-6` | `ordered-comment-continuation` | Fail: credential guard, receipt incomplete | Fail: credential guard |
| `legacy-acp-claude` | `claude-sonnet-4-6` | `continuity-restart` | Fail: credential guard, receipt incomplete | Fail: credential guard |
| `runner-codex` | `gpt-5.6-sol` | `assigned-skill-explicit-invocation` | Pass | Pass |
| `runner-codex` | `gpt-5.6-sol` | `ordered-comment-continuation` | Pass | Pass |
| `runner-codex` | `gpt-5.6-sol` | `continuity-restart` | Pass | Pass |
| `runner-acpx-claude` | `claude-sonnet-5` | `assigned-skill-explicit-invocation` | Pass | Pass |
| `runner-acpx-claude` | `claude-sonnet-5` | `ordered-comment-continuation` | Pass | Pass |
| `runner-acpx-claude` | `claude-sonnet-5` | `continuity-restart` | Pass | Pass |
| `runner-opencode` | `openrouter/deepseek/deepseek-v4-flash-0731` | `assigned-skill-explicit-invocation` | Pass | Pass |
| `runner-opencode` | `openrouter/deepseek/deepseek-v4-flash-0731` | `ordered-comment-continuation` | Fail: deadline/run-start timeout | Pass |
| `runner-opencode` | `openrouter/deepseek/deepseek-v4-flash-0731` | `continuity-restart` | Pass | Pass |
## Findings and next step
The classic Claude and classic OpenCode skill runs read the pinned skill marker and complete the task, but save zero Paperclip documents; both historical runs save exactly one. The legacy ACP Claude skill run has the same document difference under its credential-guard failure. The earlier reduced-instruction Claude pilot also failed this document oracle. The pinned skill asks for a “task document” without explicitly naming Paperclip storage, so this may reveal both reliance on the former manual and a fixture ambiguity. It remains a failed delivery result, not a regraded pass.
Keep the agreed tiny manual. Next, verify that legacy agents discover and follow the existing **Paperclip skill/API path for durable task documents and artifacts**. The skill already prohibits file-only handoff, lists issue-document routes and gives a plan-document example; audit that delivery before adding a concise generic-document recipe if needed. Run the same preserved case plus a clearly storage-specific case on both variants. Legacy agents do not receive native `paperclip_finish`/`paperclip_block` tools. Native tool descriptions are a separate item 2.3 change; guidance must follow the actual runtime capability. No production or skill instructions were changed to make these runs pass.
Classic Claude chat fails the memory assertion in both variants. Six legacy ACP cells per variant fail the persisted-credential guard: the scanner finds a scoped provider credential in persisted ACP session state. Raw session files and credential values are not published. These are existing qualification failures requiring their own investigation; they cannot be waived or interpreted as instruction-reduction quality evidence.
Historical native OpenCode ordered comments hit the case deadline without the required continuation outcome. The initial historical classic OpenCode ordered cell lost its AWS runner before result upload; its one unchanged-source recovery also hits the 12-minute deadline. Both OpenCode ordered cases pass in the candidate. Completed failures are never rerun to select a better result. Those candidate passes against historical timeouts are observed differences, not proof of a general improvement.
Historical public prompt retrieval is clipped in three legacy ACP cells (four invocations), including identity/connection suffix text. Those structural receipt failures cannot establish that the provider omitted instructions. The candidate ACP Codex chat run also lacks a complete per-run invocation receipt. Instruction presence/absence claims are bounded by the retained public receipts.
## Timing, usage and cost
| Variant | Retained cells | Pass / fail / missing | Cell duration sum | Provider duration sum | Recorded runs with tokens / with reported cost / total | Reported LLM subtotal |
| --- | ---: | --- | ---: | ---: | --- | ---: |
| baseline | 24 | 15 / 9 / 0 | 3250.195s | 2108.034s | 52 / 43 / 58 | $1.749425 |
| candidate | 24 | 15 / 9 / 0 | 2804.913s | 1540.860s | 45 / 36 / 47 | $1.743151 |
For the 13 cells that pass both variants, median cell time is 85.550s historical versus 69.761s reduced; median provider time is 57.150s versus 43.862s. This excludes failures and is descriptive, not a reliable speedup estimate.
Provider billing is incomplete. Legacy Codex/ACP usage is unpriced in several runs; native Codex/Claude often report zero with billing type `unknown`. Reported subtotals do not prove zero native spending or invoice totals. Local runner costs and interrupted partial runs are not metered. Diagnostic/pilot costs and repeated interrupted work are outside the matched-cell subtotal and remain retained. No aggregate cost-saving claim is justified.
## Campaign and evidence history
| Campaign | Source | Outcome / evidence |
| --- | --- | --- |
| [Initial diagnostic Codex](https://github.com/paperclipai/paperclip/actions/runs/37034213743) | `a63437069` | Skill passes; predates mandatory admission. |
| [Cold full attempt](https://github.com/paperclipai/paperclip/actions/runs/37037105491) | `36e987246` | Missing SDK build; cancelled before provider admission. |
| [SDK pilot](https://github.com/paperclipai/paperclip/actions/runs/37039240025) | `4163dbfd0` | Missing daemon; stops before providers. |
| [Daemon pilot](https://github.com/paperclipai/paperclip/actions/runs/37040493183) | `ac6ddefb5` | Missing fake Codex fixture; stops before providers. |
| [Complete-build Claude pilot](https://github.com/paperclipai/paperclip/actions/runs/37041741124) | `f02d8d0df` | 537 prerequisites pass; skill document oracle fails; reported $0.175308. |
| [Initial candidate matrix](https://github.com/paperclipai/paperclip/actions/runs/37042856368) | `f02d8d0df` | 4 pass, 7 fail, 13 cancelled when a same-target-branch pilot superseded it. Original attempts retained. |
| [Candidate cancelled-cell recovery](https://github.com/paperclipai/paperclip/actions/runs/37045368302) | `f02d8d0df` | Only the 13 cancelled cells: 11 pass, 2 fail. Combined candidate has 24 results, 15 pass / 9 fail. |
| [Historical matrix](https://github.com/paperclipai/paperclip/actions/runs/37042864888) | `12c5433c6` | 15 pass, 8 recorded failures, one AWS-runner shutdown without result upload. |
| [Historical interrupted-cell recovery](https://github.com/paperclipai/paperclip/actions/runs/37048838402) | `12c5433c6` | Only legacy OpenCode ordered comments: 720.616s deadline failure; evidence and cleanup valid. Historical cohort now has 24 results, 15 pass / 9 fail. |
| [Current packaging pilot](https://github.com/paperclipai/paperclip/actions/runs/37044967981) | `1eb5ba420` | Attempt 1 stopped before cell/provider execution; attempt 2 passes, evidence valid and cleanup pass. |
The same-target-branch workflow concurrency rule caused the candidate interruption; that orchestration mistake was acknowledged and corrected with a separate unchanged-source recovery branch. No completed model failure was retried. The historical missing cell is recovered once for a runner shutdown. Partial/failed attempts remain part of the history and unknown-spend accounting.
The measured matrix packages contain a prerequisite folder beside the exact campaign root. The trusted selector correctly rejects this layout, so their original published reports synthesize missing-result infrastructure failures. A separate local reconstruction nests that prerequisite folder under the campaign root and verifies byte identity for every copied file; result, scorer, usage, screenshot and prerequisite content are unchanged. The unchanged trusted selector then selects 11/24 original candidate, 13/13 recovery 23/24 original historical packages and 1/1 interrupted-cell recovery; recovered normalization validates every retained result’s evidence. Original missing packages stay missing in their original campaign; the separate recovery result has its own provenance. This is declared directory-layout recovery, not a canonical successful publication or a change to the oracle. The safe JSON records result/receipt/scoring hashes and recovery-manifest hashes.
The fix at `1eb5ba420` writes prerequisites beneath the exact campaign root. Its single selected Codex skill cell passes through the protected report and publication pipeline: [interactive pilot report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37044967981-2/index.html), [GitHub evidence](https://github.com/paperclipai/paperclip/actions/runs/37044967981#artifacts). It has `evidenceValid=true`, cleanup pass, 27.149s provider / 49.523s cell, and a reported zero subtotal with unknown billing type. It does not replace the full matched cohort or qualify untested providers.
Original historical/recovery dashboards retain the publication failure: [historical](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37042864888-1/index.html), [candidate recovery](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37045368302-1/index.html). Inspect their access-controlled raw artifacts for measured cell evidence; do not read synthetic missing-result rows as task performance.
## Verification and remaining qualification
At `1eb5ba420`, all 557 credential-free prerequisites pass (556 TypeScript plus one Rust), along with 895 E2E support tests, E2E typecheck and 24-cell discovery. The cold builder prepares SDK/shared outputs and both daemon/fake protocol binaries, records hashes, and fails closed before credentials. The 313 filtered native tests are not counted as passing coverage. This head has 52 passing CI gates and two skipped gates; Greptile reviewed that exact head at 5/5 with zero open threads. Later documentation heads require fresh checks/review; measured sources remain explicit above.
Repository typecheck and build pass. The complete local Vitest run has 14,870 passed, 83 skipped and three unrelated timing failures; the three affected files pass unchanged narrow reruns (25 auth, 16 reviewed-chat binding, four webhook tests). Original failures are retained rather than calling the first full run green.
PR #14948 stays draft while document delivery is unresolved. Unrepresented harnesses, legacy ACP credential safety, public-receipt clipping, the independent ACP base-replacement issue, existing Claude chat memory behavior and saved-session migration are not qualified away by these results. This small matrix measures skill/context/chat behavior, not broad coding quality or statistical equivalence.
## Retained delivery diagnosis and approved repair
Both failed classic agents loaded the operational Paperclip skill. This is not an observed skill-discovery failure. In the reduced Claude run, the agent wrote a workspace Markdown file before loading Paperclip and then completed the issue without a document API write. The reduced OpenCode run loaded Paperclip, accessed the assigned skill through the public skills API, and wrote a workspace task-document file; it made no document, attachment, or work-product delivery write. Both corresponding historical runs saved a Paperclip issue document.
Claude received the full operational skill, including its existing no-local-only delivery guidance. OpenCode's skill result was explicitly truncated in both variants: the early artifact rule survived, but the later generic document endpoint and plan-only write example were omitted. That truncation is existing behavior, not a newly measured regression. Removing always-on delivery reminders may interact with it, but the combined manual/shared reduction and ambiguous assigned-skill wording prevent a causal attribution to one layer.
Dotta approved a small early API-runtime skill recipe and a generic issue-document reference. The tiny manual remains eight words. The recipe checks the successful write's returned saved revision/content and links the returned key; it does not require another GET after a clear valid receipt. It preserves explicit destinations, downloadable-file delivery, and native document-tool boundaries.
The focused repair comparison will hold the tiny manual/shared prompts fixed and vary only `skills/paperclip/SKILL.md` plus the new `references/issue-documents.md`. Classic Claude and OpenCode each run the preserved original case and an added explicitly Paperclip-storage case: four cells per variant, eight expected provider turns in total. This measures the skill repair separately from the original reduction. The explicit case independently checks public saved body/revision and an agent comment linking the exact same-app document; local-only, missing-revision, wrong-document, local-path, and other-origin outcomes cannot pass. Current state: fixture preparation and source freeze, no repair provider run yet.
doc/plans/2026-10-02-stock-harness-paperclip-checklist.md
+290 -18
View File
@@ -1,7 +1,17 @@
# Stock harness, with Paperclip: working checklist
Created: 2026-10-02. Status: item 1 implemented for native Codex app-server;
PR validation in progress. Other implementation items remain open.
Created: 2026-10-02. Status: item 1 merged for native Codex app-server in
[PR #14920](https://github.com/paperclipai/paperclip/pull/14920). Item 2's default
hire manual is reduced to identity only, and common legacy startup/resume
instructions are in [PR #14948](https://github.com/paperclipai/paperclip/pull/14948).
GitHub live qualification retains earlier document-delivery failures; the latest
focused OpenCode comparison passes both cases in both variants but still exposes
deficient original-case handoff links. PR #14948 remains draft. The original
[live report](2026-10-02-stock-harness-live-comparison.md), completed
[skill repair](2026-10-02-legacy-document-skill-repair.md) and latest
[stock-selection/link comparison](2026-10-02-opencode-skill-routing-link-qualification.md)
retain their separate sources and limitations. Additional carriers and fixed
native instructions remain open; hiring templates are being reduced separately.
Goal: keep the agent's stock harness behavior and add only what it needs to work
with Paperclip. Apply this across legacy adapters, the new Runner, and their
@@ -12,7 +22,11 @@ record the proposed behavior and Dotta's direction, make a bounded change, and
verify the affected paths before moving on. New findings get stable IDs in the
ledger below so they do not disappear into conversation history.
**Current item: 1 — validate and review the native Codex instruction fix.**
For every change, record its executable test/eval coverage before continuing.
Distinguish coverage setup, deterministic results, and measured live results;
configured cells alone do not qualify behavior.
**Current item: 2 — reduce the default operating manual and shared prompt layers.**
## Agreed direction and boundaries
@@ -43,7 +57,7 @@ ledger below so they do not disappear into conversation history.
- [x] Remove default replacement of stock base instructions across those paths.
- [x] Verify the actual app-server request and retained session instructions,
including resume; checking only a prompt builder is insufficient.
- [ ] Verify Paperclip task context, tools, auth, assigned skills, and completion
- [x] Verify Paperclip task context, tools, auth, assigned skills, and completion
still work. Record applicable regressions and eval results.
Starting points: [Codex backend](../../packages/paperclip-runner/src/backends/codex-native-backend.ts),
@@ -61,14 +75,20 @@ and need a provider session reset. Do not reset active sessions automatically.
The separate Codex-through-ACP dependency patch remains a coverage follow-up
under item 6; this change covers the native app-server path.
Verification: 412 focused TypeScript/Rust tests, repository typecheck/build,
55 passing PR checks, and Greptile 5/5. The original full local test command
was stopped after setup failures; affected suites passed on rerun. No paid
live campaign was run, and no task-quality improvement is claimed.
## 2. Reduce the default operating manual and shared prompt layers
- [ ] Inventory what an agent actually receives: hire instructions, shared
prompt template, wake context, runtime prompt, bootstrap, and loaded skills.
Separate always-present text from content loaded on demand.
- [ ] Agree on the tiny common contract and the coordination details that each
runtime still requires. Legacy API coordination and native semantic tools
need appropriate instructions for their respective interfaces.
- [x] Agree on the tiny default: identity only, with no skill or native-tool
pointers. The harness supplies coordination instructions.
- [x] Reduce the generic default hire `AGENTS.md` to the agreed sentence and
update the existing creation test to expect the minimal bundle.
- [ ] Remove repeated workflow rules and stock coding/style/autonomy guidance.
- [ ] Move detailed planning, hiring, artifacts, and exceptional procedures to
discoverable references or tools where feasible.
@@ -80,7 +100,130 @@ Starting points: [default hire instructions](../../server/src/onboarding-assets/
[native runtime contract](../../packages/paperclip-runner/src/contracts/runtime-context.ts),
[Paperclip operational skill](../../skills/paperclip/SKILL.md).
Decision: exact retained paragraph and on-demand boundaries pending.
Decision: Dotta confirmed the harness already handles skill and tool delivery;
the default hire manual is now only "You are an agent in a Paperclip company."
This replaces 602 words with eight. The shared loader supplies this default
across instruction-bundle-capable adapters for non-CEO hires without explicit
instructions. Existing saved bundles retain their content; CEO, first-agent,
and role/team templates remain separate work under item 3. The common
prompt/wake reduction is complete locally below; additional carriers, native
instructions, and skill/reference corrections remain open.
Verification: the existing agent-skills route suite passed all 54 tests,
including default creation, custom bundles, and CEO/first-agent paths. The
onboarding asset suite passed all 10 tests. This initially had only deterministic
coverage. Dotta subsequently requested behavioral eval coverage for every change;
the dedicated production-default-hire suite below closes the custom QA manual
gap. Full repository typecheck/build/test were not rerun for this narrow asset
and existing-test update.
### Shared prompt follow-ups
- [x] **2.1 Reduce common legacy startup/resume instructions.** Keep
identity and connection guidance in the shared task/chat defaults; remove the
generic resumed-wake execution contract. Preserve current task/event data,
specialized wake contracts, custom templates, skills, and auth.
- [ ] **2.2 Review additional legacy carriers.** Reduce Hermes local/gateway
wrappers, review Pi system delivery, and check OpenClaw fresh-wake framing.
Preserve transport facts and user configuration. Dotta deferred this item on
2026-10-02 for a later revisit; it remains open.
- [ ] **2.3 Reduce native Runner instructions and constraints.** Improve
discoverable tool documentation first, then shorten fixed guidance and
consolidate completion rules. Verify prompt revisions, digests, and session
compatibility across native Codex, ACPX, and OpenCode. Dotta approved the
native tool-documentation slice on 2026-10-02 in a separate worktree. Native
`paperclip_finish`/`paperclip_block` guidance must not leak into legacy
completion paths, which use the operational skill and API.
### Follow-up 2.1 implementation and verification
The common task and conversation defaults now share the same identity sentence
and unchanged connection guidance: 113 words each, down from 661 and 205.
Connection guidance accounts for 108 of those words. The ordinary resumed-wake
execution contract is removed (172 words), including the old opt-in used by
OpenClaw on fresh turns. `includeExecutionContract` remains accepted as a
deprecated no-op for adapter/plugin source compatibility.
Current task facts, ordered comments, work modes, approval/review state,
checkout, holds/blockers, recovery/watchdog roles, and external-chat contracts
remain in their existing owners. Skill delivery, runtime authentication,
custom `promptTemplate`, and `bootstrapPromptTemplate` mechanics are unchanged.
Hermes's own wrappers and Pi's system carrier remain follow-up 2.2; native
fixed instructions and full-turn constraints remain follow-up 2.3.
The later read-only 2.2 audit found another OpenClaw-owned HTTP identity,
checkout/status, delegation, plan-approval and task-discovery wrapper in
`buildWakeText`. Its short conversation branch is selected when optional task
Markdown is present, including on ordinary task dispatch. Existing dispatch
coverage does not assert the absence of conversation waiting language. This is
a framing concern to verify, not a measured failure. Pi already uses additive
`--append-system-prompt` delivery and suppresses the duplicate user-prompt copy;
its next step is a delivery audit with minimal changes. These findings are
deferred with 2.2 and are not implemented in PR #14948.
The new defaults apply when prompts are assembled after deployment. Existing
provider sessions can retain earlier startup instructions in their history
until reset; no active session or saved custom template is rewritten here.
Verification: 563 distinct focused tests passed across 18 suites:
- Shared prompt selection/rendering, operational-skill selection, and retained
specialized wake contracts (130 tests).
- ACPX, Pi, OpenCode, Cursor Cloud, and Codex local execution, including
task/chat fresh/resumed/reset/fallback delivery, custom prompts, skill
mounting, and authentication (259 tests).
- Hermes local prompt rendering and gateway execution (41 tests).
- Claude, Gemini, Cursor local, Grok, Kimi, OpenClaw, and Claude/Codex ACP
fallback execution (113 tests).
- Native execution input, preserving shared wake compatibility (20 tests).
`@paperclipai/adapter-utils` typecheck and build passed; `git diff --check`
passed. Full repository typecheck/build/test and live behavioral evals were not
run for this local bounded change. These checks prove prompt and runtime
mechanics, not an improvement in coding-task quality.
### Remaining shared layers inspected (2026-10-02)
The table records the initial inspection before follow-up 2.1. Word counts
cover fixed source text, not complete assembled prompts or token counts. They
exclude task data, assigned agent instructions, and skills. This inspection
establishes instruction mechanics, not a performance result.
| Layer | Delivery at initial inspection | Disposition |
| --- | --- | --- |
| Shared legacy task/chat templates | `DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE` was 661 words and the conversation template was 205, including connection guidance. Used by Claude, Codex, Cursor local/cloud, Gemini, Grok, Kimi, OpenCode, Pi, and Hermes local. | Completed in 2.1: both defaults now 113 words. Repeated procedures removed, connection guidance retained, no replacement skill pointer. |
| Generic resumed wake contract | `renderPaperclipWakePrompt` added a 172-word execution contract on ordinary resumed turns. OpenClaw requested it on fresh turns too because it has no shared template. | Completed in 2.1: generic contract removed, including legacy opt-ins. Task/event data and conditional runtime contracts retained. |
| Additional legacy carriers | Hermes local adds a 171-word identity/API/curl wrapper before the shared task template; Hermes gateway supplies its own four-rule execution contract. Pi carries shared defaults in its system extension and suppresses the wake copy when that extension owns policy. | Separate follow-up 2.2; changing the shared constants alone does not remove every wrapper. Preserve transport facts and custom configuration. |
| Native fixed prompt | The 262-word `paperclip-execution.v5` prompt carries hiring, delegation/dependencies, connection setup, and finalization guidance across native Codex, ACPX, and OpenCode backends. | Keep one short native completion/dependency contract. Relocate uncommon procedures to the relevant tool documentation, improving that documentation where needed before removing instructions. Any change needs prompt revision/digest/session-compatibility verification. |
| Native full-turn constraints | `nativeTaskConstraints` adds assigned-skill limits, current agent-file paths, document/file delivery, answered-question handling, and accepted completion/final-response sequencing. Codex/OpenCode add another completion reminder. Prepared constraints reach the model envelope through `native-session-runtime`; compact continuations avoid replaying prior task constraints. | Consolidate duplicate completion instructions. Keep current paths, answer scope, and the required result protocol. Move detailed file/document procedure to discoverable tool descriptions where adequately covered. |
| Task/event-specific framing | Server task Markdown and wake rendering supply work mode, ordered comments, approved revisions, checkout/holds/blockers, recovery/review/watchdog roles, external-chat bookkeeping/delivery, and conversation handoff. | Retain the facts and mode boundaries in the first reduction. Use one authoritative owner for shared mode directives; review specialized wording separately. |
Custom `promptTemplate` and fresh-only `bootstrapPromptTemplate` configuration
are distinct from shipped defaults and should not be silently rewritten.
The built-in process adapter invokes a configured command; HTTP passes context
as JSON. They do not use the shared text template. External adapter plugins and
hosted configuration variants still require the broader item 6 audit.
An accepted-plan discrepancy needs resolution: task Markdown permits cohesive
implementation on the current issue, while the wake renderer's planning-mode
accepted-confirmation branch requires children and prohibits implementation on
the source issue. Existing tests assert both versions. Production reachability
after the server changes work mode has not yet been established; do not claim
this proves a live plan-continuation failure.
Dotta selected follow-up 2.1 first: reduce the common legacy startup/resume
copies, preserving conditional context. Additional carriers and native fixed
instructions remain distinct follow-ups 2.2 and 2.3. No shared runtime text was
changed by the initial inspection.
Coverage identified during inspection includes shared prompt selection/rendering,
adapter execution tests, native backend/runtime context tests, and server
task-context/native-input tests. Follow-up 2.1 extended the shared and adapter
checks; its results are above. Follow-ups 2.2 and 2.3 need their corresponding
carrier/protocol assertions. Product E2E context-integrity, completion-updates,
and direct blocker guidance are candidate lifecycle checks; their current scope
does not establish coding-task quality. The initial inspection itself ran no
tests or live evals.
## 3. Review every hiring and role template
@@ -107,7 +250,61 @@ Starting points: [onboarding assets](../../server/src/onboarding-assets/),
[hiring skill and references](../../skills/paperclip-create-agent/),
[teams catalog](../../packages/teams-catalog/catalog/).
Decision: per-template disposition and migration policy pending.
Dotta approved and merged [PR #14985](https://github.com/paperclipai/paperclip/pull/14985)
at `862a5758ba0e88a33232c1f1fa645e85c38a3113`, after corrected source
`57dcee147ed0b2d2e3cc657cd9e50fb16bf9ec25` passed full CI and fresh Greptile 5/5.
The hiring skill/references, CEO/CoS assets and team catalog are now on master.
Their live qualification remains separate from unit/CI verification.
[Prompt diffs and preserved scope](https://github.com/paperclipai/paperclip/blob/57dcee147ed0b2d2e3cc657cd9e50fb16bf9ec25/doc/plans/2026-10-02-hiring-template-prompt-diff.md).
The read-only audit found four onboarding CEO files (1,897 words), a 164-word
CoS manual, four role examples (coder 652, QA 619, UX 1,325, security 1,724 words),
and eight team-catalog manuals. Hiring step 6, the 60–150-line role guide and
review checklist would regenerate the removed operating policies.
- [x] **3.1** Reduce role drafting rules, coder/adapted hire examples and matching
catalog coder, preserving metadata, auth, permissions and skill selections.
- [x] **3.2** Reduce the CEO bundle/copy, including forced delegation, hiring,
memory and repeated API recipes.
- [x] **3.3** Review remaining roles/catalog and CoS copy.
- [ ] **3.4** Review specialized built-in/plugin contracts separately; retain
product-required behavior rather than assuming every rule is redundant.
Custom and existing bundles and governance remain deliberate rollout boundaries.
Specialized Summarizer, Reflection Coach and Wiki Maintainer prompts remain
unchanged for separate product-contract review; optional Content Lead was already
tiny. The generic non-CEO fallback belongs to PR #14948. PR #14985 does not reduce
every shipped specialized agent.
Its explicit `hiring-templates` suite selects native local Codex and ACPX Claude
`hire-coder-template-reuse` cells: a real default CEO, explicitly requested
production hiring skill/coder reference, independently scored saved JSON
artifacts and session reuse. Outcome and source/read coverage are separate;
missing or redacted receipts do not prove no regression. The existing first-task
suite covers actual wizard CoS snapshot selection. The matched union holds the
reduced manual/shared/operational skill and native completion guidance constant,
restoring only 21 historical hiring production/derived sources in baseline.
Frozen candidate `9f5404ad3aacbe76777952759414d34fd381e674` and baseline
`296a4df85e8bcc97a160fc78c291b17adb828196` passed 705 exact-source credential-free
prerequisites each (704 TypeScript + 1 Rust; zero providers), with 8,242 identical
other tracked files and matching fixture/model/auth/permissions configuration.
Only four candidate-specific single-file CEO selection assertions are filtered
symmetrically; each variant's bundle/source hashes and canonical catalog are
independently admitted. Two cells per variant, five expected turns each:
four cells / 20 turns, 15 minutes each, no automatic retries or broader selection.
The [candidate](https://github.com/paperclipai/paperclip/actions/runs/37075466208)
and [baseline](https://github.com/paperclipai/paperclip/actions/runs/37075469463)
protected campaigns are dispatched. No graded live pairs yet; pending evidence
cannot establish non-regression.
[Inspect the partial report](https://github.com/paperclipai/paperclip/blob/e3720f369df070036526dd295a7a68e49488c51a/doc/plans/2026-10-02-hiring-template-live-comparison.md).
Report-only updates preserve the measured refs and merged hiring PR.
The default-manual PR is replayed on the merged hiring base, preserving custom
CEO bundle checks, the minimal generic boundary and both eval suites. Its prior
browser failure remains retained: a deterministic process fixture replayed its
last plan command on an asynchronous `chat_task_completed` wake and wrote a
second, text-identical source plan revision. It was not paid/provider execution;
fresh rebased CI remains required before readiness.
## 4. Fix repository context while retaining Paperclip configuration
@@ -154,7 +351,7 @@ does not apply.
| Legacy Codex / Claude | Local adapters, managed auth and isolated configuration | Initial inspection only |
| Other legacy local adapters | ACPX, OpenCode, Pi, Cursor, Gemini, Grok, Kimi, Hermes | Pending |
| Other adapter transports | Cursor Cloud, Hermes/OpenClaw gateways, process, HTTP, external adapter plugins | Pending |
| Runner Codex | App-server driver, runnerd bridge, Rust provider, direct/fallback paths | Additive instruction fix implemented; PR validation pending |
| Runner Codex | App-server driver, runnerd bridge, Rust provider, direct/fallback paths | Additive instruction fix merged in PR #14920 |
| Runner ACPX | Enabled profiles, especially Claude/Grok; declared or pending profiles tracked separately | Claude initial inspection; remaining audit pending |
| Runner OpenCode | Native provider and configuration paths | Pending |
| Hosted/remote providers | Claude Managed and AWS AgentCore; identify their own baseline rather than assuming CLI semantics | Pending |
@@ -172,15 +369,18 @@ does not apply.
## Verification and evals — apply to each item
- [ ] Map existing coverage before adding cases; use [doc/evals.md](../evals.md).
- [x] Map existing coverage for all implemented changes before adding cases;
see the SH-1–SH-3 map below and [doc/evals.md](../evals.md).
Keep Runner protocol evals and Product E2E evals distinct.
- [ ] Start with narrow deterministic checks for instruction layering, effective
- [x] Set up narrow deterministic checks for the implemented instruction layering,
hire, and shared-prompt changes. Future changes still need their own map for effective
configuration, skill/auth delivery, and session behavior where appropriate.
- [ ] Use Runner evals for provider/session/tool protocol changes; use Product
E2E for real hiring, repository context, task lifecycle, and artifact delivery.
- [ ] Review existing context-integrity, hiring, completion-updates, and blocker
suites for reusable coverage; record gaps rather than claiming coverage from
a similarly named case.
- [x] Reuse existing protocol tests for native Codex request layering and add Product
E2E for production-default hiring, skill delivery, task lifecycle, and chat
continuity. Repository context and additional artifact cases remain future work.
- [x] Review existing suites for reusable coverage. Their custom QA manuals did
not exercise the tiny default hire; the new suite deliberately omits those
bundles and reuses independently graded skill, continuation, and chat journeys.
- [ ] Compare task quality as well as Paperclip protocol compliance when claiming
that fewer instructions improve agent performance. Keep model, effort,
permissions, tools, and fixture comparable.
@@ -189,19 +389,71 @@ does not apply.
- [ ] Run the relevant checks for each change and the repository's required full
verification before a PR-ready handoff. Record unrun checks and their reasons.
### Coverage for changes implemented so far
The [stock-harness runbook](../../tests/runner-e2e/STOCK-HARNESS.md) defines a
credential-free prerequisite plus 24 explicit local Product E2E cells across
eight legacy/native profiles. Each profile runs assigned-skill invocation,
ordered comment continuation, and chat continuity across restart. Public receipts
verify the identity-only managed bundle before provider execution; final receipts
check actual legacy prompts and both budget hard stops. The existing lifecycle
oracles still own task/chat success. Models, auth, skills, permissions, and
secret-reference plumbing are inherited from existing profiles.
| Coverage ID | Implemented change | Executable coverage | Live status |
| --- | --- | --- | --- |
| SH-1 | Native Codex additive developer instructions, including start/resume/recovery | TypeScript driver, runnerd transport, Runner Lab/live-session, and Rust provider tests in `pnpm test:e2e:runner:stock-harness`; native Codex cells exercise real hires. | Native Codex passes all three matched journeys in both variants; #14920 held constant, so task success is not before/after vendor-base proof. |
| SH-2 | Eight-word default hire `AGENTS.md` | Public creation/onboarding tests; exact independent public bundle oracle before and after provider execution for every stock-harness cell. | Candidate public tiny-bundle and budget receipts pass in all 24 retained cells. Behavioral delivery is not fully qualified; see F14–F17. |
| SH-3 | Reduced shared task/chat defaults and removed generic resume contract | Shared renderer and ACPX/Codex/OpenCode/Pi/Hermes/Cursor Cloud regressions; actual legacy invocation prompts checked in the new suite. | Actual legacy prompts measured. Document-delivery regressions and clipped/missing receipts retained; additional carriers remain deferred item 2.2. |
The suite is explicit-only and excluded from `--all`; it does not add paid work
to ordinary campaigns. Negative calibration covers manual regrowth, missing or
malformed prompt receipts, old startup/resume procedures, missing connection
guidance, wrong budgets, and skipped prerequisite assertions. Source revisions,
attempts, cost, partial failures, cleanup, and sanitized evidence use the existing
report pipeline. No paid providers were launched during the initial setup.
Subsequent GitHub measurements are recorded below; no broad coding-quality
improvement or full live qualification is claimed. Unrepresented harnesses remain
unqualified; Codex-through-ACP vendor-base preservation is still F7.
Local setup verification: 478 prerequisite tests passed (477 TypeScript plus one
Rust), 860 Product E2E support tests passed, E2E typecheck passed, and all 24 cells
were discovered. The 313 unrelated native tests filtered by the gate are not
passing coverage. Final prerequisite evidence is retained at
`tests/runner-e2e/results/stock-harness-preflight-2026-10-02T16-15-39.064Z/preflight.json`;
earlier interrupted, missing-Hermes-discovery, and setup/test-timeout attempts are
retained as failures. Hermes is checked through its package config because the
root Vitest project list omits it. Later exact-head prerequisites expanded to
557 passed checks (556 TypeScript and one Rust), and E2E support to 895 tests.
Repository typecheck/build passed. The complete local Vitest run retained three
unrelated timing failures; all three files passed unchanged reruns. At
`1eb5ba420`, 52 CI gates pass and two skip; Greptile is 5/5 with zero open
threads. Later documentation heads require fresh checks. See the live report
for exact source hashes, campaigns, failures and cost limitations.
## Findings ledger
Confirmed mechanics below do not by themselves establish an effect on task quality.
| ID | Finding | Work item / disposition |
| --- | --- | --- |
| F1 | Native Codex sent Paperclip text as `baseInstructions`. Probes on codex-cli 0.153.4 showed replacement; additive `developerInstructions` retained the stock base on start and cold resume. | 1; app-server fix implemented, PR validation pending |
| F1 | Native Codex sent Paperclip text as `baseInstructions`. Probes on codex-cli 0.153.4 showed replacement; additive `developerInstructions` retained the stock base on start and cold resume. | 1; app-server fix merged in PR #14920 |
| F2 | Default hires and role templates prescribe substantial operating procedures; common prompt and wake layers add further coordination text. | 2–3; mechanics confirmed, performance effect unmeasured |
| F3 | Hiring references require legacy Paperclip skill/comment procedures, while native Runner intentionally omits that operational skill and uses semantic tools. | 2–3, 5; reconcile runtime contracts |
| F4 | Local Claude appends instructions; Runner Claude preserves the Claude Code preset. Runner isolation excludes project/local settings, which can also exclude repository instruction discovery. | 4; selective context fix to design |
| F5 | Some Codex capability settings differ between the direct driver and daemon path; an intermediate configuration does not prove the final provider behavior. | 6; effective-path audit pending |
| F6 | Omitting or nulling `baseInstructions` on an old Codex thread's resume preserves its saved replacement; an empty string produces an empty base. | 1; document the required provider session reset; no automatic migration in this PR |
| F7 | The isolated Codex-through-ACP dependency patch also sets `baseInstructions` on start/resume. It is a separate path from the native app-server backend. | 6; follow-up patch/profile audit pending |
| F8 | The operational skill says target-bound confirmations default `supersedeOnUserComment` to true; the default manual and server normalizer say false. The server uses false. | 2; correct stale skill/reference guidance in a follow-up |
| F9 | The default manual required a comment on every task, while the operational skill's verified external-chat shortcut delegates comments and lifecycle bookkeeping to the harness. | 2; unconditional manual rule removed; shared layers still need review |
| F10 | Removing the default manual left the 661-word legacy task template and its 172-word resumed-wake execution contract. Hermes and Pi have additional policy carriers. | 2.1 complete locally: task/chat defaults 113 words, no generic resume contract; 2.2 wrappers pending |
| F11 | Task Markdown and the wake renderer prescribe different accepted-plan behavior for planning-mode accepted-confirmation payloads; tests currently expect both. | 2; unify the directive owner; production reachability still to trace |
| F12 | Native fixed instructions and full-turn constraints repeat completion and uncommon procedures, while reserved finish/block tool descriptions are only one sentence each. | 2; improve tool documentation before removing needed native protocol guidance |
| F13 | Existing context-integrity/chat fixtures injected a QA manual, so their green results did not qualify the production tiny hire default. | Dedicated stock-harness suite measured real default hires; retained failures prevent blanket qualification. |
| F14 | Historical classic Claude/OpenCode skill runs save one Paperclip document; reduced runs save none while completing the task. Legacy ACP Claude has the same behavior under a separate guard failure. Pinned skill storage wording is ambiguous. | Measured delivery failures; keep original oracle. Improve legacy skill/API delivery guidance separately, then compare both variants with preserved and storage-specific cases. |
| F15 | Classic Claude restart chat fails its memory assertion in both variants. | Existing behavior, not attributable to this reduction from these trials. |
| F16 | All six legacy ACP cells per variant fail the persisted-provider-credential guard. Three historical ACP cells also have clipped public prompt retrieval, and candidate ACP Codex chat lacks a complete invocation receipt. | Security/receipt qualification follow-ups; do not waive guard, expose raw sessions, or infer missing provider instructions from clipped receipts. |
| F17 | Prerequisites beside the campaign root violate trusted artifact selection. A same-target pilot superseded 13 candidate cells; one historical AWS runner shut down without upload. | Packaging fixed at `1eb5ba420` and pilot passes. Directory-layout-only copies preserve every byte; missing cells alone recovered at unchanged source, completed failures never rerun. |
Append new findings with evidence, affected paths, and the numbered item that
will address them. Record intentional behavior explicitly rather than as a bug.
@@ -214,6 +466,26 @@ will address them. Record intentional behavior explicitly rather than as a bug.
| 2026-10-02 | Paperclip-owned MCP isolation and configuration changes/session resets are acceptable. | Preserve Paperclip auth and assigned skills while fixing repository context. |
| 2026-10-02 | Dotta requested implementation and a PR for item 1. Native app-server paths now use additive developer instructions. | 139 targeted TypeScript tests and 91 Rust provider tests passed; repository typecheck/build passed. Remaining test/review results to record. |
| 2026-10-02 | Verified actual Codex instruction layering using a localhost Responses stub, without paid inference. | codex-cli 0.153.4 sent identical 14,732-character stock base instructions on start and cold resume, with the Paperclip marker retained in developer input. This is protocol evidence, not a task-quality eval. |
| 2026-10-02 | Dotta requested and confirmed the merge of item 1. | PR #14920 merged at `408f70e69f9c5e49cb4377f4886ac2001bfa67a2`, with all 55 checks passing and Greptile 5/5. |
| 2026-10-02 | Dotta directed an identity-only default manual with no skill or runtime pointers; the harness already handles coordination. | Reduced the default to eight words. Existing hire and onboarding suites passed all 64 tests. Shared prompts and role templates remain pending. |
| 2026-10-02 | Dotta requested three explicit shared-prompt follow-ups and selected common legacy startup/resume reduction first. | Follow-ups 2.1–2.3 recorded. 2.1 complete locally: both defaults 113 words; generic resume contract removed. 563 focused tests and shared utility typecheck/build passed. Extra carriers and native instruction reduction remain pending. |
| 2026-10-02 | Dotta requested executable eval coverage for everything implemented so far and all subsequent changes before continuing. | Added SH-1–SH-3 coverage map, credential-free prerequisite, and 24 production-default-hire cells. 478 prerequisite and 860 support tests passed; E2E typecheck/discovery passed. Live provider results remain `not_run`; item 2.2/2.3 unchanged. |
| 2026-10-02 | Dotta requested PRs and GitHub-runner before/after qualification while discussing subsequent work separately. | Draft [PR #14948](https://github.com/paperclipai/paperclip/pull/14948); native Codex diagnostic [passed](https://github.com/paperclipai/paperclip/actions/runs/37034213743), 34.745 s provider / 56.660 s cell. Comparison restores only the prior manual/shared prompts and holds #14920 constant; no general coding-quality claim. |
| 2026-10-02 | Full candidate cold setup failed before provider admission; stopped and retained the attempt. | [Run 37037105491](https://github.com/paperclipai/paperclip/actions/runs/37037105491), target `36e987246`: prerequisite imports lacked the plugin SDK build. Added ordinary dependency setup before credential-free prerequisites and exact-source coverage of connection guidance. Cold pilot and full matched campaigns pending. |
| 2026-10-02 | Dotta deferred 2.2 additional legacy carriers and approved the first 2.3 native tool-description slice separately. | Keep wrappers open; native finish/block documentation must not be supplied to legacy skill/API completion paths. |
| 2026-10-02 | Matched default-manual/shared-prompt campaigns measured failures, with #14920 constant. | Candidate `f02d8d0df`: 15/24 pass. Historical `12c5433c6`: 15/24 pass after the one runner-shutdown recovery also timed out. Classic Claude/OpenCode Paperclip document delivery regressed in observed trials; keep PR #14948 draft. [Live report](2026-10-02-stock-harness-live-comparison.md). |
| 2026-10-02 | Cold prerequisite and packaging faults were repaired without weakening admission or behavioral graders. | Three setup attempts stopped before providers. Current `1eb5ba420` pilot passes 557 prerequisite checks and protected report publication; source/hash/cost evidence retained. Candidate cancellation and historical runner shutdown recovered only for missing cells. |
| 2026-10-02 | Final bounded recovery completed; no further model reruns. | Both matched cohorts have all 24 results, 15 pass and nine fail. Two classic skill deliveries regress; two OpenCode ordered cases pass only with reduced instructions. Overall parity is not behavioral equivalence. All 48 retained result/receipt projections are hashed and sanitized; original interruptions and partial unknown spend remain recorded. |
| 2026-10-02 | Dotta approved the measured legacy delivery repair and narrow follow-up qualification. | Early operational skill PUT/receipt/link guidance plus generic issue-document reference; eight-word manual retained. Focused original + explicit Paperclip-storage cases on classic Claude/OpenCode, with only the two skill sources varied. All four pairs completed: Claude original Fail → Pass, Claude explicit Pass → Pass, both OpenCode cases Fail → Fail. Explicit OpenCode handoff worsened beneath the unchanged machine grade (clickable API URL → code-formatted path). [Repair report](2026-10-02-legacy-document-skill-repair.md); no full matrix rerun. Claude chat-memory (F15) and ACP credential/receipt failures (F16) remain separate and unresolved. |
| 2026-10-02 | Native tool-description comparison completed all six paired cases. | Zero newly failing cases, three unchanged completion passes, three unchanged blocker failures. Claude/OpenCode blocker API matchers pass in both variants; UI matcher wrongly demanded marker-only replies. Codex's exact-action punctuation failure is unchanged. Original failures retained; corrected 10-case browser calibration and separate retained-DOM replay pass all six visible replies. Exact Codex action failures remain. No native fixed-prompt removal measured. |
| 2026-10-02 | Dotta approved minimal operational skill selection/link correction after retained OpenCode diagnosis. | Candidate `fe9dc1e3c` and baseline `0d7ecfa96d` have two matched Pass → Pass cases, zero new failures/passes and no pending pairs. All four exact-source gates pass 587 checks, all four provider runs and cleanup pass. Candidate original loads Paperclip/reference before delivery, but uses the wrong PAP prefix; baseline original saves publicly later within the same assignment and gives a bare path. Explicit clickable UI links are correct in both. [Complete report](2026-10-02-opencode-skill-routing-link-qualification.md); prior failures retained, no causal or broad quality claim. |
- F18: Native blocker browser assertion required a marker-only reply despite asking for owner/action/reason. Correct marker-plus-explanation checks symmetrically, calibrate contradictory and future-condition replies, retain original verdicts.
- F19: Original prompt-removal cohorts loaded Paperclip; OpenCode skill truncation omitted the late API recipe. The early repair fixed Claude in one paired trial. In the repaired original OpenCode assignment, Paperclip was first loaded only during disposition recovery after a local-file write. Shared legacy operational-skill delivery/selection needs review before more recipe expansion.
- F20: Explicit repaired OpenCode reads the early recipe/reference and saves a public document/revision, but hands off a code-formatted path rather than a clickable anchor. Baseline provides a clickable API URL, rejected by the UI-only oracle. Preserve both grades; clarify future clickable UI-link fixture and review the smallest link example separately.
- F22: Legacy operational skill mounting is already mandatory, but its discovery description omitted ordinary task/heartbeat work and document delivery. The approved narrow fix expands stock metadata selection and adds a real Markdown link example in the existing reference. Original + clarified-explicit OpenCode pairs both pass in both variants, so improvement causality is not established. Candidate loads the operational skill/reference before public delivery; baseline original writes locally first, then saves publicly in the same assignment. Original candidate copies the example PAP prefix into its link while baseline supplies a bare slug path; neither is scored by the original link-free oracle. No full-body injection or native-tool leakage. ACP Claude names-only metadata, custom ACP unsupported delivery, OpenClaw wrappers and Pi HOME differences remain distinct follow-ups.
- F23: The new original OpenCode candidate copies `/PAP/issues/RUN-1#document-context-integrity-output` from the literal example despite actual prefix RUN. It passes durable-document grading but has an incorrect-prefix handoff. Baseline original also lacks a clickable canonical link. Approved reference-only correction now derives the link from `issue.identifier` and the saved receipt key. Independent provider-free calibration checks two other prefixes, redirected keys and rejects a wrong-company link; 25 focused document/source tests, canonical metadata checks and E2E typecheck pass. The recipe now also handles null/absent identifiers through the supported issue-ID route; fresh review caught the initial nullable edge. 56 focused checks pass. Source review shows the UI corrects wrong prefixes, so the frozen PAP href is noncanonical rather than proven broken. These later corrections are not live-qualified by the preserved runs; no further paid rerun.
- F21: Skill heading insertion shifted both generated capability inventories. General CI caught stale metadata after the frozen repair campaign’s narrower build admitted providers. Regenerate canonical derived files and require stale-manifest/inventory checks before future provider admission.
For each completed item, add the chosen behavior, changed paths, verification
results, remaining exceptions, and follow-ups here before checking it off.
doc/runner-created-skills.md
+21
View File
@@ -4,6 +4,27 @@ The native Runner exposes `create_skill` in Auto and skill-test work. An agent
can save a reusable, single-file skill directly in the company's skill library.
The tool is unavailable in Ask and pre-acceptance Plan modes.
The native Runner also exposes `update_skill` in the same modes. Read the
existing skill and its `currentVersionId` through the company skill API before
editing. Supply the **complete** primary file, not a fragment:
```json
{
"skillId": "<existing skill UUID>",
"expectedVersionId": "<current version UUID>",
"markdown": "---\nname: release-review\ndescription: Review release notes.\n---\n\n# Review\nCheck each release note and its tests.\n",
"idempotencyKey": "review-update-1"
}
```
The tool uses the existing Skill Studio file endpoint and company `skills.edit`
policy. It returns a skill ID, path, resulting version ID, and Skill Studio
path. Retry a lost response with **exactly** the same key and inputs; that
returns the original receipt without another version or activity record.
Reusing the key with different inputs, or updating after the version changes,
returns a conflict. After a version conflict, reread the skill and choose a
new key for the revised edit. Policy is checked again even on exact retries.
```json
{
"name": "release-review",
package.json
+1
View File
@@ -74,6 +74,7 @@
"test:e2e:browser-context": "playwright test --config tests/e2e/playwright-company-context.config.ts",
"test:e2e:runner:browser-support": "playwright test --config tests/runner-e2e/playwright-support.config.ts",
"test:e2e:runner:unit": "vitest run --config tests/runner-e2e/vitest.config.ts",
"test:e2e:runner:stock-harness": "node tests/runner-e2e/stock-harness-checks.mjs",
"test:e2e:runner:typecheck": "tsc -p tests/runner-e2e/tsconfig.json",
"test:e2e:runner:report": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/report.ts",
"test:runner-workflow-evals": "pnpm --filter @paperclipai/paperclip-eval-kernel build && pnpm --filter @paperclipai/paperclip-runner test:runner-workflow-evals",
packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.test.ts
+165
View File
@@ -0,0 +1,165 @@
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createRuntimeStore, type AcpSessionRecord, type AcpSessionStore } from "acpx/runtime";
import { afterEach, describe, expect, it, vi } from "vitest";
import { createEphemeralSessionEnvironmentStore } from "./ephemeral-session-environment.js";
const currentEnvironment = Object.freeze({
PAPERCLIP_API_KEY: "current-credential-fixture",
PAPERCLIP_RUN_ID: "current-run-fixture",
PATH: "/fixture/current-bin",
});
const oldEnvironment = { PAPERCLIP_API_KEY: "old-credential-fixture", OLD_RUN_PATH: "/fixture/old-run" };
function record(): AcpSessionRecord {
return {
schema: "acpx.session.v1",
acpxRecordId: "fixture-record",
acpSessionId: "fixture-session",
agentSessionId: "fixture-agent-session",
agentCommand: "fixture-provider",
cwd: "/fixture/project",
name: "fixture-name",
createdAt: "2026-10-03T00:00:00.000Z",
lastUsedAt: "2026-10-03T00:00:01.000Z",
lastSeq: 3,
eventLog: { active_path: "/fixture/events.jsonl", segment_count: 1, max_segment_bytes: 1024, max_segments: 2 },
messages: [{ User: { id: "message-1", content: [{ Text: "Retained conversation" }] } }],
updated_at: "2026-10-03T00:00:01.000Z",
cumulative_token_usage: { input_tokens: 11, output_tokens: 7 },
request_token_usage: { request_1: { input_tokens: 11 } },
importedFrom: { recordId: "origin-record", cwdOriginal: "/fixture/origin", exportedBy: "fixture-caller", exportedAt: "2026-10-03T00:00:00.000Z" },
acpx: {
current_mode_id: "code",
desired_config_options: { effort: "high" },
session_options: { model: "fixture-model", allowed_tools: ["Read", "Write"], max_turns: 4, system_prompt: { append: "Retained caller option" }, env: { ...currentEnvironment } },
},
};
}
function freeze<T extends object>(value: T): T {
for (const child of Object.values(value)) if (typeof child === "object" && child !== null) freeze(child);
return Object.freeze(value);
}
function stubStore(loaded: AcpSessionRecord | undefined) {
const load = vi.fn<AcpSessionStore["load"]>().mockResolvedValue(loaded);
const save = vi.fn<AcpSessionStore["save"]>().mockResolvedValue(undefined);
return { load, save };
}
const directories: string[] = [];
afterEach(async () => { await Promise.all(directories.splice(0).map(directory => rm(directory, { recursive: true, force: true }))); });
async function diskStore() {
const stateDir = await mkdtemp(join(tmpdir(), "paperclip-session-env-")); directories.push(stateDir);
return { store: createRuntimeStore({ stateDir }), file: join(stateDir, "sessions", "fixture-record.json") };
}
describe("ephemeral ACPX session environment", () => {
it("saves a fresh session to the actual file store without persisting its live environment", async () => {
const disk = await diskStore(), input = freeze(record());
const wrapper = createEphemeralSessionEnvironmentStore(disk.store, currentEnvironment);
await wrapper.save(input);
const text = await readFile(disk.file, "utf8"), saved = JSON.parse(text);
expect(saved.acpx.session_options).not.toHaveProperty("env");
for (const value of Object.values(currentEnvironment)) expect(text).not.toContain(value);
expect(input.acpx?.session_options?.env).toEqual(currentEnvironment);
expect(saved.acpx.current_mode_id).toBe("code");
expect(saved.acpx.desired_config_options).toEqual(input.acpx?.desired_config_options);
expect(saved.acpx.session_options).toEqual({ model: "fixture-model", allowed_tools: ["Read", "Write"], max_turns: 4, system_prompt: { append: "Retained caller option" } });
expect(saved.messages).toEqual(input.messages);
expect(saved.imported_from.record_id).toBe("origin-record");
const loaded = await wrapper.load(input.acpxRecordId);
expect(loaded?.acpx?.session_options?.env).toEqual(currentEnvironment);
expect(loaded?.messages).toEqual(input.messages);
});
it("loads an old env-bearing file with rotated credentials without writing, then saves neither environment", async () => {
const disk = await diskStore(), legacy = record(); legacy.acpx!.session_options!.env = { ...oldEnvironment };
await disk.store.save(legacy);
const before = await readFile(disk.file, "utf8");
const wrapper = createEphemeralSessionEnvironmentStore(disk.store, currentEnvironment);
const loaded = await wrapper.load(legacy.acpxRecordId);
expect(loaded?.acpx?.session_options?.env).toEqual(currentEnvironment);
expect(loaded?.acpx?.session_options?.env).not.toHaveProperty("OLD_RUN_PATH");
expect(await readFile(disk.file, "utf8")).toBe(before);
expect(legacy.acpx?.session_options?.env).toEqual(oldEnvironment);
await wrapper.save(loaded!);
const after = await readFile(disk.file, "utf8");
for (const value of [...Object.values(oldEnvironment), ...Object.values(currentEnvironment)]) expect(after).not.toContain(value);
expect(JSON.parse(after).acpx.session_options).not.toHaveProperty("env");
expect(loaded?.acpx?.session_options?.env).toEqual(currentEnvironment);
});
it("loads an env-free persisted session into the current environment without writing", async () => {
const disk = await diskStore(), input = record(); delete input.acpx!.session_options!.env;
await disk.store.save(input);
const before = await readFile(disk.file, "utf8");
const loaded = await createEphemeralSessionEnvironmentStore(disk.store, currentEnvironment).load(input.acpxRecordId);
expect(loaded?.acpx?.session_options?.env).toEqual(currentEnvironment);
expect(loaded?.acpx?.session_options?.model).toBe("fixture-model");
expect(await readFile(disk.file, "utf8")).toBe(before);
});
it("copies loaded records, options and current env without mutating stored or caller-owned objects", async () => {
const stored = record(); stored.acpx!.session_options!.env = { ...oldEnvironment }; freeze(stored);
const backing = stubStore(stored), wrapper = createEphemeralSessionEnvironmentStore(backing, currentEnvironment);
const loaded = await wrapper.load("fixture-record");
expect(backing.load).toHaveBeenCalledWith("fixture-record"); expect(backing.save).not.toHaveBeenCalled();
expect(loaded).not.toBe(stored); expect(loaded?.acpx).not.toBe(stored.acpx);
expect(loaded?.acpx?.session_options).not.toBe(stored.acpx?.session_options);
expect(loaded?.acpx?.session_options?.env).not.toBe(currentEnvironment);
expect(stored.acpx?.session_options?.env).toEqual(oldEnvironment);
loaded!.acpx!.session_options!.env!.PAPERCLIP_API_KEY = "isolated-loaded-fixture";
expect(currentEnvironment.PAPERCLIP_API_KEY).toBe("current-credential-fixture");
});
it("saves only a copied env-free projection, preserving unrelated and caller metadata", async () => {
const input = freeze(Object.assign(record(), { caller_metadata: { keep: "caller-state", env: { preserve: "unrelated caller metadata" } } }));
const backing = stubStore(undefined), wrapper = createEphemeralSessionEnvironmentStore(backing, currentEnvironment);
await wrapper.save(input);
const saved = backing.save.mock.calls[0]![0];
const { env: _environment, ...expectedOptions } = input.acpx!.session_options!;
expect(saved).toEqual({ ...input, acpx: { ...input.acpx, session_options: expectedOptions } });
expect(saved).not.toBe(input); expect(saved.acpx).not.toBe(input.acpx);
expect(saved.acpx?.session_options).not.toBe(input.acpx?.session_options);
expect(saved.acpx?.session_options).not.toHaveProperty("env");
expect(input.acpx?.session_options?.env).toEqual(currentEnvironment);
});
for (const [name, acpx] of [
["absent acpx", undefined],
["explicit undefined acpx", undefined],
["absent session options", { current_mode_id: "code" }],
["undefined session options", { session_options: undefined }],
["empty session options", { session_options: {} }],
["undefined env", { session_options: { env: undefined, model: "fixture-model" } }],
] as const) it(`preserves ${name} when saving and retains existing load injection semantics`, async () => {
const input = record();
if (name === "absent acpx") delete input.acpx; else input.acpx = acpx;
freeze(input);
const backing = stubStore(input), wrapper = createEphemeralSessionEnvironmentStore(backing, currentEnvironment);
await wrapper.save(input);
const saved = backing.save.mock.calls[0]![0];
if (name === "undefined env") expect(saved.acpx?.session_options).toEqual({ model: "fixture-model" });
else expect(saved).toEqual(input);
expect(Object.hasOwn(saved, "acpx")).toBe(Object.hasOwn(input, "acpx"));
if (saved.acpx && input.acpx) expect(Object.hasOwn(saved.acpx, "session_options")).toBe(Object.hasOwn(input.acpx, "session_options"));
const loaded = await wrapper.load(input.acpxRecordId);
expect(loaded?.acpx?.session_options?.env).toEqual(currentEnvironment);
expect(backing.save).toHaveBeenCalledTimes(1);
});
it("returns a missing record without saving or creating conversation state", async () => {
const backing = stubStore(undefined);
expect(await createEphemeralSessionEnvironmentStore(backing, currentEnvironment).load("missing")).toBeUndefined();
expect(backing.save).not.toHaveBeenCalled();
});
it("propagates load and save errors unchanged", async () => {
const failure = new Error("fixture store failure"), backing = stubStore(undefined);
backing.load.mockRejectedValue(failure); backing.save.mockRejectedValue(failure);
const wrapper = createEphemeralSessionEnvironmentStore(backing, currentEnvironment);
await expect(wrapper.load("fixture-record")).rejects.toBe(failure);
const input = freeze(record()); await expect(wrapper.save(input)).rejects.toBe(failure);
expect(input.acpx?.session_options?.env).toEqual(currentEnvironment);
});
});
packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.ts
+32
View File
@@ -0,0 +1,32 @@
import type { AcpSessionStore } from "acpx/runtime";
/** Keep the current run's environment in memory, outside persisted conversation state. */
export function createEphemeralSessionEnvironmentStore(
persistedStore: AcpSessionStore,
currentEnvironment: Readonly<Record<string, string>>,
): AcpSessionStore {
return {
async load(id) {
const record = await persistedStore.load(id);
if (!record) return undefined;
return {
...record,
acpx: {
...record.acpx,
session_options: {
...record.acpx?.session_options,
env: { ...currentEnvironment },
},
},
};
},
save(record) {
const persisted = { ...record };
if (record.acpx?.session_options !== undefined) {
const { env: _environment, ...sessionOptions } = record.acpx.session_options;
persisted.acpx = { ...record.acpx, session_options: sessionOptions };
}
return persistedStore.save(persisted);
},
};
}
packages/adapter-utils/src/acpx-engine/execute.test.ts
+99 -5
View File
@@ -2,7 +2,7 @@ import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { AcpRuntimeOptions } from "acpx/runtime";
import { createRuntimeStore, type AcpRuntimeOptions, type AcpSessionRecord, type AcpSessionStore } from "acpx/runtime";
import type { AdapterExecutionContext, AdapterRuntimeMcpAccess } from "@paperclipai/adapter-utils";
import {
DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC,
@@ -377,6 +377,42 @@ const ALLOWED_TURN_SPAN_ATTRIBUTE_KEYS = new Set<string>([
]);
describe("shared ACPX engine runtime behavior", () => {
it("integrates the env-free store while delivering current credentials and options to the provider", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const first = await runExecutor({ agent: "claude", agentCommand: "node ./fake-acp.js", cwd: root, stateDir,
env: { ANTHROPIC_API_KEY: "first-fixture-credential" } });
const liveEnvironment = (first.sessionInputs[0]!.sessionOptions as { env: Record<string, string> }).env;
expect(liveEnvironment.ANTHROPIC_API_KEY).toBe("first-fixture-credential");
const store = first.runtimeOptions[0]!.sessionStore as AcpSessionStore;
const record: AcpSessionRecord = {
schema: "acpx.session.v1", acpxRecordId: "integration-record", acpSessionId: "integration-session",
agentCommand: "fixture-provider", cwd: root, name: "fixture", createdAt: "2026-10-03T00:00:00.000Z",
lastUsedAt: "2026-10-03T00:00:01.000Z", lastSeq: 0,
eventLog: { active_path: path.join(stateDir, "fixture.jsonl"), segment_count: 1, max_segment_bytes: 1024, max_segments: 2 },
messages: [], updated_at: "2026-10-03T00:00:01.000Z", cumulative_token_usage: {}, request_token_usage: {},
acpx: { session_options: { env: { ...liveEnvironment }, model: "retained-model", max_turns: 4 } },
};
await store.save(record);
const file = path.join(stateDir, "sessions", "integration-record.json");
const saved = await fs.readFile(file, "utf8");
expect(saved).not.toContain("first-fixture-credential");
expect(JSON.parse(saved).acpx.session_options).toEqual({ model: "retained-model", max_turns: 4 });
expect(record.acpx?.session_options?.env).toEqual(liveEnvironment);
// A file written by the prior engine is loaded with the current run's env,
// without altering the provider launch or deleting unrelated options.
record.acpx!.session_options!.env = { ANTHROPIC_API_KEY: "legacy-fixture-credential" };
await createRuntimeStore({ stateDir }).save(record);
const second = await runExecutor({ agent: "claude", agentCommand: "node ./fake-acp.js", cwd: root, stateDir,
env: { ANTHROPIC_API_KEY: "rotated-fixture-credential" } }, { runtime: { sessionParams: first.result.sessionParams } });
expect((second.sessionInputs[0]!.sessionOptions as { env: Record<string, string> }).env.ANTHROPIC_API_KEY).toBe("rotated-fixture-credential");
const rotatedStore = second.runtimeOptions[0]!.sessionStore as AcpSessionStore;
const loaded = await rotatedStore.load(record.acpxRecordId);
expect(loaded?.acpx?.session_options).toMatchObject({ env: { ANTHROPIC_API_KEY: "rotated-fixture-credential" }, model: "retained-model", max_turns: 4 });
await rotatedStore.save(loaded!);
const resaved = await fs.readFile(file, "utf8");
for (const value of ["first-fixture-credential", "legacy-fixture-credential", "rotated-fixture-credential"]) expect(resaved).not.toContain(value);
});
it.each(["claude", "codex", "gemini", "kimi", "custom"])("defaults the legacy %s engine to full auto on fresh and resumed runs", async (agent) => {
const root = await makeTempRoot();
const config = {
@@ -597,6 +633,9 @@ describe("shared ACPX engine runtime behavior", () => {
const fresh = await runExecutor(config, { context });
expect(fresh.turnInputs).toHaveLength(1);
const prompt = String(fresh.turnInputs[0]?.text ?? "");
expect(prompt).toContain("You are agent agent-1");
expect(prompt).toContain("Connection tools:");
expect(prompt).not.toContain("Execution contract:");
expect(prompt).toContain(context.paperclipTaskMarkdownAssignment);
expect(prompt).toContain(context.paperclipTaskCommunicationGuidance);
expect(prompt).not.toContain('"objective":');
@@ -610,10 +649,18 @@ describe("shared ACPX engine runtime behavior", () => {
const resumed = await runExecutor(config, { context, runtime: { sessionParams: fresh.result.sessionParams } });
expect(resumed.sessionInputs[0]?.resumeSessionId).toBe(fresh.result.sessionId);
const resumedPrompt = String(resumed.turnInputs[0]?.text ?? "");
expect(resumedPrompt).not.toContain("Execution contract:");
expect(resumedPrompt).toContain(context.paperclipTaskMarkdownAssignmentCompact);
expect(resumedPrompt).not.toContain(context.paperclipTaskCommunicationGuidance);
expect(resumedPrompt).not.toContain('"id":"comment-first"');
expect(resumedPrompt).toContain('"id":"comment-second"');
const reset = await runExecutor(config, { context });
expect(reset.sessionInputs[0]?.resumeSessionId).toBeUndefined();
const resetPrompt = String(reset.turnInputs[0]?.text ?? "");
expect(resetPrompt).toContain("You are agent agent-1");
expect(resetPrompt).toContain("Connection tools:");
expect(resetPrompt).toContain(context.paperclipTaskMarkdownAssignment);
expect(resetPrompt).not.toContain("Execution contract:");
});
it("includes Paperclip env and API access notes in the ACPX prompt without leaking the token", async () => {
@@ -747,10 +794,10 @@ describe("shared ACPX engine runtime behavior", () => {
expect(prompt).not.toContain("Create child issues");
expect(prompt).not.toContain("Use child issues");
}
expect(String(fresh.meta[0]?.prompt)).toContain(custom ? "Custom agent instructions." : "Continue your Paperclip conversation");
expect(String(reset.meta[0]?.prompt)).toContain(custom ? "Custom agent instructions." : "Continue your Paperclip conversation");
expect(String(fresh.meta[0]?.prompt)).toContain(custom ? "Custom agent instructions." : "You are agent agent-1");
expect(String(reset.meta[0]?.prompt)).toContain(custom ? "Custom agent instructions." : "You are agent agent-1");
const ordinary = await runExecutor({ ...config, promptTemplate: "" }, { context: { ...context, conversationMode: false } });
expect(String(ordinary.meta[0]?.prompt)).toContain("Execution contract:");
expect(String(ordinary.meta[0]?.prompt)).not.toContain("Execution contract:");
expect(String(ordinary.meta[0]?.prompt)).toContain("Create child issues from the approved plan");
});
@@ -3826,9 +3873,54 @@ describe("ACPX engine Claude skill bundle staging (remote ACP lane)", () => {
};
}
it("advertises folded routing metadata and exact staged paths without injecting skill bodies", async () => {
const root = await makeTempRoot();
const cwd = path.join(root, "worktree");
const stateDir = path.join(root, "state");
await fs.mkdir(cwd, { recursive: true });
const skill = await createSkill(path.join(root, "sources"), "paperclip--versioned", [
"---", "description: >-", " Use for Paperclip-managed tasks", " and durable task documents.",
"---", "PRIVATE_SKILL_BODY_RECIPE", "",
].join("\n"));
const { meta, result } = await runExecutor({
agent: "claude", agentCommand: "node ./fake-acp.js", stateDir, cwd,
paperclipRuntimeSkills: [skill], paperclipSkillSync: { desiredSkills: [skill.key] },
});
const bundle = await onlyChildDir(path.join(stateDir, "runtime-skills", "claude"));
const stagedFile = path.join(bundle, ".claude", "skills", skill.runtimeName, "SKILL.md");
const prompt = String(meta[0]?.prompt ?? "");
expect(prompt).toContain(`- ${skill.runtimeName}: Use for Paperclip-managed tasks and durable task documents. (file: ${stagedFile})`);
expect(prompt).not.toContain("PRIVATE_SKILL_BODY_RECIPE");
expect(prompt).not.toContain(skill.source);
expect(result.sessionParams?.skills).toMatchObject({ selectedSkills: [skill.runtimeName] });
await expect(fs.readFile(stagedFile, "utf8")).resolves.toContain("PRIVATE_SKILL_BODY_RECIPE");
});
it("bounds routing descriptions and keeps files discoverable when metadata is absent or malformed", async () => {
const root = await makeTempRoot();
const cwd = path.join(root, "worktree");
const stateDir = path.join(root, "state");
await fs.mkdir(cwd, { recursive: true });
const sources = path.join(root, "sources");
const bounded = await createSkill(sources, "bounded", `---\ndescription: ${"x".repeat(600)}\n---\nBOUNDED_BODY\n`);
const absent = await createSkill(sources, "absent", "# ABSENT_BODY\n");
const malformed = await createSkill(sources, "malformed", "---\ndescription: incomplete header\nMALFORMED_BODY\n");
const skills = [bounded, absent, malformed];
const { meta } = await runExecutor({
agent: "claude", agentCommand: "node ./fake-acp.js", stateDir, cwd,
paperclipRuntimeSkills: skills, paperclipSkillSync: { desiredSkills: skills.map((skill) => skill.key) },
});
const prompt = String(meta[0]?.prompt ?? "");
expect(prompt).toContain(`- bounded: ${"x".repeat(512)} (file: `);
expect(prompt).not.toContain("x".repeat(513));
expect(prompt).toMatch(/- absent \(file: .*\/absent\/SKILL\.md\)/);
expect(prompt).toMatch(/- malformed \(file: .*\/malformed\/SKILL\.md\)/);
for (const body of ["BOUNDED_BODY", "ABSENT_BODY", "MALFORMED_BODY"]) expect(prompt).not.toContain(body);
});
it("rewrites the prompt and skill identity onto the in-sandbox skill root once the bundle is staged", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const skill = await createSkill(path.join(localCwd, "skills"), "review");
const skill = await createSkill(path.join(localCwd, "skills"), "review", "---\ndescription: Review requested task documents.\n---\n# review\nREMOTE_BODY_ONLY\n");
const { meta, result } = await runExecutor(
{
@@ -3851,6 +3943,8 @@ describe("ACPX engine Claude skill bundle staging (remote ACP lane)", () => {
const inSandboxSkillsRoot = prompt.match(/Skill root: (\S+)/)![1]!;
expect(inSandboxSkillsRoot).not.toBe(hostSkillsHome);
expect(prompt).toContain(`- review: Review requested task documents. (file: ${path.join(inSandboxSkillsRoot, "review", "SKILL.md")})`);
expect(prompt).not.toContain("REMOTE_BODY_ONLY");
await expect(
fs.readFile(path.join(inSandboxSkillsRoot, skill.runtimeName, "SKILL.md"), "utf8"),
).resolves.toContain("# review");
packages/adapter-utils/src/acpx-engine/execute.ts
+34 -36
View File
@@ -1,4 +1,5 @@
import { cancellableSandboxStartup } from "./startup-cancellation.js";
import { withAdapterExecutionPhase, type AdapterExecutionPhase } from "../execution-phase.js";
import fs from "node:fs/promises";
import fsSync from "node:fs";
import os from "node:os";
@@ -6,6 +7,7 @@ import path from "node:path";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { createHash, randomUUID } from "node:crypto";
import { parseFrontmatterMarkdown } from "@paperclipai/shared";
import { fileURLToPath } from "node:url";
import type {
AdapterBillingType,
@@ -84,6 +86,7 @@ import {
type PaperclipSkillEntry,
} from "@paperclipai/adapter-utils/server-utils";
import { shellQuote } from "@paperclipai/adapter-utils/ssh";
import { createEphemeralSessionEnvironmentStore } from "./ephemeral-session-environment.js";
import {
createAcpRuntime,
createAgentRegistry,
@@ -99,7 +102,6 @@ import {
type AcpRuntimeTurnResult,
type AcpRuntimeUsageBreakdown,
type AcpRuntimeUsageCost,
type AcpSessionStore,
} from "acpx/runtime";
import {
ACPX_DUPLEX_LOSS_CANCEL_DEADLINE_MS,
@@ -1182,11 +1184,24 @@ async function prepareClaudeSkillRuntime(input: {
}
const selectedNames = materializedNames.sort();
const skillDescriptions = await Promise.all(selectedNames.map(async (name) => {
const file = path.join(skillsHome, name, "SKILL.md");
let description = "";
try {
const parsed = parseFrontmatterMarkdown(await fs.readFile(file, "utf8"));
description = asString(parsed.frontmatter.description, "").replace(/\s+/g, " ").trim().slice(0, 512);
} catch {
// A readable skill without valid routing metadata remains available.
// Do not substitute its body for a missing description.
}
return `- ${name}${description ? `: ${description}` : ""} (file: ${file})`;
}));
const promptInstructions = selectedNames.length > 0
? [
"Paperclip has materialized selected runtime skills for this ACPX Claude session.",
`Skill root: ${skillsHome}`,
`Selected skills: ${selectedNames.join(", ")}`,
...skillDescriptions,
"When a task calls for one of these skills, read its SKILL.md from that root and follow it.",
].join("\n")
: "";
@@ -4079,9 +4094,9 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
: teardownErr instanceof Error
? teardownErr.message
: String(teardownErr);
await ctx
await withAdapterExecutionPhase(ctx, "phase_reporting", () => ctx
.onLog("stderr", `[paperclip] ACPX teardown step "${step}" failed: ${reason}\n`)
.catch(() => {});
.catch(() => {}));
};
// Emit one per-phase timing run-log event. It is not an OpenTelemetry
// export and it is not a Telemetry event: it carries the phase name
@@ -4092,15 +4107,15 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
emitRunPhaseTiming(ctx, phase, now() - startMs, outcome);
// Time a settlement step and emit its phase timing on every path. A step
// error still emits `failed` before it re-throws to the Phase 3 error policy.
const timedPhase = async (phase: string, run: () => Promise<void> | void): Promise<void> => {
const timedPhase = async (phase: AdapterExecutionPhase, run: () => Promise<void> | void): Promise<void> => {
const start = now();
try {
await run();
await withAdapterExecutionPhase(ctx, phase, run);
} catch (error) {
await emitPhase(phase, start, "failed");
await withAdapterExecutionPhase(ctx, "phase_reporting", () => emitPhase(phase, start, "failed"));
throw error;
}
await emitPhase(phase, start, "ok");
await withAdapterExecutionPhase(ctx, "phase_reporting", () => emitPhase(phase, start, "ok"));
};
// The turn the run started. It is hoisted to the run scope so the settlement
// `endSession` step can cancel a running turn before it closes the runtime
@@ -4248,26 +4263,9 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
flushChildStderr(childStderrState);
childStderrState.logPath = prepared.childStderrLogPath;
const persistedRuntimeStore = createRuntimeStore({ stateDir: prepared.stateDir });
const runtimeStore: AcpSessionStore = {
async load(id) {
const record = await persistedRuntimeStore.load(id);
if (!record) return undefined;
// ACPX resumes from the stored session options rather than the
// options passed to ensureSession. Keep conversation state, but
// launch the provider with this run's credentials and scratch paths.
return {
...record,
acpx: {
...record.acpx,
session_options: {
...record.acpx?.session_options,
env: { ...prepared.env },
},
},
};
},
save: (record) => persistedRuntimeStore.save(record),
};
// Resume with this run's launch environment; keep it out of the saved
// conversation record without mutating the live runtime's options.
const runtimeStore = createEphemeralSessionEnvironmentStore(persistedRuntimeStore, prepared.env);
const runtimeOptions: PaperclipAcpRuntimeOptions = {
cwd: prepared.cwd,
// Host-only spawn cwd for the relay proxy on the remote process-session
@@ -4860,7 +4858,7 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
if (ctx.signal?.aborted) armStopDeadline();
return {
cancel: async (reason: string) => {
await turn.cancel({ reason });
await withAdapterExecutionPhase(ctx, "cancel_turn", () => turn.cancel({ reason }));
},
};
};
@@ -5323,7 +5321,7 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
: baseSettlement;
// Cancel a running turn before the close (the turn-error path).
if (settlement.cancelTurnReason && activeTurn) {
await activeTurn.cancel({ reason: settlement.cancelTurnReason }).catch(() => {});
await withAdapterExecutionPhase(ctx, "cancel_turn", () => activeTurn!.cancel({ reason: settlement.cancelTurnReason! }).catch(() => {}));
}
const existing = warmHandles.get(prepared.sessionKey);
// Re-read the duplex control-channel disposition here, at the
@@ -5362,26 +5360,26 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
) {
// A matching warm entry closes through the warm store, which also
// clears its idle timer and flushes its child stderr.
runtimeStopConfirmed = await closeWarmHandle({
runtimeStopConfirmed = await withAdapterExecutionPhase(ctx, "close_session", () => closeWarmHandle({
handles: warmHandles,
key: prepared.sessionKey,
entry: existing,
reason: settlement.reason,
discardPersistentState: settlement.discardPersistentState,
});
}));
return;
}
const onCloseError = settlement.recordCloseError || ctx.signal?.aborted
? (closeErr: unknown) => recordTeardownError("runtime-close", closeErr)
: () => {};
await runtime
await withAdapterExecutionPhase(ctx, "close_session", () => runtime
.close({
handle: settlement.handle,
reason: settlement.reason,
discardPersistentState: settlement.discardPersistentState,
})
.then(() => { runtimeStopConfirmed = true; })
.catch(onCloseError);
.catch(onCloseError));
if (settlement.dropWarmEntry && warmHandleMatches(existing, runtime, settlement.handle) && existing) {
clearWarmHandleTimer(existing);
warmHandles.delete(prepared.sessionKey);
@@ -5412,18 +5410,18 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
// mutable provider files. Require the owned close or local OS handle.
const stopped = runtimeStopConfirmed || (!prepared.processSessionBridge && capturedProcessExited(processIdentitySink?.localProcess));
try {
if (stopped) await ctx.onProviderStopped?.();
if (stopped) await withAdapterExecutionPhase(ctx, "instruction_collection", () => ctx.onProviderStopped?.());
} catch {
// Match ACP's fail-soft teardown policy without describing this as
// a workspace restore failure or exposing paths from a raw error.
await recordTeardownError("instruction-collection", new Error("Instruction collection failed after provider stop. No instruction save is claimed."));
} finally {
await runRuntimeSpan("sandbox.syncBack", async () => {
await withAdapterExecutionPhase(ctx, "workspace_restore", () => runRuntimeSpan("sandbox.syncBack", async () => {
const restoreOutcome = await syncBackManagedHome(prepared);
if (!restoreOutcome.ok) {
workspaceRestoreFailureField = { workspaceRestoreFailure: restoreOutcome.code };
}
});
}));
}
}),
// The staging lease releases as the run's final act, AFTER the coordinator
packages/adapter-utils/src/acpx-engine/settlement-characterization.test.ts
+104
View File
@@ -292,6 +292,110 @@ describe("ACP settlement — Layer A: engine teardown orchestration", () => {
vi.clearAllMocks();
});
it.each(["close_session", "stop_transport", "instruction_collection", "workspace_restore", "phase_reporting", "close_error_reporting"])(
"keeps %s visible while its real settlement await is stalled",
async (blockedPhase) => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
let resume!: () => void;
let entered!: () => void;
const gate = new Promise<void>((resolve) => { resume = resolve; });
const reached = new Promise<void>((resolve) => { entered = resolve; });
const block = () => { entered(); return gate; };
const scopes = new Map<symbol, string>();
if (blockedPhase === "stop_transport") {
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementation(async () => ({ env: {}, stop: block }) as never);
}
const execute = createAcpxEngineExecutor({
createRuntime: () => ({
ensureSession: async () => okHandle,
startTurn: () => blockedPhase === "close_error_reporting" ? throwingTurn() : completedTurn(),
close: blockedPhase === "close_session" ? block : async () => {
if (blockedPhase === "close_error_reporting") throw new Error("close fixture failed");
},
}) as never,
prepareRemoteManagedHome: async (input) => ({
stagedRuntime: await input.stage([]),
teardown: async () => {
if (blockedPhase === "workspace_restore") await block();
return { ok: true };
},
}),
});
const execution = execute({
runId: "pending-settlement-fixture",
...remoteArgs(stateDir, localCwd, executionTarget),
onProviderStopped: blockedPhase === "instruction_collection" ? block : async () => {},
onExecutionPhase: (phase: string) => {
const token = Symbol();
scopes.set(token, phase);
return () => { scopes.delete(token); };
},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
if (blockedPhase === "phase_reporting" && event.eventType === "run.phase.timing" && event.payload?.phase === "end_session") await block();
},
onLog: async (_stream: string, text: string) => {
if (blockedPhase === "close_error_reporting" && text.includes("close fixture failed")) await block();
},
} as never);
try {
await reached;
expect([...scopes.values()].at(-1)).toBe(blockedPhase === "close_error_reporting" ? "phase_reporting" : blockedPhase);
} finally {
resume();
expect((await execution).exitCode).toBe(blockedPhase === "close_error_reporting" ? 1 : 0);
}
expect(scopes.size).toBe(0);
},
);
it("keeps the primary Stop cancellation visible until its await settles", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
const controller = new AbortController();
let started!: () => void;
let cancelled!: () => void;
let resume!: () => void;
const turnStarted = new Promise<void>((resolve) => { started = resolve; });
const cancellationStarted = new Promise<void>((resolve) => { cancelled = resolve; });
const gate = new Promise<void>((resolve) => { resume = resolve; });
const scopes = new Map<symbol, string>();
const execute = createAcpxEngineExecutor({
createRuntime: () => ({
ensureSession: async () => okHandle,
startTurn: () => {
started();
return {
events: (async function* () { await gate; })(),
result: gate.then(() => ({ status: "cancelled", stopReason: "cancelled" })),
cancel: async () => { cancelled(); await gate; },
};
},
close: async () => {},
}) as never,
});
const execution = execute({
runId: "pending-cancel-fixture",
...remoteArgs(stateDir, localCwd, executionTarget),
signal: controller.signal,
onExecutionPhase: (phase: string) => {
const token = Symbol();
scopes.set(token, phase);
return () => { scopes.delete(token); };
},
} as never);
try {
await turnStarted;
controller.abort();
await cancellationStarted;
expect([...scopes.values()].at(-1)).toBe("cancel_turn");
} finally {
resume();
expect((await execution).exitCode).toBe(1);
}
expect(scopes.size).toBe(0);
});
it("test_clean_completed_remote_teardown_runs_bridge_stop_then_sync_back_then_lease_release", async () => {
// cleanupRemoteBridges (execute.ts:2306-2326) fixes the sub-order for a clean
// exit: stop both bridges (allSettled) → run the managed-home sync-back
packages/adapter-utils/src/execution-phase.test.ts
+26
View File
@@ -0,0 +1,26 @@
import { expect, it, vi } from "vitest";
import { withAdapterExecutionPhase } from "./execution-phase.js";
it("enters before a stalled await and releases only when that operation settles", async () => {
const release = vi.fn();
const enter = vi.fn(() => release);
let resolve!: (value: number) => void;
const operation = new Promise<number>((done) => { resolve = done; });
const result = withAdapterExecutionPhase({ onExecutionPhase: enter }, "instruction_collection", () => operation);
expect(enter).toHaveBeenCalledWith("instruction_collection");
await Promise.resolve();
expect(release).not.toHaveBeenCalled();
resolve(7);
await expect(result).resolves.toBe(7);
expect(release).toHaveBeenCalledOnce();
});
it.each(["enter", "release"])("preserves values and errors when the %s diagnostic callback throws", async (where) => {
const context = { onExecutionPhase: () => {
if (where === "enter") throw new Error("diagnostic failure");
return () => { throw new Error("diagnostic failure"); };
} };
await expect(withAdapterExecutionPhase(context, "workspace_restore", () => 3)).resolves.toBe(3);
const failure = new Error("operation failure");
await expect(withAdapterExecutionPhase(context, "workspace_restore", () => { throw failure; })).rejects.toBe(failure);
});
packages/adapter-utils/src/execution-phase.ts
+28
View File
@@ -0,0 +1,28 @@
/** Content-free, in-memory diagnostic scopes. These never establish stop proof. */
export const ADAPTER_EXECUTION_PHASES = [
"host_execution", "adapter_execution", "end_session", "cancel_turn", "close_session",
"settle_reuse", "stop_transport", "sync_back", "instruction_collection",
"workspace_restore", "phase_reporting", "instruction_cleanup", "lease_release",
] as const;
export type AdapterExecutionPhase = (typeof ADAPTER_EXECUTION_PHASES)[number];
export type AdapterExecutionPhaseSink = (phase: AdapterExecutionPhase) => (() => void) | void;
export function isAdapterExecutionPhase(value: unknown): value is AdapterExecutionPhase {
return ADAPTER_EXECUTION_PHASES.some((phase) => phase === value);
}
/** A broken diagnostic sink must not change execution or teardown outcomes. */
export async function withAdapterExecutionPhase<T>(
context: { onExecutionPhase?: AdapterExecutionPhaseSink },
phase: AdapterExecutionPhase,
operation: () => T | Promise<T>,
): Promise<T> {
let release: (() => void) | void = undefined;
try { release = context.onExecutionPhase?.(phase); } catch { /* diagnostics only */ }
try {
return await operation();
} finally {
try { release?.(); } catch { /* diagnostics only */ }
}
}
packages/adapter-utils/src/prompt-sections.test.ts
+7 -6
View File
@@ -3,13 +3,14 @@ import { selectPaperclipPromptSections as selectSections } from "./server-utils.
import { createPromptContextFixture } from "./test-fixtures/prompt-context.js";
describe("task and event section ownership", () => {
it("lets a separate instruction carrier own the execution contract on a resumed turn", () => {
it("preserves resumed wake data without reintroducing generic procedures", () => {
const context = createPromptContextFixture();
const sections = selectSections(context, { resumedSession: true, includeExecutionContract: false });
expect(sections.taskContextNote).toBe(context.paperclipTaskMarkdownAssignmentCompact);
expect(sections.wakePrompt).toContain('"id":"comment-second"');
expect(sections.wakePrompt).not.toContain("Execution contract:");
expect(selectSections(context, { resumedSession: true }).wakePrompt).toContain("Execution contract:");
for (const includeExecutionContract of [undefined, false, true]) {
const sections = selectSections(context, { resumedSession: true, includeExecutionContract });
expect(sections.taskContextNote).toBe(context.paperclipTaskMarkdownAssignmentCompact);
expect(sections.wakePrompt).toContain('"id":"comment-second"');
expect(sections.wakePrompt).not.toContain("Execution contract:");
}
});
it("preserves user repetition and distinct same-body comments under their source owners", () => {
packages/adapter-utils/src/server-utils.test.ts
+30 -104
View File
@@ -925,7 +925,7 @@ describe("renderPaperclipWakePrompt", () => {
fallbackFetchNeeded: false,
};
const ordinary = renderPaperclipWakePrompt(payload, { resumedSession: true });
expect(ordinary).toContain("Execution contract:");
expect(ordinary).not.toContain("Execution contract:");
expect(ordinary).toContain("Create child issues from the approved plan");
for (const resumedSession of [false, true]) {
const chat = renderPaperclipWakePrompt(payload, {
@@ -1171,7 +1171,7 @@ describe("renderPaperclipWakePrompt", () => {
expect(incompleteResumePrompt).toContain(
"[continuation summary truncated]",
);
expect(incompleteResumePrompt).toContain(
expect(incompleteResumePrompt).not.toContain(
"a successful process exit or final response is not sufficient",
);
});
@@ -1410,76 +1410,18 @@ describe("renderPaperclipWakePrompt", () => {
);
});
it("keeps the default local-agent prompt action-oriented", () => {
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Start actionable work in this heartbeat",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"do not stop at a plan",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"clear final disposition",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"evidence, not valid liveness paths by themselves",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"keep `in_progress` only when a live continuation path exists",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Prefer the smallest verification that proves the change",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"After 2 consecutive failures of the same control-plane write",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"adapter/runtime status channel as the sanctioned fallback",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Use child issues",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"instead of polling agents, sessions, or processes",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Create child issues directly when you know what needs to be done",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"POST /api/issues/$PAPERCLIP_TASK_ID/interactions",
);
// URL paths in prompt text carry real ids or env vars, never brace
// placeholders: agents paste these lines verbatim, and a literal {issueId}
// reaches the server as /api/issues/%7BissueId%7D and 404s.
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).not.toContain(
"/api/issues/{issueId}",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).not.toContain(
"/api/issues/{id}",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"kind suggest_tasks, ask_user_questions, or request_confirmation",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Use continuationPolicy wake_assignee when you need to resume after a response (it wakes on acceptance and rejection alike; only expiry does not wake); use wake_assignee_on_accept when you want to resume only after acceptance",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).not.toContain(
"for request_confirmation this resumes only after acceptance",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Never create probe or throwaway issue-thread interactions to discover the interactions API shape or your permissions",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"confirmation:{issueId}:plan:{revisionId}",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Wait for acceptance before creating implementation subtasks",
);
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
"Respect budget, pause/cancel, approval gates, and company boundaries",
);
it("keeps task and chat defaults to identity and connection guidance", () => {
for (const template of [
DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE,
DEFAULT_PAPERCLIP_CONVERSATION_PROMPT_TEMPLATE,
]) {
expect(template).toBe(
`You are agent {{agent.id}} ({{agent.name}}).\n\n${CONNECTION_INTENT_AGENT_GUIDANCE}`,
);
}
});
it("leaves the execution contract to the heartbeat template on fresh scoped wake prompts", () => {
it("keeps current task data without generic procedures on fresh scoped wake prompts", () => {
const prompt = renderPaperclipWakePrompt({
reason: "issue_assigned",
issue: {
@@ -1499,12 +1441,12 @@ describe("renderPaperclipWakePrompt", () => {
expect(prompt).toContain("## Paperclip Wake Payload");
expect(prompt).not.toContain("Execution contract:");
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).toContain(
expect(DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE).not.toContain(
"Execution contract:",
);
});
it("adds the execution contract to resume delta prompts and opted-in fresh prompts", () => {
it("does not restore generic procedures on resume or with the legacy opt-in", () => {
const payload = {
reason: "issue_assigned",
issue: {
@@ -1522,36 +1464,19 @@ describe("renderPaperclipWakePrompt", () => {
fallbackFetchNeeded: false,
};
for (const prompt of [
renderPaperclipWakePrompt(payload, { resumedSession: true }),
renderPaperclipWakePrompt(payload, { includeExecutionContract: true }),
]) {
expect(prompt).toContain(
"Execution contract: take concrete action in this heartbeat",
);
expect(prompt).toContain("clear final disposition");
expect(prompt).toContain(
"Immediately before returning, verify that Paperclip records one of those dispositions",
);
expect(prompt).toContain(
"a successful process exit or final response is not sufficient",
);
expect(prompt).toContain(
"If no valid disposition is recorded, record it now and do not end the run",
);
expect(prompt).toContain(
"After 2 consecutive failures of the same control-plane write",
);
expect(prompt).toContain(
"adapter/runtime status channel as the sanctioned fallback",
);
expect(prompt).toContain(
"evidence, not valid liveness paths by themselves",
);
expect(prompt).toContain(
"Use child issues for long or parallel delegated work instead of polling",
);
expect(prompt).toContain("named unblock owner/action");
for (const resumedSession of [false, true]) {
for (const includeExecutionContract of [undefined, false, true]) {
const prompt = renderPaperclipWakePrompt(payload, {
resumedSession, includeExecutionContract,
});
expect(prompt).toContain(resumedSession ? "## Paperclip Resume Delta" : "## Paperclip Wake Payload");
expect(prompt).toContain("- reason: issue_assigned");
expect(prompt).toContain("- issue: PAP-1580 Update prompts");
expect(prompt).toContain("- issue status: in_progress");
expect(prompt).not.toContain("Execution contract:");
expect(prompt).not.toContain("clear final disposition");
expect(prompt).not.toContain("do not end the run");
}
}
});
@@ -1947,7 +1872,7 @@ describe("renderPaperclipWakePrompt", () => {
);
});
it("keeps exactly one execution contract in a composed fresh heartbeat prompt", () => {
it("keeps connection guidance without a generic manual in a composed fresh prompt", () => {
const wakePrompt = renderPaperclipWakePrompt({
reason: "issue_assigned",
issue: {
@@ -1967,7 +1892,8 @@ describe("renderPaperclipWakePrompt", () => {
const composed = [wakePrompt, DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE].join(
"\n\n",
);
expect(composed.match(/Execution contract/g)).toHaveLength(1);
expect(composed).toContain(CONNECTION_INTENT_AGENT_GUIDANCE);
expect(composed).not.toContain("Execution contract:");
});
it("trims comment-batch boilerplate on fresh wakes with zero pending comments", () => {
packages/adapter-utils/src/server-utils.ts
+11 -46
View File
@@ -208,41 +208,15 @@ export function resolvePaperclipInstanceRootForAdapter(
}
export const DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE = [
"You are agent {{agent.id}} ({{agent.name}}). Continue your Paperclip work.",
"",
"Execution contract:",
"- Start actionable work in this heartbeat; do not stop at a plan unless the issue asks for planning.",
"- Leave durable progress in comments, documents, or work products, then update the issue to a clear final disposition before ending the heartbeat.",
"- Comments, documents, screenshots, work products, and `Remaining` bullets are evidence, not valid liveness paths by themselves.",
"- Final disposition checklist: mark `done` when complete; use `in_review` only with a real reviewer, approval, interaction, or monitor path; use `blocked` only with first-class blockers or a named unblock owner/action; create delegated follow-up issues with blockers when another agent owns the next step; keep `in_progress` only when a live continuation path exists.",
"- Prefer the smallest verification that proves the change; do not default to full workspace typecheck/build/test on every heartbeat unless the task scope warrants it.",
"- After 2 consecutive failures of the same control-plane write, stop retrying that write for the rest of the heartbeat. Continue useful work, report the failure in the final response, and rely on the adapter/runtime status channel as the sanctioned fallback.",
"- Use child issues for parallel or long delegated work instead of polling agents, sessions, or processes.",
"- If woken by a human comment on a dependency-blocked issue, respond or triage the comment without treating the blocked deliverable work as unblocked.",
"- Create child issues directly when you know what needs to be done; use issue-thread interactions when the board/user must choose suggested tasks, answer structured questions, or confirm a proposal.",
"- Use `PAPERCLIP_SCRATCH_DIR` / `PAPERCLIP_RUN_SCRATCH_DIR` for temporary scratch files instead of ad hoc `/tmp` paths; Paperclip removes that run-owned directory after the run ends.",
"- To ask for that input, create an interaction on the current issue with POST /api/issues/$PAPERCLIP_TASK_ID/interactions using kind suggest_tasks, ask_user_questions, or request_confirmation. Use continuationPolicy wake_assignee when you need to resume after a response (it wakes on acceptance and rejection alike; only expiry does not wake); use wake_assignee_on_accept when you want to resume only after acceptance.",
"- Never create probe or throwaway issue-thread interactions to discover the interactions API shape or your permissions; schema discovery goes through the OpenAPI spec and explicit validation errors, not placeholder cards. Every ask_user_questions, suggest_tasks, or request_confirmation you post must carry a real, answerable prompt; withdraw one you no longer need instead of leaving it pending.",
"- When you intentionally restart follow-up work on a completed assigned issue, include structured `resume: true` with the POST /api/issues/$PAPERCLIP_TASK_ID/comments or PATCH /api/issues/$PAPERCLIP_TASK_ID comment payload (substitute that issue's real id when it is not the current task). Generic agent comments on closed issues are inert by default.",
"- For plan approval, update the plan document first, then create request_confirmation targeting the latest plan revision with idempotencyKey confirmation:{issueId}:plan:{revisionId}. Wait for acceptance before creating implementation subtasks, and create a fresh confirmation after superseding board/user comments if approval is still needed.",
"- If blocked, mark the issue blocked and name the unblock owner and action.",
"- Respect budget, pause/cancel, approval gates, and company boundaries.",
"- When the server-authenticated wake payload includes an External chat response contract, that narrower contract replaces the generic Paperclip comment, status, checkout, and final-disposition steps above for that turn. Follow the external-chat contract exactly; it does not relax any permission, approval, execution-policy, containment, budget, pause/cancel, or company boundary.",
"You are agent {{agent.id}} ({{agent.name}}).",
"",
CONNECTION_INTENT_AGENT_GUIDANCE,
].join("\n");
// Chat behavior is supplied centrally by the server's task-context markdown.
// Keep the ordinary task's completion/delegation contract out of this template.
export const DEFAULT_PAPERCLIP_CONVERSATION_PROMPT_TEMPLATE = [
"You are agent {{agent.id}} ({{agent.name}}). Continue your Paperclip conversation using the supplied chat mode directive.",
"Use available tools and assigned skills as needed; respect budget, pause/cancel, approval gates, and company boundaries.",
"Prefer the smallest verification that proves the action. Use PAPERCLIP_SCRATCH_DIR / PAPERCLIP_RUN_SCRATCH_DIR for temporary scratch files.",
"After 2 consecutive failures of the same control-plane write, stop retrying that write for the rest of the turn. Report the failure honestly; never claim an unconfirmed mutation succeeded.",
"Never create probe or throwaway issue-thread interactions. Every interaction must carry a real, answerable prompt; withdraw one you no longer need.",
"",
CONNECTION_INTENT_AGENT_GUIDANCE,
].join("\n");
// Task/chat modes arrive in server-owned context; operational procedures live
// in the harness-delivered Paperclip skill rather than either default template.
export const DEFAULT_PAPERCLIP_CONVERSATION_PROMPT_TEMPLATE =
DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE;
export const WATCHDOG_DEFAULT_MANDATE = [
"You are running as a task watchdog, not as the original deliverable worker.",
@@ -2242,6 +2216,7 @@ export function selectPaperclipPromptSections(
options: {
resumedSession?: boolean;
includeCommunicationGuidance?: boolean;
/** @deprecated Generic execution instructions are no longer injected. */
includeExecutionContract?: boolean;
nativeWakeReaderAvailable?: boolean;
} = {},
@@ -2277,6 +2252,7 @@ function renderPaperclipWakePromptBody(
value: unknown,
options: {
resumedSession?: boolean;
/** @deprecated Generic execution instructions are no longer injected. */
includeExecutionContract?: boolean;
// Conversation policy arrives in the server-owned task markdown. Generic
// task disposition and child-delegation instructions conflict with it.
@@ -2301,12 +2277,6 @@ function renderPaperclipWakePromptBody(
externalChatTurn ||
externalChatReaderTurn ||
externalChatQuestionResponseTurn;
// The heartbeat prompt template already carries the execution contract on
// fresh sessions; only resume deltas (which replace the template) and
// template-less adapters need the wake-payload copy. An explicit false means
// another delivery carrier owns the contract, including on resume.
const includeExecutionContract = options.conversationMode !== true &&
(options.includeExecutionContract ?? resumedSession);
const hasWakeCommentBatch =
normalized.comments.length > 0 ||
normalized.includedCount > 0 ||
@@ -2397,7 +2367,7 @@ function renderPaperclipWakePromptBody(
}
};
const executionContractLines = externalChatContract
const wakeContractLines = externalChatContract
? [
externalChatQuestionResponseTurn
? "## External chat answered-question contract"
@@ -2444,12 +2414,7 @@ function renderPaperclipWakePromptBody(
`Fallback preference order: (1) send back to ${originalAssigneeLabel} with a retry instruction; (2) fix the runtime/adapter/workspace problem, then send it back; (3) reassign to another agent with the right specialty; (4) convert to an explicit manual-review state for the board.`,
"",
]
: includeExecutionContract
? [
"Execution contract: take concrete action in this heartbeat when the issue is actionable; do not stop at a plan unless planning was requested. Leave durable progress and then give the issue a clear final disposition before ending the heartbeat: `done`, `in_review` with a real reviewer/approval/interaction path, `blocked` with first-class blockers or a named unblock owner/action, delegated follow-up issues with blockers, or `in_progress` only when a live continuation path exists. Immediately before returning, verify that Paperclip records one of those dispositions; a successful process exit or final response is not sufficient. If no valid disposition is recorded, record it now and do not end the run. After 2 consecutive failures of the same control-plane write, stop retrying it for the rest of the heartbeat, continue useful work, report the failure in the final response, and rely on the adapter/runtime status channel as the sanctioned fallback. Use child issues for long or parallel delegated work instead of polling. Comments, documents, screenshots, work products, and `Remaining` bullets are evidence, not valid liveness paths by themselves.",
"",
]
: [];
: [];
const wakeSummaryLines = [
`- reason: ${normalized.reason ?? "unknown"}`,
`- issue: ${normalized.issue?.identifier ?? normalized.issue?.id ?? "unknown"}${normalized.issue?.title ? ` ${normalized.issue.title}` : ""}`,
@@ -2507,7 +2472,7 @@ function renderPaperclipWakePromptBody(
]),
"",
...externalInteractionContinuationLines,
...executionContractLines,
...wakeContractLines,
...wakeSummaryLines,
]
: [
@@ -2537,7 +2502,7 @@ function renderPaperclipWakePromptBody(
: []),
"",
...externalInteractionContinuationLines,
...executionContractLines,
...wakeContractLines,
...wakeSummaryLines,
];
packages/adapter-utils/src/types.ts
+3
View File
@@ -5,6 +5,7 @@
import type { SshRemoteExecutionSpec } from "./ssh.js";
import type { AdapterExecutionTarget } from "./execution-target.js";
import type { RuntimeStatusSink } from "./runtime-progress.js";
import type { AdapterExecutionPhaseSink } from "./execution-phase.js";
import type { ExecutionContinuationEnvelope, NativeFinalizationResult } from "@paperclipai/shared";
export interface AdapterAgent {
@@ -228,6 +229,8 @@ export interface AdapterRuntimeEvent {
}
export interface AdapterExecutionContext {
/** Synchronous, content-free diagnostic scope; never stop or collection authority. */
onExecutionPhase?: AdapterExecutionPhaseSink;
/** Run-scoped operator cancellation; adapters must settle before returning. */
signal?: AbortSignal;
/** Opt in to signal-based cancellation before starting provider work. */
packages/adapters/cursor-cloud/src/server/execute.test.ts
+4 -1
View File
@@ -164,7 +164,7 @@ describe("cursor_cloud execute", () => {
expect(result.exitCode).toBe(0);
const prompt = String(sdkAgent.send.mock.calls[0]?.[0]);
expect(prompt).toContain(directive);
expect(prompt).toContain(custom ? "Do the work for" : "Continue your Paperclip conversation");
expect(prompt).toContain(custom ? "Do the work for" : "You are agent agent-1");
expect(prompt).not.toContain("Execution contract:");
expect(prompt).not.toContain("Create child issues");
});
@@ -178,6 +178,9 @@ describe("cursor_cloud execute", () => {
expect(result.exitCode).toBe(0);
const prompt = String(sdkAgent.send.mock.calls[0]?.[0]);
expect(prompt).toContain("## Owned assignment");
expect(prompt).toContain("You are agent agent-1 (Cursor Cloud Agent).");
expect(prompt).toContain("Connection tools:");
expect(prompt).not.toContain("Execution contract:");
expect(prompt.indexOf("Append the same ledger entry.")).toBeLessThan(prompt.indexOf("Change the final scope to the launch checklist."));
expect(prompt.split("Append the same ledger entry.")).toHaveLength(3);
});
packages/adapters/hermes/src/server/prompt-rendering.test.ts
+3 -3
View File
@@ -82,7 +82,7 @@ test("renders standard assignment wake with task authority and no backlog discov
expect(prompt).toContain("Paperclip task context:");
expect(prompt).toContain("Add focused unit tests for assignment wake and custom prompt rendering.");
expect(prompt).toContain("The harness already checked out this issue for the current run.");
expect(prompt).toContain("clear final disposition");
expect(prompt).not.toContain("clear final disposition");
expect(prompt).not.toContain("check for unassigned issues");
expect(prompt).not.toContain("status=backlog");
});
@@ -130,8 +130,8 @@ test("renders scoped planning wake authority before the Hermes default workflow"
expect(prompt).toContain("- checkout: already claimed by the harness for this run");
expect(prompt).toContain("The harness already checked out this issue for the current run.");
expect(prompt).toContain("Issue description:\n```text\nUse the wake payload as runtime authority.\n```");
expect(prompt).toContain("clear final disposition");
expect(prompt).toContain("keep `in_progress` only when a live continuation path exists");
expect(prompt).not.toContain("clear final disposition");
expect(prompt).not.toContain("keep `in_progress` only when a live continuation path exists");
expect(prompt).not.toContain("check for unassigned issues");
expect(prompt).not.toContain("status=backlog");
});
packages/adapters/opencode-local/src/server/execute.test.ts
+4 -1
View File
@@ -77,7 +77,7 @@ describe("OpenCode local skill injection", () => {
});
expect(result.exitCode).toBe(0);
expect(prompt).toContain(directive);
expect(prompt).toContain(custom ? "Custom agent instruction." : "Continue your Paperclip conversation");
expect(prompt).toContain(custom ? "Custom agent instruction." : "You are agent agent-1");
expect(prompt).not.toContain("Execution contract:");
expect(prompt).not.toContain("Create child issues");
});
@@ -102,6 +102,9 @@ describe("OpenCode local skill injection", () => {
expect(prompts).toHaveLength(2);
expect(prompts[0]).toContain("## Compact assignment");
expect(prompts[1]).toContain("## Owned assignment");
for (const prompt of prompts) expect(prompt).not.toContain("Execution contract:");
expect(prompts[1]).toContain("You are agent agent-1 (OpenCode).");
expect(prompts[1]).toContain("Connection tools:");
});
it("injects runtime skills into the configured child HOME", async () => {
packages/adapters/pi-local/src/server/execute.remote.test.ts
+8 -4
View File
@@ -696,7 +696,7 @@ describe("pi remote execution", () => {
expect(userPrompt).toBe("CUSTOM POLICY run-custom-policy");
});
it("keeps the resumed default execution contract in the system carrier only", async () => {
it("keeps resumed default identity and connection guidance in the system carrier", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-pi-resumed-policy-"));
cleanupDirs.push(rootDir);
const sessionPath = path.join(rootDir, "session.jsonl");
@@ -726,11 +726,13 @@ describe("pi remote execution", () => {
const args = call?.[2] ?? [];
const systemPrompt = args[args.indexOf("--append-system-prompt") + 1] ?? "";
const userPrompt = args.at(-1) ?? "";
expect(systemPrompt).toContain("Execution contract:");
expect(systemPrompt).toContain("You are agent agent-1 (Pi Builder).");
expect(systemPrompt).toContain("Connection tools:");
expect(systemPrompt).not.toContain("Execution contract:");
expect(userPrompt).not.toContain("Execution contract:");
});
it("keeps the default contract in system input when custom prompt uses loaded instructions", async () => {
it("keeps default identity in system input when custom prompt uses loaded instructions", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-pi-instructions-policy-"));
cleanupDirs.push(rootDir);
const sessionPath = path.join(rootDir, "session.jsonl");
@@ -769,7 +771,9 @@ describe("pi remote execution", () => {
const systemPrompt = args[args.indexOf("--append-system-prompt") + 1] ?? "";
const userPrompt = args.at(-1) ?? "";
expect(systemPrompt).toContain("Loaded instructions for this run.");
expect(systemPrompt).toContain("Execution contract:");
expect(systemPrompt).toContain("You are agent agent-1 (Pi Builder).");
expect(systemPrompt).toContain("Connection tools:");
expect(systemPrompt).not.toContain("Execution contract:");
expect(userPrompt).not.toContain("CUSTOM POLICY run-instructions-policy");
expect(userPrompt).not.toContain("Execution contract:");
});
packages/paperclip-runner/docs/capability-contract.md
+90 -87
View File
@@ -8,9 +8,9 @@ The skill/reference inventory and eval cases are the only normative behavior sou
## Baseline Counts
- Skill/reference headings: 157
- Skill/reference headings: 160
- Eval cases: 106 across 16 groups
- Total normative rows: 263
- Total normative rows: 266
- Legacy MCP aliases folded into normative rows: 42
| Eval group | Cases |
@@ -43,91 +43,35 @@ The skill/reference inventory and eval cases are the only normative behavior sou
| --- | --- | --- |
| skill:skills/paperclip/SKILL.md:paperclip-skill:10 | optional_agent_tool | skills/paperclip/SKILL.md:10 |
| skill:skills/paperclip/SKILL.md:terminology:14 | optional_agent_tool | skills/paperclip/SKILL.md:14 |
| skill:skills/paperclip/SKILL.md:authentication:18 | control_plane_owned | skills/paperclip/SKILL.md:18 |
| skill:skills/paperclip/SKILL.md:conversation-tasks:30 | optional_agent_tool | skills/paperclip/SKILL.md:30 |
| skill:skills/paperclip/SKILL.md:server-verified-external-chat-turns:47 | control_plane_owned | skills/paperclip/SKILL.md:47 |
| skill:skills/paperclip/SKILL.md:the-heartbeat-procedure:87 | optional_agent_tool | skills/paperclip/SKILL.md:87 |
| skill:skills/paperclip/SKILL.md:generated-artifacts-and-work-products:196 | always_agent_tool | skills/paperclip/SKILL.md:196 |
| skill:skills/paperclip/SKILL.md:status-quick-guide:244 | control_plane_owned | skills/paperclip/SKILL.md:244 |
| skill:skills/paperclip/SKILL.md:monitors-and-watchers-say-only-what-you-actually-scheduled:254 | optional_agent_tool | skills/paperclip/SKILL.md:254 |
| skill:skills/paperclip/SKILL.md:delegating-review-tasks:267 | always_agent_tool | skills/paperclip/SKILL.md:267 |
| skill:skills/paperclip/SKILL.md:managing-a-user-s-inbox:278 | control_plane_owned | skills/paperclip/SKILL.md:278 |
| skill:skills/paperclip/SKILL.md:issue-dependencies-blockers:286 | control_plane_owned | skills/paperclip/SKILL.md:286 |
| skill:skills/paperclip/SKILL.md:requesting-board-approval:311 | optional_agent_tool | skills/paperclip/SKILL.md:311 |
| skill:skills/paperclip/SKILL.md:issue-thread-interactions:332 | optional_agent_tool | skills/paperclip/SKILL.md:332 |
| skill:skills/paperclip/SKILL.md:standalone-decisions:361 | optional_agent_tool | skills/paperclip/SKILL.md:361 |
| skill:skills/paperclip/SKILL.md:mcp-tool-approval-gates:465 | optional_agent_tool | skills/paperclip/SKILL.md:465 |
| skill:skills/paperclip/SKILL.md:niche-workflow-pointers:507 | optional_agent_tool | skills/paperclip/SKILL.md:507 |
| skill:skills/paperclip/SKILL.md:cases:517 | optional_agent_tool | skills/paperclip/SKILL.md:517 |
| skill:skills/paperclip/SKILL.md:company-skills-workflow:522 | optional_agent_tool | skills/paperclip/SKILL.md:522 |
| skill:skills/paperclip/SKILL.md:routines:533 | optional_agent_tool | skills/paperclip/SKILL.md:533 |
| skill:skills/paperclip/SKILL.md:issue-workspace-runtime-controls:544 | optional_agent_tool | skills/paperclip/SKILL.md:544 |
| skill:skills/paperclip/SKILL.md:proposing-credentials-safely:551 | optional_agent_tool | skills/paperclip/SKILL.md:551 |
| skill:skills/paperclip/SKILL.md:reading-granted-secrets:558 | optional_agent_tool | skills/paperclip/SKILL.md:558 |
| skill:skills/paperclip/SKILL.md:critical-rules:584 | optional_agent_tool | skills/paperclip/SKILL.md:584 |
| skill:skills/paperclip/SKILL.md:comment-style-required:605 | always_agent_tool | skills/paperclip/SKILL.md:605 |
| skill:skills/paperclip/SKILL.md:update:637 | optional_agent_tool | skills/paperclip/SKILL.md:637 |
| skill:skills/paperclip/SKILL.md:planning-required-when-planning-requested:647 | optional_agent_tool | skills/paperclip/SKILL.md:647 |
| skill:skills/paperclip/SKILL.md:key-endpoints-hot-routes:680 | optional_agent_tool | skills/paperclip/SKILL.md:680 |
| skill:skills/paperclip/SKILL.md:searching-issues:709 | optional_agent_tool | skills/paperclip/SKILL.md:709 |
| skill:skills/paperclip/SKILL.md:full-reference:719 | optional_agent_tool | skills/paperclip/SKILL.md:719 |
| skill:skills/paperclip/SKILL.md:conversational-confirmation-answers:723 | always_agent_tool | skills/paperclip/SKILL.md:723 |
| skill:skills/paperclip/references/artifacts.md:generated-artifacts-and-work-products:1 | always_agent_tool | skills/paperclip/references/artifacts.md:1 |
| skill:skills/paperclip/references/artifacts.md:workspace-only-file-references:15 | optional_agent_tool | skills/paperclip/references/artifacts.md:15 |
| skill:skills/paperclip/references/cases.md:cases:1 | optional_agent_tool | skills/paperclip/references/cases.md:1 |
| skill:skills/paperclip/references/cases.md:core-model:12 | optional_agent_tool | skills/paperclip/references/cases.md:12 |
| skill:skills/paperclip/references/cases.md:upsert-semantics:29 | optional_agent_tool | skills/paperclip/references/cases.md:29 |
| skill:skills/paperclip/references/cases.md:read-and-search:67 | optional_agent_tool | skills/paperclip/references/cases.md:67 |
| skill:skills/paperclip/references/cases.md:documents:90 | always_agent_tool | skills/paperclip/references/cases.md:90 |
| skill:skills/paperclip/references/cases.md:fields:118 | optional_agent_tool | skills/paperclip/references/cases.md:118 |
| skill:skills/paperclip/references/cases.md:issue-links:151 | optional_agent_tool | skills/paperclip/references/cases.md:151 |
| skill:skills/paperclip/references/cases.md:child-cases:176 | optional_agent_tool | skills/paperclip/references/cases.md:176 |
| skill:skills/paperclip/references/cases.md:attachments:195 | optional_agent_tool | skills/paperclip/references/cases.md:195 |
| skill:skills/paperclip/references/cases.md:lifecycle:208 | optional_agent_tool | skills/paperclip/references/cases.md:208 |
| skill:skills/paperclip/references/cases.md:worked-blog-post-example:222 | optional_agent_tool | skills/paperclip/references/cases.md:222 |
| skill:skills/paperclip/references/company-skills.md:company-skills-workflow:1 | optional_agent_tool | skills/paperclip/references/company-skills.md:1 |
| skill:skills/paperclip/references/company-skills.md:what-exists:5 | optional_agent_tool | skills/paperclip/references/company-skills.md:5 |
| skill:skills/paperclip/references/company-skills.md:permission-model:22 | optional_agent_tool | skills/paperclip/references/company-skills.md:22 |
| skill:skills/paperclip/references/company-skills.md:core-endpoints:29 | optional_agent_tool | skills/paperclip/references/company-skills.md:29 |
| skill:skills/paperclip/references/company-skills.md:install-a-skill-into-the-company:65 | optional_agent_tool | skills/paperclip/references/company-skills.md:65 |
| skill:skills/paperclip/references/company-skills.md:app-shipped-catalog:75 | optional_agent_tool | skills/paperclip/references/company-skills.md:75 |
| skill:skills/paperclip/references/company-skills.md:external-source-import:101 | optional_agent_tool | skills/paperclip/references/company-skills.md:101 |
| skill:skills/paperclip/references/company-skills.md:source-types-in-order-of-preference:105 | optional_agent_tool | skills/paperclip/references/company-skills.md:105 |
| skill:skills/paperclip/references/company-skills.md:example-skills-sh-import-preferred:116 | optional_agent_tool | skills/paperclip/references/company-skills.md:116 |
| skill:skills/paperclip/references/company-skills.md:example-github-import:138 | optional_agent_tool | skills/paperclip/references/company-skills.md:138 |
| skill:skills/paperclip/references/company-skills.md:inspect-what-was-installed:164 | optional_agent_tool | skills/paperclip/references/company-skills.md:164 |
| skill:skills/paperclip/references/company-skills.md:assign-skills-to-an-existing-agent:181 | optional_agent_tool | skills/paperclip/references/company-skills.md:181 |
| skill:skills/paperclip/references/company-skills.md:include-skills-during-hire-or-create:216 | optional_agent_tool | skills/paperclip/references/company-skills.md:216 |
| skill:skills/paperclip/references/company-skills.md:notes:256 | optional_agent_tool | skills/paperclip/references/company-skills.md:256 |
| skill:skills/paperclip/references/issue-workspaces.md:issue-workspace-runtime-controls:1 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:1 |
| skill:skills/paperclip/references/issue-workspaces.md:discover-the-workspace:5 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:5 |
| skill:skills/paperclip/references/issue-workspaces.md:control-services:23 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:23 |
| skill:skills/paperclip/references/issue-workspaces.md:start-all-configured-services-waits-for-configured-readiness-checks:28 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:28 |
| skill:skills/paperclip/references/issue-workspaces.md:restart-all-configured-services:36 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:36 |
| skill:skills/paperclip/references/issue-workspaces.md:stop-all-running-services:44 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:44 |
| skill:skills/paperclip/references/issue-workspaces.md:read-the-url:63 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:63 |
| skill:skills/paperclip/references/issue-workspaces.md:mcp-tools:72 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:72 |
| skill:skills/paperclip/references/routines.md:paperclip-routines:1 | optional_agent_tool | skills/paperclip/references/routines.md:1 |
| skill:skills/paperclip/references/routines.md:lifecycle:16 | optional_agent_tool | skills/paperclip/references/routines.md:16 |
| skill:skills/paperclip/references/routines.md:creating-a-routine:27 | optional_agent_tool | skills/paperclip/references/routines.md:27 |
| skill:skills/paperclip/references/routines.md:concurrency-policies:64 | optional_agent_tool | skills/paperclip/references/routines.md:64 |
| skill:skills/paperclip/references/routines.md:catch-up-policies:76 | optional_agent_tool | skills/paperclip/references/routines.md:76 |
| skill:skills/paperclip/references/routines.md:activity-gated-scheduled-runs:87 | optional_agent_tool | skills/paperclip/references/routines.md:87 |
| skill:skills/paperclip/references/routines.md:example-skip-quiet-nights:107 | optional_agent_tool | skills/paperclip/references/routines.md:107 |
| skill:skills/paperclip/references/routines.md:adding-triggers:126 | optional_agent_tool | skills/paperclip/references/routines.md:126 |
| skill:skills/paperclip/references/routines.md:schedule-cron:136 | optional_agent_tool | skills/paperclip/references/routines.md:136 |
| skill:skills/paperclip/references/routines.md:webhook:150 | optional_agent_tool | skills/paperclip/references/routines.md:150 |
| skill:skills/paperclip/references/routines.md:api-manual-only:167 | optional_agent_tool | skills/paperclip/references/routines.md:167 |
| skill:skills/paperclip/references/routines.md:updating-and-deleting-triggers:179 | optional_agent_tool | skills/paperclip/references/routines.md:179 |
| skill:skills/paperclip/references/routines.md:manual-run:196 | optional_agent_tool | skills/paperclip/references/routines.md:196 |
| skill:skills/paperclip/references/routines.md:updating-a-routine:212 | optional_agent_tool | skills/paperclip/references/routines.md:212 |
| skill:skills/paperclip/references/routines.md:reading-routines-and-runs:223 | optional_agent_tool | skills/paperclip/references/routines.md:223 |
| skill:skills/paperclip/references/workflows.md:paperclip-workflow-playbooks:1 | optional_agent_tool | skills/paperclip/references/workflows.md:1 |
| skill:skills/paperclip/references/workflows.md:project-setup-ceo-manager:7 | optional_agent_tool | skills/paperclip/references/workflows.md:7 |
| skill:skills/paperclip/references/workflows.md:openclaw-invite-ceo:22 | optional_agent_tool | skills/paperclip/references/workflows.md:22 |
| skill:skills/paperclip/references/workflows.md:setting-agent-instructions-path:50 | optional_agent_tool | skills/paperclip/references/workflows.md:50 |
| skill:skills/paperclip/references/workflows.md:company-import-export:79 | optional_agent_tool | skills/paperclip/references/workflows.md:79 |
| skill:skills/paperclip/references/workflows.md:self-test-playbook-app-level:106 | optional_agent_tool | skills/paperclip/references/workflows.md:106 |
| skill:skills/paperclip/SKILL.md:authentication:20 | control_plane_owned | skills/paperclip/SKILL.md:20 |
| skill:skills/paperclip/SKILL.md:conversation-tasks:32 | optional_agent_tool | skills/paperclip/SKILL.md:32 |
| skill:skills/paperclip/SKILL.md:server-verified-external-chat-turns:49 | control_plane_owned | skills/paperclip/SKILL.md:49 |
| skill:skills/paperclip/SKILL.md:the-heartbeat-procedure:89 | optional_agent_tool | skills/paperclip/SKILL.md:89 |
| skill:skills/paperclip/SKILL.md:generated-artifacts-and-work-products:198 | always_agent_tool | skills/paperclip/SKILL.md:198 |
| skill:skills/paperclip/SKILL.md:status-quick-guide:246 | control_plane_owned | skills/paperclip/SKILL.md:246 |
| skill:skills/paperclip/SKILL.md:monitors-and-watchers-say-only-what-you-actually-scheduled:256 | optional_agent_tool | skills/paperclip/SKILL.md:256 |
| skill:skills/paperclip/SKILL.md:delegating-review-tasks:269 | always_agent_tool | skills/paperclip/SKILL.md:269 |
| skill:skills/paperclip/SKILL.md:managing-a-user-s-inbox:280 | control_plane_owned | skills/paperclip/SKILL.md:280 |
| skill:skills/paperclip/SKILL.md:issue-dependencies-blockers:288 | control_plane_owned | skills/paperclip/SKILL.md:288 |
| skill:skills/paperclip/SKILL.md:requesting-board-approval:313 | optional_agent_tool | skills/paperclip/SKILL.md:313 |
| skill:skills/paperclip/SKILL.md:issue-thread-interactions:334 | optional_agent_tool | skills/paperclip/SKILL.md:334 |
| skill:skills/paperclip/SKILL.md:standalone-decisions:363 | optional_agent_tool | skills/paperclip/SKILL.md:363 |
| skill:skills/paperclip/SKILL.md:mcp-tool-approval-gates:467 | optional_agent_tool | skills/paperclip/SKILL.md:467 |
| skill:skills/paperclip/SKILL.md:niche-workflow-pointers:509 | optional_agent_tool | skills/paperclip/SKILL.md:509 |
| skill:skills/paperclip/SKILL.md:cases:519 | optional_agent_tool | skills/paperclip/SKILL.md:519 |
| skill:skills/paperclip/SKILL.md:company-skills-workflow:524 | optional_agent_tool | skills/paperclip/SKILL.md:524 |
| skill:skills/paperclip/SKILL.md:routines:535 | optional_agent_tool | skills/paperclip/SKILL.md:535 |
| skill:skills/paperclip/SKILL.md:issue-workspace-runtime-controls:546 | optional_agent_tool | skills/paperclip/SKILL.md:546 |
| skill:skills/paperclip/SKILL.md:proposing-credentials-safely:553 | optional_agent_tool | skills/paperclip/SKILL.md:553 |
| skill:skills/paperclip/SKILL.md:reading-granted-secrets:560 | optional_agent_tool | skills/paperclip/SKILL.md:560 |
| skill:skills/paperclip/SKILL.md:critical-rules:586 | optional_agent_tool | skills/paperclip/SKILL.md:586 |
| skill:skills/paperclip/SKILL.md:comment-style-required:607 | always_agent_tool | skills/paperclip/SKILL.md:607 |
| skill:skills/paperclip/SKILL.md:update:639 | optional_agent_tool | skills/paperclip/SKILL.md:639 |
| skill:skills/paperclip/SKILL.md:planning-required-when-planning-requested:649 | optional_agent_tool | skills/paperclip/SKILL.md:649 |
| skill:skills/paperclip/SKILL.md:key-endpoints-hot-routes:682 | optional_agent_tool | skills/paperclip/SKILL.md:682 |
| skill:skills/paperclip/SKILL.md:searching-issues:711 | optional_agent_tool | skills/paperclip/SKILL.md:711 |
| skill:skills/paperclip/SKILL.md:full-reference:721 | optional_agent_tool | skills/paperclip/SKILL.md:721 |
| skill:skills/paperclip/SKILL.md:conversational-confirmation-answers:725 | always_agent_tool | skills/paperclip/SKILL.md:725 |
| skill:skills/paperclip/references/api-reference.md:paperclip-api-reference:1 | optional_agent_tool | skills/paperclip/references/api-reference.md:1 |
| skill:skills/paperclip/references/api-reference.md:response-schemas:9 | optional_agent_tool | skills/paperclip/references/api-reference.md:9 |
| skill:skills/paperclip/references/api-reference.md:agent-record-get-api-agents-me-or-get-api-agents-agentid:11 | optional_agent_tool | skills/paperclip/references/api-reference.md:11 |
@@ -198,6 +142,65 @@ The skill/reference inventory and eval cases are the only normative behavior sou
| skill:skills/paperclip/references/api-reference.md:agent-secret-proposals:1521 | optional_agent_tool | skills/paperclip/references/api-reference.md:1521 |
| skill:skills/paperclip/references/api-reference.md:agent-secret-access:1621 | optional_agent_tool | skills/paperclip/references/api-reference.md:1621 |
| skill:skills/paperclip/references/api-reference.md:common-mistakes:1661 | optional_agent_tool | skills/paperclip/references/api-reference.md:1661 |
| skill:skills/paperclip/references/artifacts.md:generated-artifacts-and-work-products:1 | always_agent_tool | skills/paperclip/references/artifacts.md:1 |
| skill:skills/paperclip/references/artifacts.md:workspace-only-file-references:15 | optional_agent_tool | skills/paperclip/references/artifacts.md:15 |
| skill:skills/paperclip/references/cases.md:cases:1 | optional_agent_tool | skills/paperclip/references/cases.md:1 |
| skill:skills/paperclip/references/cases.md:core-model:12 | optional_agent_tool | skills/paperclip/references/cases.md:12 |
| skill:skills/paperclip/references/cases.md:upsert-semantics:29 | optional_agent_tool | skills/paperclip/references/cases.md:29 |
| skill:skills/paperclip/references/cases.md:read-and-search:67 | optional_agent_tool | skills/paperclip/references/cases.md:67 |
| skill:skills/paperclip/references/cases.md:documents:90 | always_agent_tool | skills/paperclip/references/cases.md:90 |
| skill:skills/paperclip/references/cases.md:fields:118 | optional_agent_tool | skills/paperclip/references/cases.md:118 |
| skill:skills/paperclip/references/cases.md:issue-links:151 | optional_agent_tool | skills/paperclip/references/cases.md:151 |
| skill:skills/paperclip/references/cases.md:child-cases:176 | optional_agent_tool | skills/paperclip/references/cases.md:176 |
| skill:skills/paperclip/references/cases.md:attachments:195 | optional_agent_tool | skills/paperclip/references/cases.md:195 |
| skill:skills/paperclip/references/cases.md:lifecycle:208 | optional_agent_tool | skills/paperclip/references/cases.md:208 |
| skill:skills/paperclip/references/cases.md:worked-blog-post-example:222 | optional_agent_tool | skills/paperclip/references/cases.md:222 |
| skill:skills/paperclip/references/company-skills.md:company-skills-workflow:1 | optional_agent_tool | skills/paperclip/references/company-skills.md:1 |
| skill:skills/paperclip/references/company-skills.md:what-exists:5 | optional_agent_tool | skills/paperclip/references/company-skills.md:5 |
| skill:skills/paperclip/references/company-skills.md:permission-model:22 | optional_agent_tool | skills/paperclip/references/company-skills.md:22 |
| skill:skills/paperclip/references/company-skills.md:core-endpoints:29 | optional_agent_tool | skills/paperclip/references/company-skills.md:29 |
| skill:skills/paperclip/references/company-skills.md:install-a-skill-into-the-company:65 | optional_agent_tool | skills/paperclip/references/company-skills.md:65 |
| skill:skills/paperclip/references/company-skills.md:app-shipped-catalog:75 | optional_agent_tool | skills/paperclip/references/company-skills.md:75 |
| skill:skills/paperclip/references/company-skills.md:external-source-import:101 | optional_agent_tool | skills/paperclip/references/company-skills.md:101 |
| skill:skills/paperclip/references/company-skills.md:source-types-in-order-of-preference:105 | optional_agent_tool | skills/paperclip/references/company-skills.md:105 |
| skill:skills/paperclip/references/company-skills.md:example-skills-sh-import-preferred:116 | optional_agent_tool | skills/paperclip/references/company-skills.md:116 |
| skill:skills/paperclip/references/company-skills.md:example-github-import:138 | optional_agent_tool | skills/paperclip/references/company-skills.md:138 |
| skill:skills/paperclip/references/company-skills.md:inspect-what-was-installed:164 | optional_agent_tool | skills/paperclip/references/company-skills.md:164 |
| skill:skills/paperclip/references/company-skills.md:assign-skills-to-an-existing-agent:181 | optional_agent_tool | skills/paperclip/references/company-skills.md:181 |
| skill:skills/paperclip/references/company-skills.md:include-skills-during-hire-or-create:216 | optional_agent_tool | skills/paperclip/references/company-skills.md:216 |
| skill:skills/paperclip/references/company-skills.md:notes:256 | optional_agent_tool | skills/paperclip/references/company-skills.md:256 |
| skill:skills/paperclip/references/issue-documents.md:issue-documents-through-the-api:1 | always_agent_tool | skills/paperclip/references/issue-documents.md:1 |
| skill:skills/paperclip/references/issue-documents.md:create-a-document:8 | always_agent_tool | skills/paperclip/references/issue-documents.md:8 |
| skill:skills/paperclip/references/issue-documents.md:update-or-resolve-an-unclear-write:49 | optional_agent_tool | skills/paperclip/references/issue-documents.md:49 |
| skill:skills/paperclip/references/issue-workspaces.md:issue-workspace-runtime-controls:1 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:1 |
| skill:skills/paperclip/references/issue-workspaces.md:discover-the-workspace:5 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:5 |
| skill:skills/paperclip/references/issue-workspaces.md:control-services:23 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:23 |
| skill:skills/paperclip/references/issue-workspaces.md:start-all-configured-services-waits-for-configured-readiness-checks:28 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:28 |
| skill:skills/paperclip/references/issue-workspaces.md:restart-all-configured-services:36 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:36 |
| skill:skills/paperclip/references/issue-workspaces.md:stop-all-running-services:44 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:44 |
| skill:skills/paperclip/references/issue-workspaces.md:read-the-url:63 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:63 |
| skill:skills/paperclip/references/issue-workspaces.md:mcp-tools:72 | optional_agent_tool | skills/paperclip/references/issue-workspaces.md:72 |
| skill:skills/paperclip/references/routines.md:paperclip-routines:1 | optional_agent_tool | skills/paperclip/references/routines.md:1 |
| skill:skills/paperclip/references/routines.md:lifecycle:16 | optional_agent_tool | skills/paperclip/references/routines.md:16 |
| skill:skills/paperclip/references/routines.md:creating-a-routine:27 | optional_agent_tool | skills/paperclip/references/routines.md:27 |
| skill:skills/paperclip/references/routines.md:concurrency-policies:64 | optional_agent_tool | skills/paperclip/references/routines.md:64 |
| skill:skills/paperclip/references/routines.md:catch-up-policies:76 | optional_agent_tool | skills/paperclip/references/routines.md:76 |
| skill:skills/paperclip/references/routines.md:activity-gated-scheduled-runs:87 | optional_agent_tool | skills/paperclip/references/routines.md:87 |
| skill:skills/paperclip/references/routines.md:example-skip-quiet-nights:107 | optional_agent_tool | skills/paperclip/references/routines.md:107 |
| skill:skills/paperclip/references/routines.md:adding-triggers:126 | optional_agent_tool | skills/paperclip/references/routines.md:126 |
| skill:skills/paperclip/references/routines.md:schedule-cron:136 | optional_agent_tool | skills/paperclip/references/routines.md:136 |
| skill:skills/paperclip/references/routines.md:webhook:150 | optional_agent_tool | skills/paperclip/references/routines.md:150 |
| skill:skills/paperclip/references/routines.md:api-manual-only:167 | optional_agent_tool | skills/paperclip/references/routines.md:167 |
| skill:skills/paperclip/references/routines.md:updating-and-deleting-triggers:179 | optional_agent_tool | skills/paperclip/references/routines.md:179 |
| skill:skills/paperclip/references/routines.md:manual-run:196 | optional_agent_tool | skills/paperclip/references/routines.md:196 |
| skill:skills/paperclip/references/routines.md:updating-a-routine:212 | optional_agent_tool | skills/paperclip/references/routines.md:212 |
| skill:skills/paperclip/references/routines.md:reading-routines-and-runs:223 | optional_agent_tool | skills/paperclip/references/routines.md:223 |
| skill:skills/paperclip/references/workflows.md:paperclip-workflow-playbooks:1 | optional_agent_tool | skills/paperclip/references/workflows.md:1 |
| skill:skills/paperclip/references/workflows.md:project-setup-ceo-manager:7 | optional_agent_tool | skills/paperclip/references/workflows.md:7 |
| skill:skills/paperclip/references/workflows.md:openclaw-invite-ceo:22 | optional_agent_tool | skills/paperclip/references/workflows.md:22 |
| skill:skills/paperclip/references/workflows.md:setting-agent-instructions-path:50 | optional_agent_tool | skills/paperclip/references/workflows.md:50 |
| skill:skills/paperclip/references/workflows.md:company-import-export:79 | optional_agent_tool | skills/paperclip/references/workflows.md:79 |
| skill:skills/paperclip/references/workflows.md:self-test-playbook-app-level:106 | optional_agent_tool | skills/paperclip/references/workflows.md:106 |
## Legacy MCP Alias Index
packages/paperclip-runner/docs/capability-semantic-tools.md
+3 -3
View File
@@ -26,7 +26,7 @@ list, not a disposition — they are never exposed as tools. See
[capability disposition](capability-disposition.md) and
[authorization and exposure](capability-authorization-and-exposure.md).
The catalog holds **39 tools**: 14 always-agent tools and 25 optional tools
The catalog holds **40 tools**: 14 always-agent tools and 26 optional tools
across 10 groups.
<!-- These counts are drift-checked against src/tools/capability-semantic-tool-catalog.ts
@@ -39,7 +39,7 @@ by src/catalog/catalog-docs.test.ts; update the catalog, not the numbers. -->
`finish_task`, `block_task`, `request_review`, `write_document`,
`request_human_input`, `register_deliverable`, `inspect_operation_result`.
### Optional tools (25), by group
### Optional tools (26), by group
| Group | Tools |
| --- | --- |
@@ -49,7 +49,7 @@ by src/catalog/catalog-docs.test.ts; update the catalog, not the numbers. -->
| cases | `list_cases`, `upsert_case` |
| workspace_runtime | `get_workspace_runtime`, `control_workspace_service` |
| routines | `list_routines`, `manage_routine` |
| company_skills | `create_skill`, `list_company_skills`, `sync_company_skills` |
| company_skills | `create_skill`, `update_skill`, `list_company_skills`, `sync_company_skills` |
| secrets | `list_secret_metadata`, `read_secret_value` |
| portability_admin | `export_company`, `administer_company` |
| test_escape_hatch | `generic_api_request` |
packages/paperclip-runner/generated/capability/capabilities.yaml
+85 -58
View File
@@ -20,261 +20,261 @@
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:18",
"id": "skill:skills/paperclip/SKILL.md:20",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L18:authentication",
"sourceAnchor": "skills/paperclip/SKILL.md#L20:authentication",
"heading": "Authentication",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:30",
"id": "skill:skills/paperclip/SKILL.md:32",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L30:conversation-tasks",
"sourceAnchor": "skills/paperclip/SKILL.md#L32:conversation-tasks",
"heading": "Conversation tasks",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:47",
"id": "skill:skills/paperclip/SKILL.md:49",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L47:server-verified-external-chat-turns",
"sourceAnchor": "skills/paperclip/SKILL.md#L49:server-verified-external-chat-turns",
"heading": "Server-Verified External Chat Turns",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:87",
"id": "skill:skills/paperclip/SKILL.md:89",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L87:the-heartbeat-procedure",
"sourceAnchor": "skills/paperclip/SKILL.md#L89:the-heartbeat-procedure",
"heading": "The Heartbeat Procedure",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:196",
"id": "skill:skills/paperclip/SKILL.md:198",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L196:generated-artifacts-and-work-products",
"sourceAnchor": "skills/paperclip/SKILL.md#L198:generated-artifacts-and-work-products",
"heading": "Generated Artifacts and Work Products",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "register_deliverable",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:244",
"id": "skill:skills/paperclip/SKILL.md:246",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L244:status-quick-guide",
"sourceAnchor": "skills/paperclip/SKILL.md#L246:status-quick-guide",
"heading": "Status Quick Guide",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:254",
"id": "skill:skills/paperclip/SKILL.md:256",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L254:monitors-and-watchers-say-only-what-you-actually-scheduled",
"sourceAnchor": "skills/paperclip/SKILL.md#L256:monitors-and-watchers-say-only-what-you-actually-scheduled",
"heading": "Monitors and Watchers (say only what you actually scheduled)",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:267",
"id": "skill:skills/paperclip/SKILL.md:269",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L267:delegating-review-tasks",
"sourceAnchor": "skills/paperclip/SKILL.md#L269:delegating-review-tasks",
"heading": "Delegating review tasks",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:278",
"id": "skill:skills/paperclip/SKILL.md:280",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L278:managing-a-user-s-inbox",
"sourceAnchor": "skills/paperclip/SKILL.md#L280:managing-a-user-s-inbox",
"heading": "Managing A User's Inbox",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:286",
"id": "skill:skills/paperclip/SKILL.md:288",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L286:issue-dependencies-blockers",
"sourceAnchor": "skills/paperclip/SKILL.md#L288:issue-dependencies-blockers",
"heading": "Issue Dependencies (Blockers)",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:311",
"id": "skill:skills/paperclip/SKILL.md:313",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L311:requesting-board-approval",
"sourceAnchor": "skills/paperclip/SKILL.md#L313:requesting-board-approval",
"heading": "Requesting Board Approval",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:332",
"id": "skill:skills/paperclip/SKILL.md:334",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L332:issue-thread-interactions",
"sourceAnchor": "skills/paperclip/SKILL.md#L334:issue-thread-interactions",
"heading": "Issue-Thread Interactions",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "request_human_input",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:361",
"id": "skill:skills/paperclip/SKILL.md:363",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L361:standalone-decisions",
"sourceAnchor": "skills/paperclip/SKILL.md#L363:standalone-decisions",
"heading": "Standalone Decisions",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:465",
"id": "skill:skills/paperclip/SKILL.md:467",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L465:mcp-tool-approval-gates",
"sourceAnchor": "skills/paperclip/SKILL.md#L467:mcp-tool-approval-gates",
"heading": "MCP Tool Approval Gates",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:507",
"id": "skill:skills/paperclip/SKILL.md:509",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L507:niche-workflow-pointers",
"sourceAnchor": "skills/paperclip/SKILL.md#L509:niche-workflow-pointers",
"heading": "Niche Workflow Pointers",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:517",
"id": "skill:skills/paperclip/SKILL.md:519",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L517:cases",
"sourceAnchor": "skills/paperclip/SKILL.md#L519:cases",
"heading": "Cases",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:522",
"id": "skill:skills/paperclip/SKILL.md:524",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L522:company-skills-workflow",
"sourceAnchor": "skills/paperclip/SKILL.md#L524:company-skills-workflow",
"heading": "Company Skills Workflow",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:533",
"id": "skill:skills/paperclip/SKILL.md:535",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L533:routines",
"sourceAnchor": "skills/paperclip/SKILL.md#L535:routines",
"heading": "Routines",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:544",
"id": "skill:skills/paperclip/SKILL.md:546",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L544:issue-workspace-runtime-controls",
"sourceAnchor": "skills/paperclip/SKILL.md#L546:issue-workspace-runtime-controls",
"heading": "Issue Workspace Runtime Controls",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:551",
"id": "skill:skills/paperclip/SKILL.md:553",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L551:proposing-credentials-safely",
"sourceAnchor": "skills/paperclip/SKILL.md#L553:proposing-credentials-safely",
"heading": "Proposing Credentials Safely",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:558",
"id": "skill:skills/paperclip/SKILL.md:560",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L558:reading-granted-secrets",
"sourceAnchor": "skills/paperclip/SKILL.md#L560:reading-granted-secrets",
"heading": "Reading Granted Secrets",
"primaryDisposition": "optional_agent_tool",
"semanticOperation": "scoped_discovery",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:584",
"id": "skill:skills/paperclip/SKILL.md:586",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L584:critical-rules",
"sourceAnchor": "skills/paperclip/SKILL.md#L586:critical-rules",
"heading": "Critical Rules",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:605",
"id": "skill:skills/paperclip/SKILL.md:607",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L605:comment-style-required",
"sourceAnchor": "skills/paperclip/SKILL.md#L607:comment-style-required",
"heading": "Comment Style (Required)",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "report_progress",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:637",
"id": "skill:skills/paperclip/SKILL.md:639",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L637:update",
"sourceAnchor": "skills/paperclip/SKILL.md#L639:update",
"heading": "Update",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:647",
"id": "skill:skills/paperclip/SKILL.md:649",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L647:planning-required-when-planning-requested",
"sourceAnchor": "skills/paperclip/SKILL.md#L649:planning-required-when-planning-requested",
"heading": "Planning (Required when planning requested)",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "write_document",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/SKILL.md:680",
"id": "skill:skills/paperclip/SKILL.md:682",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L680:key-endpoints-hot-routes",
"sourceAnchor": "skills/paperclip/SKILL.md#L682:key-endpoints-hot-routes",
"heading": "Key Endpoints (Hot Routes)",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:709",
"id": "skill:skills/paperclip/SKILL.md:711",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L709:searching-issues",
"sourceAnchor": "skills/paperclip/SKILL.md#L711:searching-issues",
"heading": "Searching Issues",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:719",
"id": "skill:skills/paperclip/SKILL.md:721",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L719:full-reference",
"sourceAnchor": "skills/paperclip/SKILL.md#L721:full-reference",
"heading": "Full Reference",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/SKILL.md:723",
"id": "skill:skills/paperclip/SKILL.md:725",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/SKILL.md#L723:conversational-confirmation-answers",
"sourceAnchor": "skills/paperclip/SKILL.md#L725:conversational-confirmation-answers",
"heading": "Conversational confirmation answers",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "call_api",
@@ -1162,6 +1162,33 @@
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/references/issue-documents.md:1",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/references/issue-documents.md#L1:issue-documents-through-the-api",
"heading": "Issue documents through the API",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "write_document",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/references/issue-documents.md:8",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/references/issue-documents.md#L8:create-a-document",
"heading": "Create a document",
"primaryDisposition": "always_agent_tool",
"semanticOperation": "write_document",
"expectedMockState": "operation_result"
},
{
"id": "skill:skills/paperclip/references/issue-documents.md:49",
"kind": "skill_heading",
"sourceAnchor": "skills/paperclip/references/issue-documents.md#L49:update-or-resolve-an-unclear-write",
"heading": "Update or resolve an unclear write",
"primaryDisposition": "control_plane_owned",
"semanticOperation": "runtime_reconciliation",
"expectedMockState": "runtime_decision_record"
},
{
"id": "skill:skills/paperclip/references/issue-workspaces.md:1",
"kind": "skill_heading",
packages/paperclip-runner/generated/capability/capability-contract.md
+2 -2
View File
@@ -2,9 +2,9 @@
Generated by `scripts/generate-capability-contract.mjs`; do not edit generated files.
- Skill/reference headings: 158
- Skill/reference headings: 161
- Legacy MCP tools: 42
- Eval cases: 106 across 16 groups
- Deterministic content SHA-256: `2707de482cfbe6c7a483f6dee3bf4a2fc66e2b944bb50c3818ca6d5ef777b1de`
- Deterministic content SHA-256: `e577ca3ad6e6b045a7413ac11008013c8c12c2dc954f5a7876b86d96fbe017e8`
Every row has exactly one primary disposition, a source anchor, a semantic operation, and a mock-state expectation.
packages/paperclip-runner/generated/capability/semantic-tool-contracts.json
+1 -1
View File
@@ -1 +1 @@
[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaimLine truncated
[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaimLine truncated
packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json
+1 -1
View File
@@ -24,7 +24,7 @@
"prpVersion": 1,
"nativeExecutionVersion": 1,
"catalogVersion": 1,
"catalogSha256": "sha256:81ed59a7fa4ceb6e0f28dffabf7a7ae7bdfa9e23e6bb643a14612368262abfe3",
"catalogSha256": "sha256:1dd4904ba9ae5c2b6bb0cbd4cfef65734d855bb5ed10ed05c70b603b8589bec3",
"driverContractVersion": 1,
"driverKind": "paperclip-deterministic",
"driverVersion": "1.0.0"
packages/paperclip-runner/protocol/manifest.json
+1 -1
View File
@@ -160,7 +160,7 @@
},
{
"path": "fixtures/evals/native-execution-seeded.json",
"sha256": "a5fde52c41fe77a8872ec1d1613ba9bb426b9231835e1464d943930ab799d933",
"sha256": "b4e488f922dd328a3967b2560951c2377f4c1411fb2c4c9eef7a44195a8f7514",
"expectation": "accept",
"compatibilityCase": "canonical"
},
packages/paperclip-runner/runner/crates/runner-core/src/provider_events.rs
+46
View File
@@ -928,6 +928,18 @@ pub fn normalize_codex_notification(method: &str, params: &Value) -> Vec<Normali
},
"text": provider_item.get("text").and_then(Value::as_str).map(|value| bounded_text(value, MAX_TEXT_CHARS)),
});
// Preserve only actual terminal invocation identity in the compatibility
// projection. Arguments, arbitrary tool names and result bodies stay omitted.
if item_type == "tool_call" {
if let Some(name @ ("paperclip_finish" | "paperclip_block")) =
provider_item.get("name").and_then(Value::as_str)
{
payload
.as_object_mut()
.expect("item payload is an object")
.insert("item".to_owned(), json!({ "name": name }));
}
}
if !provider_phase.is_empty() {
payload
.as_object_mut()
@@ -1562,6 +1574,40 @@ mod tests {
}
}
#[test]
fn preserves_closed_compatibility_terminal_tool_identity() {
let fixture: Value = serde_json::from_str(include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../../../tests/runner-e2e/fixtures/native-completion/terminal-tool-carrier.json"
)))
.unwrap();
for case in fixture["cases"].as_array().unwrap() {
let events = normalize_codex_notification("item/started", &case["providerInput"]);
assert_eq!(events.len(), 1);
assert_eq!(events[0].event_type, "item.started");
assert_eq!(events[0].payload, case["normalizedPayload"]);
assert!(!events[0].payload.to_string().contains("not-for-the-log"));
}
for (item_type, name) in [
("tool_call", Some("write_document")),
("tool_call", Some("mcp.paperclip_finish")),
("tool_call", None),
("agentMessage", Some("paperclip_finish")),
("tool_result", Some("paperclip_block")),
] {
let events = normalize_codex_notification(
"item/completed",
&json!({ "item": {
"id": "terminal-call", "type": item_type, "name": name,
"status": "completed", "arguments": {"secret": "not-for-the-log"},
"result": {"secret": "not-for-the-log"}
}}),
);
assert_eq!(events[0].payload.get("item"), None);
assert!(!events[0].payload.to_string().contains("not-for-the-log"));
}
}
#[test]
fn enforces_the_declared_safe_path_contract() {
for location in [
packages/paperclip-runner/scripts/check-capability-inventory.test.mjs
+1 -1
View File
@@ -42,7 +42,7 @@ function validInventories() {
schemaVersion: 2,
inventoryRole: "normative",
generatedFrom: ["skills/paperclip/SKILL.md"],
rows: Array.from({ length: 157 }, (_, index) => row(`capability-${index}`)),
rows: Array.from({ length: 160 }, (_, index) => row(`capability-${index}`)),
},
evaluations: {
schemaVersion: 2,
packages/paperclip-runner/scripts/generate-capability-contract.d.mts
+1
View File
@@ -0,0 +1 @@
export function buildContract(): Promise<Record<string, string>>;
packages/paperclip-runner/scripts/generate-capability-contract.mjs
+1 -1
View File
@@ -159,7 +159,7 @@ function renderHandoff() {
].join("\n") + "\n";
}
async function buildContract() {
export async function buildContract() {
const contract = JSON.parse(await readFile(contractPath, "utf8"));
const capabilities = await readSkillHeadings(contract.skillSources);
const discoveredTools = await readLegacyTools();
packages/paperclip-runner/scripts/lib/capability-inventory.mjs
+6 -6
View File
@@ -221,16 +221,16 @@ export async function buildInventories({ repoRoot, evalRoot }) {
}
export async function buildSkillInventory(repoRoot) {
const skillRoot = resolve(repoRoot, "skills/paperclip");
const skillFiles = ["SKILL.md", "references/artifacts.md", "references/cases.md", "references/company-skills.md", "references/issue-workspaces.md", "references/routines.md", "references/workflows.md", "references/api-reference.md"];
const { skillSources: skillFiles } = JSON.parse(await readFile(resolve(repoRoot,
"packages/paperclip-runner/spec/capability/source-contract.json"), "utf8"));
const rows = (await Promise.all(skillFiles.map(async (file) => parseSkillHeadings(
await readFile(resolve(skillRoot, file), "utf8"),
`skills/paperclip/${file}`,
await readFile(resolve(repoRoot, file), "utf8"),
file,
)))).flat();
return {
schemaVersion: 2,
inventoryRole: "normative",
generatedFrom: skillFiles.map((file) => `skills/paperclip/${file}`),
generatedFrom: skillFiles,
rows,
};
}
@@ -247,7 +247,7 @@ export async function buildMcpInventory(repoRoot) {
export function validateInventories(inventories) {
const errors = [];
const expectedCounts = { capabilities: 157, evaluations: 106, legacyMcpAliases: 42 };
const expectedCounts = { capabilities: 160, evaluations: 106, legacyMcpAliases: 42 };
const normativeNames = ["capabilities", "evaluations"];
const normativeRows = new Map();
const globalNormativeIds = new Set();
packages/paperclip-runner/spec/capability/capabilities.yaml
+975 -929
View File
File diff suppressed because it is too large. Load diff
packages/paperclip-runner/spec/capability/protocol-coverage.json
+21 -1
View File
@@ -7,7 +7,7 @@
"src/scenarios/scenario-plan.ts"
],
"counts": {
"actions": 53,
"actions": 54,
"legacyRequirements": 106
},
"actions": [
@@ -126,6 +126,26 @@
"src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity"
]
},
{
"id": "update_skill",
"ownership": "optional_agent_tool",
"surfaces": [
"scenario",
"live"
],
"legacyAliases": [],
"contractCase": "protocol-action:update_skill",
"contractOwner": "src/catalog/protocol-action-contracts.test.ts::update_skill has a schema-valid canonical example and every declared projection",
"legacyBehavioralCases": [],
"deterministicCases": [
"protocol-action:update_skill"
],
"legacyRequirementCases": [],
"deterministicOwners": [
"src/catalog/protocol-action-contracts.test.ts::update_skill has a schema-valid canonical example and every declared projection",
"src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity"
]
},
{
"id": "create_project",
"ownership": "optional_agent_tool",
packages/paperclip-runner/spec/capability/source-contract.json
+1
View File
@@ -7,6 +7,7 @@
"skills/paperclip/references/artifacts.md",
"skills/paperclip/references/cases.md",
"skills/paperclip/references/company-skills.md",
"skills/paperclip/references/issue-documents.md",
"skills/paperclip/references/issue-workspaces.md",
"skills/paperclip/references/routines.md",
"skills/paperclip/references/workflows.md"
packages/paperclip-runner/spec/operation-groups/source.json
+3 -3
View File
@@ -87,8 +87,8 @@
},
{
"id": "company_skills",
"description": "Create or inspect company skills and synchronize the current agent's skills.",
"operationIds": ["create_skill", "list_company_skills", "sync_company_skills"]
"description": "Create, update, or inspect company skills and synchronize the current agent's skills.",
"operationIds": ["create_skill", "update_skill", "list_company_skills", "sync_company_skills"]
},
{
"id": "secrets",
@@ -264,7 +264,7 @@
"legacyGroup": 13,
"name": "Reference files",
"owner": "always instruction tools + optional domain tools + test-only escape hatch",
"operationIds": ["list_cases", "upsert_case", "list_routines", "manage_routine", "create_skill", "list_company_skills", "sync_company_skills", "list_secret_metadata", "read_secret_value", "export_company", "administer_company", "generic_api_request", "search_api", "call_api", "create_project", "read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions"],
"operationIds": ["list_cases", "upsert_case", "list_routines", "manage_routine", "create_skill", "update_skill", "list_company_skills", "sync_company_skills", "list_secret_metadata", "read_secret_value", "export_company", "administer_company", "generic_api_request", "search_api", "call_api", "create_project", "read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions"],
"controlPlaneOperationIds": ["append_audit_record"],
"realSurface": "agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services",
"mockStateDomains": ["company", "cases", "routines", "skills", "secrets", "audit", "fault"],
packages/paperclip-runner/spec/paperclip-agent-operation-groups.md
+8 -7
View File
@@ -6,7 +6,7 @@ Status: canonical explanatory contract for the Paperclip runner V1 surface.
This document keeps three independent meanings of **group** separate. PRP families describe wire evidence and controller commands; capability placement decides who owns an operation; behavioral eval groups organize the 106 scenario corpus. None of the three axes can be used as a substitute for another.
The generated totals are **105 PRP events in 31 event families**, **18 controller commands in 7 command families**, **10 control-plane operations**, **53 reconciled semantic operations** (18 always, 35 optional), and **106 scenarios in 16 behavior groups**.
The generated totals are **105 PRP events in 31 event families**, **18 controller commands in 7 command families**, **10 control-plane operations**, **54 reconciled semantic operations** (18 always, 36 optional), and **106 scenarios in 16 behavior groups**.
## Axis 1: PRP v1 event and command families
@@ -87,7 +87,7 @@ Placement has exactly three outcomes:
`answer_status_question`, `block_task`, `finish_task`, `get_agent_instruction_history`, `get_task_context`, `get_task_history`, `inspect_operation_result`, `list_document_revisions`, `list_documents`, `read_agent_instructions`, `read_document`, `register_deliverable`, `report_progress`, `request_human_input`, `request_review`, `restore_agent_instructions`, `update_agent_instructions`, `write_document`.
### Optional operations (35) and grant groups (13)
### Optional operations (36) and grant groups (13)
Grant groups are documentation/exposure bundles, not additional authority. The operation descriptor's exact `requiredClaims` remains decisive.
@@ -101,7 +101,7 @@ Grant groups are documentation/exposure bundles, not additional authority. The o
| `workspace_runtime` | `get_workspace_runtime`<br>`control_workspace_service` | `workspace:control`<br>`workspace:read` | Inspect and control the active issue workspace runtime. |
| `wake_scheduling` | `schedule_wake` | `control_plane:wakes` | Schedule a bounded continuation wake when the current task owns the future check. |
| `routines` | `list_routines`<br>`manage_routine` | `routines:read`<br>`routines:write` | Inspect or manage company routines. |
| `company_skills` | `create_skill`<br>`list_company_skills`<br>`sync_company_skills` | `company_skills:read`<br>`company_skills:write` | Create or inspect company skills and synchronize the current agent's skills. |
| `company_skills` | `create_skill`<br>`update_skill`<br>`list_company_skills`<br>`sync_company_skills` | `company_skills:read`<br>`company_skills:write` | Create, update, or inspect company skills and synchronize the current agent's skills. |
| `secrets` | `list_secret_metadata`<br>`read_secret_value` | `secrets:metadata:read`<br>`secrets:values:read` | Inspect secret metadata or use a brokered secret value without exposing plaintext evidence. |
| `portability_admin` | `export_company`<br>`administer_company` | `company:admin`<br>`portability:export` | Export portable company state; broad administration remains deferred until split into governed operations. |
| `test_escape_hatch` | `generic_api_request` | `test:generic_api_request` | Controlled skill-test transport only; never product coverage. |
@@ -164,6 +164,7 @@ Grant groups are documentation/exposure bundles, not additional authority. The o
| `set_task_title` | `optional_agent_tool` | none | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `task_write` | `required` | no | inline/no mapping | `live`<br>`live_codex` | `setIssueTitle`<br>Run-bound title update and transactional activity record.<br>catalog PRP status: `bound` |
| `sync_company_skills` | `optional_agent_tool` | `company_skills:write` | `standard`<br>`skill_test` | `admin` | `required` | no | `mock_extension:company_skills.sync` | `scenario`<br>`scenario_mock` | `unbound`<br>company admin/portability item event plus audit record<br>catalog PRP status: `audit_pending` |
| `update_agent_instructions` | `always_agent_tool` | none | `standard`<br>`planning` | `company_write` | `recommended` | no | inline/no mapping | `live`<br>`live_codex` | `agentInstructionRevisionService`<br>tool-result item event plus canonical revision receipt for writes; no scenario mock coverage<br>catalog PRP status: `bound` |
| `update_skill` | `optional_agent_tool` | none | `standard`<br>`skill_test` | `company_write` | `required` | no | `semantic_command:update_skill` | `scenario` + `live`<br>`live_codex` | `PaperclipRunnerToolAuthority`<br>Authenticated skill file API with version guard, durable receipt, and attributed activity.<br>catalog PRP status: `bound` |
| `upsert_case` | `optional_agent_tool` | `cases:write` | `standard`<br>`skill_test` | `company_write` | `required` | no | `mock_extension:cases.upsert` | `scenario`<br>`scenario_mock` | `unbound`<br>semantic-operation item event plus company-entity state diff and audit record<br>catalog PRP status: `audit_pending` |
| `write_document` | `always_agent_tool` | none | `standard`<br>`planning`<br>`skill_test` | `task_write` | `required` | no | `semantic_command:write_document` | `scenario` + `live`<br>`live_codex` | `unbound`<br>semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events<br>catalog PRP status: `audit_pending` |
@@ -187,7 +188,7 @@ Behavior groups describe expected outcomes and trajectories. They do not grant t
| [`ap` — Approvals](#behavior-group-ap-approvals) | optional governance tools + governed approver | `list_approvals`<br>`get_approval`<br>`get_approval_context`<br>`request_approval`<br>`decide_approval`<br>`comment_on_approval` | `route_wake`<br>`append_audit_record` | company approval, decision, issue-link, comment, and governed-action services | `company`<br>`task`<br>`approvals`<br>`actor`<br>`wake`<br>`audit`<br>`idempotency` | 6 | governed semantic receipts, audit references, and attention/continuation linkage | Production binding and additive governed-action receipts are unbound; board-only authority stays outside grants. |
| [`ar` — Artifacts](#behavior-group-ar-artifacts) | always tools + artifact/work-product services | `register_deliverable` | `append_audit_record` | attachment upload and issue work-product routes | `task`<br>`artifacts`<br>`workProducts`<br>`workspace`<br>`audit`<br>`idempotency` | 4 | artifact/work-product reference and durable inspectability receipt; never binary bytes | Production upload/register composite and additive durable-reference receipt are unbound. |
| [`er` — Errors and critical rules](#behavior-group-er-errors-and-critical-rules) | runner/control plane + optional workspace/wake tools | `get_workspace_runtime`<br>`control_workspace_service`<br>`schedule_wake`<br>`inspect_operation_result` | `release_task`<br>`enforce_budget`<br>`persist_run`<br>`replay_run`<br>`reconcile_run` | workspace runtime, monitor/recovery, budget, run persistence/replay, release, and terminal services | `workspace`<br>`budget`<br>`run`<br>`wake`<br>`audit`<br>`idempotency`<br>`fault` | 9 | runtime/workspace/attention/run lifecycle, typed denials, replay facts, and terminal causality | Budget stop reasons and semantic denial/conflict receipts require additive v1 envelopes; inspect_operation_result remains scenario-only. |
| [`rf` — Reference files](#behavior-group-rf-reference-files) | always instruction tools + optional domain tools + test-only escape hatch | `list_cases`<br>`upsert_case`<br>`list_routines`<br>`manage_routine`<br>`create_skill`<br>`list_company_skills`<br>`sync_company_skills`<br>`list_secret_metadata`<br>`read_secret_value`<br>`export_company`<br>`administer_company`<br>`generic_api_request`<br>`search_api`<br>`call_api`<br>`create_project`<br>`read_agent_instructions`<br>`update_agent_instructions`<br>`get_agent_instruction_history`<br>`restore_agent_instructions` | `append_audit_record` | agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services | `company`<br>`cases`<br>`routines`<br>`skills`<br>`secrets`<br>`audit`<br>`fault` | 22 | bounded domain projections, redacted broker receipts, company diffs, and audit references | Instruction read/update/history/restore require the live canonical revision service, current responsible-user permissions, and pinned base revisions for writes. Their service, authority, and product E2E tests are separate from the legacy mock scenarios. Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage. |
| [`rf` — Reference files](#behavior-group-rf-reference-files) | always instruction tools + optional domain tools + test-only escape hatch | `list_cases`<br>`upsert_case`<br>`list_routines`<br>`manage_routine`<br>`create_skill`<br>`update_skill`<br>`list_company_skills`<br>`sync_company_skills`<br>`list_secret_metadata`<br>`read_secret_value`<br>`export_company`<br>`administer_company`<br>`generic_api_request`<br>`search_api`<br>`call_api`<br>`create_project`<br>`read_agent_instructions`<br>`update_agent_instructions`<br>`get_agent_instruction_history`<br>`restore_agent_instructions` | `append_audit_record` | agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services | `company`<br>`cases`<br>`routines`<br>`skills`<br>`secrets`<br>`audit`<br>`fault` | 22 | bounded domain projections, redacted broker receipts, company diffs, and audit references | Instruction read/update/history/restore require the live canonical revision service, current responsible-user permissions, and pinned base revisions for writes. Their service, authority, and product E2E tests are separate from the legacy mock scenarios. Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage. |
| [`mh` — Multi-hop](#behavior-group-mh-multi-hop) | composed semantic operations + control-plane continuation | `create_task`<br>`set_dependencies`<br>`request_human_input`<br>`request_approval`<br>`register_deliverable` | `route_wake`<br>`reconcile_run` | delegation, dependency, interaction, approval, artifact, and terminal orchestration services | `task`<br>`blockers`<br>`interactions`<br>`approvals`<br>`artifacts`<br>`wake`<br>`run`<br>`audit` | 4 | correlated operation receipts, state diffs, attention hops, work assessment, status decision, and terminal outcome | No generic transaction tool is allowed; shared mock/real conformance must prove each composed effect. |
| [`rs` — Restraint and no-call](#behavior-group-rs-restraint-and-no-call) | policy/exposure layer | `answer_status_question`<br>`read_secret_value`<br>`generic_api_request` | `enforce_budget` | task-mode, secret-broker, test-scope, pause, and budget policy checks | `actor`<br>`task`<br>`budget`<br>`secrets`<br>`audit`<br>`fault` | 3 | absence of forbidden effects plus typed policy denial/redaction receipts when a call is attempted | Typed redaction/authorization receipts need additive v1 evidence; generic_api_request is never a product fallback. |
| [`wk` — Wake situations](#behavior-group-wk-wake-situations) | control plane + always context/history tools | `get_task_context`<br>`get_task_history`<br>`schedule_wake` | `select_work`<br>`route_wake` | wakeup requests, heartbeat context, comment/interaction/approval/blocker wake routing, and scheduled wake services | `wake`<br>`task`<br>`comments`<br>`interactions`<br>`approvals`<br>`blockers`<br>`run` | 8 | attention request routing/resolution plus resumed session/run causality | Production scheduling binding is unbound; control-plane routing remains non-callable. |
@@ -463,9 +464,9 @@ Current responsibility-based paths are normative. Numbered `phase-*` or mileston
### Catalog split and deliberate replacement
- Scenario/eval catalog: **39** operations.
- Live dispatcher catalog: **41** operations.
- Shared: **27**; union/canonical authority: **53**.
- Scenario/eval catalog: **40** operations.
- Live dispatcher catalog: **42** operations.
- Shared: **28**; union/canonical authority: **54**.
- Scenario-only: `administer_company`, `export_company`, `inspect_operation_result`, `list_cases`, `list_company_skills`, `list_goals`, `list_routines`, `list_secret_metadata`, `manage_routine`, `read_secret_value`, `sync_company_skills`, `upsert_case`.
- Live-only: `call_api`, `create_project`, `get_agent`, `get_agent_instruction_history`, `get_approval`, `get_approval_context`, `hire_agent`, `list_project_repositories`, `read_agent_instructions`, `restore_agent_instructions`, `schedule_wake`, `search_api`, `set_task_title`, `update_agent_instructions`.
- The generated provider contract contains exactly the live catalog; the canonical union remains the migration authority until all scenario-only operations are either implemented, deferred, or removed by an explicit reconciliation decision.
packages/paperclip-runner/src/catalog/catalog-docs.test.ts
+1 -1
View File
@@ -43,7 +43,7 @@ describe("capability-semantic-tools.md stays in sync with the catalog", () => {
expect(Number(alwaysHeader)).toBe(always.length);
expect(Number(optionalHeader)).toBe(optional.length);
// Guard against the historical 36-vs-37 drift the reconciliation plan flagged.
expect(SCENARIO_CATALOG.length).toBe(39);
expect(SCENARIO_CATALOG.length).toBe(40);
});
it("lists exactly the catalog's operation ids in the counts region", () => {
packages/paperclip-runner/src/catalog/reconciliation.test.ts
+7 -7
View File
@@ -19,10 +19,10 @@ const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..", "..")
describe("canonical semantic-catalog reconciliation authority", () => {
it("pins the reconciled op-set relationship between the two catalogs", () => {
const summary = capabilityCatalogReconciliation();
expect(summary.scenarioCount).toBe(39);
expect(summary.liveCount).toBe(41);
expect(summary.sharedCount).toBe(27);
expect(summary.unionCount).toBe(53);
expect(summary.scenarioCount).toBe(40);
expect(summary.liveCount).toBe(42);
expect(summary.sharedCount).toBe(28);
expect(summary.unionCount).toBe(54);
// Any operation added to or removed from either catalog without a
// reconciliation decision changes these exact sets and fails the gate.
expect(summary.liveOnly).toEqual([
@@ -58,7 +58,7 @@ describe("canonical semantic-catalog reconciliation authority", () => {
});
it("is the single source both catalogs derive their operation set from", () => {
expect(CAPABILITY_CANONICAL_OPERATIONS).toHaveLength(53);
expect(CAPABILITY_CANONICAL_OPERATIONS).toHaveLength(54);
const canonicalIds = new Set(CAPABILITY_CANONICAL_OPERATIONS.map((operation) => operation.operationId));
// Neither catalog may contain an operation absent from the canonical source.
for (const tool of SCENARIO_CATALOG) expect(canonicalIds.has(tool.operationId)).toBe(true);
@@ -92,7 +92,7 @@ describe("canonical semantic-catalog reconciliation authority", () => {
});
it("names placement, claims, task modes, side-effect class, idempotency, redaction, mock mapping, real binding status, and PRP evidence for every operation", () => {
expect(CAPABILITY_CANONICAL_CATALOG).toHaveLength(53);
expect(CAPABILITY_CANONICAL_CATALOG).toHaveLength(54);
for (const operation of CAPABILITY_CANONICAL_CATALOG) {
expect(operation.placement).toMatch(/^(always|optional)_agent_tool$/);
expect(Array.isArray(operation.requiredClaims)).toBe(true);
@@ -118,7 +118,7 @@ describe("canonical semantic-catalog reconciliation authority", () => {
it("classifies real binding status so generic_api_request is never product coverage", () => {
const summary = capabilityCatalogReconciliation();
expect(summary.byRealBindingStatus).toEqual({
live_codex: 40,
live_codex: 41,
scenario_mock: 12,
test_only: 1,
});
packages/paperclip-runner/src/catalog/semantic-action-types.ts
+1
View File
@@ -27,6 +27,7 @@ export type PaperclipSemanticActionId =
| "reassign_task"
| "set_dependencies"
| "create_skill"
| "update_skill"
| "create_project"
| "list_project_repositories"
| "list_projects"
packages/paperclip-runner/src/contracts/completion-result.test.ts
+4
View File
@@ -3,8 +3,10 @@ import { describe, expect, it } from "vitest";
import {
PRP_BLOCK_RESULT_OUTPUT_SCHEMA,
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_COMPLETION_RESULT_OUTPUT_SCHEMA,
PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_COMPLETION_TOOL_DESCRIPTION,
} from "./completion-result.js";
import { codexSemanticToolSpecs } from "../drivers/codex/codex-driver-values.js";
@@ -83,6 +85,8 @@ describe("provider-neutral completion result schema", () => {
);
expect(finish.inputSchema).toEqual(PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA);
expect(block.inputSchema).toEqual(PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA);
expect(finish.description).toBe(PRP_COMPLETION_TOOL_DESCRIPTION);
expect(block.description).toBe(PRP_BLOCK_TOOL_DESCRIPTION);
});
it("allows only a response-wake continuation when completion explicitly yields", () => {
packages/paperclip-runner/src/contracts/completion-result.ts
+5
View File
@@ -1,6 +1,11 @@
/** Provider-neutral semantic completion tools and their strict model-facing schemas. */
export const PRP_COMPLETION_TOOL_NAME = "paperclip_finish" as const;
export const PRP_BLOCK_TOOL_NAME = "paperclip_block" as const;
/** Native Runner guidance; legacy adapters use their own skill/API completion paths. */
export const PRP_COMPLETION_TOOL_DESCRIPTION =
"Report completed work (done), work requiring review (needs_review), or an explicit wait for the next response (yielded with response_wake). Use the current completion contract and supporting evidence. If rejected, correct the report and retry. After acceptance, read the returned outcome; do not claim completion while gated. Explain any required approval with its supplied link and action, then write the final response and end the turn without further tool calls.";
export const PRP_BLOCK_TOOL_DESCRIPTION =
"Report work that cannot continue because of a concrete blocker. Identify the blocker, its owner, and the action needed to unblock it; use the current completion contract and supporting evidence. If rejected, correct the report and retry. After acceptance, read the returned outcome, explain the blocker and any required action in the final response, and end the turn without further tool calls.";
export const PRP_SEMANTIC_TOOL_NAMES = [
PRP_COMPLETION_TOOL_NAME,
PRP_BLOCK_TOOL_NAME,
packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.lifecycle.test.ts
+10
View File
@@ -1,3 +1,9 @@
import {
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_COMPLETION_TOOL_DESCRIPTION,
} from "../../contracts/completion-result.js";
import {
CODEX_BLOCK_RESULT_OUTPUT_SCHEMA,
CODEX_INVALID_REQUEST,
@@ -540,6 +546,10 @@ describe("Codex app-server Codex driver", () => {
expect(started.developerInstructions).toBe(conversationMode === "direct" ? undefined : baseInstructions);
expect(resumed.developerInstructions).toBe(conversationMode === "direct" ? "" : baseInstructions);
expect(resumed.dynamicTools).toEqual(started.dynamicTools);
expect(started.dynamicTools).toEqual(conversationMode === "direct" ? [] : [
{ name: "paperclip_finish", description: PRP_COMPLETION_TOOL_DESCRIPTION, inputSchema: PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA },
{ name: "paperclip_block", description: PRP_BLOCK_TOOL_DESCRIPTION, inputSchema: PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA },
]);
expect(resumed.config).toEqual(started.config);
await recovered.session?.close({ reason: "verified additive recovery" });
});
packages/paperclip-runner/src/drivers/codex/codex-driver-values.ts
+6 -4
View File
@@ -1,5 +1,9 @@
import type { PersistedHarnessProviderIdentity } from "../../contracts/harness-driver.js";
import type { NativeUserMessage } from "../../contracts/types.js";
import {
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_COMPLETION_TOOL_DESCRIPTION,
} from "../../contracts/completion-result.js";
import {
CODEX_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
CODEX_BLOCK_TOOL_NAME,
@@ -232,8 +236,7 @@ export function differingJsonPaths(
function finishToolSpec(): Record<string, unknown> {
return {
name: CODEX_COMPLETION_TOOL_NAME,
description:
"Return the one semantic completion result for this task, including an explicit response_wake yield when waiting for the next response.",
description: PRP_COMPLETION_TOOL_DESCRIPTION,
inputSchema: CODEX_RESULT_PROVIDER_INPUT_SCHEMA,
};
}
@@ -241,8 +244,7 @@ function finishToolSpec(): Record<string, unknown> {
function blockToolSpec(): Record<string, unknown> {
return {
name: CODEX_BLOCK_TOOL_NAME,
description:
"Return the one semantic result when the task cannot continue.",
description: PRP_BLOCK_TOOL_DESCRIPTION,
inputSchema: CODEX_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
};
}
packages/paperclip-runner/src/drivers/opencode/mcp-bridge.test.ts
+8 -2
View File
@@ -1,4 +1,10 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import {
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_COMPLETION_TOOL_DESCRIPTION,
} from "../../contracts/completion-result.js";
import { canonicalOpenCodeMcpToolName, startOpenCodeMcpBridge, type OpenCodeMcpBridge } from "./mcp-bridge.js";
@@ -38,8 +44,8 @@ describe("OpenCode MCP bridge", () => {
expect(await listed.json()).toMatchObject({
result: { tools: [
{ name: "documents.read" },
{ name: "paperclip_finish" },
{ name: "paperclip_block" },
{ name: "paperclip_finish", description: PRP_COMPLETION_TOOL_DESCRIPTION, inputSchema: PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA },
{ name: "paperclip_block", description: PRP_BLOCK_TOOL_DESCRIPTION, inputSchema: PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA },
] },
});
});
packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts
+4 -2
View File
@@ -4,8 +4,10 @@ import { Ajv2020, type ValidateFunction } from "ajv/dist/2020.js";
import {
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_BLOCK_TOOL_NAME,
PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_COMPLETION_TOOL_DESCRIPTION,
PRP_COMPLETION_TOOL_NAME,
} from "../../contracts/completion-result.js";
@@ -117,8 +119,8 @@ function normalizeTools(
seen.add(name);
}
if (includeTerminalTools) for (const terminal of [
{ name: PRP_COMPLETION_TOOL_NAME, description: "Return the semantic completion result.", inputSchema: PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA },
{ name: PRP_BLOCK_TOOL_NAME, description: "Return the semantic blocked result.", inputSchema: PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA },
{ name: PRP_COMPLETION_TOOL_NAME, description: PRP_COMPLETION_TOOL_DESCRIPTION, inputSchema: PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA },
{ name: PRP_BLOCK_TOOL_NAME, description: PRP_BLOCK_TOOL_DESCRIPTION, inputSchema: PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA },
]) {
if (!seen.has(terminal.name)) tools.push(terminal);
}
packages/paperclip-runner/src/drivers/runner-tool-bridge.test.ts
+8 -2
View File
@@ -1,6 +1,12 @@
import { connect } from "node:net";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_COMPLETION_TOOL_DESCRIPTION,
} from "../contracts/completion-result.js";
import {
canonicalRunnerToolName,
@@ -48,8 +54,8 @@ describe("runner semantic MCP bridge", () => {
result: {
tools: [
{ name: "documents.read" },
{ name: "paperclip_finish" },
{ name: "paperclip_block" },
{ name: "paperclip_finish", description: PRP_COMPLETION_TOOL_DESCRIPTION, inputSchema: PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA },
{ name: "paperclip_block", description: PRP_BLOCK_TOOL_DESCRIPTION, inputSchema: PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA },
],
},
});
packages/paperclip-runner/src/drivers/runner-tool-bridge.ts
+4 -2
View File
@@ -10,8 +10,10 @@ import { Ajv2020, type ValidateFunction } from "ajv/dist/2020.js";
import {
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_BLOCK_TOOL_NAME,
PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
PRP_COMPLETION_TOOL_DESCRIPTION,
PRP_COMPLETION_TOOL_NAME,
} from "../contracts/completion-result.js";
@@ -70,12 +72,12 @@ const MAX_RESULT_CHUNK_BYTES = 64 * 1024;
const RESERVED_TOOLS: readonly RunnerToolDefinition[] = [
{
name: PRP_COMPLETION_TOOL_NAME,
description: "Return the semantic completion result.",
description: PRP_COMPLETION_TOOL_DESCRIPTION,
inputSchema: PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA,
},
{
name: PRP_BLOCK_TOOL_NAME,
description: "Return the semantic blocked result.",
description: PRP_BLOCK_TOOL_DESCRIPTION,
inputSchema: PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
},
];
packages/paperclip-runner/src/eval/workflow-evals.test.ts
+3 -3
View File
@@ -466,13 +466,13 @@ describe("workflow reports and stress traceability", () => {
candidateFailures: 36,
});
expect(report.coverage).toMatchObject({
canonicalOperations: 53,
canonicalOperations: 54,
capabilityCases: 106,
workflows: 12,
stressFindings: 44,
stressExclusions: 1,
});
expect(report.coverage.operations).toHaveLength(53);
expect(report.coverage.operations).toHaveLength(54);
expect(report.coverage.composedWorkflows).toHaveLength(12);
expect(
report.coverage.operations.find(
@@ -480,7 +480,7 @@ describe("workflow reports and stress traceability", () => {
)?.workflowIds.length,
).toBeGreaterThan(0);
expect(renderRunnerWorkflowMarkdown(report)).toContain(
"53 operations · 106 capability cases · 12 workflows",
"54 operations · 106 capability cases · 12 workflows",
);
expect(renderRunnerWorkflowJUnit(report)).toContain(
'tests="36" failures="36" skipped="0"',
packages/paperclip-runner/src/generated/capability-contract.ts
+2 -2
View File
@@ -6,9 +6,9 @@ export type CapabilityPrimaryDisposition =
| "optional_agent_tool";
export const capabilityInventoryCounts = {
"skillReferenceCapabilities": 157,
"skillReferenceCapabilities": 160,
"evalCases": 106,
"normativeRows": 263,
"normativeRows": 266,
"legacyMcpAliases": 42
} as const;
packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts
+14 -3
View File
@@ -54,6 +54,10 @@ import {
CODEX_SKILLLESS_BASE_INSTRUCTIONS,
createCodexTaskEnvelope,
} from "../contracts/codex.js";
import {
PRP_BLOCK_TOOL_DESCRIPTION,
PRP_COMPLETION_TOOL_DESCRIPTION,
} from "../contracts/completion-result.js";
import {
CodexAppServerDriver,
codexSemanticToolSpecs,
@@ -1336,14 +1340,19 @@ it("includes ACPX terminal tools in the authenticated bridge catalog", () => {
});
});
it("preserves answer and internal wait descriptions in the serialized native tool catalog", () => {
it.each(["codex", "opencode", "claude_managed", "aws_agentcore", "acpx"] as const)("preserves answer and internal wait descriptions in the serialized native %s tool catalog", (provider) => {
const catalog = JSON.parse(
JSON.stringify(authorizedToolSetForProvider("codex", codexSemanticToolSpecs())),
JSON.stringify(authorizedToolSetForProvider(provider, codexSemanticToolSpecs())),
);
const finish = catalog.operations.find(
(operation: { operationId: string }) =>
operation.operationId === "paperclip_finish",
);
const block = catalog.operations.find(
(operation: { operationId: string }) => operation.operationId === "paperclip_block",
);
expect(finish.description).toBe(PRP_COMPLETION_TOOL_DESCRIPTION);
expect(block.description).toBe(PRP_BLOCK_TOOL_DESCRIPTION);
expect(finish.inputSchema.properties.summary.description).toContain(
"complete user-facing answer",
);
@@ -4154,7 +4163,9 @@ it("captures exact provider frames and correlates Rust and TypeScript interpreta
);
const tracePath = join(traceDirectory, "trace.ndjson");
const bundle = createCapabilityRunnerdCodexTransport({
runnerBinary: defaultCapabilityRunnerdBinary(),
// Qualification builds a debug daemon for this exact source. Its selected
// binary must win over any separately staged product/runtime artifact.
runnerBinary: process.env.PAPERCLIP_STOCK_PREFLIGHT_RUNNERD ?? defaultCapabilityRunnerdBinary(),
codexCommand: fakeCodex,
codexArgs: fakeCodexArgs(traceDirectory, "--structured-activity"),
stateDirectory: join(traceDirectory, "state"),
packages/paperclip-runner/src/mock-core/capability-control-plane-types.ts
+2
View File
@@ -359,6 +359,7 @@ interface CapabilityBaseCommand {
export type CapabilitySemanticCommand =
| (CapabilityBaseCommand & { kind: "create_skill"; name: string; slug?: string; description: string; markdown: string })
| (CapabilityBaseCommand & { kind: "update_skill"; skillId: string; markdown: string; expectedVersionId: string })
| (CapabilityBaseCommand & { kind: "report_progress"; body: string })
| (CapabilityBaseCommand & {
kind: "write_document";
@@ -479,6 +480,7 @@ export type CapabilityCommandOutcome =
*/
export const CAPABILITY_COMMAND_REQUIRED_CLAIMS = {
create_skill: [],
update_skill: [],
report_progress: [],
write_document: [],
request_human_input: [],
packages/paperclip-runner/src/mock-core/capability-mock-control-plane-adapter.ts
+15
View File
@@ -734,6 +734,21 @@ export class CapabilityMockControlPlaneAdapter implements CapabilityMockControlP
entityRefs.push(`skill:${skill.id}`);
break;
}
case "update_skill": {
const skill = this.#state.skills?.find(candidate => candidate.id === command.skillId && candidate.companyId === run.companyId);
if (!skill) throw new CapabilityMockControlPlaneError("fixture_state_invalid", "Skill not found");
if (skill.versionId !== command.expectedVersionId) throw new CapabilityMockControlPlaneError("fixture_state_invalid", "Skill version changed");
const document = parseFrontmatterMarkdown(command.markdown);
if (!document.hasFrontmatter || !validateSkillFrontmatter(document.frontmatter) || !document.body.trim()) {
throw new CapabilityMockControlPlaneError("invalid_skill_document", "Provide a complete SKILL.md with valid frontmatter and a nonempty body");
}
skill.name = String(document.frontmatter.name);
skill.description = String(document.frontmatter.description);
skill.markdown = command.markdown;
skill.versionId = this.#id("skill-version");
entityRefs.push(`skill:${skill.id}`);
break;
}
case "write_document": {
requireText(command.key, "document key");
requireText(command.title, "document title");
packages/paperclip-runner/src/protocol-actions/index.ts
+2
View File
@@ -5,6 +5,7 @@ import { getAgentInstructionHistoryAction } from "./get-agent-instruction-histor
import { restoreAgentInstructionsAction } from "./restore-agent-instructions.js";
import { reassignTaskAction } from "./reassign-task.js";
import { createSkillAction } from "./create-skill.js";
import { updateSkillAction } from "./update-skill.js";
import { createProjectAction } from "./create-project.js";
import { listProjectRepositoriesAction } from "./list-project-repositories.js";
import { searchApiAction } from "./search-api.js";
@@ -61,6 +62,7 @@ export const PAPERCLIP_PROTOCOL_ACTIONS = deepFreezeProtocolAction([
restoreAgentInstructionsAction,
createSkillAction,
updateSkillAction,
createProjectAction,
listProjectRepositoriesAction,
searchApiAction,
packages/paperclip-runner/src/protocol-actions/update-skill.ts
+54
View File
@@ -0,0 +1,54 @@
/** Canonical definition and documentation for `update_skill`. */
export const updateSkillAction = {
id: "update_skill",
canonical: {
operationId: "update_skill", surfaces: ["scenario", "live"], placement: "optional_agent_tool",
optionalGroup: "company_skills", requiredClaims: [], taskModes: ["standard", "skill_test"],
sideEffectClass: "company_write", idempotency: "required", disabledByDefault: false,
realBindingStatus: "live_codex", realServiceBinding: "PaperclipRunnerToolAuthority",
prpEvidence: "Authenticated skill file API with version guard, durable receipt, and attributed activity.",
prpBindingStatus: "bound", legacyAliases: [],
},
documentation: {
title: "Update skill",
description: "Replace the primary SKILL.md of an existing company skill. Read the current skill and version first. Supply the complete file, expectedVersionId, and an idempotencyKey; reuse the key on retries. A version conflict requires a fresh read. This uses the existing file API and company edit policy.",
note: null,
},
examples: {
call: { operationId: "update_skill", input: { skillId: "00000000-0000-4000-8000-000000000001", expectedVersionId: "00000000-0000-4000-8000-000000000002", idempotencyKey: "edit-1", markdown: "---\nname: review\ndescription: Review changes.\n---\n\n# Review\nInspect the changes.\n" } },
scenarioCall: { operationId: "update_skill", idempotencyKey: "edit-1", input: { skillId: "skill-example", expectedVersionId: "version-example", markdown: "---\nname: review\ndescription: Review changes.\n---\n\n# Review\nInspect the changes.\n" } },
success: { ok: true, operationId: "update_skill", result: { skillId: "skill-example", path: "SKILL.md", versionId: "version-next", studioPath: "/skills/studio/skill-example" } },
},
live: {
order: 47,
descriptor: {
schema: "paperclip.semantic-tool.v1", operationId: "update_skill", version: 1,
title: "Update skill", description: "Replace an existing company skill's complete SKILL.md using the current version as a guard. Reuse idempotencyKey on lost-response retries; read again after a version conflict.",
exposure: "optional", requiredClaims: [], allowedModes: ["standard", "skill_test"],
inputSchema: { type: "object", properties: {
skillId: { type: "string", minLength: 1 },
markdown: { type: "string", minLength: 1, maxLength: 200000 },
expectedVersionId: { type: "string", minLength: 1 },
idempotencyKey: { type: "string", minLength: 1, maxLength: 240 },
}, required: ["skillId", "markdown", "expectedVersionId", "idempotencyKey"], additionalProperties: false },
outputSchema: { type: "object", additionalProperties: true },
},
},
scenario: {
order: 47,
successExample: { schema: "paperclip.capability.tool-result.v1", ok: true, operationId: "update_skill", operationResultId: "example-result", value: { commandId: "example", disposition: "applied", stateRevision: 1, entityRefs: ["skill:skill-example"], scheduledWakeIds: [] }, commandResult: null, authorization: {} },
descriptor: {
operationId: "update_skill", version: 1, title: "Update skill",
description: "Replace an existing company skill's complete SKILL.md with an expected version guard and a retry key.",
inputSchema: { type: "object", properties: {
skillId: { type: "string", minLength: 1 },
markdown: { type: "string", minLength: 1, maxLength: 200000 },
expectedVersionId: { type: "string", minLength: 1 },
}, required: ["skillId", "markdown", "expectedVersionId"], additionalProperties: false },
outputSchema: { type: "object", additionalProperties: true },
disposition: "optional_agent_tool", optionalGroup: "company_skills", requiredClaims: [],
taskModes: ["standard", "skill_test"], sideEffectClass: "company_write", idempotency: "required", redaction: [],
mockCommandMapping: { kind: "semantic_command", commandKind: "update_skill" },
},
},
} as const;
packages/paperclip-runner/src/semantic-tools/discovery.ts
+1 -1
View File
@@ -33,7 +33,7 @@ const NAMESPACE: Readonly<Record<CapabilitySemanticOperationId, string>> = Objec
update_agent_instructions: "agent_instructions",
get_agent_instruction_history: "agent_instructions",
restore_agent_instructions: "agent_instructions",
create_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects",
create_skill: "skills", update_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects",
set_dependencies: "delegation", reassign_task: "delegation", list_approvals: "governance", get_approval: "governance",
get_approval_context: "governance", request_approval: "governance",
decide_approval: "governance", comment_on_approval: "governance",
packages/paperclip-runner/src/semantic-tools/dispatcher.ts
+11
View File
@@ -331,6 +331,10 @@ export class CapabilitySemanticDispatcher {
slug: typeof input.slug === "string" ? input.slug : undefined,
description: requiredString(input.description), markdown: requiredString(input.markdown) };
break;
case "update_skill":
command = { kind: "update_skill", taskId, skillId: requiredString(input.skillId),
expectedVersionId: requiredString(input.expectedVersionId), markdown: requiredString(input.markdown) };
break;
case "write_document":
command = {
kind: "write_document",
@@ -420,6 +424,13 @@ export class CapabilitySemanticDispatcher {
versionId: skill.versionId, studioPath: `/skills/studio/${skill.id}`,
});
}
if (operationId === "update_skill" && outcome.ok) {
const id = outcome.result.entityRefs.find(ref => ref.startsWith("skill:"))?.slice(6);
const skill = this.port.snapshot().skills?.find(candidate => candidate.id === id);
if (skill) return readSuccess(outcome.result.stateRevision, {
skillId: skill.id, path: "SKILL.md", versionId: skill.versionId, studioPath: `/skills/studio/${skill.id}`,
});
}
return commandOutcome(outcome);
}
packages/paperclip-runner/src/semantic-tools/paperclip-discovery.ts
+1 -1
View File
@@ -38,7 +38,7 @@ const NAMESPACE: Readonly<Record<PaperclipSemanticActionId, string>> =
get_workspace_runtime: "workspace",
control_workspace_service: "workspace",
set_dependencies: "delegation", reassign_task: "delegation",
create_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects",
create_skill: "skills", update_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects",
request_approval: "governance",
decide_approval: "governance",
comment_on_approval: "governance",
packages/paperclip-runner/src/semantic-tools/semantic-tools.test.ts
+20 -1
View File
@@ -43,6 +43,25 @@ async function running(
}
describe("Capability semantic catalog and authorization", () => {
it("updates an existing skill once, rejects a stale version, and denies an ungranted capability", async () => {
const adapter = await running();
const dispatcher = new CapabilitySemanticDispatcher(adapter);
const markdown = "---\nname: release-review\ndescription: Review release notes.\n---\n# Review\nCheck each note.";
const created = await dispatcher.dispatch({ runId: OPEN.identity.runId, callId: "create", operationId: "create_skill",
input: { name: "release-review", description: "Review release notes.", markdown, idempotencyKey: "create" } });
expect(created.ok).toBe(true);
if (!created.ok) return;
const args = { skillId: (created.result as { id: string }).id, expectedVersionId: (created.result as { versionId: string }).versionId,
markdown: markdown + " Inspect tests.", idempotencyKey: "update-once" };
const call = { runId: OPEN.identity.runId, callId: "update", operationId: "update_skill" as const, input: args };
const first = await dispatcher.dispatch(call);
expect(first).toMatchObject({ ok: true, result: { skillId: args.skillId, path: "SKILL.md", versionId: expect.any(String) } });
expect(await dispatcher.dispatch({ ...call, callId: "retry" })).toMatchObject({ ok: true, result: first.ok ? first.result : {} });
expect(await dispatcher.dispatch({ ...call, callId: "stale", input: { ...args, idempotencyKey: "next" } })).toMatchObject({ ok: false });
const denied = new CapabilitySemanticDispatcher(adapter, { scenario: { id: "no-update", claims: [], denyOperations: ["update_skill"] } });
expect(await denied.dispatch({ ...call, callId: "denied", input: { ...args, idempotencyKey: "denied" } })).toMatchObject({ ok: false, denial: { code: "scenario_denied" } });
});
it("creates a durable skill once and returns its reference on retry", async () => {
const adapter = await running();
const dispatcher = new CapabilitySemanticDispatcher(adapter);
@@ -94,7 +113,7 @@ describe("Capability semantic catalog and authorization", () => {
it("publishes a stable narrow catalog without credentials or control-plane-owned tools", () => {
const names = CAPABILITY_SEMANTIC_TOOL_CATALOG.map((tool) => tool.operationId);
expect(new Set(names).size).toBe(names.length);
expect(names).toHaveLength(41);
expect(names).toHaveLength(42);
expect(names).toContain("get_task_context");
expect(names).toContain("finish_task");
expect(names).not.toContain("checkout_task");
packages/paperclip-runner/src/semantic-tools/types.ts
+1
View File
@@ -47,6 +47,7 @@ export type CapabilitySemanticOperationId =
| "reassign_task"
| "set_dependencies"
| "create_skill"
| "update_skill"
| "create_project"
| "list_project_repositories"
| "list_projects"
packages/paperclip-runner/src/tools/capability-semantic-tool-runtime.ts
+3
View File
@@ -1543,6 +1543,9 @@ function commandForOperation(
return { kind: "create_skill", name: requireString(input.name),
slug: typeof input.slug === "string" ? input.slug : undefined,
description: requireString(input.description), markdown: requireString(input.markdown) };
case "update_skill":
return { kind: "update_skill", skillId: requireString(input.skillId),
expectedVersionId: requireString(input.expectedVersionId), markdown: requireString(input.markdown) };
case "write_document":
return {
kind: "write_document",
packages/paperclip-runner/test/capability-contract.test.mjs
+14 -1
View File
@@ -18,7 +18,8 @@ test("generated Capability inventory has full source coverage", async () => {
readRows("eval-traceability.yaml"),
]);
assert.equal(capabilities.length, 158);
assert.equal(capabilities.length, 161);
assert.equal(capabilities.filter(row => row.sourceAnchor.startsWith("skills/paperclip/references/issue-documents.md#")).length, 3);
assert.equal(tools.length, 42);
assert.equal(evals.length, 106);
assert.equal(new Set(evals.map((row) => row.group)).size, 16);
@@ -51,3 +52,15 @@ test("conversational answer guidance has the same agent-operation classification
for (const row of [...generated, ...spec]) assert.equal(row.primaryDisposition, "always_agent_tool");
for (const row of generated) assert.equal(row.semanticOperation, "call_api");
});
test("both capability inventories cover every declared skill source", async () => {
const contract = JSON.parse(await readFile(resolve(import.meta.dirname, "../spec/capability/source-contract.json"), "utf8"));
const generated = await readRows("capabilities.yaml");
const normative = decodeInventory(await readFile(resolve(import.meta.dirname, "../spec/capability/capabilities.yaml"), "utf8"));
assert.deepEqual(normative.generatedFrom, contract.skillSources);
for (const source of contract.skillSources) {
assert.ok(generated.some(row => row.sourceAnchor.startsWith(`${source}#`)), `Missing generated source ${source}`);
assert.ok(normative.rows.some(row => row.sourceAnchor.startsWith(`${source}:`)), `Missing normative source ${source}`);
}
assert.equal(normative.rows.filter(row => row.sourceAnchor.startsWith("skills/paperclip/references/issue-documents.md:")).length, 3);
});
packages/shared/src/types/execution-projection.ts
+2
View File
@@ -9,6 +9,8 @@ export interface ExecutionBlocker {
runError?: string | null;
/** Candidate only: admission must still verify termination and all gates. */
canContinue?: boolean;
/** Explicit Retry candidate; the server rechecks stop proof and execution gates. */
canRetry?: boolean;
savedMessageCount?: number;
}
packages/shared/src/validators/company-skill.ts
+2
View File
@@ -395,6 +395,8 @@ export const companySkillFileUpdateSchema = z.object({
content: z.string(),
encoding: z.enum(["utf8", "base64"]).optional(),
executable: z.boolean().optional(),
expectedVersionId: z.string().guid().nullable().optional(),
idempotencyKey: z.string().min(1).max(240).optional(),
});
export const companySkillFileDeleteSchema = z.object({
server/src/__tests__/agent-live-run-routes.test.ts
+1 -1
View File
@@ -1163,7 +1163,7 @@ describe("agent live run routes", () => {
expect(res.status, JSON.stringify(res.body)).toBe(202);
expect(res.body).toEqual(receipt);
expect(mockHeartbeatService.getRun).toHaveBeenCalledWith(
failedChatRunId,
failedChatRunId, { includeExecutionEvidence: true },
);
expect(
mockChatRunRetries.prepareFailedChatRunRetry,
server/src/__tests__/agent-skills-routes.test.ts
+2 -30
View File
@@ -1184,7 +1184,7 @@ describe.sequential("agent skill routes", () => {
expect(returnedAgent.adapterConfig).not.toHaveProperty("promptTemplate");
});
it("materializes the bundled default instruction set for non-CEO agents with no prompt template", async () => {
it("materializes minimal default instructions for non-CEO agents with no prompt template", async () => {
const res = await requestApp(await createApp(), (baseUrl) => request(baseUrl)
.post("/api/companies/company-1/agents")
.send({
@@ -1204,38 +1204,10 @@ describe.sequential("agent skill routes", () => {
adapterType: "claude_local",
}),
expect.objectContaining({
"AGENTS.md": expect.stringMatching(/Start actionable work in the same heartbeat\.[\s\S]*Keep the work moving until it is done\./),
"AGENTS.md": "You are an agent in a Paperclip company.\n",
}),
{ entryFile: "AGENTS.md", replaceExisting: false },
);
expect(mockAgentInstructionsService.materializeManagedBundle).toHaveBeenCalledWith(
expect.any(Object),
expect.objectContaining({
"AGENTS.md": expect.stringContaining('kind: "request_confirmation"'),
}),
expect.any(Object),
);
expect(mockAgentInstructionsService.materializeManagedBundle).toHaveBeenCalledWith(
expect.any(Object),
expect.objectContaining({
"AGENTS.md": expect.stringContaining("confirmation:{issueId}:plan:{revisionId}"),
}),
expect.any(Object),
);
expect(mockAgentInstructionsService.materializeManagedBundle).toHaveBeenCalledWith(
expect.any(Object),
expect.objectContaining({
"AGENTS.md": expect.stringMatching(/PUT \/issues\/\{id\}\/documents\/plan[\s\S]*Re-`GET \/documents\/plan`, assert it returns `200`[\s\S]*latestRevisionId[\s\S]*target=\{ type: 'issue_document', key: 'plan', revisionId: latestRevisionId \}[\s\S]*Never present a plan only in a thread comment or through `ask_user_questions`/),
}),
expect.any(Object),
);
expect(mockAgentInstructionsService.materializeManagedBundle).toHaveBeenCalledWith(
expect.any(Object),
expect.objectContaining({
"AGENTS.md": expect.stringContaining("skills/paperclip/scripts/paperclip-upload-artifact.sh"),
}),
expect.any(Object),
);
});
});
server/src/__tests__/codex-local-execute.test.ts
+1 -2
View File
@@ -1443,9 +1443,8 @@ process.exit(1);
expect(invocationPrompt).toContain("baseRevisionId set to that latestRevisionId");
expect(capture.prompt).not.toContain("Execution contract:");
expect(capture.prompt).not.toContain("Use child issues");
} else {
expect(capture.prompt).toContain("Execution contract:");
}
expect(capture.prompt).not.toContain("Execution contract:");
expect(capture.prompt).not.toContain("Follow the paperclip heartbeat.");
if (resumedSession) {
expect(capture.prompt).not.toContain("You are managed instructions.");
server/src/__tests__/heartbeat-process-recovery.test.ts
+24 -2
View File
@@ -6221,14 +6221,28 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
expect((await db.select().from(issues).where(eq(issues.id, issueId)))[0]).toMatchObject({ status: "blocked", assigneeAgentId: agentId });
});
it("converts a continuation parked for review into a dependency wait on its open sub-tasks", async () => {
const { companyId, agentId, issueId } = await seedStrandedIssueFixture({
it.each([false, "receipt", "launch", "provider", "cleanup"])("converts a continuation parked for review into a dependency wait on its open sub-tasks (dispatch receipt: %s)", async dispatchReceipt => {
const { companyId, agentId, issueId, runId } = await seedStrandedIssueFixture({
status: "in_progress",
runStatus: "cancelled",
retryReason: "issue_continuation_needed",
runErrorCode: "issue_continuation_waiting_on_review",
runError: "Continuation parked: issue is waiting on review/approval",
});
if (dispatchReceipt) {
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, companyId));
await db.update(heartbeatRuns).set({ startedAt: null,
resultJson: { stopReason: "issue_continuation_waiting_on_review", timeoutSource: "stale_queued_run_gate" },
}).where(eq(heartbeatRuns.id, run.id));
}
if (dispatchReceipt === "launch" || dispatchReceipt === "provider") {
await db.insert(heartbeatRunEvents).values({ companyId, runId, agentId, seq: 1,
eventType: dispatchReceipt === "launch" ? "native.process_start_requested" : "provider.event",
stream: "system", level: "info", message: "Contradictory execution evidence" });
} else if (dispatchReceipt === "cleanup") {
await db.insert(environmentLeases).values({ companyId, heartbeatRunId: runId,
provider: "local", status: "pending_cleanup", cleanupStatus: "failed" });
}
const issuePrefix = `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`;
const openChildTodoId = randomUUID();
const openChildInProgressId = randomUUID();
@@ -6270,6 +6284,14 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
const heartbeat = heartbeatService(db);
const result = await heartbeat.reconcileStrandedAssignedIssues();
if (dispatchReceipt && dispatchReceipt !== "receipt") {
expect(result.waitingOnReviewResolved).toBe(0);
expect(result.escalated).toBe(1);
expect(await getExecutionBlocker(db, companyId, issueId)).toMatchObject({
cause: "legacy_execution_requires_reconciliation", runId, canRetry: false,
});
return;
}
expect(result.waitingOnReviewResolved).toBe(1);
expect(result.escalated).toBe(0);
expect(result.issueIds).toEqual([issueId]);
server/src/__tests__/issue-queued-comments-routes.test.ts
+76
View File
@@ -30,6 +30,7 @@ import { issueService } from "../services/issues.js";
import { issueThreadInteractionService } from "../services/issue-thread-interactions.js";
import { remoteTerminationReceipt } from "../services/remote-execution-termination.js";
import { initializeRunIdentity, reconcileSteeredIdentity } from "../services/run-identity.js";
import { appendHeartbeatRunEvent } from "../services/heartbeat-run-events.js";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
@@ -1449,6 +1450,22 @@ describeEmbeddedPostgres("issue queued-comment routes", () => {
});
});
it.each(["paperclip_runner", "codex_local"])("keeps Steer during native preparation even after agent settings change to %s", async adapterType => {
const seeded = await seedQueue();
const steeringCalls = steerNativeSessionMock.mock.calls.length;
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", runtimeModeResolvedAt: null,
runnerProfileJson: { adapterDispatch: { adapterType: "paperclip_runner" } },
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(agents).set({ adapterType }).where(eq(agents.id, seeded.agentId));
const client = app(seeded.companyId);
const initial = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
expect(initial.body).toMatchObject({ protocol: "paperclip_runner_v1", steeringDisposition: "temporarily_unavailable" });
const edited = await request(client).patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "Updated direction" }).expect(200);
expect(edited.body).toMatchObject({ protocol: "paperclip_runner_v1", steeringDisposition: "temporarily_unavailable" });
expect(steerNativeSessionMock.mock.calls).toHaveLength(steeringCalls);
});
it("serializes discard against queued-run claim", async () => {
const seeded = await seedQueue();
await db.delete(issueComments).where(eq(issueComments.id, seeded.commentIds[1]));
@@ -1579,6 +1596,65 @@ describeEmbeddedPostgres("issue queued-comment routes", () => {
});
});
it("allows the runner to persist its acknowledgement while steering, preserving concurrent run updates", async () => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
const originalProfile = { adapterDispatch: { adapterType: "paperclip_runner" },
nativeExecutionInput: { provider: { kind: "codex", model: "gpt-5.6-sol", reasoningEffort: "medium" } } };
await db.update(heartbeatRuns).set({ runnerProfileJson: originalProfile }).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ assigneeAdapterOverrides: { model: "gpt-6-sol", reasoningEffort: "xhigh" } })
.where(eq(issues.id, seeded.issueId));
steerNativeSessionMock.mockImplementationOnce(async () => {
await db.transaction(async tx => {
await tx.execute(sql`set local lock_timeout = '1000ms'`);
await appendHeartbeatRunEvent(tx as unknown as typeof db, {
companyId: seeded.companyId, agentId: seeded.agentId, runId: seeded.runId,
eventType: "item.completed", payload: { kind: "steering_acknowledgement" },
});
await tx.update(heartbeatRuns).set({ resultJson: { concurrentProviderReceipt: "preserve" } })
.where(eq(heartbeatRuns.id, seeded.runId));
});
return { turnId: "turn-before-model-change" };
});
const initial = await request(app(seeded.companyId)).get(`/api/issues/${seeded.issueId}/queued-comments`);
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(steered.status, JSON.stringify(steered.body)).toBe(200);
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId));
expect(run.status).toBe("running");
expect(run.runnerProfileJson).toEqual(originalProfile);
expect(steerNativeSessionMock).toHaveBeenCalledWith(expect.objectContaining({ runId: seeded.runId,
message: "First queued message", correlationId: seeded.commentIds[0] }));
expect(run.resultJson).toMatchObject({ concurrentProviderReceipt: "preserve", queuedSteeringAcknowledgements: {
[seeded.commentIds[0]]: { status: "acknowledged", turnId: "turn-before-model-change" },
} });
expect(steered.body.entries.map((entry: any) => entry.comment.id)).toEqual([seeded.commentIds[1]]);
});
it.each(["cancelled", "failed", "succeeded"] as const)("preserves queued input when the run stops during acknowledgement (%s)", async status => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
steerNativeSessionMock.mockImplementationOnce(async () => {
await db.update(heartbeatRuns).set({ status, finishedAt: new Date(), resultJson: { terminalReceipt: "preserve" } })
.where(eq(heartbeatRuns.id, seeded.runId));
return { turnId: "turn-that-stopped" };
});
const initial = await request(app(seeded.companyId)).get(`/api/issues/${seeded.issueId}/queued-comments`);
const response = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(response.status).toBe(409);
expect(response.body.details).toMatchObject({ code: "queued_comment_stale_target" });
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.status).toBe("deferred_issue_execution");
expect((wake.payload as any)._paperclipWakeContext.wakeCommentIds).toEqual(seeded.commentIds);
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId));
expect(run.resultJson).toEqual({ terminalReceipt: "preserve" });
const activity = await db.select().from(activityLog).where(eq(activityLog.action, "issue.queued_comment_steered"));
expect(activity).toHaveLength(0);
});
it("keeps the identity pending after a steering timeout, then reconciles it on a later acknowledgement", async () => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
server/src/__tests__/run-failure-sentry-real-sdk.test.ts
+7 -1
View File
@@ -35,7 +35,11 @@ afterEach(async () => {
});
describe.skipIf(!sentryPackage)("run failure context with the real Sentry SDK", () => {
it("keeps unconfirmed Stop context off unrelated events and drops arbitrary error fields", async () => {
it.each([
{ phase: "workspace_restore", phaseElapsedMs: 60_123, expectedPhase: "workspace_restore", expectedElapsedMs: 60_123 },
{ phase: "private-provider-phase", phaseElapsedMs: 100, expectedPhase: "unknown", expectedElapsedMs: null },
{ phase: "workspace_restore", phaseElapsedMs: Infinity, expectedPhase: "workspace_restore", expectedElapsedMs: null },
])("keeps bounded unconfirmed Stop context off unrelated events ($expectedPhase)", async ({ phase, phaseElapsedMs, expectedPhase, expectedElapsedMs }) => {
const Sentry = sentryPackage!;
const events: Array<Record<string, unknown>> = [];
vi.stubEnv("SENTRY_DSN_BACKEND", "https://public@example.invalid/1");
@@ -55,6 +59,7 @@ describe.skipIf(!sentryPackage)("run failure context with the real Sentry SDK",
const timeout = Object.assign(new AdapterStopTimeoutError(60_000, {
runId: "11111111-1111-4111-8111-111111111111", adapterType: "cursor",
runtimeMode: "legacy", abortRequested: true,
phase, phaseElapsedMs,
}), {
cause: new Error("private-provider-cause"),
providerResponse: { headers: "private-provider-headers", body: "private-provider-body" },
@@ -72,6 +77,7 @@ describe.skipIf(!sentryPackage)("run failure context with the real Sentry SDK",
contexts: { adapter_stop: {
runId: "11111111-1111-4111-8111-111111111111", adapterType: "cursor",
runtimeMode: "legacy", abortRequested: true, timeoutMs: 60_000,
phase: expectedPhase, phaseElapsedMs: expectedElapsedMs,
} },
});
expect(JSON.stringify(events)).not.toContain("private-provider-");
server/src/__tests__/sentry.test.ts
+6 -2
View File
@@ -136,6 +136,7 @@ describe("captureException", () => {
const error = new AdapterStopTimeoutError(60_000, {
runId: "11111111-1111-4111-8111-111111111111",
adapterType: "cursor", runtimeMode: "legacy", abortRequested: true,
phase: "instruction_collection", phaseElapsedMs: 60_321,
});
Object.assign(error, { providerResponse: "private fixture payload" });
captureException(error);
@@ -144,7 +145,8 @@ describe("captureException", () => {
expect(sdk.captureException.mock.calls[0]).toEqual([
expect.objectContaining({ message: error.message, stack: error.stack }),
{ tags: { error_code: "adapter_stop_unconfirmed" }, fingerprint: ["{{ default }}"], contexts: {
adapter_stop: { runId: "11111111-1111-4111-8111-111111111111", adapterType: "cursor", runtimeMode: "legacy", abortRequested: true, timeoutMs: 60_000 },
adapter_stop: { runId: "11111111-1111-4111-8111-111111111111", adapterType: "cursor", runtimeMode: "legacy", abortRequested: true, timeoutMs: 60_000,
phase: "instruction_collection", phaseElapsedMs: 60_321 },
} },
]);
expect(JSON.stringify(sdk.captureException.mock.calls[0])).not.toContain("private fixture payload");
@@ -159,10 +161,12 @@ describe("captureException", () => {
await sentryReady;
captureException(new AdapterStopTimeoutError(NaN, {
runId: "private fixture payload", adapterType: "private fixture payload", runtimeMode: "private fixture payload",
phase: "private fixture payload", phaseElapsedMs: Infinity,
}));
expect(JSON.stringify(sdk.captureException.mock.calls)).not.toContain("private fixture payload");
expect(sdk.captureException.mock.calls[0]).toEqual([expect.any(Error), expect.objectContaining({ contexts: {
adapter_stop: { runId: null, adapterType: "unknown", runtimeMode: "unknown", abortRequested: null, timeoutMs: null },
adapter_stop: { runId: null, adapterType: "unknown", runtimeMode: "unknown", abortRequested: null, timeoutMs: null,
phase: "unknown", phaseElapsedMs: null },
} })]);
});
server/src/__tests__/task-title-routes.test.ts
+57 -1
View File
@@ -1,5 +1,15 @@
import { randomUUID } from "node:crypto";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import * as chat from "chat";
vi.mock("chat", async importOriginal => {
const original = await importOriginal<typeof import("chat")>();
return {
...original,
parseMarkdown: vi.fn(original.parseMarkdown),
markdownToPlainText: vi.fn(original.markdownToPlainText),
};
});
import { and, eq } from "drizzle-orm";
import { activityLog, externalObjectMentions, heartbeatRuns, issues } from "@paperclipai/db";
import { createChildIssueSchema, createIssueSchema, setIssueTitleSchema } from "@paperclipai/shared";
@@ -49,6 +59,52 @@ describe("task titles", () => {
expect(setIssueTitleSchema.safeParse({ title: "x".repeat(241) }).success).toBe(false);
});
it("removes leading Markdown before truncating a provisional title", async () => {
const f = await server.fixture();
const description = [
"![Screenshot](https://example.com/a-very-long-image-name.png)",
"# Fix the [sign-in redirect](https://example.com/issues/42)",
"Keep `returnTo` working with **saved sessions**.",
].join("\n\n");
const result = await issueService(server.db).create(f.companyId, { description });
expect(result).toMatchObject({
title: "Fix the sign-in redirect Keep returnTo working with saved sessions.",
description,
titleNeedsGeneration: true,
});
await expect(issueService(server.db).create(f.companyId, {
description: "![Error dialog](https://example.com/error.png)",
})).resolves.toMatchObject({ title: "Error dialog", titleNeedsGeneration: true });
await expect(issueService(server.db).create(f.companyId, {
description: "![](https://example.com/image.png)",
})).resolves.toMatchObject({ title: "Image", titleNeedsGeneration: true });
await expect(issueService(server.db).create(f.companyId, {
description: "![Map](https://example.com/Map_(1).png)",
})).resolves.toMatchObject({ title: "Map", titleNeedsGeneration: true });
await expect(issueService(server.db).create(f.companyId, {
description: "Fix `set_task_title`",
})).resolves.toMatchObject({ title: "Fix set_task_title", titleNeedsGeneration: true });
await expect(issueService(server.db).create(f.companyId, {
description: "![Screenshot][img]\n\n[img]: https://example.com/image.png\n\nFix login",
})).resolves.toMatchObject({ title: "Fix login", titleNeedsGeneration: true });
});
it.each(["parseMarkdown", "markdownToPlainText"] as const)("falls back to a simple title if %s throws", async parser => {
const f = await server.fixture();
const description = ` Fix the login flow after a failed parser\n${"long ".repeat(30)}`;
vi.mocked(chat[parser]).mockImplementationOnce(() => { throw new Error("parser failed"); });
try {
const result = await issueService(server.db).create(f.companyId, { description });
expect(result).toMatchObject({
title: description.trim().replace(/\s+/g, " ").slice(0, 120),
description,
titleNeedsGeneration: true,
});
} finally {
vi.mocked(chat[parser]).mockClear();
}
});
it("creates prompt-only children and still accepts explicit child titles", async () => {
const f = await server.fixture();
const description = "Investigate the child sign-in redirect";
server/src/__tests__/workspace-runtime.test.ts
+22 -15
View File
@@ -6748,37 +6748,44 @@ describeEmbeddedPostgres("workspace runtime service control persistence", () =>
executionWorkspaceId,
serviceName: "web",
status: "provisioning",
healthStatus: "unknown",
providerRef: null,
});
expect(existsSync(markerPath)).toBe(false);
await waitForMarker(markerPath);
const startingRow = await waitForPersistedStatus("starting");
expect(startingRow).toMatchObject({
// Readiness begins before the PID-bearing transaction commits. Keep the
// listener independent of DB reads and verify the completed records.
const services = await startPromise;
expect(services).toHaveLength(1);
expect(services[0]).toMatchObject({
id: provisioningRow.id,
companyId,
projectId,
projectWorkspaceId,
executionWorkspaceId,
issueId,
serviceName: "web",
status: "starting",
healthStatus: "unknown",
});
// Starting is persisted before the asynchronous spawn metadata write.
// The child marker alone does not prove that write has committed.
expect(startingRow.port).toEqual(expect.any(Number));
const services = await startPromise;
expect(services).toHaveLength(1);
expect(services[0]).toMatchObject({
id: startingRow.id,
status: "running",
healthStatus: "healthy",
});
expect(services[0]!.providerRef).toMatch(/^\d+$/);
expect(services[0]!.port).toEqual(expect.any(Number));
const runningRow = await waitForPersistedStatus("running");
expect(runningRow.id).toBe(startingRow.id);
expect(runningRow.providerRef).toMatch(/^\d+$/);
expect(runningRow).toMatchObject({
id: provisioningRow.id,
companyId,
projectId,
projectWorkspaceId,
executionWorkspaceId,
issueId,
serviceName: "web",
status: "running",
healthStatus: "healthy",
providerRef: services[0]!.providerRef,
port: services[0]!.port,
});
await expect(fetch(services[0]!.url!)).resolves.toMatchObject({ ok: true });
const runtimeProvisionOperations = await db
.select()
server/src/modules/wake-queue/adapters/postgres.test.ts
+40
View File
@@ -9,6 +9,8 @@ import {
agents,
companies,
createDb,
environmentLeases,
heartbeatRunEvents,
heartbeatRuns,
issueComments,
issueRecoveryActions,
@@ -59,6 +61,8 @@ describeEmbeddedPostgres("wake-queue postgres adapter", () => {
afterEach(async () => {
await db.delete(activityLog);
await db.delete(issueComments);
await db.delete(heartbeatRunEvents);
await db.delete(environmentLeases);
// `heartbeat_runs.wakeup_request_id` references `agent_wakeup_requests.id`,
// so the run row must go first.
await db.delete(heartbeatRuns);
@@ -416,6 +420,42 @@ describeEmbeddedPostgres("wake-queue postgres adapter", () => {
expect(wake.status).toBe("deferred_issue_execution");
});
it.each(["clean", "launch", "cleanup"])("promotes saved input only after a verified pre-dispatch review wait (%s)", async evidence => {
const companyId = await seedCompany(), agentId = await seedAgent({ companyId });
const issueId = await seedIssue({ companyId, assigneeAgentId: agentId });
const runId = await seedRun({ companyId, agentId, status: "cancelled", contextSnapshot: { issueId },
errorCode: "issue_continuation_waiting_on_review" });
await db.update(heartbeatRuns).set({ finishedAt: new Date(Date.now() - 1000), resultJson: {
stopReason: "issue_continuation_waiting_on_review", timeoutSource: "stale_queued_run_gate",
} }).where(eq(heartbeatRuns.id, runId));
if (evidence === "launch") await db.insert(heartbeatRunEvents).values({ companyId, runId, agentId, seq: 1,
eventType: "native.process_start_requested", stream: "system", level: "info" });
if (evidence === "cleanup") await db.insert(environmentLeases).values({ companyId, heartbeatRunId: runId,
provider: "local", status: "pending_cleanup", cleanupStatus: "failed" });
const [comment] = await db.insert(issueComments).values({ companyId, issueId, authorType: "user",
authorUserId: "responsible-user", body: "Continue with this new message." }).returning();
const wakeId = await seedDeferredWake({ companyId, agentId, issueId, requestedByActorId: "responsible-user",
payload: { commentId: comment.id, _paperclipWakeContext: { issueId, wakeReason: "issue_commented", wakeCommentIds: [comment.id] } } });
const release = createReleaseIssueExecution({ issueLock: createPostgresWakeQueueAdapter(db, stubDeps),
recovery: { escalateStrandedAssignedIssue: async () => {}, escalateStrandedRecoveryIssueInPlace: async () => {} } });
const result = await release({ companyId, runId, now: new Date() });
const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, companyId));
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, wakeId));
if (evidence !== "clean") {
expect(result.outcome.kind).toBe("released");
expect(runs).toHaveLength(1);
expect(wake).toMatchObject({ status: "deferred_issue_execution", runId: null });
return;
}
expect(result.outcome.kind).toBe("promoted");
expect(runs).toHaveLength(2);
const successor = runs.find(run => run.status === "queued")!;
expect(wake.runId).toBe(successor.id);
expect(successor.contextSnapshot).toMatchObject({ wakeCommentIds: [comment.id] });
await release({ companyId, runId, now: new Date() });
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, companyId))).toHaveLength(2);
});
it("leaves deferred work untouched until the effective execution hold clears", async () => {
const companyId = await seedCompany();
const agentId = await seedAgent({ companyId });
server/src/modules/wake-queue/adapters/postgres.ts
+2 -2
View File
@@ -19,7 +19,7 @@ import {
nativeRunFinalizations,
} from "@paperclipai/db";
import { hasConversationContinuationPolicy } from "../../../services/conversation-continuation.js";
import { legacyExecutionNeedsReconciliation } from "../../../services/legacy-execution-recovery.js";
import { legacyExecutionNeedsReconciliationWithEvidence } from "../../../services/legacy-execution-recovery.js";
import {
authorizeFailedChatRunRetryWake,
FailedChatRunRetryAuthorizationError,
@@ -1104,7 +1104,7 @@ export function createPostgresWakeQueueAdapter(db: Db, deps: WakeQueuePostgresAd
issueStatus: issueRow?.status ?? "",
hasAssigneeUser: Boolean(issueRow?.assigneeUserId),
assigneeAgentMatchesRunAgent: issueRow?.assigneeAgentId === run.agentId,
legacyExecutionNeedsReconciliation: legacyExecutionNeedsReconciliation(run),
legacyExecutionNeedsReconciliation: await legacyExecutionNeedsReconciliationWithEvidence(tx as unknown as Db, run),
// An operator stop never promotes old queued work by itself. The
// next explicit wake adopts those messages atomically when it
// queues a run.
server/src/modules/wake-queue/adapters/queued-comment-postgres.ts
+3 -1
View File
@@ -31,7 +31,9 @@ function toWakeRow(row: WakeRow): QueuedCommentWakeRow {
}
function toRunRow(row: RunRow): QueuedCommentRunRow {
return { id: row.id, status: row.status, runtimeMode: row.runtimeMode, contextSnapshot: parseObject(row.contextSnapshot) };
return { id: row.id, status: row.status, runtimeMode: row.runtimeMode,
runtimeModeResolvedAt: row.runtimeModeResolvedAt, runnerProfileJson: row.runnerProfileJson,
contextSnapshot: parseObject(row.contextSnapshot) };
}
export type QueuedCommentQueuePostgresAdapterDeps = {
server/src/modules/wake-queue/application/queued-comment-ports.ts
+2
View File
@@ -61,6 +61,8 @@ export type QueuedCommentRunRow = {
id: string;
status: string;
runtimeMode: string | null;
runtimeModeResolvedAt?: Date | null;
runnerProfileJson?: Record<string, unknown> | null;
contextSnapshot: Record<string, unknown>;
};
server/src/onboarding-assets/default/AGENTS.md
+1 -24
View File
@@ -1,24 +1 @@
You are an agent at Paperclip company.
## Execution Contract
- Start actionable work in the same heartbeat. Do not stop at a plan unless the issue explicitly asks for planning.
- Keep the work moving until it is done. If you need QA to review it, ask them. If you need your boss to review it, ask them.
- Leave durable progress in task comments, documents, or work products, then update the issue to a clear final disposition before you exit.
- When your work produces a user-inspectable deliverable file, follow the Paperclip skill's "Generated Artifacts and Work Products" workflow before final disposition. Use `skills/paperclip/scripts/paperclip-upload-artifact.sh` when working in this repo, create/update an artifact work product when the file is the deliverable, and link the uploaded attachment in the final comment. Do not rely on local filesystem paths as the only access path. If an important file intentionally remains workspace-only, create/update a work product with `metadata.resourceRef.kind: "workspace_file"` and a workspace-relative path, then name that work product and path in the final comment. Treat browse/search as a fallback for recovering workspace files, not the preferred deliverable path.
- When your work produces or updates an operator-facing engineering output, create/update the matching work product: `pull_request` for opened PRs, `preview_url` for published previews, `runtime_service` for managed preview/dev services, `commit` for notable pushed commits, and `branch` when the branch itself is the handoff. A comment is not a substitute for the work product access path.
- Comments, documents, screenshots, work products, and `Remaining` bullets are evidence, not valid liveness paths by themselves.
- Final disposition checklist: mark `done` when complete and verified; use `in_review` only with a real reviewer, approval, interaction, or monitor path; use `blocked` only with first-class blockers or a named unblock owner/action; create delegated follow-up issues with blockers when another agent owns the next step; keep `in_progress` only when a live continuation path exists.
- Use child issues for parallel or long delegated work instead of polling agents, sessions, or processes.
- Create child issues directly when you know what needs to be done. If the board/user needs to choose suggested tasks, answer structured questions, or confirm a proposal first, create an issue-thread interaction on the current issue with `POST /api/issues/{issueId}/interactions` using `kind: "suggest_tasks"`, `kind: "ask_user_questions"`, or `kind: "request_confirmation"`.
- Use `request_confirmation` instead of asking for yes/no decisions in markdown. Before presenting a plan for review, you MUST complete this publish contract:
1. `PUT /issues/{id}/documents/plan` with `{ format: 'markdown', body, changeSummary }`.
2. Re-`GET /documents/plan`, assert it returns `200`, and capture its `latestRevisionId`.
3. Only then create `request_confirmation` with `target={ type: 'issue_document', key: 'plan', revisionId: latestRevisionId }` and `idempotencyKey=confirmation:{issueId}:plan:{revisionId}`.
4. Wait for acceptance before creating implementation subtasks.
Never present a plan only in a thread comment or through `ask_user_questions`; comments are supporting context and questions are for gathering input, not plan review.
- `ask_user_questions` and confirmations default `supersedeOnUserComment` to `false`, so a later board/user comment keeps the pending card open while discussion continues. Set it to `true` when a new comment should replace the pending request. If you wake up from a superseding comment, revise the artifact, question set, or proposal and create a fresh interaction if input is still needed.
- For human input, save a pending question/confirmation interaction and set `in_review`; prose alone does not create a waiting path. Use `blockedByIssueIds` for issue dependencies. An agent may set an `unblockDescriptor` only for itself (`owner: { "agentId": "<your-agent-id>" }` plus `action`), not for the board/user or another agent.
- Respect budget, pause/cancel, approval gates, and company boundaries.
Do not let work sit here. You must always update your task with a comment.
You are an agent in a Paperclip company.
server/src/routes/agents.ts
+7 -7
View File
@@ -13,7 +13,7 @@ import { assertAiConnectionCreateAccess, canInstallSharedAiConnectionForNewAgent
import { isAiConnectionCompatible } from "@paperclipai/shared";
import { applyConnectorSkills, resolveConnectorAssignments, annotateConnectorSkills, isConnectorSkill } from "../services/connector-runtime.js";
import { getExecutionBlocker } from "../services/execution-blocker.js";
import { canContinueCancelledRun } from "../services/run-cancellation.js";
import { canRetryStoppedRun } from "../services/cancelled-native-startup.js";
import { paperclipRunnerTransitionConfig, normalizeLegacyRunnerProvider, isPaperclipRunnerProvider } from "@paperclipai/adapter-utils";
import { executionProjectionForRun, executionProjectionsForRuns } from "../services/execution-projection.js";
import { selectDashboardRunIds } from "../services/dashboard-run-selection.js";
@@ -2759,8 +2759,8 @@ export function agentRoutes(
// first-task/chief-of-staff/AGENTS.md, placeholders filled) over the agent's
// entry file instead of the generic default. Honored only for board-authored
// requests — the onboarding wizard runs as the board — so a client marker
// alone cannot swap another actor's instructions. The generic execution
// contract (default/AGENTS.md) is still appended on every run, unchanged.
// alone cannot swap another actor's instructions. Runtime coordination is
// supplied separately by the harness.
async function resolveOnboardingFirstAgentBundle(params: {
onboardingFirstAgent: unknown;
actorType: string;
@@ -5998,7 +5998,7 @@ export function agentRoutes(
"An exact failed-run retry cannot override its execution context.",
);
}
const failedRun = await heartbeat.getRun(req.body.failedRunId);
const failedRun = await heartbeat.getRun(req.body.failedRunId, { includeExecutionEvidence: true });
if (
!failedRun ||
failedRun.companyId !== agent.companyId ||
@@ -6006,14 +6006,14 @@ export function agentRoutes(
) {
throw notFound("Failed run not found");
}
if (!["failed", "timed_out"].includes(failedRun.status) && !canContinueCancelledRun(failedRun)) {
throw conflict("Only a failed run or verified unexpected cancellation can start a new attempt.");
}
if (failedRun.runtimeMode === "native" && failedRun.errorCode === "native_session_cleanup_quarantined") {
throw conflict("The stopped native session requires cleanup and reconciliation before a new attempt.", {
code: "native_session_cleanup_quarantined",
});
}
if (!(await canRetryStoppedRun(db, failedRun))) {
throw conflict("Only a failed run or a verified eligible cancellation can start a new attempt.");
}
const failedContext = asRecord(failedRun.contextSnapshot) ?? {};
const issueId =
typeof failedContext.issueId === "string"
server/src/routes/company-skills.ts
+55 -2
View File
@@ -7,7 +7,7 @@ import { once } from "node:events";
import type { SkillSourceDiscoveryEvent } from "@paperclipai/shared";
import { createHash } from "node:crypto";
import { and, eq, sql } from "drizzle-orm";
import { activityLog } from "@paperclipai/db";
import { activityLog, companySkills } from "@paperclipai/db";
import { persistActivity, publishActivity } from "../services/activity-log.js";
import { projectToolContext } from "../services/project-tool-context.js";
import { Router, type Request } from "express";
@@ -1219,13 +1219,66 @@ export function companySkillRoutes(db: Db) {
const companyId = req.params.companyId as string;
const skillId = req.params.skillId as string;
await assertCanMutateCompanySkills(req, companyId, "skills.edit", () => skillPolicyResource({ companyId, skillId }));
const { idempotencyKey, expectedVersionId, ...input } = req.body;
if (idempotencyKey) {
const actor = getActorInfo(req);
const runContext = req.actor.type === "agent" && req.actor.source === "agent_jwt" && req.actor.runId
? await projectToolContext(db, req.actor, true, "Skill") : null;
const receiptKey = `skill-file:${companyId}:${actor.actorId}:${runContext?.issue.id ?? "board"}:${skillId}:${idempotencyKey}`;
const fingerprint = createHash("sha256").update(JSON.stringify({ ...input, expectedVersionId })).digest("hex");
const rollbackState: { current: { restore: () => Promise<void>; versionId: string | null } | null } = { current: null };
const outcome = await db.transaction(async tx => {
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${receiptKey}, 0))`);
if (runContext) await projectToolContext(tx as unknown as Db, req.actor, true, "Skill");
const [prior] = await tx.select().from(activityLog).where(and(
eq(activityLog.companyId, companyId), eq(activityLog.action, "company.skill_file_updated"),
sql`${activityLog.details}->>'idempotencyKey' = ${receiptKey}`,
));
if (prior) {
if (prior.details?.fingerprint !== fingerprint) throw conflict("Skill file idempotency key was used with different inputs");
return { receipt: prior.details?.receipt, publication: null };
}
let publication: ActivityPublication | null = null;
let receipt: { skillId: string; path: string; versionId: string | null; studioPath: string } | null = null;
await companySkillService(tx as unknown as Db).updateFile(companyId, skillId, input.path, input.content, skillActor(req), {
encoding: input.encoding, executable: input.executable, expectedVersionId,
onRollback: (restore, versionId) => { rollbackState.current = { restore, versionId }; },
afterUpdate: async (versionId) => {
receipt = { skillId, path: input.path, versionId, studioPath: `/skills/studio/${encodeURIComponent(skillId)}` };
const activity = await persistActivity(tx as unknown as Db, {
companyId, actorType: actor.actorType, actorId: actor.actorId, agentId: actor.agentId,
runId: actor.runId, agentApiKeyId: actor.agentApiKeyId, issueId: runContext?.issue.id,
action: "company.skill_file_updated", entityType: "company_skill", entityId: skillId,
details: { path: input.path, markdown: input.path === "SKILL.md", versionId,
sourceIssueId: runContext?.issue.id ?? null, idempotencyKey: receiptKey, fingerprint, receipt },
});
publication = activity.publication;
},
});
return { receipt, publication };
}).catch(async (error) => {
if (rollbackState.current) {
const { restore, versionId } = rollbackState.current;
await db.transaction(async tx => {
await tx.execute(sql`select ${companySkills.id} from ${companySkills} where ${companySkills.id} = ${skillId} and ${companySkills.companyId} = ${companyId} for update`);
const [current] = await tx.select({ versionId: companySkills.currentVersionId }).from(companySkills)
.where(and(eq(companySkills.id, skillId), eq(companySkills.companyId, companyId)));
if (current && current.versionId === versionId) await restore();
});
}
throw error;
});
if (outcome.publication) publishActivity(outcome.publication);
res.json(outcome.receipt);
return;
}
const result = await svc.updateFile(
companyId,
skillId,
String(req.body.path ?? ""),
String(req.body.content ?? ""),
skillActor(req),
{ encoding: req.body.encoding, executable: req.body.executable },
{ encoding: req.body.encoding, executable: req.body.executable, expectedVersionId },
);
const actor = getActorInfo(req);
server/src/routes/issues.ts
+23 -8
View File
@@ -7064,6 +7064,8 @@ export function issueRoutes(
queueId: string;
targetRunId?: string;
allowStoppedTarget?: boolean;
/** Steering must allow PRP event ingestion to update the run before ACK. */
lockRun?: boolean;
}) {
const [currentIssue] = await input.tx
.select()
@@ -7145,8 +7147,8 @@ export function issueRoutes(
state === "deferred"
? (input.targetRunId ?? input.issue.executionRunId ?? null)
: null;
const activeRun = activeRunId
? await input.tx
const activeRunQuery = activeRunId
? input.tx
.select()
.from(heartbeatRuns)
.where(
@@ -7156,9 +7158,11 @@ export function issueRoutes(
input.allowStoppedTarget ? undefined : eq(heartbeatRuns.status, "running"),
),
)
.for("update")
.limit(1)
.then((rows) => rows[0] ?? null)
.$dynamic()
: null;
const activeRun = activeRunQuery
? await (input.lockRun === false ? activeRunQuery : activeRunQuery.for("update"))
.limit(1).then(rows => rows[0] ?? null)
: null;
if (input.targetRunId) {
const runContext = readObject(activeRun?.contextSnapshot);
@@ -15577,7 +15581,6 @@ export function issueRoutes(
eq(heartbeatRuns.agentId, retryWake.agentId),
),
)
.for("update")
.limit(1)
.then((rows) => rows[0] ?? null)
: null;
@@ -15615,6 +15618,7 @@ export function issueRoutes(
actor,
queueId: req.body.queueId,
targetRunId: req.body.targetRunId,
lockRun: false,
});
if (!locked.activeRun) {
throw conflict("The queued message targets a stale run", {
@@ -15680,6 +15684,17 @@ export function issueRoutes(
? () => reconcileSteeredIdentity(db, steeringIdentity)
: undefined,
}));
// PRP ingestion allocates event sequences on this row. Locking it
// while awaiting the provider prevents its durable ACK from arriving.
// Re-read under lock after ACK so concurrent run receipts are retained.
const [acknowledgedRun] = await tx.select().from(heartbeatRuns)
.where(eq(heartbeatRuns.id, locked.activeRun.id)).for("update");
if (!acknowledgedRun || acknowledgedRun.status !== "running") {
throw conflict("The queued message targets a stopped run. Your message is still queued.", {
code: "queued_comment_stale_target",
});
}
const acknowledgedResult = readObject(acknowledgedRun.resultJson);
if (steeringIdentity)
await acceptSteeredIdentity(tx, steeringIdentity);
acknowledgedTurnId = acknowledgement.turnId;
@@ -15711,9 +15726,9 @@ export function issueRoutes(
.update(heartbeatRuns)
.set({
resultJson: {
...runResult,
...acknowledgedResult,
queuedSteeringAcknowledgements: {
...acknowledgements,
...readObject(acknowledgedResult.queuedSteeringAcknowledgements),
[commentId]: {
status: "acknowledged",
queueId: req.body.queueId,
server/src/routes/openapi.ts
+2 -2
View File
@@ -7740,12 +7740,12 @@ registry.registerPath({
method: "patch",
path: "/api/companies/{companyId}/skills/{skillId}/files",
tags: ["skills"],
summary: "Update a skill file",
summary: "Update a skill file (optional expectedVersionId and idempotencyKey guard agent retries)",
request: {
params: z.object({ companyId: z.string(), skillId: z.string() }),
body: jsonBody(companySkillFileUpdateSchema),
},
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized },
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 409: r.conflict },
});
registry.registerPath({
server/src/services/adapter-execution-control.test.ts
+53
View File
@@ -10,6 +10,16 @@ import {
afterEach(() => vi.useRealTimers());
async function stopTimeout(pending: Promise<void>): Promise<AdapterStopTimeoutError> {
try {
await pending;
} catch (error) {
expect(error).toBeInstanceOf(AdapterStopTimeoutError);
return error as AdapterStopTimeoutError;
}
throw new Error("Stop unexpectedly settled");
}
it("holds readiness for every exact-run no-owner Stop and releases owners idempotently", async () => {
const runId = "registration-after-two-stops";
const first = captureAdapterStopOwnership(runId);
@@ -139,6 +149,7 @@ it("records the exact unconfirmed run without finishing or removing its control"
expect(error).toBeInstanceOf(AdapterStopTimeoutError);
expect((error as AdapterStopTimeoutError).diagnostics).toEqual({
runId, adapterType: "claude_local", runtimeMode: "legacy", abortRequested: true, timeoutMs: 1000,
phase: "unknown", phaseElapsedMs: null,
});
expect(adapterExecutionControls.get(runId)).toBe(control);
expect(finished).toBe(false);
@@ -151,3 +162,45 @@ it("records the exact unconfirmed run without finishing or removing its control"
adapterExecutionControls.delete(runId);
}
});
it("samples the pending phase at each Stop timeout without settling the executor", async () => {
vi.useFakeTimers();
const runId = "11111111-1111-4111-8111-111111111111";
const control = createAdapterExecutionControl();
await registerAdapterExecutionControl(runId, control);
const read = vi.spyOn(control.phases, "snapshot");
const pending = stopTimeout(waitForAdapterStop(control.settled, 1000, { runId }, control));
expect(read).not.toHaveBeenCalled();
control.phases.enter("instruction_collection");
await vi.advanceTimersByTimeAsync(1000);
const first = await pending;
expect(first.diagnostics).toMatchObject({ phase: "instruction_collection", phaseElapsedMs: expect.any(Number) });
const secondWait = stopTimeout(waitForAdapterStop(control.settled, 1000, { runId }, control));
control.phases.enter("lease_release");
await vi.advanceTimersByTimeAsync(1000);
expect((await secondWait).diagnostics.phase).toBe("lease_release");
expect(first.diagnostics.phase).toBe("instruction_collection");
expect(Object.isFrozen(first.diagnostics)).toBe(true);
expect(adapterExecutionControls.get(runId)).toBe(control);
control.finish();
adapterExecutionControls.delete(runId);
});
it.each(["replaced", "wrong_run", "wrong_promise", "throwing_reader"])("omits pending phase for a %s control", async (scenario) => {
vi.useFakeTimers();
const runId = "11111111-1111-4111-8111-111111111111";
const owner = createAdapterExecutionControl();
const other = createAdapterExecutionControl();
await registerAdapterExecutionControl(runId, owner);
owner.phases.enter("instruction_collection");
const pending = stopTimeout(waitForAdapterStop(scenario === "wrong_promise" ? other.settled : owner.settled, 1000,
{ runId: scenario === "wrong_run" ? "22222222-2222-4222-8222-222222222222" : runId }, owner));
if (scenario === "replaced") await registerAdapterExecutionControl(runId, other);
if (scenario === "throwing_reader") vi.spyOn(owner.phases, "snapshot").mockImplementation(() => { throw new Error("private fixture payload"); });
await vi.advanceTimersByTimeAsync(1000);
expect((await pending).diagnostics).toMatchObject({ phase: "unknown", phaseElapsedMs: null });
expect(JSON.stringify((await pending).diagnostics)).not.toContain("private fixture payload");
owner.finish();
other.finish();
adapterExecutionControls.delete(runId);
});
server/src/services/adapter-execution-control.ts
+19 -2
View File
@@ -1,13 +1,15 @@
import { AdapterStopTimeoutError, type AdapterStopContext } from "./adapter-stop-timeout.js";
import { createAdapterExecutionPhaseTracker } from "./adapter-execution-phase.js";
/** Live adapter ownership shared by routes and scheduler service instances. */
export function createAdapterExecutionControl() {
const controller = new AbortController();
const phases = createAdapterExecutionPhaseTracker();
let finish!: () => void;
const settled = new Promise<void>((resolve) => {
finish = resolve;
});
return { controller, settled, finish };
return { controller, settled, phases, finish: () => { phases.finish(); finish(); } };
}
export const adapterExecutionControls = new Map<
@@ -70,7 +72,17 @@ export async function waitForAdapterStop(
settled: Promise<void>,
timeoutMs = 60_000,
diagnostics?: AdapterStopContext,
owner?: ReturnType<typeof createAdapterExecutionControl>,
) {
const currentPhase = () => {
try {
return owner && owner.settled === settled && diagnostics?.runId
&& adapterExecutionControls.get(diagnostics.runId) === owner
? owner.phases.snapshot() : null;
} catch {
return null;
}
};
let timer: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
@@ -79,7 +91,12 @@ export async function waitForAdapterStop(
timer = setTimeout(
() =>
reject(
new AdapterStopTimeoutError(timeoutMs, diagnostics),
new AdapterStopTimeoutError(timeoutMs, {
...diagnostics,
phase: undefined,
phaseElapsedMs: undefined,
...currentPhase(),
}),
),
timeoutMs,
);
server/src/services/adapter-execution-phase.test.ts
+60
View File
@@ -0,0 +1,60 @@
import { expect, it } from "vitest";
import { createAdapterExecutionPhaseTracker, MAX_EXECUTION_PHASE_ELAPSED_MS } from "./adapter-execution-phase.js";
it("returns the latest pending scope and resumes the outer scope's original age", () => {
let now = 10;
const phases = createAdapterExecutionPhaseTracker(() => now);
const outer = phases.enter("sync_back")!;
now = 30;
const inner = phases.enter("instruction_collection")!;
now = 80;
expect(phases.snapshot()).toEqual({ phase: "instruction_collection", phaseElapsedMs: 50 });
inner();
expect(phases.snapshot()).toEqual({ phase: "sync_back", phaseElapsedMs: 70 });
outer();
expect(phases.snapshot()).toBeNull();
});
it("late and repeated releases never clear a newer scope or revive a completed scope", () => {
const phases = createAdapterExecutionPhaseTracker(() => 20);
const first = phases.enter("end_session")!;
const second = phases.enter("close_session")!;
first();
first();
expect(phases.snapshot()?.phase).toBe("close_session");
second();
expect(phases.snapshot()).toBeNull();
});
it("isolates trackers and refuses late updates after the executor finishes", () => {
const first = createAdapterExecutionPhaseTracker();
const second = createAdapterExecutionPhaseTracker();
first.enter("workspace_restore");
second.enter("instruction_cleanup");
first.finish();
first.enter("lease_release");
expect(first.snapshot()).toBeNull();
expect(second.snapshot()?.phase).toBe("instruction_cleanup");
});
it("bounds retained scopes and fails closed on overflow or arbitrary labels", () => {
const phases = createAdapterExecutionPhaseTracker();
phases.enter("private fixture payload" as "sync_back");
expect(phases.snapshot()).toBeNull();
for (let i = 0; i < 100; i++) phases.enter("sync_back");
expect(phases.snapshot()).toBeNull();
phases.finish();
expect(phases.snapshot()).toBeNull();
});
it("uses a finite nonnegative capped elapsed time", () => {
let now = 10;
const phases = createAdapterExecutionPhaseTracker(() => now);
phases.enter("close_session");
now = Infinity;
expect(phases.snapshot()).toBeNull();
now = 9;
expect(phases.snapshot()).toBeNull();
now = MAX_EXECUTION_PHASE_ELAPSED_MS * 2;
expect(phases.snapshot()).toEqual({ phase: "close_session", phaseElapsedMs: MAX_EXECUTION_PHASE_ELAPSED_MS });
});
server/src/services/adapter-execution-phase.ts
+44
View File
@@ -0,0 +1,44 @@
import { performance } from "node:perf_hooks";
import {
isAdapterExecutionPhase,
type AdapterExecutionPhase,
} from "@paperclipai/adapter-utils/execution-phase";
export const MAX_EXECUTION_PHASE_ELAPSED_MS = 86_400_000;
const MAX_ACTIVE_PHASES = 16;
/** Owned by one executor, never looked up by agent, task, or provider identity. */
export function createAdapterExecutionPhaseTracker(now = () => performance.now()) {
const scopes = new Map<symbol, { phase: AdapterExecutionPhase; startedAt: number }>();
let finished = false;
let overflow = false;
return {
enter(phase: AdapterExecutionPhase) {
if (finished || !isAdapterExecutionPhase(phase)) return;
if (scopes.size >= MAX_ACTIVE_PHASES) {
// Refuse further attribution rather than retain an unbounded history or
// describe an older scope as the current one. finish clears the tracker.
overflow = true;
return;
}
const token = Symbol();
let startedAt: number;
try { startedAt = now(); } catch { return; }
scopes.set(token, { phase, startedAt });
return () => { scopes.delete(token); };
},
snapshot() {
if (finished || overflow) return null;
const current = [...scopes.values()].at(-1);
if (!current) return null;
let elapsed: number;
try { elapsed = now() - current.startedAt; } catch { return null; }
if (!Number.isFinite(elapsed) || elapsed < 0) return null;
return { phase: current.phase, phaseElapsedMs: Math.min(Math.floor(elapsed), MAX_EXECUTION_PHASE_ELAPSED_MS) };
},
finish() {
finished = true;
scopes.clear();
},
};
}
server/src/services/adapter-stop-timeout.ts
+8
View File
@@ -1,10 +1,14 @@
import { AGENT_ADAPTER_TYPES } from "@paperclipai/shared";
import { isAdapterExecutionPhase } from "@paperclipai/adapter-utils/execution-phase";
import { MAX_EXECUTION_PHASE_ELAPSED_MS } from "./adapter-execution-phase.js";
export interface AdapterStopContext {
runId?: string;
adapterType?: string;
runtimeMode?: string;
abortRequested?: boolean;
phase?: string;
phaseElapsedMs?: number;
}
/** Diagnostic identity only. This error never acknowledges termination. */
@@ -22,6 +26,10 @@ export class AdapterStopTimeoutError extends Error {
adapterType: AGENT_ADAPTER_TYPES.some((type) => type === context?.adapterType) ? context!.adapterType! : "unknown",
runtimeMode: context?.runtimeMode === "native" || context?.runtimeMode === "legacy" ? context.runtimeMode : "unknown",
abortRequested: typeof context?.abortRequested === "boolean" ? context.abortRequested : null,
phase: isAdapterExecutionPhase(context?.phase) ? context.phase : "unknown",
phaseElapsedMs: isAdapterExecutionPhase(context?.phase) && Number.isSafeInteger(context?.phaseElapsedMs)
&& context!.phaseElapsedMs! >= 0 && context!.phaseElapsedMs! <= MAX_EXECUTION_PHASE_ELAPSED_MS
? context!.phaseElapsedMs! : null,
});
}
}
server/src/services/cancelled-native-startup.ts
+30 -4
View File
@@ -1,12 +1,27 @@
import { isPreDispatchReviewWait } from "./pre-dispatch-review-wait.js";
import { and, eq, inArray, isNotNull, or } from "drizzle-orm";
import { environmentLeases, heartbeatRunEvents, heartbeatRuns, nativeRunFinalizations, type Db } from "@paperclipai/db";
import { claimedAdapterType } from "./conversation-continuation.js";
import { PROCESS_IDENTITY_RECORDED, PROCESS_START_REQUESTED } from "./native-local-process-stop.js";
import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";
import { canContinueCancelledRun } from "./run-cancellation.js";
import { legacyControllerBootId } from "./legacy-controller-lease.js";
type Run = typeof heartbeatRuns.$inferSelect;
type Coordinator = typeof nativeRunFinalizations.$inferSelect;
/** Candidate for explicit user Retry only, never automatic replay after Stop. */
export async function canRetryStoppedRun(db: Db, run: Run): Promise<boolean> {
if (run.errorCode === "native_session_cleanup_quarantined") return false;
if (["failed", "timed_out"].includes(run.status) || canContinueCancelledRun(run)) return true;
if (run.status !== "cancelled" || !run.finishedAt || run.processPid || run.processGroupId ||
run.processStartedAt || run.sessionIdAfter) return false;
const [coordinator] = await db.select().from(nativeRunFinalizations).where(and(
eq(nativeRunFinalizations.companyId, run.companyId), eq(nativeRunFinalizations.runId, run.id),
));
return isCancelledNativeStartup(db, run, coordinator);
}
/** Caller holds the coordinator and run locks when using this proof to admit
* work. Attempt zero is a durable never-claimed receipt: every native executor
* commits its first claim before it can start or attach a provider. */
@@ -14,7 +29,18 @@ export async function isCancelledNativeStartup(db: Db, run: Run, coordinator: Co
if (run.status !== "cancelled" || !run.finishedAt || run.processPid || run.processGroupId ||
run.processStartedAt || run.sessionIdAfter) return false;
const cancellation = run.resultJson?.startupCancellation as Record<string, unknown> | undefined;
const beforeSelection = run.runtimeMode === "legacy" && !run.runtimeModeResolvedAt &&
// Older builds could omit the cancellation fence or its unwind marker. Their immutable
// native-adapter claim and unresolved preparation stage still prove that
// provider dispatch did not begin. Require an expired owner from another
// server boot; neither a missing PID nor mutable agent settings is proof.
const historicalBeforeSelection = run.runtimeMode === "legacy" && !run.runtimeModeResolvedAt &&
run.executionStage === "preparing" && !run.nativeIssueId && !run.nativeSessionId && !coordinator &&
claimedAdapterType(run) === "paperclip_runner" && run.errorCode === "operator_interrupted" &&
(run.resultJson === null || cancellation?.beforeNativeSelection === true) &&
Boolean(run.controllerBootId && run.controllerBootId !== legacyControllerBootId &&
run.controllerLeaseExpiresAt && run.controllerLeaseExpiresAt <= new Date());
const beforeReviewDispatch = isPreDispatchReviewWait(run) && !coordinator;
const beforeSelection = beforeReviewDispatch || historicalBeforeSelection || run.runtimeMode === "legacy" && !run.runtimeModeResolvedAt &&
!run.nativeSessionId && !coordinator && claimedAdapterType(run) === "paperclip_runner" &&
cancellation?.beforeNativeSelection === true;
const neverClaimed = run.runtimeMode === "native" && coordinator &&
@@ -23,13 +49,13 @@ export async function isCancelledNativeStartup(db: Db, run: Run, coordinator: Co
!coordinator.controllerPid && !coordinator.leaseOwner && !coordinator.leaseExpiresAt &&
!coordinator.resultId && !coordinator.failureDetail?.successorRunId;
if (!beforeSelection && !neverClaimed) return false;
const settled = typeof run.resultJson?.startupPreparationSettledAt === "string";
const settled = beforeReviewDispatch || typeof run.resultJson?.startupPreparationSettledAt === "string";
// The old preparer can still be unwinding even though the run is terminal.
if (!settled && run.controllerLeaseExpiresAt && run.controllerLeaseExpiresAt > new Date()) return false;
const leases = await db.select().from(environmentLeases).where(and(
eq(environmentLeases.companyId, run.companyId), eq(environmentLeases.heartbeatRunId, run.id),
));
if ((!settled && leases.length === 0) || leases.some(lease =>
if ((!settled && leases.length === 0 && !historicalBeforeSelection) || leases.some(lease =>
lease.provider === "local"
? !lease.releasedAt || lease.status === "pending_cleanup" || lease.cleanupStatus === "failed"
: !hasRemoteTerminationReceipt(lease))) return false;
@@ -38,7 +64,7 @@ export async function isCancelledNativeStartup(db: Db, run: Run, coordinator: Co
const [execution] = await db.select({ id: heartbeatRunEvents.id }).from(heartbeatRunEvents).where(and(
eq(heartbeatRunEvents.companyId, run.companyId), eq(heartbeatRunEvents.runId, run.id),
or(isNotNull(heartbeatRunEvents.sourceEventId),
inArray(heartbeatRunEvents.eventType, [PROCESS_START_REQUESTED, PROCESS_IDENTITY_RECORDED,
inArray(heartbeatRunEvents.eventType, ["adapter.invoke", PROCESS_START_REQUESTED, PROCESS_IDENTITY_RECORDED,
"harness.ready", "session.started", "session.resumed", "session.updated", "turn.started",
"provider.event", "provider.rpc_result", "tool.execution.started"])),
)).limit(1);
server/src/services/company-skills.ts
+58 -33
View File
@@ -4643,6 +4643,11 @@ export function companySkillService(db: Db) {
for (const slug of [...new Set([initial.slug, ...additionalSlugs])].filter(Boolean).sort()) {
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`${companyId}:${slug}`}, 0))`);
}
await tx.execute(sql`
select ${companySkills.id} from ${companySkills}
where ${companySkills.id} = ${skillId} and ${companySkills.companyId} = ${companyId}
for update
`);
const skill = await getById(companyId, skillId, tx);
if (!skill) throw notFound("Skill not found");
if (skill.slug !== initial.slug) throw conflict("Skill was renamed. Retry the operation.");
@@ -4669,10 +4674,16 @@ export function companySkillService(db: Db) {
relativePath: string,
content: string,
actor: SkillActor | null = null,
format: { encoding?: "utf8" | "base64"; executable?: boolean } = {},
format: { encoding?: "utf8" | "base64"; executable?: boolean; expectedVersionId?: string | null;
afterUpdate?: (versionId: string | null) => Promise<void>;
onRollback?: (restore: () => Promise<void>, versionId: string | null) => void } = {},
): Promise<CompanySkillFileDetail> {
return withSkillFileMutation(companyId, skillId, async (skill, tx) => {
if (format.expectedVersionId !== undefined && skill.currentVersionId !== format.expectedVersionId) {
throw conflict("Skill version changed. Read the current version before retrying.");
}
const source = deriveSkillSourceInfo(skill);
if (!source.editable || skill.sourceType !== "local_path") {
throw unprocessable(source.editableReason ?? "This skill cannot be edited.");
@@ -4689,41 +4700,55 @@ export function companySkillService(db: Db) {
}
const previousContent = await fs.readFile(absolutePath).catch(() => null);
const previousMode = (await fs.stat(absolutePath).catch(() => null))?.mode ?? 0o644;
const restore = async () => {
if (previousContent === null) await fs.rm(absolutePath, { force: true });
else {
await fs.writeFile(absolutePath, previousContent);
await fs.chmod(absolutePath, previousMode);
}
};
format.onRollback?.(restore, skill.currentVersionId);
const mode = (format.executable ?? Boolean(previousMode & 0o111)) ? 0o755 : 0o644;
await fs.mkdir(path.dirname(absolutePath), { recursive: true });
await fs.writeFile(absolutePath, bytes);
await fs.chmod(absolutePath, mode);
try {
await fs.mkdir(path.dirname(absolutePath), { recursive: true });
await fs.writeFile(absolutePath, bytes);
await fs.chmod(absolutePath, mode);
if (normalizedPath === "SKILL.md") {
const parsed = parseFrontmatterMarkdown(content);
await tx
.update(companySkills)
.set({
name: asString(parsed.frontmatter.name) ?? skill.name,
description: asString(parsed.frontmatter.description) ?? skill.description,
markdown: content,
...readSkillStoreMetadata(parsed.frontmatter, skill.metadata),
updatedAt: new Date(),
})
.where(eq(companySkills.id, skill.id));
} else {
await tx
.update(companySkills)
.set({ updatedAt: new Date() })
.where(eq(companySkills.id, skill.id));
if (normalizedPath === "SKILL.md") {
const parsed = parseFrontmatterMarkdown(content);
await tx
.update(companySkills)
.set({
name: asString(parsed.frontmatter.name) ?? skill.name,
description: asString(parsed.frontmatter.description) ?? skill.description,
markdown: content,
...readSkillStoreMetadata(parsed.frontmatter, skill.metadata),
updatedAt: new Date(),
})
.where(eq(companySkills.id, skill.id));
} else {
await tx
.update(companySkills)
.set({ updatedAt: new Date() })
.where(eq(companySkills.id, skill.id));
}
const inventory = await refreshEditedSkillInventory(skill, tx);
if (!previousContent?.equals(bytes) || Boolean(previousMode & 0o111) !== Boolean(mode & 0o111)) {
await createVersion(companyId, skillId, {}, actor, {
database: tx, skipInventoryRefresh: true, skill, fileInventory: inventory,
});
}
const updated = await getById(companyId, skillId, tx);
const detail = updated ? await readLoadedSkillFile(updated, normalizedPath) : null;
if (!detail) throw notFound("Skill file not found");
await format.afterUpdate?.(updated?.currentVersionId ?? null);
return detail;
} catch (error) {
await restore();
throw error;
}
const inventory = await refreshEditedSkillInventory(skill, tx);
if (!previousContent?.equals(bytes) || Boolean(previousMode & 0o111) !== Boolean(mode & 0o111)) {
await createVersion(companyId, skillId, {}, actor, {
database: tx, skipInventoryRefresh: true, skill, fileInventory: inventory,
});
}
const updated = await getById(companyId, skillId, tx);
const detail = updated ? await readLoadedSkillFile(updated, normalizedPath) : null;
if (!detail) throw notFound("Skill file not found");
return detail;
});
}
Loaded 100 of 203 files, more files were not shown because too many files have changed in this diff. Show more