mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:58:52 +01:00
DottaandPaperclip
72ff3a9f27
## Thinking Path > - Paperclip manages persistent agents and their assigned work. > - Agents receive both fixed instructions and tool definitions. > - Moving a procedure into a tool description still adds model context. > - We need to measure the complete delivered catalog and test real outcomes. > - The tested reductions saved little space and introduced failing outcomes. > - This PR keeps measurement, bounded eval coverage and the original evidence. > - Production prompts, tools and runtime behavior stay unchanged. ## Linked Issues or Issue Description Refs: #15151, #14961, #14948, #14985. This adds the measurement and eval coverage needed to assess further native instruction changes. The attempted hiring and dependency reduction failed qualification and is excluded from the final diff. ## What Changed - Measure the actual standard-mode tool authority, including all 39 tools and their input schemas. Capture scripted native start, resume and continuation payloads and the OpenCode MCP declaration list. - Add OpenCode to the two explicit-only local hiring/reuse and delegation/feedback stories. Preserve the original task requests and independent oracles. - Apply one attempt per selected story and explicit company and lead-agent budget stops. - Select managed hiring credentials from the requested profile, including OpenRouter. - Run the existing Node test files under Node instead of collecting them as Vitest suites. - Retain sanitized comparison reports, original failed grades, source hashes and evidence gaps. ## Verification Final source: `2e7cef78eef7cdfe02265e0dcb03e855b8e50bd8`. Local verification passes: - Full repository `pnpm -r typecheck` and `pnpm build`. - Eval-support unit tests: 1,252 Vitest checks and 128 Node checks. - Eval TypeScript check and six final-source measurement tests. All 36 normalized components across nine scripted deliveries and the OpenCode MCP catalog match the baseline exactly; the complete standing projection is 49,200 bytes. Fresh review of this exact head is [5/5 with no remaining findings](https://github.com/paperclipai/paperclip/pull/15218#issuecomment-5996723170); all three review threads are resolved. [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/37354539126) passes all 47 jobs, including repository typecheck, build, tests, runner checks, browser shards and the canary check. One initial annotation-test timeout is retained in attempt 1; its seven-test suite passed locally, and the affected CI lane passed on one targeted retry. No source or paid eval rerun was needed. Both ready-transition security scans passed with zero annotations. The PR is out of draft and conflict-free. GitHub still requires code-owner approval for the `package.json` test-script change; its requested reviewers are already set. A byte-for-byte comparison against master context `a65ca0950834a85bb93bcc4b4042ecacdebfef53` confirms no production changes under `packages/` or production `server/` paths. The only server addition is a measurement test. No new paid rerun is needed to compare unchanged production bytes. This does not claim that existing product defects have been fixed. The rejected corrected experiment had baseline **5 PASS / 1 FAIL** and candidate **3 PASS / 3 FAIL**, including **two newly failing pairs**. The later readiness experiment had candidate **3 PASS / 3 FAIL** and baseline **3 PASS / 2 FAIL / one setup cell without a behavioral grade**. Its five comparable pairs had two new failures, two new passes and one unchanged pass. The missing baseline Codex cell never reached its provider step because Docker setup timed out. The observed missing behaviors include parent continuation, waiting for the latest child revision, revised ZIP delivery and OpenCode credential persistence. Those failures remain failures. Source review and passing CI do not regrade them. The original reduction saved 460 bytes; its first repair saved only 125 bytes, and the larger unqualified runtime repair increased the full projection. None of those production changes is shipped here. Read the [report](https://github.com/paperclipai/paperclip/blob/2e7cef78eef7cdfe02265e0dcb03e855b8e50bd8/doc/plans/2026-10-05-native-procedure-guidance.md) and its linked sanitized receipts for exact sources, original campaign links, pair-level results and evidence limitations. ## Risks - Full-catalog bytes are not model tokens, invoices, private vendor prompts, lazy-loading behavior or truncation proof. - The two stories are explicit-only and do not prove general coding quality or arbitrary resume behavior. Single trials do not establish causation or performance trends. - The retained OpenCode candidate credential guard failed. Cleanup removed the original provider database, so the precise persistence mechanism remains unknown. The guard is unchanged. - ACPX provider-execution IDs lack proven host-call mapping. No extra-work or feedback-consumption claim is inferred by matching names, order or counts. - PostgreSQL cannot start locally while the host's shared-memory slots are exhausted. Hosted CI must supply the full database checks; the full local database suite is not claimed green. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and code execution. The exact deployment model ID and context-window limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
5434 lines
194 KiB
JSON
5434 lines
194 KiB
JSON
{
|
|
"schema": "paperclip.native-procedure-public-comparison.v1",
|
|
"disposition": "hold_draft_no_regression_gate_failed",
|
|
"originalGradesPreserved": true,
|
|
"sources": {
|
|
"originalBaseline": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"originalCandidate": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"correctedBaseline": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"correctedCandidate": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"integratedCandidate": "ee096aa23fc88a001e7666215bcd338b5863f377",
|
|
"integratedProviderFreeBaseline": "ce31f5efd0d416205c3994c55dd02b645f8b8423"
|
|
},
|
|
"controls": {
|
|
"trustedWorkflowSha": "a386a599983519eb1d399f8b770bfccdb2a74762",
|
|
"commonFixtureFilesChecked": 256,
|
|
"sourceDelta": [
|
|
"packages/paperclip-runner/generated/capability/semantic-tool-contracts.json",
|
|
"packages/paperclip-runner/generated/semantic-action-catalog.json",
|
|
"packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json",
|
|
"packages/paperclip-runner/protocol/manifest.json",
|
|
"packages/paperclip-runner/src/catalog/semantic-action-catalog.ts",
|
|
"packages/paperclip-runner/src/contracts/runtime-context.ts",
|
|
"packages/paperclip-runner/src/protocol-actions/create-task.ts",
|
|
"packages/paperclip-runner/src/protocol-actions/hire-agent.ts",
|
|
"packages/paperclip-runner/src/protocol-actions/set-dependencies.ts",
|
|
"server/src/services/native-runtime/native-session-resume.ts",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.ts"
|
|
],
|
|
"cellsPerVariant": 6,
|
|
"maximumAttemptsPerCell": 1,
|
|
"companyHardStopCents": 1000,
|
|
"leadHardStopCents": 1000,
|
|
"workerIndividualHardStop": "not asserted; company hard stop applies",
|
|
"cellDeadlineMs": 720000,
|
|
"maximumStoryRunRecordsPerCell": 12
|
|
},
|
|
"commonFixtureHashesCanonicalJsonSha256": "2c1113b8995e5d004960905668d935ab77d463fa27ab979c824e3402210d1f27",
|
|
"original": {
|
|
"totals": {
|
|
"baseline": {
|
|
"pass": 4,
|
|
"fail": 2,
|
|
"retainedStartedRuns": 25
|
|
},
|
|
"candidate": {
|
|
"pass": 2,
|
|
"fail": 4,
|
|
"retainedStartedRuns": 15
|
|
}
|
|
},
|
|
"pairs": [
|
|
{
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.delegate-feedback",
|
|
"baseline": "passed",
|
|
"candidate": "failed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.hire-reuse",
|
|
"baseline": "passed",
|
|
"candidate": "failed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-codex.local.delegate-feedback",
|
|
"baseline": "passed",
|
|
"candidate": "failed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-codex.local.hire-reuse",
|
|
"baseline": "passed",
|
|
"candidate": "passed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-opencode.local.delegate-feedback",
|
|
"baseline": "failed",
|
|
"candidate": "passed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-opencode.local.hire-reuse",
|
|
"baseline": "failed",
|
|
"candidate": "failed"
|
|
}
|
|
],
|
|
"cells": [
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"ref": "refs/heads/codex/native-procedure-baseline",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325870695"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "b08ea732990fd75bea0f17df7ca06a87fc5ebe4a59c3113e810b777b7087d6a9",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "e37c96ffe166426f4c99e398142b1072d53421992cde8c7aada6303145fcdff3",
|
|
"html-report/data/5d2572f36528e8fddbc2153c632730c4f013b79a.json": "b08ea732990fd75bea0f17df7ca06a87fc5ebe4a59c3113e810b777b7087d6a9",
|
|
"playwright-output/runner-everyday-workflows--fdae5-the-browser-and-public-APIs/attachments/runner-e2e-result-eb06dc1fa3ae54172129187047897c5f1b2adb19.json": "b08ea732990fd75bea0f17df7ca06a87fc5ebe4a59c3113e810b777b7087d6a9",
|
|
"result.json": "b08ea732990fd75bea0f17df7ca06a87fc5ebe4a59c3113e810b777b7087d6a9",
|
|
"snapshots/api-state.json": "fa464c86756c6d01517cf486965a0266e9ec874fae8248ae547c58d39c6c7cea",
|
|
"snapshots/browser-diagnostics.json": "0bdbe36772ce52f8a4140cdf061596614d347229901be18b5cc17325cd0ea781",
|
|
"snapshots/chat-final-run-ledger.json": "1b4925e0c32d5369ddd23317f0a72d4060c99fc5f411d129c3db3b32a6a829dd",
|
|
"snapshots/everyday-workflow.json": "85a6509a51869f944d52b34d26a42ac48e83c455770140a55f0db58ba6aa1f0b",
|
|
"snapshots/fixtures.json": "140546a07e4fa11c45fea99741682eb5dc0d7905a2d20df22a45c4449aa07684"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.hire-reuse",
|
|
"source": {
|
|
"sha": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"ref": "refs/heads/codex/native-procedure-baseline",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325870695"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "85935339f572d46cf3dfa351bad4376bb537bafe1ada9a7d798b75dbb099941a",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "5c935a3744afc62c64fd29a664c33026aef235a92ec0500c9912d3507052e697",
|
|
"html-report/data/8bbe2058a3abe580b4ccb871d532ba41350da7aa.json": "85935339f572d46cf3dfa351bad4376bb537bafe1ada9a7d798b75dbb099941a",
|
|
"playwright-output/runner-everyday-workflows--9a7d2-the-browser-and-public-APIs/attachments/runner-e2e-result-89ae8a684c0bb5d6bb61e001e54f638ef8240960.json": "85935339f572d46cf3dfa351bad4376bb537bafe1ada9a7d798b75dbb099941a",
|
|
"result.json": "85935339f572d46cf3dfa351bad4376bb537bafe1ada9a7d798b75dbb099941a",
|
|
"snapshots/api-state.json": "7073c56ae937235b6d25f830d8da2dc1c1740f9295b737d361c31bc5c21b2302",
|
|
"snapshots/browser-diagnostics.json": "695c94cb21dc497359abc12568fe2270509df2327368590067b4eeeceb8f6784",
|
|
"snapshots/chat-final-run-ledger.json": "1437a3268c85532e7c64dec8c8b09ebb8c9b94a8213ea13e1795214df04888c7",
|
|
"snapshots/everyday-workflow.json": "acedad4621258df94eeb405fb43a180d17a449b4edc3ba4a1ace5b97ded79b63",
|
|
"snapshots/fixtures.json": "32736c391b9c292c81d7d7aeb8b3ab678784d68ed169727430ed9a429f97d5a2"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-codex.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"ref": "refs/heads/codex/native-procedure-baseline",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325870695"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 5,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "2b601d5736f3756ffb890a50821a655a49490a113922e15c199fef1fe0954e39",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "eb30cdd784087c1a34c4ecbaa7a43f051eddd95d3805fc89ab1054af70a044c2",
|
|
"html-report/data/cdbcafe148f5698ecd0df0cdf30809758fd418e8.json": "2b601d5736f3756ffb890a50821a655a49490a113922e15c199fef1fe0954e39",
|
|
"playwright-output/runner-everyday-workflows--a6fcb-the-browser-and-public-APIs/attachments/runner-e2e-result-b59bcc2342e97bbc85432cee261db26c76f3c80a.json": "2b601d5736f3756ffb890a50821a655a49490a113922e15c199fef1fe0954e39",
|
|
"result.json": "2b601d5736f3756ffb890a50821a655a49490a113922e15c199fef1fe0954e39",
|
|
"snapshots/api-state.json": "0b6a89bc7cdff9fa19c241eff210031468d7d500d9dd427526a888d740e8454a",
|
|
"snapshots/browser-diagnostics.json": "c520523de99f806d8cbe9fad497639ef813839b7dce1bbfaddaa7a1b77f66995",
|
|
"snapshots/chat-final-run-ledger.json": "ebae3fb5b1c5a87451af70817f9a6a35087b7606aed4c15437d453500675a3c6",
|
|
"snapshots/everyday-workflow.json": "691a281397c434341784cbbeabdf53d7ffbbe34cf6fccf05e05b365fffc2d2db",
|
|
"snapshots/fixtures.json": "93b762a930101110ad1953551cb5ba0094ccb609626f9ae2767d5d8c8d610424"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-codex.local.hire-reuse",
|
|
"source": {
|
|
"sha": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"ref": "refs/heads/codex/native-procedure-baseline",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325870695"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "200c06ab5dadf615bac0ec66b9a9ce851c75832ddaad5c854ed7fbbac16f90a7",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "5ff552d77ebbe35f2a123fcfe60f55008fd221b9c890ef92733b427e52634e28",
|
|
"html-report/data/cba00c1f6256ff612b9b89e78005f1a7460ae138.json": "200c06ab5dadf615bac0ec66b9a9ce851c75832ddaad5c854ed7fbbac16f90a7",
|
|
"playwright-output/runner-everyday-workflows--d934c-the-browser-and-public-APIs/attachments/runner-e2e-result-04508984e2fc501021db8f215efa4abf738af42d.json": "200c06ab5dadf615bac0ec66b9a9ce851c75832ddaad5c854ed7fbbac16f90a7",
|
|
"result.json": "200c06ab5dadf615bac0ec66b9a9ce851c75832ddaad5c854ed7fbbac16f90a7",
|
|
"snapshots/api-state.json": "e802749b93f083dbd37eae5a6cb4d63726bfa880fdad7a0cb9033a9e5132e5d1",
|
|
"snapshots/browser-diagnostics.json": "f9fd5484cc80f9fce274ba98be946d6f2826c20cf454ac5f0f09d5f29f4d4c9d",
|
|
"snapshots/chat-final-run-ledger.json": "44af7eed470338dcc0ad99eda29b65f4493e5ab48456e3196be70671550edf3e",
|
|
"snapshots/everyday-workflow.json": "36a50e25841b6a3827d7dc0c227fee06a07a5dfdfc6cdcef45e87006e2c49aec",
|
|
"snapshots/fixtures.json": "8e01cdc83c06cd80f9a7aa45844bd165c8152b40c551578f702dcd1facfe49fd"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-opencode.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"ref": "refs/heads/codex/native-procedure-baseline",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325870695"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": false
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "d9e4207efd71b1fde7b626100905f486ea758ccf149b1f8708d6f47e1b314f96",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "f12b13167c11896a77e2313295f8a4740a82d38e11179a92fdf9694af41a0014",
|
|
"html-report/data/1de72a5e503228c98ef17641b6a7e8e5520dd6d9.json": "d9e4207efd71b1fde7b626100905f486ea758ccf149b1f8708d6f47e1b314f96",
|
|
"playwright-output/runner-everyday-workflows--616ad-the-browser-and-public-APIs/attachments/runner-e2e-result-cbaa721581d55eb04872c27fef1564ad1093b5db.json": "d9e4207efd71b1fde7b626100905f486ea758ccf149b1f8708d6f47e1b314f96",
|
|
"result.json": "d9e4207efd71b1fde7b626100905f486ea758ccf149b1f8708d6f47e1b314f96",
|
|
"snapshots/api-state.json": "a88d15d5d1ea5cb4864f0ca86a9104207f90cb57424a0642c98bc63cd90ac5d5",
|
|
"snapshots/browser-diagnostics.json": "2ae30c24e8081099a72f8cb3fa5e86a633e525956cee65c09f36e6353c42efc1",
|
|
"snapshots/chat-final-run-ledger.json": "3674c59de912197327ccb8ba1d659f120b1cdc51b1f88b1bc3859485c31f1422",
|
|
"snapshots/everyday-workflow.json": "43730e83f96c51dda1cb39a7ab0509955af7bfcc18dd939a84cc1cef3c739d87",
|
|
"snapshots/fixtures.json": "9b2ef4cebf077c3ed74e55dc4faf5e5b547ba7be990214757cd479368c71717e"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-opencode.local.hire-reuse",
|
|
"source": {
|
|
"sha": "64cd2194a1d058356679b653c181f16404aefc80",
|
|
"ref": "refs/heads/codex/native-procedure-baseline",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325870695"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "permanent_infrastructure",
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 0,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [],
|
|
"resultSha256": "f8ef0e87c24ecacb8dd18ebc158771a74b5cbb8c0b673edd00c80e60a1932832",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "33b1383486d18fcb0a01b7e3488704fcfa760114958554469016725f79212508",
|
|
"html-report/data/26ec75604543a922ef760e2cfb41ed3f340a019b.json": "f8ef0e87c24ecacb8dd18ebc158771a74b5cbb8c0b673edd00c80e60a1932832",
|
|
"playwright-output/runner-everyday-workflows--90607-the-browser-and-public-APIs/attachments/runner-e2e-result-b9a31955fc83e0239705bc49c9a7cb13939d91ed.json": "f8ef0e87c24ecacb8dd18ebc158771a74b5cbb8c0b673edd00c80e60a1932832",
|
|
"result.json": "f8ef0e87c24ecacb8dd18ebc158771a74b5cbb8c0b673edd00c80e60a1932832",
|
|
"snapshots/browser-diagnostics.json": "94d2792cae9e2a1628779f7349538e0ce00418a6e448c54fcbba978b55a1b8ed"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"ref": "refs/heads/codex/native-procedure-guidance",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325777217"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "transient_infrastructure",
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 1,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "887c612dedbc4319e8ddf443d238b9ea6ded27e701748982a5b6f24ffeead10e",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "b1f020afba6828a685735014a221db1ef141b9958f4e42b075dfa14f2a574d5f",
|
|
"html-report/data/1c84a0af56116198fe02730dc6e07c03e7708a91.json": "887c612dedbc4319e8ddf443d238b9ea6ded27e701748982a5b6f24ffeead10e",
|
|
"playwright-output/runner-everyday-workflows--fdae5-the-browser-and-public-APIs/attachments/runner-e2e-result-3c37bc94f4716823d58a0e836c85db647f971bc7.json": "887c612dedbc4319e8ddf443d238b9ea6ded27e701748982a5b6f24ffeead10e",
|
|
"result.json": "887c612dedbc4319e8ddf443d238b9ea6ded27e701748982a5b6f24ffeead10e",
|
|
"snapshots/api-state.json": "a09940179b96e90991043e5305e3820c0e23b227b7d77ec93edb6c9a42e664a0",
|
|
"snapshots/browser-diagnostics.json": "44550ff6356f15e46b961ed53c4c5130c16b59360a55c2b7ffd57eda5bcf1987",
|
|
"snapshots/chat-final-run-ledger.json": "5650a472f29e83dde20802f25d913d403fd5ef590d4dedabf6da3fd0aeb9c3a1",
|
|
"snapshots/everyday-workflow.json": "495455927af91cd9686bf7441aaf078b895bdf5dbf4b0f392bcd0da930a5862b",
|
|
"snapshots/fixtures.json": "a566dedac5d1b16487d0e4d064905797a2e65a19a0c068c5f3aece0130c88f39"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.hire-reuse",
|
|
"source": {
|
|
"sha": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"ref": "refs/heads/codex/native-procedure-guidance",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325777217"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 2,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "33995933e0ba31d829c5e52a5a5169aa1bf5bd3531a33e2762c5e2f021b65db3",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "c253edc0ea3e1a215a04bdddc07b420b90ed75ae9581b299a93653939ac48149",
|
|
"html-report/data/34410e200327d4861b2fa8863ac1446eb3daaf4e.json": "33995933e0ba31d829c5e52a5a5169aa1bf5bd3531a33e2762c5e2f021b65db3",
|
|
"playwright-output/runner-everyday-workflows--9a7d2-the-browser-and-public-APIs/attachments/runner-e2e-result-a19b320e711940bd3128319ae068af3a6e41fa43.json": "33995933e0ba31d829c5e52a5a5169aa1bf5bd3531a33e2762c5e2f021b65db3",
|
|
"result.json": "33995933e0ba31d829c5e52a5a5169aa1bf5bd3531a33e2762c5e2f021b65db3",
|
|
"snapshots/api-state.json": "150a6ecc15a5a64abfd8742289301eca792b406cf51dfc95df22cd66531d09bb",
|
|
"snapshots/browser-diagnostics.json": "37fe5436927fd28b49859eca1751a2ff4b6f8683ec96274273724dd03b2a9f30",
|
|
"snapshots/chat-final-run-ledger.json": "5397a9497d49879c09dab2c9b0a27d108792a8f36c6643593f848a259868829e",
|
|
"snapshots/everyday-workflow.json": "f0dbaaa800c3c3808e62370a69bb6b8dca1d43bd51b187617a2242a12e18a2d3",
|
|
"snapshots/fixtures.json": "91da96a65a7196b1a2b3e3d359e164063e6c5ef95096641a8789f2944fff825e"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-codex.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"ref": "refs/heads/codex/native-procedure-guidance",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325777217"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 3,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "833df00d30db77e3aad32e091ed73e8161ad38ff23d98eff7f0bc4f2cc8fa8f0",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "446cae83c521fe16af543ddac6b4816b8aa11fb24aa9462442b57fc8a16ad9fa",
|
|
"html-report/data/1adabfb6931e1bbba0e3717037ae9a4cfa8cd37e.json": "833df00d30db77e3aad32e091ed73e8161ad38ff23d98eff7f0bc4f2cc8fa8f0",
|
|
"playwright-output/runner-everyday-workflows--a6fcb-the-browser-and-public-APIs/attachments/runner-e2e-result-8d7203080e1991ce5b786549e01e4580160cfcee.json": "833df00d30db77e3aad32e091ed73e8161ad38ff23d98eff7f0bc4f2cc8fa8f0",
|
|
"result.json": "833df00d30db77e3aad32e091ed73e8161ad38ff23d98eff7f0bc4f2cc8fa8f0",
|
|
"snapshots/api-state.json": "8d02b400e815af4adc73f9295a013841b5b24744ab17aace60b85240a08ecbd3",
|
|
"snapshots/browser-diagnostics.json": "c5ad3d5ac16cb2d57635abf7dffb7ee9db66b3ed9af02fa93e53fd015b3da435",
|
|
"snapshots/chat-final-run-ledger.json": "855e8c256db5cfc1bfd28a79325e7b14d205a1a7096d0fe2d95b16b3d6b3bddc",
|
|
"snapshots/everyday-workflow.json": "a86aafa04eba8193a6acd8ed927568b7fd82d2ffc69894e43c67361de3e60d33",
|
|
"snapshots/fixtures.json": "87a46bfe1470d96dbba3c61a02f82c389e7fa765c0631ae3af148e9bd9ea5bf7"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-codex.local.hire-reuse",
|
|
"source": {
|
|
"sha": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"ref": "refs/heads/codex/native-procedure-guidance",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325777217"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "94c594173473b2b40e6f8635d97f7e70e71e856cf99a84400fcdf3185782b81a",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "63babab4e851eedd8dc00474d0815c507d60343e121e770ce0f05c4aefa740dd",
|
|
"html-report/data/f84e6632cdf019513a84a0b08672da740648a5e7.json": "94c594173473b2b40e6f8635d97f7e70e71e856cf99a84400fcdf3185782b81a",
|
|
"playwright-output/runner-everyday-workflows--d934c-the-browser-and-public-APIs/attachments/runner-e2e-result-f768e9d7e6ca953c7c34ef967667dbac637d037e.json": "94c594173473b2b40e6f8635d97f7e70e71e856cf99a84400fcdf3185782b81a",
|
|
"result.json": "94c594173473b2b40e6f8635d97f7e70e71e856cf99a84400fcdf3185782b81a",
|
|
"snapshots/api-state.json": "d6b685e3fedec4993160abbb9436dfbf3f819c82f78b2acf912d9fea24150fdb",
|
|
"snapshots/browser-diagnostics.json": "21d2288d0a3b6db142c294c1a11657309147493d5295ed4f00a74c76eef34660",
|
|
"snapshots/chat-final-run-ledger.json": "667c2d841eb10e43176fe54e9344619bdebe15c7404d85f921be009e5a028984",
|
|
"snapshots/everyday-workflow.json": "5e5375d89bad4c311b4eecb0551c12d3aaad05337ab32b0dd3807c2b76fd3997",
|
|
"snapshots/fixtures.json": "9cc5dfc3ae89c49634c051c5f33be0ea1a8acc4932d2877225eac25021b1bdf8"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-opencode.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"ref": "refs/heads/codex/native-procedure-guidance",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325777217"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 3,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "41f0ad5604813feb32fa37fb6f57a4f413ef1a57dbeba2619449cfbc8a50871f",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "c3f408e84ab9a36f7abe3a58dd3b7df514f4c5f48ceed3efcf42caa36fec39c7",
|
|
"html-report/data/a829fecb87471d91b31c772c54d90f475156f3e8.json": "41f0ad5604813feb32fa37fb6f57a4f413ef1a57dbeba2619449cfbc8a50871f",
|
|
"playwright-output/runner-everyday-workflows--616ad-the-browser-and-public-APIs/attachments/runner-e2e-result-f10cd915174a4203d526eb0a386078715e179a96.json": "41f0ad5604813feb32fa37fb6f57a4f413ef1a57dbeba2619449cfbc8a50871f",
|
|
"result.json": "41f0ad5604813feb32fa37fb6f57a4f413ef1a57dbeba2619449cfbc8a50871f",
|
|
"snapshots/api-state.json": "9ef26183b198e896e7aa009caae51a43485c8eec8882e6b87fc0bb01eb86d38e",
|
|
"snapshots/browser-diagnostics.json": "4cd36a195c3a38b669c2c2be7c3724fceabf43229b27178664dee30755e97cdb",
|
|
"snapshots/chat-final-run-ledger.json": "10634af18a8d28e77c905e5688ea7289bd846a26d2dd9ab1b2a123830aff476a",
|
|
"snapshots/everyday-workflow.json": "21b9add74a177fb9433bcfeec3c41fabded5449713da1a1eeff188f973a54808",
|
|
"snapshots/fixtures.json": "1a3dedfabfc9e1114b14eedaf4725d8a3b4f81d5945cfea1bd61e34f49fdfcf1"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-opencode.local.hire-reuse",
|
|
"source": {
|
|
"sha": "eb01d073c35134bf0d373947b55a79f9e443f8ee",
|
|
"ref": "refs/heads/codex/native-procedure-guidance",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37325777217"
|
|
},
|
|
"suiteDefinitionHash": "c0b25d634e19cb18f647058f357a83cb5a14845b6fada1764cf21258d00abf54",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "permanent_infrastructure",
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 0,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [],
|
|
"resultSha256": "73d68db5221120745f90c411d265faa13f3b99bd4e22735e4ceab320ec2f8c6a",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "2607859e543f14494e344ceca6896cda6ccb7810c1fcc8c62d89009add2652bd",
|
|
"html-report/data/f2c1f121afe0207d99ecf59a23882f023b2296e0.json": "73d68db5221120745f90c411d265faa13f3b99bd4e22735e4ceab320ec2f8c6a",
|
|
"playwright-output/runner-everyday-workflows--90607-the-browser-and-public-APIs/attachments/runner-e2e-result-ae68640cfa3c5759281518d751c2991e889aaa73.json": "73d68db5221120745f90c411d265faa13f3b99bd4e22735e4ceab320ec2f8c6a",
|
|
"result.json": "73d68db5221120745f90c411d265faa13f3b99bd4e22735e4ceab320ec2f8c6a",
|
|
"snapshots/browser-diagnostics.json": "94d2792cae9e2a1628779f7349538e0ce00418a6e448c54fcbba978b55a1b8ed"
|
|
}
|
|
}
|
|
],
|
|
"evidenceHashEntryCount": 100
|
|
},
|
|
"corrected": {
|
|
"totals": {
|
|
"baseline": {
|
|
"pass": 5,
|
|
"fail": 1,
|
|
"retainedStartedRuns": 27
|
|
},
|
|
"candidate": {
|
|
"pass": 3,
|
|
"fail": 3,
|
|
"retainedStartedRuns": 33
|
|
}
|
|
},
|
|
"pairs": [
|
|
{
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.delegate-feedback",
|
|
"baseline": "passed",
|
|
"candidate": "passed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.hire-reuse",
|
|
"baseline": "passed",
|
|
"candidate": "passed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-codex.local.delegate-feedback",
|
|
"baseline": "passed",
|
|
"candidate": "failed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-codex.local.hire-reuse",
|
|
"baseline": "passed",
|
|
"candidate": "passed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-opencode.local.delegate-feedback",
|
|
"baseline": "failed",
|
|
"candidate": "failed"
|
|
},
|
|
{
|
|
"executionId": "everyday-workflows.runner-opencode.local.hire-reuse",
|
|
"baseline": "passed",
|
|
"candidate": "failed"
|
|
}
|
|
],
|
|
"cells": [
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"ref": "refs/heads/codex/native-procedure-baseline-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331642168"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "689e5154888c5ea3697b7577587c2aa8630dddb2100cb1b7786bc995c13c8097",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "07f77f464ba2e896decfb2db87db62b4c19ac37303f4b9e0cb3b89afcf707af6",
|
|
"html-report/data/580f383fcce38922446d0dd2f23a50283699a0b1.json": "689e5154888c5ea3697b7577587c2aa8630dddb2100cb1b7786bc995c13c8097",
|
|
"playwright-output/runner-everyday-workflows--fdae5-the-browser-and-public-APIs/attachments/runner-e2e-result-1a68e97dada87da3688de2deb3e9c9f12df25ad9.json": "689e5154888c5ea3697b7577587c2aa8630dddb2100cb1b7786bc995c13c8097",
|
|
"result.json": "689e5154888c5ea3697b7577587c2aa8630dddb2100cb1b7786bc995c13c8097",
|
|
"snapshots/api-state.json": "984d171f9e501c3b700e7c60e0abbed70f4abe3012dcfb33ad35ad0fa823c486",
|
|
"snapshots/browser-diagnostics.json": "26865f180e7e1f9d174b83fabcf13ed8912731f7533e257ad85bb01fb84d9d1e",
|
|
"snapshots/chat-final-run-ledger.json": "0740de3c3c6f91c79b7fe8b88ee75176d24d5fd07e12597f26d5504c682e1ee4",
|
|
"snapshots/everyday-workflow.json": "d8ee30954306a973cf4e1c412e648b87429961b1ba485e5fe76735d22e8b9b9e",
|
|
"snapshots/fixtures.json": "34b5db8164c2149e57b3bad0c249c7c738b114ccf11897533c85644d22700159"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.hire-reuse",
|
|
"source": {
|
|
"sha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"ref": "refs/heads/codex/native-procedure-baseline-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331642168"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "08ad50bd253f9551f531bd4ce0a15ee200bd4c544a171a6540b8f39bd526f3cd",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "ad4ae5c149d9a288f09510a9f40d751a0e16056a15c0e3e5fc22673f4fe89b5a",
|
|
"html-report/data/f1b2ae22fca1509f0c62f23f8d6df4ac70ed88ea.json": "08ad50bd253f9551f531bd4ce0a15ee200bd4c544a171a6540b8f39bd526f3cd",
|
|
"playwright-output/runner-everyday-workflows--9a7d2-the-browser-and-public-APIs/attachments/runner-e2e-result-345f227fa1bde20467d1efcdba85e9a90b9d2e06.json": "08ad50bd253f9551f531bd4ce0a15ee200bd4c544a171a6540b8f39bd526f3cd",
|
|
"result.json": "08ad50bd253f9551f531bd4ce0a15ee200bd4c544a171a6540b8f39bd526f3cd",
|
|
"snapshots/api-state.json": "748fcd21ef5cd800eec08adc76f8713475adb3c54605eec179001c79272347a8",
|
|
"snapshots/browser-diagnostics.json": "9cac545625ca2970a8b3dba8c8d3ab50473dfba21fd057d7d4d6fe59724f0a8e",
|
|
"snapshots/chat-final-run-ledger.json": "b8e78dbd52dff17b77f1dfcf18bf469a691e9bf30cff3dcc91e6d3d761334502",
|
|
"snapshots/everyday-workflow.json": "10552e78b7e70a050e43003419a37f5c215665e890cc928e188806c30e464589",
|
|
"snapshots/fixtures.json": "667ce63d133703653093d3217dd537ac50d5ada0bd3c6d0e15e4cc3607305314"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-codex.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"ref": "refs/heads/codex/native-procedure-baseline-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331642168"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "d0535b600235580ada1b0227b84ae5f621e1a3932a7a94e3285bfebfe7dd716a",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "8317f0026d88709a1fcd716b9373786bee39b9f61902a3c1cc48c4ff64b1b59e",
|
|
"html-report/data/0192f8929945cb662a6241d65fdd9e43994b876e.json": "d0535b600235580ada1b0227b84ae5f621e1a3932a7a94e3285bfebfe7dd716a",
|
|
"playwright-output/runner-everyday-workflows--a6fcb-the-browser-and-public-APIs/attachments/runner-e2e-result-be735800e00ca4152df495a405c0f0024010d799.json": "d0535b600235580ada1b0227b84ae5f621e1a3932a7a94e3285bfebfe7dd716a",
|
|
"result.json": "d0535b600235580ada1b0227b84ae5f621e1a3932a7a94e3285bfebfe7dd716a",
|
|
"snapshots/api-state.json": "4239640e901c755f81484e852b092a4ee99caf7421ca4ed364b418a59f6def51",
|
|
"snapshots/browser-diagnostics.json": "5b90904dacfbd32aecbe9cbe8ad59962c09242ee61ddfcb5feac326030b2e2d4",
|
|
"snapshots/chat-final-run-ledger.json": "8bb0946ed361a28a841c7d209ca18619436d087ec7f88fa7e1c1b1f326e35aae",
|
|
"snapshots/everyday-workflow.json": "668dfef6034aed2079cb9e8023fef8198074dbd08ebe3521b6c70b77de783ca6",
|
|
"snapshots/fixtures.json": "59d95bfccd4f60d1ff990483fba59210d0a8eaf63a8f0197c328e8e6c7bfb654"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-codex.local.hire-reuse",
|
|
"source": {
|
|
"sha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"ref": "refs/heads/codex/native-procedure-baseline-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331642168"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "a23a9e92d7ff47999ee6483f5a927f41c725c0cce69d91c150cf4e9ab2cbe5eb",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "0a8bef60505c30179b11c36c764ba991e3e2a56be3b61a6fa7c3848786a5ebe9",
|
|
"html-report/data/71d82d27caa117b7d6102f8fe8ff41d42c222c7f.json": "a23a9e92d7ff47999ee6483f5a927f41c725c0cce69d91c150cf4e9ab2cbe5eb",
|
|
"playwright-output/runner-everyday-workflows--d934c-the-browser-and-public-APIs/attachments/runner-e2e-result-0c090b31617607484975dfd0adaed62b9f8ea755.json": "a23a9e92d7ff47999ee6483f5a927f41c725c0cce69d91c150cf4e9ab2cbe5eb",
|
|
"result.json": "a23a9e92d7ff47999ee6483f5a927f41c725c0cce69d91c150cf4e9ab2cbe5eb",
|
|
"snapshots/api-state.json": "9df20d0d9a4a319a87271c66eb38ea57dbc649464574c10e7cf62734378a29a6",
|
|
"snapshots/browser-diagnostics.json": "467a514913b0d01e876353bf5a76813fc7fb717dd8cbc567ee13eb17fbca8f72",
|
|
"snapshots/chat-final-run-ledger.json": "215358fb7caa14e656a15464b693940b9c202b0aefee817fe5375eb7912b0df2",
|
|
"snapshots/everyday-workflow.json": "b70f39ea8147669a795deffe59ce6fcc024f9d39a0e7a9ef8a809f8a8636f4e8",
|
|
"snapshots/fixtures.json": "7d104f51b41eee886b810b583b90a242652aba9e0596d9dddab5e2b4d764da42"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-opencode.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"ref": "refs/heads/codex/native-procedure-baseline-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331642168"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 3,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": false
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "a51a5345df8812be3f754d3f0e5df30a1f826150a6c7778f0dc0726d642d7f4b",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "71573393fe99cd451ce4fdc90010c72572b769b2b0bbdfa11bc3f28bd8655d2a",
|
|
"html-report/data/0c3bfda38c3dc2d51f9f55d711970b533bf283fa.json": "a51a5345df8812be3f754d3f0e5df30a1f826150a6c7778f0dc0726d642d7f4b",
|
|
"playwright-output/runner-everyday-workflows--616ad-the-browser-and-public-APIs/attachments/runner-e2e-result-b4cc93b23776a0ae821207b52c43a3f464f99977.json": "a51a5345df8812be3f754d3f0e5df30a1f826150a6c7778f0dc0726d642d7f4b",
|
|
"result.json": "a51a5345df8812be3f754d3f0e5df30a1f826150a6c7778f0dc0726d642d7f4b",
|
|
"snapshots/api-state.json": "bab60c326f4ad0971748cf30f2de276eb7f69c985a38733d923c78b3fec659aa",
|
|
"snapshots/browser-diagnostics.json": "d164e04291112ffe4efc8d248d61608e944d7b1ae6d3fb14c10c5a655eb06790",
|
|
"snapshots/chat-final-run-ledger.json": "de0b3b353dd606474591946c3bdb42aac2084a0de01ec83b5faae0004c250003",
|
|
"snapshots/everyday-workflow.json": "6505cc96cb7ff2f371c73eeef2318e105fb8b8aaa14f7b3252ef8a432f048345",
|
|
"snapshots/fixtures.json": "46fdccbefe0f5b14de8d26798e04bf042c1b9ea9f5155cde523d292fa32c4e85"
|
|
}
|
|
},
|
|
{
|
|
"variant": "baseline",
|
|
"executionId": "everyday-workflows.runner-opencode.local.hire-reuse",
|
|
"source": {
|
|
"sha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"ref": "refs/heads/codex/native-procedure-baseline-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331642168"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "741f7e0deee7f7b7b8c8498e998100ad86e3670340a116bba7e3d7f19b139ff4",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "70ca2a5525ce594a58d9e88ceae252e03918fe9c5af5f478bd345e7623085aff",
|
|
"html-report/data/b4ef23d5b811da0b4c40ac32d7109b15b2ed3eea.json": "741f7e0deee7f7b7b8c8498e998100ad86e3670340a116bba7e3d7f19b139ff4",
|
|
"playwright-output/runner-everyday-workflows--90607-the-browser-and-public-APIs/attachments/runner-e2e-result-a3041877169260aa9a9b8c577933a9aaffdc90d6.json": "741f7e0deee7f7b7b8c8498e998100ad86e3670340a116bba7e3d7f19b139ff4",
|
|
"result.json": "741f7e0deee7f7b7b8c8498e998100ad86e3670340a116bba7e3d7f19b139ff4",
|
|
"snapshots/api-state.json": "50a969cb558df7ca61bad975ace655421f6afc9e894a344834b79a8e760ce5e4",
|
|
"snapshots/browser-diagnostics.json": "f5d63620b23461a6ea972734fe519a3e1e08cd6d4301e4520255b552d20862a8",
|
|
"snapshots/chat-final-run-ledger.json": "2688ded68ad348d62bfe745b4a26c55f1108a6c8c2bcc35e5cb0fcd421130e4b",
|
|
"snapshots/everyday-workflow.json": "ca3bf543901083b817c9e08e336a54a28a61d7ca4af5a03fbed5579ce372adce",
|
|
"snapshots/fixtures.json": "b04cca5e54ad156e911331ae8e2d6af3b8a3e31feedbd3ecab464414ca7d9a75"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"ref": "refs/heads/codex/native-procedure-candidate-v2-recovery",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37334061104"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "1d0a1e688b1f003491c27497edeeff963a5459ed99acfd32ada06fb9decd93b1",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "977cf3c0ac2c93190b71f6bb766ce44107e78f1d9b726db79d7e4c7e1efe7ed0",
|
|
"html-report/data/8e6146fc86993655e3a364a37eff7cf125570318.json": "1d0a1e688b1f003491c27497edeeff963a5459ed99acfd32ada06fb9decd93b1",
|
|
"playwright-output/runner-everyday-workflows--fdae5-the-browser-and-public-APIs/attachments/runner-e2e-result-c75e8d1cd375ccf9308dc1626ec8d9959eae7c13.json": "1d0a1e688b1f003491c27497edeeff963a5459ed99acfd32ada06fb9decd93b1",
|
|
"result.json": "1d0a1e688b1f003491c27497edeeff963a5459ed99acfd32ada06fb9decd93b1",
|
|
"snapshots/api-state.json": "e5ce82339b356c8125b32949371049d6b1ba85305ecff321e2b30ee022029b18",
|
|
"snapshots/browser-diagnostics.json": "72af20421dd49d3362dc6b5688cd96d960bf41b34ed58dd8833fea2ca9d58386",
|
|
"snapshots/chat-final-run-ledger.json": "b3d15cabdbe0819da45f08621fba6df2e29f6d2655ea3f6a51a55daed14b15ef",
|
|
"snapshots/everyday-workflow.json": "d2074d8f9c302d383ea3e25b81a5d844173725b70ba7b1fecca3bfa8f75687e6",
|
|
"snapshots/fixtures.json": "7ad274c380095f860f5bf95a31dd2e9bd772b64c7230f30288aa171560c43f11"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-acpx-claude.local.hire-reuse",
|
|
"source": {
|
|
"sha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"ref": "refs/heads/codex/native-procedure-candidate-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331580879"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "claude-sonnet-5",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "6f4ef2c3760227fc504de1030e30788afe54168b91db72f5088c5eb634295a47",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "7c04512304d32ac249741243f3081f87a6c9d3530b29f2fc31af2619ae79ba1a",
|
|
"html-report/data/e789dbe6a578c12c42b3ab46cc3c657158c41129.json": "6f4ef2c3760227fc504de1030e30788afe54168b91db72f5088c5eb634295a47",
|
|
"playwright-output/runner-everyday-workflows--9a7d2-the-browser-and-public-APIs/attachments/runner-e2e-result-02e9625499f3a6da62e679ca785ff0a376465eef.json": "6f4ef2c3760227fc504de1030e30788afe54168b91db72f5088c5eb634295a47",
|
|
"result.json": "6f4ef2c3760227fc504de1030e30788afe54168b91db72f5088c5eb634295a47",
|
|
"snapshots/api-state.json": "83a3295d9563dcbbaccbdcf5278a277af41cd4a24b723af1c28f5f6795e895d7",
|
|
"snapshots/browser-diagnostics.json": "85188804dee7b70a86a96808b1625e54b421663f3a47f8662845c44e8f675bea",
|
|
"snapshots/chat-final-run-ledger.json": "042fb89f01a7b790147efc55d0879c8fc39090518cf6bf9c3ed4a3772acf6ac6",
|
|
"snapshots/everyday-workflow.json": "063028cb1f58cf86b2c13ac2ae17a5d54f0780c5f1883a489f00904ce2e3696c",
|
|
"snapshots/fixtures.json": "366e91cf2365f156fa370ac1fb816f920ebceba5d6978b5fd6e773c2717a0601"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-codex.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"ref": "refs/heads/codex/native-procedure-candidate-v2-recovery",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37334061104"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "2b4c8af68619aca7ffc2ac40f2569b1595ba21bdabf2481c6073b7186d220adf",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "008abe42e978f9224955e623d237b53a0dfaefafec767635aef09dbf20874c77",
|
|
"html-report/data/247c57b22f37607373563c94a9554ba3c8f6a4f0.json": "2b4c8af68619aca7ffc2ac40f2569b1595ba21bdabf2481c6073b7186d220adf",
|
|
"playwright-output/runner-everyday-workflows--a6fcb-the-browser-and-public-APIs/attachments/runner-e2e-result-146fc7aa673f1bd55ea9e51ad19afdbd163cc99b.json": "2b4c8af68619aca7ffc2ac40f2569b1595ba21bdabf2481c6073b7186d220adf",
|
|
"result.json": "2b4c8af68619aca7ffc2ac40f2569b1595ba21bdabf2481c6073b7186d220adf",
|
|
"snapshots/api-state.json": "d4d98e4fa703535b9d7a02eab49f3ac69e32836c1a556285327e0c51a88ea6db",
|
|
"snapshots/browser-diagnostics.json": "791cb4bc513bbbfbd7342f09d827520a8aed93bb14b1749e911dba31be0f0a94",
|
|
"snapshots/chat-final-run-ledger.json": "3c97659721c812cc3ab632f532982c8d1889133cad37ca324aece54eb0d78152",
|
|
"snapshots/everyday-workflow.json": "5e9907a320ace5ddbdc695a846ea711bd887aaddc793e56c9e20af9a6ec386b4",
|
|
"snapshots/fixtures.json": "96d4acbe3a2da0c05d8f5d41447302e805f76e6bf66996d8d8ab26f6db99c4e5"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-codex.local.hire-reuse",
|
|
"source": {
|
|
"sha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"ref": "refs/heads/codex/native-procedure-candidate-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331580879"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "passed",
|
|
"originalFailureClass": null,
|
|
"configuredModel": "gpt-5.6-sol",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 6,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-reused",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed-revision",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator-_",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.separator--",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reused-delivery.reject-invalid-separator",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
}
|
|
],
|
|
"resultSha256": "266e8d35f8d4ef3a570ea6940c64b6a9892eb6ab1d14324625da2337b4e7b99c",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "4b60c8a396e229b271fb377c44148e7b922286093824c48ca86782ed19490c0b",
|
|
"html-report/data/7c7b49ba9c9fcdcbde63035d3af25ba0bdade045.json": "266e8d35f8d4ef3a570ea6940c64b6a9892eb6ab1d14324625da2337b4e7b99c",
|
|
"playwright-output/runner-everyday-workflows--d934c-the-browser-and-public-APIs/attachments/runner-e2e-result-2d7949d3b25761e104f66bbe3f3f56425e20a3ab.json": "266e8d35f8d4ef3a570ea6940c64b6a9892eb6ab1d14324625da2337b4e7b99c",
|
|
"result.json": "266e8d35f8d4ef3a570ea6940c64b6a9892eb6ab1d14324625da2337b4e7b99c",
|
|
"snapshots/api-state.json": "87cc674d10f53be4beecaf60e8c50813e76a2c774bf5c1e9173733be23979615",
|
|
"snapshots/browser-diagnostics.json": "139fedd6a92bed9ff5ae218d8b012da508b97cf4494eaa102472b59a230d952d",
|
|
"snapshots/chat-final-run-ledger.json": "692d37db951fe5b2852a1748f72392cb14977d07cfee009961535e8170830712",
|
|
"snapshots/everyday-workflow.json": "68fbc3745b7aa60145479bd4cbd8646e3faac56b96bd501c438491d04b881964",
|
|
"snapshots/fixtures.json": "396dcdb89dd9656c748c5a5a1e34cf774bcb2e178124bd46e8d051dba963fd10"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-opencode.local.delegate-feedback",
|
|
"source": {
|
|
"sha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"ref": "refs/heads/codex/native-procedure-candidate-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331580879"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 4,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "reply-3-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "one-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "child-consumed-feedback",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.length-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "delegated-delivery.reject-zero-length",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "feedback-delivered-to-child",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "submitted-replies-consumed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-finishes-after-child",
|
|
"passed": false
|
|
},
|
|
{
|
|
"id": "native-model-config",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-terminal-contract",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "tasks-done",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "settled",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "native-runtime",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "successful-runs",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "bounded-work",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "parent-owned-by-lead",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "no-pending-bookkeeping",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "17c1145948d1f444211d9b319f36d7e9562bb4c99e804024cfbe0bf6f780354b",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "4911bab65a4f2e24573870946b894d3f5ca4030819ae023881644f7c35e443fc",
|
|
"html-report/data/ef9ff8e5b5a82f998ba0c465bdce4ec2c19294d2.json": "17c1145948d1f444211d9b319f36d7e9562bb4c99e804024cfbe0bf6f780354b",
|
|
"playwright-output/runner-everyday-workflows--616ad-the-browser-and-public-APIs/attachments/runner-e2e-result-1f3520dbb09fae78c011e89c25533003bd91a93a.json": "17c1145948d1f444211d9b319f36d7e9562bb4c99e804024cfbe0bf6f780354b",
|
|
"result.json": "17c1145948d1f444211d9b319f36d7e9562bb4c99e804024cfbe0bf6f780354b",
|
|
"snapshots/api-state.json": "7b0631c60b459355779aa75d83554df845192bd19ee748c5ef8acda3a15a97ff",
|
|
"snapshots/browser-diagnostics.json": "cf9da33e8f81b874be572e7bce6103fc71bb0add1ecc9a48d1805379bdb1430d",
|
|
"snapshots/chat-final-run-ledger.json": "3f87d37dd18a1311f34d75ab3e07f1e7cf674d3b1180ca219e241ae8decea5dc",
|
|
"snapshots/everyday-workflow.json": "e67b30bbf111cd16281f2b6bed2a99343a62924493149b461716dce96c578efb",
|
|
"snapshots/fixtures.json": "f1b5d61b9cb04ca16642614b53a5f676a527466d31c835ecdd417ea9bda60aed"
|
|
}
|
|
},
|
|
{
|
|
"variant": "candidate",
|
|
"executionId": "everyday-workflows.runner-opencode.local.hire-reuse",
|
|
"source": {
|
|
"sha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"ref": "refs/heads/codex/native-procedure-candidate-v2",
|
|
"workflowRunUrl": "https://github.com/paperclipai/paperclip/actions/runs/37331580879"
|
|
},
|
|
"suiteDefinitionHash": "acd17b4acbd7051070531f7d7587131139900f54245b992eb7a62faacda8e334",
|
|
"attempt": 1,
|
|
"originalStatus": "failed",
|
|
"originalFailureClass": "candidate_failure",
|
|
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
"cleanup": "passed",
|
|
"startedRunCount": 9,
|
|
"resultLedgerRunSetEqual": true,
|
|
"originalChecks": [
|
|
{
|
|
"id": "exactly-one-hire",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "manager-correct",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hire-native-connection",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-agent-executed",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.one-slugify-source",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.readme-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.project-tests-present",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-0",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-1",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-2",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-3",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-4",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-5",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-6",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.base-7",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "hired-delivery.importable-function",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "reply-5-stored-once",
|
|
"passed": true
|
|
},
|
|
{
|
|
"id": "workflow-completed",
|
|
"passed": false
|
|
}
|
|
],
|
|
"resultSha256": "b001ce9d1741adc087dab22643a3f65e5dabc88bcbde432dcebe9c1f793f6a8e",
|
|
"evidenceHashes": {
|
|
"evidence-manifest.json": "1426f8127ca46828ef5320c3bf5ab2134d60172361bb1c43a2b415fe61c766ff",
|
|
"html-report/data/9777bc5b0460c2689ba53533aad6b65d6879e30b.json": "b001ce9d1741adc087dab22643a3f65e5dabc88bcbde432dcebe9c1f793f6a8e",
|
|
"playwright-output/runner-everyday-workflows--90607-the-browser-and-public-APIs/attachments/runner-e2e-result-e406f7ff0925c2cfeaed8101a63cb789dac7ec22.json": "b001ce9d1741adc087dab22643a3f65e5dabc88bcbde432dcebe9c1f793f6a8e",
|
|
"result.json": "b001ce9d1741adc087dab22643a3f65e5dabc88bcbde432dcebe9c1f793f6a8e",
|
|
"snapshots/api-state.json": "fa9852b1155d3f3753bfe711e30e8fff6847cf30d82f58089db6592365719c59",
|
|
"snapshots/browser-diagnostics.json": "0ae03cb7af8cb2e405f6a1d78f3fb86fe9193de6bc07c5b7a69f44f4956c664d",
|
|
"snapshots/chat-final-run-ledger.json": "ad8bed7e06561967f88ea2b0beb12728257db8b579f3547df55729225fbb2889",
|
|
"snapshots/everyday-workflow.json": "b5b0b3e3f07eaa9be98b1c335585fa1ba80bdda1acab3f95b81e512b376f375f",
|
|
"snapshots/fixtures.json": "1b310f2e030acbf3997ffd87f648f0106e7a06e475671c16a8164f7b1400cb4c"
|
|
}
|
|
}
|
|
],
|
|
"evidenceHashEntryCount": 108
|
|
},
|
|
"correctedPairCounts": {
|
|
"newFailures": 2,
|
|
"newPasses": 0,
|
|
"unchangedPasses": 3,
|
|
"unchangedFailures": 1,
|
|
"pending": 0
|
|
},
|
|
"sourceModelSuiteAttemptLeadBudgetCleanupAndRunSetAssertions": true,
|
|
"interruptions": {
|
|
"campaign": 37331580879,
|
|
"source": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"recoveryPolicy": "One bounded replacement cell each on identical source, distinct workflow target; preserve original interruption, unknown run/cost totals, and both attempt identities. Do not repeat successful baselines.",
|
|
"recoveryTrustedWorkflowSha": "a65ca0950834a85bb93bcc4b4042ecacdebfef53",
|
|
"workflowByteIdenticalToOriginal": true,
|
|
"workflowFileSha256": "eb570267e66043c1196338bb386b07ab8e8e8306ba04b6223f13b73b04fdc8f4",
|
|
"recoveryCampaign": 37334061104,
|
|
"cells": [
|
|
{
|
|
"profile": "claude",
|
|
"jobId": 111838521502,
|
|
"logSha256": "ad84bbdc76f9a48a16886f676335ec5fe9ba2190c47f912f34619d010c36bc1a",
|
|
"originalWorkflowConclusion": "failure",
|
|
"behavioralGrade": null,
|
|
"finalLedger": null,
|
|
"cleanupReceipt": null,
|
|
"actualRunCount": null,
|
|
"actualCharges": null,
|
|
"observedPartialRunRecordCount": 4
|
|
},
|
|
{
|
|
"profile": "codex",
|
|
"jobId": 111838521694,
|
|
"logSha256": "3f7b36ae28d0cb254a6199f548ce6c347e91a39dcc456eb7e8078290211008ea",
|
|
"originalWorkflowConclusion": "failure",
|
|
"behavioralGrade": null,
|
|
"finalLedger": null,
|
|
"cleanupReceipt": null,
|
|
"actualRunCount": null,
|
|
"actualCharges": null,
|
|
"observedPartialRunRecordCount": 4
|
|
}
|
|
]
|
|
},
|
|
"accounting": {
|
|
"originalRetainedStartedRuns": 40,
|
|
"correctedCompletedOrRecoveredRetainedStartedRuns": 60,
|
|
"allRetainedStartedRuns": 100,
|
|
"additionalInterruptedActualRuns": null,
|
|
"actualCharges": null,
|
|
"completeTotalRunCount": false,
|
|
"infrastructureRecoveryCellCount": 2,
|
|
"limit": "Completed/recovered cell ledgers match their result run sets. Two earlier interrupted candidate jobs lack final result/ledger/cleanup receipts; their actual runs/charges are additional and incompletely known. No claim of total known cost or first-attempt success."
|
|
},
|
|
"measurement": {
|
|
"corrected-baseline": {
|
|
"schema": "paperclip.native-procedure-measurement.v2",
|
|
"sourceSha": "aecfa3b6a5cab110a9b926e93705311bbcf4db47",
|
|
"sourceDirty": false,
|
|
"measuredInputsDirty": false,
|
|
"sourceHashes": {
|
|
"server/src/onboarding-assets/default/AGENTS.md": "c6318bf7e425cc0dd7acd4ff2905dab5c46ffddedef979afdf06720c82f6e95a",
|
|
"packages/paperclip-runner/src/contracts/runtime-context.ts": "a17e30a69f3cb38fba00c37e1a06753d723651a004e6050c9e77ae202a61264b",
|
|
"packages/paperclip-runner/src/protocol-actions/hire-agent.ts": "3b786405fa48e6ba9cf63789c03bbbb370fc2962a4a5c8bd80ccca57759bf912",
|
|
"packages/paperclip-runner/src/protocol-actions/create-task.ts": "4efe71c1693dd65d8087b2fdbaf5831b397130d036afb837ece20d4791631805",
|
|
"packages/paperclip-runner/src/protocol-actions/set-dependencies.ts": "53c65776ce4984f177d261c1e53957a97e1a236ce58e731119cb8ee26f141da2",
|
|
"packages/paperclip-runner/src/catalog/semantic-action-catalog.ts": "170a1e97f6402d87041b944fc45a7b3a41cf5d07dc158b3cc494ac38208dbd6d",
|
|
"packages/paperclip-runner/generated/semantic-action-catalog.json": "6b398dbf14c2cb239333321387e3978b0ef21b2c1c6a12882105d077eb0214a0",
|
|
"packages/paperclip-runner/generated/capability/semantic-tool-contracts.json": "c31cb3b21aa81c46847e6827865bc17d100a2f135822b7580bec7fc1893454a2",
|
|
"packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json": "b4e488f922dd328a3967b2560951c2377f4c1411fb2c4c9eef7a44195a8f7514",
|
|
"packages/paperclip-runner/protocol/manifest.json": "9cebc584dce7b42ed6800bef9b3e1c83880a67798a7bdceb9d9482b246f66a18",
|
|
"packages/paperclip-runner/src/backends/codex-native-backend.ts": "affecc515a623e0dfeb338f553ea53ebb7d18baa3d13e4395d17b21000dbee93",
|
|
"packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts": "ec1cd94c17be22d0556d4586fa6407b28e2309ed7c2437d95a32beeb0755cc4f",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.ts": "d2360cdaa63902cbf0c6a6109da452ba7e0b2a5aaa76ee5cab9c63b4bf12e584",
|
|
"server/src/services/native-runtime/native-session-resume.ts": "5b27465e1a328412770fc27c4bc637e88b62022adc6a67709bb74187eadc465b",
|
|
"server/src/__tests__/native-procedure-measurement.test.ts": "4e5725c0d86442c0214d96e6ab489547b919d45f8f237d9ec14d02437965268d"
|
|
},
|
|
"boundary": "scripted runnerd RPC with production standard-mode authority, API tools enabled, local workspace, no assigned external apps",
|
|
"byteAccounting": "utf8Bytes counts the unmodified JSON serialization of each extracted component/projection; normalizedComparison substitutes only the fixture workspace path. Neither is the entire transport request.",
|
|
"providerCalls": 0,
|
|
"tokenCount": null,
|
|
"upstreamLoadingOrTruncation": "unverified",
|
|
"vendorStockPrompt": "unavailable",
|
|
"receipts": [
|
|
{
|
|
"provider": "codex",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "a336ca46d52cada364ad79d6072332acf4c3920a5f1f6d595ab5c8e1b2be6117",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48949,
|
|
"characters": 48939,
|
|
"sha256": "696a27c9d8dd92d5c22a94eca82f214786f7fb12a7a00712685990450d14fc6f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48781,
|
|
"characters": 48771,
|
|
"sha256": "1678ae17dcdfedd17de242b5fce9a27d287bfd9c4203611678043a99342a7754"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "a336ca46d52cada364ad79d6072332acf4c3920a5f1f6d595ab5c8e1b2be6117",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48949,
|
|
"characters": 48939,
|
|
"sha256": "696a27c9d8dd92d5c22a94eca82f214786f7fb12a7a00712685990450d14fc6f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48781,
|
|
"characters": 48771,
|
|
"sha256": "1678ae17dcdfedd17de242b5fce9a27d287bfd9c4203611678043a99342a7754"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47087,
|
|
"characters": 47077,
|
|
"sha256": "bb7057ad43526b62652d752d504b41b9113254ebae72d461859ea12080905ca2",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 47003,
|
|
"characters": 46993,
|
|
"sha256": "9888af76927545d1e986c9f810975e6a1c4c3e395d0b25550e74d7f75146397c"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "a336ca46d52cada364ad79d6072332acf4c3920a5f1f6d595ab5c8e1b2be6117",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48949,
|
|
"characters": 48939,
|
|
"sha256": "696a27c9d8dd92d5c22a94eca82f214786f7fb12a7a00712685990450d14fc6f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48781,
|
|
"characters": 48771,
|
|
"sha256": "1678ae17dcdfedd17de242b5fce9a27d287bfd9c4203611678043a99342a7754"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "a336ca46d52cada364ad79d6072332acf4c3920a5f1f6d595ab5c8e1b2be6117",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48949,
|
|
"characters": 48939,
|
|
"sha256": "696a27c9d8dd92d5c22a94eca82f214786f7fb12a7a00712685990450d14fc6f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48781,
|
|
"characters": 48771,
|
|
"sha256": "1678ae17dcdfedd17de242b5fce9a27d287bfd9c4203611678043a99342a7754"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47087,
|
|
"characters": 47077,
|
|
"sha256": "bb7057ad43526b62652d752d504b41b9113254ebae72d461859ea12080905ca2",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 47003,
|
|
"characters": 46993,
|
|
"sha256": "9888af76927545d1e986c9f810975e6a1c4c3e395d0b25550e74d7f75146397c"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "a336ca46d52cada364ad79d6072332acf4c3920a5f1f6d595ab5c8e1b2be6117",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48949,
|
|
"characters": 48939,
|
|
"sha256": "696a27c9d8dd92d5c22a94eca82f214786f7fb12a7a00712685990450d14fc6f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48781,
|
|
"characters": 48771,
|
|
"sha256": "1678ae17dcdfedd17de242b5fce9a27d287bfd9c4203611678043a99342a7754"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "a336ca46d52cada364ad79d6072332acf4c3920a5f1f6d595ab5c8e1b2be6117",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48949,
|
|
"characters": 48939,
|
|
"sha256": "696a27c9d8dd92d5c22a94eca82f214786f7fb12a7a00712685990450d14fc6f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48781,
|
|
"characters": 48771,
|
|
"sha256": "1678ae17dcdfedd17de242b5fce9a27d287bfd9c4203611678043a99342a7754"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "f89d1d862203db2f03023c0144139925f6784c0fbeae1ac0c2957962c4d60cc4",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44944,
|
|
"characters": 44934,
|
|
"sha256": "c8f98376b8e7e5c7f92bce1c0d36ca2fb82fe0e2bd398a6b2750a9a25043f293"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47087,
|
|
"characters": 47077,
|
|
"sha256": "bb7057ad43526b62652d752d504b41b9113254ebae72d461859ea12080905ca2",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 47003,
|
|
"characters": 46993,
|
|
"sha256": "9888af76927545d1e986c9f810975e6a1c4c3e395d0b25550e74d7f75146397c"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
}
|
|
],
|
|
"mcpReceipt": {
|
|
"boundary": "authenticated loopback MCP tools/list; no OpenCode/provider process",
|
|
"serverAnnotationsForwarded": false,
|
|
"tools": {
|
|
"utf8Bytes": 44249,
|
|
"characters": 44239,
|
|
"sha256": "07d02ed46f9055af77756edc004dd3b30ec796afbbabe21623709f810d7ba1d1",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44249,
|
|
"characters": 44239,
|
|
"sha256": "07d02ed46f9055af77756edc004dd3b30ec796afbbabe21623709f810d7ba1d1"
|
|
}
|
|
},
|
|
"count": 39
|
|
},
|
|
"originalReceiptSha256": "2158cc185b2d7c5dd6082737836a7d1c5d3e20bf4539a897e41ed4e99080aadd"
|
|
},
|
|
"corrected-candidate": {
|
|
"schema": "paperclip.native-procedure-measurement.v2",
|
|
"sourceSha": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"sourceDirty": true,
|
|
"measuredInputsDirty": false,
|
|
"sourceHashes": {
|
|
"server/src/onboarding-assets/default/AGENTS.md": "c6318bf7e425cc0dd7acd4ff2905dab5c46ffddedef979afdf06720c82f6e95a",
|
|
"packages/paperclip-runner/src/contracts/runtime-context.ts": "3d2ee39cf7cc2210104e155213f04cec82981f76cd7c38384cdc11fa3ce4b013",
|
|
"packages/paperclip-runner/src/protocol-actions/hire-agent.ts": "ed8d270e2f3a8357bde5f2b180616d09f45218cd90bad22609d2e2e3d02d0319",
|
|
"packages/paperclip-runner/src/protocol-actions/create-task.ts": "8557d4030c97aaf99e8ef45dc608e4e0117c764eb459b221ce6739499d79b8aa",
|
|
"packages/paperclip-runner/src/protocol-actions/set-dependencies.ts": "89819f6741dc14009f2daf54717f5e23d583c6f5685c30459fe8a0d672a2720b",
|
|
"packages/paperclip-runner/src/catalog/semantic-action-catalog.ts": "786b597ab2199f3d943e499911fb58cfd014e78ff60923b9f44b9e2d80d73d4d",
|
|
"packages/paperclip-runner/generated/semantic-action-catalog.json": "795c5c8b98b1d7d60a2097760ffcb5a72ffa27713ca75b086ec741bbb0260f3a",
|
|
"packages/paperclip-runner/generated/capability/semantic-tool-contracts.json": "383ca779a4a4e98b9cf739ccfbfd072e470b57d8a340dbf3e86c7d502285f5ba",
|
|
"packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json": "044d8532b9a51d5ad254618a32f2d9d39227b70110d916e1a05b8072f4ee4e22",
|
|
"packages/paperclip-runner/protocol/manifest.json": "420055c0f98b46f57096ac32bdfb6a0d53ae40ac2c3345dc1fea7cfdab2addb0",
|
|
"packages/paperclip-runner/src/backends/codex-native-backend.ts": "affecc515a623e0dfeb338f553ea53ebb7d18baa3d13e4395d17b21000dbee93",
|
|
"packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts": "ec1cd94c17be22d0556d4586fa6407b28e2309ed7c2437d95a32beeb0755cc4f",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.ts": "ae5cd9c6d076cf44471b42c583f69eea9776aaad41ccbc8319d416aa88f78b44",
|
|
"server/src/services/native-runtime/native-session-resume.ts": "d9d4d7fde805c150e8278fd26f36d5e27d77c49690a0371f20e83b542f9cb2db",
|
|
"server/src/__tests__/native-procedure-measurement.test.ts": "4e5725c0d86442c0214d96e6ab489547b919d45f8f237d9ec14d02437965268d"
|
|
},
|
|
"boundary": "scripted runnerd RPC with production standard-mode authority, API tools enabled, local workspace, no assigned external apps",
|
|
"byteAccounting": "utf8Bytes counts the unmodified JSON serialization of each extracted component/projection; normalizedComparison substitutes only the fixture workspace path. Neither is the entire transport request.",
|
|
"providerCalls": 0,
|
|
"tokenCount": null,
|
|
"upstreamLoadingOrTruncation": "unverified",
|
|
"vendorStockPrompt": "unavailable",
|
|
"receipts": [
|
|
{
|
|
"provider": "codex",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "28e674540ba45792d6a31a1acef0a5b99fe8aa60c66df2d4aeda12dbe7bba3b5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48489,
|
|
"characters": 48479,
|
|
"sha256": "911dace61347be9b8515ffd249ca84bfb67df487c5a1a2cecebe6c5b822f34ce",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48321,
|
|
"characters": 48311,
|
|
"sha256": "31bc2475becb447e0f2722ad61a32938693258cca07c3b1e3f7162b8be0d8319"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "28e674540ba45792d6a31a1acef0a5b99fe8aa60c66df2d4aeda12dbe7bba3b5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48489,
|
|
"characters": 48479,
|
|
"sha256": "911dace61347be9b8515ffd249ca84bfb67df487c5a1a2cecebe6c5b822f34ce",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48321,
|
|
"characters": 48311,
|
|
"sha256": "31bc2475becb447e0f2722ad61a32938693258cca07c3b1e3f7162b8be0d8319"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 46627,
|
|
"characters": 46617,
|
|
"sha256": "8e8e003648345a706d3eb28b3e0762e8accf51291101eed78be98b0e876cd445",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 46543,
|
|
"characters": 46533,
|
|
"sha256": "20b7f03d24078f192aa8935c269f63e06b50dc2a7176f417d64ca8c1ace92591"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "28e674540ba45792d6a31a1acef0a5b99fe8aa60c66df2d4aeda12dbe7bba3b5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48489,
|
|
"characters": 48479,
|
|
"sha256": "911dace61347be9b8515ffd249ca84bfb67df487c5a1a2cecebe6c5b822f34ce",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48321,
|
|
"characters": 48311,
|
|
"sha256": "31bc2475becb447e0f2722ad61a32938693258cca07c3b1e3f7162b8be0d8319"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "28e674540ba45792d6a31a1acef0a5b99fe8aa60c66df2d4aeda12dbe7bba3b5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48489,
|
|
"characters": 48479,
|
|
"sha256": "911dace61347be9b8515ffd249ca84bfb67df487c5a1a2cecebe6c5b822f34ce",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48321,
|
|
"characters": 48311,
|
|
"sha256": "31bc2475becb447e0f2722ad61a32938693258cca07c3b1e3f7162b8be0d8319"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 46627,
|
|
"characters": 46617,
|
|
"sha256": "8e8e003648345a706d3eb28b3e0762e8accf51291101eed78be98b0e876cd445",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 46543,
|
|
"characters": 46533,
|
|
"sha256": "20b7f03d24078f192aa8935c269f63e06b50dc2a7176f417d64ca8c1ace92591"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "28e674540ba45792d6a31a1acef0a5b99fe8aa60c66df2d4aeda12dbe7bba3b5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48489,
|
|
"characters": 48479,
|
|
"sha256": "911dace61347be9b8515ffd249ca84bfb67df487c5a1a2cecebe6c5b822f34ce",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48321,
|
|
"characters": 48311,
|
|
"sha256": "31bc2475becb447e0f2722ad61a32938693258cca07c3b1e3f7162b8be0d8319"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "28e674540ba45792d6a31a1acef0a5b99fe8aa60c66df2d4aeda12dbe7bba3b5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48489,
|
|
"characters": 48479,
|
|
"sha256": "911dace61347be9b8515ffd249ca84bfb67df487c5a1a2cecebe6c5b822f34ce",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48321,
|
|
"characters": 48311,
|
|
"sha256": "31bc2475becb447e0f2722ad61a32938693258cca07c3b1e3f7162b8be0d8319"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "a415c39912f9b7c8a15b24dc27702229e2458adcef143511d77d67bd125d854b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45037,
|
|
"characters": 45027,
|
|
"sha256": "6830d63d19141fa7efcc7b53f0f6f7e044352d006c3e146ddc0073d502646e5c"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 46627,
|
|
"characters": 46617,
|
|
"sha256": "8e8e003648345a706d3eb28b3e0762e8accf51291101eed78be98b0e876cd445",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 46543,
|
|
"characters": 46533,
|
|
"sha256": "20b7f03d24078f192aa8935c269f63e06b50dc2a7176f417d64ca8c1ace92591"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
}
|
|
],
|
|
"mcpReceipt": {
|
|
"boundary": "authenticated loopback MCP tools/list; no OpenCode/provider process",
|
|
"serverAnnotationsForwarded": false,
|
|
"tools": {
|
|
"utf8Bytes": 44342,
|
|
"characters": 44332,
|
|
"sha256": "4241b7d6404f286c693d8760fb41d9dfdf22170271bdf49a0674cc6717fe689b",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44342,
|
|
"characters": 44332,
|
|
"sha256": "4241b7d6404f286c693d8760fb41d9dfdf22170271bdf49a0674cc6717fe689b"
|
|
}
|
|
},
|
|
"count": 39
|
|
},
|
|
"originalReceiptSha256": "8486d4c2897ceca5124500899e83eb9754230e80b992c971e586354253ec547e"
|
|
},
|
|
"integrated-baseline": {
|
|
"schema": "paperclip.native-procedure-measurement.v2",
|
|
"sourceSha": "ce31f5efd0d416205c3994c55dd02b645f8b8423",
|
|
"sourceDirty": false,
|
|
"measuredInputsDirty": false,
|
|
"sourceHashes": {
|
|
"server/src/onboarding-assets/default/AGENTS.md": "c6318bf7e425cc0dd7acd4ff2905dab5c46ffddedef979afdf06720c82f6e95a",
|
|
"packages/paperclip-runner/src/contracts/runtime-context.ts": "a17e30a69f3cb38fba00c37e1a06753d723651a004e6050c9e77ae202a61264b",
|
|
"packages/paperclip-runner/src/protocol-actions/hire-agent.ts": "3b786405fa48e6ba9cf63789c03bbbb370fc2962a4a5c8bd80ccca57759bf912",
|
|
"packages/paperclip-runner/src/protocol-actions/create-task.ts": "4efe71c1693dd65d8087b2fdbaf5831b397130d036afb837ece20d4791631805",
|
|
"packages/paperclip-runner/src/protocol-actions/set-dependencies.ts": "53c65776ce4984f177d261c1e53957a97e1a236ce58e731119cb8ee26f141da2",
|
|
"packages/paperclip-runner/src/catalog/semantic-action-catalog.ts": "b8f723ef39f31218d143e284ce6d379d3ba252d4f3086ce05c20db9daf56f848",
|
|
"packages/paperclip-runner/generated/semantic-action-catalog.json": "d287caff473a61800135866148f3a2a38ea46cf532343a390dfbf2557f6adc08",
|
|
"packages/paperclip-runner/generated/capability/semantic-tool-contracts.json": "3d490614baafe423d77d1ad6d46504d47c79373b33ed639614a5f67e9d9bde93",
|
|
"packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json": "a53e3d8eae39899111ff1a0110135a9cfafdd12f7843cf6872a3de8fae915a76",
|
|
"packages/paperclip-runner/protocol/manifest.json": "9cd783f47dd2b950a89404421a3e76446249693e61c934a91d6f4e529ec18f9b",
|
|
"packages/paperclip-runner/src/backends/codex-native-backend.ts": "affecc515a623e0dfeb338f553ea53ebb7d18baa3d13e4395d17b21000dbee93",
|
|
"packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts": "ec1cd94c17be22d0556d4586fa6407b28e2309ed7c2437d95a32beeb0755cc4f",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.ts": "d2360cdaa63902cbf0c6a6109da452ba7e0b2a5aaa76ee5cab9c63b4bf12e584",
|
|
"server/src/services/native-runtime/native-session-resume.ts": "5b27465e1a328412770fc27c4bc637e88b62022adc6a67709bb74187eadc465b",
|
|
"server/src/__tests__/native-procedure-measurement.test.ts": "4e5725c0d86442c0214d96e6ab489547b919d45f8f237d9ec14d02437965268d"
|
|
},
|
|
"boundary": "scripted runnerd RPC with production standard-mode authority, API tools enabled, local workspace, no assigned external apps",
|
|
"byteAccounting": "utf8Bytes counts the unmodified JSON serialization of each extracted component/projection; normalizedComparison substitutes only the fixture workspace path. Neither is the entire transport request.",
|
|
"providerCalls": 0,
|
|
"tokenCount": null,
|
|
"upstreamLoadingOrTruncation": "unverified",
|
|
"vendorStockPrompt": "unavailable",
|
|
"receipts": [
|
|
{
|
|
"provider": "codex",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "3c83a00a3403aa8b20b47ecb9320b1006e0e823116e7947743c4adc6b3ec5803",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 49368,
|
|
"characters": 49358,
|
|
"sha256": "b89d8d78a06fb98b17022a1f36644c2b309fc8bbab9bf82ddc9a547d77cfa479",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 49200,
|
|
"characters": 49190,
|
|
"sha256": "721d0f637736336c3c8a470b6c80bf40208b09526076c565a98cab364324b63f"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "3c83a00a3403aa8b20b47ecb9320b1006e0e823116e7947743c4adc6b3ec5803",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 49368,
|
|
"characters": 49358,
|
|
"sha256": "b89d8d78a06fb98b17022a1f36644c2b309fc8bbab9bf82ddc9a547d77cfa479",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 49200,
|
|
"characters": 49190,
|
|
"sha256": "721d0f637736336c3c8a470b6c80bf40208b09526076c565a98cab364324b63f"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47506,
|
|
"characters": 47496,
|
|
"sha256": "73d1774b566e6aaeca24a60a05410362609c7547c94413809bbb2cefe687477f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 47422,
|
|
"characters": 47412,
|
|
"sha256": "bdb47770fd8f092e6f34ecb85768ed24f1a8613962ea3461252c60911dba5ed6"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "3c83a00a3403aa8b20b47ecb9320b1006e0e823116e7947743c4adc6b3ec5803",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 49368,
|
|
"characters": 49358,
|
|
"sha256": "b89d8d78a06fb98b17022a1f36644c2b309fc8bbab9bf82ddc9a547d77cfa479",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 49200,
|
|
"characters": 49190,
|
|
"sha256": "721d0f637736336c3c8a470b6c80bf40208b09526076c565a98cab364324b63f"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "3c83a00a3403aa8b20b47ecb9320b1006e0e823116e7947743c4adc6b3ec5803",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 49368,
|
|
"characters": 49358,
|
|
"sha256": "b89d8d78a06fb98b17022a1f36644c2b309fc8bbab9bf82ddc9a547d77cfa479",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 49200,
|
|
"characters": 49190,
|
|
"sha256": "721d0f637736336c3c8a470b6c80bf40208b09526076c565a98cab364324b63f"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47506,
|
|
"characters": 47496,
|
|
"sha256": "73d1774b566e6aaeca24a60a05410362609c7547c94413809bbb2cefe687477f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 47422,
|
|
"characters": 47412,
|
|
"sha256": "bdb47770fd8f092e6f34ecb85768ed24f1a8613962ea3461252c60911dba5ed6"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "3c83a00a3403aa8b20b47ecb9320b1006e0e823116e7947743c4adc6b3ec5803",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 49368,
|
|
"characters": 49358,
|
|
"sha256": "b89d8d78a06fb98b17022a1f36644c2b309fc8bbab9bf82ddc9a547d77cfa479",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 49200,
|
|
"characters": 49190,
|
|
"sha256": "721d0f637736336c3c8a470b6c80bf40208b09526076c565a98cab364324b63f"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "3c83a00a3403aa8b20b47ecb9320b1006e0e823116e7947743c4adc6b3ec5803",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 49368,
|
|
"characters": 49358,
|
|
"sha256": "b89d8d78a06fb98b17022a1f36644c2b309fc8bbab9bf82ddc9a547d77cfa479",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 49200,
|
|
"characters": 49190,
|
|
"sha256": "721d0f637736336c3c8a470b6c80bf40208b09526076c565a98cab364324b63f"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1893,
|
|
"characters": 1893,
|
|
"sha256": "fd2c444ee5d225b4ec24fc46cb66b4c1e11e1f86745b50a297ce94da152b0be5",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1809,
|
|
"characters": 1809,
|
|
"sha256": "d3618f4e545ed8279a6d71e0f0e214bb713786d91df9a57a2359c8ee076df80a"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45363,
|
|
"characters": 45353,
|
|
"sha256": "ab3c8469ca02e07bd99d0a475ac7c10d8bff5b12da3a788896f4f79c675248cd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47506,
|
|
"characters": 47496,
|
|
"sha256": "73d1774b566e6aaeca24a60a05410362609c7547c94413809bbb2cefe687477f",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 47422,
|
|
"characters": 47412,
|
|
"sha256": "bdb47770fd8f092e6f34ecb85768ed24f1a8613962ea3461252c60911dba5ed6"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
}
|
|
],
|
|
"mcpReceipt": {
|
|
"boundary": "authenticated loopback MCP tools/list; no OpenCode/provider process",
|
|
"serverAnnotationsForwarded": false,
|
|
"tools": {
|
|
"utf8Bytes": 44668,
|
|
"characters": 44658,
|
|
"sha256": "9c6e59816c0a49d675a4afd18f6d05a9d3315da5af803519373bf19fd81ea8dd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44668,
|
|
"characters": 44658,
|
|
"sha256": "9c6e59816c0a49d675a4afd18f6d05a9d3315da5af803519373bf19fd81ea8dd"
|
|
}
|
|
},
|
|
"count": 39
|
|
},
|
|
"originalReceiptSha256": "356693e0c75a86b7e53b397822399b0f4a09652b3d88be280b7a4569fd40efb6"
|
|
},
|
|
"integrated-candidate": {
|
|
"schema": "paperclip.native-procedure-measurement.v2",
|
|
"sourceSha": "ee096aa23fc88a001e7666215bcd338b5863f377",
|
|
"sourceDirty": true,
|
|
"measuredInputsDirty": false,
|
|
"sourceHashes": {
|
|
"server/src/onboarding-assets/default/AGENTS.md": "c6318bf7e425cc0dd7acd4ff2905dab5c46ffddedef979afdf06720c82f6e95a",
|
|
"packages/paperclip-runner/src/contracts/runtime-context.ts": "3d2ee39cf7cc2210104e155213f04cec82981f76cd7c38384cdc11fa3ce4b013",
|
|
"packages/paperclip-runner/src/protocol-actions/hire-agent.ts": "ed8d270e2f3a8357bde5f2b180616d09f45218cd90bad22609d2e2e3d02d0319",
|
|
"packages/paperclip-runner/src/protocol-actions/create-task.ts": "8557d4030c97aaf99e8ef45dc608e4e0117c764eb459b221ce6739499d79b8aa",
|
|
"packages/paperclip-runner/src/protocol-actions/set-dependencies.ts": "89819f6741dc14009f2daf54717f5e23d583c6f5685c30459fe8a0d672a2720b",
|
|
"packages/paperclip-runner/src/catalog/semantic-action-catalog.ts": "315674491b8e2656b5d9ad77ed41cafd9647f356ed4c46f0dc2de08eb82b646d",
|
|
"packages/paperclip-runner/generated/semantic-action-catalog.json": "7238896278a78e307b7eeae1b02d86d785694d10cfdf8dbb4402f1f756e9ebf5",
|
|
"packages/paperclip-runner/generated/capability/semantic-tool-contracts.json": "4a641cd79e9a75b8247f4e756b6ef0711a76348816b17eb267d284832ed59d37",
|
|
"packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json": "1ef1b62506d7c044d68dd9ed8033bd7901f8dba47aea70188c85878d448f6a0a",
|
|
"packages/paperclip-runner/protocol/manifest.json": "4c6e1bd289db4be0d669fe79439d7adf3c1a56fc1c762e663299bcd431557597",
|
|
"packages/paperclip-runner/src/backends/codex-native-backend.ts": "affecc515a623e0dfeb338f553ea53ebb7d18baa3d13e4395d17b21000dbee93",
|
|
"packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts": "ec1cd94c17be22d0556d4586fa6407b28e2309ed7c2437d95a32beeb0755cc4f",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.ts": "ae5cd9c6d076cf44471b42c583f69eea9776aaad41ccbc8319d416aa88f78b44",
|
|
"server/src/services/native-runtime/native-session-resume.ts": "d9d4d7fde805c150e8278fd26f36d5e27d77c49690a0371f20e83b542f9cb2db",
|
|
"server/src/__tests__/native-procedure-measurement.test.ts": "4e5725c0d86442c0214d96e6ab489547b919d45f8f237d9ec14d02437965268d"
|
|
},
|
|
"boundary": "scripted runnerd RPC with production standard-mode authority, API tools enabled, local workspace, no assigned external apps",
|
|
"byteAccounting": "utf8Bytes counts the unmodified JSON serialization of each extracted component/projection; normalizedComparison substitutes only the fixture workspace path. Neither is the entire transport request.",
|
|
"providerCalls": 0,
|
|
"tokenCount": null,
|
|
"upstreamLoadingOrTruncation": "unverified",
|
|
"vendorStockPrompt": "unavailable",
|
|
"receipts": [
|
|
{
|
|
"provider": "codex",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "20af9131c3b9db8aef690c86b254aab8595f0abac36bb4e2253dd3ab59a6cb25",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48908,
|
|
"characters": 48898,
|
|
"sha256": "5137d332b7cde806c4276c725dbdb3cb558061517f5fb38f9e9140427f3fe69a",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48740,
|
|
"characters": 48730,
|
|
"sha256": "120a6211bc0b3a5e141e33e8ac21ee22948940f7698d75f60b6713829ebaac22"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "20af9131c3b9db8aef690c86b254aab8595f0abac36bb4e2253dd3ab59a6cb25",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48908,
|
|
"characters": 48898,
|
|
"sha256": "5137d332b7cde806c4276c725dbdb3cb558061517f5fb38f9e9140427f3fe69a",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48740,
|
|
"characters": 48730,
|
|
"sha256": "120a6211bc0b3a5e141e33e8ac21ee22948940f7698d75f60b6713829ebaac22"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "codex",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47046,
|
|
"characters": 47036,
|
|
"sha256": "5defafb88865b1d19aa776323faab1081d84e92c9e832d6f4f5980e3c3d70a06",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 46962,
|
|
"characters": 46952,
|
|
"sha256": "5729e7faaff14fa6de4211022fb8f43fab471dfd2d9eec2c8c950ee56bd55976"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "20af9131c3b9db8aef690c86b254aab8595f0abac36bb4e2253dd3ab59a6cb25",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48908,
|
|
"characters": 48898,
|
|
"sha256": "5137d332b7cde806c4276c725dbdb3cb558061517f5fb38f9e9140427f3fe69a",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48740,
|
|
"characters": 48730,
|
|
"sha256": "120a6211bc0b3a5e141e33e8ac21ee22948940f7698d75f60b6713829ebaac22"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "20af9131c3b9db8aef690c86b254aab8595f0abac36bb4e2253dd3ab59a6cb25",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48908,
|
|
"characters": 48898,
|
|
"sha256": "5137d332b7cde806c4276c725dbdb3cb558061517f5fb38f9e9140427f3fe69a",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48740,
|
|
"characters": 48730,
|
|
"sha256": "120a6211bc0b3a5e141e33e8ac21ee22948940f7698d75f60b6713829ebaac22"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "acpx",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47046,
|
|
"characters": 47036,
|
|
"sha256": "5defafb88865b1d19aa776323faab1081d84e92c9e832d6f4f5980e3c3d70a06",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 46962,
|
|
"characters": 46952,
|
|
"sha256": "5729e7faaff14fa6de4211022fb8f43fab471dfd2d9eec2c8c950ee56bd55976"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "start",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "20af9131c3b9db8aef690c86b254aab8595f0abac36bb4e2253dd3ab59a6cb25",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48908,
|
|
"characters": 48898,
|
|
"sha256": "5137d332b7cde806c4276c725dbdb3cb558061517f5fb38f9e9140427f3fe69a",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48740,
|
|
"characters": 48730,
|
|
"sha256": "120a6211bc0b3a5e141e33e8ac21ee22948940f7698d75f60b6713829ebaac22"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "resume",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 2077,
|
|
"characters": 2077,
|
|
"sha256": "20af9131c3b9db8aef690c86b254aab8595f0abac36bb4e2253dd3ab59a6cb25",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1993,
|
|
"characters": 1993,
|
|
"sha256": "f4a263fa69d0ca3a5ed13095b76da6b5d588540f8777708ee9e08104d066f2ff"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 48908,
|
|
"characters": 48898,
|
|
"sha256": "5137d332b7cde806c4276c725dbdb3cb558061517f5fb38f9e9140427f3fe69a",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 48740,
|
|
"characters": 48730,
|
|
"sha256": "120a6211bc0b3a5e141e33e8ac21ee22948940f7698d75f60b6713829ebaac22"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
},
|
|
{
|
|
"provider": "opencode",
|
|
"phase": "continuation",
|
|
"instructions": {
|
|
"utf8Bytes": 1340,
|
|
"characters": 1340,
|
|
"sha256": "32cebc711830c7db05593c29c6ff59b7e643a0bf0d9434ae51b779bdf3ee18ec",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 1256,
|
|
"characters": 1256,
|
|
"sha256": "1a623b330bb5ae167dacc32a5ea5d714c716ec1ef7f7e3be78a7bc8033fd42be"
|
|
}
|
|
},
|
|
"tools": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 45456,
|
|
"characters": 45446,
|
|
"sha256": "8f93bd4543a64025b161fe52a98f32491e023379b3216be0ced70cd4a72be5fd"
|
|
}
|
|
},
|
|
"input": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 215,
|
|
"characters": 215,
|
|
"sha256": "a85a9583b37a072e1707791482d5aa2a5e2cc1e4649b26e009a7abcc488643f0"
|
|
}
|
|
},
|
|
"fullProjection": {
|
|
"utf8Bytes": 47046,
|
|
"characters": 47036,
|
|
"sha256": "5defafb88865b1d19aa776323faab1081d84e92c9e832d6f4f5980e3c3d70a06",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 46962,
|
|
"characters": 46952,
|
|
"sha256": "5729e7faaff14fa6de4211022fb8f43fab471dfd2d9eec2c8c950ee56bd55976"
|
|
}
|
|
},
|
|
"deliveredToolCount": 39
|
|
}
|
|
],
|
|
"mcpReceipt": {
|
|
"boundary": "authenticated loopback MCP tools/list; no OpenCode/provider process",
|
|
"serverAnnotationsForwarded": false,
|
|
"tools": {
|
|
"utf8Bytes": 44761,
|
|
"characters": 44751,
|
|
"sha256": "ddcc592d0acc330d6fe5783473b7e35a698ef6d4030b0311dee563b9d60aeda2",
|
|
"normalizedComparison": {
|
|
"utf8Bytes": 44761,
|
|
"characters": 44751,
|
|
"sha256": "ddcc592d0acc330d6fe5783473b7e35a698ef6d4030b0311dee563b9d60aeda2"
|
|
}
|
|
},
|
|
"count": 39
|
|
},
|
|
"originalReceiptSha256": "63348f5a79226c2b24991812f9331c671c3559812852ef69d7d1826ee309a8cb"
|
|
}
|
|
},
|
|
"integration": {
|
|
"integratedHead": "ee096aa23fc88a001e7666215bcd338b5863f377",
|
|
"paidCandidate": "ffe847fabaebbf24bc93924a4b6c8922d46a1fb6",
|
|
"master": "a65ca0950834a85bb93bcc4b4042ecacdebfef53",
|
|
"paidSourcesUnchanged": true,
|
|
"resolution": "Regenerated three conflicted contract files from combined source; retained master project discovery and finalization changes.",
|
|
"liveOnIntegratedHead": false,
|
|
"integratedBaseline": "ce31f5efd0d416205c3994c55dd02b645f8b8423",
|
|
"intendedSourceDeltaIdentical": true,
|
|
"commonFixtureFilesChecked": 256,
|
|
"sourceDelta": [
|
|
"packages/paperclip-runner/generated/capability/semantic-tool-contracts.json",
|
|
"packages/paperclip-runner/generated/semantic-action-catalog.json",
|
|
"packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json",
|
|
"packages/paperclip-runner/protocol/manifest.json",
|
|
"packages/paperclip-runner/src/catalog/semantic-action-catalog.ts",
|
|
"packages/paperclip-runner/src/contracts/runtime-context.ts",
|
|
"packages/paperclip-runner/src/protocol-actions/create-task.ts",
|
|
"packages/paperclip-runner/src/protocol-actions/hire-agent.ts",
|
|
"packages/paperclip-runner/src/protocol-actions/set-dependencies.ts",
|
|
"server/src/services/native-runtime/native-session-resume.ts",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts",
|
|
"server/src/services/native-runtime/paperclip-runner-tool-authority.ts"
|
|
]
|
|
},
|
|
"limitations": [
|
|
"Single trials do not establish causation, general equivalence, coding quality, speed or cost trends.",
|
|
"All graded results retain their original measured sources. No paid result measured the integrated PR head.",
|
|
"Original setup used a company cap but left the lead budget at zero. Corrected setup applied both company and lead caps.",
|
|
"Unknown total activity and missing final result/ledger/cleanup for two interrupted jobs remain missing, not silently passed.",
|
|
"Configured model identity is not independent proof of provider-side execution identity.",
|
|
"ACPX provider-execution IDs are not proven mapped to host-call IDs. Never join by name, order or count.",
|
|
"Measurement is an extracted delivery projection; vendor private prompts, tokenization, loading and truncation remain unverified.",
|
|
"Numeric billing subtotals and raw sessions are excluded from this public summary; no complete billing or savings claim is made."
|
|
]
|
|
}
|