mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-05 21:02:34 +01:00
131fd66001ee718d0fef016d2cd1e977d752741a
171
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dotta
|
131fd66001 | Isolate goal purpose conditional context and grade business meaning | ||
|
Dotta
|
af18a43ca2 | Bind inbox summary grading to actual responsible-user access | ||
|
Dotta
|
8b27fe681d | Correct goal alignment grading to match catalog semantics | ||
|
Dotta
|
875371a262 | test(e2e): verify runtime surfaces and actual task workspace binding | ||
|
Dotta
|
f1762c5609 | Evaluate multi-page open task inventory and honest partial results | ||
|
Dotta
|
45b248d2f0 | test(e2e): cover runtime history and human primary controls | ||
|
Dotta
|
b5f0e6399f | Add authenticated inbox scope and denied archive evals | ||
|
Dotta
|
2574cbc18e | Evaluate private task owner sharing boundary with real observer | ||
|
Dotta
|
3a713fabbe | Add owner-driven project sharing eval and strengthen evidence controls | ||
|
Dotta
|
e652b0de6e | feat(evals): retain genuine pre-query chat history boundaries | ||
|
Dotta
|
5591141864 | test(e2e): add runtime agent and dependency cycle oracles | ||
|
Dotta
|
4c9de65edd | fix(evals): load source chat domains and retain canonical report IDs | ||
|
Dotta
|
ba144e8503 | test(evals): add external chat catalog checkpoint and provider prerequisites | ||
|
Dotta
|
2200f1c2e8 | test(evals): verify private projects with independent human and agent observers | ||
|
Dotta
|
10d0ce6361 | test(evals): add identity scenarios and separate browser boundary dispatch | ||
|
Dotta
|
0c67cb39be | test(evals): exercise delegated task creation and lost send recovery | ||
|
Dotta
|
6ddec3a40d | test(evals): bind goal outcomes to their actual chat phases | ||
|
Dotta
|
caba17c3f0 | test(evals): freeze complete goal chat journeys and calibration | ||
|
DottaandPaperclip
|
de73104b34 |
test: isolate chat case modules and fingerprint executable dependencies
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bc7b66912b |
test: establish frozen chat catalog campaign and executable case modules
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2ee36a4c0a |
test: add chat feature scenarios and real task workflow evals (#17794)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat lets a person assign work through a conversation. > - Feature and API maps do not prove that a natural request produces the right saved result. > - Product E2E already owns isolated fixtures, browser runs, evidence and cleanup. > - This pull request adds a chat scenario catalog and two real task workflows in that framework. > - Independent checks verify the task, its author, the saved output and the reply in chat. > - The benefit is a small executable foundation for broader chat coverage. ## Linked Issues or Issue Description **What existing behavior does this improve?** It adds real task creation and delegation coverage for Agent Chat. **Current behavior** Existing maps describe features and agent access. Existing chat evals cover specific protocols and handoffs. They do not connect a broad human-request catalog to independent saved-state checks. **Proposed behavior** Keep 244 planned scenarios across 35 feature areas. Promote two requests into the explicit-only `chat-features` suite. One saves a backlog task without starting work. The other delegates an announcement and returns the saved result to the original chat. **Reason and benefit** A successful tool call or confident reply does not prove the requested result exists. These evals check public API state, attribution and browser access after a real model run. **Breaking changes** No API or data schema changes. Chat instructions now require actual relative task links. The local E2E private-state scanner recognizes Codex account ownership metadata while continuing to reject credentials and model-visible identity leaks. **Additional context** Related prior work: #13969 and #13762. This improves verification of the existing Agent Chat roadmap item. It does not add a new core product feature. ## What Changed - Add YAML scenarios with capability IDs, starting context, user requests and expected outcomes. Validate all area files against their schema, global ID uniqueness and exact feature-map capability coverage. - Add two local native Codex subscription cells. Send the exact catalog request once through Chromium. - Seed only prerequisites through public APIs. Use shipped agent instructions and explicit disposable budget policies for unpriced subscription usage. - Check task ownership, creation audit, no premature execution, current worker-authored output, automatic completion delivery and usable links after reload. - Keep private Codex account ownership metadata distinct from bearer credentials. Scan rollout message content, all tokens and published evidence as before. - Teach normal chat instructions to use relative task links instead of invented hosts. Update tests, maps and the runbook. - Repair existing E2E contract drift: include the active Copilot identity in the image digest and update Pi profile expectations. Keep global catalog cardinality in its owning catalog test. Product E2E owns the new flow and grader. Its existing launch and report paths remain in use. The shared scanner change is limited to owner-private temporary Codex runtime metadata. Other callers retain the full secret scan. No UI, endpoint, schema or permission behavior changes. ## Verification - `node scripts/check-maps.mjs` passed: 35 areas and 191 capability IDs. - Product E2E typecheck and focused tests passed before rebase: 225 Vitest tests and 137 Node tests. Conversation service tests passed 40/40. Five existing runner turn-binding regressions passed. - Historical live measurement used real Chromium, server, database and native Codex with no automatic retries. The initial same-definition batch passed 5/6 cells. Backlog passed 3/3; delegation passed 2/3. One CEO startup failed because the runner did not receive a provider turn identity. The cause is unresolved. - An unchanged follow-up passed delegation three times and backlog twice. Combined evidence is backlog 5/5 and delegation 5/6. Each attempt passed cleanup. Later passes do not erase the earlier failure or prove a runner repair. - Earlier failed revisions exposed an invented task hostname, private account metadata false positives and missing funding for unpriced subscription usage. Those led to the specific changes above; the failed records remain retained locally. - The runbook records campaign identifiers and definition provenance. Generated result artifacts are not committed. Historical live results predate this branch rebase; fresh PR-source live evidence and final validation are recorded below. - Full workspace typecheck and production build passed on the rebased branch. Product E2E typecheck passed. - The full Product E2E support suite first reproduced four baseline failures: two missing active Copilot image-identity inputs and two stale Pi/catalog expectations. The narrow contract fixes above resolved them. The full rerun passed 4,282 Vitest tests plus 137 Node tests, with two skipped. Image checks verify mutation changes the hash, unchanged input stays stable, and an input closure without Copilot is unaffected. Original failed logs remain retained. - The repo-wide local Vitest run has reported two failures in unchanged service files. The Workspace connection test passed an isolated recheck and its full file passed 469/469 unchanged. The recovery test hit a 15-second timeout; its full file passed 89/89 unchanged. The run later hit explicit PostgreSQL `semget` resource exhaustion and was interrupted after those observed failures; no final aggregate counts or full local pass are claimed. Its complete available log is retained. No assertion or timeout was weakened. Matching remote server CI passed on the same source. - On clean PR head `39ad80b523ce194c90aafafd077d48dd9a36b4c5`, backlog passed in `local-2026-10-26T19-24-13-014Z`. The parallel delegation fixture failed during PostgreSQL initdb, before browser or model execution; its failed cleanup grade is retained. A disposable standalone initdb probe then passed. No semaphore sets or kernel settings were changed before that probe/recheck. After confirming no remaining fixture process, its private temporary files were removed. - One separate delegation recheck passed in `local-2026-10-26T19-29-24-800Z`, with one attempt, zero automatic retries and successful cleanup. Both passing current-source cells use definition `3f2c9ba2d03e3dcddde47ddab3b43bc52de977f25e256fceae0a24c9128e204b`. The earlier bootstrap failure and the historical provider startup failure remain distinct. These local results omit automatic source-SHA metadata; the clean checkout SHA is recorded here from the execution session. - The later broad local run reported `could not create semaphores: No space left on device`. A bounded cleanup removed 32 current-user PostgreSQL sets with 17 members, marker 537, a dead recorded creator and zero waiters. No live process or kernel limit was changed by that cleanup. The test process group was subsequently stopped; no broad rerun is planned. This direct later resource failure does not prove the precise cause of the earlier live initdb failure. - Review follow-up adds only static catalog validation and documentation of the existing five-minute journey bound. The executable definition is unchanged. Focused catalog/oracle tests passed 28/28; no redundant live run is claimed. - Final head `30fcb36ac9ae1f0270a589dbfafcec91ca6a03f3` passed all required and underlying CI checks in [run 40391402347](https://github.com/paperclipai/paperclip/actions/runs/40391402347), including `ci / verify` and `ci / e2e`. Greptile and Paperclip Review both scored this head 5/5, with all review threads resolved. The static follow-up leaves every executable definition input byte-identical to the live-tested source at `39ad80b`. Normal squash auto-merge is enabled. Actual merge is pending eligible code-owner approval for the owned dependency manifest; no protection is bypassed. ## Risks - Live model behavior can vary. One missing turn-start event remains unexplained; this PR does not claim provider reliability or broad catalog qualification. - Subscription costs are unpriced. Explicit fixture funding admits that usage. Run counts and deadlines remain bounded, but the dollar limit cannot cap unknown cost. - Account identity exceptions apply only to known private Codex runtime files. Token reuse, unknown files and model-visible rollout content remain blocking findings. - Deterministic checks establish saved state, output structure, required brief facts and access. They do not score all prose quality. Initial samples also require human inspection. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, browser and repository tools. The precise model revision and context window are not exposed to this session. Live eval agents used `gpt-5.6-sol`; that is the evaluated model, not a claim about the authoring model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run local tests; focused suites pass and the interrupted full-run failures are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
313c1db2c3 |
fix(evals): share bounded request policy with qualification (#17765)
## Thinking Path > - Paperclip manages agents and their work. > - Product E2E evaluations send model work through a gateway with cost limits. > - The gateway checks the request body before it reserves money or contacts a provider. > - A qualification probe needs that same check before it records a spending-cap denial. > - Namespace support already exists. Qualification must share that policy without sending a paid request. > - This pull request exposes the pure request validator and bounds the supported local tool definitions. > - The benefit is one request policy for qualification and forwarding. ## Linked Issues or Issue Description Refs: #17452, #17469. **What happened?** The request validator was private to `forwardEvalRequest`. A qualification caller could not reuse the request policy before admission. A separate check could accept a request body that forwarding rejects before a reservation. **Expected behavior** Qualification and forwarding use the same pure validator. One-level OpenAI namespaces with local function or custom tools remain supported. Invalid tool definitions fail before accounting or transport work. **Steps to reproduce** Use the synthetic native-daemon request fixture in `eval-gateway.test.ts`. It has seven local function tools and one namespace with five local function tools. Call the shared validator, then forward through the fake provider. Run the malformed namespace and tool-count cases to check rejection before side effects. The namespace behavior follows the [OpenAI function-calling guide](https://developers.openai.com/api/docs/guides/function-calling). This change preserves the one-level contract introduced in #17452 and hardened in #17469. ## What Changed - Export `boundedEvalPayload` with credential-free route fields. `forwardEvalRequest` calls that function. - Keep one namespace level with function or custom leaves. Require exact leaf types, nonblank string leaf names, and a closed namespace wrapper. - Limit OpenAI tool definitions to 1,024 per request. Count namespace wrappers and their children. - Add native-daemon-shaped forwarding coverage, malformed and hosted tool rejection, count and depth boundaries, and Anthropic compatibility checks. - Document the shared qualification policy in the Product E2E security guide. - Compile the real observability validator in the test parent. The owned CLI fixture imports JavaScript, so its process group does not include a test-only TypeScript compiler service. The parent resolves the compiler from the root package, which declares that dependency. Production cleanup checks and attempt deadlines stay intact. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts eval-gateway.test.ts cost-ledger.test.ts provider-usage.test.ts operating-overhead.test.ts operations.test.ts`: 173 passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Separate Node support checks: 137 passed. - `pnpm test:e2e:runner:unit`: 4,271 passed, two skipped, four failed in unchanged Daytona image and Pi catalog contracts. The same four failures reproduce on untouched base commit `a67a18c171816b097f46eae78fc58c7e55c9c565` (79 passed, four failed in those files). - `pnpm test:run`: interrupted and incomplete after required CI completed. Initial database startup failed because this checkout omitted the embedded PostgreSQL package postinstall aliases. The package hydrator restored them, and 14 real database checks passed. Earlier failures and the full output remain preserved; this local full run is not counted as passing verification. Its owned process group exited after SIGINT, with no owned survivors. - An offline full native authentication probe used the real Rust daemon and pinned Codex 0.160.0 Darwin SDK. The same captured request failed the older operations policy before reservation and reached a mock reservation with this policy. The OS sandbox allowed loopback only. It used synthetic credentials, retained no raw model payload, and made no provider calls. - Controller fixture suite: 70 passed. Three cold comparisons each confirmed that the old child started an owned compiler service and the corrected child did not. On Darwin the old compiler also exited cleanly, so this does not prove the exact historical Linux process race. Fresh Linux CI is required. - Independent review passed. The private qualification integration passed 14 exact-source compatibility checks and 67 shared-source tests without provider calls. - Required GitHub CI checks passed on the current head. The separate Entire approval gate still awaits an eligible reviewer; the delegated author account's explicitly AI-assisted approval does not satisfy it. ## Risks Requests with more than 1,024 OpenAI tool definitions or extra namespace wrapper fields now fail closed. New provider schema fields will need review. Anthropic tool handling, model prices, billing restrictions, output caps, and reservation ordering remain unchanged. This offline request-policy proof does not establish a completed model turn or Linux or Cloud onboarding acceptance. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model identifier and context window are not exposed in this session. The model used reasoning, tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bbe1016d81 |
feat(evals): retain bounded native tracing in the Runner controller (#17493)
Retain bounded native tracing in the Runner controller with attempt-bound relays, private export receipts, offline cost replay, conservative holds and bounded failure diagnostics.
Verified exact head
|
||
|
DottaandPaperclip
|
3f91d58dd5 |
fix(ci): provide Copilot auth only to protected paid eval cells (#17743)
## Thinking Path > - Paperclip manages AI agents and their work. > - Paid runner evals test real provider integrations. > - Copilot cells use the saved-token connection path. > - The two trusted workflows did not pass the Copilot credential to those cells. > - This PR supplies it only when the selected cell needs it. > - Existing environment and actor gates protect access in this public repository. ## Linked Issues or Issue Description Related: #15705 and #15706. A search found no open PR that adds this workflow credential binding. **Describe the bug** Copilot Product and direct-protocol eval cells require `COPILOT_GITHUB_TOKEN`. The protected paid workflows do not currently pass that environment secret to the selected execution step. **Expected behavior** Only a selected Copilot cell receives this credential. Ordinary PR, build, catalog, image and reporting jobs receive none. **Steps to reproduce** Dispatch the trusted full-stack workflow for one Copilot cell. Credential validation fails before the task starts, although the protected environment can hold a valid token. **Environment** GitHub Actions in the public `paperclipai/paperclip` repository, with the reviewed workflow on `master` and the `runner-e2e-paid` environment. ## What Changed - Pass `COPILOT_GITHUB_TOKEN` in the paid Product and direct-protocol execution steps, only when `matrix.credentialName` selects it. - Add the missing Copilot protocol-catalog credential mapping and verify it reaches the selected execution matrix. - Add security checks for the selected-cell binding and bounded protected-job scope. Extend the scan that rejects provider credentials in unrelated workflows. - Document the Copilot credential scope. No secret values, account data or private test evidence are in this PR. ## Verification - Product workflow security: 16 tests passed on this head. - Protocol catalog and workflow security: 21 tests passed, including the Copilot matrix regression reproduced red before its fix. - Current-head required `ci / verify` and `ci / e2e` are green. Greptile is 5/5 with zero unresolved review threads. - The first CI attempt retained runner-shutdown failures; one failed-job retry passed on the unchanged head. Both attempts remain in [the CI run](https://github.com/paperclipai/paperclip/actions/runs/40342429106). - `git diff --check` passed. - A selected-credential scan found zero matching secret values in the seven changed files. - The environment setting was separately checked: `master` only, required approval from existing allowlisted code owners, and administrator bypass disabled. The new secret exists only at environment scope; no repository-wide duplicate was created. - The packaged Copilot runtime verified the selected token and the explicit model without a model prompt. A fresh local UI journey then created an agent, completed its first task and verified the downloaded artifact and cleanup. ## Risks The selected paid cells gain access to one additional environment credential. Missing credentials still fail closed. Default-branch checks, initiating/rerunning actor checks, protected environment access and per-cell credential selection remain in place. This PR adds no trigger and gives no credential to builds or ordinary PR checks. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and browser regression tests. The exact runtime model identifier is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
49bd7a4ba6 |
feat(evals): retain portable Runner controller evidence (#17469)
Retain private Runner request evidence for offline cost reconciliation. Bound post-child reads, reject hidden paid tools, exclude configured credentials, bind incomplete results, and replace redacted proof atomically.
Verified at
|
||
|
DottaandPaperclip
|
41c42895fa |
feat(evals): enforce fixed Runner request allocations (#17452)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runner evals execute real provider harnesses against maintained fixtures. > - Turn-level usage arrives after requests have already spent money. > - Parallel cells need distinct allowances before requests start. > - Missing output must preserve the remaining allowance and any known charges. > - This pull request adds a controller that reuses the existing request ledger and gateway. > - It makes a bounded local Runner attempt reviewable before hosted scheduling is enabled. ## Linked Issues or Issue Description Refs #17440. That PR added the explicit Runner gateway route. This PR owns request admission and retained campaign accounting for that route. The existing hosted matrix is not enabled or changed here. Its global allocation, rerun rejection, and private evidence archive remain separate integration work. ## What Changed - Validate a trusted manifest with exact request, CLI, package, daemon, model, and source identities. Assign each selected cell a distinct allowance. Count operating overhead once. Reject allocations above the campaign limit before launch. - Run native Codex or ACPX Claude through the shared request proxy and append-only ledger. Pass only the proxy credential to the child. Refuse local attempt reuse and preserve existing output files. - Close gateway admission on cancellation. Retain the ledger, receipts, route descriptor, child result, and cleanup evidence outside the disposable workspace. - Recompute campaign accounting from retained ledgers. Missing or invalid final evidence holds the full cell allowance while preserving known charges. Reports distinguish provisional and incomplete costs. - Accept namespaces containing only local function/custom tools. Disable web search in the controlled Codex profile. Continue to reject hosted tools, including tools hidden inside a namespace. - Add controller tests to the existing Runner static test path and document this source-checkout entry point. ## Verification - 20 controller tests pass. They use real child processes, local HTTP, and the actual ledger. Cases cover reservations before dispatch, concurrent cell isolation, caps, cancellation, replay rejection, lost artifacts, changed receipts, and Braintrust Claude routing. Upstream responses are synthetic. - 52 shared gateway tests and 55 Runner CLI tests pass. The namespace acceptance and web-search configuration regressions failed before the repair and pass afterward. - Runner TypeScript/Rust checks, Runner build, shared E2E typecheck, and `git diff --check` pass. - The first actual-harness probe found that Codex advertised both a local tool namespace and web search. It reached no upstream transport. That failure is retained; the compatibility repair does not allow paid hosted tools. - An earlier test failure showed that separate dynamic imports created different admission-error classes. The shared runtime bridge now loads one module graph. Another regression showed that a missing final report hid known ledger charges; the collector now preserves those charges. - The repaired actual-harness probe passed its control checks for native Codex and ACPX Claude: each reached exactly one synthetic upstream request after the ledger reservation; both retained the unresolved hold and route evidence; both recorded clean process exit. A scan of 41 retained files found no synthetic upstream credential. Deliberate provider rejection preserved the failed Codex turn and Claude timeout; these are not semantic passes. Prices, operating bounds, and the package archive identity in this probe were synthetic. - Review regressions reproduced lost request/receipt evidence, lost admission metadata, executable replacement after verification, and a SIGTERM-resistant descendant surviving the CLI. The repairs now pass. Executable bytes are staged before launch; a trusted Node resolution hook preserves relative imports. A symlink-path regression also failed before its repair and now passes. The final staged native probe repeated both one-request checks successfully. - Full current-head [CI](https://github.com/paperclipai/paperclip/actions/runs/40115932485) passed on `4781487215116afd10a415d4db53b1ec8e004dd5`: all 58 check runs and the legacy status are successful, neutral, or skipped. Greptile and Paperclip both reviewed this commit at 5/5. All five finding threads are fixed and resolved. No paid provider request ran during this change. ## Risks - This is a trusted source-checkout controller, not a published package command. It needs the repository's installed shared cost dependencies. - The manifest declares reviewed prices and full-lifecycle bounds. Synthetic tests do not qualify real prices, invoices, storage, or Braintrust metering. - The attempt-directory guard is local. Hosted fanout still needs unique global allocations, rerun rejection, and durable private retention. Do not enable the existing matrix on this change alone. - Forced child termination can leave cleanup unverified. That attempt remains incomplete and holds its full allocation. These checks do not claim a host security boundary against an adversarial target. - The current private publisher does not archive the separate controller directory. Native trace relay, private report integration, and live qualification remain open. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. This runtime does not expose the exact model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aec86c794b |
fix(runner): preserve native work through controller outages (#17594)
Keep remote native work under its authenticated connection lease through controller outages. Hold durable ACPX tools through reconnection, align the native Claude HTTP MCP client budget, and wait for the recovered handler before dispatch. Preserve Stop, cancellation, exact identity and artifact checks. Verified with real-process regressions and a real Claude Opus 5.5 Daytona workflow: 125 seconds offline during useful work, original process and lease retained, no failed native tool calls, queued follow-up completed, independently verified ZIP and committed workspace receipts. All current-head CI gates pass and Greptile is 5/5 with all findings resolved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
e6ec781a1a |
fix(e2e): verify review outcomes and preserve report measurements (#17556)
## Thinking Path > - Paperclip's Product E2E journeys verify user outcomes in the real app and GitHub. > - The review journey must show the score and failed check for its exact reviewed commit, then the visible task status before and after reload. > - GitHub includes a hidden “Re-run failed checks” menu item before the visible check result. > - The journey selected that hidden item and failed even after native work and publication completed. A subsequent attempt exposed the same first-text-match problem with Done in the task UI. > - This change reads the selected check's status header and binds its URL, score, and commit. > - Browser fixtures retain this failure and reject plausible wrong projections. ## Linked Issues or Issue Description Refs: #17457, #17513, #17551. **What happened?** The bot completed its native run and task, then published its summary, inline precision finding, and failed 2/5 check. The browser assertion selected a hidden “Re-run failed checks” menu item before the visible result, so the journey still failed. After that was corrected, the next live attempt passed the check projection but selected hidden Done text before the visible task status. **Expected behavior** Verify the actual selected GitHub check. A hidden menu item or a word in the model's narrative cannot prove its conclusion. **Steps to reproduce** 1. Render a hidden “Re-run failed checks” item before a visible failed check. 2. Use the old first text match for “failed”. 3. Observe the false visibility failure. **Paperclip version or commit** The base includes the journey changes from #17457 and #17513. ## What Changed - Validate the check URL against the configured repository and select its exact check-run ID. - Require the expected score and full reviewed commit inside that check. - Read failure status only from that check's header. - Give each projection step its own phase so failures identify the correct boundary. - Stage the verified persisted screenshot as the standard report final-state artifact, rejecting a passing report if it is absent. - Preserve measured billing summaries during regeneration when raw run usage is absent; newer raw usage takes precedence. - Extend the catalog-wide history fixture to include the new journey evidence contract. - Read task completion from the visible Status property picker before and after reload, with separate phase labels. - Add eighteen isolated browser fixtures to the existing browser-support suite and document both boundaries. ## Verification - Reproduced the old selector failure in the live GitHub page and a credential-free browser fixture before fixing it. - All 60 browser-support cases pass, including eighteen focused projection cases. They reject the wrong check status, score, commit, page, and repository reference, plus hidden or unrelated Done text. - All 163 focused billing, history, accounting, report, and journey tests pass, as does runner E2E typechecking. - A regression reproduced discarded saved token totals. Retained totals now survive per-cell and campaign aggregation while missing dollar prices stay unknown; fresh raw usage still overrides stale summaries. - Diff and credential-value checks pass. - The earlier failed attempts remain failed. A fresh complete QA journey passed all 18 checkpoints and cleanup with a successful native run, a below-threshold review and inline precision finding, the correctly failed check, visible Done status, and reload persistence. The later reference-validation commit changes only browser fixtures; the live driver bytes are unchanged. - A report regression reproduced the missing standard final-state artifact. The packager now aliases only the hash-verified persisted screenshot for passing results; missing final-state proof is rejected. Presentation regeneration preserves the original live grade and dispatches no provider work. ## Risks This depends on GitHub's check-run header structure. A markup change fails the journey instead of accepting narrative text as a result. The product runtime and behavioral oracle stay unchanged. Usage-price coverage remains a separate evidence dimension. A saved billing summary is reused only when raw usage is absent, preserving the recorded coverage and unknown prices. The separate advisory Paperclip Review remains queued after its prior assigned attempt timed out. Its normal Retry control rejected the obsolete failed chat request as unsafe to retry. Required repository gates `ci / verify` and `ci / e2e` pass, all 55 completed CI checks are green, and final-head Greptile is 5/5 with no unresolved findings. The advisory queue is not represented as a completed assessment. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime model ID and context-window size are not exposed in this session. Used for code, browser fixtures, and live evidence analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <paperclip@paperclip.ing> Co-authored-by: Paperclip <bot@paperclip.ing> |
||
|
|
a0e5166e97 |
Fix GitHub journey registration diagnostics and native retry cleanup (#17513)
## Thinking Path > - Paperclip manages work performed by AI agents. > - The Product E2E harness verifies a GitHub review-bot journey. > - GitHub can return a registration page without the expected creation control. > - The journey could then wait for the full login allowance without useful evidence. > - A failed native run can also retain a retry after its terminal status is visible. > - This change records a safe registration boundary and uses the normal Stop route for owned retries. > - Operators can diagnose the setup failure and confirm cleanup without changing the behavioral oracle. ## Linked Issues or Issue Description Refs: #17457. **What happened?** The GitHub review journey waited at App registration with no useful handoff evidence. Its cleanup also skipped failed native runs that still owned a durable retry. **Expected behavior** The journey should report a bounded registration failure or reauthentication boundary. Cleanup should Stop an owned native retry and require its audited acknowledgement. **Steps to reproduce** 1. Run the configured GitHub review journey. 2. Return a GitHub registration page without the expected creation button, or fail a native review run while its coordinator can retry. 3. Inspect the result and cleanup evidence. **Paperclip version or commit** Source built from current master. This extends the driver merged in #17457. ## What Changed - Bound the registration render wait to 30 seconds. Keep the separate attended confirmation allowance. - Retain GitHub origin, path, and control-presence flags. Retain headings, button labels, and alerts only on known registration error pages without authentication fields. - Exclude query parameters, entered values, page bodies, grant payloads, and authentication content from that diagnostic. - Add an optional private authentication hook for ordinary GitHub access reconfirmation. Detect passkey-first sudo at the manifest URL without retaining its authentication content. - Stop owned failed native retries through the existing run-only cancellation route. Verify company, agent, task, and audit acknowledgement. - Use native Stop correlation only for native runs. Queued legacy or unresolved runs use the ordinary cancel request. Stop scheduled retries and wait for them to leave the live state set. - Keep pending accounting debt intact. Add regression coverage and journey documentation. ## Verification - Focused journey and report suite: 58 tests passed. - Runner E2E TypeScript check passed. - `git diff --check` passed. Credential-value scan passed. - Live fixture reproduced the missing creation control at the manifest handoff. The new diagnostic retained its path and control-presence flags. No model work started. Draft cleanup passed. - All 58 current-head checks are terminal and green. Greptile scored the exact head 5/5, with no unresolved review threads. ## Risks GitHub pages that take more than 30 seconds to render the creation control now fail early. Native cleanup fails closed if the server does not provide the expected Stop acknowledgement. The live setup-to-Done behavioral journey is still unqualified. These changes do not repair a missing provider usage receipt or weaken its accounting gate. ## Model Used OpenAI Codex, based on GPT-6. Exact runtime model identifier and context-window size are not exposed in this session. Used for code execution, browser tooling, and regression analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
970127a022 |
test: retain onboarding Connect failure evidence (#17529)
## Thinking Path > - Paperclip manages agents and their work through normal company controls. > - Product E2E journeys check these controls through the browser and public API. > - Cloud onboarding uses one Connect action to validate the environment and hire the first agent. > - A failed Connect action previously left only a general journey error in the report. > - This pull request retains a closed diagnostic receipt for that action. > - The report can show the observed failure boundary without retaining private response text. ## Linked Issues or Issue Description **What existing behavior does this improve?** Cloud onboarding Product E2E failure evidence and report detail. **Current behavior** A validation or hire failure can reach the journey as a general operation failure. The report does not retain which Connect response failed. **Proposed behavior** Observe only the same-origin, same-company validation and hire POST requests made during one Connect action. Retain a closed receipt with the checkpoint, response state, bounded HTTP status, and allowlisted adapter, outcome, check code, level, and server code. The Get started control and saved-agent readback remain required. **Reason and benefit** The next failure can be attributed to the exact observed browser/API boundary. This change does not claim the cause of a historical failure. **Breaking changes** The new receipt is optional. Historical results remain valid. Explicit validation and hire rejections now stop the journey with a precise closed error code. This changes only E2E source; it does not change product behavior or qualification gates. ## What Changed - Add a bounded Connect observer with exact request matching and listener cleanup. - Retain known server codes from `details.code` or `code` without response messages, hints, headers, request bodies, provider text, or credentials. - Validate the receipt as a closed schema and render its allowlisted fields in existing report detail. - Include the new helper in the journey definition hash. - Add route, privacy, malformed-body, deadline, late-response, delayed-failure, success, and historical compatibility tests. Document the evidence boundary. ## Verification - Focused Product E2E tests passed: 3 files, 58 tests on the review-fix head. - E2E typecheck, workspace typecheck, and full build passed on the preceding implementation head. The review fix changes only report labels, checkpoint timing, and the matching tests. - The local full `pnpm test:run` command stopped progressing at the general-server start header. Its owned session was interrupted. This is not recorded as a pass. - All 58 remote checks completed on the final review-fix head: 55 passed, one neutral, and two expected skips. Both Greptile and Paperclip reviews are 5/5 with no unresolved review threads. - No live campaign was run with this patch. The observed historical failure is preserved. ## Risks - Unknown or missing responses remain explicit diagnostic states. They do not establish a product root cause. - Response body reads stop after five seconds or the action deadline and accept at most 64 KiB of JSON. Checks are capped at 32 retained entries. - New server or check codes appear as `unrecognized` until the closed allowlist is updated. - The production launch classifier, native gateway qualification, and Get started requirement are unchanged. ## Model Used OpenAI Codex (GPT-6 family), with code execution and tool use. The host does not expose the exact model ID or context window to this run; these values are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
044628f59e |
Add explicit native tracing to the Runner eval CLI (#17483)
## Thinking Path > - Paperclip manages AI agents and their work. > - Runner Evals exercise real providers against a mock control plane. > - The controlled gateway route creates a disposable provider home. > - That route had no explicit native trace configuration. > - Codex skips normal home preparation for a preconfigured home. > - This change validates and applies the existing observability launch contract before provider creation. > - Operators can collect native evidence without passing vendor keys to the child. ## Linked Issues or Issue Description Refs #17440. Related: #17452 and #17469. This PR is based directly on master and does not include those pending controller changes. **Current behavior** The eval-session CLI cannot pass an explicit native observability launch. Passing environment alone cannot configure Codex in the disposable gateway home. **Proposed behavior** Accept an opt-in descriptor bound to the gateway attempt. Validate the existing provider launch contract, add a scoped relay token in memory, and apply it only to the owned provider home and environment. **Reason and benefit** A controller can route native evidence through its bounded local relay while the CLI keeps configuration distinct from verified ingestion. This is the CLI contract; hosted controller integration remains separate work. ## What Changed - Add a strict, bounded descriptor reader for native Codex and ACPX Claude. Bind campaign, execution, attempt, project, trace, span, and mock authority identifiers. - Reuse Runner observability materialization and redaction. Preserve the model gateway route. Reject upstream destinations, extra environment fields, credentials in headers, and unknown exporter settings. - Add `--observability-config`, requiring an explicit gateway. Keep direct and unconfigured paths unchanged. - Retain secret-free configured/unverified metadata in success and failure artifacts. - Document the contract and controller responsibilities. No workflow, runtime dependency, paid schedule, or public report changes. ## Verification - Eval CLI regression selection: 81 passed before the test-only relocation. The 17 new cross-package observability tests now pass from `tests/runner-e2e/runner-eval-observability.test.ts`; their integration-suite typecheck also passes. They use the actual shared launch generator, real disposable homes, CLI success/failure paths, and credential redaction. The first run exposed two test setup references introduced by a broad edit; both were corrected and its failure log remains retained. - Runner TypeScript build and release daemon build passed. Full `pnpm -r typecheck` passed. - Actual native Codex and ACPX Claude reached a local receiver with authenticated OTLP protobuf and the expected Braintrust parent. Codex sent four log and two trace batches; Claude sent three log and one trace batch. All recorded processes exited, disposable homes were removed, and 29 retained files contained neither synthetic credential. - The local model endpoint deliberately returned 401. Codex retained a failed terminal turn; Claude retained a provider timeout. These are route/export proofs, not semantic eval passes. Native retries reached only the fake endpoint. No paid provider or Braintrust request was made. - The first local probe failed before provider launch because its TypeScript loader path was wrong. It was corrected to the installed loader; both attempts remain retained. - Full repository build passed at initial head |
||
|
DottaandPaperclip
|
6d46851920 |
test(e2e): add attended GitHub review bot journey (#17457)
## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E checks verify real user paths through those agents. > - A recorded GitHub review-bot walkthrough had no repeatable registered case. > - App setup, automatic triggering, publication, and task completion need separate evidence. > - This pull request adds an explicit attended journey to the existing catalog and report tools. > - Reusable setup profiles and independent assertions preserve the same outcome across compatible targets. ## Linked Issues or Issue Description Refs: #17262. This case remains an explicit attended attached-target flow. It does not replace the shared deployment-target or paid-operations framework. **What existing behavior does this improve?** The Product E2E catalog can now select and retain evidence for GitHub review-bot setup and its first automatic review. **Current behavior** Recorded walkthroughs cannot be rerun as a fixed registered case. A published review alone can hide a failed run or unfinished task. **Proposed behavior** Create and install an owned App, save automatic new-PR settings, create a defective PR, and require correlated review delivery, an inline finding, a consistent check, a successful run, and a Done task. Retain failures and confirm owned-resource cleanup. **Reason and benefit** The case makes a useful external integration reproducible without treating setup, unit checks, or report rendering as proof of a completed live journey. ## What Changed - Add two explicit attended cells for an existing configured agent in local or Daytona execution. - Declare hosting, account, connection, and credential prerequisites through private setup bindings. - Add fixed browser controls, ownership checks, independent grading, and cleanup. - Package declared screenshots through the existing sanitizer, manifest, validator, and dashboard. - Reject operations-managed execution until budgeted routing is qualified. - Document hosted QA credential setup and evidence limits. ## Verification - Focused catalog, oracle, report, connection, and ownership checks: 127 tests passed. - Inherited-environment preflight regression failed against the previous commit and passed after the fix. Actual-run environment verification remains required. - Harness TypeScript check and full workspace typecheck passed. Full build passed. The local full-test attempt was interrupted after repeated embedded PostgreSQL startup failures; its exit code of zero does not establish a pass; Exact-head Linux CI passed the full test, typecheck, build, Runner, serialized, browser, and canary gates on `abbddeade848edefc8de75f11cc9008b2cd52764`. - A real hosted attempt published a rated review and inline finding. It failed when its native session timed out before the task reached Done. Cleanup passed. Full live qualification remains open. - Credential scan found no known credential values in the authored files or retained report corpus. ## Risks - The full case creates external test resources and uses the configured provider. It requires an explicit selection and current task authority. It is excluded from default and CI matrices. - Time and run limits are not a hard dollar cap. Missing dollar cost stays unknown. Fixture setup does not weaken budget policy. - Fresh personal OAuth, unattended authentication, runtime skill-read proof, and cross-host qualification are separate claims. - The selected live fixture needs environment/lifecycle recovery before it can adopt a longer provider deadline. ## Model Used OpenAI GPT-6 family through Codex, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, including the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open findings - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
661f33f994 |
fix(evals): verify stored gateway request coverage (#17432)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E campaigns retain a request ledger and send native traces and gateway logs to Braintrust. > - Native trace verification did not prove that each charged gateway request had a stored log row. > - A hosted probe showed that `x-bt-span-id` identifies the row `id`, while the stored `span_id` differs. Propagated campaign metadata was absent. > - This pull request joins retained request receipts to the correct Braintrust rows during scheduled finalization. > - Operators can see missing request coverage without changing costs, releasing holds, or rerunning agents. ## Linked Issues or Issue Description Refs #17354 and #17371. **Describe the bug** A campaign could pass native trace and experiment indexing checks while gateway request logs remained unverified. The runbook also claimed that propagated metadata made a request searchable after cancellation before response headers. The hosted gateway did not preserve that metadata in retained evidence. **Steps to reproduce** Inspect a retained gateway response header and query project logs by its `x-bt-span-id`. The matching record uses that value as `id`, not `span_id`. Compare every model reservation with its saved response receipt, including setup and canceled work. **Expected behavior** Verify each request against the stored row and report absent or conflicting evidence. Keep trace coverage separate from final usage and invoice reconciliation. ## What Changed - Added a bounded, read-only gateway index verifier. It matches row IDs, provider and model, and retains the actual span and trace IDs. - Included setup, failed and canceled requests in coverage. Missing receipts or row IDs, duplicate attribution, conflicting records and evidence changes prevent verification. - Added a separate scheduled gateway coverage gate and private receipt. Native trace and experiment checks remain independent. - Shared the existing bounded Braintrust query reader. Queries select only identity fields, with at most 100 IDs per batch, three rounds and a 45-second deadline. - Corrected the runbook's metadata guarantee and documented the historical `gatewaySpanId` receipt field. ## Verification - `pnpm test:e2e:runner:unit`: 4,120 passed, two existing skips; all 137 Node contracts passed. - Final repair: 54 focused tests passed, including scheduled-finalization coverage and all five optional identity fields. Each conflicting-field regression failed before the fix; absent or matching partial metadata remains supported. - `pnpm test:e2e:runner:typecheck`: passed after rebuilding stale local plugin SDK declarations. The initial stale-declaration failure remains recorded in the validation logs. - `git diff --check`: passed. - Read-only live verification of retained evidence: all ten local Codex requests and the one canceled drain-probe request matched stored rows. The hosted Claude campaign matched eight of ten; its two pre-header cancellations correctly remained incomplete. - The first hosted lookup exceeded the 1 MiB read limit when selecting full metadata. The final identity-only projection passed the same lookup. - Original journals stayed byte-identical. These checks made zero model calls and did not change provider data or release holds. - Full repository CI passed on `957ad0b684b80267d230fc4fb56d4f002ef089ad`: [run 40105676630](https://github.com/paperclipai/paperclip/actions/runs/40105676630), including typecheck, tests and build. Both current-head automated reviews scored 5/5 with no open findings. - The superseded initial CI run was canceled after the repair push to release the queue; its completed results remain available. No failed job was retried on the final commit. ## Risks Missing gateway IDs now fail scheduled trace coverage. A failed or canceled call may have no stored row, so this can require investigation even if product assertions pass. Indexing can lag; bounded query retries do not retry model work. Stored identity does not prove final billing or gateway span nesting under native attempt traces. Private identifiers remain in private evidence only. Recurrence and billing settings are unchanged. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model ID or context-window size in this session. Used reasoning, repository tools, local code execution and read-only API verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
87bf89756e |
fix: support Daytona runner upgrades and ignore obsolete Claude options (#17361)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Existing Claude and Codex agents can upgrade to Paperclip Runner. > - A Daytona computer needs platform qualification before Runner becomes its default. > - The upgrade notice required that default, so it hid the action that could start qualification. > - Saved Claude turn limits also caused conversion to fail even though Runner can run without them. > - This pull request verifies the computer on click and ignores obsolete Claude options with a warning. > - Agents retain their model, account, environment, instructions, and permission policy. ## Linked Issues or Issue Description Related: #17122. **What happened?** A legacy Claude agent on a Daytona snapshot had no upgrade button until its environment had platform qualification. A saved `maxTurnsPerRun: 1000` then caused conversion to return HTTP 422. **Expected behavior** The upgrade action verifies a supported Daytona computer and converts the agent. Unsupported tuning options produce a visible warning and do not block execution. **Steps to reproduce** 1. Create a legacy Claude agent with `maxTurnsPerRun: 1000` on an active Daytona snapshot. 2. Open Runtime before the environment has platform qualification. 3. Try to upgrade to Paperclip Runner. **Paperclip version or commit** After #17122, reproduced against `9c442cf48e`. ## What Changed - Offer upgrade on probe-capable Daytona snapshots and immutable sandbox images. Perform the public environment probe only after a click. Save after fresh adapter discovery confirms support. - Resolve the actual computer for explicit, instance-default, and hidden managed environments. Preserve pending edits and reject stale probe results. - Ignore Claude `maxTurnsPerRun`, `chrome`, `warmHandleIdleMs`, and `acpWarmHandleIdleMs`. Return structured warnings from create, hire, and update. Show warnings in Runtime and after a save. - Keep validation for commands, credentials, filesystem, network, and permission settings. - Retain the legacy turn limit in the existing live conversion fixture. Update regression tests, upgrade stories, and capability maps. ## Verification - Red: the old route returns HTTP 422 for the saved turn limit; the old form hides the Daytona upgrade notice. - Green: 417 focused server and form tests pass. A further 188 tests pass after the review fixes. Qualification coverage includes failed probes, unsupported targets, changed settings, managed defaults, unsaved environment variables, and forced Kubernetes targets. - `pnpm -r typecheck` and `pnpm build` pass. Token gates, map checks, map tests, and Storybook catalog checks pass. - Real local Claude Opus 5.5 conversion and task pass, with the downloaded artifact checked against seeded source bytes, SHA-256, and Git commit. - Real browser upgrade and full Claude Opus 5.5 task on a disposable named Daytona snapshot pass at `b81cec7d8c`. The saved agent retains its model, high effort, credentials, timeout, and grace settings while dropping the turn limit with a visible warning. Exactly one successful run completed the task. The downloaded artifact matched independently seeded source bytes, SHA-256, and Git commit; the source was unchanged, no interactions remained pending, and the delivery link survived a browser reload. The snapshot used current runtime assets built from public source. The supplied Daytona account could not access the original staging snapshot, so this verifies an equivalent local fixture rather than that exact snapshot. Disposable test resources were cleaned up. - Earlier live attempts remain recorded: an image-based sandbox creation failed, and the first named-snapshot probe returned failure with unresolved provisional cleanup identity. Later named-snapshot probing and the task succeeded; the initial creation failures are not attributed to this PR or declared resolved. A separate attempt using the Oct 18 cached runtime failed the current provider-pack compatibility check; rebuilding the runtime from matching public source removed that mismatch. Owned resources were identified and cleaned up, with diagnostic databases retained. - 54 checks passed at `b81cec7d8c`, with no failed checks; Greptile and Paperclip Review both scored 5/5, with both review threads resolved. Master advanced during the live proof, so the branch was rebased with both sides of additive import/test conflicts retained. On the rebased head `2b74222b94`, 472 focused tests, UI typecheck, and UI build pass. All required CI gates pass; Greptile and Paperclip Review both score 5/5 on this head, with no unresolved review threads. The PR is mergeable. The serial local `pnpm test:run` was stopped after the CI test lanes passed; that local run is not claimed as a pass. ## Risks - An explicit upgrade can allocate a temporary probe sandbox and take several minutes. Probe failures leave the agent on its existing runner. - Ignored options no longer enforce a turn cap, Chrome integration, or legacy idle-time tuning. The UI warns about each active ignored option. - Unsafe or ambiguous execution settings still fail validation. This change does not weaken environment qualification or runtime artifact checks. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
52a66af29d |
perf(evals): tighten measured campaign resource ceilings (#17381)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E checks real provider behavior under a campaign budget. > - Its request and trace limits also determine the operating-cost forecast. > - The initial limits greatly exceed the measured marker and feedback cases. > - This pull request lowers those defaults while keeping the same selected cases and full model context limits. > - The benefit is a smaller bounded resource envelope with explicit quota failures and readable historical evidence. ## Linked Issues or Issue Description Refs: #17365 Refs: #17371 **What existing behavior does this improve?** The resource ceilings for bounded Product E2E campaigns and their OTLP relay. **Current behavior** Daily and weekly campaigns allow 60 and 120 model requests. The native relay defaults to 256 MiB of expanded source and 16,384 records per attempt. Both October 23 two-case qualification campaigns used ten model requests. The largest native attempt used 6,531,801 expanded source bytes and 2,455 records. **Proposed behavior** Use 30 daily and 60 weekly model requests. Default native exports to 16 MiB of expanded source and 4,096 records. Preserve the older explicit relay maxima so prior receipts remain valid and deliberate controller overrides still work. **Reason and benefit** The smaller limits leave headroom for the measured cases and reduce the maximum source traffic in the operating forecast. They do not change the daily or weekly dollar cap, model context ceiling, per-request reservation, selected cases, or attempt policy. **Breaking changes** A new bounded attempt that exceeds the smaller default trace allowance receives a quota error. It cannot silently omit required evidence. Existing receipts remain readable. No paid recurrence is enabled by this change. ## What Changed - Set scheduled model request caps to 30 daily and 60 weekly. - Add smaller native relay defaults while preserving the existing explicit maximum and historical receipt validation. - Exercise default count and expanded-source exhaustion through the HTTP relay. Keep a regression for larger historical receipt policies. - Document the measured basis, default capacities, and failure behavior. ## Verification - `pnpm test:e2e:runner:unit`: 4,095 Vitest tests passed, two existing skips; 137 Node contract tests passed. - `pnpm test:e2e:runner:typecheck`: passed. - `git diff --check`: passed. - The relay test accepts two 3-MiB log payloads, rejects the third before upstream dispatch, and retains its quota receipt. - Read-only replay of the prior real hosted campaign preserved all five ingestion receipts and 123,044 payload bytes exactly. Its accounting journal remained byte-for-byte unchanged; zero provider calls. - Full repository typecheck, tests, build and reviews are pending hosted CI. No paid model call was made for this change. Live qualification of the smaller defaults remains open. ## Risks - More verbose future cases may exhaust the new defaults. Such a failure remains visible and requires evidence before increasing a limit. - Source-byte ceilings are not Braintrust invoice units. Operating-cost qualification and reconciliation remain separate requirements. - Manual controller overrides can still use the earlier larger maxima; their cost audit must account for the limits actually recorded in the receipt. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution and tool use. The host does not expose an exact model build identifier or context window size for this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
eadc560712 |
fix(evals): retain usage during bounded gateway shutdown (#17371)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests use a local gateway to record model usage and reserve costs. > - Test cleanup can disconnect a background request before its provider sends final usage. > - Immediate abort then leaves the request cost unknown and its full reservation held. > - This pull request lets an admitted response finish within a short drain period. > - The benefit is better accounting without a new request or a retry. ## Linked Issues or Issue Description Refs #17365 and #17354. **What happened?** Two hosted cases passed, but cleanup canceled background requests before the gateway received their usage. Both requests kept their full holds. A local HTTP regression reproduces this loss with OpenAI and Anthropic responses. **Expected behavior** Retain final usage when an already-dispatched response finishes within a bounded cleanup period. Keep the client outcome canceled. Preserve the hold when final usage is still unknown. **Steps to reproduce** 1. Send a request through the eval gateway. 2. Disconnect the client before the provider returns final usage. 3. Close the gateway and let the provider response arrive. 4. Before this fix, the gateway aborts immediately and loses the final receipt. **Paperclip version or commit** Reproduced against master `b6e690044a821faab52d5466db36c4330c754a2d`. ## What Changed - Drain an already-dispatched response for at most five seconds after a disconnect. Keep the original provider timeout and campaign deadline. - Stop forwarding response data to a disconnected client. Stop admission as soon as shutdown begins. - Retain `clientDisconnected` on the private receipt and keep the outcome canceled, even when final usage arrives. - Make repeated shutdown calls share completion. - Add HTTP and SSE regression tests. Document the drain and its accounting limits. ## Verification - Red: five regression cases failed against the previous gateway implementation. Four further cases reproduced late cancellation during review. - `pnpm test:e2e:runner:unit`: 4,072 Vitest tests and 137 Node contract tests passed. Two existing Vitest tests were skipped. - `pnpm exec tsc --noEmit -p tests/runner-e2e/tsconfig.json`: passed. - `git diff --check`: passed. - Regressions cover both providers, an interrupted SSE stream, grace expiry, a shorter provider deadline, no dispatch during interrupted identity resolution, no new admission, repeated shutdown, client-only disconnects, and disconnects during each final receipt/ledger write. - Full repository typecheck, tests, and build passed in [CI run 40026846417](https://github.com/paperclipai/paperclip/actions/runs/40026846417) on `40ef13bd4d5df6b2ca0a9666fb485adf2a565c28`. Greptile and Paperclip reviews are both 5/5. No paid model test was run for this change. Live qualification remains open. ## Risks - Cleanup can take up to five more seconds. The admitted request can continue generating during that period, inside its existing reservation and deadlines. - An abort does not prove that the provider stopped billing. Missing usage keeps the unknown hold. - This change does not establish invoice settlement or rewrite historical receipts. It changes only the eval gateway. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, and tool use. The host does not expose an exact model build identifier or context window size for this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a967d479bd |
fix(e2e): retain verified onboarding artifact evidence (#17370)
Retain the bounded verified onboarding download and independent oracle in private normalized results before cleanup. Preserve the receipt when later readback or trace checks fail without marking the journey as passed. Refs #17262. Refs #17253. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b6e690044a |
feat(evals): allow reviewed bounded provisional accounting (#17365)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E campaigns include model, setup, storage and observability costs. > - Some billing windows close after the tests and reports finish. > - Requiring settled receipts for retained storage can block a good campaign for weeks. > - A reservation alone does not prove that a pending charge has an upper bound. > - This change permits provisional completion only with reviewed evidence for the exact cost configuration. > - Reports preserve unknown amounts and holds until later reconciliation. ## Linked Issues or Issue Description Refs: #17360, #17354 **What existing behavior does this improve?** The Product E2E scheduled accounting gate and its retained cost reports. **Current behavior** Every pending receipt fails finalization. A completed test can wait through the retained-storage billing window. The journal has dollar reservations but no explicit qualification of their bounds. **Proposed behavior** Permit provisional completion when all pending charges have reviewed full-lifecycle bounds, no requests remain in flight, no receipts conflict, and known costs plus held amounts fit the campaign limit. Keep all existing holds and receipt-finality rules. Missing evidence still fails. **Reason and benefit** An operator can distinguish delayed but bounded billing from unbounded or missing costs. The campaign can finish without claiming that its invoice is settled. **Breaking changes** The journal accepts optional bound metadata and an invalidation record. New reports add an optional bounded-pending count. Current readers retain old journals and reports without granting them bounds. Older code must not read new bound-bearing journals. The existing daily/weekly enable gates and budget amounts stay unchanged. **Additional context** This implements the operator-approved provisional-cost policy for the existing eval harness. It does not change product budgets or the agent-eval roadmap. A targeted GitHub search found no duplicate open PR. ## What Changed - Accounting owns a narrow evidence contract: a reviewed audit hash, exact configuration hash and full-lifecycle scope. - Gateway reservations retain that evidence before dispatch. Changed limits or rates fail before dispatch. Unexpected routing, model identity or excessive usage invalidates a bound durably. - Operating allowances can carry separately reviewed bounds for each cost component, including its full storage lifetime. No existing allowance becomes qualified by default. - Reports and Braintrust experiment metadata distinguish provisional accounting, complete receipts, blocked accounting and unavailable evidence. Public output omits private audit hashes. - Scheduled finalization allows qualified provisional costs and explicitly rejects spending overruns, including overruns with complete receipts. - Add regressions for legacy data, restarts, late reconciliation, conflicts, malformed evidence, routing uncertainty and exceeded limits. Document the evidence and migration boundaries. ## Verification - Full offline eval-support suite: 4,062 Vitest tests passed, two existing tests skipped; 137 Node tests passed. - `pnpm test:e2e:runner:typecheck`: passed. - `git diff --check`: passed. - Read the retained hosted qualification journal with the new policy. All six legacy pending charges remain unqualified. Known cost `$0.421245200`, held `$19.327680000`, and committed `$19.748925200` remain unchanged. The journal's SHA-256 is unchanged. The original failed workflow remains failed. - No paid models, network provider calls, receipt imports or reservation releases were used for this verification. - Rebased onto master `4d90634f74`; 142 targeted tests and typecheck passed. - Review reproduced a lost OpenAI input-total limit check when cache details were missing (1 failed, 47 passed). Fix `5fef9ba7c149951a51eae468614a4b89f0b3d6db` preserves unpriced provider usage bounds. Then 178 focused tests, full offline support and typecheck passed. - Current-head hosted CI passed: https://github.com/paperclipai/paperclip/actions/runs/40020746585 (attempt 2). Typecheck, build, all general/Runner/serialized tests and browser gates are green. - CI attempt 1 hit a PostgreSQL deadlock in unchanged chat-test cleanup; its skipped flag restoration caused a second assertion failure. One failed-jobs-only retry passed. Original failure evidence is preserved in attempt 1. No paid eval campaign was rerun. - Greptile reviewed exact head `5fef9ba7c149951a51eae468614a4b89f0b3d6db` at 5/5; its one finding is fixed and resolved. The optional Paperclip review is paused and supplied no assessment. ## Risks - Evidence hashes bind an audit to a configuration; they do not prove the audit is true. Operators must review real provider limits, all price tiers, full resource lifetimes and enforceable infrastructure limits before configuring bounds. - Provisional completion is not invoice settlement. Later billing can reveal overruns, which must be reconciled and inspected. The ledger cannot stop external billing. - Existing pilot journals, configured allowances and price cards have no new bound approval. This PR alone does not qualify or enable recurring paid work. - The gateway cannot qualify a canceled Braintrust request without route confirmation. It preserves that unknown hold. - No new dependency, secret, repository variable, workflow, schedule or cloud resource changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment ID and context-window size are not exposed in this session. Used reasoning, repository tools and code execution. No paid evaluation model calls were made for this change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
17f1e386e8 |
feat(evals): compare retained Anthropic billing exports (#17360)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E evaluations retain request costs and failed attempts in a journal. > - Provider exports can arrive later and contain costs at a wider time scope. > - Comparing those exports by hand makes repeated accounting work hard to audit. > - This change adds a read-only command for the observed Anthropic export format. > - It binds the comparison to the retained manifest, journal and exact export bytes. > - Operators can see the unmatched amount without assigning it to canceled requests or releasing their holds. ## Linked Issues or Issue Description Refs: #17009, #17354 **Problem or motivation** A hosted campaign retained eight complete model receipts and two canceled requests. A later provider export covered the campaign, but comparing its tokens, rounded costs and the journal required a one-off script. Repeated observations must not be counted twice or treated as request-level billing proof. **Proposed solution** Add a private comparison command for one standard Claude Sonnet 5 campaign and one UTC date. Validate the selected workspace, key label, model and pricing dimensions. Compare provider totals with one authoritative observation per request. Retain source hashes and signed differences. Keep provisional exports, partial receipts and missing model identities explicit. **Alternatives considered** Manual comparisons remain possible but are hard to repeat. Importing a day total as a request cost would invent attribution. This command leaves journal and budget mutations to their existing owners. **Roadmap alignment** This is a focused operational tool for the existing Product E2E accounting subsystem. It does not change the product budgeting or agent-eval roadmap features. A targeted search found no duplicate billing-comparison PR or issue. ## What Changed - Add the provider-specific CSV parser, scope checks, token pricing comparison and private CLI in the Product E2E accounting owner. - Report complete and partial receipt coverage, source hashes, rounded provider charges, separate non-model charges, and signed unmatched amounts. - Reject ambiguous scopes, unsupported price dimensions, duplicate rows, contradictory token observations, changed journals and incomplete input. - Write a new mode-0600 output without replacing existing files or changing the journal. - Document the supported export shape and the limits of comparison evidence. ## Verification - After rebasing onto master `e48f1867b3b45f74edca4645d640846bcafeee4c`, 104 focused and adjacent tests passed across comparison, ledger, provider pricing, operating receipts and gateway behavior. - `pnpm test:e2e:runner:typecheck`: passed after rebase. - `git diff --check origin/master...HEAD`: passed. - Ran the actual CLI on retained hosted-campaign CSV exports and journal. It reproduced the prior manual calculation: provider-token estimate `$0.428027200`, with `$0.006782000` unmatched to retained request cost receipts. No provider or network call was made. - The first retained-data run exposed an empty provider key-status field and canceled receipts without a returned model identity. Those cases are now covered by the fixtures; the rerun passed. No request identity or final charge was invented. - Regression tests first reproduced loss of token fields on unknown-cost receipts (2 failed, 13 passed). The repair retains each known field independently of cost and rejects conflicting equal-authority token values. - Full PR CI passed on `f74ef1c31391ee62b686fcf300601e9cf8e73a7e`, including typecheck, tests, build and browser suites. Greptile and Paperclip each reviewed that exact head at 5/5 with no remaining actionable findings. ## Risks - The initial supported schema is standard global Claude Sonnet 5 with context buckets at or below 200k, one campaign and one UTC date. Other provider shapes fail explicitly. - Matching workspace/key labels does not prove exclusive key use or a complete provider export. The output remains a comparison and cannot release a reservation. - Provider costs are rounded and can lag. The token estimate uses retained prices and is kept separate from provider charges and non-model costs. - Config and exports are private inputs. Output includes hashes instead of raw key labels. This file is not a public report projection. - No runtime dependency, schedule, credential, paid execution, journal format or admission rule changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment ID and context-window size are not exposed in this session. Used reasoning, repository tools and code execution. No paid evaluation model calls were made for this change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e48f1867b3 |
fix(evals): correlate canceled gateway requests with their ledger (#17354)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E evaluations use a budget ledger and a local model gateway. > - The gateway writes each request reservation before it calls a provider. > - A client can cancel before the gateway returns response headers. > - Project-only logging context cannot then bind a late gateway log to its reservation. > - This change sends the reserved request identity in Braintrust log metadata before dispatch. > - Operators can use that identity to inspect late records without guessing from time or session order. ## Linked Issues or Issue Description Refs: #17009 **What happened?** A hosted evaluation passed its product checks but retained two canceled requests with no response headers. Later gateway logs had partial usage. Their time and session matched, but they had no ledger request identity. **Expected behavior** A gateway log must carry the ledger request ID even when the client receives no headers. A log match must not imply final billing. **Steps to reproduce** Run the gateway regression with a transport that receives a request and then cancels before response headers. Before this change, both requests have the same project-only parent header. ## What Changed - Add a small Braintrust context encoder owned by the eval gateway integration. It uses the existing SpanComponentsV4 project-log format and an allowlist of request, campaign, execution, attempt and phase metadata. - Save the exact context in the private request receipt before dispatch. Send it as the gateway parent header. - Test both provider routes, repeated identical requests, cancellation holds and direct-provider isolation. - Document the lookup field and the limits of historical and partial evidence. ## Verification - The new cancellation regressions failed before the change: 2 failed, 37 passed. - Gateway, operations and tracing checks: 82 passed. - `pnpm test:e2e:runner:typecheck`: passed after rebuilding stale generated plugin-SDK types from current master. The initial stale-type failure remains recorded locally. - Official Braintrust SDK 3.38.0 decoded the actual generated context. Its `startSpan` implementation emitted the reserved identity in the initial LLM log. The check captured logs in memory and forbade all network calls. - `git diff --check`: passed. - Full repository CI passed on `529446a9295f2dc6fa6499a2f976480d74626097`, including typecheck, tests, build and both required gates. Greptile is 5/5 on that head; its sole finding is fixed and resolved. - The optional Paperclip review turn ended without a final response. Its check remains queued and provides no review assessment. No passing review is inferred from it. - Live hosted acceptance of this header change remains unqualified; no paid test was dispatched for this fix. ## Risks - The encoder follows Braintrust's versioned span format. A future vendor change can require an update. It adds no package dependency. - A late log can have partial usage. Existing reservations remain held until valid cost evidence resolves them. - Historical project-only requests cannot gain the new identity retroactively. - Only synthetic eval identifiers enter the new metadata. Credentials, prompts and responses are excluded. Direct provider routes keep their current headers. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment ID and context-window size are not exposed in this session. Used reasoning, repository tools and code execution. No paid evaluation model calls were made for this change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
59d1b7b43d |
fix(evals): collect usage when GitHub skips jobs without a runner (#17352)
## Thinking Path > - Paperclip manages AI agents and their operating budgets. > - Product E2E runs retain accounting for each campaign. > - The usage collector reads completed GitHub jobs without rerunning paid work. > - GitHub returns a null runner ID for jobs that it skips before assigning a runner. > - The collector rejected those jobs and could not retain the completed campaign inventory. > - This change accepts that skipped-job representation while preserving started-job checks. ## Linked Issues or Issue Description Refs #17009. **What happened?** The usage collector rejected completed qualification run 39971835384. GitHub returned skipped jobs with null runner IDs. The collector accepted only zero for jobs without a runner. **Expected behavior** Retain completed run usage when skipped jobs have a null or zero runner ID. Do not count VM time for those jobs. Continue to reject missing runner identity for started jobs. **Steps to reproduce** Read the completed first-attempt Product E2E run with operating-receipts.ts github. Before this fix, a skipped job with runner_id set to null throws Invalid workflow job identity. **Paperclip version or commit** |
||
|
DottaandPaperclip
|
ed3f067dbd |
fix(evals): select operating allowance by cadence (#17326)
## Thinking Path > - Paperclip manages AI agents and their operating budgets. > - Product E2E runs exercise real agents through a bounded gateway. > - The daily and weekly selections share one repository configuration variable. > - Their different request limits need different operating allowances. > - The existing parser accepts only one allowance, which cannot represent both reviewed limits. > - This change selects the allowance for the actual cadence and retains all existing validation. ## Linked Issues or Issue Description Refs #17009. **What happened?** The daily selection needs an $11 operating reservation; the weekly selection needs $21. A shared $21 allowance cannot fit the $20 daily cap. A shared $11 allowance does not cover the reviewed weekly resource bound. A configuration containing both allowances was rejected before paid admission. **Expected behavior** Each cadence uses its own reviewed allowance. Missing or malformed selected allowances stop admission before paid work. **Steps to reproduce** Set the operating allowance JSON to an object with daily and weekly allowances. Prepare either campaign through prepareScheduledCampaign. Before this change, both calls reject the outer object. **Paperclip version or commit** |
||
|
DottaandPaperclip
|
3bd709df8b |
Operationalize Product E2E accounting, tracing, and scheduling (#17009)
## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E tests exercise real agents, tools, setup and cleanup. > - Repeated runs need retained costs and traces, including failed attempts. > - A passing retry must not hide earlier spending or feedback. > - Recurring work also needs admission limits, private reports and failure detection. > - This change supplies those controls while keeping paid activation behind existing gates. ## Linked Issues or Issue Description Refs #16996 and #17158. This is the consolidated landing of the reviewed eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033, #17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096, #17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127, #17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository permits squash merges only. A local rehearsal showed that squashing each slice into master causes repeated conflicts in dependent slices. This PR preserves their commits and lands their combined implementation in one reviewed integration. Superseded slices will close after this merge. ## What Changed - Retain every attempt's cost receipt, including setup, failures, interruptions and retries. Separate reported amounts, estimates, missing costs and reserved capacity in reports and dashboards. - Add bounded request admission, exact model pricecards, gateway error accounting, cache policy and reconciliation evidence. Refuse unsupported routes and requests that cannot fit the allowance. - Integrate Braintrust gateway traces and OTLP correlation, with secret filtering and experiment imports. Record phase timing, including setup, feedback capture and cleanup. - Collect complaint and suggestion evidence after each attempt, including recovery after failures. Retain explicit review dispositions independently from the product grade. - Add gated daily and weekly selections, measured forecasts, setup/readiness checks, wall-time limits and retained monitor notifications. Include declared non-model overhead in admission while reporting the existing subscription separately. - Keep recurring history private, bound uploads and retention, and use isolated archive environments. Keep public publication an explicit operation. Report and publication jobs install only the committed lockfile, with scripts disabled and isolated stores; stale pins fail closed. - Integrate current master, preserve its Product E2E journey billing and privacy checks, and test that shared setup receipts do not double-count estimated agent usage. Preserve upstream fixture repairs. ## Verification - Complete integration baseline `0649a5ce7544ab20825c2183d2f421157db49f24` includes master `a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head as ancestors. - On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012 Vitest tests passed, two skipped, across 193 files; all 137 Node source-contract tests passed. - Focused billing and Product E2E journey contract checks: 24 tests passed. They cover estimates, unknown receipts, single-owner setup costs, browser-only execution and private publication guards. - Monitor and workflow wiring: all 96 Node tests passed. `node scripts/check-token-gates.mjs`: all gates clean. - `pnpm test:e2e:runner:typecheck` passes after the standard Runner TypeScript build refreshed local declarations. The first attempt rejected the new master `runner_prepared_launch` union member because this checkout retained an older generated declaration; the source union already contained that member. - Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes dependency re-resolution from archive-authorized reporting jobs after the security scan flagged it. All 15 workflow-security tests pass after updating the old two-install assertion. - Hosted [CI run 39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180) passes on the current repaired head, including typecheck, build, general tests, Runner checks, browser shards and canary dry run. Greptile is 5/5 on this exact head with no unresolved threads. The security scan passes after the fixed lockfile-resolution thread was resolved. - Paperclip Review returned a terminal neutral check but an incomplete 0-file assessment on the repaired head: its platform blocked both diff responses as prompt-injection content. This is not a successful code review. No bypass or fabricated pass was attempted; current-head Greptile, security and hosted CI are independent gates. - No paid test, reservation release or recurring-run activation accompanies this landing. Live route qualification and invoice reconciliation remain separate evidence. <details> <summary>Retained earlier validation failures</summary> Earlier broad local runs had two 15-second test timeouts (CLI poll cadence and Retry validation), one embedded-database setup timeout, and later PostgreSQL semaphore exhaustion. The exact timed-out tests passed separately without larger timeouts; the broad-run causes were not established. Those local failures remain distinct from subsequent hosted CI results. The independent #17158 fix records its Muse fixture collision, its UUID correction and hosted verification. Local Muse execution was also blocked by macOS embedded-PostgreSQL startup. No historical failure is treated as a successful paid qualification. </details> ## Risks - Admission is a local control, not a provider invoice cap. Missing provider receipts or resource charges retain unknown amounts and holds; estimates do not become billed dollars. - The bounded path admits only documented local suites and provider routes. Ordinary legacy or separate Runner protocol campaigns have different controls. Merging this code does not qualify additional routes. - Recurrence still needs the existing enable gates and current qualification/allowance evidence. Archive permissions and resources must match the documented private-environment contract. - Legacy reports may lack complete attempt or phase evidence. Reports preserve that limitation and do not infer zero usage from missing data. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks; full-suite host limitation disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5ef385f7e6 |
Run reusable Product E2E journeys across deployment targets (#17262)
## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description **Subsystem affected** Product E2E harness, fixture registry, and reports. **Problem or motivation** Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. **Proposed solution** Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. **Roadmap alignment** This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0a5f1546ee |
feat(cases): add reusable prompt workflows (#17114)
## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
6dda8cc2b7 |
fix: preserve Stop authority and qualify complete recovery journeys (#17105)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final head |
||
|
|
507a0d7098 |
test: verify repository delivery after real runner conversion (#17089)
## Thinking Path > - Paperclip manages agents that perform useful work. > - Repository setup and Runner selection must preserve that work context. > - A successful provider turn cannot prove configuration retention or usable delivery. > - The repaired setup chain needs independent product checks with the recorded models. > - This PR adds explicit repository and conversion journeys to the existing Product E2E harness. > - The observer checks source bytes, Git identity, saved configuration, and downloaded output. > - Development and staging qualification remain separate. ## Linked Issues or Issue Description Refs #17088 and #17047. The complete repository setup repair in #17088 is merged. No competing conversion journey was found in the open PR search. **What happened?** Existing checks do not prove that a real legacy agent retains its configured context when switched to Paperclip Runner and then delivers verified repository work. The projectless repair check also uses different default models from the recorded staging models. **Expected behavior** A configured repository task delivers correct, accessible output. A real legacy-to-native conversion preserves configuration, instructions, environment, and equivalent permission policy before doing subsequent work. A projectless task continues through setup without supervision. **Steps to reproduce** List the configured-repository, legacy-conversion, and automatic-repository-configured-models suites. Run one exact ID from the README. These are explicit paid development checks, excluded from the default selection. ## What Changed - Add four configured-repository cells and four real legacy-conversion cells for Codex gpt-6.1-sol and Claude claude-opus-5-5 on local and Daytona environments. - Seed a fresh reference only in an owned disposable Git checkout. Compare the delivered JSON with its original bytes, hash, and commit. - Switch the same real legacy agent through the public Runner selection API. Compare allowlisted configuration and instruction digests before and after conversion. - Require one committed native run with the configured provider, model, and environment, a visible registered download after reload, Done, and no pending question. - Add four projectless replay cells with the recorded staging models and the unchanged strict continuation oracle. - Use production hire instructions and existing sandbox profiles, budgets, report pipeline, and cleanup. Record API-key and per-turn lifecycle differences explicitly. - Retain the required allowlisted API snapshot after successful automatic setup. Preserve lifecycle IDs, state, ordering, and receipt references without raw provider content. Verify both admissions retain the configured provider and model. ## Verification - Product E2E support after integrating the setup repair: 3,353 passed, two existing skips; source prerequisites: 128 passed. - Product E2E typecheck passes. Discovery lists all twelve new explicit cells. - Independent oracle calibration rejects wrong configuration, instructions, provider, model, source bytes, hash, commit, downloaded output, delivery, and pending questions. - The first full support run exposed a catalog count mismatch and an unprovisioned new profile ID. Both are fixed; the successful run uses the existing trusted sandbox profile IDs without changing workflow authority. - Review found fixture prerequisite defects: optional CLI-only Codex arguments and platform qualification after Runner selection. Both are corrected; the disposable Claude baseline also excludes its unsupported optional CLI turn limit. Seven new setup checks cover probe ordering, absent qualification, immutable images, local conversion, and uncertain probe accounting. The final focused cohort passes 86 checks and E2E typecheck passes. - The first qualification-regression run also had an incorrect test assumption about the legacy timeout; the corrected red reaches the unsupported extraArgs setting. No production conversion restriction was relaxed. - A retained four-cell development campaign delivered independently verified ordinary Codex output and real Codex/Claude conversion results with retained configuration. All three attempts nevertheless failed the required evidence contract because api-state.json was missing. The flow now saves allowlisted task/run lifecycle fields; it does not export prompts, comments, interactions, configuration contents, or provider outputs. The projectless cell delivered valid bytes but used a monitor fallback and failed the unchanged continuation oracle; its driver correction belongs to #17088. All four attempts remain failures; fresh exact-candidate acceptance is pending. These API-key/per-turn checks do not establish subscription authentication, warm reuse, Slack delivery, or the complete milestone. - Four fresh local ordinary-repository and conversion cells passed at `c4e409d819cc0a703736d3f1ed3552064ca690b8`. Independent downloaded artifacts and saved configuration evidence agree. - The repaired automatic setup chain passed all six journey checks for both models, but first failed because the success path omitted api-state.json. The separate capture repair now passes the fresh two-cell [campaign](https://github.com/paperclipai/paperclip/actions/runs/39633381139) at `7a7c26f6688ec6b0e0454562a43fd48ab1a81c97`. Both required API files and their run/intent/dispatch bindings were independently checked. Earlier failures remain failures. - After merging current master, 114 workspace/admission/continuation checks across six suites pass. The provider/model and evidence privacy cohort passes 65 checks, and E2E typecheck passes. The final model-binding assertion has unit calibration; fresh actual-model acceptance of that assertion is still pending. - Exact-head CI/review and complete staging acceptance remain pending. Local development passes do not establish the 24-cell milestone. ## Risks The real provider may expose additional conversion or setup defects. Failures remain evidence and cannot be replaced by catalog registration or a later retry. The conversion uses the production API and does not inject a native result. Only synthetic data is seeded in a resolved fixture-owned directory. Configuration evidence contains digests, not raw environment values. This PR changes evaluation code, not production behavior. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
|
eece07a747 |
fix: repair the complete projectless repository setup chain (#17088)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
53732c811b |
fix(evals): classify latency from observation bounds (#16829)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E probes measure when acknowledgements and replies become visible. > - A poll gives a time interval for an event. > - The first positive poll can occur after a target that the event could still meet. > - This pull request compares the full interval with a strict or inclusive target. > - Reports retain the raw observations and the original verdict when the timing verdict changes. ## Linked Issues or Issue Description **What happened?** A historical acknowledgement was absent at 1901 ms and first present at 2427 ms. A report marked the 2000 ms target as failed. The interval crosses that target, so the available evidence cannot give a pass or fail verdict. **Expected behavior** Return pass when the full interval meets the target. Return fail when the full interval rules out the target. Return inconclusive for overlap, invalid evidence, or insufficient evidence. Respect strict and inclusive targets and open and closed endpoints. **Steps to reproduce** Run the deterministic timing tests. Use the acknowledgement interval `(1901, 2427] ms` with `<= 2000 ms`. Compare the late reply interval `(52965, 53334] ms` with `<= 30000 ms`. The first is inconclusive. The second fails. **Paperclip version or commit** The source includes master at `14afc12124017d4ec30d0f4bfd6892f8ce475d21`. Related PR: [#13710](https://github.com/paperclipai/paperclip/pull/13710) changed chat latency and session reuse. This change corrects probe evidence and report verdicts. ## What Changed - Add a shared interval classifier with explicit endpoint and target operators. - Start observation before dispatch. Use a monotonic clock. Bound asynchronous reads by request start and completion. - Measure the first visible response separately from message creation and final settlement. Share browser reads with an optional acknowledgement probe. - Keep raw bounds, clock resolution, message metadata, failed reads, and original verdicts in retained report evidence. - Add deterministic coverage for threshold equality, missing data, edited messages, asynchronous reads, teardown, report correction, and a finite-domain oracle. - Bound observer shutdown to one second across pending and final reads. Retain timeout counts. Ignore late completions and start no more reads after shutdown. - Document the timing contract in the existing Product E2E guide. - Keep current startup reuse evidence after the master merge. Update four stale test expectations for qualified Copilot admission. Keep the explicit suite and provider restrictions. ## Verification - Current head: `c42e87e6d3b4b347e7cb176f56819c1d0abe106d`. - Passed: `pnpm test:e2e:runner:unit` (3,287 Vitest tests and 128 native completion tests; two explicit skips). - Passed: `pnpm test:e2e:runner:typecheck`, `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, and `git diff origin/master --check`. - Passed: [exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39607140418), including both required checks (`ci / verify` and `ci / e2e`), all general and serialized test jobs, all eight browser shards, build, canary dry run, and the clean public npm-install smoke. - Greptile reviewed the complete current diff and returned [5/5 with no actionable issues](https://github.com/paperclipai/paperclip/pull/16829#issuecomment-6266704814). All review threads are resolved. - The redundant local `pnpm test:run` was stopped after the exact-head CI test jobs passed. It did not produce a complete local verdict; the full repository test evidence is the CI run above. - The [earlier broad run](https://github.com/paperclipai/paperclip/actions/runs/39249282597) failed on the old head. Its full test command timed out after 15 minutes. Five E2E support failures came from ambient `NODE_PATH` and a missing historical Git object. Current master supplies the closed-environment launcher and retained source fixture. The full support suite passes with these fixes. - After the current master merge, four stale Copilot assertions and the nested unit launcher failed. Copilot is now qualified. Those assertions now require no qualification override and retain the rejected suite/provider cases. The full documented unit command passes. - The first local typecheck after the merge used stale generated package declarations and failed. The typechecks pass after `pnpm build`. - These checks use synthetic timing evidence. They make no live provider calls and claim no new live latency measurement. ## Risks - The browser diagnostic depends on the existing message and receipt DOM markers. Controlled tests cover visibility, receipt separation, concurrent reads, and teardown. - Corrected timing verdicts can become inconclusive. The report keeps the original verdict and does not change the overall workflow grade. - This change adds no database migration or product surface. ## Model Used - OpenAI GPT-6 Codex. Model family: GPT-6. This session does not expose a more specific model ID or context window size. The agent used reasoning, shell tools, code inspection, and deterministic tests. The earlier implementation session did not retain its exact model ID. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dff63551fe |
fix: resume unfinished work after repository preparation (#17047)
## Thinking Path
> - Paperclip manages agents that perform work for a company.
> - Task repositories are prepared at normal run admission.
> - A running native task can stage a repository while its work remains
unfinished.
> - The controller did not offer that saved setup as a continuation.
> - The agent could add a timer or wait for a user restart.
> - This change uses the existing same-agent continuation and status
ledger.
> - The accepted explanation and proven conversation remain available
after setup.
## Linked Issues or Issue Description
Refs #17045. That prerequisite has merged. This PR targets master.
**What happened?**
A projectless native task can request a repository for the next
admission. An unfinished task has no server-selected repository
continuation. An explicit same-agent continuation also loses chat
provenance.
**Expected behavior**
An unfinished standard task can yield and resume through the existing
admission queue. Its explanation and final result retain the authorized
conversation. A task that only stages a repository can complete without
another run.
**Steps to reproduce**
1. Start a native standard task in a task directory.
2. Request an authorized repository through prepare_repository.
3. Yield with unfinished work and no explicit continuation.
4. Observe that the former driver rejects the yield, although the
pending repository is saved.
## What Changed
- Offer a saved same-agent continuation for the current task's pending
repository.
- Accept that current server continuation in the shared provider driver.
- Resolve chat provenance through applied status decisions, delivered
effects, and admitted wakes.
- Preserve accepted setup explanations without repeating an owned
question card.
- Document optional reproduction details and configured repository
discovery.
- Extend database regressions and workspace access maps.
- Preserve Copilot profile v52 and bind the shared continuation source
in v53. Include the active identity in the QA image fingerprint.
## Verification
- Five focused server suites: 100 checks pass.
- Shared continuation driver: 12 checks pass.
- Server typecheck passes.
- Map contract and whitespace checks pass.
- Existing API and driver assertions fail on the former implementation.
- Real dispatcher and queue cohort: 43 checks pass. The original
reply-loss regression fails before the dispatch receipt fix.
- Continuation driver and installation identity cohort: 62 checks pass.
- Regenerated capability inventory and contract checks pass.
- Final response precedence cohort: 109 checks pass. The stale-comment
case fails on the former implementation.
- All required CI checks, build, and both source reviews pass on
|