mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 15:35:12 +01:00
3937215a058d18555e80a2bc87397ffeffe5d898
103
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dotta
|
3937215a05 |
Qualify explicit native written references in the basic question journey (#18459)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Basic task journeys must prove that people can answer an agent and receive saved work. > - The question journey asks for a choice, then a typed reference, then a saved note. > - A native choice card can offer a real written-answer control as well as predefined options. > - The old observer rejected that form before a person could type the reference. > - This pull request accepts that explicit native control and keeps exact answer and work evidence. > - The separate text-only and named-tool qualifications keep their stricter contracts. ## Linked Issues or Issue Description Refs #18346 and #18456. The integration inherits the reviewed fixture prerequisite #18387 and Agent Chat recovery #18465 through their actual merged parent. The producer fix in #18456 is separate from this basic journey contract correction. Public duplicate PR and issue searches found no other correction for this form equivalence. **What happened?** The basic question journey requires the reference card to have `answerMode: text`. It rejects a native single-select card even when its canonical form explicitly enables a written answer. This makes the observer test a widget shape in addition to the requested behavior. **Expected behavior** The person must make a real choice and type the complete reference. A native card with an explicit written-answer control can satisfy the second step. Selecting a predefined option cannot satisfy it. The agent must resume the bound work and save both actual answers in its own task document. The document must survive reload. **Steps to reproduce** Use the focused recording and real QuestionForm browser test in this PR. Keep the reference as a single-select native question with one predefined option and `customAnswer.enabled: true`. The previous grader rejects `question-resume.forms`. The previous browser helper rejects `single_select` before it uses the written-answer control. **Paperclip version / commit** Original source base `f2db909393de124caf6a5380a52d4e0e1d621661`. Current integration `a417fd827347ed6f7998774a9de6aee70b76d867` merges exactly `12b0cc630471ee151ec5bc8e41a34f09236d6723` into reviewed question source `faec4ce6d775a34d303d200987e4e84d6dc26caf`. The scheduled failure on `82d15946c5844d3ae41c6ade62fa224053f20a9a` remains a failure. Its saved card had no canonical written-answer capability and would still fail this change. The retained normalized events do not prove whether the original provider flag was omitted or false. **Deployment mode** Attached native task journeys, including QA Cloud. This PR changes the observer and grader. It does not change the app, provider prompt, model, budget, timeouts, cleanup, or runtime deployment. ## What Changed - Accept one provider-bound single-select reference card with explicit canonical custom answers and valid, distinct options. Keep pure-text reference support. - Make the browser select the actual written-answer control and fill the real rich input. Opt in only for the basic question journey. - Keep the exact no-option typed response, immutable card, board actor, native request and continuation, saved agent document, reload, model, environment, checkpoints, and existing one-to-three-run checks. - Reject missing or disabled custom capability, inferred legacy capability, malformed or extra questions, foreign native identities, changed cards, substituted options, and wrong or missing reference text. - Add a real production QuestionForm and TaskChatRichInput Chromium probe. It first selects a predefined option, then proves the helper clears it and submits only the exact custom text. - Document the contract correction and preserve the historical failed attempt and its separate cleanup outcome. Keep necessary-question and named-tool documentation qualification unchanged. ## Verification - Before the observer change, the focused grader regression failed only the new valid-written-reference case; the other 108 cases passed. - Before the browser helper change, real Chromium passed the text control and four rejection controls but failed the valid native written-answer control at the old pure-text assertion. - Initial focused Runner E2E unit selection at `af07f4de`: **256 passed across 9 files**. This includes question/resumption, clarification evidence/flow/collection/deadline, necessary question, documentation, contracts, and report catalog tests. - Initial real Chromium selection at `af07f4de`: **12 passed**. Six new production QuestionForm/control checks and all six existing document-readiness checks pass. No form or rich-input component is mocked. - Historical test-only head `faec4ce6d775a34d303d200987e4e84d6dc26caf` adds a direct canonical-absent legacy `freeText` grader negative and renders the actual legacy presentation in the browser. All four production helper blobs remain identical to the reviewed head. A temporary legacy inference mutant makes the new negative fail. After restoring production source, **179 focused unit tests and all 6 real QuestionForm checks pass**. The six document-readiness specs and their helper are unchanged. - Original [CI run 41205748589](https://github.com/paperclipai/paperclip/actions/runs/41205748589) failed the staged-listen Runner handoff test; that failure is preserved and was not retried. Its Runner source closure was identical to an earlier passing canonical lane. The symptom also recurred on other heads and is tracked independently; no infrastructure attribution or live pass is claimed. - Current integration `a417fd827347ed6f7998774a9de6aee70b76d867` preserves **all 12 question files byte-for-byte** from faec. Every other tracked path is identical to exact merged parent `12b0cc630471ee151ec5bc8e41a34f09236d6723`, which contains actual prerequisite `baf651c8d0122efc93f3ef03e891919eeb666215`. There is no manual conflict resolution or additional production edit. - On that integrated head: **257 focused Vitest tests across 9 files, 206 required Node source/reporter tests, and 12 real Chromium tests pass**. Map/module-boundary checks and catalog discovery pass. The actual `--preview` command passes with a synthetic question target and an intentionally unavailable adapter; it performs no credential read, adapter loading, provisioning or model call. It retains maxRuns3, a 600000ms campaign bound and zero retries. - Exact-head [CI run 41210806950](https://github.com/paperclipai/paperclip/actions/runs/41210806950) passes, including official typecheck, build, all eight browser shards, both Runner Vitest lanes and aggregate verify. All 59 current PR contexts are terminal with no failures. Fresh [Greptile summary](https://github.com/paperclipai/paperclip/pull/18459#issuecomment-6543878408) explicitly reviews `a417fd827347ed6f7998774a9de6aee70b76d867` at 5/5 with no findings; its exact-head check succeeds and all threads are resolved. No fresh deployed journey pass is claimed yet. - Catalog discovery lists the existing question journey. Map checks and module-boundary checks pass. Diff checks pass. - `tsc -p tests/runner-e2e/tsconfig.json` was attempted. The fresh integrated checkout still lacks compiled Runner, plugin SDK, and adapter declarations. The attempted local check reports 344 diagnostic lines from dependency/type closure and zero diagnostics in all 12 question-owned paths. Official exact-head CI establishes the full typecheck/build gates. Full repository build, typecheck, and test suites were not repeated locally for this observer change. - The first broad local report selection lacked the CLI loader and then hit sandbox IPC restrictions. Locked dependency installation and an authorized local IPC run resolved those setup errors; the full selected 256 tests pass. The initial localhost probe also needed the normal local-listen permission. - No fresh QA pass is claimed. The coordinator must qualify the final reviewed runtime and harness together. Prior failed results remain preserved. ## Risks The basic journey accepts one additional form with explicit native provenance. It still rejects the original closed card without a written-answer capability. The separate producer default fix must reach the selected remote provider pack before it can affect a deployed run. Local browser and grader tests do not prove that artifact served or that a model completed the live journey. Existing cleanup and physical-stop obligations remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, browser testing, and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked related work above - [x] I have described the issue in-PR following the bug template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
492d537340 |
fix(native): recover controllers across webserver replacement (#18343)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native execution runs in a remote Runner while the webserver owns its controller. > - Encrypted PRP checkpoints and session snapshots already survive a webserver restart in Postgres. > - Ordinary warm and per-turn admission still needs local markers and supervisor metadata to find that state. > - Replacing the webserver with an empty native controller directory can therefore lose a usable retained session. > - This change adds bounded recovery metadata to the existing snapshot transaction and reconstructs local projections from the selected durable source. > - The benefit is native execution continuity across webserver replacement, with the existing ownership and agent lifecycle policies. ## Linked Issues or Issue Description **What existing behavior does this improve?** Native remote controller recovery and retained-session continuation during a webserver upgrade. **Subsystem affected** server/ — native runtime, checkpoint persistence, recovery scheduling and internal upgrade reporting. Product E2E qualification also changes. **Current behavior** Without both the local PRP marker and supervisor file, normal continuation can return before discovering the saved Postgres checkpoint. The snapshot already exists in Postgres, but its local reuse compatibility envelope cannot be reconstructed. **Proposed behavior** Select the existing durable source, validate its identity and ownership, restore its PRP state and supervisor projection, then use normal authenticated admission. Missing or inconsistent evidence holds recovery. It does not authorize fresh provider work. **Reason and benefit** A replacement webserver must recover active execution and later continue its retained session without the old server's native files. This is one component of deployment continuity. HTTP traffic switching, workspace storage migration and Railway rollout remain separate. **Breaking changes** No new public endpoint, schema migration or database pool. The private run-profile envelope is additive. Disk-independent rollback also requires a reader for the staged controller format and compact checkpoint part; the first production release containing these readers is the floor. Older file-backed sessions need an owner checkpoint or the bounded internal importer. **Additional context** Related foundations: #17401, #17916, #17254, #14621, #14622. The implementation contract and open acceptance gates are in [the dated plan](doc/plans/2026-10-30-native-controller-checkpoints.md). ## What Changed - Persist a versioned recovery envelope, at most 4 KiB, beside the redacted session snapshot in its existing transaction. Keep it out of public run projections. - Compare frozen admission separately from prepared attachment instructions during staged recovery, preserving exact template/identity/authority checks. - Bound the existing lazy checkpoint pool at three connections after measured two-slot burst backlog; keep request/renewal capacities and SQL operation counts unchanged. The tuning is a separate commit. - Discover and restore the exact selected retained source without local native markers. Preserve the 32-candidate window, progress barriers and current authority checks. - Fence snapshot and result writes. Acquire coordinator, issue and run locks in order and check expiry after lock waits. Combine snapshot ownership/read work into one statement. - Reuse existing recovery deadlines and drain/resume scheduling. Retained-owner waits preserve the original unstarted run and actual lease deadline. Stop, ownership and provider-progress fences precede deferral. Add bounded native evidence reporting, retained-source import and retention protection. - Admit a settled warm source with an authenticated idle goal-read suffix while retaining active-work and mixed-authority holds. - Permit lifecycle `turn.stop` checkpoints after accepted results under existing ownership fences. Claim the existing successor coordinator before retiring the old warm session, and stage shared attachments only after verified retirement. Preserve omitted isolated-home defaults and existing fresh skill-selection order during recovery without hiding real configuration changes. - Carry the exact durable six-field PRP binding during active recovery after a healthy warm handoff. Keep the Runner mismatch guard and preserve opaque attachment IDs. - Preserve an already verified active or idle warm process during remote preparation; revalidate its proof/liveness and retain later authenticated PRP attachment. Ordinary resumed leases retain suspension checks. - Drain retained source events under their original authority, reconcile immutable source accounting and block unknown source tool dispatch before target work. - Persist compact staged handoff intent through authenticated adoption and recover direct/listener command-loss windows without another Runner or duplicate attachment. - Rebuild suspended lifecycle receipts for resumed prepared tasks, and retry missing lifecycle reads after partial PRP materialization without overwriting contradictory evidence. - Add normal warm/per-turn diskless continuation coverage, real Postgres races and SQL-cost checks. Add a reproducible paired load workload. - Give only the explicit outage fixture a 15-minute provider idle window, with a red/green catalog regression preserving all production timing and lifecycle policies. - Publish the goal-less task-session pointer in the existing post-acceptance terminal checkpoint transaction, with current ownership, accepted-result, reset and newer-progress fences. Keep a private bound receipt so late completion cannot duplicate or resurrect publication. - Use a retained warm sandbox in the dedicated continuity fixture, and accept an exact output token followed by an explanation. Keep the strict identity oracle. - Extend the explicit Daytona cell to a five-minute active outage and another replacement after completion. Require retained provider identity, a new tool receipt and a useful follow-up result. Add an owned loopback Postgres E2E fixture for hosts that cannot start embedded Postgres. ## Verification **Merge recommendation — `fc68584ce6231a02e47ecd743f55b036ccea034f`: ready for the qualified native-controller component.** Full CI, fresh review, local application paths, controlled performance qualification and the strict live Linux/Daytona case pass. Independent resource cleanup is complete. Broader rollout, browser reconnection and the unexecuted transport/platform combinations remain integration gates. Integrated 484 passed full local recursive typecheck/build and ordinary active→completion→cache-loss→C component qualification. Fresh review then found a valid attached-wake failure: preparation appends attachment instructions before the verifier compares execution to the frozen admission. 9f4 fixes that by separately checking the pre-preparation admission and exact prepared execution. Both hashes are captured privately per admission; the full template, identity, configuration and authority checks remain. Its regression fails before the fix; 36 focused cases, 42 staged source/target cases and server typecheck pass afterward. The review thread is resolved. Exact-head full CI passes ([run 41168228822](https://github.com/paperclipai/paperclip/actions/runs/41168228822): 48 successful jobs, one intended skip); the complete `pnpm test:run` partitions are forced, with the five native suites in the Runner lane. This is not a claim that an unsharded local command passed. The fresh exact-head Greptile check succeeds with 5/5 and all six threads resolved. The original two-connection database comparison closes **steady PASS and burst FAIL**. Every relative CPU/WAL/request-p99 gate passes and all operations/ownership remain correct, but candidate arm 3 drains in 1,015.99 ms and baseline arm 4 in 1,059.67 ms against the unchanged 1,000 ms absolute gate. Those results remain failed. A separate 2→3→4→2 diagnostic reproduces drain times 1,051.57 / 956.79 / 882.02 / 1,065.72 ms: three is the smallest tested passing cap, and the repeated two-cap failure survives run-order/cache effects. Separately reviewable `fc68584ce6` changes the existing lazy checkpoint pool default to three connections. Request capacity stays ten and liveness two; idle release, shutdown fencing, transaction counts and pool count are unchanged. Thirteen DB lifecycle/cap tests and DB typecheck pass. Full recursive typecheck/build pass at exact |
||
|
DottaandPaperclip
|
322fa6b65c |
fix(ui): restore task document authoring and add persistence journey (#18356)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Task documents keep versioned work beside each conversation. > - The default chat interface could read general documents but hid their editor. > - People also need unfinished edits to survive a tab switch or reload. > - This pull request restores the shared editor in Artifacts. > - A portable browser journey checks exact saved content and revision history. > - The benefit is a usable document flow with independent deployment checks. ## Linked Issues or Issue Description **What happened?** A person using the default task interface could read a general task document but could not create or edit one. The existing editor was available in Classic mode. A pending create could also leave newer text as a new form after its editor unmounted, causing a conflict against the person's own successful creation. **Expected behavior** A person can create and edit general documents through Artifacts. Titles, bodies and revision history persist after reopening and reloading the task. Unfinished text survives tab switches. A successful create makes retained newer text an edit of that confirmed revision. Later writes from another actor still cause a conflict. **Steps to reproduce** 1. Open an ordinary task in the default chat interface, then open Artifacts. 2. Try to create a general document. The previous interface has no authoring control. 3. With authoring restored, submit a new document, keep typing, and switch to Properties before the request finishes. 4. Reopen Artifacts. The previous draft path restores a new form despite the successful create. Related public work: #17598 adds document copy/download and #18157 adds image galleries. The selected actual merged parent is `f1d780838218f975064f38333946805e0ec103a5` from #18427. It includes the stable profile-entry ordering repair and the exact recovery-fixture prerequisite. The resulting PR diff contains 32 document and reporting files. ## What Changed - Reuse `IssueDocumentsSection` for general documents in default-chat Artifacts. Keep Classic authoring in its existing task body. Ordinary tasks, conversations and Coordinator mode share the same editor options and signed-in permission wrapper. - Preserve Plans, file-review documents, attachments, uploads and work products. Hide duplicate general-document cards. Use the same general-document key rules for visible rows, validation and refused writes. - Forward image uploads, mentions, references, author maps and feedback to both editor mounts. Keep empty mention/project values stable while queries load. - Back up unfinished forms within their signed-in account, task and editor scope. Preserve text entered before its key. Record a successful create on its matching retained newer buffer, including after unmount or an early reopen. Preserve newer title/body edits, real remote conflicts and matching cancellation cleanup. - Add the model-free `task-document-persistence` journey. Setup creates only an owned empty backlog task. The browser creates and edits the document; normal APIs independently check company, task, actor, exact content and both revision records. - Reopen the owned task through the explicit Backlog view, then reload it. Require exact UI readback and zero native runs. Add only the Doc registration, definition-hash inputs, report mapping and four closed checks to the shared harness. - Count explicit zero-run browser receipts after private journey metadata is removed for publication. Preserve missing billing evidence, positive usage and native/protocol accounting. - Update browser assertions for the restored inline editor while retaining exact download bytes, revision collisions, paused-state and no-run checks. Update the feature map, usage docs and shared editor stories. ## Verification - Latest focused checks use official Node 24.22.2: 568 journey, catalog, report, billing and attempt-accounting tests across 25 suites pass. All four admitted journeys remain registered. The seven relevant UI suites pass 529 tests, including the task-page container after the parent merge. Maps, token gates, the PR-scope diff check and the owned credential-pattern scan pass. The normal QA import preflight also passes with an empty environment and network access blocked. - Billing regression: before the repair, the explicit empty public receipt counted one run, and two positive-usage receipts were discarded by browser metadata. Those three cases fail before and pass after the repair. Two additional red regressions cover aggregate usage beside an empty run list. That ambiguous shape now keeps an unavailable receipt and a minimum count of one. Missing receipts retain unavailable billing, and native/protocol one-or-more run totals retain their measurements. - The previous fresh CI run failed its unchanged snapshot-only recovery fixture: one global sweep did not call its callback. The original failure remains recorded. An isolated PGlite probe with all canonical migrations exercises the real reconciler: it processes two earlier sources, then reaches the owned third source on the next sweep. It preserves source evidence, files and run IDs. This proves bounded pagination, not full native snapshot repair. - Local native recovery tests could not start PostgreSQL because the host semaphore limit was reached. That run executed zero cases. Canonical Linux CI later passed all 143 fixture cases, including the original snapshot-only case and the new competing-candidate control. The exact fixture repair is now inherited from #18427 and is absent from this PR diff. - The portable oracle rejects 19 stale, missing, foreign or incorrect outcomes. Result validation also rejects model/provider metadata and native runs for this case. - Five real Artifacts-to-Properties unmount regressions fail before the repair and pass after it. They cover both editor modes, reopening before and after completion, later typing, and a foreign revision that must still conflict. The edit-storage failure case also checks that Cancel clears the promoted fallback copy. - UI coverage includes default chat, Classic, conversations, Coordinator, Plan rendering, document-card deduplication, reserved keys, pending-query stability, permissions and rich editor options. - Earlier real browser checks passed five upload cases and two artifact-arrival cases. They retain exact CSV bytes, original/revised content, collision, paused-state and no-run assertions. Current-head normal CI runs the updated browser suites again. - The branch merges only the selected actual parent `f1d780838218f975064f38333946805e0ec103a5`. The merge has no conflicts or manual resolution. The SQL service and recovery fixture match that parent exactly. The own document editor, persistence flow, saved-state oracle and billing repair retain their previously qualified source blobs. Follow-up, clarification, question/resumption, screenshot helpers and the Steering map are byte-identical to the merged parent. Existing project/search/question/follow-up registrations remain present. - Required document source blobs remain exact: Doc UI `22e80663b053363e5f75e1432dd9043dc0e70243`, its unmount tests `16f3f4ed0dc9b11aaff129e06e6cf2bbf4522713`, readiness helper `800a747284db19b8484650b6759e8bd1f356839d`, and budget service `3a15d489f0aa55c22e8f7fe6a6bc6a0e14e5914e`. - Local typing is limited by borrowed declarations from another checkout. They omit current `skill_test` and sandbox creation-operation fields. No dependency declarations or repository config were changed. The authoritative clean CI Typecheck + Release Registry gate passes on exact head `2ffce29827ae170420d740940c1c069c7225a74c`. - Full monorepo typecheck, tests and build were not run locally. Borrowed build declarations from another checkout limit local typing. Authoritative [clean CI](https://github.com/paperclipai/paperclip/actions/runs/41171717008) passes on exact head `82d15946c5844d3ae41c6ade62fa224053f20a9a`: 55 successful checks, 3 skipped and 1 neutral, with zero pending or failed checks. Typecheck, build, Runner, server and browser lanes pass. Greptile gives that same head a fresh 5/5. All 15 review threads are resolved, with no further thread page. The PR is mergeable and clean. - Failure history remains recorded. The earlier recovery failure was repaired with a bounded, owned-source fixture. A later CI worker shut down after 2,375 Runner unit tests passed. Its one authorized narrow retry passed those tests and all five native integration suites. The separate AgentConfig browser failure was reproduced and fixed by the merged #18427 parent. These earlier passes do not replace current-head CI. - Final-source live qualification passed on exact harness and served application `82d15946c5844d3ae41c6ade62fa224053f20a9a`. QA run `41177696858` completed all four closed checks and all five phases with zero native runs. Cleanup passed; the durable manifest is complete with no pending evidence. The coordinator verified the original grade and manifest. - The [public qualification report](https://pages.paperclip.ing/evals-qa-smoke-41177696858-1/) records zero agent runs, empty run IDs and empty run usage. Pricing remains unavailable and billing remains incomplete; the report does not claim measured zero cost. - The earlier harness `36c8d9ddd76d0b7ddb28a9a6c674d9e538f6df88` pass against application `2ffce29827ae170420d740940c1c069c7225a74c` remains historical evidence. It is separate from the final-source measurement above. - The selected parent brings profile-entry ordering, proactive chat, a database migration, Runner updates, and task-page chat/read behavior. Its new task-page container is included in the final application measurement. The coordinator retains the original evidence and owns the primary merge and daily-plan admission. ## Risks Artifacts now mounts the existing shared editor. Its autosave, conflict, revision, annotation and server access contracts remain in place. Tests cover both interface modes, shared scopes, later remote writes and unavailable edit storage. Plans and generated file-review documents keep their existing flows. The PR adds no database schema or API change. The shared billing repair requires explicit captured empty run IDs and an unambiguous usage object containing only an empty run list for public browser-only receipts. Other fields beside an empty list keep conservative unavailable billing. Missing measurements remain unavailable. Contradicting positive usage remains counted. The inherited recovery fixture keeps the production sweep limit and its source, readback and no-agent-work assertions. The live journey must fail when authoring controls or saved outcomes are absent. It must not create the expected document through setup APIs or switch to another interface to pass. Final deployment verification must retain the actual app and harness revisions separately. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8c9dd1e444 |
test(runner): qualify Kimi Code lifecycle and recovery (#18029)
Add the Kimi lifecycle and recovery qualification harness. Preserve original grades and cleanup evidence, strict native ownership audits, bounded startup and cleanup, and early rejection of unsupported local fault execution before credentials or resources. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d9354bbfc3 |
Add a portable agent question and answer resumption journey (#18346)
## Thinking Path
> - Paperclip helps people manage agent work and keep human decisions
attached to tasks.
> - Product E2E journeys verify the browser, native agent, saved state,
and cleanup.
> - The classic suite already checks a choice question followed by a
text question.
> - Deployment-targeted journeys need the same behavior on an attached
instance.
> - Production questions can use provider input, the dedicated semantic
tool, or the authenticated native API fallback.
> - The basic journey must verify the original card and native
continuation for each supported path.
> - This pull request shares the scenario and browser actions, adds
strict API receipt evidence, and verifies the saved document after
reload.
> - Separate tool-documentation qualification and the original failed
measurements keep their existing scope and outcomes.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
It adds portable coverage for answering agent questions and resuming the
same task on an attached deployment.
**Current behavior**
The classic suite has the two-question scenario. The portable catalog
does not expose it. Its shared native-path grader recognizes provider
and dedicated semantic receipts, but rejects a valid completed native
API creation receipt.
**Proposed behavior**
Expose `question-answer-resume` as a deployment-targeted journey. Answer
one choice question and one text question in the browser. Require
server-owned creation evidence, unchanged forms, exact board answers,
bound continuation, an agent-authored task document, and readback after
reload.
**Reason and benefit**
The same story can qualify local and hosted targets through their normal
provider setup and cleanup. It catches lost answers, duplicate or
foreign work, premature output, and stale documents.
## What Changed
- Register the journey with a three-run maximum and positive model
allowance. Accept one to three attributed work runs at both collection
and retained-result boundaries.
- Share the existing two-question scenario, browser answer actions, and
behavioral grader. Keep separate tool-documentation grading unchanged.
- Recognize a completed native API creation receipt only when its HTTP
201 result retains the original pending card, exact
task/company/agent/source run, idempotency key, and full normalized
form. Require the receipt to remain unchanged after the answer and
through the final checkpoint. Missing, pending, foreign, or altered
proof still fails.
- Check signed-in board ownership, each native continuation, no
premature or extra output, document authorship, and the saved revision
after reload.
- Wait for the authenticated product region before screenshots. Recheck
origin and path after readiness. Bound readiness and capture by the
campaign deadline.
- Add closed stage-specific errors, catalog coverage, sanitized receipt
calibration, and documentation. Include the provider-form and
browser-question presentation helpers in the definition fingerprint.
## Verification
- Fresh QA staging qualification passed at
`7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks
passed, cleanup passed, and all three native runs used the configured QA
target and model. The [sanitized run
report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/)
records the measured pass. This is one successful qualification, not a
reliability claim.
- Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3`
contains the one main merge through
`fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of
the four-file qualified search fix
`4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity
matches. It preserves the project and search registrations and atomic
search readback. The question live qualification ran at the earlier SHA
above; the final integration head has separate local and CI
verification.
- Eighteen question journey, helper, and test file blobs are identical
between the qualified and integration SHAs. This includes clarification
flow and evidence, question grader and browser helper, the sanitized
receipt fixture, screenshot helper, provider-form helper, browser
question presentation, continuation scenario/readiness/flow, shared
agent validation, and user actions. The question run-count contract and
dispatch branch are also unchanged. Reviewers can compare those paths
directly between the two commits.
- All 622 selected Product E2E, catalog, question-resumption,
necessary-question, report, and readiness tests pass after the main
merge. After the exact search delta, all 145 narrow
question/search/project/contract tests and all nine real Chromium search
regressions pass. Map checks, combined credential-free preview, source
fingerprints, and the earlier workspace-link preflight pass. The local
TypeScript attempt reports stale Runner declaration errors for
`onAccountingEvent` in unchanged native-session code; fresh official CI
provides the clean-build type gate. The three search source/test blobs
match the qualified search merge, and the three project source blobs are
unchanged.
- An in-memory probe executes the actual launcher fingerprint
expression. Each project, search, and question implementation input
changes the digest. Separate red probes first reproduced the omitted
provider-form and browser presentation helpers; both pass after their
source inputs are added.
- The sanitized retained native API response-shape regression fails
before the grading correction and passes afterward. Negative controls
reject missing, pending, foreign, changed, spoofed, or wrongly bound
receipts, answers, continuation, model, environment, and document
evidence. All nine provider/semantic/API path combinations pass
calibration.
- The real local Chromium screenshot probe fails before the readiness
repair and passes afterward. Login redirects and sign-in pages at task
URLs remain excluded with no screenshot files.
- The two original attached failures remain retained: the first stopped
at early screenshot capture; the second stopped before answering at the
former API provenance gate. An earlier sandbox run also retained one
report subprocess failure; the full selected tests then passed with
local subprocess IPC access.
- The prior integration CI had an unchanged canary mock `EADDRINUSE`
failure. That history is retained; its documented retry is owned by the
coordinator. No application API, schema, production UI, model, runtime
policy, credential, or deployment configuration change is included.
Final integration head
|
||
|
DottaandPaperclip
|
ebed7f5237 |
test(evals): protect Antigravity setup and two-turn delivery (#18190)
## Thinking Path > - Paperclip manages agents and their work. > - A provider integration must work through ordinary agent setup and task controls. > - The recorded Antigravity walkthrough exposed setup, delivery, and continuation failures. > - A repeatable check must verify real work and retain failed attempts. > - This pull request converts all 47 recorded steps into one fixed Product E2E journey. > - Maintainers can rerun the case and inspect independent delivery, cleanup, and accounting evidence. ## Linked Issues or Issue Description Builds on merged #18187, #18188, and #18189, and consumes the shared repairs from merged #18230 and the equivalent manual-policy declaration from #18222. Refs #18214 for the CI task wrapper. Refs #12503 and #14938 for related legacy adapter work. **Agent or provider** Google Antigravity native Runner with a saved managed Google subscription and the exact Low model. **Why this adapter is useful** A user must be able to add an agent, connect its account, and receive real task results across a follow-up. Picker visibility alone is insufficient. **How the agent is invoked** A fixed Playwright driver uses ordinary product controls. The agent runs through native ACP. Independent checks use public APIs, actual downloads, owned process observations, and native lifetime fences. ## What Changed - Register one explicit-only case: `antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn`. - Map all 47 source steps and declare saved-account prerequisites, finite limits, and same-host macOS ARM64 scope. - Create an agent through the ordinary picker, connect the saved account, select Low, run Test, and wait for readiness. Save an instruction marker, deliver a file, and update it through an ordinary follow-up. - Require each run's visible completed file activity, exact downloaded bytes, committed Done result, same tagged conversation, distinct physical owner birth, released lease, and free native fences. Verify reload persistence. - Stop failed native runs that still own a retry. Replay an existing Stop with its durable request identity and wait for the matching audited acknowledgement. A terminal reservation without a native intent remains unconfirmed. - Keep private evidence in the existing result, report, dashboard, and accounting pipeline. Retain the paused QA agent and financial history while preserving the shared account and budget policies. - Journal only the ordinary wizard's future ID, target, name, intended harness and expected creator before forwarding its POST unchanged. Bind the response and interrupted recovery to exact identity reads; a pending 404 stays unconfirmed. - Serialize private resource snapshots through temporary-file fsync, atomic rename and directory fsync. Abort forwarding on storage failure and preserve legacy receipts. If later journal or diagnostic writes fail, retain the primary error, always close the browser, and attempt the ordinary failed report. - Add production-component browser controls and harden task-menu readiness, alias selection, scroll restoration, failure packaging, and one hermetic auth-test budget. - Wait for the existing GitHub recovery fixture to register its next deadline before asserting the original range; preserve its nested-transaction and zero-request checks. In the decision-expiry fixture, verify the exact saved expiry separately from the earlier global wake of another queue. ## Verification - Fresh registered acceptance passed on eval source `9060f3cd01b7c3445ecdf703d72e9cf30fe42b59` and merged serving product `61bd4ac6919c2309f766a25e62273b074cee3261`. It completed ordinary setup and two turns with native `.5` and `gemini-3.8-flash-low`. All 16 matchers, 14 checkpoints, and cleanup passed in **165,613 ms**. Actual downloads were **61 bytes** and **119 bytes**. Both turns passed their own run-bound browser activity checks and reload persistence. - Independent read-back confirms Done, a paused retained agent, the same connected saved default account, and no active owned run. Both accepted results have the same full tagged conversation and distinct primary process births. All 21 observed processes retired; both leases released and all three native fences were free after each turn. The selected `.5` checkpoints also showed equal agent/backend session IDs; this is an observed case requirement, not a generic ACP invariant. - Receipts report 339,273 input tokens, 7,224 output tokens, and 202,846 cached input tokens. Subscription prices are unavailable. Billing coverage remains incomplete, not free. - The normal report CLI passed. All 310 files from nine original attempts and two separate recoveries retained their hashes. The twelve current screenshots are byte-identical, and 194 local report links are contained and valid. The eight earlier reports were copied without regeneration. - Delivery source `1bf4fb04df04710b82844b3c9355767f46b5f68f` branches from `d3f2a60e795fc60d64c8b19d1eed4d48657d5492` and merges cleanly with current master `1d571de5f9f886d66e520800baec8fc2f67d0948`. All 41 prior `c8b332` authored file blobs remain identical; the 42nd path is the decision-expiry fixture contract correction. Creation ownership, the strict durable writer, and the ordinary successful workflow remain unchanged. Of 43 measured file blobs, 30 remain identical. The Antigravity oracle, artifact, billing and report gates and its suite-specific validation block remain unchanged. The actual retained agent matches its original creator; its original request `setupId` was not captured and is not reconstructed. - Later master changes include model-visible workspace guidance and rejected-terminal-report handling. Their bounded source assessment is separate: the first actual finish failed, then a same-turn replacement was accepted and committed. Later read-only results match both original native-result hashes, and the complete event catalogs have no `session.failed` event. Master #18228 also expands detection of explicit JSON file requests. The measured create/update-and-attach `.txt` task already required publication under both parsers; the publication validation gate itself is unchanged. Master #18240 changes remote workspace cache staging and adds an OpenCode index refresh. The selected local lane returns before those paths. Master #18213 adds optional numeric turn timing and trace boundaries. Its selected Antigravity model, text, IDs, result and retirement semantics remain unchanged in the bounded source review. Local diagnostic rows and awaits can change event ordinals and latency; the oracle checks relative semantic order. Collector coverage and latency improvement remain outside this proof. This does not claim that the whole current runtime is byte-identical to serving `61bd`, or relabel the measured result as a new run. - The adopted upstream cache controls passed 3/3 with pinned real Turbo. Forced work disables local and remote reads and writes; ordinary work reuses local results. The original genuine-pnpm/Turbo controls remain historical to `9060`'s earlier write-only policy. The duplicate cache patch is superseded by #18230. Its qualification-only pnpm addition is removed because the adopted upstream controls do not require it. - The E2E typecheck, map checks and 277-control replay are carried from the unchanged eval source paths. That replay passed, including the 183 Antigravity controls, catalog/connection checks and the separate product-journey validator. The carried creation controls include 91 config/recovery cases. The final launcher suite passed 19/19, including five real campaign finalization fault cases; the report suite passed 7/7. These final checks did not rerun the earlier 277-control group. The canonical production-component browser spec passed in 27.4 s under its unchanged 60 s limit before the replay. Four paused-composer cases and the product base's build/typecheck plus 376 focused regressions were verified earlier. - The shipped wizard and actual API client cover persistence before forwarding, lost response JSON, foreign/malformed requests, wrong response identity/creator and cleanup-error preservation. Real filesystem controls failed 5/5 before the writer repair and passed 5/5 afterward, preserving the old journal on partial-write, temporary-file sync and rename faults. Three persistent-sync campaign controls failed before the finalization repair and passed afterward. All five final campaign fault cases preserve the primary failure, cleanup failure, usage and failed report while attempting browser closure. The strict journal still blocks the next mutation. These are provider-free ownership controls; the Google task proof remains the original `9060` → `61bd` acceptance. - The decision restart check passed in the local 20-case fixture run. A current-source real-PostgreSQL rollback control reproduced the exact 90,000 ms earlier global wake: the old equality rejected it, while the exact durable expiry and new future-wake bound passed. The original 60 s open-state, exact-timestamp, continuation and quiet checks remain. The broader local run was **FAILED: 19 passed / 1 failed**, with a 15 s timeout and a 30 s cleanup timeout in an unchanged timestamp-pagination case. That case passed in canonical `c8b332` CI; the local timeout cause remains inconclusive. An extra local server typecheck failed against stale cached Runner declarations; it did not check this excluded test file. Both limits remain recorded for fresh canonical CI. - A broader local GitHub recovery group passed 21/22. Its unchanged pagination case observed an unexpected config request; the log does not establish the cause. That result remains retained, and fresh current-head CI is the final broader gate. - Exact delivery-head CI completed with 55 successes, two skips, and one optional neutral review. Required `ci / verify` and `ci / e2e`, build, typecheck, and Runner checks passed. Fresh Greptile is 5/5 with zero unresolved threads. Optional Paperclip is neutral 4/5; its carried recommendations are dispositioned against the public Stop contract and selected live identity evidence. Seven earlier converted attempts retain their failed grades and partial evidence. Historical Paid8 and its later read-only per-run UI observation remain separate from the fresh acceptance above. Unknown prices and the older partial startup accounting remain explicit. Full lineage and raw evidence remain private in the normal report bundle. ## Risks - This case requires an authorized saved personal default Google subscription and finite operator candidate admission. It does not test fresh OAuth or establish global qualification. - The physical observer supports a same-host loopback macOS ARM64 target. Cloud, Linux, and other models remain outside this case. - A known-cost stop cannot cap missing prices or in-flight subscription charges. Existing operator policies remain unchanged. - Raw account/native identities stay private. Missing retirement proof keeps resources retained. The selected cleanup mode does not test agent purge. ## Model Used OpenAI Codex (GPT-6). The host did not expose the exact model revision or context limit. The model used tools for code edits, tests, browser automation, and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
99d27fd5e7 |
feat(evals): add a metadata overview for the chat catalog (#18194)
## Thinking Path > - Paperclip is a control plane for AI agents that do work. > - Its evaluation reports separate recorded results from source coverage. > - A case without a main baseline measurement can still have a candidate run. > - A normal campaign card treats an expected execution without a result as missing evidence. > - Catalog planning data must not create those cards or change a recorded grade. > - This change adds a typed metadata view to the existing Evals dashboard. > - Readers can compare main outcomes, candidate execution, allowances and source review separately. ## Linked Issues or Issue Description **What existing behavior does this improve?** The existing Evals dashboard can show a campaign's retained results. It cannot show the full chat catalog as metadata without synthetic result entries. **Current behavior** An unused initial, an unknown allowance and a completed candidate run can all lack a main baseline result. Showing them as missing campaign artifacts hides these differences. **Proposed behavior** Add one validated metadata view in the existing renderer. Show one row per frozen scenario, separate execution and main outcome columns, filters, and the public request and expected outcome from static YAML. Keep normal campaign cards unchanged. **Reason and benefit** Readers can see which cases ran, which have no main measurement, and which still need preparation. Source review and candidate outcomes cannot add baseline pass credit. **Breaking changes** None to existing campaign inputs or outputs. The new metadata mode uses its own strict schema and requires a new private output directory. No matching open or closed PR or issue was found in the duplicate search. ## What Changed - Add a strict display-only catalog schema with closed enums and fixed catalog scope. - Validate the 244 row identities against static YAML without importing executable case callbacks. - Show main results, recorded execution, allowances and source review in separate columns. - Add search and filters, passing examples, qualification notes and expandable public test descriptions. - Add one dated immutable snapshot link in the existing shared report navigation. This does not change history data or update an already-published page. - Add a private metadata preview command to the existing dashboard owner. Keep existing campaign generation on its current path. - Keep allowance certainty independent of measurements and allow an optional later candidate outcome alongside a main outcome, without changing main counters. - Add focused pure, JSDOM and file-loader checks. Document the metadata command. ## Verification - The owning `catalog-overview.test.ts` Vitest suite passes all fourteen tests under the focused source configuration. The valid-main/unknown-allowance and main-fail/later-candidate-pass controls fail against the old renderer and pass with this repair. The never-run/unknown-allowance control fails against |
||
|
DottaandPaperclip
|
7f10cc0585 |
feat(evals): show full task conversations in E2E reports (#18055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests verify tasks through the application and retain reports. > - A final screenshot cannot show the complete task conversation. > - Debugging a result requires the earlier requests, replies, and answers. > - This pull request captures task conversations before fixture cleanup and adds them to Test details. > - Reviewers can read each task thread without a running test instance. ## Linked Issues or Issue Description **What existing behavior does this improve?** The task evidence shown in Product E2E campaign reports. **Current behavior** Most task results show screenshots and checks. Screenshots show the final visible state. They omit earlier messages and long replies. Only the first-task suite has a conversation view. **Proposed behavior** Every standard runner attempt captures the saved conversation before cleanup. Test details show full task messages, requests, answers, and native agent progress. Related tasks have separate threads. Failed attempts also retain available history. Partial captures are labeled. **Reason and benefit** Readers can debug a task by reading its conversation. They do not need to reconstruct it from a screenshot or private run logs. GitHub searches found no duplicate PR or issue for this change. This is an improvement to test reporting and does not overlap a planned core feature in ROADMAP.md. **Breaking changes** None. The result adds an optional `taskChat` field. Old results still render. A result without a task has no chat section. Older reports need retained conversation evidence before they can gain this view. ## What Changed - Capture comments through all pages, task briefs, visible request copy, recorded answers, and native progress before fixture cleanup. - Include delegated tasks and standing chats. Preserve available history when a read fails. Report capture limits explicitly. - Add a closed, credential-redacted conversation format to normalized results. Exclude reasoning, raw tool payloads, and provider session metadata. - Render task conversations inside Test details. Retain them through report aggregation and dashboard regeneration. - Add capture, publication-format, regeneration, and browser tests. Update the evidence and security contracts. ## Verification - Report tests: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts task-chat.test.ts first-task-report.test.ts report.test.ts history.test.ts` — 103 tests passed after review fixes. - Browser tests: `node node_modules/@playwright/test/cli.js test --config tests/runner-e2e/report-playwright.config.ts` — 6 tests passed after rebase. - Targeted TypeScript check of the new capture, renderer, validator, report, and browser-test files passed. - Tests cover a 1,105-message thread, a long final reply, event pagination, partial capture, company access, credential redaction, HTML escaping, and report regeneration. - Review regressions reproduced two failures before repair: the task-run API uses `runId`, and item reviews save verdicts in `result.items`. Capture now follows these contracts. Both regression tests pass. - Greptile reviewed commit `471236438e03e54cc4ad85de6653adf45835063c` at 5/5 with no new findings. The separate Paperclip review execution stopped without an assessment and ended neutral. Its cause is not established; this is not a completed Paperclip assessment. - Execution discovery passed for `core-compatibility.runner-codex.local.message-marker`. No paid provider run was started. - Full local commands were attempted. `pnpm -r typecheck` and `pnpm build` stop because the DB package cannot find `tsx`. `pnpm test:run` stops because the general-server lane cannot find `vitest`. These package dependency links are absent in this checkout. - [Remote CI on the final commit](https://github.com/paperclipai/paperclip/actions/runs/40728689517) passed: typecheck and release registry, all general and serialized test shards, Runner static/Rust/Vitest checks, build, canary dry run, all eight browser E2E shards, and final verification. Security and contributor checks also passed. - Open a new task-based result and expand **Test details**. Read **Task chat** and expand each related task thread. Results without tasks keep their existing layout. ## Risks - Published reports now include fixture conversation text. Capture retains only reviewed display fields and redacts credentials. Fixture authors must keep this text suitable for public reports. - Long threads increase report size. Collection has a 30-second request-admission budget and page limits. It labels incomplete evidence and does not change the workflow grade. - Old reports cannot recover conversation text from screenshots. New attempts must retain the evidence. ## Model Used OpenAI Codex (GPT-6). The exact serving snapshot and context-window size are not exposed in this session. Used reasoning, code execution, repository tools, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8fe5d5defc |
fix(evals): honor the shared feedback read deadline (#18026)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E evaluations must retain feedback from each paid attempt. > - The collector reads saved feedback and native tool events before teardown. > - A five-second request limit can stop a valid read before the shared read deadline. > - This pull request gives event reads the remaining shared read budget. > - The total deadline and reserved checkpoint time stay the same. ## Linked Issues or Issue Description **What happened?** A paid Ask attempt passed the product grade but retained partial feedback invocation coverage. The collector recorded `events_read_failed`. Its event endpoint later returned HTTP 200. The retained timing is consistent with a request that exceeded the five-second request limit. This change fixes that deterministic failure path; it does not prove the cause of every historical partial read. **Expected behavior** A valid read that takes eight seconds can complete within the 30-second collection deadline. A stalled read must still time out and leave time to save the available feedback. No read or paid attempt is retried. **Steps to reproduce** 1. Return a terminal native run and an empty saved-feedback list. 2. Make the event read take eight seconds and respect the supplied request timeout. 3. Run `collectAttemptFeedback` with a 30-second deadline. 4. Before this fix, the read times out after five seconds and invocation coverage stays partial. Related accounting and reporting work: Refs #18010. ## What Changed - Use the remaining shared read budget for event API requests. Keep the five-second cap for other reads. - Add regressions for an eight-second event response, a stalled event response that preserves saved feedback, and a stalled early feedback read that still permits the event scan. - Document the read budget, reserved persistence time, and no-retry behavior. ## Verification - Red: the initial two regression tests fail before the fix. The other 42 feedback tests pass. The early-read starvation review regression then fails against the first implementation (44 pass), and passes after narrowing the change. - Green: 58 tests pass across feedback collection, interrupted recovery, process shutdown, and scheduled finalization. - Full Product support run: 4,609 pass, three skip, and the retained embedded PostgreSQL bootstrap test fails on this host. The log remains available. This is not a full local pass. - Local Product typecheck fails on installed workspace export mismatches outside the changed files. Hosted CI must check the exact commit with clean dependencies. No local full repository build pass is claimed. - Original paid grades, partial feedback receipts, and accounting journals remain unchanged. This repair makes zero provider calls. ## Risks - A stalled event read can consume the remaining read budget. Other endpoints keep their five-second cap. The shared deadline still reserves up to five seconds for the final checkpoint. - A slow event page can leave later runs uninspected. Those runs remain partial or unavailable. The change does not treat missing evidence as success. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bd0d81c7b |
fix(e2e): follow proven workspace deferrals during onboarding (#18008)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify the user outcome through the browser and scoped APIs. > - An onboarding child can wait while its parent owns their shared workspace. > - The server records that no provider work started and schedules a durable replacement. > - The journey rejected that expected wait, and feedback counted it as unknown legacy execution. > - This pull request follows the exact replacement and records the distinct no-provider evidence. > - The child must still finish through native execution and produce the independently checked download. ## Linked Issues or Issue Description Refs #17940. **What happened?** The fresh-workspace journey failed with `onboarding_first_task_failed` when a child encountered shared-workspace contention. The cancelled admission had an explicit `workspace_wait` receipt with `providerWorkStarted: false`. Its linked replacement was still scheduled. The runtime had not been selected. The default `legacy` value did not represent a provider execution. **Expected behavior** Keep both rows and their run-limit cost. Follow only the unique replacement for the same company, agent, and issue. A pending replacement must not pass. Real failures and contradictory execution evidence must still fail. **Steps to reproduce** 1. Start a native onboarding parent and accept its child-task proposal. 2. Observe the child admission while the parent still holds their workspace. 3. Present the server's no-provider cancellation and its durable scheduled replacement to the journey and feedback collectors. ## What Changed - Add a shared predicate for the explicit no-provider workspace-deferral receipt. Reject runtime, process, output, usage, and cost evidence that contradicts it. - Follow exact, unique retry lineage during onboarding. Wait for active or scheduled replacements without repeating browser actions. Preserve the campaign deadline, run cap, native artifact attribution, and download oracle. - Retain per-run feedback inspection classifications: `native_events`, `provider_not_started`, or `incomplete`. Require exhaustive event reads. Keep pending work and uncertain execution partial. - Add causal regressions and document the evidence contract. Historical failed attempts keep their original grades. ## Verification - Red: the new journey regression failed with the observed `onboarding_first_task_failed` before the repair. The new feedback cases also failed before the repair. - Green: 70 journey tests and 74 feedback-family tests passed. The final affected journey/feedback gate passed 275 tests plus 137 Node checks. Five retry-lane reset cases failed before the review correction and passed after it. - `pnpm test:e2e:runner:typecheck` passed after the retry-counter correction. - `pnpm -r typecheck` and `pnpm build` passed before the final rebase. - The full local `pnpm test:run` and harness unit gate encountered embedded PostgreSQL startup failures. The harness gate passed 4,574 tests and failed its database-startup test. The final focused journey/feedback gate and all 137 Node checks passed. CI will provide independent full-suite results before merge. - Current-head CI passed 55 checks; two checks skipped and one was neutral. Greptile is 5/5, with zero unresolved review threads. - Live Cloud verification will use the reviewed harness and Operations revisions. This PR does not claim that a fresh Cloud campaign has passed. ## Risks A broad cancellation exemption could hide real failures. The exemption requires the server's explicit receipt, unresolved runtime selection, no contradictory execution evidence, and exact durable lineage. Missing successors remain pending until the existing deadline. A successful native child and valid downloaded bytes remain required. No server behavior, approval policy, or budget policy changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40c1ab36cd |
feat(evals): register explicit Daytona feedback persistence (#17986)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E evals check real server, browser, runner, and provider behavior. > - Feedback persistence already checks saved complaints and suggestions with exact bodies and run attribution. > - Its catalog registers only local cells, so a Daytona campaign cannot select the same workflow. > - This PR registers the existing workflow for the two native Daytona profiles and tests their gateway bindings. > - Remote cells stay explicit-only and unqualified until a bounded live run passes. ## Linked Issues or Issue Description Refs #17980 and #17975. No duplicate open PR or issue was found in the Daytona feedback search. **What existing behavior does this improve?** The `feedback-persistence` Product E2E suite and its explicit execution selectors. **Current behavior** At master `d54f776ca0`, the suite has only local Claude and Codex cells. Selecting the matching Daytona ID fails before admission. The gateway manifest already supports these native remote profiles. **Proposed behavior** Register four cells: local and Daytona for each native profile. Reuse the existing saved-body and completed-tool oracle. Require an immutable image and Daytona credentials. Keep one attempt, one expected work run, five-minute local and inherited 15-minute remote case timeouts, and the existing 60-minute provider creation TTL. **Reason and benefit** A bounded remote qualification can check both a simple task and real feedback persistence. Local evidence remains separate from remote acceptance. **Breaking changes** Explicit `--suite feedback-persistence` now lists four cells instead of two. `--all` still excludes the suite. Daily and weekly manifests still select their existing exact local IDs. ## What Changed - Register the two native Daytona feedback cells and increment the suite definition version. - Address the remote cold-start review: inherit the base task's 15-minute Daytona deadline and retain the five-minute local limit. The regression failed before the fix and now passes. - Check immutable-image requirements, credentials, lifetime, and shared gateway connections in existing contract tests. - Fix the config type in the TTL catalog test added by #17980. Keep its runtime lifetime assertions. - Document remote selection and its unqualified status in the existing eval guides. ## Verification - Before the catalog change, the registration regression failed: one failed and 16 passed. - Focused catalog, feedback, operations, ledger, manifest, and summary tests: 114 passed across six files. - Read-only discovery lists exactly the two Daytona feedback IDs. - The Node test phase passes separately: 137 tests. The retained Vitest database failure prevents the unit launcher from reaching that phase. - Full Product support suite: 4,523 passed, three skipped, and one embedded PostgreSQL bootstrap failure. The same database failure was reproduced on a clean baseline before this PR. - Fresh Linux Product typecheck passed at the review-fix commit: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=2 node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. The default run and the ineffective JavaScript-heap attempt exited 137 and remain retained. The compiler launches a native child process; its memory controls let the unchanged semantic check complete. Host typecheck still has 15 stale installed DB/Runner export errors; the two TTL test typing errors are fixed. All 58 latest-head checks are acceptable (success, neutral, or skipped). Both fresh reviews are 5/5, with no unresolved review threads. - The updated credential-free Linux controller discovers both planned remote IDs and launches Chromium 151.0.7922.34. Runtime build inputs are unchanged from the prior controller. The Daytona image content ID is still `b55f03dc9c633db300b70206a691ff45a8e779e56a3c546f9358e3282d3597c9`. - No model requests, sandbox allocation, or paid schedule changes. ## Risks - Selecting the whole explicit suite now includes remote cells and requires Daytona configuration. Use exact IDs for a bounded paid campaign. - Registration and contract tests do not prove live remote behavior or token accounting. Remote qualification remains pending. - The shared dependency checkout limits the local host typecheck. The fresh Linux semantic check passed. Earlier compiler OOM failures and the database bootstrap failure remain retained. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d54f776ca0 |
fix(daytona): bound ephemeral eval lifetime at creation (#17980)
## Thinking Path
> - Paperclip manages agent work through Runner and sandbox providers.
> - Product E2E runs paid work in disposable Daytona sandboxes.
> - A controller deadline cannot stop remote compute after the
controller crashes.
> - Inactivity cleanup does not establish an independent wall-clock
lifetime.
> - Daytona supports a lifetime in the creation request.
> - This change exposes that option in the Daytona driver and verifies
the provider expiry.
> - Ephemeral evals can retain a finite lifetime even when their
controller is unavailable.
## Linked Issues or Issue Description
Refs #17947 and #17978. No duplicate creation-lifetime PR was found.
**What happened?**
The ephemeral Product E2E fixture configures inactivity stop, archive,
and delete intervals. Ordinary task leases do not supply a requested
expiry. A crashed controller therefore has no confirmed wall-clock
sandbox lifetime for its cost bound.
**Expected behavior**
Each ephemeral eval sandbox must carry a provider-enforced lifetime from
creation. The driver must reject invalid bounds and reject a sandbox
whose provider expiry is missing, expired, or later than the configured
remaining lifetime.
**Steps to reproduce**
Acquire a Daytona lease from the ephemeral fixture. Inspect the creation
request and returned expiry. Before this patch, the request has no
lifetime and the lease has no expiry unless its caller supplies one.
**Paperclip version or commit**
Current source based on master at
|
||
|
DottaandPaperclip
|
6987c2f20b |
test(upgrades): qualify controller replacement with immutable artifacts and phase evidence (#17455)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/105/changes/paperclip/105 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip needs evidence that an admitted execution survives a main-server replacement. > - Runner mechanics and server adoption belong to #17097 and #17439. > - A healthy HTTP endpoint does not prove recovery of the original execution. > - Qualification must identify the serving artifact, natural ownership, authenticated attachment and resumed processing separately. > - It must also retain failed runs and measure normal-operation costs. > - This PR extends the existing qualification machinery and acceptance record. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operators can assess upgrade continuity from phase evidence tied to exact artifacts and profiles. **Current behavior** A same-artifact restart or synthetic executor pass cannot establish different-artifact upgrade and rollback support. Idle SQL gates also do not measure active-run capacity. **Proposed behavior** Use immutable A/B artifacts and strict phase receipts through the existing supervisor. Preserve failed evidence and add measured before/after workload costs. Builds on merged #17097 and #17439. The base is now `master`. #17456 follows for review convenience. This PR lands qualification machinery and the evidence record. The remaining hosted gates stay open for deployment qualification; merging this machinery does not claim those experiments passed. ## What Changed - Extend the main-server supervisor and installed-artifact verification for actual serving revisions, forward migrations and compatible rollback. - Distinguish HTTP readiness, natural ownership, authenticated attachment, resumed processing and backlog settlement. - Retain the original reconnect grace and execution identity checks. Never substitute a new task or edited lease for continuity. - Reuse real-harness and Daytona qualification paths, with exact profile and artifact provenance. - Bind current Copilot attestation inputs in the Daytona image fingerprint. - Keep one acceptance record, including both failed file-only stress phases and the final passing bounded-pool comparison. ## Verification After retargeting to master, all 158 focused qualification cases across 13 files pass, with E2E typecheck. The first 12-file batch has four report-lock test failures and 133 passes. Master #17811 had committed the exact already-approved output lock, while that fixture assumed HEAD must be the uncorrected input. The fixture now checks both pinned forms, executes the committed-lock path, and reconstructs the pinned input to retain all rejection tests. The production helper, policy hashes, patch and dependency resolution are unchanged. The corrected batch explicitly includes both process-preservation suites; the prior 146-case receipt has its own source and selection. Current candidate `8f1d6ec02f` integrates master `09ad9adce2`, including the merged server slice. Its diff contains only qualification code and documentation. The [qualification record](https://github.com/paperclipai/paperclip/blob/8f1d6ec02f5bf6c90e7b27da8156953cabd9a180/doc/plans/2026-10-22-hot-upgrade-qualification.md) is the acceptance ledger and retains source-attributed failures and hosted limits. On the eighth integration, all 146 cases across 12 focused qualification files pass, including report-lock preparation, workflow security and workflow targeting. E2E typecheck passes. The earlier 118-case result remains attributed to its prior source. The first sandbox invocation has 117 passes and one failure because process inspection was denied during inert fixture cleanup. The permitted repeat passes all 118 and a targeted process check finds no remaining fixture daemon. The earlier 96-case receipt remains source-attributed in the ledger. The latest two review regressions first fail: an intermediate replacement can cite another runner/session despite a valid final step, and malformed receipt JSON aborts evidence packaging. Each replacement now binds run/company/issue/runner/session, and corrupt receipts retain explicit invalid metadata plus sanitized diagnostics while other logs and receipts still package. These checks prove qualification machinery with injected supervisor operations, configuration subprocesses and real filesystem packaging. They do not qualify a hosted harness. Three request-reader cases, 14 artifact/configuration/cache/history cases, and three lost-partial-receipt cases fail before their respective corrections. The final collector preserves sanitized startup, readiness and recovery phase receipts through packaging and temporary-root deletion. Earlier embedded-Postgres initialization, report-catalog drift, CI scheduling/cleanup failures and fixture collisions remain recorded with their corrections or limits. The previous Greptile service error is retained as historical review evidence, not a current review pass. The record includes prior source `9bb9d3417e`'s actual Linux contention/fairness successes and the ordinary-warm regression proof from the server slice. The historical 300-session component campaign passes four after phases and retains earlier burst failures. Those component costs do not establish hosted capacity. The hosted API-only pair has zero agents/viewers, fails both absolute latency gates and the candidate memory gate, and does not complete the active-agent/viewer matrix. The strict ten-agent same-artifact restart remains attributed to its original artifact. It does not qualify this updated stack. Actual A→B/B→A, the normal drain-free Cloud rollout, and other hosted harnesses remain open. No new provider turn, deployment or hosted qualification was run during merge preparation. Prior source `57d3767eac` jobs 122590643944, 122590643983 and 122590644161 ended after runner shutdown/cancellation without final Vitest results. The shutdown cause is unproven. No retry was issued before this source was superseded by the required server recovery fix. These attempts remain failures/incomplete evidence. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and normal merge eligibility remain required. ## Risks Different-artifact A→B/B→A, the normal drain-free Cloud rollout path, other hosted harnesses and the complete concurrency/viewer matrix remain open. Frozen branch builds are development-build evidence. This PR owns qualification and evidence; production defects must be fixed in their earlier owning slice. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a8bd012635 |
fix(evals): apply reviewed lock corrections to reports and qualification images (#17937)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/98/changes/paperclip/98 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages agents and records their work. > - Product E2E campaigns measure real agent behavior and model spend. > - A paid hosted run completed both cases and all ten model receipts. > - Its report job could not install dependencies from the committed lock. > - The trusted checkout needs a reviewed dependency correction. > - This change applies a reviewed lock correction before the frozen report install. ## Linked Issues or Issue Description Refs: #17928 **Description** The report in [run 40576181788](https://github.com/paperclipai/paperclip/actions/runs/40576181788) failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Both paid cases had passed. The worker and build jobs already restored the resolved source lock, but the report did not. **Steps to reproduce** Dispatch the bounded daily campaign on the default branch with a stale committed lock configuration. Workers use the resolved lock. The report uses the stale lock and fails before report generation. **Expected behavior** The report installs frozen dependencies from a verified lock for the exact trusted source. Evidence can be reported without repeating paid work. ## What Changed - Apply a reviewed lock correction from the trusted report checkout before its frozen install. Use the same correction for the qualification image build, which also requires a compatible frozen lock. - Pin the original lock, correction and output SHA-256. The trusted authorization job rejects unknown inputs before any paid work. - Apply the same correction to trusted manual history publication. Qualification image builds apply it only when their target equals the workflow revision; other targets keep their own lock. - Update existing Claude ACP and postgres patch identities and link the declared yaml dependency. Existing package tarball integrity entries stay unchanged; the declared yaml dependency adds exactly yaml@2.9.1 with pinned integrity. - Keep runtime registry resolution and worker target locks out of the credential-bearing report. - Test valid correction, idempotence and changed-lock, changed-patch and wrong-output rejection. ## Verification - Focused workflow and lock-correction tests: 36 passed. - `actionlint .github/workflows/runner-full-stack-e2e.yml` passed. - Original worker evidence: 2/2 passed, ten final model receipts, zero unresolved model requests. - The original report was regenerated from retained evidence with zero provider calls and saved privately with replay provenance. - Broader repository checks run in CI. Prior full Product checks passed for #17928. ## Risks The correction is tied to the reviewed committed lock. Dependency input changes need a reviewed correction update. No new registry version or lifecycle script can enter the privileged report through this fix. The original failed workflow stays failed. No root lockfile is committed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d40f9ac377 |
fix: enforce first-task completion decisions and retain E2E evidence (#17940)
## Thinking Path > - Paperclip lets people delegate work to agents and approve proposed work. > - The onboarding first task asks for scope, proposes a task, and waits for a human decision. > - An answer to its opening question supplies scope. It does not approve a proposal. > - Task creation already checks the saved proposal decision. > - Native completion used the general completion policy and could mark the onboarding parent Done without that decision. > - This PR checks the trusted decision before completion feedback and again when the native status is committed. > - The first task stays in review until a saved human decision permits closure. Ordinary completion and acknowledged refusal keep their current behavior. > - The Product E2E guard also retains closed scalar evidence so future failures show which boundary condition was observed. ## Linked Issues or Issue Description Refs: #17914. Profile compatibility follow-up to #17922. **What happened?** A native onboarding run can submit a valid low-risk Done result after only the opening answer. The native finalizer then marks the first task Done even though no human has accepted or rejected a proposal. **Expected behavior** The opening answer must not permit completion. A trusted saved proposal decision must permit it. A saved rejection permits acknowledgment and closure, but does not authorize new work. A later pending proposal must keep the task in review. **Steps to reproduce** Create a server-owned onboarding first task with an answered opening question, no proposal decision, and a valid low-risk completion contract. Pass a Done result through native completion feedback, the control-plane port, and the finalizer. Before this fix, the persisted status is Done. The regression expects In review. Ordinary completion and acknowledged rejection pass as controls. ## What Changed - Share the trusted latest-proposal and human-response checks with completion admission. Keep task creation acceptance-only. - Refuse premature native completion feedback. Keep an already saved premature result in review during finalization. - Check the decision again under the existing task and run locks before committing Done. - Cover accepted decisions, saved refusals, forged context, pending or changed proposals, and a real database race. - Retain a closed scalar receipt when the existing E2E pre-acceptance guard fails. Validate and display it without task content, IDs, filenames, or raw rows. Preserve the original failure, blocked dependents, and historical reports. - Include the evidence module in the journey definition hash. Bind the receipt to its exact failure code and check its closed keys against the full type. - Repair base-branch profile drift found by the full harness check: include the active Copilot v66 fixture in image hashing and update stale Pi version/hash expectations from #17922. Keep all older image inputs and strict hash checks. - Update the onboarding feature and agent access maps and E2E evidence documentation. ## Verification - Semantic regression before implementation: one failure and two passing controls on PostgreSQL 18. The failed assertion observed Done instead of In review. - The local database uses an owned disposable container with an immutable image digest. Actual migrations, feedback, control-plane port, finalizer, and status commit run against PostgreSQL. A temporary test adapter replaces only the embedded database transport because this host has exhausted its PostgreSQL semaphores. - Native control-plane suite: 55 tests passed, including 17 new controls. Existing task-creation admission: 25 passed. Existing completion feedback: 19 passed. - Credential-free E2E support: 206 Vitest suites passed, 4,407 tests passed, and two skipped. All 137 Node support checks passed. Harness typecheck passed. - The failure-receipt regression had 11 failed assertions before capture was implemented. Two failure-code binding assertions fail without their gate. Eight placement/binding cases first validate the receipt-free control, then reject the misplaced receipt. - The untouched base reproduced the three profile compatibility failures. The narrow repair passed all 83 affected tests before the full harness run. - Private workflow preparation and credential-free onboarding preview: nine compatibility checks passed. Ownership, recovery, evidence, shared-contract and workflow compatibility: 77 passed. The actual provider preview also confirms immediate success deletion, earlier declared failure eligibility, limits, and zero retries without credential access. Map checks and git diff checks passed. - Required repository typecheck passed for the product fix. The combined branch build passed. The local full repository suite was stopped after repeated PostgreSQL startup failures; a separate initdb probe confirmed host SysV semaphore exhaustion. No shared IPC resources were changed. [Hosted CI](https://github.com/paperclipai/paperclip/actions/runs/40591020873) passed the full database suites, typecheck, build, runner checks, serialized suites, and browser shards on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b` (55 success, two expected skips, one advisory neutral). - Independent source review approved the product guard, atomic race regression, evidence contract, and maps with no actionable findings. Both automated reviews are 5/5 on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b`; prior findings are fixed and resolved. Final CI is green and the PR is conflict-free. - No Cloud workspace, provider call, release publication, or deployment is part of this verification. These tests do not identify a historical Cloud failure or qualify live onboarding. - Local dependency installation uses the existing CI-generated lockfile from #17811. No lockfile change is included. ## Risks - Existing onboarding runs without a trusted saved proposal decision remain in review even when the agent reports success. This is the intended approval boundary. - Decision changes can race completion. The status commit checks the decision under its existing locks and uses the existing status-race recovery path. - A saved refusal permits closure only. Creation admission still rejects it. - Evidence readers reject malformed receipts and unrelated failure codes. The version-1 key registry is checked against every receipt field at compile time. Existing attempts do not gain evidence after the fact. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), ultra reasoning, through Codex. Used repository inspection, code execution, and collaborating coding agents. The runtime did not report a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bf3ef12f3a |
fix(evals): qualify model spend separately from operating costs (#17928)
## Thinking Path > - Paperclip manages agent work and its budgets. > - Product E2E campaigns must explain each model request and attempt. > - The current completion gate also requires complete operating-cost evidence. > - Long-lived observability and storage charges can block a campaign after its model receipts are complete. > - An operator can now choose model-request readiness in a retained allowance. > - The report keeps full accounting, operating holds and model readiness separate. ## Linked Issues or Issue Description Refs #17889, #17523, #17452. **What happened?** A hosted campaign could pass both product cases and retain complete model receipts, yet fail readiness because operating charges were unresolved. **Expected behavior** An explicit model-readiness policy permits complete model receipts to qualify a run. Unknown operating charges stay visible and reserved. Missing model receipts and budget violations still block qualification. **Steps to reproduce** Use a campaign with four unresolved operating reservations and one completed model request. The regression checks readiness before and after the final model receipt, then checks an operating overrun. The default policy retains the original full-accounting gate. ## What Changed - Add the explicit `accountingReadiness: "model_requests"` allowance policy. Bind it into operating-bound configuration when present. - Derive a model subtotal and receipt-coverage projection from the retained journal. Show it beside full accounting and retained holds. - Use the selected readiness policy in scheduled finalization. Keep product, cleanup, feedback, trace and budget checks. - Refresh two existing eval prerequisites found by the full unit suite: the active Copilot image identity and the model-account catalog mapping. - Document the policy and its limit: it does not certify a full-lifecycle operating-cost ceiling. ## Verification - Regression failed on the old implementation with `Invalid operating overhead fields` and passed after the change. - Focused accounting/operations/finalization checks: 54 tests passed before the final policy assertions. - Existing image/catalog failures reproduced, then their 12 checks passed after the metadata repairs. - Product E2E unit suite: 4,360 passed and 2 skipped, plus 137 Node checks. Product E2E and workspace typechecks and workspace build passed. - Review regressions: 35 focused accounting/finalization checks passed on |
||
|
Dotta
|
051b425fdf |
fix(evals): price explicit response model identities (#17808)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E operations reserve model cost before each provider request. > - Providers can return an exact snapshot name after a request selects a model alias. > - The estimator required the response name to equal the request alias. It left valid snapshot usage unpriced and invalidated its reviewed bound. > - This pull request lets a versioned pricecard declare a small list of exact response names that share its reviewed rates. > - The benefit is correct cost coverage while request selection and unknown-model handling stay strict. ## Linked Issues or Issue Description Refs #17765 for the shared eval gateway request policy. **What happened?** An eval request can select `gpt-5.4-mini` and receive `gpt-5.4-mini-2026-03-17`. Even with complete usage, the estimator returns `null`. The gateway also rejects that response identity as evidence for its reviewed reservation bound. **Expected behavior** A pricecard can name exact response identities with the same verified rates. Unlisted snapshots stay unknown. A declared response identity must not become an allowed request identity. **Steps to reproduce** 1. Configure a route with an alias pricecard and an exact response snapshot. 2. Return complete Responses usage for that snapshot through the gateway. 3. Observe that the old estimator leaves the charge unknown and invalidates the bound. **Paperclip version or commit** Reproduced in the shared estimator and gateway at `507dc70be26c191e372f09e58dc0965e73bf8f57`. **Deployment mode** Other: the credential-free Product E2E operations harness. The same library serves attached deployment targets. ## What Changed - Add optional `responseModels` to versioned eval pricecards. Validate one to sixteen unique exact additional identities. - Reuse one response matcher in cost estimation, reviewed bound checks and supported request-policy cache-write attribution. Cache attribution also requires the reviewed provider and request identity to match. Keep exact request-model admission unchanged. - Admit the field in the strict operations manifest. Include it in the existing whole-price bound digest. - Cover fragmented JSON and SSE forwarding, persisted usage, durable ledger replay, partial responses, unknown snapshots, invalid mappings, stale bound evidence and unchanged request admission. - Document mapping provenance, version changes and preservation of historical ledgers. ## Verification - Focused provider usage, gateway and operations tests: **185 passed, zero skipped** on the cache attribution follow-up. The exact snapshot assertions failed before the initial repair. Three positive cache-write snapshot tests also failed before the follow-up and now pass. - `pnpm test:e2e:runner:typecheck`: passed after building required workspace declarations. - `pnpm test:e2e:runner:unit`: 4,311 passed, two skipped and four failures. The same four failures reproduce on unchanged base `507dc70` (79 passed, four failed in the two affected files): two Daytona Copilot image identity assertions and two Pi catalog/profile expectation assertions. This PR does not change those files or their source inputs. - `pnpm -r typecheck` and `pnpm build`: passed again on the exact cache attribution follow-up commit. - The private Cloud consumer passed 57 offline compatibility tests again on this exact follow-up source. These cover the declared snapshot estimate, exact request selection, unknown snapshot and incomplete usage handling, ledger holds, and source export admission. No provider calls were made. - `git diff --check`: passed. - `pnpm test:run`: interrupted after repeated embedded PostgreSQL startup failures in existing native-session and other database suites. The failed attempt is preserved and does not count as a full local suite pass. Only this checkout's verified test process group was stopped; it left no processes running. The first sandboxed workspace typecheck could not open a local tsx IPC socket; the later authorized check passes. An initial runner typecheck also lacked fresh-checkout workspace build declarations; its later run passes. - All forwarding tests use local synthetic response bytes. No provider calls, releases, deployments or paid eval campaigns were run for this repair. ## Risks - An incorrect mapping can apply incorrect rates. The pricecard author must verify provider provenance and change the price version. Validation bounds the list and uses exact matching only. - Existing pricecards remain exact-name only when the new field is absent. Adding or changing a mapping invalidates old reviewed bound evidence. Historical ledgers and receipts are not rewritten. ## Model Used OpenAI Codex, GPT-6 family. The exact runtime deployment ID and context window are not exposed in this session. The agent used reasoning, source editing, terminal commands and credential-free tests. No model inference was used by the eval harness for this repair. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
2ee36a4c0a |
test: add chat feature scenarios and real task workflow evals (#17794)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat lets a person assign work through a conversation. > - Feature and API maps do not prove that a natural request produces the right saved result. > - Product E2E already owns isolated fixtures, browser runs, evidence and cleanup. > - This pull request adds a chat scenario catalog and two real task workflows in that framework. > - Independent checks verify the task, its author, the saved output and the reply in chat. > - The benefit is a small executable foundation for broader chat coverage. ## Linked Issues or Issue Description **What existing behavior does this improve?** It adds real task creation and delegation coverage for Agent Chat. **Current behavior** Existing maps describe features and agent access. Existing chat evals cover specific protocols and handoffs. They do not connect a broad human-request catalog to independent saved-state checks. **Proposed behavior** Keep 244 planned scenarios across 35 feature areas. Promote two requests into the explicit-only `chat-features` suite. One saves a backlog task without starting work. The other delegates an announcement and returns the saved result to the original chat. **Reason and benefit** A successful tool call or confident reply does not prove the requested result exists. These evals check public API state, attribution and browser access after a real model run. **Breaking changes** No API or data schema changes. Chat instructions now require actual relative task links. The local E2E private-state scanner recognizes Codex account ownership metadata while continuing to reject credentials and model-visible identity leaks. **Additional context** Related prior work: #13969 and #13762. This improves verification of the existing Agent Chat roadmap item. It does not add a new core product feature. ## What Changed - Add YAML scenarios with capability IDs, starting context, user requests and expected outcomes. Validate all area files against their schema, global ID uniqueness and exact feature-map capability coverage. - Add two local native Codex subscription cells. Send the exact catalog request once through Chromium. - Seed only prerequisites through public APIs. Use shipped agent instructions and explicit disposable budget policies for unpriced subscription usage. - Check task ownership, creation audit, no premature execution, current worker-authored output, automatic completion delivery and usable links after reload. - Keep private Codex account ownership metadata distinct from bearer credentials. Scan rollout message content, all tokens and published evidence as before. - Teach normal chat instructions to use relative task links instead of invented hosts. Update tests, maps and the runbook. - Repair existing E2E contract drift: include the active Copilot identity in the image digest and update Pi profile expectations. Keep global catalog cardinality in its owning catalog test. Product E2E owns the new flow and grader. Its existing launch and report paths remain in use. The shared scanner change is limited to owner-private temporary Codex runtime metadata. Other callers retain the full secret scan. No UI, endpoint, schema or permission behavior changes. ## Verification - `node scripts/check-maps.mjs` passed: 35 areas and 191 capability IDs. - Product E2E typecheck and focused tests passed before rebase: 225 Vitest tests and 137 Node tests. Conversation service tests passed 40/40. Five existing runner turn-binding regressions passed. - Historical live measurement used real Chromium, server, database and native Codex with no automatic retries. The initial same-definition batch passed 5/6 cells. Backlog passed 3/3; delegation passed 2/3. One CEO startup failed because the runner did not receive a provider turn identity. The cause is unresolved. - An unchanged follow-up passed delegation three times and backlog twice. Combined evidence is backlog 5/5 and delegation 5/6. Each attempt passed cleanup. Later passes do not erase the earlier failure or prove a runner repair. - Earlier failed revisions exposed an invented task hostname, private account metadata false positives and missing funding for unpriced subscription usage. Those led to the specific changes above; the failed records remain retained locally. - The runbook records campaign identifiers and definition provenance. Generated result artifacts are not committed. Historical live results predate this branch rebase; fresh PR-source live evidence and final validation are recorded below. - Full workspace typecheck and production build passed on the rebased branch. Product E2E typecheck passed. - The full Product E2E support suite first reproduced four baseline failures: two missing active Copilot image-identity inputs and two stale Pi/catalog expectations. The narrow contract fixes above resolved them. The full rerun passed 4,282 Vitest tests plus 137 Node tests, with two skipped. Image checks verify mutation changes the hash, unchanged input stays stable, and an input closure without Copilot is unaffected. Original failed logs remain retained. - The repo-wide local Vitest run has reported two failures in unchanged service files. The Workspace connection test passed an isolated recheck and its full file passed 469/469 unchanged. The recovery test hit a 15-second timeout; its full file passed 89/89 unchanged. The run later hit explicit PostgreSQL `semget` resource exhaustion and was interrupted after those observed failures; no final aggregate counts or full local pass are claimed. Its complete available log is retained. No assertion or timeout was weakened. Matching remote server CI passed on the same source. - On clean PR head `39ad80b523ce194c90aafafd077d48dd9a36b4c5`, backlog passed in `local-2026-10-26T19-24-13-014Z`. The parallel delegation fixture failed during PostgreSQL initdb, before browser or model execution; its failed cleanup grade is retained. A disposable standalone initdb probe then passed. No semaphore sets or kernel settings were changed before that probe/recheck. After confirming no remaining fixture process, its private temporary files were removed. - One separate delegation recheck passed in `local-2026-10-26T19-29-24-800Z`, with one attempt, zero automatic retries and successful cleanup. Both passing current-source cells use definition `3f2c9ba2d03e3dcddde47ddab3b43bc52de977f25e256fceae0a24c9128e204b`. The earlier bootstrap failure and the historical provider startup failure remain distinct. These local results omit automatic source-SHA metadata; the clean checkout SHA is recorded here from the execution session. - The later broad local run reported `could not create semaphores: No space left on device`. A bounded cleanup removed 32 current-user PostgreSQL sets with 17 members, marker 537, a dead recorded creator and zero waiters. No live process or kernel limit was changed by that cleanup. The test process group was subsequently stopped; no broad rerun is planned. This direct later resource failure does not prove the precise cause of the earlier live initdb failure. - Review follow-up adds only static catalog validation and documentation of the existing five-minute journey bound. The executable definition is unchanged. Focused catalog/oracle tests passed 28/28; no redundant live run is claimed. - Final head `30fcb36ac9ae1f0270a589dbfafcec91ca6a03f3` passed all required and underlying CI checks in [run 40391402347](https://github.com/paperclipai/paperclip/actions/runs/40391402347), including `ci / verify` and `ci / e2e`. Greptile and Paperclip Review both scored this head 5/5, with all review threads resolved. The static follow-up leaves every executable definition input byte-identical to the live-tested source at `39ad80b`. Normal squash auto-merge is enabled. Actual merge is pending eligible code-owner approval for the owned dependency manifest; no protection is bypassed. ## Risks - Live model behavior can vary. One missing turn-start event remains unexplained; this PR does not claim provider reliability or broad catalog qualification. - Subscription costs are unpriced. Explicit fixture funding admits that usage. Run counts and deadlines remain bounded, but the dollar limit cannot cap unknown cost. - Account identity exceptions apply only to known private Codex runtime files. Token reuse, unknown files and model-visible rollout content remain blocking findings. - Deterministic checks establish saved state, output structure, required brief facts and access. They do not score all prose quality. Initial samples also require human inspection. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, browser and repository tools. The precise model revision and context window are not exposed to this session. Live eval agents used `gpt-5.6-sol`; that is the evaluated model, not a claim about the authoring model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run local tests; focused suites pass and the interrupted full-run failures are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aec86c794b |
fix(runner): preserve native work through controller outages (#17594)
Keep remote native work under its authenticated connection lease through controller outages. Hold durable ACPX tools through reconnection, align the native Claude HTTP MCP client budget, and wait for the recovered handler before dispatch. Preserve Stop, cancellation, exact identity and artifact checks. Verified with real-process regressions and a real Claude Opus 5.5 Daytona workflow: 125 seconds offline during useful work, original process and lease retained, no failed native tool calls, queued follow-up completed, independently verified ZIP and committed workspace receipts. All current-head CI gates pass and Greptile is 5/5 with all findings resolved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
970127a022 |
test: retain onboarding Connect failure evidence (#17529)
## Thinking Path > - Paperclip manages agents and their work through normal company controls. > - Product E2E journeys check these controls through the browser and public API. > - Cloud onboarding uses one Connect action to validate the environment and hire the first agent. > - A failed Connect action previously left only a general journey error in the report. > - This pull request retains a closed diagnostic receipt for that action. > - The report can show the observed failure boundary without retaining private response text. ## Linked Issues or Issue Description **What existing behavior does this improve?** Cloud onboarding Product E2E failure evidence and report detail. **Current behavior** A validation or hire failure can reach the journey as a general operation failure. The report does not retain which Connect response failed. **Proposed behavior** Observe only the same-origin, same-company validation and hire POST requests made during one Connect action. Retain a closed receipt with the checkpoint, response state, bounded HTTP status, and allowlisted adapter, outcome, check code, level, and server code. The Get started control and saved-agent readback remain required. **Reason and benefit** The next failure can be attributed to the exact observed browser/API boundary. This change does not claim the cause of a historical failure. **Breaking changes** The new receipt is optional. Historical results remain valid. Explicit validation and hire rejections now stop the journey with a precise closed error code. This changes only E2E source; it does not change product behavior or qualification gates. ## What Changed - Add a bounded Connect observer with exact request matching and listener cleanup. - Retain known server codes from `details.code` or `code` without response messages, hints, headers, request bodies, provider text, or credentials. - Validate the receipt as a closed schema and render its allowlisted fields in existing report detail. - Include the new helper in the journey definition hash. - Add route, privacy, malformed-body, deadline, late-response, delayed-failure, success, and historical compatibility tests. Document the evidence boundary. ## Verification - Focused Product E2E tests passed: 3 files, 58 tests on the review-fix head. - E2E typecheck, workspace typecheck, and full build passed on the preceding implementation head. The review fix changes only report labels, checkpoint timing, and the matching tests. - The local full `pnpm test:run` command stopped progressing at the general-server start header. Its owned session was interrupted. This is not recorded as a pass. - All 58 remote checks completed on the final review-fix head: 55 passed, one neutral, and two expected skips. Both Greptile and Paperclip reviews are 5/5 with no unresolved review threads. - No live campaign was run with this patch. The observed historical failure is preserved. ## Risks - Unknown or missing responses remain explicit diagnostic states. They do not establish a product root cause. - Response body reads stop after five seconds or the action deadline and accept at most 64 KiB of JSON. Checks are capped at 32 retained entries. - New server or check codes appear as `unrecognized` until the closed allowlist is updated. - The production launch classifier, native gateway qualification, and Get started requirement are unchanged. ## Model Used OpenAI Codex (GPT-6 family), with code execution and tool use. The host does not expose the exact model ID or context window to this run; these values are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6d46851920 |
test(e2e): add attended GitHub review bot journey (#17457)
## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E checks verify real user paths through those agents. > - A recorded GitHub review-bot walkthrough had no repeatable registered case. > - App setup, automatic triggering, publication, and task completion need separate evidence. > - This pull request adds an explicit attended journey to the existing catalog and report tools. > - Reusable setup profiles and independent assertions preserve the same outcome across compatible targets. ## Linked Issues or Issue Description Refs: #17262. This case remains an explicit attended attached-target flow. It does not replace the shared deployment-target or paid-operations framework. **What existing behavior does this improve?** The Product E2E catalog can now select and retain evidence for GitHub review-bot setup and its first automatic review. **Current behavior** Recorded walkthroughs cannot be rerun as a fixed registered case. A published review alone can hide a failed run or unfinished task. **Proposed behavior** Create and install an owned App, save automatic new-PR settings, create a defective PR, and require correlated review delivery, an inline finding, a consistent check, a successful run, and a Done task. Retain failures and confirm owned-resource cleanup. **Reason and benefit** The case makes a useful external integration reproducible without treating setup, unit checks, or report rendering as proof of a completed live journey. ## What Changed - Add two explicit attended cells for an existing configured agent in local or Daytona execution. - Declare hosting, account, connection, and credential prerequisites through private setup bindings. - Add fixed browser controls, ownership checks, independent grading, and cleanup. - Package declared screenshots through the existing sanitizer, manifest, validator, and dashboard. - Reject operations-managed execution until budgeted routing is qualified. - Document hosted QA credential setup and evidence limits. ## Verification - Focused catalog, oracle, report, connection, and ownership checks: 127 tests passed. - Inherited-environment preflight regression failed against the previous commit and passed after the fix. Actual-run environment verification remains required. - Harness TypeScript check and full workspace typecheck passed. Full build passed. The local full-test attempt was interrupted after repeated embedded PostgreSQL startup failures; its exit code of zero does not establish a pass; Exact-head Linux CI passed the full test, typecheck, build, Runner, serialized, browser, and canary gates on `abbddeade848edefc8de75f11cc9008b2cd52764`. - A real hosted attempt published a rated review and inline finding. It failed when its native session timed out before the task reached Done. Cleanup passed. Full live qualification remains open. - Credential scan found no known credential values in the authored files or retained report corpus. ## Risks - The full case creates external test resources and uses the configured provider. It requires an explicit selection and current task authority. It is excluded from default and CI matrices. - Time and run limits are not a hard dollar cap. Missing dollar cost stays unknown. Fixture setup does not weaken budget policy. - Fresh personal OAuth, unattended authentication, runtime skill-read proof, and cross-host qualification are separate claims. - The selected live fixture needs environment/lifecycle recovery before it can adopt a longer provider deadline. ## Model Used OpenAI GPT-6 family through Codex, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, including the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open findings - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
87bf89756e |
fix: support Daytona runner upgrades and ignore obsolete Claude options (#17361)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Existing Claude and Codex agents can upgrade to Paperclip Runner. > - A Daytona computer needs platform qualification before Runner becomes its default. > - The upgrade notice required that default, so it hid the action that could start qualification. > - Saved Claude turn limits also caused conversion to fail even though Runner can run without them. > - This pull request verifies the computer on click and ignores obsolete Claude options with a warning. > - Agents retain their model, account, environment, instructions, and permission policy. ## Linked Issues or Issue Description Related: #17122. **What happened?** A legacy Claude agent on a Daytona snapshot had no upgrade button until its environment had platform qualification. A saved `maxTurnsPerRun: 1000` then caused conversion to return HTTP 422. **Expected behavior** The upgrade action verifies a supported Daytona computer and converts the agent. Unsupported tuning options produce a visible warning and do not block execution. **Steps to reproduce** 1. Create a legacy Claude agent with `maxTurnsPerRun: 1000` on an active Daytona snapshot. 2. Open Runtime before the environment has platform qualification. 3. Try to upgrade to Paperclip Runner. **Paperclip version or commit** After #17122, reproduced against `9c442cf48e`. ## What Changed - Offer upgrade on probe-capable Daytona snapshots and immutable sandbox images. Perform the public environment probe only after a click. Save after fresh adapter discovery confirms support. - Resolve the actual computer for explicit, instance-default, and hidden managed environments. Preserve pending edits and reject stale probe results. - Ignore Claude `maxTurnsPerRun`, `chrome`, `warmHandleIdleMs`, and `acpWarmHandleIdleMs`. Return structured warnings from create, hire, and update. Show warnings in Runtime and after a save. - Keep validation for commands, credentials, filesystem, network, and permission settings. - Retain the legacy turn limit in the existing live conversion fixture. Update regression tests, upgrade stories, and capability maps. ## Verification - Red: the old route returns HTTP 422 for the saved turn limit; the old form hides the Daytona upgrade notice. - Green: 417 focused server and form tests pass. A further 188 tests pass after the review fixes. Qualification coverage includes failed probes, unsupported targets, changed settings, managed defaults, unsaved environment variables, and forced Kubernetes targets. - `pnpm -r typecheck` and `pnpm build` pass. Token gates, map checks, map tests, and Storybook catalog checks pass. - Real local Claude Opus 5.5 conversion and task pass, with the downloaded artifact checked against seeded source bytes, SHA-256, and Git commit. - Real browser upgrade and full Claude Opus 5.5 task on a disposable named Daytona snapshot pass at `b81cec7d8c`. The saved agent retains its model, high effort, credentials, timeout, and grace settings while dropping the turn limit with a visible warning. Exactly one successful run completed the task. The downloaded artifact matched independently seeded source bytes, SHA-256, and Git commit; the source was unchanged, no interactions remained pending, and the delivery link survived a browser reload. The snapshot used current runtime assets built from public source. The supplied Daytona account could not access the original staging snapshot, so this verifies an equivalent local fixture rather than that exact snapshot. Disposable test resources were cleaned up. - Earlier live attempts remain recorded: an image-based sandbox creation failed, and the first named-snapshot probe returned failure with unresolved provisional cleanup identity. Later named-snapshot probing and the task succeeded; the initial creation failures are not attributed to this PR or declared resolved. A separate attempt using the Oct 18 cached runtime failed the current provider-pack compatibility check; rebuilding the runtime from matching public source removed that mismatch. Owned resources were identified and cleaned up, with diagnostic databases retained. - 54 checks passed at `b81cec7d8c`, with no failed checks; Greptile and Paperclip Review both scored 5/5, with both review threads resolved. Master advanced during the live proof, so the branch was rebased with both sides of additive import/test conflicts retained. On the rebased head `2b74222b94`, 472 focused tests, UI typecheck, and UI build pass. All required CI gates pass; Greptile and Paperclip Review both score 5/5 on this head, with no unresolved review threads. The PR is mergeable. The serial local `pnpm test:run` was stopped after the CI test lanes passed; that local run is not claimed as a pass. ## Risks - An explicit upgrade can allocate a temporary probe sandbox and take several minutes. Probe failures leave the agent on its existing runner. - Ignored options no longer enforce a turn cap, Chrome integration, or legacy idle-time tuning. The UI warns about each active ignored option. - Unsafe or ambiguous execution settings still fail validation. This change does not weaken environment qualification or runtime artifact checks. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bd709df8b |
Operationalize Product E2E accounting, tracing, and scheduling (#17009)
## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E tests exercise real agents, tools, setup and cleanup. > - Repeated runs need retained costs and traces, including failed attempts. > - A passing retry must not hide earlier spending or feedback. > - Recurring work also needs admission limits, private reports and failure detection. > - This change supplies those controls while keeping paid activation behind existing gates. ## Linked Issues or Issue Description Refs #16996 and #17158. This is the consolidated landing of the reviewed eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033, #17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096, #17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127, #17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository permits squash merges only. A local rehearsal showed that squashing each slice into master causes repeated conflicts in dependent slices. This PR preserves their commits and lands their combined implementation in one reviewed integration. Superseded slices will close after this merge. ## What Changed - Retain every attempt's cost receipt, including setup, failures, interruptions and retries. Separate reported amounts, estimates, missing costs and reserved capacity in reports and dashboards. - Add bounded request admission, exact model pricecards, gateway error accounting, cache policy and reconciliation evidence. Refuse unsupported routes and requests that cannot fit the allowance. - Integrate Braintrust gateway traces and OTLP correlation, with secret filtering and experiment imports. Record phase timing, including setup, feedback capture and cleanup. - Collect complaint and suggestion evidence after each attempt, including recovery after failures. Retain explicit review dispositions independently from the product grade. - Add gated daily and weekly selections, measured forecasts, setup/readiness checks, wall-time limits and retained monitor notifications. Include declared non-model overhead in admission while reporting the existing subscription separately. - Keep recurring history private, bound uploads and retention, and use isolated archive environments. Keep public publication an explicit operation. Report and publication jobs install only the committed lockfile, with scripts disabled and isolated stores; stale pins fail closed. - Integrate current master, preserve its Product E2E journey billing and privacy checks, and test that shared setup receipts do not double-count estimated agent usage. Preserve upstream fixture repairs. ## Verification - Complete integration baseline `0649a5ce7544ab20825c2183d2f421157db49f24` includes master `a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head as ancestors. - On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012 Vitest tests passed, two skipped, across 193 files; all 137 Node source-contract tests passed. - Focused billing and Product E2E journey contract checks: 24 tests passed. They cover estimates, unknown receipts, single-owner setup costs, browser-only execution and private publication guards. - Monitor and workflow wiring: all 96 Node tests passed. `node scripts/check-token-gates.mjs`: all gates clean. - `pnpm test:e2e:runner:typecheck` passes after the standard Runner TypeScript build refreshed local declarations. The first attempt rejected the new master `runner_prepared_launch` union member because this checkout retained an older generated declaration; the source union already contained that member. - Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes dependency re-resolution from archive-authorized reporting jobs after the security scan flagged it. All 15 workflow-security tests pass after updating the old two-install assertion. - Hosted [CI run 39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180) passes on the current repaired head, including typecheck, build, general tests, Runner checks, browser shards and canary dry run. Greptile is 5/5 on this exact head with no unresolved threads. The security scan passes after the fixed lockfile-resolution thread was resolved. - Paperclip Review returned a terminal neutral check but an incomplete 0-file assessment on the repaired head: its platform blocked both diff responses as prompt-injection content. This is not a successful code review. No bypass or fabricated pass was attempted; current-head Greptile, security and hosted CI are independent gates. - No paid test, reservation release or recurring-run activation accompanies this landing. Live route qualification and invoice reconciliation remain separate evidence. <details> <summary>Retained earlier validation failures</summary> Earlier broad local runs had two 15-second test timeouts (CLI poll cadence and Retry validation), one embedded-database setup timeout, and later PostgreSQL semaphore exhaustion. The exact timed-out tests passed separately without larger timeouts; the broad-run causes were not established. Those local failures remain distinct from subsequent hosted CI results. The independent #17158 fix records its Muse fixture collision, its UUID correction and hosted verification. Local Muse execution was also blocked by macOS embedded-PostgreSQL startup. No historical failure is treated as a successful paid qualification. </details> ## Risks - Admission is a local control, not a provider invoice cap. Missing provider receipts or resource charges retain unknown amounts and holds; estimates do not become billed dollars. - The bounded path admits only documented local suites and provider routes. Ordinary legacy or separate Runner protocol campaigns have different controls. Merging this code does not qualify additional routes. - Recurrence still needs the existing enable gates and current qualification/allowance evidence. Archive permissions and resources must match the documented private-environment contract. - Legacy reports may lack complete attempt or phase evidence. Reports preserve that limitation and do not infer zero usage from missing data. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks; full-suite host limitation disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5ef385f7e6 |
Run reusable Product E2E journeys across deployment targets (#17262)
## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description **Subsystem affected** Product E2E harness, fixture registry, and reports. **Problem or motivation** Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. **Proposed solution** Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. **Roadmap alignment** This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0a5f1546ee |
feat(cases): add reusable prompt workflows (#17114)
## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
6dda8cc2b7 |
fix: preserve Stop authority and qualify complete recovery journeys (#17105)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final head |
||
|
|
507a0d7098 |
test: verify repository delivery after real runner conversion (#17089)
## Thinking Path > - Paperclip manages agents that perform useful work. > - Repository setup and Runner selection must preserve that work context. > - A successful provider turn cannot prove configuration retention or usable delivery. > - The repaired setup chain needs independent product checks with the recorded models. > - This PR adds explicit repository and conversion journeys to the existing Product E2E harness. > - The observer checks source bytes, Git identity, saved configuration, and downloaded output. > - Development and staging qualification remain separate. ## Linked Issues or Issue Description Refs #17088 and #17047. The complete repository setup repair in #17088 is merged. No competing conversion journey was found in the open PR search. **What happened?** Existing checks do not prove that a real legacy agent retains its configured context when switched to Paperclip Runner and then delivers verified repository work. The projectless repair check also uses different default models from the recorded staging models. **Expected behavior** A configured repository task delivers correct, accessible output. A real legacy-to-native conversion preserves configuration, instructions, environment, and equivalent permission policy before doing subsequent work. A projectless task continues through setup without supervision. **Steps to reproduce** List the configured-repository, legacy-conversion, and automatic-repository-configured-models suites. Run one exact ID from the README. These are explicit paid development checks, excluded from the default selection. ## What Changed - Add four configured-repository cells and four real legacy-conversion cells for Codex gpt-6.1-sol and Claude claude-opus-5-5 on local and Daytona environments. - Seed a fresh reference only in an owned disposable Git checkout. Compare the delivered JSON with its original bytes, hash, and commit. - Switch the same real legacy agent through the public Runner selection API. Compare allowlisted configuration and instruction digests before and after conversion. - Require one committed native run with the configured provider, model, and environment, a visible registered download after reload, Done, and no pending question. - Add four projectless replay cells with the recorded staging models and the unchanged strict continuation oracle. - Use production hire instructions and existing sandbox profiles, budgets, report pipeline, and cleanup. Record API-key and per-turn lifecycle differences explicitly. - Retain the required allowlisted API snapshot after successful automatic setup. Preserve lifecycle IDs, state, ordering, and receipt references without raw provider content. Verify both admissions retain the configured provider and model. ## Verification - Product E2E support after integrating the setup repair: 3,353 passed, two existing skips; source prerequisites: 128 passed. - Product E2E typecheck passes. Discovery lists all twelve new explicit cells. - Independent oracle calibration rejects wrong configuration, instructions, provider, model, source bytes, hash, commit, downloaded output, delivery, and pending questions. - The first full support run exposed a catalog count mismatch and an unprovisioned new profile ID. Both are fixed; the successful run uses the existing trusted sandbox profile IDs without changing workflow authority. - Review found fixture prerequisite defects: optional CLI-only Codex arguments and platform qualification after Runner selection. Both are corrected; the disposable Claude baseline also excludes its unsupported optional CLI turn limit. Seven new setup checks cover probe ordering, absent qualification, immutable images, local conversion, and uncertain probe accounting. The final focused cohort passes 86 checks and E2E typecheck passes. - The first qualification-regression run also had an incorrect test assumption about the legacy timeout; the corrected red reaches the unsupported extraArgs setting. No production conversion restriction was relaxed. - A retained four-cell development campaign delivered independently verified ordinary Codex output and real Codex/Claude conversion results with retained configuration. All three attempts nevertheless failed the required evidence contract because api-state.json was missing. The flow now saves allowlisted task/run lifecycle fields; it does not export prompts, comments, interactions, configuration contents, or provider outputs. The projectless cell delivered valid bytes but used a monitor fallback and failed the unchanged continuation oracle; its driver correction belongs to #17088. All four attempts remain failures; fresh exact-candidate acceptance is pending. These API-key/per-turn checks do not establish subscription authentication, warm reuse, Slack delivery, or the complete milestone. - Four fresh local ordinary-repository and conversion cells passed at `c4e409d819cc0a703736d3f1ed3552064ca690b8`. Independent downloaded artifacts and saved configuration evidence agree. - The repaired automatic setup chain passed all six journey checks for both models, but first failed because the success path omitted api-state.json. The separate capture repair now passes the fresh two-cell [campaign](https://github.com/paperclipai/paperclip/actions/runs/39633381139) at `7a7c26f6688ec6b0e0454562a43fd48ab1a81c97`. Both required API files and their run/intent/dispatch bindings were independently checked. Earlier failures remain failures. - After merging current master, 114 workspace/admission/continuation checks across six suites pass. The provider/model and evidence privacy cohort passes 65 checks, and E2E typecheck passes. The final model-binding assertion has unit calibration; fresh actual-model acceptance of that assertion is still pending. - Exact-head CI/review and complete staging acceptance remain pending. Local development passes do not establish the 24-cell milestone. ## Risks The real provider may expose additional conversion or setup defects. Failures remain evidence and cannot be replaced by catalog registration or a later retry. The conversion uses the production API and does not inject a native result. Only synthetic data is seeded in a resolved fixture-owned directory. Configuration evidence contains digests, not raw environment values. This PR changes evaluation code, not production behavior. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
|
eece07a747 |
fix: repair the complete projectless repository setup chain (#17088)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
53732c811b |
fix(evals): classify latency from observation bounds (#16829)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E probes measure when acknowledgements and replies become visible. > - A poll gives a time interval for an event. > - The first positive poll can occur after a target that the event could still meet. > - This pull request compares the full interval with a strict or inclusive target. > - Reports retain the raw observations and the original verdict when the timing verdict changes. ## Linked Issues or Issue Description **What happened?** A historical acknowledgement was absent at 1901 ms and first present at 2427 ms. A report marked the 2000 ms target as failed. The interval crosses that target, so the available evidence cannot give a pass or fail verdict. **Expected behavior** Return pass when the full interval meets the target. Return fail when the full interval rules out the target. Return inconclusive for overlap, invalid evidence, or insufficient evidence. Respect strict and inclusive targets and open and closed endpoints. **Steps to reproduce** Run the deterministic timing tests. Use the acknowledgement interval `(1901, 2427] ms` with `<= 2000 ms`. Compare the late reply interval `(52965, 53334] ms` with `<= 30000 ms`. The first is inconclusive. The second fails. **Paperclip version or commit** The source includes master at `14afc12124017d4ec30d0f4bfd6892f8ce475d21`. Related PR: [#13710](https://github.com/paperclipai/paperclip/pull/13710) changed chat latency and session reuse. This change corrects probe evidence and report verdicts. ## What Changed - Add a shared interval classifier with explicit endpoint and target operators. - Start observation before dispatch. Use a monotonic clock. Bound asynchronous reads by request start and completion. - Measure the first visible response separately from message creation and final settlement. Share browser reads with an optional acknowledgement probe. - Keep raw bounds, clock resolution, message metadata, failed reads, and original verdicts in retained report evidence. - Add deterministic coverage for threshold equality, missing data, edited messages, asynchronous reads, teardown, report correction, and a finite-domain oracle. - Bound observer shutdown to one second across pending and final reads. Retain timeout counts. Ignore late completions and start no more reads after shutdown. - Document the timing contract in the existing Product E2E guide. - Keep current startup reuse evidence after the master merge. Update four stale test expectations for qualified Copilot admission. Keep the explicit suite and provider restrictions. ## Verification - Current head: `c42e87e6d3b4b347e7cb176f56819c1d0abe106d`. - Passed: `pnpm test:e2e:runner:unit` (3,287 Vitest tests and 128 native completion tests; two explicit skips). - Passed: `pnpm test:e2e:runner:typecheck`, `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, and `git diff origin/master --check`. - Passed: [exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39607140418), including both required checks (`ci / verify` and `ci / e2e`), all general and serialized test jobs, all eight browser shards, build, canary dry run, and the clean public npm-install smoke. - Greptile reviewed the complete current diff and returned [5/5 with no actionable issues](https://github.com/paperclipai/paperclip/pull/16829#issuecomment-6266704814). All review threads are resolved. - The redundant local `pnpm test:run` was stopped after the exact-head CI test jobs passed. It did not produce a complete local verdict; the full repository test evidence is the CI run above. - The [earlier broad run](https://github.com/paperclipai/paperclip/actions/runs/39249282597) failed on the old head. Its full test command timed out after 15 minutes. Five E2E support failures came from ambient `NODE_PATH` and a missing historical Git object. Current master supplies the closed-environment launcher and retained source fixture. The full support suite passes with these fixes. - After the current master merge, four stale Copilot assertions and the nested unit launcher failed. Copilot is now qualified. Those assertions now require no qualification override and retain the rejected suite/provider cases. The full documented unit command passes. - The first local typecheck after the merge used stale generated package declarations and failed. The typechecks pass after `pnpm build`. - These checks use synthetic timing evidence. They make no live provider calls and claim no new live latency measurement. ## Risks - The browser diagnostic depends on the existing message and receipt DOM markers. Controlled tests cover visibility, receipt separation, concurrent reads, and teardown. - Corrected timing verdicts can become inconclusive. The report keeps the original verdict and does not change the overall workflow grade. - This change adds no database migration or product surface. ## Model Used - OpenAI GPT-6 Codex. Model family: GPT-6. This session does not expose a more specific model ID or context window size. The agent used reasoning, shell tools, code inspection, and deterministic tests. The earlier implementation session did not retain its exact model ID. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f89c37791c |
Add campaign sidebar and case navigation to runner E2E reports (#17010)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E campaigns test runner behavior across agent profiles and environments. > - The retained reports contain many suites, cases, and execution previews. > - A long report makes it hard to find a case or return to the campaign context. > - This pull request adds campaign navigation, case breadcrumbs, and separate global history pages. > - Users can scan all execution previews or inspect one case without losing their place. ## Linked Issues or Issue Description **What existing behavior does this improve?** Navigation in the retained Runner E2E campaign reports and history homepage. **Subsystem affected** Test report infrastructure in `tests/runner-e2e/`. **Current behavior** Reports show a long configuration matrix. The suite index and campaign history share the same page. Case links do not provide a focused inspection view. **Proposed behavior** Global overview, history, and attention pages show global navigation. A specific campaign adds a folding suite tree. All results keeps the full scrollable report. A case opens with breadcrumbs and an execution picker. **Reason and benefit** Users can find a case, compare its executions, and return to their reading position. Global pages do not show suites from an unrelated campaign. **Breaking changes** The report HTML layout changes. Retained result schemas, execution identities, and suite and execution anchors remain unchanged. **Additional context** Refs: #16996. This extends the merged report navigation and local preview work. The duplicate PR and issue search found no open PR for this change. ## What Changed - Add shared report navigation with the Paperclip brand mark and wordmark. - Scope the suite tree to the current campaign. Keep global overview, history, and attention pages separate. - Group all execution previews by suite and case. Reuse the existing evidence cards and image gallery. - Add focused case inspection, configuration selection, breadcrumbs, filter handling, and scroll restoration. - Move historical trends to the global history page. Keep billing details, missing results, and failure evidence available. - Update report documentation and browser regressions for filters, status dots, configuration reloads, breadcrumbs, and scroll restoration. ## Verification - [Final CI](https://github.com/paperclipai/paperclip/actions/runs/39500401411) passes on `89a8b5c290de3cfb9bdc061be3c88a29fd160a62`. Typecheck, build, runner checks, server suites, and all browser shards pass. - All four report browser regressions pass. They cover filters, gallery navigation, repeated suite jumps, the folding sidebar, breadcrumbs, scroll restoration, configuration selection and reload, mobile navigation, and aggregate case status. - The 63 report, history, preview-boundary, and navigation tests pass. Runner E2E typecheck passes with `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Manual browser checks covered desktop and a 390-pixel viewport. Retained campaigns covered 209 executions in eight suites, 179 executions in seven suites, and 15 executions in one suite. Browser fixture checks also confirmed filtered inspection and aggregate status. No paid evaluations ran. - The initial report browser tests still targeted the old layout and routes. They were updated for the new navigation. Review also exposed filtered-inspection and aggregate-status defects; both were fixed before the final passing browser run. - The unchanged Cases UI grouping test [failed once on the final head](https://github.com/paperclipai/paperclip/actions/runs/39500401411/job/118898559197). Its full file also failed locally, while the isolated test passed. The same UI code passed in earlier CI. A [single CI retry](https://github.com/paperclipai/paperclip/actions/runs/39500401411/job/118903333430) passed all 3,978 UI tests and 286 CLI tests without code changes. The cause remains unresolved. - Two jobs on an intermediate head were interrupted by runner shutdown signals. Those remain interrupted results. The complete final-head CI run supersedes them for merge verification. - Workspace `pnpm -r typecheck` and `pnpm build` also passed locally before the rebase. The full local `pnpm test:run` attempt encountered repeated embedded PostgreSQL startup failures and was stopped. It is not a passing local full-suite result; final CI provides the full verification. ## Risks - Navigation and hash handling can select the wrong case or hide results. Focused tests and browser checks cover these paths. - Historical campaign links must resolve from both the preview and published report locations. - The report layout changes. Graders, prompts, retained outcomes, and billing calculations do not change. ## Model Used - OpenAI Codex with `gpt-6-astra` and `gpt-6.1-sol`. Used reasoning, repository tools, code execution, and browser verification. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2b16262280 |
Improve runner E2E report navigation and local previews (#16996)
## Thinking Path > - Paperclip is the open source app that people use to manage AI agents for work. > - Product E2E campaigns check user journeys across agent profiles and environments. > - Readers use the campaign history and reports to inspect results and retained evidence. > - The reports showed image summaries, many unselected suites, and a long sticky index. > - Individual case cards also showed routine diagnostics before the visual evidence. > - This pull request adds text summaries, feature filters, folding case indexes, and simpler cards. > - A local preview uses retained public data so designers can change reports without another paid campaign. ## Linked Issues or Issue Description **Current behavior** The report is hard to scan as the journey catalog grows. Suite links can point into hidden results. Case indexes do not show whether a case passed across its configurations. Design changes also need a realistic local preview. **Proposed behavior** Show the selected campaign as text. Link journeys to current feature-map capabilities. Add folding case indexes with status indicators, reliable section links, and a back-to-top control. Keep failed outcomes visible and put routine diagnostics under Test details. Render retained public campaign data with local templates. **Reason and benefit** Readers can find a feature or failed case faster. Designers can iterate against real retained data without starting an application instance or using provider credentials. **Breaking changes** No result schema or grading change. The report stops embedding the generated summary PNG. The publisher still produces that artifact for existing consumers. Related work: #12889 added the existing matrix browser. #12846 concerns the public image publication boundary. This PR changes report presentation and local preview behavior. ## What Changed - Replace image summaries with shared HTML suite summaries on the history and campaign pages. - Show selected executions and preserve missing results for selected cells. - Add explicit journey-to-capability associations, feature filters, and a coverage view with unmapped journeys. - Add folding case indexes, section case lists, and red, green, or amber status indicators across matching configurations. - Fix anchor navigation through filters and add a floating back-to-top control. - Simplify case cards. Keep failure reasons visible. Fold routine checks, billing, and context into Test details. - Add an original-image link to the gallery. - Add a local preview utility with public data caching and offline mode. Document the direct Node command. - Test manifest-listed evidence, rejection of unlisted files and encoded traversal, cache reuse, and offline misses with a real loopback server and a stub downloader. ## Verification - Focused report, history, and preview tests: 75 passed in 6 files. - Report Playwright spec: 4 passed in Chrome with a standalone config that does not start the application server. - `node scripts/check-maps.mjs`: passed. - `git diff --check`: passed. - Manually checked retained campaign reports, feature filters, folding indexes, failure links, card details, gallery links, mobile layout, and keyboard navigation. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm test:run`: stopped after 15 native session database cases failed. The local Postgres package first lacked dynamic-library links. After link hydration, the same cases still failed. A direct `initdb` check then reported `could not create semaphores: No space left on device` and identified the macOS semaphore limit. The complete local suite has not passed. - `pnpm test:e2e:runner:unit`: 3,250 passed, 6 failed, 2 skipped. All six failures reproduced against unchanged `origin/master` report sources with the same installed dependencies. They concern Copilot admission and its Daytona image input, plus the launcher test that runs those assertions. No report or history test failed. - No live provider evaluation was run for this presentation change. - CI on `b1a9c06823` first failed browser shard 3/8. The agent-chat split-handoff test reached cleanup, where cancelling a run returned HTTP 500 from a Postgres deadlock. Cleanup did not restore the experiment setting, and a later test failed its initial setting check. This shard does not contain the report spec. One targeted rerun passed, and that full workflow then passed. This shows recovery; no production fix was made for the deadlock. The later preview-test commit requires its own fresh checks. To review the design, run `node --watch --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/preview.ts`. Open the history address printed by the command, then open a campaign. The README covers offline mode and cache refresh. ## Risks - Feature associations use the current map. They describe linked scenarios, not proof that a whole capability is covered. Historical grades remain unchanged. - New or unfamiliar cases remain unmapped until their assertions are inspected. - The preview needs the public history host on first access. Offline mode requires cached files. It binds to the loopback interface and serves manifest-listed public evidence. - Hiding routine diagnostics adds one disclosure step. Failed or incomplete outcomes remain visible. This is report polish. It does not duplicate planned core work in `ROADMAP.md`. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, code execution, and browser automation. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
505e1a5582 |
feat(cases): add persistent workflows with same-task agent handoffs (#16959)
## Thinking Path > - Paperclip manages AI agents and their work. > - An agent workflow needs persistent facts and a safe way to transfer work. > - Creating a task for each phase splits the objective, discussion, workspace, and review policy. > - A case can retain the facts while one primary task moves between agents. > - Transfers must survive process failure and preserve private access, budgets, and completion gates. > - This pull request adds durable case handoffs, native case tools, and read-only case pages. > - Prompts describe the workflow. Existing task controls decide when work can run and finish. ## Linked Issues or Issue Description **Problem or motivation** Prompt-driven agent teams need a persistent record that can move through triage, implementation, verification, and correction. Ordinary reassignment does not provide a durable instruction, safe release barrier, or recoverable delivery receipt for an agent handing off its own running task. **Proposed solution** Extend standalone cases with one primary task, a descriptive phase, retained access sources, and a handoff ledger. Apply assignments after source execution and workspace settlement. Deliver each transfer through the existing durable wake queue. Provide `create_case`, `read_case`, `update_case`, and `handoff_case` in the native runner. Creation requires kickoff and explicit workflow text. An explicit handoff to `done` closes the case when normal task completion is accepted. **Alternatives considered** Do not add a workflow engine, filesystem queue, configured state graph, or one task per phase. The workflow document combines transition prose with an optional Mermaid `stateDiagram-v2` map. The server does not execute the diagram. The Pipelines experiment removed on master stays removed. **Roadmap alignment** This extends Agent Reviews and Approvals, Enforced Outcomes, Bounded Run Recovery, and Activity Log attribution. Existing approval, review, budget, privacy, workspace, and responsible-user rules remain authoritative. Related work: [ordinary handoff wake recovery](https://github.com/paperclipai/paperclip/pull/15019). This feature adds a separate persisted case-handoff receipt and release barrier. ## What Changed - Add case routing fields, retained access sources, durable handoff history, delivery leases, and replay-safe migrations. - Keep the same primary task across agent phases. Preserve each returning agent's own session and the task's workspace. - Fence outgoing mutations, wait for provider and workspace release, and recover delivery without duplicate task comments or runs. Recheck current assignment permissions as the original caller before applying a delayed transfer. Scope a run Stop to that run’s own handoff intents, so repeated old Stop requests preserve newer work. Cancelling terminal intent without a source run also requires fenced current task ownership, including native retries and preparation with an old checkout. For Cloud callers, revalidate current stack membership with an uncached Cloud RPC and derive owner elevation from the current managed setting. - Keep case reads, documents, assets, Audit, and live events inside retained task/project permissions. - Add native case creation/read/update/handoff tools. Store required workflow text and initial documents on the first task. Page oversized case facts and referenced document bodies through the dedicated tool with bounded UTF-8 JSON, current routing versions, snapshot conflicts, and current source authorization on every page. - Add read-only case lists and detail pages, task-style list controls, compact dotted field tables, linked handoff tasks, and case filters in Audit. - Add native/legacy integration tests, migration/restart tests, browser coverage, and bounded live receipt fixtures. Extend the trusted sandbox condition to cover the new subscription fixture profile. - Hide obsolete paused-agent warnings on completed or cancelled task threads. - Synchronize the company-switch settings test with React effects after a CI failure. Preserve its security assertions and production remount behavior. ## Verification The branch directly includes master `9ef434fe79`. The required `ci / verify` and `ci / e2e` gates pass on head `765fccd67c`. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/39464348614) passes typecheck, build, tests, and all eight browser shards. Security checks pass. [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16959#issuecomment-6291156549), with no unresolved review threads. The optional Paperclip Review check remains queued because its assigned agent is paused. It is not a required branch check. No check result was overridden. Local workspace typecheck and build pass after the master merge, along with database migration checks, token gates, map checks, Storybook catalog checks, and the Copilot profile identity check. Standard builds refresh Runner vendor output. Focused checks pass 2,008 API/paging/context tests and 7 strict case protocol tests. The final commit changes one test fixture only; its ten tests, UI typecheck, and token gates pass. No production source changed in that commit. On current head `765fccd67c`, all **194 critical workflow tests** executed and passed with no skipped tests in those files: - 38 Stop/ownership tests, including board completion intent, native retries, stale Stop, and preparation with an old checkout. - 2 actual legacy process/HTTP handoff tests, including scratch failure, a closed-stdio child that continues writing, and recovery through a fresh worker after the process group stops. - 5 migration tests, including SQL column metadata, preserved null authority context, and all six prior-numbering receipts. - 3 process-settlement and 5 scratch tests. - 88 handoff-service, 18 heartbeat handoff, and 21 native case-tool integration tests. - 7 bounded case/document-read projection tests and 7 strict protocol-contract tests. The current CI checkout was `fed65d49`, merging `765fccd67c` into master `9ef434fe79`. Its new document regressions reconstruct exact Unicode/escaped bodies and revisions through real native authority and authenticated HTTP with generic API tools disabled. They cover case-owned, primary-task, and linked-task documents; routing on every page; stale document snapshots; access loss and unlinking; unrelated/foreign/private sources; system documents; and quarantined content. Active-task document tools retain their existing behavior. These are current-head test results from the first CI attempt; the browser-only retry did not rerun these suites. The full `64fdd3403f` CI failed one separate settings test: after a company switch, it observed an unsettled API-key draft. The production component and test were unchanged from prior green source. Production switches remount both keyed settings layers; the test directly reused an unkeyed view and used a timer tick instead of waiting for React effects. The final commit replaces that fixture synchronization with React `act`. The exact secret-clearing assertions and all timeouts remain unchanged. The original assertion failure remains recorded; no unchanged retry was used. The narrow unchanged baseline and fixed fixture each pass 10 tests. The upstream result-paging merge preserves case capture namespaces and current asset privacy. Its API integration suite executed 39 of 41 tests successfully, with two skipped; the optional assigned-result PRP integration test was skipped. Neither skipped suite is counted in the 194 critical tests or as live provider proof. All 361 upstream migration entries and SQL bodies remain unchanged. Six case migrations follow master as 0363–0368 with identical SQL hashes. Official Drizzle generators produced the combined metadata and five retained snapshots. Generation reports no schema changes, and the snapshot drift test passes. The earlier `ce361ae5d0` CI passed on attempt 1, including its then-current 179 critical tests. Fresh review later identified unusable document references with generic API tools disabled; the bounded document reads and ten additional authenticated HTTP tests address that finding. The full local `pnpm test:run` attempt was cancelled after this Mac exhausted PostgreSQL semaphore capacity; it is **not** counted as a pass. Later database attempts also stopped before assertions or skipped, including one sandbox socket denial. Two original stale-Stop regressions did reproduce the bug against the old source. A REST serialization assertion passed after two preserved database-startup failures. No test timeout was increased. All 194 assertions listed above executed and passed in current-head CI. The complete 385-file, 3,963-test UI shard also passed after the fixture repair. A configured-provider receipt workflow passed 13 backend checks across triage → implementation → verification → correction → done, using one task and four work runs. The saved receipt is revision 2 with total 6 USD; case and task both closed. The original canonical attempt remains failed because its browser assertion confused the terminal destination with completed-task tooltips. The corrected full browser verifier passed on the same retained database and original runs, with no additional provider calls. The last clean-build browser readback is on `6a86c54a53`; later changes affect recovery, authorization, bounded reads, and settlement. Accounted work cost is a $1.010574 rate-card estimate; provider billing and setup-probe usage are unavailable. Copilot profile 49 passed the unchanged seven-fixture private protocol pack (46/46 checks, no retries) and the product file-edit canary after master's completion-replay change. All 60 bound implementation hashes remain unchanged on the current source. Historical profile-47/48 identities and results remain intact. See the [profile-49 compatibility record](https://github.com/paperclipai/paperclip/blob/codex/persistent-case-handoffs/doc/plans/2026-10-20-copilot-completion-recovery-compatibility-v49.json). This is bounded compatibility evidence, not a fresh full matrix, installed-package product test, or Daytona rollout. Preserved CI failures and retries: - On `20662930ac`, Slack managed setup failed a five-second success assertion after reload. Its test/application path was unchanged from a prior green head. One unchanged-source retry passed all 25 tests in that shard; no source or timeout changed. - On `9402b4f17a`, two CI workers received shutdown signals at 15:24:19 UTC. Neither log had an assertion failure before interruption. One unchanged-source retry of each passed all Runner static checks and all 32 tests in browser shard 6; both aggregates passed. The next head, `7b4374edaf`, passed all CI without retries. - On `295f289b5a`, serialized shard 9 timed out during its first device-login test, then the next test found an active login lease. Direct login/test paths were unchanged from a prior green head. Cold import plus a timed-out async body using the next test's mutable actor/store is the supported fixture-race explanation; no request trace proves that sequence. The original failure remains recorded. One unchanged-source retry passed all 294 tests in 19 files, including all 41 device-login tests. No login source or timeout changed. The subsequent `ce361ae5d0` CI passed without retries. - On `64fdd3403f`, one company-switch settings assertion observed an unsettled draft. The final test-only commit uses React `act` without changing assertions, timeouts, or production code. On `765fccd67c`, all ten settings tests and the complete 385-file, 3,963-test UI shard passed. - On `765fccd67c`, browser shard 3 completed the split-handoff assertions, then cleanup received HTTP 500 from run cancellation after a one-second PostgreSQL row-lock timeout. The extra chat run was reporting completed work while teardown began. Cleanup stopped before restoring a global experiment flag, causing a later assertion to fail. Relevant production and test paths are unchanged from an earlier green commit. Stop's lock order is consistent, and no nested self-lock was found; the exact PostgreSQL blocker was not recorded. One unchanged-shard retry passed. The original failure, trace, and downstream flag failure remain recorded. No Stop implementation, cleanup assertion, or timeout changed. The retry supports intermittent cleanup contention; it does not identify the original blocking transaction. ## Risks - Local process transfers require host-owned PID/group absence. Missing or uncertain termination evidence keeps delivery pending, independently of scratch setup. - Handoff admission and task cancellation share execution ownership. New regression tests cover temporary holds, late source writes, and uncertain provider termination. - Retained private sources use an intersection of existing permissions. This intentionally fails closed when a source is missing or an existing task cannot preserve that boundary. - Migrations add tables and indexes. They preserve master history and can replay over prior feature installs. Migration 0368 adds nullable internal authorization provenance with no default or authority backfill. Historical transfers still awaiting release without this context fail closed; an operator must cancel them and request a new handoff. - Delayed Cloud handoffs depend on the current Cloud membership RPC. An unavailable, ambiguous, expired, or mismatched response holds the handoff for retry; it does not reuse cached authority or grant access. - Bounded reads reload and reauthorize the selected projection per page. Underlying work grows with the data size and number of pages. Changed data or routing requires restarting from offset zero without the old snapshot. - Phase names and workflow diagrams remain advisory. They do not satisfy review or approval gates. - This is a large, cross-cutting PR. The maintainer requested one PR for the whole feature; the commits separate data, runtime, UI, and qualification work. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and delegated code review. The exact deployment model ID and context-window size were not exposed in the session. Live fixture agents used the explicitly configured provider model recorded in their test evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7c247243e5 |
Qualify and package Copilot for ordinary installations (#15706)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A provider must work through normal installed packages on local and Daytona environments. > - Candidate-only success does not prove that the shipped package contains the manifest and native runtime needed for Daytona. > - The release must preserve saved-token access and task lifecycle guarantees. > - This pull request will qualify Copilot and include the exact-source remote manifest in normal release archives. > - The benefit is a reviewable part of the Copilot production release. ## Linked Issues or Issue Description Refs #15701, #15703 and #15705. Supersedes qualification work in #15563 and the original candidate in #14434. Base: #15705. ## What Changed - Keep Copilot discovery passive during reconnect reconciliation; refresh on explicit opening, account/environment changes and reconnect. - Retain the selected Product cases, permission/continuity evidence and cleanup checks. - Build the Linux provider manifest from the existing pinned Docker stage in the cloud. - Join all four native daemon receipts with that manifest before server packing. - Reject missing or incompatible remote authority during normal release assembly; retain historical workflow compatibility. - Promote the completed profile47 qualification marker, publish the current qualification record and preserve every failed attempt. - Keep one current release checklist and the shipped-canary gates explicit. ## Verification Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`. History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`. Remaining source [`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d) contains the four-file authorization repair. The remaining PR trees preserve six nonoverlapping files from the newer public mainline managed-settings fix and the CI-owned lockfile. No unrelated provider is enabled. The security finding is repaired: existing-agent discovery requires `agent_config:update` before connection selection or token resolution. The saved environment is the default. Authorized editors can select a prospective environment. Both denied board-viewer and agent-reader regressions fail on the original source. All49 route tests and server typecheck pass with the fix. Denied requests perform no connection selection, credential resolution or probe. Current-head security scans pass on both remaining PRs; the repaired thread is resolved. Copilot CLI1.0.88 and profile47 retain digest `sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`. The [qualification record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json) reports20/20 Product cells,154 assertions and7/7 protocol cases,46 assertions against frozen normal installed source `c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification model is `gpt-5.6-luna`. Every passing Product case confirms cleanup. Original failures remain recorded; automatic behavior retries are disabled. The route-only authorization repair changes no bound runtime/profile bytes and does not relabel the frozen campaign. The ordinary installed remote journey passed saved-token selection, model discovery, normal UI save/reopen, metadata-only Test, completed task, accepted receipt and exact19-byte browser download. An open model catalog caused no additional discovery requests during117seconds. Actual installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta and native Linux x64 on Daytona. Raw token browser entry was not exercised; encrypted connection creation used the normal API. The first Daytona file attempt timed out during delayed semantic dispatch; its precise stalled layer remains unresolved. Independent ordinary remote delivery and the other19 Product cells passed before one instrumented recheck passed8/8 with unchanged source,prompt,assertions and deadlines. No causal latency repair is claimed. Before the authorization repair, all four final PR heads passed required verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads. Current-head CI/review is running for the two repaired heads. Local focused checks retain129 connection,59 runner,72 shared-eval and88 recovery/workspace passes, plus migration safety and token gates. The additive Copilot migration is361; released master history is preserved. [Cloud normal-package run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704) builds the pre-security source4f3f. All four native producers and Linux package assembly pass; the final clean installed consumers are finishing. All18 retained archives match their hashes;37,863 regular files contain zero selected credentials. That package does not contain the later route repair. The actual final published package must include it and pass the local/Daytona live canaries. The frozen Daytona image remains published and independently verified. No separate full-image scan is claimed. Docker builds run in the cloud. **Remaining:** current-head CI/review, setup and qualification merges, final publication, and ordinary local/Daytona canaries against the actual shipped package. The user explicitly authorizes the maintainer review override; CI remains a merge gate. No production deployment is claimed. Provider USD remains unknown and GitHub-attributed. ## Risks An omitted or mismatched image manifest prevents ordinary Daytona startup. Final normal archives must contain the matching image authority and platform daemons. The completed runtime campaign remains bound to its original source; the final UI-only repair needs its own normal-install setup proof. Credit notices are not measured USD receipts. Native steering, native plan mode and detached work remain outside this release. ## Model Used OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this task. Used for code changes, review and tool-assisted verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b448710986 |
fix: derive startup warmth from per-layer reuse evidence (#16908)
## Thinking Path > - Paperclip helps people manage AI agents and inspect their work. > - Agent startup can reuse a lease, workspace, Runner, process, or session. > - A session ID can remain the same when other resources are new. > - The old cold/warm label did not show these separate decisions. > - This pull request records closed reuse receipts and derives labels from them. > - Reports now show partial reuse and missing evidence. ## Linked Issues or Issue Description Refs #11685. That issue tracks a specific ACPX fingerprint repair. This change records reuse evidence across startup resources. It does not repeat that repair. **What existing behavior does this improve?** Startup labels in the operator trace and Product E2E reports. **Current behavior** A cached runtime or a continuous session can suggest a warm startup even when workspace staging or provider launch work occurs. **Proposed behavior** Record each layer's reuse decision and a closed reason. Record observed provider launch counts. Use `warm`, `cold`, `partial_reuse`, or `unknown` from those receipts. Keep old runs unknown when they have no receipts. ## What Changed - Add local run-log receipts for the lease, workspace, Runner, provider process, and provider session. - Count physical ACPX launches, including respawns. Preserve the reason for a new process. - Reuse native workspace adoption and session lifecycle evidence. Keep native and bridged remote provider-process reuse and launch counts unknown where physical evidence is absent. - Show the per-run summary in retained results and the dashboard. Keep missing legacy values explicit. - Set the legacy trace boolean only for fully evidenced cold or warm startup. Keep the operator OTLP gate. Add no first-party Telemetry. - Test full reuse, workspace invalidation, two direct provider launches with session continuity, and report generation. - Start the final diagnostic write without waiting for persistence. Keep lease cleanup independent of a stalled event sink. - Keep remote provider evidence unknown after runtime setup fails or the host relay starts twice. - Align a stale Pi support-test expectation with current master profile 28 and its suite fingerprint. Keep the qualification rules unchanged. ## Verification - Passed: all 319 runtime, startup-timing, reuse-summary, and native workspace-history tests. Disposable local Git tests used the native Git binary. - Passed on the current commit: all 2,768 Runner E2E support tests across 128 files. This includes report and history tests for generated HTML and normalized JSON. - Passed: `pnpm --filter @paperclipai/adapter-utils typecheck`. - Passed on `df60f8dbdbfcb6d80c84cd3bf563f8d58b2f9f03`: all PR CI gates, including repository typecheck and build, all test shards, and browser tests. Both automated reviews are 5/5. GitHub reports 55 successful checks, one neutral review, and two skipped Storybook checks. - The first server shard run failed because an unchanged workspace-runtime fixture did not accept health connections within 10 seconds. Its isolated local test and the single failed-jobs rerun both pass. This establishes recovery; the cause of the first timeout remains unknown. - Passed in the hosted source run on this exact commit: recursive typecheck, token gates, the large Runner E2E typecheck, Runner E2E unit tests, and build. The retained `source.txt` matches this PR head. - The extra unsharded `pnpm test:run` reached its 60-minute limit (exit 124). It has no complete verdict. The broad workflow therefore reports failure. No assertion-failure marker appears in the retained partial log. Full-suite acceptance uses the passing canonical PR CI partitions. A read-only inventory check found no gaps across 1,023 general server files, 161 serialized suites, and 20 workspace projects. The separate chat/native Runner lanes cover the excluded server files. No tests or timeouts were weakened. The local environment has 4 GiB of memory. Use CI for the large Runner E2E typecheck. - The initial report tests failed because the harness could not resolve package imports from the repository root. Relative source imports fixed that failure. - The first broad local run had timeouts from the environment Git bridge. Native Git resolves the ancestry timeout and all runtime tests pass. The added report fixture lacked its required screenshot; the corrected report tests pass. A stale master assertion expected Pi profile 27, while the generated profile is 28. Updated its expectation and exact suite fingerprint; all 78 targeted support tests pass. The clean full support run passes all 2,768 tests. - The three review regression cases failed before repair. They showed remote counts of zero and two, and a completed run that waited for the final diagnostic sink. They now pass. The stalled-sink case keeps persistence pending while the run returns, both bridge stops run once, and the staging lock map becomes empty. - Hosted evidence for this commit: [source verification](https://github.com/paperclipai/paperclip/actions/runs/39335475416), [retained command logs and status table](https://github.com/paperclipai/paperclip/actions/runs/39335475416/artifacts/12228829282). All listed commands exited zero except the incomplete unsharded test command described above. - The physical-launch test drives the executor's spawn callback twice. The native workspace test uses real Git repositories and verifies reuse and invalidation. These are deterministic tests, not a paid provider qualification campaign. ## Risks - Consumers must handle `partial_reuse` and `unknown`. Partial evidence no longer produces a legacy cold/warm boolean. - Native and remote bridged provider process evidence is still unavailable on some paths. The report shows unknown values for those paths. - Receipt emission is best effort. Missing receipts reduce confidence in a label and must not change execution or settlement. - No schema migration, first-party Telemetry event, or endpoint default changes are included. ## Model Used OpenAI Codex, based on GPT-6, used reasoning, repository tools, and code execution. The exact provider model ID and context window are not exposed to this session. The existing implementation checkpoint was preserved from the earlier agent run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> |
||
|
DottaandPaperclip
|
5739a53f5f |
fix(tests): sanitize installed-consumer validation processes (#16797)
Start installed-consumer validation through Node and remove ambient Node startup settings before test processes start. Keep pinned historical source fixtures available in shallow checkouts and set filesystem fixture conditions explicitly. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> |
||
|
DottaandPaperclip
|
45b67f1d93 |
feat: default new Apple Silicon Claude agents to Paperclip Runner (#15993)
## Thinking Path > - Paperclip lets users choose an agent harness and account. > - The server resolves the runner for new agents. > - Claude still needs the same ordinary native-default journey as Codex. > - Existing agents must retain their runner, model and account. > - This PR defaults new Apple Silicon Claude agents to Paperclip Runner with Opus 5.5. > - Users can connect and work without a runner override. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15994 and precedes #15997. Related scope: #15422, #13978 and #15224. ## What Changed - Resolve new Apple Silicon Claude agents to `paperclip_runner`, `provider: acpx`, `acpxAgent: claude`. Keep explicit Legacy, models, provider routing and saved execution. - Use `claude-opus-5-5` for ordinary Claude defaults and retain explicit overrides. - Reuse existing API-key/subscription connections, credential storage and checks. Generalize the shared Advanced runner control with Claude branding. - Carry configured effort into the shared v9 contract. Preserve compatible permissions, instructions, skills, timeouts and session behavior. Require Legacy for unsupported settings. - Qualify a new immutable managed Claude target at explicit Connect through the existing permission-gated probe. Use fresh scoped discovery for test and hire. Discard stale results and require an explicit Connect after settings change. - Retain merged Codex onboarding recovery, auth preparation, import/effort corrections and Grok Bot gates. - Check the Claude first-task wizard's saved default directly. Update configuration/runtime documentation and exact Opus 5.5 accounting. - Align hire fixtures with the merged host-environment restriction. Native agent-authenticated hires cannot inject host env settings; both creation endpoints verify that no extra agent is saved. ## Verification - Updated onto master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through #15994. Native input v9 carries effort while preserving v7 home modes and v8 Muse. Codex defaults remain unchanged. - Focused local verification passed 285 server tests, 323 UI tests, 124 setup/registry tests, and two v9 permission-pin transport cases. Direct server/UI TypeScript checks passed. An initial 30-second hook timeout was resolved by serial verification with a longer hook budget: all 122 credential/permission cases and 232 snapshot/environment cases pass. Direct server TypeScript checking also passes. - Ordinary web creation with the experiment disabled saved native Claude, Opus 5.5 and High effort without runner/model overrides. The actual product task and follow-up both succeeded on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` with the cloud-built daemon from `04e0202ca81387b17cbe7fd094a8338648356fe9`. The final master reconciliation also changes shared runtime source. Historical proof retains its source identity; a final-source cloud qualification is running before refreshing the preview daemon. - Both actual browser downloads were checked: v1 has three sections/six checkboxes (451 bytes); v2 preserves the original body and adds two Rollback checks (612 bytes). The original attachment stayed byte-identical. The follow-up records session resume and the same provider session. Total estimated API usage: $0.518432800, with no pending/unpriced usage in the successful fixture. - Fixed the live-discovered setup incompatibility: native API-key validation no longer depends on the global Legacy Claude CLI. The existing provider endpoint verifies the key without a paid prompt; subscription/custom routing probes remain intact. - Fixed security review's native-hire credential redirection: agent-supplied URLs, proxy/launch settings and other host environment controls are rejected before inheritance. Native credential references remain supported and partial edits use the saved harness. Added create/hire/update regression coverage. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - Docker and native release builds ran in cloud infrastructure. An inadvertent local Rust prerequisite was stopped; no local Docker build was run. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - Missing packages, authentication rejection, unsupported settings and unavailable models must fail explicitly. Existing agents retain saved execution; explicit runner changes use existing revision/session invalidation. - Subscription follow-up remains unverified; remaining live acceptance uses API keys per human steering. Historical accounting holds and the separate $50 ceiling remain intact. - The subscription acceptance limit remains explicit for review. No Claude database migration is introduced. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
adb69e8f09 |
feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683)
## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto | paperclip | legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. **Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass.** The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board **Try again** and **Resume** recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal **Approve completion** reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f329b24ee1 |
feat: give tasks durable project-optional workspaces (#15749)
Give tasks durable company-scoped workspace bindings independent of project assignment, channel defaults and authorized repository preparation. Reuse verified input seeds and sparse output recovery while preserving trust, privacy and controller provisioning boundaries.
Full CI and E2E, Superagent, legacy checks and Greptile Apex 5/5 passed on reviewed head
|
||
|
Dotta
|
a9830414c7 |
Preserve repository dependencies when importing GitHub skills (#15731)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its Skills library imports reusable instructions from GitHub. > - Some skills depend on shared files and sibling skills outside their folders. > - Directory-only imports lose those dependencies. > - This pull request keeps repository files together and lets users select which skills appear in the library. > - Agents receive the original files and paths through the existing skill delivery system. ## Linked Issues or Issue Description Refs #14713. Related: #13656. **Current behavior** GitHub imports copy independent skill directories. Shared scripts, policies, helper skills, and credit files can be omitted. One shared-file error can block every skill. **Proposed behavior** New imports keep repository files together at the scanned commit. Show all skills with their folders and select every skill by default. Collapse individual package files. Omit unsupported or oversized files with a warning. Import available content without repair choices or an extra confirmation. Keep warnings on the saved source. **Reason and benefit** Skills can read their shared repository files after Paperclip delivers them to an agent. Original instructions stay unchanged. Users can inspect included files and understand omissions before and after import. ## What Changed - Store immutable company-scoped repository snapshots and associate them with skill versions. Regenerate the migration after merging master. - Preserve original bytes, binary encoding, and executable modes. Shared-file changes version each usable selected skill. Pinned versions retain their original files. - Deliver a discovery wrapper with the original tree under `.paperclip-repository/`. Preserve invocation policy when present. Fork and create-from-existing retain repository files. - Keep repository files together for new UI imports. Show every skill, select all by default, and collapse package files. Remove the packaging-mode menu. Keep the mobile import footer visible. - Warn about symlinks, submodules, oversized files, missing references, and invalid optional manifests. A malformed skill remains selected with a Needs setup warning. Usable skills still publish, and its original file remains repository support content. Source warnings persist through refresh. Unsafe execution and secret-disclosure findings omit the affected files before snapshot publication. - Expose saved repository files in the installed skill viewer, including binary encoding and executable flags. Reads use the immutable company-scoped snapshot; relative instruction links resolve from their original repository directory. - Audit delivered inventories and derive trust from all repository files. Accept evaluation prose while retaining dynamic-execution findings. Read blobs in one bounded Git batch and check budgets before reading support files. Imports never execute setup commands. - Add scanner, database, delivery, UI, and browser regressions. Update Storybooks, screenshots, and documentation. ## Verification - 69 focused scanner, Git snapshot, delivery, and PostgreSQL lifecycle tests passed. They cover partial imports, warning persistence, missing references, binary files, executable modes, copies, pinned versions, cancellation, company isolation, and batched Git reads. - 110 import, skill-detail, selection-tree, saved-file link, and Markdown renderer tests passed. A file-switch regression failed before the resolver fix and passed afterward. Browser walkthroughs cover the all-selected open tree, repository preview, non-blocking warnings, missing references, setup problems, warning persistence, and mobile layout. - Full workspace typecheck and build passed. Runner E2E TypeScript checks and design-token gates passed. The full serial local suite was attempted and stopped after over an hour. The existing 40,000-file Git streaming stress test failed after 303 seconds on macOS; the broad local suite is not counted as passed. All 55 applicable PR checks passed on the final commit, including the full sharded CI suite. Two optional Storybook workflows were skipped; the five local Storybook walkthroughs passed. - On October 12, live campaigns imported the private Poteto repository through the production GitHub Sources UI, selected architect and bro, and invoked them with native Codex on local and Daytona. The source connection was disabled before invocation. Independent filesystem inspections matched every supported Git blob, size, and executable mode. Both tasks completed and cleanup passed. Campaigns: `local-2026-10-12T15-07-39-504Z` (both environments), `local-2026-10-12T16-08-39-682Z` (local), and `local-2026-10-12T16-03-33-938Z` (Daytona). Each of the two selected skills received all 234 supported repository files. Later changes only affect the installed-file viewer and documentation. This qualifies skill delivery and instruction reads, not every Poteto workflow or adapter. - A manual source-checkout test drive imported a synthetic private GitHub repository with a symlink, an oversized file, a missing reference, and malformed skill metadata. Available skills published with their shared checklist. The walkthrough found duplicate warning text after import; the final UI fix removes that duplicate. The file viewer exposes saved repository dependencies and resolves authored relative links without changing the instructions. A follow-up live refresh omitted unsafe helper files with warnings, preserved every safe file byte, and survived a server restart. Screenshots below use the production UI with Storybook fixtures. They do not replace the live GitHub and runtime checks. <details> <summary>All skills visible and selected by default</summary>  </details> <details> <summary>Import stays available with warnings</summary>  </details> <details> <summary>Repository file inspector</summary>  </details> <details> <summary>Mobile import</summary>  </details> ## Risks - Each selected skill carries a repository copy in the runtime. The database stores each snapshot once. Runtime transfer and storage increase with the number of assigned skills. - Runtime budgets remain: 9,990 repository entries, 1 MiB per file, and 61 MiB total. File omissions are recorded as warnings. Source-access or download failures retain installed versions. - Import success does not prove that every skill can run. Environment dependencies, credentials, delegation tools, and schedules still need configuration. Imported setup scripts are not executed. - Live runtime coverage is native Codex on local and Daytona. Other adapters use the shared delivery path but were not exercised in these campaigns. The roadmap marks Skills Manager as shipped. This extends the existing importer. Related pull requests were checked. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code execution, automated tests, and browser tools. The exact deployment identifier and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
57e977be72 |
feat: integrate Pi 1.0 into the experimental Runner (#14921)
## Thinking Path > - Paperclip manages AI agents and their work. > - The experimental Runner owns provider processes and durable sessions. > - Pi needs working task execution and human controls. > - The five-PR stack must preserve changes already on master. > - Each layer now carries the complete integrated source for a safe sequential fallback. > - This PR belongs to native GitHub stack #15602, ending at #14956. ## Linked Issues or Issue Description Refs #14436, #14631, #14743 and #14956. Ship Pi 1.0 through the experimental Paperclip Runner. The five PRs are #14921, #14922, #14923, #14924 and #14956. The user authorized the complete merge after checks pass. Existing `pi_local` execution is unchanged. Accounting and wider provider/platform qualification remain deferred. ## What Changed - Recover missing final replies after workspace finalization changes owners, using accepted-turn evidence without rerunning work or granting external-chat publication. - Preserve the admitted Pi instruction root across warm runs, while retaining changed-root rejection. - Give Pi a bounded 15-second default shutdown grace so stop, drain acknowledgement and durable suspension can complete. Explicit deadlines and other providers retain their existing behavior. - Integrate the Pi 1.0 runtime and master contracts. - Use Pi profile 22. Preserve explicit caller-selected models and exact native thinking levels. Keep Pi's wrapper, helper, extension and question/control behavior unchanged from the qualified profile-19 runtime. - Preserve master's Dot lifecycle and consent fields, configured task environment, status guards and current Codex/Claude dependency versions. Cursor stays qualified. Copilot stays pending; profile 17 binds the changed shared protocol validation sources. - Exclude general AWS IAM credentials from Pi static/custom provider bindings and selected task projections; preserve the provider-scoped Bedrock bearer key. Profile 21 is retained as historical provenance. Rust and cloud install probes use the current declaration. - Patch bundled brace-expansion 5.0.9 to the exact official 5.0.12 payload. Pin the patch and complete runtime closures. Include the patch in normal installed setup tooling. Keep the upstream Pi shrinkwrap as provenance and permit only this exact security correction. - Include current attestation files in the Docker build context. Keep the repository lockfile unchanged from master. CI and private image builds resolve manifest changes before their frozen installation. ## Verification - Full local `pnpm -r typecheck` passes, including Runner Rust, server and UI. Focused integration checks pass: 194 Runner admission/environment tests, 63 profile/credential tests with one expected skip, 152 Dot/UI configuration tests, and Pi transcript/notice tests. - Full local `pnpm build` passes on the final source. - Fresh final-source checks pass: all 698 Rust workspace tests (32 binaries), 156 credential/profile/controller tests with one expected skip, Runner TypeScript typecheck, and 20 package/setup/sandbox tests. - The profile-21 Pi materializer passes on the native host with the official pinned Node 24.21.0 and its npm. It verifies all 150 locked packages, the patched dependency and the exact closure. Setup/package bundle tests and UI token gates pass. - The old hashes were reproduced for all three supported targets before calculating the patched graph. New closure hashes are darwin-arm64 `282022db10150c6632b3444df421342e7d534bdf5d5fb1097a2e79d0625a2bcf`, darwin-x64 `64e251e19009f755c0b04f73ce2138246faab71a961b0f13d75ebfcc34bef12e`, and linux-x64 `713b1fdff42fb56a1518bdc084f181d70bee8ebadc3e4b1d76321ed9108c8410`. Independent native platform execution is separate from graph identity reproduction. - Historical cloud qualification remains unchanged: all seven core cases pass on shipping source `10dc43c9ec65d88c2f782d62afb296d09494f215`, harness `1a4408a48cfb5a1f094a311141c257c92cd7a893`, image `sha256:5b3a775b383591bda1b0c1889e509acc70ce7f37c53f09733c81d59037f02280`, and accepted Sonnet 4.6/low fixture. All 215 canonical files and all seven cleanup checks pass independent verification. These are profile-19 results and are not relabeled as fresh profile-22 runs. - Current Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [The readiness plan](https://github.com/paperclipai/paperclip/blob/codex/pi-production-readiness/doc/plans/2026-10-02-pi-production-readiness.md) preserves campaign and failed-attempt provenance. - Merge only after every PR's current-head CI and fresh review pass. Linux CI covers the full suites, build and browser tests. The local embedded Postgres API-authority suite cannot start on this macOS/Node 26 host, so Linux CI must confirm that suite. ### Fresh profile-22 core qualification — 2026-10-08 All seven accepted core cases pass canonically on Pi profile 22, with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. This model is a fixture; production accepts the caller's explicit Pi provider/model. Runtime/install source: `3241a992f2a7703e59e97ed0fd3e5d6405de4401`. Frozen accepted harness: `1a4408a48cfb5a1f094a311141c257c92cd7a893`. Immutable cloud image: `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`. Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [Hosted Linux image and clean-install verification](https://github.com/paperclipai/paperclip/actions/runs/37868328023) passes, including all 20 source-bound archives, normal CLI/Pi setup, companion import and the production pack reader. This exact installation source includes the latest master integration and the corrected Pi warm instruction-root fence. Full local typecheck/build and current-head hosted CI verify the final stack. All 13 focused real-root regressions pass. The full local executor suite passed 662 tests; 15 database tests could not start the Mac embedded PostgreSQL service. Hosted Linux CI passes the full required verification and E2E checks. These fresh results keep their own source identity; profile-19 results remain historical. | Core path | Canonical campaign | Retained archive SHA-256 | | --- | --- | --- | | File edit, validation, download and Done | `pi-core22-replyfix-0-1791511228` | 23 files; `a473e8603a3dd4737863291f8d3d1e392391f0b16d433c3e0e0e9d8baf7a97b0` | | Pending question and controller restart | `pi-core22-replyfix-1-1791511376` | 33 files; `6b829c4eb74e1f32a89c692a4ae7130dbfc1c6d3cf13915effe2103d9e242c8e` | | Three-turn session/process/workspace continuity | `pi-core22-replyfix-2-1791511587` | 23 files; `7a87021f8f9a3fdd3c58bb4467f8d82c635e3ea4795d6e75f144d9aa14818df8` | | Four typed questions and browser reconnects | `pi-core22-replyfix-3-1791511881` | 42 files; `9e31755252be1f4f9cb0626c984c142d4d1ae5f5bee3a7af08444db8d12c280a` | | Plan approval and completion | `pi-core22-replyfix-4-1791512031` | 22 files; `a0383ce1aab38e7b5a25ce0e9dd3bebea5c037ebd96ae6b29dae19015da2ae2c` | | Same-turn steering and permission denial | `pi-core22-replyfix-5-1791512261` | 39 files; `c929b8c7070f0b66aedc17e65ca46e6beab1e363926ac9f7e2a75fb250f05949` | | Stop during pending permission | `pi-core22-replyfix-6-1791512390` | 33 files; `7f0a58ae0f4d5bfc76149435f4e322537089c5bd16e7ffe9b5ad71f10a621a07` | All 215 canonical files (28714587 bytes) are independently hash-verified. All seven cleanup grades pass, with no owned runtime process or temporary root after each case. Automatic retries are zero. The owned cloud host stopped normally after retention. The prior profile-22 warm attempt remains failed and separately retained: archive SHA-256 `1e54eba5ec72b50cee1534b23d1d1d4f21a090006b8a64501ba70db972abfde5`. Its original canonical classification is preserved. Diagnosis reproduced a product bug comparing an agent-files root against an unset checkpoint-only field. The fix stores the admitted physical root separately from the adopted per-run collection capability. The real-root regression fails before the fix and passes afterward, including rejection of a changed physical root. Fixture, grader, model and all seven accepted case IDs are unchanged; this fresh campaign tests final-reply publication after file registration first. The intermediate restart attempt also remains failed and retained: archive SHA-256 `5dcaefdf1d17cf4cd54fd4cf810f45e736667392339b8ce7caf08bb4e225277f`. Its original canonical classification is preserved. Pi resumed, wrote the verified answer and completed its task; exact runner suspension was proven, but idle stop consumed about 5.2s and left under 3s for the drain acknowledgement. The Pi-only default shutdown grace is now 15s, preserving a full 5s drain round trip and a finite suspension reserve. Explicit caller deadlines, other provider defaults, literal drain receipts and exact suspension identity checks remain unchanged. The timing regression fails before this correction and passes afterward; all 18 focused settlement tests and Runner typecheck pass. The final-source file attempt is also preserved as failed (`candidate_failure`), archive SHA-256 `db6767b6773ea618997927ac77bdb005a5ac81492c7b9c0ffbc900449f829bc9`. Native edit, validation, exact downloadable artifact and Done/succeeded all passed, and the exact final reply was durably recorded. A workspace recovery owner completed before the live heartbeat reached presentation, leaving that reply absent from task chat. Recovery now materializes only a completed final reply from the accepted turn of an ordinary internal Done task, preserving issue/run/contract binding, suppression, external-chat authorization and same-run deduplication. The database regression covers the generated file-preparation receipt, suppression, unapproved external continuation and replay. Server typecheck and all 49 response-selection tests pass; hosted Linux verifies the database regression because embedded PostgreSQL cannot start on this Mac. The delayed-final-answer database regression passes on [the final root-source Linux server shard](https://github.com/paperclipai/paperclip/actions/runs/37868262553/job/113628594152), alongside 1,108 passing tests. The first root Runner shard had one unchanged durable-resume test exceed its 5-second timeout; the identical top-source shard and the isolated exact test passed. One rerun of that failed job and its required aggregate passed without source or test changes. The original failed job log and the single-rerun receipt remain retained. ### October 9 merge verification Current merge head: `5a8fe63512a7166aaef5cf50065a25008aa8b44b`. All current-head checks pass, including `ci / verify` and `ci / e2e`; exact-head Greptile review is 5/5 with no unresolved threads. Current master conflicts are resolved. The user authorized the maintainer override of the code-owner review gate after these checks. The seven retained live core cases remain bound to source `3241a992f2a7703e59e97ed0fd3e5d6405de4401` and its recorded cloud image. ## Risks - The security correction changes the dependency closure and profile identity. Old sessions must reopen on the new profile. Exact identities and credential bindings fail closed. - The runner remains experimental and requires explicit selection. Legacy Pi Local is unchanged. Caller model IDs pass through; the E2E model is a fixture. - Accounting and the broad platform/provider matrix remain deferred. This merge does not publish a release or deploy a service. ## Model Used OpenAI GPT-6 through Codex assisted with reasoning, repository inspection, editing and tool use. The exact serving ID and context window are not exposed in this session. Final live qualification uses Pi 1.0.0 with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
428b617fc9 |
test(evals): qualify native question resume paths (#15616)
## Thinking Path > - Paperclip lets agents pause tasks for human answers and continue the work. > - Native questions can yield a run or pause inside the provider's running turn. > - Those paths have different run identities and different forms. > - The documentation-placement test requires a semantic three-turn journey. > - A valid provider question therefore needs separate behavioral qualification. > - This pull request adds an opt-in two-answer journey with exact source and resume checks. > - Qualification exposed a Vite startup crash in idle-handler traversal, so the branch also distinguishes Connect mount paths from Express Route objects. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E evaluation of native task questions and answer delivery. **Current behavior** The semantic documentation case rejects the provider adapter's optional Other field before answering. Its three-run expectation also cannot prove a provider question that resumes within the same run. **Proposed behavior** Keep that semantic test intact. Add a separate opt-in case that classifies each recorded question by authoritative identities. Accept the optional Other field only for a verified provider question. Require both user answers, the correct continuation, and one saved final document. Related: #15564 added strict lifecycle checks. #15522 introduced idle-request tracking; this PR corrects its Connect/Vite compatibility after reproducing the startup crash. #14591 changes ACP question drafts and provider support; this PR has no overlapping files with that change. ## What Changed - Add the explicit-only `question-resume` suite for native Codex and ACPX Claude. Reuse the existing neutral choice-then-text task request. - Bind each question to its company, task, agent and run. Require an applied semantic tool receipt or the exact provider request identity. - Require a provider answer to continue the same run. Require a semantic answer to start one new run with the matching interaction and source-run wake fields. - Preserve question forms and both real board answers through the final checkpoint. Reject missing inputs, duplicate answers, extra runs and substituted identities. - Keep the existing lifecycle, output, task ownership and budget checks. Limit the new case to one attempt and one to three runs. - Add mutation calibrations and document the separate qualification claims. Keep production guidance, scheduling, provider code and the historical semantic case unchanged. - Fix startup with Vite middleware: treat Connect string mount paths separately from Express Route objects when traversing idle-request handlers. A real-Vite regression verifies startup and retained async work through an idle hold. ## Verification - 81 focused continuation, question and calibration tests passed before review. After the reference-preservation correction, all 50 question/resume tests pass, including two new negative cases that failed against the previous grader. - Final eval support suite passes: 1,921 Vitest tests and 128 Node tests, with one intentional skip. - Eval and workspace typechecks pass. Workspace build passes. - Catalog discovery finds only the two declared opt-in cells. Existing continuation remains 23 cells; default paid scope is unchanged. - Original [campaign 37831726391](https://github.com/paperclipai/paperclip/actions/runs/37831726391), source `1ac4b2873466896ff2892f49be82904927b6c57d`, failed during server startup before any test or provider run. Its original `transient_infrastructure` result and `cleanup: not_started` are preserved. Local reproduction identifies the Connect string route traversal crash; the question grader was never reached. - Setup-corrected [campaign 37833872898](https://github.com/paperclipai/paperclip/actions/runs/37833872898) measures source `9781a772389d41421df55759004676e1b905e512` with trusted workflow `8e59efc50b162126f33816841e06509634eba65c`. The workflow bytes are unchanged from the first dispatch. Same single selected native Claude cell, one attempt, at most three runs, ten-minute cell deadline, and 1,000-cent company/agent hard stops. Original result **PASS**, 24/24 checks and cleanup pass. Exactly three successful native Claude runs: two applied Paperclip `request_human_input` questions and one finishing run. Both real board answers persist, exact interaction/source-run response wakes match, one revision-1 task document contains Afternoon and the supplied reference, and final task state is done with no active lock, retry, recovery, monitor, pending interaction or child task. No model retry. - Current review head `3c08cb7d26e7f9ac16469eeb07d1f31d828ec02e` differs from the measured source only in the question/resume grader and its tests (11 insertions, 3 deletions). The saved document must contain the complete submitted reference, including its prefix and punctuation. Separate provider-free replay of the retained successful campaign passes all 24 continuation/lifecycle checks under this stricter grader. The original result and its measured source remain unchanged; no new provider run was made. - Observed paths were **semantic → semantic**. The provider built-in question/optional Other path was not exercised by this new live attempt. Its form acceptance has retained-original calibration; same-run answer/resume has deterministic positive/negative calibration. This neutral-task pass alone does not qualify the provider path; the subsequent explicit bridge trial below is separate evidence. Neither trial regrades the old Claude failure or establishes a causal behavior/performance comparison. - Subsequent explicit provider-path [campaign 37841107684](https://github.com/paperclipai/paperclip/actions/runs/37841107684) measures the final PR source `3c08cb7d26e7f9ac16469eeb07d1f31d828ec02e`, using trusted workflow `2a5f65c9ae501b49b2a38ce7edd04209b806c9d0`. Exact cell: `continuation.runner-acpx-claude.local.provider-question-bridge`, Claude `claude-sonnet-5`, suite fingerprint `fc77203bf85fd99b06ebc53cd52982186c0ae0dc369fcc38a9ec3c737e9a31e5`. **Original PASS, 15/15 checks, cleanup passed, one actual provider run and zero eval rerolls.** The built-in question produced one choice plus its optional Other companion. The browser selected the requested reference, left Other blank, and submitted. Exact runtime request/response IDs and the selected option match the saved interaction; the same paused native run created one revision-1 document and finished Done with no pending interaction, lock, retry, recovery, monitor or child task. Independent evidence and runtime-receipt audits pass; marked initial/final screenshots were visually checked. [Original public report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37841107684-1/index.html). - This separate provider-directed trial qualifies one choice-answer bridge and traversal of an empty optional Other field. It does not establish natural tool selection, a typed Other answer, two consecutive built-in questions, free-text-only input, restart recovery or general reliability. No production, prompt or grader change was made for this dispatch. The old failure and the separate neutral semantic-path result retain their original grades. - Preserve within-run friction: one `write_task_document` input-schema rejection and two HTTP 409 “Document does not exist yet” responses preceded a successful `write_document` call. These are three unsuccessful tool calls inside the same run, not three extra provider runs; no flawless document-tool behavior is claimed. Billing reports token usage but remains unpriced/incomplete, so actual charges are unknown and local/hosted runtime is unmetered. Both company and agent hard stops were 1,000 cents, with one attempt, one selected cell, concurrency one and a ten-minute cell deadline. - Billing records all three model runs with token usage, but no priced dollar receipt (`unpriced`, `complete: false`). Actual charges are unknown; local/hosted runtime is unmetered. Numeric zero reported cost does not mean free. - Startup repair: real-Vite reproduction passes; 35 targeted idle-tracking tests pass, 34 database-dependent checks skip because embedded PostgreSQL is unavailable locally. Workspace typecheck and build pass again after the repair. - Full local repository database tests were not repeated because embedded PostgreSQL was unavailable in the preceding workspace verification. Full Linux CI now passes on the final review head. - Final-head [CI run 37837804284](https://github.com/paperclipai/paperclip/actions/runs/37837804284) passes. Complete current-head audit: 53 successful check-runs, two intentional Storybook skips, and separate Snyk success. The ready transition’s contributor and security checks also pass (security scan: no findings). [Fresh review](https://github.com/paperclipai/paperclip/pull/15616#issuecomment-6068092601) is 5/5 on `3c08cb7d26e7f9ac16469eeb07d1f31d828ec02e`; zero unresolved threads and no merge conflicts. ## Risks This is a new behavioral definition. The neutral two-question trial records semantic-tool use; the separate provider-directed trial records one built-in question. They do not establish consistent tool selection, documentation placement, default hiring, remote environments, crash recovery or general reliability. The separate explicit bridge trial covers one provider choice, an empty optional Other companion and same-run completion. Typed Other, two consecutive built-in questions and free-text-only provider input remain unqualified. The observed document-tool schema rejection and two 409 responses are retained as a separate follow-up, without attributing their cause. The historical case and its original grades remain intact. The reference question must still be text-only; the oracle does not turn an arbitrary extra question into an optional companion. Missing or unfamiliar identity evidence fails closed. The only production change distinguishes Connect string mount paths from Express route objects during idle-handler traversal. Unknown route objects still fail closed. No API, schema, migration or workflow change is included. ## Model Used OpenAI Codex, GPT-6-based, with tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
24a15c5afa |
test(evals): verify durable waiting across continuation checkpoints (#15564)
## Thinking Path > - Paperclip manages agents and durable tasks across provider runs. > - Human questions must preserve task ownership and stop work until a real answer arrives. > - The continuation eval checks the final work, but its lifecycle oracle misses several broken waiting states. > - A correct final answer can hide a stale execution lock or a lost intermediate answer receipt. > - This pull request checks each wait and retains every question and run identity. > - A source audit records which waiting operations belong to native runtimes and which still require legacy API calls. ## Linked Issues or Issue Description **What existing behavior does this improve?** The existing Product E2E continuation oracle for human question and approval journeys. **Current behavior** A pending interaction can pass the waiting check even when its task has the wrong status, a stale lock, or a retry. Only the first pending question and first run receipts are checked at the final checkpoint. **Proposed behavior** Require a healthy wait on the same task and assignee. Preserve all intermediate run receipts and every pending question's answered identity. Allow a paused native provider question only when its pending runtime request identifies the running native run and execution lock. Related: #15548 and #15554 cover earlier bookkeeping slices. #15544 changes production continuation summaries; this PR changes the eval oracle and does not overlap that fix. ## What Changed - Check waiting state, execution ownership, and all answer/run identities in the existing lifecycle oracle. - Add negative calibrations for broken records and positive coverage for both semantic waits and paused provider questions. - Retain task activity at each checkpoint for inspection of successful persisted mutations. - Verify 1,000-cent company and agent budget limits before continuation work. Disable automatic cell rerolls. - Document runtime ownership, existing coverage, and the remaining instruction decision. Keep production instructions unchanged. ## Verification - `pnpm test:e2e:runner:unit`: 1,867 Vitest tests pass, one skips; 128 Node tests pass. - `pnpm test:e2e:runner:typecheck`: passes. - `pnpm -r typecheck`: passes. - `pnpm build`: passes. - Negative calibration: 27 added cases fail against the prior oracle and pass with these checks. - Four existing local continuation cells are selected for a separate bounded live canary. Live results are pending; no behavioral pass is claimed here. - The full local `pnpm test:run` suite was not repeated because embedded PostgreSQL was unavailable in the preceding workspace verification. Required Linux PR CI must pass before readiness. ## Risks The stronger oracle can expose existing product or fixture defects. A paused provider question and a terminal semantic wait have distinct valid states. The four-cell canary does not qualify approval/review, dependency unblock, crash races, remote execution, or general task quality. Task activity records successful persisted writes, not failed API attempts; repeated progress comments are not automatically defects. Historical eval grades remain unchanged. No production scheduling, prompt, tool, schema or migration changes. ## Model Used OpenAI Codex, GPT-6-based, with tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d0db8820db |
fix(connections): repair native baseline and approval continuations (#15420)
## Thinking Path > - Paperclip manages AI agents and the tools they may use. > - Connection setup separates provider preference from permission to use a tool. > - The first native connection baseline could not exercise its intended decisions. > - The browser used mutable task titles, and the provider fixture already granted access. > - Native provider-choice instructions also disagreed with the preferred question format. Schema rejection gave no field guidance. > - This pull request repairs those test preconditions and native guidance, then fixes restart/approval defects exposed by the corrected baseline. It also restores missing OpenCode tool-error evidence. > - The benefit is an inspectable baseline before any further instruction reduction. ## Linked Issues or Issue Description Refs #15407. The original 15-cell baseline remains 0 PASS / 15 FAIL. Ten cells stopped on stale titles, two Codex cells had schema denials, two OpenCode cells used already-granted tools, and one Claude cell returned no native result. No intended user decisions were submitted. The exact invalid Codex field and underlying Claude failure cause remain unknown. [Original campaign](https://github.com/paperclipai/paperclip/actions/runs/37562577199) · [Original report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37562577199-1/index.html) ## What Changed - Match the browser's task route and visible identifier instead of a title the agent can change. - Start native provider-choice fixtures with no agent tool access. Verify the public effective-access records. - In the positive case, select Arcade, then grant its exact HubSpot tool through the real access card. Require both saved decisions and exactly one observed call. - Return a canonical `providerQuestionSet` for native input and retain the equivalent legacy `providerQuestion`. - Keep invalid input rejected. Return bounded schema locations and required field names without submitted values. - Preserve Claude's exact session/content identity while allowing authenticated registered instruction-copy paths to rotate on a new run. - Reject duplicate approval reports for an existing exact tool-action card before they create another human review. - Wait for a recorded service-approval continuation within the existing deadline; retain missing or failed continuation grades. - Forward OpenCode tool activity through the runner facade, preserving bounded errors and execution-part identity without inventing host-call joins or exposing arguments. - Preserve original grades, costs, scope limits and diagnoses in the dated repair report. ## Verification - Eval typecheck passes. Support suite: 1,801 PASS, one intentional skip; Node checks: 128 PASS. - Connection/schema tests: 51 PASS. Real-server public fixture setup: one PASS with zero providers. - Browser support regression: five PASS, including renamed and wrong tasks. - Focused Rust safe-feedback test: one PASS. - Repository typecheck and build pass before the latest master replay. Post-replay connection/shared/real-server fixture checks: 52 PASS; eval typecheck passes. The browser review fix additionally passes all five browser checks and seven suite checks. - The full local repository run was interrupted incomplete after about 45 minutes, with five integration failures retained. All five pass in a separate targeted invocation (1,250 unrelated tests skipped). No full local-suite pass or root cause for the initial local failures is claimed. - Corrected frozen source `162cc90fdabe7f505b88ae095044531b82784c92`: **10 PASS / 5 FAIL** across the [passing Codex canary](https://github.com/paperclipai/paperclip/actions/runs/37575158761) and [remaining 14 cells](https://github.com/paperclipai/paperclip/actions/runs/37576261807). The canary passes all 17 checks. Claude's two provider-choice continuations fail on restart, Claude service approval exposes an early evaluator rejection, Codex service approval creates a duplicate approval, and OpenCode provider-second times out after both decisions with no HubSpot call. No original result is regraded. - Final ledgers count 31 actual runs: 27 succeeded, two failed, two cancelled during cleanup. All 15 cleanup/budget checks pass. The late Claude continuation is absent from its earlier workflow snapshot; it remains in the result/API/final ledger. Original evidence retains 279 hashes. Recorded LLM subtotal $0.04553787 is incomplete billing, not actual total cost; local runtime is unmetered. - New repair regressions reproduce the Claude attach failure, duplicate approval acceptance and dropped OpenCode tool events before their respective fixes. Nine Rust attachment checks, 127 ACPX host/adapter tests, 33 completion/control-plane checks, nine eval deadline tests, 59 OpenCode proxy/driver tests, one Rust tool-error/redaction check, and TypeScript/Rust composer parity pass. Eval typecheck, repository typecheck and build pass. Existing support coverage is 1,802 PASS plus 128 Node PASS, one intentional support skip; two additional deadline tests also pass. - New-source full CI/review and live canaries are pending. The next bounded selection is Claude provider-decline, Codex service-approve and one OpenCode provider-second diagnostic with repaired event evidence. No broader campaign or instruction-reduction qualification is claimed. - Initial corrected campaign [37574251834](https://github.com/paperclipai/paperclip/actions/runs/37574251834) was cancelled during shared build after review found the breadcrumb whitespace assumption. Its matrix job has zero steps and no provider execution. The real adjacent-span browser regression now reproduces the old failure and passes after the fix. ## Risks - The corrected baseline remains 10/15. The new restart/approval fixes require live qualification; OpenCode evidence forwarding does not itself establish or fix its prior behavioral failure. - The positive provider case now expects three runs, including separate access approval. Its new results are distinct from the original invalid fixture. - The old Claude missing-result cause and rejected Codex field are unknown. These repairs do not retroactively explain or erase either failure. - Path rotation must preserve prompt, custom instruction, skill/content identity and protected provider settings; regression checks reject stale or changed content. No connection authorization, JSON schema, budget, cleanup, or final-result requirement is relaxed. Historical Everyday prompts and gateway setup remain unchanged. ## Model Used OpenAI Codex, GPT-6. The exact deployment variant and context window are not exposed in this session. Used repository inspection, code editing, test execution and retained-evidence analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
caf120105c |
test: prepare neutral native connection guidance evals (#15407)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents need to discover connections, obtain consent, and continue from saved decisions. > - We want to reduce repeated instructions only when measured behavior supports the change. > - The existing decline tasks tell the model not to retry. One provider-decline check can pass without an explanation or an observed service counter. > - This PR adds neutral tasks and stricter saved-evidence checks before any connection instruction reduction. > - Production instructions remain unchanged. The new cells are configured, not live-qualified. ## Linked Issues or Issue Description Refs #15218. Refs #15389. **What existing behavior does this improve?** The Product E2E connection workflow evaluation and its instruction measurement provenance. **Current behavior** Some decline prompts supply the policy they intend to test. The provider-decline workflow does not require a saved post-decision explanation. Its old no-call check can use a missing fixture counter as zero. OpenCode has no connection cases in the original Everyday matrix. **Proposed behavior** Add an explicit-only suite with five connection stories on native Codex, ACPX Claude, and OpenCode. Require an explanation attributed by exact run ID after a saved decline. Observe the provider fixture counter. Preserve the original cases and grades. ## What Changed - Add fifteen configured cells with one attempt, twelve-minute deadlines, and verified 1,000-cent company and agent budget stops. - Remove procedure hints from the three new decline prompts. Keep a user-permitted explanation fallback and the existing positive controls. - Require saved decline state, one decision, unchanged connections, observed zero service calls where applicable, and a post-decision explanation from a successful run on the same task. - Add negative grader calibration and test the actual fixture budget payloads. Exclude the suite from default and generic selection. - Extend the existing full-catalog measurement source manifest with connection descriptions and schemas. Add an audit of fixed text, tool descriptions, returned instructions, and unqualified behavior. - Rebase on master `a6306ba606eb87c89b9ef0344e9fe8e0025580f9` and preserve its new Cursor suites. No production, credential, workflow, or lockfile change. ## Verification - Before rebase: Product E2E support passed 1,424 TypeScript tests and 128 Node checks. Six catalog measurement tests, repository typecheck/build, Product E2E typecheck, and exact fifteen-cell discovery passed. - The full pre-rebase repository test run was stopped when master advanced. Its partial result is not a pass. - After rebase and the review correction: repository build/typecheck, Product E2E typecheck, 1,799 TypeScript support tests (one skipped), 128 Node checks, six measurement tests, and exact fifteen-cell discovery pass. The duplicate local full-suite run was stopped incomplete after about 20 minutes once complete CI passed; no local full-suite pass is claimed. - Review found that the initial grader read `runId` instead of public `createdByRunId`. A regression calibration reproduced both rejection of valid public comments and acceptance of the wrong alias. The fix uses the actual field and binds the evidence type to the shared `IssueComment` contract. A subsequent type-only import path correction passes Product E2E typecheck. - Final source `0de306b9664bfbdebb6709ddb54c95152740d1ad` passes [complete CI](https://github.com/paperclipai/paperclip/actions/runs/37560250545): 51 successful checks and two intentional Storybook skips, plus separate Snyk success. Fresh Greptile review is 5/5 with the single review thread resolved and no new findings. The PR is clean and mergeable. - Local commands: `pnpm build`, `pnpm -r typecheck`, `pnpm test:e2e:runner:unit`, `pnpm test:e2e:runner:typecheck`, and `pnpm test:e2e:runner -- --list --suite native-connection-guidance`. The measurement uses `PAPERCLIP_NATIVE_PROCEDURE_MEASUREMENT=/tmp/connection-measurement.json pnpm exec vitest run --project @paperclipai/server server/src/__tests__/native-procedure-measurement.test.ts`. Validation used pinned pnpm 9.15.4. - No paid provider campaign was started. There is no baseline/candidate behavior result for these new cells. - The audit records 654 UTF-8 bytes of fixed connection guidance. A clean capture at `a04b8c6a452315625014888335d45670a2094fb6` confirms 41 supplied tools, 53,341 normalized bytes at start/resume, 50,949 at compact continuation, and a 48,195-byte authenticated OpenCode MCP catalog. These are byte counts, not tokens, bills, vendor-private prompt sizes, or savings from this PR. ## Risks - This is eval preparation. Passing support tests do not establish live model behavior or qualify an instruction reduction. - The explanation oracle checks attributed saved output. It does not prove cognition or arbitrary prose truthfulness. One saved interaction also does not prove the absence of repeated idempotent tool calls. - Successful new authentication and tool refresh, existing-connection agent grants, independent work while waiting, explicit retry after decline, and blocking when mandatory work remains still need separate coverage. - Notion setup decline does not execute a real Notion service. Positive service approval uses an already installed deterministic service; it does not qualify new connection creation. - The original historical failures remain unchanged. Future comparisons must freeze source, fixture, model, input, and grading controls and retain every actual attempt. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The exact serving model ID and context-window size were not exposed in this session; they are not inferred. No model provider was invoked by the eval suite in this PR preparation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a6306ba606 |
feat(runner): consolidate Cursor production integration (#15075)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native Runner keeps provider sessions under company authority, approvals, budgets and durable recovery. > - Cursor work was spread across candidate branches. The published branch lacked later plan, permission and cleanup fixes. > - Production also needs public installation and matching runtime assets for local and Daytona execution. > - This pull request consolidates Cursor onto current mainline recovery behavior and completes that installation path. > - The installed v11 release passed focused local and Daytona qualification after the generic mode and lifecycle cleanup. The later model-selection correction and current mainline merge produce v14 artifacts that need matching release qualification. > - Cursor admission is enabled in source; publish only an artifact combination with matching qualification. Native AskQuestion and complete per-run dollar accounting remain excluded. ## Linked Issues or Issue Description Refs: #14435, #14631, #14669, #14699, #14724. This completes the Cursor implementation by @cryppadotta from combined source `22c78242a4e0c2369fecf0c2dc4e7600fbad6706`. It preserves newer mainline recovery, completion and warm-directory behavior. Pi and Copilot remain gated. ## What Changed - Generate named Rust and TypeScript ACPX release profiles from one manifest. Share runtime pins with packaging and server verification. Preserve vendor runtime versions; bind the updated ACPX patch to Cursor profile v14 and reject stale generated declarations at build/typecheck. - Remove ACPX model allowlists, including the former Codex and Pi restrictions and the duplicate developer test-drive gate. Send any explicit model ID unchanged to its provider and verify the effective selection before prompting. The bundled ACPX package forwards unlisted IDs, rejects mismatched acknowledgements, and restores the exact selection after session load. It does not expand Cursor model aliases. Provider rejection, mismatch, or missing model controls fails without a fallback. Model examples live in evaluation fixtures, outside runtime declarations. - Add pinned Cursor execution, contained instructions, exact model verification and Agent/Plan/Ask modes. - Carry an opaque generic `mode` identifier in shared native execution, sidecar, Rust and recovery contracts. The provider adapter owns supported modes, defaults, native translation and acknowledgement. - Keep native RPC recognition, accepted-plan interpretation and permission evidence behind provider adapters. Shared settlement and recovery verify normalized facts and their committed evidence. - Replace the Cursor-only warm-attachment branch with a runner-owned capability. Only Cursor opts into it. Move profile compatibility and optional usage parsing into provider metadata and adapters. - Write generic plan-wait receipts. Read exact historical Cursor receipts through a separate compatibility decoder. Reject mixed formats and preserve existing authority checks. - Carry native plans, semantic questions, todos, child activity, permission identities and partial usage diagnostics through the Runner. - Preserve durable response delivery, cancellation, warm ownership and process retirement. - Finish accepted planning runs successfully. Keep their tasks open for explicit direction. Acceptance does not start implementation. - Ship `paperclipai runtime setup cursor` and its provisioner through the public package. npm installation does not download Cursor. Setup uses the OS account's closure-keyed cache so system-wide npm packages can remain read-only. Run it as the Paperclip service account. - Include Cursor in normal provider packs and Daytona images for macOS ARM64/x64 and Linux x64. - Reject stale release packs by source revision and current ACPX/Cursor pins before assembly writes files. Verify current Cursor version/profile/closure again at runtime. - Ship all three daemon targets and the expected Linux image-pack identity. A macOS controller uses its packaged Linux daemon for Daytona. Image mismatches fail before provider launch. - Use the vendored Runner boundary for installed readiness probes. Verify the actual installed Cursor probe. - Verify compiled public Daytona plugins and their release versions in installed smokes. - Record exact artifacts, the acceptance matrix, retained failures, supported capabilities and rollback behavior in the [readiness report](https://github.com/paperclipai/paperclip/blob/codex/cursor-production-readiness/doc/plans/2026-10-03-cursor-production-readiness.md). ## Verification - Current head `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037` merges mainline `faa8e452c73bae5e044dd6379179a00106abb131`. It keeps Cursor plan and cancellation guards alongside mainline historical-question filtering. The evaluation catalog includes both Cursor and expanded adapter accounting cases (683 total). Recursive typecheck, full build, 696 lifecycle/recovery tests, 45 fixture tests and fixture typecheck passed. Current-head CI passed: 56 successful checks, one neutral and four skipped. [Complete CI](https://github.com/paperclipai/paperclip/actions/runs/37557996535). The fresh Base Greptile review is 5/5 on this exact head, with 304 files reviewed, zero new comments and zero unresolved threads. The user authorized overriding the CODEOWNER review gate after checks passed; no failing checks are overridden. Prior results below retain their own head identities. - Corrective head `3d2b168366258036f6b6a6fccb382c49138cc601` fixes the post-merge Apex finding. Automatic-review and new-evidence reconciliation preserve pending child results and recheck delivery under the status lock before completing. Account repair now excludes unrelated secret consumers and requires the failed agent's identity. Regression coverage includes the commit race, delivery statuses, current-run/current-intent exclusions, repeated reconciliation, both database reconciliation paths, and credential consumer boundaries. All 184 affected tests, server typecheck and server build passed. Current-head Base Greptile review is 5/5, with 304 files reviewed, zero new comments and zero unresolved threads. Current-head CI passed: 56 successful checks, one neutral and four skipped. [Complete CI](https://github.com/paperclipai/paperclip/actions/runs/37535994724). This Base review is distinct from the earlier Apex review. - Merge head `5957c257a` reconciles mainline `b508a05c4`. It preserves both accepted-plan waits and pending-child-completion checks, current provider selectors, task-creation response identities, and mainline ACPX missing-file handling. The combined patch is bound to Cursor profile v14; historical records keep their original identities. - Merge head `5957c257a` passed recursive typecheck, full build, 43 installed ACPX/package contracts, 107 provider UI and plan/recovery tests, 593 database-backed lifecycle tests, 49 profile/native contract tests, 45 Product E2E fixture tests, fixture typecheck, token gates, three provider-free browser task-creation cases, and Runner conformance/replay checks. Its complete CI passed (55 successful checks, one neutral and four skipped), while Apex returned 2/5 with a child-delivery finding addressed below. - The local full-suite attempt again failed the unchanged Git streaming test (360-second timeout) and was stopped. The concurrent local Rust attempt failed four unchanged Codex process/deadline tests; all four passed serially without code changes in 7.29 seconds after removing the competing test load. These failed commands are retained and are not reported as full-suite passes; the fresh Linux CI runs are tracked separately. - The previous head `907bdb2a2778c7ffeb4a662a91460c9d1ddfc9c5` earned Apex 5/5 with zero comments after fixing all three findings: per-user install cache, stale release-pack rejection, and public Linux smoke account/home handling. Its real built installer passed from read-only public packages on macOS ARM64 and Linux x64. All 137 release-registry checks and 64 ACPX package contracts passed. That review does not cover this mainline reconciliation. - Prior `beadd3654` passed the full CI matrix; its one unchanged chat test failure and successful single retry remain in the [CI history](https://github.com/paperclipai/paperclip/actions/runs/37521449327). Historical results below remain attributed to their original builds. - Fixture follow-up `dd59d7e82b103a88b7cbd7d2c38b612c0fbbff7a` removes provider-specific model choices from generic offline ACPX tests. The fake sidecar preserves the model and session identity selected at open through suspension. Affected verification passed: 106 Rust tests and 73 TypeScript tests. This commit changes test code only; the production-code checks below retain their recorded identities. Its CI and Greptile review later passed; those results belong to that historical head. - Model-selection cleanup `9a070808b48960a41fdfd369ae0636b95af82459`: 252 focused Runner tests passed (six platform skips), covering all six ACPX agents, native model acknowledgement, rejected selections, installation integrity and recovery identity. The merged branch passed recursive typecheck, full build, token gates, server admission (19 tests), and the Product E2E catalog (45 tests). The acceptance catalog passed all four tests. The full Rust suite passed: 643 tests, 2 ignored. It verifies sidecar acknowledgement of unlisted models and rejection of model mismatches. The final commits only update Rust tests; production sources match the verified build at `65ec3279ac50185e3cda109b5cfd9b4f56105de0`. No new paid provider calls were made. - The merge preserves both Cursor and the new mainline public-MCP fixture cases. Auto-merge remains disabled; the latest follow-up status is recorded above. The local `pnpm test:run` attempt hit the unchanged Git streaming test's 300-second timeout and was interrupted before merging mainline. The broad Runner attempt found obsolete single-model assertions plus three macOS fixture-path failures caused by a `/private/tmp` override. The assertions are corrected; affected TypeScript checks passed with the standard macOS temporary directory, and the complete Rust suite passed. Neither interrupted command is a full-suite pass. - Earlier declaration-cleanup head `6f4a5e9e2` passed recursive typecheck, build, Rust and focused tests. Its CI later exposed a test expecting duplicated Grok digest literals. The current source fixes that assertion to compare launcher bytes with the shared manifest. Historical successes and failed attempts are retained; no new live provider qualification is claimed. - Previous head `e75fde6098b0ddd8cec765bfb6ecaeecb88a26a6` passed complete CI (56 successful checks, one neutral, four skipped) and Greptile 5/5. [Historical complete CI](https://github.com/paperclipai/paperclip/actions/runs/37489112305). Those results are not claimed for the cleanup head. - Frozen live application: `d7b696f9b8f79095233e9e3d56d23e6a6018dd48`. Public package version: `0.0.0-cursor-verify.3d0c9b7761c6`. The declaration cleanup preserves release pins and does not relabel that tested artifact as a build of the new source. Mainline through `e34abee670` was reconciled while preserving accepted-plan waits, provider-capacity handling, and both Cursor and public-MCP fixtures. - Clean normal installation, explicit Cursor setup and daemon resolution passed on macOS ARM64, macOS x64 under Rosetta, and Linux x64. npm lifecycle hooks ran without silently downloading Cursor. - Historical v11 live matrix: **18/18 passed with cleanup** (nine local, nine Daytona) after the generic mode and lifecycle cleanup. The campaign has 23 attempts; all five failures and their diagnoses remain recorded. Exact case identities, hashes and limits are in the readiness report. All provider calls are real, use the explicit Luna model and company-bound credentials, and run without qualification or runtime-asset overrides. - The immutable Daytona image is `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:d6259b6bba094702c13fc2283bd85550849c1c53145b656fb2746778f9fa1747`. The public Daytona plugin is installed independently and its version is checked. - Recursive typecheck, full build, token gates and Runner contract/conformance/replay checks passed on the frozen application. Its complete Linux CI suite passed. The duplicate local full-suite command was incomplete after timing failures; affected repeats passed, but that command is not reported as a clean pass. - Qualification fixtures passed typecheck, 1,675 Vitest tests (one skip), 128 Node checks, three provider-free browser tests, and 150 focused lifecycle tests after the final diagnostic correction. The affected legacy Cursor command file also passed all five tests after removing its shorter 10-second override; it now inherits the suite’s standard 15-second timeout. Greptile is 5/5 on `e75fde609` with no unresolved review threads. CI results above are recorded separately from historical build results. ## Risks - Cursor v14 includes the updated ACPX dependency patch and release identity. The v11 live matrix and image below remain historical evidence. They do not certify new v14 package/image artifacts. - ACPX accepts models beyond the qualification fixtures. Availability and entitlement depend on the provider. Successful configuration is not a claim of live qualification for every model. - Shared mode is an opaque identifier. Provider adapters own its meaning. Incompatible historical sessions remain fenced; exact committed plan waits and task history remain inspectable. - Native AskQuestion is excluded. Paperclip semantic questions are supported. Authoritative per-run dollar accounting is unavailable; partial counters remain diagnostics and unknown cost is not zero. - Image input, detailed native diffs, deeper child transcripts and native plan-file export remain follow-ups. - macOS x64 has clean-install and daemon-startup proof under Rosetta, not a separate live campaign on Intel hardware. - Release only the tested package/image combination. Merging this PR does not publish npm packages or deploy that image. Later builds need their own release verification. Rollback disables new Cursor admission while preserving records and recovery inspection. - A model can fail an exact instruction: one cancelled-plan attempt returned the wrong summary marker despite correct cancellation. The unchanged repeat passed; both results remain in the report. > ROADMAP.md was checked. This completes existing native Runner/Cursor work; it does not add an independent core feature proposal. ## Model Used OpenAI Codex, GPT-6. The exact serving variant and context window are not exposed in this session. The agent used reasoning, repository inspection, code execution, protocol tests and browser-backed Product E2E tools. Cursor acceptance uses the explicit `gpt-5.6-luna[context=272k,reasoning=medium,fast=false]` model. That is the evaluated provider model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — affected suites passed; full CI and the retained local failed attempts are recorded separately above. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — 56 successful checks, one neutral and four skipped on `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037` - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — fresh Base review passed on `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037`; zero new comments and no unresolved threads. The earlier Apex finding remains fixed. - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e38d6d16b6 |
feat(connections): add advanced provider setup and live browser qualification (#15341)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Users connect accounts and choose an agent harness and model. > - The runtime change in #14970 supports custom providers on those connections. > - Normal setup must stay simple while advanced users can choose a compatible gateway. > - Shared connector rows and access controls keep these choices consistent. > - This pull request refines the agent setup UI and adds review stories and repeatable browser qualification. > - The qualification checks real tools and downloaded outputs, not only a successful run status. ## Linked Issues or Issue Description Refs #14970, #37, #13083, #14104, #14565, #12692. The core implementation in #14970 is merged. This branch incorporates its squash commit and targets `master`. Both PRs contain our implementation. #14016 is a reference only and is not a dependency. This PR has 96 changed files. ## What Changed - Complete model-provider connector presentation beside other connectors. Each row uses the existing Connect action and connection list. Tags are stored without category UI. The base PR includes the provider forms and routes. - Show persistent Subscription, API Key, and Advanced choices. Label Advanced as Custom Gateway. Reuse provider logos, connection lists, and permissions controls. Default access to the organization and all agents when permitted; keep narrowing controls under Advanced. - Keep Configure reachable before subscription sign-in, so users can select a supported environment when the default cannot sign in. Testing and saving still require a connection. Show the execution environment in Configure. Preserve the confirmed Connect choice. Editing a method, credential, saved account, or advanced choice requires that current choice to connect before testing or saving. Use matching model and thinking-effort dropdowns and retain connection icons in selected values. - Preserve the new harness model default when switching an existing OpenCode agent to Codex or Claude, and resolve user-selected model names with the effective harness. - Load popular OpenRouter models through the shared connection-model discovery path. Keep explicit model lists and manual model entry available. - Group onboarding, connection setup, agent runtime, management, recovery, and production-component stories under AI Connections / Provider routing. - Add an explicit-only provider-connections browser suite for managed local or existing local/staging targets. Use private browser profiles and credential handoffs. Support human-assisted subscription sign-in without sharing passwords or tokens in reports. - Verify persisted connection identity, runtime probes, tool execution, exact artifact bytes, completion, and context-dependent follow-up. Retain source/model provenance, cost bounds, closed error diagnostics, original failures, and cleanup evidence. - Add Gemini startup-model and skill-root fixes, Grok private-history detection, ACP filesystem regression fixtures, selected-workspace handling for local Hermes, and artifact-helper workspace fallback. - Keep managed Grok runtime homes disposable. Remove host-side transcript retention/restoration because private file modes do not isolate same-user agent processes. Ignore earlier development archives and use a fresh task handoff when history is unavailable. Verify the absence of restored transcripts with a separate same-user process. - Capture stopped-run diagnostics before deleting an attached-company fixture agent. Track creation and owned sign-in receipts; revoke only this attempt's accounts and never adopt a concurrent campaign's newly created account. Preserve failure signals and final status through cleanup. - Require the requested environment in the saved agent and every run, including follow-ups. Reject a forced incompatible target. Keep one cancellation state through startup, every cell, reporting, and teardown for SIGINT, SIGTERM, and SIGHUP. Stop further paid cells after interruption. Document qualification limits. ## Verification - Current head `b3bb3e94d577d43d9965a6b9daba039f599b2e49` includes master `d9f600043`. The security fix in `a758fde31` passes full workspace typecheck, production build, and 119 connection/Grok regressions. The unchanged UI passes all 126 configuration/model-discovery tests and token gates. The final published-guide correction passes Grok adapter typecheck. Earlier head `eebd8225c` passed the complete deterministic runner suite (1,404 Vitest tests and 128 Node tests) and all CI jobs. Current-head CI run `37520147514` passed all 47 jobs, including the full sharded Vitest and browser matrix, production build, and canary dry run. All 55 checks completed: 53 successes and two expected skips. The current-head security scan passed, Greptile is 5/5, and no review threads remain open. - A separate same-user process reproduced reading a restored Grok transcript before the security fix. The regression now finds no transcript. Existing fresh-session fallback and ordinary session metadata behavior pass. - The final account-choice and cleanup fixes pass 85 setup tests and 26 qualification-harness tests. Regressions verify that editing a connection invalidates confirmation, Configure remains reachable before sign-in, diagnostics are captured before fixture deletion, and concurrent campaigns cannot adopt or revoke each other's accounts. UI and E2E typechecks pass. - The Storybook build and actual Chromium production-component stories passed during this change. Review the neighboring AI Connections / Provider routing stories, regular connector rows, three connection modes, model discovery, and the single execution-environment control in Configure. - Cancellation smoke verified authenticated cleanup before browser close for SIGINT, SIGTERM, and SIGHUP. Regressions cover interruption during startup and reporting, missing-file ACP resource errors, and preserved permission denials. Both ACP runtime versions and 54 ACPX/Grok regressions passed. The deterministic connection-intent browser suite passed two tests. - Historical local qualification retained 43 passing API/gateway cells out of 46, with downloaded outputs and follow-up receipts. These attempts span earlier builds; they do not qualify this exact commit or staging. Subscription combinations, Gemini overloads, and the unresolved follow-up failure remain recorded rather than counted as passing. - Use `pnpm test:e2e:runner -- --list --suite provider-connections` to inspect the matrix. Follow `tests/runner-e2e/PROVIDER-CONNECTIONS.md` for credentials, target URL, sign-in assistance, budget, evidence, and cleanup. Paid live tests remain opt-in. ## Risks - The core implementation in #14970 is merged. This PR adds no database migration of its own. - Subscription login needs an interactive provider session. Dedicated accounts and staging qualification remain follow-up work; this PR does not certify every login combination for production. - Managed Grok transcript resume is deferred until provider history has an OS isolation or authorized broker solution. Follow-ups start fresh with Paperclip task context; earlier live Grok results do not qualify this behavior. - Gemini CLI 0.58.0 has an upstream ACP new-file error conversion defect. Live overloads and one unresolved follow-up timeout remain recorded. The stock CLI is unchanged, and those cases are not marked as passing. - Real-provider tests spend credits and use private credential/evidence directories. The launcher requires explicit selection and checks target ownership. It must not attach to a developer's database by accident. - OpenClaw Gateway, Hermes Gateway, Claude Managed, AWS AgentCore, Process, HTTP, and legacy ACPX local remain outside custom provider setup. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser testing. The exact deployment model ID and context window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e34abee670 |
feat(mcp): connect assistants to a team with user OAuth (#14846)
## Thinking Path > - Paperclip gives teams durable tasks, agent execution, budgets, and approvals. > - People also use assistants in Codex, Claude, and other MCP clients. > - Those assistants need a scoped connection that preserves the person’s permissions and attribution. > - Delegating a task must not turn the assistant into the assigned agent. > - This PR adds opt-in user OAuth, ten first-party tools, browser consent, and workflow packages. > - Paid product evals verify the resulting tasks, documents, attribution, retries, and access boundaries. > - The team keeps working after the assistant conversation ends. ## Linked Issues or Issue Description **Problem or motivation** A person cannot connect an external assistant to an existing team through browser consent and safely delegate durable work as themselves. **Proposed solution** Expose an opt-in `/mcp/paperclip` endpoint with individually described first-party operations. Bind every connection to a person, client, company, resource, and scopes. Reuse domain authorization and scheduling. Package shared team-review, delegation, and follow-up workflows for OpenAI/Codex and Claude. **Alternatives considered** Related PRs #9393 and #12549 cover earlier remote MCP and board-operator approaches. This change uses user OAuth and a bounded public catalog. It does not expose a generic executor, operator administration, static shared board credentials, or external agent execution. Registry listing work in #9851 is a separate distribution step. **Roadmap alignment** This maintainer-requested implementation extends the governed MCP gateway, activity attribution, durable work products, and hosted deployment direction in `ROADMAP.md`. It implements the first release of the saved design plan; external agent participation and granted third-party tools remain later releases. ## What Changed - Add MCP 2.0 discovery and task status/comment/document Events on the same authenticated endpoint. Persist subscriptions and delivery receipts, verify HTTPS callbacks, sign Standard Webhooks, encrypt callback material, recheck permissions/Cloud membership, and bound retries/expiry. Older MCP clients keep their existing tools. - Add discovery, dynamic client registration, S256 PKCE, resource validation, rotating refresh tokens, revocation, and company consent. Store credentials as hashes and recheck membership at execution. - Add tools for connection identity, agents/projects, task search/read/create, human comments, documents/deliverables, and pending-approval links. Preserve current domain permissions and scheduling. - Add durable mutation receipts across reconnects. Matching retries replay results; uncertain outcomes keep the same request ID and require inspection. - Add consent and connection-management pages, OAuth log redaction, shared plugin workflows, and separate OpenAI/Codex and Claude package outputs. - Add eight paid Product E2E cases across three models, independent durable-state grading, usage evidence, cleanup, and report integration. Add task-document guidance and regenerate the runner capability inventories. - Add migrations 0301 and 0302, the dated implementation plan, result notes, and direct-client setup instructions in `doc/public-mcp.md`. ## Verification - Merge integration `e180b1948`: resolved conflicts with current master, preserved both eval registries, regenerated capability catalogs, and regenerated migrations as 0301/0302 while keeping the original replay-safe SQL byte-identical. Local migration safety/snapshot tests (26), MCP/OAuth tests (38), redaction/OpenAPI tests (71), and eval catalog/grading tests (198) pass. Token and capability gates pass. Full recursive typecheck passed. Fresh Greptile review is 5/5 with no unresolved findings. CI is green on this exact head (55 successes, two intentional skips, one neutral result): one unchanged Cursor sandbox test timed out at 10 seconds, then passed locally in 856 ms. A single retry of that failed shard and the aggregate workflow passed. Merge remains blocked on the repository code-owner approval rule. Earlier checks passed at `6aa0962d4fb715f2190bb7bb22efacab2e58495d`: 55 successes, two intentional skips and one neutral result. [The earlier CI run](https://github.com/paperclipai/paperclip/actions/runs/36901592350) includes all test shards, browser tests, typecheck, build and canary dry run. Greptile was 5/5 on that commit with no unresolved review threads. GitHub still requires code-owner review under the repository merge rules; passing checks do not bypass that approval. Paid source fingerprints remain separate below and in the dated result note. - Paid Events qualification passes **3/3**: GPT-5.4 Mini, Claude Haiku 4.5 and Claude Sonnet 4.6. Each uses a real public HTTPS callback, signature verification and report retrieval in a fresh conversation. A final Mini regression passes after the quota/status fixes. All evidence validates. Bounded tunnel startup retries occur before provider calls and remain visible; failed earlier attempts retain their original grades. - The earlier complete seven-case matrix passes **21/21**, with a separate **3/3** delegation regression. Two preceding matrices also passed 21/21 each. A complete 24-cell matrix including Events has not been run. [The dated results](doc/plans/2026-10-01-public-mcp-paid-eval-results.md) retain exact source fingerprints, failures, model IDs and partial costs. - Node 24: repository-wide `pnpm -r typecheck` and `pnpm build` pass after merging master. Server typecheck passes after the final quota/status changes. Eval typecheck and all 892 eval-support tests pass. - All 33 real MCP/OAuth tests pass. The preceding combined MCP, redaction, private-address and DNS-rebinding run passed 129 tests; two later MCP regressions cover quota reuse and unchanged-status suppression. All 28 adjacent issue-tree/stale-lock route tests pass. CI then found a null checkout result in the existing concurrent-workspace path; logging now uses optional status access. All 12 closed-workspace tests and all 33 MCP tests pass after that correction. The exact-start event calibration exposed a timestamp gap; scanning now includes the subscription start, with all 33 MCP tests and server typecheck passing. These two narrow corrections follow the paid regression. - A real Core → Cloud → Core authority round trip passes OAuth, MCP 2.0 subscription/delivery, current membership loss, unsubscribe, legacy SDK tools, refresh and revocation. Its callback transport is a fixture with independent HMAC verification. The paid Events campaigns separately prove public HTTPS delivery. - Earlier component qualification passed UI 7,117 tests, CLI 502, shared 832, skills catalog 20, database 160 and OpenAPI 10. Token gates, module boundaries, migration order and plugin regeneration passed. CI covers general/serialized suites, eight browser shards, runner checks, typecheck, build and canary dry run. - **Local full-suite limitation:** the earlier monolithic run was not clean. It encountered overlapping schema rebuilding, Mac database shared-memory limits and isolated CLI/fixture failures. Targeted reruns passed. The existing >32 MiB Git filename stress test still hit its 300-second Mac timeout. The additional serialized sweep stopped after 62 passing suites once CI passed. Original failures and partial logs remain; this PR does not claim a wholly green local monolithic run. - Local Codex CLI and Claude Code OAuth login and MCP SDK interoperability were verified. Public-store installation, actual ChatGPT Work Cloud Events UI, staging HTTPS client behavior and hosted newcomer provisioning remain release gates. Enablement is moving to **Settings → Experimental → Assistant connections (MCP)** in the stacked follow-up [#14933](https://github.com/paperclipai/paperclip/pull/14933). Merge both for the intended setup experience. This foundation branch alone still uses `PAPERCLIP_PUBLIC_MCP_ENABLED=true`. After deployment, set `PAPERCLIP_PUBLIC_URL` to the authenticated instance's HTTPS origin, and connect to `/mcp/paperclip`. Select a team and allow writes in browser consent. Configure an available agent and budget, then delegate and retrieve results later. For Events, rescan the deployed plugin catalog in ChatGPT Work Cloud; the host supplies its webhook credentials when the user asks to watch a task. See [the setup runbook](doc/public-mcp.md). ## Risks - Events are at-least-once and may arrive out of order. No replay cursor is advertised. Clients must refresh finite subscriptions, read current state and avoid comment feedback loops. Callback material uses the instance secrets master key; hosted subscriptions require the updated Cloud broker and are bounded to five minutes/the access proof expiry. - ChatGPT Work Cloud/dot event UI, plugin rescan and a hosted staging subscription remain deployment gates. Local signed-webhook and paid model evidence does not claim those surfaces have been exercised. - Disabled by default. Merging adds schema and opt-in code; it does not deploy a public endpoint, publish a store listing, create a team, or start paid agents. - Migrations 0301 and 0302 are additive and idempotent. Their SQL is unchanged from the earlier preview numbers, so hash-aware upgrade reconciliation preserves prior staging applications. Normal instance upgrades must apply it before enabling MCP. - Task creation and comments can schedule paid agent work. Consent and tool descriptions disclose that effect. Revocation blocks future calls but does not undo delegated work. - Public deployments need edge rate limits and credential-safe logging. Internal dispatch is restricted to the closed catalog and carries a request-local verified actor. - Hosted onboarding requires the companion Cloud broker, encryption-key configuration, and tenant rollout. Self-hosted direct connections can use this PR alone. - Store acceptance and agent-mode participation are not claimed. Checked-in plugin endpoints are development defaults; rebuild packages for a real deployment before installation. ## Model Used OpenAI GPT-6 in Codex, with reasoning, tool use, and code execution. A more specific serving version and context-window size were not exposed by the session. Paid eval models: `gpt-5.4-mini-2026-03-17`, `claude-haiku-4-5-20251001`, and `claude-sonnet-4-6`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted/component checks; full local-run limitations are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
16b7db35ff |
Shorten planning skills and measure task decomposition (#15296)
## Thinking Path > - Paperclip manages work for AI agents. > - Planning guidance helps agents choose owners and dependencies. > - The runtime skill favors few tasks, but the catalog skill requires a child-task breakdown. > - Both add repeated process instructions that can distract from the requested outcome. > - This change keeps the ownership and dependency rules and removes the required matrix and repeated checklist. > - A bounded Product E2E comparison measures saved outcomes and task handoffs before qualification. ## Linked Issues or Issue Description Refs #11057. Related measurement work: #15218. **What existing behavior does this improve?** Planning and delegation through the runtime plan-to-tasks and bundled task-planning skills. **Current behavior** The two skills contain about 1,900 words and conflicting guidance on whether plans require child tasks. **Proposed behavior** Keep cohesive work with one owner. Split only for a real owner, parallel output, dependency, independent review, or follow-up lifecycle. Preserve existing authorization and planning mechanics. ## What Changed - Shorten both skills to about 400 words combined. Preserve their keys and installed-version behavior. - Remove the duplicate operational-skill pointer and regenerate affected source metadata. - Add twelve explicit Product E2E cells: four scenarios with current, short and disabled planning skills. - Use the current task composer and actual create-response ID; calibrate public skill APIs and browser creation without providers. - Eliminate an observed collision in chat-test company prefixes with a per-suite sequence. - Grade saved documents, exact author/run attribution, child count, prerequisite execution order, review boundaries and completion handoffs. - Retain current skill bytes and report source, selections, run accounting and failures. ## Verification - `pnpm test:e2e:runner:typecheck`: pass. - `pnpm test:e2e:runner:unit`: 1,287 Vitest tests and 128 Node checks pass. - `pnpm test:e2e:runner -- --list --suite plan-task-guidance`: twelve local Codex cells. - Archived current skills match master `72ff3a9f27e581a27acb49771e8658bbb0bbaa47` exactly. - Corrected fixture: three real public-API/database calibrations pass with zero provider runs; all 35 evaluator checks and Product E2E typecheck pass. - Setup campaign [37399550253](https://github.com/paperclipai/paperclip/actions/runs/37399550253) was canceled after source review found unsupported bundled edits and automatic core reinstallation. Its paid-cell step was skipped: zero provider runs, no behavioral grade. - The next setup [37401094799](https://github.com/paperclipai/paperclip/actions/runs/37401094799) failed before task creation on the old title-field selector: zero actual runs, original FAIL retained, cleanup passed. A real browser/API calibration of the new helper passes with paused non-provider agents and zero runs. - Full local typecheck/build pass. Full local tests retain one unchanged five-minute Git streaming timeout (also fails isolated), 9,591 passes and 5,796 skips. CI's chat failure was a proven random fixture-prefix collision; five affected cases pass after the test-only repair. - Paid behavior comparison and new-head CI/review remain pending. This PR remains a draft. ## Risks - The shorter text may change delegation decisions. Live outcomes are not yet qualified. - The initial comparison uses one profile and one attempt per cell. It cannot establish cross-model reliability or cost trends. - Disabled means unassigned company-owned copies; the company library remains discoverable. This does not qualify global removal, automatic accepted-plan wiring changes, or installed-copy migration. - Skill availability does not prove a model read or cognitively used it. - No provider/tool protocol, permission, timeout or runtime lifecycle behavior changes in production. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code editing and tool use. The exact backend model ID and context-window size are not exposed in this session. The declared eval model is native Codex `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
72ff3a9f27 |
Measure native tool context and expand bounded workflow evals (#15218)
## Thinking Path > - Paperclip manages persistent agents and their assigned work. > - Agents receive both fixed instructions and tool definitions. > - Moving a procedure into a tool description still adds model context. > - We need to measure the complete delivered catalog and test real outcomes. > - The tested reductions saved little space and introduced failing outcomes. > - This PR keeps measurement, bounded eval coverage and the original evidence. > - Production prompts, tools and runtime behavior stay unchanged. ## Linked Issues or Issue Description Refs: #15151, #14961, #14948, #14985. This adds the measurement and eval coverage needed to assess further native instruction changes. The attempted hiring and dependency reduction failed qualification and is excluded from the final diff. ## What Changed - Measure the actual standard-mode tool authority, including all 39 tools and their input schemas. Capture scripted native start, resume and continuation payloads and the OpenCode MCP declaration list. - Add OpenCode to the two explicit-only local hiring/reuse and delegation/feedback stories. Preserve the original task requests and independent oracles. - Apply one attempt per selected story and explicit company and lead-agent budget stops. - Select managed hiring credentials from the requested profile, including OpenRouter. - Run the existing Node test files under Node instead of collecting them as Vitest suites. - Retain sanitized comparison reports, original failed grades, source hashes and evidence gaps. ## Verification Final source: `2e7cef78eef7cdfe02265e0dcb03e855b8e50bd8`. Local verification passes: - Full repository `pnpm -r typecheck` and `pnpm build`. - Eval-support unit tests: 1,252 Vitest checks and 128 Node checks. - Eval TypeScript check and six final-source measurement tests. All 36 normalized components across nine scripted deliveries and the OpenCode MCP catalog match the baseline exactly; the complete standing projection is 49,200 bytes. Fresh review of this exact head is [5/5 with no remaining findings](https://github.com/paperclipai/paperclip/pull/15218#issuecomment-5996723170); all three review threads are resolved. [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/37354539126) passes all 47 jobs, including repository typecheck, build, tests, runner checks, browser shards and the canary check. One initial annotation-test timeout is retained in attempt 1; its seven-test suite passed locally, and the affected CI lane passed on one targeted retry. No source or paid eval rerun was needed. Both ready-transition security scans passed with zero annotations. The PR is out of draft and conflict-free. GitHub still requires code-owner approval for the `package.json` test-script change; its requested reviewers are already set. A byte-for-byte comparison against master context `a65ca0950834a85bb93bcc4b4042ecacdebfef53` confirms no production changes under `packages/` or production `server/` paths. The only server addition is a measurement test. No new paid rerun is needed to compare unchanged production bytes. This does not claim that existing product defects have been fixed. The rejected corrected experiment had baseline **5 PASS / 1 FAIL** and candidate **3 PASS / 3 FAIL**, including **two newly failing pairs**. The later readiness experiment had candidate **3 PASS / 3 FAIL** and baseline **3 PASS / 2 FAIL / one setup cell without a behavioral grade**. Its five comparable pairs had two new failures, two new passes and one unchanged pass. The missing baseline Codex cell never reached its provider step because Docker setup timed out. The observed missing behaviors include parent continuation, waiting for the latest child revision, revised ZIP delivery and OpenCode credential persistence. Those failures remain failures. Source review and passing CI do not regrade them. The original reduction saved 460 bytes; its first repair saved only 125 bytes, and the larger unqualified runtime repair increased the full projection. None of those production changes is shipped here. Read the [report](https://github.com/paperclipai/paperclip/blob/2e7cef78eef7cdfe02265e0dcb03e855b8e50bd8/doc/plans/2026-10-05-native-procedure-guidance.md) and its linked sanitized receipts for exact sources, original campaign links, pair-level results and evidence limitations. ## Risks - Full-catalog bytes are not model tokens, invoices, private vendor prompts, lazy-loading behavior or truncation proof. - The two stories are explicit-only and do not prove general coding quality or arbitrary resume behavior. Single trials do not establish causation or performance trends. - The retained OpenCode candidate credential guard failed. Cleanup removed the original provider database, so the precise persistence mechanism remains unknown. The guard is unchanged. - ACPX provider-execution IDs lack proven host-call mapping. No extra-work or feedback-consumption claim is inferred by matching names, order or counts. - PostgreSQL cannot start locally while the host's shared-memory slots are exhausted. Hosted CI must supply the full database checks; the full local database suite is not claimed green. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and code execution. The exact deployment model ID and context-window limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a386a59998 |
Reduce repeated native completion guidance and preserve final replies (#15151)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents receive task constraints and completion tools from Paperclip. > - Completion tools already define the procedure for reporting a result. > - Repeated procedure text adds instructions to each full task turn. > - The final reply must still explain a blocker and link a saved document. > - This pull request removes repeated procedure text and keeps these visible outcome requirements explicit. > - A document receipt supplies the exact link, and stricter evals check the persisted reply and browser navigation. ## Linked Issues or Issue Description Refs: #14961. Related: #14948 and #15007. **What happened?** Native task envelopes repeat completion procedure text. A reduced envelope needs explicit final-reply requirements. The `write_document` receipt also lacks a canonical document link. **Expected behavior** Keep the completion tools as the source of procedure details. Require one accepted completion result before the final reply. A blocked reply must explain the reason, owner and unblock action. A document reply must contain a working link to the saved document. **Steps to reproduce** 1. Run the native assigned-skill document case and native blocker case. 2. Inspect the run-attributed provider final and its persisted comment. 3. Check the blocker explanation or open the final reply's document link. ## What Changed - Remove repeated completion procedure text from the native task constraints and backend instructions. - Keep explicit blocker and document-link requirements in full task turns. - Return a company/task-scoped `documentHref` from `write_document`. Preserve the link in the idempotent mutation receipt. - Repeat canonical links for this run's current saved revisions in accepted completion feedback. Give blocked providers final-response guidance for the cause, owner and unblock action. - Keep internal document/comment anchors when Markdown issue links load cached issue details. - Add a manual six-cell comparison suite with strict source, build, default-instruction and budget admission. - Capture eighteen shared runnerd RPC projections and six direct OpenCode HTTP projections across start, resume and continuation phases, using scripted local transports and no provider execution. - Apply v3 checks only to the manual instruction comparison; preserve v2 checks for the existing native completion suite. Check the actual persisted blocker reason and exact saved-document link. Click the rendered document link and check the original content marker in the classic document card or the new document tab. - Forward exact OpenCode finishing calls through the controller. Wait for acceptance, keep accepted feedback and concrete rejection text, and reject malformed responses. Preserve ordinary dynamic-tool response handling. - Settle the completion decision and tool response before mapping a racing idle/error/abort event or handling explicit close/interruption. Reject a concurrent finishing call before controller admission. - Add a provider-free regression through real runnerd, the OpenCode proxy and a fake provider. Reject the first completion, accept the corrected report in the same turn, and propose one result. - Keep all original verdicts unchanged. Treat replay under new checks as separate diagnostics. ## Verification - `pnpm -r typecheck` and `pnpm build` pass locally. - Native document-authority tests pass, including company/run authorization and idempotent replay. - Native runtime-context, backend and measurement tests pass. - Final-answer calibration, protocol scoring, source-admission and catalog tests pass. Wrong reasons, absent links and wrong link targets fail. - `pnpm test:e2e:runner:typecheck` passes. Discovery lists exactly six single-attempt local cells with the declared models. - Exported `prepareNativeInstructionPreflight` then `verifyNativeInstructionPreflight` pass on this clean committed source. They build locally and make zero provider calls. - Corrective live confirmation is incomplete. Source |
