mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 21:07:40 +01:00
f03522ea8fba2dfd3b08229bfc33a753e6cbd82c
386
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
27270e9bf8 |
fix(server): wake interrupted retry cancellation from committed intent (#17667)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A parked scheduled retry has no executor to finish an interrupted Stop request. > - Its saved cancellation fence already records the work that remains. > - The orphan reaper currently scans for that work on every heartbeat, including idle periods. > - This PR registers the existing records with the shared delivery scheduler and wakes recovery after committed writes. > - Recovery keeps the existing cancellation guards and settlement behavior, then becomes quiet. ## Linked Issues or Issue Description Refs #17666, the merged preceding PR in the recurring-server-work migration stack. This PR targets master. Related cancellation work found in the dedup search: #12301 and #7340. This PR changes the trigger for interrupted parked-retry cancellation; it does not replace their cancellation or retry policies. **What happened?** The periodic orphan reaper queries scheduled retries with cancellation fences even when no cancellation needs recovery. Its first-20-row scan can also stop behind malformed evidence. **Expected behavior** Committed cancellation intent wakes bounded recovery. Failed work retains a retry. Startup reconstructs durable work. Empty or settled work performs no recurring SQL for this duty. **Steps to reproduce** 1. Start a server with heartbeat scheduling enabled and no interrupted retry cancellations. 2. Observe the orphan reaper repeatedly query for cancellation fences. 3. Cancel a scheduled retry and fail the final status write after its fence commits. Recovery must complete the existing Stop request without running a provider or dropping its metadata. **Paperclip version or commit** Current parent squash `7afee8f0622e273d10506b0febfb68f143d168b1`. Source acceptance `66107ae0a7ad85fd0476f91c4f8f98ae74a7a048`. **Deployment mode** Self-hosted server with the existing single-process transaction signal model. Local evidence uses PostgreSQL 17. ## What Changed - Add one domain-owned recovery worker over existing heartbeat run rows. No new table. It delegates to run control and preserves original cancellation, budget, and operator metadata. - Signal cancellation fences, lifecycle transitions, dispatch deferral/promotion/suppression, relevant settings changes, and deletion within their owning transactions. Wrap only the ordinary database fence in a short transaction; provider calls stay outside it. - Keep startup/manual recovery compatible and remove this duty from the periodic orphan sweep. Other reaper duties remain for separate PRs. - Extend the shared scoped worker to retain pending work with a memory-only admission retry while its owner is paused. Without pending work, it schedules nothing. Existing GitHub consumers retain their behavior. - Add a concurrent idempotent partial index and lifecycle/producer tests. Enable guarded disposable PostgreSQL URLs in affected test harnesses. - Reconstruct one hundred candidates per scope per scheduler pass. Resume from the saved scope cursor and retain fair twenty-item execution batches. Manual recovery starts cancellation before reading the full backlog. - Require the recovered browser reply to be saved before testing reload. Keep all original assertions and time limits. - Record the full producer audit, lifecycle limits, and failed attempts in `doc/plans/2026-10-25-scheduled-retry-cancellation-sleep-audit.md`. ## Verification Published head: `405e533394b26215e22d0452e9b0207d3f784a79`. Source acceptance: `a35fa43b98e2bd378d4a920d2775ce243654838b`. The final commit changes the audit only. - All 363 backend cases in ten complete owner and consumer files pass without skips. This includes actual Stop/retry producers, transaction settlement, interrupted-write recovery, pause/resume, shutdown, fair backlog processing and quiet after settlement. Seven separate fresh PostgreSQL 17 databases prevent fixture interference. - The two embedded-only backend fixtures temporarily use separate external databases on this IPC-exhausted host. Their fixture imports and native client teardown are restored byte for byte afterward. Assertions and timing limits are unchanged. CI must still pass with the original embedded fixtures. - Two paging regressions fail on the prior worker and pass on the fix. They cover cancellation before full reconstruction, a malformed first page, fair batches, a newly committed intent behind the cursor, and no later idle SQL. Per-candidate admission reads remain deliberate safety checks. - The repeated CI reload failure is reproduced with a temporary twelve-second fixture settlement delay. The trace retains the same five-second missing-reply assertion, even though a cleanup race masked it in the top-level error. Requiring exactly one saved reply from the resumed run passes under the identical delay. The fault is removed. All six browser journeys pass on the final source in 40.5 seconds, with zero retries and the original 120-second case and five-second reload limits. - Browser acceptance uses the canonical CLI/server/static-UI path and a fresh external database. Only fixture database-URI setup is temporarily adapted and then restored. This is not an unmodified embedded CI-environment reproduction. The six cases include the controlled case; repeated runs are not added to the distinct count. - Actual database build, server typecheck/build, module boundaries and exact-current migration order pass. The database source tree is unchanged from the verified 0405-to0406 upgrade: all 404 prior receipts and 300 companies stay intact, exactly one receipt is added, replay adds none, and the index is valid on upgraded and fresh databases. The five snapshots remain 0402 through 0406. - All earlier failures remain recorded in the audit. Prior head `48b3fcb83f305616df02a4337a42ec0867bb4b80` had exact-head Greptile 5/5 but failed the repeated browser assertion. It was not retried blindly. Its completed Ponytail review's backlog concern and old migration paragraph are addressed here. Old scores do not satisfy new-head gates. - Current-head CI passes. Exact-head Greptile and Ponytail both give 5/5, with no remaining actionable findings. These checks do not prove deployed instance or database suspension. ## Risks - Only supported in-process writers produce local transaction signals. Independent SQL writers or other replicas need a separate ingress contract. Startup reconstructs saved obligations. - A real failure retains bounded retries. Unknown commit outcomes retain the shared intent hold. Neither is discarded to report quiet. - Per-owner drain uses a memory-only admission recheck. The shared helper change affects existing consumers, which have focused regression coverage. - The partial index is concurrent and idempotent; it adds index storage. Each scoped load reads at most one hundred rows. The shared helper retains discovered pending IDs between passes, so a large unresolved backlog still uses memory. - Cloud alarms, idle-safety exemptions, and other reaper duties remain separate. Merge requires passing current-head CI and review gates. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, and local PostgreSQL tests. The runtime does not expose a more precise model identifier or context-window size for this turn. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7afee8f062 |
fix(server): schedule direct GitHub webhook recovery from saved deadlines (#17666)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/218/changes/paperclip/218 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub callbacks can fail before the app records an inbox row. > - Direct GitHub integrations need bounded recovery from provider delivery history. > - The chat timer checks for due recovery every second, even without an eligible integration. > - Existing recovery checkpoints already store the next scan time and runtime epoch. > - This pull request schedules those saved obligations and wakes on committed configuration changes. > - It removes the final chat maintenance interval while preserving authenticated recovery. ## Linked Issues or Issue Description **What happened?** The app queries due GitHub recovery endpoints every second. This still runs when no direct GitHub integration needs recovery. Other chat maintenance duties already have independent workers in the preceding stack. **What did you expect?** Reconstruct existing recovery deadlines at startup and after committed changes. Run only when an eligible endpoint reaches its saved deadline. Make no recurring SQL or HTTP calls for unconfigured, disabled, or Cloud-relayed integrations. **Steps to reproduce** Start an instance without an active direct GitHub integration. Leave it idle. The chat maintenance interval continues to call processFailedGitHubWebhookDeliveries each second. Follows merged #17664. Related #15981 owns the original GitHub webhook ingress/recovery separation. #17662 preserves review-check recovery, which is a separate obligation. No duplicate scheduling migration was found in the related-PR search. ## What Changed - Put deadline reconstruction, scopes, and registration in the GitHub provider owner. Read existing recovery checkpoints in pages of 100. Reload current eligibility before each dispatch. - Dispatch batches of five through the existing shared helper. Preserve company fairness and the default limit of 25 for other consumers. - Notify inside configuration, credential, checkpoint, installation-lifecycle, and Cloud-registration transactions. A signed installation deletion reproduced a missing wake before the fix. - Detect credential-only epoch changes during a future backoff. Preserve the existing floor reset without adopting old events into new credentials. - Gate every new recovery HTTP call on dispatch admission and shutdown. Join in-flight work and retain its existing outcome/uncertainty settlement. Other queues remain independent. - Remove the final chat reconciliation interval and its unused coordinator. Update the old coordinator fixtures; preserve the voice retention worker acceptance. - Restore known-unsent redelivery claims after drain/shutdown, guarded by exact attempt identity. Preserve uncertain outcomes after transport starts and never overwrite a newer receipt. - Keep all existing redelivery, source, authority, lease, receipt, and callback checks. Add no table, migration, or external wake contract. ## Verification Current source: `ab00fc5b30d51d6c85891536599e7bdfd0c822fe`. - **751 distinct cases pass** at `3d7f163bc5259b60f9bd19b7fc90b402f0a6e925`: 274 complete GitHub review cases, 294 other GitHub-named cases, 30 publication cases, 145 voice/retention cases, and eight startup cases. - All 19 new recovery worker cases and all six stop-before-transport variants pass. Coverage includes configured eligibility, callback-only admission, committed configuration changes, outer transaction settlement, restart/drain/standby, saved deadlines, fair batches, independent queues, held HTTP, joined shutdown, known-unsent restoration, and newer-receipt fencing. - The first workflow filter omitted the separate recovery group. The saved case list exposed the gap. The second selection ran the remaining GitHub cases in a new database. Passed-name sets are disjoint. The two commands filtered 1,504 and 1,484 unrelated cases respectively. No full chat or repository-suite claim is made. - Database build, server typecheck/build, module boundaries, and migration order pass. Five fresh PostgreSQL fixtures receive the unchanged parent migration history through 0403. This slice adds no database migration or table. - The parent browser-fixture repair is carried with both complete patches preserved. At predecessor `836acef0370d42323fdfdbd6d62a805b2469f1d4`, entire server, package and UI source trees and owned hashes match the 751-case acceptance. The current follow-up adds three test cases and audit evidence; production source remains unchanged. The parent has merged. - [The owning audit](https://github.com/paperclipai/paperclip/blob/ab00fc5b30d51d6c85891536599e7bdfd0c822fe/doc/plans/2026-10-25-github-webhook-recovery-sleep-audit.md) preserves earlier failures and exact acceptance scope. Fresh exact-head Greptile and all CI gates remain required. ## Risks - Direct GitHub endpoints still require the existing one-minute external history cadence, with five-minute failure/claim backoff and provider backoff up to one day. Cloud relay endpoints are excluded. This does not promise network-idle direct integrations. - The existing history, envelope, attempt, and lease limits remain. Provider history never substitutes for a genuine authenticated callback. - Supported producers share the current database owner's process-local signals. Independent SQL writers and other server processes need their own integration. Unknown transaction outcomes remain held by the existing reconciliation protocol. - Already-started HTTP keeps its existing timeout and is joined during shutdown. No later request starts after drain or stop; durable attempt uncertainty is preserved. - Host-sleep eligibility and Cloud alarm coverage stay unchanged. Local worker tests do not prove host/database suspension or fleet savings. - Each remaining sleep slice needs its own current review and CI checks before merge. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context-window size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Parent startup-test correction: #17664 fixes its legacy coordinator boundary marker. This slice removes that coordinator and timer entirely, so its existing absence assertions supersede that obsolete inspection. All other parent and upstream changes are preserved. At predecessor `836acef0370d42323fdfdbd6d62a805b2469f1d4`, complete server, package and UI trees match the 751-case acceptance; all eight startup cases also pass after the explicit conflict resolution. Fresh review and CI are required for this new head. ## Required review finding: orphan assignment investigation The reported orphan endpoint cannot be committed through the existing schema: assigned_agent_id is NOT NULL, its agent FK is ON DELETE RESTRICT, and a validated nondeferrable composite FK enforces the same company. A first reproduction failed at fixture mutation with PostgreSQL 23502 before reaching the worker; that failure remains recorded. Three new real-PostgreSQL cases verify NULL/missing/deleted assignments are rejected, then prove the preserved valid endpoint recovers once, saves its next deadline, and makes no SQL reads during a 15-second pre-deadline interval. All 22 recovery-scheduling cases pass (19 previous plus three new); unselected cases are not counted. The preceding style-only commit preserved production behavior and non-comment tokens, and the prior 751-case evidence retains its original scope (754 distinct cases across both selections). A new review must assess this evidence before merge; the old conditional verdict is not waived. Actual parent squash ancestry is carried with whole-tree identity before this test-only follow-up. Repository style follow-up removes seven new code-comment lines from the recovery ingress owner and constraint tests. Their explanations remain in the audit. Both files retain identical non-comment TypeScript token streams, preserving all guards, assertions and deadlines. Production eligibility SQL is unchanged. Fresh current-head CI and reviews remain required. ## CI test corrections The original strict deletion assertion reproduced CI's PostgreSQL-18 failure locally: the forbidden delete reports 23001 (restrict violation), versus 23503 on PostgreSQL 17. The corrected assertion accepts only these integrity codes for deletion, retains exact codes for NULL/missing assignments, and verifies the original agent and assignment remain. All 22 recovery cases pass on embedded PostgreSQL 18; the same three integrity probes pass on PostgreSQL 17. A delayed real announcement response reproduced the attachment E2E's filechooser timeout: the dismiss handler tried to click outside an already-open modal menu. It now closes the menu, dismisses through the UI, and restores that menu so the original action resumes. Controlled late-announcement acceptance passes; the temporary fault is removed. All 11 complete attachment receipt journeys pass, retries zero and original assertions/budgets unchanged, using a fresh supported external PostgreSQL-17 database with canonical isolated onboarding/server/UI/storage. An earlier embedded-cluster startup failure occurred before tests and remains recorded. Production server typecheck passes; production source and schema are unchanged. Fresh current-head CI and review are required. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
scotttongandPaperclip
|
71f0c0c99e |
feat: add opt-in Coordinator workspace for chat, tasks, and decisions (#17983)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/147/changes/paperclip/147 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People coordinate work through conversations, tasks, and requests for input. > - These surfaces must share task state and preserve the context of each decision. > - The existing workspace makes people switch surfaces to follow that context. > - This pull request adds an optional Coordinator workspace with linked conversations, tasks, documents, and decisions. > - Each person can enable it in Settings > Experimental and return to the classic workspace. > - The benefit is a coherent path from a request to an assigned task and reviewed work. ## Linked Issues or Issue Description **Subsystem affected** UI, server, shared contracts, and database. **Problem or motivation** People need a consistent way to start tasks from agent conversations, answer questions, review work, and return to the source context. Decisions and task state must agree across these surfaces. **Proposed solution** Add a personal, default-off workspace experiment. Keep one conversation per agent. Add task proposals, compact task and decision cards, contextual document panels, and a Needs You queue with search, filters, sorting, grouping, and history. **Alternatives considered** Replacing the classic workspace for everyone would remove the opt-out path. Keeping separate decision state in each surface would create conflicting answers. This change uses the existing canonical task and interaction records instead. **Roadmap alignment** This supports the existing agent chat, task management, and human review capabilities. It is an experimental presentation choice. The user requested this isolated implementation after design review. No Discord discussion is claimed. ## What Changed - Add per-person workspace preferences, a default-off setting, and an additive migration. - Add Coordinator navigation, pinned destinations, task lists, agent switching, and contextual panels. - Link task suggestions and creation to the source conversation. Default new task ownership to the current agent unless the user specifies another owner. - Share pending questions, decision receipts, exact review targets, and history across tasks, chat, and Needs You. - Preserve task access, assignment authorization, revision binding, and required output review gates. - Add compact document links, relative timestamps, composer controls, and reduced-motion support for task and decision transitions. - Add component stories, regression coverage, and capability-map documentation. ## Verification - Final head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, based on master `8bc051245f77a47bee0f8ba45896b8bfa5922e0a`. Verified October 29, 2026 at 00:10 UTC: **MERGEABLE, 56 checks passed, four conditional skips, no pending or failed checks**. [Current CI run](https://github.com/paperclipai/paperclip/actions/runs/40790639688) passed without a rerun on this head. - Fresh Greptile **5/5**, zero new comments and zero open review threads; Superagent passed with no suspicious changes. [Checks](https://github.com/paperclipai/paperclip/pull/17983/checks). - Local workspace typecheck/build passed after the latest rebase. The final account-read patch passed UI typecheck/build and all 12 related preference/settings tests, including a delayed real-client request across account changes. Earlier scoped checks cover review deduplication/replay, decision deep links, cross-surface state, access boundaries, and migration replay. - Local database/browser rechecks after the latest rebase were limited by exhausted macOS shared-memory IDs: `initdb` failed before database creation. The local run passed 31 UI/snapshot tests and skipped 24 database tests; those skips and the blocked local browser start are not claimed as passes. The independent Linux CI database and browser checks passed. No running dev database or host setting was changed. - Upstream migration 0403 is preserved. Preferences and durable review retry aliases are generated as replay-safe migrations `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`; five newest snapshots are retained. Every detected conflict was rebased and pushed before waiting for CI/review, as requested. - The PR is unmerged. The active master ruleset still requires code-owner approval for the changed Paperclip skill; no approval or rule change was performed. <details> <summary>Earlier verification failures, integration history, and evidence</summary> These entries preserve prior checkpoints and failures; the current acceptance result is above. - Focused UI and server checks from the isolated build are recorded in the implementation plan. - Browser checks covered the opt-in shell, navigation, task/document links, and decision card states. No new live provider trial is claimed for this handoff. - Current acceptance head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, rebased onto master `8bc051245` immediately after its GitHub review-repair migration took 0403. Preferences are generated as idempotent `0404_tense_silver_sable.sql`, preserving upstream SQL/journal/snapshot history and five newest snapshots. GitHub reports MERGEABLE after the immediate lease-protected push. Scott's updated instruction is to rebase and push every conflict resolution first, without waiting for CI or Greptile; verification follows the push. - The final reviewed feature tree was consolidated for repeatable single-commit rebases. The complete previous commit history remains reachable through local recovery ref `codex/coordinator-recovery-before-rebase-20261028` (resolved snapshot `0e1bbdf4e`). The actual rebase completed, and its tree hash exactly matches that resolved snapshot. The force-with-lease required the known remote head `0e6ef7c44`, protecting against concurrent branch changes. No PR merge or permission/runner setting change occurred. Fresh CI/review are in progress for the current head. - `2001132fa` received Greptile 5/5 with one nonblocking account-switch read finding. Current head detaches path-coalesced preference GETs and disables HTTP caching, retaining React Query's account-scoped sharing. A delayed real-client GET regression proves account B can load its saved choice before account A responds, and A's late response only updates A's cache. Preference/settings tests 12/12 and UI typecheck passed; its review thread is resolved. Fresh current-head CI/reviews are pending. The prior CI chat shard 3 ended with “The operation was canceled” and no assertion failure ([job 123171595631](https://github.com/paperclipai/paperclip/actions/runs/40789688333/job/123171595631), `/tmp/pr17983-2001132-chat3.log`); no code/timeout change is attributed to that cancellation. - `843e823b4` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass. All browser tests including the repaired YAML shard passed, and 54 checks passed. General server shard 9 ended with “The operation was canceled” and no failed assertion; its test step was marked cancelled (job [123164350893](https://github.com/paperclipai/paperclip/actions/runs/40787501761/job/123164350893), log `/tmp/pr17983-843e-server9.log`). A single unchanged-head rerun was started after the workflow completed, but master moved while it ran. A local nested-clone check timed out in its 30-second database-startup hook and then failed cleanup before the 20 database cases ran; this is retained as a local verification limit (`/tmp/pr17983-843e-nested-clone-check.log`), not claimed as a pass or proof of the CI cancellation cause. - Following Scott's instruction, the new conflict was rebased and pushed immediately without waiting for that rerun. Upstream's `0403_natural_cerise.sql` is preserved; regenerated preferences and retry-alias migrations are `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`, replay-safe with five newest snapshots retained. The YAML helper conflict combines upstream's click and header behavior with the editor-selection guard and injected regression. Recovery ref `codex/coordinator-recovery-843e823b4` preserves pre-rebase work. Current head is MERGEABLE. Full workspace typecheck/build passed; UI and snapshot checks passed 31 tests, but 24 database tests were skipped by the local support probe, and the local browser server could not initialize PostgreSQL. A direct disposable `initdb` diagnostic established the host limitation: `could not create shared memory segment: No space left on device` (`shmget`, shared-memory IDs exhausted, not disk space). No dev database or host setting was changed. Diagnostics: `/tmp/pr17983-2001132-initdb-probe.log`, `/tmp/pr17983-2001132-db-probe.log`; checks: `/tmp/pr17983-2001132-tests.log`, `/tmp/pr17983-2001132-browser.log`, `/tmp/pr17983-2001132-types.log`, `/tmp/pr17983-2001132-build.log`. These local skipped/blocked checks are not claimed as passes; fresh CI on independent runners is in progress. - `d18f683cd` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass, but [E2E shard 7](https://github.com/paperclipai/paperclip/actions/runs/40785966243/job/123159719925) failed parsing an Advanced YAML clipboard read beginning with “Skip to Main Content.” The log is retained at `/tmp/pr17983-d18f-e2e7.log`. Current head hardens the test helper to verify editor focus and selection, clear stale clipboard content, and require the YAML header at the start of a fresh copy before parsing. Saved-state assertions and timeouts are unchanged. All six Advanced YAML browser journeys passed on a disposable local instance (41.0s); the exact lost-response journey also passed with a one-time injected surrounding-page copy, verifying rejection/reacquisition (17.6s). The precise browser focus/clipboard timing in CI remains unproven; the observed invalid-read path is now guarded and deterministically exercised. Logs: `/tmp/pr17983-yaml-clipboard-scope-browser.log`, `/tmp/pr17983-yaml-clipboard-injected-browser.log`. Fresh current-head CI and reviews are pending. - `19e6dcb0b` passed Superagent and Greptile 5/5, with one nonblocking deep-link finding. Current head preserves attention links using `?decisionId=`: the requested pending decision is rendered even beyond the first 40 cards, focused and scrolled into view, then only that parameter is consumed so later refreshes do not refocus. Other URL parameters remain. Needs You tests 10/10 and UI typecheck passed; its review thread is resolved. The preceding post-rebase regression run passed 45/45 tests. Fresh current-head CI/reviews are pending. - `6d772b7cc` reached Greptile 5/5 with zero comments/open threads and Superagent pass. While 32 checks had passed, master moved and caused a documentation conflict in `maps/feature-map/onboarding.md`. Rebased immediately, preserving both onboarding paragraphs. The complete four-commit branch was pushed as `19e6dcb0b` before waiting on CI/review, per Scott's instruction. A transient Git index lock interrupted the first continue; the command sequence briefly pushed intermediate `34e298c07`, then the completed rebase was immediately pushed. All feature and security/retry fixes are present; local recovery ref `codex/coordinator-recovery-6d772b7cc` preserves the pre-rebase history. The only tree differences from that ref are upstream's two smoke-test files and the combined documentation paragraph. GitHub reports MERGEABLE. Fresh current-head verification is in progress; prior-head review results are not current acceptance. - `21efa3120` passed Superagent but Greptile scored 4/5 for a valid retry-key persistence bug: an acknowledged duplicate key could reopen a resolved review. Current head adds an additive, replay-safe `0404_light_swarm.sql` migration for indexed, task-scoped aliases from acknowledged retry keys to the canonical interaction. Alias registration and retry lookup occur under the existing task lock; payload/actor mismatch still conflicts, and each canonical review permits at most 100 aliases. Original and acknowledged keys replay the terminal result rather than creating a new gate. Five newest migration snapshots are retained. Service/routes tests 165/165 passed; final quality review plus migration replay/snapshot checks 25/25 passed; server typecheck passed. The P1 thread is resolved in code; fresh current-head reviews and CI are pending. - `69c2c7543` passed 55 checks and Greptile 5/5 with no open Greptile findings, but Superagent check [123140675812](https://github.com/paperclipai/paperclip/pull/17983/checks?check_run_id=123140675812) correctly flagged unbounded pending quality-card creation. Current head fixes that under the existing task-row lock: equivalent requests with fresh idempotency keys reuse the pending card; conflicting requests for an unresolved deliverable return the existing interaction ID; pending cards are capped at ten per task across all reviewers. Distinct outputs remain independent, and already-answered items in a partially pending batch can receive a fresh review. Added real-database concurrency, cross-card duplicate, partial-answer, and hard-cap regressions. Quality review 22/22 tests and final server typecheck passed; earlier service/routes 163/163 tests passed before the final partial-answer refinement. One added assertion initially expected a service HTTP error, but duplicate keys are rejected earlier by the existing shared schema; the assertion now verifies that validation. Fresh current-head CI/reviews are pending; the action-required security result is not treated as passed. - `c9c575e5f` completed all 56 checks successfully with four conditional skips, Greptile 5/5, and Superagent pass. Its two P2 review findings are fixed in the current head: independently bound quality approvals now each remain visible in attention; changing/clearing URL-backed assignee/status filters updates the URL and survives task navigation. Both review threads are resolved. Regression evidence: attention service 34/34 tests passed, CoordinatorTasks 20/20 passed, UI and server typechecks and the full workspace build passed on `69c2c7543`. The first new UI test run caught an assertion racing React Router's transition; waiting for the navigation before asserting passes without weakening the navigation/remount checks. These older-head green checks are not claimed for the new fix commit. - Preceding `0e6ef7c44` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, and passed Superagent. A fifth migration-metadata collision arrived before the final Runner job finished. Those green results are preserved as prior-head evidence, not current acceptance. - Previous fully verified migration head: `0e6ef7c44696a377f35e69521b93587576374bd0`, integrating master `f8226b051` after the fourth migration-number collision. Master added terminal GitHub-response migration 0401 while the final native check ran; upstream 0401 SQL/journal/snapshot is preserved unchanged, and preferences are generated as idempotent `0402_yielding_blink.sql` on top. Exactly the five newest snapshots remain. GitHub reports MERGEABLE after the push. All 1,155 relevant migration/review/interaction/native-execution/remote-assets tests, workspace typecheck/build, migration guards and whitespace checks pass. Fresh exact-head CI/Greptile/Superagent are pending. - Immediately preceding head `36c67cf03` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, plus passed Superagent security. No retry was needed for that head's CI. Master advanced and introduced only a migration-metadata conflict while its final native check ran, so that green result does not establish acceptance of the current integration. Four consecutive collisions (0398, 0399, 0400, 0401) during verification are retained as concrete integration churn; a brief pause of competing migration merges was requested, without assuming it happened or changing any repo setting. The PR has not been merged. - Prior fully verified head: `36c67cf03`, on the clean `30059c0ed` integration of master `efd65c112`. The last Greptile P2 is fixed: Needs attention retains the participant-agent scope through authoritative server membership queries for each pending task, including historical participation when another agent currently owns the task. It does not approximate participation with current assignment or enumerate every company task. The actual view-switch regression and all 31 focused task/routing/collection tests, UI typecheck/build, token gates and whitespace checks pass. Greptile completed a fresh 5/5 review on this exact head (review 19) with zero new comments and no open review threads. CI and Superagent remain in progress; no current-head CI failure has been reported. - Head `30059c0ed` received Greptile 5/5 with that one nonblocking participant finding, now fixed. Superagent passed that head. Current-base integration restored the ephemeral RunsOn fleet after a superseded workflow retained queued aggregate jobs: normal cancellation was insufficient; the single force-cancel request was interrupted but readback confirmed that obsolete run cancelled and current CI began on the trusted fleet. No cancellation retry or runner/security/workflow changes were made. Its partial CI was superseded by the necessary participant fix and is not current acceptance. - Previous compatibility/base-sync head: `30059c0ededc0eab4e04e8299a884f36db8b292f`, integrating master `efd65c112` and the Inbox/workspace compatibility fixes. GitHub reports MERGEABLE after the push. Coordinator keeps Inbox standalone under More without rewriting the instance's Combined Inbox preference; opting out restores that preference. Saved merged-Inbox URLs resolve to their original owner, task status presets remain usable, and canonical repeated `workspace` parameters retain scoped, deduplicated pagination (including Needs attention). All 192 broader routing/Inbox/sidebar/privacy/review tests and the final 16 task-page tests pass, as do workspace typecheck/build, token gates and whitespace checks. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The preceding `d5b9be5bd` remained mergeable but its CI routing gate intentionally fell back to GitHub-hosted runners: “stacked base merge commit does not contain the current base branch.” All 45 downstream jobs stayed queued. Current master was merged cleanly to refresh the pinned base; no runner/security settings or workflow logic were changed. Greptile scored that preceding head 4/5 with the two compatibility findings fixed here. Its automatic Superagent scan was inconclusive at checkout; the documented retry was still pending when superseded. Those results do not establish acceptance of this current head. - Previous approval/migration head: `d5b9be5bd3b8d6b0437e42bf48138377f34fa67d`, integrating master `ba5bba86d` and the stale-output approval correction. GitHub reports MERGEABLE after the push. The fresh P1 finding is fixed: every submitted item now validates and locks its bound output inside the resolution transaction, whether or not it carries a required-quality-review binding. Real database regressions verify that an advanced document revision or deleted attachment refuses the entire batch without recording approvals, and current ordinary output reviews still resolve. All 272 combined migration/review/interaction tests pass; workspace typecheck/build passed. Fresh exact-head Greptile and CI are pending. The automatic Superagent scan ended inconclusive at checkout; one documented on-demand retry is pending. These are explicit verification limits, not passed security results. The P1 thread is resolved. - Master consumed the next migration slot three times during this verification pass: confirmation recovery took 0398, continuation recovery took 0399, and GitHub acknowledgement recovery took 0400. Each upstream SQL/journal/snapshot is preserved unchanged. Preferences are now generated as idempotent `0401_white_lenny_balinger.sql` from upstream 0400; exactly five snapshots remain, and replay coverage locates this migration by table content. This is recorded concurrent integration churn, not a claim that older green results apply to the latest base. - Head `ed3b68da9` fixed the two history/URL findings with 26 focused regression tests, server/UI typechecks, workspace build and token gates passing. Greptile then scored that head 4/5 with the stale-output P1 corrected in the current head. Its partial CI is superseded, not counted as current acceptance. - Previous filter-fix head: `ed3b68da9`, based on the `8458d39d9` merge of master `063cfd438`. The two fresh Greptile findings are fixed: history outcome filters now match recorded per-item verdicts in SQL before pagination (mixed reviews remain in both applicable outcomes), and same-page URL assignee changes/back/forward navigation update that filter without clearing independent filters. All 26 focused database/navigation/collection tests, server/UI typechecks, workspace build, token gates and whitespace checks pass. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The `8458d39d9` migration integration also passed all 328 targeted migration/interaction/confirmation tests, workspace typecheck/build and Superagent security scan. Greptile scored that head 4/5 with the two functional gaps fixed in the current head; those prior-head checks are not current acceptance. - Migration integration head: `8458d39d9721f4cb312c2390070a0609f8cddd1a`, integrating master `063cfd43813cabf18417605fb9eefb9aacf95c8d`. GitHub reports MERGEABLE after the push. Master advanced twice during this conflict-resolution pass: first its confirmation-recovery migration took 0398, then its continuation-recovery migration took 0399. Both upstream migrations and journal entries remain unchanged; Coordinator preferences are now generated as idempotent `0400_sad_guardian.sql` from upstream's latest snapshot. Exactly the five newest snapshots remain. The replay regression now locates the table migration by content instead of hardcoding a renumbered filename. - All 328 targeted question/review/confirmation/migration tests passed again after the second merge. Workspace typecheck/build and fresh exact-head CI/reviews are in progress. The preceding `0c46f46d6` passed workspace typecheck/build and Superagent security scan, but was superseded because master consumed migration 0399 while those checks ran. That prior-head result is not acceptance of the current head. No production settings or runner/security permissions were changed, and the PR has not been merged. - Previous integration head: `0c46f46d6f2b23417ab872ce3cb6ff2673647287`, integrating master `287b04d72f1b9fca1412e05435c75b8302ba0710`. GitHub reports MERGEABLE after the push. The renewed conflicts were `0398_snapshot.json`, `_journal.json`, and the top imports in `issue-thread-interactions.ts`. Upstream's `0398` confirmation-recovery index and journal history are preserved unchanged; Coordinator preferences were regenerated as idempotent `0399_bouncy_miek.sql`, with replay coverage updated and exactly the five newest snapshots retained. Both terminal confirmation notifications and Coordinator task assignment/required review behavior are preserved. - All 328 targeted interaction/review/confirmation/migration tests pass, as do workspace typecheck, map validation and whitespace checks. The workspace build and fresh exact-head CI/Greptile/Superagent checks are pending. The first migration-generation attempt hit the sandbox's local IPC restriction before generating files; the unchanged authorized local generation completed successfully. Prior-head green checks below do not establish acceptance of this merge. - Previous acceptance head: `8b7dcf739ed15afc51351fdfa0e0e5fbc75ab788`, with master `1aa3a21ac` integrated and GitHub reporting MERGEABLE. Greptile completed a fresh 5/5 review on this exact head with zero new actionable findings and no open review threads. The [fresh CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40752864319) completed SUCCESS on attempt 2 at 19:40:08 UTC, including all eight E2E shards. Final completed check set: 55 succeeded, four conditional checks skipped, zero pending/queued/running checks, and one neutral/inconclusive Superagent scan. The auxiliary commitperclip review and isolated native Runner compilation/cache-reuse check both passed. This is not a claim that Superagent produced a passed security verdict. GitHub still reports MERGEABLE against current master `60b8089a44b4eaa37e627adbfdf2f130bda13335`. - Current-head E2E: all eight shards and the aggregate passed, including the previously failing YAML browser journey. Serialized server shard 9/9 failed: the protected YAML projection test exceeded its original 15-second timeout; its late update request then reached the next test's reset mock (the captured payload is the prior test's model/label update). All 88 tests in that exact unchanged file passed locally in 38.03 seconds, with no timeout or assertion changes. The original slow-request cause remains unproven. CI failure log: `/tmp/pr17983-8b7-serialized9.log`; local recovery: `/tmp/pr17983-adapter-yaml-repro.log`. The single bounded unchanged shard rerun passed, and the workflow completed SUCCESS. Rerun log: `/tmp/pr17983-8b7-serialized9-rerun.log`. The initial failure is retained; a passing retry does not establish its original slow-request cause. - Greptile scored preceding head `4873704be` 5/5 with one valid P2: opened task decision records missed company-scoped live invalidation. The current head uses the existing company decision prefix for resolution and effect-completion events, and resets private record payloads on access changes. Its thread is resolved. All 107 focused record/live-update tests, UI typecheck/build, token checks and whitespace checks pass. - CI on `4873704be` failed E2E shard 7/8 in the Advanced YAML lost-creation-response journey: its clipboard helper selected page text starting with “Skip to Main Content” instead of YAML. The exact unchanged journey passed in a disposable local browser instance (1 test, 17.1 seconds). No assertion, timeout or production behavior was weakened; the original failure cause remains unproven. Failure log: `/tmp/pr17983-487-e2e7.log`; local check: `/tmp/pr17983-agent-yaml-repro.log`. The full unchanged Advanced YAML browser suite also passed locally (6 tests, 37.8 seconds). Fresh current-head CI subsequently passed this journey and all other E2E shards. - Both the automatic and one supported on-demand Superagent scan on `4873704be` ended neutral/inconclusive: “Pinned PR scan merge-base checkout failed or timed out.” These do not count as a passed security verdict. The current head's automatic scan and one documented on-demand retry also returned the identical neutral checkout failure, with zero annotations or further diagnostics. This is an external scanner blocker, not a passed security verdict. No security settings or permissions were changed. - Recorded decision links now render and focus the full source-task record in both Classic and Coordinator, including recorded choice, submitted inputs, and the existing effect-result resolver. Fresh terminal records take precedence over older open-decision cache entries. Context and account checks keep mismatched or inaccessible records out of the DOM; no decision-history tab was added to tasks or chat. The shared company live-run feed now drives live-only filtering, including ended runs and terminal tasks with stale run entries. All 213 focused tests, UI typecheck/build, token and whitespace checks pass; the final focus follow-up passed all 188 task-detail tests and UI typecheck. The first new navigation test failed because its test host omitted company context; the host was corrected before the passing run. - Previous integration head: `375c5e89516705050d94bab99336212841b2e6e1` merges master `1aa3a21ac` after four additional upstream commits. The sole conflict was competing imports in `server/src/services/issues.ts`; both quality-review policy and inbound wake-notice notifications are preserved. The PR-specific review import now sits beside execution-policy imports instead of the frequently changing first line. GitHub mergeability is rechecked after the push. - On the combined master tree, workspace typecheck/build, all 153 affected UI tests, map validation, and whitespace checks pass. Four server suites passed 77 tests; the fifth suite's module-import hook timed out while broad verification was running. Its unchanged isolated run passed all 41 tests in 7.2 seconds with the original timeout. This records recovery without claiming the original import-timeout cause is proven. CI on `375c5e895` passed every substantive job, including all eight E2E shards and Superagent; Greptile found the two functional gaps fixed in the current head. A short-lived `669d90876` commit added the main fixes before the focused navigation follow-up. - Immediately preceding head `9d9737b49` completed all 56 checks successfully with four conditional skips, Greptile 5/5 and no open findings, plus a successful Superagent scan. The signoff E2E shard passed on its single accepted rerun; its initial cancellation/timing failure is retained below. Master moved again while those checks ran, requiring this final integration rather than treating the prior head's green result as current acceptance. - Previous head `9d9737b496911a6054d0ea22d1b42b28b115e9ab` includes master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`; that head was subsequently conflicted by moving master. The master merge preserved durable account-bound writes and Coordinator UX. Workspace typecheck/build, 416 focused UI tests, and 142 affected comment-route tests passed. A stable full UI run on preceding head `938ad1665` passed all 9,833 tests across 836 files; its final full workspace build also passed. - Greptile found nested proposal project inheritance and Classic bound-revision links on `91be74952`. Both were fixed with 142 interaction-route tests, 113 focused UI tests, and UI/server typechecks. Its next review identified a valid draft-preservation bug in the new Classic loading UI. Head `9d9737b49` keeps loading/error notices outside editors and annotations while preserving dirty drafts; 115 focused review UI tests pass, including pending/failed revision loads and continued draft editing. All related threads are resolved. Greptile completed a fresh 5/5 review of `9d9737b496911a6054d0ea22d1b42b28b115e9ab` with no open findings. - CI on `938ad1665` exposed an integration regression with the newly merged durable startup saves: the personal preference gate blocked the task page before its first session lookup. The guarded title editor now mounts while that lookup is pending, with presentation-specific tools still gated. All 216 affected component tests pass. The unchanged five-test `disconnect-write-recovery.spec.ts` browser suite passes against a newly launched disposable local-trusted PostgreSQL instance on port 33219, including the exact failed startup-save journey. No live provider, hosted authentication or production proof is claimed. The helper-owned test server exited and the port has no listener. UI typecheck, built-browser assets, token and whitespace checks pass. - Initial durable-write merge tests found duplicate imports and unscoped draft fixtures, fixed before 416 focused tests passed. A sandbox typecheck could not create its IPC socket; the supported rerun passed. The first revision-link test used a wrong Return to latest button assertion, corrected before the 113-test pass. The startup E2E reproduction initially still failed because an outer shell gate also blocked the page; the subsequent fix passed all five cases without changing the test or its timeouts. A focused loading-header assertion initially used an incorrect fixture title; correcting it preceded the 216-test pass. These failures remain recorded. - Earlier routing gates fell back to GitHub-hosted runners while master was moving or conflicted. Superseded queued runs were cancelled, then the `938ad1665` routing gate explicitly selected the ephemeral RunsOn fleet. Its CI actively executed and completed most tests; only the diagnosed startup E2E shard and aggregate failed, alongside the valid Greptile finding. Auxiliary GitHub-hosted workflows also completed successfully on earlier heads, so queueing alone is not an administrative blocker. Fresh [CI on `9d9737b49`](https://github.com/paperclipai/paperclip/actions/runs/40744012678) is the acceptance target; the previously failing startup E2E shard now passes. A different signoff E2E shard failed during executor resubmission when its fixture run was cancelled as `issue_continuation_waiting_on_review`. The exact unchanged journey passed locally (1 test, 46 seconds), and the preceding head with identical server/signoff code passed that CI shard. This supports a bounded rerun but does not establish the timing failure's root cause. After the workflow finished, GitHub accepted one rerun of that failed shard and its dependent aggregate; both passed, completing all checks on that head. The failure log is retained at `/tmp/pr17983-9d9737-e2e7.log` and the local reproduction at `/tmp/pr17983-signoff-repro.log`. No assertion or timeout was weakened. - On the first merge, full workspace typecheck/build, all 9,622 UI tests, 267 tests across seven affected server suites, two migration checks, map validation, token gates, and the PR diff whitespace check passed. Greptile then found duplicate local task-search filtering and lost completed item-review detail. Both are fixed, with 42 passing focused UI tests and passing UI typecheck/build. - A subsequent Greptile review found attachment confirmations without working links and mobile history without a visible host. Confirmation layouts now derive the authenticated immutable-content endpoint from the attachment ID, and Needs You uses the existing full-screen mobile sheet. All 119 focused tests pass, including missing/stale display URLs, both review layouts, and both mobile history entry buttons. UI typecheck/build and token checks pass. Full workspace typecheck/build also pass after the second master merge. - Greptile gave head |
||
|
DottaandPaperclip
|
8bc051245f |
fix(server): wake stale GitHub review repair from committed changes (#17664)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/215/changes/paperclip/215 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub checks and task admission can finish in either order. > - A check can reject an old head before admission creates its review row. > - The chat maintenance loop repeatedly scans for that late review. > - Existing cancellation and delivery receipts already identify the repair. > - This pull request wakes repair from committed changes and leaves settled history quiet. ## Linked Issues or Issue Description **What happened?** GitHub chat maintenance repeatedly scans all review projections. Empty instances and retained history incur database work even when no repair is owed. **What did you expect?** A committed stale-head cancellation or admitted review should wake repair. Only the exact delivery's unassessed queued/running review should become superseded. When no repair remains, this duty should stop making database calls. **Steps to reproduce** Configure a GitHub chat endpoint and leave it idle after all reviews settle. The previous maintenance callback continues to call repairSupersededReviews. A rapid push can also cancel a check before admission inserts its review, which explains why recovery must remain. Refs #17662, the merged review-check publication parent. Related #17661 and #17656 migrate terminal responses and acknowledgements. This slice owns only late stale-head projection repair. ## What Changed - Move the repair predicate and execution into the GitHub provider owner. Match the original admitted delivery and cancellation receipt. Recheck state and assessment in the update. - Register a scoped worker with the existing coordinator. Use keyset pages and fair bounded dispatch. Keep retries only for unresolved repair or failed reconstruction. - Signal before cancellation, automatic receipt insertion, automatic-review binding, and check-driven review-state writes. Preserve existing transaction ownership and outer commit behavior. - Remove the periodic provider wrapper and publication-maintenance callback. Preserve explicit manual recovery and independent missed-webhook recovery. - Add two concurrent, idempotent partial indexes. Use a guarded delivery UUID lookup and a bounded exact-receipt lookup. Add no tables. - Add ten real-database lifecycle and producer cases. Document the producer audit, failure history, and remaining limits. - Keep YAML browser reads scoped to the editor after focus changes, with the original journey assertions and deadlines. ## Verification Current source: `0886a0229b1c4f68d2f582da86aa1f248e9124f3`. - At `dd6bca0485c3444d6e3d0ee10ec4aa651e801392`, **287 distinct cases pass**: 274 complete GitHub workflow cases and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. The workflow suite uses a new, pre-migrated PostgreSQL database. - Coverage includes both orders of cancellation and admission, outer commit and rollback, restart, drain, standby, failed-update retry, concurrent assessment under an observed row lock, shutdown during that wait, more than one candidate page, manual history, and settled-history quiet. - Database build, server typecheck/build, module boundaries, and target migration order pass. - Canonical migration `0403_natural_cerise.sql` preserves both original index definitions and concurrent idempotent creation. All 401 parent migration receipts remain unchanged during upgrade. One new receipt is added. Replay adds none. Both indexes are ready and valid on fresh and upgraded databases. The five newest snapshots remain. - The normal parent merge preserved the prepared tree. The later browser fixture repair changes only its central read helper and verification note. Entire server, package, and UI trees remain identical to the accepted source. - [The owning audit](https://github.com/paperclipai/paperclip/blob/0886a0229b1c4f68d2f582da86aa1f248e9124f3/doc/plans/2026-10-25-github-review-projection-sleep-audit.md) records the exact evidence and earlier failures. Earlier query-plan measurements retain their historical revision scope. - Fresh current-head CI and Greptile review remain required. No deployed sleep, full repository-suite, or fleet-savings claim is made. Failure accounting: browser shard 7 failed twice on `1453a4447c0d4d33f713ae3e00734c7950f95426` because the YAML helper read whole-page clipboard text while main was focused. Three isolated unchanged local passes did not resolve that failure. A controlled native focus-loss probe reproduced the same error before the repair and passed after it. The helper now clears stale clipboard data, focuses the editor, and waits within the existing expectation budget for a fresh YAML copy. All six complete YAML journeys pass with zero retries in 50.9 seconds. Journey assertions and deadlines are unchanged. The temporary fault is removed. The exact CI focus-change trigger remains unknown. [Both CI attempts](https://github.com/paperclipai/paperclip/actions/runs/40777754306) remain part of the verification record. Fresh new-head checks are required. Ponytail follow-up: the 4/5 review of `b3c81ba` was conditional on fixing a stale startup-test slice marker, so the merge was held. Boundary assertions reproduced the invalid -1 end index. The test now uses the existing timer declaration and checks both bounds; all 13 startup/hostname cases pass. The optional no-op coordinator cleanup is already in #17666. The six YAML journeys and all production code remain unchanged. Fresh review and CI are required for this corrected head. ## Risks - Missing a future producer signal could delay discovery until restart. Supported writers share the current database owner's process-local notification scope. Independent SQL writers and other server processes need their own integration. - The migration builds indexes concurrently. The existing migration runner handles interruption and invalid index recovery. - Direct GitHub missed-webhook discovery remains periodic. This change does not make the whole chat service idle. - Existing Cloud wake coverage and sleep blockers remain unchanged. These tests do not prove deployed host or database suspension. - Each remaining sleep slice needs its own review and CI checks before merge. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context-window size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
c2163b3ecf |
test(release): verify paused onboarding setup records (#18102)
## Thinking Path > - Paperclip manages agents and controls when their work starts. > - Onboarding saves an unfinished agent before account connection. > - The setup hold keeps that record paused until Connect completes. > - The published canary smoke still required an empty agent list at that point. > - That old assertion rejected the documented setup state and blocked release work. > - These tests now check the setup hold and the absence of early work. > - The nightly smoke also checks that Connect completes the same record. ## Linked Issues or Issue Description **What happened?** Published canary onboarding failed at the empty-agent-list assertion after reaching Connect a model. The API returned the expected paused setup record. The same failure affected recent canaries and prevented their promotion to a nightly. **Expected behavior** Require exactly one named setup record before Connect, paused with the `setup` hold and heartbeat disabled. Require no tasks or runs. After successful Connect, complete that same agent record. **Steps to reproduce** Run the published canary onboarding smoke with `PAPERCLIPAI_VERSION=2026.1028.0-canary.15`. Enter the organization and agent names, then open Connect a model. The old assertion fails on the held setup record. **Paperclip version or commit** The published artifact is `2026.1028.0-canary.15` at `743698e700638cddc4d3fc4c0ad89e6c132fab40`. The test branch starts at `65fa1cd617b43a45c9d22e10480f4fb6e52dbd21`. **Deployment mode** The canary check installs the exact public npm package in a fresh local-trusted instance. The authenticated nightly smoke uses the existing Docker harness and its provider mock. Refs #17976 and #12667. The existing agent setup contract is in `doc/SPEC-implementation.md` section 10.3. ## What Changed - Replace the stale empty-agent assertion with a bounded check for one paused setup record, disabled heartbeats and no tasks or runs. - Check the same setup state in the authenticated nightly smoke. Require Connect to complete that record without creating a second agent. - Update the onboarding verification recipe with the scope of both release checks. ## Verification - Red: the unchanged smoke failed locally against the exact published `.15` artifact at the empty-agent assertion. This matches the [release failure](https://github.com/paperclipai/paperclip/actions/runs/40761030399/job/123112508486). - Green: the revised smoke passed against the same `.15` artifact, with a real browser and fresh data. No model credential was required. - Full `pnpm build` and `node scripts/check-maps.mjs` pass. - Full recursive `pnpm -r typecheck` passes. - Full local `pnpm test:run` hit embedded PostgreSQL initialization failures. A standalone probe of the same bundled `initdb` reproduced `could not create shared memory segment: No space left on device`; this Mac had exhausted its shared-memory IDs. The local suite was stopped after those failures. No full local-suite pass is claimed. - The additional old-nightly artifact probe stopped before the browser journey with the same PostgreSQL bootstrap failure. Its failure is retained separately from the canary red/green result. - All 54 current-head checks pass; two optional checks are skipped. [CI run](https://github.com/paperclipai/paperclip/actions/runs/40779358071). - [Apex review is 5/5](https://github.com/paperclipai/paperclip/pull/18102#issuecomment-6482273653) for `40c63db4e9bf66e915843a9f325763d449aee049`, with no unresolved review threads. - The first [authenticated Linux canary smoke](https://github.com/paperclipai/paperclip/actions/runs/40779405110) stopped before the browser because the canary tag moved to `.16` before its matching server dependency was published. That dependency is now available and the failed job has been retried. Both the canary and [nightly smoke](https://github.com/paperclipai/paperclip/actions/runs/40780591068) passed background-service installation. Their authenticated Docker browser jobs are still queued for GitHub-hosted Ubuntu runners. Full authenticated browser acceptance remains unverified pending runner allocation. ## Risks These are test and verification-document changes. Unexpected agent states, extra records, early tasks and early execution still fail the checks. The authenticated smoke uses the existing mocked provider acceptance; it does not qualify real provider authentication. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. The session does not expose a more specific model ID or the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — targeted canary smoke; full local-suite limitation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
github-actions[bot]andlockfile-bot
|
efd65c1125 |
chore(lockfile): refresh pnpm-lock.yaml (#18059)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/189/changes/paperclip/189 <!-- entire-trail-link-end --> Auto-generated lockfile refresh after dependencies changed on master. This PR only updates pnpm-lock.yaml. Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
287b04d72f |
fix(evals): keep report dependency proof with lockfile refresh (#18063)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/192/changes/paperclip/192 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and runs paid Product E2E evaluations. > - Paid work needs a report that can install its trusted dependencies. > - The report uses a reviewed offline lock correction. > - New UI dependencies can make that correction stale while its lock hash still matches. > - The lockfile bot must refresh the correction with its lock changes. > - This PR also checks manifest consistency offline before model admission. ## Linked Issues or Issue Description **What happened?** Current master declares new CodeMirror and YAML UI dependencies. Its committed lock omits those UI importer entries. The existing report correction accepts that lock, but a frozen install then fails with `ERR_PNPM_OUTDATED_LOCKFILE`. This can waste completed paid work before the report fails. Refs #18059. That bot PR supplies the matching lock. This PR keeps the root lock bot-owned and supplies its exact offline report correction. The correction changes 21 importer lines and leaves all package records unchanged. **Expected behavior** The existing bot includes matching report proof in its dependency PR. Paid authorization rejects stale manifests before model work. Report installation still accepts only reviewed hashes and applies no registry resolution. **Steps to reproduce** 1. Run a frozen pnpm 9.15.4 install on master at `7f10cc058568e3c1ac133cf24f3b4000cbdfca0e`. 2. Observe the missing UI importer entries. 3. Apply the reviewed correction in this PR. The frozen install passes. ## What Changed - Add an explicit refresh mode that derives the offline patch and hashes from a known committed lock and the bot's resolved lock. - Keep the existing unknown-lock, patch hash and output hash guards. - Include the proof files in the existing lockfile bot PR. New bot PRs use complete descriptions; existing descriptions stay intact. - Validate the frozen lock against current manifests offline before paid admission, using the existing pinned pnpm version. - Refresh the current correction and document this process. The root lockfile is not committed here. ## Verification - 24 focused report-lock and workflow-security tests pass. They cover offline replay, tampered input/patch/output rejection, real Git refresh, unknown-base refusal and unchanged proof. - A full frozen install passes after the reviewed correction, with lifecycle scripts disabled. - A fresh source archive validates offline with an empty package store. - A deliberate manifest change fails that same offline check with `ERR_PNPM_OUTDATED_LOCKFILE`. - Both workflow files parse as YAML. All 95 shell steps pass Bash syntax checks. - `git diff --check` passes. These checks make no model requests. - Full repository checks and fresh reviews must pass on the final PR head before merge. ## Risks - Dependency changes must still pass review. Refresh refuses an unknown committed baseline. - The bot now commits two additional proof files. Its write permissions and target branch stay unchanged. Existing PR descriptions are preserved. - The paid authorization job adds an offline manifest check. It runs no package lifecycle scripts. - No budget, provider credential, environment permission, schedule or paid enable gate changes. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
75754ffb7a |
fix: offer setup instead of Run for unfinished agents (#17976)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent can have a saved record before its setup is complete. > - This record stays paused with the setup reason and cannot run. > - Its shared action controls still offer Run beside Finish setup. > - Run then produces a generic refusal instead of taking the user to setup. > - This change makes Finish setup replace Run for that state. > - The user returns to the saved agent and can run it after setup completes. ## Linked Issues or Issue Description **What happened?** The header of a saved agent with unfinished setup offers Run. Clicking it produces "Agent is not invokable in its current state." Finish setup is a separate control that the user must discover. **Expected behavior** Show one Finish setup action in place of Run and Resume while the setup hold is present. Open the existing setup record with its selected runner. Restore normal controls when setup finishes. **Steps to reproduce** 1. Start creating an agent and save it before setup completes. 2. Open the agent from the roster to view its header. 3. Click Run and observe the generic refusal. Related: #17965 fixes the managed Runner connection-test result. This change fixes the controls for a saved setup record. No open duplicate was found in the title search. ## What Changed - Replace Run with one Finish setup action for a unfinished setup record. - Use the existing paused-state fallback when lifecycle details are absent. - Keep the saved agent ID and runner selection in the setup route. - Check unsaved changes before leaving the detail page. - Refresh pause reason and time with lifecycle state so completing setup in another tab restores Run without overwriting editable configuration. - Keep setup available when a caller hides Run or disables work actions. - Extend the agent action tests, onboarding browser journey and feature map. ## Verification - Red: three component regressions fail before the fix. The old UI keeps Run, lacks setup without lifecycle details, and skips the navigation guard. - Three more regressions fail before the preparation-state fix. Preparation, verification and pause transitions retain Finish setup; terminal states cannot reopen setup. - Green: 211 focused tests pass across agent actions, duplication, detail pages and saved setup. The final control suite passes 36 tests. - A real QueryClient, lifecycle hook and action-control regression fails before the pause-field refresh fix, then passes after it. The short-URL editable snapshot stays unchanged while the UUID lifecycle query restores Run. - UI typecheck and production build pass. Token and map checks pass. - The real Chromium onboarding journey passes on final source `67c4c59f6fbfc5bc8eca8f70e12b8dc1c3763f4f`: no Run or Resume, one Finish setup after reload, and the same saved agent reopens. The first attempt could not initialize the disposable PostgreSQL instance at the local shared-memory limit; no app assertion ran. After removing only a verified detached segment from this task’s completed browser run, the unchanged journey passes. No staging deployment is claimed. - A separate local integration branch combines this PR with the final write-recovery fixes from #17960. Its 36 action-control tests and the real onboarding browser journey pass too. This integration preview is not pushed to the PR branch. - All hosted CI gates pass on the final commit, including typecheck, tests, build and E2E. Apex is 5/5 with no unresolved findings. ## Risks - The change uses the existing setup pause reason through preparation and verification. Terminating and terminal records cannot reopen setup. Manual pauses retain Resume, and ready agents retain Run. - The API remains authoritative if state changes while the page is open. No admission, permission, account or credential rule changes. - The browser journey stops at setup configuration. It does not claim provider execution or deployment acceptance. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
d30c8f42b8 |
feat(ui): keep writes durable through temporary disconnects (#17960)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/123/changes/paperclip/123 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People save tasks, write documents, make decisions and send board messages. > - A disconnect can hide a response after the server accepts a write. > - Recovery must keep the original account, task and request key. > - The merged connection foundation in #15754 handles reads and shared outage state. > - This change adds bounded write recovery and explicit actions when recovery stops. ## Linked Issues or Issue Description **What happened?** Temporary failures could lose edits, report accepted writes as failures, or require people to copy and send input again. A retry could target a newly selected task or account. Restoring attachments could create a second task. **Expected behavior** Keep the draft, its owner and its target. Retry only operations with a server contract that prevents duplicate writes. Verify uncertain results. Show conflicts and permanent failures, with an explicit resend or cancel action. **Steps to reproduce** 1. Save a task title during an outage, then open another task before recovery. 2. Restore a task creation that needs files, selecting those files in another order. 3. Lose a document or decision response, then interrupt the verification read. 4. Switch accounts while a create, upload or connected-channel publication is pending. 5. Type and erase a new-task draft before the first account lookup completes, then reopen it. This PR carries the write layer from #15769 and its review fixes. It targets master after #15754 merged. Related test cleanup #16952 already covers the unchanged copy-link toast test; this PR does not duplicate that work. ## What Changed - Add bounded autosave and submit jobs for supported writes, with pending, conflict and manual retry states. Retain admitted autosaves after navigation and preserve newer task revisions. - Keep each job with its original account, task and operation. First account resolution adopts admitted startup work; a later account change cancels old work. Remount inline and document editors when their owner or document subject changes. - Preserve task-creation keys and immutable request snapshots through file reattachment. Restore generated document keys before comparing reselected files. Reject changed requests instead of creating a second task. - Check the original account scope before creates and each upload. Save successful upload receipts before stopping on an account change. Identify attachments by content digest and metadata. - Verify uncertain document and decision results. Replay only plain, keyed comments automatically; combined reassignment, reopen and run-setting changes require receipt reconciliation or cancel. Settle each request once when a receipt races its retry. - Scope drafts and connected-channel publication reads to their owner. Keep unowned older records unread and unreplayed, with an explicit confirmed discard action after Activity inspection. Preserve typing through the first session lookup, keep erased text erased, and report browser persistence only when storage succeeds. - Wait for the first account lookup before sending startup status changes or decisions. Retry failed required lookups with a fixed limit and surface exhaustion to the original action. Preserve uncertain comment keys on account cancellation. Check the account generation before a comment retry calls its sender and again immediately before POST after asynchronous preparation. Retire only that request’s optimistic echo on completion. - Return an ordinary comment’s accepted same-company, same-task, same-user receipt before workspace reopening, task status changes or wakeups. Reused keys with changed text still conflict. Complete reference and activity records through the same finalizer for initial sends and accepted retries. It locks the task before its comment, matching task and company deletion, comment removal and tombstoning, rejects removed or tombstoned comments, repairs references and inserts a missing activity record in one transaction, and publishes after commit. - Remove reader-only record and comment-body size caps so accepted browser writes remain readable, including JSON-escaped comment requests in local and session storage. Update four existing Storybook fixtures to use owned draft keys. - Document the write contracts and add regression and real browser recovery coverage. - Add the exact reviewed-save recovery regression for both composer implementations and reloads. Align receipt completion, both comment deletion methods and company deletion with task deletion by locking tasks first. Cover real task and company deletion races. ## Verification - The exact reviewed-unknown-save regression fails in all four cases on original commit `1ed0af4186`. It passes with this PR for both composers, before and after reload. Cancel keeps the text and uploaded receipt. The retained draft sends, and the next message also sends. Run `pnpm exec vitest run --project @paperclipai/ui ui/src/components/composer-recovery.test.tsx`. - The new real PostgreSQL task-deletion race fails with SQLSTATE `40P01` on parent commit `f3253a00c4`. The company-deletion race also fails with `40P01` on `e8cf0e02dd`. All six receipt tests pass after the final lock-order fixes. They cover company deletion, task deletion, comment removal, tombstoning, concurrent cross-task references and interrupted reference repair. Run `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/issue-comment-receipt-postgres.test.ts`. - The full UI run passed 9,656 of 9,659 tests across 816 files. Three cases failed under that run: two large-tree timeouts and one plugin-panel assertion. All 67 tests in those two files passed in an isolated rerun. These failures remain recorded; this does not claim a clean full UI run. - `pnpm -r typecheck`, `pnpm build`, token gates and map checks pass on the final change. The local full repository test command was started, then stopped when the delivery moved to this existing PR. Fresh PR CI must provide the full-suite gate. - A local Storybook walkthrough used the real rich editor. Cancel kept the restored text, Send cleared it after the simulated save, and a new draft enabled Send again. Storybook simulates submission; it is not staging acceptance. - Earlier verification on `f3253a00c4` recorded five browser journeys for title saves, cached-task offline recovery, startup recovery, accepted comment replay and session lookup failure. Those results remain prior-head evidence. No new staging deployment is claimed. - On `e8cf0e02dd`, all CI checks passed after one browser-shard rerun. Its first failure copied page text instead of YAML in an unrelated agent-configuration test. On final commit `35131d4380`, Greptile is 5/5 with no unresolved review threads. The first CI attempt has two failures outside this diff: runner close-attachment suspension and scheduled-retry teardown. The scheduled-retry file passes all 14 tests in isolation; the isolated runner test still fails locally. The single standard CI rerun passed both jobs, and the aggregate verification gate passed. All CI checks are green on `35131d4380`; Apex is 5/5 on that same commit with no unresolved findings. ## Risks - Automatic replay requires server duplicate prevention or result verification. Uncertain combined mutations require receipt observation or cancel and a fresh intentional send. - Old unowned form drafts remain stored but are not assigned to an account. Unowned channel records can be retired after Activity inspection; this never cancels an accepted remote update. - Browser storage and hashing can be unavailable. Storage failure is reported honestly. Without a digest, replay can duplicate an upload rather than silently drop a selected file. - Detached autosaves remain bounded and check account ownership on each attempt. Conflicts or exhaustion retain the draft for explicit recovery. - No schema, credential or agent permission change is included. This PR does not claim staging deployment or original-task acceptance. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) produced the original implementation with tool use. OpenAI Codex, GPT-6 family, assisted with restoration, integration, verification and review fixes. This Codex session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Amy (Opus) <claude@users.noreply.github.com> Co-authored-by: Dotta <bippadotta@protonmail.com> |
||
|
DottaandPaperclip
|
7f10cc0585 |
feat(evals): show full task conversations in E2E reports (#18055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests verify tasks through the application and retain reports. > - A final screenshot cannot show the complete task conversation. > - Debugging a result requires the earlier requests, replies, and answers. > - This pull request captures task conversations before fixture cleanup and adds them to Test details. > - Reviewers can read each task thread without a running test instance. ## Linked Issues or Issue Description **What existing behavior does this improve?** The task evidence shown in Product E2E campaign reports. **Current behavior** Most task results show screenshots and checks. Screenshots show the final visible state. They omit earlier messages and long replies. Only the first-task suite has a conversation view. **Proposed behavior** Every standard runner attempt captures the saved conversation before cleanup. Test details show full task messages, requests, answers, and native agent progress. Related tasks have separate threads. Failed attempts also retain available history. Partial captures are labeled. **Reason and benefit** Readers can debug a task by reading its conversation. They do not need to reconstruct it from a screenshot or private run logs. GitHub searches found no duplicate PR or issue for this change. This is an improvement to test reporting and does not overlap a planned core feature in ROADMAP.md. **Breaking changes** None. The result adds an optional `taskChat` field. Old results still render. A result without a task has no chat section. Older reports need retained conversation evidence before they can gain this view. ## What Changed - Capture comments through all pages, task briefs, visible request copy, recorded answers, and native progress before fixture cleanup. - Include delegated tasks and standing chats. Preserve available history when a read fails. Report capture limits explicitly. - Add a closed, credential-redacted conversation format to normalized results. Exclude reasoning, raw tool payloads, and provider session metadata. - Render task conversations inside Test details. Retain them through report aggregation and dashboard regeneration. - Add capture, publication-format, regeneration, and browser tests. Update the evidence and security contracts. ## Verification - Report tests: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts task-chat.test.ts first-task-report.test.ts report.test.ts history.test.ts` — 103 tests passed after review fixes. - Browser tests: `node node_modules/@playwright/test/cli.js test --config tests/runner-e2e/report-playwright.config.ts` — 6 tests passed after rebase. - Targeted TypeScript check of the new capture, renderer, validator, report, and browser-test files passed. - Tests cover a 1,105-message thread, a long final reply, event pagination, partial capture, company access, credential redaction, HTML escaping, and report regeneration. - Review regressions reproduced two failures before repair: the task-run API uses `runId`, and item reviews save verdicts in `result.items`. Capture now follows these contracts. Both regression tests pass. - Greptile reviewed commit `471236438e03e54cc4ad85de6653adf45835063c` at 5/5 with no new findings. The separate Paperclip review execution stopped without an assessment and ended neutral. Its cause is not established; this is not a completed Paperclip assessment. - Execution discovery passed for `core-compatibility.runner-codex.local.message-marker`. No paid provider run was started. - Full local commands were attempted. `pnpm -r typecheck` and `pnpm build` stop because the DB package cannot find `tsx`. `pnpm test:run` stops because the general-server lane cannot find `vitest`. These package dependency links are absent in this checkout. - [Remote CI on the final commit](https://github.com/paperclipai/paperclip/actions/runs/40728689517) passed: typecheck and release registry, all general and serialized test shards, Runner static/Rust/Vitest checks, build, canary dry run, all eight browser E2E shards, and final verification. Security and contributor checks also passed. - Open a new task-based result and expand **Test details**. Read **Task chat** and expand each related task thread. Results without tasks keep their existing layout. ## Risks - Published reports now include fixture conversation text. Capture retains only reviewed display fields and redacts credentials. Fixture authors must keep this text suitable for public reports. - Long threads increase report size. Collection has a 30-second request-admission budget and page limits. It labels incomplete evidence and does not change the workflow grade. - Old reports cannot recover conversation text from screenshots. New attempts must retain the evidence. ## Model Used OpenAI Codex (GPT-6). The exact serving snapshot and context-window size are not exposed in this session. Used reasoning, code execution, repository tools, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b14e1ecaf8 |
feat: add a complete agent YAML configuration editor (#18036)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/175/changes/paperclip/175 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators configure each agent through several settings tabs. > - Operators also need one complete, editable configuration document. > - The document must retain each resource service's access and validation rules. > - This pull request defines the `paperclip/v1` Agent YAML format and adds an Advanced editor. > - The editor previews changes, rejects stale core writes, and explains partial saves. > - Operators can edit agent settings in one place and verify the saved state after reload. ## Linked Issues or Issue Description Refs #2219. Related settings work: #10974. **Problem or motivation** Agent configuration spans profile, runtime, instructions, skills, tools, channels, permissions, secrets, keys, and revision history. Operators cannot inspect and edit those settings as one versioned YAML document. **Proposed solution** Add an Advanced agent tab with a YAML editor. Define a shared document schema. Read and write resources through their existing APIs. Separate editable settings, explicit commands, and read-only observed state. **Alternatives considered** A raw agent-row editor would omit related resources. A generic database update would bypass resource ownership. The editor therefore uses the existing resource contracts and records partial results when a later request fails. **Roadmap alignment** This extends the shipped agent configuration capability. It relates to easy AGENTS.md configuration, multi-harness teams, governed tool access, and secrets management. Stored credentials remain redacted. This format is an editable settings view; it does not add company portability or provider consent. ## What Changed - Add the `paperclip/v1` Agent document schema and format documentation. - Add the Advanced route, sidebar entry, YAML editor, validation, change summary, save progress, and reload recovery. - Cover core settings, instruction files, skill pins, tool installs/profiles/policies, channels, permissions, secret operations, API keys, and revision restore. - Add private-keyed configuration revisions checked under the agent write lock. Read the current key through its owner so file-based rotation does not require restarting the server. Preserve unchanged secret markers at their original paths. - Retain drafts and returned credentials after partial saves. Require reload before retrying commands. Prevent duplicate submissions. Keep new key tokens visible through rename and disable saves with an empty change plan. - Send only changed channel settings. Require explicit intent for shared profile edits. Load destination instruction files when changing their location. - Update Storybook fixtures, verification maps, and regression tests. Add browser acceptance through real APIs and PostgreSQL. ## Verification - `pnpm -r typecheck` passes. - `pnpm build` passes. - The focused UI suite passes all 50 tests. The API/revision helper suites pass all 93 tests. - All 65 lifecycle tests pass with a dedicated PostgreSQL database. They exercise real row locks, redaction, revision history, audit, and company access. - All six browser acceptance cases pass. They check saved settings and clears, secret redaction, key creation/revocation, stale drafts, lost-response recovery, instruction relocation, revision restore, and key-token retention through rename. The tests use real resource APIs and storage. The MCP target uses the existing local provider fixture. No model run or provider login is claimed. - Signing-key regressions cover both environment changes and real in-place key-file rotation. The protected database save/fencing cases pass after removing the cache. A local measurement of 1,000 file-backed revision calculations took 48 ms. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and Storybook catalog checks pass. - The full [PR CI run](https://github.com/paperclipai/paperclip/actions/runs/40725425470) passes on `34ed97bbf9`: all 48 jobs, including all eight browser shards. Greptile is 5/5 on this commit, with no unresolved review threads. This final commit refreshes CI after a runner-routing fallback; its source tree matches `fb4cf488cf`. - Paperclip Review and Superagent Security Scan ended neutral because their external agents did not complete a valid assessment/receipt. Superagent passed on `fb4cf488cf`, which has the identical source tree. One request to retry the final scan returned HTTP 404. Neither check reports a code finding. - The local full-suite attempt was blocked by host System V semaphore exhaustion. Acceptance and lifecycle verification used an isolated PostgreSQL 18 container. Earlier CI runners were interrupted. A local combined run hit hook/test timeouts; the full 88-test route file passed after the builds finished. The final full CI run passes. Earlier failure logs and browser traces remain retained outside the repository. To run the browser regression: ```sh pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/agent-configuration.spec.ts ``` ## Risks - Multi-resource saves are sequential. Earlier operations can remain saved if a later operation fails. The editor reports the completed steps and requires reload. - Core settings and managed instruction files have atomic revision/hash checks. Other unversioned resource writes and external supporting files retain a race window after preflight. - Shared profile definitions affect other bindings. The editor requires `updateSharedDefinition: true` for those edits. - The editor follows native service normalization and experimental settings. Canonical reload can add generated access profiles or mask environment values. - New editor dependencies require the repository's normal automated lockfile update. This pull request does not edit the lockfile or add a database migration. ## Model Used OpenAI Codex, GPT-6 family. The exact current model version was not exposed. The work used reasoning, repository tools, code execution, and browser automation. Earlier work in this chat used the configured Codex model before a model switch; its exact model ID and context window were not exposed. The current context window size is also not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7d75ae0569 |
fix(server): recover Teams attempts from saved claim deadlines (#17592)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves a claim before each provider operation. > - A crashed operation can leave that claim unfinished. > - Periodic maintenance scans for abandoned claims even when none exist. > - This PR wakes recovery from committed changes and the saved claim deadline. > - Empty work becomes quiet while unknown effects remain safe to review. ## Linked Issues or Issue Description **What existing behavior does this improve?** Teams abandoned file-transfer recovery. This is slice 27, stacked on consent expiry #17590. **Current behavior** The remaining Teams maintenance callback scans every 30 seconds for expired transport claims. It runs from shared publication maintenance. **Proposed behavior** The Teams owner reconstructs existing claim deadlines at startup and reloads a transfer after its committed update. At expiry it clears the claim and preserves the existing unknown-outcome or conflict state. Normal receipt settlement removes the deadline. **Reason and benefit** Remove the last periodic Teams maintenance scan without resending a possibly delivered card or file. **Breaking changes** No schema or public API change. Explicit diagnostic recovery methods remain compatible. Consent expiry stays with its separate parent worker. ## What Changed - Add a Teams-owned worker for consent_sending, uploading, file_info_sending and conflict rows with a saved attempt expiry. - Notify from the existing versioned update before writes. Preserve transaction ownership and shared uncertain-write reconciliation. - Expose a narrow per-transfer recovery method that uses the existing lock order, integrity checks and atomic projection. - Remove the periodic Teams maintenance callback and its publication-maintenance invocation. - Add real claim, receipt, conflict, backlog, restart, precision, failure, drain and shutdown tests. Wire the owner through existing app admission gates. - Retain the existing signoff cancelled-run selection test and GitHub inline retry-window race assertion. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-attempt-recovery.md` and its linked audit. ## Verification Published head: `24c54399909105984e2a10f5869064984f69c472`, targeting master after consent #17590 merged at `4a89456bb3f9aef87cfe0e922e8b9d1a777a6c7e`. A normal ancestry merge reconciles the landed squash with the previously verified child. The complete 13-path binary patch is identical before and after reconciliation, and all 30 unrelated master paths are preserved. This adds no new worker behavior; the 239-case local evidence below belongs to the verified unchanged patch. All 58 current-head checks are now clean, including the complete required CI run. Greptile is 5/5 at this exact head. The current Ponytail turn is paused without a verdict, which is an explicitly authorized nonblocker; no failing review is waived. The complete 13-path slice preserves the reviewed attempt implementation and inherits the parent's report migration 0397, bounded consent worker and retention CI test repair. No new schema, queue table, shared scheduler contract, workflow or lockfile change is added by this duty. - **239 distinct selected cases pass:** 221 across five complete transfer/publication/consent/runtime/app-wiring suites, 16 selected real Board/Teams-authority/GitHub-retry integration cases, and two delegated Slack policy cases. Other chat integration cases (1,579) and connection-management cases (81) were skipped, not passed. Each database owner uses a new isolated PostgreSQL database. - Real Board publication saves an active attempt deadline; its confirmed receipt retires recovery, and SQL stays quiet afterward. The inherited consent producer and authority cases still pass. Lifecycle coverage exercises claim stages, unknown-outcome quarantine without repeating provider I/O, conflicts, rollback, restart, precision, backlog/fairness, failure recovery, drain and shutdown. The changed GitHub metadata/retry race passes without duplicate assessments. - The old server10 CI failure, run `40557802358` job `122390238707`, was a delegated Slack invitation rate-window assertion (2,122 passed/one failed/15 skipped in that shard). The verified parent already contains its Date-only fixture fix. Both linked and invitation cases now pass their full caller/target/shared-policy scenarios on a fresh server fixture; no duplicate policy change or new owned file was added. That old failure is preserved rather than called a pass. - Server production typecheck, build and module-boundary checks pass. Test files were executed; production typechecking does not statically check them. The existing test-only signoff cancelled-run selection fix is retained but was not executed locally in this acceptance; fresh remote E2E gates remain required. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-25-teams-attempt-recovery-sleep-audit.md#current-consent-parent-preparation-and-acceptance-october-28) records current scope, commands, counts and limits. Earlier 235-case and combined-stack counts belong to their stated revisions and are not added to 239. The [stack ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository suite, live Teams/Slack tenant, external wake, physical suspension or savings claim is made. - **Exact-head Greptile 5/5 and all 58 checks are clean.** Actual failing Ponytail findings must be repaired; a missing/stopped/paused turn without a verdict is an authorized nonblocker. An active no-verdict turn waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older reviews and parent checks are not treated as current passes. ## Risks - Recovery must not infer that an expired claim means a provider effect did not happen. It retains unknown outcomes and does not resend. - Late receipts must still match current claim identity. Current-state rechecks and existing protocol tests preserve this fence. - Supported deployment uses one active writer process through its owning DB client. Independent SQL and additional processes need a separate wake contract. - Cloud external alarm coverage and sleep eligibility are unchanged. Other chat duties still run periodically until their own migrations. - Voice-report parent review repairs are propagated: explicit single-active-writer scope and an indexed report window. Thirty report-worker/startup tests pass on the combined child. Fresh remote checks, older stack findings and final master reconciliation remain required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d8bd8fe9fb |
feat(evals): browse E2E suites across targets and models (#18027)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - E2E tests show whether people can complete real app and agent journeys. > - Operators need to choose a suite and inspect its evidence across deployments and models. > - The history page listed campaigns without a suite browser or clear coverage filters. > - QA smoke also appeared separate from the other retained E2E measurements. > - This pull request puts suites, deployment targets, and named selections in one history browser. > - The benefit is that a passing subset stays distinct from a full suite run, with direct links to evidence. ## Linked Issues or Issue Description Related: #18009 added clear per-test details and links to the tested application. This PR extends the shared E2E history browser. **What happened?** The public directory called full app testing “Product E2E Evals” and made users choose campaigns before they could find a suite. The history list could not filter independently by tested app, agent execution, model, or run selection. A recent passing subset could be mistaken for a full suite result. **Expected behavior** Browse current and retained suites in one E2E category. Keep QA smoke in normal campaign history. Show the latest matching run and latest full suite run. Preserve the recorded definition and denominator when filters change. **Steps to reproduce** 1. Open the E2E history directory. 2. Find a suite, then compare a narrow passing run with a full run across execution environments and models. 3. Find the QA smoke measurement and open its retained screenshots. **Paperclip version or commit** The previous report UI was at `8279c99008`. This branch is rebased on current master. **Deployment mode** Static E2E history and campaign reports generated from retained measurements. ## What Changed - Call full-product testing “E2E tests.” Keep Runner protocol evals separate because they use a mock control plane. - Add a searchable suite sidebar that includes current suites and suites found only in retained history. - Add correlated suite, coverage, deployment target, execution, provider, model, profile, runtime, result, date, and named-selection filters. Keep filters in the URL for reload and back navigation. Sidebar suite links clear other filters; the Suite dropdown keeps deliberate intersections. Incompatible suite/preset URLs explain the empty selection and link to the suite runs. - Show latest matching and latest full run links. Distinguish full, subset, unfinished, and unknown coverage from the recorded suite definition and execution counts. Keep failed full runs visible. - Retain the tested application URL and runtime mode in new history records. Older records show “Not recorded.” Browser-only journeys do not gain an inferred agent or model. - Add an index-only refresh command that can import a reviewed public campaign once. It rejects private Cloud bundles and does not replace retained campaigns or upload evidence. - Keep existing report URLs, test IDs, commands, and measurement timestamps stable. Campaign links carry suite/profile/environment; target/model/provider/runtime remain history-level filters. - Add unit and synthetic browser regressions, including mobile navigation, long source refs, sidebar filter reset, and empty-selection repair. Inset dropdown carets in the shared report styles. Document direct Node entry points without adding package.json shortcuts. - Add a prominent Reset all filters button, blue active-filter borders and tint, and visible active counts for the main and advanced filters. Reset clears all 13 history filters, sidebar search, URL state, and the attention view. Campaign-page filters use the same active styling. ## Verification - Post-rebase focused Vitest: 127 tests passed across 10 files. The latest follow-up selection passed 100 tests across 9 files. E2E support TypeScript passed after both changes. - The synthetic Chromium journey covers full and subset coverage, unfinished selections, correlated filters, browser-only QA selection, URL/back/reload, and desktop and touch navigation. The expanded journey verifies 13 active history fields, the collapsed advanced-filter count, reset of every field and sidebar search, and active border color changes that revert after reset. It makes zero provider calls. - Hub Python tests: 7 passed. The browser harness also exits promptly when Chromium is unavailable; its HTTP server closes on launch failure. - Real history verification covered 419 campaigns and 59 suites. All 418 previous campaign records remained deeply equal after importing the reviewed QA campaign. Publication used the authoritative history ETag. Only mutable index/history/pointer files were written; immutable campaign bundles stayed unchanged. - A live browser walkthrough passed from the two-family eval hub through QA smoke, Basic app journeys, target filtering, the retained report, screenshot gallery, history, and the logo link. The QA URL was clickable and the image loaded. No browser JavaScript errors occurred. The original QA normalized results and all three PNG hashes remained unchanged. - Live [E2E history](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history), [core compatibility suite](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history&suite=core-compatibility), and [QA smoke runs](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history&preset=qa-smoke). - The live follow-up verified the incompatible Stock harness/QA-smoke URL, the repair link to 21 Stock harness campaigns, and sidebar navigation that clears preset/model filters. Dropdown carets were inspected in dark, light, and mobile views. No JavaScript errors occurred. - The final live follow-up verified reset of history controls, URL state, sidebar search, and the attention view, plus reload/back restoration and mobile touch reset. Active filter borders and tint were inspected in dark and light views. Campaign-page reset restored all three QA cases. All 419 history records and original QA measurement/evidence hashes stayed unchanged. - Full repository typecheck, test, build, and E2E CI gates passed on final head `1334cb79a5`. Both enforced aggregates, `ci / verify` and `ci / e2e`, completed successfully. Fresh Greptile review scored that exact head 5/5 with no unresolved review threads. ## Risks - Historical metadata can be missing. Unknown targets and runtime details stay explicit instead of being inferred from a worker environment. - Full suite coverage follows the saved definition and selection. Filtering to one model does not redefine that denominator. The latest full-run shortcut can cross coverage selections and says so in its detail text. - This changes shared report presentation. It adds no application runtime or database changes. The browser harness and index refresh do not invoke models. - Public history publication must keep its existing conditional-write and retained-evidence boundaries. Private bundles, raw logs, credentials, and screenshots are not part of this PR. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8fe5d5defc |
fix(evals): honor the shared feedback read deadline (#18026)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E evaluations must retain feedback from each paid attempt. > - The collector reads saved feedback and native tool events before teardown. > - A five-second request limit can stop a valid read before the shared read deadline. > - This pull request gives event reads the remaining shared read budget. > - The total deadline and reserved checkpoint time stay the same. ## Linked Issues or Issue Description **What happened?** A paid Ask attempt passed the product grade but retained partial feedback invocation coverage. The collector recorded `events_read_failed`. Its event endpoint later returned HTTP 200. The retained timing is consistent with a request that exceeded the five-second request limit. This change fixes that deterministic failure path; it does not prove the cause of every historical partial read. **Expected behavior** A valid read that takes eight seconds can complete within the 30-second collection deadline. A stalled read must still time out and leave time to save the available feedback. No read or paid attempt is retried. **Steps to reproduce** 1. Return a terminal native run and an empty saved-feedback list. 2. Make the event read take eight seconds and respect the supplied request timeout. 3. Run `collectAttemptFeedback` with a 30-second deadline. 4. Before this fix, the read times out after five seconds and invocation coverage stays partial. Related accounting and reporting work: Refs #18010. ## What Changed - Use the remaining shared read budget for event API requests. Keep the five-second cap for other reads. - Add regressions for an eight-second event response, a stalled event response that preserves saved feedback, and a stalled early feedback read that still permits the event scan. - Document the read budget, reserved persistence time, and no-retry behavior. ## Verification - Red: the initial two regression tests fail before the fix. The other 42 feedback tests pass. The early-read starvation review regression then fails against the first implementation (44 pass), and passes after narrowing the change. - Green: 58 tests pass across feedback collection, interrupted recovery, process shutdown, and scheduled finalization. - Full Product support run: 4,609 pass, three skip, and the retained embedded PostgreSQL bootstrap test fails on this host. The log remains available. This is not a full local pass. - Local Product typecheck fails on installed workspace export mismatches outside the changed files. Hosted CI must check the exact commit with clean dependencies. No local full repository build pass is claimed. - Original paid grades, partial feedback receipts, and accounting journals remain unchanged. This repair makes zero provider calls. ## Risks - A stalled event read can consume the remaining read budget. Other endpoints keep their five-second cap. The shared deadline still reserves up to five seconds for the final checkpoint. - A slow event page can leave later runs uninspected. Those runs remain partial or unavailable. The change does not treat missing evidence as success. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8279c99008 |
fix(evals): clarify report details and link tested applications (#18009)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E reports help operators inspect real application journeys. > - A worker environment does not identify the application that a journey tested. > - Reports hid that URL and gave missing diagnostics too much space. > - Standalone reports also sent the logo link to a missing parent page. > - This pull request shows the recorded target and checks in the shared report. > - The benefit is a clear report with useful links and inspectable diagnostics. ## Linked Issues or Issue Description **What happened?** A standalone Product E2E report had no application link in test details or the screenshot gallery. Its details showed several empty diagnostic sections before useful information. The logo could resolve to the page host root and return 404. Campaign notes added outside the report shell could overlap the fixed sidebar. **Expected behavior** Show the recorded application URL beside each test and screenshot. Put recorded checks before optional diagnostics. Keep campaign context inside the shared layout. Link the logo to the evaluations directory. **Steps to reproduce** 1. Render a retained campaign at a standalone page path without a history index. 2. Open a test with screenshot evidence and missing startup timing. 3. Inspect test details and gallery information, then click the report logo. **Paperclip version or commit** Reproduced in the report renderer at `39e811c9c5`. This branch applies the fix to current master. **Deployment mode** Static Product E2E HTML report generated from retained normalized results. ## What Changed - Add optional recorded `targetUrl` metadata and resolve existing Product E2E and provider-connection targets. Only HTTP and HTTPS links without embedded credentials are shown. - Show the target URL in test details and screenshot gallery information. - Put recorded checks and execution facts first. Keep raw usage, timing, and startup details under Diagnostics. Preserve missing metadata as unknown. - Keep campaign titles, descriptions, notes, and evidence links inside the shared report layout during retained-evidence regeneration. - Fix standalone navigation and let the brand lockup wrap at narrow sidebar widths. - Match report profiles by suite and recorded provider/model before falling back to an inferred profile. - Add report regressions and document the retained metadata contract. ## Verification - Focused report tests: 86 passed across 8 files on the isolated branch. Command: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts report-target.test.ts report-navigation.test.ts report-catalog.test.ts first-task-report.test.ts report.test.ts dashboard-regenerate.test.ts`. - Browser verification covered dark desktop widths 1440 and 1000, light intermediate width 780, and dark mobile width 390. It covered the drawer, URL links, gallery keyboard navigation, focus return, and the logo destination. - The published retained report keeps all 3 original passing cases and 7 recorded checks. Its campaign and screenshot bytes remain unchanged. This presentation refresh made zero provider calls. - Full `pnpm build`, `pnpm -r typecheck`, and the eval TypeScript check passed after rebuilding the reused package dependencies. The first typecheck attempt had stale local dependency builds; those errors are resolved. - The local `pnpm test:run` attempt hit repeated embedded PostgreSQL bootstrap failures in server suites. I stopped that attempt and retained its log. It is not counted as a passing check. Current-head CI remains the full test gate. - Review follow-ups add provider/model and cross-suite profile regressions. Malformed campaign presentation metadata is rejected before any report or retained-result writes; ten malformed input cases verify that retained results stay unchanged. ## Risks - This changes presentation across Product E2E reports. Historical reports without a target show “Not recorded” rather than an inferred URL. - Unknown startup reuse data remains in raw metadata but no longer creates an empty table. - Standalone logo links open the shared public evaluations directory. Campaign history links still use a matching history index when present. - No application runtime, database, or measurement verdict changes are included. Private QA configuration and raw campaign evidence stay outside this PR. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bd0d81c7b |
fix(e2e): follow proven workspace deferrals during onboarding (#18008)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify the user outcome through the browser and scoped APIs. > - An onboarding child can wait while its parent owns their shared workspace. > - The server records that no provider work started and schedules a durable replacement. > - The journey rejected that expected wait, and feedback counted it as unknown legacy execution. > - This pull request follows the exact replacement and records the distinct no-provider evidence. > - The child must still finish through native execution and produce the independently checked download. ## Linked Issues or Issue Description Refs #17940. **What happened?** The fresh-workspace journey failed with `onboarding_first_task_failed` when a child encountered shared-workspace contention. The cancelled admission had an explicit `workspace_wait` receipt with `providerWorkStarted: false`. Its linked replacement was still scheduled. The runtime had not been selected. The default `legacy` value did not represent a provider execution. **Expected behavior** Keep both rows and their run-limit cost. Follow only the unique replacement for the same company, agent, and issue. A pending replacement must not pass. Real failures and contradictory execution evidence must still fail. **Steps to reproduce** 1. Start a native onboarding parent and accept its child-task proposal. 2. Observe the child admission while the parent still holds their workspace. 3. Present the server's no-provider cancellation and its durable scheduled replacement to the journey and feedback collectors. ## What Changed - Add a shared predicate for the explicit no-provider workspace-deferral receipt. Reject runtime, process, output, usage, and cost evidence that contradicts it. - Follow exact, unique retry lineage during onboarding. Wait for active or scheduled replacements without repeating browser actions. Preserve the campaign deadline, run cap, native artifact attribution, and download oracle. - Retain per-run feedback inspection classifications: `native_events`, `provider_not_started`, or `incomplete`. Require exhaustive event reads. Keep pending work and uncertain execution partial. - Add causal regressions and document the evidence contract. Historical failed attempts keep their original grades. ## Verification - Red: the new journey regression failed with the observed `onboarding_first_task_failed` before the repair. The new feedback cases also failed before the repair. - Green: 70 journey tests and 74 feedback-family tests passed. The final affected journey/feedback gate passed 275 tests plus 137 Node checks. Five retry-lane reset cases failed before the review correction and passed after it. - `pnpm test:e2e:runner:typecheck` passed after the retry-counter correction. - `pnpm -r typecheck` and `pnpm build` passed before the final rebase. - The full local `pnpm test:run` and harness unit gate encountered embedded PostgreSQL startup failures. The harness gate passed 4,574 tests and failed its database-startup test. The final focused journey/feedback gate and all 137 Node checks passed. CI will provide independent full-suite results before merge. - Current-head CI passed 55 checks; two checks skipped and one was neutral. Greptile is 5/5, with zero unresolved review threads. - Live Cloud verification will use the reviewed harness and Operations revisions. This PR does not claim that a fresh Cloud campaign has passed. ## Risks A broad cancellation exemption could hide real failures. The exemption requires the server's explicit receipt, unresolved runtime selection, no contradictory execution evidence, and exact durable lineage. Missing successors remain pending until the existing deadline. A successful native child and valid downloaded bytes remain required. No server behavior, approval policy, or budget policy changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e7ea2e2961 |
fix(evals): export grades for expanded qualification batches (#18010)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E campaigns retain each attempt and its independent grades. > - Braintrust receives product and cleanup grades for each retained attempt. > - A six-case qualification produces twelve scores. The export allowed only eight. > - Experiment creation succeeded, but grade insertion stopped before dispatch. > - This fix allows fourteen scores so the six- and seven-case batches retain every result. > - Operators can inspect failed and successful grades without running an agent again. ## Linked Issues or Issue Description Refs: #17044 **What happened?** A retained six-case Product E2E campaign created its Braintrust experiment, then failed with `Experiment ingestion allowance exhausted`. Its twelve deterministic scores exceeded the eight-score cap. No insert request was sent. Agent results and model accounting remained saved. **Expected behavior** The bounded qualification batches must export both grades for every selected attempt. Failed grades must remain visible. **Steps to reproduce** Create an experiment through `openExperimentIngestion`. Insert six or seven events with `product_pass` and `cleanup_pass` scores. The old limit rejects the insert before it reaches the transport. **Paperclip version or commit** Reproduced on master `cff331f6ca0a652b5e41641c68dfa5fa584c22ae`. **Deployment mode** Local Docker controller built from source. The failure is in the shared Product export path. ## What Changed - Raise the finite experiment score cap from eight to fourteen. Keep the three-request, 4 MiB and 60-second limits. - Add regressions for the six- and seven-case batches. Keep the failed product grade in each test batch. - Verify that the receipt collector accepts the old eight-score policy and the new fourteen-score policy. It rejects a fifteen-score policy. - Document the seven-attempt export ceiling. This change does not add agent calls or automatic retries. ## Verification - Before the fix: both new batch regressions failed with `Experiment ingestion allowance exhausted`. - After the fix: 67 tests pass across experiment ingestion, ingestion accounting, experiment projection and scheduled finalization. - Command: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/experiment-ingestion.test.ts tests/runner-e2e/ingestion-usage.test.ts tests/runner-e2e/scheduled-finalization.test.ts tests/runner-e2e/braintrust-experiment.test.ts --configLoader runner --cache=false`. - `git diff --check` passes. All 58 current-head checks are acceptable, including typecheck, build and tests. Both reviews are 5/5, with no unresolved threads. - Live agent attempts will not be repeated to verify this reporting fix. Recovery uses the saved results and a separate bounded ingestion receipt. ## Risks - New exports can submit at most six more deterministic scores per experiment. These are saved grades, not model judge calls. - Larger campaigns still fail closed. Existing receipt files keep their original policies and traffic counts. - This fix does not resolve the plan-lifecycle behavior failures observed in the qualification. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cff331f6ca |
fix(evals): retain local controller cost origin (#17990)
## Thinking Path > - Paperclip manages AI agents and their work. > - Its eval journals retain model costs and operating reservations under one cap. > - The operating policy records the controller's execution origin. > - The schema currently accepts only GitHub-hosted and AWS Fleet origins. > - A local controller must therefore use an inaccurate label or fail admission. > - This PR adds the local origin and keeps the hosted-only schedule gate. ## Linked Issues or Issue Description Refs #17986. No duplicate open PR or issue was found in the operating-overhead/local search. **What existing behavior does this improve?** The Product E2E operating allowance and its retained journal policy. **Current behavior** At master `40c1ab36cd`, an unscheduled local controller cannot set `runner: "local"`. The generic allowance validator rejects it. Local qualification can use the same accounting controls but cannot record its origin correctly. **Proposed behavior** Accept the local origin in the generic allowance. Keep all component, price, cap, expiry, and receipt rules. The scheduled validator still requires `github-hosted` and rejects a local allowance. **Reason and benefit** Local qualification can retain an accurate execution origin without claiming that its reservations are billed GitHub charges. ## What Changed - Add `local` to the operating allowance's runner type and validator. - Exercise local journal creation, model admission, retained overhead, restart, and reconciliation in the existing boundary test. - Check that a local allowance fails Product scheduled admission. - Pin new hosted Runner controller admission to `github-hosted`. Preserve original labels when reading historical manifests; those reads cannot authorize a new worker. - Exercise the hosted-origin rejection and historical/local reads in both source and copied portable runtimes. - Document the local label and its accounting limits. ## Verification - Before the fix: the new local persistence case and schedule-boundary assertion failed; eight tests passed. - Focused ledger, overhead, operations, and scheduled finalization tests: 71 passed across five files. - Controlled Runner source and portable tests: 86 passed. The new hosted-origin regression failed before the fix. - Full Product support suite at the first commit: 4,524 passed, three skipped, and one retained embedded PostgreSQL bootstrap failure. The same database failure was previously reproduced on a clean baseline. - Separate Node test phase on the host: 137 passed. - Actual Linux controller: database lifecycle tests pass. After installing its missing `zip` utility, the full Product suite has 4,527 passes and one Copilot parent-version coverage failure. That same failure reproduces on clean master `40c1ab36cd` with the same Linux filesystem and dependencies (14 passes, one failure). Copilot files are unchanged. The initial seven ZIP-related failures and the watcher failure remain retained. - The final fresh Linux semantic check passes: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=1 node node_modules/typescript/bin/tsc --singleThreaded -p tests/runner-e2e/tsconfig.json`. The parallel compiler OOM is retained. No semantic checks were omitted. - Both fresh reviews are 5/5, with the addressed thread resolved. Hosted Build first failed during unchanged Pi setup prefetch with an operation timeout. One job rerun recovered; the original failure remains retained. All 58 latest-head checks are now acceptable. - No paid model requests, sandbox creation, schedule changes, or historical journal edits. ## Risks - The caller supplies this origin. It must describe the actual controller. - `local` does not prove zero compute cost, provide a default allowance, or release a hold. - Both maintained hosted admission paths require the GitHub-hosted controller origin. A previously accepted hosted policy with an AWS/local label now fails new admission. Historical reads preserve the original evidence. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40c1ab36cd |
feat(evals): register explicit Daytona feedback persistence (#17986)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E evals check real server, browser, runner, and provider behavior. > - Feedback persistence already checks saved complaints and suggestions with exact bodies and run attribution. > - Its catalog registers only local cells, so a Daytona campaign cannot select the same workflow. > - This PR registers the existing workflow for the two native Daytona profiles and tests their gateway bindings. > - Remote cells stay explicit-only and unqualified until a bounded live run passes. ## Linked Issues or Issue Description Refs #17980 and #17975. No duplicate open PR or issue was found in the Daytona feedback search. **What existing behavior does this improve?** The `feedback-persistence` Product E2E suite and its explicit execution selectors. **Current behavior** At master `d54f776ca0`, the suite has only local Claude and Codex cells. Selecting the matching Daytona ID fails before admission. The gateway manifest already supports these native remote profiles. **Proposed behavior** Register four cells: local and Daytona for each native profile. Reuse the existing saved-body and completed-tool oracle. Require an immutable image and Daytona credentials. Keep one attempt, one expected work run, five-minute local and inherited 15-minute remote case timeouts, and the existing 60-minute provider creation TTL. **Reason and benefit** A bounded remote qualification can check both a simple task and real feedback persistence. Local evidence remains separate from remote acceptance. **Breaking changes** Explicit `--suite feedback-persistence` now lists four cells instead of two. `--all` still excludes the suite. Daily and weekly manifests still select their existing exact local IDs. ## What Changed - Register the two native Daytona feedback cells and increment the suite definition version. - Address the remote cold-start review: inherit the base task's 15-minute Daytona deadline and retain the five-minute local limit. The regression failed before the fix and now passes. - Check immutable-image requirements, credentials, lifetime, and shared gateway connections in existing contract tests. - Fix the config type in the TTL catalog test added by #17980. Keep its runtime lifetime assertions. - Document remote selection and its unqualified status in the existing eval guides. ## Verification - Before the catalog change, the registration regression failed: one failed and 16 passed. - Focused catalog, feedback, operations, ledger, manifest, and summary tests: 114 passed across six files. - Read-only discovery lists exactly the two Daytona feedback IDs. - The Node test phase passes separately: 137 tests. The retained Vitest database failure prevents the unit launcher from reaching that phase. - Full Product support suite: 4,523 passed, three skipped, and one embedded PostgreSQL bootstrap failure. The same database failure was reproduced on a clean baseline before this PR. - Fresh Linux Product typecheck passed at the review-fix commit: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=2 node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. The default run and the ineffective JavaScript-heap attempt exited 137 and remain retained. The compiler launches a native child process; its memory controls let the unchanged semantic check complete. Host typecheck still has 15 stale installed DB/Runner export errors; the two TTL test typing errors are fixed. All 58 latest-head checks are acceptable (success, neutral, or skipped). Both fresh reviews are 5/5, with no unresolved review threads. - The updated credential-free Linux controller discovers both planned remote IDs and launches Chromium 151.0.7922.34. Runtime build inputs are unchanged from the prior controller. The Daytona image content ID is still `b55f03dc9c633db300b70206a691ff45a8e779e56a3c546f9358e3282d3597c9`. - No model requests, sandbox allocation, or paid schedule changes. ## Risks - Selecting the whole explicit suite now includes remote cells and requires Daytona configuration. Use exact IDs for a bounded paid campaign. - Registration and contract tests do not prove live remote behavior or token accounting. Remote qualification remains pending. - The shared dependency checkout limits the local host typecheck. The fresh Linux semantic check passed. Earlier compiler OOM failures and the database bootstrap failure remain retained. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d54f776ca0 |
fix(daytona): bound ephemeral eval lifetime at creation (#17980)
## Thinking Path
> - Paperclip manages agent work through Runner and sandbox providers.
> - Product E2E runs paid work in disposable Daytona sandboxes.
> - A controller deadline cannot stop remote compute after the
controller crashes.
> - Inactivity cleanup does not establish an independent wall-clock
lifetime.
> - Daytona supports a lifetime in the creation request.
> - This change exposes that option in the Daytona driver and verifies
the provider expiry.
> - Ephemeral evals can retain a finite lifetime even when their
controller is unavailable.
## Linked Issues or Issue Description
Refs #17947 and #17978. No duplicate creation-lifetime PR was found.
**What happened?**
The ephemeral Product E2E fixture configures inactivity stop, archive,
and delete intervals. Ordinary task leases do not supply a requested
expiry. A crashed controller therefore has no confirmed wall-clock
sandbox lifetime for its cost bound.
**Expected behavior**
Each ephemeral eval sandbox must carry a provider-enforced lifetime from
creation. The driver must reject invalid bounds and reject a sandbox
whose provider expiry is missing, expired, or later than the configured
remaining lifetime.
**Steps to reproduce**
Acquire a Daytona lease from the ephemeral fixture. Inspect the creation
request and returned expiry. Before this patch, the request has no
lifetime and the lease has no expiry unless its caller supplies one.
**Paperclip version or commit**
Current source based on master at
|
||
|
DottaandPaperclip
|
4c314117d2 |
fix(evals): bind active image identity and isolate routing checks (#17978)
## Thinking Path > - Paperclip manages agent work through Runner. > - Product E2E uses a content hash to identify its Daytona image. > - The hash must include the active provider identity declaration. > - The active Copilot fixture advanced past the manually listed image inputs. > - The runner routing test also captured later YAML steps as shell commands. > - This change binds the active fixture and stops shell extraction at the next step. > - Operators can verify image identity and routing against the current source. ## Linked Issues or Issue Description Refs #17937 and #17947. The related Copilot work includes #15563. This change repairs the shared image contract; it does not add a Copilot qualification claim. **What happened?** Two image tests fail because the active Copilot identity fixture is absent from the image hash inputs. Four routing tests execute later workflow YAML as shell commands after checkout and dependency steps were added to the authorization job. **Expected behavior** The image hash must change when the active identity fixture changes. The routing tests must execute only the selected routing step. **Steps to reproduce** Run the Daytona image and paid runner routing unit files on current master. Six assertions fail before this patch. The same failures occur on the clean |
||
|
DottaandPaperclip
|
9b7d5b35eb |
fix(evals): refresh forecasts from complete request receipts (#17975)
## Thinking Path
> - Paperclip coordinates agent work through Runner.
> - Scheduled Product E2E campaigns need measured cost and time
forecasts.
> - The saved forecast samples came from October 21.
> - The October 27 qualifications retain final usage for all 19 model
requests.
> - The Claude measurements are hosted; the Codex measurements are
local.
> - This change refreshes the samples and keeps measurement hosts
separate.
> - Operators can inspect the latest evidence without treating it as
current qualification or an invoice.
## Linked Issues or Issue Description
Refs #17127 and #17946.
**What happened?**
The scheduled forecast still uses the older pilot amounts. It reports no
hosted measurement and does not record the measurement host per sample.
**What did you expect to happen?**
The forecast must retain the latest complete model receipts and their
source hashes. It must keep local and hosted timing cohorts separate. A
changed definition must remain unqualified.
**Steps to reproduce**
Prepare the daily or weekly forecast from the default branch. Inspect
its saved measurement date, totals, and hosted fields.
## What Changed
- Refresh four samples from two intact request ledgers and four retained
Product results.
- Record measurement hosts, final request counts, and original campaign
outcomes.
- Add historical hosted totals. Current hosted totals still require
matching evidence for every selected cell.
- Keep hosts in separate per-cell cohorts. Preserve definition drift and
unknown slow-tail estimates.
- Document the model-cost scope and timing limits.
## Verification
- Forecast and scheduled-preflight tests: 33 passed.
- The extraction replays both hash-checked ledgers, requires final
receipts, and checks the retained pricecards. The four samples cover 19
final requests.
- Daily historical model subtotal: $0.428650100; summed attempt time:
107.286 seconds.
- Weekly historical model subtotal: $0.793676500; summed attempt time:
158.093 seconds. The Codex samples are local.
- Full Product support suite at the initial head: 4,510 passed, seven
failed, three skipped. A clean
|
||
|
Devin FoleyandPaperclip
|
c9213d1be4 |
fix: send saved queued messages after Stop (#17959)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People can save follow-up messages while an agent runs. > - Stop preserves those messages and stops the active execution. > - The queue removed its delivery controls when that execution ended. > - Native admission also refused an explicit resend of older queued input. > - This change adds Send for a stopped queue and records fresh board intent. > - The saved message can continue through normal cleanup and admission checks. ## Linked Issues or Issue Description **What happened?** A person queued a message with a screenshot while an agent was starting. Steering was not yet available. They stopped the run. The message remained visible, but no control could send it. Native queue delivery also rejected this transition. **Expected behavior** A stopped queue offers Send. The click preserves message IDs, content, attachments and original authors. It starts at most one eligible continuation. Stop remains effective until the person gives new direction. **Steps to reproduce** 1. Start a native task and queue a second message. 2. Stop the active run before it consumes that message. 3. Open the task after cleanup. Observe the saved message with no send control. Refs #16965 for the queue-control change and #15015 for stopped conversation recovery. No open repair for this exact transition was found. ## What Changed - Show Send for a saved deferred queue with no active run. - Accept the existing board queue action for a stopped native target. Keep live native interruption restricted. - Use the server-recorded click time to authorize older native messages. Reject stale, malformed and wrong-actor intents. - Reuse normal queue dispatch, revision checks, cleanup proofs and execution admission, including a failed native predecessor after a verified Send. - Ignore a fully committed, successful recovered export when admitting a fresh message, while retaining the wait for results still committing. - Honor an applied `/new` conversation boundary while retaining native execution holds until the existing stop and cleanup checks verify them. Workspace holds also survive reset. Check the newest failed or cancelled native execution even when older recovery actions exist. - Keep native automatic delivery separate from the explicit Send action. Update execution and access documentation. ## Verification - Meaningful regressions failed before the fixes: stopped queues lacked Send, native admission refused fresh intent, cleanup and dispatch receipts were missing, deferred rows disappeared, live plan Interrupt rejected its own receipt, and Agent Chat `/new` inherited an old non-workspace execution hold. - Six assembled-wakeup cases fail before the reset cleanup fix. All 11 reset/history cases pass after it: live processes, missing identity, quarantined cleanup, remote leases and live controllers remain held; verified cleanup proceeds and retires the hold. Two more cases fail before the fallback fix and pass after it: Send verifies a stopped or live native predecessor despite a later reset receipt and no recovery record. - **525 server tests pass** across seven queue admission, route, dispatch, envelope, provenance, blocker and conversation files. **343 queue/thread UI tests** also pass. Server/UI TypeScript, token and map guards pass. The final scoped server typecheck also builds native Runner successfully. - The latest regressions prove that completed recovered exports permit fresh input, and clearing conversation history still requires old native stop and cleanup verification; an unsafe workspace still blocks after `/new`; failed native Send reaches actual continuation dispatch with the original queued comment. A result still being committed remains deferred. History plus a newer Stop cannot hide a missing process/controller/lease receipt. - The real Chromium Stop/Send journey passes on current head `75fa5e7222c59f43ff7ac326e81093bc9da28330`: queue a follow-up, Stop, reload, explicitly Send, then verify exactly one useful response and no live run. Earlier browser journeys cover legacy Interrupt, interruption during a write and Stop followed by a new direction. These use an isolated app/database and controlled ACP fixture, not an external AI account or a staging deployment. - Negative cases retain actor, company, issue, assignee, queue and cancellation-receipt bindings; cleanup/process/workspace/controller holds; approvals; pause and budget gates. Typed responses keep their original resolver and immutable result. - Full local recursive typecheck stopped in unchanged Rust Runner compilation with ENOSPC, so the chained full tests/build did not run. All hosted CI gates pass on final commit `75fa5e7222c59f43ff7ac326e81093bc9da28330`, including typecheck, tests, build and E2E. Apex review on that commit is 5/5, with no unresolved findings. Earlier unrelated CI failures involved PostgreSQL fixture-cleanup deadlock and an unchanged decision-expiry deadline assertion; the latter's 20 cases pass locally without source changes. No assertions or limits were weakened. Local resource cleanup was limited to this task's verified completed test processes and rebuildable npm caches. ## Risks - Send can authorize a new turn for a saved message after Stop. A Send click must be newer than the stopped execution; a live plan Interrupt is bound to the cancellation receipt produced by that same click. It does not imply that cleanup finished or that an external action is safe to replay. - The existing endpoint and intent format are reused. No schema or credential change is required. Live native steering and legacy interruption keep their existing paths. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose the exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6cc765bd81 |
feat(live): deliver recoverable transcripts to interested views (#17456)
## Thinking Path > - Paperclip should show live agent output without repeating presentation work for every view. > - Execution, acknowledgements and durable checkpoints must remain independent of browser interest. > - The server already owns authorization and redaction at delivery. > - A shared browser connection can declare which visible runs need full transcripts. > - Catch-up must handle subscription races, disconnects, terminal output and slow consumers. > - This PR delivers the backend and UI contract together without a database migration. ## Linked Issues or Issue Description **What existing behavior does this improve?** Visible transcript consumers receive recoverable output through one shared browser connection. **Current behavior** Independent views can create duplicate connections, presentation work and recurring history reads. A subscription race or filtered native history can lose the saved tail. **Proposed behavior** Negotiate run interests, subscribe before catch-up and retain independent canonical recovery cursors. Keep lightweight lifecycle and attention events available without full transcript delivery. PR 4 of #17097 → #17439 → #17455 → #17456. The runtime feature is independent of hot-upgrade recovery; the stack order is for review convenience. ## What Changed - Add additive WebSocket capability negotiation, replacement interests, revision acknowledgement and resynchronization notices. - Reference-count visible consumer interests on one company-scoped browser connection. Preserve older-client/server behavior. - Reuse canonical structural/current-user/registered-secret redaction for plugin run notifications through the same bounded delivery queue. Keep producer and uninterested streams free of recipient lookups. - Refresh application queries after the matching recovery acknowledgement, including one follow-up after an older in-flight read. Ordinary interest acknowledgements add no refresh. - Keep authorization and redaction at delivery. Browser interest cannot change harness execution, PRP ACKs, checkpoints, accounting or canonical logs. - Deduplicate by sequence and log identity. Catch up terminal output and saved tails without recurring reads during healthy negotiated streaming. - Bound delivery to 1,000 items and 1 MiB. Overflow requires resynchronization; persistent congestion closes the connection. - Preserve compact-history expansion from current master across paginated catch-up. Snapshot the original read window before React can defer its update. ## Verification On current source `f6710f0b3c`, Linux [job 122604880503](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880503) executes all five run-read permission cases successfully (2.574 seconds), including grant and revocation on the same connection. Its shard passes all 2,430 cases. Current [server-shard job 122604880416](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880416) has 1,513 passes, one failure and three skips: an unchanged nested-clone expiration/restore test exceeds its 15-second deadline. The preceding `5efc6c48bf` source has identical non-document bytes; [its corresponding job 122601359272](https://github.com/paperclipai/paperclip/actions/runs/40623017917/job/122601359272) passes all 21 nested-clone cases, including that case in 9.505 seconds. The exact timing cause is unproven. The single diagnostic [retry job 122609332597](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122609332597) passes all 21 nested-clone cases (61.316 seconds), including the formerly timed-out case in 7.003 seconds, and all 1,514 shard cases (three skips). The failure remains recorded; no timeout or assertion was changed. After the server squash merge and qualification retarget, all 50 focused server cases and all 91 focused UI cases pass again, with both typechecks and token gates. Runner vendor preparation rebuilds current master inputs before those tests. This source needs fresh CI and reviews; prior CI and browser receipts do not qualify it by inference. Current candidate `f6710f0b3c` now targets master `6987c2f20b`, after merged #17097, #17439 and #17455. Its diff contains only transcript delivery and its documentation. It adds no database migration. Retargeting changes only the evidence document; all production and test code is byte-identical to `5efc6c48bf`. On the eighth integration, all 50 focused server delivery cases and all 91 focused UI cases pass, with both typechecks and token gates. The following 35-case receipt belongs to the earlier delivery correction. Four low-queue authorization/decoration stall variants and the late-rejection case fail before correction. They now verify the five-second read deadline, old and negotiated clients, revisions that cannot reset the deadline, unaffected sibling connections, rejected late results, and timer cleanup. The first combined toast run has four loopback EPERM failures and 31 passes; its permitted repeat passes all 35. Earlier 28-case evidence excluded the toast file. The permission correction requires the current HTTP company-scope transcript permission, plus task access, for both native events and legacy logs. Prior Linux source `49a8783e9b`, job 122549913643, executes all five run-read cases, including grant/revocation on the same connection, successfully. That source placed this file in server shard 11; later shard assignments can differ and must be checked against their actual inventories. The older `405bedd189` receipt retains a separate scheduled-retry teardown race and inherited fixture collision; neither is waived. The new head must pass again in Linux CI. The seven extracted stream-recovery tests preserve their assertions. All 39 related legacy/native cases, UI typecheck and token gates pass after restoring the initially missed fixture helper. Earlier full workspace build/typecheck and Chrome journeys at `445a76563c` cover two views, company switches, reconnect, terminal catch-up, reload and zero recurring healthy transcript reads. Those browser receipts are historical source evidence and do not qualify this new head by inference. The [qualification record](https://github.com/paperclipai/paperclip/blob/f6710f0b3c84c6fdb2021df67206261b446dfc97/doc/plans/2026-10-22-hot-upgrade-qualification.md) retains exact earlier UI/history, source-boundary and cleanup failures with their fixes and limits. The workload-cost test preserves bounds and authorization callback counts for zero interests, one viewer, multiple browsers and a slow viewer. Its socket congestion is simulated. It measures neither SQL nor network costs. Current permission checks and the final hosted active-agent/viewer matrix remain unmeasured; no production savings are claimed. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and ordinary merge eligibility remain required. The earlier recipient corrections pass all 47 registry/plugin/delivery/WebSocket/toast cases and all 174 UI provider/recovery/interest cases, both typechecks and token gates on source `b202e59a25`. A real bus/host-service test registers a synthetic run secret through the production registry and reproduces its raw notification before the fix; native/legacy/final notifications now redact it, disposal prevents late worker notification. A later P1 review reproduces lost final notifications on settings-read and secret-resolution failure. Both cases fail before correction. Any delivery failure or subscription timeout now sends one content-free stream error and removes the plugin subscription. It never changes the agent execution or run status. Later events start no new redaction reads. The final integrated six-file server lane passes all 50 cases, with server typecheck and token gates. Existing browser delivery behavior is unchanged. Storage and secret-provider seams are scripted. Two real React Query/provider regressions first miss a run during resync; both now refresh after the valid replacement ACK, including pending-read and stale-scope cases. The prerequisite qualification batch passes all 118 cases and E2E typecheck after master integration. Fresh CI, security scan and reviews remain pending on this head. The contribution-rule review correction removes 104 explanatory comments introduced by this slice and keeps the ordering, deadline and plugin-delivery explanations in the delivery document. Existing comments and functional test/lint directives remain. All edited source files have identical parsed non-comment tokens before and after the change. All 50 server delivery cases and all 91 cases in six affected UI suites pass, along with token gates. The new head requires fresh CI, security and review. ## Risks A slow or disconnected viewer must resynchronize from authorized canonical history. It may see a display gap until catch-up completes. Company changes and revoked access must remove delivery interests. This slice adds no database migration and cannot change execution or recovery authority. Full transcript delivery evaluates current company-scope permission per delivery. Active plugin recipients also perform canonical redaction reads; viewer-load database costs on this final source remain unmeasured. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6987c2f20b |
test(upgrades): qualify controller replacement with immutable artifacts and phase evidence (#17455)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/105/changes/paperclip/105 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip needs evidence that an admitted execution survives a main-server replacement. > - Runner mechanics and server adoption belong to #17097 and #17439. > - A healthy HTTP endpoint does not prove recovery of the original execution. > - Qualification must identify the serving artifact, natural ownership, authenticated attachment and resumed processing separately. > - It must also retain failed runs and measure normal-operation costs. > - This PR extends the existing qualification machinery and acceptance record. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operators can assess upgrade continuity from phase evidence tied to exact artifacts and profiles. **Current behavior** A same-artifact restart or synthetic executor pass cannot establish different-artifact upgrade and rollback support. Idle SQL gates also do not measure active-run capacity. **Proposed behavior** Use immutable A/B artifacts and strict phase receipts through the existing supervisor. Preserve failed evidence and add measured before/after workload costs. Builds on merged #17097 and #17439. The base is now `master`. #17456 follows for review convenience. This PR lands qualification machinery and the evidence record. The remaining hosted gates stay open for deployment qualification; merging this machinery does not claim those experiments passed. ## What Changed - Extend the main-server supervisor and installed-artifact verification for actual serving revisions, forward migrations and compatible rollback. - Distinguish HTTP readiness, natural ownership, authenticated attachment, resumed processing and backlog settlement. - Retain the original reconnect grace and execution identity checks. Never substitute a new task or edited lease for continuity. - Reuse real-harness and Daytona qualification paths, with exact profile and artifact provenance. - Bind current Copilot attestation inputs in the Daytona image fingerprint. - Keep one acceptance record, including both failed file-only stress phases and the final passing bounded-pool comparison. ## Verification After retargeting to master, all 158 focused qualification cases across 13 files pass, with E2E typecheck. The first 12-file batch has four report-lock test failures and 133 passes. Master #17811 had committed the exact already-approved output lock, while that fixture assumed HEAD must be the uncorrected input. The fixture now checks both pinned forms, executes the committed-lock path, and reconstructs the pinned input to retain all rejection tests. The production helper, policy hashes, patch and dependency resolution are unchanged. The corrected batch explicitly includes both process-preservation suites; the prior 146-case receipt has its own source and selection. Current candidate `8f1d6ec02f` integrates master `09ad9adce2`, including the merged server slice. Its diff contains only qualification code and documentation. The [qualification record](https://github.com/paperclipai/paperclip/blob/8f1d6ec02f5bf6c90e7b27da8156953cabd9a180/doc/plans/2026-10-22-hot-upgrade-qualification.md) is the acceptance ledger and retains source-attributed failures and hosted limits. On the eighth integration, all 146 cases across 12 focused qualification files pass, including report-lock preparation, workflow security and workflow targeting. E2E typecheck passes. The earlier 118-case result remains attributed to its prior source. The first sandbox invocation has 117 passes and one failure because process inspection was denied during inert fixture cleanup. The permitted repeat passes all 118 and a targeted process check finds no remaining fixture daemon. The earlier 96-case receipt remains source-attributed in the ledger. The latest two review regressions first fail: an intermediate replacement can cite another runner/session despite a valid final step, and malformed receipt JSON aborts evidence packaging. Each replacement now binds run/company/issue/runner/session, and corrupt receipts retain explicit invalid metadata plus sanitized diagnostics while other logs and receipts still package. These checks prove qualification machinery with injected supervisor operations, configuration subprocesses and real filesystem packaging. They do not qualify a hosted harness. Three request-reader cases, 14 artifact/configuration/cache/history cases, and three lost-partial-receipt cases fail before their respective corrections. The final collector preserves sanitized startup, readiness and recovery phase receipts through packaging and temporary-root deletion. Earlier embedded-Postgres initialization, report-catalog drift, CI scheduling/cleanup failures and fixture collisions remain recorded with their corrections or limits. The previous Greptile service error is retained as historical review evidence, not a current review pass. The record includes prior source `9bb9d3417e`'s actual Linux contention/fairness successes and the ordinary-warm regression proof from the server slice. The historical 300-session component campaign passes four after phases and retains earlier burst failures. Those component costs do not establish hosted capacity. The hosted API-only pair has zero agents/viewers, fails both absolute latency gates and the candidate memory gate, and does not complete the active-agent/viewer matrix. The strict ten-agent same-artifact restart remains attributed to its original artifact. It does not qualify this updated stack. Actual A→B/B→A, the normal drain-free Cloud rollout, and other hosted harnesses remain open. No new provider turn, deployment or hosted qualification was run during merge preparation. Prior source `57d3767eac` jobs 122590643944, 122590643983 and 122590644161 ended after runner shutdown/cancellation without final Vitest results. The shutdown cause is unproven. No retry was issued before this source was superseded by the required server recovery fix. These attempts remain failures/incomplete evidence. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and normal merge eligibility remain required. ## Risks Different-artifact A→B/B→A, the normal drain-free Cloud rollout path, other hosted harnesses and the complete concurrency/viewer matrix remain open. Frozen branch builds are development-build evidence. This PR owns qualification and evidence; production defects must be fixed in their earlier owning slice. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e4b30163bb |
fix(evals): preserve the complete trusted controller artifact inventory (#17943)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/103/changes/paperclip/103 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages real agent work. > - Controlled Runner evaluations record model spend and traces. > - A trusted build creates one portable controller with a complete file inventory. > - GitHub artifact upload omitted one hidden dependency metadata file. > - The next job correctly refused the incomplete inventory before any paid worker started. > - This change preserves all inventoried bytes without relaxing verification. ## Linked Issues or Issue Description Refs #17889, #17929. **What happened?** [Run 40585138437](https://github.com/paperclipai/paperclip/actions/runs/40585138437) failed at copied controller verification. Its artifact contains 294 files; its signed inventory expects 295. The only missing file is `server/node_modules/protobufjs/node_modules/@protobufjs/pool/.npmignore`. No extra files exist. Every paid worker was skipped and no model request ran. **Expected behavior** The trusted controller artifact preserves its full inventory, including hidden package metadata. The strict verifier rejects any actual change or missing file. **Steps to reproduce** Build the portable runtime, upload the directory with the action's default hidden-file exclusion, then verify the downloaded artifact against the original receipt. ## What Changed - Enable hidden files only on the fresh trusted controller code upload. - Download that exact upload and run a real artifact-service round-trip test before any private preparation. The test fails if the upload drops hidden files again; it tests the actual transport instead of matching a YAML flag. - Keep the strict byte inventory and trusted source/receipt checks. - Document why this code artifact differs from private attempt evidence, which still uses the protected archive. ## Verification - Exact retained artifact comparison: one missing `.npmignore`, zero extra files. - `actionlint .github/workflows/runner-protocol-live-evals.yml` passed. - Protocol workflow security tests: 14 passed. - The artifact-service integration test is explicit-only locally. The controlled workflow supplies the real downloaded artifact, trusted source and receipt digest. - No product runtime, provider routing or grading code changed. CI runs broader checks. Fresh controlled workflow qualification follows merge. ## Risks Hidden files are allowed only in the code-only output of the provider-credential-free build. Raw private evidence and credentials do not enter that upload. The artifact verifier still requires every file and SHA-256 from the trusted receipt. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a8bd012635 |
fix(evals): apply reviewed lock corrections to reports and qualification images (#17937)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/98/changes/paperclip/98 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages agents and records their work. > - Product E2E campaigns measure real agent behavior and model spend. > - A paid hosted run completed both cases and all ten model receipts. > - Its report job could not install dependencies from the committed lock. > - The trusted checkout needs a reviewed dependency correction. > - This change applies a reviewed lock correction before the frozen report install. ## Linked Issues or Issue Description Refs: #17928 **Description** The report in [run 40576181788](https://github.com/paperclipai/paperclip/actions/runs/40576181788) failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Both paid cases had passed. The worker and build jobs already restored the resolved source lock, but the report did not. **Steps to reproduce** Dispatch the bounded daily campaign on the default branch with a stale committed lock configuration. Workers use the resolved lock. The report uses the stale lock and fails before report generation. **Expected behavior** The report installs frozen dependencies from a verified lock for the exact trusted source. Evidence can be reported without repeating paid work. ## What Changed - Apply a reviewed lock correction from the trusted report checkout before its frozen install. Use the same correction for the qualification image build, which also requires a compatible frozen lock. - Pin the original lock, correction and output SHA-256. The trusted authorization job rejects unknown inputs before any paid work. - Apply the same correction to trusted manual history publication. Qualification image builds apply it only when their target equals the workflow revision; other targets keep their own lock. - Update existing Claude ACP and postgres patch identities and link the declared yaml dependency. Existing package tarball integrity entries stay unchanged; the declared yaml dependency adds exactly yaml@2.9.1 with pinned integrity. - Keep runtime registry resolution and worker target locks out of the credential-bearing report. - Test valid correction, idempotence and changed-lock, changed-patch and wrong-output rejection. ## Verification - Focused workflow and lock-correction tests: 36 passed. - `actionlint .github/workflows/runner-full-stack-e2e.yml` passed. - Original worker evidence: 2/2 passed, ten final model receipts, zero unresolved model requests. - The original report was regenerated from retained evidence with zero provider calls and saved privately with replay provenance. - Broader repository checks run in CI. Prior full Product checks passed for #17928. ## Risks The correction is tied to the reviewed committed lock. Dependency input changes need a reviewed correction update. No new registry version or lifecycle script can enter the privileged report through this fix. The original failed workflow stays failed. No root lockfile is committed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5787962454 |
fix(evals): report setup-inclusive model token totals (#17946)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/108/changes/paperclip/108 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip Product E2E evaluations run real agent requests. > - Model accounting includes setup, canceled requests and earlier attempts. > - Heartbeat token totals can omit setup requests. > - The shared gateway journal retains final normalized usage for those requests. > - Reports should expose that usage once, with coverage and conflicts visible. ## Linked Issues or Issue Description Refs #17928. **What happened?** The two-case qualification reports showed complete model dollars from the shared ledger, while their headline heartbeat tokens omitted setup. For example, the verified Codex campaign has nine final requests, including setup requests, whose aggregate is only available in private receipts. **Expected behavior** The existing ledger report shows all-request token totals, a setup subtotal and missing/conflicting usage counts. Heartbeat and request totals remain separate. **Steps to reproduce** Run a budgeted native marker case with an asynchronous setup probe, then compare the heartbeat totals with final request observations in its retained ledger. ## What Changed - Aggregate final provider/gateway usage once per model request, including setup and canceled requests. Ignore provisional observations and infrastructure reservations. - Expose uncached input, cache-read input, cache-write input and output separately. Label reasoning as included in output. Preserve agreeing optional reasoning from partial final receipts only when it fits the complete output count. - Check every final provider/gateway observation for conflicting supplied fields, including partial receipts. Exclude ambiguous requests and report their conflicting coverage. Missing token usage remains visible. - Preserve retained reports without the optional token field; validate new counts and totals before dashboard regeneration. - Keep the journal-bound experiment identity stable when adding derived token presentation, so old accepted experiments can still be linked without a reimport. Journal changes still invalidate the receipt. ## Verification - Focused report and scheduled-operations tests: 54 passed. - Eval TypeScript check passed after refreshing ignored Runner declaration outputs. - Regression covers duplicate sources, provisional usage, setup, canceled requests, missing usage, conflicting final usage, infrastructure exclusion and invalid retained totals, including nonzero totals with zero contributing requests. All 32 partial-field subsets are checked in both observation orders. - Regenerated both real retained Claude and Codex reports successfully with all final request tokens and existing experiment receipts, using zero model calls. - Broader checks run in CI. No model calls are needed for report generation. ## Risks The subtotal requires final normalized gateway/provider token observations. Older journals and other telemetry sources can have incomplete coverage, which remains explicit. Token totals do not establish invoice reconciliation. The change does not alter paid selection, routing or budgets. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
64a42d2cf0 |
fix(evals): honor the Daytona setup attempt deadline (#17947)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/110/changes/paperclip/110 <!-- entire-trail-link-end --> ## Thinking Path > - Budgeted Product E2E cases must attribute asynchronous hello probes to setup. > - Daytona can take longer than local startup to provision an isolated environment. > - The setup helper imposed a 45-second local limit on both environments. > - A qualified remote marker stopped while the agent was still verifying. > - Remote readiness should use its existing finite attempt deadline. ## Linked Issues or Issue Description Refs #17936. **What happened?** The bounded Daytona marker campaign stopped at the setup helper after 45 seconds while the agent was still verifying. The case permits a 15-minute attempt and the environment allows up to five minutes to provision. It recorded zero model requests; retained cleanup passed and a provider-side company-label query confirmed no sandbox remained. **Expected behavior** Daytona readiness can finish within the existing attempt budget. Local setup keeps the current short limit; observed errors, holds and identity changes still stop either environment immediately. **Steps to reproduce** Run a budgeted native marker against a cold Daytona environment whose readiness takes longer than 45 seconds. ## What Changed - Pass the execution environment to the budgeted setup helper. - Use the existing finite attempt deadline for Daytona; retain the 45-second local limit. - Set a five-minute saved-account hello-probe allowance only for explicit budgeted Daytona fixtures, matching the environment provisioning allowance. A later qualification exposed the inner probe’s separate 90-second timeout. - Keep the gateway in the setup phase until the same agent is ready. Model limits, attempt limits, cleanup and automatic retries are unchanged. - Update the operations guide with the two timing windows. ## Verification - Focused setup and operations regression suites: 48 passed. - Eval TypeScript check passed. - A fixture regression verifies the five-minute probe allowance is remote-only and preserves gateway credential routing. The timing regression advances the observed clock past 45 seconds: Daytona reaches ready while local setup still stops. Existing tests retain identity/error/hold and attempt-deadline guards. - Fresh real Daytona qualification follows this correction with one attempt and the existing $20 stage cap. ## Risks A failed remote setup can now consume more of its existing attempt window. It cannot extend that deadline, increase model allowance, or automatically retry. This change does not claim the prior failed campaign passed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d40f9ac377 |
fix: enforce first-task completion decisions and retain E2E evidence (#17940)
## Thinking Path > - Paperclip lets people delegate work to agents and approve proposed work. > - The onboarding first task asks for scope, proposes a task, and waits for a human decision. > - An answer to its opening question supplies scope. It does not approve a proposal. > - Task creation already checks the saved proposal decision. > - Native completion used the general completion policy and could mark the onboarding parent Done without that decision. > - This PR checks the trusted decision before completion feedback and again when the native status is committed. > - The first task stays in review until a saved human decision permits closure. Ordinary completion and acknowledged refusal keep their current behavior. > - The Product E2E guard also retains closed scalar evidence so future failures show which boundary condition was observed. ## Linked Issues or Issue Description Refs: #17914. Profile compatibility follow-up to #17922. **What happened?** A native onboarding run can submit a valid low-risk Done result after only the opening answer. The native finalizer then marks the first task Done even though no human has accepted or rejected a proposal. **Expected behavior** The opening answer must not permit completion. A trusted saved proposal decision must permit it. A saved rejection permits acknowledgment and closure, but does not authorize new work. A later pending proposal must keep the task in review. **Steps to reproduce** Create a server-owned onboarding first task with an answered opening question, no proposal decision, and a valid low-risk completion contract. Pass a Done result through native completion feedback, the control-plane port, and the finalizer. Before this fix, the persisted status is Done. The regression expects In review. Ordinary completion and acknowledged rejection pass as controls. ## What Changed - Share the trusted latest-proposal and human-response checks with completion admission. Keep task creation acceptance-only. - Refuse premature native completion feedback. Keep an already saved premature result in review during finalization. - Check the decision again under the existing task and run locks before committing Done. - Cover accepted decisions, saved refusals, forged context, pending or changed proposals, and a real database race. - Retain a closed scalar receipt when the existing E2E pre-acceptance guard fails. Validate and display it without task content, IDs, filenames, or raw rows. Preserve the original failure, blocked dependents, and historical reports. - Include the evidence module in the journey definition hash. Bind the receipt to its exact failure code and check its closed keys against the full type. - Repair base-branch profile drift found by the full harness check: include the active Copilot v66 fixture in image hashing and update stale Pi version/hash expectations from #17922. Keep all older image inputs and strict hash checks. - Update the onboarding feature and agent access maps and E2E evidence documentation. ## Verification - Semantic regression before implementation: one failure and two passing controls on PostgreSQL 18. The failed assertion observed Done instead of In review. - The local database uses an owned disposable container with an immutable image digest. Actual migrations, feedback, control-plane port, finalizer, and status commit run against PostgreSQL. A temporary test adapter replaces only the embedded database transport because this host has exhausted its PostgreSQL semaphores. - Native control-plane suite: 55 tests passed, including 17 new controls. Existing task-creation admission: 25 passed. Existing completion feedback: 19 passed. - Credential-free E2E support: 206 Vitest suites passed, 4,407 tests passed, and two skipped. All 137 Node support checks passed. Harness typecheck passed. - The failure-receipt regression had 11 failed assertions before capture was implemented. Two failure-code binding assertions fail without their gate. Eight placement/binding cases first validate the receipt-free control, then reject the misplaced receipt. - The untouched base reproduced the three profile compatibility failures. The narrow repair passed all 83 affected tests before the full harness run. - Private workflow preparation and credential-free onboarding preview: nine compatibility checks passed. Ownership, recovery, evidence, shared-contract and workflow compatibility: 77 passed. The actual provider preview also confirms immediate success deletion, earlier declared failure eligibility, limits, and zero retries without credential access. Map checks and git diff checks passed. - Required repository typecheck passed for the product fix. The combined branch build passed. The local full repository suite was stopped after repeated PostgreSQL startup failures; a separate initdb probe confirmed host SysV semaphore exhaustion. No shared IPC resources were changed. [Hosted CI](https://github.com/paperclipai/paperclip/actions/runs/40591020873) passed the full database suites, typecheck, build, runner checks, serialized suites, and browser shards on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b` (55 success, two expected skips, one advisory neutral). - Independent source review approved the product guard, atomic race regression, evidence contract, and maps with no actionable findings. Both automated reviews are 5/5 on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b`; prior findings are fixed and resolved. Final CI is green and the PR is conflict-free. - No Cloud workspace, provider call, release publication, or deployment is part of this verification. These tests do not identify a historical Cloud failure or qualify live onboarding. - Local dependency installation uses the existing CI-generated lockfile from #17811. No lockfile change is included. ## Risks - Existing onboarding runs without a trusted saved proposal decision remain in review even when the agent reports success. This is the intended approval boundary. - Decision changes can race completion. The status commit checks the decision under its existing locks and uses the existing status-race recovery path. - A saved refusal permits closure only. Creation admission still rejects it. - Evidence readers reject malformed receipts and unrelated failure codes. The version-1 key registry is checked against every receipt field at compile time. Existing attempts do not gain evidence after the fact. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), ultra reasoning, through Codex. Used repository inspection, code execution, and collaborating coding agents. The runtime did not report a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2c70d04bf6 |
feat(evals): account for explicit Daytona model requests (#17936)
## Thinking Path > - Paperclip evals need complete model-spend evidence. > - The bounded Product E2E controller already owns request admission and token receipts. > - Its loopback endpoints cannot serve a Daytona agent. > - Explicit remote cases therefore need a scoped transport to the same controller. > - This PR exposes only authenticated model and OTLP endpoints through one owned quick tunnel. > - Remote requests keep the same budget, request identity, usage journal and cancellation drain. ## Linked Issues or Issue Description Refs #17928, #17929, #17523. **What happened?** The operations manifest and fixture reject Daytona. The parent gateway and native trace relay bind loopback, so normal remote agent traffic cannot use their cost accounting. **Expected behavior** An explicit native marker or feedback case on Daytona can route requests through the bounded parent. Upstream provider and Braintrust keys stay on that controller. Every paid request keeps its usage receipt and the attempt retains ordinary feedback and cleanup evidence. **Steps to reproduce** Use the explicit `remoteGateway: "cloudflare"` manifest option for a native Codex or Claude Daytona marker/feedback case. Without that option, the remote selection remains rejected. The parent verifies its public health nonce before fixture setup. ## What Changed - Add a deadline-limited bridge with exact model/OTLP paths, separate scoped tokens, streamed responses and owned tunnel cleanup. - Start one HTTPS quick tunnel with a private home and no provider credentials or ambient Cloudflare configuration. Do not retry creation or paid work. - Admit only explicit native Daytona marker/feedback cells and require the provisioning credential before opening the journal. - Bind fixture model/OTLP endpoints to the assigned remote origin. Keep provisioning access in the fixture/server controller. - Preserve final token usage when a remote client disconnects during shutdown. - Document the manual route. Daily and weekly selections remain local. ## Verification - Full Product E2E unit suite: 4,370 passed and 2 skipped across 206 files, plus 137 Node checks. - Focused remote/usage/operations/trace checks: 71 tests passed; final bridge/drain checks: 15 passed. - Product E2E typecheck passed. - Tests cover missing tunnel executable, expiry, authentication, blocked paths, token separation, missing provisioning credentials, endpoint assignment and listener cleanup. - The cancellation regression uses the real gateway and ledger with a synthetic upstream. It retains the final receipt once after remote disconnect. - Latest source and architecture/cleanup checks: 167 passed; eval typecheck passed. Both reviews are 5/5 and exact-head CI is green. Live qualification follows merge. ## Risks The explicit route sends synthetic eval requests and scoped attempt tokens through Cloudflare. Those tokens expire with the existing campaign deadline and budget. Upstream keys stay on the controller. Public readiness has a finite wait; failure stops before fixture setup. The operator still needs the existing immutable Daytona image and verified native artifacts. Source and deterministic tests do not establish live remote acceptance. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bf3ef12f3a |
fix(evals): qualify model spend separately from operating costs (#17928)
## Thinking Path > - Paperclip manages agent work and its budgets. > - Product E2E campaigns must explain each model request and attempt. > - The current completion gate also requires complete operating-cost evidence. > - Long-lived observability and storage charges can block a campaign after its model receipts are complete. > - An operator can now choose model-request readiness in a retained allowance. > - The report keeps full accounting, operating holds and model readiness separate. ## Linked Issues or Issue Description Refs #17889, #17523, #17452. **What happened?** A hosted campaign could pass both product cases and retain complete model receipts, yet fail readiness because operating charges were unresolved. **Expected behavior** An explicit model-readiness policy permits complete model receipts to qualify a run. Unknown operating charges stay visible and reserved. Missing model receipts and budget violations still block qualification. **Steps to reproduce** Use a campaign with four unresolved operating reservations and one completed model request. The regression checks readiness before and after the final model receipt, then checks an operating overrun. The default policy retains the original full-accounting gate. ## What Changed - Add the explicit `accountingReadiness: "model_requests"` allowance policy. Bind it into operating-bound configuration when present. - Derive a model subtotal and receipt-coverage projection from the retained journal. Show it beside full accounting and retained holds. - Use the selected readiness policy in scheduled finalization. Keep product, cleanup, feedback, trace and budget checks. - Refresh two existing eval prerequisites found by the full unit suite: the active Copilot image identity and the model-account catalog mapping. - Document the policy and its limit: it does not certify a full-lifecycle operating-cost ceiling. ## Verification - Regression failed on the old implementation with `Invalid operating overhead fields` and passed after the change. - Focused accounting/operations/finalization checks: 54 tests passed before the final policy assertions. - Existing image/catalog failures reproduced, then their 12 checks passed after the metadata repairs. - Product E2E unit suite: 4,360 passed and 2 skipped, plus 137 Node checks. Product E2E and workspace typechecks and workspace build passed. - Review regressions: 35 focused accounting/finalization checks passed on |
||
|
DottaandPaperclip
|
8548173fc9 |
feat(evals): connect hosted Runner costs and traces (#17889)
## Thinking Path > - Paperclip manages AI agents and measures their behavior with Runner evaluations. > - The cost controller, native tracing, private receipts and portable runtime are on master. > - The maintained live workflow still launches the provider CLI directly. > - Independent workers need one shared selection and disjoint budget allocations. > - Reports must keep unresolved allocations when a worker produces no evidence. > - This PR connects the maintained workflow to the existing controller and private archive. > - Operators can inspect known costs, conservative holds and trace evidence before qualifying paid recurrence. ## Linked Issues or Issue Description Refs #17452, #17469, #17493, #17506, #17523. **Current behavior** The live Runner workflow does not use the merged cost/tracing controller. Attempt-only aggregation cannot reconstruct a campaign when every worker archive is absent. Ordinary Actions artifacts in this public repository are readable by authenticated public GitHub users. **Proposed behavior** Controlled runs prepare one manifest, bind every selected request, enforce disjoint allocations and retain evidence in private storage. The report starts from the full retained manifest. Missing workers keep their allocations. The workflow rejects retries, unsupported providers, changed requests and absent or stale operating bounds before provider work. **Reason and benefit** The maintained workflow needs auditable costs and traces, including failures and missing receipts, before paid recurrence can be qualified. ## What Changed - Add controlled mode as the manual default and the only scheduled mode. Legacy mode remains explicit for other manual provider qualifications. - Prepare the full request selection and one shared manifest. Bind current run, first attempt, code, definitions, executable hashes and exact profile allocations. - Verify the copied controller and manifest before provider credentials. Require the controlled target to equal the trusted workflow revision before private archive access. - Reuse private archive retention for the controller policy, cell attempts and canonical report. AWS credentials remain transfer-step outputs. Controlled raw evidence is never uploaded as an Actions artifact. - Replay a retained manifest with partial or empty worker evidence. Count operating holds once and preserve known partial charges. - Pin the controller to an explicitly reviewed compatible lockfile digest. Reject resolver or byte drift before a frozen, script-free install into a job-local store. Record the lock digest in the runtime receipt and verify it again in private preparation. - Build the portable controller without secrets, private environment access or OIDC. Private preparation verifies that artifact and performs no dependency install. OIDC jobs use no mutable dependency resolution or shared cache. - Add synthetic source/copied-runtime tests, executable workflow admission tests, private archive tests and an operator runbook. ## Verification - Controller, campaign, portable runtime and workflow tests: 104 passed. This includes real aggregation from an isolated reporter without source dependencies, safe diagnostics, driver drift, distinct evidence binding and reviewed-lock drift. Real private Codex/Claude requests also pass through an independently verified copied runtime with fake transports. - Private transport and workflow contracts: 35 passed. Executable/static workflow guards: 13 passed. - Paired private report tests: 166 passed; Evalbook discovery: 7 passed. - `pnpm -r typecheck` and `pnpm build`: passed. - `pnpm test:e2e:runner:typecheck`, `actionlint .github/workflows/runner-protocol-live-evals.yml` and `git diff --check`: passed. - `pnpm test:run` hit repeated local embedded PostgreSQL initialization failures in unrelated DB-dependent server tests. Its owned process tree was stopped after preserving the log. This is a local verification limitation; current-head CI remains required. - An earlier head hit one unchanged Rust cleanup assertion. The exact test passed locally and the final head passed the full Rust lane; the initial failure log is retained. - On the final head, one unchanged chat idle-queue assertion failed once in CI. Its single scoped retry passed. All test jobs and the final aggregate gate passed on commit `2e328fa53dd296174d59d19f14f068aeeb0b82d8` (CI run 40535692048, attempt 2). - No paid model, Braintrust or cloud calls. No live variables, secrets, schedules or resources changed. ## Risks - Controlled mode fails closed until a reviewed `RUNNER_PROTOCOL_EVAL_CONTROLLER_POLICY_JSON` and a compatible private eval SHA exist. `all` does not silently become a supported subset. - The reviewed policy must cover setup, model requests, trace/gateway processing, archive transfer and retained storage. Hash binding does not establish vendor billing bounds. This PR does not enable recurrence or release historical holds. - The controlled policy must pin `controllerLockSha256`; it never accepts a resolver-generated digest automatically. No lockfile is committed. A changed compatible lock needs review. - Legacy browser verification now requires the committed frozen lockfile. A stale lockfile fails closed; controlled preparation has no dependency install. - Workflow cancellation can prevent the final report. The full manifest is retained before provider work and remains the starting point for reconciliation. - The workflow exceeds 1,000 lines because the shared shard anchor, dependencies, environment expressions and credential boundaries belong to one declarative job graph. Request preparation, accounting, archive safety and runtime verification stay in separately tested modules. ## Model Used OpenAI Codex, GPT-6-based coding agent with tool use and code execution. The host does not expose an exact deployment model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the relevant tests locally and they pass; the broad local suite limitation is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the final head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups; the secondary reviewer is also 5/5 - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e71a23c01c |
fix(ui): animate late task sidebar loading (#17880)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/61/changes/paperclip/61 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task pages load the conversation and related sidebar data separately. > - The sidebar host did not exist until its content arrived. > - Its first appearance skipped the width transition and moved the thread at once. > - This pull request keeps an empty host at zero width until content is available. > - The sidebar and thread now move together, with immediate changes for reduced motion. ## Linked Issues or Issue Description **What happened?** A cold task load showed a wide skeleton. The right sidebar then appeared at full width and shifted the thread at once. **Expected behavior** The sidebar should enter smoothly. The task thread should move with it. Unknown or hidden sidebars should take no space. **Steps to reproduce** 1. Open a populated task in a fresh browser context. 2. Delay the task metadata response. 3. Watch the skeleton and thread as the right sidebar becomes available. Related work: [task cold-open reads](https://github.com/paperclipai/paperclip/pull/17756). ## What Changed - Keep the shared sidebar host mounted at zero width while its content is unknown. Make hidden hosts inert and hide them from assistive technology. - Use shared motion tokens for width and opacity. Disable motion for reduced-motion preferences and keep pointer resizing immediate. - Clamp late content to the current viewport without overwriting the saved width. Keep the motion catalog aligned with the reveal tokens. - Clip the panel body during reveal. Keep the resize handle inside the viewport during the first frame. - Clear empty panes immediately when content is removed. Release maximized geometry on that render. Remove the unused width/opacity token. - Add four replayable Storybook scenarios using the production skeleton and panel components. Add browser checks for both arrival orders, absent and hidden sidebars, reduced motion, resizing, and maximize/restore. - Document the shared sidebar loading behavior in DESIGN.md. - Load the comment-route test modules in suite setup. This prevents a slow module load from timing out the first test and affecting the next test. This test-only change fixes a reproducible CI failure; route behavior and assertions stay the same. ## Verification - 201 focused task-detail, properties-panel, and motion-catalog tests passed. New host-continuity regressions failed against the original component in both task interfaces. - Eight focused Storybook browser checks passed without retries after the review fix. They measure intermediate widths and matching thread widths, check overflow, exercise resize and maximize controls, and confirm immediate removal of docked and maximized content. - The first removal checks sampled before replay cleanup committed. Sampling now starts after the host becomes inert. All three removal checks fail against the prior commit. Restoring the source during that comparison left a stale Storybook module in the preview. After a preview restart, all eight cases passed without retries on the fixed source. - Verified a real populated local task in a fresh browser context with delayed real API responses. Recorded a screencast for visual review. The captured reveal had 20 intermediate widths, no horizontal overflow, and no browser errors. - Repository typecheck, full build, Storybook build, token gates, map checks, and 30 catalog/map tests passed. Full typecheck, full build, focused UI tests, and UI typecheck were repeated after the upstream rebase. - Full `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL initialization failures. The host has exhausted its semaphore pool. The local app test used a disposable container database instead. - Replay `tasks-sidebar-loading--sidebar-during-skeleton` in Storybook. Run the focused browser suite with `STORYBOOK_TEST_BASE_URL` set to that Storybook server. - CI serialized shard 6/9 failed when the first comment-route test hit its 15-second timeout. Its work then affected the next test's mock count. Both failures reproduced locally on Node 24. After moving module loading to suite setup, all 103 tests and server typecheck passed with the same test timeout. A first local attempt could not bind the Supertest listener inside the sandbox; it was repeated with local network access. ## Risks - The shared host also serves other properties panels. Hidden preferences, default widths, pointer resizing, maximize/restore, and both task interfaces have focused coverage. - Width animation can reflow text while the panel enters. Reduced motion removes the transition. Navigation removes empty panes immediately. - Visual approval is complete. The full CI suite must pass before merge. No schema or API contracts change. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
051b425fdf |
fix(evals): price explicit response model identities (#17808)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E operations reserve model cost before each provider request. > - Providers can return an exact snapshot name after a request selects a model alias. > - The estimator required the response name to equal the request alias. It left valid snapshot usage unpriced and invalidated its reviewed bound. > - This pull request lets a versioned pricecard declare a small list of exact response names that share its reviewed rates. > - The benefit is correct cost coverage while request selection and unknown-model handling stay strict. ## Linked Issues or Issue Description Refs #17765 for the shared eval gateway request policy. **What happened?** An eval request can select `gpt-5.4-mini` and receive `gpt-5.4-mini-2026-03-17`. Even with complete usage, the estimator returns `null`. The gateway also rejects that response identity as evidence for its reviewed reservation bound. **Expected behavior** A pricecard can name exact response identities with the same verified rates. Unlisted snapshots stay unknown. A declared response identity must not become an allowed request identity. **Steps to reproduce** 1. Configure a route with an alias pricecard and an exact response snapshot. 2. Return complete Responses usage for that snapshot through the gateway. 3. Observe that the old estimator leaves the charge unknown and invalidates the bound. **Paperclip version or commit** Reproduced in the shared estimator and gateway at `507dc70be26c191e372f09e58dc0965e73bf8f57`. **Deployment mode** Other: the credential-free Product E2E operations harness. The same library serves attached deployment targets. ## What Changed - Add optional `responseModels` to versioned eval pricecards. Validate one to sixteen unique exact additional identities. - Reuse one response matcher in cost estimation, reviewed bound checks and supported request-policy cache-write attribution. Cache attribution also requires the reviewed provider and request identity to match. Keep exact request-model admission unchanged. - Admit the field in the strict operations manifest. Include it in the existing whole-price bound digest. - Cover fragmented JSON and SSE forwarding, persisted usage, durable ledger replay, partial responses, unknown snapshots, invalid mappings, stale bound evidence and unchanged request admission. - Document mapping provenance, version changes and preservation of historical ledgers. ## Verification - Focused provider usage, gateway and operations tests: **185 passed, zero skipped** on the cache attribution follow-up. The exact snapshot assertions failed before the initial repair. Three positive cache-write snapshot tests also failed before the follow-up and now pass. - `pnpm test:e2e:runner:typecheck`: passed after building required workspace declarations. - `pnpm test:e2e:runner:unit`: 4,311 passed, two skipped and four failures. The same four failures reproduce on unchanged base `507dc70` (79 passed, four failed in the two affected files): two Daytona Copilot image identity assertions and two Pi catalog/profile expectation assertions. This PR does not change those files or their source inputs. - `pnpm -r typecheck` and `pnpm build`: passed again on the exact cache attribution follow-up commit. - The private Cloud consumer passed 57 offline compatibility tests again on this exact follow-up source. These cover the declared snapshot estimate, exact request selection, unknown snapshot and incomplete usage handling, ledger holds, and source export admission. No provider calls were made. - `git diff --check`: passed. - `pnpm test:run`: interrupted after repeated embedded PostgreSQL startup failures in existing native-session and other database suites. The failed attempt is preserved and does not count as a full local suite pass. Only this checkout's verified test process group was stopped; it left no processes running. The first sandboxed workspace typecheck could not open a local tsx IPC socket; the later authorized check passes. An initial runner typecheck also lacked fresh-checkout workspace build declarations; its later run passes. - All forwarding tests use local synthetic response bytes. No provider calls, releases, deployments or paid eval campaigns were run for this repair. ## Risks - An incorrect mapping can apply incorrect rates. The pricecard author must verify provider provenance and change the price version. Validation bounds the list and uses exact matching only. - Existing pricecards remain exact-name only when the new field is absent. Adding or changing a mapping invalidates old reviewed bound evidence. Historical ledgers and receipts are not rewritten. ## Model Used OpenAI Codex, GPT-6 family. The exact runtime deployment ID and context window are not exposed in this session. The agent used reasoning, source editing, terminal commands and credential-free tests. No model inference was used by the eval harness for this repair. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
2ee36a4c0a |
test: add chat feature scenarios and real task workflow evals (#17794)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat lets a person assign work through a conversation. > - Feature and API maps do not prove that a natural request produces the right saved result. > - Product E2E already owns isolated fixtures, browser runs, evidence and cleanup. > - This pull request adds a chat scenario catalog and two real task workflows in that framework. > - Independent checks verify the task, its author, the saved output and the reply in chat. > - The benefit is a small executable foundation for broader chat coverage. ## Linked Issues or Issue Description **What existing behavior does this improve?** It adds real task creation and delegation coverage for Agent Chat. **Current behavior** Existing maps describe features and agent access. Existing chat evals cover specific protocols and handoffs. They do not connect a broad human-request catalog to independent saved-state checks. **Proposed behavior** Keep 244 planned scenarios across 35 feature areas. Promote two requests into the explicit-only `chat-features` suite. One saves a backlog task without starting work. The other delegates an announcement and returns the saved result to the original chat. **Reason and benefit** A successful tool call or confident reply does not prove the requested result exists. These evals check public API state, attribution and browser access after a real model run. **Breaking changes** No API or data schema changes. Chat instructions now require actual relative task links. The local E2E private-state scanner recognizes Codex account ownership metadata while continuing to reject credentials and model-visible identity leaks. **Additional context** Related prior work: #13969 and #13762. This improves verification of the existing Agent Chat roadmap item. It does not add a new core product feature. ## What Changed - Add YAML scenarios with capability IDs, starting context, user requests and expected outcomes. Validate all area files against their schema, global ID uniqueness and exact feature-map capability coverage. - Add two local native Codex subscription cells. Send the exact catalog request once through Chromium. - Seed only prerequisites through public APIs. Use shipped agent instructions and explicit disposable budget policies for unpriced subscription usage. - Check task ownership, creation audit, no premature execution, current worker-authored output, automatic completion delivery and usable links after reload. - Keep private Codex account ownership metadata distinct from bearer credentials. Scan rollout message content, all tokens and published evidence as before. - Teach normal chat instructions to use relative task links instead of invented hosts. Update tests, maps and the runbook. - Repair existing E2E contract drift: include the active Copilot identity in the image digest and update Pi profile expectations. Keep global catalog cardinality in its owning catalog test. Product E2E owns the new flow and grader. Its existing launch and report paths remain in use. The shared scanner change is limited to owner-private temporary Codex runtime metadata. Other callers retain the full secret scan. No UI, endpoint, schema or permission behavior changes. ## Verification - `node scripts/check-maps.mjs` passed: 35 areas and 191 capability IDs. - Product E2E typecheck and focused tests passed before rebase: 225 Vitest tests and 137 Node tests. Conversation service tests passed 40/40. Five existing runner turn-binding regressions passed. - Historical live measurement used real Chromium, server, database and native Codex with no automatic retries. The initial same-definition batch passed 5/6 cells. Backlog passed 3/3; delegation passed 2/3. One CEO startup failed because the runner did not receive a provider turn identity. The cause is unresolved. - An unchanged follow-up passed delegation three times and backlog twice. Combined evidence is backlog 5/5 and delegation 5/6. Each attempt passed cleanup. Later passes do not erase the earlier failure or prove a runner repair. - Earlier failed revisions exposed an invented task hostname, private account metadata false positives and missing funding for unpriced subscription usage. Those led to the specific changes above; the failed records remain retained locally. - The runbook records campaign identifiers and definition provenance. Generated result artifacts are not committed. Historical live results predate this branch rebase; fresh PR-source live evidence and final validation are recorded below. - Full workspace typecheck and production build passed on the rebased branch. Product E2E typecheck passed. - The full Product E2E support suite first reproduced four baseline failures: two missing active Copilot image-identity inputs and two stale Pi/catalog expectations. The narrow contract fixes above resolved them. The full rerun passed 4,282 Vitest tests plus 137 Node tests, with two skipped. Image checks verify mutation changes the hash, unchanged input stays stable, and an input closure without Copilot is unaffected. Original failed logs remain retained. - The repo-wide local Vitest run has reported two failures in unchanged service files. The Workspace connection test passed an isolated recheck and its full file passed 469/469 unchanged. The recovery test hit a 15-second timeout; its full file passed 89/89 unchanged. The run later hit explicit PostgreSQL `semget` resource exhaustion and was interrupted after those observed failures; no final aggregate counts or full local pass are claimed. Its complete available log is retained. No assertion or timeout was weakened. Matching remote server CI passed on the same source. - On clean PR head `39ad80b523ce194c90aafafd077d48dd9a36b4c5`, backlog passed in `local-2026-10-26T19-24-13-014Z`. The parallel delegation fixture failed during PostgreSQL initdb, before browser or model execution; its failed cleanup grade is retained. A disposable standalone initdb probe then passed. No semaphore sets or kernel settings were changed before that probe/recheck. After confirming no remaining fixture process, its private temporary files were removed. - One separate delegation recheck passed in `local-2026-10-26T19-29-24-800Z`, with one attempt, zero automatic retries and successful cleanup. Both passing current-source cells use definition `3f2c9ba2d03e3dcddde47ddab3b43bc52de977f25e256fceae0a24c9128e204b`. The earlier bootstrap failure and the historical provider startup failure remain distinct. These local results omit automatic source-SHA metadata; the clean checkout SHA is recorded here from the execution session. - The later broad local run reported `could not create semaphores: No space left on device`. A bounded cleanup removed 32 current-user PostgreSQL sets with 17 members, marker 537, a dead recorded creator and zero waiters. No live process or kernel limit was changed by that cleanup. The test process group was subsequently stopped; no broad rerun is planned. This direct later resource failure does not prove the precise cause of the earlier live initdb failure. - Review follow-up adds only static catalog validation and documentation of the existing five-minute journey bound. The executable definition is unchanged. Focused catalog/oracle tests passed 28/28; no redundant live run is claimed. - Final head `30fcb36ac9ae1f0270a589dbfafcec91ca6a03f3` passed all required and underlying CI checks in [run 40391402347](https://github.com/paperclipai/paperclip/actions/runs/40391402347), including `ci / verify` and `ci / e2e`. Greptile and Paperclip Review both scored this head 5/5, with all review threads resolved. The static follow-up leaves every executable definition input byte-identical to the live-tested source at `39ad80b`. Normal squash auto-merge is enabled. Actual merge is pending eligible code-owner approval for the owned dependency manifest; no protection is bypassed. ## Risks - Live model behavior can vary. One missing turn-start event remains unexplained; this PR does not claim provider reliability or broad catalog qualification. - Subscription costs are unpriced. Explicit fixture funding admits that usage. Run counts and deadlines remain bounded, but the dollar limit cannot cap unknown cost. - Account identity exceptions apply only to known private Codex runtime files. Token reuse, unknown files and model-visible rollout content remain blocking findings. - Deterministic checks establish saved state, output structure, required brief facts and access. They do not score all prose quality. Initial samples also require human inspection. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, browser and repository tools. The precise model revision and context window are not exposed to this session. Live eval agents used `gpt-5.6-sol`; that is the evaluated model, not a claim about the authoring model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run local tests; focused suites pass and the interrupted full-run failures are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8c702d59db |
feat(evals): package a portable Runner controller runtime (#17506)
Package a verified portable Runner eval controller with complete bounded file receipts, copied dependency closure, private evidence replay and symlink-safe CLI entrypoints. Preserve fail-closed cost holds, native trace receipts and all parent security repairs.
Verified head:
|
||
|
DottaandPaperclip
|
313c1db2c3 |
fix(evals): share bounded request policy with qualification (#17765)
## Thinking Path > - Paperclip manages agents and their work. > - Product E2E evaluations send model work through a gateway with cost limits. > - The gateway checks the request body before it reserves money or contacts a provider. > - A qualification probe needs that same check before it records a spending-cap denial. > - Namespace support already exists. Qualification must share that policy without sending a paid request. > - This pull request exposes the pure request validator and bounds the supported local tool definitions. > - The benefit is one request policy for qualification and forwarding. ## Linked Issues or Issue Description Refs: #17452, #17469. **What happened?** The request validator was private to `forwardEvalRequest`. A qualification caller could not reuse the request policy before admission. A separate check could accept a request body that forwarding rejects before a reservation. **Expected behavior** Qualification and forwarding use the same pure validator. One-level OpenAI namespaces with local function or custom tools remain supported. Invalid tool definitions fail before accounting or transport work. **Steps to reproduce** Use the synthetic native-daemon request fixture in `eval-gateway.test.ts`. It has seven local function tools and one namespace with five local function tools. Call the shared validator, then forward through the fake provider. Run the malformed namespace and tool-count cases to check rejection before side effects. The namespace behavior follows the [OpenAI function-calling guide](https://developers.openai.com/api/docs/guides/function-calling). This change preserves the one-level contract introduced in #17452 and hardened in #17469. ## What Changed - Export `boundedEvalPayload` with credential-free route fields. `forwardEvalRequest` calls that function. - Keep one namespace level with function or custom leaves. Require exact leaf types, nonblank string leaf names, and a closed namespace wrapper. - Limit OpenAI tool definitions to 1,024 per request. Count namespace wrappers and their children. - Add native-daemon-shaped forwarding coverage, malformed and hosted tool rejection, count and depth boundaries, and Anthropic compatibility checks. - Document the shared qualification policy in the Product E2E security guide. - Compile the real observability validator in the test parent. The owned CLI fixture imports JavaScript, so its process group does not include a test-only TypeScript compiler service. The parent resolves the compiler from the root package, which declares that dependency. Production cleanup checks and attempt deadlines stay intact. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts eval-gateway.test.ts cost-ledger.test.ts provider-usage.test.ts operating-overhead.test.ts operations.test.ts`: 173 passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Separate Node support checks: 137 passed. - `pnpm test:e2e:runner:unit`: 4,271 passed, two skipped, four failed in unchanged Daytona image and Pi catalog contracts. The same four failures reproduce on untouched base commit `a67a18c171816b097f46eae78fc58c7e55c9c565` (79 passed, four failed in those files). - `pnpm test:run`: interrupted and incomplete after required CI completed. Initial database startup failed because this checkout omitted the embedded PostgreSQL package postinstall aliases. The package hydrator restored them, and 14 real database checks passed. Earlier failures and the full output remain preserved; this local full run is not counted as passing verification. Its owned process group exited after SIGINT, with no owned survivors. - An offline full native authentication probe used the real Rust daemon and pinned Codex 0.160.0 Darwin SDK. The same captured request failed the older operations policy before reservation and reached a mock reservation with this policy. The OS sandbox allowed loopback only. It used synthetic credentials, retained no raw model payload, and made no provider calls. - Controller fixture suite: 70 passed. Three cold comparisons each confirmed that the old child started an owned compiler service and the corrected child did not. On Darwin the old compiler also exited cleanly, so this does not prove the exact historical Linux process race. Fresh Linux CI is required. - Independent review passed. The private qualification integration passed 14 exact-source compatibility checks and 67 shared-source tests without provider calls. - Required GitHub CI checks passed on the current head. The separate Entire approval gate still awaits an eligible reviewer; the delegated author account's explicitly AI-assisted approval does not satisfy it. ## Risks Requests with more than 1,024 OpenAI tool definitions or extra namespace wrapper fields now fail closed. New provider schema fields will need review. Anthropic tool handling, model prices, billing restrictions, output caps, and reservation ordering remain unchanged. This offline request-policy proof does not establish a completed model turn or Linux or Cloud onboarding acceptance. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model identifier and context window are not exposed in this session. The model used reasoning, tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bbe1016d81 |
feat(evals): retain bounded native tracing in the Runner controller (#17493)
Retain bounded native tracing in the Runner controller with attempt-bound relays, private export receipts, offline cost replay, conservative holds and bounded failure diagnostics.
Verified exact head
|
||
|
DottaandPaperclip
|
3f91d58dd5 |
fix(ci): provide Copilot auth only to protected paid eval cells (#17743)
## Thinking Path > - Paperclip manages AI agents and their work. > - Paid runner evals test real provider integrations. > - Copilot cells use the saved-token connection path. > - The two trusted workflows did not pass the Copilot credential to those cells. > - This PR supplies it only when the selected cell needs it. > - Existing environment and actor gates protect access in this public repository. ## Linked Issues or Issue Description Related: #15705 and #15706. A search found no open PR that adds this workflow credential binding. **Describe the bug** Copilot Product and direct-protocol eval cells require `COPILOT_GITHUB_TOKEN`. The protected paid workflows do not currently pass that environment secret to the selected execution step. **Expected behavior** Only a selected Copilot cell receives this credential. Ordinary PR, build, catalog, image and reporting jobs receive none. **Steps to reproduce** Dispatch the trusted full-stack workflow for one Copilot cell. Credential validation fails before the task starts, although the protected environment can hold a valid token. **Environment** GitHub Actions in the public `paperclipai/paperclip` repository, with the reviewed workflow on `master` and the `runner-e2e-paid` environment. ## What Changed - Pass `COPILOT_GITHUB_TOKEN` in the paid Product and direct-protocol execution steps, only when `matrix.credentialName` selects it. - Add the missing Copilot protocol-catalog credential mapping and verify it reaches the selected execution matrix. - Add security checks for the selected-cell binding and bounded protected-job scope. Extend the scan that rejects provider credentials in unrelated workflows. - Document the Copilot credential scope. No secret values, account data or private test evidence are in this PR. ## Verification - Product workflow security: 16 tests passed on this head. - Protocol catalog and workflow security: 21 tests passed, including the Copilot matrix regression reproduced red before its fix. - Current-head required `ci / verify` and `ci / e2e` are green. Greptile is 5/5 with zero unresolved review threads. - The first CI attempt retained runner-shutdown failures; one failed-job retry passed on the unchanged head. Both attempts remain in [the CI run](https://github.com/paperclipai/paperclip/actions/runs/40342429106). - `git diff --check` passed. - A selected-credential scan found zero matching secret values in the seven changed files. - The environment setting was separately checked: `master` only, required approval from existing allowlisted code owners, and administrator bypass disabled. The new secret exists only at environment scope; no repository-wide duplicate was created. - The packaged Copilot runtime verified the selected token and the explicit model without a model prompt. A fresh local UI journey then created an agent, completed its first task and verified the downloaded artifact and cleanup. ## Risks The selected paid cells gain access to one additional environment credential. Missing credentials still fail closed. Default-branch checks, initiating/rerunning actor checks, protected environment access and per-cell credential selection remain in place. This PR adds no trigger and gives no credential to builds or ordinary PR checks. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and browser regression tests. The exact runtime model identifier is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
49bd7a4ba6 |
feat(evals): retain portable Runner controller evidence (#17469)
Retain private Runner request evidence for offline cost reconciliation. Bound post-child reads, reject hidden paid tools, exclude configured credentials, bind incomplete results, and replace redacted proof atomically.
Verified at
|
||
|
DottaandPaperclip
|
a1f77821b9 |
perf(tasks): show recent conversation before historical activity (#17593)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - A task page must show the instruction and current conversation
quickly.
> - The current page waits for related task data and historical run
transcripts.
> - These reads grow with task history, even when the reader only needs
the latest answer.
> - This pull request gives cold opens a small initial view and loads
older history on demand.
> - Messages and runs observed while the page stays open remain visible.
> - The benefit is a readable task before historical diagnostics finish
loading.
## Linked Issues or Issue Description
Related: #14667 and #16229. The first changes the reveal gate and scroll
anchors. The second adds general history pagination. This PR adds a core
task read and a folded recent-conversation view. It preserves the
default full-read API contracts. The shared files will need coordination
if either related PR lands first.
**What existing behavior does this improve?**
Cold opening an ordinary task with a long conversation or large run
logs.
**Current behavior**
Task enrichment and historical transcripts delay the readable
conversation. A cold open can fetch several megabytes of tool activity
before revealing saved messages.
**Proposed behavior**
Show the task instruction, recent saved replies, latest human input, and
the latest run's conversation data first. Fetch older messages and full
run activity when the reader chooses **More messages**. Retain new
messages while the page stays open.
## What Changed
- Add an authorized core issue view that skips related enrichment. Keep
enrichment on the existing detail read. Share task mutation, rollback
and invalidation updates with the core cache.
- Add a bounded recent-comment read. Preserve the first human
instruction when no description exists, including legacy rows with a
NULL author type. Keep company and task access checks and comment
redaction.
- Add optional recent activity and run-summary views. Fetch original
input receipts for visible comments separately so delivered messages
stay out of the queue. Reconcile watched unfinished runs after a
reconnect beyond the preview window, removing deleted runs after
reconciliation.
- Filter the latest native run's initial events to assistant messages,
request lifecycles and terminal results. Full event reads remain
available on expansion. Legacy fallback reads use a small tail extended
to the first complete record. HTTP cancellation aborts the storage read.
- Retain observed messages across cursor catch-up and new runs. Apply
local cancellation and deletion to both caches and their task aliases.
Refetch locally seeded history pages on expansion. Preserve expansion
across identifier and UUID links. Keep deep links, classic tasks and
Agent Chat on their existing full-history path.
- Update API documentation, design guidance, capability maps and
regression coverage.
## Verification
- Final acceptance on `9ea0b15ae44d9f6d2c9a9f824b4835f299f92979`: all CI
gates passed, including server/UI tests, all eight browser shards,
runner checks, full typecheck and build. [Final CI
run](https://github.com/paperclipai/paperclip/actions/runs/40326990328).
GitHub reports 55 successful checks, two intentionally skipped checks,
and one neutral Paperclip Review check described below. No checks are
pending or failed; Greptile is 5/5 and no review threads remain open.
- October 26 merge update: merged current `master`. The only conflict
was in task-page imports; both the compact-history hook and the new
live-update hooks remain. All CI tests and build jobs passed on
`044fd6733d`. Fresh review found canceled-comment cache drift, legacy
human-comment filtering, send-then-expand truncation, and partial legacy
log records. These defects were fixed in `af3d66a5bd`.
- Review-fix verification: 470 tests pass across ten affected suites,
including all six PostgreSQL cold-open tests without a helper override.
Cancellation/delete and send-then-expand regression tests failed before
the fixes and pass afterward. Coverage includes task identifier/UUID
reopen, stale in-flight reads, legacy NULL author types, local-file and
object-storage tails cut inside UTF-8 text, route forwarding, and
request cancellation. Full workspace typecheck and production build,
token gates, map validation and diff checks pass.
- Follow-up review found that a failed expanded legacy read lost its
record-alignment state on retry. `9ea0b15ae4` commits the expanded
window only after a successful read. The new regression fails before the
fix and passes afterward; all 19 transcript tests, UI typecheck, UI
build and token gates pass. Greptile reviewed this commit at 5/5 with no
actionable findings, and all review threads are resolved.
- The final Paperclip Review check ended neutral because its execution
did not complete; no passing Paperclip assessment is claimed. Its
earlier four concrete findings were fixed and regression-tested.
Greptile and the security scans completed successfully on the final
commit.
- The preceding `af3d66a5bd` CI run was canceled when superseded by the
retry fix. Its server shard 11 check briefly reported failure, but the
job API records cancellation and its test step passed all 1,750 tests.
This superseded run is not used as final CI acceptance.
- 549 focused tests pass on the final implementation across nine task
UI, activity-route, OpenAPI and PostgreSQL suites. These cover pending
history, desktop/mobile visibility, expansion, retained runs, message
bursts, cancellation recovery and queued-comment removal in both views.
- Legacy expansion keeps the latest answer visible while fetching and
deduplicating older output.
- Five PostgreSQL tests pass for core access denial, recent comments,
missing-description instructions, event paging and bounded activity/run
reads. macOS could not start another PostgreSQL cluster because it
exhausted its semaphore limit. These tests used a separate disposable
database on an existing local server through a temporary test setup
override. The helper was restored afterward.
- Full workspace typecheck and build pass with the final state fixes, as
do token gates and capability-map validation.
- Token gates, capability-map validation and `git diff --check` pass.
- Browser proof used a disposable task with 80 comments and six large
run histories. The cold view showed the instruction and recent exchange.
**More messages** revealed older comments. Navigation away and back
preserved expansion.
- Local HTTP sample: core 50 ms, recent comments 24 ms, recent run
summaries 17 ms. The latest native conversation response was 839 JSON
characters / 8 ms; its full event response was 1,099,005 / 86 ms.
Browser content paint was 1.34 seconds with development assets cached.
These are local observations, not staging latency claims.
- The first CI run passed every job except four board-send browser cases
and the review threshold. Those browser fixtures only mocked the old
comments endpoint; they now serve both recent and full reads, retaining
all message and attachment assertions. Both initial review findings were
fixed with regression tests. The next CI run passed every job. The
deeper review identified core-cache mutation, older delivery receipt and
retained run-status gaps; those are now fixed with regression coverage.
Final review is 5/5 with zero unresolved findings. Final-head CI
initially failed one unchanged runner shutdown test
(`runnerd-codex-transport.test.ts`, timeout-attachment case) with
`NativeSessionCloseUnrecoverableError`; all other CI jobs passed. The
same runner code passed on
|
||
|
DottaandPaperclip
|
41c42895fa |
feat(evals): enforce fixed Runner request allocations (#17452)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runner evals execute real provider harnesses against maintained fixtures. > - Turn-level usage arrives after requests have already spent money. > - Parallel cells need distinct allowances before requests start. > - Missing output must preserve the remaining allowance and any known charges. > - This pull request adds a controller that reuses the existing request ledger and gateway. > - It makes a bounded local Runner attempt reviewable before hosted scheduling is enabled. ## Linked Issues or Issue Description Refs #17440. That PR added the explicit Runner gateway route. This PR owns request admission and retained campaign accounting for that route. The existing hosted matrix is not enabled or changed here. Its global allocation, rerun rejection, and private evidence archive remain separate integration work. ## What Changed - Validate a trusted manifest with exact request, CLI, package, daemon, model, and source identities. Assign each selected cell a distinct allowance. Count operating overhead once. Reject allocations above the campaign limit before launch. - Run native Codex or ACPX Claude through the shared request proxy and append-only ledger. Pass only the proxy credential to the child. Refuse local attempt reuse and preserve existing output files. - Close gateway admission on cancellation. Retain the ledger, receipts, route descriptor, child result, and cleanup evidence outside the disposable workspace. - Recompute campaign accounting from retained ledgers. Missing or invalid final evidence holds the full cell allowance while preserving known charges. Reports distinguish provisional and incomplete costs. - Accept namespaces containing only local function/custom tools. Disable web search in the controlled Codex profile. Continue to reject hosted tools, including tools hidden inside a namespace. - Add controller tests to the existing Runner static test path and document this source-checkout entry point. ## Verification - 20 controller tests pass. They use real child processes, local HTTP, and the actual ledger. Cases cover reservations before dispatch, concurrent cell isolation, caps, cancellation, replay rejection, lost artifacts, changed receipts, and Braintrust Claude routing. Upstream responses are synthetic. - 52 shared gateway tests and 55 Runner CLI tests pass. The namespace acceptance and web-search configuration regressions failed before the repair and pass afterward. - Runner TypeScript/Rust checks, Runner build, shared E2E typecheck, and `git diff --check` pass. - The first actual-harness probe found that Codex advertised both a local tool namespace and web search. It reached no upstream transport. That failure is retained; the compatibility repair does not allow paid hosted tools. - An earlier test failure showed that separate dynamic imports created different admission-error classes. The shared runtime bridge now loads one module graph. Another regression showed that a missing final report hid known ledger charges; the collector now preserves those charges. - The repaired actual-harness probe passed its control checks for native Codex and ACPX Claude: each reached exactly one synthetic upstream request after the ledger reservation; both retained the unresolved hold and route evidence; both recorded clean process exit. A scan of 41 retained files found no synthetic upstream credential. Deliberate provider rejection preserved the failed Codex turn and Claude timeout; these are not semantic passes. Prices, operating bounds, and the package archive identity in this probe were synthetic. - Review regressions reproduced lost request/receipt evidence, lost admission metadata, executable replacement after verification, and a SIGTERM-resistant descendant surviving the CLI. The repairs now pass. Executable bytes are staged before launch; a trusted Node resolution hook preserves relative imports. A symlink-path regression also failed before its repair and now passes. The final staged native probe repeated both one-request checks successfully. - Full current-head [CI](https://github.com/paperclipai/paperclip/actions/runs/40115932485) passed on `4781487215116afd10a415d4db53b1ec8e004dd5`: all 58 check runs and the legacy status are successful, neutral, or skipped. Greptile and Paperclip both reviewed this commit at 5/5. All five finding threads are fixed and resolved. No paid provider request ran during this change. ## Risks - This is a trusted source-checkout controller, not a published package command. It needs the repository's installed shared cost dependencies. - The manifest declares reviewed prices and full-lifecycle bounds. Synthetic tests do not qualify real prices, invoices, storage, or Braintrust metering. - The attempt-directory guard is local. Hosted fanout still needs unique global allocations, rerun rejection, and durable private retention. Do not enable the existing matrix on this change alone. - Forced child termination can leave cleanup unverified. That attempt remains incomplete and holds its full allocation. These checks do not claim a host security boundary against an adversarial target. - The current private publisher does not archive the separate controller directory. Native trace relay, private report integration, and live qualification remain open. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. This runtime does not expose the exact model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
7703220d38 |
perf(github): narrow response authority reads (#17686)
Read current GitHub response identity and policy fields without transferring saved message bodies, instructions, or automatic policy wrappers. Preserve company predicates, shared locks, publication leases, malformed input handling, and stored content. Verify workspace typecheck, server build, 414 selected GitHub tests, three focused browser cases, all 55 checks, and Apex 5/5. Preserve the original browser cleanup deadlock failure; always restore experiment settings while reporting cancellation errors. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5cea9c7c0e |
fix(ui): show eligible wake targets when resuming tasks (#17521)
## Thinking Path > - Paperclip manages agents and their assigned tasks. > - A task pause controls execution without changing task status. > - Resume can remove that pause and request work from eligible assignees. > - The preview counted blocked tasks and former executors as wake targets. > - The release route correctly skipped those targets, so the dialog promised work that did not start. > - Resume also trusted stale pause snapshots after tasks were reopened. > - This PR aligns preview with execution eligibility and rechecks current task state when releasing a pause. ## Linked Issues or Issue Description **What happened?** Pause an assigned blocked task, then open Resume. The dialog says that one task will resume and offers to wake one agent. Applying it only removes the pause. The route skips blocked and backlog tasks. A former executor can also inflate the wake count after reassignment. **Expected behavior** The dialog must distinguish removing a pause from starting work. Its wake count must use the current assignees and statuses accepted by the release route. Blocked work must stay blocked. **Steps to reproduce** 1. Open an assigned blocked task. 2. Choose Pause work, then Resume task. 3. Inspect the wake count and apply Resume work. Related runtime recovery work is tracked separately in #17097 and #17439. This PR does not change remote execution recovery. ## What Changed - Share the resume status list between preview and release. Count only current assignees on todo, in-progress, and in-review tasks for resume wakes. - Explain that Resume removes the pause. Show the existing preview warnings, including tasks that remain idle. - Disable and uncheck the wake option when no agents qualify. Send the same choice to the server. - Reevaluate previously skipped pause members after reopening or releasing another hold. Retain active-pause and unsafe-execution guards. - Add real PostgreSQL and HTTP-route regressions plus browser coverage for both task interfaces. Keep draft, ancestor-pause, and Stop/reconciliation coverage. - Update the task coordination maps. Add an opt-in dedicated test database URL for hosts where embedded PostgreSQL cannot start. ## Verification - Final head `9548eb16736f632cc111a5c13a9a994ec1d95a13`: all 55 completed checks pass, with two optional Storybook checks skipped. [CI 40188094904](https://github.com/paperclipai/paperclip/actions/runs/40188094904) passes, including typecheck, build, all general/serialized suites, and all eight E2E shards. Greptile is 5/5 on this commit; the reopened-task finding is fixed and resolved. - Red: the real PostgreSQL regression receives three agents instead of the one eligible current assignee on the original code. - Green: all 27 tree-control service, route, and unit tests pass. They cover current assignees, blocked/backlog exclusions, reopened tasks, company boundaries, remaining pause holds, and execution blockers. - The reopened-task route regression fails on the previous implementation: skipped members never reach preflight/wake, and an unsafe replay releases the hold instead of returning 409. All cases pass with the fix. - Browser: all five applicable cases pass together (90 seconds): blocked-work resume in both interfaces, draft/ancestor-pause cases in both interfaces, and the process Stop/pause/reconciliation/restart/cancel lifecycle. The optional native-runner fixture is skipped locally because its fixture binary is not configured. - All 157 task-page component tests pass on the prior commit; subsequent changes affect the server route and Stop fixture only. - `pnpm -r typecheck` and `pnpm build` pass before rebase, and CI typecheck/build passed on the previous PR commit. Server typecheck passes again after the route fix. The final-head CI run also passes these full checks. - `node scripts/check-maps.mjs`, `node scripts/check-token-gates.mjs`, and `git diff --check` pass. - Full `pnpm test:run` was attempted but did not complete: this host cannot initialize embedded PostgreSQL, and unrelated workspace-runtime tests timed out. The focused database tests and E2E suite use disposable PostgreSQL 18 instead. All general-test and serialized-test CI shards passed on the previous PR commit; the final-head CI run also passes every test shard. - The first CI run caught the existing Stop test trying to check a disabled wake option for blocked legacy work. It now asserts zero wake targets and retains the no-execution-before-reconciliation checks. - Earlier E2E attempts exposed fixture problems: assignment started the fixture before it was paused, and the classic interface had two matching editor nodes. The fixture now enables dispatch after the hold exists and scopes the final assertion to the task composer. No product assertion was removed. ## Risks Resume preview is a snapshot. The release route still checks current status, assignment, and execution blockers. It does not bypass approval, budget, or recovery gates. Pause, cancel, and restore keep their existing affected-agent semantics. The test database override only accepts a dedicated database name ending in `_test`; it runs migrations before the suite and closes its clients afterward. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, browser control, and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
7ebf98eafd |
fix(recovery): avoid duplicate terminal run status notifications (#17614)
Retain pending status deliveries when Native or legacy terminal recovery projects the same state. Preserve completion time on unchanged Native terminal status. Real state, metadata, error, explicit patch, and retained-file changes still notify. The publisher acknowledgement, accepted results, retry deadlines, recovery holds, ownership checks, and workspace cleanup remain intact. Correct the queued Retry browser fixture race by releasing its old-process guard at the exact Retry request, requiring a successful response, and retaining fixture cleanup after failure. Validated by 159 focused tests, 653 wider Native recovery tests, six real Chromium/PostgreSQL browser cases, five queued Retry repeats, build/typecheck, exact-head Apex 5/5, and green required CI. The original detached-button CI timeout is documented in the PR. The additional full local repository Vitest command is still running and is not claimed complete. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aec86c794b |
fix(runner): preserve native work through controller outages (#17594)
Keep remote native work under its authenticated connection lease through controller outages. Hold durable ACPX tools through reconnection, align the native Claude HTTP MCP client budget, and wait for the recovered handler before dispatch. Preserve Stop, cancellation, exact identity and artifact checks. Verified with real-process regressions and a real Claude Opus 5.5 Daytona workflow: 125 seconds offline during useful work, original process and lease retained, no failed native tool calls, queued follow-up completed, independently verified ZIP and committed workspace receipts. All current-head CI gates pass and Greptile is 5/5 with all findings resolved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
e6ec781a1a |
fix(e2e): verify review outcomes and preserve report measurements (#17556)
## Thinking Path > - Paperclip's Product E2E journeys verify user outcomes in the real app and GitHub. > - The review journey must show the score and failed check for its exact reviewed commit, then the visible task status before and after reload. > - GitHub includes a hidden “Re-run failed checks” menu item before the visible check result. > - The journey selected that hidden item and failed even after native work and publication completed. A subsequent attempt exposed the same first-text-match problem with Done in the task UI. > - This change reads the selected check's status header and binds its URL, score, and commit. > - Browser fixtures retain this failure and reject plausible wrong projections. ## Linked Issues or Issue Description Refs: #17457, #17513, #17551. **What happened?** The bot completed its native run and task, then published its summary, inline precision finding, and failed 2/5 check. The browser assertion selected a hidden “Re-run failed checks” menu item before the visible result, so the journey still failed. After that was corrected, the next live attempt passed the check projection but selected hidden Done text before the visible task status. **Expected behavior** Verify the actual selected GitHub check. A hidden menu item or a word in the model's narrative cannot prove its conclusion. **Steps to reproduce** 1. Render a hidden “Re-run failed checks” item before a visible failed check. 2. Use the old first text match for “failed”. 3. Observe the false visibility failure. **Paperclip version or commit** The base includes the journey changes from #17457 and #17513. ## What Changed - Validate the check URL against the configured repository and select its exact check-run ID. - Require the expected score and full reviewed commit inside that check. - Read failure status only from that check's header. - Give each projection step its own phase so failures identify the correct boundary. - Stage the verified persisted screenshot as the standard report final-state artifact, rejecting a passing report if it is absent. - Preserve measured billing summaries during regeneration when raw run usage is absent; newer raw usage takes precedence. - Extend the catalog-wide history fixture to include the new journey evidence contract. - Read task completion from the visible Status property picker before and after reload, with separate phase labels. - Add eighteen isolated browser fixtures to the existing browser-support suite and document both boundaries. ## Verification - Reproduced the old selector failure in the live GitHub page and a credential-free browser fixture before fixing it. - All 60 browser-support cases pass, including eighteen focused projection cases. They reject the wrong check status, score, commit, page, and repository reference, plus hidden or unrelated Done text. - All 163 focused billing, history, accounting, report, and journey tests pass, as does runner E2E typechecking. - A regression reproduced discarded saved token totals. Retained totals now survive per-cell and campaign aggregation while missing dollar prices stay unknown; fresh raw usage still overrides stale summaries. - Diff and credential-value checks pass. - The earlier failed attempts remain failed. A fresh complete QA journey passed all 18 checkpoints and cleanup with a successful native run, a below-threshold review and inline precision finding, the correctly failed check, visible Done status, and reload persistence. The later reference-validation commit changes only browser fixtures; the live driver bytes are unchanged. - A report regression reproduced the missing standard final-state artifact. The packager now aliases only the hash-verified persisted screenshot for passing results; missing final-state proof is rejected. Presentation regeneration preserves the original live grade and dispatches no provider work. ## Risks This depends on GitHub's check-run header structure. A markup change fails the journey instead of accepting narrative text as a result. The product runtime and behavioral oracle stay unchanged. Usage-price coverage remains a separate evidence dimension. A saved billing summary is reused only when raw usage is absent, preserving the recorded coverage and unknown prices. The separate advisory Paperclip Review remains queued after its prior assigned attempt timed out. Its normal Retry control rejected the obsolete failed chat request as unsafe to retry. Required repository gates `ci / verify` and `ci / e2e` pass, all 55 completed CI checks are green, and final-head Greptile is 5/5 with no unresolved findings. The advisory queue is not represented as a completed assessment. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime model ID and context-window size are not exposed in this session. Used for code, browser fixtures, and live evidence analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <paperclip@paperclip.ing> Co-authored-by: Paperclip <bot@paperclip.ing> |
||
|
|
a0e5166e97 |
Fix GitHub journey registration diagnostics and native retry cleanup (#17513)
## Thinking Path > - Paperclip manages work performed by AI agents. > - The Product E2E harness verifies a GitHub review-bot journey. > - GitHub can return a registration page without the expected creation control. > - The journey could then wait for the full login allowance without useful evidence. > - A failed native run can also retain a retry after its terminal status is visible. > - This change records a safe registration boundary and uses the normal Stop route for owned retries. > - Operators can diagnose the setup failure and confirm cleanup without changing the behavioral oracle. ## Linked Issues or Issue Description Refs: #17457. **What happened?** The GitHub review journey waited at App registration with no useful handoff evidence. Its cleanup also skipped failed native runs that still owned a durable retry. **Expected behavior** The journey should report a bounded registration failure or reauthentication boundary. Cleanup should Stop an owned native retry and require its audited acknowledgement. **Steps to reproduce** 1. Run the configured GitHub review journey. 2. Return a GitHub registration page without the expected creation button, or fail a native review run while its coordinator can retry. 3. Inspect the result and cleanup evidence. **Paperclip version or commit** Source built from current master. This extends the driver merged in #17457. ## What Changed - Bound the registration render wait to 30 seconds. Keep the separate attended confirmation allowance. - Retain GitHub origin, path, and control-presence flags. Retain headings, button labels, and alerts only on known registration error pages without authentication fields. - Exclude query parameters, entered values, page bodies, grant payloads, and authentication content from that diagnostic. - Add an optional private authentication hook for ordinary GitHub access reconfirmation. Detect passkey-first sudo at the manifest URL without retaining its authentication content. - Stop owned failed native retries through the existing run-only cancellation route. Verify company, agent, task, and audit acknowledgement. - Use native Stop correlation only for native runs. Queued legacy or unresolved runs use the ordinary cancel request. Stop scheduled retries and wait for them to leave the live state set. - Keep pending accounting debt intact. Add regression coverage and journey documentation. ## Verification - Focused journey and report suite: 58 tests passed. - Runner E2E TypeScript check passed. - `git diff --check` passed. Credential-value scan passed. - Live fixture reproduced the missing creation control at the manifest handoff. The new diagnostic retained its path and control-presence flags. No model work started. Draft cleanup passed. - All 58 current-head checks are terminal and green. Greptile scored the exact head 5/5, with no unresolved review threads. ## Risks GitHub pages that take more than 30 seconds to render the creation control now fail early. Native cleanup fails closed if the server does not provide the expected Stop acknowledgement. The live setup-to-Done behavioral journey is still unqualified. These changes do not repair a missing provider usage receipt or weaken its accounting gate. ## Model Used OpenAI Codex, based on GPT-6. Exact runtime model identifier and context-window size are not exposed in this session. Used for code execution, browser tooling, and regression analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
970127a022 |
test: retain onboarding Connect failure evidence (#17529)
## Thinking Path > - Paperclip manages agents and their work through normal company controls. > - Product E2E journeys check these controls through the browser and public API. > - Cloud onboarding uses one Connect action to validate the environment and hire the first agent. > - A failed Connect action previously left only a general journey error in the report. > - This pull request retains a closed diagnostic receipt for that action. > - The report can show the observed failure boundary without retaining private response text. ## Linked Issues or Issue Description **What existing behavior does this improve?** Cloud onboarding Product E2E failure evidence and report detail. **Current behavior** A validation or hire failure can reach the journey as a general operation failure. The report does not retain which Connect response failed. **Proposed behavior** Observe only the same-origin, same-company validation and hire POST requests made during one Connect action. Retain a closed receipt with the checkpoint, response state, bounded HTTP status, and allowlisted adapter, outcome, check code, level, and server code. The Get started control and saved-agent readback remain required. **Reason and benefit** The next failure can be attributed to the exact observed browser/API boundary. This change does not claim the cause of a historical failure. **Breaking changes** The new receipt is optional. Historical results remain valid. Explicit validation and hire rejections now stop the journey with a precise closed error code. This changes only E2E source; it does not change product behavior or qualification gates. ## What Changed - Add a bounded Connect observer with exact request matching and listener cleanup. - Retain known server codes from `details.code` or `code` without response messages, hints, headers, request bodies, provider text, or credentials. - Validate the receipt as a closed schema and render its allowlisted fields in existing report detail. - Include the new helper in the journey definition hash. - Add route, privacy, malformed-body, deadline, late-response, delayed-failure, success, and historical compatibility tests. Document the evidence boundary. ## Verification - Focused Product E2E tests passed: 3 files, 58 tests on the review-fix head. - E2E typecheck, workspace typecheck, and full build passed on the preceding implementation head. The review fix changes only report labels, checkpoint timing, and the matching tests. - The local full `pnpm test:run` command stopped progressing at the general-server start header. Its owned session was interrupted. This is not recorded as a pass. - All 58 remote checks completed on the final review-fix head: 55 passed, one neutral, and two expected skips. Both Greptile and Paperclip reviews are 5/5 with no unresolved review threads. - No live campaign was run with this patch. The observed historical failure is preserved. ## Risks - Unknown or missing responses remain explicit diagnostic states. They do not establish a product root cause. - Response body reads stop after five seconds or the action deadline and accept at most 64 KiB of JSON. Checks are capped at 32 retained entries. - New server or check codes appear as `unrecognized` until the closed allowlist is updated. - The production launch classifier, native gateway qualification, and Get started requirement are unchanged. ## Model Used OpenAI Codex (GPT-6 family), with code execution and tool use. The host does not expose the exact model ID or context window to this run; these values are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |