mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-05 21:02:34 +01:00
b7dfdbbefdca8b1f17e1eb7fe3c93ab4c4c41a18
878
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
b7dfdbbefd |
test(evals): preserve consumed commentary identity response
Dispose only unread browser-session response bodies. Exercise the actual callback with real Node Responses so successful JSON is not followed by locked-stream cancellation. Preserve real reader and owner identity checks, restoration, denied-body disposal, malformed JSON failures, feedback privacy oracles, and original clocks. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0ecd43035a |
fix(evals): bound status-card setup by original case clock
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fa644b9b20 |
test(evals): bind deferred question delivery to its actual successor
Accept a null-target durable wake receipt only with the unique succeeded answer-bound continuation from the server response path. Keep actual human answer, company/task/agent, ordering, duplicate conflict, saved output and full run accounting checks. Version the observer and retain failed history. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
41743cb0ae |
test(evals): observe required question validation on Next
Replace the disabled-button assumption with real invalid-click validation, no page advance, no saved answer and no new work. Keep all subsequent exact answers, page retention and continuation checks. Calibrate the real QuestionForm with a pure React test; no product UI behavior changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
12e6def200 |
test(evals): mutate readonly context only through fixture setup
Keep the exhausted-budget negative while respecting the readonly public context type. No production observer or grader changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd4dc1f10b |
test(evals): inspect authored skills through normal Studio navigation
For the reusable-skill request, prove ordinary library discovery, editable Studio checklist content and reload with the same saved version instead of prescribing a chat-link format. Keep current-version/content/attribution/assignment checks and all other skill contracts unchanged; record the explicit authoring v2 conditions. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d2826233b3 |
test(evals): align goal link calibration with UI evidence
Keep the aggregate no-chat-link expectation consistent with the reviewed v2 normal Goals UI/readback oracle. Missing hierarchy inspection remains a failing negative control. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b65a906c87 |
test(evals): name the requested Fast response feature
Clarify the feature-specific request without injecting evaluator context or changing the fixture, service oracle, permissions, model or clocks. Retain a regression that rejects an instruction-only acknowledgement change. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
750c9f517b |
test(evals): grade inspectable goal hierarchies with optional owners
Version the goal-tree oracle against the product contract: ownership is optional unless requested, assigned owners must belong to the company, and normal Goals navigation plus public readback proves inspectability. Keep exact hierarchy, attribution, native work, explicit retry identity and progress assertions; keep original failed history. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
615b8e6a9a |
fix(evals): copy readonly skill source inventory
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cbecbfc788 |
fix(evals): read immutable skill version details for content evidence
Version listings contain inventory metadata without file bytes. Fetch source-scoped immutable version details and preserve version, author and inventory identity for content and history observations. Keep link, import, semantic and native feature graders unchanged. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0697887371 |
fix(evals): establish and preserve independent tree budget holds
Use the existing positive-cap and explicitly synthetic accounting-import fixture pattern instead of zero, which disables enforcement. Preserve the held branch policy while independently verifying company, CEO and active branches remain funded; retain all real tree stop/pause/hold oracles. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6e5a430118 |
fix(evals): preserve assignment permission in team fixture request
Use the complete public permissions schema and retain the independently observed same-company CEO assignment capability. Reject foreign or unverified actor metadata before writing, and verify the effective capability remains unchanged. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
78d91b0e2f |
fix(evals): admit verified missing personal secret prerequisite
Prove the controller and worker ready before introducing the missing required current-person secret. Preserve that specific scenario only after exact owner-scoped missing-binding resolution; reject unrelated lifecycle, hold, identity and configuration failures. Keep feature grading, human secure entry, native delivery, clocks and privacy assertions unchanged. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bb33df5320 |
test(evals): bound wrong-reviewer observation to its case clock
Use the existing remaining original deadline for the wrong-reviewer native reply before the real human denial probe. Preserve reviewer identity, request state, permission and no-work assertions. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fce64fe2f2 |
test(evals): keep CLI guidance inside the original clock
Pass the already established case deadline into the two CLI guidance waits and validate it before and after observation. Preserve command grammar, scoped CLI actor and saved-state oracles. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9ab0f951a5 |
test(evals): retain readonly company deadline typing
Set synthetic invalid context values without assigning to the readonly production contract. Runtime observer behavior is unchanged. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2387c95569 |
fix(evals): preserve controller secrets during secure entry fixture
Authenticate only exact bootstrap secret references as the pre-entry baseline. Preserve their company ownership, metadata and versions through human deployment-key creation; reject foreign extras or a precompleted target while retaining secure UI, permission, binding and privacy checks. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a5408a736d |
test(evals): share the original company settings clock
Observe declared company settings native turns within the existing setup-inclusive case deadline instead of the unrelated 150-second helper cap. Keep the original case limits, model, queries and feature oracles. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b2f86ff593 |
fix(evals): use public membership for incidental human fixtures
Keep invitation enrollment quarantine for invitation journeys. Establish the same real scoped non-admin identities through the existing public prerequisite helper for feedback, timeline, and primary-agent fixtures. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a8ef1cea37 |
test(evals): initialize owned SDK retention staging
Create the SDK staging parent before retention when private capture uses the external handoff root. Preserve existing source, compiler, path and privacy validation. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3b66c3c144 |
test(evals): admit the finite chat catalog through TestOps
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
de0941835c |
test(evals): admit the bounded TestOps chat completion pilot (#18515)
## Thinking Path > - Paperclip evals verify real agent workflows. > - The preserved chat catalog runs on authenticated AWS hosts. > - Its host guard currently admits only the full-stack workflow. > - A dedicated TestOps pilot needs the same host verification. > - This change admits only its three selected scenarios and fixed profile. > - The existing workflow and all other catalog cases retain their gates. ## Linked Issues or Issue Description Refs #18377 and #18516. This is intentionally the catalog-only prerequisite for a dedicated TestOps connection to project work, project defaults and instance restart. The full catalog remains on the integration branch. This admission is inert until the companion main-only workflow in #18516 exists. No TestOps dispatch may run until both PRs pass their gates and the companion source pin is final. The companion exercises its real YAML environment against the actual pinned source host function before provider access. Registration and source checks do not establish a new live TestOps pass. ## What Changed - Admit the main-only `chat-completion-testops.yml` entry point for the three exact pilot IDs. - Require its admission marker, job-run identity and original attempt one. Preserve profile, source, actual AWS metadata, lifecycle and zero-retry checks. - Record the actual admitted workflow in the existing host receipt. Document the limited entry point. ## Verification - Red: all three new positive controls fail at the old workflow guard. The other 22 host controls pass. - Green: all 30 host controls pass. Negative controls reject foreign cases, missing authority, missing job identity, replay and candidate workflows before metadata or inference. - This guard and the new dedicated dispatcher both require at most 16 digits and a positive JavaScript-safe integer. The old backlog dispatcher is a separate entry point; its 20-digit string bound is not used here. Controls cover leading zeroes, unsafe 16-digit and 17-digit IDs, and exact maximum-safe ID preservation. - The receipt retains the admitted TestOps job-run ID and marker. Scenario admission cross-checks both execution ID and actual task ID. - Full catalog typecheck passes with the existing qualified dependency graph. The first fresh-checkout attempts lacked dependency links; the first broad unit startup also lacked prepared provider bundles. Those setup failures remain retained. The focused host controls make no native or provider calls. - Current-head CI and review are pending. No live run has been dispatched from this new entry point. ## Risks The new entry point has access only through a trusted main workflow. That workflow must authenticate the exact TestOps selection before provider access and retain private evidence through its existing owner. The prerequisite alone does not register cases, expose credentials, or grant a live pass. Model, queries, case clocks and graders are unchanged. ## Model Used OpenAI GPT-6 through Codex, with repository inspection and code execution. Exact hosted model revision and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
192d83e828 |
Fix chat catalog prerequisite clocks and retained evidence (#18377)
## Thinking Path > - Paperclip evals must prove actual work and retain inspectable evidence. > - Project defaults must reach new work while prior work remains active. > - A hidden HTTP deadline invalidated that prerequisite before the case clock. > - Report rewriting and missing evidence files obscured real outcomes. > - Observer assumptions also rejected valid history, reads and lease bookkeeping. > - This PR repairs those layers and keeps the original feature requirements and clocks. ## Linked Issues or Issue Description Refs #18364 for product repairs. Refs #18170 for protected catalog evidence. The held worker's native fetch had a five-minute header timeout. Successful cases lacked required files. Restart compared stored Markdown to rendered text. Diagnostic redaction changed trusted Playwright program files, causing the existing producer guard to reject failed-report retention. Valid dedicated project reads and automatic environment leases were also misclassified. This PR targets **`codex/chat-catalog-master-integration`**, not master. Its base combines master `51ea624344`, preserved catalog `e2b35be76f` and all product/test changes from #18364, including the native chat response-stage contract repair. The frozen preservation branch remains unchanged. The full catalog stays off master. Shipped subscription aliases retain their 120-second turn limit. ## What Changed - Hold the exact worker with native HTTP, a one-use agent and the original absolute deadline. Keep its capability in an owned 0700 directory / 0600 script outside evidence trees. Put only its path in task/tool inputs. Register the token with the existing secret scanner. Remove file/listener after normal execution and failed setup. - Use the already-selected original native case deadline for prerequisite readiness. Consume only public lifecycle state and fixed recovery diagnostics, preserve manual-repair/terminal failures, and record bounded state transitions and recovery flags. Other lanes retain their authored 45-second setup cap. Nine tests pass actual server diagnostics through the public agent normalizer. - Poll canonical source state with the existing settlement predicate; collect full domain/browser evidence after settlement. - Capture required `final-state.png` and actual `api-state.json`. Keep restart logs inside owned roots and recover abnormal evidence from the actual capture root. - Compare attributed rendered history before/after restart. Independently require exact stored bytes, identity and no duplicate admissions. - Authenticate and scan canonical Playwright program files with the existing validator before preserving their exact bytes. Keep ordinary redaction and every original path, producer, secret, size and transfer guard. Trusted protocol files remain unchanged. - Expose only fixed diagnostics, recognized tool names and bounded counts. Keep raw provider output private. - Accept correlated `list_projects` reads only when the same native call delivered the exact full saved target-project defaults. Cover real OpenCode and server event producers; reject stale, truncated, foreign and unpaired reads. - Version project-defaults as **v4**. Accept natural “newly admitted” future-scope wording; still reject replies missing the future/existing distinction. Exclude only automatic lease acquisition/release with the exact `environment_lease` entity type from settings-change rejection. Retain rejection of actual budget/environment edits, mismatched types and unknown events. Keep active-work, inheritance and reply checks strict. ## Verification **Qualified: [pilot 41256592937](https://github.com/paperclipai/paperclip/actions/runs/41256592937) passes 3/3, with all 35 independent checks passing.** | Case | Checks | Full case duration | Evidence and cleanup | | --- | --- | --- | --- | | Instance restart | 9/9 | 140.1 s | complete | | Project defaults | 11/11 | 204.2 s | complete | | Project work | 15/15 | 204.0 s | complete | All three are original-attempt results from source `c51e079f7f2364305b131ffd549e263ff78c5335`, containing the exact product files from `13c2b61f31d873a475ea0c3c026b19926ff42301`. Actual saved conversation contracts were inspected, as were ordinary native worker contracts where present. Each protected archive includes the actual API state and final PNG; cleanup completes. Original queries, `openrouter/deepseek/deepseek-v4.1-flash-1017`, finite case clocks, concurrency one and zero eval retries are preserved. Earlier passes do not transfer into this qualification. Both current heads pass all checks: product **62** (56 success, one neutral, five skipped), catalog **56** (54 success, one neutral, one skipped). Both have exact-head Greptile **5/5**, zero unresolved review threads and clean mergeability. The three trusted private-evidence protocol files remain byte-identical to the dispatch workflow. Raw evidence stays protected. - **202 focused catalog/evidence tests and catalog typecheck pass.** Nine readiness controls consume actual lifecycle diagnostics through the public agent normalizer, which omits private operation state. Manual repair, terminal states and unknown errors still fail. - Real Node 24 control: old fetch fails at 301,464 ms; native HTTP completes after release at 310,054 ms under the same 340-second control deadline. Actual native OpenCode/Runner also completes a separate source while the held worker stays blocked. Failed-setup regressions verify resource cleanup. - All **45 privacy/report controls pass**. The old failed-report trace rewriting reproduces the producer-guard rejection; authenticated canonical program bytes now survive unchanged. Contaminated assets still fail. No secret, producer, path, transfer, expansion or size guard is weakened. - Oracle **`paperclip.chat-catalog.project-defaults.v4`** retains active-work, inheritance, actual mutation/read and reply requirements. Positive and negative controls cover correlated full saved-default reads, automatic lease bookkeeping and natural “newly admitted” wording. Original failed grades remain retained alongside explicit versioned regrades. - Product validation in #18364 includes 2,044 discovery tests, 112 real-database/HTTP/native-runner/OpenAPI consumer tests, and passing typecheck/build. Task-owned Docker PostgreSQL unblocked local DB checks and was removed afterward. Retained attempt ledger; no earlier result substitutes for the qualified run: | Attempt | Recorded outcome / finding | | --- | --- | | 41114636294 / 41119215255 / 41203017450 | Initial report/setup and unavailable-archive failures. Functional partial success did not qualify the reports. | | [41207155025](https://github.com/paperclipai/paperclip/actions/runs/41207155025) | 0/3. Rendered-Markdown mismatch; other archives unavailable. | | [41211541655](https://github.com/paperclipai/paperclip/actions/runs/41211541655) | Work-only diagnostic: 15/15 with required files and cleanup. | | [41212946325](https://github.com/paperclipai/paperclip/actions/runs/41212946325) | 2/3. Defaults did not settle; transfer failed. | | [41216880087](https://github.com/paperclipai/paperclip/actions/runs/41216880087) | Defaults failed; counters did not identify finish outcomes. | | [41219519639](https://github.com/paperclipai/paperclip/actions/runs/41219519639) | Native work succeeded but grader did not; report assets and expansion limits rejected transfer. | | [41222196525](https://github.com/paperclipai/paperclip/actions/runs/41222196525) | 1/3. Restart exposed fabricated empty-response success; defaults exposed backfill and missing dedicated-read recognition. All archives retained. | | 41226224666 / 41226710271 | Cancelled before paid cells to repair generated outputs, credential custody and native event-shape coverage. | | [41227965230](https://github.com/paperclipai/paperclip/actions/runs/41227965230) | 2/3. Defaults 9/11 correctly rejected existing-task backfill and reply scope. | | [41233723348](https://github.com/paperclipai/paperclip/actions/runs/41233723348) | Restart 9/9. Defaults 10/11: v2 misclassified automatic lease events. Work was cancelled because replacement was dispatched prematurely; no work pass inferred. | | [41236679567](https://github.com/paperclipai/paperclip/actions/runs/41236679567) | 1/3. Defaults 11/11. Restart 4/9: no answer. Work failed and the unchanged ZIP expansion guard rejected full retention; missing outcome remains inconclusive. | | [41240010122](https://github.com/paperclipai/paperclip/actions/runs/41240010122) | Restart 4/9 under authority guidance: its literal native criterion still required the unavailable action. Defaults 10/11: v3 rejected valid “newly admitted” wording; v4 explicitly regrades 11/11 while retaining the original grade and privacy findings. Work 10/15: created/repaired a task without final chat reply. All three archives retained. | | [41243833425](https://github.com/paperclipai/paperclip/actions/runs/41243833425) | Source `6fa0c83674` / product `dddd2d5206`: 2/3. Restart 9/9 and work 15/15; defaults failed prerequisite setup before any canonical query/native run because its 45-second helper cap cut short a normal 60-second probe. | | 41247798414 | Cancelled before paid cells after review caught a private-field assumption in the readiness helper. The repair uses public lifecycle state and fixed recovery diagnostics. | | 41248428190 | Cancelled before paid cells after the unresolved-thread sweep caught a conversation identity edge case. The repair uses canonical identity and carries the user ID through heartbeat. | | [41250502608](https://github.com/paperclipai/paperclip/actions/runs/41250502608) | Source `7b1de24ce1` / product `c0bf249807`: 1/3. Work 15/15. Restart 4/9 despite the repaired contract actually being admitted; it continued discovery after board-only 403 and had no reply. Nearby HTTP 200 reads are not call-correlated to the MCP timeouts. Defaults timed out before settlement (zero finish proposals), then exceeded the unchanged 256 MiB original-evidence bound; detailed domain behavior remains inconclusive. | ## Risks The prerequisite needs native Node in the worker environment. Trusted program bytes remain subject to the existing authentication and scanners. The audit exception rejects unknown events. Oversized ZIPs can still be rejected, as in the retained work attempt; this PR does not weaken that guard or claim an unobserved outcome. A passing pilot qualifies the selected attempts, not full-catalog reliability. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use, code execution and live eval tooling. Exact hosted model revision and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge - [x] Final same-source three-cell pilot passes 3/3 with required evidence and cleanup --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
1e423103b3 | Record refreshed master recovery baseline for catalog pilot | ||
|
Dotta
|
e7522f48bb |
Merge commit '51ea62434413df7a0485e8b0c5f6700671b698a5' into codex/chat-catalog-master-integration
* commit '51ea62434413df7a0485e8b0c5f6700671b698a5': fix(server): reject a native run's own task status change (#18411) Make saved execution receipt recovery react to durable work (#18409) Drive issue recovery from committed changes and exact deadlines (#18402) |
||
|
DottaandPaperclip
|
3ee4c5b613 |
Make saved execution receipt recovery react to durable work (#18409)
Reviewed and verified at
|
||
|
Dotta
|
76a3af8a43 | docs(evals): record the refreshed chat recovery baseline | ||
|
Dotta
|
f1602d654e |
Merge remote-tracking branch 'origin/master' into codex/chat-catalog-master-integration
* origin/master: fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) |
||
|
DottaandPaperclip
|
47f81e850b |
fix(testops): verify unexpanded result job runs (#18462)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat feature evaluations send their results to TestOps. > - The final reader checks the result against its current GitHub run. > - A real result omitted the nested job object and had no presentation links. > - The reader rejected that passed result after a successful upload. > - This change joins the embedded job-run ID to the separate job-run response. > - The reader keeps the source, case, status, and attachment checks. ## Linked Issues or Issue Description Refs #18394 and #18322. **What happened?** The native chat test and cleanup passed. TestOps stored a passed result and three attachments. The final read-only step failed with `chat_testops_upload_mismatch`. The result response had a job-run ID, launch ID, and current GitHub run URL. It did not include the nested job object. Its presentation links were empty. **Expected behavior** The reader must accept that response when the separate job-run response proves the exact job, launch, and current GitHub run. It must reject conflicting identity or source data. **Steps to reproduce** 1. Return a valid result with an unexpanded job run and empty links. 2. Return the matching full job run from the existing job-run endpoint. 3. Call `verifyChatUpload`. The old reader rejects the result. **Paperclip version or commit** Observed on |
||
|
Dotta
|
2e45e86734 | docs(evals): record current master integration boundaries | ||
|
Dotta
|
6af11aacac |
Merge remote-tracking branch 'origin/master' into codex/chat-catalog-master-integration
* origin/master: fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
Dotta
|
6c1a5b9e07 | test(evals): reconcile preserved catalog with current harness contracts | ||
|
Dotta
|
b1e3950e6d |
Merge master into the preserved catalog integration candidate
Preserve the frozen catalog and its historical qualification separately. Keep current supervisor ownership, installed-artifact and recovery controls, the shipped TestOps cell, and the historical OpenCode candidate's exact model. Production source matches master
|
||
|
DottaandPaperclip
|
ab51cabbbd |
test: await async outcomes and drive saved deadlines deterministically (#18416)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
f2db909393 |
fix(ui): retain startup autosave through session recovery (#18412)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
d5766e4074 |
perf(chat): scope task questions and batch inbox queue checks (#18422)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Standing chats show recent messages and current task questions. > - Recent-message pagination bounds history work, but it does not bound question work. > - The question read built the full company attention feed before it selected the chat's tasks. > - Each feed read also checked seeded queue membership with one insert per item. > - This pull request scopes question collection in the attention owner and batches queue membership checks. > - Chats keep their current question cards and initial layout while doing less work. ## Linked Issues or Issue Description Follow-up to #18358 and #18323. Related work: #18312. **What happened?** A hosted standing chat remained slow after recent-message pagination shipped. On commit `4eaa435bf4`, an embedded-browser reload made the conversation usable at about 8.6 seconds. Recent comments finished at about 1.9 seconds. The pending-question request held the initial reveal until about 8.4 seconds. That request called the complete company attention feed and then filtered its results. The separate badge summary also did broad work. **Expected behavior** Chat question cost should depend on the relevant tasks. Reading an existing queue must not require a separate insert for every member. Current access and decision policy must remain in force. **Steps to reproduce** 1. Create a standing chat with questions on tasks assigned to its agent. 2. Add many unrelated tasks and pending questions to the company. 3. Open the chat from a fresh application cache. 4. Compare recent-comment arrival with the pending-question response and content reveal. **Paperclip version or commit** Baseline: `909e29dd071b1aeeb24c2933af049da90a6066d3`. Its affected services match the observed hosted revision `4eaa435bf4`. ## What Changed - Add an internal task-question projection to the attention service. Filter pending interactions by company, task IDs and question kind before card enrichment. - Keep the same access, audience, dismissal, queue, snooze, retention, archive and training-example logic. Ranks are local to the returned question list. - Use the projection from the conversation receipt service. Keep current chat audience and session checks there. - Batch seeded membership inserts in groups of 500. Keep the queue lock, conflict protection, audit events and activity logging for each new membership. Existing members produce no new audit records. - Add real PostgreSQL regression cases and a reproducible paired benchmark. No schema migration or UI reveal change is needed. ## Verification - `pnpm -r typecheck` passed. The final server typecheck also passed after the queue change. - `pnpm build` passed. - Real PostgreSQL service coverage: 57 attention and chat-handoff tests passed. Seven queue-route tests passed, including concurrent reads, mixed existing/new items, duplicates, disabled seeds and audit preservation. - Red/green proof: the unrelated-attention regression fails with the baseline conversation implementation and passes with the scoped read. - `node scripts/check-module-boundaries.mjs` and `git diff --check` passed. - Full `pnpm test:run` was attempted and stopped with exit 130 after embedded PostgreSQL startup failures and 17 failures in the untouched native-session executor suite. It is not a local pass. The focused Docker/PostgreSQL results above completed before Docker Desktop became unavailable. A direct native PostgreSQL probe identified the host semaphore limit (`could not create semaphores: No space left on device`). No shared kernel settings or other test clusters were changed. [Full CI](https://github.com/paperclipai/paperclip/actions/runs/41162314995) passed on `2809f096b3fb2c01b44a96a1b82a27809c9359df`, including all eight browser shards. Final status: 55 successful checks/statuses, three skipped and one neutral. Greptile is 5/5 with no findings or unresolved threads. The fresh local browser retest is blocked because Docker Desktop cannot start and native PostgreSQL cannot allocate semaphores. The original hosted reproduction above is complete; no candidate browser pass is claimed. Local service benchmark: PostgreSQL 17.11, two relevant tasks, five alternating pairs per scenario, after warmup. These are local service timings, not a claim about deployed latency. | Fixture | Read | Baseline median (range) | Candidate median (range) | SQL calls | | --- | --- | --- | --- | --- | | 1,000 unrelated tasks/questions | Chat questions | 670 ms (632–1,905) | 12 ms (11–15) | 1,076 → 22 | | 1,000 unrelated tasks/questions | Company badge summary | 641 ms (622–717) | 167 ms (161–171) | 1,068 → 69 | | Small company | Chat questions | 38 ms (37–52) | 13 ms (12–16) | 52 → 22 | | Small company | Company badge summary | 34 ms (33–39) | 34 ms (32–40) | 44 → 43 | All question fields and ordering matched, except rank is relative to the scoped list. Summary fields matched except the generation timestamp. The interaction plan changed from 1,002 selected rows to two rows through the existing company/issue/status index. The benchmark loads baseline attention, conversation and queue modules together so the queue change cannot contaminate the baseline. ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-questions.ts \ --baseline-ref=909e29dd07 --unrelated=1000 ``` ## Risks - Scoped and complete feeds must share policy. This change reuses the same attention collection and finalization path and tests dismissals, named audiences, snoozing, archives and private access. - Batched inserts must audit only new members. The concurrent PostgreSQL test checks unique membership and one event/activity record per new item across batch boundaries. - Full inbox summaries still collect the company feed. The small-company summary timing is unchanged. Other chat requests and transcript hydration can still contribute to hosted latency. - The first reveal still waits for question cards to preserve layout. This follow-up has not been deployed, so its hosted effect remains unverified. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution and browser profiling. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
492d537340 |
fix(native): recover controllers across webserver replacement (#18343)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native execution runs in a remote Runner while the webserver owns its controller. > - Encrypted PRP checkpoints and session snapshots already survive a webserver restart in Postgres. > - Ordinary warm and per-turn admission still needs local markers and supervisor metadata to find that state. > - Replacing the webserver with an empty native controller directory can therefore lose a usable retained session. > - This change adds bounded recovery metadata to the existing snapshot transaction and reconstructs local projections from the selected durable source. > - The benefit is native execution continuity across webserver replacement, with the existing ownership and agent lifecycle policies. ## Linked Issues or Issue Description **What existing behavior does this improve?** Native remote controller recovery and retained-session continuation during a webserver upgrade. **Subsystem affected** server/ — native runtime, checkpoint persistence, recovery scheduling and internal upgrade reporting. Product E2E qualification also changes. **Current behavior** Without both the local PRP marker and supervisor file, normal continuation can return before discovering the saved Postgres checkpoint. The snapshot already exists in Postgres, but its local reuse compatibility envelope cannot be reconstructed. **Proposed behavior** Select the existing durable source, validate its identity and ownership, restore its PRP state and supervisor projection, then use normal authenticated admission. Missing or inconsistent evidence holds recovery. It does not authorize fresh provider work. **Reason and benefit** A replacement webserver must recover active execution and later continue its retained session without the old server's native files. This is one component of deployment continuity. HTTP traffic switching, workspace storage migration and Railway rollout remain separate. **Breaking changes** No new public endpoint, schema migration or database pool. The private run-profile envelope is additive. Disk-independent rollback also requires a reader for the staged controller format and compact checkpoint part; the first production release containing these readers is the floor. Older file-backed sessions need an owner checkpoint or the bounded internal importer. **Additional context** Related foundations: #17401, #17916, #17254, #14621, #14622. The implementation contract and open acceptance gates are in [the dated plan](doc/plans/2026-10-30-native-controller-checkpoints.md). ## What Changed - Persist a versioned recovery envelope, at most 4 KiB, beside the redacted session snapshot in its existing transaction. Keep it out of public run projections. - Compare frozen admission separately from prepared attachment instructions during staged recovery, preserving exact template/identity/authority checks. - Bound the existing lazy checkpoint pool at three connections after measured two-slot burst backlog; keep request/renewal capacities and SQL operation counts unchanged. The tuning is a separate commit. - Discover and restore the exact selected retained source without local native markers. Preserve the 32-candidate window, progress barriers and current authority checks. - Fence snapshot and result writes. Acquire coordinator, issue and run locks in order and check expiry after lock waits. Combine snapshot ownership/read work into one statement. - Reuse existing recovery deadlines and drain/resume scheduling. Retained-owner waits preserve the original unstarted run and actual lease deadline. Stop, ownership and provider-progress fences precede deferral. Add bounded native evidence reporting, retained-source import and retention protection. - Admit a settled warm source with an authenticated idle goal-read suffix while retaining active-work and mixed-authority holds. - Permit lifecycle `turn.stop` checkpoints after accepted results under existing ownership fences. Claim the existing successor coordinator before retiring the old warm session, and stage shared attachments only after verified retirement. Preserve omitted isolated-home defaults and existing fresh skill-selection order during recovery without hiding real configuration changes. - Carry the exact durable six-field PRP binding during active recovery after a healthy warm handoff. Keep the Runner mismatch guard and preserve opaque attachment IDs. - Preserve an already verified active or idle warm process during remote preparation; revalidate its proof/liveness and retain later authenticated PRP attachment. Ordinary resumed leases retain suspension checks. - Drain retained source events under their original authority, reconcile immutable source accounting and block unknown source tool dispatch before target work. - Persist compact staged handoff intent through authenticated adoption and recover direct/listener command-loss windows without another Runner or duplicate attachment. - Rebuild suspended lifecycle receipts for resumed prepared tasks, and retry missing lifecycle reads after partial PRP materialization without overwriting contradictory evidence. - Add normal warm/per-turn diskless continuation coverage, real Postgres races and SQL-cost checks. Add a reproducible paired load workload. - Give only the explicit outage fixture a 15-minute provider idle window, with a red/green catalog regression preserving all production timing and lifecycle policies. - Publish the goal-less task-session pointer in the existing post-acceptance terminal checkpoint transaction, with current ownership, accepted-result, reset and newer-progress fences. Keep a private bound receipt so late completion cannot duplicate or resurrect publication. - Use a retained warm sandbox in the dedicated continuity fixture, and accept an exact output token followed by an explanation. Keep the strict identity oracle. - Extend the explicit Daytona cell to a five-minute active outage and another replacement after completion. Require retained provider identity, a new tool receipt and a useful follow-up result. Add an owned loopback Postgres E2E fixture for hosts that cannot start embedded Postgres. ## Verification **Merge recommendation — `fc68584ce6231a02e47ecd743f55b036ccea034f`: ready for the qualified native-controller component.** Full CI, fresh review, local application paths, controlled performance qualification and the strict live Linux/Daytona case pass. Independent resource cleanup is complete. Broader rollout, browser reconnection and the unexecuted transport/platform combinations remain integration gates. Integrated 484 passed full local recursive typecheck/build and ordinary active→completion→cache-loss→C component qualification. Fresh review then found a valid attached-wake failure: preparation appends attachment instructions before the verifier compares execution to the frozen admission. 9f4 fixes that by separately checking the pre-preparation admission and exact prepared execution. Both hashes are captured privately per admission; the full template, identity, configuration and authority checks remain. Its regression fails before the fix; 36 focused cases, 42 staged source/target cases and server typecheck pass afterward. The review thread is resolved. Exact-head full CI passes ([run 41168228822](https://github.com/paperclipai/paperclip/actions/runs/41168228822): 48 successful jobs, one intended skip); the complete `pnpm test:run` partitions are forced, with the five native suites in the Runner lane. This is not a claim that an unsharded local command passed. The fresh exact-head Greptile check succeeds with 5/5 and all six threads resolved. The original two-connection database comparison closes **steady PASS and burst FAIL**. Every relative CPU/WAL/request-p99 gate passes and all operations/ownership remain correct, but candidate arm 3 drains in 1,015.99 ms and baseline arm 4 in 1,059.67 ms against the unchanged 1,000 ms absolute gate. Those results remain failed. A separate 2→3→4→2 diagnostic reproduces drain times 1,051.57 / 956.79 / 882.02 / 1,065.72 ms: three is the smallest tested passing cap, and the repeated two-cap failure survives run-order/cache effects. Separately reviewable `fc68584ce6` changes the existing lazy checkpoint pool default to three connections. Request capacity stays ten and liveness two; idle release, shutdown fencing, transaction counts and pool count are unchanged. Thirteen DB lifecycle/cap tests and DB typecheck pass. Full recursive typecheck/build pass at exact |
||
|
DottaandPaperclip
|
322fa6b65c |
fix(ui): restore task document authoring and add persistence journey (#18356)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Task documents keep versioned work beside each conversation. > - The default chat interface could read general documents but hid their editor. > - People also need unfinished edits to survive a tab switch or reload. > - This pull request restores the shared editor in Artifacts. > - A portable browser journey checks exact saved content and revision history. > - The benefit is a usable document flow with independent deployment checks. ## Linked Issues or Issue Description **What happened?** A person using the default task interface could read a general task document but could not create or edit one. The existing editor was available in Classic mode. A pending create could also leave newer text as a new form after its editor unmounted, causing a conflict against the person's own successful creation. **Expected behavior** A person can create and edit general documents through Artifacts. Titles, bodies and revision history persist after reopening and reloading the task. Unfinished text survives tab switches. A successful create makes retained newer text an edit of that confirmed revision. Later writes from another actor still cause a conflict. **Steps to reproduce** 1. Open an ordinary task in the default chat interface, then open Artifacts. 2. Try to create a general document. The previous interface has no authoring control. 3. With authoring restored, submit a new document, keep typing, and switch to Properties before the request finishes. 4. Reopen Artifacts. The previous draft path restores a new form despite the successful create. Related public work: #17598 adds document copy/download and #18157 adds image galleries. The selected actual merged parent is `f1d780838218f975064f38333946805e0ec103a5` from #18427. It includes the stable profile-entry ordering repair and the exact recovery-fixture prerequisite. The resulting PR diff contains 32 document and reporting files. ## What Changed - Reuse `IssueDocumentsSection` for general documents in default-chat Artifacts. Keep Classic authoring in its existing task body. Ordinary tasks, conversations and Coordinator mode share the same editor options and signed-in permission wrapper. - Preserve Plans, file-review documents, attachments, uploads and work products. Hide duplicate general-document cards. Use the same general-document key rules for visible rows, validation and refused writes. - Forward image uploads, mentions, references, author maps and feedback to both editor mounts. Keep empty mention/project values stable while queries load. - Back up unfinished forms within their signed-in account, task and editor scope. Preserve text entered before its key. Record a successful create on its matching retained newer buffer, including after unmount or an early reopen. Preserve newer title/body edits, real remote conflicts and matching cancellation cleanup. - Add the model-free `task-document-persistence` journey. Setup creates only an owned empty backlog task. The browser creates and edits the document; normal APIs independently check company, task, actor, exact content and both revision records. - Reopen the owned task through the explicit Backlog view, then reload it. Require exact UI readback and zero native runs. Add only the Doc registration, definition-hash inputs, report mapping and four closed checks to the shared harness. - Count explicit zero-run browser receipts after private journey metadata is removed for publication. Preserve missing billing evidence, positive usage and native/protocol accounting. - Update browser assertions for the restored inline editor while retaining exact download bytes, revision collisions, paused-state and no-run checks. Update the feature map, usage docs and shared editor stories. ## Verification - Latest focused checks use official Node 24.22.2: 568 journey, catalog, report, billing and attempt-accounting tests across 25 suites pass. All four admitted journeys remain registered. The seven relevant UI suites pass 529 tests, including the task-page container after the parent merge. Maps, token gates, the PR-scope diff check and the owned credential-pattern scan pass. The normal QA import preflight also passes with an empty environment and network access blocked. - Billing regression: before the repair, the explicit empty public receipt counted one run, and two positive-usage receipts were discarded by browser metadata. Those three cases fail before and pass after the repair. Two additional red regressions cover aggregate usage beside an empty run list. That ambiguous shape now keeps an unavailable receipt and a minimum count of one. Missing receipts retain unavailable billing, and native/protocol one-or-more run totals retain their measurements. - The previous fresh CI run failed its unchanged snapshot-only recovery fixture: one global sweep did not call its callback. The original failure remains recorded. An isolated PGlite probe with all canonical migrations exercises the real reconciler: it processes two earlier sources, then reaches the owned third source on the next sweep. It preserves source evidence, files and run IDs. This proves bounded pagination, not full native snapshot repair. - Local native recovery tests could not start PostgreSQL because the host semaphore limit was reached. That run executed zero cases. Canonical Linux CI later passed all 143 fixture cases, including the original snapshot-only case and the new competing-candidate control. The exact fixture repair is now inherited from #18427 and is absent from this PR diff. - The portable oracle rejects 19 stale, missing, foreign or incorrect outcomes. Result validation also rejects model/provider metadata and native runs for this case. - Five real Artifacts-to-Properties unmount regressions fail before the repair and pass after it. They cover both editor modes, reopening before and after completion, later typing, and a foreign revision that must still conflict. The edit-storage failure case also checks that Cancel clears the promoted fallback copy. - UI coverage includes default chat, Classic, conversations, Coordinator, Plan rendering, document-card deduplication, reserved keys, pending-query stability, permissions and rich editor options. - Earlier real browser checks passed five upload cases and two artifact-arrival cases. They retain exact CSV bytes, original/revised content, collision, paused-state and no-run assertions. Current-head normal CI runs the updated browser suites again. - The branch merges only the selected actual parent `f1d780838218f975064f38333946805e0ec103a5`. The merge has no conflicts or manual resolution. The SQL service and recovery fixture match that parent exactly. The own document editor, persistence flow, saved-state oracle and billing repair retain their previously qualified source blobs. Follow-up, clarification, question/resumption, screenshot helpers and the Steering map are byte-identical to the merged parent. Existing project/search/question/follow-up registrations remain present. - Required document source blobs remain exact: Doc UI `22e80663b053363e5f75e1432dd9043dc0e70243`, its unmount tests `16f3f4ed0dc9b11aaff129e06e6cf2bbf4522713`, readiness helper `800a747284db19b8484650b6759e8bd1f356839d`, and budget service `3a15d489f0aa55c22e8f7fe6a6bc6a0e14e5914e`. - Local typing is limited by borrowed declarations from another checkout. They omit current `skill_test` and sandbox creation-operation fields. No dependency declarations or repository config were changed. The authoritative clean CI Typecheck + Release Registry gate passes on exact head `2ffce29827ae170420d740940c1c069c7225a74c`. - Full monorepo typecheck, tests and build were not run locally. Borrowed build declarations from another checkout limit local typing. Authoritative [clean CI](https://github.com/paperclipai/paperclip/actions/runs/41171717008) passes on exact head `82d15946c5844d3ae41c6ade62fa224053f20a9a`: 55 successful checks, 3 skipped and 1 neutral, with zero pending or failed checks. Typecheck, build, Runner, server and browser lanes pass. Greptile gives that same head a fresh 5/5. All 15 review threads are resolved, with no further thread page. The PR is mergeable and clean. - Failure history remains recorded. The earlier recovery failure was repaired with a bounded, owned-source fixture. A later CI worker shut down after 2,375 Runner unit tests passed. Its one authorized narrow retry passed those tests and all five native integration suites. The separate AgentConfig browser failure was reproduced and fixed by the merged #18427 parent. These earlier passes do not replace current-head CI. - Final-source live qualification passed on exact harness and served application `82d15946c5844d3ae41c6ade62fa224053f20a9a`. QA run `41177696858` completed all four closed checks and all five phases with zero native runs. Cleanup passed; the durable manifest is complete with no pending evidence. The coordinator verified the original grade and manifest. - The [public qualification report](https://pages.paperclip.ing/evals-qa-smoke-41177696858-1/) records zero agent runs, empty run IDs and empty run usage. Pricing remains unavailable and billing remains incomplete; the report does not claim measured zero cost. - The earlier harness `36c8d9ddd76d0b7ddb28a9a6c674d9e538f6df88` pass against application `2ffce29827ae170420d740940c1c069c7225a74c` remains historical evidence. It is separate from the final-source measurement above. - The selected parent brings profile-entry ordering, proactive chat, a database migration, Runner updates, and task-page chat/read behavior. Its new task-page container is included in the final application measurement. The coordinator retains the original evidence and owns the primary merge and daily-plan admission. ## Risks Artifacts now mounts the existing shared editor. Its autosave, conflict, revision, annotation and server access contracts remain in place. Tests cover both interface modes, shared scopes, later remote writes and unavailable edit storage. Plans and generated file-review documents keep their existing flows. The PR adds no database schema or API change. The shared billing repair requires explicit captured empty run IDs and an unambiguous usage object containing only an empty run list for public browser-only receipts. Other fields beside an empty list keep conservative unavailable billing. Missing measurements remain unavailable. Contradicting positive usage remains counted. The inherited recovery fixture keeps the production sweep limit and its source, readback and no-agent-work assertions. The live journey must fail when authoring controls or saved outcomes are absent. It must not create the expected document through setup APIs or switch to another interface to pass. Final deployment verification must retain the actual app and harness revisions separately. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8c9dd1e444 |
test(runner): qualify Kimi Code lifecycle and recovery (#18029)
Add the Kimi lifecycle and recovery qualification harness. Preserve original grades and cleanup evidence, strict native ownership audits, bounded startup and cleanup, and early rejection of unsupported local fault execution before credentials or resources. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b866fa4bdc |
feat(runner): add pinned Kimi Code native runtime (#18028)
Add the pinned Kimi Code runtime, native instruction and model admission, scoped credentials, and pending-candidate recovery contracts. Retain current master release attestations and provider-pack compatibility. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d06c5e0eff |
fix(evals): preserve opaque TestOps parameters (#18394)
## Thinking Path > - Paperclip verifies real agent work through Product E2E tests. > - The fixed Chat feature workflow uses TestOps to select and report one case. > - TestOps exposes job parameters as ID/value pairs. > - The schema does not establish that those IDs identify global environment variables. > - The reader used that unsupported relationship to infer parameter names. > - This pull request keeps IDs opaque, preserves the strict allowed values, and records closed read diagnostics. ## Linked Issues or Issue Description Refs: #18322, #18375, #18385. **What happened?** A zero-model bootstrap returned HTTP 404 during context readback. The exact failed GET was not retained and remains unknown. Schema review also found no established relationship between job-parameter IDs and global environment-variable IDs, although the reader queried that namespace to label parameters. **Expected behavior** Read only the existing job-run, launch, and job records. Preserve the job's exact permitted ID/value vector without inventing parameter names. Record which fixed GET class failed, its positive resource ID, state, and available HTTP status without raw responses or URLs. **Steps to reproduce** Use the actual API helpers with a job-parameter ID that has no global environment-variable definition. The new pure controls reject an unexpected definition lookup and verify the opaque vector and diagnostics without a backend or model call. **Paperclip version or commit** This change starts from `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` after #18385. **Deployment mode** GitHub Actions metadata setup in the protected TestOps reporting environment. ## What Changed - Remove the unsupported global environment-variable lookup from job-parameter handling. - Preserve an exact positive ID and `refs/heads/master` value. Use the fixed workflow's existing `evaluate` default. - Treat an operation parameter as such only when its ID matches the protected configured ID and its value is allowed. Keep rejecting empty, duplicate, conflicting, or unknown vectors before writes. - Project only an opaque ID with a fixed-default-ref kind and value, or an unresolved kind with the value withheld. - Add closed diagnostics for the existing job-run, launch, and job GETs. A successful call keeps HTTP status null because the API abstraction does not expose the status. - Sync one reporter-test assertion and document opaque IDs. Reporter runtime and workflow source are unchanged. ## Verification - Four new actual-helper controls fail on the parent. - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs tests/runner-e2e/testops-chat-reporter.test.mjs`: 58 affected pure/mock/FS controls pass on Node 26.4.0. - Three MJS syntax checks and `git diff --check` pass. TypeScript noEmit was not run for this MJS/documentation change. - Two intermediate exact-projection assertion failures were retained before the assertions were synced to the new diagnostic contract. - [Normal Node 24 CI](https://github.com/paperclipai/paperclip/actions/runs/41137009579) passes on exact head `3c256fb8f7559093c5fb063170e26c01b6e8a0a4`: 48 jobs passed and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed this exact head at 5/5 with no actionable findings or unresolved threads. At this verification cut, the optional Contributor trust check was still in progress; it is not an effective required status context. - No actual CLI, API, provider, model, app, database, build, install, or additional bootstrap was run to validate this change locally. ## Risks The original HTTP 404 route remains unknown. Removing an unsupported schema assumption does not prove that it caused that failure. The live job mapping and dispatch remain unqualified. The strict source, actor, selection, run-number, nonce, once-only send, private custody, case, model, grader, and clock checks are unchanged. Unknown metadata still stops. This PR adds no endpoint, workflow mode, permission, credential path, or passing result. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8ff812cdde |
feat(chat): deliver proactive task messages to agent chats (#18374)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat gives each person a persistent conversation with each
agent.
> - Tasks and routines produce results outside that conversation.
> - Those runs need a safe way to bring a requested result back to the
person.
> - The server must select the agent's own chat and check the source
task's access rules.
> - This pull request adds proactive chat messages and unread
indicators.
> - People can see the result, open its source task, and reply in the
same chat.
## Linked Issues or Issue Description
**Subsystem affected**
Server, native runner tools, shared contracts, database, and UI.
**Problem or motivation**
An agent can report progress on a task or comment on an authorized task.
It cannot send a routine result to its own standing chat with the
responsible person. A morning review routine therefore cannot leave a
visible digest in that person's chat.
**Proposed solution**
Add `send_chat_message({body, idempotencyKey})` and `POST
/api/agents/me/chat/messages`. The server selects the company, agent,
and responsible person from the active run. It saves an attributed
comment and audit record in one transaction. The chat shows a
source-task link and per-person unread state.
**Alternatives considered**
A destination parameter on a generic comment tool would expose
unnecessary routing authority. A new delivery worker would add another
durable delivery path. This change uses a dedicated tool and the
existing comment and live-event paths.
**Roadmap alignment**
This extends the experimental Agent Chat capability in `ROADMAP.md`. The
existing `enableAgentChat` setting controls it. Related public work:
Refs #13282, Refs #13283, and Refs #14408. Searches for proactive chat
and unread chat found no duplicate PR.
## What Changed
- Add a focused messaging service shared by the native tool and HTTP
route. It checks current run ownership, execution mode, recipient
membership, task access, and task privacy.
- Reuse the canonical company/agent/person conversation. Save the
comment and audit record together. Keep task completion and chat
execution separate from delivery.
- Add source-task provenance and source-scoped retry keys. Identical
retries return the saved receipt. Changed content returns a conflict.
- Enforce the conversation's fixed agent at the shared comment-write
boundary. Keep normal reply cancellation and session checks.
- Add chat unread state, accessible agent dots, aggregate Chat
indicators, and visible-message read cutoffs. Refresh through existing
live events.
- Add the request-key constraint, native tool contracts, documentation,
capability maps, stories, and regression coverage.
## Verification
- Local delivery/security proof: all 38 proactive-message cases pass.
Eight credential-scope regressions and four quota regressions fail
against the prior source. The repaired delivery and canonical
cross-issue cap suites pass all 49 cases; all 31 OpenAPI tests, server
typecheck, and capability-map checks pass. The shared per-run cap covers
native and HTTP sends, mixed ordinary mutations, concurrent last-slot
contention, and retry/recovery without double charging.
- Credential tests preserve the authenticated person and authorizer;
reject bridge/test credentials and persisted skill-test runs after
task-mode changes; and recheck API-key revocation, scope, and
responsible person after destination locking. The earlier
delivery/native-authority suites passed all 74 cases before the four
quota additions.
- Four PostgreSQL contention cases cover native and HTTP sends plus
retries during a native chat mutation. All four deadlock against the
earlier source and pass with destination-before-source/run locking.
Recovery and human-reply delivery enqueue only the human comment; a
proactive digest never wakes the chat agent.
- Passed 108 companion conversation, native-authority, and OpenAPI
tests; 112 runner catalog/workflow-report tests; 47 completion-delivery
tests; and 25 project-handoff tests. Focused runner tests cover native
tool availability without generic API tools.
- All 327 affected UI tests, UI typecheck, token gates, and catalog
checks pass. The read hook retries at most three times, respects server
retry delay, coalesces only seen cutoffs, recovers on reconnect, and
cancels on scope changes.
- Real browser journeys with a deterministic process agent verify
routine-to-chat delivery, first-chat creation, unread state across
reload, the source link, secondary chat navigation, and a follow-up
reply referring to the digest. This does not qualify live GitHub
retrieval or model output quality.
- The Storybook showcase uses production components with deterministic
network fixtures. Rendered interactions pass for the landing, ordinary
read success, transient read recovery, and persistent failure after
bounded retries. Both Chat navigation variants update from two unread
chats to one while only Kodi's dot clears. Mobile and light variants
were reviewed. The existing Coordinator Recents label contrast finding
is unchanged; no Linux visual baseline archive exists.
- Current master's independent sandbox migration retains 0418.
Regenerated the feature migration as 0419 from the combined schema and
retained exactly the newest five snapshots; numbering/safety, replay,
and schema-drift checks pass.
- Workspace typecheck and build passed before PR preparation; canonical
server/UI typechecks passed again on the combined tree. Migration
`0419_slippery_impossible_man.sql` passes safety and schema drift checks
on disposable PostgreSQL, including interrupted index repair, concurrent
migrators, one-time owner read-state backfill, preserved receipts, and
two SQL replays. A default five-second local snapshot comparison timed
out; a diagnostic 30-second run passed without drift.
- Earlier failures are retained in the record: full local `pnpm
test:run` and broad shards encountered embedded-PostgreSQL startup,
timing, and provider-recovery failures. Thirteen selected
external-channel cases pass in isolation on the original and changed
sources; an unordered comment-row assertion now explicitly orders both
reads. The earlier route registry, native tool count, runner catalog,
and accessible Chat badge-name failures were fixed and passed full
remote CI on `920b3bb7db`.
- Repeated managed-skill server8 failures came from CI's newer synthetic
merge: its Cloud guard correctly rejected the old local-disk fixture.
Master independently repaired the fixture in #18385; this branch keeps
that repair unchanged. All 57 managed-skill and Cloud prerequisite cases
pass on the combined tree, and that fixture has no remaining PR diff.
- Current reviewed head `dc16f314739a6d977bed21cb1446d0e80a541578`:
Greptile 5/5 with no findings, Superagent security assessment passed,
advisory Paperclip Review 5/5 with no findings, and zero open review
threads. All 55 current-head check runs pass, with five expected skips
and the advisory review neutral at 5/5; Snyk also passes. No checks are
pending or failing. Protected squash auto-merge is enabled and is
waiting for required code-owner approval. A normal exact-head merge was
refused by the base-branch policy; no admin bypass was used. The
previous head's Superagent quota finding is fixed and covered above.
## Risks
- Chat remains company-visible. Delivery rejects a source task with
private-task, project, or ancestor restrictions.
- Read state uses a comment timestamp cutoff. The server retains
database precision and never moves a read receipt backward.
- The new unique constraint changes comment query planning. An existing
regression now uses an explicit row order. The migration builds the
unique index concurrently, then takes a brief table lock to attach it.
It uses the existing top-level migration and interrupted-index recovery
path.
- The upgrade backfills missing chat-owner read-state rows at the latest
existing agent comment. Existing read receipts remain intact. New chats
can still be unread before the human participates.
- Quota observation commits atomically with delivery. A rejected send
rolls back its attempted denial audit row and any first-chat creation;
the canonical warning remains in server logs, and previously spent quota
remains enforced. Identical/conflicting retries and receipt recovery
spend no extra slot.
- This version delivers in-app only. It adds no external-channel, push,
or email delivery.
- Local full-suite acceptance remains limited by the failures recorded
above. The feature stays behind `enableAgentChat`.
## Model Used
OpenAI Codex. PR preparation and CI work used `gpt-6.1-sol` with tool
use and code execution. The earlier implementation also used Codex, but
its exact model ID was not recorded in the visible session. The
context-window size was not provided.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass; the broad local
failures are recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
github-actions[bot]andlockfile-bot
|
4eaa435bf4 |
chore(lockfile): refresh pnpm-lock.yaml (#18414)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
1353a232f2 |
test(e2e): add a portable active follow-up journey (#18345)
## Thinking Path > - Paperclip lets people direct agents while their work is active. > - Product E2E checks must prove that saved user input reaches the intended work. > - The existing active chat story depends on a file in a local agent workspace. > - That setup cannot qualify an attached deployment with remote agent execution. > - This change adds one shared active follow-up journey through the normal task UI and API. > - Independent native operation, saved input, final reply, and cleanup checks make the result inspectable across targets. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for a user follow-up sent while a native agent is working. **Current behavior** The classic chat story holds work open with a local file. The shared deployment-targeted harness has no active follow-up journey. A running heartbeat alone cannot prove that work was active when input arrived. **Proposed behavior** Run one bounded command through the real native provider. Observe its active execution before and after one follow-up is saved. Require one final reply with the late requirement, correct task and agent identity, an empty input queue, and visible readback after reload. Use the same story on managed and attached targets. ## What Changed - Add a shared `followup-while-running` journey with normal task creation and reply submission. - Require a bound native turn and pending shell execution at the input boundary. Retain a closed receipt with source identity and a row hash. - Accept one steered work run or one normal successor (1–2 native runs). Preserve every other journey's declared run contract. - Reuse the saved native final reply observer, run binding checks, usage collection, target lifecycle and private evidence path. - Accept canonical builtin/process execution events independent of provider tool display names; calibrate against real OpenCode, Codex and ACP mappers. Reject Paperclip semantic tool names across native wrappers/prefixes and unnamed or ambiguous builtin calls. - Recognize only the observed legacy PRP v1 builtin/bash/unknown/readOnly=false command shape and fail promptly when owned work finishes before a live boundary can be observed. - Ready the task page/composer before the observed command boundary, use a bounded 90-second command, and preserve active-after-persistence proof. - Add adversarial oracle and retained-result calibration. Register the journey, include its sources in the definition hash, and document its coverage. - Keep six real browser document-readiness checks and verify shared-deadline behavior with a deterministic clock/state test that rejects timeout-reset mutations. ## Verification Exact current head `e72e20ad111bd94728d38606015d1fdac3c92929` passed [full CI run 41142552195](https://github.com/paperclipai/paperclip/actions/runs/41142552195): all 49 workflow jobs reached accepted terminal results, including typecheck, build, every test lane and aggregate verify. All 59 reported checks are terminal: 55 success, three expected skips and one neutral 5/5 Paperclip review. Snyk passed. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/18345#issuecomment-6528913235) explicitly names this exact head and gives 5/5. No review threads remain open. The PR is mergeable. These source gates do not establish a live QA pass at this head. Fresh protected QA at this exact source and the separately recorded deployed application/runtime remains required before primary merge and daily admission. Historical live QA staging **PASS** on qualified commit `af9e091b15b5e24b6b2cc1b424523430cc2ab024`: 6/6 checks, all five phases including cleanup, two native work runs, and the configured model/environment check passed. The [sanitized qualification report](https://pages.paperclip.ing/evals-qa-smoke-41122620146-1/) records that exact measurement. Final integration head `e72e20ad111bd94728d38606015d1fdac3c92929` merges exact actual squash `18b1637b6371a6a6adade857e2a73654ee626fef` from [#18254](https://github.com/paperclipai/paperclip/pull/18254) into previous head `e54d64743964b1a19d165537bebbe62c50389a23`. The merge had no conflicts. All 18 files in this PR's change set, the complete 45-file shared journey directory, and six shared native/evidence helpers retain their exact previous-head Git blobs: 59 scoped files in total. The production clarification helper remains blob `800a747284db19b8484650b6759e8bd1f356839d`. Follow-up 1–2 runs, question 1–3 runs, older exact counts, owned native event checks, active-before/after persistence proof, cleanup/accounting and feature mappings remain unchanged. Source integration head `e55d74491f5bcd99c1455e9ace02bd7ad7319fae` integrates exact actual squash `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` from [#18385](https://github.com/paperclipai/paperclip/pull/18385). The merge had no conflicts. All 17 files in this PR's change set and six shared native/evidence helpers retain their exact Git blobs from previous head `162e341cda5b627639b486b4019a1675bb018ea2`. This preserves the follow-up and question readiness checks, feature mappings, six closed follow-up check IDs, immutable before/after active-operation proof, follow-up 1–2 runs, question 1–3 runs and older exact counts. Previously reviewed head `e54d64743964b1a19d165537bebbe62c50389a23` adds a separate test-only reliability repair on top of that source integration. It replaces one scheduler-sensitive browser case with a deterministic Vitest clock and locator-state model that calls the actual production helper. The six other real Chromium checks remain byte-for-byte unchanged and pass **6/6**. Focused unit suites pass **51/51**, including the deterministic deadline case. This is six browser checks plus focused unit coverage, not seven browser checks. The model makes the panel ready at 100 ms, the supplied answer ready at 600 ms, and full valid content ready at 1,300 ms. It requires rejection at the shared 1,000 ms deadline and records immutable rejection state before simulating a 10-second scheduler stall. Both restarted-timeout mutations fail: resetting all waits wrongly resolves at 1,300 ms; resetting matcher timeouts while keeping the final real deadline guard rejects only at 1,300 ms with valid content present. The genuine helper rejects at 1,000 ms with the answer still incomplete. The production helper is unchanged, Git blob `800a747284db19b8484650b6759e8bd1f356839d`. That two-file test-only patch changed no journey, application, run, accounting, model or runtime behavior. Its six-browser/51-unit results are prior-head evidence; the complete patch is unchanged in the new integration. The final source also retains the actual merged question-primary squash `d9354bbfc35ef74333e90e596ce21dc8e59149b9` from [#18346](https://github.com/paperclipai/paperclip/pull/18346). Later commits `409117451da260a03fc28a3107c7808ecbe68f8f` and `ea3e7b1d1bed562351fcf4722c7175001b987b59` bound question document readiness to one campaign deadline and test that waits cannot restart it. Commit `162e341cda5b627639b486b4019a1675bb018ea2` adds question/follow-up report feature mappings and coverage. These files are unchanged by the final merge. The imported upstream source has real application changes: [#18334](https://github.com/paperclipai/paperclip/pull/18334) makes managed skill content durable and extends native skill-test contracts; [#18350](https://github.com/paperclipai/paperclip/pull/18350) requires managed Cloud shared storage and bootstrap prerequisites. #18385 updates TestOps job metadata and repairs their combined maintenance fixture. The new merge additionally imports [#18383](https://github.com/paperclipai/paperclip/pull/18383) at `81ecaf0654fe79a2a2dbda6588f033e39ee610e4`, which bounds failed lifecycle verification; [#18357](https://github.com/paperclipai/paperclip/pull/18357) at `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`, which persists sandbox-creation ownership and updates cleanup/adapter/plugin contracts; and #18254 at `18b1637b6371a6a6adade857e2a73654ee626fef`, which accepts a valid late Antigravity authorization URL while native authentication drains. This PR's diff against exact upstream `18b1637b6371a6a6adade857e2a73654ee626fef` contains only journey tests, report coverage and documentation. Harness source and deployed application/runtime identity are separate inputs. This source integration does not deploy the imported application changes or claim the QA target runs this application revision. This final SHA has not been measured by the historical run above: **fresh QA at this exact final head is required before primary merge**, followed by normal TestOps verification/admission. Fresh exact-head CI and review have passed as recorded above. The live measurement must record the actual separately deployed QA application/runtime identity. Seven whole-file blobs remain identical to the qualified SHA. All paths are relative to `tests/runner-e2e/`: | Journey/helper | Unchanged Git blob | | --- | --- | | `product-journeys/active-followup-evidence.ts` | `83a5d9fab52abb9849a40e83eca6b6a9c29bf587` | | `product-journeys/active-followup.test.ts` | `728160c610c18292d2cf06d4a87d25a8719f1734` | | `native-task-reply.ts` | `59d11f1a178dad9db3da8fa2342ff93643cb9922` | | `native-event-envelope.ts` | `305b0b456c59c511ee328e529ad574c0583075a0` | | `run-observations.ts` | `08c8ad184a821a7cd536926b3d7e3d1156a1e3da` | | `user-actions.ts` | `e9611348dc41e17899367dab2e3b36d3d1be9480` | | `stop-redirect-flow.ts` | `4580bd605a425bc72511e02d988492acbec8d60e` | The follow-up preservation comparison against historical `af9` has three explicit exceptions; no whole-file identity is claimed for them. The separately documented question readiness and report additions above also require final-source verification: 1. `product-journeys/agent-flows.ts` adds `question-answer-resume` to the `AgentJourney` type exclusion, alongside follow-up. Its remaining contents and runtime helper bodies are byte-for-byte unchanged. Qualified blob `a0afaa968c1d854476159fef21de7b6dd50a4a98`; final blob `0cc797c0ae594e7769338e63a80c4f67e29e4a54`. 2. The common run-count helper is centralized in `success-contract.ts` and re-exported by the catalog. It accepts follow-up **1–2**, clarification **1–3**, and each older paid journey's exact count. The follow-up catalog entry, maximum of two runs, six required checks, native dispatch, collection and retained-run contract remain byte-for-byte unchanged. The helper's text differs; its preserved count semantics are verified explicitly. 3. `product-journeys/active-followup-flow.ts` changes exactly one composer readiness assertion from Playwright's implicit five-second default to an explicit 30-second timeout, matching `submitTaskReply` for attached targets. Prompts, actions, native evidence, before/after persistence proof and grading otherwise compare byte-for-byte unchanged. Qualified blob `64392329198a1a4d9828e32834d4507e175216e6`; final blob `2746b5ba89e2af26b4097ba7897913186786ac5d`. This correction is why a fresh final-head QA run is required. - **310 focused tests passed at current final head `e72e20ad111bd94728d38606015d1fdac3c92929`**: 286 journey/evidence/catalog/report/shared-deadline tests in 14 files and 24 native authentication/real-child stream-order tests in two files. Both invocations used Node24 and one Vitest worker. The earlier `e55` integration also passed 365 focused tests, including 40 Cloud configuration and 40 TestOps metadata cases; those results remain prior-source evidence. - **157 explicit count assertions passed again at the current final head**, including both interactive ranges, older exact counts, invalid/fractional/non-number values and catalog/contract export identity. The maximum bounds and six closed follow-up check IDs also matched. - Maps and diff checks passed. The official QA runtime-import preflight passed on Node24 at the current final head without network access. It uses a dummy adapter and no credentials. It does not prove target connectivity, sign-in, model readiness or deployed application/runtime identity. Those require the normal protected QA run. No model/provider work or optional Runner release was dispatched. - Current local runtime dependencies use ignored isolated workspace-source views. No shared dependencies or compilers were changed. A previous local typecheck attempt hit stale reused Runner declarations. No full repository typecheck, test run or build was repeated locally; the successful exact-head full CI run above now supplies these gates. - CI on previous head `162e341cda5b627639b486b4019a1675bb018ea2` failed the Cloud/local maintenance fixture in [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133716802/job/124331643100), plus dependent verify. The actual tested merge combined that head with master `198218bf554877fe3b7a83742eefb3b8f12392c6`. A Cloud child selected local-disk storage and hit the new shared-storage policy. This was a deterministic upstream fixture mismatch; no unchanged failed-job retry was used. - The actual #18385 repair configures a valid Cloud child, verifies local bytes independently, checks the expected provider mismatch and retains exact stored-config readback. Its own final head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57` passed 59 checks, including [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133876636/job/124331450635). That prior result supports the repair; it does not replace this final head's fresh CI. Separate repair candidate #18392 and its source/evidence are retained as superseded history. - A local attempt to run the database fixture could not start PostgreSQL because the host exhausted macOS semaphores. A first setup attempt also failed package resolution before test execution. Skipped database cases and these setup failures are not passes. No kernel settings, shared processes or databases were changed. - The complete previous-head CI run [41138618869](https://github.com/paperclipai/paperclip/actions/runs/41138618869) finished with one underlying failure: [Runner Vitest shard 1](https://github.com/paperclipai/paperclip/actions/runs/41138618869/job/124347859445) expected native Antigravity login success but observed failure at `antigravity-auth.test.ts:101`. The lane had one failed test, 1,994 passing tests and eight skipped tests. Aggregate verify failed only because of this lane. All other 57 checks were accepted, including build, typecheck, all server/browser suites, Runner shard 2, Snyk and exact-head 5/5 reviews. No unchanged retry was requested. - The independently reviewed #18254 repair removes an invalid cross-pipe ordering assumption: a valid URL on stderr can arrive after the authenticate response on stdout during drain. Its real-child regression forces both orders. The current final source imports that actual squash; the 24 focused authentication cases pass here. Fresh full CI and exact-head review have now passed for this source, as recorded above. The earlier failed result is not relabeled. - The original live compatibility failure is retained: an older PRP v1 pack labeled builtin `bash` as operation `unknown`, causing the observer to miss the command and never send the follow-up. Its exact shape was reproduced red before repair. Generic unknown operations, semantic tools, wrappers, read-only work and foreign provenance remain rejected. The historical qualified pass used the repaired oracle; no failed result was relabeled. - Earlier CI on `6fad965d69` hit an unchanged heartbeat test's three-second adapter-call wait; its original failure and one narrow rerun remain recorded. The qualified SHA subsequently passed CI. An earlier report-catalog subprocess failure is retained. The final focused Node24 run now includes that suite and passes after the documented report feature mappings. ## Risks The fixture asks the native agent to run one bounded `sleep 90` command. The observer must see actual active native execution; a fast or noncompliant turn fails rather than passing from elapsed time. A completed result requires one or two bounded native work runs, source attribution, configured model/environment, and collection. Unknown provider prices stay unknown. This change does not alter production execution or grant new access. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with available version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs; existing follow-up execution fixes do not provide this deployment-targeted test - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the new focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8595efc20b |
fix(evals): support official TestOps job metadata (#18385)
## Thinking Path > - Paperclip manages AI agents and verifies real work through Product E2E tests. > - The fixed Chat feature workflow uses TestOps to select and report one backlog case. > - Official registration exposes a workflow-query URL and a Branch parameter. > - The existing reader accepts only workflow-file or numeric URLs, and the admin launcher requires an operation parameter. > - These assumptions reject valid official metadata before the feature can run. > - This pull request supports the exact observed form while preserving source, selection, and once-only execution checks. ## Linked Issues or Issue Description Refs: #18322, #18375. **What happened?** The second zero-model bootstrap stopped with a generic job-mismatch code. Ordinary TestOps UI inspection then showed an official workflow-query URL outside the accepted literals and a Branch-only job. The exact first failed predicate in that bootstrap remains unknown. **Expected behavior** Accept that exact fixed-workflow URL. Preserve the existing master Branch value when the job has no operation parameter. Keep rejecting foreign URLs, unknown parameters, wrong branches, and conflicting operation IDs before mutation. **Steps to reproduce** Pass the official query URL and Branch-only metadata through the existing bootstrap, selection, and admin-launch owners. The new pure controls exercise those actual helpers without a CLI, backend, or model call. **Paperclip version or commit** This change starts from `0e15c92073a09e50df99190a37f69a86286b989b` after #18375. **Deployment mode** GitHub Actions and the protected TestOps reporting environment. ## What Changed - Share one exact URL predicate across bootstrap, evaluation selection, and admin preparation. Add only the observed official workflow-query literal. - Support Branch-only, operation-only, and Branch-plus-operation jobs with strict definition IDs, names, values, and duplicate checks. - Preserve the exact master Branch value. Set an existing operation parameter to `evaluate`; otherwise use the fixed workflow's existing default. - Use the same checked parameter vector for job and plan writes and their readbacks. - Report distinct fixed bootstrap predicate codes without response values. - Document when the protected operation-parameter ID is required. - Align one existing Cloud-maintenance test with the inherited managed-Cloud S3 policy. Preserve the same-version local-storage proof in the parent, use valid synthetic Cloud settings in the child, and check the expected provider mismatch without S3 requests. Retain the no-seed and unchanged persisted-row assertions. ## Verification - URL controls: three failures on the parent; all pass with the correction. - Branch controls: two failures and one correct rejection on the predecessor; all pass with the correction. - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs`: 40 owning pure/mock controls pass on Node 26.4.0. - Two MJS syntax checks and `git diff --check` pass. TypeScript noEmit was not run for this MJS/documentation change. - The first [normal CI run](https://github.com/paperclipai/paperclip/actions/runs/41131094974) failed only in the managed-skill Cloud fixture and its dependent verification gate: 46 jobs passed, two failed, and one skipped. The tested merge included a new upstream policy that rejects the test's local-disk Cloud configuration. The exact caught runtime exception remains unknown. - Three pure controls using the actual policy validator pass: the old invalid Cloud fixture rejects, the new synthetic S3 fixture admits, and an invalid master key rejects. An initial tsx wrapper IPC permission error executed zero controls and remains a separate setup failure. - The PostgreSQL/maintenance-child test was not run locally. On new-head AWS CI, `managed-skill-content.test.ts` passed all 17 tests in server shard 9. - [Normal CI](https://github.com/paperclipai/paperclip/actions/runs/41133876636) is green on exact head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57`: 48 jobs passed and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed this exact head at 5/5 with no actionable findings or unresolved review threads. - No actual CLI, backend, model, provider, app, database, build, or bootstrap was run to validate this correction locally. ## Risks The exact original bootstrap predicate is still unknown. This fixes a demonstrated metadata false negative and preserves the prior failed run. Actual backend parameter types and integration dispatch still need verification. Unknown metadata stops before the once marker and writes. This PR changes no workflow, credential, case, model, grader, clock, or private-evidence policy. It does not claim a passing feature result. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0e15c92073 |
feat(skills): make managed content independent of web-server disk (#18334)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use installed skills and exact version pins during execution. > - Managed skills still depended on mutable directories on the web server. > - A replacement server could lose acknowledged edits or remove an unassigned skill whose directory was absent. > - This pull request makes immutable versions the authority for managed content and installed repository reads. > - The server can rebuild delivery files from durable content after replacement. > - This establishes skills readiness for ephemeral web servers. General server draining and workspace recovery remain separate work. ## Linked Issues or Issue Description **Subsystem affected** Skills services, shared contracts, database receipts, Skill Studio, and runtime delivery. **Problem or motivation** A fresh web server must discover, read, edit, and deliver protected installed skills using the same database, object storage, secrets, and supported application artifacts. It must not need its predecessor's files or process memory. **Proposed solution** Use `managed` ownership with a null source locator. Read installed content by immutable version. Prepare packages before a short publication transaction. Persist execution selections before materialization. Recover exact pinned runtime assets and reject deletion while active or recovering pins exist. **Alternatives considered** Reconstructing a mutable installed directory preserves the old source-of-truth ambiguity. Permanent dual writes create two competing authorities. This change uses one managed model after a bounded legacy conversion. **Roadmap alignment** This improves the existing Skills Manager, Studio, version history, and Cloud execution capabilities. The maintainer supplied the implementation contract. This does not add a separate skills product or deploy fleet changes. Related work: #18082 supplies immutable package storage. #17273 addresses edits outside the API for linked local skills; this PR keeps that source category separate. #2648 is a closed earlier ephemeral-filesystem proposal. ## What Changed - Add managed ownership and focused content publication and execution materialization modules. Keep the archive format and package storage tables. - Publish managed create, save, delete, fork, rename, restore, and catalog content with version checks. Add durable create receipts. Preserve file-save receipt responses. - Return file version identities. Keep Studio drafts after a conflict and provide an explicit reload action. A delayed reload cannot overwrite newer typing or navigation. - Persist exact runtime selections, including empty selections. Rebuild pinned assets and verify their admitted digests on a replacement server. - Preserve source access on forks and disconnected copies. Keep the existing workspace policy category for managed skills. - Add bounded conversion, eligibility, and independent verification phases. Load persisted Cloud configuration in the standalone CLI without seeding or changing it. Reject partial current versions against installed inventories; recover only complete saved content or the exact source. Document storage migration blockers and the compatible rollback floor. - Add separate-process PostgreSQL/S3 qualification and update domain, retention, policy, and UI regressions. - Pass the plugin worker manager into Studio execution, admit restricted `skill_test` mode through native Runner contracts, and preserve admitted scope across native and REST tools. Native tests cannot create ordinary tasks, browse company data, or use external tools. - Select the exact durable Studio revision even when unassigned or Beta skills is off. Persist Studio output and audit atomically with native task completion, preserving the result on replay. ## Verification - Final revision `13e8af480b88ba6fea5737bac4bd81104d604792` passed full CI: all 49 workflow jobs completed successfully ([run 41127396098](https://github.com/paperclipai/paperclip/actions/runs/41127396098)). This includes typecheck, test suites, Runner checks, browser checks and build. Greptile review 15 is 5/5 on that same revision, with zero unresolved threads. The PR is ready for review. - Local workspace typecheck and build passed after integrating master. Focused checks passed: 33 execution/completion cases, 52 Runner HTTP cases, 16 managed-content cases, 17 Studio cases, the new real-process CLI regression, map checks and UI token gates. The CLI regression proves inspection does not seed missing Cloud configuration. Server typecheck passed after that fix. - The independent-process PostgreSQL/S3 matrix passed through normal HTTP skill routes. It covers A → B → C replacement, lost commit responses, upload crashes, concurrent saves, binary contents and executable delivery. Metadata listing and warm selected delivery retain zero object downloads. Local timing measurements are recorded; they do not establish hosted latency. - Hosted A → B → C passed on the owned disposable test tenant with external Postgres, S3 and a real remote Paperclip Computer. Predecessor temporary homes were absent. C read and executed B's acknowledged managed save. New runs used repository v1 while v2 was installed. Hashes matched an independently checked exact upstream commit. - Compatible image rollback to `6518b323` passed. Studio retained the unassigned fork's C save. A fresh remote run executed the newer managed content and historical repository pin. The actual shell event confirmed expected hashes, stdout, exit 0 and mode 0555. - Runtime candidate `9200e3fcb4` deployed successfully ([Cloud run 41124147876](https://github.com/paperclipai/paperclip-cloud/actions/runs/41124147876)). The final Studio test delivered only the unassigned fork's exact v3, executed its script and persisted a successful output snapshot. Independent command-event inspection confirmed all three hashes, stdout, exit 0 and mode 0555. All 14 installations were version-backed. Independent verification checked all 25 versions: 23 remote objects, two bundled database snapshots, zero blockers. - The deployed image is `9200e3fcb4`. Later master integration, the client-only delayed-reload draft guard and the CLI bootstrap have separate automated proof. The CLI bootstrap was also exercised against the tenant's stored configuration and storage through the same read-only services. This PR does not claim that the deployed image is the final PR revision. - Live Studio checks exposed worker wiring, admission, access scope, exact selection and completion-publication gaps. They are fixed. Failed attempts remain in the evidence. The full local suite exceeded 80 minutes under host contention and was stopped. Later combined/isolated local attempts also timed out; they are not counted as passes. Full CI covers the integrated suites. Earlier CI found an obsolete restart-scope fixture, which was corrected and passed all 52 HTTP cases. - Reproduction and migration: [`doc/skill-ephemeral-web-servers.md`](doc/skill-ephemeral-web-servers.md). Source revisions, deployments, versions, run IDs, hashes and failure history: [`doc/plans/2026-10-30-ephemeral-web-skills.md`](doc/plans/2026-10-30-ephemeral-web-skills.md). ## Risks - Legacy conversion requires stopped admission and retired filesystem-authoritative writers. Pre-conversion binaries are not supported rollback targets afterward. - Cloud readiness requires remote object storage. Local provider receipts and provider mismatches remain blockers until objects are copied and verified. - Linked local sources retain filesystem semantics and are outside the cloud guarantee. - Cold reads still fetch the existing archive format. Warm runtime preparation retains the verified-cache path. - Bundled reconciliation preserves an installed head. Existing release snapshots remain selectable. An older server must not replace a newer head with its shipped files. - No production volumes, fleet defaults, or general deployment coordination change in this PR. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and browser use. The exact backend model ID and context-window size are not exposed in this session. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d9354bbfc3 |
Add a portable agent question and answer resumption journey (#18346)
## Thinking Path
> - Paperclip helps people manage agent work and keep human decisions
attached to tasks.
> - Product E2E journeys verify the browser, native agent, saved state,
and cleanup.
> - The classic suite already checks a choice question followed by a
text question.
> - Deployment-targeted journeys need the same behavior on an attached
instance.
> - Production questions can use provider input, the dedicated semantic
tool, or the authenticated native API fallback.
> - The basic journey must verify the original card and native
continuation for each supported path.
> - This pull request shares the scenario and browser actions, adds
strict API receipt evidence, and verifies the saved document after
reload.
> - Separate tool-documentation qualification and the original failed
measurements keep their existing scope and outcomes.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
It adds portable coverage for answering agent questions and resuming the
same task on an attached deployment.
**Current behavior**
The classic suite has the two-question scenario. The portable catalog
does not expose it. Its shared native-path grader recognizes provider
and dedicated semantic receipts, but rejects a valid completed native
API creation receipt.
**Proposed behavior**
Expose `question-answer-resume` as a deployment-targeted journey. Answer
one choice question and one text question in the browser. Require
server-owned creation evidence, unchanged forms, exact board answers,
bound continuation, an agent-authored task document, and readback after
reload.
**Reason and benefit**
The same story can qualify local and hosted targets through their normal
provider setup and cleanup. It catches lost answers, duplicate or
foreign work, premature output, and stale documents.
## What Changed
- Register the journey with a three-run maximum and positive model
allowance. Accept one to three attributed work runs at both collection
and retained-result boundaries.
- Share the existing two-question scenario, browser answer actions, and
behavioral grader. Keep separate tool-documentation grading unchanged.
- Recognize a completed native API creation receipt only when its HTTP
201 result retains the original pending card, exact
task/company/agent/source run, idempotency key, and full normalized
form. Require the receipt to remain unchanged after the answer and
through the final checkpoint. Missing, pending, foreign, or altered
proof still fails.
- Check signed-in board ownership, each native continuation, no
premature or extra output, document authorship, and the saved revision
after reload.
- Wait for the authenticated product region before screenshots. Recheck
origin and path after readiness. Bound readiness and capture by the
campaign deadline.
- Add closed stage-specific errors, catalog coverage, sanitized receipt
calibration, and documentation. Include the provider-form and
browser-question presentation helpers in the definition fingerprint.
## Verification
- Fresh QA staging qualification passed at
`7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks
passed, cleanup passed, and all three native runs used the configured QA
target and model. The [sanitized run
report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/)
records the measured pass. This is one successful qualification, not a
reliability claim.
- Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3`
contains the one main merge through
`fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of
the four-file qualified search fix
`4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity
matches. It preserves the project and search registrations and atomic
search readback. The question live qualification ran at the earlier SHA
above; the final integration head has separate local and CI
verification.
- Eighteen question journey, helper, and test file blobs are identical
between the qualified and integration SHAs. This includes clarification
flow and evidence, question grader and browser helper, the sanitized
receipt fixture, screenshot helper, provider-form helper, browser
question presentation, continuation scenario/readiness/flow, shared
agent validation, and user actions. The question run-count contract and
dispatch branch are also unchanged. Reviewers can compare those paths
directly between the two commits.
- All 622 selected Product E2E, catalog, question-resumption,
necessary-question, report, and readiness tests pass after the main
merge. After the exact search delta, all 145 narrow
question/search/project/contract tests and all nine real Chromium search
regressions pass. Map checks, combined credential-free preview, source
fingerprints, and the earlier workspace-link preflight pass. The local
TypeScript attempt reports stale Runner declaration errors for
`onAccountingEvent` in unchanged native-session code; fresh official CI
provides the clean-build type gate. The three search source/test blobs
match the qualified search merge, and the three project source blobs are
unchanged.
- An in-memory probe executes the actual launcher fingerprint
expression. Each project, search, and question implementation input
changes the digest. Separate red probes first reproduced the omitted
provider-form and browser presentation helpers; both pass after their
source inputs are added.
- The sanitized retained native API response-shape regression fails
before the grading correction and passes afterward. Negative controls
reject missing, pending, foreign, changed, spoofed, or wrongly bound
receipts, answers, continuation, model, environment, and document
evidence. All nine provider/semantic/API path combinations pass
calibration.
- The real local Chromium screenshot probe fails before the readiness
repair and passes afterward. Login redirects and sign-in pages at task
URLs remain excluded with no screenshot files.
- The two original attached failures remain retained: the first stopped
at early screenshot capture; the second stopped before answering at the
former API provenance gate. An earlier sandbox run also retained one
report subprocess failure; the full selected tests then passed with
local subprocess IPC access.
- The prior integration CI had an unchanged canary mock `EADDRINUSE`
failure. That history is retained; its documented retry is owned by the
coordinator. No application API, schema, production UI, model, runtime
policy, credential, or deployment configuration change is included.
Final integration head
|
||
|
DottaandPaperclip
|
a1845b3b64 |
fix(evals): capture TestOps registration after watch (#18375)
## Thinking Path > - Paperclip manages AI agents and verifies their work through Product E2E tests. > - The fixed Chat feature workflow prepares a TestOps case and plan before any model run. > - Its first zero-result bootstrap completed, but the watched child did not receive the generated job-run ID. > - The official watch log contained a job-run ID, so the missing child variable did not prove that registration was absent. > - This pull request reads official context after watch and validates the linked backend records. > - The workflow retains closed registration metadata and stops on unknown or foreign context. ## Linked Issues or Issue Description Refs: #18322. **What happened?** The successful zero-result bootstrap retained `official_job_run_id_missing` from its watched child. It could not prove the backend job and workflow binding needed for the next setup step. **Expected behavior** After successful watch, read the generated context, validate its current GitHub run and fixed workflow, and retain only closed IDs and parameter metadata. **Steps to reproduce** Run the fixed workflow's `bootstrap` operation on master. Inspect its setup artifact after watch completes. No evaluation or synthetic passing result is needed. **Paperclip version or commit** The original bootstrap used merged #18322. This follow-up starts from `76fde891ccc29283b3f5c8dcdce089c5b3d937d1`. **Deployment mode** GitHub Actions metadata-only setup in the protected reporting environment. ## What Changed - Run an internal context reader after successful bootstrap watch in the same protected step. - Parse only numeric official job-run and launch IDs in memory. Do not log or save raw command output. - Validate current-run, project, launch, job, workflow, and bounded parameter metadata before writing an exclusive registration receipt. - Retain the two known registration receipt paths with the existing setup artifact. - Preserve the original inside-watch receipt, case reconciliation, provider isolation, and evaluation limits. ## Verification - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs tests/runner-e2e/testops-chat-reporter.test.mjs`: 48 pure/mock/FS controls pass (36 prior and 12 new). The actual local Node version was 26.4.0. Normal Node 24 CI remains required. - Four MJS syntax checks and `git diff --check` pass. No TypeScript noEmit claim is made for this MJS-only helper change. - Actionlint passes for the changed workflow. - Normal [GitHub Actions CI](https://github.com/paperclipai/paperclip/actions/runs/41123894542) passes on exact head `58396de10f0c49294b0bbc0f4709c29990d57d53`: 48 jobs succeeded and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed that exact head at 5/5 with no actionable findings or unresolved review threads. - No local CLI, provider, model, app, database, or build qualification was run. No replacement bootstrap or paid evaluation was dispatched. ## Risks The official context or backend DTO shape can differ from the supported format. The reader then stops with a closed failure receipt. Pure tests do not qualify the remote CLI or backend. The original zero-result bootstrap and its missing child-context observation remain part of the history. The case's actual outcome remains unmeasured. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4edd10804e |
fix(evals): batch task search result readback across renders (#18370)
## Thinking Path
> - Paperclip lets people manage AI agents and saved work.
> - Product journeys check user actions against saved state.
> - The task search journey compares the exact visible task identities.
> - A search can replace rows while the harness reads an old indexed
locator.
> - The attribute read can then time out after the correct results
appear.
> - This pull request reads the visible links together and preserves
strict result checks.
## Linked Issues or Issue Description
Refs #18355.
**What happened?**
An attached Cloud attempt passed the task search journey. A later
attempt on the same harness and app revision failed at search-result
readback. Its owned screenshot showed the correct target and completed
control. Authentication, collection, and cleanup passed. The failure
remains recorded and the journey has not been admitted to the daily
plan.
The reader called `all()`, then awaited `isVisible()` and
`getAttribute()` for each indexed locator. When a search replaced the
old collection between those reads, the next attribute read waited for
an index that no longer existed.
**Expected behavior**
Read the matching visible links together. Compare the complete identity
set against the owned target and controls. A render transition must not
consume the whole poll deadline on a stale row index.
**Steps to reproduce**
1. Render three task links in the task collection.
2. Let the reader observe the third link as visible.
3. Replace the collection with the two correct search matches before its
attribute read.
4. The old reader times out waiting for `nth(2)`, even though the two
correct rows are present.
## What Changed
- Read visible link attributes in one batch. Discard a detached snapshot
and retry within the existing poll deadline.
- Keep the exact identity comparison, duplicate detection, control
exclusion, saved-content checks, and zero-run checks.
- Add a real Chromium render-transition regression and invalid-result
cases. Extend the stateful flow test for a detached snapshot.
- Document the readback boundary. No app UI, target configuration, or
cleanup behavior changes.
## Verification
- Review follow-up: the existing journey wait helper is now exercised
with the real Playwright poll. Its first read evaluates the actual
reader callback on detached browser DOM nodes and returns no evidence.
The second read contains an extra owned control. Only the third, exact
pair succeeds. The test observes all three reads. The 30-second deadline
and grading behavior are unchanged.
- Rebased onto current master, including the merged project journey.
Credential-free discovery confirms exactly one `project-task-context`
and one `task-search-filter` entry.
- Latest rebased checks passed: all nine Chromium search regressions,
all 381 Product journey unit tests, runner typecheck, discovery, maps,
and diff checks. Exact head
|