32 Commits

Include renames

Author	SHA1	Message	Date
3 people	a497bae83a	test(e2e): verify Cloud workspace access isolation (#18655) ... ## Thinking Path > - Paperclip keeps agent work inside company boundaries. > - Cloud users switch workspaces through the tenant account menu. > - A company-bound credential must work inside its permitted company. > - A missing resource cannot prove an access denial. > - This journey uses real controls in two registered test workspaces. > - It checks reads and task creation in both directions and revokes both keys. ## Linked Issues or Issue Description Refs: #18728, #18729, #18652. **What existing behavior does this improve?** Product E2E coverage for Cloud workspace selection and company-bound agent API access. **Current behavior** The journey catalog lacks a two-workspace boundary check with real allowed and denied controls. **Proposed behavior** Use the visible Cloud menu to switch workspaces and return. Read a real task and create one task in each credential's own company. Require real foreign reads and the same task-creation operation to return 401 or 403 in both directions. Verify that no foreign task was created and that protected content was not returned. Revoke both credentials and require their earlier allowed reads to return 401. ## What Changed - Add a browser journey and a narrow target-provider fixture contract. - Require independent owner reads, exact agent authorship, zero-to-one task creation, stable run history, no unexpected work and no foreign creation. - Validate observed workspace entry links against the exact registered targets. - Add wrong, missing, leaked, foreign and invalid-entry evidence controls. Saved passes reject collected runs or nonempty model usage. - Register all seven required checks, definition-hash inputs, feature associations. The feature-map documents are unchanged, as requested. - Base the final tree on verified repository master `7b8aa60b6d79b6cd9e4d46ca1369515c33e2d5d4` plus only 15 access journey and catalog files. Preserve all other master files and modes. The held completion journey is outside this PR's diff. ## Verification - 851 catalog, journey, completion-helper, artifact and browser-action support tests passed across 40 files. - E2E TypeScript, credential-free discovery, feature-map validation, full workspace recursive typecheck, full build and `git diff --check` passed. - The broad local `pnpm test:run` attempt stopped in chat integration shard 1 before any assertion. Embedded Postgres initialization exited with code null. No chat tests or later unit shards ran. The startup cause is unresolved, the failure is retained, and the command was not retried. The database/chat/test-runner source is unchanged from the fixed master snapshot. The separate hosted full CI result is recorded below. - The first local E2E typecheck read stale compiled plugin SDK declarations after the merge. The SDK source already declared the field. Rebuilding its declarations cleared the diagnostic without a source edit. - The complete tree comparison proves exactly 17 changed paths before the documentation removal against the fixed master snapshot. Every other master file and mode is preserved. All four access flow, contract and test blobs match the previously passing source `fab9a1fb2fefb41a7248e4fc1aa239d77455f5e2` byte for byte. - The historical staging attempt on source `fab9a1fb2fefb41a7248e4fc1aa239d77455f5e2`, the earlier Core42 deployment and the earlier adapter passed all seven checks: owner resource controls, visible Cloud switch and return, scoped data, both restricted-agent positive controls, cross-workspace read/task-creation denials, credential revocation and no agent work. Both credentials were revoked, owned credential agents had terminal receipts, and evidence cleanup was complete. No agent or model work was recorded. - The earlier attempt failed before journey checks while reading the borrowed fixture agent's historical details. That failed result remains preserved. The provider then supplied a separately owned primary identity. The public history assertions remain unchanged. - The strict root catalog expectation first failed because it omitted the new access case. Adding only that expected entry made the test pass. The ordered list and default-selection assertions remain unchanged. - The first combined-head full CI run failed two unchanged attention-retention tests at their original 15-second limits. The catalog fix does not change those tests or their timing. The initial failures are retained, their cause is unresolved, and that failed run was not retried. - Normal full CI passed on the new commit in [run 41595755161](https://github.com/paperclipai/paperclip/actions/runs/41595755161), attempt 1: 48 successful jobs and one intended skip. The two unchanged attention-retention cases passed at about 12 seconds; the earlier timeout cause is still unknown. The fresh review on commit `4c0ea682444738ad6fa17017a6afd25a49037559` is 5/5 with no open findings. The fresh staging result is recorded below. This PR changes no daily plan, private configuration, model or live workspace. - Fresh staging qualification on commit `4c0ea682444738ad6fa17017a6afd25a49037559` passed all seven checks and all five phases. Both ordinary same-workspace task creations returned 201. Both cross-workspace task creations returned 401. Both credential agents reached terminal state. Both keys were revoked, and both revoked-key reads returned 401. The manifest records complete cleanup and no active operations. No model or task run was recorded. All three workflow jobs passed. - The coordinator verified TestOps case 719 and result 3761 as passed, then closed launch 1987. Its report shows model none and the current qualification run. The automatic Slack report recorded one pass and zero failures. - The original normalized result was passed. The first evidence collector left its own qualification false because its allowlist omitted the standard `dashboard.html` artifact. The adapter already declared that file, and the public launcher generated it. An exact-object read matched the 118,791-byte object, ETag and SHA256 archive receipt. This completed the three-artifact inventory. The original result, collector proof and seal remain unchanged. No HTML was executed or rendered, and no test was rerun or regraded. | Fresh staging provenance | Verified value | | --- | --- | | Public commit | `4c0ea682444738ad6fa17017a6afd25a49037559` | | Qualification workflow | `41597774599`, attempt 1 | | Serving Core commit | `09162fe893b464dd2ccdafd14bd3ef1611202a86` | | Target adapter commit | `7367cf6fd0ee1cf52c2651a8a6671e087728a9c1` | | Actual workflow bootstrap commit | `55351ffba2704a4e13c1ead3e50665acd51aef87` | | Exact 117-file QA contract SHA256 | `14d95c29059c82563ab0e1d65a01aa4ac6c4f60f77a236803cf06ad5d9e7d27b` | | Provider-pack configuration version | `9` | - At the user’s request, follow-up commit `7e0dbc4bdba8a25ff6486648b0ca0d0db2616a4c` removes both feature-map additions. The final diff contains 15 paths and no `maps/feature-map/` changes. An entire-tree comparison against the QA-passed commit proves these two Markdown files are the only changes; every executable file, test, fixture, registration and file mode is unchanged. `node scripts/check-maps.mjs` and `git diff --check` pass after the removal. The live QA run above was not repeated for this documentation-only change. Fresh current-head full CI passed in [run 41633385961](https://github.com/paperclipai/paperclip/actions/runs/41633385961), attempt 1. The current-head Greptile review is 5/5 with zero unresolved threads. The normal merge preserves every newer master file outside the 15 PR paths and keeps all 15 PR files byte-for-byte intact. ## Risks The private target provider must register two exact test workspaces and preserve their separate ownership and cleanup. The case verifies company-bound agent credentials. It does not certify human membership roles. It tests task creation because ordinary agent task edits require a real execution context. No execution context is forged and no product authorization guard changes. The local broad unit run could not start embedded Postgres, so it does not establish a full local unit pass. The earlier hosted attention-retention timeouts also remain unexplained. Passing a later run would not prove the source of those timeouts. The four case files retain their historical passing bytes. The fresh qualification verifies the executable test code retained in this PR against the deployed staging stack. Daily admission remains a coordinator action after the normal source merge. The case does not cover future deployments without another run. ## Model Used OpenAI GPT-6 through Codex with reasoning and code execution. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com>	2026-11-03 06:24:08 -06:00
DottaandPaperclip	7b8aa60b6d	test(e2e): add repository continuation journey (#18729) ... ## Thinking Path > - Paperclip keeps agent work in project workspaces. > - A later user request must continue from the saved repository state. > - A successful reply alone cannot prove file or Git history preservation. > - The repository journey in #18652 passed against QA staging with independent byte and commit checks. > - Its separate completion journey remains unpassed and must stay open. > - This PR delivers only repository continuation and reuses the merged shared support from #18728. ## Linked Issues or Issue Description Refs: #18652. Builds on merged #18728. This is the repository-only delivery from #18652. The original completion implementation and all failed attempts remain on that open PR. ## What Changed - Add `repository-continuation` with the exact measured source fixture, two native turns, and eight required checks. - Start from measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189`. Keep the repository flow and Git/byte oracle unchanged. Clarify the two file-content instructions to require the word first or verified followed by exactly one LF newline; retain the exact expected bytes and the original random-reference contract. - Document the two-turn proof boundary: final saved hidden Git metadata is not independently read. Keep all eight success assertions. - Use the original-deadline and routine failure-reporting repairs from merged #18728. Integrate its actual master squash `e5713d95c6d9a258882649789ea5eb071cc17dc0` through one normal merge. - Use a fixed public read-only repository commit. Verify downloaded Git objects, parent commits, changed paths, saved workspace identity, and exact source bytes across both turns. Reopen the registered outputs through the browser. - Keep the shared artifact reader behavior unchanged and narrow its type to the API capabilities it uses. - Add only this case to the catalog, feature association, dispatcher, source hash, types, run/check contract, strict catalog expectation, and journey documentation. ## Verification - Current head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed fresh [QA staging run 41589522430](https://github.com/paperclipai/paperclip-cloud/actions/runs/41589522430) and its [full report](https://pages.paperclip.ing/evals-qa-smoke-41589522430-1/). All eight checks and five phases passed. Both native turns succeeded and committed in the same workspace. The independent same-buffer Git audit verified that the second commit has the first commit as its parent. The saved original files and browser downloads matched their expected bytes. This does not add a final hidden `.git`/HEAD assertion. - The fresh run used OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on QA staging/Daytona. It used Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`, Cloud `cloud-41570439002-1`, and environment configuration version 9. Actual checkouts matched public source `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31`, adapter `7367cf6fd0ee1cf52c2651a8a6671e087728a9c1`, and protected workflow bootstrap `a9ad6877de62fe3f65aacadc8a9dfba7d90729cf`. The bootstrap matched the accepted 117-file QA contract. - Normal cleanup completed. The original manifest is complete, evidence cleanup is complete, no operations remain active, and the owned agent is terminal. The coordinator verified ACTIVE TestOps case 685 / result 3695 and closed [launch 1921](https://paperclip.testops.cloud/launch/1921). This records normal cleanup closure; no separate physical provider-stop acknowledgement was collected. - Measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189` passed [QA staging run 41573331189](https://pages.paperclip.ing/evals-qa-smoke-41573331189-1/) against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch 1723 / result 3501 maps to case 685. All eight checks and five phases passed. The two run bindings, independent saved-byte/Git audit, and completed cleanup were verified. - Initial extraction `54103639850d08d4d125ebe0493d8d3d49b58211` preserved the measured case files. The current head also clarifies the two LF instructions, documents the final Git-metadata limit, and inherits the reviewed helper repairs. This changed the fixture conditions. The current head now has its own live pass, listed below. The original ac4 result remains a pass under its original conditions; it was not used to qualify this head. - The emitted-prompt control failed on the old quoted backslash+n instructions, while 14 sibling controls passed. All 47 focused repository/artifact controls then passed after the two-sentence repair. Local: 891 related support tests, runner E2E typecheck, recursive typecheck, build, maps, token gates, and `git diff --check` passed. The inherited routine source has 62 causal read/cleanup controls and 26 unchanged real component controls. - The initial repository head's normal full CI run 41578686494 passed. Its 4/5 review and all original findings remain recorded. The later stacked head `037b979477161881a4877230500e6f3de8faea5c` also passed full CI run 41581342503 and a fresh 5/5 review. Current integration head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed normal full [CI run 41584981651](https://github.com/paperclipai/paperclip/actions/runs/41584981651), attempt 1. Its fresh exact-head Greptile review is 5/5, with zero unresolved threads and no open findings. The broad local suite was not repeated. The earlier full local run recorded in #18643 had four failures and did not reach 164 serialized groups; this PR does not claim an all-local pass. - Earlier repository failures remain recorded: provider-pack transfer timed out in one attempt; a later attempt could not read saved workspace files. The later pass does not regrade those failures. The product repairs are separate from this unchanged case delivery. - The fixed actual #18728 squash is now integrated. The PR targets master. Its aggregate diff has exactly 14 repository-only paths, each identical to `037b979477161881a4877230500e6f3de8faea5c`; all other master changes are preserved. No later master head was chased. The current CI, review, and fresh staging qualification all passed for this integration head. ## Risks - Git bundles and workspace bytes are private evidence. Public reports retain closed checks; the fixture never pushes to the public source repository. - The second committed output must retain the first commit as its parent, the same workspace binding, and original saved file bytes. Final hidden `.git`/HEAD after the second save is not independently exposed by the supported remote workspace API; this test does not claim third-turn Git metadata durability. - Only JSON metadata observations retry transient transport failures. Binary attachment and browser downloads remain single attempts. Transport failures and digest mismatches remain visible failures. - The existing artifact reader is shared. Its runtime code is unchanged and its regression coverage remains included. - Daily source pins and plan selection stay with the coordinator. This PR does not register or bypass the unpassed completion case. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment ID and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 00:46:13 -06:00
DottaandPaperclip	e5713d95c6	test(e2e): add scheduled routine journey (#18728) ... ## Thinking Path > - Paperclip manages agent work through tasks, projects, and routines. > - Daily QA must exercise the staging app through normal browser and API surfaces. > - A schedule must create real work and retain its result and history. > - Pausing that schedule must prevent the next invocation. > - The routine journey in #18652 passed in QA, while its separate completion journey has not passed. > - This PR delivers the routine test independently and preserves its measured flow and assertions. ## Linked Issues or Issue Description Refs: #18652. This is the routine-only delivery from that PR. The original completion implementation and its failed attempts remain on #18652. ## What Changed - Add the target-independent `routine-scheduled-run` journey with one native run and seven required checks. - Start from the measured routine source `ac4efb7c6f4782ea806661683c656fd2b4cc3189`. Keep its request, seven success assertions, one native-run limit, and 65-second pause check. - Bound each observed GET by the original remaining journey deadline. Preserve the normal API timeout for other callers. Keep the original routine error if final trigger disabling also fails, save a closed secondary receipt, and fail an otherwise successful journey when disabling fails. - Narrow the shared output-reader type to its one-argument GET capability without changing its runtime code. - Register only this case in the catalog, types, feature associations, dispatcher, and definition hash. Keep the strict catalog expectation in sync. - Retain the real Routines component controls and the existing project-flow coverage for the shared browser helper. - Document resource cleanup and the 65-second pause observation. Preserve the current routine map content. ## Verification - The original source passed [QA staging run 41567574185](https://pages.paperclip.ing/evals-qa-smoke-41567574185-1/). TestOps launch 1657 / result 3431 maps to case 718. All seven checks and five phases passed. One scheduler-bound native run, saved output attribution, the full pause window, and cleanup were verified. - Current source `8f62137f130f59fd3e39276e3d60185ddff24517` passed [fresh QA staging run 41582706162](https://pages.paperclip.ing/evals-qa-smoke-41582706162-1/) against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch 1855. All seven checks, five phases, and all three workflow jobs passed. The run used one scheduler-bound native turn and observed 65.352 seconds after pause and completion. Canonical evidence collection and normal cleanup completed, with no active operations and the owned agent terminal. - This fresh pass covers the reviewed deadline and failure-reporting changes in the current source. The earlier ac4 pass and the byte-preserving b940 extraction remain historical records under their original conditions; neither failed attempts nor prior source conditions are regraded. - Causal controls: six assertions failed on the old behavior, then all 62 focused controls passed after the repair. Local: 834 related support tests, 41 shared chat/output-reader tests, and 26 unchanged real component controls passed. Runner E2E typecheck, recursive typecheck, build, maps, token gates, and `git diff --check` passed. The temporary output-reader type mismatch exposed by typecheck was corrected by narrowing the capability type. - The first restricted build/typecheck invocations failed because local tsx IPC was denied. A restricted support run also lacked the report CLI output. The same source passed with normal owned local IPC allowed; the original logs remain retained. - The original head's hosted e2e shard 2/8 failed at the unchanged mobile preference frame-count test: expected more than five frames, observed five. The cause is unknown. Its original log remains retained; it is not labeled a proven flake. No CI rerun was requested. - Current-head full CI run 41580423918, attempt 1, passed. The fresh review is 5/5 with zero unresolved threads. The broad local suite was not repeated. The earlier full local run recorded in #18643 had four failures and did not reach 164 serialized groups; this PR does not claim an all-local pass. - Prior failed routine attempts remain recorded. They exposed composer focus, collapsed schedule controls, pending browser events, and Markdown serialization problems. The measured later pass does not regrade those attempts. ## Risks - The shared browser-event helper also serves task creation and project creation. Its regression tests and existing consumers remain included. - The staging pass qualifies the captured source and configuration. The coordinator retains source merging, daily pinning, and plan selection. - The repository test and the unpassed completion test are not registered by this PR. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment ID and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub issue/PR references) --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 23:25:56 -06:00
DottaandPaperclip	82c033ed12	test(e2e): cover uploaded input and personal inbox triage (#18643) ... ## Thinking Path > - Paperclip lets people assign work to agents and manage their tasks. > - Product E2E journeys verify the saved outcome through the browser and normal APIs. > - Uploaded input and personal inbox actions need repeatable coverage. > - An agent answer must match the input file, and an inbox action must retain its saved state. > - This change adds two journeys with independent evidence checks. > - A blocked fixture action must use the normal human-owned unblock contract. > - Registration stays separate from deployed qualification and daily admission. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for file attachments, saved task documents, and personal inbox read and archive actions. **Current behavior** The journey catalog lacks an uploaded-input calculation and a persisted personal inbox triage path. A successful agent run alone cannot prove either outcome. **Proposed behavior** Upload a generated CSV through the task composer. Require an exact saved answer and unchanged input bytes. Read and archive one owned task through the inbox, reload each state, and verify that a new Blocked event brings it back without agent work. ## What Changed - Add a file-input journey with a random reference held only in the uploaded CSV. - Check exact result values, document authorship, native run binding, original input bytes, and persistence after reload. - Add a browser inbox journey with an independently authenticated human author, saved read and archive state, unchanged work status, and activity resurfacing checks. - Reject wrong, missing, duplicate, and foreign evidence in assertion controls. - Register both cases and document their scope and privacy rules. - Supply a current-human unblock owner and action for the archived-task fixture. Test that exact payload through the production route and retain the bare-blocked422 negative control. - Include the fixture constructor in the source definition hash and both new cases in the exact catalog assertion. - Reject collected inbox receipts with unexpected runs, providers, models, or native/legacy execution modes. ## Verification - The frozen source is `cdb6659fb4e4b67e9321bc3fd963cf11e9b8b2f3`. It includes one fixed master integration at `fd0d61b8e329efc89ff541dc2cac44e7c7a37f78`. The single map conflict retains both coverage paragraphs. - The production HTTP route rejects the original bare blocked payload with422 and accepts the exact human-owned fixture payload with200. Neither control wakes an agent. The full related route file passes all200 tests. - The frozen current head passes recursive workspace typecheck, runner typecheck, full build, map checks, and diff checks. Its full related journey/catalog suite passes723 tests, the production route suite passes200, and the separate Node support suite passes347. The retained contract file passes all38 tests. Fresh Greptile review is 5/5 on the frozen current head. Current-head hosted CI passes after one authorized failed-jobs-only recheck. The original Runner close test failed; the unchanged, properly cargo-prepared focused local control passed in2.99s, and the hosted recheck then passed. The cause remains unknown; no Runner code, test timing, or assertion changed. The original hosted diagnostic and the initial invalid local invocation without required fake binaries remain recorded. The ordinary local full-test run is in progress. - The unrestricted local runner support run on the fixed-master integration is not green:5993 tests passed,12 failed,3 skipped. The unchanged failures cover onboarding cleanup expectations, Pi metadata, workflow-security expectations, and an expired test allowance. An untouched743 baseline reproduces the same non-PG failures; their source/test blobs also match pre-feature40b807. This change does not extend an allowance or alter paid admission. The original5983-pass/15-failure run remains recorded. Restoring the local Postgres package's own library aliases resolved its two startup failures. The feature-related catalog omission is corrected. - The uploaded-file case passed its first complete deployed QA attempt: one configured native run, all six independent checks, original evidence, complete cleanup, and no active operations. - The first inbox attempt stopped at the agent comment execution-context guard. That guard stays intact. The fixture uses an approved, independently authenticated QA human. - The next original inbox attempt verified protected-secret retrieval, human attribution, and session closure, then failed its unread prerequisite. The product unread defect was repaired in #18670. - The fresh deployed inbox attempt passed external-human activity, saved read state, archive persistence, and unchanged task status after the product rollout. It then stopped at the fixture's bare blocked PATCH before any resurfacing assertion. The response status was discarded;422 is an explicit contract inference independently reproduced through the production route. That failed attempt remains retained. A fresh full inbox qualification on the current frozen head now passes all six checks and all five phases; the normal human comment receipt, closed human session, zero agent work, complete cleanup, and absence of active operations are verified in the original evidence. Daily admission remains a separate coordinator action. - The retained-result review gap failed a negative receipt control before the repair. All38 retained journey contract tests now pass. The two old unread advisory threads are resolved only after the fresh full deployed inbox pass. - Original failures remain recorded, including the earlier unchanged environment-runtime assertion and unchanged runner/occupied-port startup failures. Closed operation codes and failure screenshots retain the failing step without raw browser errors. Actual retry qualification remains separate from source tests. ## Risks The target must supply the declared external-human fixture. Inbox control selectors may differ across deployed UI revisions. File processing depends on the configured native agent. Exact saved-state and source checks must remain required when those dependencies fail. The change adds test journeys and does not change production behavior or schemas. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (relevant suites; broader baseline failures are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 18:12:16 -06:00
DottaandPaperclip	7e595fd055	test(e2e): verify native hiring, delegation and teammate reuse (#18351) ... ## Thinking Path > - Paperclip manages real work by AI agents. > - Daily QA must verify that work on the staging deployment. > - Classic Agent Chat coverage already exercises hiring and teammate reuse. > - The deployment journey should share that behavior and its ownership oracle. > - Saved documents, exact callback replies and physical cleanup establish success. > - The corrected journey and complete daily suite now pass on QA. ## Linked Issues or Issue Description **Subsystem affected** Product E2E deployment journeys and their shared browser controls. **Problem or motivation** QA must prove that one newly hired teammate completes two different tasks and returns both saved results to the lead conversation. Names, chat claims and successful-run counts alone cannot establish authorship or completion. **Proposed solution** Reuse the classic hiring oracle and production composer. Verify immutable creation audits, initial assignment, current project and ownership, saved document revisions, exact native authors, durable callbacks and their own task links after reload. Bound execution to five lead turns and two child turns. **Alternatives considered** A separate QA implementation would duplicate the classic journey. Name matching or later reassignment would accept broken initial ownership. Links elsewhere in the conversation cannot substitute for either callback's own result link. **Roadmap alignment** This verifies existing features. Related merged repairs: #18389, #18465, #18459, #18463, #18502, #18503, #18504, #18506, #18507 and #18514. Separate dependency #18517 repairs file preservation after lifecycle cancellation and is merged as Core `050365e`, which also contains #18514. This primary PR now differs from that protected mainline in only 16 unchanged H harness/documentation paths; it adds no application production delta. The measured QA deployment serves Core `050365e`, with its matching bundled provider pack and unchanged approved model/ROM. ## What Changed - Add the shared `hire-delegate-reuse` journey, catalog, types, dispatch, launch fingerprints, journey-feature entries and documentation. Hire one teammate, delegate a checklist, then reuse the same teammate for a saved review. Require seven canonical native runs: five lead and two child turns. - Require exact company, lead, chat and native creation receipts. Both tasks must be created for the intended teammate using immutable creation-audit assignee fields. Independently verify current project and ownership. Creation-time project is not recorded in that audit and is not claimed by this assertion. - Verify both current document revisions belong to the exact child runs and preserve the original checklist. - Use ordinary production rich-editor agent/project autocomplete controls. Verify exact reference chips, then send once. Preserve the shared default sender. - Require each durable, run-owned callback reply to contain its own exact completed-task link. Collect links through the actual production Markdown renderer and issue-reference decoration, then apply the existing company-scoped route authority. An unrelated table or code fence cannot hide a rendered link; foreign, wrong-task and code-only links remain failures. - Preserve approved connection, account, grant and responsible-user attribution. Verify the child's independent installation and retain the immutable binding snapshot. Do not copy credentials or broaden grants. - Record ownership before later observations can fail. Retry only typed transient observation GET failures. Preserve authority and cleanup failures. - Retain all 14 daily journeys, the model matrix, deadlines, seven-run cap and 500-cent aggregate allowance split 300/200. ## Verification - **Current composed head:** `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`. It normally merges mainline `1e8e322` and exact #18517 source `adb1d4e`, retaining the intermediate `b2d9eb4` and `6fd1042` ancestry. The only merge conflict was the Playwright support registration; it is resolved to the exact previously qualified union. All 16 H harness/documentation paths are byte-identical to `5c0830f`; all four #18517 paths are byte-identical to `adb1d4e`. Every other tracked path matches actual protected mainline `050365e`. A clean merge preview of `050365e` and this head equals the current head tree, so no additional source commit is needed. GitHub's three-dot display still lists the four already merged helper files because #18517 used a squash merge; those blobs equal protected mainline and add no net application change. No journey, oracle, cap or model change accompanies this composition. A final clean merge preview on intervening mainline `006d5691a6245d5a710e25ba3df6d609754e163a` preserves all unrelated changes and adds exactly the same 16 measured H blobs, with no net App production delta. That later mainline is not represented as the AppCore measured by the QA run. - **Local qualification:** 149/149 focused tests across six suites, including the report-catalog case, and 11/11 real Chromium checks passed on `b2d9eb4`. All H modules, browser inputs and parser/package dependencies are byte-identical at `afa3f55`; those unchanged suites were not repeated. Current-source transitive types, maps, module boundaries and diff checks pass. Official Node24 QA runtime imports pass again using exact public `afa3f55` and private `8671fa9`, with a clean credential-free environment and network disabled. The initial local report-catalog failure is retained: the child `tsx` process was denied its local IPC socket before importing the report or writing an artifact. The same source and selected case pass after correcting only sandbox IPC permission; no source workaround or deselection was used. - **Current canonical CI and reviews pass.** [CI 41275109691](https://github.com/paperclipai/paperclip/actions/runs/41275109691) completed successfully at exact `afa3f55`: all 59 contexts are terminal and clean (55 success, three skipped, one neutral). Fresh exact-head Greptile and official Paperclip reviews both scored 5/5; all 27 review threads are resolved with pagination complete. The official review is static and its check is neutral; it did not execute tests or live QA. A separate exact-head independent composition assessment found no actionable issues at 4/5 confidence and performed no live actions. **Fresh QA qualification passed: all 14 daily journeys, including the corrected H7, passed against deployed Core `050365e`.** Prior check conclusions and review scores retain their actual heads. The previous `5c0830f` completed all 59 contexts cleanly, actual exact-head Greptile and official Paperclip reviews both scored 5/5, and all 27 review threads were resolved. Its separate independent journey-source review at `2b803f08` remained 4/5 with a unit-helper authorship caveat. Intermediate `b2d9eb4` received an actual exact-head Greptile 5/5 with zero open findings, then was updated to compose the valid dependency review fix before live QA. That score and its pending CI are not attributed to `afa3f55`. - Existing causal controls remain unchanged: invalid immutable assignee, either missing callback link, foreign/code/wrong-task links and an earlier handoff fail against the relevant prior source. Renderer controls prove that tables and fenced code elsewhere in a valid callback do not hide its actual task link. Both composer and document-navigation browser registrations remain present. - #18503 merged after its actual QA document case passed four canonical checks and five phases, positively established zero native work, and completed cleanup. Its measured files are unchanged in this composition. #18506 and #18507 merged after their own checks; their causal worker-settlement controls remain intact. - #18514 is merged as `1e8e322`. It reproduces the selected-task metadata-only-project startup failure against deployed `756599e` and passes 19 real-PostgreSQL selected controls after repair. Both direct and assembled baseline controls fail before it. Real source, explicit selection, policy and workspace-isolation guards remain enforced. It is included in measured Core `050365e`; fresh H7 reached and completed both delegated native tasks. - #18517 initial source `6fd1042` passed 320/320 focused and sibling Stop/recovery tests, including 20 preservation controls. Its actual official review was 4/5 and found a valid follow-up: the new live-controller guard should wait for actual lease expiry rather than repeat locked checks every 30 seconds. Both an already-live lease and a real under-lock renewal reproduce that issue against `6fd1042`. Corrected source `adb1d4e` passes all 22 preservation controls and 124 directly affected tests across three suites, exact-head server build/stamp, typecheck and structural checks. It defers before locking and rechecks renewed expiry afterward, preserving the transactional owner guard. The old score/failure and successful owner tests remain attributed to their actual sources. Independent exact-head review of `adb1d4e` found no actionable issues at 4/5 confidence. This is local/source evidence, not physical QA recovery. Its current [CI 41274963476](https://github.com/paperclipai/paperclip/actions/runs/41274963476) completed successfully with all 59 contexts clean; both actual current reviews are 5/5 with zero unresolved threads. It merged normally as `050365e` after those gates. QA now runs that image. The historical allocation is positively stopped and retained; its files-only export remains a separately tracked repair, not a passing-test assertion. - The historical [first H7 pass](https://pages.paperclip.ing/evals-qa-smoke-41226065043-1/) remains attributed to public `5d3f143`, private `dabad62` and App `b7f1c37`: all checks and five phases passed, seven native runs succeeded, both saved documents and exact callbacks were verified, and physical cleanup completed. An optional runtime-metadata receipt was inconclusive and was not grade authority. Material failed attempts remain failed: | Attempt | Actual result and correction | | --- | --- | | [41231853638](https://pages.paperclip.ing/evals-qa-smoke-41231853638-1/) | 13 passed; hiring omitted initial teammate/project assignment and started extra work. The production composer repair retains the ownership negative. Later recovery stopped physical work without changing the failed grade. | | [41238711372](https://pages.paperclip.ing/evals-qa-smoke-41238711372-1/) | All seven native runs succeeded, but the final callback had no durable chat reply. Cleanup passed. #18502 repairs premature completion-outbox retargeting; the fresh measured H7 now verifies both durable, run-owned callback replies and their own task links. | | [41253428446](https://pages.paperclip.ing/evals-qa-smoke-41253428446-1/) | 12 passed; hiring rejected a mentioned metadata-only project before delegation, and document reopening hit an immediate-control-probe race. #18504 and #18503 repair those separate boundaries. Historical missing-task absence remains quarantined; physical termination does not manufacture absence evidence. | | [41266385118](https://pages.paperclip.ing/evals-qa-smoke-41266385118-1/) | On `5c0830f` and App `756599e`, 13 passed, including the exact composed document helper. Hiring created the correctly assigned task, but its child admission failed before runtime selection because the organizational project had no repository source. #18514 repairs that selected-task boundary. The cancelled native lead retained an active prepared-workspace lease; cleanup 41271035231 failed with `owned_work_stop_unconfirmed`. #18517 addresses its file-preservation admission. Launch 1110, original evidence and automatic Slack FAIL remain failed. | | [CI 41257779002](https://github.com/paperclipai/paperclip/actions/runs/41257779002) / [CI 41263476916](https://github.com/paperclipai/paperclip/actions/runs/41263476916) | Existing enrollment and snapshot-retention fixtures observed intermediate I/O before worker settlement. #18506 and #18507 have controlled causal failures, complete-settlement observers and preserved deadlines/negative controls. Runtime scheduling behavior is unchanged. | **Fresh measured acceptance:** [QA run 41285131220](https://github.com/paperclipai/paperclip-cloud/actions/runs/41285131220) and its [permanent report](https://pages.paperclip.ing/evals-qa-smoke-41285131220-1/) passed **14/14**, zero failed or incomplete, with all five phases and complete cleanup for every case. The strict verifier independently matched original grades, current provenance and completed manifests: public harness `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`, private workflow `8671fa93e27501fabbfffa377bec6dea453b5db7`, AppCore `050365e09d88411960fc3bacc1c08c46c0d9b6fa` and matching Cloud image/provider pack. All model, ROM, deadline and run-budget controls remained unchanged. All three workflow jobs completed successfully; the automatic Slack notification reported PASS with links to this run. The passing TestOps launch was closed normally after evidence verification. The hiring case passed all 11 check entries across nine IDs and five phases, with exactly five lead and two child native runs. Both saved documents match their live current revision/body hashes and exact hired-agent run attribution. The immutable original checklist remains intact; the same teammate authored the review. Both own callback replies contain their exact delegated task link. Independent canonical readback confirmed all seven native runs succeeded and committed their accepted result, assessment and decision, with no live owner and zero pending or failed effects; each of the seven exact owned leases has its own bound DESTROYED receipt and successful cleanup. All seven final snapshot operations match their source run, workspace, task, epoch, SHA, generation and export location. The four earlier lead snapshots are retained SUPERSEDED ancestors of the current committed snapshot; the latest lead and both child snapshots are COMMITTED. The initial diagnostic required current COMMITTED status for every historical snapshot and correctly remained false until the separate lineage diagnostic established this supported supersession. The product oracle and original grade were not changed. Canonical Suite metadata is `qa-basics`; the daily plan was saved with its standard suite-only query and synchronized, retaining all 14 ACTIVE cases. The older failed allocation remains separately stopped, retained, isolated from reuse and owned by terminated actors. That permits fresh campaigns without replaying old work. Its same-allocation files export is still unresolved and has separate bounded files-only recovery work; it is not claimed by this successful campaign. Original failed grades, reports and automatic FAIL notifications remain unchanged. No failed result was closed, relabelled, hidden or retried unchanged to obtain green. ## Risks The journey requires Agent Chat, normal hiring permission, a scoped lead and a provider that finalizes owned resources. Missing or extra work, wrong ownership, absent saved documents, missing callbacks and unconfirmed cleanup must fail. The deployed app and bundled runner pack must match the verified Core. The old run's cancellation ACK and terminal database row do not prove physical termination or saved files. Its coordinator has no accepted result or decision; pending-effect absence cannot be inferred from that missing decision. File recovery must preserve the exact allocation and original destroy disposition without a provider turn. Current source checks, reviews, fresh H7 and full-14 aggregate all pass. Canonical Suite metadata and actual 14-case daily-plan synchronization were verified after closing the genuine passing launch. The original failed allocation still has a separate files-export debt; its STOPPED receipt is not a SAVED or DESTROYED claim. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and browser test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge - [x] Historical passes and material failed attempts retain their actual source and outcome - [x] The composed document helper passed normal QA with complete cleanup on its actual source - [x] The corrected hiring journey passes fresh normal H7 with complete cleanup - [x] The active hiring case has canonical Suite and daily-plan membership after the passing launch is closed --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-01 02:33:59 -06:00
DottaandPaperclip	9dbbfa4fb2	fix(qa): wait for task document controls after reload (#18503) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task documents keep saved work and its revision history with each task. > - The browser document journey must reopen the saved task and read the revised document after reload. > - Reload can finish before the task controls mount. > - The helper checked visibility once and could wait for a tab launcher inside a panel it had never opened. > - This pull request waits for the existing usable controls before it selects the normal UI path. > - The saved content, revision, ownership and no-agent-work checks stay unchanged. ## Linked Issues or Issue Description Refs: #18356. **What happened?** The document case in QA run 41253428446 passed document creation and revision-history checks, then failed during reopen or readback. The preserved screen shows the revised document and revision 2 in chat. The original result has the generic `journey_operation_failed` code. Its exception stack was intentionally excluded, so the exact historical failing instruction is not known. A separate Chromium control reproduces a concrete navigation race in the same helper. If task controls mount 200 ms after reload, the helper can select `Open a new tab` before it opens the properties panel. That control never becomes visible. The same race affects a delayed classic document editor or an existing Artifacts tab. **Expected behavior** The helper waits for the current task's normal authoring or panel controls. It then opens Artifacts through the same visible controls. A task without document authoring must still fail. A stale body, stale title or damaged revision history must still fail. **Steps to reproduce** 1. Open a task from the Backlog list and reload it. 2. Delay mounting the task controls by 200 ms. 3. Run the unchanged document reopen helper with its properties panel closed. 4. Observe that it waits for `Open a new tab` while `Show properties` is visible. **Paperclip version or commit** Frozen source base: `b889ac39b7fbed558c5f4cfbbc2b73c4ca916810`. No duplicate document reopen PR was found. The related document feature is #18356. ## What Changed - Wait for an existing authoring, properties, Artifacts or launcher control before choosing the navigation path. - Wait for usable panel content after opening properties. - Add five credential-free Chromium controls for delayed collapsed, Artifacts, launcher and classic layouts, plus missing-authoring rejection. - Keep the existing Backlog navigation test compatible with the new readiness probes. - Register the browser regression in the existing support suite. No product code, model, budget, run cap or campaign deadline changed. The existing document contract needs no documentation change. ## Verification - Original failure is preserved. The [sanitized QA report](https://pages.paperclip.ing/evals-qa-smoke-41253428446-1/) records both partial checks and the failed journey. This PR does not change that grade. - Before the fix, the real Chromium control failed for delayed collapsed, Artifacts and classic controls. The launcher path and missing-authoring rejection passed. A 750 ms action timeout bounded this diagnostic; final tests use the existing suite timeouts. - Final Chromium: `node node_modules/@playwright/test/cli.js test --config tests/runner-e2e/playwright-support.config.ts task-document-navigation.spec.ts` — 5 passed. - Document units: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/product-journeys/task-document-flow.test.ts tests/runner-e2e/product-journeys/task-document-evidence.test.ts --maxWorkers 1` — 23 passed. This includes 19 negative content, ownership and revision controls. - Scoped strict TypeScript, map checks, token gates and diff checks passed. - Node 24 normal QA import preflight passed with network disabled. - The entire `taskDocumentPersistence` function and the canonical evidence module are byte-identical to the base. The existing exact title/body checks, independent API reads, revision pointers and no-agent-work checks remain in force. - Repository-wide build, typecheck and tests were not run locally for this four-file harness change. [Canonical CI 41255897484](https://github.com/paperclipai/paperclip/actions/runs/41255897484) passed on exact head `ffc04500cb0728f6fbc8b3bc2e704c547015641d`: all 59 checks are terminal (55 success, 3 skipped, 1 neutral), with no failed or pending checks. Greptile and Paperclip each gave this head 5/5 with no findings. All review threads were checked; none are open and there is no next page. The branch is CLEAN and MERGEABLE. - Actual QA measurement now passes in [workflow 41266385118](https://github.com/paperclipai/paperclip-cloud/actions/runs/41266385118), campaign `journey-1793502283113-8f658807`. The measured harness `5c0830f693da2ccef062612146b76629a0177049` contains the three exact reviewed document source/test blobs and unchanged canonical evidence module from this head. The target was the QA cloud instance on App `756599e4ea069c9338c320fe8846523d3dfb92a0`. All four canonical checks and all five phases passed; the manifest is COMPLETE with evidence cleanup complete and nothing pending. A positive no-agent-work check establishes zero native work despite omitted raw run IDs. A separate read-only verifier reread the exact original final grade and confirmed the retained-result validator. Raw document snapshots were not retained; content, revision and author proof comes from the executed unchanged exact UI/API predicates. The full 14-case launch remains failed because hiring failed separately; no aggregate pass is claimed. ## Risks The historical exception was intentionally sanitized. The local control proves this helper race, but cannot prove which instruction failed in the prior QA attempt. The current staging measurement passes this document case; the exact historical failing instruction remains unproven. The new readiness waits use the existing assertion timeout and campaign deadline. They do not accept missing controls or weaken document evidence. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code execution and browser automation. The exact serving model revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Live qualification: the document case is verified on QA with complete cleanup. Historical failed results remain unchanged. Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-31 22:30:05 -05:00
DottaandPaperclip	5bc029bf6c	test: wait for native work and accounting before Stop redirect (#18463) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys test the browser, native work and saved outcomes on a selected deployment. > - The Stop journey must interrupt real work before it sends a new direction. > - Its old boundary accepted a turn-start event before any command or usage report. > - A stopped attempt with incomplete usage correctly blocks later work under the existing budget policy. > - This pull request waits for a live native command and bound usage, then checks actual accounting settlement before it sends the redirect. > - The provider reports first usage after its initial command completes, so the fixture finishes a harmless readiness call before the long interruptible hold. > - The benefit is a stronger test of Stop and continuation with the same three-run limit. ## Linked Issues or Issue Description Refs: #18098, #18345, #18354. Related accounting work: #18035. Integrated upstream prerequisites: #18387, #18465 and #18409. Duplicate selector PR #18472 was closed as superseded by actual merged #18409. **What happened?** A QA attempt clicked Stop soon after a native turn started. The owned event stream had no command or usage report. Its cancelled run retained incomplete accounting. The saved redirect received a skipped wake, and no third run started. The budget guard must keep that missing usage visible. **Expected behavior** The journey must first prove that actual native work is active and that its current turn has reported complete usage. The fixture must finish a harmless readiness command before it starts a separate bounded hold. After Stop and physical cleanup, it must verify a complete accounting receipt and saved settlement before sending the redirect. **Steps to reproduce** The fresh staging attempt failed before Stop because its first 90-second command ended before the provider emitted its first complete usage. Its only later eligible commands lasted less than one second. The controlled browser sequence reproduces that ordering without accepting it. Completed readiness usage followed by a separate still-active hold qualifies under the unchanged oracle. Startup alone, command alone, usage alone, partial usage, incomplete accounting and a missing settlement timestamp still hold the next action. ## What Changed - Request a harmless `printf` readiness command first. Wait for its result, then request `sleep 90` exactly once in a separate shell call before file edits. This gives the provider an opportunity to emit current-turn usage before the interruptible hold. - Reuse the strict native-command observer. Require a bound current-turn complete usage report with valid nonzero token counts. Reject stale, partial, foreign and semantic-tool evidence. - Wait for the stopped run's complete accounting receipt, `accountingReceiptReady: true`, `costAccountingPending: false` and a valid saved settlement time before redirecting. A receipt already ready before Stop is intentional only when the exact cancelled run is actually accounted with no pending debt; no source/sequence advance or callback clock order is assumed. - Exclude canonical raw, partial and complete `item.completed` usage only when the API row and PRP envelope match the authoritative owned run, company and agent, and their source event ID, instance ID and positive sequence agree. Missing context keeps the strict work check. A self-consistent accounting source can change after Stop. Keep the original snapshot and reject real tools, turns, messages and other item events, including productive output from a new source. - Retain private boundary and accounting proof. Source and sequence observations do not impose unsupported controller or clock order. Include the new oracle in the suite definition hash. Keep existing success-check IDs, exact three runs, model, budget, deadline, Stop proof and final outcome checks. - Reuse one binding validator in the active-work boundary and post-Stop filter. Pre-Stop keeps its current Runner/current-turn restriction. Add focused evidence tests and a standalone Chromium/control-API sequence check. The optional pre-Stop hook leaves existing classic callers unchanged; they do not call the accounting filter. ## Verification - Current source is `47578138ad357fc11b8b346bfada36d6aa5ab484`, one narrow fixture commit on reviewed `b9668ccfba61c59eb496ba18269aee06e2921d8f`. Only the prompt constant, two ordering controls, the existing Chromium sequence and the journey documentation changed. All oracle functions, run counts, model, budget, deadline and check IDs are byte-identical to b966. There is no new upstream merge or app change. - Fresh staging failure is preserved in the [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41227920892-1/). It measured harness `5d3f143d0aed4dc7cf472b7891ea3d4d95ab843b` against app `b7f1c3769c00a107ffe4128b48b0ef8cc452cf3f`; the reviewed Stop helpers and its flow were unchanged in that harness. The original grade has `stop_boundary_already_settled`, two succeeded native runs, and complete cleanup. Authentication, setup, collection and cleanup passed; the journey failed. This attempt remains failed. - Causal timing evidence: the first shell command started at event108 and completed at111 about 90 seconds later. The first structured complete current-turn usage arrived at113, after the hold ended. Replaying the exact retained event metadata never qualifies that hold. The only later eligible command intervals lasted about 0.90 and 0.18 seconds. The emitter publishes positive usage when the provider sends its assistant-message usage update. The fixture now completes a harmless first call before requesting the long hold; no usage is supplied or invented by setup. - On current source, all 137 focused tests and journey contracts passed. Use `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts product-journeys/stop-redirect-evidence.test.ts product-journeys/agent-evidence.test.ts product-journeys/active-followup.test.ts native-event-envelope.test.ts product-journeys/contracts.test.ts --maxWorkers 1`. New controls reject every prefix when first usage arrives only after the hold completes. They accept the same-turn sequence of completed readiness, genuine complete usage and a separate active hold, and reject it after that hold completes. - Manual Chromium qualification passed: `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/stop-redirect-boundary.browser.mjs`. The browser does not click Stop for the failed ordering or completed readiness alone. It clicks once for a separate active hold with bound usage. It sends one redirect only after actual settlement. Foreign usage, productive output and missing accounting still fail. This isolated control uses controlled responses and does not call a model or the deployed app. - Strict scoped TypeScript, maps, diff and secret checks passed. The official credential-free Node24 QA runtime-import preflight passed without network access. Full local E2E typing previously encountered old borrowed workspace declarations; clean CI typecheck passed on b966. A full local repository build/test was not repeated for this four-file fixture change. Fresh canonical CI must supply the clean full gates. - Historical binding proof remains valid: the exact old d499 filter failed 17 of the final61 Stop tests. The shared validator checks authoritative run/company/agent plus consistent row/envelope source identity. Bound raw/partial/complete accounting is allowed after Stop, including a consistent source change; productive output remains strict. Pre-Stop still requires its current Runner and turn, complete nonzero usage and a running command. - Historical b966 CI `41224054720` passed all59 terminal checks, with exact-head Greptile and Paperclip Review each5/5 and no open findings. These are not approvals of current475. Current475 has fresh exact-head Greptile5/5 and Paperclip Review5/5, with zero unresolved findings and no omitted thread pages. Canonical CI [`41229185200`](https://github.com/paperclipai/paperclip/actions/runs/41229185200) passed on exact475. All59 checks are terminal:55 success,3 expected skips and1 optional neutral review, with zero failed or pending checks. No old job was retried. Fresh normal staging run41229965802 passed. Its [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41229965802-1/) agrees with the independently read final original grade:11 checks, all5 phases, exactly3 native runs and cleanup passed. The final manifest is complete, with cleanup complete and no pending evidence. Smoke, report publication and Slack outcome jobs all succeeded. - The live pass measured harness `6be20bef6d0c5077adc1ae99562ad84e756ab167` against app `b7f1c3769c00a107ffe4128b48b0ef8cc452cf3f`. Its Stop oracle, flow, shared binding validator, unit cases and Chromium control are byte-identical to reviewed475. The shared agent-flow file adds only the hiring import and early hiring dispatch; all remaining bytes match. This records the actual harness and app identities separately. - Canonical run readback confirmed the first run succeeded, the second was cancelled and the third succeeded. The cancelled run had a ready native-producer accounting receipt, no pending accounting and a saved settlement time. After verified file copyback, Core queued final destruction and followed its normal five-minute cleanup backoff. The exact lease naturally reached expired/cleanup-success with a bound destroyed receipt before the third run started. No operator cleanup, source change, timeout extension or workflow retry was used. The earlier failed staging attempt remains failed. - Current475 is ready for the normal reviewed merge and QA admission process. Source stays frozen; merge and configuration changes belong to that process. ## Risks - Some provider profiles may not expose a valid current-turn usage report while the command runs. The stronger boundary fails closed in that case. - Accounting may remain incomplete after Stop. The redirect stays unsent within the original deadline. This change does not waive missing debt, treat it as zero, raise a budget or change provider behavior. - This repairs the test boundary. It does not prove or repair a general provider shutdown-usage issue. A separate local late-usage ordering control is not attributed to this QA incident. ## Model Used - OpenAI Codex, GPT-6 family. The exact deployment variant and context-window size are not exposed in this session. Used code reasoning, repository tools, test execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-31 13:49:02 -05:00
Dotta	3937215a05	Qualify explicit native written references in the basic question journey (#18459) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Basic task journeys must prove that people can answer an agent and receive saved work. > - The question journey asks for a choice, then a typed reference, then a saved note. > - A native choice card can offer a real written-answer control as well as predefined options. > - The old observer rejected that form before a person could type the reference. > - This pull request accepts that explicit native control and keeps exact answer and work evidence. > - The separate text-only and named-tool qualifications keep their stricter contracts. ## Linked Issues or Issue Description Refs #18346 and #18456. The integration inherits the reviewed fixture prerequisite #18387 and Agent Chat recovery #18465 through their actual merged parent. The producer fix in #18456 is separate from this basic journey contract correction. Public duplicate PR and issue searches found no other correction for this form equivalence. **What happened?** The basic question journey requires the reference card to have `answerMode: text`. It rejects a native single-select card even when its canonical form explicitly enables a written answer. This makes the observer test a widget shape in addition to the requested behavior. **Expected behavior** The person must make a real choice and type the complete reference. A native card with an explicit written-answer control can satisfy the second step. Selecting a predefined option cannot satisfy it. The agent must resume the bound work and save both actual answers in its own task document. The document must survive reload. **Steps to reproduce** Use the focused recording and real QuestionForm browser test in this PR. Keep the reference as a single-select native question with one predefined option and `customAnswer.enabled: true`. The previous grader rejects `question-resume.forms`. The previous browser helper rejects `single_select` before it uses the written-answer control. **Paperclip version / commit** Original source base `f2db909393de124caf6a5380a52d4e0e1d621661`. Current integration `a417fd827347ed6f7998774a9de6aee70b76d867` merges exactly `12b0cc630471ee151ec5bc8e41a34f09236d6723` into reviewed question source `faec4ce6d775a34d303d200987e4e84d6dc26caf`. The scheduled failure on `82d15946c5844d3ae41c6ade62fa224053f20a9a` remains a failure. Its saved card had no canonical written-answer capability and would still fail this change. The retained normalized events do not prove whether the original provider flag was omitted or false. **Deployment mode** Attached native task journeys, including QA Cloud. This PR changes the observer and grader. It does not change the app, provider prompt, model, budget, timeouts, cleanup, or runtime deployment. ## What Changed - Accept one provider-bound single-select reference card with explicit canonical custom answers and valid, distinct options. Keep pure-text reference support. - Make the browser select the actual written-answer control and fill the real rich input. Opt in only for the basic question journey. - Keep the exact no-option typed response, immutable card, board actor, native request and continuation, saved agent document, reload, model, environment, checkpoints, and existing one-to-three-run checks. - Reject missing or disabled custom capability, inferred legacy capability, malformed or extra questions, foreign native identities, changed cards, substituted options, and wrong or missing reference text. - Add a real production QuestionForm and TaskChatRichInput Chromium probe. It first selects a predefined option, then proves the helper clears it and submits only the exact custom text. - Document the contract correction and preserve the historical failed attempt and its separate cleanup outcome. Keep necessary-question and named-tool documentation qualification unchanged. ## Verification - Before the observer change, the focused grader regression failed only the new valid-written-reference case; the other 108 cases passed. - Before the browser helper change, real Chromium passed the text control and four rejection controls but failed the valid native written-answer control at the old pure-text assertion. - Initial focused Runner E2E unit selection at `af07f4de`: **256 passed across 9 files**. This includes question/resumption, clarification evidence/flow/collection/deadline, necessary question, documentation, contracts, and report catalog tests. - Initial real Chromium selection at `af07f4de`: **12 passed**. Six new production QuestionForm/control checks and all six existing document-readiness checks pass. No form or rich-input component is mocked. - Historical test-only head `faec4ce6d775a34d303d200987e4e84d6dc26caf` adds a direct canonical-absent legacy `freeText` grader negative and renders the actual legacy presentation in the browser. All four production helper blobs remain identical to the reviewed head. A temporary legacy inference mutant makes the new negative fail. After restoring production source, **179 focused unit tests and all 6 real QuestionForm checks pass**. The six document-readiness specs and their helper are unchanged. - Original [CI run 41205748589](https://github.com/paperclipai/paperclip/actions/runs/41205748589) failed the staged-listen Runner handoff test; that failure is preserved and was not retried. Its Runner source closure was identical to an earlier passing canonical lane. The symptom also recurred on other heads and is tracked independently; no infrastructure attribution or live pass is claimed. - Current integration `a417fd827347ed6f7998774a9de6aee70b76d867` preserves **all 12 question files byte-for-byte** from faec. Every other tracked path is identical to exact merged parent `12b0cc630471ee151ec5bc8e41a34f09236d6723`, which contains actual prerequisite `baf651c8d0122efc93f3ef03e891919eeb666215`. There is no manual conflict resolution or additional production edit. - On that integrated head: **257 focused Vitest tests across 9 files, 206 required Node source/reporter tests, and 12 real Chromium tests pass**. Map/module-boundary checks and catalog discovery pass. The actual `--preview` command passes with a synthetic question target and an intentionally unavailable adapter; it performs no credential read, adapter loading, provisioning or model call. It retains maxRuns3, a 600000ms campaign bound and zero retries. - Exact-head [CI run 41210806950](https://github.com/paperclipai/paperclip/actions/runs/41210806950) passes, including official typecheck, build, all eight browser shards, both Runner Vitest lanes and aggregate verify. All 59 current PR contexts are terminal with no failures. Fresh [Greptile summary](https://github.com/paperclipai/paperclip/pull/18459#issuecomment-6543878408) explicitly reviews `a417fd827347ed6f7998774a9de6aee70b76d867` at 5/5 with no findings; its exact-head check succeeds and all threads are resolved. No fresh deployed journey pass is claimed yet. - Catalog discovery lists the existing question journey. Map checks and module-boundary checks pass. Diff checks pass. - `tsc -p tests/runner-e2e/tsconfig.json` was attempted. The fresh integrated checkout still lacks compiled Runner, plugin SDK, and adapter declarations. The attempted local check reports 344 diagnostic lines from dependency/type closure and zero diagnostics in all 12 question-owned paths. Official exact-head CI establishes the full typecheck/build gates. Full repository build, typecheck, and test suites were not repeated locally for this observer change. - The first broad local report selection lacked the CLI loader and then hit sandbox IPC restrictions. Locked dependency installation and an authorized local IPC run resolved those setup errors; the full selected 256 tests pass. The initial localhost probe also needed the normal local-listen permission. - No fresh QA pass is claimed. The coordinator must qualify the final reviewed runtime and harness together. Prior failed results remain preserved. ## Risks The basic journey accepts one additional form with explicit native provenance. It still rejects the original closed card without a written-answer capability. The separate producer default fix must reach the selected remote provider pack before it can affect a deployed run. Local browser and grader tests do not prove that artifact served or that a model completed the live journey. Existing cleanup and physical-stop obligations remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, browser testing, and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked related work above - [x] I have described the issue in-PR following the bug template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-31 13:16:40 -05:00
DottaandPaperclip	322fa6b65c	fix(ui): restore task document authoring and add persistence journey (#18356) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Task documents keep versioned work beside each conversation. > - The default chat interface could read general documents but hid their editor. > - People also need unfinished edits to survive a tab switch or reload. > - This pull request restores the shared editor in Artifacts. > - A portable browser journey checks exact saved content and revision history. > - The benefit is a usable document flow with independent deployment checks. ## Linked Issues or Issue Description **What happened?** A person using the default task interface could read a general task document but could not create or edit one. The existing editor was available in Classic mode. A pending create could also leave newer text as a new form after its editor unmounted, causing a conflict against the person's own successful creation. **Expected behavior** A person can create and edit general documents through Artifacts. Titles, bodies and revision history persist after reopening and reloading the task. Unfinished text survives tab switches. A successful create makes retained newer text an edit of that confirmed revision. Later writes from another actor still cause a conflict. **Steps to reproduce** 1. Open an ordinary task in the default chat interface, then open Artifacts. 2. Try to create a general document. The previous interface has no authoring control. 3. With authoring restored, submit a new document, keep typing, and switch to Properties before the request finishes. 4. Reopen Artifacts. The previous draft path restores a new form despite the successful create. Related public work: #17598 adds document copy/download and #18157 adds image galleries. The selected actual merged parent is `f1d780838218f975064f38333946805e0ec103a5` from #18427. It includes the stable profile-entry ordering repair and the exact recovery-fixture prerequisite. The resulting PR diff contains 32 document and reporting files. ## What Changed - Reuse `IssueDocumentsSection` for general documents in default-chat Artifacts. Keep Classic authoring in its existing task body. Ordinary tasks, conversations and Coordinator mode share the same editor options and signed-in permission wrapper. - Preserve Plans, file-review documents, attachments, uploads and work products. Hide duplicate general-document cards. Use the same general-document key rules for visible rows, validation and refused writes. - Forward image uploads, mentions, references, author maps and feedback to both editor mounts. Keep empty mention/project values stable while queries load. - Back up unfinished forms within their signed-in account, task and editor scope. Preserve text entered before its key. Record a successful create on its matching retained newer buffer, including after unmount or an early reopen. Preserve newer title/body edits, real remote conflicts and matching cancellation cleanup. - Add the model-free `task-document-persistence` journey. Setup creates only an owned empty backlog task. The browser creates and edits the document; normal APIs independently check company, task, actor, exact content and both revision records. - Reopen the owned task through the explicit Backlog view, then reload it. Require exact UI readback and zero native runs. Add only the Doc registration, definition-hash inputs, report mapping and four closed checks to the shared harness. - Count explicit zero-run browser receipts after private journey metadata is removed for publication. Preserve missing billing evidence, positive usage and native/protocol accounting. - Update browser assertions for the restored inline editor while retaining exact download bytes, revision collisions, paused-state and no-run checks. Update the feature map, usage docs and shared editor stories. ## Verification - Latest focused checks use official Node 24.22.2: 568 journey, catalog, report, billing and attempt-accounting tests across 25 suites pass. All four admitted journeys remain registered. The seven relevant UI suites pass 529 tests, including the task-page container after the parent merge. Maps, token gates, the PR-scope diff check and the owned credential-pattern scan pass. The normal QA import preflight also passes with an empty environment and network access blocked. - Billing regression: before the repair, the explicit empty public receipt counted one run, and two positive-usage receipts were discarded by browser metadata. Those three cases fail before and pass after the repair. Two additional red regressions cover aggregate usage beside an empty run list. That ambiguous shape now keeps an unavailable receipt and a minimum count of one. Missing receipts retain unavailable billing, and native/protocol one-or-more run totals retain their measurements. - The previous fresh CI run failed its unchanged snapshot-only recovery fixture: one global sweep did not call its callback. The original failure remains recorded. An isolated PGlite probe with all canonical migrations exercises the real reconciler: it processes two earlier sources, then reaches the owned third source on the next sweep. It preserves source evidence, files and run IDs. This proves bounded pagination, not full native snapshot repair. - Local native recovery tests could not start PostgreSQL because the host semaphore limit was reached. That run executed zero cases. Canonical Linux CI later passed all 143 fixture cases, including the original snapshot-only case and the new competing-candidate control. The exact fixture repair is now inherited from #18427 and is absent from this PR diff. - The portable oracle rejects 19 stale, missing, foreign or incorrect outcomes. Result validation also rejects model/provider metadata and native runs for this case. - Five real Artifacts-to-Properties unmount regressions fail before the repair and pass after it. They cover both editor modes, reopening before and after completion, later typing, and a foreign revision that must still conflict. The edit-storage failure case also checks that Cancel clears the promoted fallback copy. - UI coverage includes default chat, Classic, conversations, Coordinator, Plan rendering, document-card deduplication, reserved keys, pending-query stability, permissions and rich editor options. - Earlier real browser checks passed five upload cases and two artifact-arrival cases. They retain exact CSV bytes, original/revised content, collision, paused-state and no-run assertions. Current-head normal CI runs the updated browser suites again. - The branch merges only the selected actual parent `f1d780838218f975064f38333946805e0ec103a5`. The merge has no conflicts or manual resolution. The SQL service and recovery fixture match that parent exactly. The own document editor, persistence flow, saved-state oracle and billing repair retain their previously qualified source blobs. Follow-up, clarification, question/resumption, screenshot helpers and the Steering map are byte-identical to the merged parent. Existing project/search/question/follow-up registrations remain present. - Required document source blobs remain exact: Doc UI `22e80663b053363e5f75e1432dd9043dc0e70243`, its unmount tests `16f3f4ed0dc9b11aaff129e06e6cf2bbf4522713`, readiness helper `800a747284db19b8484650b6759e8bd1f356839d`, and budget service `3a15d489f0aa55c22e8f7fe6a6bc6a0e14e5914e`. - Local typing is limited by borrowed declarations from another checkout. They omit current `skill_test` and sandbox creation-operation fields. No dependency declarations or repository config were changed. The authoritative clean CI Typecheck + Release Registry gate passes on exact head `2ffce29827ae170420d740940c1c069c7225a74c`. - Full monorepo typecheck, tests and build were not run locally. Borrowed build declarations from another checkout limit local typing. Authoritative [clean CI](https://github.com/paperclipai/paperclip/actions/runs/41171717008) passes on exact head `82d15946c5844d3ae41c6ade62fa224053f20a9a`: 55 successful checks, 3 skipped and 1 neutral, with zero pending or failed checks. Typecheck, build, Runner, server and browser lanes pass. Greptile gives that same head a fresh 5/5. All 15 review threads are resolved, with no further thread page. The PR is mergeable and clean. - Failure history remains recorded. The earlier recovery failure was repaired with a bounded, owned-source fixture. A later CI worker shut down after 2,375 Runner unit tests passed. Its one authorized narrow retry passed those tests and all five native integration suites. The separate AgentConfig browser failure was reproduced and fixed by the merged #18427 parent. These earlier passes do not replace current-head CI. - Final-source live qualification passed on exact harness and served application `82d15946c5844d3ae41c6ade62fa224053f20a9a`. QA run `41177696858` completed all four closed checks and all five phases with zero native runs. Cleanup passed; the durable manifest is complete with no pending evidence. The coordinator verified the original grade and manifest. - The [public qualification report](https://pages.paperclip.ing/evals-qa-smoke-41177696858-1/) records zero agent runs, empty run IDs and empty run usage. Pricing remains unavailable and billing remains incomplete; the report does not claim measured zero cost. - The earlier harness `36c8d9ddd76d0b7ddb28a9a6c674d9e538f6df88` pass against application `2ffce29827ae170420d740940c1c069c7225a74c` remains historical evidence. It is separate from the final-source measurement above. - The selected parent brings profile-entry ordering, proactive chat, a database migration, Runner updates, and task-page chat/read behavior. Its new task-page container is included in the final application measurement. The coordinator retains the original evidence and owns the primary merge and daily-plan admission. ## Risks Artifacts now mounts the existing shared editor. Its autosave, conflict, revision, annotation and server access contracts remain in place. Tests cover both interface modes, shared scopes, later remote writes and unavailable edit storage. Plans and generated file-review documents keep their existing flows. The PR adds no database schema or API change. The shared billing repair requires explicit captured empty run IDs and an unambiguous usage object containing only an empty run list for public browser-only receipts. Other fields beside an empty list keep conservative unavailable billing. Missing measurements remain unavailable. Contradicting positive usage remains counted. The inherited recovery fixture keeps the production sweep limit and its source, readback and no-agent-work assertions. The live journey must fail when authoring controls or saved outcomes are absent. It must not create the expected document through setup APIs or switch to another interface to pass. Final deployment verification must retain the actual app and harness revisions separately. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-31 01:49:24 -05:00
DottaandPaperclip	1353a232f2	test(e2e): add a portable active follow-up journey (#18345) ... ## Thinking Path > - Paperclip lets people direct agents while their work is active. > - Product E2E checks must prove that saved user input reaches the intended work. > - The existing active chat story depends on a file in a local agent workspace. > - That setup cannot qualify an attached deployment with remote agent execution. > - This change adds one shared active follow-up journey through the normal task UI and API. > - Independent native operation, saved input, final reply, and cleanup checks make the result inspectable across targets. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for a user follow-up sent while a native agent is working. **Current behavior** The classic chat story holds work open with a local file. The shared deployment-targeted harness has no active follow-up journey. A running heartbeat alone cannot prove that work was active when input arrived. **Proposed behavior** Run one bounded command through the real native provider. Observe its active execution before and after one follow-up is saved. Require one final reply with the late requirement, correct task and agent identity, an empty input queue, and visible readback after reload. Use the same story on managed and attached targets. ## What Changed - Add a shared `followup-while-running` journey with normal task creation and reply submission. - Require a bound native turn and pending shell execution at the input boundary. Retain a closed receipt with source identity and a row hash. - Accept one steered work run or one normal successor (1–2 native runs). Preserve every other journey's declared run contract. - Reuse the saved native final reply observer, run binding checks, usage collection, target lifecycle and private evidence path. - Accept canonical builtin/process execution events independent of provider tool display names; calibrate against real OpenCode, Codex and ACP mappers. Reject Paperclip semantic tool names across native wrappers/prefixes and unnamed or ambiguous builtin calls. - Recognize only the observed legacy PRP v1 builtin/bash/unknown/readOnly=false command shape and fail promptly when owned work finishes before a live boundary can be observed. - Ready the task page/composer before the observed command boundary, use a bounded 90-second command, and preserve active-after-persistence proof. - Add adversarial oracle and retained-result calibration. Register the journey, include its sources in the definition hash, and document its coverage. - Keep six real browser document-readiness checks and verify shared-deadline behavior with a deterministic clock/state test that rejects timeout-reset mutations. ## Verification Exact current head `e72e20ad111bd94728d38606015d1fdac3c92929` passed [full CI run 41142552195](https://github.com/paperclipai/paperclip/actions/runs/41142552195): all 49 workflow jobs reached accepted terminal results, including typecheck, build, every test lane and aggregate verify. All 59 reported checks are terminal: 55 success, three expected skips and one neutral 5/5 Paperclip review. Snyk passed. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/18345#issuecomment-6528913235) explicitly names this exact head and gives 5/5. No review threads remain open. The PR is mergeable. These source gates do not establish a live QA pass at this head. Fresh protected QA at this exact source and the separately recorded deployed application/runtime remains required before primary merge and daily admission. Historical live QA staging **PASS** on qualified commit `af9e091b15b5e24b6b2cc1b424523430cc2ab024`: 6/6 checks, all five phases including cleanup, two native work runs, and the configured model/environment check passed. The [sanitized qualification report](https://pages.paperclip.ing/evals-qa-smoke-41122620146-1/) records that exact measurement. Final integration head `e72e20ad111bd94728d38606015d1fdac3c92929` merges exact actual squash `18b1637b6371a6a6adade857e2a73654ee626fef` from [#18254](https://github.com/paperclipai/paperclip/pull/18254) into previous head `e54d64743964b1a19d165537bebbe62c50389a23`. The merge had no conflicts. All 18 files in this PR's change set, the complete 45-file shared journey directory, and six shared native/evidence helpers retain their exact previous-head Git blobs: 59 scoped files in total. The production clarification helper remains blob `800a747284db19b8484650b6759e8bd1f356839d`. Follow-up 1–2 runs, question 1–3 runs, older exact counts, owned native event checks, active-before/after persistence proof, cleanup/accounting and feature mappings remain unchanged. Source integration head `e55d74491f5bcd99c1455e9ace02bd7ad7319fae` integrates exact actual squash `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` from [#18385](https://github.com/paperclipai/paperclip/pull/18385). The merge had no conflicts. All 17 files in this PR's change set and six shared native/evidence helpers retain their exact Git blobs from previous head `162e341cda5b627639b486b4019a1675bb018ea2`. This preserves the follow-up and question readiness checks, feature mappings, six closed follow-up check IDs, immutable before/after active-operation proof, follow-up 1–2 runs, question 1–3 runs and older exact counts. Previously reviewed head `e54d64743964b1a19d165537bebbe62c50389a23` adds a separate test-only reliability repair on top of that source integration. It replaces one scheduler-sensitive browser case with a deterministic Vitest clock and locator-state model that calls the actual production helper. The six other real Chromium checks remain byte-for-byte unchanged and pass **6/6**. Focused unit suites pass **51/51**, including the deterministic deadline case. This is six browser checks plus focused unit coverage, not seven browser checks. The model makes the panel ready at 100 ms, the supplied answer ready at 600 ms, and full valid content ready at 1,300 ms. It requires rejection at the shared 1,000 ms deadline and records immutable rejection state before simulating a 10-second scheduler stall. Both restarted-timeout mutations fail: resetting all waits wrongly resolves at 1,300 ms; resetting matcher timeouts while keeping the final real deadline guard rejects only at 1,300 ms with valid content present. The genuine helper rejects at 1,000 ms with the answer still incomplete. The production helper is unchanged, Git blob `800a747284db19b8484650b6759e8bd1f356839d`. That two-file test-only patch changed no journey, application, run, accounting, model or runtime behavior. Its six-browser/51-unit results are prior-head evidence; the complete patch is unchanged in the new integration. The final source also retains the actual merged question-primary squash `d9354bbfc35ef74333e90e596ce21dc8e59149b9` from [#18346](https://github.com/paperclipai/paperclip/pull/18346). Later commits `409117451da260a03fc28a3107c7808ecbe68f8f` and `ea3e7b1d1bed562351fcf4722c7175001b987b59` bound question document readiness to one campaign deadline and test that waits cannot restart it. Commit `162e341cda5b627639b486b4019a1675bb018ea2` adds question/follow-up report feature mappings and coverage. These files are unchanged by the final merge. The imported upstream source has real application changes: [#18334](https://github.com/paperclipai/paperclip/pull/18334) makes managed skill content durable and extends native skill-test contracts; [#18350](https://github.com/paperclipai/paperclip/pull/18350) requires managed Cloud shared storage and bootstrap prerequisites. #18385 updates TestOps job metadata and repairs their combined maintenance fixture. The new merge additionally imports [#18383](https://github.com/paperclipai/paperclip/pull/18383) at `81ecaf0654fe79a2a2dbda6588f033e39ee610e4`, which bounds failed lifecycle verification; [#18357](https://github.com/paperclipai/paperclip/pull/18357) at `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`, which persists sandbox-creation ownership and updates cleanup/adapter/plugin contracts; and #18254 at `18b1637b6371a6a6adade857e2a73654ee626fef`, which accepts a valid late Antigravity authorization URL while native authentication drains. This PR's diff against exact upstream `18b1637b6371a6a6adade857e2a73654ee626fef` contains only journey tests, report coverage and documentation. Harness source and deployed application/runtime identity are separate inputs. This source integration does not deploy the imported application changes or claim the QA target runs this application revision. This final SHA has not been measured by the historical run above: **fresh QA at this exact final head is required before primary merge**, followed by normal TestOps verification/admission. Fresh exact-head CI and review have passed as recorded above. The live measurement must record the actual separately deployed QA application/runtime identity. Seven whole-file blobs remain identical to the qualified SHA. All paths are relative to `tests/runner-e2e/`: | Journey/helper | Unchanged Git blob | | --- | --- | | `product-journeys/active-followup-evidence.ts` | `83a5d9fab52abb9849a40e83eca6b6a9c29bf587` | | `product-journeys/active-followup.test.ts` | `728160c610c18292d2cf06d4a87d25a8719f1734` | | `native-task-reply.ts` | `59d11f1a178dad9db3da8fa2342ff93643cb9922` | | `native-event-envelope.ts` | `305b0b456c59c511ee328e529ad574c0583075a0` | | `run-observations.ts` | `08c8ad184a821a7cd536926b3d7e3d1156a1e3da` | | `user-actions.ts` | `e9611348dc41e17899367dab2e3b36d3d1be9480` | | `stop-redirect-flow.ts` | `4580bd605a425bc72511e02d988492acbec8d60e` | The follow-up preservation comparison against historical `af9` has three explicit exceptions; no whole-file identity is claimed for them. The separately documented question readiness and report additions above also require final-source verification: 1. `product-journeys/agent-flows.ts` adds `question-answer-resume` to the `AgentJourney` type exclusion, alongside follow-up. Its remaining contents and runtime helper bodies are byte-for-byte unchanged. Qualified blob `a0afaa968c1d854476159fef21de7b6dd50a4a98`; final blob `0cc797c0ae594e7769338e63a80c4f67e29e4a54`. 2. The common run-count helper is centralized in `success-contract.ts` and re-exported by the catalog. It accepts follow-up **1–2**, clarification **1–3**, and each older paid journey's exact count. The follow-up catalog entry, maximum of two runs, six required checks, native dispatch, collection and retained-run contract remain byte-for-byte unchanged. The helper's text differs; its preserved count semantics are verified explicitly. 3. `product-journeys/active-followup-flow.ts` changes exactly one composer readiness assertion from Playwright's implicit five-second default to an explicit 30-second timeout, matching `submitTaskReply` for attached targets. Prompts, actions, native evidence, before/after persistence proof and grading otherwise compare byte-for-byte unchanged. Qualified blob `64392329198a1a4d9828e32834d4507e175216e6`; final blob `2746b5ba89e2af26b4097ba7897913186786ac5d`. This correction is why a fresh final-head QA run is required. - **310 focused tests passed at current final head `e72e20ad111bd94728d38606015d1fdac3c92929`**: 286 journey/evidence/catalog/report/shared-deadline tests in 14 files and 24 native authentication/real-child stream-order tests in two files. Both invocations used Node24 and one Vitest worker. The earlier `e55` integration also passed 365 focused tests, including 40 Cloud configuration and 40 TestOps metadata cases; those results remain prior-source evidence. - **157 explicit count assertions passed again at the current final head**, including both interactive ranges, older exact counts, invalid/fractional/non-number values and catalog/contract export identity. The maximum bounds and six closed follow-up check IDs also matched. - Maps and diff checks passed. The official QA runtime-import preflight passed on Node24 at the current final head without network access. It uses a dummy adapter and no credentials. It does not prove target connectivity, sign-in, model readiness or deployed application/runtime identity. Those require the normal protected QA run. No model/provider work or optional Runner release was dispatched. - Current local runtime dependencies use ignored isolated workspace-source views. No shared dependencies or compilers were changed. A previous local typecheck attempt hit stale reused Runner declarations. No full repository typecheck, test run or build was repeated locally; the successful exact-head full CI run above now supplies these gates. - CI on previous head `162e341cda5b627639b486b4019a1675bb018ea2` failed the Cloud/local maintenance fixture in [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133716802/job/124331643100), plus dependent verify. The actual tested merge combined that head with master `198218bf554877fe3b7a83742eefb3b8f12392c6`. A Cloud child selected local-disk storage and hit the new shared-storage policy. This was a deterministic upstream fixture mismatch; no unchanged failed-job retry was used. - The actual #18385 repair configures a valid Cloud child, verifies local bytes independently, checks the expected provider mismatch and retains exact stored-config readback. Its own final head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57` passed 59 checks, including [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133876636/job/124331450635). That prior result supports the repair; it does not replace this final head's fresh CI. Separate repair candidate #18392 and its source/evidence are retained as superseded history. - A local attempt to run the database fixture could not start PostgreSQL because the host exhausted macOS semaphores. A first setup attempt also failed package resolution before test execution. Skipped database cases and these setup failures are not passes. No kernel settings, shared processes or databases were changed. - The complete previous-head CI run [41138618869](https://github.com/paperclipai/paperclip/actions/runs/41138618869) finished with one underlying failure: [Runner Vitest shard 1](https://github.com/paperclipai/paperclip/actions/runs/41138618869/job/124347859445) expected native Antigravity login success but observed failure at `antigravity-auth.test.ts:101`. The lane had one failed test, 1,994 passing tests and eight skipped tests. Aggregate verify failed only because of this lane. All other 57 checks were accepted, including build, typecheck, all server/browser suites, Runner shard 2, Snyk and exact-head 5/5 reviews. No unchanged retry was requested. - The independently reviewed #18254 repair removes an invalid cross-pipe ordering assumption: a valid URL on stderr can arrive after the authenticate response on stdout during drain. Its real-child regression forces both orders. The current final source imports that actual squash; the 24 focused authentication cases pass here. Fresh full CI and exact-head review have now passed for this source, as recorded above. The earlier failed result is not relabeled. - The original live compatibility failure is retained: an older PRP v1 pack labeled builtin `bash` as operation `unknown`, causing the observer to miss the command and never send the follow-up. Its exact shape was reproduced red before repair. Generic unknown operations, semantic tools, wrappers, read-only work and foreign provenance remain rejected. The historical qualified pass used the repaired oracle; no failed result was relabeled. - Earlier CI on `6fad965d69` hit an unchanged heartbeat test's three-second adapter-call wait; its original failure and one narrow rerun remain recorded. The qualified SHA subsequently passed CI. An earlier report-catalog subprocess failure is retained. The final focused Node24 run now includes that suite and passes after the documented report feature mappings. ## Risks The fixture asks the native agent to run one bounded `sleep 90` command. The observer must see actual active native execution; a fast or noncompliant turn fails rather than passing from elapsed time. A completed result requires one or two bounded native work runs, source attribution, configured model/environment, and collection. Unknown provider prices stay unknown. This change does not alter production execution or grant new access. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with available version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs; existing follow-up execution fixes do not provide this deployment-targeted test - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the new focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 18:37:17 -05:00
DottaandPaperclip	d9354bbfc3	Add a portable agent question and answer resumption journey (#18346) ... ## Thinking Path > - Paperclip helps people manage agent work and keep human decisions attached to tasks. > - Product E2E journeys verify the browser, native agent, saved state, and cleanup. > - The classic suite already checks a choice question followed by a text question. > - Deployment-targeted journeys need the same behavior on an attached instance. > - Production questions can use provider input, the dedicated semantic tool, or the authenticated native API fallback. > - The basic journey must verify the original card and native continuation for each supported path. > - This pull request shares the scenario and browser actions, adds strict API receipt evidence, and verifies the saved document after reload. > - Separate tool-documentation qualification and the original failed measurements keep their existing scope and outcomes. ## Linked Issues or Issue Description **What existing behavior does this improve?** It adds portable coverage for answering agent questions and resuming the same task on an attached deployment. **Current behavior** The classic suite has the two-question scenario. The portable catalog does not expose it. Its shared native-path grader recognizes provider and dedicated semantic receipts, but rejects a valid completed native API creation receipt. **Proposed behavior** Expose `question-answer-resume` as a deployment-targeted journey. Answer one choice question and one text question in the browser. Require server-owned creation evidence, unchanged forms, exact board answers, bound continuation, an agent-authored task document, and readback after reload. **Reason and benefit** The same story can qualify local and hosted targets through their normal provider setup and cleanup. It catches lost answers, duplicate or foreign work, premature output, and stale documents. ## What Changed - Register the journey with a three-run maximum and positive model allowance. Accept one to three attributed work runs at both collection and retained-result boundaries. - Share the existing two-question scenario, browser answer actions, and behavioral grader. Keep separate tool-documentation grading unchanged. - Recognize a completed native API creation receipt only when its HTTP 201 result retains the original pending card, exact task/company/agent/source run, idempotency key, and full normalized form. Require the receipt to remain unchanged after the answer and through the final checkpoint. Missing, pending, foreign, or altered proof still fails. - Check signed-in board ownership, each native continuation, no premature or extra output, document authorship, and the saved revision after reload. - Wait for the authenticated product region before screenshots. Recheck origin and path after readiness. Bound readiness and capture by the campaign deadline. - Add closed stage-specific errors, catalog coverage, sanitized receipt calibration, and documentation. Include the provider-form and browser-question presentation helpers in the definition fingerprint. ## Verification - Fresh QA staging qualification passed at `7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks passed, cleanup passed, and all three native runs used the configured QA target and model. The [sanitized run report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/) records the measured pass. This is one successful qualification, not a reliability claim. - Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3` contains the one main merge through `fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of the four-file qualified search fix `4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity matches. It preserves the project and search registrations and atomic search readback. The question live qualification ran at the earlier SHA above; the final integration head has separate local and CI verification. - Eighteen question journey, helper, and test file blobs are identical between the qualified and integration SHAs. This includes clarification flow and evidence, question grader and browser helper, the sanitized receipt fixture, screenshot helper, provider-form helper, browser question presentation, continuation scenario/readiness/flow, shared agent validation, and user actions. The question run-count contract and dispatch branch are also unchanged. Reviewers can compare those paths directly between the two commits. - All 622 selected Product E2E, catalog, question-resumption, necessary-question, report, and readiness tests pass after the main merge. After the exact search delta, all 145 narrow question/search/project/contract tests and all nine real Chromium search regressions pass. Map checks, combined credential-free preview, source fingerprints, and the earlier workspace-link preflight pass. The local TypeScript attempt reports stale Runner declaration errors for `onAccountingEvent` in unchanged native-session code; fresh official CI provides the clean-build type gate. The three search source/test blobs match the qualified search merge, and the three project source blobs are unchanged. - An in-memory probe executes the actual launcher fingerprint expression. Each project, search, and question implementation input changes the digest. Separate red probes first reproduced the omitted provider-form and browser presentation helpers; both pass after their source inputs are added. - The sanitized retained native API response-shape regression fails before the grading correction and passes afterward. Negative controls reject missing, pending, foreign, changed, spoofed, or wrongly bound receipts, answers, continuation, model, environment, and document evidence. All nine provider/semantic/API path combinations pass calibration. - The real local Chromium screenshot probe fails before the readiness repair and passes afterward. Login redirects and sign-in pages at task URLs remain excluded with no screenshot files. - The two original attached failures remain retained: the first stopped at early screenshot capture; the second stopped before answering at the former API provenance gate. An earlier sandbox run also retained one report subprocess failure; the full selected tests then passed with local subprocess IPC access. - The prior integration CI had an unchanged canary mock `EADDRINUSE` failure. That history is retained; its documented retry is owned by the coordinator. No application API, schema, production UI, model, runtime policy, credential, or deployment configuration change is included. Final integration head ae0d9d2898 passed all 58 CI checks, the external Snyk status, and fresh Greptile 5/5 with no unresolved findings. This clean CI includes the official TypeScript and build gates. Recurring selection follows merge and a separate normal TestOps job verification. ## Risks An API result must prove native creation; form shape and run count cannot establish it. The grader checks the server receipt, exact scope, unchanged card and receipt, real board answers, and bound native continuation. The shared answer helper still serves classic journeys. The screenshot wait affects attached journeys and remains limited to the authenticated product region and campaign deadline. One live QA pass establishes the measured journey outcome at the qualified SHA. It does not establish reliability across models or deployments. The integration SHA preserves its journey and helper blobs, while shared registry changes receive fresh CI and review. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 16:18:21 -05:00
DottaandPaperclip	4edd10804e	fix(evals): batch task search result readback across renders (#18370) ... ## Thinking Path > - Paperclip lets people manage AI agents and saved work. > - Product journeys check user actions against saved state. > - The task search journey compares the exact visible task identities. > - A search can replace rows while the harness reads an old indexed locator. > - The attribute read can then time out after the correct results appear. > - This pull request reads the visible links together and preserves strict result checks. ## Linked Issues or Issue Description Refs #18355. **What happened?** An attached Cloud attempt passed the task search journey. A later attempt on the same harness and app revision failed at search-result readback. Its owned screenshot showed the correct target and completed control. Authentication, collection, and cleanup passed. The failure remains recorded and the journey has not been admitted to the daily plan. The reader called `all()`, then awaited `isVisible()` and `getAttribute()` for each indexed locator. When a search replaced the old collection between those reads, the next attribute read waited for an index that no longer existed. **Expected behavior** Read the matching visible links together. Compare the complete identity set against the owned target and controls. A render transition must not consume the whole poll deadline on a stale row index. **Steps to reproduce** 1. Render three task links in the task collection. 2. Let the reader observe the third link as visible. 3. Replace the collection with the two correct search matches before its attribute read. 4. The old reader times out waiting for `nth(2)`, even though the two correct rows are present. ## What Changed - Read visible link attributes in one batch. Discard a detached snapshot and retry within the existing poll deadline. - Keep the exact identity comparison, duplicate detection, control exclusion, saved-content checks, and zero-run checks. - Add a real Chromium render-transition regression and invalid-result cases. Extend the stateful flow test for a detached snapshot. - Document the readback boundary. No app UI, target configuration, or cleanup behavior changes. ## Verification - Review follow-up: the existing journey wait helper is now exercised with the real Playwright poll. Its first read evaluates the actual reader callback on detached browser DOM nodes and returns no evidence. The second read contains an extra owned control. Only the third, exact pair succeeds. The test observes all three reads. The 30-second deadline and grading behavior are unchanged. - Rebased onto current master, including the merged project journey. Credential-free discovery confirms exactly one `project-task-context` and one `task-search-filter` entry. - Latest rebased checks passed: all nine Chromium search regressions, all 381 Product journey unit tests, runner typecheck, discovery, maps, and diff checks. Exact head 183c3e1d19 passed all 58 CI checks, external Snyk status, and fresh Greptile 5/5 with no unresolved findings. - Red: the real Chromium transition test failed on the old reader with `locator.getAttribute: Timeout 150ms exceeded`, waiting for `nth(2)`. The short timeout bounds only this regression fixture. - Green: all nine credential-free Chromium search tests passed. They cover the transition, duplicate/extra/unknown/wrong destinations, visible-link scope, and status-label behavior. - All 131 focused search/oracle/contract/campaign tests passed. - Runner typecheck passed: `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - `node scripts/check-maps.mjs` and `git diff --check` passed. - Fresh QA staging acceptance passed at this exact head: all five checks, authentication/setup/journey/collection/cleanup phases, and zero agent runs. The [measured QA report](https://pages.paperclip.ing/evals-qa-smoke-41124961876-1/) preserves current source and measurement provenance. Daily admission follows merge and a normal TestOps job verification; the earlier failure remains recorded. ## Risks - The fix uses the existing Playwright visible-element predicate and task-link selector. A future UI selector change can still fail the journey. - A detached snapshot returns no evidence. It cannot satisfy a nonempty expected result set. The original poll deadline stays in place; there is no added journey retry. - Browser fixtures prove the readback mechanism. They do not replace the required live Cloud pass. ## Model Used OpenAI Codex, GPT-6 family. The host does not expose an exact model version or context limit. Used repository tools, TypeScript checks, and fresh local Chromium contexts. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 15:51:28 -05:00
Dotta	a4308b9791	test(e2e): add browser project and task context journey (#18353) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People group tasks into projects and use those projects to keep work in context. > - Deployment journeys already test task creation and assignment. > - They do not yet prove that a new project and its task keep the same saved relationship after reload. > - This pull request adds an explicit browser journey for that gap. > - Independent API reads and visible task properties must agree on the exact records and selected settings. > - The journey uses no model runs. Each deployment must pass before a daily plan can include it. ## Linked Issues or Issue Description **What existing behavior does this improve?** The deployment-targeted Product E2E suite can now check project creation and task context through the production dialogs. **Current behavior** The suite covers standalone task creation and assignment. It does not prove that a newly created project contains the task selected in the composer, or that the relation survives reload. **Proposed behavior** Select `project-task-context`. Create a unique project and a self-assigned task with that project selected. Reopen the project, open its task, and reload. Require exact saved company, project, title, prompt, user assignment and standard work mode. Require no agent runs before or after reload. **Reason and benefit** The same journey can qualify different deployments. It catches lost project links and incorrect saved context without a model call or a separate QA-only test implementation. ## What Changed - Add the browser flow and an independent saved-state oracle with positive and negative cases. - Scope project and task navigation to unique links in the main content so sidebar links cannot bypass list coverage. - Bind reopened routes to saved project and task references. Verify the main project heading and visible task content instead of a hidden duplicate sidebar label. Keep browser failures as closed stage codes. - Register the journey, its closed success checks, and its source files in the definition hash. Default model-matrix selections stay unchanged. - Associate qualification-report coverage with project lifecycle, project task context and task creation. Reject retained browser-only passes containing run IDs, provider/model metadata or native/legacy execution. - Add `project` to the existing durable ownership resource contract. A target provider remains responsible for cleanup of its own project fixtures. - Update discovery expectations, the deployment journey guide and the projects feature map. ## Verification - Focused runner tests: 110 passed across the new oracle, discovery, journey contracts and existing report subprocess checks. The first sandboxed attempt blocked the existing report subprocess IPC socket; the same selection passed with local IPC enabled. - Late review regression checks: both report association and unexpected browser-only model work reproduced as failing tests before the fix. The final contract, saved-state oracle and report catalog selection passed all 72 tests. Runner typecheck passed again after these changes. - Browser readback: the old project-name selector failed in Chromium when a hidden sidebar label preceded the production project heading. All six browser regression cases now pass, including missing rows, wrong project/task destinations and a wrong project property. The fixture uses the production heading and task-row overlay conventions. Credential-free Node 24 imports pass. - Current-master integration: 208 focused catalog, project/search, campaign and reporting tests passed. All nine project/search browser regressions passed. Runner typecheck, map checks and Node 24 imports for both journeys passed. The combined catalog, dispatch, source hashes, success checks, browser-only guard and feature associations retain both selections. - The first deployment qualification created both owned records but failed during later readback with a generic browser code. Authentication, setup, collection and cleanup passed. That failed grade is retained. The corrected source subsequently passed a fresh attached QA campaign; the original failure remains preserved. - New oracle alone: 7 tests passed after the final catalog mapping change. - `node scripts/check-maps.mjs` — passed; structural checks only. - Credential-free `--suite product-journeys --list` — passed and lists this browser journey. - `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json` — passed after building required plugin, runner and eval-kernel declarations. The initial fresh-checkout check failed because those declarations were not built. - All current-head CI gates passed, including repository typecheck, tests, build and browser checks. Greptile reviewed the exact head at 5/5 with no unresolved findings. Fresh attached QA passed at `f19557c9eebde14c2def37195d6b154c23eb9690`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41119722363-1/). All five independent checks and authentication, setup, journey, collection and cleanup passed; zero agent runs started. ## Risks - This adds an opt-in test and changes no production behavior or schema. UI control changes can fail the journey. - The target provider must support durable `project` ownership before it can run this case. Project cleanup must settle owned tasks, reject foreign linked tasks and archive the project while preserving history. - Source registration alone does not include the candidate in any saved daily plan. A fresh pass against that deployment is required first. ## Model Used - OpenAI Codex, GPT-6 family, with code inspection and tool execution. The host does not expose the exact underlying model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicate journey found) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-30 15:07:21 -05:00
DottaandPaperclip	a0e3ac62a7	test(evals): add reusable task search and filter journey (#18355) ... ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - The task list lets people find saved work and open its context. > - Product journeys can exercise the same task UI on local and Cloud targets. > - The shared suite lacks a search and status-filter journey with control tasks. > - This pull request adds that journey with exact task identity checks. > - The benefit is reusable evidence that search, filters, and task navigation work together. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for task search, status filters, and saved task navigation. **Current behavior** The shared journey suite covers task creation and assignment. It does not prove that a search excludes an unrelated task, that a status filter excludes a matching completed task, or that the selected task retains its saved content. **Proposed behavior** An explicit `task-search-filter` selection creates three owned tasks. It searches through the production task list, applies Backlog, and opens the exact target. It verifies reload and Back navigation, saved task fields, and zero agent runs for all three tasks. **Reason and benefit** The same journey can run against a managed local stack or an attached Cloud target. Independent assertions reject missing evidence and plausible wrong results. Deployment qualification and daily-plan admission remain separate gates. **Breaking changes** None. Existing journey selections keep their behavior. Related work: #16979 fixes status filtering before pagination. #15403 covers saved task views. This journey tests the existing search/filter interaction and does not add a new UI or saved-view feature. ## What Changed - Add a zero-run browser journey with one target and two owned controls. - Compare exact visible task identities after search, filtering, reload, and Back navigation. - Verify saved company, owner, title, description, status, and zero agent runs through the public API. - Register dispatch, source hashing, closed success checks, and browser reporting validation. - Add adversarial oracle tests and a campaign dispatch test. Document the selection and its feature-map scope. - Exercise the real browser flow with a stateful UI/API port across Coordinator, combined Tasks, and legacy shells. Inject missing ownership, wrong destinations, lost filters, changed saved context, and unexpected native work. - Use the combined Tasks shell's named Backlog view for reload persistence, as required by its existing view-preset contract. Preserve the successful polled result snapshot and link the case to feature coverage. ## Verification - First attached Cloud attempt failed during Backlog selection. Its cleanup passed and it started no model work. The original failed result is preserved; the corrected candidate passed a fresh attached campaign below. - Local Chromium reproduces the cause: the legacy StatusIcon and text label produce the accessible checkbox name `BacklogBacklog`, so the prior exact `Backlog` lookup finds no status control. - The selector now stays inside the Status group and accepts the explicit and duplicated names. Three real Chromium regressions pass, including a separate assignee called Backlog. - Latest focused journey checks: 130 passed. Runner typecheck and map checks passed. Unexpected browser/API failures retain only closed source-owned stage codes. Screenshot privacy guards and all exact identity, content, filter, and zero-run checks are unchanged. - Fresh attached Cloud qualification passed at `e3dfc0ae37e8ac74e9a07ca24421ea6c96f9bcfc`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41114233380-1/). All five independent search/filter/reload/reopen/no-agent checks and all five phases, including cleanup, passed. The browser journey started zero agent runs. - Runner typecheck passed: `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Focused oracle, contract, and campaign tests passed: 101 unique tests. The campaign suite was rerun after its new dispatch test: 30 passed. - Direct browser-flow tests passed: 23 tests. These call the real `taskSearchFilter` function through a stateful UI/API port. They do not establish live browser or Cloud qualification. - Latest focused search, oracle, catalog, and report checks passed: 127 tests across five files. - Full runner Vitest completed with 5,204 passes, two skips, and three failures outside this change. Two local embedded PostgreSQL fixtures failed at bootstrap. The Pi controls test expects profile version 35 while the unchanged base declares 37. These failures remain recorded; the PR does not change those fixtures or pins. - The remaining Node support suite passed separately: 137 tests. The standard unit launcher stops before these tests after a Vitest failure. - Credential-free discovery passed: `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/launch.ts --suite product-journeys --list`. - `node scripts/check-maps.mjs` passed. `git diff --check` passed. - All 58 current-head check runs are green, the additional security status passed, Greptile reviewed this exact head at 5/5, and no review threads remain unresolved. Typecheck, test, build, browser CI, and fresh attached Cloud acceptance passed. Daily-plan admission follows merge. ## Risks - Task-list control labels and link structure can change. Such drift fails the journey and needs an update. - The journey creates three board-assigned tasks. It records ownership before creation and records each returned ID immediately. The existing target provider owns cleanup. - Local assertion tests do not prove that a deployed Cloud target passes. The target owner must record a fresh live pass before admission to a daily plan. ## Model Used OpenAI Codex, GPT-6 family. The host does not expose an exact model version or context limit. Used repository search, TypeScript tools, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 14:26:53 -05:00
DottaandPaperclip	ac4f13d205	fix(e2e): report committed child blockers promptly (#18306) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E checks the ordinary onboarding conversation and its delivered work. > - A native run can finish successfully while its task remains blocked for a human. > - The onboarding journey kept waiting for such a child until the campaign deadline. > - This pull request reports the committed child blocker when no owned work remains pending. > - The journey still requires the same independent checks before it can pass. ## Linked Issues or Issue Description **What happened?** An approved child can stop with a committed current-track blocker owned by the board. Its native run succeeds, but the child task stays blocked. The onboarding journey reports a deadline failure instead of the observed child blocker. **Expected behavior** Report `onboarding_child_task_blocked` when the latest child run commits that blocker at the child's current status version. Keep waiting for parent dependencies and pending native or workspace recovery. **Steps to reproduce** Use the existing onboarding fixture with an approved blocked child. Give its latest native run a succeeded status, committed finalization, blocked work disposition, and the matching task status version. The old journey waits until its deadline. The new journey fails at that settled blocker without downloading or accepting any delivery. Related public context: #18298. Public issue and PR searches found no duplicate blocked-child harness fix. This change reports the failure boundary; it does not repair artifact delivery. ## What Changed - Check the owned child's board blocker after the existing run and recovery checks. - Require a unique latest producer, native committed results, and the current task status version. - Preserve parent dependency waits and all artifact, document, download, link, scope, and budget checks. - Add a behavioral regression and controls for active work, native recovery, workspace replacement, stale versions, newer producers, and agent-owned blockers. - Document the distinct failure code and the existing control-plane writers for its metadata. ## Verification - The new behavioral regression failed on the old code with `onboarding_first_task_deadline`; 147 controls passed. - The current head passed all 148 onboarding tests, including the valid agent-owner object control. - The initial implementation passed 305 tests across all 10 Product E2E files. The later update changes only writer links and the owner control; its 148 focused tests and harness typecheck also pass. - `pnpm test:e2e:runner:typecheck` passed after building the isolated checkout's workspace declarations. - `git diff --check` passed. - All required CI checks passed on `1b3bd04d193fc98d0d9f2eb3b826a4ef4bdb6dea`, including full tests and build. They were not repeated locally. Greptile reviewed that exact head at 5/5. - The full Cloud journey remains unverified by these source and CI checks. This change does not claim that the underlying artifact delivery issue is fixed. ## Risks The early failure depends on the public task and run projections. Missing, stale, or ambiguous producer metadata keeps the existing bounded wait. A pending recovery or newer producer prevents the old blocker from ending the journey. No success requirement changes. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`, reasoning effort `ultra`. The model used source inspection, code editing, and local test tools. The runtime did not expose its context window size. Root supplied the model and effort metadata for this agent session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 08:26:14 -05:00
DottaandPaperclip	8f50704135	fix(e2e): wait for pending native recovery (#18298) ... ## Thinking Path > - Paperclip manages agents and their work. > - Native runs can retain a bounded automatic recovery on the same run. > - Product E2E observes those runs before it checks the delivered work. > - The onboarding journey treated every failed run as a final failure, even when the server had saved its next recovery attempt. > - This pull request waits only for the explicit agent-owned native recovery projection. > - The journey still requires successful execution and the same independent delivery checks. ## Linked Issues or Issue Description **What happened?** The onboarding journey can stop after a native turn completes without its semantic result. The server marks that attempt failed and saves a same-run retry. The journey rejects the failed row before the saved retry is due. This also affects a native child that follows a proven workspace deferral. **Expected behavior** Observe the saved automatic recovery within the original campaign deadline and run cap. A pending recovery cannot satisfy completion. Ordinary failures and exhausted, terminal, or board-owned recovery must still fail. **Steps to reproduce** Use the existing onboarding fixture with a native failed row whose nativePhase is retryable_failure. Give it the server-owned native_session_interrupted cause, an agent-owned retry_scheduled projection, a valid retry time, and attempt 1 of 3. Transition that same row through reconnecting and running to success. The old journey rejects the first row. Related public context: #18292. A search of related native-recovery PRs found no duplicate harness fix. ## What Changed - Observe the authenticated native recovery state in the existing workspace-deferral module. Require matching nonempty cause strings and every owner, phase, attempt, schedule, deadline, scope, and successor guard. - Apply it to the proposal wait, failed native rows, workspace successor tails, and final settlement. - Keep defensive rejection of known permanent, permission-declined, and exhausted causes. Diagnostic cause strings are not a positive retry policy. - Add behavioral recovery, deadline, and rejection controls to the existing workflow tests. - Document that this wait preserves the original campaign and final delivery requirements. ## Verification - Baseline workflow regression: 4 expected premature failures with 90 controls passing. The final structural calibration rejected 20 valid pending-recovery cases before the correction, with 120 controls passing. - Final regression: 140/140 pass, including existing and new diagnostic causes on direct and workspace-successor paths. Known terminal, ambiguous, malformed, mismatched, and exhausted states still fail. - Focused Product E2E compatibility: 297/297 pass across 10 files. - pnpm test:e2e:runner:typecheck: pass after building the normal SDK and Runner TypeScript dependencies. - git diff --check: pass. - Final source passed these checks before the reviewed commit. - Full repository tests/build and live Cloud acceptance were not run locally for this harness-only change. Required CI run 41049692194 passed on 04dedd7d. Live acceptance remains separate. ## Risks The execution projection is a polling observation, not dispatch authority. A server that never resumes can hold this wait until the original deadline. The predicate requires matching native interruption state and cause, an agent owner, a bounded attempt and schedule, and no successor. Known terminal-code rejection is a defensive check, not the authority to retry. The journey must still observe actual success and pass its document, artifact, download, and parent-link checks. ## Model Used OpenAI gpt-6.1-sol, ultra reasoning, inherited from the parent runtime metadata. Context window size is unavailable. Used source inspection, code edits, shell tools, Git, and deterministic tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with Fixes/Closes/Refs OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 06:55:41 -05:00
DottaandPaperclip	8035f8b245	test: require saved onboarding approval cards (#18292) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests exercise real user workflows through the browser. > - The onboarding journey requires a human to accept one proposed child task. > - A live agent tried to create the child before it saved a proposal card. The product correctly rejected that action. > - The scripted request named the required result but did not name the proposal tool or require its saved receipt. > - This pull request makes that handoff explicit. It keeps the existing human approval and independent result checks. > - The benefit is a clearer scripted workflow without a change to product authority. ## Linked Issues or Issue Description Refs #18288 and #18266. **What happened?** The agent treated the opening task request as a step toward acceptance without first saving a proposal. It then stopped after child creation was denied. Its visible response referred to a pending proposal, but no proposal card existed. **Expected behavior** The agent saves a native confirmation card, checks its receipt, and waits for the human to accept it through the browser. It must not record its own approval. Only an accepted card authorizes the child task. **Steps to reproduce** Run the scripted Cloud onboarding journey. Give the canonical onboarding task its first task request. The observed agent skipped the saved proposal and tried to create the child. Model behavior varies, so this is a fixture guidance change rather than a deterministic product defect claim. **Paperclip version or commit** Fixture base: `7a7ade4b9360aaf8e1a2808bb2b1a752c080f564`. The serving product was Core `965184d2968f40b1183803529b88f288264c832f`. **Deployment mode** Testing Cloud workspace with native Codex. This pull request selects no product release or deployment. ## What Changed - Name the existing `request_human_input` confirmation or revision-bound checkbox contract and `wake_assignee` policy in the parent request. - Require a saved pending-card receipt. Require corrections when tool arguments are rejected. - Forbid self-approval. Require the actual accepted human card decision before child creation. - Document this scripted handoff. Keep the child brief, final download links, browser actions, and all grading assertions. ## Verification - Focused onboarding and Cloud journey tests: 79 passed. - Shared evidence, campaign, and contract tests: 66 passed. - `pnpm test:e2e:runner:typecheck`: passed after building the existing workspace type dependencies. - Credential-free preview under a network-deny guard: passed with zero external calls. Its preparation plan matches the base fixture. - `git diff --check`: passed. - No prompt-string test was added. Existing tests continue to enforce approval order, child scope, independent file checks, and real result links. - Full local repository build and database suites were not repeated for this fixture text change. Required PR CI must pass before merge. - The original live result remains unchanged. This local work does not prove that a later model run will comply. ## Risks - The model can still ignore the instructions. A new real campaign must prove the workflow. - More explicit guidance makes this a scripted product journey. It is not a model discovery benchmark. - There are no product runtime, schema, credential, model, budget, or approval-policy changes. ## Model Used OpenAI Codex, exact model `gpt-6.1-sol`, ultra reasoning effort. Parent runtime metadata confirmed this model. Context window size was not reported. Used source inspection, local code execution, and an independent agent review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 05:04:21 -05:00
DottaandPaperclip	7a7ade4b93	test: require real onboarding result links (#18288) ... ## Thinking Path > - Paperclip lets people assign work to agents and use their saved results. > - The Cloud onboarding journey tests this handoff through a real parent and child task. > - The child can publish a correct file while the parent names that file without linking it. > - The existing journey correctly rejects that reply because the user cannot download the named result from it. > - This pull request makes the scripted parent request explicit about real Markdown links and returned attachment paths. > - The independent artifact and parent-link checks remain unchanged. ## Linked Issues or Issue Description Refs #18266 and #18258. **What happened?** A Cloud onboarding attempt produced a correct child artifact and saved document. The parent then named the child, document key, and filename in inline code. It omitted the attachment URL. The attempt failed with `onboarding_parent_result_links_missing`. **Expected behavior** The parent's final reply must link the saved child results. The file link must use the real published attachment reference. **Steps to reproduce** Run the existing explicit Cloud onboarding journey. A reply that only names the file must remain a failure, even when the child artifact is correct. **Paperclip version or commit** The branch starts from `965184d2968f40b1183803529b88f288264c832f`. ## What Changed - Ask the parent for real Markdown links in its final reply. Use the published attachment's returned content or download path. Do not invent IDs or URLs. - Add a filename-only reply to the existing negative oracle cases. - Document this scripted fixture requirement. ## Verification - Focused onboarding tests: 79 passed. - Shared task, campaign, and contract tests: 66 passed. - Existing artifact and native-completion Node checks: 137 passed. - `git diff --check`: passed. - Harness typecheck, `pnpm -r typecheck`, and `pnpm build`: passed after building the existing workspace dependencies. The initial missing-declaration and local-IPC setup failures remain recorded. - The full local database test suite was not rerun. Required repository CI runs the broader suites on Linux. - Required repository CI passed on the exact PR head. No paid model calls or Cloud operations were run for this patch. - [Initial CI](https://github.com/paperclipai/paperclip/actions/runs/41027635545/attempts/1) failed one unchanged retention timing test (3310 ms versus the strict 3000 ms bound). The unchanged case passed once on an isolated local Postgres17 fixture. One failed-jobs-only rerun passed in [attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41027635545/attempts/2). The original failure remains visible. No timing threshold or retention source changed. - The previous live attempt stays failed. Local tests do not prove that a model follows the revised request. A new bounded live attempt must establish that result. ## Risks - The extra instruction can still be ignored by a model. The unchanged oracle will reject missing links. - This changes the fixture definition hash. Historical attempts retain their original source and result. - No product runtime, API, approval policy, model profile, budget, or cleanup behavior changes. ## Model Used OpenAI `gpt-6.1-sol`, ultra reasoning, with code execution and repository tools. Model metadata was confirmed in the parent task. The context window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 04:01:33 -05:00
DottaandPaperclip	81ebe479e8	fix(evals): bind rendered Markdown replies to saved run evidence (#18264) ... ## Thinking Path > - Paperclip lets people assign work to AI agents and read saved results. > - Product journeys must prove that a result is visible and belongs to the completed run. > - The shared observer compares stored Markdown with browser text. > - A document link can render as an issue chip, so those strings differ. > - This PR keeps run and comment attribution, then compares full text and exact link destinations. > - The benefit is a valid browser result without a false timeout or a weaker provenance check. ## Linked Issues or Issue Description Refs: #18248, #18255. **What happened?** The protected-action story completed both decisions and all four native turns. Its final explanation contained a document link. The browser rendered that link as an issue chip. The observer waited for raw Markdown to equal rendered text, so it could not accept the valid result. **Expected behavior** Accept complete matching content from the final run after reload. Keep the exact link destination, task anchor, author, run and durable final-message checks. Reject missing or changed content. **Steps to reproduce** 1. Save an attributed reply with bold text and a Markdown document link. 2. Render the link as an issue chip in the task bubble. 3. Run `readBoundTaskFinalReply`. The original observer times out. The new browser regression preserves that failure. ## What Changed - Add a UI-owned content projection using its existing Markdown renderer and GFM/issue-reference plugins. Keep DOM observation, attribution and destination checks in the task-reply observer. Keep source Markdown and observed text separately. - Read visible DOM text, link labels and destinations. Require the structured projection to match the actual visible text. Allow only recognized product chips to replace a link label. Ordinary links keep their source label. Reuse the production issue-reference and company-route helpers, plus the current task URL. Ordinary root-relative anchors keep their original path. Preserve exact marker checks and all run/comment/native-event attribution checks. - Check visible decline explanations and unread data in preserved native source text. - Add browser regressions and source/content tests. Reuse the UI’s existing `react-markdown` and `remark-gfm` dependencies. The final diff changes no package manifest or lockfile. - Match generated GFM checkbox states and reject changed states. Document supported content and fail-closed behavior. ## Verification - Original observer: the decorated-link browser regression fails at the final-reply predicate. - Fixed observer: all 21 browser regressions pass, without credentials or model calls. - Focused observer, decline, content and journey contract suites: 64 tests pass. - E2E and UI TypeScript checks pass. All UI token gates pass. The unchanged frozen install passes. - Full CI passed on the preceding 6eba7ff head (41003112166). Full CI passed on the preceding b9a2571 head (41006205475), with both reviewers at 5/5. Current-head full CI passed at b3d2d08305 (41009669966), including typecheck, build and all eight E2E shards. Both reviewers scored this commit 5/5, with no unresolved threads. The initial lockfile policy failure was corrected. The final diff has no manifest or lock changes, and its unchanged frozen install passed. - The optional positive GFM strikethrough assertion also passes. - The Node 24/tsx end-to-end observer probe passes. The callback has no external compiler helper; a browser regression covers actual tsx serialization. - The Paperclip review finding is also fixed: bare URLs and issue keys use the production plugins and the UI’s `scope=accessible` company-prefix context, with RED/GREEN browser coverage. Source Markdown is required in the receipt. - All Greptile findings are fixed, including matching the UI’s accessible-company scope: ordinary link labels, issue-prefix/fragment resolution and ordinary root-relative paths have positive and negative browser coverage. - The deployed attempt remains a failed original measurement. Fresh qualification is required after this repair. ## Risks The comparison normalizes whitespace and inline formatting. It verifies full text and exact link targets, while link labels may become product chips. Images, raw HTML, tables and fenced code fail closed. Those constructs may require a future explicit extension if a journey uses them. No application behavior, model, budget, approval or connection permission changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution and browser tools. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 00:57:59 -05:00
DottaandPaperclip	145214201d	test: preserve the onboarding child brief at delegation (#18266) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys check onboarding through the real app. > - The onboarding story asks a parent agent to delegate a file and document task. > - A parent can preserve the goal but omit the exact tool instructions from its child description. > - The child can then publish a note instead of the required JSON file and task document. > - This pull request asks the parent to copy the full child brief without changing it. > - The existing independent checks still decide whether the result passes. ## Linked Issues or Issue Description Refs #18258. Includes merged prerequisite #18270 for the existing retention scheduling failure found by required CI. **What happened?** The scripted onboarding request gave the parent an ordered tool recipe. It did not require the parent to keep that recipe in the child description. A delegated child could compute the result but publish a Markdown note and omit the Paperclip task document. **Expected behavior** The child receives the full approved brief. It must publish `connection-proof.json`, save its Paperclip task document with the returned file link, and report completion through the current contract. **Steps to reproduce** Run the explicit Cloud onboarding journey. Inspect the approved child description and its published result when the parent summarizes the request. **Paperclip version or commit** This change is based on public master `b3c2172d417ea6b783e0c796aaed4a91e1f9d381`. **Deployment mode** Scripted Product E2E against a live Cloud testing workspace. ## What Changed - Put the existing ordered instructions in one marked child brief. - Require the parent to copy that brief verbatim into `create_task.description`. - State the exact JSON filename and workspace-relative content reference for `register_deliverable`. - State that a Markdown workspace note does not replace the `write_document` step. - Document this scripted scope in the Product journey guide. ## Verification - Existing focused Product journey tests: 227 passed in 10 files. - Credential-free Product journey discovery passed through the direct tsx loader. - Source comparison confirms that all code after the opening reply is unchanged. The download, calculation, hash, provenance, document, parent-link, approval and run-limit checks retain their original behavior. - No prompt word-matching tests were added. This is an instruction-only change. - `pnpm build`, `pnpm -r typecheck`, and `pnpm test:e2e:runner:typecheck` passed. - Full support Vitest check: 5,042 passed, 2 skipped, 2 failed. Both failures occur during embedded PostgreSQL startup in the controller database lifecycle fixtures. The log includes a missing bundled `libzstd.1.dylib`. The separate Node support checks passed all 137 tests. - `pnpm test:run` was attempted. It reached the general server suite and repeated embedded PostgreSQL startup failures. I stopped only this owned local test process group. The full local test command is incomplete. Required Linux CI remains the merge gate. - The first harness typecheck ran before workspace build outputs existed and reported missing workspace modules. It passed after the full build. The earlier failure remains recorded. - The initial CLI discovery wrapper hit a local tsx IPC permission error. The direct loader passed with the same source and arguments. - The original required CI run found an existing skill-retention page delay. Its unchanged test passed once against a dedicated local PostgreSQL fixture. Separate PR #18270 added a deterministic regression and fixed the scheduler. After that PR merged, this branch received one normal merge from master. Its diff remains two fixture files. The original failed CI run remains preserved. - After the branch refresh, all 70 onboarding first-task tests and 7 retention/deadline controls passed. Harness typecheck passed. Fresh required CI and review passed on head `608e266c6567c19981fd3f8a07305c785346753b`. - The refreshed CI run passed the retention shard but failed one unchanged Antigravity authentication fixture. The same Runner lane passed on the prior fixture head and the merged prerequisite. The exact test passed once locally. Source review identified a possible stdout/stderr ordering race, but the failed run did not retain its exact error subtype. The one approved retry of the failed CI job and its verification aggregate passed on unchanged source. The original failure remains preserved. This proves recovery, not its cause. - The live Cloud journey has not been rerun with this source. Historical failed attempts retain their original results. ## Risks - An agent can still fail to follow the brief. Existing independent assertions will reject the wrong file or missing task document. - The fixture definition changes. A later live attempt must record this new harness revision. - No application, model, effort, budget, cleanup or acceptance rule changes. ## Model Used - OpenAI `gpt-6.1-sol`, ultra reasoning effort, with repository tools and code execution. The parent verified this model ID from current runtime metadata. The context window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 00:13:37 -05:00
DottaandPaperclip	b3c2172d41	fix(evals): verify native protected-action outcomes (#18255) ... ## Thinking Path > - Paperclip manages work assigned to agents. > - Product E2E stories ask for useful user-facing results. > - Classic runner and deployed QA reuse the same service approve/decline task prompts. > - The service returns a receipt marker that belongs in a saved briefing document. > - State that destination explicitly and ask for the document link when complete. > - The deployed decline path also needs the same durable final-reply proof as Plan and Stop. ## Linked Issues or Issue Description Refs #18222. Builds on the merged reply observer in #18248. **What happened?** The shared service tasks ask for titles and a “verification code” without specifying that the code belongs in the document or asking for its final link. A native QA attempt created its briefing but then repeatedly failed the completion tool’s array validation. Those invalid arguments remain a separate observed failure; the wording change does not establish their cause or relax validation. Source inspection also found that the declined-action oracle requires a projected comment even when the settled native thread renders a durable final message. **Expected behavior** Ask for page titles and the service receipt code in the saved briefing, then a document link. After a declined action, require the existing explanation-only fallback without another service call. **Steps to reproduce** Run the shared service-approve/service-decline stories through the classic runner or QA Product E2E adapter. ## What Changed - Clarify the natural-language task for both shared service stories: put page titles and service receipt code in the document and provide its link. - Use the reloaded run-bound final reply for QA service declines. Pass an explicit native receipt to the existing decline oracle; do not synthesize issue comments. Require the final native run to start after the saved decline and keep exact decision, truthful explanation, no unread marker, zero calls and unchanged connection checks. - Keep classic callers on their existing attributed-comment path. No internal tool instructions are added to the user prompt; ownership, service data and model/runtime limits stay strict. ## Verification - Decline regression observed RED: two cases reject a valid saved native explanation without projected comments. After the repair, 79 focused tests passed across native reply, decline guidance, agent evidence and journey contracts. A two-successful-run control rejects the earlier receipt and accepts the later one. Negatives cover stale/wrong run, wrong agent/task, missing final proof, summary/unsettled/hidden output, false explanations and unread service data. - Product E2E TypeScript and `git diff --check` passed. - Live protected-action qualification is required after merging this and the companion private service/cleanup fixes. The previous failed result remains failed; this is a bounded task-clarity experiment, not a claimed model or completion-schema repair. - Current-head Greptile and Paperclip reviews are 5/5 with zero open threads. CI initially failed the unrelated agent-chat-projects cleanup POST with a PostgreSQL lock timeout after its assertions completed. One bounded retry passed on unchanged source; all current-head gates passed at `64bc169e147290ed4dbaa18221e0268362ce81ee`. The original failing attempt and its logs remain available. The task-clarity change is still a bounded experiment; the observer repair addresses a demonstrated storage-assumption defect. ## Risks A clearer task can still fail with the selected model. All acceptance checks and completion input validation remain strict; a saved briefing alone cannot qualify the complete journey. Only test prompts and test evidence code change. Missing or contradictory native evidence fails closed. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed by this session. Used reasoning, source editing, shell execution and test inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 22:14:37 -05:00
DottaandPaperclip	59b76777db	test: clarify onboarding artifact completion instructions (#18258) ... ## Thinking Path > - Paperclip helps people manage AI agents and inspect their work. > - Product E2E journeys test that work through normal browser and API controls. > - The Cloud onboarding journey asks one native child to publish a file and save a document. > - Its task request did not explain the completion report format or make each output step distinct. > - A live attempt published the file, then submitted an invalid finish report and declared a blocker. > - This change orders the scripted request and explains the existing completion contract. > - The agent must still produce the result and pass the same independent checks. ## Linked Issues or Issue Description Refs #17940. Refs #18228. **What happened?** The scripted onboarding request left completion details implicit. A live child published its proof file, then submitted a finish payload without verification and with an invalid schema value. Its later blocked report also listed local paths as artifacts. This does not establish a Core validator defect. **Expected behavior** The request should explain the existing tool contract. The child must publish the file, save the document with its returned link, then submit a valid completion report. Argument correction must not replace artifact verification or bypass approval. **Steps to reproduce** Run the explicit Cloud onboarding Product E2E journey through its private target provider. Approve the proposed child through the UI. Inspect the native finish report and the independently checked attachment, document, and parent links. **Paperclip version or commit** The observed deployment used public Core `6a674716447e8fa268c5cdeb1db1020319368a77`. This fixture change starts from `a8e07500db68f187b19de83823eb7fabf1196021`. **Deployment mode** Cloud testing with the scripted Product E2E harness. ## What Changed - Order the child request: create files, publish the proof, save its document, then finish. - Require verification for actual checks and omit the optional schema field. - Explain receipt-derived references in evidence and file artifacts. Keep local input paths in verification details. - Document same-attempt argument correction and preserve historical failed results. ## Verification - Existing onboarding-first-task and artifact-evidence suites: 83 checks passed. - The maintained harness unit wrapper also passed 137 Node contract checks. - Independent source review found no contract, privacy, or oracle change. - Full `pnpm build` and `pnpm -r typecheck` passed at the preceding commit `c3900cf2`. The only later change names the filename required by the existing document assertion. - Current `d75180b4`: the 83 fixture/artifact checks, 137 contract checks, and `pnpm test:e2e:runner:typecheck` passed again. - Credential-free `--list --suite product-journeys` passed. - The local full repository Vitest run at `c3900cf2` was stopped before the filename correction. It is incomplete and is not reported as a pass. Current-head CI passed: 54 successful checks and two optional Storybook skips. The separate advisory review is still running and is not reported as passed. Initial fresh-worktree dependency and sandbox IPC failures remain recorded. - No paid model, Cloud workspace, release, or deployment was run for this change. These checks establish harness compatibility, not a live model pass. The prior failed campaign remains failed. ## Risks The model may still submit an invalid report or fail another step. A fresh live journey is required to measure the instruction change. This PR changes fixture text and documentation only. It supplies no result, checksum, real attachment ID, or successful status. Models, limits, validators, and independent assertions stay the same. ## Model Used OpenAI Codex, `gpt-6.1-sol`, ultra reasoning. The model identity comes from confirmed parent runtime metadata. Context window size is unavailable. Tools and code execution assisted the change and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 22:02:51 -05:00
DottaandPaperclip	2badf8a4a5	fix(evals): verify native QA replies on the reloaded task thread (#18248) ... ## Thinking Path > - Paperclip manages AI agents that do work for users. > - Product E2E journeys check user outcomes on real deployments. > - Native task threads can render saved final replies from a settled run. > - The new QA Plan and Stop checks looked only for agent comments. > - Both flows reached the requested terminal state and showed the correct reply, but that storage assumption rejected them. > - This PR checks the reloaded visible reply and binds it to the final successful run. Explicit presentation decisions remain authoritative. ## Linked Issues or Issue Description Refs #18222 and #18244. **What happened?** The deployed Plan journey completed all three native runs, including revision and approval. Stop completed native cancellation, resource release and a new direction. Their final-answer checks failed because the comments endpoint did not contain the rendered native reply. The original failed grades remain unchanged. **Expected behavior** Require exactly one correct visible final answer after reload, attributed to the final successful owned run. Reject an earlier run, progress text, hidden content, duplicate answers or a cancelled run. If its selected completion comment is projected, require that comment and matching author/run/text. Otherwise require a single durable final assistant message and its settled native final-response anchor. **Steps to reproduce** Run the Plan or Stop journey on a native deployment whose task thread renders the settled run before a completion comment is projected. ## What Changed - Add a narrow reply observer over the existing task-thread hooks. It reloads the page and refreshes saved run/comment metadata while waiting. - Bind native replies to a validated durable final assistant event and the successful run's settled final-response anchor. Require the selected comment when projected; support pending or suppressed projection without accepting a summary fallback. Accept payload and item-normalized finals, rejecting contradictory fields. - Use the observer for Plan completion and Stop's new direction. Keep existing run counts, deployment binding, cancellation and artifact checks. - Add negative grading tests, include the helper in the definition hash, and document its evidence contract. Classic protocol comment matchers are unchanged. ## Verification - 63 focused tests passed across native reply evidence, agent evidence and journey contracts. - Product E2E TypeScript passed. The reply observer uses the remaining journey deadline and backs off polling. - Live Plan and Stop attempts exposed this grading failure. Their original outcomes remain failed; fresh qualification is required after merge. - Full current-head CI passed at `40840ed742977853630d733292461ad7003fc7fb`; Greptile and Paperclip Review both scored 5/5, with zero unresolved review threads. ## Risks The observer depends on existing task-thread test hooks and run anchors. Missing or changed hooks fail closed. It reads rendered user-facing text; a finish summary alone cannot satisfy it. Native event provenance, run ownership and exact reply equality are required. No backend, model, provider, budget or cancellation policy changes are included. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed by this session. Used reasoning, source editing, shell execution and test inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 21:35:15 -05:00
DottaandPaperclip	8941db9ce6	fix(evals): accept editor escapes and cancel sensitive QA requests (#18244) ... ## Thinking Path > - Paperclip manages AI agents that do work for users. > - Product E2E journeys check real browser actions and saved results. > - A rich editor can store plain underscores as escaped Markdown. > - The QA reply check treated that valid saved message as a failure. > - A private target adapter also needs to cancel credential-safe HTTP requests before cleanup. > - This PR fixes reply comparison and adds abort and timeout options to the existing sensitive transport. ## Linked Issues or Issue Description Refs #18222. **What happened?** The artifact revision and Stop journeys rejected the correct single user reply because its underscore was stored as `\_`. Sensitive connection setup also lacked a way to pass its lifecycle signal to the actual HTTP request. **Expected behavior** The check accepts the editor's underscore escape and still rejects changed, duplicate, missing, or foreign replies. A cancelled sensitive request closes before setup cleanup proceeds. **Steps to reproduce** Run an artifact revision with an underscore in the reply. For the transport case, abort a pending sensitive request or let its deadline expire. ## What Changed - Compare the fixture reply after undoing only underscore escaping. Keep exact text, count, and author checks. - Let `ConnectionApi.postSensitive` accept a bounded timeout and abort signal. Check cancellation again after cookie loading and before dispatch. - Test the retained reply failure and both cancellation paths with an actual loopback HTTP server. Document the adapter contract. ## Verification - Focused Product E2E tests: 30 passed across two files. - Product E2E TypeScript check passed. After rebase, a final check first found four diagnostics from stale runner/plugin SDK declarations; rebuilding those two workspace packages cleared all four, and the unchanged E2E check passed. - The first sandbox run failed because loopback `listen` returned `EPERM`. The unchanged HTTP tests passed with loopback permission. The test now reports listener errors directly. - Both independent reviewers found a possible pre-arrival deadline-test hang. The follow-up races arrival against request settlement and adds one second of deadline headroom. All 30 affected tests passed after that fix. - Full current-head CI passed in run 40982914525. Greptile and Paperclip Review both scored 5/5 at `f0526f5fbbf13de5ea43e868eb012069ca3d8907`; all review threads are resolved. Live QA requalification follows the merged public and private fixes; no live pass is claimed here. ## Risks The text normalization applies only to plain fixture replies and only removes underscore escapes. It does not remove arbitrary Markdown. Sensitive calls retain their existing 30-second default and closed error messages. API wrappers must forward the optional signal and await the actual request. ## Model Used OpenAI Codex, GPT-6 family. The serving model ID and context window are not exposed by this session. Used reasoning, code editing, shell execution, and test inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 20:14:56 -05:00
DottaandPaperclip	d3f2a60e79	test(e2e): reuse core journeys across deployment targets (#18222) ... ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests verify work through the browser and public APIs. > - The classic matrix already tests several common task stories. > - Its older stories depend on local fixture setup and workspace access. > - Deployment tests need the same story checks with a separate target adapter. > - This PR shares those checks and adds five explicit journey selections. > - A deployment pass still requires evidence from that deployment. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E journey reuse across managed instances and deployed Cloud instances. **Current behavior** The deployed-target catalog covers task creation, assignment, Ask, and fresh onboarding. Plan revision and Everyday stories run through the classic fixture driver. Several story steps assume a local workspace. **Proposed behavior** Add explicit Plan revision, artifact revision, protected-action approval and decline, Skill Studio, and Stop/redirect journeys. Share the main browser sequences and content checks with the classic driver. Let target adapters provide authentication, native agents, a reachable service fixture, and owned cleanup. **Reason and benefit** A local pass cannot verify the deployed application. Shared story code lets a deployed target exercise the same behavior without copying an entire test or starting a replacement server. **Breaking changes** No product API or default matrix selection changes. Passed deployment results must satisfy the declared story checks and work-run count. Unsupported target capabilities fail before a story runs. Related public work: #17262 and #18098. Upstream prerequisite: #18230. ## What Changed - Extract shared Plan, artifact download/oracle, protected service, Skill Studio, and Stop/redirect helpers. Keep classic callers on those helpers. - Add five explicit paid journey selections with owned-task observations and native company, agent, model, and environment checks. - Include classic prompts, delivery/decision code, and the decline grader in the source-definition hash. Keep classic transient-read retries and reject deployment binding failures immediately. Match cookie directory boundaries in sensitive fixture requests. - Require complete story checks and expected run receipts before retaining a passing result. Keep failed evidence and private downloads. - Add a secret-safe request method for deployment fixture credentials. Add the missing declaration for the existing manual evidence policy module so the E2E project can typecheck. - Document target responsibilities and promotion after a real deployment pass. This source catalog does not enroll candidates in a daily plan. ## Verification - E2E TypeScript check: passed, including the review follow-up. - Review follow-up: 30 tests passed across four files, covering transient Plan reads, immediate binding rejection, cookie-path scope, agent receipts, and artifact evidence. Stop now waits for a saved, still-active native turn before the browser click and retains its startup events. Counterexamples reject claimed-but-unstarted, already-completed, interrupted, and non-native work. Shared observation and API helpers also enter the definition fingerprint. Cancellation snapshots are captured before lease cleanup. A direct terminal-boundary assertion rejects native work after cancellation even if it is already in the first snapshot; five cleanup-window counterexamples are covered. The Stop-output regression first failed for both native `item.delta` and `item.completed`; the fixed check detects both while allowing accounting-only updates. A report-catalog test first failed because the sandbox denied the tsx child IPC pipe (`listen EPERM`); the unchanged test passed outside the sandbox. - Affected journey, catalog, API, service fixture, and delivery checks: 313 tests passed across 19 files. - Workspace build: passed after rebase. - Workspace typecheck: passed. The default repository run was attempted. Repeated embedded PostgreSQL startup failures prevented local database coverage; it was stopped without claiming a full pass. Linux CI must supply that coverage. - Broad E2E support run: 4,793 passed, 23 failed, 2 skipped. Failures include existing stale image/Pi assertions and process/startup timeouts under host load. The affected tests pass separately. This is not a claim that the broad suite is green. - Upstream CI prerequisites: this branch is rebased on the merged #18230 repair. The earlier conflicting Turbo flags no longer apply. The shell-hook fix is also supplied by that upstream change. Linux shell-hook investigation: 19 focused tests and adapter TypeScript passed. The exact staged-config sequence reproduced CI's 500 in a pinned Node 24 Linux container. Node's default stdin socket at shell level zero selects Bash remote-shell startup and reads `.bashrc` instead of `BASH_ENV`. Ignored stdin and a nested shell each produce 1000. The fixture now ignores stdin; the production hook and 1000 assertion are unchanged. See [Bash startup rules](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files). - GitHub CI on the earlier head passed most suites but failed two browser tests (a fixture cleanup lock timeout and a fixture run staying queued) plus the shell-hook assertion. The lock-timeout case received one retry, which was cancelled by a later source push and does not establish recovery. These failures remain part of the verification record. - Real Cloud qualification: the private target adapter is reviewed and merged. These candidates must pass there before daily-plan promotion. Current source review: Greptile 5/5 and Paperclip Review 5/5 on `a9df1d6e8d880710fce39b87e689e07e1c8e6edc`, with zero unresolved review threads. The full [GitHub CI run](https://github.com/paperclipai/paperclip/actions/runs/40974946081) passed on that exact head, including both required aggregate checks. ## Risks Shared helper changes affect classic callers. The independent ZIP oracle and visible delivery requirements remain in place. Stop requires native acknowledgement, terminal event evidence, resource cleanup, and no later original-turn activity. The managed local target still lacks paid-agent fixtures and reports these journeys as unsupported. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, terminal use, and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 18:58:26 -05:00
DottaandPaperclip	b81b870732	Add QA smoke journeys and OpenCode run traces (#18098) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify saved application behavior across deployments. > - Task creation and questions need reusable smoke selections and exact run attribution. > - Operators also need to inspect the OpenCode execution behind a measured question. > - Company observability does not currently support native OpenCode. > - This change adds the journeys and a bounded projection of normalized OpenCode run events. > - The projection follows the configured capture policy and retains the current run identity across warm reuse. ## Linked Issues or Issue Description Related foundation: #17262. Related alternative: #17989 exports metadata from OpenCode's native OTLP stream. This change uses normalized run events to support the existing content switches and per-run linkage. It does not forward the native stream. The two PRs overlap in launch registration and must be reconciled before both land. **What existing behavior does this improve?** Reusable Product E2E smoke selections and company observability for native OpenCode. **Current behavior** The journey catalog lacks reusable task creation and attached-agent question checks. Native OpenCode is marked unsupported for company observability, so a successful destination connection test does not prove that agent traces will appear. **Proposed behavior** Add explicit task-create and ask-question selections. Verify saved state and exact company, issue, agent, model, environment and successful run attribution. Export normalized OpenCode turns, tools and usage to the configured OTLP destination. Honor each capture switch and redact known secrets. **Reason and benefit** CI can run a small measured smoke selection, retain its screenshots and checks, and link results to traces belonging to the same run. **Breaking changes** Existing selectors remain unchanged. OpenCode export is opt-in through existing company settings. Legacy OpenCode stays unsupported. ## What Changed - Add reusable task creation and question journeys, configured-agent capability, owned-fixture checkpoints and strict evidence validation. - Add an OpenCode-owned projection of normalized run events. No provider SDK or raw payload schema enters heartbeat coordination. - Extend the private launch union and reserved environment allowlist for native OpenCode. With export disabled, no receiver request runs. - Keep destination headers in the parent wrapper and out of the actual provider child environment and configuration. - Bound each turn to 64 tool and 64 usage entries, 8 KiB per text field and a 1 MiB request. Use a three-second request deadline, four concurrent exports, and no retries or idle worker. - Pass the rotated runner authority to each proxy turn. A retained process no longer attributes later turns to its original launch run. - Update settings copy, observability documentation and capability maps. ## Verification - `pnpm -r typecheck` and `pnpm build` passed after the implementation changes. - Adapter launch/redaction: 9 tests passed. Projection and actual proxy: 12 tests passed before the review fix; the expanded projection suite passed all 12 cases after the fix. The proxy test exercises two runs in one process against a real local HTTP receiver and a fake provider. - Full OpenCode driver/projection suite: 72 passed, one existing response_wake test timed out at five seconds. A focused check then passed both existing omitted-continuation cases and the new trace export case. The timeout cause remains unconfirmed. - Runner package TypeScript and Rust `cargo check` passed. Map and UI token checks passed. - Prior journey validation: 273 tests across 12 files and the journey TypeScript check passed. - Broad local `pnpm test:run` encountered native-runtime integration failures with repeated embedded PostgreSQL initialization failures and was stopped. All 16 failing cases passed in an isolated run outside the filesystem sandbox (961 unrelated cases unselected). The broad local run is not a clean acceptance result. A later focused database-backed chat/heartbeat run passed all six selected observability cases, including native OpenCode launch and successful allocation (1,809 unrelated cases were unselected). - All CI gates passed for `21085f880d8babc9efeb22d0623db99613d33093`, including packaged-runner, clean-install, browser and build checks: https://github.com/paperclipai/paperclip/actions/runs/40884658995 . Greptile reviewed the same commit at 5/5 with no unresolved findings. - Verified OTLP JSON acceptance and indexing with an explicitly synthetic transport marker. No live OpenCode model trace or deployed smoke acceptance is claimed yet. ## Risks Export is best effort. Abrupt termination, recovered in-flight turns without a new submitted event, limits or receiver failure can lose traces. Export failure does not change task success. The projection represents observed normalized events, not raw native model spans or hidden reasoning. Known-secret redaction is not an OS isolation boundary. UI changes can break journey locators. Deployment adapters retain credential, ownership and cleanup responsibility. Both the controller and native runner bundle must be updated before live qualification. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution and browser tools. The exact serving model ID and context size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 08:57:12 -05:00
DottaandPaperclip	3bd0d81c7b	fix(e2e): follow proven workspace deferrals during onboarding (#18008) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify the user outcome through the browser and scoped APIs. > - An onboarding child can wait while its parent owns their shared workspace. > - The server records that no provider work started and schedules a durable replacement. > - The journey rejected that expected wait, and feedback counted it as unknown legacy execution. > - This pull request follows the exact replacement and records the distinct no-provider evidence. > - The child must still finish through native execution and produce the independently checked download. ## Linked Issues or Issue Description Refs #17940. **What happened?** The fresh-workspace journey failed with `onboarding_first_task_failed` when a child encountered shared-workspace contention. The cancelled admission had an explicit `workspace_wait` receipt with `providerWorkStarted: false`. Its linked replacement was still scheduled. The runtime had not been selected. The default `legacy` value did not represent a provider execution. **Expected behavior** Keep both rows and their run-limit cost. Follow only the unique replacement for the same company, agent, and issue. A pending replacement must not pass. Real failures and contradictory execution evidence must still fail. **Steps to reproduce** 1. Start a native onboarding parent and accept its child-task proposal. 2. Observe the child admission while the parent still holds their workspace. 3. Present the server's no-provider cancellation and its durable scheduled replacement to the journey and feedback collectors. ## What Changed - Add a shared predicate for the explicit no-provider workspace-deferral receipt. Reject runtime, process, output, usage, and cost evidence that contradicts it. - Follow exact, unique retry lineage during onboarding. Wait for active or scheduled replacements without repeating browser actions. Preserve the campaign deadline, run cap, native artifact attribution, and download oracle. - Retain per-run feedback inspection classifications: `native_events`, `provider_not_started`, or `incomplete`. Require exhaustive event reads. Keep pending work and uncertain execution partial. - Add causal regressions and document the evidence contract. Historical failed attempts keep their original grades. ## Verification - Red: the new journey regression failed with the observed `onboarding_first_task_failed` before the repair. The new feedback cases also failed before the repair. - Green: 70 journey tests and 74 feedback-family tests passed. The final affected journey/feedback gate passed 275 tests plus 137 Node checks. Five retry-lane reset cases failed before the review correction and passed after it. - `pnpm test:e2e:runner:typecheck` passed after the retry-counter correction. - `pnpm -r typecheck` and `pnpm build` passed before the final rebase. - The full local `pnpm test:run` and harness unit gate encountered embedded PostgreSQL startup failures. The harness gate passed 4,574 tests and failed its database-startup test. The final focused journey/feedback gate and all 137 Node checks passed. CI will provide independent full-suite results before merge. - Current-head CI passed 55 checks; two checks skipped and one was neutral. Greptile is 5/5, with zero unresolved review threads. - Live Cloud verification will use the reviewed harness and Operations revisions. This PR does not claim that a fresh Cloud campaign has passed. ## Risks A broad cancellation exemption could hide real failures. The exemption requires the server's explicit receipt, unresolved runtime selection, no contradictory execution evidence, and exact durable lineage. Missing successors remain pending until the existing deadline. A successful native child and valid downloaded bytes remain required. No server behavior, approval policy, or budget policy changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 07:58:41 -05:00
DottaandPaperclip	d40f9ac377	fix: enforce first-task completion decisions and retain E2E evidence (#17940) ... ## Thinking Path > - Paperclip lets people delegate work to agents and approve proposed work. > - The onboarding first task asks for scope, proposes a task, and waits for a human decision. > - An answer to its opening question supplies scope. It does not approve a proposal. > - Task creation already checks the saved proposal decision. > - Native completion used the general completion policy and could mark the onboarding parent Done without that decision. > - This PR checks the trusted decision before completion feedback and again when the native status is committed. > - The first task stays in review until a saved human decision permits closure. Ordinary completion and acknowledged refusal keep their current behavior. > - The Product E2E guard also retains closed scalar evidence so future failures show which boundary condition was observed. ## Linked Issues or Issue Description Refs: #17914. Profile compatibility follow-up to #17922. **What happened?** A native onboarding run can submit a valid low-risk Done result after only the opening answer. The native finalizer then marks the first task Done even though no human has accepted or rejected a proposal. **Expected behavior** The opening answer must not permit completion. A trusted saved proposal decision must permit it. A saved rejection permits acknowledgment and closure, but does not authorize new work. A later pending proposal must keep the task in review. **Steps to reproduce** Create a server-owned onboarding first task with an answered opening question, no proposal decision, and a valid low-risk completion contract. Pass a Done result through native completion feedback, the control-plane port, and the finalizer. Before this fix, the persisted status is Done. The regression expects In review. Ordinary completion and acknowledged rejection pass as controls. ## What Changed - Share the trusted latest-proposal and human-response checks with completion admission. Keep task creation acceptance-only. - Refuse premature native completion feedback. Keep an already saved premature result in review during finalization. - Check the decision again under the existing task and run locks before committing Done. - Cover accepted decisions, saved refusals, forged context, pending or changed proposals, and a real database race. - Retain a closed scalar receipt when the existing E2E pre-acceptance guard fails. Validate and display it without task content, IDs, filenames, or raw rows. Preserve the original failure, blocked dependents, and historical reports. - Include the evidence module in the journey definition hash. Bind the receipt to its exact failure code and check its closed keys against the full type. - Repair base-branch profile drift found by the full harness check: include the active Copilot v66 fixture in image hashing and update stale Pi version/hash expectations from #17922. Keep all older image inputs and strict hash checks. - Update the onboarding feature and agent access maps and E2E evidence documentation. ## Verification - Semantic regression before implementation: one failure and two passing controls on PostgreSQL 18. The failed assertion observed Done instead of In review. - The local database uses an owned disposable container with an immutable image digest. Actual migrations, feedback, control-plane port, finalizer, and status commit run against PostgreSQL. A temporary test adapter replaces only the embedded database transport because this host has exhausted its PostgreSQL semaphores. - Native control-plane suite: 55 tests passed, including 17 new controls. Existing task-creation admission: 25 passed. Existing completion feedback: 19 passed. - Credential-free E2E support: 206 Vitest suites passed, 4,407 tests passed, and two skipped. All 137 Node support checks passed. Harness typecheck passed. - The failure-receipt regression had 11 failed assertions before capture was implemented. Two failure-code binding assertions fail without their gate. Eight placement/binding cases first validate the receipt-free control, then reject the misplaced receipt. - The untouched base reproduced the three profile compatibility failures. The narrow repair passed all 83 affected tests before the full harness run. - Private workflow preparation and credential-free onboarding preview: nine compatibility checks passed. Ownership, recovery, evidence, shared-contract and workflow compatibility: 77 passed. The actual provider preview also confirms immediate success deletion, earlier declared failure eligibility, limits, and zero retries without credential access. Map checks and git diff checks passed. - Required repository typecheck passed for the product fix. The combined branch build passed. The local full repository suite was stopped after repeated PostgreSQL startup failures; a separate initdb probe confirmed host SysV semaphore exhaustion. No shared IPC resources were changed. [Hosted CI](https://github.com/paperclipai/paperclip/actions/runs/40591020873) passed the full database suites, typecheck, build, runner checks, serialized suites, and browser shards on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b` (55 success, two expected skips, one advisory neutral). - Independent source review approved the product guard, atomic race regression, evidence contract, and maps with no actionable findings. Both automated reviews are 5/5 on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b`; prior findings are fixed and resolved. Final CI is green and the PR is conflict-free. - No Cloud workspace, provider call, release publication, or deployment is part of this verification. These tests do not identify a historical Cloud failure or qualify live onboarding. - Local dependency installation uses the existing CI-generated lockfile from #17811. No lockfile change is included. ## Risks - Existing onboarding runs without a trusted saved proposal decision remain in review even when the agent reports success. This is the intended approval boundary. - Decision changes can race completion. The status commit checks the decision under its existing locks and uses the existing status-race recovery path. - A saved refusal permits closure only. Creation admission still rejects it. - Evidence readers reject malformed receipts and unrelated failure codes. The version-1 key registry is checked against every receipt field at compile time. Existing attempts do not gain evidence after the fact. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), ultra reasoning, through Codex. Used repository inspection, code execution, and collaborating coding agents. The runtime did not report a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 17:07:50 -05:00
DottaandPaperclip	970127a022	test: retain onboarding Connect failure evidence (#17529) ... ## Thinking Path > - Paperclip manages agents and their work through normal company controls. > - Product E2E journeys check these controls through the browser and public API. > - Cloud onboarding uses one Connect action to validate the environment and hire the first agent. > - A failed Connect action previously left only a general journey error in the report. > - This pull request retains a closed diagnostic receipt for that action. > - The report can show the observed failure boundary without retaining private response text. ## Linked Issues or Issue Description **What existing behavior does this improve?** Cloud onboarding Product E2E failure evidence and report detail. **Current behavior** A validation or hire failure can reach the journey as a general operation failure. The report does not retain which Connect response failed. **Proposed behavior** Observe only the same-origin, same-company validation and hire POST requests made during one Connect action. Retain a closed receipt with the checkpoint, response state, bounded HTTP status, and allowlisted adapter, outcome, check code, level, and server code. The Get started control and saved-agent readback remain required. **Reason and benefit** The next failure can be attributed to the exact observed browser/API boundary. This change does not claim the cause of a historical failure. **Breaking changes** The new receipt is optional. Historical results remain valid. Explicit validation and hire rejections now stop the journey with a precise closed error code. This changes only E2E source; it does not change product behavior or qualification gates. ## What Changed - Add a bounded Connect observer with exact request matching and listener cleanup. - Retain known server codes from `details.code` or `code` without response messages, hints, headers, request bodies, provider text, or credentials. - Validate the receipt as a closed schema and render its allowlisted fields in existing report detail. - Include the new helper in the journey definition hash. - Add route, privacy, malformed-body, deadline, late-response, delayed-failure, success, and historical compatibility tests. Document the evidence boundary. ## Verification - Focused Product E2E tests passed: 3 files, 58 tests on the review-fix head. - E2E typecheck, workspace typecheck, and full build passed on the preceding implementation head. The review fix changes only report labels, checkpoint timing, and the matching tests. - The local full `pnpm test:run` command stopped progressing at the general-server start header. Its owned session was interrupted. This is not recorded as a pass. - All 58 remote checks completed on the final review-fix head: 55 passed, one neutral, and two expected skips. Both Greptile and Paperclip reviews are 5/5 with no unresolved review threads. - No live campaign was run with this patch. The observed historical failure is preserved. ## Risks - Unknown or missing responses remain explicit diagnostic states. They do not establish a product root cause. - Response body reads stop after five seconds or the action deadline and accept at most 64 KiB of JSON. Checks are capped at 32 retained entries. - New server or check codes appear as `unrecognized` until the closed allowlist is updated. - The production launch classifier, native gateway qualification, and Get started requirement are unchanged. ## Model Used OpenAI Codex (GPT-6 family), with code execution and tool use. The host does not expose the exact model ID or context window to this run; these values are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 21:32:42 -07:00
DottaandPaperclip	a967d479bd	fix(e2e): retain verified onboarding artifact evidence (#17370) ... Retain the bounded verified onboarding download and independent oracle in private normalized results before cleanup. Preserve the receipt when later readback or trace checks fail without marking the journey as passed. Refs #17262. Refs #17253. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-23 14:23:08 -07:00
DottaandPaperclip	3bd709df8b	Operationalize Product E2E accounting, tracing, and scheduling (#17009) ... ## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E tests exercise real agents, tools, setup and cleanup. > - Repeated runs need retained costs and traces, including failed attempts. > - A passing retry must not hide earlier spending or feedback. > - Recurring work also needs admission limits, private reports and failure detection. > - This change supplies those controls while keeping paid activation behind existing gates. ## Linked Issues or Issue Description Refs #16996 and #17158. This is the consolidated landing of the reviewed eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033, #17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096, #17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127, #17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository permits squash merges only. A local rehearsal showed that squashing each slice into master causes repeated conflicts in dependent slices. This PR preserves their commits and lands their combined implementation in one reviewed integration. Superseded slices will close after this merge. ## What Changed - Retain every attempt's cost receipt, including setup, failures, interruptions and retries. Separate reported amounts, estimates, missing costs and reserved capacity in reports and dashboards. - Add bounded request admission, exact model pricecards, gateway error accounting, cache policy and reconciliation evidence. Refuse unsupported routes and requests that cannot fit the allowance. - Integrate Braintrust gateway traces and OTLP correlation, with secret filtering and experiment imports. Record phase timing, including setup, feedback capture and cleanup. - Collect complaint and suggestion evidence after each attempt, including recovery after failures. Retain explicit review dispositions independently from the product grade. - Add gated daily and weekly selections, measured forecasts, setup/readiness checks, wall-time limits and retained monitor notifications. Include declared non-model overhead in admission while reporting the existing subscription separately. - Keep recurring history private, bound uploads and retention, and use isolated archive environments. Keep public publication an explicit operation. Report and publication jobs install only the committed lockfile, with scripts disabled and isolated stores; stale pins fail closed. - Integrate current master, preserve its Product E2E journey billing and privacy checks, and test that shared setup receipts do not double-count estimated agent usage. Preserve upstream fixture repairs. ## Verification - Complete integration baseline `0649a5ce7544ab20825c2183d2f421157db49f24` includes master `a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head as ancestors. - On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012 Vitest tests passed, two skipped, across 193 files; all 137 Node source-contract tests passed. - Focused billing and Product E2E journey contract checks: 24 tests passed. They cover estimates, unknown receipts, single-owner setup costs, browser-only execution and private publication guards. - Monitor and workflow wiring: all 96 Node tests passed. `node scripts/check-token-gates.mjs`: all gates clean. - `pnpm test:e2e:runner:typecheck` passes after the standard Runner TypeScript build refreshed local declarations. The first attempt rejected the new master `runner_prepared_launch` union member because this checkout retained an older generated declaration; the source union already contained that member. - Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes dependency re-resolution from archive-authorized reporting jobs after the security scan flagged it. All 15 workflow-security tests pass after updating the old two-install assertion. - Hosted [CI run 39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180) passes on the current repaired head, including typecheck, build, general tests, Runner checks, browser shards and canary dry run. Greptile is 5/5 on this exact head with no unresolved threads. The security scan passes after the fixed lockfile-resolution thread was resolved. - Paperclip Review returned a terminal neutral check but an incomplete 0-file assessment on the repaired head: its platform blocked both diff responses as prompt-injection content. This is not a successful code review. No bypass or fabricated pass was attempted; current-head Greptile, security and hosted CI are independent gates. - No paid test, reservation release or recurring-run activation accompanies this landing. Live route qualification and invoice reconciliation remain separate evidence. <details> <summary>Retained earlier validation failures</summary> Earlier broad local runs had two 15-second test timeouts (CLI poll cadence and Retry validation), one embedded-database setup timeout, and later PostgreSQL semaphore exhaustion. The exact timed-out tests passed separately without larger timeouts; the broad-run causes were not established. Those local failures remain distinct from subsequent hosted CI results. The independent #17158 fix records its Muse fixture collision, its UUID correction and hosted verification. Local Muse execution was also blocked by macOS embedded-PostgreSQL startup. No historical failure is treated as a successful paid qualification. </details> ## Risks - Admission is a local control, not a provider invoice cap. Missing provider receipts or resource charges retain unknown amounts and holds; estimates do not become billed dollars. - The bounded path admits only documented local suites and provider routes. Ordinary legacy or separate Runner protocol campaigns have different controls. Merging this code does not qualify additional routes. - Recurrence still needs the existing enable gates and current qualification/allowance evidence. Archive permissions and resources must match the documented private-environment contract. - Legacy reports may lack complete attempt or phase evidence. Reports preserve that limitation and do not infer zero usage from missing data. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks; full-suite host limitation disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:18:37 -07:00
DottaandPaperclip	5ef385f7e6	Run reusable Product E2E journeys across deployment targets (#17262) ... ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description **Subsystem affected** Product E2E harness, fixture registry, and reports. **Problem or motivation** Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. **Proposed solution** Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. **Roadmap alignment** This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 06:27:49 -07:00