34 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	0884e73b46	test(e2e): verify saved assigned skill use (#18783) ... ## Thinking Path > - Paperclip lets people manage agent work and reusable company skills. > - A saved assigned skill must affect a task's actual execution. > - Existing QA coverage creates and edits skills. > - The daily target needs coverage for ordinary saved-skill use. > - The task receives its receipt data only through one saved skill. > - The admitted immutable version, native read and saved document establish the outcome. > - This PR adds one bounded journey for that gap. > - A shared lease test also needs a stable clock to assert its exact timeout. ## Linked Issues or Issue Description Refs: #18305. Supporting repair #18885 merged as `057898530c702ec14c1e14d163e96dfb045cd8c0`. It closes the recovery-conflict prerequisite. This branch does not duplicate that production patch. **What existing behavior does this improve?** Product E2E coverage for ordinary assigned company skill use on an existing deployment target. **Current behavior** The deployment journey catalog can create and edit a company skill. It has no separate case that requires a native task to load its assigned saved skill and use data available only inside that source. **Proposed behavior** Create one synthetic managed company skill through the public API. Assign its canonical key with the normal Latest selection. Create the task through the production browser composer. Require one native work run, the exact created immutable version in its admitted skill-version map, a successful native source read or load, and an agent-authored saved receipt. Reload and verify the result, revision and original source. ## What Changed - Add `saved-skill-use` with one native work run and seven required checks. The first check is `saved-skill-assignment-bound`. - Use ordinary `desiredSkills: [canonicalKey]` assignment. Require the returned stored assignment to have `versionId: null`. This works with Beta skills off and reads or changes no experimental setting. - Bind the creation receipt, pre-dispatch source read and version file inventory. Require a managed skill with the exact company key. Require the run's authoritative `skillSelectionSnapshot: true` and admitted version map to match the created immutable version. - Keep the random receipt reference and numeric inputs only in the saved source. The task prompt names the skill without supplying its answer. - Require exact company, task, agent, run, turn and runner bindings for a complete successful native source receipt. Accept the deployed OpenCode read wrapper only when both the saved assignment and native execution input bind its runtime directory to the created immutable skill version. Also accept the first-party assigned-skill read with a provider-independent canonical key/version binding from both the saved API assignment and admitted native input. Require the exact full source, checksum, byte count and complete page. Keep the materialized directory-name guard specific to the OpenCode filesystem path. Reject claims, shell output, foreign or incomplete reads and wrong skill loads. - Add a narrow Codex reader for the retained controller `semantic_tool.input` and `semantic_tool.result` pair. Require the actual native provider, runner, session, call, turn and item; the admitted immutable key/version; the read-only `read_assigned_skill` operation; and its successful active-task receipt. Recompute the input digest and the exact complete seven-field read-result digest from the saved source. Raw dynamic or MCP names grant no authority. Keep the builtin OpenCode/filesystem reader body unchanged. - Require authoritative native commit before qualifying the saved receipt. Independently check its value and same-run document revision authorship. Reload and verify the result and unchanged source. Retain those existing readback receipts in the private checkpoint for independent verification. - Keep the exact raw campaign ID in the journal, API name, frontmatter and saved name. A lost successful creation response remains recoverable by name. - Register discovery, source fingerprints, closed checks and existing feature associations. Update the case and fixture docs. Feature-map files are unchanged. - Stabilize one existing shared sandbox lease regression with a scoped `Date.now` spy and `finally` restoration. Keep its exact timeout, no-destroy, no-replacement and recorded lease assertions. Change no production service code. ## Verification - **Genuine QA staging qualification passed on this exact `2fb654f7be0bac3bfeb4ec6b7a2eb7a06b8de7e4` head**: [workflow41763421118](https://github.com/paperclipai/paperclip-cloud/actions/runs/41763421118), [TestOps launch2257/result4034](https://paperclip.testops.cloud/launch/2257/tree/4034?treeId=0), [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41763421118-1/). All seven required checks and all five phases passed, with exactly one configured OpenCode/DeepSeek native run succeeded and committed. Owned cleanup closed. A separate source-pinned replay verified25 original-evidence controls and16 rejection controls without altering originals. Automatic #testops PASS was independently read back. Both earlier failed QA grades remain unchanged. - Current-head CI `41758727042` attempt 2 passed with 48 successful jobs and one skip. The coordinator authorized one unchanged-source failed-jobs diagnostic. Start and end timestamps prove three fresh executions (Runner package, server shard 7 and aggregate verification), 45 retained successful executions and one skipped non-execution. The prepared-context test passed in 2,534 ms. All 28 repository rehome tests passed. The original failed attempt remains sealed. Its timeout causes are still unknown. No source, timeout or assertion changed, and no reliability fix is claimed. - Fresh exact-head Greptile review is 5/5 with zero unresolved threads. All 58 current check-runs and the status rollup are terminal and clean. The native Paperclip review ended neutral and incomplete, so it is not a validated assessment. The PR remains mergeable after the dependency case merged in #18901. The source remains frozen. A coordinator-only qualification draft is prepared with approval false. A genuine QA staging pass is still required before daily enrollment. - Current head `2fb654f7be0bac3bfeb4ec6b7a2eb7a06b8de7e4`: 269 focused tests across eight files passed. These include 42 Codex producer, converter and false-positive controls. The canonical E2E typecheck passed with its own declaration prerequisites. Credential-free discovery lists 24 journeys. Fresh exact-head Greptile review passed at 5/5 with zero unresolved threads. CI `41758727042` attempt 1 failed. It completed 45 successful jobs, one skip and three failed jobs. The primary failures were one existing OpenCode prepared-context test at its unchanged 5-second timeout and three existing repository rehome tests at their unchanged 15-second timeouts. The third failed job aggregates those results. The timeout causes are unknown. The original attempt remains sealed. A direct passing twin ran the prepared-context test in 2,533 ms. All package, script, CLI, UI, workflow and root inputs match between the actual tested merges. Three server files differ, including `task-retry.ts` inside the broader rehome import graph; no full-server equality or causal fix is claimed. The later single diagnostic pass is recorded above. The native Paperclip review ended neutral with an incomplete execution, so it is not a validated assessment. No live QA pass is claimed. - The valid review on `e1a474` found that Codex displays its first-party tool as dynamic. Source inspection confirmed that its raw name has no authority. The retained controller semantic input/result pair supplies the real operation authority. Core `09162fe893b464dd2ccdafd14bd3ef1611202a86` and current source have identical relevant envelope, digest, result-unwrapping and immutable-read implementation regions. - Three real-producer positive controls failed against the old grader. All 42 now pass. The tests call the actual immutable-file reader, production protocol digest builders, Codex display converter and API redaction. They validate the event schema and cover omitted defaults, the exact runtime alias and Unicode bytes. Missing, duplicate, reordered, failed, foreign, MCP, unbound, partial-page, wrong-digest, wrong-key/version and nonce-only evidence fails. Generic dynamic output fails even with copied full text and invented read-only markers. - The original builtin reader body and source-only fixture function are byte-identical to `e1a474`. All seven checks, the one-run limit, 500-cent guard, ordinary Latest assignment and Beta-off behavior remain unchanged. The independent verifier is pinned to this new head. Its synthetic semantic receipt passes 24 negative controls and its filesystem receipt passes 16. Older helpers and both original failed QA seals remain unchanged. - Earlier integrated head `e1a47438b1c7e51a5bb2896a4ec9b6ed6560318b`: 376 focused checks across eleven files passed, including Saved, settings and Chat overlap controls. All seven canonical server type layers and exact-head E2E types passed. All four lease controls passed. Credential-free discovery lists 24 catalog journeys. No new staging run has occurred. - Normal integrations consume actual master `fbb7055b9020e505cd1bf903b1b1c301ba8f3eef` and actual Chat squash `d7283e875772ca8c684dc86e471e6b48d543b7b1`. Preserve all five Saved case/parser files byte-for-byte and retain settings and Chat registrations. The only manual resolutions were two additive handler/type anchors. All seven checks, one-run limit and 500-cent guard remain unchanged. The independent verifier is pinned to this exact head. - Earlier dbdf CI passed after one authorized failed-jobs recheck of a confirmed Spot-terminated Build runner. Timestamp comparison proves two fresh executions (Build and verify), 46 retained successful executions and one skip. The original failed attempt remains sealed. Its native Paperclip Review completed neutral with an incomplete execution and is not a validated assessment. - New master entries made the earlier dbdf PR conflict. The frozen qualification guard correctly blocked dispatch before any pin or receipt. Preserve its unapproved draft as history; it did not authorize a QA run. This normal integration resolves the observed conflict. Fresh current-head CI/review and one genuine staging pass remain required. - Earlier merged head `dbdf1f3cbeaacc7cf9a2a457618035c945bd09f0`: 227 focused checks passed across seven files. All seven canonical server type layers passed. The E2E typecheck passed using the declarations built in this same checkout on this exact head. All four existing clock and lease controls passed. Credential-free discovery lists 22 catalog journeys, including both saved-skill use and task lifecycle. These are local checks, not a live staging qualification. - Normal merge of master `8771606fd87f69ed4057779e3dce3a9bca8088e6` retains the reviewed task-lifecycle case and its Cursor fixture repair from #18782. Resolve six shared journey registrations by preserving both cases. The five saved-skill case files remain byte-for-byte equal to reviewed `ff3814`. All seven required checks, one-run limit, 500-cent bound and ordinary Beta-off Latest assignment remain unchanged. No feature-map delta was added. - The independent sealed-run verifier is now pinned to this exact merged head. Synthetic semantic evidence passed with 24 negative controls. Synthetic filesystem evidence passed with 16 negative controls. Foreign document authorship and missing reopen/source readback still fail. The old `ff3814` verifier and both original failed QA seals remain unchanged. - Common generated outputs were copied only after all 4,225 package, script and root inputs matched, installed lock/TypeScript hashes matched, and all 4,955 source output paths and hashes passed before and after copy. No server declarations or editable source/module links were copied. The server declarations were rebuilt locally. A full frozen workspace install disabled lifecycle scripts and used the same locked packages. No Rust build ran locally. - Previous `ff3814` CI `41733864320` exposed a real wrapped PostgreSQL uniqueness error in a recovery server test. That production defect is independently fixed and tested in #18885. The later supporting PR merged separately. This branch does not duplicate its production patch or regrade the original failure. Preserve the original failed result and the separate Spot interruption. New full CI and a fresh exact-head 5/5 review remain required. - Previously, **225 tests passed across six files** in the focused evidence, flow, contract, catalog and report set after the semantic-read correction. The prior 181-test proof and all original failed attempts remain retained. This correction changes only the saved-skill evidence helper, flow argument, their tests and case documentation. The shared clock repair remains unchanged. - E2E TypeScript check passed on Node **24.22.2**. - Earlier credential-free discovery listed 21 cases and exposes the ordinary saved-skill journey without a Beta prerequisite. These local checks make no provider or model calls. - Source inspection of Core `09162fe893b464dd2ccdafd14bd3ef1611202a86` confirms key-only selection becomes a stored null Latest selection. Managed execution resolves its current immutable version and records the admitted map in `contextSnapshot`. The public run API preserves that map independently of optional result presentation data. - Seven invalid admitted-version fixtures wrongly qualified without the new guard. The regressions now reject them before the read or document can qualify. Beta-off and Beta-on ordinary request controls use the production sync validator. Wrong assignments, missing null selection, source drift, creation-version mismatch and another skill's complete native output cannot pass. - Existing native commit, null/compact result data, read provenance, nonce, saved-result, reload and lost-creation-response controls remain required. Their original red results are retained. - The earlier experimental version-pin variant remains unqualified. Its missing-prerequisite regression and the denied target-setting attempt remain separate failure evidence. This case qualifies ordinary assignment and the version actually admitted by the server. - Normal merge of master `5f1b59ba2c9adfb40a681bf419a8b7e7837d1125` includes the reviewed inbox snapshot fix #18798 and App/Shell access-fixture fix #18799. The merge has no conflicts. All 17 journey source, helper, registration and documentation files remain byte-identical to reviewed head `eab260955ff43eeacfdafc9b75dfc9b116f0eb79`. - Previous CI `41672686614` failed two UI test shards with obsolete user-ID fixtures; its aggregate verify job also failed. The merged access-fixture repair addresses those failures. This is a new source CI run; the failed run is retained. - Dependency output reuse follows exact path and byte checks for 10,977 tracked build inputs and 3,411 generated outputs. The first copy guard detected 12 bundles regenerated after its initial manifest and stopped before any destination mutation. The stable replacement manifest passed all checks before copying. Final focused tests, E2E typecheck and discovery passed after the verified copy. No editable source or module link was shared. - An earlier integration found stale generated Cursor Cloud declarations. Canonical dependency builds refreshed them, then typecheck passed. The original typecheck failure remains recorded. - The initial report support failure came from a blocked `tsx` IPC pipe before report execution. The unchanged test passed with normal local IPC permission. The original failure and diagnostic remain recorded. - `git diff --check` passes. Full workspace recursive typecheck, unit run and build remain assigned to hosted CI and coordinator verification. Full CI `41679720041` passed on the prior `f3f288a54177f2b9a7b225b1d1ab35440dfc597c` source. A clean 5/5 review passed on `0f445cf1c622afb25a1af92398784209df542696`. Fresh CI and review are required after the shared clock fixture repair. - The first real QA staging attempt on `f3f288a54177f2b9a7b225b1d1ab35440dfc597c` failed `saved-skill-native-read-attributed`. Assignment, immutable admitted version and cleanup passed. The test stopped before document, reload and source-after observations. The original failed attempt and its complete evidence remain sealed. - The original successful builtin `read` event contained the full saved procedure. OpenCode returned its file wrapper with an unknown normalized operation, null target and a hashed runtime skill name. The old parser rejected that shape. A synthetic regression failed before this correction. The corrected parser recognizes that exact retained read in memory, with no change to the original grade. Wrong paths, copied shell text, missing starts, mismatched calls, failed/truncated reads and wrong version/name bindings still fail. A later staging attempt is described below. Neither failed original was regraded. - Original CI `41703535552` failed one shared server test and its dependent verification job. The lease-resume test expected an RPC timeout of 31,234 ms, but the runtime correctly supplied 31,233 ms after one elapsed millisecond. The test and runtime service were byte-identical to the previous passing source. The failed run remains recorded. - A scoped advancing clock reproduced that exact 31,233-versus-31,234 failure locally. A stable clock around only the acquisition call now preserves the exact original assertion. The repaired test and three related sandbox lifecycle controls pass on disposable embedded Postgres. No production timeout, lease preservation, cleanup or replacement behavior changed. - The first local reproduction stopped before test execution because six same-package native Postgres library aliases were missing. Their exact existing versioned libraries matched the canonical dependency. Restoring only those internal relative aliases allowed the separate deterministic red and green measurements. This prerequisite failure remains recorded. - The second real staging attempt on `02582aadd23fc128ae29daa69482113cf0f95ad1` also failed `saved-skill-native-read-attributed`. Authentication, setup, assignment, source-only input, collection and cleanup passed. It completed one native run. Later document, reload and source-after checks were not executed. The complete original remains sealed as failed. - That attempt used the ordinary first-party `paperclip_read_assigned_skill` tool. Its complete native receipt contained the exact canonical skill key, created version, `SKILL.md` bytes, SHA-256, byte count and null next-page offset. Core verifies the pinned manifest and file checksum before returning that receipt. The old grader recognized only filesystem and native skill-load output. An actual-shape regression failed before this correction. The corrected parser recognizes the retained read in memory. Wrong keys, versions, source bytes, hashes, partial pages, semantic failures, copied shell output and foreign or unpaired native calls still fail. No model prompt, product source or original grade changed. A live pass on this corrected source remains required. - CI `41708636059` on the previous `02582` source passed at attempt 3. The first two interrupted attempts remain recorded. AWS evidence identified Spot capacity termination for the second Rust runner. The targeted retry executed Rust and aggregate verification; 46 successful jobs retained their exact prior execution timestamps and one job remained skipped. New metadata IDs do not establish new executions. Fresh full CI and review remain required for this source. - Earlier CI `41730743279` passed on `1ee86838ea528edd8c60fb9ddd858faf3c83e783`. A later review finding correctly showed that the semantic read still depended on an OpenCode-only directory-name helper. That finding remains recorded. Three local Codex, ACPX Cursor and Cursor Cloud flow controls reproduced the rejection before this correction. - The semantic path now binds the canonical key and immutable version through the common native runtime skill selection contract. Its checksum, complete source, page and native call attribution checks stay unchanged. Missing, duplicate, wrong-key and foreign-version API/native assignment controls still fail for all three provider configurations. The OpenCode filesystem wrapper still requires its exact provider and materialized directory name. These local controls test the existing canonical receipt shape; they do not claim live provider qualification or add a guessed provider encoding. Fresh CI and review are required after this correction. - This public PR does not enroll the case in the daily plan or change protected QA configuration. ## Risks The target must supply a disposable native agent with normal company skill-write access and cleanup. It must preserve the authoritative admitted version map and provide a complete source read or named native skill-load receipt. Missing evidence fails the case even when the receipt document has the right value. The OpenCode wrapper path requires matching assignment and native input records for the same immutable version; it does not accept a generic targetless read. The builtin first-party path requires the same version binding and exact full-source receipt. Codex must provide its authoritative controller pair and a digest that commits to that exact complete result. Partial pages, unknown operations and semantic failures do not qualify. Local flow controls simulate the target API and browser boundaries. They do not establish live Cloud qualification. Experimental assignment pins need separate qualification. The journey changes only its owned skill, assignment and task. It changes no shared agent, setting, model, connection or budget. ## Model Used OpenAI GPT-6 through Codex with reasoning and code execution. The runtime does not expose a more specific model identifier or context-window size. Local checks made no product-model calls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 21:13:40 -06:00
DottaandPaperclip	630ed6624c	test(e2e): add verified dependency continuation QA (#18901) ... ## Thinking Path > - Paperclip lets people manage work assigned to AI agents. > - Task dependencies must hold assigned work until its blockers are complete. > - The reusable Product E2E set needs a browser journey for that full path. > - The journey must verify saved records, automatic wake identity, and the result after reload. > - The original implementation passed staging QA, but its automated review workspace has an export hold. > - This replacement integrates current master and earns a separate normal review while preserving the original attempt. ## Linked Issues or Issue Description Refs #18789, the original dependency QA pull request. Related: #7542 and #17524 for dependency wake and explicit resume behavior; #18782 for the merged lifecycle registration layout. **What existing behavior does this improve?** The Product E2E catalog can verify the complete blocked-task continuation path against a configured native Cloud target. **Current behavior** The main catalog lacks this full browser journey. The original PR's automated review task has a retained workspace export hold (`native_workspace_sync_out_retry_exhausted`), with neither Retry nor Continue available. Its task, files, checks, and failed export attempts remain preserved. **Proposed behavior** Add the same independently checked `task-dependency-resume` journey through this replacement. Its new commit normally merges the actual fetched master `4e29f001396c9fc76075850165ccfa3a6743e3ef` into verified source `e53dd51407b901b14bd582eeb2071fabe6955d9d`. That integration contains 39 real upstream path updates. The replacement must pass fresh CI and automated review on its own head. **Reason and benefit** Daily QA can detect premature work, a missing or duplicate continuation, and a result from the wrong run. A separate review workspace lets the normal review process inspect the current integration while the original held workspace stays intact. **Breaking changes** The change adds an explicit journey. Existing journeys and their registrations remain present. ## What Changed - Add a browser journey that creates a human-owned blocker and dependent task, saves their relation, selects Blocked, and assigns the configured native agent. - Require a measured ten-second blocked wait, a board-owned blocker completion, and exactly one automatic successful native run. - Independently verify the saved wake source and blocker identity, configured provider/model/environment, committed native phase, document revision authorship, and reloaded result. - Return `unsupported` before observations or task creation when the native provider or model identity is missing or blank. - Attempt both evidence writes, preserve an existing exact error object and outcome, and fail otherwise successful work when required evidence cannot be saved. - Grade runtime identity, document authorship, reload visibility, and retained history from their own evidence bound to the same company, task, agent, and run. All eight checks remain required. - Keep the random review reference in the blocker description. Save allowlisted private observations and read failures through the existing evidence pipeline. - Register the case at the distinct dependency anchors in the catalog, types, checks, source hash, dispatcher, features, and documentation. The delta against fetched master is exactly the original 15 QA paths. The fixture, contract tests, shared registrations, and dispatch remain unchanged. The browser actions and observations are byte-identical to `d41558b5`; only evidence finalization changes in that flow. Six shared-file AST comparisons still match fetched master after removing only the dependency registrations. All upstream integration changes remain preserved. ## Verification Prior acceptance remains bound to its actual source: - [Staging QA run 41731870131](https://pages.paperclip.ing/evals-qa-smoke-41731870131-1/) passed all eight checks and all five phases, with closed cleanup, on `dc9ea42d46d3015c010d0532b48e0da973d3611a`. - [Original full CI run 41745149524](https://github.com/paperclipai/paperclip/actions/runs/41745149524) passed on `e53dd51407b901b14bd582eeb2071fabe6955d9d`: 48 successful jobs and one skip on attempt 1. - The original failed staging result and the original held review remain intact. These prior passes are not new checks for this replacement. Passed locally on the current replacement source with Node 24.22.2 and pinned pnpm 9.15.4: - One complete `install --frozen-lockfile --ignore-scripts`, followed by the maintained `node scripts/link-plugin-dev-sdk.mjs` for links inside this checkout. - `node scripts/ensure-plugin-build-deps.mjs` and `pnpm --filter @paperclipai/db run build`. - `node scripts/typecheck-test-projects.mjs --low-memory tests/runner-e2e`, including the current Runner TypeScript build, fresh server declarations, and E2E typecheck. - 239 focused controls across eight files. These cover missing/null/empty/blank execution identity, real filesystem write failures, synchronous writer failures, frozen original errors, and independent defect attribution. - 5,302 comparisons against the prior grader, with zero aggregate acceptance changes or new throws. The sealed actual staging observations from run 41731870131 still produce the same eight passing checks. This is finite calibration of retained evidence, not a new staging run. - Credential-free discovery: 22 unique cases, including dependency and lifecycle. - Source-preservation checks and whitespace checks for both the normal integration and the replacement delta. The focused command is: ```sh node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/product-journeys/task-dependency-resume-evidence.test.ts tests/runner-e2e/product-journeys/task-dependency-resume.test.ts tests/runner-e2e/product-journeys/task-dependency-resume-contract.test.ts tests/runner-e2e/product-journeys/contracts.test.ts tests/runner-e2e/product-journeys/observation-read.test.ts tests/runner-e2e/catalog.test.ts tests/runner-e2e/journey-features.test.ts ``` The old generated cache failed current-source equality and was not copied. Canonical current outputs were built and sealed with their input hashes, including CLI, UI, and provider inputs. No Rust build, QA dispatch, model call, or action on the held workspace was performed for this replacement. Review follow-up: the first replacement head `b4408c5732e1800f7ccf66c639cf382b5ff2f01a` passed full CI run [41749940687](https://github.com/paperclipai/paperclip/actions/runs/41749940687), with 48 successful jobs and one skip. The native review then found the missing execution-identity preflight. The preflight now rejects that setup before work. Completed current-head checks are recorded below. The current E2E typecheck reused the same canonical declarations only after all 11,273 build inputs, two installed toolchain files, and 6,747 generated files matched their saved hashes. The five changed case files are outside that build-input set. One sandbox test run passed 216 controls and lacked the report subprocess output. A direct `tsx` diagnostic confirmed a temporary pipe `listen EPERM`. The unchanged 217 controls then passed with ordinary local IPC permissions. Both results remain recorded. The `d41558b5807e609032bbadddb103d6b0a7db4a12` attempt [41753398775](https://github.com/paperclipai/paperclip/actions/runs/41753398775) is retained: 45 successful jobs, one skip, two failed server shards, and a failed aggregate. The runtime readiness case refused connections within its ten-second window; the issue-deadline and large repository cases hit their existing test limits. Their causes remain unknown. A source-matched Linux CI run executed and passed all three cases, and one unchanged local runtime case passed. No timeout or assertion changed and no old-head diagnostic recheck was issued. The native review then identified error masking during evidence persistence and cumulative diagnostic flags. The current head fixes those two owned-case issues. Its completed current-head checks are recorded below. Seventeen of twenty new regression controls failed before these fixes; all now pass. Two additional controls exercise synchronous writer failures. Both original review and CI evidence remain intact. The eight required checks, one-run limit, 500-cent configured budget, and bounded ten-second observation are unchanged. Fresh full CI, current-head Greptile 5/5 with no open findings, and every actual review context must complete before merge. Prior statuses are not copied or overridden. Current-head verification is complete for `d9bc055f1f6609463679dc78fdcb96034d2c3727`: - [Full CI run 41758005698](https://github.com/paperclipai/paperclip/actions/runs/41758005698), attempt 2, completed successfully at 02:26:19 UTC on November 4: 48 successful jobs and one skip. - Attempt 1 lost one exact Spot runner. AWS confirmed `Server.SpotInstanceTermination` and `instance-terminated-no-capacity`; its log recorded the shutdown. That failed job and the failed aggregate remain preserved. - One normal failed-jobs recheck ran only the interrupted static checks (02:19:13–02:25:54 UTC) and aggregate (02:26:13–02:26:18 UTC). The other 46 successful start/finish pairs remained unchanged. The skipped selector was not an execution. - Greptile and Paperclip Review both scored this exact head 5/5. There are no unresolved review findings. ## Risks - The staging pass remains attributed to `dc9ea42`; final admission must retain that source provenance and the current preservation proof. - The case uses one paid work run when executed. Agent setup usage remains separate from task-run accounting. - The ten-second blocked observation proves a finite wait. Missing, incomplete, foreign, premature, or duplicate evidence fails the independent checks. - The original held workspace still needs its own disposition. This PR leaves it preserved for its owner. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, and code execution. The exact model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 20:35:43 -06:00
DottaandPaperclip	d7283e8757	test(e2e): cover chat reload and fresh sessions (#18784) ... ## Thinking Path > - Paperclip keeps agent work attached to saved company records. > - Agent Chat keeps one conversation for each person and agent. > - Reload must preserve context without starting another turn. > - The ordinary `/new` command must create fresh provider context and preserve history. > - Existing local chat journeys cover these behaviors, but the deployment-targeted daily catalog does not. > - This PR adds one reusable journey with independent saved-state and rendered-reply checks. ## Linked Issues or Issue Description Refs: #2242, #18655. **What existing behavior does this improve?** Product E2E coverage for Agent Chat persistence and session reset on an attached deployment. **Current behavior** The deployment-targeted catalog has no bounded check for recall after reopening and a fresh session in the same saved conversation. **Proposed behavior** Remember a random phrase, reopen and reload the chat, then ask for the phrase without supplying it. Send `/new` through the ordinary composer. Require a new native session and a fresh-context answer while the original chat and history remain saved. ## What Changed - Add the `chat-continuity` flow and independent evidence grader. - Reuse the existing chat sender and run-bound native reply reader. - Require exact board-user message attribution, three successful native provider turns and one synthetic reset row. Count all four persisted run IDs. A reset with provider input or a native session cannot pass as synthetic work. - Use the original bound issue-run summary to prove reset completion when the detail result is null or compact. Preserve the raw detail and require its reset context, company, agent, issue and wake comment to agree. Native provider turns require committed native state and independent output evidence; an optional duplicate result field is not required. - Require an empty agent-filtered run history before the first send. Reused agents are unsupported for this case. Validate every agent-scoped activity row and reject mixed or truncated responses; do not filter global history into a pass. - Require the fresh native detail to contain an explicit null `sessionIdBefore`. Missing or empty admission evidence cannot prove a fresh provider session. Keep optional result blobs optional. - Check provider, model, environment, native session, finalization and reset generation. Reject stale replies, duplicate sends, extra runs, missing history and unexpected work. - Permit authenticated chat screenshots through the existing same-origin and mounted-Layout guard. Authentication and unrelated settings pages remain excluded. - Register the case, exact checks, source hash inputs, discovery and documentation. Feature-map files are unchanged. ## Verification - Normal merge commit `ddc37bb05b87cbc1fa449e4b2fc5a4677f0fe3b6` has exactly two real parents: the staging-qualified chat source `4c6cc9d67d2ae577e3a8ae7856bd6ae22a8489c1` and lifecycle squash `8771606fd87f69ed4057779e3dce3a9bca8088e6` from #18782. Thirteen dedicated case, support and reused runtime/oracle files are byte-for-byte preserved. The eight shared registry placements match the finite integration forecast. The nine required checks and four-run limit are unchanged. - The integrated source passes 99 focused tests across three files and 925 related Product E2E/catalog/native-reply tests across 42 files. Credential-free discovery lists 22 unique journeys, including chat and lifecycle exactly once. The PR diff against the actual lifecycle squash passes its whitespace check and remains the original 17 chat paths. - The current canonical `node scripts/typecheck-test-projects.mjs --low-memory tests/runner-e2e` passes, including Runner TypeScript and server declaration prerequisites. The DB migration checks and build pass against the actual squash's execution-admission schema and migration 0428. No Rust build was run locally. Canonical generated outputs are bound to current tracked inputs and were rechecked after tests. - The first direct typecheck found a stale generated Runner export, `taskDocumentKeySchema`; the current canonical prerequisites resolved it. A frozen offline install lacked a cached locked tarball, and an initial child command reached the global pnpm version switch and failed signature retrieval. Fetching only locked packages and pinning child pnpm to 9.15.4 corrected the setup. The original diagnostics remain retained; no tracked source was changed to bypass them. - The unchanged chat implementation passed a genuine staging campaign, workflow 41725181387, at source `4c6cc9d67d2ae577e3a8ae7856bd6ae22a8489c1`: all nine checks, all five phases, three provider turns plus one synthetic reset, all four usage IDs, and closed cleanup. This evidence remains bound to that original source. The new integration head requires its own current gates and coordinator qualification decision; this PR does not enroll anything in the daily plan. - The earlier staging workflow 41701828968 remains failed. Its nine behavior checks passed, but private collection rejected the valid reset's capture of the old session and cleanup failed. The private adapter fix and evidence-first recovery were handled separately. The original failed grade and archive are preserved. Reset prior-session capture is distinct from the third native turn's explicit-null fresh admission. - Three missing/undefined/empty fresh-admission controls reproduced false passes on `247827a066af01fa583ff030df79b61cb2c87730`, then passed after the strict null fix. Four earlier controls reproduced compact-detail, mixed-activity, missing fresh-agent receipt and unscoped-activity problems. Their original failures remain retained. Positive and wrong outcomes still enforce bound reset summaries, committed native state, configured identity and independent rendered output. Board provisioning uses the acting-agent filter; it does not grandfather earlier work by the test agent. - The first broad support attempt passed 861 tests and failed one report-generator check. A separate reproduction showed `listen EPERM` for the tsx local IPC socket before report output; the unchanged support suite passed with normal local IPC permission. An early flow fixture also returned an object where the events API returns an array; the real event reader rejected it, and only that fixture was corrected. Both original failures remain recorded. - Original commit `d6226b30acaf5a2dad28030f5b295ac1dd119565` failed full CI [41653171240](https://github.com/paperclipai/paperclip/actions/runs/41653171240): a CLI child-server runtime-file wait and two 15-second attention-retention timeouts, followed by database setup/cleanup errors. Their exact causes remain unresolved; the failed jobs and logs were retained and not retried. A later pass does not explain those timeouts. - Strict-admission source `055a7fa99e7cb8aad12b85fccf23880f0dc98a3f` failed full CI [41669527762](https://github.com/paperclipai/paperclip/actions/runs/41669527762): private-invitation and pending-shell access fixtures. The retained assertions match the account-identity fixture repair in #18799, now included through master, together with #18798 and #18650. No old-head job was retried. Subsequent source `4c6cc9d67d2ae577e3a8ae7856bd6ae22a8489c1` passed full CI 41680411215 and current-source review at 5/5. - Current full CI [41743580793, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41743580793/attempts/2) completed successfully on unchanged `ddc37bb05b87cbc1fa449e4b2fc5a4677f0fe3b6` at 2026-11-04 00:34:52 UTC: 48 successful jobs and one intentional selector skip. Attempt 1 remains failed: three run-bound runners were independently confirmed to have lost Spot capacity, and the two aggregate failures reflected those interrupted lanes. The build and boot health smoke had passed before interruption. Exactly one failed-job recheck was used; no source change or further retry was made. Fresh exact-head Greptile review is 5/5 with no new actionable issues and both review threads resolved. The separate Ponytail assessment says Ship with no must-fix or should-fix item; its optional direct board-row grader control remains documented. Whole-workspace recursive typecheck, full tests and full build were covered by hosted verification and were not repeated locally; the bounded local checks above passed. These gates do not claim that a later master merge tree has been live-qualified. ## Risks The attached target must supply a fresh disposable native agent and durable pre-send ownership for its chat. Its private adapter must collect all four run IDs, distinguish the synthetic reset from the three provider turns, and stop and clean up owned resources after evidence is saved. This case verifies browser reload, reopen and explicit session reset. It does not verify controller restart. Unknown setup or provider cost remains unknown. No shared QA source agent, model setting, production behavior or budget policy is changed by this public PR. ## Model Used OpenAI GPT-6 through Codex with reasoning, code execution and repository tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 19:20:34 -06:00
DottaandPaperclip	985b175e83	test(e2e): verify saved agent instructions on the current QA harness (#18891) ... ## Thinking Path > - Paperclip lets users configure agents and verify their work. > - Daily QA must exercise saved instructions on the QA staging instance. > - A fresh owned agent can test an instruction change without changing a shared agent. > - The test must prove that saved instructions caused the next run's output. > - It must also prove that the original instructions were restored. > - This pull request brings the qualified settings journey onto the current shared registry layout. > - The benefit is a reusable daily QA candidate with independent evidence and fresh current-source checks. ## Linked Issues or Issue Description Refs #18788. This is the replacement review PR for the same settings journey after integration with the merged lifecycle registry layout in #18782. **What happened?** The settings journey passed its original staging qualification at source898ae8ab4e49d2408688088ce82e3316b91cc09c. The earlier PR's review workspace remains retained for file preservation. That workspace and its recovery evidence remain intact. **Expected behavior** Review the integrated settings source through this PR's own normal CI and fresh reviews. The journey saves instructions through ordinary UI controls, verifies their effect on one current native run, and restores the original bytes. **Steps to reproduce** Use the documented dedicated agent-settings-persistence campaign. The provider must supply a fresh owned agent and the agent-settings-write capability. Save the instruction edit, reload it, complete the ordinary assigned task, verify its independently saved result and current-run attribution, then restore and verify the original instruction entry. ## What Changed - Add the settings browser journey, its independent evidence grader, fixtures, retained evidence contract and focused controls from #18788. - Require one fresh dedicated agent. Select its exact configured instruction entry through the file tree. Confirm both board saves through persisted source and audit evidence. - Require one committed native run on the configured provider, model and environment. Keep the synthetic instruction reference out of the task request. - Restore the original instruction bytes through the same ordinary UI and verify the saved result after reload. - Normally merge actual lifecycle squash8771606fd87f69ed4057779e3dce3a9bca8088e6 once. Place only settings registrations at the documented distinct anchors. Preserve all eight dedicated source/oracle/fixture/docs blobs from qualified898 exactly. ## Verification - Current integrated head: 330 focused checks pass across eight settings, catalog, contract, observation, result/report and operations test files. - Current E2E types pass: `node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations tests/runner-e2e`. - Declaration reuse was bound to all11091 matching package/script/server/UI/root inputs, installed lock and TypeScript, plus all source and target generated-output hashes. No source or module-resolution links were copied. All copied output hashes remain unchanged after local checks. - Maintained product-journey discovery passes and includes both lifecycle and settings. The integrated catalog has22 entries, including the separate fresh-cloud onboarding case. - Static preservation checks pass for seven shared registry ASTs after only declared order changes. The settings dispatch body, required seven checks, one-run bound and all dedicated implementation/oracle blobs remain intact. - Historical staging evidence: [QA workflow41689186453](https://github.com/paperclipai/paperclip-cloud/actions/runs/41689186453) passed all seven settings checks and all five phases at source898ae8ab4e49d2408688088ce82e3316b91cc09c. This is the original qualification receipt. It is not a new run of this integrated head. - This PR must earn its own current-head full CI, fresh5/5 review, resolved threads and terminal passing normal check contexts. Those checks are pending. No prior review or check status was copied. - Full workspace checks and a new live QA/model run were not run locally. The coordinator owns current-source acceptance and daily-plan admission. ## Risks - The journey has one paid provider run and needs a fresh dedicated agent. It rejects a campaign that shares this agent with another paid journey. - Browser saves and restoration must persist before navigation. Wrong, missing, foreign or incomplete evidence fails the independent grader. Failed or active work is preserved for normal owned cleanup. - Registration and hash-input order changed during integration, so the definition/source identity changes. The historical QA receipt remains bound to its original source. - The retained earlier review workspace is still held for file preservation. This PR does not recover, reset, replace or destroy it, and submits no turn to it. - No product runtime, permission, model, shared agent, configuration, feature map, QA pin or TestOps-plan changes are included. ## Model Used - OpenAI GPT-6 (Codex), with reasoning, tool use and code execution. The host does not expose the exact deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub issue/PR references; historical QA workflow is linked as qualification evidence) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 18:55:49 -06:00
DottaandPaperclip	8771606fd8	test(e2e): cover browser task editing and reopening (#18782) ... ## Thinking Path > - Paperclip lets people manage tasks and agent work. > - Daily QA uses reusable Product E2E journeys against a named target. > - Existing task journeys cover creation and assignment. > - They lack saved title and label edits across completion and reopening. > - This pull request adds one browser-only lifecycle journey with independent saved-state checks. > - It also puts shared registry entries on separate lines so independent case PRs can use distinct insertion points. > - The registry formatting preserves every token, value, entry order and case body. ## Linked Issues or Issue Description Refs: #18655. **What existing behavior does this improve?** Product E2E coverage of ordinary task editing and completion. **Current behavior** The reusable catalog covers task creation and assignment. It lacks one journey that edits a task, reloads it, completes it and reopens it while retaining its saved context. Several shared registries put all entries on one line, so independent case additions conflict. **Proposed behavior** Create a task assigned to the signed-in person. Edit its title and create one label through the visible UI. Change its status to In Progress, reload it, mark it Done, reload it, reopen it to Todo and reload it again. Require the exact saved task and label throughout. Preserve its description, assignee, project and priority. Allow no agent run or model usage. Put the existing shared registry entries on separate lines without changing their contents or order. ## What Changed - Add `task-edit-lifecycle` and six required checks for browser creation, saved edits, reload, completion and reopening, preserved context and no agent work. - Add valid, wrong and missing evidence controls. Retain allowlisted observations and bounded GET failure receipts in private evidence. - Register the case, feature associations, catalog expectation, source-hash inputs and browser-only result validation. - Extend target ownership for run-owned labels and one exact predeclared title change. Require compatible scoped cleanup before selecting the case. - Open the real side panel or Properties control when needed. Exclude inert and aria-hidden Labels rows and require visible readback. Document inline title editing and the case prerequisites. - Keep priority editing disabled. Verify that the saved priority remains unchanged. - Keep Cursor test observations outside its existing Stop deadline, preserve the full rejection/Stop overlap matrix and add pre-release no-provider-call controls. Production behavior and deadline values stay unchanged. - Format five shared registry files with one entry per line: the JourneyId and AgentJourney unions, definition filenames, expected catalog IDs and agent run counts. No registry entry moves in this PR. ## Verification - Current head is `d1af3aae3712455c90db937fa7a5141555710506`. Its only change from integrated head d40f28 is the Cursor driver test-fixture ordering repair described below. The prior normal merge consumes actual master `d59791828482f213713ca55000c40e6c9c6c03e5`, including reviewed UI repair #18848 and the existing idle-history fixture repair from #18853. It has both real parents, no conflicts and the required trailer. All seven lifecycle case/oracle/catalog/documentation blobs and six selected registry constructs remain exact 10625b. The idle-history fixture is byte-identical to master. The PR diff has 13 paths, including the one Cursor test file, and no feature-map or production-driver changes. - 167 focused tests pass across catalog, target contracts, agent evidence, lifecycle evidence and observation reads. E2E TypeScript and credential-free discovery of21 Product E2E cases pass. The real merged idle-history and exposure-reservation suites pass all 109 tests in132 seconds, including the owned HTTP-process adoption assertion and explicit unknown-listener denial/drift controls. Canonical shared/SDK, database and Runner TypeScript dependencies were rebuilt for the changed master inputs. All4,220 tracked package/script/root inputs and 4,954 dependency outputs are recorded; the outputs remain unchanged after the tests and browser journey. The first database build stopped because child PATH selected a different global pnpm and its version switch could not fetch signed metadata. The preserved attempt made no package or source change. Using the already-installed verified pnpm9 on child PATH passed. No install, Rust or provider-build global setup ran. - One fresh canonical managed-local browser journey passes on exact d40f28: all six checks and authentication, setup, journey, collection and cleanup pass. Cleanup releases the disposable instance. The journey takes 12.059 seconds and has explicit zero-run readback at all five saved states. Its runtime commit isd40f28. Definition hash is `f262707de2e544cecdb9c5c4c5222d61f04a122575b438658dec069e1a93f9cb`. The prior10625b local6-check/5-phase pass remains separately recorded with its own source and definition hash. - A real closed-panel control previously reproduced the original label-interaction failure using only the presentation preference `paperclip:panel-visible=false`. The corrected journey passed all six unchanged checks through an ordinary Show properties click. A separate canonical743 local run passed all six checks and five phases with released cleanup. No domain outcome was supplied by that control. - A separate QA staging qualification passed all six checks and five phases on exact743, with closed cleanup and verified zero agent work. That original pass remains bound to743. It is not regraded or relabeled as a staging run of10625b. Daily admission remains a separate coordinator action. - Earlier material failures remain part of the verification record. The first managed-local setup failed before authentication with an embedded PostgreSQL exit code of null; its original cause remains unresolved. A later loader diagnostic found missing library aliases in this isolated checkout. Existing bundled libraries were hash-verified and the aliases were restored only there. Subsequent real runs exposed an overlong label name, a status picker name mismatch and the closed-panel issue. Each failed result and cleanup record is retained. The fixes preserve the full campaign name, validate the production48-character label limit before mutations, and use the actual visible controls. - Original full CI [41652807327](https://github.com/paperclipai/paperclip/actions/runs/41652807327) failed an unchanged Discord fixture assertion, attention-retention fixture limits and the CLI import/export startup limit. One bounded recheck recovered Discord but repeated the other failures. The reviewed merged retention-fixture repair was consumed normally through #18770. Full CI [41667811850](https://github.com/paperclipai/paperclip/actions/runs/41667811850) passed all E2E shards but exposed inbox/account-fixture failures. This branch consumed the actual reviewed repairs from #18798 and #18799. Original logs and failed assertions remain retained; no timeout or oracle was weakened. - Original743 CI [41682757346](https://github.com/paperclipai/paperclip/actions/runs/41682757346) failed the unchanged CLI suite's30-second child-server startup hook. A bounded local phase probe passed the same startup oracle but did not prove the original CI cause. One coordinator-approved official recheck followed. Timestamp comparison shows46 non-skipped jobs retained their original start and end times; only the failed CLI job and final aggregate have new start and end times. The skipped selection job remains unclassified. New GitHub job IDs alone did not prove new executions. The original startup failure remains preserved and its cause remains unproven. - Formatting-head CI [41726104217](https://github.com/paperclipai/paperclip/actions/runs/41726104217) failed the upstream idle-history process-adoption test and dependent verify job:46 jobs passed, one was skipped and two failed. Its actual synthetic checkout had parents masterf187 and10625b; the failed test is absent from 10625b itself. The assertion expected one adopted process and received zero. A pure production-function control proves a conditional reserved-port drift path, but the actual failed CI port and full reconciliation counters were not logged. The actual CI cause remains unproven. This branch consumes the already-reviewed bounded fixture-port repair in #18853 through its normal actual-master merge. The original failure is retained. No unchanged-code retry, timeout increase, skipped assertion or ownership-policy change was used. - Integrated-head CI [41730231620](https://github.com/paperclipai/paperclip/actions/runs/41730231620) passed the formerly failing server1/12 lane, Typecheck, Build and E2E. It failed the Cursor package fixture below and the dependent verify job. A separate server4/12 job lost runner communication; the exact runner instance and Spot-request readback confirm a capacity interruption. Its unavailable log is not treated as a test pass. The original failures remain retained. No old-head retry was made. - The Cursor fixture sets a 300 ms cancellation budget, holds a rejected-create checkpoint and originally performed directory/file observations while that clock ran. Controlled400ms observation latency on the original fixture reproduces the same fenced Stop error. The same diagnostic passes after moving file observations before the cancellation clock, or while the initial Stop-checkpoint gate still prevents it from starting. The driver,300ms configuration and timeout negatives are unchanged. Every original assertion AST remains, and three new assertions forbid provider observation/cancel calls before releasing the checkpoint. All81 driver tests,12 client tests and43 checkpoint-archive tests pass; Runner TypeScript build and E2E TypeScript pass. The actual CI inner error and observation duration were not logged, so its precise original cause remains unproven. - Final d1af3a full CI [41733938939](https://github.com/paperclipai/paperclip/actions/runs/41733938939) attempt3 is successful. Attempt1 lost four exact-bound Spot runners; attempt2 recovered the Runner and both server lanes but lost the replacement E2E runner to another confirmed Spot capacity interruption. Two separate coordinator-authorized official failed-job rechecks were recorded. Attempt3 passed the E2E path and final aggregate. Successful executions retain their original timestamps; new job IDs alone were not used as proof of re-execution. All original failures remain retained. - Fresh Greptile check126258142754 completed successfully on exact d1af3a with 5/5 and zero unresolved threads. Full recursive workspace typecheck, all workspace tests and the full build were not repeated locally; the corresponding hosted full CI gates pass. ## Risks The case requires target support for `label` ownership and scoped `taskEdit.revisedTitle` cleanup. Older target adapters must not select it. Cleanup must reconcile lost creation responses and exact predeclared title saves without touching other resources. Registry formatting changes the harness definition hash because that hash includes source bytes. Historical runs must keep their original source and hash. The formatting does not change runtime values, order, dispatch, case checks or product behavior. Later case PRs must verify their actual merge resolutions and final-head checks before admission. ## Model Used OpenAI GPT-6 through Codex with reasoning, tool use and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 17:47:13 -06:00
3 people	a497bae83a	test(e2e): verify Cloud workspace access isolation (#18655) ... ## Thinking Path > - Paperclip keeps agent work inside company boundaries. > - Cloud users switch workspaces through the tenant account menu. > - A company-bound credential must work inside its permitted company. > - A missing resource cannot prove an access denial. > - This journey uses real controls in two registered test workspaces. > - It checks reads and task creation in both directions and revokes both keys. ## Linked Issues or Issue Description Refs: #18728, #18729, #18652. **What existing behavior does this improve?** Product E2E coverage for Cloud workspace selection and company-bound agent API access. **Current behavior** The journey catalog lacks a two-workspace boundary check with real allowed and denied controls. **Proposed behavior** Use the visible Cloud menu to switch workspaces and return. Read a real task and create one task in each credential's own company. Require real foreign reads and the same task-creation operation to return 401 or 403 in both directions. Verify that no foreign task was created and that protected content was not returned. Revoke both credentials and require their earlier allowed reads to return 401. ## What Changed - Add a browser journey and a narrow target-provider fixture contract. - Require independent owner reads, exact agent authorship, zero-to-one task creation, stable run history, no unexpected work and no foreign creation. - Validate observed workspace entry links against the exact registered targets. - Add wrong, missing, leaked, foreign and invalid-entry evidence controls. Saved passes reject collected runs or nonempty model usage. - Register all seven required checks, definition-hash inputs, feature associations. The feature-map documents are unchanged, as requested. - Base the final tree on verified repository master `7b8aa60b6d79b6cd9e4d46ca1369515c33e2d5d4` plus only 15 access journey and catalog files. Preserve all other master files and modes. The held completion journey is outside this PR's diff. ## Verification - 851 catalog, journey, completion-helper, artifact and browser-action support tests passed across 40 files. - E2E TypeScript, credential-free discovery, feature-map validation, full workspace recursive typecheck, full build and `git diff --check` passed. - The broad local `pnpm test:run` attempt stopped in chat integration shard 1 before any assertion. Embedded Postgres initialization exited with code null. No chat tests or later unit shards ran. The startup cause is unresolved, the failure is retained, and the command was not retried. The database/chat/test-runner source is unchanged from the fixed master snapshot. The separate hosted full CI result is recorded below. - The first local E2E typecheck read stale compiled plugin SDK declarations after the merge. The SDK source already declared the field. Rebuilding its declarations cleared the diagnostic without a source edit. - The complete tree comparison proves exactly 17 changed paths before the documentation removal against the fixed master snapshot. Every other master file and mode is preserved. All four access flow, contract and test blobs match the previously passing source `fab9a1fb2fefb41a7248e4fc1aa239d77455f5e2` byte for byte. - The historical staging attempt on source `fab9a1fb2fefb41a7248e4fc1aa239d77455f5e2`, the earlier Core42 deployment and the earlier adapter passed all seven checks: owner resource controls, visible Cloud switch and return, scoped data, both restricted-agent positive controls, cross-workspace read/task-creation denials, credential revocation and no agent work. Both credentials were revoked, owned credential agents had terminal receipts, and evidence cleanup was complete. No agent or model work was recorded. - The earlier attempt failed before journey checks while reading the borrowed fixture agent's historical details. That failed result remains preserved. The provider then supplied a separately owned primary identity. The public history assertions remain unchanged. - The strict root catalog expectation first failed because it omitted the new access case. Adding only that expected entry made the test pass. The ordered list and default-selection assertions remain unchanged. - The first combined-head full CI run failed two unchanged attention-retention tests at their original 15-second limits. The catalog fix does not change those tests or their timing. The initial failures are retained, their cause is unresolved, and that failed run was not retried. - Normal full CI passed on the new commit in [run 41595755161](https://github.com/paperclipai/paperclip/actions/runs/41595755161), attempt 1: 48 successful jobs and one intended skip. The two unchanged attention-retention cases passed at about 12 seconds; the earlier timeout cause is still unknown. The fresh review on commit `4c0ea682444738ad6fa17017a6afd25a49037559` is 5/5 with no open findings. The fresh staging result is recorded below. This PR changes no daily plan, private configuration, model or live workspace. - Fresh staging qualification on commit `4c0ea682444738ad6fa17017a6afd25a49037559` passed all seven checks and all five phases. Both ordinary same-workspace task creations returned 201. Both cross-workspace task creations returned 401. Both credential agents reached terminal state. Both keys were revoked, and both revoked-key reads returned 401. The manifest records complete cleanup and no active operations. No model or task run was recorded. All three workflow jobs passed. - The coordinator verified TestOps case 719 and result 3761 as passed, then closed launch 1987. Its report shows model none and the current qualification run. The automatic Slack report recorded one pass and zero failures. - The original normalized result was passed. The first evidence collector left its own qualification false because its allowlist omitted the standard `dashboard.html` artifact. The adapter already declared that file, and the public launcher generated it. An exact-object read matched the 118,791-byte object, ETag and SHA256 archive receipt. This completed the three-artifact inventory. The original result, collector proof and seal remain unchanged. No HTML was executed or rendered, and no test was rerun or regraded. | Fresh staging provenance | Verified value | | --- | --- | | Public commit | `4c0ea682444738ad6fa17017a6afd25a49037559` | | Qualification workflow | `41597774599`, attempt 1 | | Serving Core commit | `09162fe893b464dd2ccdafd14bd3ef1611202a86` | | Target adapter commit | `7367cf6fd0ee1cf52c2651a8a6671e087728a9c1` | | Actual workflow bootstrap commit | `55351ffba2704a4e13c1ead3e50665acd51aef87` | | Exact 117-file QA contract SHA256 | `14d95c29059c82563ab0e1d65a01aa4ac6c4f60f77a236803cf06ad5d9e7d27b` | | Provider-pack configuration version | `9` | - At the user’s request, follow-up commit `7e0dbc4bdba8a25ff6486648b0ca0d0db2616a4c` removes both feature-map additions. The final diff contains 15 paths and no `maps/feature-map/` changes. An entire-tree comparison against the QA-passed commit proves these two Markdown files are the only changes; every executable file, test, fixture, registration and file mode is unchanged. `node scripts/check-maps.mjs` and `git diff --check` pass after the removal. The live QA run above was not repeated for this documentation-only change. Fresh current-head full CI passed in [run 41633385961](https://github.com/paperclipai/paperclip/actions/runs/41633385961), attempt 1. The current-head Greptile review is 5/5 with zero unresolved threads. The normal merge preserves every newer master file outside the 15 PR paths and keeps all 15 PR files byte-for-byte intact. ## Risks The private target provider must register two exact test workspaces and preserve their separate ownership and cleanup. The case verifies company-bound agent credentials. It does not certify human membership roles. It tests task creation because ordinary agent task edits require a real execution context. No execution context is forged and no product authorization guard changes. The local broad unit run could not start embedded Postgres, so it does not establish a full local unit pass. The earlier hosted attention-retention timeouts also remain unexplained. Passing a later run would not prove the source of those timeouts. The four case files retain their historical passing bytes. The fresh qualification verifies the executable test code retained in this PR against the deployed staging stack. Daily admission remains a coordinator action after the normal source merge. The case does not cover future deployments without another run. ## Model Used OpenAI GPT-6 through Codex with reasoning and code execution. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com>	2026-11-03 06:24:08 -06:00
DottaandPaperclip	7b8aa60b6d	test(e2e): add repository continuation journey (#18729) ... ## Thinking Path > - Paperclip keeps agent work in project workspaces. > - A later user request must continue from the saved repository state. > - A successful reply alone cannot prove file or Git history preservation. > - The repository journey in #18652 passed against QA staging with independent byte and commit checks. > - Its separate completion journey remains unpassed and must stay open. > - This PR delivers only repository continuation and reuses the merged shared support from #18728. ## Linked Issues or Issue Description Refs: #18652. Builds on merged #18728. This is the repository-only delivery from #18652. The original completion implementation and all failed attempts remain on that open PR. ## What Changed - Add `repository-continuation` with the exact measured source fixture, two native turns, and eight required checks. - Start from measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189`. Keep the repository flow and Git/byte oracle unchanged. Clarify the two file-content instructions to require the word first or verified followed by exactly one LF newline; retain the exact expected bytes and the original random-reference contract. - Document the two-turn proof boundary: final saved hidden Git metadata is not independently read. Keep all eight success assertions. - Use the original-deadline and routine failure-reporting repairs from merged #18728. Integrate its actual master squash `e5713d95c6d9a258882649789ea5eb071cc17dc0` through one normal merge. - Use a fixed public read-only repository commit. Verify downloaded Git objects, parent commits, changed paths, saved workspace identity, and exact source bytes across both turns. Reopen the registered outputs through the browser. - Keep the shared artifact reader behavior unchanged and narrow its type to the API capabilities it uses. - Add only this case to the catalog, feature association, dispatcher, source hash, types, run/check contract, strict catalog expectation, and journey documentation. ## Verification - Current head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed fresh [QA staging run 41589522430](https://github.com/paperclipai/paperclip-cloud/actions/runs/41589522430) and its [full report](https://pages.paperclip.ing/evals-qa-smoke-41589522430-1/). All eight checks and five phases passed. Both native turns succeeded and committed in the same workspace. The independent same-buffer Git audit verified that the second commit has the first commit as its parent. The saved original files and browser downloads matched their expected bytes. This does not add a final hidden `.git`/HEAD assertion. - The fresh run used OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on QA staging/Daytona. It used Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`, Cloud `cloud-41570439002-1`, and environment configuration version 9. Actual checkouts matched public source `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31`, adapter `7367cf6fd0ee1cf52c2651a8a6671e087728a9c1`, and protected workflow bootstrap `a9ad6877de62fe3f65aacadc8a9dfba7d90729cf`. The bootstrap matched the accepted 117-file QA contract. - Normal cleanup completed. The original manifest is complete, evidence cleanup is complete, no operations remain active, and the owned agent is terminal. The coordinator verified ACTIVE TestOps case 685 / result 3695 and closed [launch 1921](https://paperclip.testops.cloud/launch/1921). This records normal cleanup closure; no separate physical provider-stop acknowledgement was collected. - Measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189` passed [QA staging run 41573331189](https://pages.paperclip.ing/evals-qa-smoke-41573331189-1/) against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch 1723 / result 3501 maps to case 685. All eight checks and five phases passed. The two run bindings, independent saved-byte/Git audit, and completed cleanup were verified. - Initial extraction `54103639850d08d4d125ebe0493d8d3d49b58211` preserved the measured case files. The current head also clarifies the two LF instructions, documents the final Git-metadata limit, and inherits the reviewed helper repairs. This changed the fixture conditions. The current head now has its own live pass, listed below. The original ac4 result remains a pass under its original conditions; it was not used to qualify this head. - The emitted-prompt control failed on the old quoted backslash+n instructions, while 14 sibling controls passed. All 47 focused repository/artifact controls then passed after the two-sentence repair. Local: 891 related support tests, runner E2E typecheck, recursive typecheck, build, maps, token gates, and `git diff --check` passed. The inherited routine source has 62 causal read/cleanup controls and 26 unchanged real component controls. - The initial repository head's normal full CI run 41578686494 passed. Its 4/5 review and all original findings remain recorded. The later stacked head `037b979477161881a4877230500e6f3de8faea5c` also passed full CI run 41581342503 and a fresh 5/5 review. Current integration head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed normal full [CI run 41584981651](https://github.com/paperclipai/paperclip/actions/runs/41584981651), attempt 1. Its fresh exact-head Greptile review is 5/5, with zero unresolved threads and no open findings. The broad local suite was not repeated. The earlier full local run recorded in #18643 had four failures and did not reach 164 serialized groups; this PR does not claim an all-local pass. - Earlier repository failures remain recorded: provider-pack transfer timed out in one attempt; a later attempt could not read saved workspace files. The later pass does not regrade those failures. The product repairs are separate from this unchanged case delivery. - The fixed actual #18728 squash is now integrated. The PR targets master. Its aggregate diff has exactly 14 repository-only paths, each identical to `037b979477161881a4877230500e6f3de8faea5c`; all other master changes are preserved. No later master head was chased. The current CI, review, and fresh staging qualification all passed for this integration head. ## Risks - Git bundles and workspace bytes are private evidence. Public reports retain closed checks; the fixture never pushes to the public source repository. - The second committed output must retain the first commit as its parent, the same workspace binding, and original saved file bytes. Final hidden `.git`/HEAD after the second save is not independently exposed by the supported remote workspace API; this test does not claim third-turn Git metadata durability. - Only JSON metadata observations retry transient transport failures. Binary attachment and browser downloads remain single attempts. Transport failures and digest mismatches remain visible failures. - The existing artifact reader is shared. Its runtime code is unchanged and its regression coverage remains included. - Daily source pins and plan selection stay with the coordinator. This PR does not register or bypass the unpassed completion case. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment ID and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 00:46:13 -06:00
DottaandPaperclip	e5713d95c6	test(e2e): add scheduled routine journey (#18728) ... ## Thinking Path > - Paperclip manages agent work through tasks, projects, and routines. > - Daily QA must exercise the staging app through normal browser and API surfaces. > - A schedule must create real work and retain its result and history. > - Pausing that schedule must prevent the next invocation. > - The routine journey in #18652 passed in QA, while its separate completion journey has not passed. > - This PR delivers the routine test independently and preserves its measured flow and assertions. ## Linked Issues or Issue Description Refs: #18652. This is the routine-only delivery from that PR. The original completion implementation and its failed attempts remain on #18652. ## What Changed - Add the target-independent `routine-scheduled-run` journey with one native run and seven required checks. - Start from the measured routine source `ac4efb7c6f4782ea806661683c656fd2b4cc3189`. Keep its request, seven success assertions, one native-run limit, and 65-second pause check. - Bound each observed GET by the original remaining journey deadline. Preserve the normal API timeout for other callers. Keep the original routine error if final trigger disabling also fails, save a closed secondary receipt, and fail an otherwise successful journey when disabling fails. - Narrow the shared output-reader type to its one-argument GET capability without changing its runtime code. - Register only this case in the catalog, types, feature associations, dispatcher, and definition hash. Keep the strict catalog expectation in sync. - Retain the real Routines component controls and the existing project-flow coverage for the shared browser helper. - Document resource cleanup and the 65-second pause observation. Preserve the current routine map content. ## Verification - The original source passed [QA staging run 41567574185](https://pages.paperclip.ing/evals-qa-smoke-41567574185-1/). TestOps launch 1657 / result 3431 maps to case 718. All seven checks and five phases passed. One scheduler-bound native run, saved output attribution, the full pause window, and cleanup were verified. - Current source `8f62137f130f59fd3e39276e3d60185ddff24517` passed [fresh QA staging run 41582706162](https://pages.paperclip.ing/evals-qa-smoke-41582706162-1/) against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch 1855. All seven checks, five phases, and all three workflow jobs passed. The run used one scheduler-bound native turn and observed 65.352 seconds after pause and completion. Canonical evidence collection and normal cleanup completed, with no active operations and the owned agent terminal. - This fresh pass covers the reviewed deadline and failure-reporting changes in the current source. The earlier ac4 pass and the byte-preserving b940 extraction remain historical records under their original conditions; neither failed attempts nor prior source conditions are regraded. - Causal controls: six assertions failed on the old behavior, then all 62 focused controls passed after the repair. Local: 834 related support tests, 41 shared chat/output-reader tests, and 26 unchanged real component controls passed. Runner E2E typecheck, recursive typecheck, build, maps, token gates, and `git diff --check` passed. The temporary output-reader type mismatch exposed by typecheck was corrected by narrowing the capability type. - The first restricted build/typecheck invocations failed because local tsx IPC was denied. A restricted support run also lacked the report CLI output. The same source passed with normal owned local IPC allowed; the original logs remain retained. - The original head's hosted e2e shard 2/8 failed at the unchanged mobile preference frame-count test: expected more than five frames, observed five. The cause is unknown. Its original log remains retained; it is not labeled a proven flake. No CI rerun was requested. - Current-head full CI run 41580423918, attempt 1, passed. The fresh review is 5/5 with zero unresolved threads. The broad local suite was not repeated. The earlier full local run recorded in #18643 had four failures and did not reach 164 serialized groups; this PR does not claim an all-local pass. - Prior failed routine attempts remain recorded. They exposed composer focus, collapsed schedule controls, pending browser events, and Markdown serialization problems. The measured later pass does not regrade those attempts. ## Risks - The shared browser-event helper also serves task creation and project creation. Its regression tests and existing consumers remain included. - The staging pass qualifies the captured source and configuration. The coordinator retains source merging, daily pinning, and plan selection. - The repository test and the unpassed completion test are not registered by this PR. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment ID and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub issue/PR references) --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 23:25:56 -06:00
DottaandPaperclip	82c033ed12	test(e2e): cover uploaded input and personal inbox triage (#18643) ... ## Thinking Path > - Paperclip lets people assign work to agents and manage their tasks. > - Product E2E journeys verify the saved outcome through the browser and normal APIs. > - Uploaded input and personal inbox actions need repeatable coverage. > - An agent answer must match the input file, and an inbox action must retain its saved state. > - This change adds two journeys with independent evidence checks. > - A blocked fixture action must use the normal human-owned unblock contract. > - Registration stays separate from deployed qualification and daily admission. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for file attachments, saved task documents, and personal inbox read and archive actions. **Current behavior** The journey catalog lacks an uploaded-input calculation and a persisted personal inbox triage path. A successful agent run alone cannot prove either outcome. **Proposed behavior** Upload a generated CSV through the task composer. Require an exact saved answer and unchanged input bytes. Read and archive one owned task through the inbox, reload each state, and verify that a new Blocked event brings it back without agent work. ## What Changed - Add a file-input journey with a random reference held only in the uploaded CSV. - Check exact result values, document authorship, native run binding, original input bytes, and persistence after reload. - Add a browser inbox journey with an independently authenticated human author, saved read and archive state, unchanged work status, and activity resurfacing checks. - Reject wrong, missing, duplicate, and foreign evidence in assertion controls. - Register both cases and document their scope and privacy rules. - Supply a current-human unblock owner and action for the archived-task fixture. Test that exact payload through the production route and retain the bare-blocked422 negative control. - Include the fixture constructor in the source definition hash and both new cases in the exact catalog assertion. - Reject collected inbox receipts with unexpected runs, providers, models, or native/legacy execution modes. ## Verification - The frozen source is `cdb6659fb4e4b67e9321bc3fd963cf11e9b8b2f3`. It includes one fixed master integration at `fd0d61b8e329efc89ff541dc2cac44e7c7a37f78`. The single map conflict retains both coverage paragraphs. - The production HTTP route rejects the original bare blocked payload with422 and accepts the exact human-owned fixture payload with200. Neither control wakes an agent. The full related route file passes all200 tests. - The frozen current head passes recursive workspace typecheck, runner typecheck, full build, map checks, and diff checks. Its full related journey/catalog suite passes723 tests, the production route suite passes200, and the separate Node support suite passes347. The retained contract file passes all38 tests. Fresh Greptile review is 5/5 on the frozen current head. Current-head hosted CI passes after one authorized failed-jobs-only recheck. The original Runner close test failed; the unchanged, properly cargo-prepared focused local control passed in2.99s, and the hosted recheck then passed. The cause remains unknown; no Runner code, test timing, or assertion changed. The original hosted diagnostic and the initial invalid local invocation without required fake binaries remain recorded. The ordinary local full-test run is in progress. - The unrestricted local runner support run on the fixed-master integration is not green:5993 tests passed,12 failed,3 skipped. The unchanged failures cover onboarding cleanup expectations, Pi metadata, workflow-security expectations, and an expired test allowance. An untouched743 baseline reproduces the same non-PG failures; their source/test blobs also match pre-feature40b807. This change does not extend an allowance or alter paid admission. The original5983-pass/15-failure run remains recorded. Restoring the local Postgres package's own library aliases resolved its two startup failures. The feature-related catalog omission is corrected. - The uploaded-file case passed its first complete deployed QA attempt: one configured native run, all six independent checks, original evidence, complete cleanup, and no active operations. - The first inbox attempt stopped at the agent comment execution-context guard. That guard stays intact. The fixture uses an approved, independently authenticated QA human. - The next original inbox attempt verified protected-secret retrieval, human attribution, and session closure, then failed its unread prerequisite. The product unread defect was repaired in #18670. - The fresh deployed inbox attempt passed external-human activity, saved read state, archive persistence, and unchanged task status after the product rollout. It then stopped at the fixture's bare blocked PATCH before any resurfacing assertion. The response status was discarded;422 is an explicit contract inference independently reproduced through the production route. That failed attempt remains retained. A fresh full inbox qualification on the current frozen head now passes all six checks and all five phases; the normal human comment receipt, closed human session, zero agent work, complete cleanup, and absence of active operations are verified in the original evidence. Daily admission remains a separate coordinator action. - The retained-result review gap failed a negative receipt control before the repair. All38 retained journey contract tests now pass. The two old unread advisory threads are resolved only after the fresh full deployed inbox pass. - Original failures remain recorded, including the earlier unchanged environment-runtime assertion and unchanged runner/occupied-port startup failures. Closed operation codes and failure screenshots retain the failing step without raw browser errors. Actual retry qualification remains separate from source tests. ## Risks The target must supply the declared external-human fixture. Inbox control selectors may differ across deployed UI revisions. File processing depends on the configured native agent. Exact saved-state and source checks must remain required when those dependencies fail. The change adds test journeys and does not change production behavior or schemas. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (relevant suites; broader baseline failures are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 18:12:16 -06:00
DottaandPaperclip	7e595fd055	test(e2e): verify native hiring, delegation and teammate reuse (#18351) ... ## Thinking Path > - Paperclip manages real work by AI agents. > - Daily QA must verify that work on the staging deployment. > - Classic Agent Chat coverage already exercises hiring and teammate reuse. > - The deployment journey should share that behavior and its ownership oracle. > - Saved documents, exact callback replies and physical cleanup establish success. > - The corrected journey and complete daily suite now pass on QA. ## Linked Issues or Issue Description **Subsystem affected** Product E2E deployment journeys and their shared browser controls. **Problem or motivation** QA must prove that one newly hired teammate completes two different tasks and returns both saved results to the lead conversation. Names, chat claims and successful-run counts alone cannot establish authorship or completion. **Proposed solution** Reuse the classic hiring oracle and production composer. Verify immutable creation audits, initial assignment, current project and ownership, saved document revisions, exact native authors, durable callbacks and their own task links after reload. Bound execution to five lead turns and two child turns. **Alternatives considered** A separate QA implementation would duplicate the classic journey. Name matching or later reassignment would accept broken initial ownership. Links elsewhere in the conversation cannot substitute for either callback's own result link. **Roadmap alignment** This verifies existing features. Related merged repairs: #18389, #18465, #18459, #18463, #18502, #18503, #18504, #18506, #18507 and #18514. Separate dependency #18517 repairs file preservation after lifecycle cancellation and is merged as Core `050365e`, which also contains #18514. This primary PR now differs from that protected mainline in only 16 unchanged H harness/documentation paths; it adds no application production delta. The measured QA deployment serves Core `050365e`, with its matching bundled provider pack and unchanged approved model/ROM. ## What Changed - Add the shared `hire-delegate-reuse` journey, catalog, types, dispatch, launch fingerprints, journey-feature entries and documentation. Hire one teammate, delegate a checklist, then reuse the same teammate for a saved review. Require seven canonical native runs: five lead and two child turns. - Require exact company, lead, chat and native creation receipts. Both tasks must be created for the intended teammate using immutable creation-audit assignee fields. Independently verify current project and ownership. Creation-time project is not recorded in that audit and is not claimed by this assertion. - Verify both current document revisions belong to the exact child runs and preserve the original checklist. - Use ordinary production rich-editor agent/project autocomplete controls. Verify exact reference chips, then send once. Preserve the shared default sender. - Require each durable, run-owned callback reply to contain its own exact completed-task link. Collect links through the actual production Markdown renderer and issue-reference decoration, then apply the existing company-scoped route authority. An unrelated table or code fence cannot hide a rendered link; foreign, wrong-task and code-only links remain failures. - Preserve approved connection, account, grant and responsible-user attribution. Verify the child's independent installation and retain the immutable binding snapshot. Do not copy credentials or broaden grants. - Record ownership before later observations can fail. Retry only typed transient observation GET failures. Preserve authority and cleanup failures. - Retain all 14 daily journeys, the model matrix, deadlines, seven-run cap and 500-cent aggregate allowance split 300/200. ## Verification - **Current composed head:** `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`. It normally merges mainline `1e8e322` and exact #18517 source `adb1d4e`, retaining the intermediate `b2d9eb4` and `6fd1042` ancestry. The only merge conflict was the Playwright support registration; it is resolved to the exact previously qualified union. All 16 H harness/documentation paths are byte-identical to `5c0830f`; all four #18517 paths are byte-identical to `adb1d4e`. Every other tracked path matches actual protected mainline `050365e`. A clean merge preview of `050365e` and this head equals the current head tree, so no additional source commit is needed. GitHub's three-dot display still lists the four already merged helper files because #18517 used a squash merge; those blobs equal protected mainline and add no net application change. No journey, oracle, cap or model change accompanies this composition. A final clean merge preview on intervening mainline `006d5691a6245d5a710e25ba3df6d609754e163a` preserves all unrelated changes and adds exactly the same 16 measured H blobs, with no net App production delta. That later mainline is not represented as the AppCore measured by the QA run. - **Local qualification:** 149/149 focused tests across six suites, including the report-catalog case, and 11/11 real Chromium checks passed on `b2d9eb4`. All H modules, browser inputs and parser/package dependencies are byte-identical at `afa3f55`; those unchanged suites were not repeated. Current-source transitive types, maps, module boundaries and diff checks pass. Official Node24 QA runtime imports pass again using exact public `afa3f55` and private `8671fa9`, with a clean credential-free environment and network disabled. The initial local report-catalog failure is retained: the child `tsx` process was denied its local IPC socket before importing the report or writing an artifact. The same source and selected case pass after correcting only sandbox IPC permission; no source workaround or deselection was used. - **Current canonical CI and reviews pass.** [CI 41275109691](https://github.com/paperclipai/paperclip/actions/runs/41275109691) completed successfully at exact `afa3f55`: all 59 contexts are terminal and clean (55 success, three skipped, one neutral). Fresh exact-head Greptile and official Paperclip reviews both scored 5/5; all 27 review threads are resolved with pagination complete. The official review is static and its check is neutral; it did not execute tests or live QA. A separate exact-head independent composition assessment found no actionable issues at 4/5 confidence and performed no live actions. **Fresh QA qualification passed: all 14 daily journeys, including the corrected H7, passed against deployed Core `050365e`.** Prior check conclusions and review scores retain their actual heads. The previous `5c0830f` completed all 59 contexts cleanly, actual exact-head Greptile and official Paperclip reviews both scored 5/5, and all 27 review threads were resolved. Its separate independent journey-source review at `2b803f08` remained 4/5 with a unit-helper authorship caveat. Intermediate `b2d9eb4` received an actual exact-head Greptile 5/5 with zero open findings, then was updated to compose the valid dependency review fix before live QA. That score and its pending CI are not attributed to `afa3f55`. - Existing causal controls remain unchanged: invalid immutable assignee, either missing callback link, foreign/code/wrong-task links and an earlier handoff fail against the relevant prior source. Renderer controls prove that tables and fenced code elsewhere in a valid callback do not hide its actual task link. Both composer and document-navigation browser registrations remain present. - #18503 merged after its actual QA document case passed four canonical checks and five phases, positively established zero native work, and completed cleanup. Its measured files are unchanged in this composition. #18506 and #18507 merged after their own checks; their causal worker-settlement controls remain intact. - #18514 is merged as `1e8e322`. It reproduces the selected-task metadata-only-project startup failure against deployed `756599e` and passes 19 real-PostgreSQL selected controls after repair. Both direct and assembled baseline controls fail before it. Real source, explicit selection, policy and workspace-isolation guards remain enforced. It is included in measured Core `050365e`; fresh H7 reached and completed both delegated native tasks. - #18517 initial source `6fd1042` passed 320/320 focused and sibling Stop/recovery tests, including 20 preservation controls. Its actual official review was 4/5 and found a valid follow-up: the new live-controller guard should wait for actual lease expiry rather than repeat locked checks every 30 seconds. Both an already-live lease and a real under-lock renewal reproduce that issue against `6fd1042`. Corrected source `adb1d4e` passes all 22 preservation controls and 124 directly affected tests across three suites, exact-head server build/stamp, typecheck and structural checks. It defers before locking and rechecks renewed expiry afterward, preserving the transactional owner guard. The old score/failure and successful owner tests remain attributed to their actual sources. Independent exact-head review of `adb1d4e` found no actionable issues at 4/5 confidence. This is local/source evidence, not physical QA recovery. Its current [CI 41274963476](https://github.com/paperclipai/paperclip/actions/runs/41274963476) completed successfully with all 59 contexts clean; both actual current reviews are 5/5 with zero unresolved threads. It merged normally as `050365e` after those gates. QA now runs that image. The historical allocation is positively stopped and retained; its files-only export remains a separately tracked repair, not a passing-test assertion. - The historical [first H7 pass](https://pages.paperclip.ing/evals-qa-smoke-41226065043-1/) remains attributed to public `5d3f143`, private `dabad62` and App `b7f1c37`: all checks and five phases passed, seven native runs succeeded, both saved documents and exact callbacks were verified, and physical cleanup completed. An optional runtime-metadata receipt was inconclusive and was not grade authority. Material failed attempts remain failed: | Attempt | Actual result and correction | | --- | --- | | [41231853638](https://pages.paperclip.ing/evals-qa-smoke-41231853638-1/) | 13 passed; hiring omitted initial teammate/project assignment and started extra work. The production composer repair retains the ownership negative. Later recovery stopped physical work without changing the failed grade. | | [41238711372](https://pages.paperclip.ing/evals-qa-smoke-41238711372-1/) | All seven native runs succeeded, but the final callback had no durable chat reply. Cleanup passed. #18502 repairs premature completion-outbox retargeting; the fresh measured H7 now verifies both durable, run-owned callback replies and their own task links. | | [41253428446](https://pages.paperclip.ing/evals-qa-smoke-41253428446-1/) | 12 passed; hiring rejected a mentioned metadata-only project before delegation, and document reopening hit an immediate-control-probe race. #18504 and #18503 repair those separate boundaries. Historical missing-task absence remains quarantined; physical termination does not manufacture absence evidence. | | [41266385118](https://pages.paperclip.ing/evals-qa-smoke-41266385118-1/) | On `5c0830f` and App `756599e`, 13 passed, including the exact composed document helper. Hiring created the correctly assigned task, but its child admission failed before runtime selection because the organizational project had no repository source. #18514 repairs that selected-task boundary. The cancelled native lead retained an active prepared-workspace lease; cleanup 41271035231 failed with `owned_work_stop_unconfirmed`. #18517 addresses its file-preservation admission. Launch 1110, original evidence and automatic Slack FAIL remain failed. | | [CI 41257779002](https://github.com/paperclipai/paperclip/actions/runs/41257779002) / [CI 41263476916](https://github.com/paperclipai/paperclip/actions/runs/41263476916) | Existing enrollment and snapshot-retention fixtures observed intermediate I/O before worker settlement. #18506 and #18507 have controlled causal failures, complete-settlement observers and preserved deadlines/negative controls. Runtime scheduling behavior is unchanged. | **Fresh measured acceptance:** [QA run 41285131220](https://github.com/paperclipai/paperclip-cloud/actions/runs/41285131220) and its [permanent report](https://pages.paperclip.ing/evals-qa-smoke-41285131220-1/) passed **14/14**, zero failed or incomplete, with all five phases and complete cleanup for every case. The strict verifier independently matched original grades, current provenance and completed manifests: public harness `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`, private workflow `8671fa93e27501fabbfffa377bec6dea453b5db7`, AppCore `050365e09d88411960fc3bacc1c08c46c0d9b6fa` and matching Cloud image/provider pack. All model, ROM, deadline and run-budget controls remained unchanged. All three workflow jobs completed successfully; the automatic Slack notification reported PASS with links to this run. The passing TestOps launch was closed normally after evidence verification. The hiring case passed all 11 check entries across nine IDs and five phases, with exactly five lead and two child native runs. Both saved documents match their live current revision/body hashes and exact hired-agent run attribution. The immutable original checklist remains intact; the same teammate authored the review. Both own callback replies contain their exact delegated task link. Independent canonical readback confirmed all seven native runs succeeded and committed their accepted result, assessment and decision, with no live owner and zero pending or failed effects; each of the seven exact owned leases has its own bound DESTROYED receipt and successful cleanup. All seven final snapshot operations match their source run, workspace, task, epoch, SHA, generation and export location. The four earlier lead snapshots are retained SUPERSEDED ancestors of the current committed snapshot; the latest lead and both child snapshots are COMMITTED. The initial diagnostic required current COMMITTED status for every historical snapshot and correctly remained false until the separate lineage diagnostic established this supported supersession. The product oracle and original grade were not changed. Canonical Suite metadata is `qa-basics`; the daily plan was saved with its standard suite-only query and synchronized, retaining all 14 ACTIVE cases. The older failed allocation remains separately stopped, retained, isolated from reuse and owned by terminated actors. That permits fresh campaigns without replaying old work. Its same-allocation files export is still unresolved and has separate bounded files-only recovery work; it is not claimed by this successful campaign. Original failed grades, reports and automatic FAIL notifications remain unchanged. No failed result was closed, relabelled, hidden or retried unchanged to obtain green. ## Risks The journey requires Agent Chat, normal hiring permission, a scoped lead and a provider that finalizes owned resources. Missing or extra work, wrong ownership, absent saved documents, missing callbacks and unconfirmed cleanup must fail. The deployed app and bundled runner pack must match the verified Core. The old run's cancellation ACK and terminal database row do not prove physical termination or saved files. Its coordinator has no accepted result or decision; pending-effect absence cannot be inferred from that missing decision. File recovery must preserve the exact allocation and original destroy disposition without a provider turn. Current source checks, reviews, fresh H7 and full-14 aggregate all pass. Canonical Suite metadata and actual 14-case daily-plan synchronization were verified after closing the genuine passing launch. The original failed allocation still has a separate files-export debt; its STOPPED receipt is not a SAVED or DESTROYED claim. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and browser test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge - [x] Historical passes and material failed attempts retain their actual source and outcome - [x] The composed document helper passed normal QA with complete cleanup on its actual source - [x] The corrected hiring journey passes fresh normal H7 with complete cleanup - [x] The active hiring case has canonical Suite and daily-plan membership after the passing launch is closed --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-01 02:33:59 -06:00
DottaandPaperclip	5bc029bf6c	test: wait for native work and accounting before Stop redirect (#18463) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys test the browser, native work and saved outcomes on a selected deployment. > - The Stop journey must interrupt real work before it sends a new direction. > - Its old boundary accepted a turn-start event before any command or usage report. > - A stopped attempt with incomplete usage correctly blocks later work under the existing budget policy. > - This pull request waits for a live native command and bound usage, then checks actual accounting settlement before it sends the redirect. > - The provider reports first usage after its initial command completes, so the fixture finishes a harmless readiness call before the long interruptible hold. > - The benefit is a stronger test of Stop and continuation with the same three-run limit. ## Linked Issues or Issue Description Refs: #18098, #18345, #18354. Related accounting work: #18035. Integrated upstream prerequisites: #18387, #18465 and #18409. Duplicate selector PR #18472 was closed as superseded by actual merged #18409. **What happened?** A QA attempt clicked Stop soon after a native turn started. The owned event stream had no command or usage report. Its cancelled run retained incomplete accounting. The saved redirect received a skipped wake, and no third run started. The budget guard must keep that missing usage visible. **Expected behavior** The journey must first prove that actual native work is active and that its current turn has reported complete usage. The fixture must finish a harmless readiness command before it starts a separate bounded hold. After Stop and physical cleanup, it must verify a complete accounting receipt and saved settlement before sending the redirect. **Steps to reproduce** The fresh staging attempt failed before Stop because its first 90-second command ended before the provider emitted its first complete usage. Its only later eligible commands lasted less than one second. The controlled browser sequence reproduces that ordering without accepting it. Completed readiness usage followed by a separate still-active hold qualifies under the unchanged oracle. Startup alone, command alone, usage alone, partial usage, incomplete accounting and a missing settlement timestamp still hold the next action. ## What Changed - Request a harmless `printf` readiness command first. Wait for its result, then request `sleep 90` exactly once in a separate shell call before file edits. This gives the provider an opportunity to emit current-turn usage before the interruptible hold. - Reuse the strict native-command observer. Require a bound current-turn complete usage report with valid nonzero token counts. Reject stale, partial, foreign and semantic-tool evidence. - Wait for the stopped run's complete accounting receipt, `accountingReceiptReady: true`, `costAccountingPending: false` and a valid saved settlement time before redirecting. A receipt already ready before Stop is intentional only when the exact cancelled run is actually accounted with no pending debt; no source/sequence advance or callback clock order is assumed. - Exclude canonical raw, partial and complete `item.completed` usage only when the API row and PRP envelope match the authoritative owned run, company and agent, and their source event ID, instance ID and positive sequence agree. Missing context keeps the strict work check. A self-consistent accounting source can change after Stop. Keep the original snapshot and reject real tools, turns, messages and other item events, including productive output from a new source. - Retain private boundary and accounting proof. Source and sequence observations do not impose unsupported controller or clock order. Include the new oracle in the suite definition hash. Keep existing success-check IDs, exact three runs, model, budget, deadline, Stop proof and final outcome checks. - Reuse one binding validator in the active-work boundary and post-Stop filter. Pre-Stop keeps its current Runner/current-turn restriction. Add focused evidence tests and a standalone Chromium/control-API sequence check. The optional pre-Stop hook leaves existing classic callers unchanged; they do not call the accounting filter. ## Verification - Current source is `47578138ad357fc11b8b346bfada36d6aa5ab484`, one narrow fixture commit on reviewed `b9668ccfba61c59eb496ba18269aee06e2921d8f`. Only the prompt constant, two ordering controls, the existing Chromium sequence and the journey documentation changed. All oracle functions, run counts, model, budget, deadline and check IDs are byte-identical to b966. There is no new upstream merge or app change. - Fresh staging failure is preserved in the [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41227920892-1/). It measured harness `5d3f143d0aed4dc7cf472b7891ea3d4d95ab843b` against app `b7f1c3769c00a107ffe4128b48b0ef8cc452cf3f`; the reviewed Stop helpers and its flow were unchanged in that harness. The original grade has `stop_boundary_already_settled`, two succeeded native runs, and complete cleanup. Authentication, setup, collection and cleanup passed; the journey failed. This attempt remains failed. - Causal timing evidence: the first shell command started at event108 and completed at111 about 90 seconds later. The first structured complete current-turn usage arrived at113, after the hold ended. Replaying the exact retained event metadata never qualifies that hold. The only later eligible command intervals lasted about 0.90 and 0.18 seconds. The emitter publishes positive usage when the provider sends its assistant-message usage update. The fixture now completes a harmless first call before requesting the long hold; no usage is supplied or invented by setup. - On current source, all 137 focused tests and journey contracts passed. Use `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts product-journeys/stop-redirect-evidence.test.ts product-journeys/agent-evidence.test.ts product-journeys/active-followup.test.ts native-event-envelope.test.ts product-journeys/contracts.test.ts --maxWorkers 1`. New controls reject every prefix when first usage arrives only after the hold completes. They accept the same-turn sequence of completed readiness, genuine complete usage and a separate active hold, and reject it after that hold completes. - Manual Chromium qualification passed: `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/stop-redirect-boundary.browser.mjs`. The browser does not click Stop for the failed ordering or completed readiness alone. It clicks once for a separate active hold with bound usage. It sends one redirect only after actual settlement. Foreign usage, productive output and missing accounting still fail. This isolated control uses controlled responses and does not call a model or the deployed app. - Strict scoped TypeScript, maps, diff and secret checks passed. The official credential-free Node24 QA runtime-import preflight passed without network access. Full local E2E typing previously encountered old borrowed workspace declarations; clean CI typecheck passed on b966. A full local repository build/test was not repeated for this four-file fixture change. Fresh canonical CI must supply the clean full gates. - Historical binding proof remains valid: the exact old d499 filter failed 17 of the final61 Stop tests. The shared validator checks authoritative run/company/agent plus consistent row/envelope source identity. Bound raw/partial/complete accounting is allowed after Stop, including a consistent source change; productive output remains strict. Pre-Stop still requires its current Runner and turn, complete nonzero usage and a running command. - Historical b966 CI `41224054720` passed all59 terminal checks, with exact-head Greptile and Paperclip Review each5/5 and no open findings. These are not approvals of current475. Current475 has fresh exact-head Greptile5/5 and Paperclip Review5/5, with zero unresolved findings and no omitted thread pages. Canonical CI [`41229185200`](https://github.com/paperclipai/paperclip/actions/runs/41229185200) passed on exact475. All59 checks are terminal:55 success,3 expected skips and1 optional neutral review, with zero failed or pending checks. No old job was retried. Fresh normal staging run41229965802 passed. Its [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41229965802-1/) agrees with the independently read final original grade:11 checks, all5 phases, exactly3 native runs and cleanup passed. The final manifest is complete, with cleanup complete and no pending evidence. Smoke, report publication and Slack outcome jobs all succeeded. - The live pass measured harness `6be20bef6d0c5077adc1ae99562ad84e756ab167` against app `b7f1c3769c00a107ffe4128b48b0ef8cc452cf3f`. Its Stop oracle, flow, shared binding validator, unit cases and Chromium control are byte-identical to reviewed475. The shared agent-flow file adds only the hiring import and early hiring dispatch; all remaining bytes match. This records the actual harness and app identities separately. - Canonical run readback confirmed the first run succeeded, the second was cancelled and the third succeeded. The cancelled run had a ready native-producer accounting receipt, no pending accounting and a saved settlement time. After verified file copyback, Core queued final destruction and followed its normal five-minute cleanup backoff. The exact lease naturally reached expired/cleanup-success with a bound destroyed receipt before the third run started. No operator cleanup, source change, timeout extension or workflow retry was used. The earlier failed staging attempt remains failed. - Current475 is ready for the normal reviewed merge and QA admission process. Source stays frozen; merge and configuration changes belong to that process. ## Risks - Some provider profiles may not expose a valid current-turn usage report while the command runs. The stronger boundary fails closed in that case. - Accounting may remain incomplete after Stop. The redirect stays unsent within the original deadline. This change does not waive missing debt, treat it as zero, raise a budget or change provider behavior. - This repairs the test boundary. It does not prove or repair a general provider shutdown-usage issue. A separate local late-usage ordering control is not attributed to this QA incident. ## Model Used - OpenAI Codex, GPT-6 family. The exact deployment variant and context-window size are not exposed in this session. Used code reasoning, repository tools, test execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-31 13:49:02 -05:00
Dotta	3937215a05	Qualify explicit native written references in the basic question journey (#18459) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Basic task journeys must prove that people can answer an agent and receive saved work. > - The question journey asks for a choice, then a typed reference, then a saved note. > - A native choice card can offer a real written-answer control as well as predefined options. > - The old observer rejected that form before a person could type the reference. > - This pull request accepts that explicit native control and keeps exact answer and work evidence. > - The separate text-only and named-tool qualifications keep their stricter contracts. ## Linked Issues or Issue Description Refs #18346 and #18456. The integration inherits the reviewed fixture prerequisite #18387 and Agent Chat recovery #18465 through their actual merged parent. The producer fix in #18456 is separate from this basic journey contract correction. Public duplicate PR and issue searches found no other correction for this form equivalence. **What happened?** The basic question journey requires the reference card to have `answerMode: text`. It rejects a native single-select card even when its canonical form explicitly enables a written answer. This makes the observer test a widget shape in addition to the requested behavior. **Expected behavior** The person must make a real choice and type the complete reference. A native card with an explicit written-answer control can satisfy the second step. Selecting a predefined option cannot satisfy it. The agent must resume the bound work and save both actual answers in its own task document. The document must survive reload. **Steps to reproduce** Use the focused recording and real QuestionForm browser test in this PR. Keep the reference as a single-select native question with one predefined option and `customAnswer.enabled: true`. The previous grader rejects `question-resume.forms`. The previous browser helper rejects `single_select` before it uses the written-answer control. **Paperclip version / commit** Original source base `f2db909393de124caf6a5380a52d4e0e1d621661`. Current integration `a417fd827347ed6f7998774a9de6aee70b76d867` merges exactly `12b0cc630471ee151ec5bc8e41a34f09236d6723` into reviewed question source `faec4ce6d775a34d303d200987e4e84d6dc26caf`. The scheduled failure on `82d15946c5844d3ae41c6ade62fa224053f20a9a` remains a failure. Its saved card had no canonical written-answer capability and would still fail this change. The retained normalized events do not prove whether the original provider flag was omitted or false. **Deployment mode** Attached native task journeys, including QA Cloud. This PR changes the observer and grader. It does not change the app, provider prompt, model, budget, timeouts, cleanup, or runtime deployment. ## What Changed - Accept one provider-bound single-select reference card with explicit canonical custom answers and valid, distinct options. Keep pure-text reference support. - Make the browser select the actual written-answer control and fill the real rich input. Opt in only for the basic question journey. - Keep the exact no-option typed response, immutable card, board actor, native request and continuation, saved agent document, reload, model, environment, checkpoints, and existing one-to-three-run checks. - Reject missing or disabled custom capability, inferred legacy capability, malformed or extra questions, foreign native identities, changed cards, substituted options, and wrong or missing reference text. - Add a real production QuestionForm and TaskChatRichInput Chromium probe. It first selects a predefined option, then proves the helper clears it and submits only the exact custom text. - Document the contract correction and preserve the historical failed attempt and its separate cleanup outcome. Keep necessary-question and named-tool documentation qualification unchanged. ## Verification - Before the observer change, the focused grader regression failed only the new valid-written-reference case; the other 108 cases passed. - Before the browser helper change, real Chromium passed the text control and four rejection controls but failed the valid native written-answer control at the old pure-text assertion. - Initial focused Runner E2E unit selection at `af07f4de`: **256 passed across 9 files**. This includes question/resumption, clarification evidence/flow/collection/deadline, necessary question, documentation, contracts, and report catalog tests. - Initial real Chromium selection at `af07f4de`: **12 passed**. Six new production QuestionForm/control checks and all six existing document-readiness checks pass. No form or rich-input component is mocked. - Historical test-only head `faec4ce6d775a34d303d200987e4e84d6dc26caf` adds a direct canonical-absent legacy `freeText` grader negative and renders the actual legacy presentation in the browser. All four production helper blobs remain identical to the reviewed head. A temporary legacy inference mutant makes the new negative fail. After restoring production source, **179 focused unit tests and all 6 real QuestionForm checks pass**. The six document-readiness specs and their helper are unchanged. - Original [CI run 41205748589](https://github.com/paperclipai/paperclip/actions/runs/41205748589) failed the staged-listen Runner handoff test; that failure is preserved and was not retried. Its Runner source closure was identical to an earlier passing canonical lane. The symptom also recurred on other heads and is tracked independently; no infrastructure attribution or live pass is claimed. - Current integration `a417fd827347ed6f7998774a9de6aee70b76d867` preserves **all 12 question files byte-for-byte** from faec. Every other tracked path is identical to exact merged parent `12b0cc630471ee151ec5bc8e41a34f09236d6723`, which contains actual prerequisite `baf651c8d0122efc93f3ef03e891919eeb666215`. There is no manual conflict resolution or additional production edit. - On that integrated head: **257 focused Vitest tests across 9 files, 206 required Node source/reporter tests, and 12 real Chromium tests pass**. Map/module-boundary checks and catalog discovery pass. The actual `--preview` command passes with a synthetic question target and an intentionally unavailable adapter; it performs no credential read, adapter loading, provisioning or model call. It retains maxRuns3, a 600000ms campaign bound and zero retries. - Exact-head [CI run 41210806950](https://github.com/paperclipai/paperclip/actions/runs/41210806950) passes, including official typecheck, build, all eight browser shards, both Runner Vitest lanes and aggregate verify. All 59 current PR contexts are terminal with no failures. Fresh [Greptile summary](https://github.com/paperclipai/paperclip/pull/18459#issuecomment-6543878408) explicitly reviews `a417fd827347ed6f7998774a9de6aee70b76d867` at 5/5 with no findings; its exact-head check succeeds and all threads are resolved. No fresh deployed journey pass is claimed yet. - Catalog discovery lists the existing question journey. Map checks and module-boundary checks pass. Diff checks pass. - `tsc -p tests/runner-e2e/tsconfig.json` was attempted. The fresh integrated checkout still lacks compiled Runner, plugin SDK, and adapter declarations. The attempted local check reports 344 diagnostic lines from dependency/type closure and zero diagnostics in all 12 question-owned paths. Official exact-head CI establishes the full typecheck/build gates. Full repository build, typecheck, and test suites were not repeated locally for this observer change. - The first broad local report selection lacked the CLI loader and then hit sandbox IPC restrictions. Locked dependency installation and an authorized local IPC run resolved those setup errors; the full selected 256 tests pass. The initial localhost probe also needed the normal local-listen permission. - No fresh QA pass is claimed. The coordinator must qualify the final reviewed runtime and harness together. Prior failed results remain preserved. ## Risks The basic journey accepts one additional form with explicit native provenance. It still rejects the original closed card without a written-answer capability. The separate producer default fix must reach the selected remote provider pack before it can affect a deployed run. Local browser and grader tests do not prove that artifact served or that a model completed the live journey. Existing cleanup and physical-stop obligations remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, browser testing, and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked related work above - [x] I have described the issue in-PR following the bug template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-31 13:16:40 -05:00
DottaandPaperclip	322fa6b65c	fix(ui): restore task document authoring and add persistence journey (#18356) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Task documents keep versioned work beside each conversation. > - The default chat interface could read general documents but hid their editor. > - People also need unfinished edits to survive a tab switch or reload. > - This pull request restores the shared editor in Artifacts. > - A portable browser journey checks exact saved content and revision history. > - The benefit is a usable document flow with independent deployment checks. ## Linked Issues or Issue Description **What happened?** A person using the default task interface could read a general task document but could not create or edit one. The existing editor was available in Classic mode. A pending create could also leave newer text as a new form after its editor unmounted, causing a conflict against the person's own successful creation. **Expected behavior** A person can create and edit general documents through Artifacts. Titles, bodies and revision history persist after reopening and reloading the task. Unfinished text survives tab switches. A successful create makes retained newer text an edit of that confirmed revision. Later writes from another actor still cause a conflict. **Steps to reproduce** 1. Open an ordinary task in the default chat interface, then open Artifacts. 2. Try to create a general document. The previous interface has no authoring control. 3. With authoring restored, submit a new document, keep typing, and switch to Properties before the request finishes. 4. Reopen Artifacts. The previous draft path restores a new form despite the successful create. Related public work: #17598 adds document copy/download and #18157 adds image galleries. The selected actual merged parent is `f1d780838218f975064f38333946805e0ec103a5` from #18427. It includes the stable profile-entry ordering repair and the exact recovery-fixture prerequisite. The resulting PR diff contains 32 document and reporting files. ## What Changed - Reuse `IssueDocumentsSection` for general documents in default-chat Artifacts. Keep Classic authoring in its existing task body. Ordinary tasks, conversations and Coordinator mode share the same editor options and signed-in permission wrapper. - Preserve Plans, file-review documents, attachments, uploads and work products. Hide duplicate general-document cards. Use the same general-document key rules for visible rows, validation and refused writes. - Forward image uploads, mentions, references, author maps and feedback to both editor mounts. Keep empty mention/project values stable while queries load. - Back up unfinished forms within their signed-in account, task and editor scope. Preserve text entered before its key. Record a successful create on its matching retained newer buffer, including after unmount or an early reopen. Preserve newer title/body edits, real remote conflicts and matching cancellation cleanup. - Add the model-free `task-document-persistence` journey. Setup creates only an owned empty backlog task. The browser creates and edits the document; normal APIs independently check company, task, actor, exact content and both revision records. - Reopen the owned task through the explicit Backlog view, then reload it. Require exact UI readback and zero native runs. Add only the Doc registration, definition-hash inputs, report mapping and four closed checks to the shared harness. - Count explicit zero-run browser receipts after private journey metadata is removed for publication. Preserve missing billing evidence, positive usage and native/protocol accounting. - Update browser assertions for the restored inline editor while retaining exact download bytes, revision collisions, paused-state and no-run checks. Update the feature map, usage docs and shared editor stories. ## Verification - Latest focused checks use official Node 24.22.2: 568 journey, catalog, report, billing and attempt-accounting tests across 25 suites pass. All four admitted journeys remain registered. The seven relevant UI suites pass 529 tests, including the task-page container after the parent merge. Maps, token gates, the PR-scope diff check and the owned credential-pattern scan pass. The normal QA import preflight also passes with an empty environment and network access blocked. - Billing regression: before the repair, the explicit empty public receipt counted one run, and two positive-usage receipts were discarded by browser metadata. Those three cases fail before and pass after the repair. Two additional red regressions cover aggregate usage beside an empty run list. That ambiguous shape now keeps an unavailable receipt and a minimum count of one. Missing receipts retain unavailable billing, and native/protocol one-or-more run totals retain their measurements. - The previous fresh CI run failed its unchanged snapshot-only recovery fixture: one global sweep did not call its callback. The original failure remains recorded. An isolated PGlite probe with all canonical migrations exercises the real reconciler: it processes two earlier sources, then reaches the owned third source on the next sweep. It preserves source evidence, files and run IDs. This proves bounded pagination, not full native snapshot repair. - Local native recovery tests could not start PostgreSQL because the host semaphore limit was reached. That run executed zero cases. Canonical Linux CI later passed all 143 fixture cases, including the original snapshot-only case and the new competing-candidate control. The exact fixture repair is now inherited from #18427 and is absent from this PR diff. - The portable oracle rejects 19 stale, missing, foreign or incorrect outcomes. Result validation also rejects model/provider metadata and native runs for this case. - Five real Artifacts-to-Properties unmount regressions fail before the repair and pass after it. They cover both editor modes, reopening before and after completion, later typing, and a foreign revision that must still conflict. The edit-storage failure case also checks that Cancel clears the promoted fallback copy. - UI coverage includes default chat, Classic, conversations, Coordinator, Plan rendering, document-card deduplication, reserved keys, pending-query stability, permissions and rich editor options. - Earlier real browser checks passed five upload cases and two artifact-arrival cases. They retain exact CSV bytes, original/revised content, collision, paused-state and no-run assertions. Current-head normal CI runs the updated browser suites again. - The branch merges only the selected actual parent `f1d780838218f975064f38333946805e0ec103a5`. The merge has no conflicts or manual resolution. The SQL service and recovery fixture match that parent exactly. The own document editor, persistence flow, saved-state oracle and billing repair retain their previously qualified source blobs. Follow-up, clarification, question/resumption, screenshot helpers and the Steering map are byte-identical to the merged parent. Existing project/search/question/follow-up registrations remain present. - Required document source blobs remain exact: Doc UI `22e80663b053363e5f75e1432dd9043dc0e70243`, its unmount tests `16f3f4ed0dc9b11aaff129e06e6cf2bbf4522713`, readiness helper `800a747284db19b8484650b6759e8bd1f356839d`, and budget service `3a15d489f0aa55c22e8f7fe6a6bc6a0e14e5914e`. - Local typing is limited by borrowed declarations from another checkout. They omit current `skill_test` and sandbox creation-operation fields. No dependency declarations or repository config were changed. The authoritative clean CI Typecheck + Release Registry gate passes on exact head `2ffce29827ae170420d740940c1c069c7225a74c`. - Full monorepo typecheck, tests and build were not run locally. Borrowed build declarations from another checkout limit local typing. Authoritative [clean CI](https://github.com/paperclipai/paperclip/actions/runs/41171717008) passes on exact head `82d15946c5844d3ae41c6ade62fa224053f20a9a`: 55 successful checks, 3 skipped and 1 neutral, with zero pending or failed checks. Typecheck, build, Runner, server and browser lanes pass. Greptile gives that same head a fresh 5/5. All 15 review threads are resolved, with no further thread page. The PR is mergeable and clean. - Failure history remains recorded. The earlier recovery failure was repaired with a bounded, owned-source fixture. A later CI worker shut down after 2,375 Runner unit tests passed. Its one authorized narrow retry passed those tests and all five native integration suites. The separate AgentConfig browser failure was reproduced and fixed by the merged #18427 parent. These earlier passes do not replace current-head CI. - Final-source live qualification passed on exact harness and served application `82d15946c5844d3ae41c6ade62fa224053f20a9a`. QA run `41177696858` completed all four closed checks and all five phases with zero native runs. Cleanup passed; the durable manifest is complete with no pending evidence. The coordinator verified the original grade and manifest. - The [public qualification report](https://pages.paperclip.ing/evals-qa-smoke-41177696858-1/) records zero agent runs, empty run IDs and empty run usage. Pricing remains unavailable and billing remains incomplete; the report does not claim measured zero cost. - The earlier harness `36c8d9ddd76d0b7ddb28a9a6c674d9e538f6df88` pass against application `2ffce29827ae170420d740940c1c069c7225a74c` remains historical evidence. It is separate from the final-source measurement above. - The selected parent brings profile-entry ordering, proactive chat, a database migration, Runner updates, and task-page chat/read behavior. Its new task-page container is included in the final application measurement. The coordinator retains the original evidence and owns the primary merge and daily-plan admission. ## Risks Artifacts now mounts the existing shared editor. Its autosave, conflict, revision, annotation and server access contracts remain in place. Tests cover both interface modes, shared scopes, later remote writes and unavailable edit storage. Plans and generated file-review documents keep their existing flows. The PR adds no database schema or API change. The shared billing repair requires explicit captured empty run IDs and an unambiguous usage object containing only an empty run list for public browser-only receipts. Other fields beside an empty list keep conservative unavailable billing. Missing measurements remain unavailable. Contradicting positive usage remains counted. The inherited recovery fixture keeps the production sweep limit and its source, readback and no-agent-work assertions. The live journey must fail when authoring controls or saved outcomes are absent. It must not create the expected document through setup APIs or switch to another interface to pass. Final deployment verification must retain the actual app and harness revisions separately. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-31 01:49:24 -05:00
DottaandPaperclip	1353a232f2	test(e2e): add a portable active follow-up journey (#18345) ... ## Thinking Path > - Paperclip lets people direct agents while their work is active. > - Product E2E checks must prove that saved user input reaches the intended work. > - The existing active chat story depends on a file in a local agent workspace. > - That setup cannot qualify an attached deployment with remote agent execution. > - This change adds one shared active follow-up journey through the normal task UI and API. > - Independent native operation, saved input, final reply, and cleanup checks make the result inspectable across targets. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for a user follow-up sent while a native agent is working. **Current behavior** The classic chat story holds work open with a local file. The shared deployment-targeted harness has no active follow-up journey. A running heartbeat alone cannot prove that work was active when input arrived. **Proposed behavior** Run one bounded command through the real native provider. Observe its active execution before and after one follow-up is saved. Require one final reply with the late requirement, correct task and agent identity, an empty input queue, and visible readback after reload. Use the same story on managed and attached targets. ## What Changed - Add a shared `followup-while-running` journey with normal task creation and reply submission. - Require a bound native turn and pending shell execution at the input boundary. Retain a closed receipt with source identity and a row hash. - Accept one steered work run or one normal successor (1–2 native runs). Preserve every other journey's declared run contract. - Reuse the saved native final reply observer, run binding checks, usage collection, target lifecycle and private evidence path. - Accept canonical builtin/process execution events independent of provider tool display names; calibrate against real OpenCode, Codex and ACP mappers. Reject Paperclip semantic tool names across native wrappers/prefixes and unnamed or ambiguous builtin calls. - Recognize only the observed legacy PRP v1 builtin/bash/unknown/readOnly=false command shape and fail promptly when owned work finishes before a live boundary can be observed. - Ready the task page/composer before the observed command boundary, use a bounded 90-second command, and preserve active-after-persistence proof. - Add adversarial oracle and retained-result calibration. Register the journey, include its sources in the definition hash, and document its coverage. - Keep six real browser document-readiness checks and verify shared-deadline behavior with a deterministic clock/state test that rejects timeout-reset mutations. ## Verification Exact current head `e72e20ad111bd94728d38606015d1fdac3c92929` passed [full CI run 41142552195](https://github.com/paperclipai/paperclip/actions/runs/41142552195): all 49 workflow jobs reached accepted terminal results, including typecheck, build, every test lane and aggregate verify. All 59 reported checks are terminal: 55 success, three expected skips and one neutral 5/5 Paperclip review. Snyk passed. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/18345#issuecomment-6528913235) explicitly names this exact head and gives 5/5. No review threads remain open. The PR is mergeable. These source gates do not establish a live QA pass at this head. Fresh protected QA at this exact source and the separately recorded deployed application/runtime remains required before primary merge and daily admission. Historical live QA staging **PASS** on qualified commit `af9e091b15b5e24b6b2cc1b424523430cc2ab024`: 6/6 checks, all five phases including cleanup, two native work runs, and the configured model/environment check passed. The [sanitized qualification report](https://pages.paperclip.ing/evals-qa-smoke-41122620146-1/) records that exact measurement. Final integration head `e72e20ad111bd94728d38606015d1fdac3c92929` merges exact actual squash `18b1637b6371a6a6adade857e2a73654ee626fef` from [#18254](https://github.com/paperclipai/paperclip/pull/18254) into previous head `e54d64743964b1a19d165537bebbe62c50389a23`. The merge had no conflicts. All 18 files in this PR's change set, the complete 45-file shared journey directory, and six shared native/evidence helpers retain their exact previous-head Git blobs: 59 scoped files in total. The production clarification helper remains blob `800a747284db19b8484650b6759e8bd1f356839d`. Follow-up 1–2 runs, question 1–3 runs, older exact counts, owned native event checks, active-before/after persistence proof, cleanup/accounting and feature mappings remain unchanged. Source integration head `e55d74491f5bcd99c1455e9ace02bd7ad7319fae` integrates exact actual squash `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` from [#18385](https://github.com/paperclipai/paperclip/pull/18385). The merge had no conflicts. All 17 files in this PR's change set and six shared native/evidence helpers retain their exact Git blobs from previous head `162e341cda5b627639b486b4019a1675bb018ea2`. This preserves the follow-up and question readiness checks, feature mappings, six closed follow-up check IDs, immutable before/after active-operation proof, follow-up 1–2 runs, question 1–3 runs and older exact counts. Previously reviewed head `e54d64743964b1a19d165537bebbe62c50389a23` adds a separate test-only reliability repair on top of that source integration. It replaces one scheduler-sensitive browser case with a deterministic Vitest clock and locator-state model that calls the actual production helper. The six other real Chromium checks remain byte-for-byte unchanged and pass **6/6**. Focused unit suites pass **51/51**, including the deterministic deadline case. This is six browser checks plus focused unit coverage, not seven browser checks. The model makes the panel ready at 100 ms, the supplied answer ready at 600 ms, and full valid content ready at 1,300 ms. It requires rejection at the shared 1,000 ms deadline and records immutable rejection state before simulating a 10-second scheduler stall. Both restarted-timeout mutations fail: resetting all waits wrongly resolves at 1,300 ms; resetting matcher timeouts while keeping the final real deadline guard rejects only at 1,300 ms with valid content present. The genuine helper rejects at 1,000 ms with the answer still incomplete. The production helper is unchanged, Git blob `800a747284db19b8484650b6759e8bd1f356839d`. That two-file test-only patch changed no journey, application, run, accounting, model or runtime behavior. Its six-browser/51-unit results are prior-head evidence; the complete patch is unchanged in the new integration. The final source also retains the actual merged question-primary squash `d9354bbfc35ef74333e90e596ce21dc8e59149b9` from [#18346](https://github.com/paperclipai/paperclip/pull/18346). Later commits `409117451da260a03fc28a3107c7808ecbe68f8f` and `ea3e7b1d1bed562351fcf4722c7175001b987b59` bound question document readiness to one campaign deadline and test that waits cannot restart it. Commit `162e341cda5b627639b486b4019a1675bb018ea2` adds question/follow-up report feature mappings and coverage. These files are unchanged by the final merge. The imported upstream source has real application changes: [#18334](https://github.com/paperclipai/paperclip/pull/18334) makes managed skill content durable and extends native skill-test contracts; [#18350](https://github.com/paperclipai/paperclip/pull/18350) requires managed Cloud shared storage and bootstrap prerequisites. #18385 updates TestOps job metadata and repairs their combined maintenance fixture. The new merge additionally imports [#18383](https://github.com/paperclipai/paperclip/pull/18383) at `81ecaf0654fe79a2a2dbda6588f033e39ee610e4`, which bounds failed lifecycle verification; [#18357](https://github.com/paperclipai/paperclip/pull/18357) at `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`, which persists sandbox-creation ownership and updates cleanup/adapter/plugin contracts; and #18254 at `18b1637b6371a6a6adade857e2a73654ee626fef`, which accepts a valid late Antigravity authorization URL while native authentication drains. This PR's diff against exact upstream `18b1637b6371a6a6adade857e2a73654ee626fef` contains only journey tests, report coverage and documentation. Harness source and deployed application/runtime identity are separate inputs. This source integration does not deploy the imported application changes or claim the QA target runs this application revision. This final SHA has not been measured by the historical run above: **fresh QA at this exact final head is required before primary merge**, followed by normal TestOps verification/admission. Fresh exact-head CI and review have passed as recorded above. The live measurement must record the actual separately deployed QA application/runtime identity. Seven whole-file blobs remain identical to the qualified SHA. All paths are relative to `tests/runner-e2e/`: | Journey/helper | Unchanged Git blob | | --- | --- | | `product-journeys/active-followup-evidence.ts` | `83a5d9fab52abb9849a40e83eca6b6a9c29bf587` | | `product-journeys/active-followup.test.ts` | `728160c610c18292d2cf06d4a87d25a8719f1734` | | `native-task-reply.ts` | `59d11f1a178dad9db3da8fa2342ff93643cb9922` | | `native-event-envelope.ts` | `305b0b456c59c511ee328e529ad574c0583075a0` | | `run-observations.ts` | `08c8ad184a821a7cd536926b3d7e3d1156a1e3da` | | `user-actions.ts` | `e9611348dc41e17899367dab2e3b36d3d1be9480` | | `stop-redirect-flow.ts` | `4580bd605a425bc72511e02d988492acbec8d60e` | The follow-up preservation comparison against historical `af9` has three explicit exceptions; no whole-file identity is claimed for them. The separately documented question readiness and report additions above also require final-source verification: 1. `product-journeys/agent-flows.ts` adds `question-answer-resume` to the `AgentJourney` type exclusion, alongside follow-up. Its remaining contents and runtime helper bodies are byte-for-byte unchanged. Qualified blob `a0afaa968c1d854476159fef21de7b6dd50a4a98`; final blob `0cc797c0ae594e7769338e63a80c4f67e29e4a54`. 2. The common run-count helper is centralized in `success-contract.ts` and re-exported by the catalog. It accepts follow-up **1–2**, clarification **1–3**, and each older paid journey's exact count. The follow-up catalog entry, maximum of two runs, six required checks, native dispatch, collection and retained-run contract remain byte-for-byte unchanged. The helper's text differs; its preserved count semantics are verified explicitly. 3. `product-journeys/active-followup-flow.ts` changes exactly one composer readiness assertion from Playwright's implicit five-second default to an explicit 30-second timeout, matching `submitTaskReply` for attached targets. Prompts, actions, native evidence, before/after persistence proof and grading otherwise compare byte-for-byte unchanged. Qualified blob `64392329198a1a4d9828e32834d4507e175216e6`; final blob `2746b5ba89e2af26b4097ba7897913186786ac5d`. This correction is why a fresh final-head QA run is required. - **310 focused tests passed at current final head `e72e20ad111bd94728d38606015d1fdac3c92929`**: 286 journey/evidence/catalog/report/shared-deadline tests in 14 files and 24 native authentication/real-child stream-order tests in two files. Both invocations used Node24 and one Vitest worker. The earlier `e55` integration also passed 365 focused tests, including 40 Cloud configuration and 40 TestOps metadata cases; those results remain prior-source evidence. - **157 explicit count assertions passed again at the current final head**, including both interactive ranges, older exact counts, invalid/fractional/non-number values and catalog/contract export identity. The maximum bounds and six closed follow-up check IDs also matched. - Maps and diff checks passed. The official QA runtime-import preflight passed on Node24 at the current final head without network access. It uses a dummy adapter and no credentials. It does not prove target connectivity, sign-in, model readiness or deployed application/runtime identity. Those require the normal protected QA run. No model/provider work or optional Runner release was dispatched. - Current local runtime dependencies use ignored isolated workspace-source views. No shared dependencies or compilers were changed. A previous local typecheck attempt hit stale reused Runner declarations. No full repository typecheck, test run or build was repeated locally; the successful exact-head full CI run above now supplies these gates. - CI on previous head `162e341cda5b627639b486b4019a1675bb018ea2` failed the Cloud/local maintenance fixture in [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133716802/job/124331643100), plus dependent verify. The actual tested merge combined that head with master `198218bf554877fe3b7a83742eefb3b8f12392c6`. A Cloud child selected local-disk storage and hit the new shared-storage policy. This was a deterministic upstream fixture mismatch; no unchanged failed-job retry was used. - The actual #18385 repair configures a valid Cloud child, verifies local bytes independently, checks the expected provider mismatch and retains exact stored-config readback. Its own final head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57` passed 59 checks, including [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133876636/job/124331450635). That prior result supports the repair; it does not replace this final head's fresh CI. Separate repair candidate #18392 and its source/evidence are retained as superseded history. - A local attempt to run the database fixture could not start PostgreSQL because the host exhausted macOS semaphores. A first setup attempt also failed package resolution before test execution. Skipped database cases and these setup failures are not passes. No kernel settings, shared processes or databases were changed. - The complete previous-head CI run [41138618869](https://github.com/paperclipai/paperclip/actions/runs/41138618869) finished with one underlying failure: [Runner Vitest shard 1](https://github.com/paperclipai/paperclip/actions/runs/41138618869/job/124347859445) expected native Antigravity login success but observed failure at `antigravity-auth.test.ts:101`. The lane had one failed test, 1,994 passing tests and eight skipped tests. Aggregate verify failed only because of this lane. All other 57 checks were accepted, including build, typecheck, all server/browser suites, Runner shard 2, Snyk and exact-head 5/5 reviews. No unchanged retry was requested. - The independently reviewed #18254 repair removes an invalid cross-pipe ordering assumption: a valid URL on stderr can arrive after the authenticate response on stdout during drain. Its real-child regression forces both orders. The current final source imports that actual squash; the 24 focused authentication cases pass here. Fresh full CI and exact-head review have now passed for this source, as recorded above. The earlier failed result is not relabeled. - The original live compatibility failure is retained: an older PRP v1 pack labeled builtin `bash` as operation `unknown`, causing the observer to miss the command and never send the follow-up. Its exact shape was reproduced red before repair. Generic unknown operations, semantic tools, wrappers, read-only work and foreign provenance remain rejected. The historical qualified pass used the repaired oracle; no failed result was relabeled. - Earlier CI on `6fad965d69` hit an unchanged heartbeat test's three-second adapter-call wait; its original failure and one narrow rerun remain recorded. The qualified SHA subsequently passed CI. An earlier report-catalog subprocess failure is retained. The final focused Node24 run now includes that suite and passes after the documented report feature mappings. ## Risks The fixture asks the native agent to run one bounded `sleep 90` command. The observer must see actual active native execution; a fast or noncompliant turn fails rather than passing from elapsed time. A completed result requires one or two bounded native work runs, source attribution, configured model/environment, and collection. Unknown provider prices stay unknown. This change does not alter production execution or grant new access. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with available version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs; existing follow-up execution fixes do not provide this deployment-targeted test - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the new focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 18:37:17 -05:00
DottaandPaperclip	d9354bbfc3	Add a portable agent question and answer resumption journey (#18346) ... ## Thinking Path > - Paperclip helps people manage agent work and keep human decisions attached to tasks. > - Product E2E journeys verify the browser, native agent, saved state, and cleanup. > - The classic suite already checks a choice question followed by a text question. > - Deployment-targeted journeys need the same behavior on an attached instance. > - Production questions can use provider input, the dedicated semantic tool, or the authenticated native API fallback. > - The basic journey must verify the original card and native continuation for each supported path. > - This pull request shares the scenario and browser actions, adds strict API receipt evidence, and verifies the saved document after reload. > - Separate tool-documentation qualification and the original failed measurements keep their existing scope and outcomes. ## Linked Issues or Issue Description **What existing behavior does this improve?** It adds portable coverage for answering agent questions and resuming the same task on an attached deployment. **Current behavior** The classic suite has the two-question scenario. The portable catalog does not expose it. Its shared native-path grader recognizes provider and dedicated semantic receipts, but rejects a valid completed native API creation receipt. **Proposed behavior** Expose `question-answer-resume` as a deployment-targeted journey. Answer one choice question and one text question in the browser. Require server-owned creation evidence, unchanged forms, exact board answers, bound continuation, an agent-authored task document, and readback after reload. **Reason and benefit** The same story can qualify local and hosted targets through their normal provider setup and cleanup. It catches lost answers, duplicate or foreign work, premature output, and stale documents. ## What Changed - Register the journey with a three-run maximum and positive model allowance. Accept one to three attributed work runs at both collection and retained-result boundaries. - Share the existing two-question scenario, browser answer actions, and behavioral grader. Keep separate tool-documentation grading unchanged. - Recognize a completed native API creation receipt only when its HTTP 201 result retains the original pending card, exact task/company/agent/source run, idempotency key, and full normalized form. Require the receipt to remain unchanged after the answer and through the final checkpoint. Missing, pending, foreign, or altered proof still fails. - Check signed-in board ownership, each native continuation, no premature or extra output, document authorship, and the saved revision after reload. - Wait for the authenticated product region before screenshots. Recheck origin and path after readiness. Bound readiness and capture by the campaign deadline. - Add closed stage-specific errors, catalog coverage, sanitized receipt calibration, and documentation. Include the provider-form and browser-question presentation helpers in the definition fingerprint. ## Verification - Fresh QA staging qualification passed at `7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks passed, cleanup passed, and all three native runs used the configured QA target and model. The [sanitized run report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/) records the measured pass. This is one successful qualification, not a reliability claim. - Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3` contains the one main merge through `fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of the four-file qualified search fix `4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity matches. It preserves the project and search registrations and atomic search readback. The question live qualification ran at the earlier SHA above; the final integration head has separate local and CI verification. - Eighteen question journey, helper, and test file blobs are identical between the qualified and integration SHAs. This includes clarification flow and evidence, question grader and browser helper, the sanitized receipt fixture, screenshot helper, provider-form helper, browser question presentation, continuation scenario/readiness/flow, shared agent validation, and user actions. The question run-count contract and dispatch branch are also unchanged. Reviewers can compare those paths directly between the two commits. - All 622 selected Product E2E, catalog, question-resumption, necessary-question, report, and readiness tests pass after the main merge. After the exact search delta, all 145 narrow question/search/project/contract tests and all nine real Chromium search regressions pass. Map checks, combined credential-free preview, source fingerprints, and the earlier workspace-link preflight pass. The local TypeScript attempt reports stale Runner declaration errors for `onAccountingEvent` in unchanged native-session code; fresh official CI provides the clean-build type gate. The three search source/test blobs match the qualified search merge, and the three project source blobs are unchanged. - An in-memory probe executes the actual launcher fingerprint expression. Each project, search, and question implementation input changes the digest. Separate red probes first reproduced the omitted provider-form and browser presentation helpers; both pass after their source inputs are added. - The sanitized retained native API response-shape regression fails before the grading correction and passes afterward. Negative controls reject missing, pending, foreign, changed, spoofed, or wrongly bound receipts, answers, continuation, model, environment, and document evidence. All nine provider/semantic/API path combinations pass calibration. - The real local Chromium screenshot probe fails before the readiness repair and passes afterward. Login redirects and sign-in pages at task URLs remain excluded with no screenshot files. - The two original attached failures remain retained: the first stopped at early screenshot capture; the second stopped before answering at the former API provenance gate. An earlier sandbox run also retained one report subprocess failure; the full selected tests then passed with local subprocess IPC access. - The prior integration CI had an unchanged canary mock `EADDRINUSE` failure. That history is retained; its documented retry is owned by the coordinator. No application API, schema, production UI, model, runtime policy, credential, or deployment configuration change is included. Final integration head ae0d9d2898 passed all 58 CI checks, the external Snyk status, and fresh Greptile 5/5 with no unresolved findings. This clean CI includes the official TypeScript and build gates. Recurring selection follows merge and a separate normal TestOps job verification. ## Risks An API result must prove native creation; form shape and run count cannot establish it. The grader checks the server receipt, exact scope, unchanged card and receipt, real board answers, and bound native continuation. The shared answer helper still serves classic journeys. The screenshot wait affects attached journeys and remains limited to the authenticated product region and campaign deadline. One live QA pass establishes the measured journey outcome at the qualified SHA. It does not establish reliability across models or deployments. The integration SHA preserves its journey and helper blobs, while shared registry changes receive fresh CI and review. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 16:18:21 -05:00
DottaandPaperclip	4edd10804e	fix(evals): batch task search result readback across renders (#18370) ... ## Thinking Path > - Paperclip lets people manage AI agents and saved work. > - Product journeys check user actions against saved state. > - The task search journey compares the exact visible task identities. > - A search can replace rows while the harness reads an old indexed locator. > - The attribute read can then time out after the correct results appear. > - This pull request reads the visible links together and preserves strict result checks. ## Linked Issues or Issue Description Refs #18355. **What happened?** An attached Cloud attempt passed the task search journey. A later attempt on the same harness and app revision failed at search-result readback. Its owned screenshot showed the correct target and completed control. Authentication, collection, and cleanup passed. The failure remains recorded and the journey has not been admitted to the daily plan. The reader called `all()`, then awaited `isVisible()` and `getAttribute()` for each indexed locator. When a search replaced the old collection between those reads, the next attribute read waited for an index that no longer existed. **Expected behavior** Read the matching visible links together. Compare the complete identity set against the owned target and controls. A render transition must not consume the whole poll deadline on a stale row index. **Steps to reproduce** 1. Render three task links in the task collection. 2. Let the reader observe the third link as visible. 3. Replace the collection with the two correct search matches before its attribute read. 4. The old reader times out waiting for `nth(2)`, even though the two correct rows are present. ## What Changed - Read visible link attributes in one batch. Discard a detached snapshot and retry within the existing poll deadline. - Keep the exact identity comparison, duplicate detection, control exclusion, saved-content checks, and zero-run checks. - Add a real Chromium render-transition regression and invalid-result cases. Extend the stateful flow test for a detached snapshot. - Document the readback boundary. No app UI, target configuration, or cleanup behavior changes. ## Verification - Review follow-up: the existing journey wait helper is now exercised with the real Playwright poll. Its first read evaluates the actual reader callback on detached browser DOM nodes and returns no evidence. The second read contains an extra owned control. Only the third, exact pair succeeds. The test observes all three reads. The 30-second deadline and grading behavior are unchanged. - Rebased onto current master, including the merged project journey. Credential-free discovery confirms exactly one `project-task-context` and one `task-search-filter` entry. - Latest rebased checks passed: all nine Chromium search regressions, all 381 Product journey unit tests, runner typecheck, discovery, maps, and diff checks. Exact head 183c3e1d19 passed all 58 CI checks, external Snyk status, and fresh Greptile 5/5 with no unresolved findings. - Red: the real Chromium transition test failed on the old reader with `locator.getAttribute: Timeout 150ms exceeded`, waiting for `nth(2)`. The short timeout bounds only this regression fixture. - Green: all nine credential-free Chromium search tests passed. They cover the transition, duplicate/extra/unknown/wrong destinations, visible-link scope, and status-label behavior. - All 131 focused search/oracle/contract/campaign tests passed. - Runner typecheck passed: `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - `node scripts/check-maps.mjs` and `git diff --check` passed. - Fresh QA staging acceptance passed at this exact head: all five checks, authentication/setup/journey/collection/cleanup phases, and zero agent runs. The [measured QA report](https://pages.paperclip.ing/evals-qa-smoke-41124961876-1/) preserves current source and measurement provenance. Daily admission follows merge and a normal TestOps job verification; the earlier failure remains recorded. ## Risks - The fix uses the existing Playwright visible-element predicate and task-link selector. A future UI selector change can still fail the journey. - A detached snapshot returns no evidence. It cannot satisfy a nonempty expected result set. The original poll deadline stays in place; there is no added journey retry. - Browser fixtures prove the readback mechanism. They do not replace the required live Cloud pass. ## Model Used OpenAI Codex, GPT-6 family. The host does not expose an exact model version or context limit. Used repository tools, TypeScript checks, and fresh local Chromium contexts. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 15:51:28 -05:00
Dotta	a4308b9791	test(e2e): add browser project and task context journey (#18353) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People group tasks into projects and use those projects to keep work in context. > - Deployment journeys already test task creation and assignment. > - They do not yet prove that a new project and its task keep the same saved relationship after reload. > - This pull request adds an explicit browser journey for that gap. > - Independent API reads and visible task properties must agree on the exact records and selected settings. > - The journey uses no model runs. Each deployment must pass before a daily plan can include it. ## Linked Issues or Issue Description **What existing behavior does this improve?** The deployment-targeted Product E2E suite can now check project creation and task context through the production dialogs. **Current behavior** The suite covers standalone task creation and assignment. It does not prove that a newly created project contains the task selected in the composer, or that the relation survives reload. **Proposed behavior** Select `project-task-context`. Create a unique project and a self-assigned task with that project selected. Reopen the project, open its task, and reload. Require exact saved company, project, title, prompt, user assignment and standard work mode. Require no agent runs before or after reload. **Reason and benefit** The same journey can qualify different deployments. It catches lost project links and incorrect saved context without a model call or a separate QA-only test implementation. ## What Changed - Add the browser flow and an independent saved-state oracle with positive and negative cases. - Scope project and task navigation to unique links in the main content so sidebar links cannot bypass list coverage. - Bind reopened routes to saved project and task references. Verify the main project heading and visible task content instead of a hidden duplicate sidebar label. Keep browser failures as closed stage codes. - Register the journey, its closed success checks, and its source files in the definition hash. Default model-matrix selections stay unchanged. - Associate qualification-report coverage with project lifecycle, project task context and task creation. Reject retained browser-only passes containing run IDs, provider/model metadata or native/legacy execution. - Add `project` to the existing durable ownership resource contract. A target provider remains responsible for cleanup of its own project fixtures. - Update discovery expectations, the deployment journey guide and the projects feature map. ## Verification - Focused runner tests: 110 passed across the new oracle, discovery, journey contracts and existing report subprocess checks. The first sandboxed attempt blocked the existing report subprocess IPC socket; the same selection passed with local IPC enabled. - Late review regression checks: both report association and unexpected browser-only model work reproduced as failing tests before the fix. The final contract, saved-state oracle and report catalog selection passed all 72 tests. Runner typecheck passed again after these changes. - Browser readback: the old project-name selector failed in Chromium when a hidden sidebar label preceded the production project heading. All six browser regression cases now pass, including missing rows, wrong project/task destinations and a wrong project property. The fixture uses the production heading and task-row overlay conventions. Credential-free Node 24 imports pass. - Current-master integration: 208 focused catalog, project/search, campaign and reporting tests passed. All nine project/search browser regressions passed. Runner typecheck, map checks and Node 24 imports for both journeys passed. The combined catalog, dispatch, source hashes, success checks, browser-only guard and feature associations retain both selections. - The first deployment qualification created both owned records but failed during later readback with a generic browser code. Authentication, setup, collection and cleanup passed. That failed grade is retained. The corrected source subsequently passed a fresh attached QA campaign; the original failure remains preserved. - New oracle alone: 7 tests passed after the final catalog mapping change. - `node scripts/check-maps.mjs` — passed; structural checks only. - Credential-free `--suite product-journeys --list` — passed and lists this browser journey. - `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json` — passed after building required plugin, runner and eval-kernel declarations. The initial fresh-checkout check failed because those declarations were not built. - All current-head CI gates passed, including repository typecheck, tests, build and browser checks. Greptile reviewed the exact head at 5/5 with no unresolved findings. Fresh attached QA passed at `f19557c9eebde14c2def37195d6b154c23eb9690`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41119722363-1/). All five independent checks and authentication, setup, journey, collection and cleanup passed; zero agent runs started. ## Risks - This adds an opt-in test and changes no production behavior or schema. UI control changes can fail the journey. - The target provider must support durable `project` ownership before it can run this case. Project cleanup must settle owned tasks, reject foreign linked tasks and archive the project while preserving history. - Source registration alone does not include the candidate in any saved daily plan. A fresh pass against that deployment is required first. ## Model Used - OpenAI Codex, GPT-6 family, with code inspection and tool execution. The host does not expose the exact underlying model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicate journey found) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-30 15:07:21 -05:00
DottaandPaperclip	a0e3ac62a7	test(evals): add reusable task search and filter journey (#18355) ... ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - The task list lets people find saved work and open its context. > - Product journeys can exercise the same task UI on local and Cloud targets. > - The shared suite lacks a search and status-filter journey with control tasks. > - This pull request adds that journey with exact task identity checks. > - The benefit is reusable evidence that search, filters, and task navigation work together. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for task search, status filters, and saved task navigation. **Current behavior** The shared journey suite covers task creation and assignment. It does not prove that a search excludes an unrelated task, that a status filter excludes a matching completed task, or that the selected task retains its saved content. **Proposed behavior** An explicit `task-search-filter` selection creates three owned tasks. It searches through the production task list, applies Backlog, and opens the exact target. It verifies reload and Back navigation, saved task fields, and zero agent runs for all three tasks. **Reason and benefit** The same journey can run against a managed local stack or an attached Cloud target. Independent assertions reject missing evidence and plausible wrong results. Deployment qualification and daily-plan admission remain separate gates. **Breaking changes** None. Existing journey selections keep their behavior. Related work: #16979 fixes status filtering before pagination. #15403 covers saved task views. This journey tests the existing search/filter interaction and does not add a new UI or saved-view feature. ## What Changed - Add a zero-run browser journey with one target and two owned controls. - Compare exact visible task identities after search, filtering, reload, and Back navigation. - Verify saved company, owner, title, description, status, and zero agent runs through the public API. - Register dispatch, source hashing, closed success checks, and browser reporting validation. - Add adversarial oracle tests and a campaign dispatch test. Document the selection and its feature-map scope. - Exercise the real browser flow with a stateful UI/API port across Coordinator, combined Tasks, and legacy shells. Inject missing ownership, wrong destinations, lost filters, changed saved context, and unexpected native work. - Use the combined Tasks shell's named Backlog view for reload persistence, as required by its existing view-preset contract. Preserve the successful polled result snapshot and link the case to feature coverage. ## Verification - First attached Cloud attempt failed during Backlog selection. Its cleanup passed and it started no model work. The original failed result is preserved; the corrected candidate passed a fresh attached campaign below. - Local Chromium reproduces the cause: the legacy StatusIcon and text label produce the accessible checkbox name `BacklogBacklog`, so the prior exact `Backlog` lookup finds no status control. - The selector now stays inside the Status group and accepts the explicit and duplicated names. Three real Chromium regressions pass, including a separate assignee called Backlog. - Latest focused journey checks: 130 passed. Runner typecheck and map checks passed. Unexpected browser/API failures retain only closed source-owned stage codes. Screenshot privacy guards and all exact identity, content, filter, and zero-run checks are unchanged. - Fresh attached Cloud qualification passed at `e3dfc0ae37e8ac74e9a07ca24421ea6c96f9bcfc`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41114233380-1/). All five independent search/filter/reload/reopen/no-agent checks and all five phases, including cleanup, passed. The browser journey started zero agent runs. - Runner typecheck passed: `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Focused oracle, contract, and campaign tests passed: 101 unique tests. The campaign suite was rerun after its new dispatch test: 30 passed. - Direct browser-flow tests passed: 23 tests. These call the real `taskSearchFilter` function through a stateful UI/API port. They do not establish live browser or Cloud qualification. - Latest focused search, oracle, catalog, and report checks passed: 127 tests across five files. - Full runner Vitest completed with 5,204 passes, two skips, and three failures outside this change. Two local embedded PostgreSQL fixtures failed at bootstrap. The Pi controls test expects profile version 35 while the unchanged base declares 37. These failures remain recorded; the PR does not change those fixtures or pins. - The remaining Node support suite passed separately: 137 tests. The standard unit launcher stops before these tests after a Vitest failure. - Credential-free discovery passed: `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/launch.ts --suite product-journeys --list`. - `node scripts/check-maps.mjs` passed. `git diff --check` passed. - All 58 current-head check runs are green, the additional security status passed, Greptile reviewed this exact head at 5/5, and no review threads remain unresolved. Typecheck, test, build, browser CI, and fresh attached Cloud acceptance passed. Daily-plan admission follows merge. ## Risks - Task-list control labels and link structure can change. Such drift fails the journey and needs an update. - The journey creates three board-assigned tasks. It records ownership before creation and records each returned ID immediately. The existing target provider owns cleanup. - Local assertion tests do not prove that a deployed Cloud target passes. The target owner must record a fresh live pass before admission to a daily plan. ## Model Used OpenAI Codex, GPT-6 family. The host does not expose an exact model version or context limit. Used repository search, TypeScript tools, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 14:26:53 -05:00
DottaandPaperclip	ac4f13d205	fix(e2e): report committed child blockers promptly (#18306) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E checks the ordinary onboarding conversation and its delivered work. > - A native run can finish successfully while its task remains blocked for a human. > - The onboarding journey kept waiting for such a child until the campaign deadline. > - This pull request reports the committed child blocker when no owned work remains pending. > - The journey still requires the same independent checks before it can pass. ## Linked Issues or Issue Description **What happened?** An approved child can stop with a committed current-track blocker owned by the board. Its native run succeeds, but the child task stays blocked. The onboarding journey reports a deadline failure instead of the observed child blocker. **Expected behavior** Report `onboarding_child_task_blocked` when the latest child run commits that blocker at the child's current status version. Keep waiting for parent dependencies and pending native or workspace recovery. **Steps to reproduce** Use the existing onboarding fixture with an approved blocked child. Give its latest native run a succeeded status, committed finalization, blocked work disposition, and the matching task status version. The old journey waits until its deadline. The new journey fails at that settled blocker without downloading or accepting any delivery. Related public context: #18298. Public issue and PR searches found no duplicate blocked-child harness fix. This change reports the failure boundary; it does not repair artifact delivery. ## What Changed - Check the owned child's board blocker after the existing run and recovery checks. - Require a unique latest producer, native committed results, and the current task status version. - Preserve parent dependency waits and all artifact, document, download, link, scope, and budget checks. - Add a behavioral regression and controls for active work, native recovery, workspace replacement, stale versions, newer producers, and agent-owned blockers. - Document the distinct failure code and the existing control-plane writers for its metadata. ## Verification - The new behavioral regression failed on the old code with `onboarding_first_task_deadline`; 147 controls passed. - The current head passed all 148 onboarding tests, including the valid agent-owner object control. - The initial implementation passed 305 tests across all 10 Product E2E files. The later update changes only writer links and the owner control; its 148 focused tests and harness typecheck also pass. - `pnpm test:e2e:runner:typecheck` passed after building the isolated checkout's workspace declarations. - `git diff --check` passed. - All required CI checks passed on `1b3bd04d193fc98d0d9f2eb3b826a4ef4bdb6dea`, including full tests and build. They were not repeated locally. Greptile reviewed that exact head at 5/5. - The full Cloud journey remains unverified by these source and CI checks. This change does not claim that the underlying artifact delivery issue is fixed. ## Risks The early failure depends on the public task and run projections. Missing, stale, or ambiguous producer metadata keeps the existing bounded wait. A pending recovery or newer producer prevents the old blocker from ending the journey. No success requirement changes. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`, reasoning effort `ultra`. The model used source inspection, code editing, and local test tools. The runtime did not expose its context window size. Root supplied the model and effort metadata for this agent session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 08:26:14 -05:00
DottaandPaperclip	0c70cedce9	Wait for the saved Studio file before editing a skill (#18305) ... ## Thinking Path > - Paperclip lets people manage AI agents and use their saved work. > - Product E2E journeys check these user paths in classic and Cloud environments. > - The shared Skill Studio helper edits a skill created by an agent. > - Studio can mount the editor before its saved file finishes loading. > - Typing into that empty draft can replace or lose content when the file arrives. > - This change waits for the existing saved body before it starts the edit. > - Browser regressions check delayed loading and refusal to edit an unloaded file. ## Linked Issues or Issue Description Refs #18222 for the shared Product E2E journeys. No duplicate hydration fix was found. **What happened?** The shared Studio helper started typing as soon as the editor control was visible. A delayed file query could replace that draft. The helper could then lose the original content or fail to save its complete marker. **Expected behavior** The helper must load the original file before editing it. It must preserve that content, save the new marker, reload, and read the result from the task card. **Steps to reproduce** Run the controlled Studio browser fixture with a 1.5-second file load. The old helper saves the marker without the original `Verify checks.` body. With no file load, it still sends input events to the empty editor. ## What Changed - Wait for the original saved body before typing in the shared Studio helper. - Add browser checks for immediate load, delayed load, and an unavailable file. - Include these checks in the credential-free browser support suite. - Document the saved-file readiness requirement. ## Verification - Against the old helper, both new failure cases fail: the delayed case loses the original body, and the unloaded case sends 13 input events. - With the change, all three browser checks pass. They verify the saved original body and marker, reload, task-card readback, and zero input or save calls for an unloaded file. - `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json --noEmit` passes. - `git diff --check` passes. Full repository CI passes in [41058312896](https://github.com/paperclipai/paperclip/actions/runs/41058312896), including typecheck, tests, build and all eight E2E shards. Greptile reviewed this exact head at 5/5 with no open threads. No live acceptance result is claimed by these controlled fixtures. ## Risks This adds a readiness assertion to an existing test helper. Both callers already use the same skill fixture and verify `Verify checks.` in the task card. An unavailable file now fails before an edit can occur. The change does not alter application behavior, model instructions, grading, or deadlines. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the bug template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all reviewer comments before merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 08:05:04 -05:00
DottaandPaperclip	8f50704135	fix(e2e): wait for pending native recovery (#18298) ... ## Thinking Path > - Paperclip manages agents and their work. > - Native runs can retain a bounded automatic recovery on the same run. > - Product E2E observes those runs before it checks the delivered work. > - The onboarding journey treated every failed run as a final failure, even when the server had saved its next recovery attempt. > - This pull request waits only for the explicit agent-owned native recovery projection. > - The journey still requires successful execution and the same independent delivery checks. ## Linked Issues or Issue Description **What happened?** The onboarding journey can stop after a native turn completes without its semantic result. The server marks that attempt failed and saves a same-run retry. The journey rejects the failed row before the saved retry is due. This also affects a native child that follows a proven workspace deferral. **Expected behavior** Observe the saved automatic recovery within the original campaign deadline and run cap. A pending recovery cannot satisfy completion. Ordinary failures and exhausted, terminal, or board-owned recovery must still fail. **Steps to reproduce** Use the existing onboarding fixture with a native failed row whose nativePhase is retryable_failure. Give it the server-owned native_session_interrupted cause, an agent-owned retry_scheduled projection, a valid retry time, and attempt 1 of 3. Transition that same row through reconnecting and running to success. The old journey rejects the first row. Related public context: #18292. A search of related native-recovery PRs found no duplicate harness fix. ## What Changed - Observe the authenticated native recovery state in the existing workspace-deferral module. Require matching nonempty cause strings and every owner, phase, attempt, schedule, deadline, scope, and successor guard. - Apply it to the proposal wait, failed native rows, workspace successor tails, and final settlement. - Keep defensive rejection of known permanent, permission-declined, and exhausted causes. Diagnostic cause strings are not a positive retry policy. - Add behavioral recovery, deadline, and rejection controls to the existing workflow tests. - Document that this wait preserves the original campaign and final delivery requirements. ## Verification - Baseline workflow regression: 4 expected premature failures with 90 controls passing. The final structural calibration rejected 20 valid pending-recovery cases before the correction, with 120 controls passing. - Final regression: 140/140 pass, including existing and new diagnostic causes on direct and workspace-successor paths. Known terminal, ambiguous, malformed, mismatched, and exhausted states still fail. - Focused Product E2E compatibility: 297/297 pass across 10 files. - pnpm test:e2e:runner:typecheck: pass after building the normal SDK and Runner TypeScript dependencies. - git diff --check: pass. - Final source passed these checks before the reviewed commit. - Full repository tests/build and live Cloud acceptance were not run locally for this harness-only change. Required CI run 41049692194 passed on 04dedd7d. Live acceptance remains separate. ## Risks The execution projection is a polling observation, not dispatch authority. A server that never resumes can hold this wait until the original deadline. The predicate requires matching native interruption state and cause, an agent owner, a bounded attempt and schedule, and no successor. Known terminal-code rejection is a defensive check, not the authority to retry. The journey must still observe actual success and pass its document, artifact, download, and parent-link checks. ## Model Used OpenAI gpt-6.1-sol, ultra reasoning, inherited from the parent runtime metadata. Context window size is unavailable. Used source inspection, code edits, shell tools, Git, and deterministic tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with Fixes/Closes/Refs OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 06:55:41 -05:00
DottaandPaperclip	8035f8b245	test: require saved onboarding approval cards (#18292) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests exercise real user workflows through the browser. > - The onboarding journey requires a human to accept one proposed child task. > - A live agent tried to create the child before it saved a proposal card. The product correctly rejected that action. > - The scripted request named the required result but did not name the proposal tool or require its saved receipt. > - This pull request makes that handoff explicit. It keeps the existing human approval and independent result checks. > - The benefit is a clearer scripted workflow without a change to product authority. ## Linked Issues or Issue Description Refs #18288 and #18266. **What happened?** The agent treated the opening task request as a step toward acceptance without first saving a proposal. It then stopped after child creation was denied. Its visible response referred to a pending proposal, but no proposal card existed. **Expected behavior** The agent saves a native confirmation card, checks its receipt, and waits for the human to accept it through the browser. It must not record its own approval. Only an accepted card authorizes the child task. **Steps to reproduce** Run the scripted Cloud onboarding journey. Give the canonical onboarding task its first task request. The observed agent skipped the saved proposal and tried to create the child. Model behavior varies, so this is a fixture guidance change rather than a deterministic product defect claim. **Paperclip version or commit** Fixture base: `7a7ade4b9360aaf8e1a2808bb2b1a752c080f564`. The serving product was Core `965184d2968f40b1183803529b88f288264c832f`. **Deployment mode** Testing Cloud workspace with native Codex. This pull request selects no product release or deployment. ## What Changed - Name the existing `request_human_input` confirmation or revision-bound checkbox contract and `wake_assignee` policy in the parent request. - Require a saved pending-card receipt. Require corrections when tool arguments are rejected. - Forbid self-approval. Require the actual accepted human card decision before child creation. - Document this scripted handoff. Keep the child brief, final download links, browser actions, and all grading assertions. ## Verification - Focused onboarding and Cloud journey tests: 79 passed. - Shared evidence, campaign, and contract tests: 66 passed. - `pnpm test:e2e:runner:typecheck`: passed after building the existing workspace type dependencies. - Credential-free preview under a network-deny guard: passed with zero external calls. Its preparation plan matches the base fixture. - `git diff --check`: passed. - No prompt-string test was added. Existing tests continue to enforce approval order, child scope, independent file checks, and real result links. - Full local repository build and database suites were not repeated for this fixture text change. Required PR CI must pass before merge. - The original live result remains unchanged. This local work does not prove that a later model run will comply. ## Risks - The model can still ignore the instructions. A new real campaign must prove the workflow. - More explicit guidance makes this a scripted product journey. It is not a model discovery benchmark. - There are no product runtime, schema, credential, model, budget, or approval-policy changes. ## Model Used OpenAI Codex, exact model `gpt-6.1-sol`, ultra reasoning effort. Parent runtime metadata confirmed this model. Context window size was not reported. Used source inspection, local code execution, and an independent agent review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 05:04:21 -05:00
DottaandPaperclip	7a7ade4b93	test: require real onboarding result links (#18288) ... ## Thinking Path > - Paperclip lets people assign work to agents and use their saved results. > - The Cloud onboarding journey tests this handoff through a real parent and child task. > - The child can publish a correct file while the parent names that file without linking it. > - The existing journey correctly rejects that reply because the user cannot download the named result from it. > - This pull request makes the scripted parent request explicit about real Markdown links and returned attachment paths. > - The independent artifact and parent-link checks remain unchanged. ## Linked Issues or Issue Description Refs #18266 and #18258. **What happened?** A Cloud onboarding attempt produced a correct child artifact and saved document. The parent then named the child, document key, and filename in inline code. It omitted the attachment URL. The attempt failed with `onboarding_parent_result_links_missing`. **Expected behavior** The parent's final reply must link the saved child results. The file link must use the real published attachment reference. **Steps to reproduce** Run the existing explicit Cloud onboarding journey. A reply that only names the file must remain a failure, even when the child artifact is correct. **Paperclip version or commit** The branch starts from `965184d2968f40b1183803529b88f288264c832f`. ## What Changed - Ask the parent for real Markdown links in its final reply. Use the published attachment's returned content or download path. Do not invent IDs or URLs. - Add a filename-only reply to the existing negative oracle cases. - Document this scripted fixture requirement. ## Verification - Focused onboarding tests: 79 passed. - Shared task, campaign, and contract tests: 66 passed. - Existing artifact and native-completion Node checks: 137 passed. - `git diff --check`: passed. - Harness typecheck, `pnpm -r typecheck`, and `pnpm build`: passed after building the existing workspace dependencies. The initial missing-declaration and local-IPC setup failures remain recorded. - The full local database test suite was not rerun. Required repository CI runs the broader suites on Linux. - Required repository CI passed on the exact PR head. No paid model calls or Cloud operations were run for this patch. - [Initial CI](https://github.com/paperclipai/paperclip/actions/runs/41027635545/attempts/1) failed one unchanged retention timing test (3310 ms versus the strict 3000 ms bound). The unchanged case passed once on an isolated local Postgres17 fixture. One failed-jobs-only rerun passed in [attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41027635545/attempts/2). The original failure remains visible. No timing threshold or retention source changed. - The previous live attempt stays failed. Local tests do not prove that a model follows the revised request. A new bounded live attempt must establish that result. ## Risks - The extra instruction can still be ignored by a model. The unchanged oracle will reject missing links. - This changes the fixture definition hash. Historical attempts retain their original source and result. - No product runtime, API, approval policy, model profile, budget, or cleanup behavior changes. ## Model Used OpenAI `gpt-6.1-sol`, ultra reasoning, with code execution and repository tools. Model metadata was confirmed in the parent task. The context window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 04:01:33 -05:00
DottaandPaperclip	81ebe479e8	fix(evals): bind rendered Markdown replies to saved run evidence (#18264) ... ## Thinking Path > - Paperclip lets people assign work to AI agents and read saved results. > - Product journeys must prove that a result is visible and belongs to the completed run. > - The shared observer compares stored Markdown with browser text. > - A document link can render as an issue chip, so those strings differ. > - This PR keeps run and comment attribution, then compares full text and exact link destinations. > - The benefit is a valid browser result without a false timeout or a weaker provenance check. ## Linked Issues or Issue Description Refs: #18248, #18255. **What happened?** The protected-action story completed both decisions and all four native turns. Its final explanation contained a document link. The browser rendered that link as an issue chip. The observer waited for raw Markdown to equal rendered text, so it could not accept the valid result. **Expected behavior** Accept complete matching content from the final run after reload. Keep the exact link destination, task anchor, author, run and durable final-message checks. Reject missing or changed content. **Steps to reproduce** 1. Save an attributed reply with bold text and a Markdown document link. 2. Render the link as an issue chip in the task bubble. 3. Run `readBoundTaskFinalReply`. The original observer times out. The new browser regression preserves that failure. ## What Changed - Add a UI-owned content projection using its existing Markdown renderer and GFM/issue-reference plugins. Keep DOM observation, attribution and destination checks in the task-reply observer. Keep source Markdown and observed text separately. - Read visible DOM text, link labels and destinations. Require the structured projection to match the actual visible text. Allow only recognized product chips to replace a link label. Ordinary links keep their source label. Reuse the production issue-reference and company-route helpers, plus the current task URL. Ordinary root-relative anchors keep their original path. Preserve exact marker checks and all run/comment/native-event attribution checks. - Check visible decline explanations and unread data in preserved native source text. - Add browser regressions and source/content tests. Reuse the UI’s existing `react-markdown` and `remark-gfm` dependencies. The final diff changes no package manifest or lockfile. - Match generated GFM checkbox states and reject changed states. Document supported content and fail-closed behavior. ## Verification - Original observer: the decorated-link browser regression fails at the final-reply predicate. - Fixed observer: all 21 browser regressions pass, without credentials or model calls. - Focused observer, decline, content and journey contract suites: 64 tests pass. - E2E and UI TypeScript checks pass. All UI token gates pass. The unchanged frozen install passes. - Full CI passed on the preceding 6eba7ff head (41003112166). Full CI passed on the preceding b9a2571 head (41006205475), with both reviewers at 5/5. Current-head full CI passed at b3d2d08305 (41009669966), including typecheck, build and all eight E2E shards. Both reviewers scored this commit 5/5, with no unresolved threads. The initial lockfile policy failure was corrected. The final diff has no manifest or lock changes, and its unchanged frozen install passed. - The optional positive GFM strikethrough assertion also passes. - The Node 24/tsx end-to-end observer probe passes. The callback has no external compiler helper; a browser regression covers actual tsx serialization. - The Paperclip review finding is also fixed: bare URLs and issue keys use the production plugins and the UI’s `scope=accessible` company-prefix context, with RED/GREEN browser coverage. Source Markdown is required in the receipt. - All Greptile findings are fixed, including matching the UI’s accessible-company scope: ordinary link labels, issue-prefix/fragment resolution and ordinary root-relative paths have positive and negative browser coverage. - The deployed attempt remains a failed original measurement. Fresh qualification is required after this repair. ## Risks The comparison normalizes whitespace and inline formatting. It verifies full text and exact link targets, while link labels may become product chips. Images, raw HTML, tables and fenced code fail closed. Those constructs may require a future explicit extension if a journey uses them. No application behavior, model, budget, approval or connection permission changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution and browser tools. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 00:57:59 -05:00
DottaandPaperclip	145214201d	test: preserve the onboarding child brief at delegation (#18266) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys check onboarding through the real app. > - The onboarding story asks a parent agent to delegate a file and document task. > - A parent can preserve the goal but omit the exact tool instructions from its child description. > - The child can then publish a note instead of the required JSON file and task document. > - This pull request asks the parent to copy the full child brief without changing it. > - The existing independent checks still decide whether the result passes. ## Linked Issues or Issue Description Refs #18258. Includes merged prerequisite #18270 for the existing retention scheduling failure found by required CI. **What happened?** The scripted onboarding request gave the parent an ordered tool recipe. It did not require the parent to keep that recipe in the child description. A delegated child could compute the result but publish a Markdown note and omit the Paperclip task document. **Expected behavior** The child receives the full approved brief. It must publish `connection-proof.json`, save its Paperclip task document with the returned file link, and report completion through the current contract. **Steps to reproduce** Run the explicit Cloud onboarding journey. Inspect the approved child description and its published result when the parent summarizes the request. **Paperclip version or commit** This change is based on public master `b3c2172d417ea6b783e0c796aaed4a91e1f9d381`. **Deployment mode** Scripted Product E2E against a live Cloud testing workspace. ## What Changed - Put the existing ordered instructions in one marked child brief. - Require the parent to copy that brief verbatim into `create_task.description`. - State the exact JSON filename and workspace-relative content reference for `register_deliverable`. - State that a Markdown workspace note does not replace the `write_document` step. - Document this scripted scope in the Product journey guide. ## Verification - Existing focused Product journey tests: 227 passed in 10 files. - Credential-free Product journey discovery passed through the direct tsx loader. - Source comparison confirms that all code after the opening reply is unchanged. The download, calculation, hash, provenance, document, parent-link, approval and run-limit checks retain their original behavior. - No prompt word-matching tests were added. This is an instruction-only change. - `pnpm build`, `pnpm -r typecheck`, and `pnpm test:e2e:runner:typecheck` passed. - Full support Vitest check: 5,042 passed, 2 skipped, 2 failed. Both failures occur during embedded PostgreSQL startup in the controller database lifecycle fixtures. The log includes a missing bundled `libzstd.1.dylib`. The separate Node support checks passed all 137 tests. - `pnpm test:run` was attempted. It reached the general server suite and repeated embedded PostgreSQL startup failures. I stopped only this owned local test process group. The full local test command is incomplete. Required Linux CI remains the merge gate. - The first harness typecheck ran before workspace build outputs existed and reported missing workspace modules. It passed after the full build. The earlier failure remains recorded. - The initial CLI discovery wrapper hit a local tsx IPC permission error. The direct loader passed with the same source and arguments. - The original required CI run found an existing skill-retention page delay. Its unchanged test passed once against a dedicated local PostgreSQL fixture. Separate PR #18270 added a deterministic regression and fixed the scheduler. After that PR merged, this branch received one normal merge from master. Its diff remains two fixture files. The original failed CI run remains preserved. - After the branch refresh, all 70 onboarding first-task tests and 7 retention/deadline controls passed. Harness typecheck passed. Fresh required CI and review passed on head `608e266c6567c19981fd3f8a07305c785346753b`. - The refreshed CI run passed the retention shard but failed one unchanged Antigravity authentication fixture. The same Runner lane passed on the prior fixture head and the merged prerequisite. The exact test passed once locally. Source review identified a possible stdout/stderr ordering race, but the failed run did not retain its exact error subtype. The one approved retry of the failed CI job and its verification aggregate passed on unchanged source. The original failure remains preserved. This proves recovery, not its cause. - The live Cloud journey has not been rerun with this source. Historical failed attempts retain their original results. ## Risks - An agent can still fail to follow the brief. Existing independent assertions will reject the wrong file or missing task document. - The fixture definition changes. A later live attempt must record this new harness revision. - No application, model, effort, budget, cleanup or acceptance rule changes. ## Model Used - OpenAI `gpt-6.1-sol`, ultra reasoning effort, with repository tools and code execution. The parent verified this model ID from current runtime metadata. The context window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 00:13:37 -05:00
DottaandPaperclip	b3c2172d41	fix(evals): verify native protected-action outcomes (#18255) ... ## Thinking Path > - Paperclip manages work assigned to agents. > - Product E2E stories ask for useful user-facing results. > - Classic runner and deployed QA reuse the same service approve/decline task prompts. > - The service returns a receipt marker that belongs in a saved briefing document. > - State that destination explicitly and ask for the document link when complete. > - The deployed decline path also needs the same durable final-reply proof as Plan and Stop. ## Linked Issues or Issue Description Refs #18222. Builds on the merged reply observer in #18248. **What happened?** The shared service tasks ask for titles and a “verification code” without specifying that the code belongs in the document or asking for its final link. A native QA attempt created its briefing but then repeatedly failed the completion tool’s array validation. Those invalid arguments remain a separate observed failure; the wording change does not establish their cause or relax validation. Source inspection also found that the declined-action oracle requires a projected comment even when the settled native thread renders a durable final message. **Expected behavior** Ask for page titles and the service receipt code in the saved briefing, then a document link. After a declined action, require the existing explanation-only fallback without another service call. **Steps to reproduce** Run the shared service-approve/service-decline stories through the classic runner or QA Product E2E adapter. ## What Changed - Clarify the natural-language task for both shared service stories: put page titles and service receipt code in the document and provide its link. - Use the reloaded run-bound final reply for QA service declines. Pass an explicit native receipt to the existing decline oracle; do not synthesize issue comments. Require the final native run to start after the saved decline and keep exact decision, truthful explanation, no unread marker, zero calls and unchanged connection checks. - Keep classic callers on their existing attributed-comment path. No internal tool instructions are added to the user prompt; ownership, service data and model/runtime limits stay strict. ## Verification - Decline regression observed RED: two cases reject a valid saved native explanation without projected comments. After the repair, 79 focused tests passed across native reply, decline guidance, agent evidence and journey contracts. A two-successful-run control rejects the earlier receipt and accepts the later one. Negatives cover stale/wrong run, wrong agent/task, missing final proof, summary/unsettled/hidden output, false explanations and unread service data. - Product E2E TypeScript and `git diff --check` passed. - Live protected-action qualification is required after merging this and the companion private service/cleanup fixes. The previous failed result remains failed; this is a bounded task-clarity experiment, not a claimed model or completion-schema repair. - Current-head Greptile and Paperclip reviews are 5/5 with zero open threads. CI initially failed the unrelated agent-chat-projects cleanup POST with a PostgreSQL lock timeout after its assertions completed. One bounded retry passed on unchanged source; all current-head gates passed at `64bc169e147290ed4dbaa18221e0268362ce81ee`. The original failing attempt and its logs remain available. The task-clarity change is still a bounded experiment; the observer repair addresses a demonstrated storage-assumption defect. ## Risks A clearer task can still fail with the selected model. All acceptance checks and completion input validation remain strict; a saved briefing alone cannot qualify the complete journey. Only test prompts and test evidence code change. Missing or contradictory native evidence fails closed. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed by this session. Used reasoning, source editing, shell execution and test inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 22:14:37 -05:00
DottaandPaperclip	59b76777db	test: clarify onboarding artifact completion instructions (#18258) ... ## Thinking Path > - Paperclip helps people manage AI agents and inspect their work. > - Product E2E journeys test that work through normal browser and API controls. > - The Cloud onboarding journey asks one native child to publish a file and save a document. > - Its task request did not explain the completion report format or make each output step distinct. > - A live attempt published the file, then submitted an invalid finish report and declared a blocker. > - This change orders the scripted request and explains the existing completion contract. > - The agent must still produce the result and pass the same independent checks. ## Linked Issues or Issue Description Refs #17940. Refs #18228. **What happened?** The scripted onboarding request left completion details implicit. A live child published its proof file, then submitted a finish payload without verification and with an invalid schema value. Its later blocked report also listed local paths as artifacts. This does not establish a Core validator defect. **Expected behavior** The request should explain the existing tool contract. The child must publish the file, save the document with its returned link, then submit a valid completion report. Argument correction must not replace artifact verification or bypass approval. **Steps to reproduce** Run the explicit Cloud onboarding Product E2E journey through its private target provider. Approve the proposed child through the UI. Inspect the native finish report and the independently checked attachment, document, and parent links. **Paperclip version or commit** The observed deployment used public Core `6a674716447e8fa268c5cdeb1db1020319368a77`. This fixture change starts from `a8e07500db68f187b19de83823eb7fabf1196021`. **Deployment mode** Cloud testing with the scripted Product E2E harness. ## What Changed - Order the child request: create files, publish the proof, save its document, then finish. - Require verification for actual checks and omit the optional schema field. - Explain receipt-derived references in evidence and file artifacts. Keep local input paths in verification details. - Document same-attempt argument correction and preserve historical failed results. ## Verification - Existing onboarding-first-task and artifact-evidence suites: 83 checks passed. - The maintained harness unit wrapper also passed 137 Node contract checks. - Independent source review found no contract, privacy, or oracle change. - Full `pnpm build` and `pnpm -r typecheck` passed at the preceding commit `c3900cf2`. The only later change names the filename required by the existing document assertion. - Current `d75180b4`: the 83 fixture/artifact checks, 137 contract checks, and `pnpm test:e2e:runner:typecheck` passed again. - Credential-free `--list --suite product-journeys` passed. - The local full repository Vitest run at `c3900cf2` was stopped before the filename correction. It is incomplete and is not reported as a pass. Current-head CI passed: 54 successful checks and two optional Storybook skips. The separate advisory review is still running and is not reported as passed. Initial fresh-worktree dependency and sandbox IPC failures remain recorded. - No paid model, Cloud workspace, release, or deployment was run for this change. These checks establish harness compatibility, not a live model pass. The prior failed campaign remains failed. ## Risks The model may still submit an invalid report or fail another step. A fresh live journey is required to measure the instruction change. This PR changes fixture text and documentation only. It supplies no result, checksum, real attachment ID, or successful status. Models, limits, validators, and independent assertions stay the same. ## Model Used OpenAI Codex, `gpt-6.1-sol`, ultra reasoning. The model identity comes from confirmed parent runtime metadata. Context window size is unavailable. Tools and code execution assisted the change and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 22:02:51 -05:00
DottaandPaperclip	2badf8a4a5	fix(evals): verify native QA replies on the reloaded task thread (#18248) ... ## Thinking Path > - Paperclip manages AI agents that do work for users. > - Product E2E journeys check user outcomes on real deployments. > - Native task threads can render saved final replies from a settled run. > - The new QA Plan and Stop checks looked only for agent comments. > - Both flows reached the requested terminal state and showed the correct reply, but that storage assumption rejected them. > - This PR checks the reloaded visible reply and binds it to the final successful run. Explicit presentation decisions remain authoritative. ## Linked Issues or Issue Description Refs #18222 and #18244. **What happened?** The deployed Plan journey completed all three native runs, including revision and approval. Stop completed native cancellation, resource release and a new direction. Their final-answer checks failed because the comments endpoint did not contain the rendered native reply. The original failed grades remain unchanged. **Expected behavior** Require exactly one correct visible final answer after reload, attributed to the final successful owned run. Reject an earlier run, progress text, hidden content, duplicate answers or a cancelled run. If its selected completion comment is projected, require that comment and matching author/run/text. Otherwise require a single durable final assistant message and its settled native final-response anchor. **Steps to reproduce** Run the Plan or Stop journey on a native deployment whose task thread renders the settled run before a completion comment is projected. ## What Changed - Add a narrow reply observer over the existing task-thread hooks. It reloads the page and refreshes saved run/comment metadata while waiting. - Bind native replies to a validated durable final assistant event and the successful run's settled final-response anchor. Require the selected comment when projected; support pending or suppressed projection without accepting a summary fallback. Accept payload and item-normalized finals, rejecting contradictory fields. - Use the observer for Plan completion and Stop's new direction. Keep existing run counts, deployment binding, cancellation and artifact checks. - Add negative grading tests, include the helper in the definition hash, and document its evidence contract. Classic protocol comment matchers are unchanged. ## Verification - 63 focused tests passed across native reply evidence, agent evidence and journey contracts. - Product E2E TypeScript passed. The reply observer uses the remaining journey deadline and backs off polling. - Live Plan and Stop attempts exposed this grading failure. Their original outcomes remain failed; fresh qualification is required after merge. - Full current-head CI passed at `40840ed742977853630d733292461ad7003fc7fb`; Greptile and Paperclip Review both scored 5/5, with zero unresolved review threads. ## Risks The observer depends on existing task-thread test hooks and run anchors. Missing or changed hooks fail closed. It reads rendered user-facing text; a finish summary alone cannot satisfy it. Native event provenance, run ownership and exact reply equality are required. No backend, model, provider, budget or cancellation policy changes are included. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed by this session. Used reasoning, source editing, shell execution and test inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 21:35:15 -05:00
DottaandPaperclip	d3f2a60e79	test(e2e): reuse core journeys across deployment targets (#18222) ... ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests verify work through the browser and public APIs. > - The classic matrix already tests several common task stories. > - Its older stories depend on local fixture setup and workspace access. > - Deployment tests need the same story checks with a separate target adapter. > - This PR shares those checks and adds five explicit journey selections. > - A deployment pass still requires evidence from that deployment. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E journey reuse across managed instances and deployed Cloud instances. **Current behavior** The deployed-target catalog covers task creation, assignment, Ask, and fresh onboarding. Plan revision and Everyday stories run through the classic fixture driver. Several story steps assume a local workspace. **Proposed behavior** Add explicit Plan revision, artifact revision, protected-action approval and decline, Skill Studio, and Stop/redirect journeys. Share the main browser sequences and content checks with the classic driver. Let target adapters provide authentication, native agents, a reachable service fixture, and owned cleanup. **Reason and benefit** A local pass cannot verify the deployed application. Shared story code lets a deployed target exercise the same behavior without copying an entire test or starting a replacement server. **Breaking changes** No product API or default matrix selection changes. Passed deployment results must satisfy the declared story checks and work-run count. Unsupported target capabilities fail before a story runs. Related public work: #17262 and #18098. Upstream prerequisite: #18230. ## What Changed - Extract shared Plan, artifact download/oracle, protected service, Skill Studio, and Stop/redirect helpers. Keep classic callers on those helpers. - Add five explicit paid journey selections with owned-task observations and native company, agent, model, and environment checks. - Include classic prompts, delivery/decision code, and the decline grader in the source-definition hash. Keep classic transient-read retries and reject deployment binding failures immediately. Match cookie directory boundaries in sensitive fixture requests. - Require complete story checks and expected run receipts before retaining a passing result. Keep failed evidence and private downloads. - Add a secret-safe request method for deployment fixture credentials. Add the missing declaration for the existing manual evidence policy module so the E2E project can typecheck. - Document target responsibilities and promotion after a real deployment pass. This source catalog does not enroll candidates in a daily plan. ## Verification - E2E TypeScript check: passed, including the review follow-up. - Review follow-up: 30 tests passed across four files, covering transient Plan reads, immediate binding rejection, cookie-path scope, agent receipts, and artifact evidence. Stop now waits for a saved, still-active native turn before the browser click and retains its startup events. Counterexamples reject claimed-but-unstarted, already-completed, interrupted, and non-native work. Shared observation and API helpers also enter the definition fingerprint. Cancellation snapshots are captured before lease cleanup. A direct terminal-boundary assertion rejects native work after cancellation even if it is already in the first snapshot; five cleanup-window counterexamples are covered. The Stop-output regression first failed for both native `item.delta` and `item.completed`; the fixed check detects both while allowing accounting-only updates. A report-catalog test first failed because the sandbox denied the tsx child IPC pipe (`listen EPERM`); the unchanged test passed outside the sandbox. - Affected journey, catalog, API, service fixture, and delivery checks: 313 tests passed across 19 files. - Workspace build: passed after rebase. - Workspace typecheck: passed. The default repository run was attempted. Repeated embedded PostgreSQL startup failures prevented local database coverage; it was stopped without claiming a full pass. Linux CI must supply that coverage. - Broad E2E support run: 4,793 passed, 23 failed, 2 skipped. Failures include existing stale image/Pi assertions and process/startup timeouts under host load. The affected tests pass separately. This is not a claim that the broad suite is green. - Upstream CI prerequisites: this branch is rebased on the merged #18230 repair. The earlier conflicting Turbo flags no longer apply. The shell-hook fix is also supplied by that upstream change. Linux shell-hook investigation: 19 focused tests and adapter TypeScript passed. The exact staged-config sequence reproduced CI's 500 in a pinned Node 24 Linux container. Node's default stdin socket at shell level zero selects Bash remote-shell startup and reads `.bashrc` instead of `BASH_ENV`. Ignored stdin and a nested shell each produce 1000. The fixture now ignores stdin; the production hook and 1000 assertion are unchanged. See [Bash startup rules](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files). - GitHub CI on the earlier head passed most suites but failed two browser tests (a fixture cleanup lock timeout and a fixture run staying queued) plus the shell-hook assertion. The lock-timeout case received one retry, which was cancelled by a later source push and does not establish recovery. These failures remain part of the verification record. - Real Cloud qualification: the private target adapter is reviewed and merged. These candidates must pass there before daily-plan promotion. Current source review: Greptile 5/5 and Paperclip Review 5/5 on `a9df1d6e8d880710fce39b87e689e07e1c8e6edc`, with zero unresolved review threads. The full [GitHub CI run](https://github.com/paperclipai/paperclip/actions/runs/40974946081) passed on that exact head, including both required aggregate checks. ## Risks Shared helper changes affect classic callers. The independent ZIP oracle and visible delivery requirements remain in place. Stop requires native acknowledgement, terminal event evidence, resource cleanup, and no later original-turn activity. The managed local target still lacks paid-agent fixtures and reports these journeys as unsupported. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, terminal use, and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 18:58:26 -05:00
DottaandPaperclip	b81b870732	Add QA smoke journeys and OpenCode run traces (#18098) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify saved application behavior across deployments. > - Task creation and questions need reusable smoke selections and exact run attribution. > - Operators also need to inspect the OpenCode execution behind a measured question. > - Company observability does not currently support native OpenCode. > - This change adds the journeys and a bounded projection of normalized OpenCode run events. > - The projection follows the configured capture policy and retains the current run identity across warm reuse. ## Linked Issues or Issue Description Related foundation: #17262. Related alternative: #17989 exports metadata from OpenCode's native OTLP stream. This change uses normalized run events to support the existing content switches and per-run linkage. It does not forward the native stream. The two PRs overlap in launch registration and must be reconciled before both land. **What existing behavior does this improve?** Reusable Product E2E smoke selections and company observability for native OpenCode. **Current behavior** The journey catalog lacks reusable task creation and attached-agent question checks. Native OpenCode is marked unsupported for company observability, so a successful destination connection test does not prove that agent traces will appear. **Proposed behavior** Add explicit task-create and ask-question selections. Verify saved state and exact company, issue, agent, model, environment and successful run attribution. Export normalized OpenCode turns, tools and usage to the configured OTLP destination. Honor each capture switch and redact known secrets. **Reason and benefit** CI can run a small measured smoke selection, retain its screenshots and checks, and link results to traces belonging to the same run. **Breaking changes** Existing selectors remain unchanged. OpenCode export is opt-in through existing company settings. Legacy OpenCode stays unsupported. ## What Changed - Add reusable task creation and question journeys, configured-agent capability, owned-fixture checkpoints and strict evidence validation. - Add an OpenCode-owned projection of normalized run events. No provider SDK or raw payload schema enters heartbeat coordination. - Extend the private launch union and reserved environment allowlist for native OpenCode. With export disabled, no receiver request runs. - Keep destination headers in the parent wrapper and out of the actual provider child environment and configuration. - Bound each turn to 64 tool and 64 usage entries, 8 KiB per text field and a 1 MiB request. Use a three-second request deadline, four concurrent exports, and no retries or idle worker. - Pass the rotated runner authority to each proxy turn. A retained process no longer attributes later turns to its original launch run. - Update settings copy, observability documentation and capability maps. ## Verification - `pnpm -r typecheck` and `pnpm build` passed after the implementation changes. - Adapter launch/redaction: 9 tests passed. Projection and actual proxy: 12 tests passed before the review fix; the expanded projection suite passed all 12 cases after the fix. The proxy test exercises two runs in one process against a real local HTTP receiver and a fake provider. - Full OpenCode driver/projection suite: 72 passed, one existing response_wake test timed out at five seconds. A focused check then passed both existing omitted-continuation cases and the new trace export case. The timeout cause remains unconfirmed. - Runner package TypeScript and Rust `cargo check` passed. Map and UI token checks passed. - Prior journey validation: 273 tests across 12 files and the journey TypeScript check passed. - Broad local `pnpm test:run` encountered native-runtime integration failures with repeated embedded PostgreSQL initialization failures and was stopped. All 16 failing cases passed in an isolated run outside the filesystem sandbox (961 unrelated cases unselected). The broad local run is not a clean acceptance result. A later focused database-backed chat/heartbeat run passed all six selected observability cases, including native OpenCode launch and successful allocation (1,809 unrelated cases were unselected). - All CI gates passed for `21085f880d8babc9efeb22d0623db99613d33093`, including packaged-runner, clean-install, browser and build checks: https://github.com/paperclipai/paperclip/actions/runs/40884658995 . Greptile reviewed the same commit at 5/5 with no unresolved findings. - Verified OTLP JSON acceptance and indexing with an explicitly synthetic transport marker. No live OpenCode model trace or deployed smoke acceptance is claimed yet. ## Risks Export is best effort. Abrupt termination, recovered in-flight turns without a new submitted event, limits or receiver failure can lose traces. Export failure does not change task success. The projection represents observed normalized events, not raw native model spans or hidden reasoning. Known-secret redaction is not an OS isolation boundary. UI changes can break journey locators. Deployment adapters retain credential, ownership and cleanup responsibility. Both the controller and native runner bundle must be updated before live qualification. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution and browser tools. The exact serving model ID and context size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 08:57:12 -05:00
DottaandPaperclip	3bd0d81c7b	fix(e2e): follow proven workspace deferrals during onboarding (#18008) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify the user outcome through the browser and scoped APIs. > - An onboarding child can wait while its parent owns their shared workspace. > - The server records that no provider work started and schedules a durable replacement. > - The journey rejected that expected wait, and feedback counted it as unknown legacy execution. > - This pull request follows the exact replacement and records the distinct no-provider evidence. > - The child must still finish through native execution and produce the independently checked download. ## Linked Issues or Issue Description Refs #17940. **What happened?** The fresh-workspace journey failed with `onboarding_first_task_failed` when a child encountered shared-workspace contention. The cancelled admission had an explicit `workspace_wait` receipt with `providerWorkStarted: false`. Its linked replacement was still scheduled. The runtime had not been selected. The default `legacy` value did not represent a provider execution. **Expected behavior** Keep both rows and their run-limit cost. Follow only the unique replacement for the same company, agent, and issue. A pending replacement must not pass. Real failures and contradictory execution evidence must still fail. **Steps to reproduce** 1. Start a native onboarding parent and accept its child-task proposal. 2. Observe the child admission while the parent still holds their workspace. 3. Present the server's no-provider cancellation and its durable scheduled replacement to the journey and feedback collectors. ## What Changed - Add a shared predicate for the explicit no-provider workspace-deferral receipt. Reject runtime, process, output, usage, and cost evidence that contradicts it. - Follow exact, unique retry lineage during onboarding. Wait for active or scheduled replacements without repeating browser actions. Preserve the campaign deadline, run cap, native artifact attribution, and download oracle. - Retain per-run feedback inspection classifications: `native_events`, `provider_not_started`, or `incomplete`. Require exhaustive event reads. Keep pending work and uncertain execution partial. - Add causal regressions and document the evidence contract. Historical failed attempts keep their original grades. ## Verification - Red: the new journey regression failed with the observed `onboarding_first_task_failed` before the repair. The new feedback cases also failed before the repair. - Green: 70 journey tests and 74 feedback-family tests passed. The final affected journey/feedback gate passed 275 tests plus 137 Node checks. Five retry-lane reset cases failed before the review correction and passed after it. - `pnpm test:e2e:runner:typecheck` passed after the retry-counter correction. - `pnpm -r typecheck` and `pnpm build` passed before the final rebase. - The full local `pnpm test:run` and harness unit gate encountered embedded PostgreSQL startup failures. The harness gate passed 4,574 tests and failed its database-startup test. The final focused journey/feedback gate and all 137 Node checks passed. CI will provide independent full-suite results before merge. - Current-head CI passed 55 checks; two checks skipped and one was neutral. Greptile is 5/5, with zero unresolved review threads. - Live Cloud verification will use the reviewed harness and Operations revisions. This PR does not claim that a fresh Cloud campaign has passed. ## Risks A broad cancellation exemption could hide real failures. The exemption requires the server's explicit receipt, unresolved runtime selection, no contradictory execution evidence, and exact durable lineage. Missing successors remain pending until the existing deadline. A successful native child and valid downloaded bytes remain required. No server behavior, approval policy, or budget policy changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 07:58:41 -05:00
DottaandPaperclip	d40f9ac377	fix: enforce first-task completion decisions and retain E2E evidence (#17940) ... ## Thinking Path > - Paperclip lets people delegate work to agents and approve proposed work. > - The onboarding first task asks for scope, proposes a task, and waits for a human decision. > - An answer to its opening question supplies scope. It does not approve a proposal. > - Task creation already checks the saved proposal decision. > - Native completion used the general completion policy and could mark the onboarding parent Done without that decision. > - This PR checks the trusted decision before completion feedback and again when the native status is committed. > - The first task stays in review until a saved human decision permits closure. Ordinary completion and acknowledged refusal keep their current behavior. > - The Product E2E guard also retains closed scalar evidence so future failures show which boundary condition was observed. ## Linked Issues or Issue Description Refs: #17914. Profile compatibility follow-up to #17922. **What happened?** A native onboarding run can submit a valid low-risk Done result after only the opening answer. The native finalizer then marks the first task Done even though no human has accepted or rejected a proposal. **Expected behavior** The opening answer must not permit completion. A trusted saved proposal decision must permit it. A saved rejection permits acknowledgment and closure, but does not authorize new work. A later pending proposal must keep the task in review. **Steps to reproduce** Create a server-owned onboarding first task with an answered opening question, no proposal decision, and a valid low-risk completion contract. Pass a Done result through native completion feedback, the control-plane port, and the finalizer. Before this fix, the persisted status is Done. The regression expects In review. Ordinary completion and acknowledged rejection pass as controls. ## What Changed - Share the trusted latest-proposal and human-response checks with completion admission. Keep task creation acceptance-only. - Refuse premature native completion feedback. Keep an already saved premature result in review during finalization. - Check the decision again under the existing task and run locks before committing Done. - Cover accepted decisions, saved refusals, forged context, pending or changed proposals, and a real database race. - Retain a closed scalar receipt when the existing E2E pre-acceptance guard fails. Validate and display it without task content, IDs, filenames, or raw rows. Preserve the original failure, blocked dependents, and historical reports. - Include the evidence module in the journey definition hash. Bind the receipt to its exact failure code and check its closed keys against the full type. - Repair base-branch profile drift found by the full harness check: include the active Copilot v66 fixture in image hashing and update stale Pi version/hash expectations from #17922. Keep all older image inputs and strict hash checks. - Update the onboarding feature and agent access maps and E2E evidence documentation. ## Verification - Semantic regression before implementation: one failure and two passing controls on PostgreSQL 18. The failed assertion observed Done instead of In review. - The local database uses an owned disposable container with an immutable image digest. Actual migrations, feedback, control-plane port, finalizer, and status commit run against PostgreSQL. A temporary test adapter replaces only the embedded database transport because this host has exhausted its PostgreSQL semaphores. - Native control-plane suite: 55 tests passed, including 17 new controls. Existing task-creation admission: 25 passed. Existing completion feedback: 19 passed. - Credential-free E2E support: 206 Vitest suites passed, 4,407 tests passed, and two skipped. All 137 Node support checks passed. Harness typecheck passed. - The failure-receipt regression had 11 failed assertions before capture was implemented. Two failure-code binding assertions fail without their gate. Eight placement/binding cases first validate the receipt-free control, then reject the misplaced receipt. - The untouched base reproduced the three profile compatibility failures. The narrow repair passed all 83 affected tests before the full harness run. - Private workflow preparation and credential-free onboarding preview: nine compatibility checks passed. Ownership, recovery, evidence, shared-contract and workflow compatibility: 77 passed. The actual provider preview also confirms immediate success deletion, earlier declared failure eligibility, limits, and zero retries without credential access. Map checks and git diff checks passed. - Required repository typecheck passed for the product fix. The combined branch build passed. The local full repository suite was stopped after repeated PostgreSQL startup failures; a separate initdb probe confirmed host SysV semaphore exhaustion. No shared IPC resources were changed. [Hosted CI](https://github.com/paperclipai/paperclip/actions/runs/40591020873) passed the full database suites, typecheck, build, runner checks, serialized suites, and browser shards on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b` (55 success, two expected skips, one advisory neutral). - Independent source review approved the product guard, atomic race regression, evidence contract, and maps with no actionable findings. Both automated reviews are 5/5 on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b`; prior findings are fixed and resolved. Final CI is green and the PR is conflict-free. - No Cloud workspace, provider call, release publication, or deployment is part of this verification. These tests do not identify a historical Cloud failure or qualify live onboarding. - Local dependency installation uses the existing CI-generated lockfile from #17811. No lockfile change is included. ## Risks - Existing onboarding runs without a trusted saved proposal decision remain in review even when the agent reports success. This is the intended approval boundary. - Decision changes can race completion. The status commit checks the decision under its existing locks and uses the existing status-race recovery path. - A saved refusal permits closure only. Creation admission still rejects it. - Evidence readers reject malformed receipts and unrelated failure codes. The version-1 key registry is checked against every receipt field at compile time. Existing attempts do not gain evidence after the fact. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), ultra reasoning, through Codex. Used repository inspection, code execution, and collaborating coding agents. The runtime did not report a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 17:07:50 -05:00
DottaandPaperclip	a967d479bd	fix(e2e): retain verified onboarding artifact evidence (#17370) ... Retain the bounded verified onboarding download and independent oracle in private normalized results before cleanup. Preserve the receipt when later readback or trace checks fail without marking the journey as passed. Refs #17262. Refs #17253. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-23 14:23:08 -07:00
DottaandPaperclip	5ef385f7e6	Run reusable Product E2E journeys across deployment targets (#17262) ... ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description **Subsystem affected** Product E2E harness, fixture registry, and reports. **Problem or motivation** Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. **Proposed solution** Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. **Roadmap alignment** This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 06:27:49 -07:00