test(e2e): reuse core journeys across deployment targets (#18222)

## Thinking Path

> - Paperclip manages agents and their work.
> - Product E2E tests verify work through the browser and public APIs.
> - The classic matrix already tests several common task stories.
> - Its older stories depend on local fixture setup and workspace
access.
> - Deployment tests need the same story checks with a separate target
adapter.
> - This PR shares those checks and adds five explicit journey
selections.
> - A deployment pass still requires evidence from that deployment.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Product E2E journey reuse across managed instances and deployed Cloud
instances.

**Current behavior**

The deployed-target catalog covers task creation, assignment, Ask, and
fresh onboarding. Plan revision and Everyday stories run through the
classic fixture driver. Several story steps assume a local workspace.

**Proposed behavior**

Add explicit Plan revision, artifact revision, protected-action approval
and decline, Skill Studio, and Stop/redirect journeys. Share the main
browser sequences and content checks with the classic driver. Let target
adapters provide authentication, native agents, a reachable service
fixture, and owned cleanup.

**Reason and benefit**

A local pass cannot verify the deployed application. Shared story code
lets a deployed target exercise the same behavior without copying an
entire test or starting a replacement server.

**Breaking changes**

No product API or default matrix selection changes. Passed deployment
results must satisfy the declared story checks and work-run count.
Unsupported target capabilities fail before a story runs.

Related public work: #17262 and #18098. Upstream prerequisite: #18230.

## What Changed

- Extract shared Plan, artifact download/oracle, protected service,
Skill Studio, and Stop/redirect helpers. Keep classic callers on those
helpers.
- Add five explicit paid journey selections with owned-task observations
and native company, agent, model, and environment checks.
- Include classic prompts, delivery/decision code, and the decline
grader in the source-definition hash. Keep classic transient-read
retries and reject deployment binding failures immediately. Match cookie
directory boundaries in sensitive fixture requests.
- Require complete story checks and expected run receipts before
retaining a passing result. Keep failed evidence and private downloads.
- Add a secret-safe request method for deployment fixture credentials.
Add the missing declaration for the existing manual evidence policy
module so the E2E project can typecheck.
- Document target responsibilities and promotion after a real deployment
pass. This source catalog does not enroll candidates in a daily plan.

## Verification

- E2E TypeScript check: passed, including the review follow-up.
- Review follow-up: 30 tests passed across four files, covering
transient Plan reads, immediate binding rejection, cookie-path scope,
agent receipts, and artifact evidence. Stop now waits for a saved,
still-active native turn before the browser click and retains its
startup events. Counterexamples reject claimed-but-unstarted,
already-completed, interrupted, and non-native work. Shared observation
and API helpers also enter the definition fingerprint. Cancellation
snapshots are captured before lease cleanup. A direct terminal-boundary
assertion rejects native work after cancellation even if it is already
in the first snapshot; five cleanup-window counterexamples are covered.
The Stop-output regression first failed for both native `item.delta` and
`item.completed`; the fixed check detects both while allowing
accounting-only updates. A report-catalog test first failed because the
sandbox denied the tsx child IPC pipe (`listen EPERM`); the unchanged
test passed outside the sandbox.
- Affected journey, catalog, API, service fixture, and delivery checks:
313 tests passed across 19 files.
- Workspace build: passed after rebase.
- Workspace typecheck: passed. The default repository run was attempted.
Repeated embedded PostgreSQL startup failures prevented local database
coverage; it was stopped without claiming a full pass. Linux CI must
supply that coverage.
- Broad E2E support run: 4,793 passed, 23 failed, 2 skipped. Failures
include existing stale image/Pi assertions and process/startup timeouts
under host load. The affected tests pass separately. This is not a claim
that the broad suite is green.
- Upstream CI prerequisites: this branch is rebased on the merged #18230
repair. The earlier conflicting Turbo flags no longer apply. The
shell-hook fix is also supplied by that upstream change. Linux
shell-hook investigation: 19 focused tests and adapter TypeScript
passed. The exact staged-config sequence reproduced CI's 500 in a pinned
Node 24 Linux container. Node's default stdin socket at shell level zero
selects Bash remote-shell startup and reads `.bashrc` instead of
`BASH_ENV`. Ignored stdin and a nested shell each produce 1000. The
fixture now ignores stdin; the production hook and 1000 assertion are
unchanged. See [Bash startup
rules](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files).
- GitHub CI on the earlier head passed most suites but failed two
browser tests (a fixture cleanup lock timeout and a fixture run staying
queued) plus the shell-hook assertion. The lock-timeout case received
one retry, which was cancelled by a later source push and does not
establish recovery. These failures remain part of the verification
record.
- Real Cloud qualification: the private target adapter is reviewed and
merged. These candidates must pass there before daily-plan promotion.

Current source review: Greptile 5/5 and Paperclip Review 5/5 on
`a9df1d6e8d880710fce39b87e689e07e1c8e6edc`, with zero unresolved review
threads. The full [GitHub CI
run](https://github.com/paperclipai/paperclip/actions/runs/40974946081)
passed on that exact head, including both required aggregate checks.

## Risks

Shared helper changes affect classic callers. The independent ZIP oracle
and visible delivery requirements remain in place. Stop requires native
acknowledgement, terminal event evidence, resource cleanup, and no later
original-turn activity. The managed local target still lacks paid-agent
fixtures and reports these journeys as unsupported.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, terminal use, and code
execution. The exact serving model ID and context window are not exposed
to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes

- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-29 18:58:26 -05:00
1 parent fe3e0f2d5f
commit d3f2a60e79
30 files changed
+1106 -387

No files matched your search

@@ -0,0 +1,41 @@
# Five additional daily Cloud QA journeys
The existing daily QA plan runs three deployment-targeted Product E2E journeys.
Extend it with the five workflows requested in this chat, qualifying every one
against the registered testing Cloud QA instance before promotion. GitHub hosts
the browser driver; the application and native agents remain on Cloud/Daytona.
## Acceptance
| Journey | Required live evidence |
| --- | --- |
| Plan, revise, approve, complete | Persisted canonical Plan, a distinct revision-bound replacement confirmation, browser revision and approval, no premature completion, final native completion. |
| Deliver and revise an artifact | Browser downloads of both versions, independent content validation, different hashes, original download still available. |
| Approve and decline protected actions | Authenticated controlled service, zero accepted invocations before approval, exactly one after approval, zero after decline, saved UI decisions and truthful responses. |
| Create and edit a company skill | Native skill-creation receipt, persisted owned skill, activity card, browser Skill Studio edit, reload and task readback. |
| Stop and redirect | Observed active native work, browser Stop, authoritative retirement of the original work, one saved new-direction message, correct new response, no old work continuing. |
## Ownership and delivery
- Shared Product journey code owns user actions, prompts, assertions and evidence.
- Target adapters own authentication, registered target identity, independent
fixtures, native agent/connection setup, run accounting and owned cleanup.
- The private Cloud QA workflow owns TestOps selection, upload, reports and Slack.
- Reuse the existing classic story logic and graders where applicable; keep
deployment-specific setup outside shared stories. Keep local and Cloud target
selection explicit and fail closed on missing capabilities or evidence.
- Retain unsuccessful attempts. Product grades, infrastructure failures,
cleanup, usage coverage and trace delivery remain distinct observations.
- Promote each new case to the saved daily TestOps plan only after its real
Cloud pass, independent evidence inspection and successful cleanup.
- Completion requires all five new journeys passing and present in the daily
plan, the existing three controls still passing, passing applicable repository
checks/reviews, and a real combined CI/TestOps/report/Slack verification.
## Feedback path
Calibrate assertions and selectors without paid work first. Qualify a narrow
Cloud case, inspect its retained evidence, then add the other cases. Prefer the
existing protected QA credential and workflow paths; do not weaken source review
or secret boundaries to run candidate harness code. Record configuration,
source revisions, target deployment identity, duration and measured usage.