Commit Graph
6 Commits
Author SHA1 Message Date
DottaandPaperclip d9354bbfc3 Add a portable agent question and answer resumption journey (#18346)
## Thinking Path

> - Paperclip helps people manage agent work and keep human decisions
attached to tasks.
> - Product E2E journeys verify the browser, native agent, saved state,
and cleanup.
> - The classic suite already checks a choice question followed by a
text question.
> - Deployment-targeted journeys need the same behavior on an attached
instance.
> - Production questions can use provider input, the dedicated semantic
tool, or the authenticated native API fallback.
> - The basic journey must verify the original card and native
continuation for each supported path.
> - This pull request shares the scenario and browser actions, adds
strict API receipt evidence, and verifies the saved document after
reload.
> - Separate tool-documentation qualification and the original failed
measurements keep their existing scope and outcomes.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

It adds portable coverage for answering agent questions and resuming the
same task on an attached deployment.

**Current behavior**

The classic suite has the two-question scenario. The portable catalog
does not expose it. Its shared native-path grader recognizes provider
and dedicated semantic receipts, but rejects a valid completed native
API creation receipt.

**Proposed behavior**

Expose `question-answer-resume` as a deployment-targeted journey. Answer
one choice question and one text question in the browser. Require
server-owned creation evidence, unchanged forms, exact board answers,
bound continuation, an agent-authored task document, and readback after
reload.

**Reason and benefit**

The same story can qualify local and hosted targets through their normal
provider setup and cleanup. It catches lost answers, duplicate or
foreign work, premature output, and stale documents.

## What Changed

- Register the journey with a three-run maximum and positive model
allowance. Accept one to three attributed work runs at both collection
and retained-result boundaries.
- Share the existing two-question scenario, browser answer actions, and
behavioral grader. Keep separate tool-documentation grading unchanged.
- Recognize a completed native API creation receipt only when its HTTP
201 result retains the original pending card, exact
task/company/agent/source run, idempotency key, and full normalized
form. Require the receipt to remain unchanged after the answer and
through the final checkpoint. Missing, pending, foreign, or altered
proof still fails.
- Check signed-in board ownership, each native continuation, no
premature or extra output, document authorship, and the saved revision
after reload.
- Wait for the authenticated product region before screenshots. Recheck
origin and path after readiness. Bound readiness and capture by the
campaign deadline.
- Add closed stage-specific errors, catalog coverage, sanitized receipt
calibration, and documentation. Include the provider-form and
browser-question presentation helpers in the definition fingerprint.

## Verification

- Fresh QA staging qualification passed at
`7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks
passed, cleanup passed, and all three native runs used the configured QA
target and model. The [sanitized run
report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/)
records the measured pass. This is one successful qualification, not a
reliability claim.
- Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3`
contains the one main merge through
`fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of
the four-file qualified search fix
`4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity
matches. It preserves the project and search registrations and atomic
search readback. The question live qualification ran at the earlier SHA
above; the final integration head has separate local and CI
verification.
- Eighteen question journey, helper, and test file blobs are identical
between the qualified and integration SHAs. This includes clarification
flow and evidence, question grader and browser helper, the sanitized
receipt fixture, screenshot helper, provider-form helper, browser
question presentation, continuation scenario/readiness/flow, shared
agent validation, and user actions. The question run-count contract and
dispatch branch are also unchanged. Reviewers can compare those paths
directly between the two commits.
- All 622 selected Product E2E, catalog, question-resumption,
necessary-question, report, and readiness tests pass after the main
merge. After the exact search delta, all 145 narrow
question/search/project/contract tests and all nine real Chromium search
regressions pass. Map checks, combined credential-free preview, source
fingerprints, and the earlier workspace-link preflight pass. The local
TypeScript attempt reports stale Runner declaration errors for
`onAccountingEvent` in unchanged native-session code; fresh official CI
provides the clean-build type gate. The three search source/test blobs
match the qualified search merge, and the three project source blobs are
unchanged.
- An in-memory probe executes the actual launcher fingerprint
expression. Each project, search, and question implementation input
changes the digest. Separate red probes first reproduced the omitted
provider-form and browser presentation helpers; both pass after their
source inputs are added.
- The sanitized retained native API response-shape regression fails
before the grading correction and passes afterward. Negative controls
reject missing, pending, foreign, changed, spoofed, or wrongly bound
receipts, answers, continuation, model, environment, and document
evidence. All nine provider/semantic/API path combinations pass
calibration.
- The real local Chromium screenshot probe fails before the readiness
repair and passes afterward. Login redirects and sign-in pages at task
URLs remain excluded with no screenshot files.
- The two original attached failures remain retained: the first stopped
at early screenshot capture; the second stopped before answering at the
former API provenance gate. An earlier sandbox run also retained one
report subprocess failure; the full selected tests then passed with
local subprocess IPC access.
- The prior integration CI had an unchanged canary mock `EADDRINUSE`
failure. That history is retained; its documented retry is owned by the
coordinator. No application API, schema, production UI, model, runtime
policy, credential, or deployment configuration change is included.
Final integration head ae0d9d2898 passed
all 58 CI checks, the external Snyk status, and fresh Greptile 5/5 with
no unresolved findings. This clean CI includes the official TypeScript
and build gates. Recurring selection follows merge and a separate normal
TestOps job verification.

## Risks

An API result must prove native creation; form shape and run count
cannot establish it. The grader checks the server receipt, exact scope,
unchanged card and receipt, real board answers, and bound native
continuation. The shared answer helper still serves classic journeys.
The screenshot wait affects attached journeys and remains limited to the
authenticated product region and campaign deadline. One live QA pass
establishes the measured journey outcome at the qualified SHA. It does
not establish reliability across models or deployments. The integration
SHA preserves its journey and helper blobs, while shared registry
changes receive fresh CI and review.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, source inspection, and code
execution. The host does not expose the exact underlying model version
or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 16:18:21 -05:00
DottaandPaperclip 81ebe479e8 fix(evals): bind rendered Markdown replies to saved run evidence (#18264)
## Thinking Path

> - Paperclip lets people assign work to AI agents and read saved
results.
> - Product journeys must prove that a result is visible and belongs to
the completed run.
> - The shared observer compares stored Markdown with browser text.
> - A document link can render as an issue chip, so those strings
differ.
> - This PR keeps run and comment attribution, then compares full text
and exact link destinations.
> - The benefit is a valid browser result without a false timeout or a
weaker provenance check.

## Linked Issues or Issue Description

Refs: #18248, #18255.

**What happened?**

The protected-action story completed both decisions and all four native
turns. Its final explanation contained a document link. The browser
rendered that link as an issue chip. The observer waited for raw
Markdown to equal rendered text, so it could not accept the valid
result.

**Expected behavior**

Accept complete matching content from the final run after reload. Keep
the exact link destination, task anchor, author, run and durable
final-message checks. Reject missing or changed content.

**Steps to reproduce**

1. Save an attributed reply with bold text and a Markdown document link.
2. Render the link as an issue chip in the task bubble.
3. Run `readBoundTaskFinalReply`. The original observer times out. The
new browser regression preserves that failure.

## What Changed

- Add a UI-owned content projection using its existing Markdown renderer
and GFM/issue-reference plugins. Keep DOM observation, attribution and
destination checks in the task-reply observer. Keep source Markdown and
observed text separately.
- Read visible DOM text, link labels and destinations. Require the
structured projection to match the actual visible text. Allow only
recognized product chips to replace a link label. Ordinary links keep
their source label. Reuse the production issue-reference and
company-route helpers, plus the current task URL. Ordinary root-relative
anchors keep their original path. Preserve exact marker checks and all
run/comment/native-event attribution checks.
- Check visible decline explanations and unread data in preserved native
source text.
- Add browser regressions and source/content tests. Reuse the UI’s
existing `react-markdown` and `remark-gfm` dependencies. The final diff
changes no package manifest or lockfile.
- Match generated GFM checkbox states and reject changed states.
Document supported content and fail-closed behavior.

## Verification

- Original observer: the decorated-link browser regression fails at the
final-reply predicate.
- Fixed observer: all 21 browser regressions pass, without credentials
or model calls.
- Focused observer, decline, content and journey contract suites: 64
tests pass.
- E2E and UI TypeScript checks pass. All UI token gates pass. The
unchanged frozen install passes.
- Full CI passed on the preceding 6eba7ff head (41003112166). Full CI
passed on the preceding b9a2571 head (41006205475), with both reviewers
at 5/5. Current-head full CI passed at
b3d2d08305 (41009669966), including
typecheck, build and all eight E2E shards. Both reviewers scored this
commit 5/5, with no unresolved threads. The initial lockfile policy
failure was corrected. The final diff has no manifest or lock changes,
and its unchanged frozen install passed.
- The optional positive GFM strikethrough assertion also passes.
- The Node 24/tsx end-to-end observer probe passes. The callback has no
external compiler helper; a browser regression covers actual tsx
serialization.
- The Paperclip review finding is also fixed: bare URLs and issue keys
use the production plugins and the UI’s `scope=accessible`
company-prefix context, with RED/GREEN browser coverage. Source Markdown
is required in the receipt.
- All Greptile findings are fixed, including matching the UI’s
accessible-company scope: ordinary link labels, issue-prefix/fragment
resolution and ordinary root-relative paths have positive and negative
browser coverage.
- The deployed attempt remains a failed original measurement. Fresh
qualification is required after this repair.

## Risks

The comparison normalizes whitespace and inline formatting. It verifies
full text and exact link targets, while link labels may become product
chips. Images, raw HTML, tables and fenced code fail closed. Those
constructs may require a future explicit extension if a journey uses
them. No application behavior, model, budget, approval or connection
permission changes.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, code execution and browser
tools. The host does not expose an exact underlying model version or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal issue
identifier
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 00:57:59 -05:00
DottaandPaperclip b3c2172d41 fix(evals): verify native protected-action outcomes (#18255)
## Thinking Path

> - Paperclip manages work assigned to agents.
> - Product E2E stories ask for useful user-facing results.
> - Classic runner and deployed QA reuse the same service
approve/decline task prompts.
> - The service returns a receipt marker that belongs in a saved
briefing document.
> - State that destination explicitly and ask for the document link when
complete.
> - The deployed decline path also needs the same durable final-reply
proof as Plan and Stop.

## Linked Issues or Issue Description

Refs #18222. Builds on the merged reply observer in #18248.

**What happened?**
The shared service tasks ask for titles and a “verification code”
without specifying that the code belongs in the document or asking for
its final link. A native QA attempt created its briefing but then
repeatedly failed the completion tool’s array validation. Those invalid
arguments remain a separate observed failure; the wording change does
not establish their cause or relax validation. Source inspection also
found that the declined-action oracle requires a projected comment even
when the settled native thread renders a durable final message.

**Expected behavior**
Ask for page titles and the service receipt code in the saved briefing,
then a document link. After a declined action, require the existing
explanation-only fallback without another service call.

**Steps to reproduce**
Run the shared service-approve/service-decline stories through the
classic runner or QA Product E2E adapter.

## What Changed

- Clarify the natural-language task for both shared service stories: put
page titles and service receipt code in the document and provide its
link.
- Use the reloaded run-bound final reply for QA service declines. Pass
an explicit native receipt to the existing decline oracle; do not
synthesize issue comments. Require the final native run to start after
the saved decline and keep exact decision, truthful explanation, no
unread marker, zero calls and unchanged connection checks.
- Keep classic callers on their existing attributed-comment path. No
internal tool instructions are added to the user prompt; ownership,
service data and model/runtime limits stay strict.

## Verification

- Decline regression observed RED: two cases reject a valid saved native
explanation without projected comments. After the repair, 79 focused
tests passed across native reply, decline guidance, agent evidence and
journey contracts. A two-successful-run control rejects the earlier
receipt and accepts the later one. Negatives cover stale/wrong run,
wrong agent/task, missing final proof, summary/unsettled/hidden output,
false explanations and unread service data.
- Product E2E TypeScript and `git diff --check` passed.
- Live protected-action qualification is required after merging this and
the companion private service/cleanup fixes. The previous failed result
remains failed; this is a bounded task-clarity experiment, not a claimed
model or completion-schema repair.
- Current-head Greptile and Paperclip reviews are 5/5 with zero open
threads. CI initially failed the unrelated agent-chat-projects cleanup
POST with a PostgreSQL lock timeout after its assertions completed. One
bounded retry passed on unchanged source; all current-head gates passed
at `64bc169e147290ed4dbaa18221e0268362ce81ee`. The original failing
attempt and its logs remain available. The task-clarity change is still
a bounded experiment; the observer repair addresses a demonstrated
storage-assumption defect.

## Risks

A clearer task can still fail with the selected model. All acceptance
checks and completion input validation remain strict; a saved briefing
alone cannot qualify the complete journey. Only test prompts and test
evidence code change. Missing or contradictory native evidence fails
closed.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and context
window are not exposed by this session. Used reasoning, source editing,
shell execution and test inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 22:14:37 -05:00
DottaandPaperclip 2badf8a4a5 fix(evals): verify native QA replies on the reloaded task thread (#18248)
## Thinking Path

> - Paperclip manages AI agents that do work for users.
> - Product E2E journeys check user outcomes on real deployments.
> - Native task threads can render saved final replies from a settled
run.
> - The new QA Plan and Stop checks looked only for agent comments.
> - Both flows reached the requested terminal state and showed the
correct reply, but that storage assumption rejected them.
> - This PR checks the reloaded visible reply and binds it to the final
successful run. Explicit presentation decisions remain authoritative.

## Linked Issues or Issue Description

Refs #18222 and #18244.

**What happened?**

The deployed Plan journey completed all three native runs, including
revision and approval. Stop completed native cancellation, resource
release and a new direction. Their final-answer checks failed because
the comments endpoint did not contain the rendered native reply. The
original failed grades remain unchanged.

**Expected behavior**

Require exactly one correct visible final answer after reload,
attributed to the final successful owned run. Reject an earlier run,
progress text, hidden content, duplicate answers or a cancelled run. If
its selected completion comment is projected, require that comment and
matching author/run/text. Otherwise require a single durable final
assistant message and its settled native final-response anchor.

**Steps to reproduce**

Run the Plan or Stop journey on a native deployment whose task thread
renders the settled run before a completion comment is projected.

## What Changed

- Add a narrow reply observer over the existing task-thread hooks. It
reloads the page and refreshes saved run/comment metadata while waiting.
- Bind native replies to a validated durable final assistant event and
the successful run's settled final-response anchor. Require the selected
comment when projected; support pending or suppressed projection without
accepting a summary fallback. Accept payload and item-normalized finals,
rejecting contradictory fields.
- Use the observer for Plan completion and Stop's new direction. Keep
existing run counts, deployment binding, cancellation and artifact
checks.
- Add negative grading tests, include the helper in the definition hash,
and document its evidence contract. Classic protocol comment matchers
are unchanged.

## Verification

- 63 focused tests passed across native reply evidence, agent evidence
and journey contracts.
- Product E2E TypeScript passed. The reply observer uses the remaining
journey deadline and backs off polling.
- Live Plan and Stop attempts exposed this grading failure. Their
original outcomes remain failed; fresh qualification is required after
merge.
- Full current-head CI passed at
`40840ed742977853630d733292461ad7003fc7fb`; Greptile and Paperclip
Review both scored 5/5, with zero unresolved review threads.

## Risks

The observer depends on existing task-thread test hooks and run anchors.
Missing or changed hooks fail closed. It reads rendered user-facing
text; a finish summary alone cannot satisfy it. Native event provenance,
run ownership and exact reply equality are required. No backend, model,
provider, budget or cancellation policy changes are included.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and context
window are not exposed by this session. Used reasoning, source editing,
shell execution and test inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 21:35:15 -05:00
DottaandPaperclip 8941db9ce6 fix(evals): accept editor escapes and cancel sensitive QA requests (#18244)
## Thinking Path

> - Paperclip manages AI agents that do work for users.
> - Product E2E journeys check real browser actions and saved results.
> - A rich editor can store plain underscores as escaped Markdown.
> - The QA reply check treated that valid saved message as a failure.
> - A private target adapter also needs to cancel credential-safe HTTP
requests before cleanup.
> - This PR fixes reply comparison and adds abort and timeout options to
the existing sensitive transport.

## Linked Issues or Issue Description

Refs #18222.

**What happened?**

The artifact revision and Stop journeys rejected the correct single user
reply because its underscore was stored as `\_`. Sensitive connection
setup also lacked a way to pass its lifecycle signal to the actual HTTP
request.

**Expected behavior**

The check accepts the editor's underscore escape and still rejects
changed, duplicate, missing, or foreign replies. A cancelled sensitive
request closes before setup cleanup proceeds.

**Steps to reproduce**

Run an artifact revision with an underscore in the reply. For the
transport case, abort a pending sensitive request or let its deadline
expire.

## What Changed

- Compare the fixture reply after undoing only underscore escaping. Keep
exact text, count, and author checks.
- Let `ConnectionApi.postSensitive` accept a bounded timeout and abort
signal. Check cancellation again after cookie loading and before
dispatch.
- Test the retained reply failure and both cancellation paths with an
actual loopback HTTP server. Document the adapter contract.

## Verification

- Focused Product E2E tests: 30 passed across two files.
- Product E2E TypeScript check passed. After rebase, a final check first
found four diagnostics from stale runner/plugin SDK declarations;
rebuilding those two workspace packages cleared all four, and the
unchanged E2E check passed.
- The first sandbox run failed because loopback `listen` returned
`EPERM`. The unchanged HTTP tests passed with loopback permission. The
test now reports listener errors directly.
- Both independent reviewers found a possible pre-arrival deadline-test
hang. The follow-up races arrival against request settlement and adds
one second of deadline headroom. All 30 affected tests passed after that
fix.
- Full current-head CI passed in run 40982914525. Greptile and Paperclip
Review both scored 5/5 at `f0526f5fbbf13de5ea43e868eb012069ca3d8907`;
all review threads are resolved. Live QA requalification follows the
merged public and private fixes; no live pass is claimed here.

## Risks

The text normalization applies only to plain fixture replies and only
removes underscore escapes. It does not remove arbitrary Markdown.
Sensitive calls retain their existing 30-second default and closed error
messages. API wrappers must forward the optional signal and await the
actual request.

## Model Used

OpenAI Codex, GPT-6 family. The serving model ID and context window are
not exposed by this session. Used reasoning, code editing, shell
execution, and test inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 20:14:56 -05:00
DottaandPaperclip d3f2a60e79 test(e2e): reuse core journeys across deployment targets (#18222)
## Thinking Path

> - Paperclip manages agents and their work.
> - Product E2E tests verify work through the browser and public APIs.
> - The classic matrix already tests several common task stories.
> - Its older stories depend on local fixture setup and workspace
access.
> - Deployment tests need the same story checks with a separate target
adapter.
> - This PR shares those checks and adds five explicit journey
selections.
> - A deployment pass still requires evidence from that deployment.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Product E2E journey reuse across managed instances and deployed Cloud
instances.

**Current behavior**

The deployed-target catalog covers task creation, assignment, Ask, and
fresh onboarding. Plan revision and Everyday stories run through the
classic fixture driver. Several story steps assume a local workspace.

**Proposed behavior**

Add explicit Plan revision, artifact revision, protected-action approval
and decline, Skill Studio, and Stop/redirect journeys. Share the main
browser sequences and content checks with the classic driver. Let target
adapters provide authentication, native agents, a reachable service
fixture, and owned cleanup.

**Reason and benefit**

A local pass cannot verify the deployed application. Shared story code
lets a deployed target exercise the same behavior without copying an
entire test or starting a replacement server.

**Breaking changes**

No product API or default matrix selection changes. Passed deployment
results must satisfy the declared story checks and work-run count.
Unsupported target capabilities fail before a story runs.

Related public work: #17262 and #18098. Upstream prerequisite: #18230.

## What Changed

- Extract shared Plan, artifact download/oracle, protected service,
Skill Studio, and Stop/redirect helpers. Keep classic callers on those
helpers.
- Add five explicit paid journey selections with owned-task observations
and native company, agent, model, and environment checks.
- Include classic prompts, delivery/decision code, and the decline
grader in the source-definition hash. Keep classic transient-read
retries and reject deployment binding failures immediately. Match cookie
directory boundaries in sensitive fixture requests.
- Require complete story checks and expected run receipts before
retaining a passing result. Keep failed evidence and private downloads.
- Add a secret-safe request method for deployment fixture credentials.
Add the missing declaration for the existing manual evidence policy
module so the E2E project can typecheck.
- Document target responsibilities and promotion after a real deployment
pass. This source catalog does not enroll candidates in a daily plan.

## Verification

- E2E TypeScript check: passed, including the review follow-up.
- Review follow-up: 30 tests passed across four files, covering
transient Plan reads, immediate binding rejection, cookie-path scope,
agent receipts, and artifact evidence. Stop now waits for a saved,
still-active native turn before the browser click and retains its
startup events. Counterexamples reject claimed-but-unstarted,
already-completed, interrupted, and non-native work. Shared observation
and API helpers also enter the definition fingerprint. Cancellation
snapshots are captured before lease cleanup. A direct terminal-boundary
assertion rejects native work after cancellation even if it is already
in the first snapshot; five cleanup-window counterexamples are covered.
The Stop-output regression first failed for both native `item.delta` and
`item.completed`; the fixed check detects both while allowing
accounting-only updates. A report-catalog test first failed because the
sandbox denied the tsx child IPC pipe (`listen EPERM`); the unchanged
test passed outside the sandbox.
- Affected journey, catalog, API, service fixture, and delivery checks:
313 tests passed across 19 files.
- Workspace build: passed after rebase.
- Workspace typecheck: passed. The default repository run was attempted.
Repeated embedded PostgreSQL startup failures prevented local database
coverage; it was stopped without claiming a full pass. Linux CI must
supply that coverage.
- Broad E2E support run: 4,793 passed, 23 failed, 2 skipped. Failures
include existing stale image/Pi assertions and process/startup timeouts
under host load. The affected tests pass separately. This is not a claim
that the broad suite is green.
- Upstream CI prerequisites: this branch is rebased on the merged #18230
repair. The earlier conflicting Turbo flags no longer apply. The
shell-hook fix is also supplied by that upstream change. Linux
shell-hook investigation: 19 focused tests and adapter TypeScript
passed. The exact staged-config sequence reproduced CI's 500 in a pinned
Node 24 Linux container. Node's default stdin socket at shell level zero
selects Bash remote-shell startup and reads `.bashrc` instead of
`BASH_ENV`. Ignored stdin and a nested shell each produce 1000. The
fixture now ignores stdin; the production hook and 1000 assertion are
unchanged. See [Bash startup
rules](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files).
- GitHub CI on the earlier head passed most suites but failed two
browser tests (a fixture cleanup lock timeout and a fixture run staying
queued) plus the shell-hook assertion. The lock-timeout case received
one retry, which was cancelled by a later source push and does not
establish recovery. These failures remain part of the verification
record.
- Real Cloud qualification: the private target adapter is reviewed and
merged. These candidates must pass there before daily-plan promotion.

Current source review: Greptile 5/5 and Paperclip Review 5/5 on
`a9df1d6e8d880710fce39b87e689e07e1c8e6edc`, with zero unresolved review
threads. The full [GitHub CI
run](https://github.com/paperclipai/paperclip/actions/runs/40974946081)
passed on that exact head, including both required aggregate checks.

## Risks

Shared helper changes affect classic callers. The independent ZIP oracle
and visible delivery requirements remain in place. Stop requires native
acknowledgement, terminal event evidence, resource cleanup, and no later
original-turn activity. The managed local target still lacks paid-agent
fixtures and reports these journeys as unsupported.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, terminal use, and code
execution. The exact serving model ID and context window are not exposed
to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes

- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 18:58:26 -05:00