Files
PaperClipAI/maps
DottaandPaperclip 1353a232f2 test(e2e): add a portable active follow-up journey (#18345)
## Thinking Path

> - Paperclip lets people direct agents while their work is active.
> - Product E2E checks must prove that saved user input reaches the
intended work.
> - The existing active chat story depends on a file in a local agent
workspace.
> - That setup cannot qualify an attached deployment with remote agent
execution.
> - This change adds one shared active follow-up journey through the
normal task UI and API.
> - Independent native operation, saved input, final reply, and cleanup
checks make the result inspectable across targets.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Product E2E coverage for a user follow-up sent while a native agent is
working.

**Current behavior**

The classic chat story holds work open with a local file. The shared
deployment-targeted harness has no active follow-up journey. A running
heartbeat alone cannot prove that work was active when input arrived.

**Proposed behavior**

Run one bounded command through the real native provider. Observe its
active execution before and after one follow-up is saved. Require one
final reply with the late requirement, correct task and agent identity,
an empty input queue, and visible readback after reload. Use the same
story on managed and attached targets.

## What Changed

- Add a shared `followup-while-running` journey with normal task
creation and reply submission.
- Require a bound native turn and pending shell execution at the input
boundary. Retain a closed receipt with source identity and a row hash.
- Accept one steered work run or one normal successor (1–2 native runs).
Preserve every other journey's declared run contract.
- Reuse the saved native final reply observer, run binding checks, usage
collection, target lifecycle and private evidence path.
- Accept canonical builtin/process execution events independent of
provider tool display names; calibrate against real OpenCode, Codex and
ACP mappers. Reject Paperclip semantic tool names across native
wrappers/prefixes and unnamed or ambiguous builtin calls.
- Recognize only the observed legacy PRP v1
builtin/bash/unknown/readOnly=false command shape and fail promptly when
owned work finishes before a live boundary can be observed.
- Ready the task page/composer before the observed command boundary, use
a bounded 90-second command, and preserve active-after-persistence
proof.
- Add adversarial oracle and retained-result calibration. Register the
journey, include its sources in the definition hash, and document its
coverage.
- Keep six real browser document-readiness checks and verify
shared-deadline behavior with a deterministic clock/state test that
rejects timeout-reset mutations.

## Verification

Exact current head `e72e20ad111bd94728d38606015d1fdac3c92929` passed
[full CI run
41142552195](https://github.com/paperclipai/paperclip/actions/runs/41142552195):
all 49 workflow jobs reached accepted terminal results, including
typecheck, build, every test lane and aggregate verify. All 59 reported
checks are terminal: 55 success, three expected skips and one neutral
5/5 Paperclip review. Snyk passed. [Fresh Greptile
review](https://github.com/paperclipai/paperclip/pull/18345#issuecomment-6528913235)
explicitly names this exact head and gives 5/5. No review threads remain
open. The PR is mergeable.

These source gates do not establish a live QA pass at this head. Fresh
protected QA at this exact source and the separately recorded deployed
application/runtime remains required before primary merge and daily
admission.

Historical live QA staging **PASS** on qualified commit
`af9e091b15b5e24b6b2cc1b424523430cc2ab024`: 6/6 checks, all five phases
including cleanup, two native work runs, and the configured
model/environment check passed. The [sanitized qualification
report](https://pages.paperclip.ing/evals-qa-smoke-41122620146-1/)
records that exact measurement.

Final integration head `e72e20ad111bd94728d38606015d1fdac3c92929` merges
exact actual squash `18b1637b6371a6a6adade857e2a73654ee626fef` from
[#18254](https://github.com/paperclipai/paperclip/pull/18254) into
previous head `e54d64743964b1a19d165537bebbe62c50389a23`. The merge had
no conflicts. All 18 files in this PR's change set, the complete 45-file
shared journey directory, and six shared native/evidence helpers retain
their exact previous-head Git blobs: 59 scoped files in total. The
production clarification helper remains blob
`800a747284db19b8484650b6759e8bd1f356839d`. Follow-up 1–2 runs, question
1–3 runs, older exact counts, owned native event checks,
active-before/after persistence proof, cleanup/accounting and feature
mappings remain unchanged.

Source integration head `e55d74491f5bcd99c1455e9ace02bd7ad7319fae`
integrates exact actual squash
`8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` from
[#18385](https://github.com/paperclipai/paperclip/pull/18385). The merge
had no conflicts. All 17 files in this PR's change set and six shared
native/evidence helpers retain their exact Git blobs from previous head
`162e341cda5b627639b486b4019a1675bb018ea2`. This preserves the follow-up
and question readiness checks, feature mappings, six closed follow-up
check IDs, immutable before/after active-operation proof, follow-up 1–2
runs, question 1–3 runs and older exact counts.

Previously reviewed head `e54d64743964b1a19d165537bebbe62c50389a23` adds
a separate test-only reliability repair on top of that source
integration. It replaces one scheduler-sensitive browser case with a
deterministic Vitest clock and locator-state model that calls the actual
production helper. The six other real Chromium checks remain
byte-for-byte unchanged and pass **6/6**. Focused unit suites pass
**51/51**, including the deterministic deadline case. This is six
browser checks plus focused unit coverage, not seven browser checks.

The model makes the panel ready at 100 ms, the supplied answer ready at
600 ms, and full valid content ready at 1,300 ms. It requires rejection
at the shared 1,000 ms deadline and records immutable rejection state
before simulating a 10-second scheduler stall. Both restarted-timeout
mutations fail: resetting all waits wrongly resolves at 1,300 ms;
resetting matcher timeouts while keeping the final real deadline guard
rejects only at 1,300 ms with valid content present. The genuine helper
rejects at 1,000 ms with the answer still incomplete. The production
helper is unchanged, Git blob
`800a747284db19b8484650b6759e8bd1f356839d`. That two-file test-only
patch changed no journey, application, run, accounting, model or runtime
behavior. Its six-browser/51-unit results are prior-head evidence; the
complete patch is unchanged in the new integration.

The final source also retains the actual merged question-primary squash
`d9354bbfc35ef74333e90e596ce21dc8e59149b9` from
[#18346](https://github.com/paperclipai/paperclip/pull/18346). Later
commits `409117451da260a03fc28a3107c7808ecbe68f8f` and
`ea3e7b1d1bed562351fcf4722c7175001b987b59` bound question document
readiness to one campaign deadline and test that waits cannot restart
it. Commit `162e341cda5b627639b486b4019a1675bb018ea2` adds
question/follow-up report feature mappings and coverage. These files are
unchanged by the final merge.

The imported upstream source has real application changes:
[#18334](https://github.com/paperclipai/paperclip/pull/18334) makes
managed skill content durable and extends native skill-test contracts;
[#18350](https://github.com/paperclipai/paperclip/pull/18350) requires
managed Cloud shared storage and bootstrap prerequisites. #18385 updates
TestOps job metadata and repairs their combined maintenance fixture. The
new merge additionally imports
[#18383](https://github.com/paperclipai/paperclip/pull/18383) at
`81ecaf0654fe79a2a2dbda6588f033e39ee610e4`, which bounds failed
lifecycle verification;
[#18357](https://github.com/paperclipai/paperclip/pull/18357) at
`45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`, which persists
sandbox-creation ownership and updates cleanup/adapter/plugin contracts;
and #18254 at `18b1637b6371a6a6adade857e2a73654ee626fef`, which accepts
a valid late Antigravity authorization URL while native authentication
drains. This PR's diff against exact upstream
`18b1637b6371a6a6adade857e2a73654ee626fef` contains only journey tests,
report coverage and documentation. Harness source and deployed
application/runtime identity are separate inputs. This source
integration does not deploy the imported application changes or claim
the QA target runs this application revision.

This final SHA has not been measured by the historical run above:
**fresh QA at this exact final head is required before primary merge**,
followed by normal TestOps verification/admission. Fresh exact-head CI
and review have passed as recorded above. The live measurement must
record the actual separately deployed QA application/runtime identity.

Seven whole-file blobs remain identical to the qualified SHA. All paths
are relative to `tests/runner-e2e/`:

| Journey/helper | Unchanged Git blob |
| --- | --- |
| `product-journeys/active-followup-evidence.ts` |
`83a5d9fab52abb9849a40e83eca6b6a9c29bf587` |
| `product-journeys/active-followup.test.ts` |
`728160c610c18292d2cf06d4a87d25a8719f1734` |
| `native-task-reply.ts` | `59d11f1a178dad9db3da8fa2342ff93643cb9922` |
| `native-event-envelope.ts` |
`305b0b456c59c511ee328e529ad574c0583075a0` |
| `run-observations.ts` | `08c8ad184a821a7cd536926b3d7e3d1156a1e3da` |
| `user-actions.ts` | `e9611348dc41e17899367dab2e3b36d3d1be9480` |
| `stop-redirect-flow.ts` | `4580bd605a425bc72511e02d988492acbec8d60e` |

The follow-up preservation comparison against historical `af9` has three
explicit exceptions; no whole-file identity is claimed for them. The
separately documented question readiness and report additions above also
require final-source verification:

1. `product-journeys/agent-flows.ts` adds `question-answer-resume` to
the `AgentJourney` type exclusion, alongside follow-up. Its remaining
contents and runtime helper bodies are byte-for-byte unchanged.
Qualified blob `a0afaa968c1d854476159fef21de7b6dd50a4a98`; final blob
`0cc797c0ae594e7769338e63a80c4f67e29e4a54`.
2. The common run-count helper is centralized in `success-contract.ts`
and re-exported by the catalog. It accepts follow-up **1–2**,
clarification **1–3**, and each older paid journey's exact count. The
follow-up catalog entry, maximum of two runs, six required checks,
native dispatch, collection and retained-run contract remain
byte-for-byte unchanged. The helper's text differs; its preserved count
semantics are verified explicitly.
3. `product-journeys/active-followup-flow.ts` changes exactly one
composer readiness assertion from Playwright's implicit five-second
default to an explicit 30-second timeout, matching `submitTaskReply` for
attached targets. Prompts, actions, native evidence, before/after
persistence proof and grading otherwise compare byte-for-byte unchanged.
Qualified blob `64392329198a1a4d9828e32834d4507e175216e6`; final blob
`2746b5ba89e2af26b4097ba7897913186786ac5d`. This correction is why a
fresh final-head QA run is required.

- **310 focused tests passed at current final head
`e72e20ad111bd94728d38606015d1fdac3c92929`**: 286
journey/evidence/catalog/report/shared-deadline tests in 14 files and 24
native authentication/real-child stream-order tests in two files. Both
invocations used Node24 and one Vitest worker. The earlier `e55`
integration also passed 365 focused tests, including 40 Cloud
configuration and 40 TestOps metadata cases; those results remain
prior-source evidence.
- **157 explicit count assertions passed again at the current final
head**, including both interactive ranges, older exact counts,
invalid/fractional/non-number values and catalog/contract export
identity. The maximum bounds and six closed follow-up check IDs also
matched.
- Maps and diff checks passed. The official QA runtime-import preflight
passed on Node24 at the current final head without network access. It
uses a dummy adapter and no credentials. It does not prove target
connectivity, sign-in, model readiness or deployed application/runtime
identity. Those require the normal protected QA run. No model/provider
work or optional Runner release was dispatched.
- Current local runtime dependencies use ignored isolated
workspace-source views. No shared dependencies or compilers were
changed. A previous local typecheck attempt hit stale reused Runner
declarations. No full repository typecheck, test run or build was
repeated locally; the successful exact-head full CI run above now
supplies these gates.
- CI on previous head `162e341cda5b627639b486b4019a1675bb018ea2` failed
the Cloud/local maintenance fixture in [server shard
9](https://github.com/paperclipai/paperclip/actions/runs/41133716802/job/124331643100),
plus dependent verify. The actual tested merge combined that head with
master `198218bf554877fe3b7a83742eefb3b8f12392c6`. A Cloud child
selected local-disk storage and hit the new shared-storage policy. This
was a deterministic upstream fixture mismatch; no unchanged failed-job
retry was used.
- The actual #18385 repair configures a valid Cloud child, verifies
local bytes independently, checks the expected provider mismatch and
retains exact stored-config readback. Its own final head
`b76d5dd40d7711f1b5f6b6316b1114c0aae02d57` passed 59 checks, including
[server shard
9](https://github.com/paperclipai/paperclip/actions/runs/41133876636/job/124331450635).
That prior result supports the repair; it does not replace this final
head's fresh CI. Separate repair candidate #18392 and its
source/evidence are retained as superseded history.
- A local attempt to run the database fixture could not start PostgreSQL
because the host exhausted macOS semaphores. A first setup attempt also
failed package resolution before test execution. Skipped database cases
and these setup failures are not passes. No kernel settings, shared
processes or databases were changed.
- The complete previous-head CI run
[41138618869](https://github.com/paperclipai/paperclip/actions/runs/41138618869)
finished with one underlying failure: [Runner Vitest shard
1](https://github.com/paperclipai/paperclip/actions/runs/41138618869/job/124347859445)
expected native Antigravity login success but observed failure at
`antigravity-auth.test.ts:101`. The lane had one failed test, 1,994
passing tests and eight skipped tests. Aggregate verify failed only
because of this lane. All other 57 checks were accepted, including
build, typecheck, all server/browser suites, Runner shard 2, Snyk and
exact-head 5/5 reviews. No unchanged retry was requested.
- The independently reviewed #18254 repair removes an invalid cross-pipe
ordering assumption: a valid URL on stderr can arrive after the
authenticate response on stdout during drain. Its real-child regression
forces both orders. The current final source imports that actual squash;
the 24 focused authentication cases pass here. Fresh full CI and
exact-head review have now passed for this source, as recorded above.
The earlier failed result is not relabeled.
- The original live compatibility failure is retained: an older PRP v1
pack labeled builtin `bash` as operation `unknown`, causing the observer
to miss the command and never send the follow-up. Its exact shape was
reproduced red before repair. Generic unknown operations, semantic
tools, wrappers, read-only work and foreign provenance remain rejected.
The historical qualified pass used the repaired oracle; no failed result
was relabeled.
- Earlier CI on `6fad965d69` hit an unchanged heartbeat test's
three-second adapter-call wait; its original failure and one narrow
rerun remain recorded. The qualified SHA subsequently passed CI. An
earlier report-catalog subprocess failure is retained. The final focused
Node24 run now includes that suite and passes after the documented
report feature mappings.

## Risks

The fixture asks the native agent to run one bounded `sleep 90` command.
The observer must see actual active native execution; a fast or
noncompliant turn fails rather than passing from elapsed time. A
completed result requires one or two bounded native work runs, source
attribution, configured model/environment, and collection. Unknown
provider prices stay unknown. This change does not alter production
execution or grant new access.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The host
does not expose an exact underlying model version or context-window
size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with available version and
capability details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs; existing
follow-up execution fixes do not provide this deployment-targeted test
- [x] I have described the issue in-PR following the enhancement
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
ticket or instance-derived details
- [x] I have run the new focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 18:37:17 -05:00
..

Paperclip maps

The maps describe the same capabilities from two perspectives:

  • Feature map: what a person can do, where to reach it, and how to verify the user journey. Preserve its Poteto format.
  • Agent access map: the APIs and agent entry points for those capabilities, their permission boundaries, and access verification.

The chat scenario catalog adds natural-language requests, starting context and expected conversational outcomes for the same capability IDs. It lives with Product E2E as planning data; a scenario reference is neither a registered execution nor evidence that a journey passed.

These are sibling directories under maps/. Each feature file has a same-named agent access companion. Reuse the product map's sub-feature IDs, so tasks/create identifies the same capability in both maps. Product behavior remains governed by the implementation spec; a map cannot grant authority or redefine a permission denial as success.

Capability identifiers

The feature ID is the recipe filename without .md, such as onboarding. The full capability ID joins that feature ID and a Sub-features key with /: the company key in feature-map/onboarding.md is onboarding/company. Its agent-access-map/onboarding.md companion uses the same local company key. The feature-map index lists the existing feature and full capability IDs.

Each child page states Feature ID: `onboarding`. after its introduction or product-map link. In feature recipes, show the full ID beside the parsed local key: - `company`: `onboarding/company` — create a company. In access companions and Storybook verification guidance, keep the existing local heading and put Capability ID: `onboarding/company`. beneath it. These labels make the existing identifiers visible without changing keys, anchors, or ID generation.

Use full capability IDs in cross-feature references, Storybook annotations, tests, issues, PRs, and verification evidence. Both segments use lowercase letters, digits, and hyphens. Keep filenames and keys stable when display titles or navigation labels change; renaming either changes the capability ID.

Entry-point IDs describe a surface, such as web-wizard, and are scoped to their recipe. Reference them with a recipe link, such as onboarding.md#web-wizard, and record the capability exercised separately. Navigation section labels are not capability IDs. See IDs and references for examples.

Access contract

Every business operation needs both an API and a usable agent entry point for an actor authorized to perform it. A dedicated tool is optional when REST, the CLI, an assigned MCP client, or native search_api / call_api already provides the operation. Record which agent surfaces actually have that path; an installed CLI or optional MCP package cannot be assumed available to every native/remote run.

Track these independently:

Dimension Values and meaning
API present, missing, not-applicable, or unreviewed. Use exact methods/paths; distinguish protocols from REST.
Agent access available, partial, missing, restricted, operator-only, not-applicable, or unreviewed. Name the supported surface and actual entry point.
Policy Actor, action/grant, company/resource/ownership scope, responsible-user intersection, run requirements, work modes, and gates relevant to the operation.
Evidence source-only, verified, failed, blocked, or not-run. Source presence and successful board calls are not agent verification.
Review depth baseline records representative operations and known gaps; operation-reviewed enumerates the capability's meaningful operations and their individual access contracts.

Broad capabilities such as task properties must be split into operations before they count as operation-reviewed: renaming, metadata edits, assignment, dependencies, and lifecycle changes can use different tools and permission rules. An available read path cannot certify a write path. List/get/create/update/delete, execution/control, credential handling, and resolution are separate where their authority or transport differs. Do not duplicate request schemas from OpenAPI.

Exceptions and restrictions

Presentation-only state, static developer fixtures, and pre-start/local operator work may use not-applicable for a domain API, with a concrete reason and the existing local path. When a capability combines shared data with presentation, split the operations: browser recents do not exempt resource lookup from API coverage. Promoting local state to a shared feature requires revisiting the entry.

A board-only endpoint is restricted for agent identity, not an automatic exception or a missing API. Cite the enforced policy and record whether the agent can request the authorized human's action and observe its result. An unresolved delegation question remains in Gaps. Company access, a job title, a personal assistant's OAuth grant, or an accepted interaction does not grant board identity. Keep budget, approval, secret and execution authority intact.

Check actual route/service authorization and production tool admission. OpenAPI actor labels and scenario/evaluation tool declarations are supporting metadata; record disagreements rather than using them as proof of access. Native API fallback restrictions, tool field/scope limits, and rollout settings are part of the agent path, even when a legacy agent can call the REST endpoint directly.

Companion file format

Each agent-access-map/<area>.md has an H1, a relative link to its product file, and its feature ID, followed by one H2 backticked local key per product sub-feature. Each capability starts with its full capability ID and contains these fields:

  • Review: baseline or operation-reviewed.
  • API: presence state and exact operation IDs, or a reasoned exception.
  • Agent access: availability state and concrete entry points by surface.
  • Policy: relevant permission and runtime boundaries, with source references.
  • Evidence: evidence state, reviewed commit and sources or executed proof.
  • Verification: allowed/denied assertions and observable result; record what has actually run separately from the proposed recipe.
  • Gaps: missing APIs/paths, discovery drift, unreviewed operations or policy decisions; use none only for the enumerated scope that was reviewed.

For an operation-reviewed capability, use H3 backticked operation IDs and repeat API, Agent access, Policy, Evidence, Verification and Gaps for each operation. Use small operation names such as create, update, read, or run-now. The enclosing capability has Review: operation-reviewed only after its meaningful operations have been enumerated. The following excerpt shows one operation; include the others before using that review label:

## `definition`

Review: `operation-reviewed`.

### `create`

API: `present` — POST /api/companies/{companyId}/routines.
Agent access: `available` — agent REST, CLI routine create and native call_api
under the enabled rollout and active-run/mode gates.
Policy: same company; assigneeAgentId must be the calling agent; cite the
routine authorization and native API policy sources.
Evidence: `source-only` — reviewed commit and source links; no live proof.
Verification: `not-run` — create an owned routine; reject assignment to another
agent; inspect saved configuration, attribution and retry behavior.
Gaps: verify ownership and responsible-user resolution with a permitted agent
context; source presence alone does not qualify this operation.

Keeping the maps synchronized

When a change adds/removes a capability or changes its behavior, API, permission, tool binding, CLI support, or work mode, update the affected companion and product recipe as appropriate in the same change. Preserve IDs across renames where the capability is unchanged; update the affected cross-links together. Indexes discover recipes automatically. An API-only change may update the companion without rewriting unchanged user instructions. Update the corresponding chat scenarios when their request, assumptions or expected outcome changes; use multiple cases for meaningfully different starting conditions rather than treating an API path as conversational coverage.

maintain-verification-skill owns reconciliation of both maps. It follows changed source and authorization, checks real agent reachability, and records product gaps without implementing them during a maintenance pass. The creation skill uses this structure. verify-paperclip consumes the relevant recipes and reports evidence/drift; map editing belongs to maintenance.

Run the structural check after map changes:

node scripts/check-maps.mjs

It checks paired areas/IDs, navigation metadata, required fields, and local references. It does not certify endpoint authorization, production bindings, operation completeness, or live behavior. Source audits must check those against the existing API and tool contracts; avoid a second schema catalog or runtime.

For live access proof, use Paperclip's existing API/runner/MCP fixtures or an owned isolated instance. Use the actual agent identity, target and work mode. Assert the allowed result and relevant denial (another company/private resource, wrong owner, missing grant, invalid run, restricted mode or resolver). Mutations also need persisted state/attribution and retry behavior. Capture evidence using the existing artifact workflow; keep executed run results separate from generated maps/generated/coverage.json, which lists page references.

Local authorship and generated indexes

Change only the recipes, access contracts and stories whose behavior or intent changed. Feature recipes own navigation metadata; stories own small parameters.paperclip annotations. Keep the feature-map README's readable ID index aligned with those sources; it does not register entries for generation. Do not refresh counts in shared READMEs, inventories, fingerprints or review journals.

node scripts/storybook-catalog.mjs validates and generates aggregate navigation, Storybook data, verification routes and the page inventory. All outputs live in ignored maps/generated/, ui/storybook/generated/, and the ignored Storybook preview.tsx wrapper. Start/build and UI typechecking generate their own inputs. A clean merged checkout therefore assembles both branches' changes automatically.

Git provides history and diffs. PRs, issues and CI artifacts retain review/run evidence for the named revision. Capability associations do not create whole-file review dependencies. Keep durable scenario limits in the affected recipe or story.