Files
PaperClipAI/tests/runner-e2e/journey-features.ts
T
DottaandPaperclip 7e595fd055 test(e2e): verify native hiring, delegation and teammate reuse (#18351)
## Thinking Path

> - Paperclip manages real work by AI agents.
> - Daily QA must verify that work on the staging deployment.
> - Classic Agent Chat coverage already exercises hiring and teammate
reuse.
> - The deployment journey should share that behavior and its ownership
oracle.
> - Saved documents, exact callback replies and physical cleanup
establish success.
> - The corrected journey and complete daily suite now pass on QA.

## Linked Issues or Issue Description

**Subsystem affected**

Product E2E deployment journeys and their shared browser controls.

**Problem or motivation**

QA must prove that one newly hired teammate completes two different
tasks and returns both saved results to the lead conversation. Names,
chat claims and successful-run counts alone cannot establish authorship
or completion.

**Proposed solution**

Reuse the classic hiring oracle and production composer. Verify
immutable creation audits, initial assignment, current project and
ownership, saved document revisions, exact native authors, durable
callbacks and their own task links after reload. Bound execution to five
lead turns and two child turns.

**Alternatives considered**

A separate QA implementation would duplicate the classic journey. Name
matching or later reassignment would accept broken initial ownership.
Links elsewhere in the conversation cannot substitute for either
callback's own result link.

**Roadmap alignment**

This verifies existing features. Related merged repairs: #18389, #18465,
#18459, #18463, #18502, #18503, #18504, #18506, #18507 and #18514.
Separate dependency #18517 repairs file preservation after lifecycle
cancellation and is merged as Core `050365e`, which also contains
#18514. This primary PR now differs from that protected mainline in only
16 unchanged H harness/documentation paths; it adds no application
production delta. The measured QA deployment serves Core `050365e`, with
its matching bundled provider pack and unchanged approved model/ROM.

## What Changed

- Add the shared `hire-delegate-reuse` journey, catalog, types,
dispatch, launch fingerprints, journey-feature entries and
documentation. Hire one teammate, delegate a checklist, then reuse the
same teammate for a saved review. Require seven canonical native runs:
five lead and two child turns.
- Require exact company, lead, chat and native creation receipts. Both
tasks must be created for the intended teammate using immutable
creation-audit assignee fields. Independently verify current project and
ownership. Creation-time project is not recorded in that audit and is
not claimed by this assertion.
- Verify both current document revisions belong to the exact child runs
and preserve the original checklist.
- Use ordinary production rich-editor agent/project autocomplete
controls. Verify exact reference chips, then send once. Preserve the
shared default sender.
- Require each durable, run-owned callback reply to contain its own
exact completed-task link. Collect links through the actual production
Markdown renderer and issue-reference decoration, then apply the
existing company-scoped route authority. An unrelated table or code
fence cannot hide a rendered link; foreign, wrong-task and code-only
links remain failures.
- Preserve approved connection, account, grant and responsible-user
attribution. Verify the child's independent installation and retain the
immutable binding snapshot. Do not copy credentials or broaden grants.
- Record ownership before later observations can fail. Retry only typed
transient observation GET failures. Preserve authority and cleanup
failures.
- Retain all 14 daily journeys, the model matrix, deadlines, seven-run
cap and 500-cent aggregate allowance split 300/200.

## Verification

- **Current composed head:** `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`.
It normally merges mainline `1e8e322` and exact #18517 source `adb1d4e`,
retaining the intermediate `b2d9eb4` and `6fd1042` ancestry. The only
merge conflict was the Playwright support registration; it is resolved
to the exact previously qualified union. All 16 H harness/documentation
paths are byte-identical to `5c0830f`; all four #18517 paths are
byte-identical to `adb1d4e`. Every other tracked path matches actual
protected mainline `050365e`. A clean merge preview of `050365e` and
this head equals the current head tree, so no additional source commit
is needed. GitHub's three-dot display still lists the four already
merged helper files because #18517 used a squash merge; those blobs
equal protected mainline and add no net application change. No journey,
oracle, cap or model change accompanies this composition. A final clean
merge preview on intervening mainline
`006d5691a6245d5a710e25ba3df6d609754e163a` preserves all unrelated
changes and adds exactly the same 16 measured H blobs, with no net App
production delta. That later mainline is not represented as the AppCore
measured by the QA run.
- **Local qualification:** 149/149 focused tests across six suites,
including the report-catalog case, and 11/11 real Chromium checks passed
on `b2d9eb4`. All H modules, browser inputs and parser/package
dependencies are byte-identical at `afa3f55`; those unchanged suites
were not repeated. Current-source transitive types, maps, module
boundaries and diff checks pass. Official Node24 QA runtime imports pass
again using exact public `afa3f55` and private `8671fa9`, with a clean
credential-free environment and network disabled. The initial local
report-catalog failure is retained: the child `tsx` process was denied
its local IPC socket before importing the report or writing an artifact.
The same source and selected case pass after correcting only sandbox IPC
permission; no source workaround or deselection was used.
- **Current canonical CI and reviews pass.** [CI
41275109691](https://github.com/paperclipai/paperclip/actions/runs/41275109691)
completed successfully at exact `afa3f55`: all 59 contexts are terminal
and clean (55 success, three skipped, one neutral). Fresh exact-head
Greptile and official Paperclip reviews both scored 5/5; all 27 review
threads are resolved with pagination complete. The official review is
static and its check is neutral; it did not execute tests or live QA. A
separate exact-head independent composition assessment found no
actionable issues at 4/5 confidence and performed no live actions.
**Fresh QA qualification passed: all 14 daily journeys, including the
corrected H7, passed against deployed Core `050365e`.** Prior check
conclusions and review scores retain their actual heads. The previous
`5c0830f` completed all 59 contexts cleanly, actual exact-head Greptile
and official Paperclip reviews both scored 5/5, and all 27 review
threads were resolved. Its separate independent journey-source review at
`2b803f08` remained 4/5 with a unit-helper authorship caveat.
Intermediate `b2d9eb4` received an actual exact-head Greptile 5/5 with
zero open findings, then was updated to compose the valid dependency
review fix before live QA. That score and its pending CI are not
attributed to `afa3f55`.
- Existing causal controls remain unchanged: invalid immutable assignee,
either missing callback link, foreign/code/wrong-task links and an
earlier handoff fail against the relevant prior source. Renderer
controls prove that tables and fenced code elsewhere in a valid callback
do not hide its actual task link. Both composer and document-navigation
browser registrations remain present.
- #18503 merged after its actual QA document case passed four canonical
checks and five phases, positively established zero native work, and
completed cleanup. Its measured files are unchanged in this composition.
#18506 and #18507 merged after their own checks; their causal
worker-settlement controls remain intact.
- #18514 is merged as `1e8e322`. It reproduces the selected-task
metadata-only-project startup failure against deployed `756599e` and
passes 19 real-PostgreSQL selected controls after repair. Both direct
and assembled baseline controls fail before it. Real source, explicit
selection, policy and workspace-isolation guards remain enforced. It is
included in measured Core `050365e`; fresh H7 reached and completed both
delegated native tasks.
- #18517 initial source `6fd1042` passed 320/320 focused and sibling
Stop/recovery tests, including 20 preservation controls. Its actual
official review was 4/5 and found a valid follow-up: the new
live-controller guard should wait for actual lease expiry rather than
repeat locked checks every 30 seconds. Both an already-live lease and a
real under-lock renewal reproduce that issue against `6fd1042`.
Corrected source `adb1d4e` passes all 22 preservation controls and 124
directly affected tests across three suites, exact-head server
build/stamp, typecheck and structural checks. It defers before locking
and rechecks renewed expiry afterward, preserving the transactional
owner guard. The old score/failure and successful owner tests remain
attributed to their actual sources. Independent exact-head review of
`adb1d4e` found no actionable issues at 4/5 confidence. This is
local/source evidence, not physical QA recovery. Its current [CI
41274963476](https://github.com/paperclipai/paperclip/actions/runs/41274963476)
completed successfully with all 59 contexts clean; both actual current
reviews are 5/5 with zero unresolved threads. It merged normally as
`050365e` after those gates. QA now runs that image. The historical
allocation is positively stopped and retained; its files-only export
remains a separately tracked repair, not a passing-test assertion.
- The historical [first H7
pass](https://pages.paperclip.ing/evals-qa-smoke-41226065043-1/) remains
attributed to public `5d3f143`, private `dabad62` and App `b7f1c37`: all
checks and five phases passed, seven native runs succeeded, both saved
documents and exact callbacks were verified, and physical cleanup
completed. An optional runtime-metadata receipt was inconclusive and was
not grade authority.

Material failed attempts remain failed:

| Attempt | Actual result and correction |
| --- | --- |
|
[41231853638](https://pages.paperclip.ing/evals-qa-smoke-41231853638-1/)
| 13 passed; hiring omitted initial teammate/project assignment and
started extra work. The production composer repair retains the ownership
negative. Later recovery stopped physical work without changing the
failed grade. |
|
[41238711372](https://pages.paperclip.ing/evals-qa-smoke-41238711372-1/)
| All seven native runs succeeded, but the final callback had no durable
chat reply. Cleanup passed. #18502 repairs premature completion-outbox
retargeting; the fresh measured H7 now verifies both durable, run-owned
callback replies and their own task links. |
|
[41253428446](https://pages.paperclip.ing/evals-qa-smoke-41253428446-1/)
| 12 passed; hiring rejected a mentioned metadata-only project before
delegation, and document reopening hit an immediate-control-probe race.
#18504 and #18503 repair those separate boundaries. Historical
missing-task absence remains quarantined; physical termination does not
manufacture absence evidence. |
|
[41266385118](https://pages.paperclip.ing/evals-qa-smoke-41266385118-1/)
| On `5c0830f` and App `756599e`, 13 passed, including the exact
composed document helper. Hiring created the correctly assigned task,
but its child admission failed before runtime selection because the
organizational project had no repository source. #18514 repairs that
selected-task boundary. The cancelled native lead retained an active
prepared-workspace lease; cleanup 41271035231 failed with
`owned_work_stop_unconfirmed`. #18517 addresses its file-preservation
admission. Launch 1110, original evidence and automatic Slack FAIL
remain failed. |
| [CI
41257779002](https://github.com/paperclipai/paperclip/actions/runs/41257779002)
/ [CI
41263476916](https://github.com/paperclipai/paperclip/actions/runs/41263476916)
| Existing enrollment and snapshot-retention fixtures observed
intermediate I/O before worker settlement. #18506 and #18507 have
controlled causal failures, complete-settlement observers and preserved
deadlines/negative controls. Runtime scheduling behavior is unchanged. |

**Fresh measured acceptance:** [QA run
41285131220](https://github.com/paperclipai/paperclip-cloud/actions/runs/41285131220)
and its [permanent
report](https://pages.paperclip.ing/evals-qa-smoke-41285131220-1/)
passed **14/14**, zero failed or incomplete, with all five phases and
complete cleanup for every case. The strict verifier independently
matched original grades, current provenance and completed manifests:
public harness `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`, private
workflow `8671fa93e27501fabbfffa377bec6dea453b5db7`, AppCore
`050365e09d88411960fc3bacc1c08c46c0d9b6fa` and matching Cloud
image/provider pack. All model, ROM, deadline and run-budget controls
remained unchanged. All three workflow jobs completed successfully; the
automatic Slack notification reported PASS with links to this run. The
passing TestOps launch was closed normally after evidence verification.

The hiring case passed all 11 check entries across nine IDs and five
phases, with exactly five lead and two child native runs. Both saved
documents match their live current revision/body hashes and exact
hired-agent run attribution. The immutable original checklist remains
intact; the same teammate authored the review. Both own callback replies
contain their exact delegated task link. Independent canonical readback
confirmed all seven native runs succeeded and committed their accepted
result, assessment and decision, with no live owner and zero pending or
failed effects; each of the seven exact owned leases has its own bound
DESTROYED receipt and successful cleanup. All seven final snapshot
operations match their source run, workspace, task, epoch, SHA,
generation and export location. The four earlier lead snapshots are
retained SUPERSEDED ancestors of the current committed snapshot; the
latest lead and both child snapshots are COMMITTED. The initial
diagnostic required current COMMITTED status for every historical
snapshot and correctly remained false until the separate lineage
diagnostic established this supported supersession. The product oracle
and original grade were not changed. Canonical Suite metadata is
`qa-basics`; the daily plan was saved with its standard suite-only query
and synchronized, retaining all 14 ACTIVE cases.

The older failed allocation remains separately stopped, retained,
isolated from reuse and owned by terminated actors. That permits fresh
campaigns without replaying old work. Its same-allocation files export
is still unresolved and has separate bounded files-only recovery work;
it is not claimed by this successful campaign. Original failed grades,
reports and automatic FAIL notifications remain unchanged. No failed
result was closed, relabelled, hidden or retried unchanged to obtain
green.

## Risks

The journey requires Agent Chat, normal hiring permission, a scoped lead
and a provider that finalizes owned resources. Missing or extra work,
wrong ownership, absent saved documents, missing callbacks and
unconfirmed cleanup must fail.

The deployed app and bundled runner pack must match the verified Core.
The old run's cancellation ACK and terminal database row do not prove
physical termination or saved files. Its coordinator has no accepted
result or decision; pending-effect absence cannot be inferred from that
missing decision. File recovery must preserve the exact allocation and
original destroy disposition without a provider turn.

Current source checks, reviews, fresh H7 and full-14 aggregate all pass.
Canonical Suite metadata and actual 14-case daily-plan synchronization
were verified after closing the genuine passing launch. The original
failed allocation still has a separate files-export debt; its STOPPED
receipt is not a SAVED or DESTROYED claim.

## Model Used

OpenAI Codex, GPT-6 family. The session does not expose a more specific
backend model ID or fixed context window. Used reasoning, repository
tools, TypeScript and browser test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
- [x] Historical passes and material failed attempts retain their actual
source and outcome
- [x] The composed document helper passed normal QA with complete
cleanup on its actual source
- [x] The corrected hiring journey passes fresh normal H7 with complete
cleanup
- [x] The active hiring case has canonical Suite and daily-plan
membership after the passing launch is closed

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-01 02:33:59 -06:00

136 lines
8.0 KiB
TypeScript

import { fileURLToPath } from "node:url";
import { featureIndex } from "../../scripts/feature-map.mjs";
import type { ReportExecution } from "./report-catalog.js";
export const journeyFeatureMap = featureIndex(fileURLToPath(new URL("../../", import.meta.url)));
export const journeyFeatures = journeyFeatureMap.sections.flatMap((section) =>
section.features.map((feature) => ({ ...feature, section: section.title })),
);
const capabilities = new Map(journeyFeatures.flatMap((feature) =>
feature.capabilities.map((capability) => [capability.id, { ...capability, feature }] as const),
));
const associations = new Map<string, readonly string[]>();
function associate(suites: readonly string[], cases: Record<string, readonly string[]>) {
for (const [caseId, ids] of Object.entries(cases)) {
for (const id of ids) if (!capabilities.has(id)) throw new Error(`Unknown journey capability: ${id}`);
for (const suite of suites) associations.set(`${suite}/${caseId}`, ids);
}
}
associate(["product-journeys"], { "task-create": ["tasks/create"], "ask-question": ["questions-and-approvals/questions"],
"question-answer-resume": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
"followup-while-running": ["steering/queue"],
"hire-delegate-reuse": ["agents/hire", "documents-artifacts/documents"],
"project-task-context": ["projects/lifecycle", "projects/task-context", "tasks/create"],
"task-properties": ["tasks/properties"], "task-document-persistence": ["documents-artifacts/documents"], "task-search-filter": ["tasks/organize"], "cloud-onboarding": ["onboarding/first-task"] });
associate(["antigravity-agent-journey"], {
"saved-subscription-two-turn": ["agents/hire", "agents/instructions", "runs-adapters/validation", "runs-adapters/session",
"tasks/create", "tasks/completion", "documents-artifacts/attachments", "documents-artifacts/work-products"],
});
associate(["core-compatibility", "openrouter-model-breadth", "extended-harnesses", "grok-qualification", "grok-subscription-qualification", "kimi-qualification"], {
"message-marker": ["runs-adapters/runs"],
"hello-complete": ["runs-adapters/runs"],
"plan-revise-accept": ["questions-and-approvals/plan-review"],
"plan-approve-complete": ["questions-and-approvals/plan-review"],
"ask-question": ["questions-and-approvals/questions"],
"question-resume-complete": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
});
associate(["local-session-integrity", "extended-harnesses", "grok-qualification", "grok-subscription-qualification", "kimi-qualification"], {
"structured-question-resume": ["questions-and-approvals/continuation", "runs-adapters/session"],
"structured-question-restart-resume": ["questions-and-approvals/continuation", "runs-adapters/session"],
});
associate(["question-resume"], {
"question-answer-resume": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
});
associate(["feedback-persistence"], {
"complaint-and-suggestion": ["activity/agent-commentary"],
});
associate(["continuation"], {
"answer-updates-scope": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
"clarification-not-approval": ["questions-and-approvals/questions", "questions-and-approvals/confirmations"],
"revision-preserves-approval": ["questions-and-approvals/plan-review"],
"completed-action-resume": ["questions-and-approvals/continuation"],
"question-tool-documentation": ["questions-and-approvals/questions"],
"provider-question-bridge": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
});
associate(["first-task", "confirmation-replies", "completion-updates"], Object.fromEntries([
"interview-first-response", "clear-task-first-response", "ambiguous-task-first-response",
"plain-message-first-response", "plan-first-response", "interview-plan-accept",
"task-card-accept", "accept-while-running", "task-reply-accept", "clarify-propose-accept",
"revise-accept", "reject-no-execution",
].map((id) => [id, ["onboarding/first-task"]])));
associate(["first-task"], { "ordinary-task-control": ["tasks/completion"] });
associate(["confirmation-replies"], {
"confirmation-ambiguous": ["questions-and-approvals/confirmations", "questions-and-approvals/audience"],
"unanswered-question-return": ["questions-and-approvals/dismiss-and-reopen", "questions-and-approvals/continuation"],
});
associate(["agent-chat", "agent-chat-hardening", "grok-qualification", "grok-subscription-qualification", "kimi-qualification", "stock-harness"], {
"create-backlog": ["tasks/create"],
"reassign-task": ["tasks/properties"],
"continuity-restart": ["agent-chat/history", "runs-adapters/session"],
"new-session": ["agent-chat/history", "runs-adapters/session"],
"stop-new-resume": ["steering/stop-and-pause", "runs-adapters/session"],
"stop-startup-new-resume": ["steering/stop-and-pause", "runs-adapters/session"],
"plan-handoff": ["agent-chat/handoff", "questions-and-approvals/plan-review"],
"clarify-reuse": ["projects/task-context"],
"multi-repository": ["projects/repositories"],
"hire-delegate-reuse": ["agents/hire", "task-coordination/delegation"],
"committed-send-retry": ["recovery/unknown-outcome"],
});
associate(["agent-chat-qualification"], {
"active-reassignment": ["tasks/properties"],
"worker-crash-retry": ["recovery/source-action"],
});
associate(["agent-chat-stories"], {
"enable-disable-resume": ["agent-chat/history"],
"followup-while-running": ["steering/queue"],
"revise-while-running": ["steering/steer"],
});
associate(["completion-updates"], Object.fromEntries([
"handoff-completion-idle", "handoff-completion-busy", "handoff-completion-multiple", "handoff-completion-restart",
].map((id) => [id, ["agent-chat/handoff", "tasks/completion"]])));
associate(["everyday-workflows", "native-connection-guidance", "grok-qualification", "grok-subscription-qualification"], {
"build-revise": ["documents-artifacts/work-products"],
"delegate-feedback": ["task-coordination/delegation", "steering/steer"],
"agent-review-handoff": ["task-coordination/signoff"],
"hire-reuse": ["agents/hire", "task-coordination/delegation"],
"service-approve": ["questions-and-approvals/tool-review", "app-permissions/action-policy"],
"service-decline": ["questions-and-approvals/tool-review", "app-permissions/action-policy"],
"connection-decline": ["connection-setup/task-request"],
"provider-decline": ["connection-setup/task-request"],
"provider-second": ["connection-setup/task-request"],
"provider-native": ["connection-setup/task-request"],
"agentmail-setup": ["connection-setup/task-request"],
"recover-controller": ["recovery/user-continuation", "documents-artifacts/work-products"],
"stop-redirect": ["steering/stop-and-pause"],
"create-skill-studio": ["skills/authoring"],
});
associate(["hiring-templates"], { "hire-coder-template-reuse": ["agents/hire", "task-coordination/delegation"] });
associate(["kimi-qualification"], {
"file-edit-validate": ["documents-artifacts/work-products"],
"native-write-permission-stop": ["steering/stop-and-pause", "questions-and-approvals/tool-review"],
"native-write-provider-loss": ["recovery/diagnosis", "questions-and-approvals/tool-review"],
});
associate(["task-workspaces"], {
"task-directory-repository-resume": ["workspaces/binding", "workspaces/provisioning", "projects/repositories"],
});
associate(["task-titles"], {
"prompt-title-standard": ["tasks/create"],
"prompt-title-ask": ["tasks/create"],
"preserve-explicit-title": ["tasks/create"],
});
associate(["pi-controls", "native-active-stop"], {
"pending-permission-stop": ["steering/stop-and-pause"],
"same-turn-steering": ["steering/steer"],
});
export function journeyCapabilities(execution: Pick<ReportExecution, "suite" | "task">) {
return (associations.get(`${execution.suite.id}/${execution.task.id}`) ?? [])
.map((id) => capabilities.get(id)!);
}
export function journeyFeatureIds(execution: Pick<ReportExecution, "suite" | "task">) {
return [...new Set(journeyCapabilities(execution).map((capability) => capability.feature.id))];
}