Files
PaperClipAI/tests/runner-e2e/journey-features.ts
T
DottaandPaperclip 7b8aa60b6d test(e2e): add repository continuation journey (#18729)
## Thinking Path

> - Paperclip keeps agent work in project workspaces.
> - A later user request must continue from the saved repository state.
> - A successful reply alone cannot prove file or Git history
preservation.
> - The repository journey in #18652 passed against QA staging with
independent byte and commit checks.
> - Its separate completion journey remains unpassed and must stay open.
> - This PR delivers only repository continuation and reuses the merged
shared support from #18728.

## Linked Issues or Issue Description

Refs: #18652. Builds on merged #18728. This is the repository-only
delivery from #18652. The original completion implementation and all
failed attempts remain on that open PR.

## What Changed

- Add `repository-continuation` with the exact measured source fixture,
two native turns, and eight required checks.
- Start from measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189`.
Keep the repository flow and Git/byte oracle unchanged. Clarify the two
file-content instructions to require the word first or verified followed
by exactly one LF newline; retain the exact expected bytes and the
original random-reference contract.
- Document the two-turn proof boundary: final saved hidden Git metadata
is not independently read. Keep all eight success assertions.
- Use the original-deadline and routine failure-reporting repairs from
merged #18728. Integrate its actual master squash
`e5713d95c6d9a258882649789ea5eb071cc17dc0` through one normal merge.
- Use a fixed public read-only repository commit. Verify downloaded Git
objects, parent commits, changed paths, saved workspace identity, and
exact source bytes across both turns. Reopen the registered outputs
through the browser.
- Keep the shared artifact reader behavior unchanged and narrow its type
to the API capabilities it uses.
- Add only this case to the catalog, feature association, dispatcher,
source hash, types, run/check contract, strict catalog expectation, and
journey documentation.

## Verification

- Current head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed fresh
[QA staging run
41589522430](https://github.com/paperclipai/paperclip-cloud/actions/runs/41589522430)
and its [full
report](https://pages.paperclip.ing/evals-qa-smoke-41589522430-1/). All
eight checks and five phases passed. Both native turns succeeded and
committed in the same workspace. The independent same-buffer Git audit
verified that the second commit has the first commit as its parent. The
saved original files and browser downloads matched their expected bytes.
This does not add a final hidden `.git`/HEAD assertion.
- The fresh run used OpenCode with
`openrouter/deepseek/deepseek-v4.1-flash-1017` on QA staging/Daytona. It
used Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`, Cloud
`cloud-41570439002-1`, and environment configuration version 9. Actual
checkouts matched public source
`86808b7ec17e1727bd55a3fd46bc7e0409ca0e31`, adapter
`7367cf6fd0ee1cf52c2651a8a6671e087728a9c1`, and protected workflow
bootstrap `a9ad6877de62fe3f65aacadc8a9dfba7d90729cf`. The bootstrap
matched the accepted 117-file QA contract.
- Normal cleanup completed. The original manifest is complete, evidence
cleanup is complete, no operations remain active, and the owned agent is
terminal. The coordinator verified ACTIVE TestOps case 685 / result 3695
and closed [launch 1921](https://paperclip.testops.cloud/launch/1921).
This records normal cleanup closure; no separate physical provider-stop
acknowledgement was collected.
- Measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189` passed [QA
staging run
41573331189](https://pages.paperclip.ing/evals-qa-smoke-41573331189-1/)
against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch
1723 / result 3501 maps to case 685. All eight checks and five phases
passed. The two run bindings, independent saved-byte/Git audit, and
completed cleanup were verified.
- Initial extraction `54103639850d08d4d125ebe0493d8d3d49b58211`
preserved the measured case files. The current head also clarifies the
two LF instructions, documents the final Git-metadata limit, and
inherits the reviewed helper repairs. This changed the fixture
conditions. The current head now has its own live pass, listed below.
The original ac4 result remains a pass under its original conditions; it
was not used to qualify this head.
- The emitted-prompt control failed on the old quoted backslash+n
instructions, while 14 sibling controls passed. All 47 focused
repository/artifact controls then passed after the two-sentence repair.
Local: 891 related support tests, runner E2E typecheck, recursive
typecheck, build, maps, token gates, and `git diff --check` passed. The
inherited routine source has 62 causal read/cleanup controls and 26
unchanged real component controls.
- The initial repository head's normal full CI run 41578686494 passed.
Its 4/5 review and all original findings remain recorded. The later
stacked head `037b979477161881a4877230500e6f3de8faea5c` also passed full
CI run 41581342503 and a fresh 5/5 review. Current integration head
`86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed normal full [CI run
41584981651](https://github.com/paperclipai/paperclip/actions/runs/41584981651),
attempt 1. Its fresh exact-head Greptile review is 5/5, with zero
unresolved threads and no open findings. The broad local suite was not
repeated. The earlier full local run recorded in #18643 had four
failures and did not reach 164 serialized groups; this PR does not claim
an all-local pass.
- Earlier repository failures remain recorded: provider-pack transfer
timed out in one attempt; a later attempt could not read saved workspace
files. The later pass does not regrade those failures. The product
repairs are separate from this unchanged case delivery.
- The fixed actual #18728 squash is now integrated. The PR targets
master. Its aggregate diff has exactly 14 repository-only paths, each
identical to `037b979477161881a4877230500e6f3de8faea5c`; all other
master changes are preserved. No later master head was chased. The
current CI, review, and fresh staging qualification all passed for this
integration head.

## Risks

- Git bundles and workspace bytes are private evidence. Public reports
retain closed checks; the fixture never pushes to the public source
repository.
- The second committed output must retain the first commit as its
parent, the same workspace binding, and original saved file bytes. Final
hidden `.git`/HEAD after the second save is not independently exposed by
the supported remote workspace API; this test does not claim third-turn
Git metadata durability.
- Only JSON metadata observations retry transient transport failures.
Binary attachment and browser downloads remain single attempts.
Transport failures and digest mismatches remain visible failures.
- The existing artifact reader is shared. Its runtime code is unchanged
and its regression coverage remains included.
- Daily source pins and plan selection stay with the coordinator. This
PR does not register or bypass the unpassed completion case.

## Model Used

OpenAI GPT-6 through Codex, with tool use and code execution. The exact
deployment ID and context window are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 00:46:13 -06:00

140 lines
8.3 KiB
TypeScript

import { fileURLToPath } from "node:url";
import { featureIndex } from "../../scripts/feature-map.mjs";
import type { ReportExecution } from "./report-catalog.js";
export const journeyFeatureMap = featureIndex(fileURLToPath(new URL("../../", import.meta.url)));
export const journeyFeatures = journeyFeatureMap.sections.flatMap((section) =>
section.features.map((feature) => ({ ...feature, section: section.title })),
);
const capabilities = new Map(journeyFeatures.flatMap((feature) =>
feature.capabilities.map((capability) => [capability.id, { ...capability, feature }] as const),
));
const associations = new Map<string, readonly string[]>();
function associate(suites: readonly string[], cases: Record<string, readonly string[]>) {
for (const [caseId, ids] of Object.entries(cases)) {
for (const id of ids) if (!capabilities.has(id)) throw new Error(`Unknown journey capability: ${id}`);
for (const suite of suites) associations.set(`${suite}/${caseId}`, ids);
}
}
associate(["product-journeys"], {
"repository-continuation": ["projects/task-context", "documents-artifacts/attachments"],
"routine-scheduled-run": ["routines/definition", "routines/triggers", "routines/history", "routines/management"],
"file-input-answer": ["documents-artifacts/attachments", "documents-artifacts/documents"],
"inbox-triage": ["inbox-search/views", "inbox-search/triage"], "task-create": ["tasks/create"], "ask-question": ["questions-and-approvals/questions"],
"question-answer-resume": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
"followup-while-running": ["steering/queue"],
"hire-delegate-reuse": ["agents/hire", "documents-artifacts/documents"],
"project-task-context": ["projects/lifecycle", "projects/task-context", "tasks/create"],
"task-properties": ["tasks/properties"], "task-document-persistence": ["documents-artifacts/documents"], "task-search-filter": ["tasks/organize"], "cloud-onboarding": ["onboarding/first-task"] });
associate(["antigravity-agent-journey"], {
"saved-subscription-two-turn": ["agents/hire", "agents/instructions", "runs-adapters/validation", "runs-adapters/session",
"tasks/create", "tasks/completion", "documents-artifacts/attachments", "documents-artifacts/work-products"],
});
associate(["core-compatibility", "openrouter-model-breadth", "extended-harnesses", "grok-qualification", "grok-subscription-qualification", "kimi-qualification"], {
"message-marker": ["runs-adapters/runs"],
"hello-complete": ["runs-adapters/runs"],
"plan-revise-accept": ["questions-and-approvals/plan-review"],
"plan-approve-complete": ["questions-and-approvals/plan-review"],
"ask-question": ["questions-and-approvals/questions"],
"question-resume-complete": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
});
associate(["local-session-integrity", "extended-harnesses", "grok-qualification", "grok-subscription-qualification", "kimi-qualification"], {
"structured-question-resume": ["questions-and-approvals/continuation", "runs-adapters/session"],
"structured-question-restart-resume": ["questions-and-approvals/continuation", "runs-adapters/session"],
});
associate(["question-resume"], {
"question-answer-resume": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
});
associate(["feedback-persistence"], {
"complaint-and-suggestion": ["activity/agent-commentary"],
});
associate(["continuation"], {
"answer-updates-scope": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
"clarification-not-approval": ["questions-and-approvals/questions", "questions-and-approvals/confirmations"],
"revision-preserves-approval": ["questions-and-approvals/plan-review"],
"completed-action-resume": ["questions-and-approvals/continuation"],
"question-tool-documentation": ["questions-and-approvals/questions"],
"provider-question-bridge": ["questions-and-approvals/questions", "questions-and-approvals/continuation"],
});
associate(["first-task", "confirmation-replies", "completion-updates"], Object.fromEntries([
"interview-first-response", "clear-task-first-response", "ambiguous-task-first-response",
"plain-message-first-response", "plan-first-response", "interview-plan-accept",
"task-card-accept", "accept-while-running", "task-reply-accept", "clarify-propose-accept",
"revise-accept", "reject-no-execution",
].map((id) => [id, ["onboarding/first-task"]])));
associate(["first-task"], { "ordinary-task-control": ["tasks/completion"] });
associate(["confirmation-replies"], {
"confirmation-ambiguous": ["questions-and-approvals/confirmations", "questions-and-approvals/audience"],
"unanswered-question-return": ["questions-and-approvals/dismiss-and-reopen", "questions-and-approvals/continuation"],
});
associate(["agent-chat", "agent-chat-hardening", "grok-qualification", "grok-subscription-qualification", "kimi-qualification", "stock-harness"], {
"create-backlog": ["tasks/create"],
"reassign-task": ["tasks/properties"],
"continuity-restart": ["agent-chat/history", "runs-adapters/session"],
"new-session": ["agent-chat/history", "runs-adapters/session"],
"stop-new-resume": ["steering/stop-and-pause", "runs-adapters/session"],
"stop-startup-new-resume": ["steering/stop-and-pause", "runs-adapters/session"],
"plan-handoff": ["agent-chat/handoff", "questions-and-approvals/plan-review"],
"clarify-reuse": ["projects/task-context"],
"multi-repository": ["projects/repositories"],
"hire-delegate-reuse": ["agents/hire", "task-coordination/delegation"],
"committed-send-retry": ["recovery/unknown-outcome"],
});
associate(["agent-chat-qualification"], {
"active-reassignment": ["tasks/properties"],
"worker-crash-retry": ["recovery/source-action"],
});
associate(["agent-chat-stories"], {
"enable-disable-resume": ["agent-chat/history"],
"followup-while-running": ["steering/queue"],
"revise-while-running": ["steering/steer"],
});
associate(["completion-updates"], Object.fromEntries([
"handoff-completion-idle", "handoff-completion-busy", "handoff-completion-multiple", "handoff-completion-restart",
].map((id) => [id, ["agent-chat/handoff", "tasks/completion"]])));
associate(["everyday-workflows", "native-connection-guidance", "grok-qualification", "grok-subscription-qualification"], {
"build-revise": ["documents-artifacts/work-products"],
"delegate-feedback": ["task-coordination/delegation", "steering/steer"],
"agent-review-handoff": ["task-coordination/signoff"],
"hire-reuse": ["agents/hire", "task-coordination/delegation"],
"service-approve": ["questions-and-approvals/tool-review", "app-permissions/action-policy"],
"service-decline": ["questions-and-approvals/tool-review", "app-permissions/action-policy"],
"connection-decline": ["connection-setup/task-request"],
"provider-decline": ["connection-setup/task-request"],
"provider-second": ["connection-setup/task-request"],
"provider-native": ["connection-setup/task-request"],
"agentmail-setup": ["connection-setup/task-request"],
"recover-controller": ["recovery/user-continuation", "documents-artifacts/work-products"],
"stop-redirect": ["steering/stop-and-pause"],
"create-skill-studio": ["skills/authoring"],
});
associate(["hiring-templates"], { "hire-coder-template-reuse": ["agents/hire", "task-coordination/delegation"] });
associate(["kimi-qualification"], {
"file-edit-validate": ["documents-artifacts/work-products"],
"native-write-permission-stop": ["steering/stop-and-pause", "questions-and-approvals/tool-review"],
"native-write-provider-loss": ["recovery/diagnosis", "questions-and-approvals/tool-review"],
});
associate(["task-workspaces"], {
"task-directory-repository-resume": ["workspaces/binding", "workspaces/provisioning", "projects/repositories"],
});
associate(["task-titles"], {
"prompt-title-standard": ["tasks/create"],
"prompt-title-ask": ["tasks/create"],
"preserve-explicit-title": ["tasks/create"],
});
associate(["pi-controls", "native-active-stop"], {
"pending-permission-stop": ["steering/stop-and-pause"],
"same-turn-steering": ["steering/steer"],
});
export function journeyCapabilities(execution: Pick<ReportExecution, "suite" | "task">) {
return (associations.get(`${execution.suite.id}/${execution.task.id}`) ?? [])
.map((id) => capabilities.get(id)!);
}
export function journeyFeatureIds(execution: Pick<ReportExecution, "suite" | "task">) {
return [...new Set(journeyCapabilities(execution).map((capability) => capability.feature.id))];
}