Files
PaperClipAI/tests/runner-e2e/plan-revision-flow.ts
T
DottaandClaude Opus 5.5 6400698a8e test(e2e): find the native Plan chip in the Coordinator chat thread (#21727)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The QA Cloud plan-revise-accept journey checks that a person can
revise and approve an agent's Plan
> - It first checks that the saved Plan chip is visible, scoped to a
settled `task-chat-turn` so a live "Writing plan" chip cannot count
> - The Coordinator workspace renders the Plan chip directly in the chat
thread; there is no `task-chat-turn` element and no `data-settled`
attribute
> - So the journey never finds the chip and fails before the person
reviews the plan
> - This pull request also accepts the chip in the chat thread
> - The benefit is that the plan journey reaches the revision and
approval steps it exists to test

## Linked Issues or Issue Description

**What happened?**
On QA Cloud (build 5acf27bc), plan-revise-accept failed with
`journey_operation_failed`, no checks passed, one run. The agent saved a
correct two-step Plan and a revision-bound confirmation. The
confirmation stayed pending until cleanup closed the task, so the
browser never reached it.

**Steps to reproduce**
1. Open the failed journey task on QA (Coordinator workspace).
2. Query `[data-testid="task-chat-turn"][data-settled="true"]
[data-testid="task-chat-plan-preview"]`: 0 matches.
3. Query `[data-testid="task-chat-thread"]
[data-testid="task-chat-plan-preview"]`: 1 visible match, aria-label
"Open Plan revision 1 in task sidebar".

**Expected behavior**
The journey finds the saved Plan chip and continues to "Request
changes".

## What Changed

- `planStageRootSelector(native)`: native root is a settled task-chat
turn or the task chat thread. `expectPlanStageVisible` uses it. The
revision-name check from #21701 is unchanged.

Why this still detects the failure that matters:
`runPlanRevisionAcceptance` waits through the API until every task run
is terminal before this check, and the aria-label must name the exact
revision, which a live "Writing plan" chip does not.

## Verification

- On the live failed QA page: old scope finds 0 chips; new scope finds 1
visible chip whose label matches `planPreviewLabel(1)`.
- `tests/runner-e2e` plan-revision tests: 4 pass (new selector test
included).
- QA Cloud rerun of plan-revise-accept follows after merge and the
harness pin update.

## Risks

- Low risk: journey code only.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 06:12:41 +00:00

207 lines
7.4 KiB
TypeScript

import { expect, type Page } from "@playwright/test";
import { pollUntil } from "./api.js";
import { numberedPlanStepCount } from "./run-observations.js";
export interface PlanInteraction {
id: string; kind?: string; status: string;
payload?: { acceptLabel?: string; rejectLabel?: string; target?: { type?: string; key?: string; revisionId?: string } };
}
export interface PlanDocument { id: string; key: string; body?: string | null; latestRevisionId?: string | null; latestRevisionNumber?: number }
interface PlanState {
currentIssue: { status: string };
taskRuns: Array<{ status: string }>;
interactions: PlanInteraction[];
}
const TERMINAL_RUN_STATUSES = new Set(["succeeded", "interrupted", "failed", "cancelled", "timed_out"]);
export function isPendingPlanConfirmation(interaction: PlanInteraction) {
return interaction.kind === "request_confirmation" && interaction.status === "pending"
&& interaction.payload?.target?.type === "issue_document" && interaction.payload.target.key === "plan"
&& typeof interaction.payload.target.revisionId === "string";
}
export function normalizePlanMarkdown(body: string | null | undefined) { return (body ?? "").replaceAll("\\_", "_"); }
export function planPreviewLabel(revision: number): RegExp {
return new RegExp(`^Open Plan revision ${revision}(?: in task sidebar)?$`);
}
export function planStageRootSelector(native: boolean): string {
return native ? '[data-testid="task-chat-turn"][data-settled="true"], [data-testid="task-chat-thread"]' : "body";
}
export async function expectPlanStageVisible(page: Page, options: { native: boolean; revision: number | null }) {
const root = page.locator(planStageRootSelector(options.native));
const preview = root.getByTestId("task-chat-plan-preview").last();
await expect(preview).toBeVisible({ timeout: 30_000 });
if (options.revision !== null) await expect(preview).toHaveAttribute("aria-label", planPreviewLabel(options.revision));
}
/** The classic matrix and deployment-targeted journeys share this exact user story. */
export async function runPlanRevisionAcceptance<S extends PlanState>(input: {
page: Page; api: { get<T>(route: string): Promise<T> }; origin: string;
issue: { id: string; identifier?: string | null }; issuePrefix: string; native: boolean;
planMarkers: { draft: string; revised: string }; revisionRequest: string; deadlineAt: number;
load(): Promise<S>;
retryOnError?: (error: unknown) => boolean;
reject(state: { taskRuns: S["taskRuns"]; interactions: S["interactions"]; minimumRunCount: number }): string | undefined;
capture(id: string, label: string, file: string): Promise<void>;
}) {
const { page, api, origin, issue, issuePrefix, native, planMarkers, revisionRequest, deadlineAt, load, reject, capture } = input;
const draftState = await pollUntil({
label: `initial plan confirmation for issue ${issue.id}`,
deadlineAt,
load,
retryOnError: input.retryOnError,
accept: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status !== "done" &&
taskRuns.length >= 1 &&
taskRuns.every((run) => TERMINAL_RUN_STATUSES.has(run.status)) &&
interactions.some(isPendingPlanConfirmation),
reject: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status === "done" ? "Plan completed before approval" : reject({
taskRuns,
interactions,
minimumRunCount: 1,
}),
});
const draftInteraction = draftState.interactions.find(
isPendingPlanConfirmation,
)!;
const draftPlan = await api.get<PlanDocument>(
`/api/issues/${issue.id}/documents/plan`,
);
if (
!normalizePlanMarkdown(draftPlan.body).includes(planMarkers.draft)
) {
throw new Error(
`Initial Plan document did not contain ${planMarkers.draft}`,
);
}
if (numberedPlanStepCount(draftPlan.body) !== 2) {
throw new Error(
"Initial Plan must contain exactly two numbered steps",
);
}
if (
draftInteraction.payload?.target?.revisionId !==
draftPlan.latestRevisionId
) {
throw new Error(
"Initial plan confirmation did not target the latest Plan revision",
);
}
await page.goto(
`${origin}/${encodeURIComponent(issuePrefix)}/issues/${encodeURIComponent(issue.identifier ?? issue.id)}`,
{ waitUntil: "domcontentloaded" },
);
await expectPlanStageVisible(page, {
native,
revision: draftPlan.latestRevisionNumber ?? null,
});
await capture(
"plan-draft",
"Initial plan awaiting revision",
"plan-draft.png",
);
await page
.getByRole("button", {
name: draftInteraction.payload?.rejectLabel ?? "Reject",
exact: true,
})
.last()
.click();
const revisionComposer = page
.getByTestId("plan-revision-composer")
.last();
await expect(revisionComposer).toBeVisible({ timeout: 10_000 });
await revisionComposer
.locator('[contenteditable="true"], textarea')
.first()
.fill(revisionRequest);
await page
.getByRole("button", {
name: draftInteraction.payload?.rejectLabel ?? "Reject",
exact: true,
})
.last()
.click();
const revisedState = await pollUntil({
label: `revised plan confirmation for issue ${issue.id}`,
deadlineAt,
load,
retryOnError: input.retryOnError,
accept: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status !== "done" &&
taskRuns.length >= 2 &&
taskRuns.every((run) => TERMINAL_RUN_STATUSES.has(run.status)) &&
interactions.some(
(interaction) =>
isPendingPlanConfirmation(interaction) &&
interaction.id !== draftInteraction.id,
),
reject: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status === "done" ? "Plan completed before approval" : reject({
taskRuns,
interactions,
minimumRunCount: 2,
}),
});
const revisedInteraction = revisedState.interactions.find(
(interaction) =>
isPendingPlanConfirmation(interaction) &&
interaction.id !== draftInteraction.id,
)!;
const revisedPlan = await api.get<PlanDocument>(
`/api/issues/${issue.id}/documents/plan`,
);
const normalizedRevisedPlan = normalizePlanMarkdown(revisedPlan.body);
if (
!normalizedRevisedPlan.includes(planMarkers.revised) ||
normalizedRevisedPlan.includes(planMarkers.draft)
) {
throw new Error(
`Revised Plan must replace ${planMarkers.draft} with ${planMarkers.revised}`,
);
}
if (numberedPlanStepCount(revisedPlan.body) !== 3) {
throw new Error(
"Revised Plan must contain exactly three numbered steps",
);
}
if (
revisedInteraction.payload?.target?.revisionId !==
revisedPlan.latestRevisionId ||
revisedPlan.latestRevisionId === draftPlan.latestRevisionId
) {
throw new Error(
"Revised confirmation did not target a new latest Plan revision",
);
}
await page.goto(
`${origin}/${encodeURIComponent(issuePrefix)}/issues/${encodeURIComponent(issue.identifier ?? issue.id)}`,
{ waitUntil: "domcontentloaded" },
);
await expectPlanStageVisible(page, {
native,
revision: revisedPlan.latestRevisionNumber ?? null,
});
await capture(
"plan-revised",
"Revised plan awaiting acceptance",
"plan-revised.png",
);
await page
.getByRole("button", {
name: revisedInteraction.payload?.acceptLabel ?? "Approve",
exact: true,
})
.last()
.click();
return {
draftInteraction,
draftPlan,
revisedInteraction,
revisedPlan,
revisionRequest,
};
}