Files
PaperClipAI/tests/runner-e2e/plan-revision-flow.ts
T
DottaandClaude Opus 5.5 92308f839d test(e2e): use the Coordinator task panel, inbox and plan controls in QA journeys (#21701)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The QA Cloud product journeys check real user paths on a hosted test
stack
> - #19360 (2026-11-05) made the Coordinator workspace standard; its
task side panel starts with no tabs, and the header control opens a tab
chooser
> - The journeys were written for the classic panel, which opened
straight on Properties, and for older inbox and plan controls
> - On QA the smoke suite passed 8 of 24 journeys; 13 failures trace to
these out-of-date journey steps
> - This pull request updates the journeys to use the shipped
Coordinator controls the way a person does
> - The benefit is that the QA journeys test the product again instead
of failing on removed controls

## Linked Issues or Issue Description

**What happened?**
The QA staging smoke suite passed 8 of 24 journeys. 13 failures come
from journey steps that target the classic UI:
- Task side panel: the journeys click "Show properties" and expect
property rows. In Coordinator the click opens a tab chooser ("Open a
side panel tab"), and the user picks Properties or Artifacts. The
product's own e2e tests (`tests/e2e/artifact-tab-arrival.spec.ts`,
`artifact-uploads.spec.ts`, `disconnect-resilience.spec.ts`) and the
chat-catalog runtime cases already use the chooser.
- Inbox: desktop Coordinator inbox rows show unread state as a bold
title and have no per-row "Mark as read" button
(`doc/plans/2026-11-06-chat-contacts-pins.md`). The `r` shortcut marks
the hovered row read on the Mine view.
- Plan: the Coordinator Plan chip is named "Open Plan revision N in task
sidebar", not "Open Plan revision N".

**Steps to reproduce**
1. Run `pnpm test:e2e:runner -- --target-config <managed local config>`
with task-properties, task-edit-lifecycle, task-document-persistence and
project-task-context.
2. See 0/4 pass on master; each stops at the tab chooser.

**Expected behavior**
The journeys reach the same saved-state checks as before through the
shipped controls.

## What Changed

- New `product-journeys/task-panel.ts` helper: opens the side panel and
selects a tab (Properties or Artifacts) from the chooser, an open tab,
or the "Open a new tab" button. It waits for the page instead of a
single non-waiting visibility check, and does not close an already open
panel.
- task-properties, task-edit-lifecycle, task-dependency-resume,
project-task-context and task-document-persistence (and routine/agent
journeys that reuse `openDocumentAuthoring`) use the helper.
task-properties reopens the panel after reload before it checks "You".
- inbox-triage: when the row has no visible "Mark as read" button, open
the Mine view, blur search, hover the row and press `r` (as
`tests/e2e/coordinator-inbox-views.spec.ts` does).
- Plan journey: `planPreviewLabel()` accepts the optional " in task
sidebar" suffix and still requires the exact revision.
- No product code changes. An earlier revision of this PR changed "Show
properties" to select Properties; it is reverted because the product e2e
suite asserts the chooser is the intended design.

All saved-state assertions are unchanged: assignment, labels, status,
documents, read state and plan revision are still read back through the
API, so a broken feature still fails.

## Verification

- Real browser journeys on a managed local instance against unmodified
master UI: 4/4 pass (task-edit-lifecycle, task-properties,
task-document-persistence, project-task-context); 0/4 before.
- Inbox fallback checked by hand on a local instance: an unread task on
the Mine view became `isUnreadForMe=false` after blur + hover + `r`.
- `tests/runner-e2e` unit tests for product-journeys and plan-revision:
47 files, 1240 tests pass. The task-properties contract test now models
the chooser and asserts the opener and the Properties option are used
before assignment and after reload. New `planPreviewLabel` test covers
revision 12 vs 1 and a missing revision.
- QA Cloud rerun follows once the QA harness pin moves to a master
commit that contains this change.

## Risks

- Low risk: test code only. If the chooser changes again, the helper
fails visibly at the panel step.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 03:08:23 +00:00

203 lines
7.3 KiB
TypeScript

import { expect, type Page } from "@playwright/test";
import { pollUntil } from "./api.js";
import { numberedPlanStepCount } from "./run-observations.js";
export interface PlanInteraction {
id: string; kind?: string; status: string;
payload?: { acceptLabel?: string; rejectLabel?: string; target?: { type?: string; key?: string; revisionId?: string } };
}
export interface PlanDocument { id: string; key: string; body?: string | null; latestRevisionId?: string | null; latestRevisionNumber?: number }
interface PlanState {
currentIssue: { status: string };
taskRuns: Array<{ status: string }>;
interactions: PlanInteraction[];
}
const TERMINAL_RUN_STATUSES = new Set(["succeeded", "interrupted", "failed", "cancelled", "timed_out"]);
export function isPendingPlanConfirmation(interaction: PlanInteraction) {
return interaction.kind === "request_confirmation" && interaction.status === "pending"
&& interaction.payload?.target?.type === "issue_document" && interaction.payload.target.key === "plan"
&& typeof interaction.payload.target.revisionId === "string";
}
export function normalizePlanMarkdown(body: string | null | undefined) { return (body ?? "").replaceAll("\\_", "_"); }
export function planPreviewLabel(revision: number): RegExp {
return new RegExp(`^Open Plan revision ${revision}(?: in task sidebar)?$`);
}
export async function expectPlanStageVisible(page: Page, options: { native: boolean; revision: number | null }) {
const root = options.native ? page.locator('[data-testid="task-chat-turn"][data-settled="true"]') : page.locator("body");
const preview = root.getByTestId("task-chat-plan-preview").last();
await expect(preview).toBeVisible({ timeout: 30_000 });
if (options.revision !== null) await expect(preview).toHaveAttribute("aria-label", planPreviewLabel(options.revision));
}
/** The classic matrix and deployment-targeted journeys share this exact user story. */
export async function runPlanRevisionAcceptance<S extends PlanState>(input: {
page: Page; api: { get<T>(route: string): Promise<T> }; origin: string;
issue: { id: string; identifier?: string | null }; issuePrefix: string; native: boolean;
planMarkers: { draft: string; revised: string }; revisionRequest: string; deadlineAt: number;
load(): Promise<S>;
retryOnError?: (error: unknown) => boolean;
reject(state: { taskRuns: S["taskRuns"]; interactions: S["interactions"]; minimumRunCount: number }): string | undefined;
capture(id: string, label: string, file: string): Promise<void>;
}) {
const { page, api, origin, issue, issuePrefix, native, planMarkers, revisionRequest, deadlineAt, load, reject, capture } = input;
const draftState = await pollUntil({
label: `initial plan confirmation for issue ${issue.id}`,
deadlineAt,
load,
retryOnError: input.retryOnError,
accept: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status !== "done" &&
taskRuns.length >= 1 &&
taskRuns.every((run) => TERMINAL_RUN_STATUSES.has(run.status)) &&
interactions.some(isPendingPlanConfirmation),
reject: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status === "done" ? "Plan completed before approval" : reject({
taskRuns,
interactions,
minimumRunCount: 1,
}),
});
const draftInteraction = draftState.interactions.find(
isPendingPlanConfirmation,
)!;
const draftPlan = await api.get<PlanDocument>(
`/api/issues/${issue.id}/documents/plan`,
);
if (
!normalizePlanMarkdown(draftPlan.body).includes(planMarkers.draft)
) {
throw new Error(
`Initial Plan document did not contain ${planMarkers.draft}`,
);
}
if (numberedPlanStepCount(draftPlan.body) !== 2) {
throw new Error(
"Initial Plan must contain exactly two numbered steps",
);
}
if (
draftInteraction.payload?.target?.revisionId !==
draftPlan.latestRevisionId
) {
throw new Error(
"Initial plan confirmation did not target the latest Plan revision",
);
}
await page.goto(
`${origin}/${encodeURIComponent(issuePrefix)}/issues/${encodeURIComponent(issue.identifier ?? issue.id)}`,
{ waitUntil: "domcontentloaded" },
);
await expectPlanStageVisible(page, {
native,
revision: draftPlan.latestRevisionNumber ?? null,
});
await capture(
"plan-draft",
"Initial plan awaiting revision",
"plan-draft.png",
);
await page
.getByRole("button", {
name: draftInteraction.payload?.rejectLabel ?? "Reject",
exact: true,
})
.last()
.click();
const revisionComposer = page
.getByTestId("plan-revision-composer")
.last();
await expect(revisionComposer).toBeVisible({ timeout: 10_000 });
await revisionComposer
.locator('[contenteditable="true"], textarea')
.first()
.fill(revisionRequest);
await page
.getByRole("button", {
name: draftInteraction.payload?.rejectLabel ?? "Reject",
exact: true,
})
.last()
.click();
const revisedState = await pollUntil({
label: `revised plan confirmation for issue ${issue.id}`,
deadlineAt,
load,
retryOnError: input.retryOnError,
accept: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status !== "done" &&
taskRuns.length >= 2 &&
taskRuns.every((run) => TERMINAL_RUN_STATUSES.has(run.status)) &&
interactions.some(
(interaction) =>
isPendingPlanConfirmation(interaction) &&
interaction.id !== draftInteraction.id,
),
reject: ({ currentIssue, taskRuns, interactions }) =>
currentIssue.status === "done" ? "Plan completed before approval" : reject({
taskRuns,
interactions,
minimumRunCount: 2,
}),
});
const revisedInteraction = revisedState.interactions.find(
(interaction) =>
isPendingPlanConfirmation(interaction) &&
interaction.id !== draftInteraction.id,
)!;
const revisedPlan = await api.get<PlanDocument>(
`/api/issues/${issue.id}/documents/plan`,
);
const normalizedRevisedPlan = normalizePlanMarkdown(revisedPlan.body);
if (
!normalizedRevisedPlan.includes(planMarkers.revised) ||
normalizedRevisedPlan.includes(planMarkers.draft)
) {
throw new Error(
`Revised Plan must replace ${planMarkers.draft} with ${planMarkers.revised}`,
);
}
if (numberedPlanStepCount(revisedPlan.body) !== 3) {
throw new Error(
"Revised Plan must contain exactly three numbered steps",
);
}
if (
revisedInteraction.payload?.target?.revisionId !==
revisedPlan.latestRevisionId ||
revisedPlan.latestRevisionId === draftPlan.latestRevisionId
) {
throw new Error(
"Revised confirmation did not target a new latest Plan revision",
);
}
await page.goto(
`${origin}/${encodeURIComponent(issuePrefix)}/issues/${encodeURIComponent(issue.identifier ?? issue.id)}`,
{ waitUntil: "domcontentloaded" },
);
await expectPlanStageVisible(page, {
native,
revision: revisedPlan.latestRevisionNumber ?? null,
});
await capture(
"plan-revised",
"Revised plan awaiting acceptance",
"plan-revised.png",
);
await page
.getByRole("button", {
name: revisedInteraction.payload?.acceptLabel ?? "Approve",
exact: true,
})
.last()
.click();
return {
draftInteraction,
draftPlan,
revisedInteraction,
revisedPlan,
revisionRequest,
};
}