Files
PaperClipAI/tests/runner-e2e/user-actions.ts
T
DottaandPaperclip b81b870732 Add QA smoke journeys and OpenCode run traces (#18098)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Product E2E journeys verify saved application behavior across
deployments.
> - Task creation and questions need reusable smoke selections and exact
run attribution.
> - Operators also need to inspect the OpenCode execution behind a
measured question.
> - Company observability does not currently support native OpenCode.
> - This change adds the journeys and a bounded projection of normalized
OpenCode run events.
> - The projection follows the configured capture policy and retains the
current run identity across warm reuse.

## Linked Issues or Issue Description

Related foundation: #17262. Related alternative: #17989 exports metadata
from OpenCode's native OTLP stream. This change uses normalized run
events to support the existing content switches and per-run linkage. It
does not forward the native stream. The two PRs overlap in launch
registration and must be reconciled before both land.

**What existing behavior does this improve?**

Reusable Product E2E smoke selections and company observability for
native OpenCode.

**Current behavior**

The journey catalog lacks reusable task creation and attached-agent
question checks. Native OpenCode is marked unsupported for company
observability, so a successful destination connection test does not
prove that agent traces will appear.

**Proposed behavior**

Add explicit task-create and ask-question selections. Verify saved state
and exact company, issue, agent, model, environment and successful run
attribution. Export normalized OpenCode turns, tools and usage to the
configured OTLP destination. Honor each capture switch and redact known
secrets.

**Reason and benefit**

CI can run a small measured smoke selection, retain its screenshots and
checks, and link results to traces belonging to the same run.

**Breaking changes**

Existing selectors remain unchanged. OpenCode export is opt-in through
existing company settings. Legacy OpenCode stays unsupported.

## What Changed

- Add reusable task creation and question journeys, configured-agent
capability, owned-fixture checkpoints and strict evidence validation.
- Add an OpenCode-owned projection of normalized run events. No provider
SDK or raw payload schema enters heartbeat coordination.
- Extend the private launch union and reserved environment allowlist for
native OpenCode. With export disabled, no receiver request runs.
- Keep destination headers in the parent wrapper and out of the actual
provider child environment and configuration.
- Bound each turn to 64 tool and 64 usage entries, 8 KiB per text field
and a 1 MiB request. Use a three-second request deadline, four
concurrent exports, and no retries or idle worker.
- Pass the rotated runner authority to each proxy turn. A retained
process no longer attributes later turns to its original launch run.
- Update settings copy, observability documentation and capability maps.

## Verification

- `pnpm -r typecheck` and `pnpm build` passed after the implementation
changes.
- Adapter launch/redaction: 9 tests passed. Projection and actual proxy:
12 tests passed before the review fix; the expanded projection suite
passed all 12 cases after the fix. The proxy test exercises two runs in
one process against a real local HTTP receiver and a fake provider.
- Full OpenCode driver/projection suite: 72 passed, one existing
response_wake test timed out at five seconds. A focused check then
passed both existing omitted-continuation cases and the new trace export
case. The timeout cause remains unconfirmed.
- Runner package TypeScript and Rust `cargo check` passed. Map and UI
token checks passed.
- Prior journey validation: 273 tests across 12 files and the journey
TypeScript check passed.
- Broad local `pnpm test:run` encountered native-runtime integration
failures with repeated embedded PostgreSQL initialization failures and
was stopped. All 16 failing cases passed in an isolated run outside the
filesystem sandbox (961 unrelated cases unselected). The broad local run
is not a clean acceptance result. A later focused database-backed
chat/heartbeat run passed all six selected observability cases,
including native OpenCode launch and successful allocation (1,809
unrelated cases were unselected).
- All CI gates passed for `21085f880d8babc9efeb22d0623db99613d33093`,
including packaged-runner, clean-install, browser and build checks:
https://github.com/paperclipai/paperclip/actions/runs/40884658995 .
Greptile reviewed the same commit at 5/5 with no unresolved findings.
- Verified OTLP JSON acceptance and indexing with an explicitly
synthetic transport marker. No live OpenCode model trace or deployed
smoke acceptance is claimed yet.

## Risks

Export is best effort. Abrupt termination, recovered in-flight turns
without a new submitted event, limits or receiver failure can lose
traces. Export failure does not change task success. The projection
represents observed normalized events, not raw native model spans or
hidden reasoning. Known-secret redaction is not an OS isolation
boundary. UI changes can break journey locators. Deployment adapters
retain credential, ownership and cleanup responsibility. Both the
controller and native runner bundle must be updated before live
qualification.

## Model Used

OpenAI Codex, GPT-6, with reasoning, code execution and browser tools.
The exact serving model ID and context size are not exposed to this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted suites; broad
local limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 08:57:12 -05:00

145 lines
6.8 KiB
TypeScript

import { expect, type Locator, type Page } from "@playwright/test";
/** Rich editors interpret pasted Markdown; fill inserts literal paragraph text. */
export async function fillTaskPrompt(editor: Locator, prompt: string): Promise<void> {
if (!/^ {0,3}(?:`{3,}|~{3,})/m.test(prompt)
|| await editor.evaluate(element => element.tagName === "TEXTAREA")) {
await editor.fill(prompt);
return;
}
await editor.fill("");
await editor.focus();
await editor.evaluate((element, text) => {
const clipboardData = new DataTransfer();
clipboardData.setData("text/plain", text);
const event = new ClipboardEvent("paste", { clipboardData, bubbles: true, cancelable: true });
element.dispatchEvent(event);
if (!event.defaultPrevented) throw new Error("Task editor did not accept the Markdown paste");
}, prompt);
}
export async function createTaskThroughUi(input: {
page: Page;
origin?: string;
issuePrefix: string;
agentName: string;
title: string;
prompt: string;
workMode: "standard" | "planning" | "ask";
projectName?: string;
requireExplicitTitle?: boolean;
attachments?: readonly string[];
beforeDispatch?: () => void | Promise<void>;
}) {
// Search's public creation action exposes the explicit title field. Strict
// native-operation fixtures use it so automatic task naming is not an extra
// provider mutation before the operation whose permission is under test.
const issuesUrl = input.requireExplicitTitle
? `/${encodeURIComponent(input.issuePrefix)}/search?scope=issues&q=${encodeURIComponent(`"${input.title}"`)}`
: `/${encodeURIComponent(input.issuePrefix)}/issues`;
const newTask = input.requireExplicitTitle
? input.page.getByRole("button", { name: "Create task from this query", exact: true })
: input.page.getByRole("button", { name: /^New task$/i }).first();
let bootstrapError: unknown;
for (let bootstrapAttempt = 1; bootstrapAttempt <= 3; bootstrapAttempt += 1) {
try {
await input.page.goto(input.origin ? new URL(issuesUrl, input.origin).href : issuesUrl, {
waitUntil: "domcontentloaded",
timeout: 30_000,
});
await newTask.waitFor({ state: "visible", timeout: 20_000 });
bootstrapError = undefined;
break;
} catch (error) {
bootstrapError = error;
if (bootstrapAttempt < 3) await input.page.waitForTimeout(1_000);
}
}
if (bootstrapError) {
throw new Error(
`Browser bootstrap failed before task creation: ${bootstrapError instanceof Error ? bootstrapError.message : String(bootstrapError)}`,
{ cause: bootstrapError },
);
}
await newTask.click();
const dialog = input.page.getByRole("dialog", { name: "New task", exact: true });
await expect(dialog).toBeVisible();
const titleInput = dialog.getByRole("textbox", { name: "Task title", exact: true });
if (input.requireExplicitTitle) {
await expect(titleInput).toBeVisible();
await titleInput.fill(input.title);
} else if (await titleInput.isVisible()) await titleInput.fill(input.title);
await fillTaskPrompt(dialog.getByRole("textbox", { name: "editable markdown", exact: true }), input.prompt);
if (input.workMode !== "standard") {
await dialog.getByRole("button", { name: "Add to composer", exact: true }).click();
await input.page.getByTestId(input.workMode === "planning" ? "composer-add-plan" : "composer-add-ask").click();
}
await dialog
.getByRole("button", { name: "Select assignee", exact: true })
.click();
await input.page.getByRole("searchbox", { name: "Search assignees", exact: true }).fill(input.agentName);
await input.page.getByRole("option").filter({ has: input.page.getByText(input.agentName, { exact: true }) }).click();
await expect(input.page.getByRole("searchbox", { name: "Search assignees", exact: true })).toBeHidden();
if (input.projectName) {
// Selecting the assignee advances focus to this selector and opens it.
// Focus is idempotent here; clicking would toggle an already-open popover
// closed before the search field can be filled.
const projectTrigger = dialog.getByRole("group", { name: "Task project and worktrees", exact: true }).getByRole("button").first();
await projectTrigger.focus();
const projectSearch = input.page.getByPlaceholder(/^Search projects(?:…|\.\.\.)?$/i);
await expect(projectSearch).toBeVisible({ timeout: 2_000 }).catch(async () => {
if (await projectTrigger.getAttribute("aria-expanded") !== "true") await projectTrigger.click();
await expect(projectSearch).toBeVisible();
});
await projectSearch.fill(input.projectName);
await input.page.getByText(input.projectName, { exact: true }).last().click();
}
if (input.attachments?.length) {
const chooser = input.page.waitForEvent("filechooser");
const upload = dialog.getByRole("button", { name: "Upload", exact: true });
if (await upload.count()) await upload.click();
else {
await dialog.getByRole("button", { name: "Add to composer", exact: true }).click();
await input.page.getByRole("menuitem", { name: /^Files and images/ }).click();
}
await (await chooser).setFiles([...input.attachments]);
// Upload finishes as part of task creation; the dialog retains the selected files.
}
// The prompt-only composer generates a provisional title which the provider
// can rename immediately. Bind the fixture to the actual creation response.
await input.beforeDispatch?.();
const submittedAtMs = Date.now();
const [response] = await Promise.all([
input.page.waitForResponse(response => response.request().method() === "POST"
&& /^\/api\/companies\/[^/]+\/issues$/.test(new URL(response.url()).pathname)),
dialog.getByRole("button", { name: "Create task", exact: true }).click(),
]);
expect(response.status()).toBe(201);
const issue = await response.json() as { id: string; companyId: string; title: string; titleNeedsGeneration: boolean };
expect(new URL(response.url()).pathname).toBe(`/api/companies/${issue.companyId}/issues`);
expect(issue.id).toEqual(expect.any(String));
expect(issue.id.length).toBeGreaterThan(0);
if (input.requireExplicitTitle) {
expect(issue.title).toBe(input.title);
expect(issue.titleNeedsGeneration).toBe(false);
}
// Creation precedes uploads; allow them to finish before navigation.
await expect(dialog).not.toBeVisible({ timeout: 30_000 });
return { submittedAtMs, issueId: issue.id };
}
export async function submitTaskReply(
page: Page,
body: string,
): Promise<number> {
const composer = page.getByTestId("task-chat-composer-input").last();
await expect(composer).toBeVisible({ timeout: 30_000 });
await composer
.locator('[contenteditable="true"], textarea')
.first()
.fill(body);
const submittedAtMs = Date.now();
await page.getByTestId("task-chat-composer-send").last().click();
return submittedAtMs;
}