mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 08:28:48 +01:00
## Thinking Path > - Paperclip manages agents and their work. > - A provider integration must work through ordinary agent setup and task controls. > - The recorded Antigravity walkthrough exposed setup, delivery, and continuation failures. > - A repeatable check must verify real work and retain failed attempts. > - This pull request converts all 47 recorded steps into one fixed Product E2E journey. > - Maintainers can rerun the case and inspect independent delivery, cleanup, and accounting evidence. ## Linked Issues or Issue Description Builds on merged #18187, #18188, and #18189, and consumes the shared repairs from merged #18230 and the equivalent manual-policy declaration from #18222. Refs #18214 for the CI task wrapper. Refs #12503 and #14938 for related legacy adapter work. **Agent or provider** Google Antigravity native Runner with a saved managed Google subscription and the exact Low model. **Why this adapter is useful** A user must be able to add an agent, connect its account, and receive real task results across a follow-up. Picker visibility alone is insufficient. **How the agent is invoked** A fixed Playwright driver uses ordinary product controls. The agent runs through native ACP. Independent checks use public APIs, actual downloads, owned process observations, and native lifetime fences. ## What Changed - Register one explicit-only case: `antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn`. - Map all 47 source steps and declare saved-account prerequisites, finite limits, and same-host macOS ARM64 scope. - Create an agent through the ordinary picker, connect the saved account, select Low, run Test, and wait for readiness. Save an instruction marker, deliver a file, and update it through an ordinary follow-up. - Require each run's visible completed file activity, exact downloaded bytes, committed Done result, same tagged conversation, distinct physical owner birth, released lease, and free native fences. Verify reload persistence. - Stop failed native runs that still own a retry. Replay an existing Stop with its durable request identity and wait for the matching audited acknowledgement. A terminal reservation without a native intent remains unconfirmed. - Keep private evidence in the existing result, report, dashboard, and accounting pipeline. Retain the paused QA agent and financial history while preserving the shared account and budget policies. - Journal only the ordinary wizard's future ID, target, name, intended harness and expected creator before forwarding its POST unchanged. Bind the response and interrupted recovery to exact identity reads; a pending 404 stays unconfirmed. - Serialize private resource snapshots through temporary-file fsync, atomic rename and directory fsync. Abort forwarding on storage failure and preserve legacy receipts. If later journal or diagnostic writes fail, retain the primary error, always close the browser, and attempt the ordinary failed report. - Add production-component browser controls and harden task-menu readiness, alias selection, scroll restoration, failure packaging, and one hermetic auth-test budget. - Wait for the existing GitHub recovery fixture to register its next deadline before asserting the original range; preserve its nested-transaction and zero-request checks. In the decision-expiry fixture, verify the exact saved expiry separately from the earlier global wake of another queue. ## Verification - Fresh registered acceptance passed on eval source `9060f3cd01b7c3445ecdf703d72e9cf30fe42b59` and merged serving product `61bd4ac6919c2309f766a25e62273b074cee3261`. It completed ordinary setup and two turns with native `.5` and `gemini-3.8-flash-low`. All 16 matchers, 14 checkpoints, and cleanup passed in **165,613 ms**. Actual downloads were **61 bytes** and **119 bytes**. Both turns passed their own run-bound browser activity checks and reload persistence. - Independent read-back confirms Done, a paused retained agent, the same connected saved default account, and no active owned run. Both accepted results have the same full tagged conversation and distinct primary process births. All 21 observed processes retired; both leases released and all three native fences were free after each turn. The selected `.5` checkpoints also showed equal agent/backend session IDs; this is an observed case requirement, not a generic ACP invariant. - Receipts report 339,273 input tokens, 7,224 output tokens, and 202,846 cached input tokens. Subscription prices are unavailable. Billing coverage remains incomplete, not free. - The normal report CLI passed. All 310 files from nine original attempts and two separate recoveries retained their hashes. The twelve current screenshots are byte-identical, and 194 local report links are contained and valid. The eight earlier reports were copied without regeneration. - Delivery source `1bf4fb04df04710b82844b3c9355767f46b5f68f` branches from `d3f2a60e795fc60d64c8b19d1eed4d48657d5492` and merges cleanly with current master `1d571de5f9f886d66e520800baec8fc2f67d0948`. All 41 prior `c8b332` authored file blobs remain identical; the 42nd path is the decision-expiry fixture contract correction. Creation ownership, the strict durable writer, and the ordinary successful workflow remain unchanged. Of 43 measured file blobs, 30 remain identical. The Antigravity oracle, artifact, billing and report gates and its suite-specific validation block remain unchanged. The actual retained agent matches its original creator; its original request `setupId` was not captured and is not reconstructed. - Later master changes include model-visible workspace guidance and rejected-terminal-report handling. Their bounded source assessment is separate: the first actual finish failed, then a same-turn replacement was accepted and committed. Later read-only results match both original native-result hashes, and the complete event catalogs have no `session.failed` event. Master #18228 also expands detection of explicit JSON file requests. The measured create/update-and-attach `.txt` task already required publication under both parsers; the publication validation gate itself is unchanged. Master #18240 changes remote workspace cache staging and adds an OpenCode index refresh. The selected local lane returns before those paths. Master #18213 adds optional numeric turn timing and trace boundaries. Its selected Antigravity model, text, IDs, result and retirement semantics remain unchanged in the bounded source review. Local diagnostic rows and awaits can change event ordinals and latency; the oracle checks relative semantic order. Collector coverage and latency improvement remain outside this proof. This does not claim that the whole current runtime is byte-identical to serving `61bd`, or relabel the measured result as a new run. - The adopted upstream cache controls passed 3/3 with pinned real Turbo. Forced work disables local and remote reads and writes; ordinary work reuses local results. The original genuine-pnpm/Turbo controls remain historical to `9060`'s earlier write-only policy. The duplicate cache patch is superseded by #18230. Its qualification-only pnpm addition is removed because the adopted upstream controls do not require it. - The E2E typecheck, map checks and 277-control replay are carried from the unchanged eval source paths. That replay passed, including the 183 Antigravity controls, catalog/connection checks and the separate product-journey validator. The carried creation controls include 91 config/recovery cases. The final launcher suite passed 19/19, including five real campaign finalization fault cases; the report suite passed 7/7. These final checks did not rerun the earlier 277-control group. The canonical production-component browser spec passed in 27.4 s under its unchanged 60 s limit before the replay. Four paused-composer cases and the product base's build/typecheck plus 376 focused regressions were verified earlier. - The shipped wizard and actual API client cover persistence before forwarding, lost response JSON, foreign/malformed requests, wrong response identity/creator and cleanup-error preservation. Real filesystem controls failed 5/5 before the writer repair and passed 5/5 afterward, preserving the old journal on partial-write, temporary-file sync and rename faults. Three persistent-sync campaign controls failed before the finalization repair and passed afterward. All five final campaign fault cases preserve the primary failure, cleanup failure, usage and failed report while attempting browser closure. The strict journal still blocks the next mutation. These are provider-free ownership controls; the Google task proof remains the original `9060` → `61bd` acceptance. - The decision restart check passed in the local 20-case fixture run. A current-source real-PostgreSQL rollback control reproduced the exact 90,000 ms earlier global wake: the old equality rejected it, while the exact durable expiry and new future-wake bound passed. The original 60 s open-state, exact-timestamp, continuation and quiet checks remain. The broader local run was **FAILED: 19 passed / 1 failed**, with a 15 s timeout and a 30 s cleanup timeout in an unchanged timestamp-pagination case. That case passed in canonical `c8b332` CI; the local timeout cause remains inconclusive. An extra local server typecheck failed against stale cached Runner declarations; it did not check this excluded test file. Both limits remain recorded for fresh canonical CI. - A broader local GitHub recovery group passed 21/22. Its unchanged pagination case observed an unexpected config request; the log does not establish the cause. That result remains retained, and fresh current-head CI is the final broader gate. - Exact delivery-head CI completed with 55 successes, two skips, and one optional neutral review. Required `ci / verify` and `ci / e2e`, build, typecheck, and Runner checks passed. Fresh Greptile is 5/5 with zero unresolved threads. Optional Paperclip is neutral 4/5; its carried recommendations are dispositioned against the public Stop contract and selected live identity evidence. Seven earlier converted attempts retain their failed grades and partial evidence. Historical Paid8 and its later read-only per-run UI observation remain separate from the fresh acceptance above. Unknown prices and the older partial startup accounting remain explicit. Full lineage and raw evidence remain private in the normal report bundle. ## Risks - This case requires an authorized saved personal default Google subscription and finite operator candidate admission. It does not test fresh OAuth or establish global qualification. - The physical observer supports a same-host loopback macOS ARM64 target. Cloud, Linux, and other models remain outside this case. - A known-cost stop cannot cap missing prices or in-flight subscription charges. Existing operator policies remain unchanged. - Raw account/native identities stay private. Missing retirement proof keeps resources retained. The selected cleanup mode does not test agent purge. ## Model Used OpenAI Codex (GPT-6). The host did not expose the exact model revision or context limit. The model used tools for code edits, tests, browser automation, and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
177 lines
12 KiB
TypeScript
177 lines
12 KiB
TypeScript
import { expect, type Page } from "@playwright/test";
|
|
import { readFile } from "node:fs/promises";
|
|
import { ConnectionFailure } from "./connection-config.js";
|
|
|
|
/** Recent sidebar entries are bounded; the searchable roster contains every
|
|
* agent. Scope the exact visible name to the main roster, not sidebar recency. */
|
|
export async function openAntigravityAgentFromRoster(page: Page, agentName: string, timeout = 30_000) {
|
|
const navigation = page.locator("aside").getByRole("link", { name: "Agents", exact: true });
|
|
await expect(navigation).toHaveCount(1, { timeout });
|
|
await navigation.click({ timeout });
|
|
const toolbar = page.getByRole("toolbar", { name: "Agents controls", exact: true });
|
|
await toolbar.getByRole("textbox", { name: "Search agents", exact: true }).fill(agentName, { timeout });
|
|
const row = page.getByRole("main").getByRole("link").filter({ has: page.getByText(agentName, { exact: true }) });
|
|
await expect(row).toHaveCount(1, { timeout });
|
|
await row.click({ timeout });
|
|
}
|
|
|
|
/** Both sidebar presentations use ordinary task links: the coordinator uses
|
|
* the identifier and recent lists can repeat the same task or its immutable ID. */
|
|
export async function openAntigravityCreatedTask(page: Page, issue: { id: string; identifier?: string | null }, timeout = 30_000) {
|
|
const paths = [issue.id, issue.identifier].filter((value): value is string => Boolean(value))
|
|
.map(value => `a[href$="/issues/${encodeURIComponent(value)}"]:visible`);
|
|
// Every match is an exact alias of this captured task. Recents and expanded
|
|
// project lists may legitimately expose it more than once.
|
|
const task = page.locator("aside").locator(paths.join(", ")).first();
|
|
await expect(task).toHaveCount(1, { timeout });
|
|
await task.click({ timeout });
|
|
}
|
|
|
|
/** The ordinary picker is measured before any direct setup-route navigation. */
|
|
export async function selectAntigravityFromOrdinaryPicker(page: Page, timeout = 30_000) {
|
|
const dialog = page.getByRole("dialog", { name: "Choose an adapter", exact: true });
|
|
await expect(dialog).toBeVisible({ timeout });
|
|
const advanced = dialog.locator("details").filter({ has: page.locator("summary", { hasText: /^Advanced$/ }) });
|
|
if (await advanced.count()) await expect(advanced).not.toHaveAttribute("open", "", { timeout });
|
|
const radio = dialog.getByRole("radio", { name: /^Google Antigravity(?:\s+Candidate)?$/ });
|
|
await radio.scrollIntoViewIfNeeded({ timeout });
|
|
// The native input is visually hidden; its enclosing ordinary card is visible.
|
|
await expect(radio.locator("..")).toBeVisible({ timeout });
|
|
await radio.locator("..").click({ timeout });
|
|
await expect(radio).toBeChecked();
|
|
await expect(dialog.getByRole("button", { name: "Configure agent", exact: true })).toBeEnabled();
|
|
}
|
|
|
|
export async function assertAntigravityReady(page: Page, agentName: string, timeout = 120_000) {
|
|
const assign = page.getByRole("button", { name: `Assign ${agentName} a Task`, exact: true });
|
|
const finishing = page.getByRole("heading", { name: "Finishing agent setup", exact: true });
|
|
if (await finishing.isVisible()) await expect(assign).toBeDisabled({ timeout });
|
|
await expect(page.getByRole("heading", { name: "Your agent is ready", exact: true })).toBeVisible({ timeout });
|
|
await expect(assign).toBeEnabled({ timeout });
|
|
}
|
|
|
|
export async function assertAntigravityCleanComposer(page: Page, agentName: string, model: string, timeout = 30_000) {
|
|
const dialog = page.getByRole("dialog", { name: "New task", exact: true });
|
|
await expect(dialog).toBeVisible({ timeout });
|
|
await expect(dialog.getByTestId("new-issue-paused-assignee-note")).toBeHidden({ timeout });
|
|
await expect(dialog.getByRole("button", { name: "Select assignee", exact: true })).toContainText(agentName, { timeout });
|
|
const modelLabel = dialog.getByRole("button", { name: "Select model and effort", exact: true }).getByTestId("task-chat-composer-model-label");
|
|
await expect(modelLabel).toHaveText(model === "gemini-3.8-flash-low" ? /^(?:gemini-3\.8-flash-low|Gemini 3\.8 Flash \(Low\))$/ : model, { timeout });
|
|
}
|
|
|
|
export async function assertAntigravityTaskDone(page: Page, timeout = 30_000) {
|
|
const header = page.locator('[data-testid="issue-detail-header"]:visible');
|
|
await expect(header).toBeVisible({ timeout });
|
|
// The thread restores its bottom anchor after reload. Scrolling its entire
|
|
// large header fights that restoration and is unnecessary to read status.
|
|
const control = header.getByRole("button", { name: "Change status (current: Done)", exact: true });
|
|
await expect(control).toHaveCount(1, { timeout });
|
|
await expect(control).toBeVisible({ timeout });
|
|
const current = page.getByRole("navigation", { name: "breadcrumb", exact: true }).locator('[aria-current="page"]:visible');
|
|
if (await current.count()) {
|
|
await expect(current).toHaveCount(1, { timeout });
|
|
await expect(current.getByRole("img", { name: "Done", exact: true })).toBeVisible({ timeout });
|
|
}
|
|
}
|
|
|
|
export async function assertAntigravityFileActivity(page: Page, runId: string, timeout = 30_000) {
|
|
if (typeof runId !== "string" || !/^[a-zA-Z0-9_-]{1,128}$/u.test(runId)) throw new ConnectionFailure("browser_activity_run_identity_invalid");
|
|
// The shipped adapter prefixes both phase and item IDs with their run ID.
|
|
// Older successful rows must not satisfy this turn, or poison its checks.
|
|
const groups = page.locator(`section[data-testid="task-chat-activity-phase"][data-activity-group^="${runId}:"]`);
|
|
const histories = groups.getByTestId("task-chat-activity-phase-toggle");
|
|
const older = page.getByRole("button", { name: "More messages", exact: true });
|
|
await expect.poll(async () => await older.isVisible() || await histories.count() > 0, { timeout }).toBe(true);
|
|
if (await older.isVisible()) await older.click();
|
|
await expect.poll(() => histories.count(), { timeout }).toBeGreaterThan(0);
|
|
for (const toggle of await histories.all()) if (await toggle.getAttribute("aria-expanded") !== "true") await toggle.click();
|
|
const items = groups.locator(`li[data-activity-item-id^="${runId}:"]:visible`).filter({ has: page.locator("[data-activity-row]") });
|
|
await expect.poll(() => items.count(), { timeout }).toBeGreaterThanOrEqual(2);
|
|
await expect.poll(async () => {
|
|
let reads = 0, edits = 0;
|
|
for (const item of await items.all()) {
|
|
const row = item.locator("[data-activity-row]");
|
|
const family = await row.getAttribute("data-activity-family");
|
|
if (family !== null && family !== "tool_execution") continue;
|
|
let name: string | undefined;
|
|
if (family === null) {
|
|
// Persisted native tool_call/result entries use the legacy tool model.
|
|
// Its mono target is the adapter's summarized real call input.
|
|
const input = row.locator("span[title].font-mono");
|
|
if (await input.count() !== 1) continue;
|
|
const match = /^operation:\s*(read|edit|write),\s*target:\s*([^,]+)(?:,|$)/u.exec(await input.getAttribute("title") ?? "");
|
|
if (!match || match[2]!.trim() !== "antigravity-proof.txt") continue;
|
|
name = match[1]!;
|
|
}
|
|
const toggle = item.locator(":scope > button");
|
|
await expect(toggle).toHaveCount(1, { timeout });
|
|
if (await toggle.getAttribute("aria-expanded") !== "true") await toggle.click();
|
|
if (family === "tool_execution") {
|
|
const details = item.getByTestId("task-chat-provider-detail-list");
|
|
const value = (label: string) => details.locator(":scope > div").filter({ has: page.locator("dt").getByText(label, { exact: true }) }).locator("dd");
|
|
const operation = value("Operation");
|
|
if (await operation.count() !== 1) continue;
|
|
name = (await operation.innerText()).trim();
|
|
if (!["read", "edit", "write"].includes(name)) continue;
|
|
const target = value("Target");
|
|
await expect(target).toHaveCount(1, { timeout });
|
|
if ((await target.innerText()).trim() !== "antigravity-proof.txt") continue;
|
|
await expect(value("Status")).toHaveText("completed", { timeout });
|
|
} else {
|
|
// A non-latest legacy call can display completed-style copy while its
|
|
// phase is live. Require the visible settled-phase summary as well.
|
|
const phase = item.locator('xpath=ancestor::section[@data-testid="task-chat-activity-phase"][1]');
|
|
await expect(phase.locator("[data-completed-summary]")).toBeVisible({ timeout });
|
|
const result = item.getByTestId("task-chat-runner-activity-detail").locator("pre");
|
|
await expect(result).toBeVisible({ timeout });
|
|
await expect(result).toContainText(/\S/u, { timeout });
|
|
await expect(row.locator(":scope > span:nth-child(2) > span:first-child"))
|
|
.toHaveText(/^(?:Ran|Read|Edited|Wrote|Updated|Applied|Created)\b/u, { timeout });
|
|
}
|
|
await expect(row).toBeVisible({ timeout });
|
|
await expect(row).not.toContainText(/·\s*(?:failed|stopped)/u, { timeout });
|
|
await expect(row.locator('[data-conversation-tool-running="true"]')).toHaveCount(0, { timeout });
|
|
if (name === "read") reads++; else if (name === "edit" || name === "write") edits++;
|
|
}
|
|
return reads > 0 && edits > 0;
|
|
}, { timeout }).toBe(true);
|
|
}
|
|
|
|
/** Open the card bound to the independently observed attachment, then download
|
|
* through its real file viewer. A stale same-named first-turn card cannot pass. */
|
|
export async function inspectAntigravityArtifact(page: Page, attachmentId: string, expected: Buffer, timeout = 30_000) {
|
|
if (!/^[a-zA-Z0-9-]+$/u.test(attachmentId)) throw new ConnectionFailure("browser_attachment_identity_invalid");
|
|
const contentPath = `/api/attachments/${attachmentId}/content`;
|
|
const boundLink = page.locator(`a[href="${contentPath}"], a[href="${contentPath}?download=1"]`);
|
|
const cards = page.locator('article[data-testid="task-chat-rich-work-product-artifact"]:visible').filter({ has: boundLink });
|
|
// Work-product titles are human labels, for example "Antigravity Proof File".
|
|
// Their exact attachment link, not that title, binds the ordinary open action.
|
|
const opening = page.getByRole("button", { name: /^Open in tab: /u }).and(cards.getByRole("button", { name: /^Open in tab: /u }));
|
|
const rows = page.locator('[data-artifact-row]:visible, [data-slot="attachment"]:visible')
|
|
.filter({ has: page.getByRole("button", { name: "Open antigravity-proof.txt", exact: true }).or(page.getByRole("link", { name: "Open antigravity-proof.txt", exact: true })) })
|
|
.filter({ has: page.getByText(new RegExp(`(?:^|\\s)${expected.length} B$`, "u")) });
|
|
const boundRows = rows.filter({ has: boundLink });
|
|
const rowOpening = boundRows.getByRole("link", { name: "Open antigravity-proof.txt", exact: true });
|
|
const buttonOnly = rows.filter({ hasNot: page.locator("a[href]") }).getByRole("button", { name: "Open antigravity-proof.txt", exact: true });
|
|
await expect.poll(async () => await opening.count() + await rowOpening.count() + await buttonOnly.count(), { timeout }).toBeGreaterThan(0);
|
|
if (await opening.count()) {
|
|
await expect(opening).toHaveCount(1, { timeout });
|
|
const card = opening.locator('xpath=ancestor::*[self::article or @data-slot="card"][1]');
|
|
await expect(card.getByText(new RegExp(`(?:^|\\s)${expected.length} B$`, "u"))).toBeVisible({ timeout });
|
|
await opening.click({ timeout });
|
|
} else {
|
|
const control = await rowOpening.count() ? rowOpening : buttonOnly;
|
|
await expect(control).toHaveCount(1, { timeout });
|
|
await control.click({ timeout });
|
|
}
|
|
const raw = page.locator('pre[aria-label="antigravity-proof.txt raw text"]:visible');
|
|
await expect(raw).toHaveCount(1, { timeout });
|
|
await expect(raw).toHaveText(expected.toString("utf8"), { useInnerText: false, timeout });
|
|
const download = page.getByRole("link", { name: "Download antigravity-proof.txt", exact: true })
|
|
.and(page.locator(`a[download][href="${contentPath}?download=1"]`));
|
|
await expect(download).toHaveAttribute("href", `${contentPath}?download=1`, { timeout });
|
|
const [file] = await Promise.all([page.waitForEvent("download", { timeout }), download.click({ timeout })]);
|
|
const downloaded = await file.path();
|
|
if (!downloaded || !(await readFile(downloaded)).equals(expected)) throw new ConnectionFailure("browser_download_bytes_mismatch");
|
|
}
|