Files
PaperClipAI/tests/e2e/cloud-auth.spec.ts
T
DottaandPaperclip 48b57e0c94 test(evals): pin held scenario repairs and await Cloud return (#19188)
## Thinking Path

> - Paperclip helps people manage AI agents and their work.
> - Protected TestOps runs admit one immutable catalog source.
> - Three held scenario observers now have reviewed, locally qualified
repairs.
> - A separate Cloud recovery test started its heading wait before the
Cloud return.
> - This PR pins the qualified successor and corrects that test
boundary.
> - It changes no product behavior, model, execution budget, or private
evidence protocol.

## Linked Issues or Issue Description

Refs: #19155, #19093, #19095, #19112.

The next candidate retains the reviewed skill, company-secret, and
team-install observer repairs. It adds a narrow timeline v9 correction:
a recorded task identifier can be the subject of its actual
reassignment. The real timeline attempt remains failed; the diagnostic
correction is not a regrade or a new native pass. Earlier failed
attempts stay retained; a source pin and component tests do not
establish a native pass.

The Cloud recovery fixture failed both direct-task variants in the first
CI attempt of #19155. Its five-second heading check started before the
external Cloud roundtrip ended. The failure image already showed the
expected task shortly afterward. One unchanged local direct-task control
passed. The single authorized CI recovery also passed; this did not
establish a reliable boundary. This PR waits for one handoff and the
exact return URL before the original heading check.

## What Changed

- Pin source `5aef4bffa2b106f2066bf2485338798f04a702bd`, tree
`464be838a448c728d44e2d5593b9d7faea266663`.
- Update the two manifest target literals and expected pin test literal.
Bind the exact v9 descriptor snapshot and its one changed timeline
oracle leaf.
- Add the reviewed two-line Cloud return readiness witness. Keep the
original five-second heading assertion and existing overall test limit.
- Use descriptor
`744a566faaa90b8048fff526e0a30b7475b86c2de88d697783f8da9e02f70e20`. Keep
the launcher, all private helpers, 244 identities, requests, setup,
human/native counts, clocks, profile, model, retry policy, permissions,
and the 196/244 gate unchanged.

## Verification

- Current infrastructure worktree: **131 pin, admission, selection,
streaming, API, and reconciliation controls passed**.
- The fixture owner ran the exact patched Cloud spec: **all five real
auth cases passed** in 37.4s. A six-second Cloud hold passed the
unchanged heading check after return. Missing handoff and missing return
controls rejected. This PR's fresh master base is verified by
current-head CI, not credited from the earlier run.
- Fresh v9 source qualification: **268 focused timeline controls**, the
242-declaration gate, fresh server declarations, and full E2E typecheck
passed under local Node **26.4.0**. The preceding held-helper
qualification (239 Node controls and one offline Chromium route control)
remains evidence for the byte-identical unaffected helper closures. The
original browser sandbox failure is retained. Actual AWS Node and model
configuration remain unchanged.
- All 506 registered SDK input Git blobs and registry
`efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8` equal
the prior qualified source. No new SDK build or runtime refresh.
- All 75 private protocol files, production trees, YAML, lockfile,
launcher, identities, counts, clocks, and sourceFiles declarations equal
the prior source. Only the timeline descriptor oracle advances v8 to v9.
- Prior head `ce18f2bd52ed00ce750302f63aecc4167a3daff3` finished CI with
43 passed, one expected skip, four originating runner interruptions, and
a failed dependent verify aggregate. Two logs show runner shutdown
signals; two annotations show lost runner communication. No assertion
failure was observed. Those outcomes are retained; no unchanged recovery
was requested.
- Required fresh current-head CI, genuine review, thread clearance, and
Root's exact-diff acceptance are pending. No merge or new paid run is
authorized by this PR's publication.
- The earlier 3 PASS / 7 FAIL uniform attempt and all later original
outcomes remain unchanged. No new native pass or whole-catalog readiness
claim is made.

## Risks

The source remains a frozen candidate with unmerged catalog repairs. A
real measured run is still required. The readiness witness requires both
a real handoff and the exact task URL; it does not extend the heading
timeout, bypass auth, skip a case, or change production code. The active
measurement on the previous pin is not altered.

## Model Used

OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and
repository tools. The host did not expose a context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched for and linked related public PRs above
- [x] I have linked public PRs and described the issue
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] Relevant local pin tests passed; source and fixture qualification
scope is stated above
- [x] I have updated the expected pin test and the observed fixture
boundary
- [x] Owning source metadata and this description document the change
- [x] I have considered and documented the risks
- [ ] All current-head infrastructure CI gates are green
- [ ] Greptile is 5/5 with no unresolved material findings
- [x] I will address reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-04 23:14:14 -06:00

69 lines
3.9 KiB
TypeScript

import { randomUUID } from "node:crypto";
import { expect, test } from "@playwright/test";
test.use({ trace: "retain-on-failure" });
// The tenant UI and task database are real. Cloud is an external dependency:
// simulate its entry endpoint and independent session states at the HTTP edge.
for (const cloudOrigin of ["https://my.paperclip.app", "https://my-staging.paperclip.app"]) {
for (const entry of ["auth", "task"] as const) {
test(`Cloud recovery preserves the task (${cloudOrigin}, ${entry})`, async ({ page, request, baseURL }, testInfo) => {
const companyResponse = await request.post("/api/companies", { data: { name: `Cloud auth ${randomUUID()}` } });
expect(companyResponse.ok()).toBe(true);
const company = await companyResponse.json();
const issueResponse = await request.post(`/api/companies/${company.id}/issues`, {
data: { title: "Return here after Cloud sign-in", status: "backlog" },
});
expect(issueResponse.ok()).toBe(true);
const issue = await issueResponse.json();
const target = `/${company.issuePrefix}/issues/${issue.identifier}?view=activity#recovered`;
let authenticated = false;
let handoffs = 0;
let formWasRendered = false;
await page.exposeFunction("reportInstanceLoginForm", () => { formWasRendered = true; });
await page.addInitScript(() => {
new MutationObserver(() => {
if (document.querySelector('input[autocomplete="current-password"]')) {
void (window as unknown as { reportInstanceLoginForm: () => Promise<void> }).reportInstanceLoginForm();
}
}).observe(document, { childList: true, subtree: true });
});
await page.route("**/api/health", async (route) => {
const response = await route.fetch();
await route.fulfill({ response, json: {
...await response.json(), deploymentMode: "authenticated", bootstrapStatus: "ready",
cloud: { managed: true, managedBy: "paperclip-cloud", cloudBaseUrl: cloudOrigin, stackSlug: "test-workspace" },
} });
});
await page.route("**/api/auth/get-session", (route) => route.fulfill(authenticated ? {
json: { session: { id: "test-session", userId: "local-board" }, user: { id: "local-board", email: "test@example.test", name: "Test user", image: null }, sentryDsn: null },
} : { status: 401, json: { error: "Board authentication required" } }));
await page.route(`${cloudOrigin}/v1/stacks/test-workspace/entry-redirect?**`, async (route) => {
handoffs++;
expect(new URL(route.request().url()).searchParams.get("returnTo")).toBe(target);
authenticated = true;
await route.fulfill({ status: 302, headers: { Location: `${baseURL}${target}` } });
});
const returnedFromCloud = page.waitForURL((url) => handoffs === 1 && url.href === `${baseURL}${target}`);
await page.goto(entry === "auth" ? `/auth?next=${encodeURIComponent(target)}` : target);
await returnedFromCloud;
await expect(page.getByRole("heading", { name: issue.title, exact: true })).toBeVisible();
expect(page.url()).toBe(`${baseURL}${target}`);
expect(handoffs).toBe(1);
expect(formWasRendered).toBe(false);
await page.screenshot({ path: testInfo.outputPath("cloud-session-recovered.png") });
await page.reload();
await expect(page.getByRole("heading", { name: issue.title, exact: true })).toBeVisible();
expect(handoffs).toBe(1);
expect(formWasRendered).toBe(false);
});
}
}
test("self-hosted auth still shows the instance sign-in form", async ({ page }) => {
await page.route("**/api/auth/get-session", (route) => route.fulfill({ status: 401, json: { error: "Board authentication required" } }));
await page.goto("/auth");
await expect(page.getByRole("heading", { name: "Sign in to Paperclip", exact: true })).toBeVisible();
await expect(page.getByLabel("Password", { exact: true })).toBeVisible();
});