mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:58:52 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The web server owns request handling and remote execution coordination. > - Deployment must start a ready replacement while the old server still serves. > - An outage test does not prove that two live servers can overlap safely. > - This pull request adds a separate Product E2E cell for that boundary. > - The existing run, file, recovery, and retained-session checks still apply. ## Linked Issues or Issue Description Refs #18518. The runtime prerequisite is merged; this PR qualifies its ephemeral server and deployment contracts. ## What Changed - Add the explicit `rolling-web-deployment` Codex Daytona workflow. - Start independent A/B/C homes and compiled builds behind a test ingress. - Wait for readiness before routing, then drain and delete the old home. - Observe active authority before handoff and use the existing read-only recovery oracle. - Require ordinary reads and saves to succeed throughout replacement. Read back every accepted save exactly once. - Preserve the five-minute outage suites and their existing validators. - Add a model-free supervisor smoke and an unready replacement control. - Require graceful exit before deleting an old home; forced cleanup cannot satisfy the oracle. - Preserve a durable Stop through lease release/expiry until the recovered execution owner acknowledges it. Permit only fenced reattachment of the original execution for Stop cleanup; submit no new turn. - Exclude events saved by A during shutdown from evidence of B making progress. ## Verification - E2E TypeScript check passed. - Affected harness checks: 143 passed. Catalog, fixture and report checks: 120 passed. - Actual supervisor smoke passed two replacements: 436 reads, 45 saves, zero request failures, and confirmed owned resource cleanup. - B became ready in 18.7 seconds; C became ready in 109.2 seconds. A prior C timeout remains failed evidence. - Fresh browser proof: 11 checks passed, 246 reads, 50 acknowledged saves, zero failures, 1,058ms reconnect, both old homes deleted and cleanup confirmed. - Stop lease-gap regression: three PostgreSQL delivery tests passed with a simulated provider edge. Current server TypeScript build/vendoring passed. - Full harness: 5,814 passed, 17 failed, four skipped. Two shutdown races passed the focused file; two report timeouts passed an isolated 42-test rerun. Onboarding/workflow-security/Pi/date-expired fixtures are unchanged on inspected master; the host embedded PostgreSQL fixture remains a setup limitation. This is not a green full local suite. - CI at `172509b8be` passed all gates. Greptile is 5/5 with every review thread resolved. The earlier access-service race failure passed its isolated regression and the subsequent full CI. - Final review corrections: six focused executor cases, three PostgreSQL command-delivery cases, 19 harness cases and server/E2E typechecks passed. - Three live attempts failed before agent creation: two harness setup gaps were fixed, then provider authentication failed. The saved OpenAI key returns 401; a refresh is pending. The Daytona probe reported cleanup uncertain, and no sandbox with that attempt label was found through the authenticated staging manager. No live model execution is claimed. - Live Runner overlap, released-container qualification and hosted deployment proof remain pending. Merging this opt-in qualification lane does not claim those results. ## Risks - The source lane shares workspace dependencies and does not establish container packaging correctness. - The live workflow needs an explicitly owned S3 bucket, PostgreSQL, pinned remote image and provider credentials. - Qualification cleanup stops all owned processes after failure. A failed replacement must not be reported as a pass. ## Model Used OpenAI Codex, GPT-6. Tool use, code execution and repository review. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
96 lines
3.7 KiB
TypeScript
96 lines
3.7 KiB
TypeScript
import path from "node:path";
|
|
import { defineConfig } from "@playwright/test";
|
|
import {
|
|
runnerE2EWebServerCommand,
|
|
runnerE2EWebServerGracefulShutdown,
|
|
} from "./web-server-command.js";
|
|
|
|
function required(name: string) {
|
|
const value = process.env[name]?.trim();
|
|
if (!value) throw new Error(`${name} is required`);
|
|
return value;
|
|
}
|
|
|
|
const port = Number(required("PAPERCLIP_RUNNER_E2E_PORT"));
|
|
const temporaryRoot = required("PAPERCLIP_RUNNER_E2E_TEMP_ROOT");
|
|
const privateDir = required("PAPERCLIP_RUNNER_E2E_PRIVATE_DIR");
|
|
const paperclipHome = required("PAPERCLIP_HOME");
|
|
const configPath = required("PAPERCLIP_CONFIG");
|
|
const baseURL = `http://127.0.0.1:${port}`;
|
|
const playwrightChannel = process.env.PAPERCLIP_PLAYWRIGHT_CHANNEL?.trim();
|
|
const chromiumExecutable =
|
|
process.env.PAPERCLIP_RUNNER_E2E_CHROMIUM_EXECUTABLE?.trim();
|
|
if (playwrightChannel && chromiumExecutable) {
|
|
throw new Error(
|
|
"PAPERCLIP_PLAYWRIGHT_CHANNEL and PAPERCLIP_RUNNER_E2E_CHROMIUM_EXECUTABLE are mutually exclusive",
|
|
);
|
|
}
|
|
if (chromiumExecutable && !path.isAbsolute(chromiumExecutable)) {
|
|
throw new Error(
|
|
"PAPERCLIP_RUNNER_E2E_CHROMIUM_EXECUTABLE must be an absolute path",
|
|
);
|
|
}
|
|
required("PAPERCLIP_INSTANCE_ID");
|
|
required("PAPERCLIP_AGENT_JWT_SECRET");
|
|
required("PAPERCLIP_DECISION_SIGNING_SECRET");
|
|
required("PAPERCLIP_TOOL_ACTION_SIGNING_SECRET");
|
|
required("BETTER_AUTH_SECRET");
|
|
if (
|
|
!paperclipHome.startsWith(`${temporaryRoot}${path.sep}`) ||
|
|
!configPath.startsWith(`${temporaryRoot}${path.sep}`)
|
|
) {
|
|
throw new Error("Paperclip server paths escape the isolated temporary root");
|
|
}
|
|
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
|
|
|
export default defineConfig({
|
|
testDir: ".",
|
|
testMatch: process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_STARTUP_ONLY === "1" ? "installed-startup.spec.ts" : "runner.spec.ts",
|
|
timeout: Number(process.env.PAPERCLIP_RUNNER_E2E_TEST_TIMEOUT_MS ?? 600_000),
|
|
expect: { timeout: 30_000 },
|
|
fullyParallel: false,
|
|
workers: 1,
|
|
retries: 0,
|
|
use: {
|
|
baseURL,
|
|
browserName: "chromium",
|
|
...(playwrightChannel ? { channel: playwrightChannel } : {}),
|
|
...(chromiumExecutable
|
|
? { launchOptions: { executablePath: chromiumExecutable } }
|
|
: {}),
|
|
headless: true,
|
|
actionTimeout: 30_000,
|
|
navigationTimeout: 30_000,
|
|
screenshot: process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : "only-on-failure",
|
|
trace: process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : "retain-on-failure",
|
|
// A developer-supplied system Chromium keeps the local smoke loop
|
|
// installation-free; CI's managed browser retains failure video as usual.
|
|
video: chromiumExecutable || process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : "retain-on-failure",
|
|
},
|
|
webServer: {
|
|
// Do not put an env object here: Playwright serializes webServer config in
|
|
// blob reports. The wrapper inherits the test process and strips provider
|
|
// keys before spawning the real Paperclip process.
|
|
command: runnerE2EWebServerCommand(repositoryRoot),
|
|
gracefulShutdown: { ...runnerE2EWebServerGracefulShutdown,
|
|
...(JSON.parse(process.env.PAPERCLIP_RUNNER_E2E_EXECUTION_IDS ?? "[]").some((id: string) => id.startsWith("rolling-web-deployment.")) ? { timeout: 105_000 } : {}),
|
|
},
|
|
url: `${baseURL}/api/health`,
|
|
reuseExistingServer: false,
|
|
// Fresh databases apply the complete migration history before health is ready.
|
|
timeout: 300_000,
|
|
stdout: "pipe",
|
|
stderr: "pipe",
|
|
},
|
|
outputDir: path.join(privateDir, "playwright-output"),
|
|
reporter: [
|
|
["list"],
|
|
["blob", { outputDir: path.join(privateDir, "blob-report") }],
|
|
["junit", { outputFile: path.join(privateDir, "junit.xml") }],
|
|
[
|
|
"html",
|
|
{ open: "never", outputFolder: path.join(privateDir, "html-report") },
|
|
],
|
|
],
|
|
});
|