test(restart-chaos): pin each wave's API URL to its own loopback server (#20462)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart-chaos harness starts an isolated Paperclip server for
each wave and runs real agents against it
> - Agents find the server through `PAPERCLIP_API_URL`, which the server
derives at startup
> - On a developer machine with a public URL for the main instance (for
example a tailnet origin), the wave server derived that public URL
instead of its own loopback origin
> - So `process` agents sent the run key for the temporary instance to a
different host, which returned 502
> - This pull request pins each wave server's `PAPERCLIP_API_URL` to its
own loopback origin and removes the Q3 script workaround
> - The benefit is that a wave's agent credentials only go to the wave's
own server

## Linked Issues or Issue Description

**What happened?**
In isolated restart-chaos wave instances, `process` agents received
`PAPERCLIP_API_URL` set to the machine's public tailnet origin. That
origin serves the developer's main instance, not the wave. The agent's
run key for the temporary instance was sent to that host, which returned
502. The Q3 agent script worked around this with a hard-coded loopback
base URL.

**Expected behavior**
A wave's agents call the wave's own server.

**Steps to reproduce**
1. Configure a public URL (for example a tailnet origin) for the
developer's main Paperclip instance on the machine.
2. Run a restart-chaos wave with a `process` agent that reads
`PAPERCLIP_API_URL`.
3. The agent's API calls go to the public origin, not to the wave's
`127.0.0.1` port.

**Paperclip version or commit**
master at 2c2186706e

**Additional context**
This is a harness isolation gap, not a product bug. The server
intentionally lets an explicit `PAPERCLIP_API_URL` win, and otherwise
prefers the configured public URL (`choosePrimaryRuntimeApiUrl`),
because an operator's public URL is the address agents must use in many
deployments. The isolated wave server is the component that must name
its own origin. `tests/runner-e2e/server.ts` already pins its public URL
to loopback in its public-MCP mode for the same reason.

## What Changed

- `tests/restart-chaos/supervisor.ts`: new
`waveServerUrlEnvironment(port)` sets `PAPERCLIP_API_URL` to
`http://127.0.0.1:<port>` in every wave server environment (also after
restarts and a K14 switch).
- `tests/restart-chaos/journey-driver.ts`: the Q3 agent script reads
`PAPERCLIP_API_URL` again. The hard-coded loopback workaround is
removed.
- `tests/restart-chaos/journeys.test.ts`: two tests.

## Verification

- Red before, green after (`npx vitest run -c
tests/restart-chaos/vitest.config.ts
tests/restart-chaos/journeys.test.ts`, 22 passed):
  - `pins the wave's loopback origin over an ambient public API URL`
- `sends the Q3 agent's run key only to the PAPERCLIP_API_URL its server
assigned` (runs the real Q3 script against a local HTTP server and
checks that every request goes there with the run key)
- `npx tsc --noEmit -p tests/restart-chaos/tsconfig.json` passes.

## Risks

- Low risk. Test harness only. No product code changes.
- The pin applies to `PAPERCLIP_API_URL` only. A public URL from the
developer environment can still set the wave's auth base URL. No current
workload depends on it.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
DottaandClaude authored and GitHub committed 2026-11-10 00:24:21 -06:00
1 parent 400bb1f0f9
commit bf1524c5f8
3 files changed
+56 -6

No files matched your search

+4 -5
View File
@@ -65,11 +65,10 @@ export function questionAnswer(interaction: Json, choice: string): { questionId:
}
/** The scripted Q3 agent: one connected-app call, then report the reviewed result. Adapted from tests/e2e/connection-reviews.spec.ts. */
export function q3AgentScript(connectionId: string, title: string, serverBaseUrl: string): string {
// The wave's own loopback server. On a tailnet host, PAPERCLIP_API_URL names the machine's
// public URL, which reaches a different instance.
export function q3AgentScript(connectionId: string, title: string): string {
// The supervisor pins PAPERCLIP_API_URL to the wave's own loopback server.
return `
const base = ${JSON.stringify(serverBaseUrl)} + "/api";
const base = process.env.PAPERCLIP_API_URL.replace(/\\/+$/, "").replace(/\\/api$/, "") + "/api";
const headers = { authorization: "Bearer " + process.env.PAPERCLIP_API_KEY, "content-type": "application/json", "x-paperclip-run-id": process.env.PAPERCLIP_RUN_ID };
const api = async (path, method = "GET", body) => {
const response = await fetch(base + path, { method, headers, body: body === undefined ? undefined : JSON.stringify(body) });
@@ -164,7 +163,7 @@ export class JourneyDriver {
const agent = await api.post<Json>(`/api/companies/${lab.companyId}/agents`, {
name: `Chaos pages agent ${plan.nonce}`, role: "engineer", title: "Connected-app journey agent",
capabilities: "Creates one page with the chaos app after review.", adapterType: "process",
adapterConfig: { command: process.execPath, args: ["--input-type=module", "-e", q3AgentScript(state.connectionId, q3ToolArgument(plan.nonce), api.baseUrl)] },
adapterConfig: { command: process.execPath, args: ["--input-type=module", "-e", q3AgentScript(state.connectionId, q3ToolArgument(plan.nonce))] },
});
state.agentId = String(agent.id);
// A scripted process run records usage with no price, which the company's default budget
+41 -1
View File
@@ -1,6 +1,12 @@
import { describe, expect, it } from "vitest";
import { parseCommand } from "./config.js";
import { actionRequestRows, JourneyDriver, questionAnswer, reviewClockNames, shiftReviewClockWith, type JourneyHost } from "./journey-driver.js";
import { actionRequestRows, JourneyDriver, q3AgentScript, questionAnswer, reviewClockNames, shiftReviewClockWith, type JourneyHost } from "./journey-driver.js";
import { waveServerUrlEnvironment } from "./supervisor.js";
import { buildPaperclipServerEnvironment } from "../runner-e2e/harness-env.js";
import { execFile } from "node:child_process";
import { once } from "node:events";
import { createServer } from "node:http";
import type { AddressInfo } from "node:net";
import { matrixCells, matrixMayLaunch, planMatrixWave } from "./journey-matrix.js";
import { pkceChallenge, startOAuthApp } from "./journey-oauth-app.js";
import {
@@ -312,3 +318,37 @@ describe("Q3 review clock", () => {
expect(connectionUpdate.params[2]).toBeNull();
});
});
describe("wave server URL", () => {
it("pins the wave's loopback origin over an ambient public API URL", () => {
const environment = buildPaperclipServerEnvironment(
{ PAPERCLIP_API_URL: "https://dev-machine.example.ts.net", PATH: "/bin" },
{ PORT: "50187", ...waveServerUrlEnvironment(50187) });
expect(environment.PAPERCLIP_API_URL).toBe("http://127.0.0.1:50187");
});
it("sends the Q3 agent's run key only to the PAPERCLIP_API_URL its server assigned", async () => {
const requests: Array<{ url: string; authorization: string | undefined }> = [];
const server = createServer((request, response) => {
requests.push({ url: request.url ?? "", authorization: request.headers.authorization });
response.setHeader("content-type", "application/json");
response.end(JSON.stringify(request.url?.endsWith("/interactions")
? [{ status: "pending", payload: { toolAction: { tool: "create_page" } } }] : {}));
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
const origin = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
try {
const output = await new Promise<string>((resolve, reject) => execFile(process.execPath,
["--input-type=module", "-e", q3AgentScript("connection-1", "Chaos page")],
{ env: { PATH: process.env.PATH, PAPERCLIP_API_URL: `${origin}/api/`, PAPERCLIP_API_KEY: "run-key",
PAPERCLIP_RUN_ID: "run-1", PAPERCLIP_TASK_ID: "issue-1" }, timeout: 20_000 },
(error, stdout, stderr) => error ? reject(new Error(`${error.message}\n${stderr}`)) : resolve(stdout)));
expect(output).toContain("Waiting for the saved review");
expect(requests.map(request => request.url)).toContain("/api/issues/issue-1/interactions");
expect(requests.every(request => request.url.startsWith("/api/") && request.authorization === "Bearer run-key")).toBe(true);
} finally {
server.close();
}
});
});
+11
View File
@@ -61,6 +61,16 @@ export function crashLoopShouldFire(target: (typeof CRASH_LOOP_TARGETS)[number],
return phase === "ready";
}
/**
* Agents reach the server at PAPERCLIP_API_URL. An explicit value wins over the URL the server
* derives from its public URL (a developer's .paperclip/.env or shell can name a tailnet origin
* that serves a different instance). Pin this wave's own loopback origin so an agent's key for
* the temporary instance is only ever sent to that instance.
*/
export function waveServerUrlEnvironment(port: number): { PAPERCLIP_API_URL: string } {
return { PAPERCLIP_API_URL: `http://127.0.0.1:${port}` };
}
export interface SupervisorPaths { temporaryRoot: string; home: string; configPath: string; serverLogPath: string }
/**
@@ -218,6 +228,7 @@ export class ChaosSupervisor {
// set values. A worktree-initialized checkout's file sets
// PAPERCLIP_IN_WORKTREE=true, which suppresses all scheduling: no run starts.
PAPERCLIP_IN_WORKTREE: "false",
...waveServerUrlEnvironment(this.port),
...extraEnvironment,
};
const environment = buildPaperclipServerEnvironment(source, overrides);