refactor(runner): remove unreferenced Paperclip Runner test fixtures (#16543)

## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner package keeps fake agent servers and versioned profile identity declarations under `test/fixtures/` for its test suites. > - Five of those fixture files have no reference anywhere in the repository: no test, script, generator, workflow step, or document loads or spawns them. > - Dead fixtures add maintenance cost and make the referenced sibling fixtures harder to trust. > - This pull request deletes only those five unreferenced fixture files and adds an audit receipt with the reference evidence. > - The benefit is a smaller test-fixture surface with zero loss of executed coverage. ## Linked Issues or Issue Description **What existing behavior does this improve?** The test-suite hygiene of the Paperclip Runner package. The repository keeps unused test-support files that no suite loads. This slows audits and makes the referenced fixtures harder to trust. **Subsystem affected** `packages/paperclip-runner` test fixtures. **Current behavior** Five files under `packages/paperclip-runner/test/fixtures/` have zero references anywhere in the repository: - `fake-native-resume-codex-app-server.mjs` (277 LOC): an old fake codex app-server for resume testing. Resume coverage now runs through `fake-final-burst-codex-app-server.mjs` (used by `server/src/services/native-runtime/native-session-resume.test.ts:287`) and `fake-durable-codex-app-server.mjs` (used by `packages/paperclip-runner/src/live/live-session.test.ts:2192`). - `copilot-profile-v{3,4,5,6}-identity.json` (64 LOC total): orphaned profile identity declarations. Copilot profile tests, `native-backend-factory.test.ts`, and `scripts/generate-acpx-profiles.mjs` reference only v7 through v19 by literal path. No dynamic `copilot-profile-v${...}` path exists in the repository (the only template-literal fixture path is `test-fixtures/pi-acp/profile-v${version}-identity.json`). **Proposed behavior** Delete only the five unreferenced fixture files. Keep every referenced fixture. Keep all registered test cases unchanged. **Reason and benefit** Smaller test-fixture surface: 341 test/support lines removed with zero loss of executed coverage. The audit receipt `doc/plans/2026-10-16-runner-dead-fixture-test-audit.md` records the per-file reference evidence. **Breaking changes** None. No production code, no registered test case, and no test selection path changes. **Additional context** A repository-wide `git grep` for each removed filename stem returns zero external references across all tracked file types. The `hashFiles('packages/paperclip-runner/test/fixtures/**')` occurrences in the workflows are Rust cache-key inputs, not execution selectors; deleting files only changes that cache hash. ## What Changed - Deleted `packages/paperclip-runner/test/fixtures/fake-native-resume-codex-app-server.mjs`. - Deleted `packages/paperclip-runner/test/fixtures/copilot-profile-v{3,4,5,6}-identity.json`. - Added the audit receipt `doc/plans/2026-10-16-runner-dead-fixture-test-audit.md` with per-file reference evidence. - 341 test/support lines removed. Zero production, documentation, or registered test-case changes. ## Verification - Repo-wide `git grep -i` for each removed filename stem (`fake-native-resume`, `native-resume-codex`, `copilot-profile-v3` … `copilot-profile-v6`) across all tracked file types returns zero external references. - Selection audit: `packages/paperclip-runner/vitest.config.ts` includes only `src/**`, `devtools/**`, `examples/**`, and two named `scripts/*.test.mjs`; the server project selects `server/src/**` and `server/scripts/**`; the `test:typescript:prep` `node --test` list names no fixture. No workflow executes anything under `test/fixtures/` — the `pr-trusted.yml`/`release-verify.yml` `hashFiles('packages/paperclip-runner/test/fixtures/**')` occurrences are Rust cache-key inputs, not execution selectors. - Rust tests use their own `fake-codex-app-server` binary in `runner/crates/runner-core`, not these `.mjs` fixtures. - Searched open and merged PRs for each filename: zero overlap (`gh search prs`). - Repo-wide `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were not run this session: the removed files are unreferenced, absent from every vitest/node-test include pattern, and excluded from all tsconfigs, so no compiled or executed path changes. No runtime CI savings are claimed. - All Paperclip CI gates green on head `ce24ab3` (verified 2026-10-16 ~09:00Z): every `ci /` shard (General tests, e2e shards, serialized server suites, Verify Paperclip Runner rust/vitest/static, Build, Typecheck + Release Registry, policy, Docker context integrity, Canary Dry Run, gate), `review`, `Superagent Security Scan`, `Socket Security` PR alerts + project report, and `Contributor trust` all pass; the only skipped jobs are the Storybook deploy/visual-regression jobs, which are not required for this change. Greptile Review: 5/5 confidence, "Low impact", no actionable issues, last reviewed commit `ce24ab3`. ## Risks Low risk. No executed coverage or selection path changes. The only side effect is a one-time change to the `hashFiles` Rust cache key that lists `test/fixtures/**`, which simply invalidates that cache on the next CI run. ## Model Used Zhipu `z-ai/glm-5.3` (GLM), served through OpenRouter, running as the Paperclip Test Gardener agent. Context window size unavailable to the agent. Capabilities used: shell tooling, git, GitHub CLI search, and file inspection; no image or code execution capabilities were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
2026-10-21 08:22:20 +02:00 · 2026-10-17 21:18:11 -05:00
1 parent e30550ef39
commit cfa9dce295
@@ -0,0 +1,51 @@
+# Test audit: remove unreferenced Paperclip Runner test fixtures
+
+Date: 2026-10-16
+Branch: `PAP-458-nightly-test-gardening`
+Base: `78b833c4958465b5e345b0e337018011b00dab72` (origin/master)
+Scope: focused sweep of test-support files under `packages/paperclip-runner/test/fixtures/`.
+
+## Removed files
+
+| Candidate | Decision and reason | Contract | Evidence | Benefit |
+| --- | --- | --- | --- | --- |
+| `test/fixtures/fake-native-resume-codex-app-server.mjs` (277 lines) | Delete: orphaned fake codex app-server fixture | Nothing loads or spawns it; resume coverage runs through `fake-final-burst-codex-app-server.mjs` (server `native-session-resume.test.ts:287`) and `fake-durable-codex-app-server.mjs` (`live-session.test.ts:2192`) | Repo-wide `git grep -i "fake-native-resume"` and `"native-resume-codex"` → zero tracked references in code, docs, workflows, or path templates; sibling `.mjs` fakes are all referenced | 277 test-support LOC |
+| `test/fixtures/copilot-profile-v{3,4,5,6}-identity.json` (64 lines) | Delete: orphaned versioned identity declarations | Copilot profile tests and `native-backend-factory.test.ts` exercise v7–v19 by literal path; no dynamic `copilot-profile-v${...}` path exists (only `test-fixtures/pi-acp/profile-v${version}`) | Repo-wide `git grep` per exact filename stem → zero references; `generate-acpx-profiles.mjs` pins v19 only | 64 test-support LOC |
+
+## Retention evidence
+
+- Reference proof: `git grep` across all tracked file types (ts/tsx/mjs/js/json/yml/toml/rs/md) for each
+  removed stem returns zero external references.
+- Selection proof: `packages/paperclip-runner/vitest.config.ts` include list covers
+  `src/**`, `devtools/**`, `examples/**`, and two named `scripts/*.test.mjs` — never
+  `test/fixtures/**`. The root server project also selects `server/src/**` only. The
+  `test:typescript:prep` `node --test` list names no fixture. No workflow step executes
+  files under `test/fixtures/`; the `pr-trusted.yml`/`release-verify.yml`
+  `hashFiles('packages/paperclip-runner/test/fixtures/**')` occurrences are Rust
+  cache-key inputs, not execution selectors (deleting files only changes that hash).
+- Rust tests use their own `fake-codex-app-server` binary (`runner/crates/runner-core`),
+  not these `.mjs` fixtures.
+- No open or merged PR references any removed file (`gh search prs` per filename: zero
+  hits). `scripts/tests/native-*-codex.mjs` opt-in checks and the three other fake
+  `.mjs` servers are documented/referenced and retained.
+
+## Validation
+
+- Deletion-only diff: 341 test/support LOC removed, 0 production LOC, 0 docs LOC,
+  0 registered test cases removed (no case existed for these files).
+- Repo-wide `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were **not** run this
+  session: the removed files are unreferenced, unselected by every vitest/node-test
+  include pattern, and excluded from all tsconfigs, so no compiled or executed path
+  changes. No runtime CI savings are claimed; the benefit is a smaller fixture
+  maintenance surface.
+
+Model: Zhipu `z-ai/glm-5.3` via OpenRouter (Test Gardener (GLM)); context window unavailable;
+shell tooling and git/GitHub search only.
+
+Delivery: PR #16543, retitled to the `refactor(runner):` prefix because the repository
+quality gate requires a test file or a `refactor:` title for test-file-less PRs — no
+registered case exists for the removed fixtures. The PR body also carries the in-PR
+issue description using the literal enhancement-template labels
+(`What existing behavior does this improve?`, `Subsystem affected`, `Current behavior`,
+`Proposed behavior`, `Reason and benefit`, `Breaking changes`, `Additional context`)
+required by the same gate when no linked issue is referenced.
@@ -1,15 +0,0 @@
-{
-  "commandDigest": "sha256:527f9cbbad2f3e75169cae70d7aa5b189200e57f1ab7f9e1523357b5606b0939",
-  "declaration": {
-    "schema": "paperclip.acpx_profile_declaration.v1",
-    "agent": "copilot",
-    "agentProfileVersion": 3,
-    "acpxVersion": "0.13.1",
-    "agentServerVersion": "1.0.88",
-    "protocolPolicyRevision": "copilot-agent-manual-v1",
-    "acpxPatchSha256": "a64405199b381688ffd72fdbb88396e423394cb407813d7b6f1db43bae9c3933",
-    "policySha256": "a4be69be43db84052560420a3ca546a665c9165573c24d543eb33d144ba138a3",
-    "distributionSourceSha256": "bb3dcf14972e82f6578c8250540cfbc6ec2ad953bea5d4f10d28c8b461011f3f",
-    "agentFilesBinding": "registered-runtime-context-v1"
-  }
-}
@@ -1,16 +0,0 @@
-{
-  "commandDigest": "sha256:a119e436b4ad13ee70e1f58bedad4a0f20761b5fd982b752981f0cb1bc65f797",
-  "declaration": {
-    "schema": "paperclip.acpx_profile_declaration.v1",
-    "agent": "copilot",
-    "agentProfileVersion": 4,
-    "acpxVersion": "0.13.1",
-    "agentServerVersion": "1.0.88",
-    "protocolPolicyRevision": "copilot-agent-manual-v1",
-    "acpxPatchSha256": "64180c194ab841d6f4bba7e6eca5ead621c6222c9a8057e4bb2bb6f007d8bb3c",
-    "policySha256": "a4be69be43db84052560420a3ca546a665c9165573c24d543eb33d144ba138a3",
-    "distributionSourceSha256": "bb3dcf14972e82f6578c8250540cfbc6ec2ad953bea5d4f10d28c8b461011f3f",
-    "agentFilesBinding": "registered-runtime-context-v1",
-    "systemInstructionDelivery": "COPILOT_HOME/copilot-instructions.md:replace-under-lifetime-lease-before-launch:v1"
-  }
-}
@@ -1,16 +0,0 @@
-{
-  "commandDigest": "sha256:ece77e40876631a69a828b91813722001d71b6fc81be47ecd4b3dced84ff9473",
-  "declaration": {
-    "schema": "paperclip.acpx_profile_declaration.v1",
-    "agent": "copilot",
-    "agentProfileVersion": 5,
-    "acpxVersion": "0.13.1",
-    "agentServerVersion": "1.0.88",
-    "protocolPolicyRevision": "copilot-agent-manual-v1",
-    "acpxPatchSha256": "79aad2d688b03362e8cfcbf7a08f78a8383869f6882a9c9ed66f1d18efb94f2b",
-    "policySha256": "a4be69be43db84052560420a3ca546a665c9165573c24d543eb33d144ba138a3",
-    "distributionSourceSha256": "bb3dcf14972e82f6578c8250540cfbc6ec2ad953bea5d4f10d28c8b461011f3f",
-    "agentFilesBinding": "registered-runtime-context-v1",
-    "systemInstructionDelivery": "COPILOT_HOME/copilot-instructions.md:replace-under-lifetime-lease-before-launch:v1"
-  }
-}
@@ -1,17 +0,0 @@
-{
-  "commandDigest": "sha256:0fe49c2f8a2b144344d0de745fed1e4568df305de3a26c3a121dec21c8d4b751",
-  "declaration": {
-    "schema": "paperclip.acpx_profile_declaration.v1",
-    "agent": "copilot",
-    "agentProfileVersion": 6,
-    "acpxVersion": "0.13.1",
-    "agentServerVersion": "1.0.88",
-    "protocolPolicyRevision": "copilot-agent-manual-v1",
-    "acpxPatchSha256": "79aad2d688b03362e8cfcbf7a08f78a8383869f6882a9c9ed66f1d18efb94f2b",
-    "policySha256": "a4be69be43db84052560420a3ca546a665c9165573c24d543eb33d144ba138a3",
-    "distributionSourceSha256": "bb3dcf14972e82f6578c8250540cfbc6ec2ad953bea5d4f10d28c8b461011f3f",
-    "agentFilesBinding": "registered-runtime-context-v1",
-    "systemInstructionDelivery": "COPILOT_HOME/copilot-instructions.md:replace-under-lifetime-lease-before-launch:v1",
-    "sharedRuntimeContract": "paperclip.acpx-runtime-contract.v1"
-  }
-}
@@ -1,277 +0,0 @@
-#!/usr/bin/env node
-import { createHash } from "node:crypto";
-import { readFileSync, unlinkSync, writeFileSync } from "node:fs";
-import { createConnection, createServer } from "node:net";
-
-const websocketGuid = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
-
-function serverFrame(payload, opcode = 0x1) {
-  const body = Buffer.isBuffer(payload) ? payload : Buffer.from(payload);
-  const extended = body.length < 126 ? 0 : body.length <= 0xffff ? 2 : 8;
-  const frame = Buffer.alloc(2 + extended + body.length);
-  frame[0] = 0x80 | opcode;
-  frame[1] = body.length < 126 ? body.length : extended === 2 ? 126 : 127;
-  let offset = 2;
-  if (extended === 2) {
-    frame.writeUInt16BE(body.length, offset);
-    offset += 2;
-  } else if (extended === 8) {
-    frame.writeBigUInt64BE(BigInt(body.length), offset);
-    offset += 8;
-  }
-  body.copy(frame, offset);
-  return frame;
-}
-
-const [mode, socketPath, statePath] = process.argv.slice(2);
-if (!mode || !socketPath) throw new Error("mode and socket path are required");
-
-if (mode === "proxy") {
-  const socket = createConnection(socketPath);
-  socket.once("connect", () => {
-    process.stdin.pipe(socket);
-    socket.pipe(process.stdout);
-  });
-  socket.once("error", (error) => {
-    process.stderr.write(`${error.message}\n`);
-    process.exitCode = 1;
-  });
-} else if (mode === "server") {
-  if (!statePath) throw new Error("state path is required in server mode");
-
-  function load() {
-    try {
-      return JSON.parse(readFileSync(statePath, "utf8"));
-    } catch (error) {
-      if (error?.code !== "ENOENT") throw error;
-      return {
-        threadId: "thread-native-resume",
-        sessionId: "session-native-resume",
-        turnId: "turn-native-resume",
-        pendingTool: null,
-        completed: false,
-      };
-    }
-  }
-
-  let state = load();
-
-  function save() {
-    writeFileSync(statePath, `${JSON.stringify(state)}\n`, { mode: 0o600 });
-  }
-
-  function send(socket, value) {
-    socket.write(serverFrame(JSON.stringify(value)));
-  }
-
-  function sendPendingTool(socket) {
-    if (!state.pendingTool) return;
-    send(socket, {
-      id: state.pendingTool.requestId,
-      method: "item/tool/call",
-      params: {
-        threadId: state.threadId,
-        turnId: state.turnId,
-        callId: state.pendingTool.callId,
-        tool: "report_progress",
-        arguments: {
-          idempotencyKey: "native-resume-progress",
-          body: "One native resume effect.",
-        },
-      },
-    });
-  }
-
-  function handleRequest(socket, message) {
-    const { id, method } = message;
-    if (method === "initialize") {
-      send(socket, { id, result: { user: { sessionId: state.sessionId } } });
-      return;
-    }
-    if (method === "thread/start" || method === "thread/resume") {
-      send(socket, {
-        id,
-        result: {
-          model: "gpt-native-resume-fixture",
-          modelProvider: "openai-fixture",
-          thread: { id: state.threadId, sessionId: state.sessionId },
-        },
-      });
-      if (method === "thread/resume") sendPendingTool(socket);
-      return;
-    }
-    if (method === "thread/read") {
-      send(socket, {
-        id,
-        result: {
-          thread: {
-            id: state.threadId,
-            sessionId: state.sessionId,
-            turns: [{
-              id: state.turnId,
-              status: state.completed ? "completed" : "inProgress",
-            }],
-            tokenUsage: {
-              total: {
-                inputTokens: 10,
-                cachedInputTokens: 0,
-                outputTokens: 2,
-                reasoningOutputTokens: 0,
-              },
-            },
-          },
-        },
-      });
-      return;
-    }
-    if (method === "turn/start") {
-      state.pendingTool = {
-        requestId: "rpc-native-resume",
-        callId: "call-native-resume",
-      };
-      save();
-      send(socket, {
-        id,
-        result: { turn: { id: state.turnId, status: "inProgress" } },
-      });
-      send(socket, {
-        method: "turn/started",
-        params: {
-          threadId: state.threadId,
-          turn: { id: state.turnId, status: "inProgress" },
-        },
-      });
-      sendPendingTool(socket);
-      return;
-    }
-    send(socket, {
-      id,
-      error: { code: -32601, message: `unsupported method ${method}` },
-    });
-  }
-
-  function handle(socket, message) {
-    if (message.method) {
-      if (message.id !== undefined) handleRequest(socket, message);
-      return;
-    }
-    if (!state.pendingTool || String(message.id) !== state.pendingTool.requestId)
-      return;
-    const toolResult = message.result;
-    state.pendingTool = null;
-    state.completed = true;
-    save();
-    send(socket, {
-      method: "item/completed",
-      params: {
-        threadId: state.threadId,
-        turnId: state.turnId,
-        item: {
-          id: "message-native-resume",
-          type: "agentMessage",
-          text: `Tool result: ${JSON.stringify(toolResult)}`,
-        },
-      },
-    });
-    send(socket, {
-      method: "thread/tokenUsage/updated",
-      params: {
-        threadId: state.threadId,
-        turnId: state.turnId,
-        tokenUsage: {
-          total: {
-            inputTokens: 10,
-            cachedInputTokens: 0,
-            outputTokens: 2,
-            reasoningTokens: 0,
-          },
-          runDelta: {
-            requests: 1,
-            inputTokens: 10,
-            cacheReadTokens: 0,
-            outputTokens: 2,
-            reasoningTokens: 0,
-            providerCostUsd: 0,
-          },
-        },
-      },
-    });
-    send(socket, {
-      method: "turn/completed",
-      params: {
-        threadId: state.threadId,
-        turn: { id: state.turnId, status: "completed" },
-      },
-    });
-  }
-
-  try {
-    unlinkSync(socketPath);
-  } catch (error) {
-    if (error?.code !== "ENOENT") throw error;
-  }
-  const server = createServer((socket) => {
-    let upgraded = false;
-    let buffer = Buffer.alloc(0);
-    socket.on("data", (chunk) => {
-      buffer = Buffer.concat([buffer, chunk]);
-      if (!upgraded) {
-        const boundary = buffer.indexOf("\r\n\r\n");
-        if (boundary < 0) return;
-        const headers = buffer.subarray(0, boundary).toString("utf8");
-        const key = headers.match(/^sec-websocket-key:\s*(.+)$/im)?.[1]?.trim();
-        if (!key) throw new Error("WebSocket upgrade omitted Sec-WebSocket-Key");
-        const accept = createHash("sha1")
-          .update(`${key}${websocketGuid}`)
-          .digest("base64");
-        socket.write(
-          `HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: ${accept}\r\n\r\n`,
-        );
-        upgraded = true;
-        buffer = buffer.subarray(boundary + 4);
-      }
-
-      while (buffer.length >= 2) {
-        const first = buffer[0];
-        const second = buffer[1];
-        let length = second & 0x7f;
-        let offset = 2;
-        if (length === 126) {
-          if (buffer.length < 4) return;
-          length = buffer.readUInt16BE(2);
-          offset = 4;
-        } else if (length === 127) {
-          if (buffer.length < 10) return;
-          length = Number(buffer.readBigUInt64BE(2));
-          offset = 10;
-        }
-        if ((second & 0x80) === 0)
-          throw new Error("WebSocket client frame was not masked");
-        if (buffer.length < offset + 4 + length) return;
-        const mask = buffer.subarray(offset, offset + 4);
-        offset += 4;
-        const payload = Buffer.from(buffer.subarray(offset, offset + length));
-        buffer = buffer.subarray(offset + length);
-        for (let index = 0; index < payload.length; index += 1)
-          payload[index] ^= mask[index % 4];
-
-        const opcode = first & 0x0f;
-        if (opcode === 0x8) {
-          socket.end(serverFrame(payload, 0x8));
-          return;
-        }
-        if (opcode === 0x9) {
-          socket.write(serverFrame(payload, 0xa));
-          continue;
-        }
-        if (opcode !== 0x1)
-          throw new Error(`unsupported WebSocket opcode ${opcode}`);
-        handle(socket, JSON.parse(payload.toString("utf8")));
-      }
-    });
-  });
-  server.listen(socketPath);
-  process.once("SIGTERM", () => server.close());
-} else {
-  throw new Error(`unsupported mode ${mode}`);
-}