3 Commits

Include renames

Author	SHA1	Message	Date
DottaandClaude Opus 5.5	871a9b3180	test(restart-chaos): add Phase 1 control-plane restart chaos harness (#19477) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane owns task execution, recovery and Retry for every agent run > - Server restarts were the largest remaining source of stalled tasks in the staging soak campaign > - The soak could only observe restarts that staging happened to perform, so a failure could not be repeated > - This pull request adds a script harness that causes the restarts on purpose and grades each task on fixed rules > - The benefit is a repeatable restart failure with a replay command and an evidence bundle for each fixer ## Linked Issues or Issue Description **What existing behavior does this improve?** The restart test coverage. Today the runner-e2e restart cells test one fixed restart point per cell. They cannot run many tasks at once, cannot choose a random fault time, and cannot replay a failure. **Subsystem affected** Test tooling (`tests/restart-chaos/`). It reuses the runner-e2e process code in `tests/runner-e2e/`. Product code does not change. **Current behavior** Restart robustness is checked through a small set of fixed cells. A random restart during a real task has no defined pass rule. **Proposed behavior** A harness starts an isolated control plane and seeds a "Chaos Lab" company. It launches seeded workloads, injects K1, K2 or K4 faults at seeded times, and grades each task on rules R1 to R6 plus a deterministic task oracle. `--replay <wave>/<task>` repeats one task. **Reason and benefit** Fixers get a replay command and an evidence bundle for each failure signature. The first smoke wave already found three product signatures (dead Retry after a restart, a false over-budget stop after restarts, and an OpenCode turn killed mid-turn that ends as a provider failure). **Breaking changes** None. The change adds new test files, one entry in the test-project typecheck list and a gitignore entry. ## What Changed - Add `tests/restart-chaos/`: - `supervisor.ts` owns one isolated server. It reuses `replaceController`, the process-tree stopper, the environment isolation helpers, the config seed, the port reservation, the wrapper-owned Postgres and `server-entry.ts` from `tests/runner-e2e/`. It does not copy them. Faults: K1 (SIGKILL, immediate restart), K2 (SIGTERM, restart) and K4 (SIGKILL, 2 to 5 minute hold). Postgres and the controller runtime cache stay across a fault. - `seed.ts` creates the company, encrypted provider secrets, native Codex, ACPX Claude and OpenCode agents from the runner-e2e catalog, and the `chaos-sandbox` project. The project is a git repository with a Python CLI, one failing test and a four-minute `bench.sh`. - `scheduler.ts` makes a seeded plan of workload, adapter, fault, fire time, hold and nonce for each task. It saves the plan so that `--replay` can run one task again. - `observer.ts` polls each issue and its runs. When a task stops and offers Retry, it sends `POST /api/issues/:id/retry` one time, with the same body as `useTaskRetry.ts`. - `grader.ts` grades R1 to R6 as pure functions. `workloads.ts` defines workloads A1, B1, B5 and C1 and their oracles. - `evidence.ts` and `ledger.ts` write an evidence bundle for each failure, a JSONL ledger and `SUMMARY.md`. All results go to the gitignored `tests/restart-chaos/results/`. Known provider key values are redacted. - Unit tests: `scheduler.test.ts` (determinism, replay and CLI parsing) and `grader.test.ts` (R1 to R6 and the oracles on fixture timelines). These tests need no credentials. - Add `tests/restart-chaos` to the test-project typecheck list (`scripts/typecheck-test-projects.mjs`), with a `tsconfig.layers.json` like `tests/runner-e2e`. The harness runs through direct script invocations, not `package.json` scripts. - After the first Greptile review: - A failed fault or an unexpected server exit now gets one bounded recovery. If the recovery fails, the wave fails and does not hang. - Incomplete cleanup is now a harness error, and the harness keeps the instance. - After the C1 follow-up comment, the observer needs a fresh rest window before it grades the task. - `--replay` now fires the recorded faults that crossed the task, at the same offsets, including the faults that other tasks requested. - A failed workspace-operation read is now a sweep error. It cannot give a clean rest. - R2 now checks that workspace files from before the first fault still exist at rest. - B5 now needs the completion record that `bench.sh` writes for each task. - After the second round of reviews: - An idle `todo` or `in_progress` task with no live run and no named wait is no longer treated as resting. The observer keeps watching it until the task resumes or passes the R5 stall limit. - A failed injected fault now fails the wave with exit 2. - T-oracle commands run agent-editable code. They now run under a macOS Seatbelt sandbox with a scratch `HOME` and no network, and they cannot read the operator's home directory. On other platforms these commands are refused unless the operator explicitly allows them. - R4 now counts every active run. A wake comment may be served again only after a run that did not succeed. - Evidence redaction now covers every environment variable name that ends in `KEY`, `TOKEN`, `SECRET` or `PASSWORD`. - If the pre-fault snapshot fails, only R2 for that task is left unchecked. The wave continues. - B5 also needs bench progress that the harness observed while it polled. - Add `doc/plans/2026-11-06-control-plane-restart-chaos.md` and its campaign ledger. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`: 38 tests pass. The tests include observer tests with a fake API, a real `bench.sh` run, and a sandbox test that cannot read a canary secret. - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: passes. - `server/scripts/typecheck-layers.test.mjs`: 45 tests pass, including "covers every project under tests/". - A supervisor check with no credentials passed. K1 was ready again in about 6 seconds. K2 with a 3-second hold was ready again in about 9 seconds. Company data stayed readable. - One paid smoke wave ran: `--env local --adapters codex,claude,opencode --workloads A1,B1,C1 --faults K1 --tasks 3 --concurrency 2 --seed 7`. The harness worked from start to end, and all three tasks crossed a restart. The 0/3 result comes from product failures, which the campaign ledger records. A scan found no provider key in the results. - To run a wave: ```sh pnpm --filter @paperclipai/paperclip-runner build:runner-binaries set -a; source ~/.secrets; set +a node cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts --env local --adapters codex,claude,opencode --concurrency 4 --tasks 8 --faults K1,K2,K4 --seed 42 node cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts --replay <wave>/<task> ``` ## Risks - Low product risk. The change adds test tooling only and does not change product code. - A wave spends provider money. The company budget hard cap (`--budget-cents`, default 3000) is the only spend limit. The plan's per-wave and per-night refusal is not implemented yet. - Known gaps: - `--env daytona` is reserved. It fails with a TODO in `seed.ts`. The Daytona sandbox sweep for R6 is not implemented yet. - The project requests isolated git-worktree workspaces. A B1 task still ran in the shared fixture repository. Concurrent B1 and B5 tasks can share files. The oracles use marker files for each task. - Only K1, K2 and K4 at seeded random times are available. Boundary triggers (B1 to B16), other faults and the UI probe come in Phase 2. - The paid smoke did not exercise B5 or K4. - On macOS, B1 oracle commands need `python3` outside the operator's home directory, for example `/usr/bin` or Homebrew. On Linux they are refused until a sandbox is added. - The B5 proof is best-effort against a deliberately adversarial agent, because that agent can write to the workspace. - Startup phase timings come from polling `/api/ready`. The server records its own timings only as OpenTelemetry spans. - `pnpm verify` and the CI Vitest jobs do not select these unit tests. The root Vitest projects cover workspace packages only. CI typechecks the files, but the tests run only through the vitest command above. - After the smoke, rest detection and the review fixes changed. A recovery action that the board owns now counts as a rest, and R6 reports it. A paid wave did not run again after these changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used in Claude Code as an agent with tool use (shell, file edits) and extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:45:52 -06:00
Nicky LeachandClaude Opus 5.5	83c6832ccb	fix(build): typecheck package, Playwright, and e2e test files in CI (#18839) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner, plugin, and Playwright tests verify how those agents run, connect, and appear in the board > - Most package tsconfigs exclude tests, and Vitest strips types, so a type error in those tests fails nothing > - The tests drifted to 803 real type errors, and some of those errors hid assertions that checked nothing > - paperclipai/paperclip#18739 and paperclipai/paperclip#18738 check the server test tree and four test projects, but not these files > - This pull request adds a required check for these test files, fixes every error, and fails CI if a test file is in no program > - The benefit is that a type error in any test file outside `server/src` now fails CI ## Linked Issues or Issue Description Refs #18739 Refs #18738 **What happened?** 517 test files were in no program that required CI checks. A type error in those files did not fail CI. **Expected behavior** CI fails on a type error in any test file. **Steps to reproduce** Add `export const value: number = "text";` to a test file under `packages/paperclip-runner/src` on master. CI stays green, because `tsconfig.json` excludes test files and Vitest strips types. **Measured gap** Each row is a program with its tests included, on one thread. File and error counts are for head `7d4b5e907a`. Memory peaks were measured before the two latest merges of master. | Group | Test files | Real errors | Peak | | --- | ---: | ---: | ---: | | `paperclip-runner` Node source | 290 | 644 | 1.09 GiB | | `paperclip-runner` surfaces | 9 | 12 | 0.42 GiB | | `paperclip-runner` devtools and examples | 14 | 0 | 0.68 GiB | | `sandbox-providers/kubernetes` | 21 | 65 | 0.44 GiB | | 8 other sandbox providers and `bridge-template` | 32 | 4 | 0.34 GiB | | `plugins/sdk` | 22 | 15 | 0.41 GiB | | `plugin-workspace-diff` | 5 | 0 | 0.37 GiB | | `plugin-orchestration-smoke-example` | 1 | 1 | 0.15 GiB | | `kv-demo-mcp-server` | 3 | 0 | 0.13 GiB | | `packages/db` scripts | 1 | 0 | 1.01 GiB | | 8 Playwright suites under `tests/` | 52 | 25 | 1.72 GiB | | `tests/runner-acceptance` | 3 | 0 | 0.58 GiB | | `tests/e2e`, through layer declarations | 63 | 37 | at most 1.59 GiB | | `ui` `AttentionInteractionResolver.test.tsx` | 1 | 0 | n/a | | **Total** | **517** | **803** | | 27 of the 803 errors came from master while this pull request was open. Each merge of master brought new test code that no check had read. Errors that were config errors, not real errors: - One TS2688 for each provider, because the root install does not install provider dependencies. - 9 in the Runner Node program (TS7016 and TS6059). - 2 in `tsconfig.browser.json`. TypeScript 7 removed `baseUrl`, so that config was invalid. - 213 in the Playwright suites (`@/*` aliases and Vite types). - About 60 in `tests/e2e` (import extensions and aliases). `packages/plugins/create-paperclip-plugin` has no gap. Its tsconfig excludes its test, but `cli/tsconfig.json` reaches that test through an import. `server/test-baselines` also has no gap, because `tests/lifecycle-baseline` reads it. ## What Changed - Added `scripts/typecheck-test-programs.mjs` and its tests. The `Typecheck + Release Registry` job runs it. It checks each program in `PROGRAMS` on one thread. - The script fails if a tracked `*.test.ts(x)` or `*.spec.ts(x)` file outside `server/src` is in no program. The compiler file list decides this, because `include` drops `foo.test.tsx` when `foo.test.ts` exists. - Added a `tsconfig.test.json` to each uncovered package, and `tests/tsconfig.playwright.json`. - Added `tests/e2e` to `PROJECTS` in `scripts/typecheck-test-projects.mjs`, with a `tsconfig.json` and a `tsconfig.layers.json`. - Added `scripts/typecheck-sandbox-providers`. It is a small workspace that installs provider SDKs from one lockfile. I did not add a lockfile inside each provider, because `scripts/build-standalone-public-packages.mjs` uses a frozen install when one exists, and that changes what a release resolves. - Corrected `packages/paperclip-runner/tsconfig.browser.json` for TypeScript 7. - Set `rootDir` to `..` in `packages/paperclip-runner/tsconfig.test.json`, because one Runner test imports from `packages/adapter-utils/src`. The program does not emit files, so this changes no build output. - Moved the Kubernetes provider from `vitest ^3.2.4` to `^4.1.8`, the version the other providers use. Vitest 3 requires `tinypool ^1.1.1`, which has two critical advisories (GHSA-85c8-ppgw-ccpr and GHSA-5gmw-xhrv-c9v3). The first patched version is 2.1.2. Its tests have the same result on both versions: 189 passed, and the same 12 `file-sync` tests failed. - Moved the Kubernetes provider from `zod ^3.24.2` to `^4.6.6`, and gave its two `z.record` calls a key schema. This is the only production change, and it stays in this pull request. See "Production change for review". - Renamed `AttentionInteractionResolver.test.ts` to `AttentionInteractionResolver.replace.test.ts`, so TypeScript reads the `.test.tsx` beside it. - Fixed all 803 errors, with changes to test files and test support only. The fix for `contracts/native-execution.test.ts` is the same as the fix in paperclipai/paperclip#18739, so those two changes merge cleanly. - Updated `doc/DEVELOPING.md`. This pull request does not edit `server/scripts/typecheck-layers.mjs`. paperclipai/paperclip#18739 and paperclipai/paperclip#18738 have both merged, and this branch includes them. ### Production change for review The new lockfile made dependency review flag `zod@3.25.76` as a high-severity advisory (GHSA-mrhx-c7xc-m3w5). The first patched version is 4.6.6, so 3.x has no fix. The published Kubernetes plugin uses zod to validate operator config. The zod 3 source does not compile against zod 4, so a typecheck-only override cannot fix this. The existing tests do not prove this change, because they also passed with the invalid one-argument `z.record`. So I parsed 45 inputs with the same schema on zod 3.25.76 and zod 4.6.7: | Result | Inputs | Detail | | --- | ---: | --- | | Identical | 23 | Includes defaults, valid config, and a `__proto__` key | | Same outcome, new error wording | 21 | For example, `Invalid url` is now `Invalid URL`. `validateConfig` shows this text to operators. | | Different outcome | 1 | `jobTtlSecondsAfterFinished: 2 ** 60` was valid and is now rejected. The new limit is `Number.MAX_SAFE_INTEGER`. | I did not test this against a real cluster. The decision is to keep this change in this pull request. Since then, paperclipai/paperclip#17364 has moved the rest of the repository to zod 4.6.x, so the Kubernetes plugin now matches it. ### Behavior change for review `reducedMotion` under `use` is not a Playwright option. It had no effect in 11 configs and 2 `test.use` calls, so those suites ran with `no-preference`. The options are now under `contextOptions`, so reduced motion now takes effect. `app-shell-design.spec.ts` asserts a real transition, so it now asks for `no-preference` itself. The baseline manifest has `snapshotCount: 0`, so this invalidates no published baseline. A frozen trial reference PNG can differ. ### Findings These are product questions. I did not change them, and I did not hide them with casts: 1. **`PrpEvent` and its schema disagree.** The TypeScript type in `src/protocol/replay-contract.ts` requires `normalizedSessionId`. The `required` list in `protocol/schemas/event.schema.json` does not include it. `validatePrpEvent` accepts events that the type forbids. 2. **`requestCheckboxConfirmation` is stricter than its runtime.** It uses the output type, so plugin authors must pass `continuationPolicy` and `allowDeclineReason`, although the schema gives them defaults. `z.input` would fix this. 3. **`RetainedRunnerdMaintenanceInput.backend.kind` is a free string.** The quarantine domain comes from `descriptor.kind`, but the release uses `proof.backend.kind`. If they differ, `completeRetainedNativeSessionCleanup` returns 0 and the quarantine stays. The production caller passes `"runner"` today, so this is not a live bug. 4. **`CodexHarnessSession.close(input?)` is optional, but the contract requires `{ reason }`.** These are tests that verified nothing. I corrected them, and they pass: - `backends/native-instruction-measurement.test.ts` filtered on `placement === "core"`, which is not a valid placement. It measured an empty tool list. It now measures 14 tools. - `drivers/acpx/codex-acpx-driver.test.ts` checked for `usage.updated`, which is not an event type. - The `hostFixture()` helper in `drivers/acpx/runtime-host.test.ts` ignored two overrides, so `expect(prepareSandbox).not.toHaveBeenCalled()` could not fail. - `live/live-session.test.ts` set `agent: "pi"` before a spread that overwrote it. - `native-distribution-integrity.test.ts` passed an AbortSignal where `openCommand` expects `{ signal }`. These are gaps that this pull request does not close: - **The root lockfile still has some old zod versions.** paperclipai/paperclip#17364 moved the direct zod dependencies to 4.6.x. `pnpm-lock.yaml` still resolves `zod@3.25.76`, `4.1.11`, and `4.4.3`, which are below the patched 4.6.6. `packages/paperclip-runner/scripts/pi-distribution/package.json` pins `zod` to `3.25.76`, and some agent SDKs resolve `4.4.3`. Dependency review does not flag them, because this pull request does not add them. - **Dead GitHub coverage.** `chat-adapters-ui-providers.spec.ts` returned early for GitHub, so four later `provider === "github"` branches never ran. I removed them. They included the private key file and paste match, and the reconnect flow. Nothing asserts those behaviors in that spec now. - **Provider tests do not run in CI.** The root `vitest.config.ts` lists only Daytona. The other 8 providers, `bridge-template` included, are typechecked now, but CI does not run their tests. - **One `packages/db` test never runs.** `scripts/clean-poisoned-claude-sessions.test.ts` is outside the `include` of `packages/db/vitest.config.ts`. - **`plugin-orchestration-smoke-example` does not collect.** `@paperclipai/shared` is not linked in its `node_modules`. It passes 3 of 3 with an alias. ## Verification Run these commands: ```sh node scripts/typecheck-test-programs.mjs --low-memory pnpm --filter @paperclipai/server typecheck node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations node --test scripts/typecheck-test-programs.test.mjs ``` Results on macOS (arm64), Node 26.12. CI uses Node 24, and its `Typecheck + Release Registry` job also passes. At head `7d4b5e907a`, the test programs, `tests/e2e`, and the script tests pass again with 0 errors. The times and peaks below come from the run before the two latest merges of master: | Check | Result | | --- | --- | | Test programs, `--low-memory` | Exit 0, 0 errors, 42 seconds, peak 1.78 GiB | | Server layers, low memory | Exit 0, 0 errors, 38 seconds, peak 3.41 GiB | | Test projects, including `tests/e2e` | Exit 0, 0 errors, 13 seconds, peak 1.59 GiB | | Script tests | 12 passed | | Injected type error in each of 23 places | Each one failed its program with TS2322 | | Coverage check | 0 unchecked files, 10 seconds | I did not run this in a 4 GiB container. The peaks above are `maximum resident set size` for the largest single process. **Runtime check of the fixes.** Types do not prove that a test still works, so I ran the suites before the two latest merges of master: | Suite | Result | | --- | --- | | Runner, full suite | 4,885 passed, 3 failed, 13 skipped | | Sandbox providers, 10 packages | 939 passed, 12 failed, 36 skipped | | Plugin SDK | 200 passed | | `kv-demo-mcp-server` | 12 passed | | Renamed UI test and its sibling | 4 passed | All 15 failures exist without these fixes or come from the local machine: - **3 Cursor Cloud timeouts.** `artifact-delivery.test.ts` has 1 and `cursor-cloud-driver.test.ts` has 2. They time out in a full local run, and all 133 tests in those files with `codex-security-config.test.ts` pass when run alone. They fail in the same way with those two files at their unedited versions. - **12 Kubernetes `file-sync.test.ts` failures.** They fail in the same way before and after the fixes. The first failure is that `/proc/self/fd` does not exist on macOS. After each of the two latest merges of master, I ran only the Runner test files that I changed: | Merge | Result | | --- | --- | | `885c249cb9`, 8 files | 534 passed | | `7d4b5e907a`, 7 files | 672 passed, 16 failed | The 16 failures are in the two `source-reconciliation` test files. They fail in the same way with master's unedited versions of those files. 15 of them fail with `ACPX sidecar exited at initialize`, so I think my local build of the Rust and sidecar binaries is out of date. I did not rebuild them to check. CI builds them from scratch, and all Runner jobs pass in CI at `7d4b5e907a`. **Not run:** - `tests/e2e` and the 8 Playwright suites need a server, so most of them have only compile checks. `playwright test --list` loads 308 tests from the main e2e config. - `packages/plugins/sandbox-providers/kubernetes/test/integration` needs a cluster, so it has only compile checks. - I did not run the full `pnpm test:run` or root `pnpm build`. Required CI runs them. **Reduced motion.** I ran 6 motion-sensitive Storybook specs in system Chrome with a temporary config, once with `reduce` and once with `no-preference`. Both runs had 45 passed and the same 5 failed. So reduced motion did not change those results. The 5 failures do not depend on motion. At least one comes from the browser, because system Chrome prints `oklch(55.6% 0 0)` where the test expects `oklch(0.556 0 0)`. I did not run the other Storybook specs. The `storybook-visual` label runs that workflow. ## Risks - **A new lockfile to maintain.** A change to a provider `package.json` now fails the typecheck job until someone refreshes `scripts/typecheck-sandbox-providers/pnpm-lock.yaml`. The failure message gives the command. - **The job has a new install.** The step downloads provider SDKs, so it depends on the npm registry. It uses `--frozen-lockfile` and `--ignore-scripts`. - **The local install replaces provider `node_modules`.** A contributor with a standalone provider install must reinstall that provider after running the script. - **Reduced motion can change Storybook results.** See "Behavior change for review". That workflow is not part of required CI. - **Kubernetes plugin validation changed.** A release of this plugin ships zod 4. Error wording changes, and very large integers are now rejected. See "Production change for review". - **Master moves fast.** Each merge of master brought new test code with type errors, 27 so far. After this merges, the check reports those errors in the pull request that adds them. ## Model Used - Provider: Anthropic Claude, through Claude Code - Model: Claude Opus 5.5 (`claude-opus-5-5`) - Capabilities: tool use, code execution, and subagents for the type fixes in separate files ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details. The app generated the branch name. - [ ] I have run tests locally and they pass. See the failures explained in Verification. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 07:33:59 -08:00
Nicky LeachandClaude Opus 5.5	8ab89f41e4	fix(build): typecheck test projects in 4 GiB (#18738) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Contributors and agents typecheck this repository, often on machines with limited memory > - paperclipai/paperclip#18737 made `pnpm -r typecheck` fit in 4 GiB because the server now checks in layers > - The test projects outside `pnpm -r` still import server source through relative paths, so each one checks most of the server as one program > - These projects peaked between 6.9 GiB and 14.9 GiB, so they fail on a 4 GiB machine > - This pull request makes these projects read the server layers as declarations, and adds a CI step that keeps them that way > - The benefit is that every TypeScript project in the repository can now be checked in 4 GiB, and a later change cannot silently undo this ## Linked Issues or Issue Description Refs paperclipai/paperclip#18737, which is merged. This pull request builds on the server layers that it added. No issue exists for this gap. **What happened?** `tsc -p tests/runner-e2e/tsconfig.json` peaked at 14.9 GiB at default settings. `tests/speko-native` peaked at 12.2 GiB, `server/src/__tests__/helpers/maintenance` at 9.6 GiB, and `tests/lifecycle-baseline` at 6.9 GiB. On one thread, `tests/runner-e2e` still peaked at 7.7 GiB. **Expected behavior** Each project passes with 4 GiB of memory. **Steps to reproduce** Run `tsc -p tests/runner-e2e/tsconfig.json` in a Linux container with `--memory=4g --memory-swap=4g`. The kernel stops the process. ## What Changed - Added `tsconfig.layers.json` to the four projects. Each file extends the project's `tsconfig.json` and references the generated server declarations. - Added `scripts/typecheck-test-projects.mjs`. It builds the server declarations and then checks the named projects, or all four. It takes `--low-memory`. - Changed `test:e2e:runner:typecheck` and `test:lifecycle-baseline:typecheck` to call that script. `docker-runner-check.yml` calls the first script by name, so that workflow needs no edit. - Added `--declarations-only` to `server/scripts/typecheck-layers.mjs`. The script now also generates `tsconfig.declarations.json`, so hand-written configs do not name individual layers. - Exported `runPnpm` from `scripts/typecheck-low-memory.mjs`, so the new script can use it. - Added a step to the `Typecheck + Release Registry` job. It runs straight after the server typecheck and reuses its declarations through `--reuse-declarations`. - The script now fails if a project loads server source, even when the types pass. Test support files, declaration files and JSON files do not count. This check does not enforce a memory limit. - Added a unit test that fails if a new project under `tests/` is neither in `PROJECTS` nor named as a single program. `tests/runner-acceptance` is the one named project. - Added `server/src/types/express.d.ts` to `tests/lifecycle-baseline/tsconfig.json`. That plain config failed alone with 11 TS2339 errors, and it now passes. - Exported `isTestFile` from `server/scripts/typecheck-layers.mjs`, so the source check uses the same rule as the layers. - Turned off `declaration` in the layered configs. These projects emit nothing, and declaration checks gave a false TS2883 error when `tests/runner-e2e` read the server as declarations. - Added tests, one of which runs the real compiler. Updated `doc/DEVELOPING.md` and `tests/lifecycle-baseline/README.md`. The references are not in each project's `tsconfig.json` because Vitest reads that file. A test showed that Vitest fails when a referenced config does not exist yet. The `tsconfig.json` files do not change, so editors and Vitest see the same configs as before. No new `package.json` scripts were added, as `.greptile/rules.md` requires. ## Verification I tested commit `9763880fab`, which is based on `1c9ca5829f`. I copied the commit into a named Docker volume and ran a fresh install. Each row ran in its own Linux arm64 container with `--memory=4g --memory-swap=4g` and 8 CPUs. The first row started with no build output. All rows used `--low-memory`. CI does not set a memory limit. I used the limit only for this measurement. | Project | Errors | Peak (cgroup) | Max process RSS | OOM kills | Time | | --- | --- | --- | --- | --- | --- | | `tests/runner-e2e` | 0 | 3.34 GiB | 3.06 GiB | 0 | 57 s | | `tests/speko-native` | 0 | 3.60 GiB | 3.27 GiB | 0 | 51 s | | `tests/lifecycle-baseline` | 0 | 3.43 GiB | 3.15 GiB | 0 | 50 s | | `server/src/__tests__/helpers/maintenance` | 0 | 3.39 GiB | 3.13 GiB | 0 | 52 s | `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm test:e2e:runner:typecheck` and `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm test:lifecycle-baseline:typecheck` also passed in the same limit, with peaks of 3.55 GiB and 3.52 GiB. I also ran the CI sequence in one container with the same limit: `pnpm --filter @paperclipai/server typecheck`, then `node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations`. Both passed, with a peak of 3.46 GiB. The new step took 16 s. The server layer build sets the peak in each row. Against the declarations, the projects peaked between 0.9 GiB and 2.2 GiB at default settings on macOS. Server source files that each project loads, without test support, declaration or JSON files: | Project | Plain `tsconfig.json` | `tsconfig.layers.json` | | --- | --- | --- | | `tests/runner-e2e` | 1,345 | 0 | | `tests/speko-native` | 1,647 | 0 | | `tests/lifecycle-baseline` | 1,343 | 0 | | `server/src/__tests__/helpers/maintenance` | 1,716 | 0 | I counted these on `3f0f31b2f9`. The source check in the script confirmed 0 for all four projects on this commit. **Limits of this proof:** - Without `--low-memory`, the 4 GiB run does not pass. The kernel stopped the layer build for `tests/runner-e2e` on `3f0f31b2f9`, and the script exited with status 1. I did not repeat this run on the final commit. The script does not turn on low-memory mode by itself. - The Linux runs used arm64, not x64. - The new CI step has not run in CI. `pr-trusted.yml` runs from `@master`, so the step first runs after this pull request merges. Check its first run. **Other checks on this commit:** - `server/scripts/typecheck-layers.test.mjs`: 38 tests pass. One test uses the real compiler to show that the source check tells a layered config from a config with no references. - With the references removed from `tests/speko-native/tsconfig.layers.json`, the script exits with status 1 and reports 1,644 server source files. - With the declarations removed, `--reuse-declarations` exits with status 1 and reports TS6053. - A type error in a test project makes the script exit with status 1. - `tests/lifecycle-baseline/tsconfig.json` passes with 0 errors. - `tests/lifecycle-baseline` Vitest suite: 533 tests pass. - Workflow tests that read `pr-trusted.yml` and CI routing: 114 tests pass. **Earlier checks:** - `server/src/__tests__/server-package-build-script.test.ts` passed 10 tests on `3f0f31b2f9`. This test failed in `ci / General tests (server (9/12))` on an older head. It tested a build script split that the final #18737 removed. - A changed signature in `server/src/services/run-liveness.ts` failed `tests/lifecycle-baseline/authority.test.ts` through the rebuilt declarations, before the rebases. When I restored the signature, the check passed. **Not run locally:** - `pnpm test:run` and `pnpm build`. CI covers these. **Failures that this pull request does not cause:** `pnpm test:e2e:runner:unit` has 13 failures on this commit, in `cursor-cloud-replay-fault`, `first-task`, `operating-receipts`, `pi-controls-evidence` and `workflow-security`. The same 13 tests fail on `master` at `6ef37398c8`. A comparison of the test names shows no differences. PR CI does not run this suite. ## Risks - Low risk to runtime behavior. No shipped code changes. - **Slower check:** `pnpm test:e2e:runner:typecheck` now builds the server layers on each run. Before the rebase, it took 30 s on macOS at default settings, about the same as the old single program for that project. On this commit it took 57 s in low-memory mode in the 4 GiB container. The new CI step reuses the declarations, so it took 16 s. - **Narrower check:** Server source is now checked only by the server's own layers, not again under each project's compiler options. The projects also skip declaration-portability checks. These projects emit nothing, so those checks had no output to protect. - **Unchanged path:** `tests/runner-e2e/native-completion-checks.mjs` still runs `tsc -p tests/runner-e2e/tsconfig.json` as one program. It needs about 15 GiB at default settings. I did not change it because that file and its inputs are part of a fingerprinted source contract, and no workflow calls it. - **Layer growth:** The new checks do not measure memory. If the layers or the import cycle grow, the checks still pass and the peak rises. This is intentional. Crossing 4 GiB does not fail a build. - **Project list:** The CI step checks only the projects in `PROJECTS`. The unit test covers new projects under `tests/`, but not a new project in another directory. - **Reused declarations:** `--reuse-declarations` trusts the declarations on disk. CI uses it straight after the server typecheck on the same commit. If that typecheck fails, the job stops first. Stale declarations can pass locally, so `doc/DEVELOPING.md` warns against this. - **Job dependency:** A type error in a test project now fails `Typecheck + Release Registry`, and so `ci / verify`. - **Not covered:** The new step runs no tests. `node scripts/typecheck-low-memory.mjs` and the Docker image build also have no new guard. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: Anthropic. Model: Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-03 11:09:12 -08:00