Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-05 21:02:34 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,271 Commits 1,495 Branches 2,299 Tags
4e29f001396c9fc76075850165ccfa3a6743e3ef
Commit Graph
3 Commits
Author SHA1 Message Date
Nicky LeachandClaude Opus 5.5 11ac90e31f fix(build): typecheck server/src/__tests__ and fix every type error (#18739) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server has about 1,100 test and helper files in
`server/src/__tests__`
> - `server/tsconfig.json` excludes that directory, Vitest does not
check types, and no script runs the tsconfig in that directory
> - So a test can call a function with the wrong arguments and still
pass, and no tool reports it
> - This gap hid 2,168 real type errors, and some of them hid tests that
did not test what they claim
> - This pull request fixes all of these errors and adds the directory
to the server layers from paperclipai/paperclip#18737
> - The benefit is that a type error in a server test now fails the
typecheck, with no baseline and no backlog

## Linked Issues or Issue Description

Refs paperclipai/paperclip#18737, which is merged and added the server
layers. Refs paperclipai/paperclip#18738, which is merged and checks the
test projects outside `pnpm -r`, so it is not a duplicate. Refs
paperclipai/paperclip#18137, which records that this directory has no
check.

**What happened?**

No tool checked the types of 1,074 of the 1,101 files in
`server/src/__tests__`. I searched `package.json`, the workflows, and
all tsconfig files. 27 files are in the program of another tsconfig, and
these 27 have 0 errors. 11 of the 27 are in
`helpers/maintenance/tsconfig.json`. No script ran that project when I
measured this. paperclipai/paperclip#18738 now runs it.

**Expected behavior**

The server typecheck checks these files, and a type error fails the
check.

**Steps to reproduce**

1. On the base commit, run `tsc -p server/src/__tests__/tsconfig.json
--noEmit`. It reports 6,536 errors and needs 12.3 GiB.
2. Add a type error to any file in `server/src/__tests__`.
3. Run `pnpm --filter @paperclipai/server typecheck`. It passes.

**Paperclip version or commit**

I measured the error breakdown below on commit `8dfd632452`, with
TypeScript 7.0.2. The branch now includes master up to
paperclipai/paperclip#18865.

**Deployment mode**

Not applicable. This issue applies to development and CI. No production
source file in the server changes.

### Error breakdown

Each row adds one change to the row above.

| Step | Total errors | Change | Cause |
| --- | --- | --- | --- |
| Config as committed | 6,536 | | |
| Add `src/types/*.d.ts` | 4,447 | -2,089 | Config. The Express request
types were missing. |
| Set `rootDir` to the repository root | 2,436 | -2,011 | Config. |
| Set `allowImportingTsExtensions` | 2,181 | -255 | Config. Vitest
accepts `.ts` imports. |
| Add `ES2024.Promise` to `lib` | 2,138 | -43 | Config. Node 24 has
`Promise.withResolvers`. |
| Set `allowJs` for `.mjs` imports | 2,148 | +10 | Config. It removes 8
`TS7016` errors and shows new real errors. |
| Set `declaration` to false | 2,142 | -6 | Config. Tests emit nothing.
|
| Count only files in `server/src` | 2,058 | -84 | Other packages: 53
Daytona, 29 Runner test, 2 UI. |

So 4,478 of the 6,536 errors (69%) came from config or scope, and 2,058
are real type errors in 314 files. The 29 Runner test errors are also
real, which gives 2,087 errors. Tests that master added after this
measurement had 81 more errors, so this pull request fixes 2,168 errors.

This table classifies the first 2,058 errors.

| Kind of real error | Errors |
| --- | --- |
| Wrong argument or assignment type | 1,023 |
| Stale or partial fixture objects | 525 |
| Property that the type does not have | 317 |
| Missing narrowing of null or unknown | 101 |
| Implicit any | 60 |
| Missing name or module | 16 |
| Other | 16 |

### What the type errors hid

Most errors were stale fixtures and loose mocks. These errors hid real
test problems, and this pull request fixes them:

- **A test that never ran.** `instrumentation.test.ts` required
`sdk-trace-base`, which is not installed. So 3 span tests were skipped.
They now run and pass.
- **Assertions that could not fail.**
- `issue-thread-interactions-service` used a company ID as an issue ID.
- `status-card-evaluation-work` compared `[]` with `[]` in a rollback
check.
- `slack-conversation-lifecycle` read `item.issueId`, which does not
exist.
  - `email-channels` used a permission key that does not exist.
- `heartbeat-project-env` imported `LOW_TRUST_REVIEW_PRESET`, which is
not exported. The value was undefined.
- **Postgres processes that did not stop.** 11 test files called `stop`
on the test database helper. That method does not exist, and the helper
method is `cleanup`.
- **Arguments that the service ignored.**
- `methodKey` is now `connectionMethodKey` in the Daytona, Unikraft,
1Password, and Railway tests.
- 108 calls passed `"owner-user"` to a parameter that expects an options
object.
- 9 chat wizard fixtures passed `managementContext` as `source`, so they
ran `existing_app` and not `manifest`.
  - 2 tests passed a `{ userId }` actor that the function ignores.
- Inserts into `environments` passed a `companyId` column that the table
does not have.
- **Problems in tests that master added during this work.**
- `sandbox-creation-operations` read `database.connectionString` when an
external test database was set. In that mode `database` is undefined, so
the fixture would stop with a TypeError. It now uses the external URL.
- `idle-completed-history` set a Dot operation to `dispatched`. That is
a Muse status, and the Dot type does not have it. The fixture now uses
`admitted`, which is a real in-flight Dot status.

### Possible product problems, not changed

These may be real problems in production code or types. I did not change
them, because this pull request does not change production behavior:

- `decisions-service` `service.create` does not validate the lenient
`cancel_issue_tree` input.
- `createConnection` ignores `input.connectionPurpose`.
- The `createConnection` route schema default stores `{}` for
`transportConfig`.
- `prepareManagedAiRuntime` accepted the invalid value `mode:
"connection"`.
- `createBinding` accepts invalid `projectionClass` strings outside
class 3.
- `ensureManagedProjectWorkspace` forwards an untyped `repoRef` by
spread.
- `routineVariableSchema.label` is optional, but `RoutineVariable.label`
is required.
- `ToolRunContext.projectId` may be too strict.
- `withdrawInteraction` reads `issue.status`, which its type omits.
- `issueService.list` and `reconcileNativeFinalizations` omit fields
that they return at runtime.
- `createGitHubExternalObjectProvider` handles `tokenProvider: null`,
but its type forbids this value.

## What Changed

- Fixed the 2,168 type errors. The diff changes 322 test and helper
files.
- Corrected `server/src/__tests__/tsconfig.json`. Editors and the
typecheck now read the same options and report the same errors.
- Added three `test-tree` layers to
`server/scripts/typecheck-layers.mjs`. They read the server layers as
declarations, and any error fails them. A hash of the path selects the
shard, so a new file does not move other files.
- Added `server/scripts/typecheck-stubs/daytona-sdk.d.ts`. Server tests
reach Daytona plugin source, and the root install does not have
`@daytonaio/sdk`.
- Changed two imports in `ui/src/lib/issue-execution-policy.ts` to use
`.js` extensions, because server tests import this file.
- Made `declarationsConfig` and `--declarations-only` skip the
`test-tree` layers, because these layers emit nothing. Other projects
read only the source layers. Added a test for this.
- Updated `doc/DEVELOPING.md`.

## Verification

Run these commands:

```sh
pnpm --filter @paperclipai/server typecheck
pnpm exec vitest run server/scripts/typecheck-layers.test.mjs
```

Results on macOS (arm64), Node 24, 16 cores. Each check ran after the
merge of paperclipai/paperclip#18738 unless the row says otherwise:

| Check | Commit | Result |
| --- | --- | --- |
| Server typecheck, `PAPERCLIP_TYPECHECK_LOW_MEMORY=1`, all 7 layers |
`6e45a08573` | Exit 0, 0 errors, 62 seconds, peak 3.52 GiB |
| `node scripts/typecheck-test-projects.mjs --low-memory` | `a3306bd13e`
| Exit 0, 0 errors, 42 seconds, peak 3.25 GiB |
| The same script with `--reuse-declarations` | `1b878aabc0` | Exit 0, 0
errors |
| `typecheck-layers.test.mjs` and `live-recovery-work.test.ts` |
`01db5634e4` | 90 passed. These include real compiler runs for a pass, a
type error, and an invalid tsconfig. |
| `tsconfig.declarations.json` after a full server typecheck |
`0fb1e4fb0a` | It references only `below`, `cycle`, and `above`. No
generated config names a `tsconfig.layers.json` file. |
| Owned runtime test in `idle-completed-history` | `e8f2f448ac` |
Passed, with the fix that master later replaced |
| `idle-incoming-wake`, `plugin-idle-manager`, and the changed
`chat-channels` test | `6e45a08573` | 30 passed |
| Runner `native-execution.test.ts` | `eab4dcc1f4` | 114 passed |
| The 19 server test files that changed since `eab4dcc1f4`, and
`typecheck-layers.test.mjs` | `259737b700` | 4,275 of 4,277 passed. The
2 failures are in `chat-channels` and are in the list of 7 below. |

The peak is the largest single process, from `/usr/bin/time -l`. Peaks
for the server typecheck were between 3.42 GiB and 3.56 GiB across 8
runs. Peaks for the test projects were 3.25 GiB and 3.37 GiB across 2
runs. I did not measure a cgroup.

**Runtime check of the fixes.** Types do not prove that a test still
works. So I ran all 309 changed server test files, 11,929 tests, before
the rebase onto paperclipai/paperclip#18737:

| Result | Tests | Detail |
| --- | --- | --- |
| Passed | 11,913 | Includes the 2 tests and the 54 `dot-runner` tests
that passed in a later run, as described below. |
| Failed | 7 | The same 7 tests also fail on the commit before the
fixes. |
| Skipped | 9 | These tests have their own skip conditions: `sentry` 5,
and 1 each in `bundled-plugin-registration`, `customer-success-cloud`,
`onepassword-connection`, and `workspace-runtime-https-live-exercise`. |

The 7 tests that fail with and without this change, on my machine:

- 3 tests in `chat-channels` and `email-channels` stop with `ENOENT` on
a `SKILL.md` file. The path is in the parent checkout, above the git
worktree that I used, so I expect that the worktree is the cause. I did
not prove this.
- `workspace-instance-cleanup` compares `/private/var/...` with
`/var/...`. This is a macOS symlink difference.
- 2 tests in `git-reproducible-managed-checkouts` get
`unknown_base_git_files` and expect `unpushed_managed_checkout`. I did
not find the cause.
- 1 more test in `chat-channels`, `materializes direct external-chat
finals once`, gets `undefined`. I did not find the cause.

CI is the check for these 7 tests. `ci / verify` passed on `eab4dcc1f4`.

Failures during this work:

- My fix to `agent-runtime-connector-access` changed an invalid
permission key in the setup, but not in the assertion. The runtime check
found this. I corrected the assertion, and the file passes.
- `heartbeat-plugin-environment` failed one wait in the full run. It
passed three times alone, so I think that load caused the failure. It
can fail again under load.
- `dot-runner` skipped all 54 tests in the full run because `cargo` was
not on my `PATH`. It then passed 54 of 54 with the toolchain on the
`PATH`.
- My first `lib` value removed the default DOM types, and the error
count went up by 375. I added the DOM libraries.
- My Daytona stub with bare `any` left 48 errors, because generic
helpers then inferred `unknown`. I added shapes that return promises.
- One fix to `agent-secrets-routes` used `class_3_static_lease`, and the
test failed. That class allows only specific paths. I reverted to
`unclassified`.
- `ci / General tests (server (8/12))` failed on `259737b700` because of
the flaky port in `idle-completed-history`. paperclipai/paperclip#18853
fixed this test on master, so this pull request keeps that fix.

**Audit of the diff.** I counted these patterns in added and removed
lines, to find fixes that only hide an error:

| Pattern | Added | Removed |
| --- | --- | --- |
| `@ts-ignore` | 0 | 0 |
| `@ts-expect-error` | 0 | 1 |
| `.skip` | 0 | 0 |
| `as any` | 9 | 9 |
| `: any` | 11 | 2 |
| `as never` | 8 | 2 |
| `as unknown as` | 35 | 4 |
| `Promise<unknown>` | 16 | 5 |
| `expect(` | 128 | 113 |

No hunk removes more `expect(` calls than it adds. Three removals of
test content were intentional, and I checked each one:

- `muse-control` removed `signalToken`. Line 1012 already asserts that
it is absent.
- `merged-pr-confirmation-sweep` removed legacy aliases. Other tests
cover them.
- `heartbeat-workspace-branch-containment` removed a `finalize` branch
that no test row can reach.

Some partial fixtures stay unchecked through `as unknown as` and
`Promise<unknown>`, mostly in `issue-agent-mutation-ownership-routes`
and `issue-comment-reopen-routes`.

**Accuracy of the check.** One program over the tree and the three
shards reported the same errors at the same positions. Memory, as the
peak of the largest process on one thread, was 3.93 GiB for one program
and 2.43 GiB for the largest shard. I measured these two values before
the fixes.

Not run or not proven:

- I did not run the full `pnpm test:run`, or a root `pnpm build`. I
expect CI to run the full suites.
- I did not run the 4 GiB container check, or the full
`scripts/typecheck-low-memory.mjs`. The peaks above come from macOS, so
they are not a Linux cgroup measurement.
- I did not run the UI typecheck after the merges.
- `workspace-runtime-https-live-exercise` is skipped, so its `cleanup`
fix did not run.

## Risks

- **A large diff in test files.** 328 files change. A pull request that
edits the same tests can get merge conflicts. New tests on master can
also add type errors before this merges. Any such error fails this check
and needs a fix here.
- **A higher cost for changes to source types.** A change to a type can
add errors in test files that the author did not touch. The author must
then fix them. This is the purpose of the check, but it is new work.
- **Changed test behavior.** Some tests now run paths that they did not
run before, such as the 9 chat wizard fixtures and the 3 span tests.
They pass here, but they can be more sensitive to timing or to the
environment.
- **The Daytona stub can go stale.** The stub is written by hand. If the
plugin uses a new SDK member, the typecheck reports an error until
someone updates the stub.
- **The release path.** `release-verify.yml` and `scripts/release.sh`
run `pnpm -r typecheck`, so they now run this check.
- **The tsconfig change.** The test transform reads
`server/src/__tests__/tsconfig.json`. The options that I changed are for
type checks, and the runs above passed.
- **A gap that remains.** Tsconfigs in `packages/paperclip-runner` and
in most sandbox provider plugins exclude `*.test.ts`. This check does
not read most of those test files, so they can still drift. That work is
separate from this pull request.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with
reasoning, tool use, code execution, and subagents. The runtime does not
show the context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (7 tests fail locally, with
and without this change. See Verification.)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-03 15:27:38 -08:00
Nicky LeachandClaude Opus 5.5 8ab89f41e4 fix(build): typecheck test projects in 4 GiB (#18738) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Contributors and agents typecheck this repository, often on machines
with limited memory
> - paperclipai/paperclip#18737 made `pnpm -r typecheck` fit in 4 GiB
because the server now checks in layers
> - The test projects outside `pnpm -r` still import server source
through relative paths, so each one checks most of the server as one
program
> - These projects peaked between 6.9 GiB and 14.9 GiB, so they fail on
a 4 GiB machine
> - This pull request makes these projects read the server layers as
declarations, and adds a CI step that keeps them that way
> - The benefit is that every TypeScript project in the repository can
now be checked in 4 GiB, and a later change cannot silently undo this

## Linked Issues or Issue Description

Refs paperclipai/paperclip#18737, which is merged. This pull request
builds on the server layers that it added. No issue exists for this gap.

**What happened?**

`tsc -p tests/runner-e2e/tsconfig.json` peaked at 14.9 GiB at default
settings. `tests/speko-native` peaked at 12.2 GiB,
`server/src/__tests__/helpers/maintenance` at 9.6 GiB, and
`tests/lifecycle-baseline` at 6.9 GiB. On one thread, `tests/runner-e2e`
still peaked at 7.7 GiB.

**Expected behavior**

Each project passes with 4 GiB of memory.

**Steps to reproduce**

Run `tsc -p tests/runner-e2e/tsconfig.json` in a Linux container with
`--memory=4g --memory-swap=4g`. The kernel stops the process.

## What Changed

- Added `tsconfig.layers.json` to the four projects. Each file extends
the project's `tsconfig.json` and references the generated server
declarations.
- Added `scripts/typecheck-test-projects.mjs`. It builds the server
declarations and then checks the named projects, or all four. It takes
`--low-memory`.
- Changed `test:e2e:runner:typecheck` and
`test:lifecycle-baseline:typecheck` to call that script.
`docker-runner-check.yml` calls the first script by name, so that
workflow needs no edit.
- Added `--declarations-only` to `server/scripts/typecheck-layers.mjs`.
The script now also generates `tsconfig.declarations.json`, so
hand-written configs do not name individual layers.
- Exported `runPnpm` from `scripts/typecheck-low-memory.mjs`, so the new
script can use it.
- Added a step to the `Typecheck + Release Registry` job. It runs
straight after the server typecheck and reuses its declarations through
`--reuse-declarations`.
- The script now fails if a project loads server source, even when the
types pass. Test support files, declaration files and JSON files do not
count. This check does not enforce a memory limit.
- Added a unit test that fails if a new project under `tests/` is
neither in `PROJECTS` nor named as a single program.
`tests/runner-acceptance` is the one named project.
- Added `server/src/types/express.d.ts` to
`tests/lifecycle-baseline/tsconfig.json`. That plain config failed alone
with 11 TS2339 errors, and it now passes.
- Exported `isTestFile` from `server/scripts/typecheck-layers.mjs`, so
the source check uses the same rule as the layers.
- Turned off `declaration` in the layered configs. These projects emit
nothing, and declaration checks gave a false TS2883 error when
`tests/runner-e2e` read the server as declarations.
- Added tests, one of which runs the real compiler. Updated
`doc/DEVELOPING.md` and `tests/lifecycle-baseline/README.md`.

The references are not in each project's `tsconfig.json` because Vitest
reads that file. A test showed that Vitest fails when a referenced
config does not exist yet. The `tsconfig.json` files do not change, so
editors and Vitest see the same configs as before.

No new `package.json` scripts were added, as `.greptile/rules.md`
requires.

## Verification

I tested commit `9763880fab`, which is based on `1c9ca5829f`. I copied
the commit into a named Docker volume and ran a fresh install. Each row
ran in its own Linux arm64 container with `--memory=4g --memory-swap=4g`
and 8 CPUs. The first row started with no build output. All rows used
`--low-memory`. CI does not set a memory limit. I used the limit only
for this measurement.

| Project | Errors | Peak (cgroup) | Max process RSS | OOM kills | Time
|
| --- | --- | --- | --- | --- | --- |
| `tests/runner-e2e` | 0 | 3.34 GiB | 3.06 GiB | 0 | 57 s |
| `tests/speko-native` | 0 | 3.60 GiB | 3.27 GiB | 0 | 51 s |
| `tests/lifecycle-baseline` | 0 | 3.43 GiB | 3.15 GiB | 0 | 50 s |
| `server/src/__tests__/helpers/maintenance` | 0 | 3.39 GiB | 3.13 GiB |
0 | 52 s |

`PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm test:e2e:runner:typecheck` and
`PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm
test:lifecycle-baseline:typecheck` also passed in the same limit, with
peaks of 3.55 GiB and 3.52 GiB.

I also ran the CI sequence in one container with the same limit: `pnpm
--filter @paperclipai/server typecheck`, then `node
scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations`.
Both passed, with a peak of 3.46 GiB. The new step took 16 s.

The server layer build sets the peak in each row. Against the
declarations, the projects peaked between 0.9 GiB and 2.2 GiB at default
settings on macOS.

Server source files that each project loads, without test support,
declaration or JSON files:

| Project | Plain `tsconfig.json` | `tsconfig.layers.json` |
| --- | --- | --- |
| `tests/runner-e2e` | 1,345 | 0 |
| `tests/speko-native` | 1,647 | 0 |
| `tests/lifecycle-baseline` | 1,343 | 0 |
| `server/src/__tests__/helpers/maintenance` | 1,716 | 0 |

I counted these on `3f0f31b2f9`. The source check in the script
confirmed 0 for all four projects on this commit.

**Limits of this proof:**

- Without `--low-memory`, the 4 GiB run does not pass. The kernel
stopped the layer build for `tests/runner-e2e` on `3f0f31b2f9`, and the
script exited with status 1. I did not repeat this run on the final
commit. The script does not turn on low-memory mode by itself.
- The Linux runs used arm64, not x64.
- The new CI step has not run in CI. `pr-trusted.yml` runs from
`@master`, so the step first runs after this pull request merges. Check
its first run.

**Other checks on this commit:**

- `server/scripts/typecheck-layers.test.mjs`: 38 tests pass. One test
uses the real compiler to show that the source check tells a layered
config from a config with no references.
- With the references removed from
`tests/speko-native/tsconfig.layers.json`, the script exits with status
1 and reports 1,644 server source files.
- With the declarations removed, `--reuse-declarations` exits with
status 1 and reports TS6053.
- A type error in a test project makes the script exit with status 1.
- `tests/lifecycle-baseline/tsconfig.json` passes with 0 errors.
- `tests/lifecycle-baseline` Vitest suite: 533 tests pass.
- Workflow tests that read `pr-trusted.yml` and CI routing: 114 tests
pass.

**Earlier checks:**

- `server/src/__tests__/server-package-build-script.test.ts` passed 10
tests on `3f0f31b2f9`. This test failed in `ci / General tests (server
(9/12))` on an older head. It tested a build script split that the final
#18737 removed.
- A changed signature in `server/src/services/run-liveness.ts` failed
`tests/lifecycle-baseline/authority.test.ts` through the rebuilt
declarations, before the rebases. When I restored the signature, the
check passed.

**Not run locally:**

- `pnpm test:run` and `pnpm build`. CI covers these.

**Failures that this pull request does not cause:**

`pnpm test:e2e:runner:unit` has 13 failures on this commit, in
`cursor-cloud-replay-fault`, `first-task`, `operating-receipts`,
`pi-controls-evidence` and `workflow-security`. The same 13 tests fail
on `master` at `6ef37398c8`. A comparison of the test names shows no
differences. PR CI does not run this suite.

## Risks

- Low risk to runtime behavior. No shipped code changes.
- **Slower check:** `pnpm test:e2e:runner:typecheck` now builds the
server layers on each run. Before the rebase, it took 30 s on macOS at
default settings, about the same as the old single program for that
project. On this commit it took 57 s in low-memory mode in the 4 GiB
container. The new CI step reuses the declarations, so it took 16 s.
- **Narrower check:** Server source is now checked only by the server's
own layers, not again under each project's compiler options. The
projects also skip declaration-portability checks. These projects emit
nothing, so those checks had no output to protect.
- **Unchanged path:** `tests/runner-e2e/native-completion-checks.mjs`
still runs `tsc -p tests/runner-e2e/tsconfig.json` as one program. It
needs about 15 GiB at default settings. I did not change it because that
file and its inputs are part of a fingerprinted source contract, and no
workflow calls it.
- **Layer growth:** The new checks do not measure memory. If the layers
or the import cycle grow, the checks still pass and the peak rises. This
is intentional. Crossing 4 GiB does not fail a build.
- **Project list:** The CI step checks only the projects in `PROJECTS`.
The unit test covers new projects under `tests/`, but not a new project
in another directory.
- **Reused declarations:** `--reuse-declarations` trusts the
declarations on disk. CI uses it straight after the server typecheck on
the same commit. If that typecheck fails, the job stops first. Stale
declarations can pass locally, so `doc/DEVELOPING.md` warns against
this.
- **Job dependency:** A type error in a test project now fails
`Typecheck + Release Registry`, and so `ci / verify`.
- **Not covered:** The new step runs no tests. `node
scripts/typecheck-low-memory.mjs` and the Docker image build also have
no new guard.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Provider: Anthropic. Model: Claude Opus 5.5 (`claude-opus-5-5`),
through Claude Code with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-03 11:09:12 -08:00
Nicky LeachandClaude 6ef37398c8 fix(build): fit typecheck and image build in 4 GiB (#18737) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Contributors, agents, and image builds run `tsc` to check and
compile the repository.
> - The repository uses TypeScript 7, which is a native binary. Node
heap flags do not limit it.
> - The server is one large program, so one `tsc` process needs about 10
GiB.
> - The operating system stops `tsc` under a 4 GiB memory limit.
> - This pull request checks the server in layers, removes the server
from the CLI program, and removes declarations and checks from the image
emit.
> - The benefit is that the full typecheck and the image `tsc` step now
complete in 4 GiB.

## Linked Issues or Issue Description

I found no duplicate pull request or issue. A search for `tsc`, memory,
typecheck, and OOM found only runtime memory work, which is a different
subject.

**What happened?**

`tsc` used too much memory, and the kernel stopped it with exit code 137
in resource-constrained environments. Measured peaks before this change:

| Command | Peak |
| --- | --- |
| Server `tsc --noEmit` | About 10 GiB |
| Server `tsc` emit, as in the Docker build | About 10.7 GiB |
| CLI `tsc --noEmit` | About 7 GiB |
| `pnpm -r typecheck`, whole process tree | About 10.2 GiB |

**Expected behavior**

The typecheck and the image `tsc` step complete under a 4 GiB memory
limit.

**Steps to reproduce**

1. Start a Linux container with `--memory=4g --memory-swap=4g --cpus=4`.
2. Install the dependencies on the base commit.
3. Run `pnpm -r typecheck`. The kernel stops the process with exit code
137.
4. Run `pnpm --filter @paperclipai/server build`. The `tsc` step needs
more than 4 GiB.

**Paperclip version or commit**

Base commit `09162fe893`, with TypeScript 7.0.2.

**Deployment mode**

Not applicable. This issue applies to development, CI, and image builds.
It does not change the server at runtime.

## What Changed

- Add `server/scripts/typecheck-layers.mjs`. It reads the import graph
on each run and runs one `tsc` process per layer. The layers are the
files below the largest import cycle, the cycle, the files above the
cycle, and the tests. Each layer reads the earlier layers as
declarations.
- Add `scripts/run-tsc.mjs`. It adds `--singleThreaded` when
`PAPERCLIP_TYPECHECK_LOW_MEMORY=1` is set. The server layers and the UI
typecheck use it.
- Add `scripts/typecheck-low-memory.mjs`. It checks one package at a
time in this mode. It has no `package.json` entry, as
`.greptile/rules.md` requires.
- Change the server `typecheck` script to build only the Runner
TypeScript output. It does not build the Runner release binary now.
`pnpm build` and `typecheck:build-gaps` still run the full Runner build
and its generated-file checks.
- Change the `tsc` step of the server `build` script to
`scripts/run-tsc.mjs`. The `Dockerfile` sets `PAPERCLIP_TSC_EMIT_ONLY=1`
for that build, so the image emits JavaScript without checks or
declarations. Without this variable, `build` still does a full check for
npm packages and CI.
- Use a non-literal specifier for the `@paperclipai/server` import in
`cli/src/commands/run.ts`. The import result already had no useful type,
and the literal specifier added the full server to the CLI program.
- Add a layered server typecheck step to the `pr-trusted` typecheck job,
because `typecheck:build-gaps` does not run this script.
- Update `AGENTS.md` and `doc/DEVELOPING.md`.

## Verification

Results on Linux use a clean copy of this branch, a new `pnpm install`,
Node 24, Rust 1.97.1, 4 CPUs, and a 4 GiB memory limit with no swap.
Results before the merge of `master`:

| Command | Result | Peak non-reclaimable memory |
| --- | --- | --- |
| `node scripts/typecheck-low-memory.mjs` | Exit 0, 0 OOM kills | 3,492
MiB |
| `PAPERCLIP_TSC_EMIT_ONLY=1 pnpm --filter @paperclipai/server build` |
Exit 0 | 3,656 MiB, from the Rust release build |
| Server single-program `tsc --noEmit --singleThreaded` | Exit 137, 1
OOM kill | Reached the limit |

Results on macOS on the merged head, with peak memory for the full
process tree:

| Command | Before | After |
| --- | --- | --- |
| `pnpm -r typecheck` | 10.2 GiB, exit 0 | 5.47 GiB, exit 0 |
| `node scripts/typecheck-low-memory.mjs` | Not available | 3.81 GiB,
exit 0 |
| CLI typecheck | 7.06 GiB | 1.68 GiB, before the merge |

Other checks:

- Compiler tests: new tests run the real compiler on a small fixture.
They prove that the layers emit declarations and pass, that a renamed
export fails the layer that imports it, and that a changed type fails
the test layer.
- Declaration imports: a new compiler test proves that types from
`.d.mts` and JSON imports stay correct in later layers. On the real
server, a wrong use of `AgentFileManifest` in the cycle layer failed
with `TS2353`.
- Coverage: a test on the real server source asserts that each source
file is in one layer and that no import goes to a later layer. The test
layer contains the remaining files. `src/__tests__` stays out of the
root list, as in `server/tsconfig.json`.
- Error detection: I added one type error to one file in each of the
four layers. The typecheck reported all four errors and exited with 1.
- Cross-layer errors: I renamed an export in the lower layer. The cycle
and upper layers reported the error and the typecheck exited with 1. I
reverted these temporary edits.
- Emit: full `tsc` and `--noCheck --declaration false --declarationMap
false` gave byte-identical JavaScript and source maps for 2,249 files on
the base commit.
- Runtime: `scripts/server-boot-smoke.sh node
server/dist-runtime/index.mjs` passed with a bundle from the emit-only
build.
- CLI: the esbuild bundle keeps the dynamic import, and `node --check`
passed.
- Tests on the merged head: `typecheck-layers`, `prepare-runner-vendor`,
`cli-invocation-safety`, the CLI `runtime` tests, and all 503
`.github/scripts/tests` tests passed. Before the merge,
`release-verify-workflow` and `runner-release-crate-cache` passed.
- Full `pnpm --filter @paperclipai/server build` passed before the
merge.

Failures during this work:

- The first low-memory run failed in the UI with `TS5023: Unknown
compiler option '-b'`, because the flag came before `-b`. I corrected
the argument order and added a test that runs build mode with the flag.
- The first rename check did not rename the export, because BSD `sed`
does not support `\b`. Its exit 0 was not evidence. I ran the check
again with `perl`, and the result above is from that run.
- The first version of the real-compiler fixture had a `TS2774` error in
its own source. The test found this error, and I corrected the fixture.
- CI failed `server-package-build-script.test.ts` on `1abc5e5a43`,
because that test reads the steps of the server `build` script, and I
had moved them to `build:prepare` and `build:assets`. My first search
for readers of this script did not include `server/src`. I removed those
scripts and `build:image`, restored one `build` script, and kept the
test unchanged.
- `GOMEMLIMIT` did not help enough, and I removed it from the plan. On
these steps, `3GiB` saved at most about 300 MiB. With default checkers,
the cycle layer was stopped with and without it, and the lower layer
went from 7 seconds to 151 seconds.

Not run or not proven:

- I did not run the full `pnpm test:run`, a root `pnpm build`, or a full
`docker build`. I expect CI to run the test and build gates.
- I did not measure the Docker build steps other than the server build.
- I did not run the 4 GiB checks on x86-64.
- I did not measure the merged head in the 4 GiB container. The macOS
result on the merged head is 3.81 GiB, and the merge added one source
file to the lower layer.

## Risks

- The margin is small. In the 4 GiB container, the cycle layer peaks
about 600 MiB below the limit, and the Rust release build in the image
build peaks about 440 MiB below it. Growth of the largest import cycle
uses this margin first.
- `node scripts/typecheck-low-memory.mjs` applies only to `pnpm -r`
packages. These commands still check the server as one program and need
more than 4 GiB: `pnpm build`, `test:e2e:runner:typecheck`,
`test:lifecycle-baseline:typecheck`, and the `tests/speko-native`
project.
- A local flow that used the Runner binary from a previous typecheck
must now run `pnpm --filter @paperclipai/paperclip-runner build`.
- The image does not include server declarations now. I found no
consumer in this repository. A downstream image that reads
`server/dist/*.d.ts` must change.
- An image build does not find type errors now. A typecheck in CI must
find them before the image build.
- The import scan uses a regular expression. A false edge can only make
a layer larger. A missed edge causes a `TS6307` failure, so it cannot
skip a file.
- The new CI step adds about one minute to the typecheck job.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with
reasoning, tool use, and code execution. The runtime does not show the
context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-11-03 08:53:35 -08:00