Files
PaperClipAI/server/src/__tests__/startup-refusals.test.ts
T
Devin Foley a59f5a8adc fix(server): stop reporting expected managed-cloud transients to Sentry (#13323)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server reports crashes to Sentry so operators can find real
faults
> - Three expected conditions report as crashes: a client that closes
the connection mid-request, one stale pooled database socket after a
pooled endpoint recycles, and the short boot window where a supervised
cloud stack runs a new app image before its migration runner has caught
up
> - These events arrive in the hundreds and bury real errors
> - This pull request classifies each condition as expected and stops
the Sentry capture for exactly that condition, with behavior unchanged
everywhere else
> - The benefit is a Sentry feed where each event is a real fault

## Linked Issues or Issue Description

**What happened?**

Three noise classes fill the backend Sentry project on managed cloud
fleets:

1. `Error: aborted` (ECONNRESET) reports as a 500 crash when a client
closes the tab or loses its network mid-request. Observed 18 times in
one week from routine client disconnects.
2. `Error: write CONNECTION_CLOSED ...` reports from many query paths
after a pooled Postgres endpoint suspends. The existing single retry in
cloud actor resolution still fails, because a suspended endpoint kills
every pooled socket at once and the one replay draws another dead
socket.
3. `Error: PostgreSQL has pending migrations (...). Refusing to start`
reports from every supervised stack during a fleet upgrade. The
supervisor delivers the new app image before it runs the migration
runner, so each stack crash-loops briefly by design. One fleet roll
produced 329 events (11 per container).

**Expected behavior**

A client disconnect ends the request quietly. A transient dead socket is
replayed until a live socket answers. A supervised mid-upgrade boot
refusal logs and exits nonzero without a Sentry capture, while the same
refusal on a self-hosted deployment keeps reporting.

**Steps to reproduce**

1. Abort an HTTP request mid-flight: the error handler reports a crash
to Sentry.
2. Suspend a pooled Postgres endpoint under an idle server, then issue
two quick requests: the first replay can draw a second dead socket and
surface `CONNECTION_CLOSED`.
3. On a deployment with `PAPERCLIP_CLOUD_API_ORIGIN` set, add a
migration file without running the migration runner and boot: the
refusal reports to Sentry.

**Paperclip version or commit**

master (0e14c61da)

## What Changed

- `server/src/middleware/error-handler.ts`: a request abort (`Error:
aborted` with `ECONNRESET`) ends the response with status 499 and skips
crash reporting and telemetry.
- `server/src/middleware/auth.ts`: `retryOnTransientDbConnectionError`
replays up to twice with a short pause, so a pool-wide recycle does not
defeat the retry.
- `server/src/startup-refusals.ts`: pending migrations on a database
with applied history classify as a supervised-transient refusal
(`schema-migration-pending`). The capture skip applies only when
`PAPERCLIP_CLOUD_API_ORIGIN` is set. A wiped journal beside real tables
keeps reporting. Self-hosted behavior is unchanged.
- Tests updated and added for all three behaviors.

## Verification

- `pnpm vitest run src/__tests__/startup-refusals.test.ts
src/__tests__/cloud-tenant-transient-db-retry.test.ts
src/__tests__/error-handler.test.ts` in `server/` — 25 tests, all pass.
- The abort test asserts no `captureException` and no telemetry crash
track.
- The refusal tests pin all three classifications: never-migrated,
pending-with-history, wiped journal.

## Risks

- Low risk. The abort path only triggers on the exact `aborted` +
`ECONNRESET` pair; every other error keeps reporting.
- The refusal reclassification suppresses a capture only under a cloud
supervisor. If an operator breaks a migration runner, the supervisor's
own deploy gates surface it; self-hosted deployments still report.
- The retry widening adds at most ~150 ms before a genuine connection
fault surfaces.

## Model Used

Claude (Anthropic) — claude-fable-5 (Claude Fable 5), Claude Code
harness, extended thinking with tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-12 11:15:43 -07:00

79 lines
2.9 KiB
TypeScript

import { describe, expect, it } from "vitest";
import {
StartupRefusalError,
migrationRefusalError,
shouldReportStartupFailure,
} from "../startup-refusals.ts";
describe("migrationRefusalError", () => {
const message = "PostgreSQL has pending migrations (…). Refusing to start.";
it("classifies a never-migrated database as a supervised-transient refusal", () => {
const error = migrationRefusalError({ appliedMigrations: [], tableCount: 0 }, message);
expect(error).toBeInstanceOf(StartupRefusalError);
expect((error as StartupRefusalError).kind).toBe("schema-not-yet-migrated");
expect(error.message).toContain("Refusing to start");
});
it("classifies pending migrations on a migrated database as a supervised-transient refusal", () => {
// Managed fleet rolls deliver the new app image before the migration
// runner, so a briefly-behind schema is the routine mid-upgrade phase
// under a supervisor — suppressed there, still reported self-hosted.
const error = migrationRefusalError(
{ appliedMigrations: ["0000_init.sql"], tableCount: 41 },
message,
);
expect(error).toBeInstanceOf(StartupRefusalError);
expect((error as StartupRefusalError).kind).toBe("schema-migration-pending");
});
it("treats an empty journal beside existing tables as drift, not a fresh database", () => {
// A wiped or never-populated migration journal next to real tables is
// a persistent failure; it must keep reporting.
const error = migrationRefusalError({ appliedMigrations: [], tableCount: 17 }, message);
expect(error).toBeInstanceOf(Error);
expect(error).not.toBeInstanceOf(StartupRefusalError);
});
});
describe("shouldReportStartupFailure", () => {
const refusal = new StartupRefusalError(
"database-contract-unmet",
"authenticated public deployments require DATABASE_URL",
);
it("always reports non-refusal startup failures, managed cloud or not", () => {
expect(shouldReportStartupFailure(new Error("boom"), {})).toBe(true);
expect(
shouldReportStartupFailure(new Error("boom"), {
PAPERCLIP_CLOUD_API_ORIGIN: "https://cloud.example.com",
}),
).toBe(true);
});
it("reports supervised-transient refusals outside managed-cloud deployments", () => {
expect(shouldReportStartupFailure(refusal, {})).toBe(true);
});
it("suppresses supervised-transient refusals when a cloud supervisor owns the deployment", () => {
expect(
shouldReportStartupFailure(refusal, {
PAPERCLIP_CLOUD_API_ORIGIN: "https://cloud.example.com",
}),
).toBe(false);
});
it("treats a blank cloud origin as unset", () => {
expect(shouldReportStartupFailure(refusal, { PAPERCLIP_CLOUD_API_ORIGIN: " " })).toBe(true);
});
it("reports non-Error throwables unconditionally", () => {
expect(
shouldReportStartupFailure("string failure", {
PAPERCLIP_CLOUD_API_ORIGIN: "https://cloud.example.com",
}),
).toBe(true);
});
});