Files
PaperClipAI/server/src/__tests__/instance-settings-cache.test.ts
T
DottaandClaude Opus 5.5 8f1d39da79 perf(server): cache the instance settings row per process (#21768)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - One server process runs every agent on one Node main thread, and
each database statement costs main-thread time
> - The `default` instance settings row is read on hot paths: live run
event delivery, run log redaction, the stranded-task recovery check,
runner tool calls, adapter routes and run preparation
> - A staging measurement with 7 live runs showed the full-row settings
read at 10.24 statements per second, and 3.5 to 5.4 per second when idle
> - The row changes only when an admin or a boot step writes it, so
almost every read returns the same value
> - This pull request adds a per-process read-through cache for that
row, with in-process invalidation on every write and a 2 second age
bound for changes made by another process
> - The benefit is at most one settings read per 2 seconds per server
process, whatever the load

## Linked Issues or Issue Description

**Is your feature request related to a problem?**
The full `instance_settings` row read (`select id, singleton_key,
default_environment_id, general, experimental ...`) runs at 10.24
statements per second on staging with 7 live runs, and 3.5 to 5.4 per
second at idle. It scales with run events and recovery checks, not with
settings changes.

**Describe the solution you would like**
Cache the row per process. Drop the cached row when a settings write
commits, in this process and, through the existing work-signal
notification, in the other process during a deploy overlap. Bound the
age of a cached row to 2 seconds, so any other change (a direct database
edit, a second process outside an overlap) takes effect within 2
seconds.

**Describe alternatives you have considered**
A database trigger with LISTEN/NOTIFY would also catch direct edits, but
it needs a migration, and work notifications cross processes only during
a deploy overlap today. The existing signal plus a 2 second bound keeps
the change small.

## What Changed

- New `server/src/services/instance-settings-cache.ts`: a read-through
cache keyed by the database client. Concurrent misses share one read. A
cached row is at most `INSTANCE_SETTINGS_CACHE_TTL_MS` (2000 ms) old,
measured from the start of the read. Every caller gets its own copy.
Reads through a transaction handle skip the cache.
- `instanceSettingsService` `get`, `getGeneral` and `getExperimental`
use the cache. Write paths (`update`, `updateGeneral`,
`updateExperimental`, and the feedback vote save) still read the row
fresh.
- Every settings write publishes an `instance-settings-changed` work
signal through the existing work-signal channel
(`@paperclipai/db/work-signals`). Inside a transaction it is the first
signal of the write, so its `pg_notify` payload goes out before the
other wakes of the same commit (for example `status-card-evaluation`).
This process drops the cached row when the write commits. During a
deploy overlap, when work notifications cross processes, the other
process drops its cached row on receipt, before a worker woken by the
same write reads the settings. A reconnect of the notification listener
also drops the row.
- `fillUnsetInstanceDefault` writes through `update({ db: tx })`, so its
signal is sent when its transaction commits.
- A read that starts before an invalidation never stores its result, so
an old value can not come back after a write.
- The test setup clears the cache before each test, because many tests
write the row directly.

Hot callers of the cached row:

- Live run event delivery: `live-run-event-redaction.ts`
(`censorUsernameInLogs`), once per delivery batch.
- Run output and run lifecycle redaction: `heartbeat.ts`
`getCurrentUserRedactionOptions`, used by `run-output.ts`,
`run-lifecycle.ts`, `run-completion.ts`.
- The stranded-task recovery check: `recovery/service.ts` reads
`getExperimental().enableAgentChat` once per checked conversation issue.
- Runner tool calls: `paperclip-runner-tool-authority.ts`
(`enableIsolatedWorkspaces`, `enableExternalObjects`).
- Adapter routes: `app.ts` (`enableNativeRunner`, `enableOpenAiDot`) per
request.
- Run preparation: `heartbeat.ts` (`instance_settings` preparation
step), `queued-run-line.ts`, `queued-run-dispatch-liveness.ts`,
`run-control.ts`.
- Issue and comment paths in `issues.ts` (`censorUsernameInLogs`,
`enableAgentChat`).

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/instance-settings-cache.test.ts` (embedded PostgreSQL):
- 150 hot reads (`getGeneral`, `getExperimental`, `get` from new service
objects) after one warm read: 0 statements. Without the cache the same
test measures 150 statements (red run).
  - 20 concurrent misses: 1 statement.
- `updateExperimental` and `updateGeneral` are visible at once to a new
read in the same process.
- `fillUnsetInstanceDefault` (transaction write of the default
environment) is visible at once after it returns.
- A change written by a second client (another process) becomes visible
within 2000 ms plus a 500 ms test margin.
- A caller that changes the returned object does not change the cached
row.
- A read through a transaction handle sees the transaction's uncommitted
write, two such reads issue 2 statements, and the cache does not keep
the write after rollback.
  - A read that started before an invalidation is not stored.
- A received `instance-settings-changed` notification from another
process drops the cached row (next read issues 1 statement).
- `updateExperimental({ enableStatusCards: true })` sends
`instance-settings-changed` as its first notification, before
`status-card-evaluation`; `updateGeneral` sends it alone.
- The server tests that use instance settings or write the row directly
(230 files): see the run notes in the first comment.

## Risks

- A direct database edit, or a change from a second process outside a
deploy overlap, takes up to 2 seconds to apply in this process. Before
this change it applied at the next read. This includes access flags such
as `enableOwnerInstanceAdmin` (read per cloud-tenant request) and the
default environment.
- A change made through the settings service applies at once in this
process, and during a deploy overlap at once in the other process.
- A new code path that writes the `default` row must call
`announceInstanceSettingsChange`, or its change applies within 2
seconds.
- No schema change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
user-facing setting or command changed)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 09:50:19 -06:00

169 lines
8.3 KiB
TypeScript

import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { createDb, environments, companies, instanceSettings, type Db } from "@paperclipai/db";
import { eq, sql } from "drizzle-orm";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { createStatementCensus } from "./helpers/statement-census.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { INSTANCE_SETTINGS_CACHE_TTL_MS, INSTANCE_SETTINGS_CHANGED_TOPIC, invalidateInstanceSettingsCache, readThroughInstanceSettings } from "../services/instance-settings-cache.js";
import { DATABASE_WORK_CHANNEL, enableDatabaseWorkNotifications, receiveDatabaseWorkNotification } from "@paperclipai/db/work-signals";
import { fillUnsetInstanceDefault } from "../services/managed-instance-default.js";
const support = await getEmbeddedPostgresTestSupport();
async function waitUntil(check: () => Promise<boolean>, timeoutMs: number) {
const started = Date.now();
while (Date.now() - started < timeoutMs) {
if (await check()) return Date.now() - started;
await new Promise(resolve => setTimeout(resolve, 50));
}
throw new Error(`condition not met within ${timeoutMs} ms`);
}
(support.supported ? describe : describe.skip)("instance settings read-through cache", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let otherProcess: Db;
beforeAll(async () => {
database = await startEmbeddedPostgresTestDatabase("instance-settings-cache-");
db = createDb(database.connectionString);
otherProcess = createDb(database.connectionString);
await db.insert(companies).values({ name: "Settings cache", issuePrefix: "SCC" });
}, 90_000);
afterAll(async () => {
await Promise.all([db?.$client.end({ timeout: 0 }), otherProcess?.$client.end({ timeout: 0 })]);
await database?.cleanup();
}, 60_000);
beforeEach(async () => {
await db.delete(instanceSettings);
invalidateInstanceSettingsCache();
});
it("serves repeated hot reads from one statement", async () => {
const census = createStatementCensus(db);
try {
const settings = instanceSettingsService(db);
await settings.get();
const measured = await census.measure(async () => {
for (let index = 0; index < 50; index++) {
await Promise.all([settings.getGeneral(), settings.getExperimental(), instanceSettingsService(db).get()]);
}
});
expect(measured.statements).toBe(0);
invalidateInstanceSettingsCache();
const concurrent = await census.measure(() => Promise.all(Array.from({ length: 20 }, () => settings.getExperimental())));
expect(concurrent.statements).toBe(1);
} finally {
census.uninstall();
}
});
it("reads its own writes in this process at once", async () => {
const settings = instanceSettingsService(db);
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
await settings.updateExperimental({ enableIsolatedWorkspaces: true });
expect((await instanceSettingsService(db).getExperimental()).enableIsolatedWorkspaces).toBe(true);
expect((await settings.getGeneral()).censorUsernameInLogs).toBe(false);
await settings.updateGeneral({ censorUsernameInLogs: true });
expect((await settings.getGeneral()).censorUsernameInLogs).toBe(true);
});
it("sees a default environment committed in a caller's transaction once it commits", async () => {
const settings = instanceSettingsService(db);
expect((await settings.get()).defaultEnvironmentId).toBeNull();
const [environment] = await db.insert(environments).values({
name: "Managed Sandbox", driver: "sandbox", status: "active", config: {},
}).returning({ id: environments.id });
expect((await fillUnsetInstanceDefault(db, environment!.id)).filled).toBe(true);
expect((await settings.get()).defaultEnvironmentId).toBe(environment!.id);
});
it("takes a change made by another process within the cache bound", async () => {
const settings = instanceSettingsService(db);
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
const changedAt = Date.now();
await otherProcess.update(instanceSettings)
.set({ experimental: sql`coalesce(${instanceSettings.experimental}, '{}'::jsonb) || '{"enableIsolatedWorkspaces":true}'::jsonb` })
.where(eq(instanceSettings.singletonKey, "default"));
await waitUntil(async () => (await settings.getExperimental()).enableIsolatedWorkspaces === true,
INSTANCE_SETTINGS_CACHE_TTL_MS + 1_000);
expect(Date.now() - changedAt).toBeLessThanOrEqual(INSTANCE_SETTINGS_CACHE_TTL_MS + 500);
});
it("never shares cached rows with callers that mutate them", async () => {
const settings = instanceSettingsService(db);
const first = await settings.get();
(first.experimental as unknown as Record<string, unknown>).enableIsolatedWorkspaces = true;
expect((await settings.get()).experimental.enableIsolatedWorkspaces).toBe(false);
});
it("reads inside a transaction bypass the cache", async () => {
const settings = instanceSettingsService(db);
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
await db.transaction(async tx => {
await tx.update(instanceSettings).set({ experimental: { enableIsolatedWorkspaces: true } })
.where(eq(instanceSettings.singletonKey, "default"));
const inside = instanceSettingsService(tx as unknown as Db);
expect((await inside.getExperimental()).enableIsolatedWorkspaces).toBe(true);
const census = createStatementCensus(db);
try {
const reads = await census.measure(async () => { await inside.getExperimental(); await inside.getGeneral(); });
expect(reads.statements).toBe(2);
} finally {
census.uninstall();
}
throw new Error("rollback");
}).catch(() => undefined);
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
});
it("never stores a read that started before an invalidation", async () => {
const owner = {};
let resolveFirst!: (value: string) => void;
const first = readThroughInstanceSettings(owner, () => new Promise<string>(resolve => { resolveFirst = resolve; }));
invalidateInstanceSettingsCache();
resolveFirst("before the write");
expect(await first).toBe("before the write");
let loads = 0;
expect(await readThroughInstanceSettings(owner, async () => { loads += 1; return "after the write"; })).toBe("after the write");
expect(await readThroughInstanceSettings(owner, async () => { loads += 1; return "unexpected"; })).toBe("after the write");
expect(loads).toBe(1);
});
it("drops the cached row when another process announces a settings change", async () => {
const settings = instanceSettingsService(db);
await settings.getExperimental();
const census = createStatementCensus(db);
try {
const cached = await census.measure(() => settings.getExperimental());
expect(cached.statements).toBe(0);
receiveDatabaseWorkNotification(db, JSON.stringify({ version: 1, origin: "another-process", topic: INSTANCE_SETTINGS_CHANGED_TOPIC }));
const refreshed = await census.measure(() => settings.getExperimental());
expect(refreshed.statements).toBe(1);
} finally {
census.uninstall();
}
});
it("announces a settings change before the other wakes of the same write", async () => {
const payloads: string[] = [];
const listener = await otherProcess.$client.listen(DATABASE_WORK_CHANNEL, payload => { payloads.push(payload); });
enableDatabaseWorkNotifications(db);
try {
await instanceSettingsService(db).updateExperimental({ enableStatusCards: true });
await waitUntil(async () => payloads.length > 1, 5_000);
const topics = payloads.map(payload => (JSON.parse(payload) as { topic: string }).topic);
expect(topics[0]).toBe(INSTANCE_SETTINGS_CHANGED_TOPIC);
expect(topics).toContain("status-card-evaluation");
payloads.length = 0;
await instanceSettingsService(db).updateGeneral({ censorUsernameInLogs: true });
await waitUntil(async () => payloads.length > 0, 5_000);
expect(payloads.map(payload => (JSON.parse(payload) as { topic: string }).topic)).toEqual([INSTANCE_SETTINGS_CHANGED_TOPIC]);
} finally {
await listener.unlisten();
}
});
});