mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
perf(server): cache the instance settings row per process (#21768)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - One server process runs every agent on one Node main thread, and
each database statement costs main-thread time
> - The `default` instance settings row is read on hot paths: live run
event delivery, run log redaction, the stranded-task recovery check,
runner tool calls, adapter routes and run preparation
> - A staging measurement with 7 live runs showed the full-row settings
read at 10.24 statements per second, and 3.5 to 5.4 per second when idle
> - The row changes only when an admin or a boot step writes it, so
almost every read returns the same value
> - This pull request adds a per-process read-through cache for that
row, with in-process invalidation on every write and a 2 second age
bound for changes made by another process
> - The benefit is at most one settings read per 2 seconds per server
process, whatever the load
## Linked Issues or Issue Description
**Is your feature request related to a problem?**
The full `instance_settings` row read (`select id, singleton_key,
default_environment_id, general, experimental ...`) runs at 10.24
statements per second on staging with 7 live runs, and 3.5 to 5.4 per
second at idle. It scales with run events and recovery checks, not with
settings changes.
**Describe the solution you would like**
Cache the row per process. Drop the cached row when a settings write
commits, in this process and, through the existing work-signal
notification, in the other process during a deploy overlap. Bound the
age of a cached row to 2 seconds, so any other change (a direct database
edit, a second process outside an overlap) takes effect within 2
seconds.
**Describe alternatives you have considered**
A database trigger with LISTEN/NOTIFY would also catch direct edits, but
it needs a migration, and work notifications cross processes only during
a deploy overlap today. The existing signal plus a 2 second bound keeps
the change small.
## What Changed
- New `server/src/services/instance-settings-cache.ts`: a read-through
cache keyed by the database client. Concurrent misses share one read. A
cached row is at most `INSTANCE_SETTINGS_CACHE_TTL_MS` (2000 ms) old,
measured from the start of the read. Every caller gets its own copy.
Reads through a transaction handle skip the cache.
- `instanceSettingsService` `get`, `getGeneral` and `getExperimental`
use the cache. Write paths (`update`, `updateGeneral`,
`updateExperimental`, and the feedback vote save) still read the row
fresh.
- Every settings write publishes an `instance-settings-changed` work
signal through the existing work-signal channel
(`@paperclipai/db/work-signals`). Inside a transaction it is the first
signal of the write, so its `pg_notify` payload goes out before the
other wakes of the same commit (for example `status-card-evaluation`).
This process drops the cached row when the write commits. During a
deploy overlap, when work notifications cross processes, the other
process drops its cached row on receipt, before a worker woken by the
same write reads the settings. A reconnect of the notification listener
also drops the row.
- `fillUnsetInstanceDefault` writes through `update({ db: tx })`, so its
signal is sent when its transaction commits.
- A read that starts before an invalidation never stores its result, so
an old value can not come back after a write.
- The test setup clears the cache before each test, because many tests
write the row directly.
Hot callers of the cached row:
- Live run event delivery: `live-run-event-redaction.ts`
(`censorUsernameInLogs`), once per delivery batch.
- Run output and run lifecycle redaction: `heartbeat.ts`
`getCurrentUserRedactionOptions`, used by `run-output.ts`,
`run-lifecycle.ts`, `run-completion.ts`.
- The stranded-task recovery check: `recovery/service.ts` reads
`getExperimental().enableAgentChat` once per checked conversation issue.
- Runner tool calls: `paperclip-runner-tool-authority.ts`
(`enableIsolatedWorkspaces`, `enableExternalObjects`).
- Adapter routes: `app.ts` (`enableNativeRunner`, `enableOpenAiDot`) per
request.
- Run preparation: `heartbeat.ts` (`instance_settings` preparation
step), `queued-run-line.ts`, `queued-run-dispatch-liveness.ts`,
`run-control.ts`.
- Issue and comment paths in `issues.ts` (`censorUsernameInLogs`,
`enableAgentChat`).
## Verification
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/instance-settings-cache.test.ts` (embedded PostgreSQL):
- 150 hot reads (`getGeneral`, `getExperimental`, `get` from new service
objects) after one warm read: 0 statements. Without the cache the same
test measures 150 statements (red run).
- 20 concurrent misses: 1 statement.
- `updateExperimental` and `updateGeneral` are visible at once to a new
read in the same process.
- `fillUnsetInstanceDefault` (transaction write of the default
environment) is visible at once after it returns.
- A change written by a second client (another process) becomes visible
within 2000 ms plus a 500 ms test margin.
- A caller that changes the returned object does not change the cached
row.
- A read through a transaction handle sees the transaction's uncommitted
write, two such reads issue 2 statements, and the cache does not keep
the write after rollback.
- A read that started before an invalidation is not stored.
- A received `instance-settings-changed` notification from another
process drops the cached row (next read issues 1 statement).
- `updateExperimental({ enableStatusCards: true })` sends
`instance-settings-changed` as its first notification, before
`status-card-evaluation`; `updateGeneral` sends it alone.
- The server tests that use instance settings or write the row directly
(230 files): see the run notes in the first comment.
## Risks
- A direct database edit, or a change from a second process outside a
deploy overlap, takes up to 2 seconds to apply in this process. Before
this change it applied at the next read. This includes access flags such
as `enableOwnerInstanceAdmin` (read per cloud-tenant request) and the
default environment.
- A change made through the settings service applies at once in this
process, and during a deploy overlap at once in the other process.
- A new code path that writes the `default` row must call
`announceInstanceSettingsChange`, or its change applies within 2
seconds.
- No schema change.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
user-facing setting or command changed)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3706f520fd
commit
8f1d39da79
13 files changed
+278
-8
No files matched your search
@@ -15,6 +15,7 @@ import { toolAccessService } from "../services/tool-access.js";
|
||||
import { createToolGatewayService } from "../services/tool-gateway.js";
|
||||
import { connectionIntentService } from "../services/connection-intents.js";
|
||||
import { aiConnectionService } from "../services/ai-connections.js";
|
||||
import { invalidateInstanceSettingsCache } from "../services/instance-settings-cache.js";
|
||||
|
||||
vi.mock("../services/subscription-refresh.js", () => ({ refreshSubscriptionConnection: vi.fn().mockResolvedValue(undefined) }));
|
||||
|
||||
@@ -170,6 +171,7 @@ describeDatabase("connection assignment synchronization", () => {
|
||||
await pluginAgentRuntimeState(db, pluginId, companyId, agentId);
|
||||
const before = (await events()).length;
|
||||
await db.update(instanceSettings).set({ experimental: { enableChatConnectors: true } });
|
||||
invalidateInstanceSettingsCache();
|
||||
expect(await events()).toHaveLength(before + 2);
|
||||
expect((await pluginAgentRuntimeState(db, pluginId, companyId, agentId))!.connectionAssignments.features).toEqual({ publicMcp: false, chatConnectors: true });
|
||||
});
|
||||
|
||||
@@ -9,6 +9,7 @@ import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi }
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { pluginAgentRuntimeState } from "../services/plugin-agent-runtime-state.js";
|
||||
import { instanceSettingsService } from "../services/instance-settings.js";
|
||||
import { invalidateInstanceSettingsCache } from "../services/instance-settings-cache.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
const describeDatabase = support.supported ? describe : describe.skip;
|
||||
@@ -143,6 +144,7 @@ describeDatabase("skill runtime invalidation", () => {
|
||||
it("resolves immutable pinned and latest skill sources and removes unassigned skills", async () => {
|
||||
await instanceSettingsService(db).getExperimental();
|
||||
await db.update(instanceSettings).set({ experimental: { enableBetaSkills: true } });
|
||||
invalidateInstanceSettingsCache();
|
||||
const first = await publish("first version", 1);
|
||||
const pinned = await agent([{ key, versionId: first.id }]);
|
||||
const latest = await agent();
|
||||
|
||||
@@ -49,6 +49,7 @@ import * as copilotProbe from "../services/copilot-connection-probe.js";
|
||||
import * as settingsModule from "../services/instance-settings.js";
|
||||
import { agentRoutes } from "../routes/agents.js";
|
||||
import type { PluginWorkerManager } from "../services/plugin-worker-manager.js";
|
||||
import { invalidateInstanceSettingsCache } from "../services/instance-settings-cache.js";
|
||||
vi.mock("../services/local-ai-browser-login.js", () => ({
|
||||
startLocalBrowserLogin: () => ({ authorizationUrl: "https://auth.openai.com/codex/device", code: "ABCD-EFGHJ", abort: () => {} }),
|
||||
}));
|
||||
@@ -2519,6 +2520,7 @@ console.log(JSON.stringify({ type: "result", subtype: "success", result: "hello"
|
||||
await db.insert(toolConnections).values(ids.map(id => ({ ...connection, id, uid: id, name: `Cost ${id}` })));
|
||||
await db.insert(connectionGrants).values(ids.map(connectionId => ({ ...grant, id: randomUUID(), connectionId })));
|
||||
selects.mockClear();
|
||||
invalidateInstanceSettingsCache();
|
||||
const large = await request(app).get(url);
|
||||
largeQueries = selects.mock.calls.length;
|
||||
expect(large.status).toBe(200);
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
|
||||
import { createDb, environments, companies, instanceSettings, type Db } from "@paperclipai/db";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { createStatementCensus } from "./helpers/statement-census.js";
|
||||
import { instanceSettingsService } from "../services/instance-settings.js";
|
||||
import { INSTANCE_SETTINGS_CACHE_TTL_MS, INSTANCE_SETTINGS_CHANGED_TOPIC, invalidateInstanceSettingsCache, readThroughInstanceSettings } from "../services/instance-settings-cache.js";
|
||||
import { DATABASE_WORK_CHANNEL, enableDatabaseWorkNotifications, receiveDatabaseWorkNotification } from "@paperclipai/db/work-signals";
|
||||
import { fillUnsetInstanceDefault } from "../services/managed-instance-default.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
|
||||
async function waitUntil(check: () => Promise<boolean>, timeoutMs: number) {
|
||||
const started = Date.now();
|
||||
while (Date.now() - started < timeoutMs) {
|
||||
if (await check()) return Date.now() - started;
|
||||
await new Promise(resolve => setTimeout(resolve, 50));
|
||||
}
|
||||
throw new Error(`condition not met within ${timeoutMs} ms`);
|
||||
}
|
||||
|
||||
(support.supported ? describe : describe.skip)("instance settings read-through cache", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: Db;
|
||||
let otherProcess: Db;
|
||||
|
||||
beforeAll(async () => {
|
||||
database = await startEmbeddedPostgresTestDatabase("instance-settings-cache-");
|
||||
db = createDb(database.connectionString);
|
||||
otherProcess = createDb(database.connectionString);
|
||||
await db.insert(companies).values({ name: "Settings cache", issuePrefix: "SCC" });
|
||||
}, 90_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await Promise.all([db?.$client.end({ timeout: 0 }), otherProcess?.$client.end({ timeout: 0 })]);
|
||||
await database?.cleanup();
|
||||
}, 60_000);
|
||||
|
||||
beforeEach(async () => {
|
||||
await db.delete(instanceSettings);
|
||||
invalidateInstanceSettingsCache();
|
||||
});
|
||||
|
||||
it("serves repeated hot reads from one statement", async () => {
|
||||
const census = createStatementCensus(db);
|
||||
try {
|
||||
const settings = instanceSettingsService(db);
|
||||
await settings.get();
|
||||
const measured = await census.measure(async () => {
|
||||
for (let index = 0; index < 50; index++) {
|
||||
await Promise.all([settings.getGeneral(), settings.getExperimental(), instanceSettingsService(db).get()]);
|
||||
}
|
||||
});
|
||||
expect(measured.statements).toBe(0);
|
||||
invalidateInstanceSettingsCache();
|
||||
const concurrent = await census.measure(() => Promise.all(Array.from({ length: 20 }, () => settings.getExperimental())));
|
||||
expect(concurrent.statements).toBe(1);
|
||||
} finally {
|
||||
census.uninstall();
|
||||
}
|
||||
});
|
||||
|
||||
it("reads its own writes in this process at once", async () => {
|
||||
const settings = instanceSettingsService(db);
|
||||
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
|
||||
await settings.updateExperimental({ enableIsolatedWorkspaces: true });
|
||||
expect((await instanceSettingsService(db).getExperimental()).enableIsolatedWorkspaces).toBe(true);
|
||||
expect((await settings.getGeneral()).censorUsernameInLogs).toBe(false);
|
||||
await settings.updateGeneral({ censorUsernameInLogs: true });
|
||||
expect((await settings.getGeneral()).censorUsernameInLogs).toBe(true);
|
||||
});
|
||||
|
||||
it("sees a default environment committed in a caller's transaction once it commits", async () => {
|
||||
const settings = instanceSettingsService(db);
|
||||
expect((await settings.get()).defaultEnvironmentId).toBeNull();
|
||||
const [environment] = await db.insert(environments).values({
|
||||
name: "Managed Sandbox", driver: "sandbox", status: "active", config: {},
|
||||
}).returning({ id: environments.id });
|
||||
expect((await fillUnsetInstanceDefault(db, environment!.id)).filled).toBe(true);
|
||||
expect((await settings.get()).defaultEnvironmentId).toBe(environment!.id);
|
||||
});
|
||||
|
||||
it("takes a change made by another process within the cache bound", async () => {
|
||||
const settings = instanceSettingsService(db);
|
||||
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
|
||||
const changedAt = Date.now();
|
||||
await otherProcess.update(instanceSettings)
|
||||
.set({ experimental: sql`coalesce(${instanceSettings.experimental}, '{}'::jsonb) || '{"enableIsolatedWorkspaces":true}'::jsonb` })
|
||||
.where(eq(instanceSettings.singletonKey, "default"));
|
||||
await waitUntil(async () => (await settings.getExperimental()).enableIsolatedWorkspaces === true,
|
||||
INSTANCE_SETTINGS_CACHE_TTL_MS + 1_000);
|
||||
expect(Date.now() - changedAt).toBeLessThanOrEqual(INSTANCE_SETTINGS_CACHE_TTL_MS + 500);
|
||||
});
|
||||
|
||||
it("never shares cached rows with callers that mutate them", async () => {
|
||||
const settings = instanceSettingsService(db);
|
||||
const first = await settings.get();
|
||||
(first.experimental as unknown as Record<string, unknown>).enableIsolatedWorkspaces = true;
|
||||
expect((await settings.get()).experimental.enableIsolatedWorkspaces).toBe(false);
|
||||
});
|
||||
|
||||
it("reads inside a transaction bypass the cache", async () => {
|
||||
const settings = instanceSettingsService(db);
|
||||
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
|
||||
await db.transaction(async tx => {
|
||||
await tx.update(instanceSettings).set({ experimental: { enableIsolatedWorkspaces: true } })
|
||||
.where(eq(instanceSettings.singletonKey, "default"));
|
||||
const inside = instanceSettingsService(tx as unknown as Db);
|
||||
expect((await inside.getExperimental()).enableIsolatedWorkspaces).toBe(true);
|
||||
const census = createStatementCensus(db);
|
||||
try {
|
||||
const reads = await census.measure(async () => { await inside.getExperimental(); await inside.getGeneral(); });
|
||||
expect(reads.statements).toBe(2);
|
||||
} finally {
|
||||
census.uninstall();
|
||||
}
|
||||
throw new Error("rollback");
|
||||
}).catch(() => undefined);
|
||||
expect((await settings.getExperimental()).enableIsolatedWorkspaces).toBe(false);
|
||||
});
|
||||
|
||||
it("never stores a read that started before an invalidation", async () => {
|
||||
const owner = {};
|
||||
let resolveFirst!: (value: string) => void;
|
||||
const first = readThroughInstanceSettings(owner, () => new Promise<string>(resolve => { resolveFirst = resolve; }));
|
||||
invalidateInstanceSettingsCache();
|
||||
resolveFirst("before the write");
|
||||
expect(await first).toBe("before the write");
|
||||
let loads = 0;
|
||||
expect(await readThroughInstanceSettings(owner, async () => { loads += 1; return "after the write"; })).toBe("after the write");
|
||||
expect(await readThroughInstanceSettings(owner, async () => { loads += 1; return "unexpected"; })).toBe("after the write");
|
||||
expect(loads).toBe(1);
|
||||
});
|
||||
|
||||
it("drops the cached row when another process announces a settings change", async () => {
|
||||
const settings = instanceSettingsService(db);
|
||||
await settings.getExperimental();
|
||||
const census = createStatementCensus(db);
|
||||
try {
|
||||
const cached = await census.measure(() => settings.getExperimental());
|
||||
expect(cached.statements).toBe(0);
|
||||
receiveDatabaseWorkNotification(db, JSON.stringify({ version: 1, origin: "another-process", topic: INSTANCE_SETTINGS_CHANGED_TOPIC }));
|
||||
const refreshed = await census.measure(() => settings.getExperimental());
|
||||
expect(refreshed.statements).toBe(1);
|
||||
} finally {
|
||||
census.uninstall();
|
||||
}
|
||||
});
|
||||
|
||||
it("announces a settings change before the other wakes of the same write", async () => {
|
||||
const payloads: string[] = [];
|
||||
const listener = await otherProcess.$client.listen(DATABASE_WORK_CHANNEL, payload => { payloads.push(payload); });
|
||||
enableDatabaseWorkNotifications(db);
|
||||
try {
|
||||
await instanceSettingsService(db).updateExperimental({ enableStatusCards: true });
|
||||
await waitUntil(async () => payloads.length > 1, 5_000);
|
||||
const topics = payloads.map(payload => (JSON.parse(payload) as { topic: string }).topic);
|
||||
expect(topics[0]).toBe(INSTANCE_SETTINGS_CHANGED_TOPIC);
|
||||
expect(topics).toContain("status-card-evaluation");
|
||||
payloads.length = 0;
|
||||
await instanceSettingsService(db).updateGeneral({ censorUsernameInLogs: true });
|
||||
await waitUntil(async () => payloads.length > 0, 5_000);
|
||||
expect(payloads.map(payload => (JSON.parse(payload) as { topic: string }).topic)).toEqual([INSTANCE_SETTINGS_CHANGED_TOPIC]);
|
||||
} finally {
|
||||
await listener.unlisten();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -15,6 +15,7 @@ import { projectRoutes } from "../routes/projects.js";
|
||||
import { errorHandler } from "../middleware/index.js";
|
||||
import { createPluginWorkspaceFixture } from "./helpers/plugin-workspaces.js";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { invalidateInstanceSettingsCache } from "../services/instance-settings-cache.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
@@ -43,6 +44,7 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
{ name: "Build host", driver: "ssh", config: { host: "build.example.com" } },
|
||||
]).returning();
|
||||
await db.insert(instanceSettings).values({ defaultEnvironmentId: sandbox.id });
|
||||
invalidateInstanceSettingsCache();
|
||||
return { ...f, plugin, computer, local, sandbox, ssh };
|
||||
}
|
||||
|
||||
@@ -85,6 +87,7 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
await db.update(agents).set({ defaultEnvironmentId: null }).where(eq(agents.id, f.input.agentId));
|
||||
expect((await projectEnvironmentOptions(db, f.companyId, { userId: "person" })).defaultEnvironmentId).toBe(f.sandbox.id);
|
||||
await db.update(instanceSettings).set({ defaultEnvironmentId: null });
|
||||
invalidateInstanceSettingsCache();
|
||||
expect((await projectEnvironmentOptions(db, f.companyId)).defaultEnvironmentId).toBe(f.local.id);
|
||||
});
|
||||
|
||||
@@ -101,6 +104,7 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
expect(await withDefaultProjectStorage(db, f.companyId, explicit)).toEqual(explicit);
|
||||
await expect(withDefaultProjectStorage(db, f.companyId, explicit, { environmentId: f.computer.id })).rejects.toMatchObject({ status: 409 });
|
||||
await db.update(instanceSettings).set({ defaultEnvironmentId: f.computer.id });
|
||||
invalidateInstanceSettingsCache();
|
||||
expect((await svc.getById(sandboxProject.id))?.workspaceProvider?.pluginId).toBeNull();
|
||||
});
|
||||
|
||||
@@ -175,6 +179,7 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
it("keeps a cold worker's storage identity, and uses the only storage left after its default is archived", async () => {
|
||||
const f = await fixture();
|
||||
await db.update(instanceSettings).set({ defaultEnvironmentId: f.computer.id });
|
||||
invalidateInstanceSettingsCache();
|
||||
await db.update(plugins).set({ status: "error" }).where(eq(plugins.id, f.pluginId));
|
||||
expect(await withDefaultProjectStorage(db, f.companyId, null)).toMatchObject({ storageProviderPluginId: f.pluginId });
|
||||
await db.update(environments).set({ status: "archived" }).where(eq(environments.id, f.computer.id));
|
||||
@@ -186,6 +191,7 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
const f = await fixture();
|
||||
const [fake] = await db.insert(environments).values({ name: "Fake", driver: "sandbox", config: { provider: "fake" } }).returning();
|
||||
await db.update(instanceSettings).set({ defaultEnvironmentId: fake.id });
|
||||
invalidateInstanceSettingsCache();
|
||||
const options = await projectEnvironmentOptions(db, f.companyId);
|
||||
expect(options.groups).toHaveLength(2);
|
||||
expect(options.defaultEnvironmentId).toBeNull();
|
||||
@@ -197,6 +203,7 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
const f = await fixture();
|
||||
await db.update(environments).set({ status: "archived" }).where(inArray(environments.id, [f.sandbox.id, f.ssh.id]));
|
||||
await db.update(instanceSettings).set({ experimental: { enableManagedSandboxOnly: true } });
|
||||
invalidateInstanceSettingsCache();
|
||||
const options = await projectEnvironmentOptions(db, f.companyId);
|
||||
expect(options).toEqual({
|
||||
groups: [{ storageProviderPluginId: f.pluginId, environments: [{ id: f.computer.id, name: f.computer.name }] }],
|
||||
@@ -214,8 +221,10 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
await db.insert(companyMemberships).values({ companyId: f.companyId, principalType: "user", principalId: userId,
|
||||
status: "active", membershipRole: "operator" });
|
||||
await db.update(environments).set({ status: "archived" }).where(eq(environments.id, f.computer.id));
|
||||
if (source === "instance") await db.update(instanceSettings).set({ defaultEnvironmentId: f.computer.id });
|
||||
else {
|
||||
if (source === "instance") {
|
||||
await db.update(instanceSettings).set({ defaultEnvironmentId: f.computer.id });
|
||||
invalidateInstanceSettingsCache();
|
||||
} else {
|
||||
await db.update(agents).set({ defaultEnvironmentId: f.computer.id }).where(eq(agents.id, f.input.agentId));
|
||||
await db.insert(userCompanyPreferences).values({ companyId: f.companyId, userId, primaryAgentId: f.input.agentId });
|
||||
}
|
||||
@@ -231,10 +240,12 @@ describe.skipIf(!support.supported)("project environment compatibility", () => {
|
||||
const f = await fixture();
|
||||
await db.update(environments).set({ metadata: { managedByPaperclip: true, managedKubernetesSandbox: true } }).where(eq(environments.id, f.sandbox.id));
|
||||
await db.update(instanceSettings).set({ defaultEnvironmentId: f.local.id, experimental: { enableManagedSandboxOnly: true } });
|
||||
invalidateInstanceSettingsCache();
|
||||
const managed = await projectEnvironmentOptions(db, f.companyId);
|
||||
expect(managed.defaultEnvironmentId).toBe(f.sandbox.id);
|
||||
expect(managed.groups.flatMap(group => group.environments).map(value => value.id)).not.toContain(f.local.id);
|
||||
await db.update(instanceSettings).set({ general: { executionMode: "kubernetes" } });
|
||||
invalidateInstanceSettingsCache();
|
||||
const forced = await projectEnvironmentOptions(db, f.companyId);
|
||||
expect(forced.groups).toEqual([{ storageProviderPluginId: null, environments: [{ id: f.sandbox.id, name: f.sandbox.name }] }]);
|
||||
await expect(withDefaultProjectStorage(db, f.companyId, null, { environmentId: f.computer.id })).rejects.toMatchObject({ status: 409 });
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import fs from "node:fs";
|
||||
import { createIsolatedTestHome } from "./helpers/isolated-test-home.js";
|
||||
import { beforeEach } from "vitest";
|
||||
import { createRequire } from "node:module";
|
||||
import type { AddressInfo, Server as NetServer } from "node:net";
|
||||
import os from "node:os";
|
||||
@@ -76,3 +77,8 @@ if (!SupertestTest.prototype.__paperclipLoopbackPatched) {
|
||||
|
||||
SupertestTest.prototype.__paperclipLoopbackPatched = true;
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
const { invalidateInstanceSettingsCache } = await import("../services/instance-settings-cache.js");
|
||||
invalidateInstanceSettingsCache();
|
||||
});
|
||||
@@ -28,6 +28,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } fro
|
||||
import { smokeLabRoutes } from "../routes/smoke-lab.js";
|
||||
import { SMOKE_LAB_OAUTH_SCOPE } from "../services/smoke-lab.js";
|
||||
import { errorHandler } from "../middleware/index.js";
|
||||
import { invalidateInstanceSettingsCache } from "../services/instance-settings-cache.js";
|
||||
|
||||
const externalTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_SMOKE_LAB_TEST_DATABASE_URL");
|
||||
const embeddedPostgresSupport = externalTestDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
@@ -50,6 +51,7 @@ async function enableSmokeLab(db: TestDb) {
|
||||
target: [instanceSettings.singletonKey],
|
||||
set: { experimental: { enableSmokeLab: true }, updatedAt: new Date() },
|
||||
});
|
||||
invalidateInstanceSettingsCache();
|
||||
}
|
||||
|
||||
async function createAgent(db: TestDb, companyId: string) {
|
||||
|
||||
@@ -279,7 +279,7 @@ describe("resolveCloudTenantActor (shared-pool hardening)", () => {
|
||||
expect(insertedTables.filter((table) => table === companyMemberships)).toHaveLength(3);
|
||||
});
|
||||
|
||||
it("coalesces concurrent identical syncs while keeping authority reads and stale-admin purges per request", async () => {
|
||||
it("coalesces concurrent identical syncs and settings reads while keeping membership reads and stale-admin purges per request", async () => {
|
||||
let release!: () => void;
|
||||
let entered!: () => void;
|
||||
const blocked = new Promise<void>(resolve => { release = resolve; });
|
||||
@@ -294,7 +294,7 @@ describe("resolveCloudTenantActor (shared-pool hardening)", () => {
|
||||
const actors = await Promise.all(requests);
|
||||
expect(insertedTables.filter(table => table === companyMemberships)).toHaveLength(1);
|
||||
expect(deletedTables.filter(table => table === instanceUserRoles)).toHaveLength(20);
|
||||
expect(selectWheres.filter(entry => entry.table === instanceSettings)).toHaveLength(20);
|
||||
expect(selectWheres.filter(entry => entry.table === instanceSettings)).toHaveLength(1);
|
||||
expect(selectWheres.filter(entry => entry.table === companyMemberships && !entry.locked)).toHaveLength(20);
|
||||
expect(selectWheres.filter(entry => entry.table === companyMemberships && entry.locked)).toHaveLength(1);
|
||||
expect(actors.every(actor => actor?.memberships?.[0]?.membershipRole === "owner")).toBe(true);
|
||||
|
||||
@@ -11,6 +11,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } fro
|
||||
import { cloudActorHeaderSourceFromHeaders, resolveCloudTenantActor } from "./auth.js";
|
||||
import { errorHandler } from "./error-handler.js";
|
||||
import { userProfileRoutes } from "../routes/user-profiles.js";
|
||||
import { invalidateInstanceSettingsCache } from "../services/instance-settings-cache.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
const aliasFields = ["name", "email-local", "email", "principal"] as const;
|
||||
@@ -172,11 +173,13 @@ const aliasFields = ["name", "email-local", "email", "principal"] as const;
|
||||
await db.update(companyMemberships).set({ status: "inactive" }).where(and(eq(companyMemberships.companyId, importedCompanyId), eq(companyMemberships.principalId, userId)));
|
||||
await db.insert(instanceSettings).values({ singletonKey: "default", experimental: { enableOwnerInstanceAdmin: true } })
|
||||
.onConflictDoUpdate({ target: instanceSettings.singletonKey, set: { experimental: { enableOwnerInstanceAdmin: true } } });
|
||||
invalidateInstanceSettingsCache();
|
||||
const elevated = await resolve(db, value);
|
||||
expect(elevated?.isInstanceAdmin).toBe(true);
|
||||
expect(elevated?.companyIds).toEqual(actor!.companyIds);
|
||||
expect(await db.select().from(instanceUserRoles).where(eq(instanceUserRoles.userId, userId))).toEqual([]);
|
||||
await db.update(instanceSettings).set({ experimental: { enableOwnerInstanceAdmin: false } });
|
||||
invalidateInstanceSettingsCache();
|
||||
expect((await resolve(db, value))?.isInstanceAdmin).toBe(false);
|
||||
});
|
||||
|
||||
|
||||
@@ -51,6 +51,7 @@ import { getRunLogStore, isRunLogStoreType } from "./run-log-store.js";
|
||||
import { isObjectJournalBusyError } from "./object-byte-journal.js";
|
||||
import { recoverSettledNativeRunLog } from "./native-runtime/native-run-log-recovery.js";
|
||||
import { getOperatorSettingDefaults } from "./setting-defaults.js";
|
||||
import { announceInstanceSettingsChange } from "./instance-settings-cache.js";
|
||||
|
||||
const FEEDBACK_SCHEMA_VERSION = "paperclip-feedback-envelope-v2";
|
||||
const FEEDBACK_BUNDLE_VERSION = "paperclip-feedback-bundle-v2";
|
||||
@@ -2027,6 +2028,7 @@ export function feedbackService(db: Db, options: FeedbackServiceOptions = {}) {
|
||||
getOperatorSettingDefaults(),
|
||||
);
|
||||
if (currentInstanceSettings && currentGeneral.feedbackDataSharingPreference === "prompt") {
|
||||
await announceInstanceSettingsChange(tx);
|
||||
const nextSharingPreference = sharedWithLabs ? "allowed" : "not_allowed";
|
||||
const currentGeneralRaw = asRecord(currentInstanceSettings.general) ?? {};
|
||||
await tx
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import { is } from "drizzle-orm";
|
||||
import { PgTransaction } from "drizzle-orm/pg-core";
|
||||
import { publishDatabaseWorkHint, publishDatabaseWorkSettled, subscribeDatabaseWork } from "@paperclipai/db/work-signals";
|
||||
|
||||
export const INSTANCE_SETTINGS_CACHE_TTL_MS = 2_000;
|
||||
export const INSTANCE_SETTINGS_CHANGED_TOPIC = "instance-settings-changed";
|
||||
|
||||
type Entry = { generation: number; at: number; value?: unknown; pending?: Promise<unknown> };
|
||||
|
||||
let generation = 0;
|
||||
const entries = new WeakMap<object, Entry>();
|
||||
const subscribed = new WeakSet<object>();
|
||||
|
||||
export function invalidateInstanceSettingsCache() {
|
||||
generation += 1;
|
||||
}
|
||||
|
||||
export async function announceInstanceSettingsChange(connection: object) {
|
||||
invalidateInstanceSettingsCache();
|
||||
publishDatabaseWorkSettled(connection, INSTANCE_SETTINGS_CHANGED_TOPIC);
|
||||
if (!is(connection, PgTransaction)) {
|
||||
await publishDatabaseWorkHint(connection, INSTANCE_SETTINGS_CHANGED_TOPIC).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
function subscribeToChanges(db: object) {
|
||||
if (subscribed.has(db)) return;
|
||||
subscribed.add(db);
|
||||
try {
|
||||
subscribeDatabaseWork(db, INSTANCE_SETTINGS_CHANGED_TOPIC, invalidateInstanceSettingsCache);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
export async function readThroughInstanceSettings<T>(db: object, load: () => Promise<T>): Promise<T> {
|
||||
if (is(db, PgTransaction)) return load();
|
||||
subscribeToChanges(db);
|
||||
const current = entries.get(db);
|
||||
if (current && current.generation === generation) {
|
||||
if ("value" in current && Date.now() - current.at < INSTANCE_SETTINGS_CACHE_TTL_MS) {
|
||||
return structuredClone(current.value as T);
|
||||
}
|
||||
if (current.pending) return structuredClone(await (current.pending as Promise<T>));
|
||||
}
|
||||
const startedAt = Date.now();
|
||||
const startedGeneration = generation;
|
||||
const entry: Entry = { generation: startedGeneration, at: startedAt };
|
||||
const pending = load();
|
||||
entry.pending = pending;
|
||||
entries.set(db, entry);
|
||||
try {
|
||||
const value = await pending;
|
||||
if (entries.get(db) === entry) {
|
||||
if (generation === startedGeneration) entries.set(db, { generation: startedGeneration, at: startedAt, value });
|
||||
else entries.delete(db);
|
||||
}
|
||||
return structuredClone(value);
|
||||
} catch (error) {
|
||||
if (entries.get(db) === entry) entries.delete(db);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -59,6 +59,7 @@ import { getManagedInstanceConfig, type ManagedInstanceConfig } from "./managed-
|
||||
import { getOperatorSettingDefaults } from "./setting-defaults.js";
|
||||
import { isCloudStagingExperimentalSetting, isCloudStagingInstance } from "./cloud-instance.js";
|
||||
import { forbidden } from "../errors.js";
|
||||
import { announceInstanceSettingsChange, readThroughInstanceSettings } from "./instance-settings-cache.js";
|
||||
|
||||
const DEFAULT_SINGLETON_KEY = "default";
|
||||
const instanceGeneralSettingsStorageSchema = instanceGeneralSettingsSchema.strip();
|
||||
@@ -552,8 +553,12 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService
|
||||
throw new Error("Failed to initialize instance settings row");
|
||||
}
|
||||
|
||||
function getCachedRow(runner: InstanceSettingsWriteDb = db) {
|
||||
return readThroughInstanceSettings(runner, () => getOrCreateRow(runner));
|
||||
}
|
||||
|
||||
return {
|
||||
get: async (): Promise<InstanceSettings> => toInstanceSettings(await getOrCreateRow()),
|
||||
get: async (): Promise<InstanceSettings> => toInstanceSettings(await getCachedRow()),
|
||||
|
||||
update: async (
|
||||
patch: PatchInstanceSettings,
|
||||
@@ -576,13 +581,14 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService
|
||||
})
|
||||
.where(eq(instanceSettings.id, current.id))
|
||||
.returning();
|
||||
await announceInstanceSettingsChange(runner);
|
||||
return toInstanceSettings(updated ?? current);
|
||||
},
|
||||
|
||||
getGeneral: async (
|
||||
readOptions?: { db?: InstanceSettingsWriteDb },
|
||||
): Promise<InstanceGeneralSettings> => {
|
||||
const row = await getOrCreateRow(readOptions?.db);
|
||||
const row = await getCachedRow(readOptions?.db);
|
||||
return toGeneralView(row.general);
|
||||
},
|
||||
|
||||
@@ -593,7 +599,7 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService
|
||||
},
|
||||
|
||||
getExperimental: async (): Promise<InstanceExperimentalSettingsWithManaged> => {
|
||||
const row = await getOrCreateRow();
|
||||
const row = await getCachedRow();
|
||||
return toExperimentalView(row.experimental);
|
||||
},
|
||||
|
||||
@@ -617,6 +623,7 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService
|
||||
})
|
||||
.where(eq(instanceSettings.id, current.id))
|
||||
.returning();
|
||||
await announceInstanceSettingsChange(db);
|
||||
return toInstanceSettings(updated ?? current);
|
||||
},
|
||||
|
||||
@@ -651,6 +658,7 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService
|
||||
&& toExperimentalView(nextExperimental).enableAgentChat;
|
||||
const now = new Date();
|
||||
const [updated] = await db.transaction(async tx => {
|
||||
await announceInstanceSettingsChange(tx);
|
||||
if (patch.enableStatusCards !== undefined) await notifyStatusCardEvaluation(tx);
|
||||
if (chatBecameEnabled) await notifyDeferredExecutionEligibility(tx);
|
||||
if(toExperimentalView(current.experimental).enablePublicMcp!==toExperimentalView(nextExperimental).enablePublicMcp)await prepareMcpClientMirrorPolicy(tx as unknown as Db,toExperimentalView(nextExperimental).enablePublicMcp);
|
||||
|
||||
@@ -18,6 +18,7 @@ import { createObjectRunLogStore } from "../object-run-log-store.js";
|
||||
import { nativeSha256 } from "./canonical.js";
|
||||
import { createNativeRunLogCoverageVerifier, formatNativeRunLogEvent } from "./native-run-log-export.js";
|
||||
import { readRecoverableRunLog, recoverSettledNativeRunLog } from "./native-run-log-recovery.js";
|
||||
import { invalidateInstanceSettingsCache } from "../instance-settings-cache.js";
|
||||
|
||||
// Use an explicit guarded test database or a new owned embedded fixture.
|
||||
const externalUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_NATIVE_LOG_WATERMARK_DATABASE_URL");
|
||||
@@ -348,7 +349,7 @@ describe.skipIf(!support.supported)("native run log source watermarks", () => {
|
||||
const f = await fixture();
|
||||
const setMasking = async (value: boolean) => db.execute(sql`insert into instance_settings(general)
|
||||
values(${JSON.stringify({ censorUsernameInLogs: value })}::jsonb)
|
||||
on conflict(singleton_key) do update set general=excluded.general`);
|
||||
on conflict(singleton_key) do update set general=excluded.general`).finally(invalidateInstanceSettingsCache);
|
||||
const redaction = async () => ({ enabled: (await instanceSettingsService(db).getGeneral()).censorUsernameInLogs,
|
||||
userNames: ["fixture-operator"], homeDirs: ["/home/fixture-operator"] });
|
||||
await setMasking(enabled);
|
||||
|
||||
Reference in new issue
Block a user