fix(db): attribute bounded captures to execution owners (#18634)

## Thinking Path

> - Paperclip manages agent work through shared server and database
services.
> - High query volume can spend main-thread time on SQL construction and
result decoding.
> - The existing bounded response capture counts rows and bytes, but it
cannot name the operation that executed a query.
> - Bundled callers can be missing, and measuring every parsed JSON
value adds work to the diagnostic.
> - This pull request carries a semantic execution owner into each
measured query and bounds response sizing.
> - Operators can rank query volume by owner before choosing a
performance fix.

## Linked Issues or Issue Description

Refs #17504, #17522, #17537, #17564.

**What existing behavior does this improve?**

The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic.

**Subsystem affected**

Database diagnostics and existing server execution boundaries.

**Current behavior**

The capture groups by table, operation, and builder caller. It cannot
distinguish execution owners that reuse a prepared query. Bundled
deployments need source maps for caller locations. The capture visits
every returned value and serializes parsed JSON again to estimate its
size.

**Proposed behavior**

Version 2 captures use fixed semantic owners for HTTP requests, delivery
queues, execution phases, and budget work. They count completed calls,
returned rows, errors, and completion wall time. They sample successful
results 1, 17, 33, and so on, with fixed cell and string/byte limits.
Complex values remain explicitly unmeasured. Semantic ownership works
without source maps.

**Reason and benefit**

A short census can identify which operations issue the most queries or
return the most rows. It preserves each query's execution owner through
pooled connections, concurrent operations, and prepared-query reuse. It
reduces diagnostic work without claiming per-query CPU attribution.

**Breaking changes**

Opt-in capture output moves from `entry.v1`/`summary.v1` to
`entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace
all-result byte estimates. Consumers must use the new fields and must
not extrapolate them into a total. There is no database migration or
application API change.

## What Changed

- Added `withDatabaseOperation(owner, action)` to the DB package. The
server owns semantic labels. Queries retain execution ownership in their
completion closure.
- Added fixed owners at existing HTTP, delivery, budget, and execution
timing boundaries. Delivery work replaces the context of the producer
that woke it.
- Added complete call/row/error counters and completion wall durations
within the existing bounded window. Durations include waits and are not
CPU measurements.
- Sampled and bounded primitive-value sizing. Complex and oversized
values remain unmeasured. Known owners skip repeated stack extraction.
- Preserved the deadline, group limit, paced output, query outcomes, and
existing coverage exclusions. Updated the operator documentation.
- Added real PostgreSQL owner-isolation and prepared-query tests, server
scope tests, and plain bundled-runtime proof.
- Moved projection-test and benchmark full-response sizing into an
explicit test helper. Existing payload-size thresholds remain unchanged;
these synthetic checks do not use the production sampler.

## Verification

- Final-commit DB capture suites: 43 tests passed in 13.86 seconds
against the actual PostgreSQL driver, including concurrent owners,
prepared reuse, transaction failure, a dedicated client, sampling, and a
query crossing the deadline. DB typecheck passed.
- Server attribution and existing delivery/execution/budget suites: 53
tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5
tests passed.
- Runtime bundle suite: 8 existing tests passed. The new no-source-map
owner test passed after its fixture was corrected to use a driver
thenable. An ordinary native Promise can bypass an overridden `.then`;
the real PostgreSQL tests separately cover driver integration.
- Observer-only experiment: three alternating pairs per case on Node
24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an
already-materialized synthetic JSON result, median capture CPU fell from
82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell
from 8.10 to 4.40 microseconds/call. Every result retained identity and
all calls were counted. The new diagnostic intentionally measures less
byte data; these are observer costs, not database latency or a hosted
application speedup. Disabled-case timings were below one
microsecond/call and too small for a performance conclusion.
- Final lifecycle regression: 28 unit tests and DB typecheck passed.
Closing the last capture disables and replaces owner storage;
overlapping captures retain it.
- Final-commit full recursive typecheck passed in 38.38 seconds and full
build passed in 40.81 seconds. The build stamp matches
`83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution
tests also passed 5/5 before the test-helper-only migration. Full test
acceptance will use the repository's parallel CI gates; the unsharded
local command takes hours.
- Verification failures retained: initial sandbox runs could not open
local IPC/PostgreSQL listeners; permitted reruns exercised the real
services. One old test expected JSON serialization and was updated to
assert explicit unmeasured complex values. The first bundle fixture used
the wrong promise boundary, as described above. A retained-callback
regression also demonstrated that disabling and reusing the same
AsyncLocalStorage instance could resurrect an old owner; replacing the
instance after the last capture closes fixes that case.
- The first CI run found older projection tests consuming the previous
full-response fields. Those consumers now use the full-response test
helper, preserving their byte thresholds. The two complete affected
route suites plus helper tests passed 115/115 in 73.08 seconds. A
temporary full-row query negative control then failed the unchanged
2,048-byte limit at 5,000,812 bytes and was restored; five other service
suites passed 34/34 and the affected chat cases passed 3/3. All three
previously failing CI shards passed on the corrected head. A local
skill-list benchmark smoke check completed 255 captured queries with no
in-flight queries or overflow; this was integration verification under
concurrent test load, not a timing benchmark.
- Two final-head runner jobs lost communication with GitHub before
reporting a test result. One targeted retry of those jobs and their
dependent aggregate passed. All 53 final-head checks passed, with 3
intentional skips; previously passing suites were preserved.
- Hosted capture remains pending until the reviewed revision is deployed
and a bounded window is explicitly configured.

Reproduce the focused checks with Node 24 and the repository pnpm
version:

```sh
pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts
pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs
pnpm -r typecheck
pnpm build
```

## Risks

- Owner context is active while a valid future capture window is armed,
up to 30 minutes before accounting starts. No-window and expired calls
take the direct path. Diagnostics still add bounded work while enabled.
- A nearest nested owner is reported. Broad `http.request` and
unattributed groups remain visible; this does not identify every service
callsite or SQL shape.
- The existing census excludes tagged/raw explicit-execute/streaming
paths and internal protocol commands. Its counts are not PostgreSQL
transaction totals.
- Completion wall time includes queue/database/network/scheduling waits.
Concurrent durations overlap. Sampled bytes exclude complex values,
large strings, and values beyond the sizing limits.
- Activation is startup-only. Operators must choose the window after
artifact preparation, allow rollout/warm-up time, and remove the expired
setting afterward.

## Model Used

OpenAI Codex, GPT-6 family, with code execution, repository inspection,
and parallel agents. The exact deployment model ID and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-11-02 12:12:21 -06:00
1 parent 6aabcff7d1
commit a92f616cd0
43 files changed
+844 -228

No files matched your search

+21 -12
View File
@@ -1,18 +1,27 @@
import { trace, type Span } from "@opentelemetry/api";
import { withDatabaseOperation } from "@paperclipai/db";
type BudgetOperation = "budget.accounting.lock_wait" | "budget.accounting.recover"
| "budget.accounting.settlement" | "budget.accounting.transaction"
| "budget.admission.evaluate" | "budget.admission.reserve"
| "budget.cancellation.deliver" | "budget.enforcement.deliver"
| "budget.enforcement.scope" | "budget.exposure.evaluate";
/** Operator-gated OpenTelemetry only; these spans contain no prompt, receipt,
* credential, or provider error text. Diagnostics never change accounting. */
export async function withBudgetSpan<T>(name: string, work: () => Promise<T>, attributes?: Record<string, number | boolean>) {
if (!process.env.OTEL_EXPORTER_OTLP_ENDPOINT?.trim()) return work();
let operation: Promise<T> | undefined;
const run = (span?: Span) => operation ??= Promise.resolve().then(work).finally(() => {
try { span?.end(); } catch { /* Diagnostics cannot change accounting. */ }
export function withBudgetSpan<T>(name: BudgetOperation, work: () => Promise<T>, attributes?: Record<string, number | boolean>) {
return withDatabaseOperation(name, async () => {
if (!process.env.OTEL_EXPORTER_OTLP_ENDPOINT?.trim()) return work();
let operation: Promise<T> | undefined;
const run = (span?: Span) => operation ??= Promise.resolve().then(work).finally(() => {
try { span?.end(); } catch { /* Diagnostics cannot change accounting. */ }
});
try {
const traced = trace.getTracer("paperclip.budgets").startActiveSpan(name, { attributes }, span => run(span));
// Some external SDK implementations can fail before or after the callback.
// Own the accounting promise so either failure still executes work once.
void Promise.resolve(traced).catch(() => undefined);
} catch { /* Preserve the accounting operation if span creation fails. */ }
return operation ?? run();
});
try {
const traced = trace.getTracer("paperclip.budgets").startActiveSpan(name, { attributes }, span => run(span));
// Some external SDK implementations can fail before or after the callback.
// Own the accounting promise so either failure still executes work once.
void Promise.resolve(traced).catch(() => undefined);
} catch { /* Preserve the accounting operation if span creation fails. */ }
return operation ?? run();
}
@@ -1,6 +1,6 @@
import { randomBytes, randomUUID } from "node:crypto";
import { agents, companies, createDb, heartbeatRuns } from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
@@ -49,7 +49,7 @@ describe.skipIf(!support.supported)("chat authorization run reads", () => {
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, scope.failedRunId));
const events: Array<Record<string, unknown>> = [];
const now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
try {
const selected = await db.transaction(tx => readChatAuthorizationRun(tx, scope, true));
@@ -58,7 +58,7 @@ describe.skipIf(!support.supported)("chat authorization run reads", () => {
expect(selected.run).not.toHaveProperty("runnerProfileJson");
expect(selected.run).not.toHaveProperty("stdoutExcerpt");
} finally { capture.flush(); }
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
expect(summary.completedQueries).toBeGreaterThan(0);
expect(summary.responseValueBytes).toBeLessThan(4096);
const retry = await db.transaction(tx => readChatAuthorizationRun(tx, scope, false));
@@ -2,7 +2,7 @@ import { createHmac } from "node:crypto";
import { eq } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { applyPendingMigrations, chatActions, createDb } from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../../__tests__/helpers/full-database-response-capture.js";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../../__tests__/helpers/embedded-postgres.js";
import { resolveExternalTestDatabaseUrl } from "../../../__tests__/helpers/external-test-database.js";
import { cleanupIngressFixture, githubIngressFixture as createFixture } from "../../../__tests__/helpers/github-webhook-ingress.js";
@@ -36,7 +36,7 @@ describe.skipIf(!support.supported)("GitHub webhook ingress claim PostgreSQL beh
const f = await githubIngressFixture(db, createGitHubWebhookIngress, 768 * 1024);
const action = await f.insertAction();
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { chatActions }, capture);
try { expect((await f.service.processGitHubWebhookIngress(action.id))?.status).toBe(202); }
finally { capture.flush(); }
@@ -59,7 +59,7 @@ describe.skipIf(!support.supported)("GitHub webhook ingress claim PostgreSQL beh
// A corrupt body must not let a nonclaimant overwrite a live or delayed claim.
const action = await f.insertAction({ status, result, payload: { ...f.payload, bodySha256: "0".repeat(64) } });
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { chatActions }, capture);
let response: Response | null;
try { response = await f.service.processGitHubWebhookIngress(action.id); }
@@ -67,7 +67,7 @@ describe.skipIf(!support.supported)("GitHub webhook ingress claim PostgreSQL beh
expect(response?.status ?? null).toBe(accepted ? 202 : null);
expect(await f.readAction(action.id)).toEqual(action);
expect(f.calls).toEqual([]);
expect(events.find(event => event.event === "paperclip.db_response_capture.summary.v1")?.responseValueBytes).toBeLessThan(1024);
expect(events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")?.responseValueBytes).toBeLessThan(1024);
});
it.each(["received", "failed", "processing"])("settles eligible malformed %s payloads without replay", async status => {
@@ -1,4 +1,5 @@
import { subscribeDatabaseWork, databaseWorkPending, reconcileDatabaseWork } from "@paperclipai/db/work-signals";
import { withDatabaseOperation } from "@paperclipai/db";
import { createWorkScheduler } from "./work-scheduler.js";
import { type DeliveryQueue } from "./delivery-work-notifications.js";
import { beginIdleTrackedWork } from "./task-admission.js";
@@ -70,7 +71,7 @@ export function createDeliveryWorkCoordinator(input: {
const attempt = new AbortController();
let runFailed = false;
controller = attempt;
running = Promise.resolve().then(() => reconcileDatabaseWork(input.owner, queue))
running = withDatabaseOperation(`delivery.${queue}`, () => Promise.resolve().then(() => reconcileDatabaseWork(input.owner, queue))
.then(() => workerStopped || !(input.canRun() || (task.runDuringDrain && input.canReconcile?.())) ? undefined : task.run(attempt.signal).catch(error => { runFailed = true; throw error; })).then(() => workerStopped ? false : task.hasPending()).then(pending => {
pendingWork = pending || databaseWorkPending(input.owner, queue);
if (pendingWork) schedule(task.retryMs);
@@ -98,7 +99,7 @@ export function createDeliveryWorkCoordinator(input: {
controller = null;
finish();
if (dirty) schedule(0);
});
}));
}
function wake() {
if (workerStopped) return;
@@ -1,4 +1,5 @@
import { AsyncLocalStorage } from "node:async_hooks";
import { withDatabaseOperation } from "@paperclipai/db";
import { EXECUTION_PREPARATION_SPAN_ATTRS, getActiveStepContext, runWithRuntimeParent } from "@paperclipai/adapter-utils/acpx-engine/startup-timing";
import { createHash } from "node:crypto";
import { isSpanContextValid, ROOT_CONTEXT, SpanStatusCode, trace, type Context } from "@opentelemetry/api";
@@ -156,7 +157,7 @@ function measure<T>(runId: string, name: string, work: () => T | Promise<T>, ind
ended: false,
details: !independent && previous?.runId === runId ? previous.details : new Map<string, number>(),
};
operation = Promise.resolve().then(() =>
operation = withDatabaseOperation(name, () => Promise.resolve().then(() =>
ownership.run(owned, () => runWithRuntimeParent(owned.parent, work))).then(value => {
try { span?.end(); } catch {}
owned.ended = true;
@@ -166,10 +167,10 @@ function measure<T>(runId: string, name: string, work: () => T | Promise<T>, ind
try { span?.end(); } catch {}
owned.ended = true;
throw error;
});
}));
return operation;
};
if (!endpointEnabled()) return (async () => work())();
if (!endpointEnabled()) return withDatabaseOperation(name, async () => work());
try {
const parent = independent ? undefined : activeParent ?? getRunExecutionTraceContext(runId);
const trigger = parent ? undefined : trace.getActiveSpan()?.spanContext();
@@ -202,20 +203,22 @@ export function withRunWorkspaceFinalizationTiming<T>(runId: string, work: () =>
export function withRunResponsePresentationTiming<T>(runId: string, work: () => T | Promise<T>): Promise<T> {
const owned = ownership.getStore();
if (owned?.runId === runId && owned.phase === "task.response_presentation" && !owned.ended) {
return (async () => work())();
return withDatabaseOperation("task.response_presentation", async () => work());
}
return measure(runId, "task.response_presentation", work, true);
}
export async function measureRunPhase<T>(phase: ExecutionPhase, work: () => T | Promise<T>, counts?: { fileCount?: number; bytes?: number }): Promise<T> {
const owned = ownership.getStore();
if (!endpointEnabled() || !owned || owned.ended) return work();
const count = owned.details.get(phase) ?? 0;
if (count >= 16) return work();
owned.details.set(phase, count + 1);
// A startup step can be more specific than the owning run phase. Keep that
// causal parent while the child publishes its own context to plugin workers.
return measure(owned.runId, phase, work, false, counts, getActiveStepContext()?.parentContext as Context | undefined);
export function measureRunPhase<T>(phase: ExecutionPhase, work: () => T | Promise<T>, counts?: { fileCount?: number; bytes?: number }): Promise<T> {
return withDatabaseOperation(phase, async () => {
const owned = ownership.getStore();
if (!endpointEnabled() || !owned || owned.ended) return work();
const count = owned.details.get(phase) ?? 0;
if (count >= 16) return work();
owned.details.set(phase, count + 1);
// A startup step can be more specific than the owning run phase. Keep that
// causal parent while the child publishes its own context to plugin workers.
return measure(owned.runId, phase, work, false, counts, getActiveStepContext()?.parentContext as Context | undefined);
});
}
export function beginRunPhase(phase: ExecutionPhase): (failed?: boolean) => void {
@@ -1,6 +1,6 @@
import { randomBytes, randomUUID } from "node:crypto";
import { heartbeatRuns, issues, type Db } from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { eq, sql } from "drizzle-orm";
import { describe, expect, it, vi } from "vitest";
import { isAcknowledgedNativeStop } from "../acknowledged-native-stop.js";
@@ -31,7 +31,7 @@ describe.skipIf(!runControlPostgresSupport.supported)("native Stop evidence read
contextSnapshot: { savedInput: bulk }, runnerProfileJson: { savedInput: bulk }, stdoutExcerpt: bulk,
stderrExcerpt: bulk, resultJson: result }).returning();
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
let evidence: Awaited<ReturnType<typeof readNativeStopExecutionEvidence>>;
try { evidence = await readNativeStopExecutionEvidence(db, company.id, id); }
@@ -68,7 +68,7 @@ describe.skipIf(!runControlPostgresSupport.supported)("native Stop evidence read
runtimeMode: "native", status: "cancelled" }).returning();
await db.execute(sql`update ${heartbeatRuns} set result_json = ${JSON.stringify(saved)}::jsonb where id = ${run.id}`);
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
try {
const evidence = await readNativeStopExecutionEvidence(db, company.id, run.id);
@@ -1,5 +1,5 @@
import { randomBytes, randomUUID } from "node:crypto";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import type { Db } from "@paperclipai/db";
import {
agents,
@@ -186,13 +186,13 @@ describePostgres("heartbeat run state database wiring", () => {
stdoutExcerpt: payload, stderrExcerpt: payload,
}).returning();
const observed = createDb(database.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
try {
const state = extracted.createHeartbeatRunState(observed);
const read = await state.getRunExecutionWait(companyId, run.id);
capture.flush();
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
expect(summary.completedQueries).toBe(2);
expect(summary.responseValueBytes).toBeLessThan(4096);
expect(read).toMatchObject({ id: run.id, companyId, agentId: agent.id, status: "cancelled", contextSnapshot: context,
@@ -228,7 +228,7 @@ describePostgres("heartbeat run state database wiring", () => {
resultJson: { stdout: payload }, stdoutExcerpt: payload, stderrExcerpt: payload,
}).returning();
const observed = createDb(database.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
try {
const state = extracted.createHeartbeatRunState(observed);
@@ -236,7 +236,7 @@ describePostgres("heartbeat run state database wiring", () => {
id: run.id, companyId, scopeKind: "issue", issueId: issue.id, executionWorkspaceId: workspaceId,
});
capture.flush();
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
expect(summary.completedQueries).toBe(1);
expect(summary.responseValueBytes).toBeLessThan(1024);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]).toEqual(run);
@@ -260,7 +260,7 @@ describePostgres("heartbeat run state database wiring", () => {
}).returning();
const observed = createDb(database.connectionString);
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
try {
const state = extracted.createHeartbeatRunState(observed);
@@ -270,7 +270,7 @@ describePostgres("heartbeat run state database wiring", () => {
runtimeModeResolvedAt: null, startedAt: run.startedAt, createdAt: run.createdAt,
contextSnapshot: { issueId: issue.id }, runnerProfileJson: { adapterDispatch: { adapterType: "paperclip_runner" } } });
capture.flush();
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
expect(summary.completedQueries).toBe(2);
expect(summary.responseValueBytes).toBeLessThan(2048);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]).toEqual(run);
@@ -321,7 +321,7 @@ describePostgres("heartbeat run state database wiring", () => {
await state.getRunWorkspaceSettlement(run.id);
const events: Array<Record<string, unknown>> = [];
const now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
expect(await state.getRunCleanupState(run.id)).toEqual({
id: run.id, companyId, agentId: agent.id, runtimeMode: run.runtimeMode,
@@ -334,7 +334,7 @@ describePostgres("heartbeat run state database wiring", () => {
contextSnapshot: { timerClaimWasFirstHeartbeat: true },
});
capture.flush();
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
expect(summary.completedQueries).toBe(2);
expect(summary.responseValueBytes).toBeLessThan(2000);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]).toEqual(run);
@@ -1,5 +1,5 @@
import { heartbeatRuns, type Db } from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../../../__tests__/helpers/full-database-response-capture.js";
import { eq } from "drizzle-orm";
import { expect } from "vitest";
@@ -21,11 +21,11 @@ export async function verifyCommittedReplayPayloadReads(
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
const events: Array<Record<string, unknown>> = [];
const now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
try { await replay(); }
finally { capture.flush(); }
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
expect(summary.completedQueries).toBeGreaterThan(0);
expect(summary.responseValueBytes).toBeLessThan(100_000);
const [after] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
@@ -29,7 +29,7 @@ import {
toolApplications,
toolConnections,
} from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
import { createLocalDiskStorageProvider } from "../../storage/local-disk-provider.js";
@@ -322,7 +322,7 @@ describe("native same-conversation chat attachment reuse", () => {
const deliveries = await db.select().from(chatDeliveries).where(eq(chatDeliveries.endpointId, endpointId));
const bulk = randomBytes(64 * 1024).toString("base64");
const events: Array<Record<string, unknown>> = [];
let capture: ReturnType<typeof createDatabaseResponseCapture> | undefined;
let capture: ReturnType<typeof createFullDatabaseResponseCapture> | undefined;
try {
await db.update(chatEndpoints).set({ communicationInstructions: bulk,
setup: { ...endpoint.setup, unusedContent: bulk } as typeof endpoint.setup,
@@ -335,7 +335,7 @@ describe("native same-conversation chat attachment reuse", () => {
.where(eq(chatDeliveries.id, delivery.id));
}
const now = Date.now();
capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { chatMessageLinks, chatDeliveries, chatConversations, chatEndpoints,
chatEndpointResources, chatExternalPrincipals, chatIdentityLinks, companyMemberships }, capture);
for (const lockMode of ["read", "blocking", "nonblocking"] as const) {
@@ -8,7 +8,7 @@ import { tmpdir } from "node:os";
import path from "node:path";
import { and, eq, ne, sql } from "drizzle-orm";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import {
agentWakeupRequests,
@@ -922,7 +922,7 @@ describe("native external-chat response wait", () => {
await authorizeChatConversationForBoundRun(db, fixture, contextSnapshot, "read");
const observed = createDb(temporary?.connectionString ?? externalTestDatabaseUrl!);
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { issues, heartbeatRuns }, capture);
try {
expect(await listExternalReviewWaits(observed, fixture.companyId, [fixture.issueId])).toEqual(governed ? [] : [
@@ -5462,7 +5462,7 @@ describe("native external-chat response wait", () => {
if (reply) await db.update(chatActions).set({ result: { ...reply.result, savedProviderOutput: payload } }).where(eq(chatActions.id, reply.id));
const observed = createDb(temporary?.connectionString ?? externalTestDatabaseUrl!);
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { heartbeatRuns, chatActions }, capture);
try {
const context = await new PaperclipRunnerToolAuthority(observed, fixture)
@@ -19,7 +19,7 @@ import {
issueThreadInteractions,
issues,
} from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { startRecoveryPostgres } from "../../__tests__/helpers/recovery-postgres.js";
import { initializeRunIdentity, reserveSteeredIdentity, reconcileSteeredIdentity } from "../run-identity.js";
import { documentService } from "../documents.js";
@@ -248,7 +248,7 @@ describe("PaperclipRunnerToolAuthority", () => {
}).where(eq(heartbeatRuns.id, runId));
const [expanded] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
const observed = createDb(temporary!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { heartbeatRuns, issues, agents, companies }, capture);
try {
const reader = new PaperclipRunnerToolAuthority(observed, binding);
@@ -291,7 +291,7 @@ describe("PaperclipRunnerToolAuthority", () => {
await db.update(heartbeatRuns).set({ contextSnapshot: context }).where(eq(heartbeatRuns.id, runId));
const [expanded] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
const observed = createDb(temporary!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(observed, { heartbeatRuns, issues, agents, companies }, capture);
try {
const reader = new PaperclipRunnerToolAuthority(observed, binding);
@@ -1,6 +1,6 @@
import { randomBytes, randomUUID } from "node:crypto";
import { agents, companies, createDb, issueRecoveryActions, issues } from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
@@ -60,7 +60,7 @@ describe.skipIf(!support.supported)("recovery issue state reads", () => {
const service = recoveryService(db, { enqueueWakeup });
const events: Array<Record<string, unknown>> = [];
const now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { issues, issueRecoveryActions }, capture);
try {
await service.reconcileStrandedAssignedIssues();
@@ -100,7 +100,7 @@ describe.skipIf(!support.supported)("recovery issue state reads", () => {
const service = recoveryService(db, { enqueueWakeup });
const events: Array<Record<string, unknown>> = [];
const now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { agents, issues, issueRecoveryActions }, capture);
try {
expect(await service.reconcileStrandedAssignedIssues()).toMatchObject({ skipped: 1 });
@@ -1,6 +1,6 @@
import { randomBytes, randomUUID } from "node:crypto";
import { agents, companies, createDb, heartbeatRuns } from "@paperclipai/db";
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
@@ -37,7 +37,7 @@ describe.skipIf(!support.supported)("legacy disposition source reads", () => {
errorCode: "adapter_failed", resultJson: { executionRecovery: { providerWorkStarted: true }, stdout: bulk } }).returning();
const events: Array<Record<string, unknown>> = [];
const now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
try {
expect(await readLegacyDispositionSource(db, { companyId, issueId, runId: before.id }))
@@ -77,7 +77,7 @@ describe.skipIf(!support.supported)("legacy disposition source reads", () => {
const enqueueWakeup = vi.fn(async () => { throw new Error("native work must not enter legacy recovery"); });
const service = recoveryService(db, { enqueueWakeup });
const events: Array<Record<string, unknown>> = [], now = Date.now();
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
try { expect(await service.reconcileLegacyContinuation(before.id)).toBe("skipped"); }
finally { capture.flush(); }