mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 17:51:58 +01:00
fix(db): attribute bounded captures to execution owners (#18634)
## Thinking Path > - Paperclip manages agent work through shared server and database services. > - High query volume can spend main-thread time on SQL construction and result decoding. > - The existing bounded response capture counts rows and bytes, but it cannot name the operation that executed a query. > - Bundled callers can be missing, and measuring every parsed JSON value adds work to the diagnostic. > - This pull request carries a semantic execution owner into each measured query and bounds response sizing. > - Operators can rank query volume by owner before choosing a performance fix. ## Linked Issues or Issue Description Refs #17504, #17522, #17537, #17564. **What existing behavior does this improve?** The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic. **Subsystem affected** Database diagnostics and existing server execution boundaries. **Current behavior** The capture groups by table, operation, and builder caller. It cannot distinguish execution owners that reuse a prepared query. Bundled deployments need source maps for caller locations. The capture visits every returned value and serializes parsed JSON again to estimate its size. **Proposed behavior** Version 2 captures use fixed semantic owners for HTTP requests, delivery queues, execution phases, and budget work. They count completed calls, returned rows, errors, and completion wall time. They sample successful results 1, 17, 33, and so on, with fixed cell and string/byte limits. Complex values remain explicitly unmeasured. Semantic ownership works without source maps. **Reason and benefit** A short census can identify which operations issue the most queries or return the most rows. It preserves each query's execution owner through pooled connections, concurrent operations, and prepared-query reuse. It reduces diagnostic work without claiming per-query CPU attribution. **Breaking changes** Opt-in capture output moves from `entry.v1`/`summary.v1` to `entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace all-result byte estimates. Consumers must use the new fields and must not extrapolate them into a total. There is no database migration or application API change. ## What Changed - Added `withDatabaseOperation(owner, action)` to the DB package. The server owns semantic labels. Queries retain execution ownership in their completion closure. - Added fixed owners at existing HTTP, delivery, budget, and execution timing boundaries. Delivery work replaces the context of the producer that woke it. - Added complete call/row/error counters and completion wall durations within the existing bounded window. Durations include waits and are not CPU measurements. - Sampled and bounded primitive-value sizing. Complex and oversized values remain unmeasured. Known owners skip repeated stack extraction. - Preserved the deadline, group limit, paced output, query outcomes, and existing coverage exclusions. Updated the operator documentation. - Added real PostgreSQL owner-isolation and prepared-query tests, server scope tests, and plain bundled-runtime proof. - Moved projection-test and benchmark full-response sizing into an explicit test helper. Existing payload-size thresholds remain unchanged; these synthetic checks do not use the production sampler. ## Verification - Final-commit DB capture suites: 43 tests passed in 13.86 seconds against the actual PostgreSQL driver, including concurrent owners, prepared reuse, transaction failure, a dedicated client, sampling, and a query crossing the deadline. DB typecheck passed. - Server attribution and existing delivery/execution/budget suites: 53 tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5 tests passed. - Runtime bundle suite: 8 existing tests passed. The new no-source-map owner test passed after its fixture was corrected to use a driver thenable. An ordinary native Promise can bypass an overridden `.then`; the real PostgreSQL tests separately cover driver integration. - Observer-only experiment: three alternating pairs per case on Node 24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an already-materialized synthetic JSON result, median capture CPU fell from 82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell from 8.10 to 4.40 microseconds/call. Every result retained identity and all calls were counted. The new diagnostic intentionally measures less byte data; these are observer costs, not database latency or a hosted application speedup. Disabled-case timings were below one microsecond/call and too small for a performance conclusion. - Final lifecycle regression: 28 unit tests and DB typecheck passed. Closing the last capture disables and replaces owner storage; overlapping captures retain it. - Final-commit full recursive typecheck passed in 38.38 seconds and full build passed in 40.81 seconds. The build stamp matches `83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution tests also passed 5/5 before the test-helper-only migration. Full test acceptance will use the repository's parallel CI gates; the unsharded local command takes hours. - Verification failures retained: initial sandbox runs could not open local IPC/PostgreSQL listeners; permitted reruns exercised the real services. One old test expected JSON serialization and was updated to assert explicit unmeasured complex values. The first bundle fixture used the wrong promise boundary, as described above. A retained-callback regression also demonstrated that disabling and reusing the same AsyncLocalStorage instance could resurrect an old owner; replacing the instance after the last capture closes fixes that case. - The first CI run found older projection tests consuming the previous full-response fields. Those consumers now use the full-response test helper, preserving their byte thresholds. The two complete affected route suites plus helper tests passed 115/115 in 73.08 seconds. A temporary full-row query negative control then failed the unchanged 2,048-byte limit at 5,000,812 bytes and was restored; five other service suites passed 34/34 and the affected chat cases passed 3/3. All three previously failing CI shards passed on the corrected head. A local skill-list benchmark smoke check completed 255 captured queries with no in-flight queries or overflow; this was integration verification under concurrent test load, not a timing benchmark. - Two final-head runner jobs lost communication with GitHub before reporting a test result. One targeted retry of those jobs and their dependent aggregate passed. All 53 final-head checks passed, with 3 intentional skips; previously passing suites were preserved. - Hosted capture remains pending until the reviewed revision is deployed and a bounded window is explicitly configured. Reproduce the focused checks with Node 24 and the repository pnpm version: ```sh pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs pnpm -r typecheck pnpm build ``` ## Risks - Owner context is active while a valid future capture window is armed, up to 30 minutes before accounting starts. No-window and expired calls take the direct path. Diagnostics still add bounded work while enabled. - A nearest nested owner is reported. Broad `http.request` and unattributed groups remain visible; this does not identify every service callsite or SQL shape. - The existing census excludes tagged/raw explicit-execute/streaming paths and internal protocol commands. Its counts are not PostgreSQL transaction totals. - Completion wall time includes queue/database/network/scheduling waits. Concurrent durations overlap. Sampled bytes exclude complex values, large strings, and values beyond the sizing limits. - Activation is startup-only. Operators must choose the window after artifact preparation, allow rollout/warm-up time, and remove the expired setting afterward. ## Model Used OpenAI Codex, GPT-6 family, with code execution, repository inspection, and parallel agents. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
6aabcff7d1
commit
a92f616cd0
43 files changed
+844
-228
No files matched your search
@@ -4,7 +4,7 @@ import request from "supertest";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { agents, companies, createDb, heartbeatRunEvents, heartbeatRuns, issueComments, issues } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { createLocalAgentJwt } from "../agent-auth-jwt.js";
|
||||
import { actorMiddleware } from "../middleware/auth.js";
|
||||
@@ -53,7 +53,7 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
it("bounds actual middleware responses and preserves the complete stored run", async () => {
|
||||
const fixture = await seed();
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const response = await request(fixture.app).get("/actor").set("Authorization", `Bearer ${fixture.token}`);
|
||||
@@ -61,7 +61,7 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
expect(response.body).toMatchObject({ type: "agent", runId: fixture.runId,
|
||||
identityContextId: fixture.identity.id, onBehalfOfUserId: "owner" });
|
||||
} finally { capture.flush(); }
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "heartbeat_runs" && (event.operation === "select" || event.operation === "execute"));
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeGreaterThanOrEqual(3);
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.responseValueBytes), 0)).toBeLessThan(16_384);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { agents, companies, costEvents, createDb } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
@@ -61,7 +61,7 @@ describe.skipIf(!support.supported)("agent roster read projections", () => {
|
||||
|
||||
async function observe<T>(run: (observed: ReturnType<typeof createDb>) => Promise<T>) {
|
||||
const observed = createDb(database.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { agents, costEvents }, capture);
|
||||
try {
|
||||
const result = await run(observed);
|
||||
|
||||
@@ -4,7 +4,7 @@ import { randomUUID } from "node:crypto";
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
agents,
|
||||
@@ -1164,13 +1164,13 @@ describeEmbeddedPostgres("attention service", () => {
|
||||
})));
|
||||
}
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const feed = await attentionService(db).list(companyId, { userId: "board-user" });
|
||||
expect(feed.items.filter(item => item.sourceKind === "failed_run").map(item => item.subject.id)).toEqual([failedId]);
|
||||
} finally { capture.flush(); }
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "heartbeat_runs" && (event.operation === "select" || event.operation === "execute"));
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeGreaterThan(0);
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.rows), 0)).toBeLessThan(64);
|
||||
@@ -1209,7 +1209,7 @@ describeEmbeddedPostgres("attention service", () => {
|
||||
});
|
||||
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const feed = await attentionService(db).list(companyId, {
|
||||
@@ -1218,7 +1218,7 @@ describeEmbeddedPostgres("attention service", () => {
|
||||
expect(feed.items.filter((item) => item.sourceKind === "failed_run").map((item) => item.subject.id).sort())
|
||||
.toEqual([...keptIds].sort());
|
||||
} finally { capture.flush(); }
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "heartbeat_runs" && (event.operation === "select" || event.operation === "execute"));
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeGreaterThan(0);
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeLessThan(4);
|
||||
@@ -1407,7 +1407,7 @@ describeEmbeddedPostgres("attention service", () => {
|
||||
const interactionItems = feed.items.filter((item) => item.sourceKind === "issue_thread_interaction");
|
||||
const detailsByKind = new Map(interactionItems.map((item) => [item.detail?.kind, item]));
|
||||
const summaryEvents: Array<Record<string, unknown>> = [], captureNow = Date.now();
|
||||
const summaryCapture = createDatabaseResponseCapture({ start: captureNow - 100, end: captureNow + 59_000 }, event => summaryEvents.push(event));
|
||||
const summaryCapture = createFullDatabaseResponseCapture({ start: captureNow - 100, end: captureNow + 59_000 }, event => summaryEvents.push(event));
|
||||
installDatabaseResponseCapture(db, { documents, assets, issueAttachments }, summaryCapture);
|
||||
try {
|
||||
const summary = await attentionService(db).summary(companyId, { userId: "board-user" });
|
||||
@@ -1418,7 +1418,7 @@ describeEmbeddedPostgres("attention service", () => {
|
||||
expect(index.totalCount).toBe(feed.totalCount);
|
||||
|
||||
} finally { summaryCapture.flush(); }
|
||||
const cardReads = summaryEvents.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const cardReads = summaryEvents.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& ["documents", "assets", "issue_attachments"].includes(String(event.table)));
|
||||
expect(cardReads).toEqual([]);
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ import { createSystemRecovery } from "../services/system-recovery-state.js";
|
||||
import { remoteTerminationReceipt } from "../services/remote-execution-termination.js";
|
||||
import { stageReceiptReactionRemovals } from "../services/chat-receipt-reactions.js";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { githubResponseAuthorityFields } from "../services/chat-channels/github/response-authority-read.js";
|
||||
import * as activityLogModule from "../services/activity-log.js";
|
||||
import * as connectionCredentialBindings from "../services/connection-credential-bindings.js";
|
||||
@@ -2890,7 +2890,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
const normalizedEvent = { ...delivery.normalizedEvent, unusedContent: bulk };
|
||||
await db.update(chatDeliveries).set({ normalizedEvent }).where(eq(chatDeliveries.id, delivery.id));
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatMessageLinks, chatDeliveries, chatEndpoints, chatConversations,
|
||||
chatEndpointResources, chatGitHubConfigurations, chatExternalPrincipals, chatIdentityLinks,
|
||||
companyMemberships, chatActions, heartbeatRuns }, capture);
|
||||
@@ -2919,7 +2919,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
} };
|
||||
await db.update(chatDeliveries).set({ normalizedEvent }).where(eq(chatDeliveries.id, delivery.id));
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatDeliveries }, capture);
|
||||
let projected: { event: Record<string, unknown> };
|
||||
try {
|
||||
@@ -64493,7 +64493,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
status: "active", membershipRole: "operator" });
|
||||
}
|
||||
const queryEvents: Record<string, unknown>[] = [];
|
||||
const capture = mode === "backlog_51" ? createDatabaseResponseCapture({ start: Date.now() - 1,
|
||||
const capture = mode === "backlog_51" ? createFullDatabaseResponseCapture({ start: Date.now() - 1,
|
||||
end: Date.now() + 59_000 }, event => queryEvents.push(event)) : undefined;
|
||||
if (capture) installDatabaseResponseCapture(db, { chatActions, chatDeliveries, chatMessageLinks, issueComments,
|
||||
agentWakeupRequests, chatEndpoints, chatConversations }, capture);
|
||||
@@ -64513,7 +64513,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
idempotencyKey: mode === "concurrent_clicks" ? randomUUID() : input.idempotencyKey })]);
|
||||
if (capture) {
|
||||
capture.flush();
|
||||
expect(Number(queryEvents.find(event => event.event === "paperclip.db_response_capture.summary.v1")?.startedQueries))
|
||||
expect(Number(queryEvents.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")?.startedQueries))
|
||||
.toBeLessThan(180);
|
||||
}
|
||||
expect(two.requestId).toBe(one.requestId);
|
||||
@@ -67713,7 +67713,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
})));
|
||||
const scope = { companyId: context.fixture.companyId, issueId: context.issue.id };
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatActions, agentWakeupRequests }, capture);
|
||||
let scopes: Array<{ companyId: string; issueId: string }>;
|
||||
try {
|
||||
@@ -67730,7 +67730,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
}
|
||||
expect(events.at(-1)).toMatchObject({ startedQueries: mode === "task" ? 0 : 1,
|
||||
completedQueries: mode === "task" ? 0 : 1, inFlightAtEnd: 0 });
|
||||
expect(events.filter(event => event.event === "paperclip.db_response_capture.entry.v1")
|
||||
expect(events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1")
|
||||
.reduce((count, event) => count + Number(event.rows), 0)).toBe(mode === "task" ? 0 : scopes.length);
|
||||
} finally {
|
||||
await context.service.shutdown(); await other.service.shutdown();
|
||||
@@ -77438,7 +77438,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
})));
|
||||
const scope = { companyId: context.fixture.companyId, issueId: context.conversation.issueId };
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatActions, agentWakeupRequests }, capture);
|
||||
let scopes: Array<{ companyId: string; issueId: string }>;
|
||||
try {
|
||||
@@ -77455,7 +77455,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
}
|
||||
expect(events.at(-1)).toMatchObject({ startedQueries: mode === "task" ? 0 : 1,
|
||||
completedQueries: mode === "task" ? 0 : 1, inFlightAtEnd: 0 });
|
||||
expect(events.filter(event => event.event === "paperclip.db_response_capture.entry.v1")
|
||||
expect(events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1")
|
||||
.reduce((count, event) => count + Number(event.rows), 0)).toBe(mode === "task" ? 0 : scopes.length);
|
||||
} finally {
|
||||
await context.service.shutdown(); await other.service.shutdown();
|
||||
|
||||
@@ -17,7 +17,7 @@ import {
|
||||
projectWorkspaces,
|
||||
} from "@paperclipai/db";
|
||||
import { parseFrontmatterMarkdown } from "@paperclipai/shared";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
@@ -651,17 +651,17 @@ describeEmbeddedPostgres("companySkillService.list", () => {
|
||||
]);
|
||||
await svc.list(companyId);
|
||||
const events: Record<string, unknown>[] = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { agents, costEvents }, capture);
|
||||
let listed: Awaited<ReturnType<typeof svc.list>>;
|
||||
try { listed = await svc.list(companyId); } finally { capture.flush(); }
|
||||
expect(listed.find(item => item.id === first.id)?.attachedAgentCount).toBe(2);
|
||||
expect(listed.find(item => item.id === second.id)?.attachedAgentCount).toBe(1);
|
||||
expect(listed.find(item => item.id === ambiguous.id)?.attachedAgentCount).toBe(0);
|
||||
const agentReads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1" && event.table === "agents");
|
||||
const agentReads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1" && event.table === "agents");
|
||||
expect(agentReads).toHaveLength(1);
|
||||
expect(Number(agentReads[0]?.responseValueBytes)).toBeLessThan(2_000);
|
||||
expect(events.filter(event => event.event === "paperclip.db_response_capture.entry.v1" && event.table === "cost_events")).toHaveLength(0);
|
||||
expect(events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1" && event.table === "cost_events")).toHaveLength(0);
|
||||
await db.update(agents).set({ adapterConfig: { paperclipSkillSync: { desiredSkills: [second.key] } } }).where(eq(agents.id, readerId));
|
||||
const refreshed = await svc.list(companyId, { q: "First" });
|
||||
expect(refreshed.find(item => item.id === first.id)?.attachedAgentCount).toBe(1);
|
||||
@@ -680,12 +680,12 @@ describeEmbeddedPostgres("companySkillService.list", () => {
|
||||
const before = await svc.list(companyId);
|
||||
expect(before.find(item => item.id === inserted[0]!.id)?.folderPath).toBe("bundled/paperclip-core");
|
||||
const events: Record<string, unknown>[] = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { companySkills }, capture);
|
||||
let after: Awaited<ReturnType<typeof svc.list>>;
|
||||
try { after = await svc.list(companyId); } finally { capture.flush(); }
|
||||
expect(after).toEqual(before);
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1" && event.table === "company_skills");
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1" && event.table === "company_skills");
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.responseValueBytes), 0)).toBeLessThan(950_000);
|
||||
const stored = await svc.getById(companyId, inserted[0]!.id);
|
||||
expect(stored?.markdown).toBe(savedMarkdown);
|
||||
@@ -762,11 +762,11 @@ describeEmbeddedPostgres("companySkillService.list", () => {
|
||||
.orderBy(companySkillVersions.id);
|
||||
expect(before.length).toBeGreaterThan(0);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { companySkillVersions }, capture);
|
||||
try { expect(await companySkillService(db).list(companyId)).toEqual(expected); }
|
||||
finally { capture.flush(); }
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "company_skill_versions" && event.operation === "select");
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeGreaterThan(0);
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.responseValueBytes), 0)).toBeLessThan(4096);
|
||||
@@ -783,11 +783,11 @@ describeEmbeddedPostgres("companySkillService.list", () => {
|
||||
? { type, companyId: otherCompanyId, agentId: randomUUID() }
|
||||
: { type, companyIds: [otherCompanyId] };
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { companySkillVersions }, capture);
|
||||
try { expect(await companySkillService(db, { actor }).list(companyId)).toEqual([]); }
|
||||
finally { capture.flush(); }
|
||||
expect(events.find(event => event.event === "paperclip.db_response_capture.summary.v1")?.completedQueries).toBe(0);
|
||||
expect(events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")?.completedQueries).toBe(0);
|
||||
});
|
||||
|
||||
it("restores a deleted release after warm validation", async () => {
|
||||
|
||||
@@ -2,7 +2,7 @@ import { randomUUID } from "node:crypto";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { agents, companies, createDb, heartbeatRuns, issueComments, issueThreadInteractions, issues } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { settleConversationTurnForRun } from "../services/agent-conversations.js";
|
||||
|
||||
@@ -52,11 +52,11 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
it.each(["response", "interaction", "reset"] as const)("bounds %s settlement reads and retains all saved text and run payloads", async reply => {
|
||||
const input = await seed(reply), before = await sourceDigests(input);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns, issues, issueComments }, capture);
|
||||
try { expect(await settleConversationTurnForRun(db, input.companyId, input.runId)).toBe(true); }
|
||||
finally { capture.flush(); }
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& ["heartbeat_runs", "issues", "issue_comments"].includes(String(event.table))
|
||||
&& (event.operation === "select" || event.operation === "execute"));
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeGreaterThanOrEqual(4);
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { QueryPromise } from "drizzle-orm/query-promise";
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import { withDatabaseOperation } from "@paperclipai/db";
|
||||
import { installDatabaseWorkSignals } from "@paperclipai/db/work-signals";
|
||||
import { databaseOperationMiddleware } from "../middleware/database-operation.js";
|
||||
import { withBudgetSpan } from "../services/budget-tracing.js";
|
||||
import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../services/delivery-work-notifications.js";
|
||||
import {
|
||||
createRunPreparationTiming,
|
||||
measureRunPhase,
|
||||
withRunExecutionTiming,
|
||||
withRunResponsePresentationTiming,
|
||||
withRunWorkspaceFinalizationTiming,
|
||||
} from "../services/execution-preparation-timing.js";
|
||||
|
||||
const { owners } = await vi.hoisted(async () => {
|
||||
const { AsyncLocalStorage } = await import("node:async_hooks");
|
||||
return { owners: new AsyncLocalStorage<string>() };
|
||||
});
|
||||
vi.mock("@paperclipai/db", async importOriginal => ({
|
||||
...await importOriginal<typeof import("@paperclipai/db")>(),
|
||||
withDatabaseOperation: <T>(owner: string, action: () => T): T => owners.run(owner, action),
|
||||
}));
|
||||
|
||||
class OwnerQuery extends QueryPromise<string | undefined> {
|
||||
async execute() { return owners.getStore(); }
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it.each(["", "http://collector.test"])("attributes lazy queries to execution, preparation, and budget owners with endpoint %s", async endpoint => {
|
||||
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", endpoint);
|
||||
await withDatabaseOperation("http.request", async () => {
|
||||
const query = new OwnerQuery();
|
||||
await expect(withRunExecutionTiming("private-run", () => query)).resolves.toBe("task.execution_attempt");
|
||||
await expect(withRunWorkspaceFinalizationTiming("private-run", () => query)).resolves.toBe("task.workspace_finalization");
|
||||
await expect(withRunResponsePresentationTiming("private-run", () => query)).resolves.toBe("task.response_presentation");
|
||||
await expect(createRunPreparationTiming("private-run")("agent_read", () => query)).resolves.toBe("heartbeat.prepare.agent_read");
|
||||
await expect(createRunPreparationTiming("private-run", "native.controller")("backend_setup", () => query)).resolves.toBe("native.controller.backend_setup");
|
||||
await expect(measureRunPhase("skills.inventory", () => query)).resolves.toBe("skills.inventory");
|
||||
await expect(withBudgetSpan("budget.accounting.transaction", () => query)).resolves.toBe("budget.accounting.transaction");
|
||||
await expect(query).resolves.toBe("http.request");
|
||||
});
|
||||
expect(owners.getStore()).toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps owner attribution after the span detail limit and restores nested scopes after failure", async () => {
|
||||
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.test");
|
||||
const failure = new Error("private accounting failure");
|
||||
await withRunExecutionTiming("private-run", async () => {
|
||||
for (let i = 0; i < 18; i++) {
|
||||
await expect(measureRunPhase("skills.inventory", () => new OwnerQuery())).resolves.toBe("skills.inventory");
|
||||
}
|
||||
await expect(measureRunPhase("execution.budget_reservation", async () => {
|
||||
await expect(withBudgetSpan("budget.admission.reserve", async () => {
|
||||
await Promise.resolve();
|
||||
expect(owners.getStore()).toBe("budget.admission.reserve");
|
||||
throw failure;
|
||||
})).rejects.toBe(failure);
|
||||
return new OwnerQuery();
|
||||
})).resolves.toBe("execution.budget_reservation");
|
||||
await expect(new OwnerQuery()).resolves.toBe("task.execution_attempt");
|
||||
});
|
||||
expect(owners.getStore()).toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps concurrent HTTP requests and nested work in fixed scopes without using private URLs", async () => {
|
||||
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "");
|
||||
const app = express();
|
||||
app.use(databaseOperationMiddleware);
|
||||
app.use(express.json());
|
||||
let arrived = 0;
|
||||
let release!: () => void;
|
||||
const both = new Promise<void>(resolve => { release = resolve; });
|
||||
app.all("/:privateId", async (req, res) => {
|
||||
const before = await new OwnerQuery();
|
||||
const work = async () => {
|
||||
if (++arrived === 2) release();
|
||||
await both;
|
||||
return new OwnerQuery();
|
||||
};
|
||||
const during = req.params.privateId === "first"
|
||||
? await withBudgetSpan("budget.admission.reserve", work)
|
||||
: await measureRunPhase("skills.inventory", work);
|
||||
res.json({ before, during, after: await new OwnerQuery() });
|
||||
});
|
||||
const [first, second] = await Promise.all([
|
||||
request(app).get("/first?secret=private-first").expect(200),
|
||||
request(app).post("/second?secret=private-second").send({ private: "request-body" }).expect(200),
|
||||
]);
|
||||
expect(first.body).toEqual({ before: "http.request", during: "budget.admission.reserve", after: "http.request" });
|
||||
expect(second.body).toEqual({ before: "http.request", during: "skills.inventory", after: "http.request" });
|
||||
expect(owners.getStore()).toBeUndefined();
|
||||
});
|
||||
|
||||
it("replaces producer ownership on each delivery dispatch, including retry and pending checks", async () => {
|
||||
vi.useFakeTimers();
|
||||
const execute = vi.fn(async () => [{ xid: "42" }]);
|
||||
const database = installDatabaseWorkSignals({ execute, async transaction<T>(action: (tx: object) => Promise<T>): Promise<T> {
|
||||
return action({ execute });
|
||||
} });
|
||||
const seen: Array<{ queue: string; phase: string; owner: string | undefined }> = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: database, canRun: () => true,
|
||||
onError: (_error, queue) => { seen.push({ queue, phase: "failure", owner: owners.getStore() }); },
|
||||
});
|
||||
let fail = true;
|
||||
try {
|
||||
const workers = withDatabaseOperation("http.request", () => [DELIVERY_QUEUES.feedback, DELIVERY_QUEUES.email].map(queue =>
|
||||
coordinator.register(queue, { retryMs: 5000,
|
||||
run: async () => {
|
||||
await Promise.resolve();
|
||||
seen.push({ queue, phase: "run", owner: await new OwnerQuery() });
|
||||
if (queue === DELIVERY_QUEUES.feedback && fail) { fail = false; throw new Error("retry"); }
|
||||
},
|
||||
hasPending: async () => {
|
||||
seen.push({ queue, phase: "pending", owner: await new OwnerQuery() });
|
||||
return false;
|
||||
},
|
||||
})));
|
||||
await Promise.all(workers.map(worker => worker.ready));
|
||||
await withDatabaseOperation("budget.accounting.transaction", () => database.transaction(tx =>
|
||||
notifyDeliveryWork(tx, DELIVERY_QUEUES.feedback)));
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
expect(seen.map(({ phase }) => phase).sort()).toEqual(["failure", "pending", "pending", "run", "run", "run"]);
|
||||
for (const entry of seen) expect(entry.owner).toBe(`delivery.${entry.queue}`);
|
||||
expect(owners.getStore()).toBeUndefined();
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
} finally {
|
||||
await coordinator.stop();
|
||||
}
|
||||
});
|
||||
@@ -8,7 +8,7 @@ import { createRuntimeToolsToken } from "../runtime-tools-token.js";
|
||||
import { errorHandler } from "../middleware/index.js";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
agents,
|
||||
@@ -87,7 +87,7 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu
|
||||
});
|
||||
await db.insert(toolConnections).values(irrelevant);
|
||||
const observed = createDb(external ?? database!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { toolConnections }, capture);
|
||||
try {
|
||||
expect(await resolveManagedGitHubIdentitySelection(observed, input.companyId, { agentId: input.agentId, responsibleUserId: "A" }))
|
||||
@@ -119,7 +119,7 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu
|
||||
}).where(eq(heartbeatRuns.id, input.runId));
|
||||
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, input.runId));
|
||||
const observed = createDb(external ?? database!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
|
||||
try {
|
||||
expect(await resolveGitHubOperationCredentials(observed, input)).toMatchObject({
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import {
|
||||
agents,
|
||||
companies,
|
||||
@@ -63,11 +63,11 @@ describe("P6-11..13 / P6-17 canonical event allocator", () => {
|
||||
protocolSchemaVersion: 1, canonicalPayload: { sourceSeq } },
|
||||
});
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRunEvents }, capture);
|
||||
try { expect((await append(5002)).highestContiguousSourceSeq).toBe(5000); }
|
||||
finally { capture.flush(); }
|
||||
const reads = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "heartbeat_run_events" && (event.operation === "select" || event.operation === "execute"));
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.calls), 0)).toBeGreaterThan(0);
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.rows), 0)).toBeLessThan(16);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { drizzle } from "drizzle-orm/postgres-js";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
@@ -268,7 +268,7 @@ describeEmbeddedPostgres("heartbeat run privacy routes", { concurrent: false },
|
||||
}).where(eq(heartbeatRuns.id, f.privateRunId));
|
||||
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.privateRunId));
|
||||
const observed = createDb(externalDatabaseUrl ?? tempDb!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const app = createApp(f.companyId, f.grantedAgentId, observed);
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { createFullDatabaseResponseCapture } from "./full-database-response-capture.js";
|
||||
|
||||
const metadata = { operation: "select", table: "heartbeat_runs", callers: [] };
|
||||
|
||||
it("measures every oversized string and complex value without sampling or truncation", () => {
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const capture = createFullDatabaseResponseCapture({ start: 0, end: 1000 }, event => events.push(event), () => 0);
|
||||
const text = "private-payload🙂".repeat(30_000);
|
||||
const nested = { instructions: text, additional: [{ prompt: text }] };
|
||||
const rows = Object.assign([[text, nested]], { columns: [{ name: "stdout_excerpt" }, { name: "context_snapshot" }] });
|
||||
const bytes = Buffer.byteLength(text) + Buffer.byteLength(JSON.stringify(nested));
|
||||
try {
|
||||
for (let index = 0; index < 17; index++) expect(capture.execute(() => rows, metadata)).toBe(rows);
|
||||
capture.flush();
|
||||
expect(events[0]).toMatchObject({ event: "paperclip.test_db_response_capture.entry.v1",
|
||||
calls: 17, rows: 17, responseValueBytes: 17 * bytes, maxResponseValueBytes: bytes,
|
||||
estimatedValues: 17, serializationErrors: 0,
|
||||
});
|
||||
expect(events[1]).toMatchObject({ event: "paperclip.test_db_response_capture.summary.v1",
|
||||
startedQueries: 17, completedQueries: 17, inFlightAtEnd: 0, overflowCalls: 0, responseValueBytes: 17 * bytes,
|
||||
});
|
||||
expect(JSON.stringify(events)).not.toContain("private-payload");
|
||||
expect(nested.instructions).toBe(text);
|
||||
} finally { capture.flush(); }
|
||||
});
|
||||
|
||||
it("preserves asynchronous results and original query errors", async () => {
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const capture = createFullDatabaseResponseCapture({ start: 0, end: 1000 }, event => events.push(event), () => 0);
|
||||
const rows = Object.assign([{ value: { nested: "result" } }], { columns: [{ name: "value" }] });
|
||||
const failure = new Error("private query failure");
|
||||
try {
|
||||
await expect(capture.execute(async () => rows, metadata)).resolves.toBe(rows);
|
||||
await expect(capture.execute(async () => { throw failure; }, metadata)).rejects.toBe(failure);
|
||||
expect(() => capture.execute(() => { throw failure; }, metadata)).toThrow(failure);
|
||||
capture.flush();
|
||||
expect(events[0]).toMatchObject({ calls: 3, rows: 1, queryErrors: 2,
|
||||
responseValueBytes: Buffer.byteLength(JSON.stringify(rows[0].value)), serializationErrors: 0,
|
||||
});
|
||||
expect(events[1]).toMatchObject({ startedQueries: 3, completedQueries: 3, inFlightAtEnd: 0 });
|
||||
expect(JSON.stringify(events)).not.toContain(failure.message);
|
||||
} finally { capture.flush(); }
|
||||
});
|
||||
|
||||
it("reports unfinished work at the deadline without cancelling it or counting a late result", async () => {
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
let now = 0;
|
||||
const capture = createFullDatabaseResponseCapture({ start: 0, end: 1000 }, event => events.push(event), () => now);
|
||||
let release!: (value: string[]) => void;
|
||||
const original = new Promise<string[]>(resolve => { release = resolve; });
|
||||
const pending = capture.execute(() => original, metadata);
|
||||
now = 1000;
|
||||
capture.flush();
|
||||
release(["result"]);
|
||||
await expect(pending).resolves.toEqual(["result"]);
|
||||
capture.flush();
|
||||
expect(events).toHaveLength(1);
|
||||
expect(events[0]).toMatchObject({ event: "paperclip.test_db_response_capture.summary.v1",
|
||||
startedQueries: 1, completedQueries: 0, inFlightAtEnd: 1, responseValueBytes: 0,
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,115 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { createDatabaseResponseCapture, type ResponseCaptureWindow } from "@paperclipai/db/response-capture";
|
||||
|
||||
export { installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
|
||||
type Capture = ReturnType<typeof createDatabaseResponseCapture>;
|
||||
type Metadata = Parameters<Capture["execute"]>[1];
|
||||
type Entry = Metadata & { calls: number; rows: number; bytes: number; maxBytes: number;
|
||||
queryErrors: number; serializationErrors: number; estimatedValues: number };
|
||||
|
||||
function fullResponseValueSize(value: unknown): { bytes: number; estimatedValues: number } {
|
||||
const columns = value && typeof value === "object" ? (value as { columns?: unknown }).columns : undefined;
|
||||
if (!Array.isArray(value) || !Array.isArray(columns)) {
|
||||
const serialized = JSON.stringify(value, (_, item: unknown) => typeof item === "bigint" ? item.toString() : item);
|
||||
return { bytes: serialized === undefined ? 0 : Buffer.byteLength(serialized), estimatedValues: 1 };
|
||||
}
|
||||
let bytes = 0, estimatedValues = 0;
|
||||
for (const row of value) {
|
||||
const cells = Array.isArray(row) ? row : columns.map(column => row[column.name]);
|
||||
for (const cell of cells) {
|
||||
if (cell === null || cell === undefined) continue;
|
||||
if (typeof cell === "string") bytes += Buffer.byteLength(cell);
|
||||
else if (typeof cell === "boolean") bytes += 1;
|
||||
else if (typeof cell === "number" || typeof cell === "bigint") bytes += Buffer.byteLength(String(cell));
|
||||
else {
|
||||
estimatedValues += 1;
|
||||
const serialized = JSON.stringify(cell, (_, item: unknown) => typeof item === "bigint" ? item.toString() : item);
|
||||
if (serialized !== undefined) bytes += Buffer.byteLength(serialized);
|
||||
}
|
||||
}
|
||||
}
|
||||
return { bytes, estimatedValues };
|
||||
}
|
||||
|
||||
export function createFullDatabaseResponseCapture(
|
||||
window: ResponseCaptureWindow,
|
||||
emit: (event: Record<string, unknown>) => void,
|
||||
now: () => number = Date.now,
|
||||
): Capture {
|
||||
const capture = createDatabaseResponseCapture(window, () => {}, now);
|
||||
const id = randomUUID(), entries = new Map<string, Entry>();
|
||||
let closed = false, inFlight = 0, started = 0, completed = 0, overflowCalls = 0;
|
||||
const send = (event: Record<string, unknown>) => { try { emit(event); } catch {} };
|
||||
function flush() {
|
||||
if (closed) return;
|
||||
closed = true;
|
||||
clearTimeout(timer);
|
||||
capture.flush();
|
||||
const ordered = [...entries.values()].sort((a, b) => b.bytes - a.bytes);
|
||||
for (const [groupIndex, entry] of ordered.entries()) send({
|
||||
event: "paperclip.test_db_response_capture.entry.v1", captureId: id,
|
||||
groupIndex, groupPage: Math.floor(groupIndex / 128),
|
||||
operation: entry.operation, table: entry.table, callers: entry.callers,
|
||||
calls: entry.calls, rows: entry.rows, responseValueBytes: entry.bytes,
|
||||
maxResponseValueBytes: entry.maxBytes, estimatedValues: entry.estimatedValues,
|
||||
queryErrors: entry.queryErrors, serializationErrors: entry.serializationErrors,
|
||||
});
|
||||
send({ event: "paperclip.test_db_response_capture.summary.v1", captureId: id,
|
||||
windowStart: new Date(window.start).toISOString(), windowEnd: new Date(window.end).toISOString(),
|
||||
closedAt: new Date(now()).toISOString(), startedQueries: started, completedQueries: completed,
|
||||
inFlightAtEnd: inFlight, groups: entries.size, groupPageSize: 128, groupPages: Math.ceil(ordered.length / 128),
|
||||
overflowCalls, responseValueBytes: ordered.reduce((sum, entry) => sum + entry.bytes, 0),
|
||||
rowValuesLogged: false, sqlOrParametersLogged: false, wireBytesMeasured: false,
|
||||
measurementBoundary: "postgres-js results before Drizzle column decoding",
|
||||
relationalQueriesMeasured: true, unsafeDriverQueriesMeasured: true,
|
||||
taggedDriverQueriesMeasured: false, streamingQueriesMeasured: false,
|
||||
});
|
||||
entries.clear();
|
||||
}
|
||||
const timer = setTimeout(flush, Math.max(0, window.end - now()));
|
||||
timer.unref();
|
||||
function finish(metadata: Metadata, value: unknown, failed: boolean) {
|
||||
if (!capture.active()) return;
|
||||
inFlight -= 1;
|
||||
completed += 1;
|
||||
let key = JSON.stringify(metadata);
|
||||
if (!entries.has(key) && entries.size >= 2047) {
|
||||
overflowCalls += 1;
|
||||
metadata = { operation: "other", table: "other", callers: [] };
|
||||
key = "overflow";
|
||||
}
|
||||
let entry = entries.get(key);
|
||||
if (!entry) {
|
||||
entry = { ...metadata, calls: 0, rows: 0, bytes: 0, maxBytes: 0,
|
||||
queryErrors: 0, serializationErrors: 0, estimatedValues: 0 };
|
||||
entries.set(key, entry);
|
||||
}
|
||||
entry.calls += 1;
|
||||
if (failed) { entry.queryErrors += 1; return; }
|
||||
entry.rows += Array.isArray(value) ? value.length : 0;
|
||||
try {
|
||||
const size = fullResponseValueSize(value);
|
||||
entry.bytes += size.bytes;
|
||||
entry.maxBytes = Math.max(entry.maxBytes, size.bytes);
|
||||
entry.estimatedValues += size.estimatedValues;
|
||||
} catch { entry.serializationErrors += 1; }
|
||||
}
|
||||
function execute(action: () => unknown, metadata: Metadata) {
|
||||
if (!capture.active()) return action();
|
||||
inFlight += 1;
|
||||
started += 1;
|
||||
let result: unknown;
|
||||
try { result = action(); }
|
||||
catch (error) { finish(metadata, undefined, true); throw error; }
|
||||
let asynchronous = false;
|
||||
try { asynchronous = Boolean(result && typeof (result as { then?: unknown }).then === "function"); } catch {}
|
||||
if (asynchronous) return Promise.resolve(result).then(
|
||||
value => { finish(metadata, value, false); return value; },
|
||||
error => { finish(metadata, undefined, true); throw error; },
|
||||
);
|
||||
finish(metadata, result, false);
|
||||
return result;
|
||||
}
|
||||
return { ...capture, execute, flush };
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { activityLog, agents, companies, createDb, heartbeatRuns, issues } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
@@ -55,7 +55,7 @@ describe.skipIf(!support.supported)("creation source run projections", () => {
|
||||
expect(run.issueId).toBe(binding === "scoped" ? sourceId : null);
|
||||
expect(run.nativeIssueId).toBeNull();
|
||||
const observed = createDb(database.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns, issues }, capture);
|
||||
try {
|
||||
const result = await resolveIssueCreationSource(observed, child);
|
||||
|
||||
@@ -4,7 +4,7 @@ import { randomUUID } from "node:crypto";
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
agentRuntimeState,
|
||||
@@ -202,7 +202,7 @@ describeEmbeddedPostgres("issue queued-comment routes", () => {
|
||||
if (fallback) await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, f.issueId));
|
||||
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
|
||||
const observed = createDb(tempDb!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const response = await request(app(f.companyId, "queue-owner", undefined, observed))
|
||||
@@ -238,7 +238,7 @@ describeEmbeddedPostgres("issue queued-comment routes", () => {
|
||||
}).where(eq(heartbeatRuns.companyId, f.companyId));
|
||||
const [before] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, f.wakeId));
|
||||
const observed = createDb(tempDb!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { agentWakeupRequests, heartbeatRuns }, capture);
|
||||
try {
|
||||
const response = await request(app(f.companyId, "queue-owner", undefined, observed)).get(path).expect(200);
|
||||
@@ -275,7 +275,7 @@ describeEmbeddedPostgres("issue queued-comment routes", () => {
|
||||
})),
|
||||
]);
|
||||
const observed = createDb(tempDb!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { agentWakeupRequests }, capture);
|
||||
try {
|
||||
const response = await request(app(f.companyId, "queue-owner", undefined, observed)).get(path).expect(200);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { agents, companies, createDb, environmentLeases, environments, executionWorkspaces, heartbeatRuns, issueRecoveryActions, issues, projects } from "@paperclipai/db";
|
||||
@@ -54,12 +54,12 @@ const support = externalTestDatabaseUrl ? { supported: true } : await getEmbedde
|
||||
runnerProfileJson: { savedProfile: bulk }, stdoutExcerpt: bulk, stderrExcerpt: bulk,
|
||||
}).where(eq(heartbeatRuns.id, f.runId)).returning();
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
let returned: Awaited<ReturnType<typeof terminalizeLegacyExecution>>;
|
||||
try { returned = await terminalizeLegacyExecution({ db, run: saved!, status: "failed", patch }); }
|
||||
finally { capture.flush(); }
|
||||
const groups = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1");
|
||||
const groups = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1");
|
||||
const reads = groups.filter(group => group.table === "heartbeat_runs" && group.operation === "select");
|
||||
expect(reads.reduce((sum, group) => sum + Number(group.calls), 0)).toBe(1);
|
||||
expect(reads.reduce((sum, group) => sum + Number(group.responseValueBytes), 0)).toBeLessThan(4096);
|
||||
@@ -86,7 +86,7 @@ const support = externalTestDatabaseUrl ? { supported: true } : await getEmbedde
|
||||
patch: { resultJson: { workspaceRestoreFailure: "restore_failed", futureResult: bulk } },
|
||||
}))!;
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
let returned: Awaited<ReturnType<typeof terminalizeLegacyExecution>>;
|
||||
try { returned = await terminalizeLegacyExecution({ db, run: failed, status: "failed", recordRestoreFailureOnly,
|
||||
@@ -94,7 +94,7 @@ const support = externalTestDatabaseUrl ? { supported: true } : await getEmbedde
|
||||
...(recordRestoreFailureOnly ? { patch: { resultJson: failed.resultJson } } : {}),
|
||||
}); }
|
||||
finally { capture.flush(); }
|
||||
const writes = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const writes = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "heartbeat_runs" && event.operation === "update");
|
||||
expect(writes.reduce((sum, group) => sum + Number(group.rows), 0)).toBe(1);
|
||||
expect(returned).toMatchObject({ id: f.runId, status: "failed", finishedAt: failed.finishedAt,
|
||||
|
||||
@@ -11,7 +11,7 @@ import { setTimeout as delay } from "node:timers/promises";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { agents, companies, completionContracts, createDb, environmentLeases, environments, heartbeatRuns, issues, nativeRunFinalizations, type Db } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js";
|
||||
import { heartbeatService } from "../services/heartbeat.js";
|
||||
import { environmentRuntimeService, type EnvironmentDriver } from "../services/environment-runtime.js";
|
||||
@@ -180,7 +180,7 @@ describe("native environment release scheduling", () => {
|
||||
}
|
||||
const events: Record<string, unknown>[] = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { nativeRunFinalizations, heartbeatRuns, environmentLeases }, capture);
|
||||
const settle = vi.fn(async () => {});
|
||||
const work = nativeEnvironmentSettlementWork(db, settle);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js";
|
||||
import { EMBEDDED_POSTGRES_TEST_TIMEOUT_MS } from "@paperclipai/db";
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { spawn } from "node:child_process";
|
||||
import os from "node:os";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
@@ -222,11 +222,11 @@ suite("settled native run process identity", () => {
|
||||
const saved = await readRun(runId);
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try { expect(await releaseSettledNativeRunProcess(db, runId)).toBe("released"); }
|
||||
finally { capture.flush(); }
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.responseValueBytes).toBeLessThan(8192);
|
||||
const after = await readRun(runId);
|
||||
expect(after).toEqual({ ...saved, processPid: null, processGroupId: null, processStartedAt: null,
|
||||
|
||||
@@ -2,7 +2,7 @@ import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { eq, sql, type SQL } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from "vitest";
|
||||
import { createDb, plugins, type Db } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import type { PaperclipPluginManifestV1, PluginEnvironmentDriverDeclaration } from "@paperclipai/shared";
|
||||
import { resolvePluginSandboxProviderConfigSchema, resolvePluginSandboxProviderDriverByKey } from "../services/plugin-environment-driver.js";
|
||||
import { resolveEnvironmentDriverConfigForRuntime, resolveSandboxProviderSecretRefPaths } from "../services/environment-config.js";
|
||||
@@ -39,12 +39,12 @@ describe.skipIf(!support.supported)("sandbox provider schema reads", () => {
|
||||
const lookup = (client: Db, driverKey = key) => resolvePluginSandboxProviderConfigSchema({ db: client, driverKey });
|
||||
async function measure<T>(client: Db, operation: () => Promise<T>) {
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event), () => now);
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event), () => now);
|
||||
installDatabaseResponseCapture(client, { plugins }, capture);
|
||||
try {
|
||||
const value = await operation();
|
||||
capture.flush();
|
||||
const entries = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1");
|
||||
const entries = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1");
|
||||
return { value, bytes: entries.reduce((sum, entry) => sum + Number(entry.responseValueBytes), 0),
|
||||
calls: entries.reduce((sum, entry) => sum + Number(entry.calls), 0) };
|
||||
} finally { capture.flush(); }
|
||||
|
||||
@@ -5,7 +5,7 @@ import {
|
||||
agents, companies, createDb, toolApplications, toolCatalogEntries,
|
||||
toolConnections, toolProfileBindings, toolProfileEntries, toolProfiles, type Db,
|
||||
} from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "./helpers/full-database-response-capture.js";
|
||||
import { toolAccessService } from "../services/tool-access.js";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
|
||||
@@ -149,14 +149,14 @@ describeDatabase("effective tool assignments", () => {
|
||||
const before = await service.getEffectiveToolAssignmentsForAgent(f.company.id, f.agent.id);
|
||||
expect(before.allowedTools).toHaveLength(102);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { toolCatalogEntries, toolProfileEntries }, capture);
|
||||
let scoped: Awaited<ReturnType<typeof service.getConnectionAssignmentsForAgent>>;
|
||||
try { scoped = await service.getConnectionAssignmentsForAgent(f.company.id, f.agent.id, [f.connection.id]); }
|
||||
finally { capture.flush(); }
|
||||
expect(Number(events.at(-1)?.completedQueries)).toBeGreaterThan(0);
|
||||
expect(Number(events.at(-1)?.responseValueBytes)).toBeLessThan(16 * 1024);
|
||||
const catalogQueries = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1"
|
||||
const catalogQueries = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "tool_catalog_entries" && event.operation === "select");
|
||||
expect(catalogQueries).toHaveLength(1);
|
||||
expect(catalogQueries[0]).toMatchObject({ calls: 1, rows: 2 });
|
||||
@@ -192,13 +192,13 @@ describeDatabase("effective tool assignments", () => {
|
||||
const service = toolAccessService(db), before = await compare(f);
|
||||
const savedRules = await db.select().from(toolProfileEntries);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { toolProfileEntries, toolProfileBindings }, capture);
|
||||
try {
|
||||
const scoped = await service.getConnectionAssignmentsForAgent(f.company.id, f.agent.id, [f.connection.id]);
|
||||
capture.flush();
|
||||
const rules = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1" && event.table === "tool_profile_entries");
|
||||
const bindings = events.filter(event => event.event === "paperclip.db_response_capture.entry.v1" && event.table === "tool_profile_bindings");
|
||||
const rules = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1" && event.table === "tool_profile_entries");
|
||||
const bindings = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1" && event.table === "tool_profile_bindings");
|
||||
expect(rules).toHaveLength(1);
|
||||
expect(Number(rules[0].responseValueBytes)).toBeLessThan(4096);
|
||||
expect(rules[0]).toMatchObject({ calls: 1, rows: 2 });
|
||||
|
||||
@@ -12,6 +12,7 @@ import { DELIVERY_QUEUES } from "./services/delivery-work-notifications.js";
|
||||
import { createLifecycleDriver } from "./services/agent-lifecycle-driver.js";
|
||||
import { startAgentLifecycle } from "./services/agent-lifecycle.js";
|
||||
import { idleAdmissionMiddleware, trackIdleRequestHandlers } from "./middleware/idle-admission.js";
|
||||
import { databaseOperationMiddleware } from "./middleware/database-operation.js";
|
||||
import { staticUiFiles } from "./middleware/static-ui.js";
|
||||
import { retainedUiAssets } from "./retained-ui-assets.js";
|
||||
import { createStorageProviderFromConfig } from "./storage/provider-registry.js";
|
||||
@@ -470,6 +471,7 @@ export async function createApp(
|
||||
},
|
||||
) {
|
||||
const app = express();
|
||||
app.use(databaseOperationMiddleware);
|
||||
app.use(idleAdmissionMiddleware);
|
||||
const staticUi = express.Router();
|
||||
app.locals.paperclipDb = db;
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
import { withDatabaseOperation } from "@paperclipai/db";
|
||||
import type { RequestHandler } from "express";
|
||||
|
||||
export const databaseOperationMiddleware: RequestHandler = (_req, _res, next) =>
|
||||
withDatabaseOperation("http.request", next);
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { agents, companies, createDb, heartbeatRuns } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../../__tests__/helpers/embedded-postgres.js";
|
||||
@@ -40,7 +40,7 @@ describe.skipIf(!support.supported)("wake release run reads", () => {
|
||||
const scope = { companyId, runId: before.id };
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const native = await db.transaction(tx => readReleaseRun(tx as unknown as typeof db, scope));
|
||||
@@ -51,7 +51,7 @@ describe.skipIf(!support.supported)("wake release run reads", () => {
|
||||
expect(native.run).not.toHaveProperty("stdoutExcerpt");
|
||||
expect(native.run).not.toHaveProperty("stderrExcerpt");
|
||||
} finally { capture.flush(); }
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBe(1);
|
||||
expect(summary.responseValueBytes).toBeLessThan(8192);
|
||||
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, before.id)))[0]).toEqual(before);
|
||||
@@ -97,12 +97,12 @@ describe.skipIf(!support.supported)("wake release run reads", () => {
|
||||
contextSnapshot: { ...context, ...launch }, resultJson: evidence }).returning();
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
let selected: Awaited<ReturnType<typeof readReleaseRun>>;
|
||||
try { selected = await db.transaction(tx => readReleaseRun(tx as unknown as typeof db, { companyId, runId: before.id })); }
|
||||
finally { capture.flush(); }
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBe(1);
|
||||
expect(summary.responseValueBytes).toBeLessThan(8192);
|
||||
expect(selected!.run?.contextSnapshot).toEqual(context);
|
||||
|
||||
@@ -1,18 +1,27 @@
|
||||
import { trace, type Span } from "@opentelemetry/api";
|
||||
import { withDatabaseOperation } from "@paperclipai/db";
|
||||
|
||||
type BudgetOperation = "budget.accounting.lock_wait" | "budget.accounting.recover"
|
||||
| "budget.accounting.settlement" | "budget.accounting.transaction"
|
||||
| "budget.admission.evaluate" | "budget.admission.reserve"
|
||||
| "budget.cancellation.deliver" | "budget.enforcement.deliver"
|
||||
| "budget.enforcement.scope" | "budget.exposure.evaluate";
|
||||
|
||||
/** Operator-gated OpenTelemetry only; these spans contain no prompt, receipt,
|
||||
* credential, or provider error text. Diagnostics never change accounting. */
|
||||
export async function withBudgetSpan<T>(name: string, work: () => Promise<T>, attributes?: Record<string, number | boolean>) {
|
||||
if (!process.env.OTEL_EXPORTER_OTLP_ENDPOINT?.trim()) return work();
|
||||
let operation: Promise<T> | undefined;
|
||||
const run = (span?: Span) => operation ??= Promise.resolve().then(work).finally(() => {
|
||||
try { span?.end(); } catch { /* Diagnostics cannot change accounting. */ }
|
||||
export function withBudgetSpan<T>(name: BudgetOperation, work: () => Promise<T>, attributes?: Record<string, number | boolean>) {
|
||||
return withDatabaseOperation(name, async () => {
|
||||
if (!process.env.OTEL_EXPORTER_OTLP_ENDPOINT?.trim()) return work();
|
||||
let operation: Promise<T> | undefined;
|
||||
const run = (span?: Span) => operation ??= Promise.resolve().then(work).finally(() => {
|
||||
try { span?.end(); } catch { /* Diagnostics cannot change accounting. */ }
|
||||
});
|
||||
try {
|
||||
const traced = trace.getTracer("paperclip.budgets").startActiveSpan(name, { attributes }, span => run(span));
|
||||
// Some external SDK implementations can fail before or after the callback.
|
||||
// Own the accounting promise so either failure still executes work once.
|
||||
void Promise.resolve(traced).catch(() => undefined);
|
||||
} catch { /* Preserve the accounting operation if span creation fails. */ }
|
||||
return operation ?? run();
|
||||
});
|
||||
try {
|
||||
const traced = trace.getTracer("paperclip.budgets").startActiveSpan(name, { attributes }, span => run(span));
|
||||
// Some external SDK implementations can fail before or after the callback.
|
||||
// Own the accounting promise so either failure still executes work once.
|
||||
void Promise.resolve(traced).catch(() => undefined);
|
||||
} catch { /* Preserve the accounting operation if span creation fails. */ }
|
||||
return operation ?? run();
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { agents, companies, createDb, heartbeatRuns } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
@@ -49,7 +49,7 @@ describe.skipIf(!support.supported)("chat authorization run reads", () => {
|
||||
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, scope.failedRunId));
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const selected = await db.transaction(tx => readChatAuthorizationRun(tx, scope, true));
|
||||
@@ -58,7 +58,7 @@ describe.skipIf(!support.supported)("chat authorization run reads", () => {
|
||||
expect(selected.run).not.toHaveProperty("runnerProfileJson");
|
||||
expect(selected.run).not.toHaveProperty("stdoutExcerpt");
|
||||
} finally { capture.flush(); }
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBeGreaterThan(0);
|
||||
expect(summary.responseValueBytes).toBeLessThan(4096);
|
||||
const retry = await db.transaction(tx => readChatAuthorizationRun(tx, scope, false));
|
||||
|
||||
@@ -2,7 +2,7 @@ import { createHmac } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { applyPendingMigrations, chatActions, createDb } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../../__tests__/helpers/embedded-postgres.js";
|
||||
import { resolveExternalTestDatabaseUrl } from "../../../__tests__/helpers/external-test-database.js";
|
||||
import { cleanupIngressFixture, githubIngressFixture as createFixture } from "../../../__tests__/helpers/github-webhook-ingress.js";
|
||||
@@ -36,7 +36,7 @@ describe.skipIf(!support.supported)("GitHub webhook ingress claim PostgreSQL beh
|
||||
const f = await githubIngressFixture(db, createGitHubWebhookIngress, 768 * 1024);
|
||||
const action = await f.insertAction();
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatActions }, capture);
|
||||
try { expect((await f.service.processGitHubWebhookIngress(action.id))?.status).toBe(202); }
|
||||
finally { capture.flush(); }
|
||||
@@ -59,7 +59,7 @@ describe.skipIf(!support.supported)("GitHub webhook ingress claim PostgreSQL beh
|
||||
// A corrupt body must not let a nonclaimant overwrite a live or delayed claim.
|
||||
const action = await f.insertAction({ status, result, payload: { ...f.payload, bodySha256: "0".repeat(64) } });
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatActions }, capture);
|
||||
let response: Response | null;
|
||||
try { response = await f.service.processGitHubWebhookIngress(action.id); }
|
||||
@@ -67,7 +67,7 @@ describe.skipIf(!support.supported)("GitHub webhook ingress claim PostgreSQL beh
|
||||
expect(response?.status ?? null).toBe(accepted ? 202 : null);
|
||||
expect(await f.readAction(action.id)).toEqual(action);
|
||||
expect(f.calls).toEqual([]);
|
||||
expect(events.find(event => event.event === "paperclip.db_response_capture.summary.v1")?.responseValueBytes).toBeLessThan(1024);
|
||||
expect(events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")?.responseValueBytes).toBeLessThan(1024);
|
||||
});
|
||||
|
||||
it.each(["received", "failed", "processing"])("settles eligible malformed %s payloads without replay", async status => {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { subscribeDatabaseWork, databaseWorkPending, reconcileDatabaseWork } from "@paperclipai/db/work-signals";
|
||||
import { withDatabaseOperation } from "@paperclipai/db";
|
||||
import { createWorkScheduler } from "./work-scheduler.js";
|
||||
import { type DeliveryQueue } from "./delivery-work-notifications.js";
|
||||
import { beginIdleTrackedWork } from "./task-admission.js";
|
||||
@@ -70,7 +71,7 @@ export function createDeliveryWorkCoordinator(input: {
|
||||
const attempt = new AbortController();
|
||||
let runFailed = false;
|
||||
controller = attempt;
|
||||
running = Promise.resolve().then(() => reconcileDatabaseWork(input.owner, queue))
|
||||
running = withDatabaseOperation(`delivery.${queue}`, () => Promise.resolve().then(() => reconcileDatabaseWork(input.owner, queue))
|
||||
.then(() => workerStopped || !(input.canRun() || (task.runDuringDrain && input.canReconcile?.())) ? undefined : task.run(attempt.signal).catch(error => { runFailed = true; throw error; })).then(() => workerStopped ? false : task.hasPending()).then(pending => {
|
||||
pendingWork = pending || databaseWorkPending(input.owner, queue);
|
||||
if (pendingWork) schedule(task.retryMs);
|
||||
@@ -98,7 +99,7 @@ export function createDeliveryWorkCoordinator(input: {
|
||||
controller = null;
|
||||
finish();
|
||||
if (dirty) schedule(0);
|
||||
});
|
||||
}));
|
||||
}
|
||||
function wake() {
|
||||
if (workerStopped) return;
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { AsyncLocalStorage } from "node:async_hooks";
|
||||
import { withDatabaseOperation } from "@paperclipai/db";
|
||||
import { EXECUTION_PREPARATION_SPAN_ATTRS, getActiveStepContext, runWithRuntimeParent } from "@paperclipai/adapter-utils/acpx-engine/startup-timing";
|
||||
import { createHash } from "node:crypto";
|
||||
import { isSpanContextValid, ROOT_CONTEXT, SpanStatusCode, trace, type Context } from "@opentelemetry/api";
|
||||
@@ -156,7 +157,7 @@ function measure<T>(runId: string, name: string, work: () => T | Promise<T>, ind
|
||||
ended: false,
|
||||
details: !independent && previous?.runId === runId ? previous.details : new Map<string, number>(),
|
||||
};
|
||||
operation = Promise.resolve().then(() =>
|
||||
operation = withDatabaseOperation(name, () => Promise.resolve().then(() =>
|
||||
ownership.run(owned, () => runWithRuntimeParent(owned.parent, work))).then(value => {
|
||||
try { span?.end(); } catch {}
|
||||
owned.ended = true;
|
||||
@@ -166,10 +167,10 @@ function measure<T>(runId: string, name: string, work: () => T | Promise<T>, ind
|
||||
try { span?.end(); } catch {}
|
||||
owned.ended = true;
|
||||
throw error;
|
||||
});
|
||||
}));
|
||||
return operation;
|
||||
};
|
||||
if (!endpointEnabled()) return (async () => work())();
|
||||
if (!endpointEnabled()) return withDatabaseOperation(name, async () => work());
|
||||
try {
|
||||
const parent = independent ? undefined : activeParent ?? getRunExecutionTraceContext(runId);
|
||||
const trigger = parent ? undefined : trace.getActiveSpan()?.spanContext();
|
||||
@@ -202,20 +203,22 @@ export function withRunWorkspaceFinalizationTiming<T>(runId: string, work: () =>
|
||||
export function withRunResponsePresentationTiming<T>(runId: string, work: () => T | Promise<T>): Promise<T> {
|
||||
const owned = ownership.getStore();
|
||||
if (owned?.runId === runId && owned.phase === "task.response_presentation" && !owned.ended) {
|
||||
return (async () => work())();
|
||||
return withDatabaseOperation("task.response_presentation", async () => work());
|
||||
}
|
||||
return measure(runId, "task.response_presentation", work, true);
|
||||
}
|
||||
|
||||
export async function measureRunPhase<T>(phase: ExecutionPhase, work: () => T | Promise<T>, counts?: { fileCount?: number; bytes?: number }): Promise<T> {
|
||||
const owned = ownership.getStore();
|
||||
if (!endpointEnabled() || !owned || owned.ended) return work();
|
||||
const count = owned.details.get(phase) ?? 0;
|
||||
if (count >= 16) return work();
|
||||
owned.details.set(phase, count + 1);
|
||||
// A startup step can be more specific than the owning run phase. Keep that
|
||||
// causal parent while the child publishes its own context to plugin workers.
|
||||
return measure(owned.runId, phase, work, false, counts, getActiveStepContext()?.parentContext as Context | undefined);
|
||||
export function measureRunPhase<T>(phase: ExecutionPhase, work: () => T | Promise<T>, counts?: { fileCount?: number; bytes?: number }): Promise<T> {
|
||||
return withDatabaseOperation(phase, async () => {
|
||||
const owned = ownership.getStore();
|
||||
if (!endpointEnabled() || !owned || owned.ended) return work();
|
||||
const count = owned.details.get(phase) ?? 0;
|
||||
if (count >= 16) return work();
|
||||
owned.details.set(phase, count + 1);
|
||||
// A startup step can be more specific than the owning run phase. Keep that
|
||||
// causal parent while the child publishes its own context to plugin workers.
|
||||
return measure(owned.runId, phase, work, false, counts, getActiveStepContext()?.parentContext as Context | undefined);
|
||||
});
|
||||
}
|
||||
|
||||
export function beginRunPhase(phase: ExecutionPhase): (failed?: boolean) => void {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { heartbeatRuns, issues, type Db } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { isAcknowledgedNativeStop } from "../acknowledged-native-stop.js";
|
||||
@@ -31,7 +31,7 @@ describe.skipIf(!runControlPostgresSupport.supported)("native Stop evidence read
|
||||
contextSnapshot: { savedInput: bulk }, runnerProfileJson: { savedInput: bulk }, stdoutExcerpt: bulk,
|
||||
stderrExcerpt: bulk, resultJson: result }).returning();
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
let evidence: Awaited<ReturnType<typeof readNativeStopExecutionEvidence>>;
|
||||
try { evidence = await readNativeStopExecutionEvidence(db, company.id, id); }
|
||||
@@ -68,7 +68,7 @@ describe.skipIf(!runControlPostgresSupport.supported)("native Stop evidence read
|
||||
runtimeMode: "native", status: "cancelled" }).returning();
|
||||
await db.execute(sql`update ${heartbeatRuns} set result_json = ${JSON.stringify(saved)}::jsonb where id = ${run.id}`);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const evidence = await readNativeStopExecutionEvidence(db, company.id, run.id);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
agents,
|
||||
@@ -186,13 +186,13 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
stdoutExcerpt: payload, stderrExcerpt: payload,
|
||||
}).returning();
|
||||
const observed = createDb(database.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const state = extracted.createHeartbeatRunState(observed);
|
||||
const read = await state.getRunExecutionWait(companyId, run.id);
|
||||
capture.flush();
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBe(2);
|
||||
expect(summary.responseValueBytes).toBeLessThan(4096);
|
||||
expect(read).toMatchObject({ id: run.id, companyId, agentId: agent.id, status: "cancelled", contextSnapshot: context,
|
||||
@@ -228,7 +228,7 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
resultJson: { stdout: payload }, stdoutExcerpt: payload, stderrExcerpt: payload,
|
||||
}).returning();
|
||||
const observed = createDb(database.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const state = extracted.createHeartbeatRunState(observed);
|
||||
@@ -236,7 +236,7 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
id: run.id, companyId, scopeKind: "issue", issueId: issue.id, executionWorkspaceId: workspaceId,
|
||||
});
|
||||
capture.flush();
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBe(1);
|
||||
expect(summary.responseValueBytes).toBeLessThan(1024);
|
||||
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]).toEqual(run);
|
||||
@@ -260,7 +260,7 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
}).returning();
|
||||
const observed = createDb(database.connectionString);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns }, capture);
|
||||
try {
|
||||
const state = extracted.createHeartbeatRunState(observed);
|
||||
@@ -270,7 +270,7 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
runtimeModeResolvedAt: null, startedAt: run.startedAt, createdAt: run.createdAt,
|
||||
contextSnapshot: { issueId: issue.id }, runnerProfileJson: { adapterDispatch: { adapterType: "paperclip_runner" } } });
|
||||
capture.flush();
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBe(2);
|
||||
expect(summary.responseValueBytes).toBeLessThan(2048);
|
||||
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]).toEqual(run);
|
||||
@@ -321,7 +321,7 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
await state.getRunWorkspaceSettlement(run.id);
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
expect(await state.getRunCleanupState(run.id)).toEqual({
|
||||
id: run.id, companyId, agentId: agent.id, runtimeMode: run.runtimeMode,
|
||||
@@ -334,7 +334,7 @@ describePostgres("heartbeat run state database wiring", () => {
|
||||
contextSnapshot: { timerClaimWasFirstHeartbeat: true },
|
||||
});
|
||||
capture.flush();
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBe(2);
|
||||
expect(summary.responseValueBytes).toBeLessThan(2000);
|
||||
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]).toEqual(run);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { heartbeatRuns, type Db } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { expect } from "vitest";
|
||||
|
||||
@@ -21,11 +21,11 @@ export async function verifyCommittedReplayPayloadReads(
|
||||
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try { await replay(); }
|
||||
finally { capture.flush(); }
|
||||
const summary = events.find(event => event.event === "paperclip.db_response_capture.summary.v1")!;
|
||||
const summary = events.find(event => event.event === "paperclip.test_db_response_capture.summary.v1")!;
|
||||
expect(summary.completedQueries).toBeGreaterThan(0);
|
||||
expect(summary.responseValueBytes).toBeLessThan(100_000);
|
||||
const [after] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
|
||||
|
||||
@@ -29,7 +29,7 @@ import {
|
||||
toolApplications,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
|
||||
import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
import { createLocalDiskStorageProvider } from "../../storage/local-disk-provider.js";
|
||||
@@ -322,7 +322,7 @@ describe("native same-conversation chat attachment reuse", () => {
|
||||
const deliveries = await db.select().from(chatDeliveries).where(eq(chatDeliveries.endpointId, endpointId));
|
||||
const bulk = randomBytes(64 * 1024).toString("base64");
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
let capture: ReturnType<typeof createDatabaseResponseCapture> | undefined;
|
||||
let capture: ReturnType<typeof createFullDatabaseResponseCapture> | undefined;
|
||||
try {
|
||||
await db.update(chatEndpoints).set({ communicationInstructions: bulk,
|
||||
setup: { ...endpoint.setup, unusedContent: bulk } as typeof endpoint.setup,
|
||||
@@ -335,7 +335,7 @@ describe("native same-conversation chat attachment reuse", () => {
|
||||
.where(eq(chatDeliveries.id, delivery.id));
|
||||
}
|
||||
const now = Date.now();
|
||||
capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { chatMessageLinks, chatDeliveries, chatConversations, chatEndpoints,
|
||||
chatEndpointResources, chatExternalPrincipals, chatIdentityLinks, companyMemberships }, capture);
|
||||
for (const lockMode of ["read", "blocking", "nonblocking"] as const) {
|
||||
|
||||
@@ -8,7 +8,7 @@ import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import { and, eq, ne, sql } from "drizzle-orm";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
agentWakeupRequests,
|
||||
@@ -922,7 +922,7 @@ describe("native external-chat response wait", () => {
|
||||
await authorizeChatConversationForBoundRun(db, fixture, contextSnapshot, "read");
|
||||
const observed = createDb(temporary?.connectionString ?? externalTestDatabaseUrl!);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { issues, heartbeatRuns }, capture);
|
||||
try {
|
||||
expect(await listExternalReviewWaits(observed, fixture.companyId, [fixture.issueId])).toEqual(governed ? [] : [
|
||||
@@ -5462,7 +5462,7 @@ describe("native external-chat response wait", () => {
|
||||
if (reply) await db.update(chatActions).set({ result: { ...reply.result, savedProviderOutput: payload } }).where(eq(chatActions.id, reply.id));
|
||||
const observed = createDb(temporary?.connectionString ?? externalTestDatabaseUrl!);
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns, chatActions }, capture);
|
||||
try {
|
||||
const context = await new PaperclipRunnerToolAuthority(observed, fixture)
|
||||
|
||||
@@ -19,7 +19,7 @@ import {
|
||||
issueThreadInteractions,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { startRecoveryPostgres } from "../../__tests__/helpers/recovery-postgres.js";
|
||||
import { initializeRunIdentity, reserveSteeredIdentity, reconcileSteeredIdentity } from "../run-identity.js";
|
||||
import { documentService } from "../documents.js";
|
||||
@@ -248,7 +248,7 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
}).where(eq(heartbeatRuns.id, runId));
|
||||
const [expanded] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
|
||||
const observed = createDb(temporary!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns, issues, agents, companies }, capture);
|
||||
try {
|
||||
const reader = new PaperclipRunnerToolAuthority(observed, binding);
|
||||
@@ -291,7 +291,7 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
await db.update(heartbeatRuns).set({ contextSnapshot: context }).where(eq(heartbeatRuns.id, runId));
|
||||
const [expanded] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
|
||||
const observed = createDb(temporary!.connectionString), events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(observed, { heartbeatRuns, issues, agents, companies }, capture);
|
||||
try {
|
||||
const reader = new PaperclipRunnerToolAuthority(observed, binding);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { agents, companies, createDb, issueRecoveryActions, issues } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
@@ -60,7 +60,7 @@ describe.skipIf(!support.supported)("recovery issue state reads", () => {
|
||||
const service = recoveryService(db, { enqueueWakeup });
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { issues, issueRecoveryActions }, capture);
|
||||
try {
|
||||
await service.reconcileStrandedAssignedIssues();
|
||||
@@ -100,7 +100,7 @@ describe.skipIf(!support.supported)("recovery issue state reads", () => {
|
||||
const service = recoveryService(db, { enqueueWakeup });
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { agents, issues, issueRecoveryActions }, capture);
|
||||
try {
|
||||
expect(await service.reconcileStrandedAssignedIssues()).toMatchObject({ skipped: 1 });
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { agents, companies, createDb, heartbeatRuns } from "@paperclipai/db";
|
||||
import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
|
||||
import { createFullDatabaseResponseCapture, installDatabaseResponseCapture } from "../../__tests__/helpers/full-database-response-capture.js";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
@@ -37,7 +37,7 @@ describe.skipIf(!support.supported)("legacy disposition source reads", () => {
|
||||
errorCode: "adapter_failed", resultJson: { executionRecovery: { providerWorkStarted: true }, stdout: bulk } }).returning();
|
||||
const events: Array<Record<string, unknown>> = [];
|
||||
const now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try {
|
||||
expect(await readLegacyDispositionSource(db, { companyId, issueId, runId: before.id }))
|
||||
@@ -77,7 +77,7 @@ describe.skipIf(!support.supported)("legacy disposition source reads", () => {
|
||||
const enqueueWakeup = vi.fn(async () => { throw new Error("native work must not enter legacy recovery"); });
|
||||
const service = recoveryService(db, { enqueueWakeup });
|
||||
const events: Array<Record<string, unknown>> = [], now = Date.now();
|
||||
const capture = createDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 100, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { heartbeatRuns }, capture);
|
||||
try { expect(await service.reconcileLegacyContinuation(before.id)).toBe("skipped"); }
|
||||
finally { capture.flush(); }
|
||||
|
||||
Reference in new issue
Block a user