mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 05:11:49 +01:00
perf(workspaces): probe workspace run bindings through indexes (#20568)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The terminal workspace sweep checks each Git workspace for recent activity and for live consumers before it expires or moves the workspace > - These checks find the runs of a workspace with one SQL scope, `expirationScope`. The scope used one `OR` over all run bindings, and one branch decoded the context JSON of each run > - PostgreSQL could not use an index for that `OR`. Each check read every run of the company and its large context from TOAST. On a hosted database, each call took 2 to 5 seconds, about 5,200 s in total in one window > - This pull request splits the scope into one index probe per binding and adds one concurrent expression index for the legacy `executionWorkspaceId` context key > - The benefit is that each check reads a few index entries instead of all company runs. The cleanup decisions do not change ## Linked Issues or Issue Description **What existing behavior does this improve?** The workspace activity and cleanup-safety checks: `gitWorkspaceLastActivityAt`, `isGitWorkspaceExpirationEligible`, `workspaceMoveEligible` and the orphan-folder consumer check. All of them use `expirationScope` in `server/src/services/git-workspace-expiration.ts`. **Subsystem affected** Execution workspace cleanup in the server (`git-workspace-expiration.ts`, `isolated-repository-rehome.ts`, `isolated-task-directory-retention.ts` through the shared scope) and the `heartbeat_runs` schema in `packages/db`. **Current behavior** The `relevant_runs` CTE selects company runs where any of these is true: `issue_id` or `native_issue_id` is in the issue tree or the bound issues, the run is a bound issue's checkout or execution run, or the context JSON has a matching `issueId`, `taskId` or `executionWorkspaceId`. The planner does one sequential scan of the company runs and reads each context from TOAST. The cost grows with the run history of the company, not with the size of the workspace. **Proposed behavior** Each binding is a separate index probe. The `UNION` of run IDs goes to a primary-key probe. The SQL gives the same rows. **Reason and benefit** Local SQL measurements show a 34x to 169x decrease in time. The decrease grows with the run history (see Verification). ## What Changed - `expirationScope` builds `relevant_runs` from a `UNION` of index probes, one for each binding. The key sets go in as arrays (`= ANY(ARRAY(...))`). With `IN (subquery)`, the planner chose a hash join over a full scan, because it does not count the cost of reading TOASTed contexts. - The legacy context keys use `context_snapshot ->> 'key'`. This expression matches the existing indexes `heartbeat_runs_company_ctx_issue_created_idx` and `heartbeat_runs_company_ctx_task_created_idx`. For a non-object context, `->>` gives `NULL`, as the strict JSONPath decode did. Such a run can still match through a native binding. - New partial index `heartbeat_runs_company_ctx_execution_workspace_idx` on `(company_id, context_snapshot ->> 'executionWorkspaceId') WHERE context_snapshot ? 'executionWorkspaceId'`. The query repeats the predicate. For an array context, `?` can be true, but `->>` gives `NULL`, so the result does not change. The index is partial so that it serves only this probe. A full index that starts with `company_id` made the planner choose it for an unrelated company-scoped query. Migration 0460 builds it with `CREATE INDEX CONCURRENTLY IF NOT EXISTS`. The migration is in the list of migrations that run outside a transaction. - New tests: a legacy `issueId` or `executionWorkspaceId` binding on a run that is bound to a different issue still blocks cleanup. A plan test with `enable_seqscan = off` checks that no binding scans `heartbeat_runs` and that the three context indexes are used. - The benchmark script `scripts/benchmark-workspace-activity.ts` gets a `--runs-per-workspace` option. Unrelated fixture runs now have distinct context IDs, as real history does. - `doc/DATABASE.md` describes the probes and the new index. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/git-workspace-activity.test.ts src/__tests__/native-harness-backup-expiration.test.ts`: 132 of 132 tests pass. The second file checks that the native session index stays the planner's choice for its query. These include the existing route matrix (root, child, bound, native, context issue, context task, context workspace, checkout, execution run), exactly-once counts, malformed and uppercase contexts, company isolation, controller leases and lease vetoes. - `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter @paperclipai/db check:migrations` pass. - Benchmark (local SQL measurement, PostgreSQL 18.1, synthetic fixture, 8 workspaces per sample, before and after alternate in each iteration). The script compares the complete ordered caller results and the complete `relevant_runs` multisets of the baseline and treatment. They are equal in every run. `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-activity.ts --baseline-ref=origin/master --runs-per-workspace=<N>` | Fixture | Check | Before median (min–max) | After median (min–max) | Speedup | | --- | --- | --- | --- | --- | | 4,800 runs, 330 MB | last activity | 523 ms (460–598) | 12 ms (8–15) | 44x | | 4,800 runs, 330 MB | eligibility | 452 ms (432–473) | 13 ms (11–15) | 34x | | 4,800 runs, 330 MB | idle eligibility | 465 ms (442–573) | 12 ms (10–13) | 40x | | 24,000 runs, 1.7 GB | last activity | 2,240 ms (2,214–2,454) | 13 ms (11–14) | 169x | | 24,000 runs, 1.7 GB | eligibility | 2,245 ms (2,097–2,377) | 17 ms (14–19) | 133x | | 24,000 runs, 1.7 GB | idle eligibility | 2,196 ms (2,179–2,381) | 15 ms (13–18) | 143x | Before: one `Seq Scan` on `heartbeat_runs`. After: `heartbeat_runs_company_issue_created_idx`, `heartbeat_runs_company_native_issue_contract_id_uq`, the three context indexes and `heartbeat_runs_pkey`. - Index build time: on a synthetic table with 200,000 runs and 8 KiB contexts (1.8 GB), the full form of this index took 3.7 s to build concurrently and used 15 MB. The partial index reads the same rows and holds only runs that have the key. ## Risks - Cleanup safety: a wrong "allowed" result could delete an active workspace. The scope keeps every binding of the old query. The benchmark checks exact equality of the run sets, and the existing tests check each binding route and each veto. - Migration: the index build reads every run context once. It runs concurrently and does not block writes. Until migration 0460 builds the index, the new query is still correct but the `executionWorkspaceId` probe scans. - Plan choice: the array form makes index probes the cheapest plan for a small tree. A very large issue tree gives a large array, but each element is still one index probe. - The sweep frequency does not change. This pull request reduces only the cost of each check. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed
1 parent
17136a6c25
commit
1805a1d247
9 files changed
+264
-25
No files matched your search
@@ -422,6 +422,23 @@ local fixture:
|
||||
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-cancellation-scan.ts --runs=20000 --payload-kib=16
|
||||
```
|
||||
|
||||
Workspace activity and cleanup checks (`expirationScope` in
|
||||
`server/src/services/git-workspace-expiration.ts`) find a workspace's runs with
|
||||
one index probe per binding: `issue_id`, `native_issue_id`, the bound issues'
|
||||
checkout and execution runs, and the legacy context keys `issueId`, `taskId`
|
||||
and `executionWorkspaceId`. Migration 0460 builds
|
||||
`heartbeat_runs_company_ctx_execution_workspace_idx` concurrently for the last
|
||||
key. The index is partial (`context_snapshot ? 'executionWorkspaceId'`), so it
|
||||
serves only queries that repeat that predicate and does not compete with other
|
||||
company-scoped indexes. The context expressions and the predicate must stay
|
||||
textually equal to the index definitions; a regression test checks that no
|
||||
binding scans `heartbeat_runs`.
|
||||
To compare both plans on a disposable local fixture:
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-activity.ts --baseline-ref=<pre-fix commit> --runs-per-workspace=3000
|
||||
```
|
||||
|
||||
GitHub review lookups use four indexes from migration 0456, which builds them
|
||||
concurrently. `chat_message_links_company_comment_idx` finds a source comment's
|
||||
link for inherited review waits and run-origin checks.
|
||||
|
||||
@@ -622,6 +622,7 @@ const MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS = new Set([
|
||||
"0457_native_session_run_index.sql",
|
||||
"0458_chat_admission_issue_index.sql",
|
||||
"0459_environment_lease_backup_scope_index.sql",
|
||||
"0460_workspace_activity_context_index.sql",
|
||||
|
||||
|
||||
"0384_funny_adam_warlock.sql",
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS "heartbeat_runs_company_ctx_execution_workspace_idx" ON "heartbeat_runs" USING btree ("company_id",("context_snapshot" ->> 'executionWorkspaceId')) WHERE "heartbeat_runs"."context_snapshot" ? 'executionWorkspaceId';
|
||||
+177
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"id": "2be53987-902a-41b6-8cb4-6e89a768f8cf",
|
||||
"prevId": "f98af245-4a2e-4c3d-b89e-6ed3e81a074c",
|
||||
"id": "f93ccf18-7e86-4788-be77-b17430c86b2f",
|
||||
"prevId": "e070677d-cfb8-4438-847c-29c4da7ca8c5",
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"tables": {
|
||||
@@ -10188,6 +10188,40 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"chat_actions_company_issue_kind_idx": {
|
||||
"name": "chat_actions_company_issue_kind_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "(\"payload\"->>'issueId')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "kind",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "status",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "\"chat_actions\".\"kind\" IN ('inbound_wakeup', 'failed_run_retry')",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"chat_actions_github_stale_head_idx": {
|
||||
"name": "chat_actions_github_stale_head_idx",
|
||||
"columns": [
|
||||
@@ -10955,6 +10989,28 @@
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"chat_deliveries_github_automatic_delivery_idx": {
|
||||
"name": "chat_deliveries_github_automatic_delivery_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "endpoint_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "(\"normalized_event\"->'githubAutomatic'->'context'->>'deliveryId')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "(\"chat_deliveries\".\"normalized_event\"->'githubAutomatic'->'context'->>'deliveryId') is not null",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
@@ -12393,6 +12449,44 @@
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"chat_message_links_company_comment_idx": {
|
||||
"name": "chat_message_links_company_comment_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "comment_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "\"chat_message_links\".\"comment_id\" is not null",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"chat_message_links_delivery_idx": {
|
||||
"name": "chat_message_links_delivery_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "delivery_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "\"chat_message_links\".\"delivery_id\" is not null",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
@@ -29073,6 +29167,22 @@
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"environment_leases_native_backup_scope_idx": {
|
||||
"name": "environment_leases_native_backup_scope_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "(\"metadata\"->'nativeHarnessBackup'->>'sessionScopeSha256')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "(\"environment_leases\".\"metadata\"->'nativeHarnessBackup'->>'sessionScopeSha256') is not null",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
@@ -35475,6 +35585,27 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_company_wake_comment_idx": {
|
||||
"name": "heartbeat_runs_company_wake_comment_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "(\"context_snapshot\"->>'wakeCommentId')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_cost_accounting_pending_idx": {
|
||||
"name": "heartbeat_runs_cost_accounting_pending_idx",
|
||||
"columns": [
|
||||
@@ -35627,6 +35758,28 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_company_native_session_idx": {
|
||||
"name": "heartbeat_runs_company_native_session_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "native_session_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "\"heartbeat_runs\".\"native_session_id\" is not null",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_company_issue_created_idx": {
|
||||
"name": "heartbeat_runs_company_issue_created_idx",
|
||||
"columns": [
|
||||
@@ -35864,6 +36017,28 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_company_ctx_execution_workspace_idx": {
|
||||
"name": "heartbeat_runs_company_ctx_execution_workspace_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "(\"context_snapshot\" ->> 'executionWorkspaceId')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "\"heartbeat_runs\".\"context_snapshot\" ? 'executionWorkspaceId'",
|
||||
"concurrently": true,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_company_ctx_paperclip_issue_created_idx": {
|
||||
"name": "heartbeat_runs_company_ctx_paperclip_issue_created_idx",
|
||||
"columns": [
|
||||
+7
@@ -3207,6 +3207,13 @@
|
||||
"when": 1794329161946,
|
||||
"tag": "0459_environment_lease_backup_scope_index",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 460,
|
||||
"version": "7",
|
||||
"when": 1794332988056,
|
||||
"tag": "0460_workspace_activity_context_index",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -258,6 +258,14 @@ export const heartbeatRuns = pgTable(
|
||||
sql`(${table.contextSnapshot} ->> 'taskKey')`,
|
||||
table.createdAt.desc(),
|
||||
),
|
||||
// Runs that recorded an execution workspace in their legacy context. Workspace
|
||||
// activity and cleanup checks probe it instead of decoding every run context.
|
||||
// Partial, so it serves only queries that repeat the predicate and never
|
||||
// competes with other company-scoped indexes.
|
||||
companyCtxExecutionWorkspaceIdx: index("heartbeat_runs_company_ctx_execution_workspace_idx")
|
||||
.on(table.companyId, sql`(${table.contextSnapshot} ->> 'executionWorkspaceId')`)
|
||||
.where(sql`${table.contextSnapshot} ? 'executionWorkspaceId'`)
|
||||
.concurrently(),
|
||||
companyCtxPaperclipIssueCreatedIdx: index("heartbeat_runs_company_ctx_paperclip_issue_created_idx")
|
||||
.on(table.companyId, sql`(${table.contextSnapshot} -> 'paperclipIssue' ->> 'id')`, table.createdAt.desc())
|
||||
.concurrently(),
|
||||
|
||||
@@ -24,7 +24,9 @@ const sha256 = (source: string) => createHash("sha256").update(source).digest("h
|
||||
const baselineOnly = process.argv.includes("--baseline-only");
|
||||
const dockerPostgres = process.argv.includes("--docker-postgres=17");
|
||||
const small = process.argv.includes("--small");
|
||||
const runsPerWorkspace = small ? 16 : 600;
|
||||
const runsPerWorkspace = Number(process.argv.find(arg => arg.startsWith("--runs-per-workspace="))?.slice(21) ?? (small ? 16 : 600));
|
||||
assert.ok(Number.isInteger(runsPerWorkspace) && runsPerWorkspace >= 16 && runsPerWorkspace <= 20000,
|
||||
"--runs-per-workspace must be an integer between 16 and 20000");
|
||||
const bindingRows = Math.max(1, Math.floor(runsPerWorkspace / 20));
|
||||
const descendantsPerWorkspace = Number(process.argv.find(arg => arg.startsWith("--descendants="))?.slice(14) ?? 1);
|
||||
assert.ok(Number.isInteger(descendantsPerWorkspace) && descendantsPerWorkspace >= 1 && descendantsPerWorkspace <= 500,
|
||||
@@ -114,7 +116,8 @@ try {
|
||||
title: `Child ${n}/${child}`, status: "done", completedAt: old, updatedAt: old })));
|
||||
}
|
||||
const padding = randomBytes(24 * 1024).toString("base64");
|
||||
// Most history is unrelated, as in the reported scan; contexts and results are
|
||||
// Most history is unrelated, as in the reported scan, and unrelated runs carry
|
||||
// distinct legacy issue, task and workspace IDs as real history does; contexts and results are
|
||||
// incompressible and wide enough to live in TOAST. Existing indexes are retained.
|
||||
for (let n = 0; n < 8; n++) {
|
||||
const workspace = workspaces[n];
|
||||
@@ -122,9 +125,9 @@ try {
|
||||
SELECT $1::uuid, $2::uuid, 'succeeded', CASE WHEN i <= ${bindingRows} THEN 'issue' ELSE 'company' END,
|
||||
CASE WHEN i <= ${bindingRows} THEN $3::uuid END, CASE WHEN i > ${bindingRows} AND i <= ${bindingRows * 2} THEN $3::uuid END,
|
||||
$6::timestamptz + i * interval '1 second', $6::timestamptz + i * interval '2 seconds', $6::timestamptz + i * interval '3 seconds',
|
||||
jsonb_build_object('issueId', CASE WHEN i > ${bindingRows * 2} AND i <= ${bindingRows * 3} THEN $3::text ELSE $5::text END,
|
||||
'taskId', CASE WHEN i > ${bindingRows * 3} AND i <= ${bindingRows * 4} THEN $3::text ELSE $5::text END,
|
||||
'executionWorkspaceId', CASE WHEN i > ${bindingRows * 4} AND i <= ${bindingRows * 5} THEN $4::text ELSE 'unrelated' END,
|
||||
jsonb_build_object('issueId', CASE WHEN i > ${bindingRows * 2} AND i <= ${bindingRows * 3} THEN $3::text WHEN i % 50 = 0 THEN $5::text ELSE gen_random_uuid()::text END,
|
||||
'taskId', CASE WHEN i > ${bindingRows * 3} AND i <= ${bindingRows * 4} THEN $3::text ELSE gen_random_uuid()::text END,
|
||||
'executionWorkspaceId', CASE WHEN i > ${bindingRows * 4} AND i <= ${bindingRows * 5} THEN $4::text ELSE gen_random_uuid()::text END,
|
||||
'fixtureRun', ${n * runsPerWorkspace} + i, 'padding', $7::text),
|
||||
jsonb_build_object('stdout', $7::text)
|
||||
FROM generate_series(1, ${runsPerWorkspace}) i`, [companyId, agentId, workspace.sourceIssueId, workspace.id, unrelatedIssueId, old.toISOString(), padding]);
|
||||
|
||||
@@ -72,6 +72,32 @@ describeDatabase("Git workspace activity bindings", () => {
|
||||
expect(await gitWorkspaceLastActivityAt(db, f.workspace)).toEqual(recent);
|
||||
});
|
||||
|
||||
it.each(["issueId", "executionWorkspaceId"])("keeps a legacy %s binding on a run bound to another issue", async key => {
|
||||
const f = await fixture();
|
||||
const [unrelated] = await db.insert(issues).values({ companyId: f.companyId, title: "Unrelated", updatedAt: old }).returning();
|
||||
await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, scopeKind: "issue", issueId: unrelated.id,
|
||||
nativeIssueId: unrelated.id, status: "running", createdAt: recent,
|
||||
contextSnapshot: { [key]: key === "issueId" ? f.child.id : f.workspace.id } });
|
||||
expect(await f.eligible()).toBe(false);
|
||||
expect(await workspaceMoveEligible(db, f.workspace)).toBe(false);
|
||||
expect(await gitWorkspaceLastActivityAt(db, f.workspace)).toEqual(recent);
|
||||
});
|
||||
|
||||
it("reaches every run binding through an index", async () => {
|
||||
const f = await fixture();
|
||||
// With sequential scans disabled, a binding expression that no index
|
||||
// matches still forces a sequential scan of heartbeat_runs.
|
||||
const plan = await db.transaction(async tx => {
|
||||
await tx.execute(sql`SET LOCAL enable_seqscan = off`);
|
||||
return tx.execute<{ "QUERY PLAN": string }>(sql`EXPLAIN ${expirationScope(f.workspace)} SELECT id FROM relevant_runs`);
|
||||
});
|
||||
const text = plan.map(row => row["QUERY PLAN"]).join("\n");
|
||||
expect(text).not.toMatch(/Seq Scan on heartbeat_runs/);
|
||||
expect(text).toMatch(/heartbeat_runs_company_ctx_execution_workspace_idx/);
|
||||
expect(text).toMatch(/heartbeat_runs_company_ctx_issue_created_idx/);
|
||||
expect(text).toMatch(/heartbeat_runs_company_ctx_task_created_idx/);
|
||||
});
|
||||
|
||||
it("includes a multiply bound run exactly once", async () => {
|
||||
const f = await fixture(), runId = randomUUID();
|
||||
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, scopeKind: "issue", issueId: f.issueId,
|
||||
|
||||
@@ -104,9 +104,16 @@ async function restorePointBoundary(cwd: string, repoUrl: string, headOnRemote:
|
||||
|
||||
/** Re-read the whole tree and every runtime/recovery consumer immediately before deletion. */
|
||||
export function expirationScope(workspace: Workspace) {
|
||||
// Decode the three legacy keys together only after cheap native/pointer
|
||||
// matches fail. Strict JSONPath leaves non-object inputs unmatched, as ->>
|
||||
// did, while those inputs can still match an independent native binding.
|
||||
// Each binding is a separate index probe. A single OR over all bindings made
|
||||
// PostgreSQL read every company run and decode its context JSON. Each key set
|
||||
// is passed as an array so the planner probes the index instead of hashing a
|
||||
// full scan; it does not count the cost of reading TOASTed contexts. The legacy
|
||||
// context expressions match the heartbeat_runs expression indexes exactly,
|
||||
// and the key test matches the partial workspace index predicate;
|
||||
// ->> yields NULL for non-object contexts, so those rows stay unmatched there
|
||||
// and can still match through an independent native binding. The UNION
|
||||
// removes duplicate IDs, so the outer primary-key probe keeps each run once.
|
||||
const companyRuns = sql`SELECT id FROM ${heartbeatRuns} WHERE company_id = ${workspace.companyId}`;
|
||||
return sql`
|
||||
WITH RECURSIVE tree AS (
|
||||
SELECT id, status, completed_at, cancelled_at, updated_at FROM ${issues}
|
||||
@@ -117,23 +124,17 @@ export function expirationScope(workspace: Workspace) {
|
||||
), bound AS (
|
||||
SELECT id, updated_at, checkout_run_id, execution_run_id FROM ${issues}
|
||||
WHERE company_id = ${workspace.companyId} AND execution_workspace_id = ${workspace.id}
|
||||
), legacy_issue_ids AS MATERIALIZED (
|
||||
SELECT unnest(array_agg(id::text)) AS id FROM tree
|
||||
), relevant_runs AS (
|
||||
SELECT r.id, r.status, r.created_at, r.started_at, r.finished_at, r.controller_lease_expires_at
|
||||
FROM ${heartbeatRuns} r WHERE r.company_id = ${workspace.companyId} AND (
|
||||
r.issue_id IN (SELECT id FROM tree) OR r.native_issue_id IN (SELECT id FROM tree)
|
||||
OR r.issue_id IN (SELECT id FROM bound) OR r.native_issue_id IN (SELECT id FROM bound)
|
||||
OR r.id IN (SELECT unnest(ARRAY[checkout_run_id, execution_run_id]) FROM bound)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM jsonb_to_record(
|
||||
jsonb_path_query_first(r.context_snapshot, 'strict $ ? (@.type() == "object")')
|
||||
) context("issueId" text, "taskId" text, "executionWorkspaceId" text)
|
||||
WHERE context."issueId" IN (SELECT id FROM legacy_issue_ids)
|
||||
OR context."taskId" IN (SELECT id FROM legacy_issue_ids)
|
||||
OR context."executionWorkspaceId" = ${workspace.id}
|
||||
)
|
||||
)
|
||||
FROM ${heartbeatRuns} r WHERE r.company_id = ${workspace.companyId} AND r.id = ANY(ARRAY(
|
||||
${companyRuns} AND issue_id = ANY(ARRAY(SELECT id FROM tree UNION SELECT id FROM bound))
|
||||
UNION ${companyRuns} AND native_issue_id = ANY(ARRAY(SELECT id FROM tree UNION SELECT id FROM bound))
|
||||
UNION SELECT unnest(ARRAY[checkout_run_id, execution_run_id]) FROM bound
|
||||
UNION ${companyRuns} AND (context_snapshot ->> 'issueId') = ANY(ARRAY(SELECT id::text FROM tree))
|
||||
UNION ${companyRuns} AND (context_snapshot ->> 'taskId') = ANY(ARRAY(SELECT id::text FROM tree))
|
||||
UNION ${companyRuns} AND context_snapshot ? 'executionWorkspaceId'
|
||||
AND (context_snapshot ->> 'executionWorkspaceId') = ${workspace.id}
|
||||
))
|
||||
)`;
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user