Files
PaperClipAI/server/src/services/issue-creation-origin.ts
T
DottaandClaude Opus 5.5 8e4c5e81b9 perf(issues): read created-from tasks by index and inbox review waits by page (#20573)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The task page shows "Created from this task". The inbox shows
blocked-inbox attention for a page of tasks.
> - On a hosted database, the created-from list ran about 4,000 times in
one afternoon at about 499 ms each. The task page sends it each time a
task opens.
> - Its filter was an `OR` of three provenance paths over the list's own
rows. PostgreSQL could not use an index for it. It read every task of
the company and tested each one, with the list's other per-row
conditions (such as the chat conversation state subquery).
> - The blocked-inbox attention map also read external review waits for
every in-review task of the company, not only for the tasks of the page.
> - This pull request reads created-from tasks from three index probes,
and reads review waits only for the in-review tasks that can change a
finding of the page.
> - The benefit is less database work on each task open and each inbox
page. The responses stay the same.

## Linked Issues or Issue Description

Refs #20400 (shares review-wait reads between the attention maps of one
request).
Refs #20551 (authorizes the review waits of a page with one read; this
change narrows the waits that the page asks for, and adds a test that
pins the batched read).
Refs #13241 (adds the created-from list).

**What happened**

- `GET /api/companies/:id/issues?createdFromIssueId=…` used
`createdFromIssueCondition`. Its filter was `origin_issue_id = source OR
(origin_issue_id IS NULL AND origin_run_id IN source runs) OR (… AND
EXISTS creation activity of a source run)`. The `OR` stops index use, so
the statement read every task of the company. The list's other
conditions, which include the chat conversation state subquery, ran for
each task before the task was excluded.
- `listIssueBlockedInboxAttentionMap` read review waits for every
in-review task of the company graph. The caller uses the findings of the
page rows only.

**Expected behavior**

The created-from list reads only the created tasks. The blocked-inbox
map reads review waits only for in-review tasks that the liveness
classifier can read for a page row. The rows, the order, and the
attention values do not change.

**Steps to reproduce**

1. Use a company with tens of thousands of tasks, runs, and activity
rows.
2. Open a task page. Look at the created-from list statement: it reads
the full task set of the company.
3. Open an inbox page with blocked-inbox attention. Look at the
review-wait read: it gets the ID of every in-review task of the company.

## What Changed

- `server/src/services/issue-creation-origin.ts`:
`createdFromIssueCondition` is now `issues.id <> source AND issues.id IN
(three branches)`. Each branch starts from an index:
  - recorded origin task: `issues_company_origin_issue_idx`;
- legacy origin run: the source task's runs
(`heartbeat_runs_company_origin_issue_idx`), then the new partial index
on `issues (company_id, origin_run_id)`;
- creation activity: `activity_log_run_id_idx` for the source runs, then
the task by primary key.
The precedence does not change: a recorded origin task wins, the run
path applies only without one, and the activity path applies only
without both. The activity path matches only an `entity_id` that is the
canonical text of the task ID, as before (it also compares `id::text =
entity_id`).
- `packages/db`: add `issues_company_legacy_origin_run_idx` on `issues
(company_id, origin_run_id) WHERE origin_issue_id IS NULL AND
origin_run_id IS NOT NULL`. Migration `0461` creates it with `CREATE
INDEX CONCURRENTLY IF NOT EXISTS` and is registered with the other
concurrent migrations. `doc/DATABASE.md` documents the index and the
benchmark command.
- `server/src/services/issues.ts`: add `blockedInboxReviewWaitIssueIds`.
It returns the in-review tasks in the blocker closure of the page rows.
The blocked-inbox map reads review waits only for these tasks. The
liveness classifier reads a wait only for a row or for a blocker that it
reaches from a row, and the caller reads only the findings of the rows.
The classifier does not change.
- Tests:
- `issue-created-from-routes.test.ts`: new cases for precedence in both
directions, a source-run comment, an activity row whose task has another
origin run, a non-canonical and a non-UUID activity entity ID, the
source's own creation activity, and other-company rows.
- `blocked-inbox-review-wait-scope.test.ts`: the narrowed set for
chains, cycles, shared blockers and off-page tasks. The rows get the
same findings with the narrowed waits as with the waits of every
in-review task, and each narrowed wait is needed.
- `issue-list-review-wait-authorization.integration.test.ts`: on real
PostgreSQL with two authorized waits, `listExternalReviewWaits` and one
issue-list request with blocked-inbox attention each send one bound-run
message-link statement. With the per-wait authorization from before
#20551, both tests fail (2 statements).
- `scripts/benchmark-created-from-issues.ts`: local benchmark. It loads
the baseline `issues.ts` and `issue-creation-origin.ts` from a pinned
commit, asserts identical complete row payloads, and reports alternating
timings and `EXPLAIN (ANALYZE, BUFFERS)`.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/issue-created-from-routes.test.ts
src/__tests__/blocked-inbox-review-wait-scope.test.ts
src/__tests__/issue-list-review-wait-authorization.integration.test.ts
src/__tests__/agent-task-hierarchy-depth.test.ts
src/__tests__/issue-blocker-attention.test.ts
src/__tests__/issue-list-assignee-filter-routes.test.ts
src/__tests__/issues-parent-id-alias.test.ts
src/__tests__/batched-chat-authorization.integration.test.ts
src/__tests__/issue-liveness.test.ts` passes (126 tests). One timing
test in `issue-list-assignee-filter-routes` failed one time under
machine load and passed on rerun.
- `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter
@paperclipai/db check:migrations` pass.
- `pnpm verify` asks for broader verification because the change adds a
file under `scripts/`. Required CI supplies that coverage.
- Benchmark: `pnpm --filter @paperclipai/server exec tsx
../scripts/benchmark-created-from-issues.ts --baseline-ref=bd30ef8510`

Fixture (synthetic, local PostgreSQL 18.1): one company with 30,000
tasks (10,628 with a recorded origin task, 10,445 with only a legacy
origin run), 60,030 runs with wide context snapshots (145 MB), and
261,079 activity rows; a second company with 10,000 tasks and 20,000
runs. The busy source task has 63 created tasks through all three paths,
plus decoys. The quiet source task has 3. The UI's page: `sortField=id`,
`asc`, `limit=500`, routine executions included. Both versions return
identical complete payloads. 9 alternating pairs after 2 warm-up pairs:

| Case | List statement before | List statement after | List call
before, median (min to max) | List call after, median (min to max) |
| --- | --- | --- | --- | --- |
| Busy source (63 tasks) | 234.9 ms, 47,437 buffers | 0.90 ms, 437
buffers | 156.0 ms (114.5 to 480.7) | 31.6 ms (19.9 to 62.7) |
| Quiet source (3 tasks) | 90.7 ms, 47,413 buffers | 0.25 ms, 46 buffers
| 126.1 ms (90.6 to 300.4) | 10.5 ms (7.2 to 26.1) |

The list call time includes the enrichment reads (labels, runs,
attention), which do not change.

Inbox page (50 open tasks, blocked-inbox attention): the review-wait
read goes from 2 statements with 1,215 parameters (every in-review task
of the company) to 1 statement with 31 parameters. The page time does
not change in this fixture (median 801 ms before, 873 ms after, both
with a wide spread), because the fixture has no review waits and the
other graph reads of the map are most of the time. With real waits, the
server does an inherited GitHub review read for each wait it finds, so
the gain grows with the number of off-page waits.

Migration timing (local PostgreSQL 18.1): `0461` (measured as the same
statement under its earlier number) on 500,000 tasks (846 MB, 20
companies, one third with a legacy origin run) takes 1.6 s. The index is
12 MB.

## Risks

- The new index adds a small write cost to tasks with only an origin
run. New tasks with a source task record `origin_issue_id`, so the index
holds mostly older tasks. The migration does not lock writes.
- The created-from rewrite must keep the same rows. The new tests and
the benchmark payload check cover precedence, company scope, and ID text
forms. Visibility and read filters are the list's other conditions and
do not change.
- The review-wait narrowing depends on the liveness classifier reading
waits only for rows and their blocker chains. A unit test compares the
findings of the rows with full and narrowed waits. A future classifier
change that reads waits of other tasks must update
`blockedInboxReviewWaitIssueIds`. The function comment says this.
- The local timings are synthetic. Hosted acceptance needs a fresh
query-statistics window after deployment.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
file edits, local PostgreSQL) in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 12:38:51 -06:00

65 lines
3.2 KiB
TypeScript

import { sql } from "drizzle-orm";
import { alias } from "drizzle-orm/pg-core";
import { activityLog, heartbeatRuns, issues } from "@paperclipai/db";
import { isUuidLike } from "@paperclipai/shared";
import { unprocessable } from "../errors.js";
// The canonical text of a UUID. Only this form can equal `issues.id::text`.
const CANONICAL_UUID_PATTERN = "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$";
/** Creation provenance is independent of the task's current parent or project. */
export function createdFromIssueCondition(companyId: string, sourceIssueId: string) {
if (!isUuidLike(sourceIssueId)) {
throw unprocessable("createdFromIssueId must be a UUID");
}
const source = alias(issues, "creation_source_issue");
const created = alias(issues, "creation_candidate_issue");
// The source issue's runs, through heartbeat_runs_company_origin_issue_idx.
const originatingRuns = sql`
SELECT ${heartbeatRuns.id} FROM ${heartbeatRuns}
INNER JOIN ${issues} AS ${source} ON ${source.id} = ${sourceIssueId}
AND ${source.companyId} = ${companyId}
WHERE ${heartbeatRuns.companyId} = ${companyId}
AND coalesce(
${heartbeatRuns.nativeIssueId}::text,
nullif(${heartbeatRuns.contextSnapshot}->>'issueId', ''),
nullif(${heartbeatRuns.contextSnapshot}->>'taskId', ''),
nullif(${heartbeatRuns.contextSnapshot}->>'taskKey', '')
) IN (${source.id}::text, ${source.identifier})
`;
// A recorded origin task wins; run and activity fallbacks apply only to rows
// without one, so the list agrees with the detail's createdFrom.
//
// Each branch starts from an index (origin task, source runs, run activity)
// so the list reads only the created tasks. An OR over the list's own rows
// would test every task of the company, with the list's other per-row
// conditions, before it could exclude one.
return sql<boolean>`
${issues.id} <> ${sourceIssueId}
AND ${issues.id} IN (
SELECT ${created.id} FROM ${issues} AS ${created}
WHERE ${created.companyId} = ${companyId}
AND ${created.originIssueId} = ${sourceIssueId}
UNION ALL
SELECT ${created.id} FROM ${issues} AS ${created}
WHERE ${created.companyId} = ${companyId}
-- The predicate of issues_company_legacy_origin_run_idx.
AND ${created.originIssueId} is null and ${created.originRunId} is not null
AND ${created.originRunId} IN (SELECT source_run.id::text FROM (${originatingRuns}) AS source_run)
UNION ALL
SELECT ${created.id} FROM ${activityLog}
INNER JOIN ${issues} AS ${created}
ON ${created.id} = CASE WHEN ${activityLog.entityId} ~ ${CANONICAL_UUID_PATTERN}
THEN ${activityLog.entityId}::uuid END
AND ${created.id}::text = ${activityLog.entityId}
WHERE ${activityLog.companyId} = ${companyId}
AND ${activityLog.entityType} = 'issue'
AND ${activityLog.action} IN ('issue.created', 'issue.child_created')
AND ${activityLog.runId} IN (SELECT source_run.id FROM (${originatingRuns}) AS source_run)
AND ${created.companyId} = ${companyId}
AND ${created.originIssueId} IS NULL
AND ${created.originRunId} IS NULL
)
`;
}