mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
DottaandClaude Opus 5.5
17014cd7db
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Each finished agent run gets a liveness classification. The task
page shows it, and recovery reads it.
> - Run finalization writes the classification. A backfill repairs runs
that do not have it, for example older runs or runs whose finalization
failed.
> - The task page runs read (`GET /issues/:id/runs`) starts this
backfill on each read, for each task.
> - The candidate lookup reads every unclassified finished run in the
company. For each row, it reads the run context and probes
`activity_log`. Rows of other tasks never match, so each read repeats
the same work.
> - On a hosted test instance, this lookup used approximately 1 s per
call, 2,636 calls, and 3,500 s in total.
> - This pull request makes the lookup read only the rows of the task,
and skips a task after a pass finds nothing left to repair.
> - The benefit is less database work per task page read. The
classification still gets written.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The run liveness backfill that the task runs read starts
(`scheduleRunLivenessBackfill` in `server/src/services/activity.ts`).
**Subsystem affected**
Server: activity service, task runs read, run liveness backfill.
**Current behavior**
Each task runs read runs this lookup:
```sql
select ... from heartbeat_runs
where company_id = $1 and liveness_state is null and status not in ('queued', 'running')
and (context_snapshot ->> 'issueId' = $2
or exists (select 1 from activity_log where company_id = $3 and entity_type = 'issue'
and entity_id = $4 and run_id = heartbeat_runs.id))
limit 20
```
PostgreSQL reads all unclassified finished runs of the company through
`heartbeat_runs_company_liveness_idx`. It then reads the context of each
row, which can be large, and probes `activity_log` for each row.
Unclassified runs of other tasks, and runs without a task, stay
unclassified, because no read of their own task repairs them. Thus the
cost grows with the company, not with the task. The lookup also runs
again on each read when the task has nothing to repair.
**Proposed behavior**
- The lookup reads only the runs of this task. One branch uses
`heartbeat_runs_company_ctx_issue_created_idx` (context `issueId`). The
other branch uses `activity_log_company_entity_created_idx` (issue
activity rows with a run id). The result keeps the same filter: same
company, liveness missing, status not `queued` or `running`, at most 20
rows.
- When a pass finds fewer than 20 rows, it has repaired all rows of the
task. The service then skips the task for 10 minutes. A full batch of 20
does not start the skip window, so the next read repairs the next batch.
**Reason and benefit**
The lookup is now proportional to the history of one task. Most reads
find nothing, and those reads now do no database work for 10 minutes.
**Breaking changes**
None. No migration, no API change, no new background work.
**Additional context**
The chat bootstrap read already skips the backfill
(`skipLivenessBackfill`). That option still works, and a test covers it.
## What Changed
- `server/src/services/run-liveness-backfill-query.ts`: new
`missingRunLivenessCandidatesQuery`. It replaces the `OR` and the
correlated `exists` with `id in (context branch union all activity
branch)`. Both branches are filtered by company and task.
- `server/src/services/activity.ts`: the backfill uses the new query and
returns the number of rows it found. `scheduleRunLivenessBackfill` keeps
a bounded in-memory map (at most 10,000 tasks, oldest entry is removed
first) of tasks with nothing left to repair. It skips these tasks for 10
minutes.
- `server/src/__tests__/activity-service.test.ts`: four new
real-database tests.
- The backfill repairs runs found by context and runs found only by
issue activity. It does not change a running run, an unclassified run of
another task, or a run of another company that names the same task id.
- After a pass leaves nothing to repair, more reads do not repair a new
unclassified run. After the skip window ends, the next read repairs it.
- A task with 25 unclassified runs (more than one batch) gets all of
them repaired over later reads.
- A read with `skipLivenessBackfill` does not repair. The next normal
read does.
- `scripts/benchmark-run-liveness-backfill.ts`: a local SQL benchmark
that compares the pre-fix query with the new query.
## Verification
Local SQL benchmark (embedded PostgreSQL 18.1, macOS, synthetic data):
```sh
cd server
pnpm exec tsx ../scripts/benchmark-run-liveness-backfill.ts --baseline-ref=02bfe71a6bbd7c5676cb7e72778b361c5bc72765
```
- Baseline: `02bfe71a6b` (pre-fix master). Treatment: `80fcfcf7f1`,
clean tree.
- Fixture assumptions (staging row counts were not available): one
company with 30,000 classified runs, 6,000 unclassified finished runs of
other tasks or of no task, each with an approximately 20 KiB hex
context, 400 other tasks, and 300,000 `activity_log` rows (half with a
run id). The target task has 40 classified runs, 200 activity rows, and
the missing rows of each case. Decoys: one `running` run of the task,
and one run in another company that names the task id. A small control
uses 500 classified and 50 unclassified runs.
- Each arm: 2 warm-up runs and 11 measured runs, in alternating order.
| Case | Candidates | Baseline median (min to max) | New median (min to
max) | Speedup | Shared buffers (baseline to new) |
| --- | --- | --- | --- | --- | --- |
| Task with nothing to repair | 0 | 37.2 ms (31.1 to 44.1) | 1.21 ms
(0.42 to 1.79) | 30.8x | 34,247 to 138 |
| Task with 3 missing | 3 | 37.8 ms (27.2 to 39.6) | 1.16 ms (0.77 to
2.05) | 32.6x | 34,265 to 148 |
| Task with 25 missing | 25 | 37.5 ms (34.6 to 69.8) | 1.63 ms (1.06 to
3.47) | 23.0x | 34,361 to 154 |
| Small control | 3 | 0.79 ms (0.70 to 1.06) | 0.70 ms (0.48 to 1.70) |
1.1x (neutral) | 314 to 18 |
The `EXPLAIN ANALYZE` execution time for the large cases goes from
approximately 35 ms to approximately 0.15 ms. These are local SQL
measurements. The hosted latency improvement is not yet measured.
The benchmark also checks result equality. For each case, it runs both
queries without a limit and compares every column of every row (as
sorted positional rows). It also checks that the limited batch of 20
comes from the baseline candidate set. All cases match.
Beyond the query, the skip window removes the lookup for most reads of a
task with nothing to repair.
Tests:
```sh
cd server
pnpm exec vitest run src/__tests__/activity-service.test.ts # 24 passed
pnpm exec vitest run src/__tests__/activity-routes.test.ts src/__tests__/task-cold-open.test.ts src/__tests__/liveness.test.ts
pnpm exec tsc --noEmit -p .
```
- The skip-window test fails on the pre-fix `activity.ts` (red), and
passes with this change (green).
- `pnpm verify` asks for broader coverage, because the change adds a
script. Required CI supplies that coverage.
## Risks
- A run that misses its classification after a pass finds nothing for
its task gets repaired up to 10 minutes later. Normal runs get their
classification at finalization, so only the repair path is late.
- The skip map is in memory for each server process. A restart clears
it. That only causes one more lookup for each task.
- The new query keeps the same rows, filter, and limit. The query has no
order, as before. Thus the 20 rows that a pass picks can differ when a
task has more than 20 candidates. Later reads repair the rest in both
versions.
- Low risk. No migration and no API change.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>