PaperClipAI/doc/workspace-activity-query-performance.md

# Workspace activity query performance

The workspace activity query reads three compatibility keys from each retained
run context. Wide contexts made those separate JSON extractions expensive. The
same scope also materialized complete run rows for cleanup checks that consume
only six lifecycle fields.

The implementation extracts `issueId`, `taskId`, and `executionWorkspaceId`
together through `jsonb_to_record`, after native task bindings and bound
checkout/execution-run pointers have had a chance to match. Strict JSONPath
passes only a top-level object to the record function. Arrays, scalars and JSON
null therefore retain their former unmatched legacy-key behavior; their native
bindings remain independent. PostgreSQL documents the
[strict path and type semantics](https://www.postgresql.org/docs/17/functions-json.html#FUNCTIONS-SQLJSON-PATH-MODES).

The full tree's textual IDs are materialized once for the decoder's membership
checks. This also prevents recursive cardinality estimates from inflating the
correlated decoder's startup cost enough to trigger expensive JIT compilation.
The key set has no result limit. Empty trees produce an empty set.

## Contract and blast radius

The safety fact is that the same runs supply the same lifecycle fields. Native
and legacy bindings remain independent, including conflicting bindings. The
run scope retains descendants, tasks bound outside the tree, explicit bound
checkout/execution pointers, exact textual UUID comparisons, company scope,
duplicates and null behavior.

| Consumer | Fields read from the run scope | Consequence of a missing run |
| --- | --- | --- |
| `gitWorkspaceLastActivityAt` | Created, started and finished timestamps | Expiration may be scheduled too early. |
| `isGitWorkspaceExpirationEligible` | ID, status, all three timestamps, controller lease expiry | A live run or its persistence/cleanup obligation could stop vetoing deletion. |
| Task-directory `orphanConsumer` | ID, status, finish time, controller lease expiry | An orphan folder could be removed while a run, finalization or lease still needs it. |
| `workspaceMoveEligible` | ID, status, finish time, controller lease expiry | Re-home could move a workspace used by a live controller. |

All six fields remain projected. The task, recovery, finalization, environment
lease and runtime-service checks keep their existing mode-specific rules.
Deletion still rechecks current eligibility after the `beforeDelete` boundary,
under the existing source/lifecycle locks. Shared-path rejection still precedes
run-history evaluation.

The deterministic database tests exercise nine independent binding paths,
downstream pending-cleanup leases referenced only through a run ID, conflicting
native/legacy bindings, multiply bound runs, case-sensitive legacy IDs,
cross-company decoys, current controller leases, new consumers between reads,
and null/scalar/array/nested-array contexts with and without native bindings.
Fixture insertion disables the scope-normalization trigger in disposable
databases so legacy-path tests cannot silently become native-binding tests.

The benchmark compares complete positional caller results and the complete
multiset of all six relevant-run fields. This is evidence level 4 from the
blast-radius workflow: real code and real PostgreSQL. Deployed behavior and
production latency still require a rollout and fresh measurements.

## Reproduce the measurements

The baseline is `db70c5acd92f50fa37fb4f0dea3da0e7685089a2`. The harness loads
that service source with its unchanged current imports and captures the actual
SQL generated by the baseline and checked-out service. Reports save source
snapshots, SHA-256 digests, the source diff, all samples, complete SQL/parameters,
database settings, and `EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)` plans.

The Docker option creates and removes its own disposable PostgreSQL 17
container, applies normal migrations, and uses synthetic data. It does not use
`DATABASE_URL`. Without that option the harness uses the normal embedded
PostgreSQL test helper.

```sh
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-activity.ts \
  --baseline-ref=db70c5acd92f50fa37fb4f0dea3da0e7685089a2 \
  --docker-postgres=17 --reject-new-jit --output-dir=/tmp/workspace-activity-stress
```

Run the other controls with `--small`, `--descendants=250`, and
`--small --descendants=250`, using separate output directories. Defaults are two
warmups and five measured samples per version, with baseline/treatment order
alternating. `--samples=1 --warmups=0` is diagnostic only. `--reject-new-jit`
saves non-executing preflight plans and stops before repeating a newly expensive
JIT plan.

Each timing sample includes eight sequential workspace queries. The stress
fixture has 4,813 runs; the small control has 141. Every wide run has roughly
32 KiB of incompressible context padding and 32 KiB of result padding, with a
unique context nonce. The large-tree controls have 250 descendants per
workspace. Statistics are refreshed before timing.

The accepted source SHA-256 is
`cd784320265af7857b534d705dbdbe09e732014f2be160971dd89e6d6219029a`.
All four reports have that digest. These measurements used PostgreSQL 17.11 on
Linux/aarch64 in Docker, with `work_mem=4MB`. The run relation including indexes
and TOAST was about 331.8 MB for stress and 9.15 MB for the small controls.

| Fixture | Query | Before median, ms/8 | After median, ms/8 | Before / after |
| --- | --- | ---: | ---: | ---: |
| Stress, 1 descendant | Last activity | 1,071.54 | 743.45 | 1.44x |
| Stress, 1 descendant | Closed eligibility | 814.29 | 713.86 | 1.14x |
| Stress, 1 descendant | Idle eligibility | 721.46 | 481.69 | 1.50x |
| Small, 1 descendant | Last activity | 20.53 | 15.06 | 1.36x |
| Small, 1 descendant | Closed eligibility | 21.85 | 16.64 | 1.31x |
| Small, 1 descendant | Idle eligibility | 21.91 | 17.74 | 1.23x |
| Stress, 250 descendants | Last activity | 740.35 | 582.55 | 1.27x |
| Stress, 250 descendants | Closed eligibility | 678.95 | 549.16 | 1.24x |
| Stress, 250 descendants | Idle eligibility | 948.03 | 728.78 | 1.30x |
| Small, 250 descendants | Last activity | 45.51 | 38.09 | 1.19x |
| Small, 250 descendants | Closed eligibility | 61.11 | 62.82 | 0.97x, neutral |
| Small, 250 descendants | Idle eligibility | 46.28 | 38.02 | 1.22x |

The stress last-activity samples did not overlap: baseline 920.27–1,111.22 ms
and treatment 689.14–880.40 ms per eight workspaces. Closed-eligibility timings
were noisier; its stress range was 649.58–2,138.92 ms versus 546.43–842.67 ms.
Treat that smaller median gain as directional. The combined closed-eligibility
control also had overlapping ranges, 59.01–82.96 versus 48.41–121.52 ms; its
1.72 ms median difference per eight workspaces is neutral evidence.

The plans support the mechanism independently of timing noise. Stress
last-activity buffer accesses fell from 113,202 to 38,081. The decoder ran
4,752 times for 4,812 company rows, retaining native short-circuit behavior.
Materialized eligibility run width fell from about 1,414 to 57 planner bytes.
All final plans retained one history scan without JIT. Every measured caller
result and complete relevant-run multiset matched the baseline.

## Rejected query shapes

These failures informed the final shape and are not accepted performance results:

| Candidate | Decisive result |
| --- | --- |
| Split binding joins without parameterization | Legacy hash joins caused additional wide-context scans; roughly twice the baseline SQL time. |
| Per-tree-node indexed probes | A 250-descendant fixture activated JIT and became roughly 5–8 times slower. |
| Aggregate-key-set per-node probes | Removed JIT, but small history plus 250 descendants chose one sequential scan per node and became 43–56 times slower. |
| Separate `ANY(ARRAY(...))` binding queries | Cleared the small/large-tree regression but performed separate history scans on the stress fixture and became roughly 1.7–1.8 times slower. |
| Correlated record decoder reading the recursive tree directly | A non-executing preflight found new JIT with estimated cost 156,621; rejected before repeated timing. |

The final query retains one history scan and shares the materialized membership
key set. It changes neither scheduling frequency nor retention policy. Batching
the advisory last-activity reads is deferred so this change keeps the existing
reaper lifecycle and destructive freshness checks intact.

## Before merging or rolling out

Run the binding/eligibility tests and the existing Git expiration, directory
retention and repository re-home suites. They cover actual filesystem removal,
restoration, unique work, unresolved consumers, reopening and concurrent changes.

```sh
pnpm --filter @paperclipai/server exec vitest run \
  src/__tests__/git-workspace-activity.test.ts \
  src/__tests__/git-workspace-expiration-eligibility.test.ts \
  src/__tests__/git-workspace-expiration.test.ts \
  src/__tests__/isolated-task-directory-retention.test.ts \
  src/__tests__/isolated-repository-rehome.test.ts
```

Also complete the repository's required typecheck, full tests and build on the
final source. After an authorized rollout, confirm the deployed revision and
compare fresh query-statistics deltas and cleanup behavior. Local synthetic SQL
measurements do not establish a production latency improvement.