Files
PaperClipAI/server/src/services/execution-workspace-root-readers.ts
T
DottaandPaperclip 1f27427ffd perf(server): batch retained workspace reader identities and policies (#16683)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task startup checks who can read each physical workspace.
> - A shared root retains the identities of earlier runs.
> - The service reloads the same agents, tasks, projects and run
policies for each reader.
> - Database round trips grow with the retained run history and hold the
company provenance lock.
> - This pull request batches these reads within each guarded operation.
Each run keeps its own identity and policy.
> - The benefit is less task preparation time while current access
checks remain in place.

## Linked Issues or Issue Description

Refs #16424.

**What existing behavior does this improve?**

Workspace admission, repository publication and batched read
authorization.

**Current behavior**

A synthetic shared workspace with 64 retained readers makes 680 SQL
calls during a publication check. The same principals are loaded
repeatedly.

**Proposed behavior**

Load retained identities and saved policies in batches of at most 500
IDs. Load project settings once per guarded operation. Reload records
and permission grants on the next operation.

**Reason and benefit**

The local service benchmark with an injected 4 ms loopback round trip
changed from about 7.0 seconds to 0.5 seconds and from 680 SQL calls to
47. This measures synthetic PostgreSQL data, not hosted response
latency.

## What Changed

- Extract retained-reader identity loading and acquisition trust checks
into a focused service module. Select only the identity and trust fields
used by these checks.
- Batch company-scoped run, agent and task lookups. Keep each run's task
binding, responsible user and policy checks.
- Batch saved run-policy reads in read authorization. Keep company and
agent identity checks and ordinary lookup behavior for noncanonical IDs.
- Reuse project settings within one root authorization operation.
- Add query budgets, identity failures, batch-boundary coverage, policy
comparisons and grant-revocation tests.
- Document snapshot lifetime and query scope.

## Verification

- PASS: 197 focused workspace and authorization tests.
- PASS: `pnpm -r typecheck`.
- PASS: `pnpm build`.
- PASS: all current-head hosted CI test groups, Runner checks, E2E
shards, security checks and builds (55 successful checks; two unrelated
Storybook jobs skipped). The local monolithic `pnpm test:run` was
stopped after all equivalent hosted CI groups passed. It has no complete
local result; the hosted matrix supplies the full-suite proof.
- Synthetic real-PostgreSQL benchmark: 383 workspaces and 64 retained
readers; 4 ms simulated loopback round trip. Publication check: 680 ->
47 SQL calls; 7.0-7.1 s -> 0.48-0.51 s. Three concurrent checks:
20.0-23.1 s -> 1.52-2.12 s. Setup and provider execution are excluded.
Live reply timing remains unverified.
- Regression coverage compares complete batched and ordinary
authorization decisions. It checks distinct saved policies for the same
principals, missing identities, company mismatch, task mismatch,
responsible-user mismatch, revoked project grants and changes after the
first operation.
- No new schema, API or UI contract.

## Risks

- A batch must not combine different runs' identities or policies. Each
lookup and policy decision retains its run, company and agent checks.
- Snapshots last for one guarded operation. Later admissions and
publications reload data. No authorization cache survives the operation.
- Restricted project trust checks keep their existing per-run validation
path. Hosted latency remains an acceptance gate.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with xhigh reasoning, code execution
and tool use. The runtime does not expose the configured context-window
size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-17 21:52:08 -05:00

80 lines
5.2 KiB
TypeScript

import { and, eq, inArray, sql } from "drizzle-orm";
import { agents, heartbeatRuns, issues, projects, executionWorkspaceRepositoryAccess, type Db } from "@paperclipai/db";
import { forbidden } from "../errors.js";
import type { AuthorizationActor } from "./authorization.js";
import { measureRunPhase } from "./execution-preparation-timing.js";
import { resolveCoreTrustPreset } from "./trust-preset-resolver.js";
type ReaderDb = Db | Parameters<Parameters<Db["transaction"]>[0]>[0];
type ReaderEntry = Pick<typeof executionWorkspaceRepositoryAccess.$inferSelect, "scopeId" | "issueId" | "agentId" | "responsibleUserId">;
type ReaderAgent = Pick<typeof agents.$inferSelect, "id" | "companyId" | "permissions">;
type ReaderIssue = Pick<typeof issues.$inferSelect, "id" | "companyId" | "projectId" | "executionPolicy">;
type ReaderProject = Pick<typeof projects.$inferSelect, "companyId" | "executionWorkspacePolicy">;
export const rootReaderRunColumns = {
id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
responsibleUserId: heartbeatRuns.responsibleUserId, nativeIssueId: heartbeatRuns.nativeIssueId, issueId: heartbeatRuns.issueId,
contextSnapshot: sql<Record<string, unknown>>`jsonb_build_object('issueId', ${heartbeatRuns.contextSnapshot}->'issueId', 'taskId', ${heartbeatRuns.contextSnapshot}->'taskId', 'executionPolicy', ${heartbeatRuns.contextSnapshot}->'executionPolicy')`,
};
// This snapshot belongs to one guarded operation. Never retain it across admissions
// or publications: identities and permissions must be read again next time.
export async function loadRepositoryRootReaders<T extends ReaderEntry>(db: ReaderDb, companyId: string, entries: T[]) {
const runIds = [...new Set(entries.map(entry => entry.scopeId))];
const agentIds = [...new Set(entries.flatMap(entry => entry.agentId ? [entry.agentId] : []))];
const issueIds = [...new Set(entries.flatMap(entry => entry.issueId ? [entry.issueId] : []))];
const runs = new Map<string, Pick<typeof heartbeatRuns.$inferSelect, keyof typeof rootReaderRunColumns>>();
const agentRows = new Map<string, ReaderAgent>();
const issueRows = new Map<string, ReaderIssue>();
await measureRunPhase("workspace.readers.identity", async () => {
for (let offset = 0; offset < runIds.length; offset += 500) {
for (const row of await db.select(rootReaderRunColumns).from(heartbeatRuns).where(and(
eq(heartbeatRuns.companyId, companyId), inArray(heartbeatRuns.id, runIds.slice(offset, offset + 500))))) runs.set(row.id, row);
}
for (let offset = 0; offset < agentIds.length; offset += 500) {
for (const row of await db.select({ id: agents.id, companyId: agents.companyId, permissions: agents.permissions }).from(agents).where(and(
eq(agents.companyId, companyId), inArray(agents.id, agentIds.slice(offset, offset + 500))))) agentRows.set(row.id, row);
}
for (let offset = 0; offset < issueIds.length; offset += 500) {
for (const row of await db.select({ id: issues.id, companyId: issues.companyId, projectId: issues.projectId,
executionPolicy: issues.executionPolicy }).from(issues).where(and(
eq(issues.companyId, companyId), inArray(issues.id, issueIds.slice(offset, offset + 500))))) issueRows.set(row.id, row);
}
});
return entries.map(entry => {
const run = runs.get(entry.scopeId);
const agent = entry.agentId ? agentRows.get(entry.agentId) : undefined;
const issue = entry.issueId ? issueRows.get(entry.issueId) : undefined;
if (!run || !agent || !issue || run.agentId !== entry.agentId || run.responsibleUserId !== entry.responsibleUserId
|| (run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId ?? run.issueId) !== entry.issueId) {
throw forbidden("Repository root retains a reader whose identity is no longer available");
}
const actor: AuthorizationActor = { type: "agent", source: "agent_jwt", companyId,
agentId: agent.id, runId: run.id, onBehalfOfUserId: entry.responsibleUserId === "local-board" ? null : entry.responsibleUserId };
return { entry, run, agent, issue, actor };
});
}
export function requiresAnonymousRepositoryAcquisition(input: {
companyId: string;
issue: ReaderIssue;
agent: ReaderAgent | undefined;
project: ReaderProject | undefined;
workspaceSourceProject: ReaderProject | undefined;
run: Pick<typeof heartbeatRuns.$inferSelect, keyof typeof rootReaderRunColumns> | undefined;
agentId: string | null;
runId: string | null;
explicitRunId?: string | null;
}) {
const { companyId, issue, agent, project, workspaceSourceProject, run, agentId, runId } = input;
if (agentId && !agent) throw forbidden("Task repository agent is unavailable");
if (runId && (!run || (input.explicitRunId && run.agentId !== agentId)
|| (run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId ?? run.issueId) !== issue.id)) {
throw forbidden("Task repository run identity is unavailable");
}
const trust = resolveCoreTrustPreset({ companyId, agent, project, workspaceSourceProject, issue,
run: run ? { companyId: run.companyId, executionPolicy: run.contextSnapshot?.executionPolicy } : null });
if (trust.kind === "denied") throw forbidden(trust.detail);
return trust.kind === "low_trust_review";
}