mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
perf(server): batch retained workspace reader identities and policies (#16683)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task startup checks who can read each physical workspace. > - A shared root retains the identities of earlier runs. > - The service reloads the same agents, tasks, projects and run policies for each reader. > - Database round trips grow with the retained run history and hold the company provenance lock. > - This pull request batches these reads within each guarded operation. Each run keeps its own identity and policy. > - The benefit is less task preparation time while current access checks remain in place. ## Linked Issues or Issue Description Refs #16424. **What existing behavior does this improve?** Workspace admission, repository publication and batched read authorization. **Current behavior** A synthetic shared workspace with 64 retained readers makes 680 SQL calls during a publication check. The same principals are loaded repeatedly. **Proposed behavior** Load retained identities and saved policies in batches of at most 500 IDs. Load project settings once per guarded operation. Reload records and permission grants on the next operation. **Reason and benefit** The local service benchmark with an injected 4 ms loopback round trip changed from about 7.0 seconds to 0.5 seconds and from 680 SQL calls to 47. This measures synthetic PostgreSQL data, not hosted response latency. ## What Changed - Extract retained-reader identity loading and acquisition trust checks into a focused service module. Select only the identity and trust fields used by these checks. - Batch company-scoped run, agent and task lookups. Keep each run's task binding, responsible user and policy checks. - Batch saved run-policy reads in read authorization. Keep company and agent identity checks and ordinary lookup behavior for noncanonical IDs. - Reuse project settings within one root authorization operation. - Add query budgets, identity failures, batch-boundary coverage, policy comparisons and grant-revocation tests. - Document snapshot lifetime and query scope. ## Verification - PASS: 197 focused workspace and authorization tests. - PASS: `pnpm -r typecheck`. - PASS: `pnpm build`. - PASS: all current-head hosted CI test groups, Runner checks, E2E shards, security checks and builds (55 successful checks; two unrelated Storybook jobs skipped). The local monolithic `pnpm test:run` was stopped after all equivalent hosted CI groups passed. It has no complete local result; the hosted matrix supplies the full-suite proof. - Synthetic real-PostgreSQL benchmark: 383 workspaces and 64 retained readers; 4 ms simulated loopback round trip. Publication check: 680 -> 47 SQL calls; 7.0-7.1 s -> 0.48-0.51 s. Three concurrent checks: 20.0-23.1 s -> 1.52-2.12 s. Setup and provider execution are excluded. Live reply timing remains unverified. - Regression coverage compares complete batched and ordinary authorization decisions. It checks distinct saved policies for the same principals, missing identities, company mismatch, task mismatch, responsible-user mismatch, revoked project grants and changes after the first operation. - No new schema, API or UI contract. ## Risks - A batch must not combine different runs' identities or policies. Each lookup and policy decision retains its run, company and agent checks. - Snapshots last for one guarded operation. Later admissions and publications reload data. No authorization cache survives the operation. - Restricted project trust checks keep their existing per-run validation path. Hosted latency remains an acceptance gate. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with xhigh reasoning, code execution and tool use. The runtime does not expose the configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
cfa9dce295
commit
1f27427ffd
5 files changed
+211
-41
No files matched your search
@@ -43,6 +43,13 @@ launch context does not travel with each policy or parent-report check. JSON
|
||||
types, task-key precedence, and company and agent identity checks are preserved.
|
||||
Each decision still reads current data; this is not an authorization cache.
|
||||
|
||||
Repository-root checks load retained run, agent and task identities in batches
|
||||
of at most 500 IDs. They keep each run's task binding, responsible user and saved
|
||||
policy separate, even when several runs use the same agent. Project settings and
|
||||
saved policies for batched read authorization are also loaded once per operation.
|
||||
The next admission or publication reloads these records and permission grants.
|
||||
The queries exclude run results and unused launch context.
|
||||
|
||||
The initial agent JWT run lookup reads only the cancellation state and
|
||||
conversation flag from saved run JSON. It does not transfer the full context or
|
||||
result. Run status, responsible-user identity, company and agent boundaries,
|
||||
|
||||
@@ -9,6 +9,7 @@ import { afterAll, beforeAll, expect, it, vi } from "vitest";
|
||||
import { agents, authUsers, companies, companyMemberships, createDb, executionWorkspaceRepositoryAccess,
|
||||
executionWorkspaces, heartbeatRuns, issues, projects, projectAccessMembers, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS } from "@paperclipai/db";
|
||||
import { executionWorkspaceRepositoryService } from "../services/execution-workspace-repositories.js";
|
||||
import { loadRepositoryRootReaders } from "../services/execution-workspace-root-readers.js";
|
||||
import { authorizationService, type AuthorizationActor } from "../services/authorization.js";
|
||||
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
|
||||
@@ -55,12 +56,43 @@ it("bounds repeated-principal queries without loading retained run payloads", as
|
||||
const queries: string[] = [];
|
||||
const recorded = drizzle(db.$client, { schema: db._.fullSchema, logger: { logQuery: query => queries.push(query) } }) as unknown as typeof db;
|
||||
await executionWorkspaceRepositoryService(recorded).guardForPublication(input());
|
||||
expect(queries.length).toBeLessThan(800);
|
||||
expect(queries.length).toBeLessThan(100);
|
||||
const runs = queries.filter(query => query.includes('from "heartbeat_runs"'));
|
||||
expect(runs.length).toBeGreaterThan(0);
|
||||
expect(runs.length).toBeLessThan(10);
|
||||
expect(runs.every(query => !query.includes('"result_json"') && !query.includes('"runner_profile_json"'))).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects missing, foreign-company and mismatched retained run identities", async () => {
|
||||
const invalidId = runIds[31];
|
||||
const otherCompanyId = randomUUID();
|
||||
await db.insert(companies).values({ id: otherCompanyId, name: "Foreign readers", issuePrefix: "FRQ" });
|
||||
const service = executionWorkspaceRepositoryService(db);
|
||||
for (const change of [
|
||||
{ companyId: otherCompanyId },
|
||||
{ contextSnapshot: { issueId: randomUUID(), taskId: issueId } },
|
||||
{ responsibleUserId: randomUUID() },
|
||||
]) {
|
||||
try {
|
||||
await db.update(heartbeatRuns).set(change).where(eq(heartbeatRuns.id, invalidId));
|
||||
await expect(service.guardForPublication(input())).rejects.toThrow("identity");
|
||||
} finally {
|
||||
await db.update(heartbeatRuns).set({ companyId, nativeIssueId: null, contextSnapshot: { issueId }, responsibleUserId: userId })
|
||||
.where(eq(heartbeatRuns.id, invalidId));
|
||||
}
|
||||
}
|
||||
for (const change of [{ scopeId: randomUUID() }, { agentId: randomUUID() }, { issueId: randomUUID() }]) {
|
||||
try {
|
||||
await db.update(executionWorkspaceRepositoryAccess).set(change).where(eq(executionWorkspaceRepositoryAccess.scopeId, invalidId));
|
||||
await expect(service.guardForPublication(input())).rejects.toThrow("identity");
|
||||
} finally {
|
||||
await db.update(executionWorkspaceRepositoryAccess).set({ scopeId: invalidId, agentId, issueId })
|
||||
.where(eq(executionWorkspaceRepositoryAccess.scopeId, "scopeId" in change ? change.scopeId : invalidId));
|
||||
}
|
||||
}
|
||||
await expect(service.guardForPublication(input())).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps each retained run identity and policy distinct for the same principals", async () => {
|
||||
const service = executionWorkspaceRepositoryService(db);
|
||||
const invalidId = runIds[31];
|
||||
@@ -77,6 +109,54 @@ it("keeps each retained run identity and policy distinct for the same principals
|
||||
await expect(service.guardForPublication(input())).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("checks identities after the first query batch and reloads them on the next operation", async () => {
|
||||
const ids = Array.from({ length: 501 }, () => randomUUID());
|
||||
await db.insert(heartbeatRuns).values(ids.map(id => ({ id, companyId, agentId, responsibleUserId: userId,
|
||||
status: "succeeded", contextSnapshot: { taskId: issueId } })));
|
||||
const entries = ids.map(scopeId => ({ scopeId, issueId, agentId, responsibleUserId: userId }));
|
||||
try {
|
||||
const queries: string[] = [];
|
||||
const recorded = drizzle(db.$client, { schema: db._.fullSchema, logger: { logQuery: query => queries.push(query) } }) as unknown as typeof db;
|
||||
const readers = await loadRepositoryRootReaders(recorded, companyId, entries);
|
||||
expect(readers.map(reader => reader.actor.runId)).toEqual(ids);
|
||||
expect(readers.every(reader => reader.issue.id === issueId && reader.actor.onBehalfOfUserId === userId)).toBe(true);
|
||||
const agentQueries = queries.filter(query => query.includes('from "agents"'));
|
||||
expect(agentQueries).toHaveLength(1);
|
||||
expect(agentQueries.every(query =>
|
||||
!query.includes('"adapter_config"') && !query.includes('"runtime_config"') && !query.includes('"metadata"'))).toBe(true);
|
||||
await db.update(heartbeatRuns).set({ responsibleUserId: randomUUID() }).where(eq(heartbeatRuns.id, ids.at(-1)!));
|
||||
await expect(loadRepositoryRootReaders(db, companyId, entries)).rejects.toThrow("identity");
|
||||
} finally {
|
||||
await db.delete(heartbeatRuns).where(inArray(heartbeatRuns.id, ids));
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves each actor's saved policy in batched read decisions", async () => {
|
||||
const actor: AuthorizationActor = { type: "agent", source: "agent_jwt", companyId, agentId,
|
||||
runId: runIds[0], onBehalfOfUserId: userId };
|
||||
const policyRunId = runIds[31];
|
||||
const actors = [actor, { ...actor, runId: policyRunId }, { ...actor, runId: runIds[0].toUpperCase() },
|
||||
{ ...actor, runId: randomUUID() }];
|
||||
const resources = [{ type: "project" as const, companyId, projectId }, { type: "company" as const, companyId }];
|
||||
const service = authorizationService(db);
|
||||
try {
|
||||
await db.update(heartbeatRuns).set({ contextSnapshot: { issueId, executionPolicy: { trustPreset: "unsupported" } } })
|
||||
.where(eq(heartbeatRuns.id, policyRunId));
|
||||
const decisions = await service.decideReadsForActors(actors, resources);
|
||||
for (const [index, candidate] of actors.entries()) {
|
||||
for (const [resourceIndex, resource] of resources.entries()) {
|
||||
expect(decisions[index][resourceIndex]).toEqual(await service.decide({ actor: { ...candidate }, resource,
|
||||
action: resource.type === "project" ? "project:read" : "company_scope:read" }));
|
||||
}
|
||||
}
|
||||
expect(decisions[0][0].allowed).toBe(true);
|
||||
expect(decisions[1][0].allowed).toBe(false);
|
||||
} finally {
|
||||
await db.update(heartbeatRuns).set({ contextSnapshot: { issueId } }).where(eq(heartbeatRuns.id, policyRunId));
|
||||
}
|
||||
expect((await service.decideReadsForActors(actors, resources))[1][0].allowed).toBe(true);
|
||||
});
|
||||
|
||||
it("reloads project grants on every publication attempt", async () => {
|
||||
const service = executionWorkspaceRepositoryService(db);
|
||||
await db.update(projects).set({ visibility: "private" }).where(eq(projects.id, projectId));
|
||||
|
||||
@@ -732,6 +732,7 @@ type DbTransaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
|
||||
type ReadAuthorizationSnapshot = {
|
||||
issues: Map<string, IssueAuthorizationRow & IssuePrivacyRow & { readable: boolean }>;
|
||||
projects: Map<string, ProjectAuthorizationRow & ProjectPrivacyRow & { readable: boolean }>;
|
||||
runs: Map<string, { companyId: string; agentId: string; contextSnapshot: Record<string, unknown> } | null>;
|
||||
memo: Map<string, Promise<unknown>>;
|
||||
};
|
||||
|
||||
@@ -756,8 +757,21 @@ function createAuthorizationService(db: Db | DbTransaction, readSnapshot?: ReadA
|
||||
async function decideReadsForActors(actors: AuthorizationActor[], resources: ReadAuthorizationResource[]): Promise<AuthorizationDecision[][]> {
|
||||
if (!actors.length) return [];
|
||||
const memo = new Map<string, Promise<unknown>>();
|
||||
const snapshots: ReadAuthorizationSnapshot[] = actors.map(() => ({ issues: new Map(), projects: new Map(), memo }));
|
||||
const runs: ReadAuthorizationSnapshot["runs"] = new Map();
|
||||
const snapshots: ReadAuthorizationSnapshot[] = actors.map(() => ({ issues: new Map(), projects: new Map(), runs, memo }));
|
||||
const batchActors: ResponsibleUserActorWithMemo[] = actors.map(actor => ({ ...actor, __responsibleUserSnapshotMemo: undefined }));
|
||||
// Only canonical IDs take this fast path. Other inputs keep the ordinary
|
||||
// lookup behavior, including PostgreSQL's handling of invalid UUIDs.
|
||||
const runIds = resources.length ? [...new Set(batchActors.flatMap(actor => actor.type === "agent" && actor.runId
|
||||
&& /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/.test(actor.runId) ? [actor.runId] : []))] : [];
|
||||
for (let offset = 0; offset < runIds.length; offset += 500) {
|
||||
const ids = runIds.slice(offset, offset + 500);
|
||||
for (const id of ids) runs.set(id, null);
|
||||
const rows = await db.select({ id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
||||
contextSnapshot: sql<Record<string, unknown>>`jsonb_build_object('executionPolicy', ${heartbeatRuns.contextSnapshot}->'executionPolicy')`,
|
||||
}).from(heartbeatRuns).where(inArray(heartbeatRuns.id, ids));
|
||||
for (const row of rows) runs.set(row.id, row);
|
||||
}
|
||||
const privacyKey = (actor: AuthorizationActor) => JSON.stringify([actor.type, actor.agentId, actor.userId, actor.onBehalfOfUserId]);
|
||||
const privacyActors = [...new Map(batchActors.map(actor => [privacyKey(actor), actor])).values()];
|
||||
const privacyIndexes = batchActors.map(actor => privacyActors.findIndex(peer => privacyKey(peer) === privacyKey(actor)));
|
||||
@@ -1048,7 +1062,7 @@ function createAuthorizationService(db: Db | DbTransaction, readSnapshot?: ReadA
|
||||
async function loadRunPolicy(runId: string | null | undefined, companyId: string, agentId: string) {
|
||||
return batchRead(`run:${companyId}:${agentId}:${runId}`, async () => {
|
||||
if (!runId) return null;
|
||||
const row = await db
|
||||
const row = readSnapshot?.runs.has(runId) ? readSnapshot.runs.get(runId) : await db
|
||||
.select({
|
||||
id: heartbeatRuns.id,
|
||||
companyId: heartbeatRuns.companyId,
|
||||
|
||||
@@ -21,6 +21,7 @@ import { toolAccessService } from "./tool-access.js";
|
||||
import { normalizeProjectRepositoryUrl } from "./project-repositories.js";
|
||||
import { buildGitAuthInvocation, createGitRemoteAuthProvider } from "./git-credentials.js";
|
||||
import { materializeManagedProjectWorkspace, ensureManagedRepositoriesIgnored } from "./managed-repository-checkout.js";
|
||||
import { loadRepositoryRootReaders, requiresAnonymousRepositoryAcquisition, rootReaderRunColumns } from "./execution-workspace-root-readers.js";
|
||||
|
||||
import { canonicalRepositoryRoot, resolveRepositoryRoots, repositoryRootsOverlap, legacyRepositoryRootMayMatch, configuredWorkspaceRoot, verifiedIsolatedWorktree, unusedIsolatedCopyTarget, isolatedWorkspaceSourceRoots, type WorkspaceSourceCopyKind } from "./execution-workspace-source-roots.js";
|
||||
export { canonicalRepositoryRoot } from "./execution-workspace-source-roots.js";
|
||||
@@ -28,6 +29,7 @@ export { canonicalRepositoryRoot } from "./execution-workspace-source-roots.js";
|
||||
const execFile = promisify(execFileCallback);
|
||||
type RepositoryRow = typeof executionWorkspaceRepositories.$inferSelect;
|
||||
type DbTransaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
|
||||
type RootProject = Pick<typeof projects.$inferSelect, "id" | "companyId" | "executionWorkspacePolicy">;
|
||||
|
||||
function repositoryAccessKeys(entries: Array<Pick<typeof executionWorkspaceRepositoryAccess.$inferSelect, "kind" | "scopeId">>) {
|
||||
return new Set(entries.map(entry => `${entry.kind}:${entry.scopeId}`));
|
||||
@@ -52,11 +54,6 @@ async function retainRepositorySources(tx: DbTransaction,
|
||||
|
||||
const controllerAcquisitionMessage = "New controller repository acquisition requires a trusted task. Existing verified public checkouts remain available.";
|
||||
const lowTrustRepositoryMessage = "Low-trust repository reuse permits anonymous public URLs only; new acquisition is unavailable. Use an authorized configured source for private repositories.";
|
||||
const rootReaderRunColumns = {
|
||||
id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
||||
responsibleUserId: heartbeatRuns.responsibleUserId, nativeIssueId: heartbeatRuns.nativeIssueId, issueId: heartbeatRuns.issueId,
|
||||
contextSnapshot: sql<Record<string, unknown>>`jsonb_build_object('issueId', ${heartbeatRuns.contextSnapshot}->'issueId', 'taskId', ${heartbeatRuns.contextSnapshot}->'taskId', 'executionPolicy', ${heartbeatRuns.contextSnapshot}->'executionPolicy')`,
|
||||
};
|
||||
|
||||
function resolveAcquisitionSource(request: PrepareWorkspaceRepository["repository"], available: ProjectRepository[], anonymousOnly: boolean) {
|
||||
if (anonymousOnly && request.kind === "catalog") throw forbidden(lowTrustRepositoryMessage);
|
||||
@@ -233,19 +230,6 @@ async function assertRootProjectsReadAccessForActors(db: Db | DbTransaction, act
|
||||
}
|
||||
}
|
||||
|
||||
async function retainedRootReader(tx: DbTransaction, companyId: string, reader: Pick<typeof executionWorkspaceRepositoryAccess.$inferSelect, "scopeId" | "issueId" | "agentId" | "responsibleUserId">) {
|
||||
const [run] = await measureRunPhase("workspace.sources.inventory", async () => tx.select(rootReaderRunColumns).from(heartbeatRuns).where(and(eq(heartbeatRuns.id, reader.scopeId), eq(heartbeatRuns.companyId, companyId))));
|
||||
const [agent] = reader.agentId ? await measureRunPhase("workspace.sources.inventory", async () => tx.select({ id: agents.id }).from(agents).where(and(eq(agents.id, reader.agentId!), eq(agents.companyId, companyId)))) : [];
|
||||
const [issue] = reader.issueId ? await measureRunPhase("workspace.sources.inventory", async () => tx.select().from(issues).where(and(eq(issues.id, reader.issueId!), eq(issues.companyId, companyId)))) : [];
|
||||
if (!run || !agent || !issue || run.agentId !== reader.agentId || run.responsibleUserId !== reader.responsibleUserId
|
||||
|| (run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId ?? run.issueId) !== reader.issueId) {
|
||||
throw forbidden("Repository root retains a reader whose identity is no longer available");
|
||||
}
|
||||
const actor: AuthorizationActor = { type: "agent", source: "agent_jwt", companyId,
|
||||
agentId: reader.agentId!, runId: reader.scopeId, onBehalfOfUserId: reader.responsibleUserId === "local-board" ? null : reader.responsibleUserId };
|
||||
return { actor, issue };
|
||||
}
|
||||
|
||||
export async function lockExecutionWorkspaceRootProvenance(tx: DbTransaction, companyId: string) {
|
||||
await measureRunPhase("workspace.provenance_lock", () => tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`task-repository-roots:${companyId}`}, 0))`));
|
||||
}
|
||||
@@ -305,8 +289,7 @@ async function retainGitMetadataOwners(tx: DbTransaction, input: {
|
||||
const workspaceIds = new Set([...closure.sourceIds, ...input.workspaceIds]);
|
||||
const projectIds = new Set([...closure.sourceProjectIds, ...input.projectIds]);
|
||||
const readerActors: AuthorizationActor[] = [];
|
||||
for (const reader of closure.overlappingEntries.filter(entry => entry.kind === "reader")) {
|
||||
const { actor } = await measureRunPhase("workspace.readers.identity", () => retainedRootReader(tx, input.companyId, reader));
|
||||
for (const { actor } of await loadRepositoryRootReaders(tx, input.companyId, closure.overlappingEntries.filter(entry => entry.kind === "reader"))) {
|
||||
readerActors.push(actor);
|
||||
}
|
||||
await measureRunPhase("workspace.readers.projects", () => assertRootProjectsReadAccessForActors(tx, readerActors, input.companyId, projectIds));
|
||||
@@ -394,8 +377,7 @@ export async function retainIsolatedWorkspaceSourceAccess(db: Db, input: {
|
||||
await measureRunPhase("workspace.readers.projects", () => assertRootProjectAccess(tx, input.actor!, input.companyId, target.sourceProjectIds,
|
||||
{ sharedProjectIds: target.sharedProjectIds, issueId: input.issueId }));
|
||||
const readerActors: AuthorizationActor[] = [];
|
||||
for (const reader of target.overlappingEntries.filter(entry => entry.kind === "reader")) {
|
||||
const { actor, issue } = await measureRunPhase("workspace.readers.identity", () => retainedRootReader(tx, input.companyId, reader));
|
||||
for (const { actor, issue } of await loadRepositoryRootReaders(tx, input.companyId, target.overlappingEntries.filter(entry => entry.kind === "reader"))) {
|
||||
readerActors.push(actor);
|
||||
await measureRunPhase("workspace.readers.trust", () => assertRootProjectTrust(tx, { companyId: input.companyId, actor, issueId: issue.id, projectIds: target.sourceProjectIds }));
|
||||
}
|
||||
@@ -419,11 +401,11 @@ export type TaskWorkspaceCreationTrustContext = {
|
||||
async function assertRootProjectTrust(db: Db | DbTransaction, input: {
|
||||
companyId: string; projectIds: Set<string>; actor: AuthorizationActor;
|
||||
issueId?: string | null; sourceWorkspaceId?: string; assigneeAgentId?: string | null; taskCreation?: TaskWorkspaceCreationTrustContext;
|
||||
}) {
|
||||
}, owners?: RootProject[]) {
|
||||
if (!input.projectIds.size) return;
|
||||
const owners = await db.select().from(projects).where(and(eq(projects.companyId, input.companyId),
|
||||
const sourceOwners = owners ?? await db.select().from(projects).where(and(eq(projects.companyId, input.companyId),
|
||||
or(...[...input.projectIds].map(id => eq(projects.id, id)))));
|
||||
const restricted = owners.filter(owner => resolveCoreTrustPreset({ companyId: input.companyId, workspaceSourceProject: owner }).kind !== "standard");
|
||||
const restricted = sourceOwners.filter(owner => resolveCoreTrustPreset({ companyId: input.companyId, workspaceSourceProject: owner }).kind !== "standard");
|
||||
if (!restricted.length) return;
|
||||
const incompatible = () => forbidden("Physical workspace source trust is incompatible with the task's execution policy");
|
||||
const runId = input.actor.runId;
|
||||
@@ -514,13 +496,8 @@ export function executionWorkspaceRepositoryService(db: Db) {
|
||||
const [workspaceSourceProject] = workspace?.projectId ? await reader.select().from(projects).where(and(eq(projects.id, workspace.projectId), eq(projects.companyId, issue.companyId))) : [];
|
||||
const runId = input.runId ?? issue.executionRunId;
|
||||
const [run] = runId ? await reader.select(rootReaderRunColumns).from(heartbeatRuns).where(and(eq(heartbeatRuns.id, runId), eq(heartbeatRuns.companyId, issue.companyId))) : [];
|
||||
if (runId && (!run || (input.runId && run.agentId !== agentId) || (run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId ?? run.issueId) !== issue.id)) {
|
||||
throw forbidden("Task repository run identity is unavailable");
|
||||
}
|
||||
const trust = resolveCoreTrustPreset({ companyId: issue.companyId, agent, project, workspaceSourceProject, issue,
|
||||
run: run ? { companyId: run.companyId, executionPolicy: run.contextSnapshot?.executionPolicy } : null });
|
||||
if (trust.kind === "denied") throw forbidden(trust.detail);
|
||||
return trust.kind === "low_trust_review";
|
||||
return requiresAnonymousRepositoryAcquisition({ companyId: issue.companyId, issue, agent, project,
|
||||
workspaceSourceProject, run, agentId, runId, explicitRunId: input.runId });
|
||||
}
|
||||
async function assertCurrentControllerAcquisition(input: { companyId: string; issueId: string; workspaceId: string; agentId: string; runId: string }) {
|
||||
const [issue] = await db.select().from(issues).where(and(eq(issues.id, input.issueId), eq(issues.companyId, input.companyId)));
|
||||
@@ -589,10 +566,20 @@ export function executionWorkspaceRepositoryService(db: Db) {
|
||||
const sourceActors: AuthorizationActor[] = [];
|
||||
const projectReaders: AuthorizationActor[] = [];
|
||||
await measureRunPhase("workspace.readers", async () => {
|
||||
for (const reader of readers) {
|
||||
const { actor, issue } = await measureRunPhase("workspace.readers.identity", () => retainedRootReader(tx, input.companyId, reader));
|
||||
const identities = await loadRepositoryRootReaders(tx, input.companyId, readers);
|
||||
const projectIds = [...new Set([...sourceProjectIds, ...sourceProjects.flatMap(source => source.projectId ? [source.projectId] : []),
|
||||
...identities.flatMap(reader => reader.issue.projectId ? [reader.issue.projectId] : [])])];
|
||||
const projectRows: RootProject[] = [];
|
||||
for (let offset = 0; offset < projectIds.length; offset += 500) {
|
||||
projectRows.push(...await measureRunPhase("workspace.sources.inventory", () => tx.select({ id: projects.id,
|
||||
companyId: projects.companyId, executionWorkspacePolicy: projects.executionWorkspacePolicy }).from(projects).where(and(
|
||||
eq(projects.companyId, input.companyId), inArray(projects.id, projectIds.slice(offset, offset + 500))))));
|
||||
}
|
||||
const projectsById = new Map(projectRows.map(project => [project.id, project]));
|
||||
const sourceOwners = projectRows.filter(project => sourceProjectIds.has(project.id));
|
||||
for (const { entry: reader, actor, issue, agent, run } of identities) {
|
||||
await measureRunPhase("workspace.readers.trust", () => assertRootProjectTrust(tx, { companyId: input.companyId, projectIds: sourceProjectIds, actor, issueId: issue.id,
|
||||
sourceWorkspaceId: input.reader?.runId === reader.scopeId ? input.sourceWorkspaceId : undefined }));
|
||||
sourceWorkspaceId: input.reader?.runId === reader.scopeId ? input.sourceWorkspaceId : undefined }, sourceOwners));
|
||||
if (input.reader?.runId === reader.scopeId) {
|
||||
await measureRunPhase("workspace.readers.projects", () => assertRootProjectAccess(tx, actor, input.companyId, sourceProjectIds, { sharedProjectIds, issueId: issue.id }));
|
||||
} else projectReaders.push(actor);
|
||||
@@ -606,7 +593,10 @@ export function executionWorkspaceRepositoryService(db: Db) {
|
||||
if (!ownSource) throw forbidden("Task workspace access is no longer available");
|
||||
} else sourceActors.push(actor);
|
||||
for (const sourceWorkspace of sourceProjects.length ? sourceProjects : [null]) {
|
||||
if (await requiresAnonymousAcquisition(tx, { issue, workspace: sourceWorkspace, agentId: reader.agentId!, runId: reader.scopeId })) {
|
||||
if (requiresAnonymousRepositoryAcquisition({ companyId: input.companyId, issue, agent, run,
|
||||
agentId: reader.agentId, runId: reader.scopeId, explicitRunId: reader.scopeId,
|
||||
project: issue.projectId ? projectsById.get(issue.projectId) : undefined,
|
||||
workspaceSourceProject: sourceWorkspace?.projectId ? projectsById.get(sourceWorkspace.projectId) : undefined })) {
|
||||
await reserveRootPolicy(tx, input.companyId, canonicalCwd, "anonymous_only");
|
||||
}
|
||||
}
|
||||
@@ -721,9 +711,9 @@ export function executionWorkspaceRepositoryService(db: Db) {
|
||||
}
|
||||
await requiresAnonymousAcquisition(tx, { issue, workspace, agentId: input.agentId, runId: input.runId });
|
||||
const [run] = await measureRunPhase("workspace.sources.inventory", async () => tx.select({ responsibleUserId: heartbeatRuns.responsibleUserId }).from(heartbeatRuns).where(and(eq(heartbeatRuns.id, input.runId), eq(heartbeatRuns.companyId, input.companyId))));
|
||||
const { actor } = await measureRunPhase("workspace.readers.identity", () => retainedRootReader(tx, input.companyId, {
|
||||
const [{ actor }] = await loadRepositoryRootReaders(tx, input.companyId, [{
|
||||
scopeId: input.runId, issueId: input.issueId, agentId: input.agentId, responsibleUserId: run?.responsibleUserId ?? null,
|
||||
}));
|
||||
}]);
|
||||
return { workspace, actor };
|
||||
}
|
||||
async function guardForPublication(input: PublicationInput) {
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { agents, heartbeatRuns, issues, projects, executionWorkspaceRepositoryAccess, type Db } from "@paperclipai/db";
|
||||
import { forbidden } from "../errors.js";
|
||||
import type { AuthorizationActor } from "./authorization.js";
|
||||
import { measureRunPhase } from "./execution-preparation-timing.js";
|
||||
import { resolveCoreTrustPreset } from "./trust-preset-resolver.js";
|
||||
|
||||
type ReaderDb = Db | Parameters<Parameters<Db["transaction"]>[0]>[0];
|
||||
type ReaderEntry = Pick<typeof executionWorkspaceRepositoryAccess.$inferSelect, "scopeId" | "issueId" | "agentId" | "responsibleUserId">;
|
||||
type ReaderAgent = Pick<typeof agents.$inferSelect, "id" | "companyId" | "permissions">;
|
||||
type ReaderIssue = Pick<typeof issues.$inferSelect, "id" | "companyId" | "projectId" | "executionPolicy">;
|
||||
type ReaderProject = Pick<typeof projects.$inferSelect, "companyId" | "executionWorkspacePolicy">;
|
||||
|
||||
export const rootReaderRunColumns = {
|
||||
id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
||||
responsibleUserId: heartbeatRuns.responsibleUserId, nativeIssueId: heartbeatRuns.nativeIssueId, issueId: heartbeatRuns.issueId,
|
||||
contextSnapshot: sql<Record<string, unknown>>`jsonb_build_object('issueId', ${heartbeatRuns.contextSnapshot}->'issueId', 'taskId', ${heartbeatRuns.contextSnapshot}->'taskId', 'executionPolicy', ${heartbeatRuns.contextSnapshot}->'executionPolicy')`,
|
||||
};
|
||||
|
||||
// This snapshot belongs to one guarded operation. Never retain it across admissions
|
||||
// or publications: identities and permissions must be read again next time.
|
||||
export async function loadRepositoryRootReaders<T extends ReaderEntry>(db: ReaderDb, companyId: string, entries: T[]) {
|
||||
const runIds = [...new Set(entries.map(entry => entry.scopeId))];
|
||||
const agentIds = [...new Set(entries.flatMap(entry => entry.agentId ? [entry.agentId] : []))];
|
||||
const issueIds = [...new Set(entries.flatMap(entry => entry.issueId ? [entry.issueId] : []))];
|
||||
const runs = new Map<string, Pick<typeof heartbeatRuns.$inferSelect, keyof typeof rootReaderRunColumns>>();
|
||||
const agentRows = new Map<string, ReaderAgent>();
|
||||
const issueRows = new Map<string, ReaderIssue>();
|
||||
await measureRunPhase("workspace.readers.identity", async () => {
|
||||
for (let offset = 0; offset < runIds.length; offset += 500) {
|
||||
for (const row of await db.select(rootReaderRunColumns).from(heartbeatRuns).where(and(
|
||||
eq(heartbeatRuns.companyId, companyId), inArray(heartbeatRuns.id, runIds.slice(offset, offset + 500))))) runs.set(row.id, row);
|
||||
}
|
||||
for (let offset = 0; offset < agentIds.length; offset += 500) {
|
||||
for (const row of await db.select({ id: agents.id, companyId: agents.companyId, permissions: agents.permissions }).from(agents).where(and(
|
||||
eq(agents.companyId, companyId), inArray(agents.id, agentIds.slice(offset, offset + 500))))) agentRows.set(row.id, row);
|
||||
}
|
||||
for (let offset = 0; offset < issueIds.length; offset += 500) {
|
||||
for (const row of await db.select({ id: issues.id, companyId: issues.companyId, projectId: issues.projectId,
|
||||
executionPolicy: issues.executionPolicy }).from(issues).where(and(
|
||||
eq(issues.companyId, companyId), inArray(issues.id, issueIds.slice(offset, offset + 500))))) issueRows.set(row.id, row);
|
||||
}
|
||||
});
|
||||
return entries.map(entry => {
|
||||
const run = runs.get(entry.scopeId);
|
||||
const agent = entry.agentId ? agentRows.get(entry.agentId) : undefined;
|
||||
const issue = entry.issueId ? issueRows.get(entry.issueId) : undefined;
|
||||
if (!run || !agent || !issue || run.agentId !== entry.agentId || run.responsibleUserId !== entry.responsibleUserId
|
||||
|| (run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId ?? run.issueId) !== entry.issueId) {
|
||||
throw forbidden("Repository root retains a reader whose identity is no longer available");
|
||||
}
|
||||
const actor: AuthorizationActor = { type: "agent", source: "agent_jwt", companyId,
|
||||
agentId: agent.id, runId: run.id, onBehalfOfUserId: entry.responsibleUserId === "local-board" ? null : entry.responsibleUserId };
|
||||
return { entry, run, agent, issue, actor };
|
||||
});
|
||||
}
|
||||
|
||||
export function requiresAnonymousRepositoryAcquisition(input: {
|
||||
companyId: string;
|
||||
issue: ReaderIssue;
|
||||
agent: ReaderAgent | undefined;
|
||||
project: ReaderProject | undefined;
|
||||
workspaceSourceProject: ReaderProject | undefined;
|
||||
run: Pick<typeof heartbeatRuns.$inferSelect, keyof typeof rootReaderRunColumns> | undefined;
|
||||
agentId: string | null;
|
||||
runId: string | null;
|
||||
explicitRunId?: string | null;
|
||||
}) {
|
||||
const { companyId, issue, agent, project, workspaceSourceProject, run, agentId, runId } = input;
|
||||
if (agentId && !agent) throw forbidden("Task repository agent is unavailable");
|
||||
if (runId && (!run || (input.explicitRunId && run.agentId !== agentId)
|
||||
|| (run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId ?? run.issueId) !== issue.id)) {
|
||||
throw forbidden("Task repository run identity is unavailable");
|
||||
}
|
||||
const trust = resolveCoreTrustPreset({ companyId, agent, project, workspaceSourceProject, issue,
|
||||
run: run ? { companyId: run.companyId, executionPolicy: run.contextSnapshot?.executionPolicy } : null });
|
||||
if (trust.kind === "denied") throw forbidden(trust.detail);
|
||||
return trust.kind === "low_trust_review";
|
||||
}
|
||||
Reference in new issue
Block a user