mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 18:26:40 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Every board request goes through authorization, and the server runs
all requests on one Node main thread
> - The cost on that thread is per database statement
> - One `GET /api/issues/:id` makes several authorization decisions (the
issue, its project, mentioned projects, related task edges), and each
decision reads `instance_user_roles` and `company_memberships` again
> - On a task with a blocker and a project mention, that is 4 role reads
and 4 membership reads in the authorization service for one request, all
with the same answer
> - This pull request shares those two reads among the decisions of one
GET or HEAD request
> - The benefit is fewer statements on every polled board read, with no
authorization cache across requests
## Linked Issues or Issue Description
**What existing behavior does this improve?**
`authorizationService.isInstanceAdmin` and `getActiveMembership` read
the database for each decision in a request.
**Subsystem affected**
Server: authorization service and app middleware.
**Current behavior**
`GET /api/issues/:id` for a plain task sends 3 `instance_user_roles`
reads and 3 `company_memberships` reads (1 each in the session
middleware, 2 each in authorization). A richer task sends 5 and 5.
**Proposed behavior**
A new middleware, `authorizationReadScope`, opens a per-request memo for
GET and HEAD requests. For a session actor, it seeds the instance admin
answer that the session middleware read at the start of the same
request. Inside the scope, `isInstanceAdmin`, `getActiveMembership`,
`issueReadSqlCondition` and `projectReadSqlCondition` read each key
once. The plain task now sends 1 role read (the session middleware) and
2 membership reads (1 in the session middleware, 1 in authorization).
The issue list and summaries routes also drop their extra role reads.
Other methods, background work and other callers keep the old reads. The
memo ends with the request, so a revoked membership takes effect on the
very next request.
**Reason and benefit**
Fewer statements per polled read, with the same decisions.
**Breaking changes**
None.
## Memo lifetime rule
- The memo exists only for GET and HEAD requests.
- It wraps `res.writeHead`, `res.flushHeaders` and `res.write` once and
never reassigns them, so hooks that later middleware installs on the
response stay in place.
- It closes at the first of: `res.writeHead`, `res.flushHeaders`,
`res.write`, `finish` or `close`. After it closes, every read goes to
the database. A stream, a long poll, a download or a callback that runs
after the response therefore never uses the memo.
- Each entry is used for at most 2 seconds
(`AUTHORIZATION_READ_MAX_AGE_MS`), then read again. Correctness does not
depend on a list of routes.
- A rejected read is removed from the memo, so the next decision reads
again.
- Nothing is kept across requests.
Long-lived and streaming routes that I checked:
- `GET /plugins/:pluginId/bridge/stream/:channel` (SSE): authorizes once
at connect with `assertBoardOrgAccess` and `assertCompanyAccess` on
`req.actor`. It does not re-authorize per event. The memo closes at
`writeHead`.
- Streaming downloads (`object.stream.pipe(res)` in assets, access and
issue attachments): they authorize before the stream starts. The memo
closes at the first write.
- `POST /board/chat/stream` and `POST
/companies/:companyId/skill-sources/discover` (NDJSON): POST, so no
memo.
- Live events, plugin API, runner and terminal WebSockets: they use
`server.on("upgrade")`, so they do not pass through Express middleware.
## What Changed
- New `server/src/services/authorization-read-scope.ts`:
`authorizationReadScope` (an AsyncLocalStorage scope for GET and HEAD)
and `readOncePerRequest`.
- `authorizationService.isInstanceAdmin` and `getActiveMembership` use
`readOncePerRequest`. The existing batch snapshot memo still comes
first.
- `issueReadSqlCondition` and `projectReadSqlCondition` use the same
`admin:<userId>` key for their instance admin check.
- For `source: "session"` board actors, the scope seeds `admin:<userId>`
with `req.actor.isInstanceAdmin`, which the session middleware read from
`instance_user_roles` in the same request.
- `createApp` mounts `authorizationReadScope` after the actor and Cloud
control middleware.
- `authorizationReadScope` closes the memo when the response starts or
ends, ignores entries older than 2 seconds, and drops rejected reads.
- New exported `readInstanceAdmin(db, userId)`: the one predicate for
`isInstanceAdmin`, `issueReadSqlCondition` and
`projectReadSqlCondition`.
- New test `authorization-request-reads.test.ts`: one role read and one
membership read in authorization per issue read, and a removed
membership gives 404 on the next request.
## Verification
- Red: without the scope, the issue read sends 3 `instance_user_roles`
statements (`expected ... to have a length of 2 but got 3`).
- Green: with the scope, it sends 1 role read and 2 membership reads.
The issue list plus the summaries request send 1 role read each.
- Revocation: delete the membership row, and the next GET returns 404.
HEAD returns 200, then 404 after the revoke.
- Stream: a GET that flushes headers, then re-reads the membership after
it is deleted mid-stream, sees the deletion.
- Background callback: a callback scheduled in a GET that runs after the
response and after a revoke reads again and sees the deletion.
- Age bound: an entry older than 2 seconds is read again. A rejected
read is not kept.
- Seed predicate: for users with an `instance_admin` row, another role
row, and no row, the session middleware value equals
`readInstanceAdmin`.
- Other actors: an agent key, a board key and a cloud tenant actor get
no seeded value.
- Response hooks: a later middleware that wraps `res.writeHead` and
`res.write` keeps working for JSON and streamed responses. This test
fails with a version that restores the original methods.
- Red: with the previous implementation, the stream, callback, age and
rejection tests fail.
- `pnpm exec vitest run` on `access-routes-hidden-floor`,
`access-routes-permissions-upgrade`, `access-service`,
`agent-auth-middleware`, `authorization-privacy-query`,
`authorization-request-reads`, `authorization-run-context`,
`authorization-service`: 8 files, 177 tests pass. Also
`inbox-mine-candidates`, `issue-read-filter-batching`,
`attention-summary-counts`, `issues-unread-inbox`,
`live-runs-batched-reads`, `privacy-production-review`,
`activity-routes`, `authorization-privacy-query`,
`muse-native-visibility`: 224 tests pass (one
`privacy-production-review` test failed once under machine load and
passed on two reruns).
- `pnpm --filter @paperclipai/server typecheck` passes.
## Risks
- This changes where authorization gets two facts inside one request, so
it needs a human review before merge.
- A GET route that changes a membership or an instance role and then
authorizes again before its response starts, within 2 seconds, would see
the earlier value. GET routes do not change memberships or roles. POST,
PATCH, PUT and DELETE requests do not use the memo.
- A board actor built by the session middleware still carries its own
connect-time `memberships` and `isInstanceAdmin` for the life of a
stream. That is the existing behavior, and this change does not affect
it.
- Nothing is cached across requests.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
291 lines
14 KiB
TypeScript
291 lines
14 KiB
TypeScript
import express from "express";
|
|
import request from "supertest";
|
|
import { eq } from "drizzle-orm";
|
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
import { and, eq as equals } from "drizzle-orm";
|
|
import { authUsers, companyMemberships, instanceUserRoles, issues, projects } from "@paperclipai/db";
|
|
import { actorMiddleware } from "../middleware/auth.js";
|
|
import { errorHandler } from "../middleware/error-handler.js";
|
|
import { issueRoutes } from "../routes/issues.js";
|
|
import { AUTHORIZATION_READ_MAX_AGE_MS, authorizationReadScope, readOncePerRequest } from "../services/authorization-read-scope.js";
|
|
import { readInstanceAdmin } from "../services/authorization.js";
|
|
import { meterDatabaseStatements } from "./helpers/database-statement-meter.js";
|
|
import { describeEmbeddedPostgres, seedCompanyWithBoardAccess, useEmbeddedPostgres } from "./helpers/route-test-harness.js";
|
|
|
|
describeEmbeddedPostgres("authorization reads per request", () => {
|
|
const ctx = useEmbeddedPostgres("authorization-request-reads-");
|
|
|
|
function appFor(userId: string) {
|
|
const app = express();
|
|
app.use(express.json());
|
|
app.use(actorMiddleware(ctx.db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => ({
|
|
session: { id: `session-${userId}`, userId },
|
|
user: { id: userId, name: "Reader", email: `${userId}@example.test` },
|
|
}),
|
|
}));
|
|
app.use(authorizationReadScope);
|
|
app.use("/api", issueRoutes(ctx.db, {} as never));
|
|
app.use(errorHandler);
|
|
return app;
|
|
}
|
|
|
|
async function seed(name: string) {
|
|
const company = await seedCompanyWithBoardAccess(ctx.db, name);
|
|
const now = new Date();
|
|
await ctx.db.insert(authUsers).values({
|
|
id: company.userId, name, email: `${company.userId}@example.test`, emailVerified: true, createdAt: now, updatedAt: now,
|
|
});
|
|
const [project] = await ctx.db.insert(projects).values({ companyId: company.companyId, name: `${name} project` }).returning();
|
|
const [issue] = await ctx.db.insert(issues).values({
|
|
companyId: company.companyId, title: `${name} task`, status: "todo", projectId: project!.id, createdByUserId: company.userId,
|
|
}).returning();
|
|
return { ...company, issue: issue! };
|
|
}
|
|
|
|
it("reads the instance role once per request and the membership once for the authorization of one issue read", async () => {
|
|
const { host, port } = (ctx.db.$client as unknown as { options: { host: string[]; port: number[] } }).options;
|
|
const meter = meterDatabaseStatements(ctx.db, `postgres://${host[0]}:${port[0]}/authorization`);
|
|
try {
|
|
const fixture = await seed("Reads");
|
|
const app = appFor(fixture.userId);
|
|
await request(app).get(`/api/issues/${fixture.issue.id}`).expect(200);
|
|
meter.startLog();
|
|
await request(app).get(`/api/issues/${fixture.issue.id}`).expect(200);
|
|
const log = meter.stopLog();
|
|
const roleReads = log.filter((entry) => /from "instance_user_roles"/.test(entry.sql));
|
|
const membershipReads = log.filter((entry) => /from "company_memberships"/.test(entry.sql));
|
|
expect(roleReads).toHaveLength(1);
|
|
expect(membershipReads).toHaveLength(2);
|
|
|
|
meter.startLog();
|
|
await request(app).get(`/api/companies/${fixture.companyId}/issues`).expect(200);
|
|
await request(app).get(`/api/companies/${fixture.companyId}/issues/summaries?ids=${fixture.issue.id}`).expect(200);
|
|
const listLog = meter.stopLog();
|
|
expect(listLog.filter((entry) => /from "instance_user_roles"/.test(entry.sql))).toHaveLength(2);
|
|
} finally {
|
|
meter.uninstall();
|
|
}
|
|
}, 60_000);
|
|
|
|
it("denies the very next request after the membership is removed", async () => {
|
|
const fixture = await seed("Revoked");
|
|
const app = appFor(fixture.userId);
|
|
await request(app).get(`/api/issues/${fixture.issue.id}`).expect(200);
|
|
await ctx.db.delete(companyMemberships).where(eq(companyMemberships.principalId, fixture.userId));
|
|
await request(app).get(`/api/issues/${fixture.issue.id}`).expect(404);
|
|
}, 60_000);
|
|
function membershipRead(companyId: string, userId: string) {
|
|
return readOncePerRequest(`membership:${companyId}:user:${userId}`, () => ctx.db.select({ id: companyMemberships.id })
|
|
.from(companyMemberships)
|
|
.where(and(equals(companyMemberships.companyId, companyId), equals(companyMemberships.principalId, userId),
|
|
equals(companyMemberships.status, "active")))
|
|
.then((rows) => rows.length > 0));
|
|
}
|
|
|
|
function streamApp(userId: string, companyId: string, events: { afterRevoke: Promise<void>; late: (allowed: boolean) => void }) {
|
|
const app = express();
|
|
app.use(actorMiddleware(ctx.db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => ({
|
|
session: { id: `session-${userId}`, userId },
|
|
user: { id: userId, name: "Reader", email: `${userId}@example.test` },
|
|
}),
|
|
}));
|
|
app.use(authorizationReadScope);
|
|
app.get("/stream", async (_req, res) => {
|
|
const atConnect = await membershipRead(companyId, userId);
|
|
res.writeHead(200, { "Content-Type": "text/event-stream" });
|
|
res.flushHeaders();
|
|
res.write(`data: ${atConnect}\n\n`);
|
|
await events.afterRevoke;
|
|
res.end(`data: ${await membershipRead(companyId, userId)}\n\n`);
|
|
});
|
|
app.get("/later", async (_req, res) => {
|
|
await membershipRead(companyId, userId);
|
|
setTimeout(() => { void events.afterRevoke.then(() => membershipRead(companyId, userId)).then(events.late); }, 0);
|
|
res.json({ ok: true });
|
|
});
|
|
return app;
|
|
}
|
|
|
|
it("re-reads the membership during a stream after it is removed", async () => {
|
|
const fixture = await seed("Stream");
|
|
let revoked!: () => void;
|
|
const afterRevoke = new Promise<void>((resolve) => { revoked = resolve; });
|
|
const app = streamApp(fixture.userId, fixture.companyId, { afterRevoke, late: () => {} });
|
|
const pending = request(app).get("/stream").buffer(true).parse((res, done) => {
|
|
let text = "";
|
|
res.on("data", (chunk: Buffer) => {
|
|
text += chunk.toString();
|
|
if (text.includes("data: true") && !text.includes("data: false")) {
|
|
void ctx.db.delete(companyMemberships).where(eq(companyMemberships.principalId, fixture.userId)).then(revoked);
|
|
}
|
|
});
|
|
res.on("end", () => done(null, text));
|
|
});
|
|
const response = await pending;
|
|
expect(response.body).toBe("data: true\n\ndata: false\n\n");
|
|
}, 60_000);
|
|
|
|
it("re-reads in a callback that runs after the response and after a revoke", async () => {
|
|
const fixture = await seed("Later");
|
|
let revoked!: () => void;
|
|
const afterRevoke = new Promise<void>((resolve) => { revoked = resolve; });
|
|
let late!: (allowed: boolean) => void;
|
|
const lateResult = new Promise<boolean>((resolve) => { late = resolve; });
|
|
const app = streamApp(fixture.userId, fixture.companyId, { afterRevoke, late });
|
|
await request(app).get("/later").expect(200);
|
|
await ctx.db.delete(companyMemberships).where(eq(companyMemberships.principalId, fixture.userId));
|
|
revoked();
|
|
expect(await lateResult).toBe(false);
|
|
}, 60_000);
|
|
|
|
it("seeds the instance role from a session read that matches the authorization predicate", async () => {
|
|
const now = new Date();
|
|
const cases = [
|
|
{ userId: `user-admin-${Date.now()}`, roles: ["instance_admin"], expected: true },
|
|
{ userId: `user-other-${Date.now()}`, roles: ["instance_viewer"], expected: false },
|
|
{ userId: `user-none-${Date.now()}`, roles: [], expected: false },
|
|
];
|
|
for (const entry of cases) {
|
|
await ctx.db.insert(authUsers).values({ id: entry.userId, name: entry.userId, email: `${entry.userId}@example.test`,
|
|
emailVerified: true, createdAt: now, updatedAt: now });
|
|
if (entry.roles.length) await ctx.db.insert(instanceUserRoles).values(entry.roles.map((role) => ({ userId: entry.userId, role })));
|
|
const app = express();
|
|
app.use(actorMiddleware(ctx.db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => ({ session: { id: `session-${entry.userId}`, userId: entry.userId },
|
|
user: { id: entry.userId, name: "Admin check", email: `${entry.userId}@example.test` } }),
|
|
}));
|
|
app.get("/actor", (req, res) => { res.json({ isInstanceAdmin: req.actor.isInstanceAdmin }); });
|
|
const seeded = (await request(app).get("/actor").expect(200)).body.isInstanceAdmin;
|
|
expect({ seeded, read: await readInstanceAdmin(ctx.db, entry.userId) }).toEqual({ seeded: entry.expected, read: entry.expected });
|
|
}
|
|
await ctx.db.delete(instanceUserRoles);
|
|
}, 60_000);
|
|
|
|
it("treats HEAD like GET for an authorized read and denies it after a revoke", async () => {
|
|
const fixture = await seed("Head");
|
|
const app = appFor(fixture.userId);
|
|
await request(app).head(`/api/issues/${fixture.issue.id}`).expect(200);
|
|
await ctx.db.delete(companyMemberships).where(eq(companyMemberships.principalId, fixture.userId));
|
|
await request(app).head(`/api/issues/${fixture.issue.id}`).expect(404);
|
|
}, 60_000);
|
|
});
|
|
|
|
describe("authorization read scope boundaries", () => {
|
|
afterEach(() => { vi.useRealTimers(); });
|
|
|
|
function countingApp(actor: Express.Request["actor"] = { type: "board", source: "session", userId: "user-1", isInstanceAdmin: false }) {
|
|
let reads = 0;
|
|
const gates: Array<() => void> = [];
|
|
const app = express();
|
|
app.use((req, _res, next) => {
|
|
req.actor = actor;
|
|
next();
|
|
});
|
|
app.use(authorizationReadScope);
|
|
app.all("/read", async (req, res) => {
|
|
const read = () => readOncePerRequest("membership:company-1:user:user-1", async () => ++reads);
|
|
const first = await read();
|
|
if (req.query.wait) await new Promise<void>((resolve) => gates.push(resolve));
|
|
res.json({ first, second: await read(), admin: await readOncePerRequest("admin:user-1", async () => "read") });
|
|
});
|
|
return { app, gates, reads: () => reads };
|
|
}
|
|
|
|
it("shares a read only inside one GET request", async () => {
|
|
const { app, reads } = countingApp();
|
|
expect((await request(app).get("/read")).body).toEqual({ first: 1, second: 1, admin: false });
|
|
expect((await request(app).get("/read")).body).toEqual({ first: 2, second: 2, admin: false });
|
|
expect(reads()).toBe(2);
|
|
});
|
|
|
|
it("reads again for every call in a write request", async () => {
|
|
const { app, reads } = countingApp();
|
|
expect((await request(app).post("/read")).body).toEqual({ first: 1, second: 2, admin: "read" });
|
|
expect((await request(app).delete("/read")).body).toEqual({ first: 3, second: 4, admin: "read" });
|
|
expect(reads()).toBe(4);
|
|
});
|
|
|
|
it("keeps overlapping GET requests apart", async () => {
|
|
const { app, gates } = countingApp();
|
|
const slow = request(app).get("/read?wait=1").then((res) => res.body);
|
|
while (gates.length === 0) await new Promise((resolve) => setTimeout(resolve, 5));
|
|
const fast = await request(app).get("/read");
|
|
gates[0]!();
|
|
expect(await slow).toEqual({ first: 1, second: 1, admin: false });
|
|
expect(fast.body).toEqual({ first: 2, second: 2, admin: false });
|
|
});
|
|
it("ignores an entry older than the age bound", async () => {
|
|
const { app, gates, reads } = countingApp();
|
|
vi.useFakeTimers({ toFake: ["Date"] });
|
|
const pending = request(app).get("/read?wait=1").then((res) => res.body);
|
|
while (gates.length === 0) await new Promise((resolve) => setTimeout(resolve, 5));
|
|
vi.setSystemTime(Date.now() + AUTHORIZATION_READ_MAX_AGE_MS + 1);
|
|
gates[0]!();
|
|
expect(await pending).toEqual({ first: 1, second: 2, admin: "read" });
|
|
expect(reads()).toBe(2);
|
|
});
|
|
|
|
it("does not keep a rejected read", async () => {
|
|
let attempts = 0;
|
|
const app = express();
|
|
app.use((req, _res, next) => {
|
|
req.actor = { type: "board", source: "session", userId: "user-1", isInstanceAdmin: false };
|
|
next();
|
|
});
|
|
app.use(authorizationReadScope);
|
|
app.get("/read", async (_req, res) => {
|
|
const read = () => readOncePerRequest("membership:company-1:user:user-1", async () => {
|
|
attempts += 1;
|
|
if (attempts === 1) throw new Error("connection reset");
|
|
return attempts;
|
|
});
|
|
const first = await read().catch((error: Error) => error.message);
|
|
res.json({ first, second: await read(), third: await read() });
|
|
});
|
|
expect((await request(app).get("/read")).body).toEqual({ first: "connection reset", second: 2, third: 2 });
|
|
});
|
|
|
|
it.each([
|
|
["an agent key", { type: "agent", source: "agent_key", agentId: "agent-1", companyId: "company-1" }],
|
|
["a board key", { type: "board", source: "board_key", userId: "user-1", isInstanceAdmin: true }],
|
|
["a cloud tenant", { type: "board", source: "cloud_tenant", userId: "user-1", isInstanceAdmin: true }],
|
|
] as const)("seeds nothing for %s", async (_label, actor) => {
|
|
const { app } = countingApp(actor as Express.Request["actor"]);
|
|
expect((await request(app).get("/read")).body.admin).toBe("read");
|
|
});
|
|
it("keeps response hooks that later middleware installs", async () => {
|
|
const app = express();
|
|
app.use(authorizationReadScope);
|
|
app.use((_req, res, next) => {
|
|
const { writeHead, write } = res;
|
|
let writes = 0;
|
|
res.writeHead = function (this: typeof res, ...args: Parameters<typeof res.writeHead>) {
|
|
this.setHeader("x-late-hook", "kept");
|
|
return writeHead.apply(this, args);
|
|
} as typeof res.writeHead;
|
|
res.write = function (this: typeof res, ...args: Parameters<typeof res.write>) {
|
|
writes += 1;
|
|
res.locals.writes = writes;
|
|
return write.apply(this, args);
|
|
} as typeof res.write;
|
|
next();
|
|
});
|
|
app.get("/json", (_req, res) => { res.json({ ok: true }); });
|
|
app.get("/stream", (_req, res) => {
|
|
res.write("a");
|
|
res.write("b");
|
|
res.end(String(res.locals.writes));
|
|
});
|
|
const json = await request(app).get("/json").expect(200);
|
|
expect(json.headers["x-late-hook"]).toBe("kept");
|
|
const stream = await request(app).get("/stream").expect(200);
|
|
expect(stream.headers["x-late-hook"]).toBe("kept");
|
|
expect(stream.text).toBe("ab2");
|
|
});
|
|
});
|