Files
PaperClipAI/server/src/__tests__/assets.test.ts
T
DottaandPaperclip 3447609d22 fix(runner): stream and page large API responses within capture budgets (#14301)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use governed API tools to inspect task evidence.
> - Large API results become saved assets with short previews.
> - Reading an asset through the same tool used to create another asset,
so the agent could not reach the rest of the evidence.
> - The 10 MiB response cap also blocked useful large results. Removing
all bounds allowed excessive disk use.
> - This pull request streams responses up to 1 GiB and makes saved text
readable in bounded pages. It adds durable run budgets and capture
admission limits.
> - Agents can inspect complete evidence while tool results, memory use,
and capture work stay bounded.

## Linked Issues or Issue Description

**What happened?**
A large response became an asset. Reading that asset returned another
asset and the same preview. Responses above 10 MiB failed before the
agent could read any page.

**Expected behavior**
The agent can fetch a large response and read its saved text to EOF.
Each page stays bounded. New snapshots have a generous finite limit and
a durable run budget. Existing larger assets remain readable through
byte ranges.

**Steps to reproduce**
1. Call a GET operation that returns more than 10 MiB of text or JSON.
2. Before the fix, the tool returns `api_transport_failure`.
3. With this change, responses up to 1 GiB become streamed snapshots
with artifact references.
4. Read `GET /api/assets/{assetId}/content` with `responseText:
{offsetBytes: 0, limitBytes: 8192}`. Follow `nextOffsetBytes` until
null.

Related work: #14186 added the API fallback tools. #14218 bounded API
discovery.

## What Changed

- Add authenticated UTF-8 text windows to `call_api`, with byte offsets
and total size. Keep each page at or below 24 KiB.
- Stream new responses above 24 KiB through private temporary files into
company-owned assets. Bound each capture to 1 GiB of decoded bytes.
Reject oversized declared lengths before reading and count streamed
bytes before writing.
- Reserve capture budget in the run record before spilling. Allow 4 GiB
per run. Settle successful captures to their actual size. Failed or
interrupted captures retain their full 1 GiB reservation. Run restarts
do not reset the budget.
- Enforce a 20 GiB company snapshot quota with database reservations.
Count legacy snapshots and unfinished storage work across runs and
processes. Asset deletion frees quota.
- Limit large captures to two per company and four per server process.
Hold slots through storage upload and temporary-file cleanup. Use a
10-minute download deadline and 30-second connection/idle-read timeouts.
- Return explicit size, budget, busy, and timeout errors. Preserve
unknown outcomes for mutations whose response cannot be captured.
- Read saved assets through authenticated storage ranges, with at most
two extra bytes for UTF-8 and EOF handling. Unpaged reads return the
existing asset and digest with a bounded preview. Reads create no copies
and do not consume capture budget.
- Keep existing assets above 1 GiB readable in pages. Use safe integer
offsets and PostgreSQL `bigint` asset sizes.
- Stream large S3 uploads through ordered multipart requests. Abort
failed uploads and remove partial local files.
- Revalidate run authority during downloads. Keep company authorization,
GET-only text paging, redirect denial, and mutation replay receipts.
- Document the separate 10 MiB upload limits. This PR does not raise
memory-buffered attachment ingestion limits. Future large video uploads
need streamed ingestion and storage quotas.

## Verification

- Full workspace `pnpm -r typecheck` and `pnpm build` pass after
rebasing on master.
- Focused API and response tests: 1,761 pass. Cover declared and chunked
oversize responses, incorrect Content-Length, exact-limit success,
active-stream deadline, cancellation, cleanup, concurrency admission,
and mutation outcome handling.
- Real HTTP integration: 28 tests pass, including runnerd → PRP →
authority → HTTP, a 12 MiB snapshot, final-page/EOF reads, cross-company
denial, a persisted 3 GiB sparse asset, and large mutation receipt
replay.
- The HTTP suite verifies durable run-budget accounting, simultaneous
runs competing for company quota, legacy snapshot accounting, deletion
refunds, failed-storage reservations, cleaned-failure refunds,
metadata-rollback cleanup refunds, preservation after a lost commit
acknowledgement, and small/saved reads after capture-budget exhaustion.
- A standalone proof streams exactly 1 GiB through the production
capture helper, verifies the final bytes, and removes its temporary
file. It uses repeated 256 KiB chunks and records a peak process RSS of
191 MiB.
- Earlier storage verification covers exact S3 multipart boundaries,
cleanup/abort failures, and a 17 MiB transfer through the real AWS SDK
to a local HTTP S3 endpoint. No cloud S3 qualification was run for this
follow-up.
- The local full test run was interrupted for the company-quota changes.
A later targeted run hit exhausted macOS shared-memory slots before
tests started; two unattached PostgreSQL segments with dead owners were
reclaimed before retrying. All 55 current-head checks pass at
`aebb80ceeeee77d5a56b67bfffd835f2f846878c`, including the full CI test
suite, typecheck, build, browser suites, security scan, and Greptile
(5/5). There are no unresolved review threads. The combined rebased test
catalog also passes (48 tests).
- Earlier paging acceptance passed Daytona and separate staging at
`7739879e9`. Those runs predate the streaming and budget changes.

## Risks

- The 1 GiB response cap and 10-minute active-download deadline are
intentional product limits. Larger live results must use endpoint
pagination or a direct file workflow. Existing larger assets remain
readable through bounded ranges.
- A durable 20 GiB company snapshot quota counts stored runner-api
assets and active/orphan reservations across runs and processes. The
operator can set PAPERCLIP_RUNNER_API_COMPANY_CAPTURE_MAX_BYTES to a
finite value of at least 1 GiB. Deleting snapshots frees capacity;
possible orphan storage must be reconciled before releasing its
reservation.
- A failed capture uses its full reservation. A new large capture needs
a full 1 GiB available, even if it later completes at a smaller size.
Small reads and existing asset pages remain available.
- Concurrency limits apply per server process. The run byte budget is
shared through the database.
- The `integer` to `bigint` migration rewrites asset metadata and takes
an exclusive table lock. File bytes stay in storage.
- A live endpoint is fetched once before returning its snapshot.
Continue reading the saved artifact for stable pages. Mutations may
commit before any size or transport error; inspect state before
retrying.
- Attachment uploads and native file handoffs still default to 10 MiB.
Raising buffered ingestion paths to GiB sizes is separate work.

## Model Used

OpenAI Codex, based on GPT-6, with code execution and repository tools.
The runtime does not expose an exact serving model variant or
context-window size. The earlier paging work also used browser testing
and subagents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-28 10:36:32 -05:00

520 lines
18 KiB
TypeScript

import { beforeEach, describe, expect, it, vi } from "vitest";
import express from "express";
import { Readable } from "node:stream";
import request from "supertest";
import { MAX_ATTACHMENT_BYTES } from "../attachment-types.js";
import type { StorageService } from "../storage/types.js";
const { createAssetMock, getAssetByIdMock, logActivityMock } = vi.hoisted(() => ({
createAssetMock: vi.fn(),
getAssetByIdMock: vi.fn(),
logActivityMock: vi.fn(),
}));
function registerModuleMocks() {
vi.doMock("../services/activity-log.js", () => ({
logActivity: logActivityMock,
}));
vi.doMock("../services/assets.js", () => ({
assetService: vi.fn(() => ({
create: createAssetMock,
getById: getAssetByIdMock,
})),
}));
vi.doMock("../services/index.js", () => ({
assetService: vi.fn(() => ({
create: createAssetMock,
getById: getAssetByIdMock,
})),
logActivity: logActivityMock,
}));
}
function createAsset() {
const now = new Date("2026-01-01T00:00:00.000Z");
return {
id: "asset-1",
companyId: "company-1",
provider: "local",
objectKey: "assets/abc",
contentType: "image/png",
byteSize: 40,
sha256: "sha256-sample",
originalFilename: "logo.png",
createdByAgentId: null,
createdByUserId: "user-1",
createdAt: now,
updatedAt: now,
};
}
type TestStorageService = StorageService & {
__calls: {
putFileInputs: Array<{
companyId: string;
namespace: string;
originalFilename: string | null;
contentType: string;
body: Buffer;
}>;
};
};
function createStorageService(contentType = "image/png"): TestStorageService {
const calls: TestStorageService["__calls"] = { putFileInputs: [] };
const putFile: StorageService["putFile"] = async (input: {
companyId: string;
namespace: string;
originalFilename: string | null;
contentType: string;
body: Buffer;
}) => {
calls.putFileInputs.push(input);
return {
provider: "local_disk" as const,
objectKey: `${input.namespace}/${input.originalFilename ?? "upload"}`,
contentType: contentType || input.contentType,
byteSize: input.body.length,
sha256: "sha256-sample",
originalFilename: input.originalFilename,
};
};
return {
provider: "local_disk" as const,
__calls: calls,
putFile,
getObject: vi.fn(),
headObject: vi.fn(),
deleteObject: vi.fn(),
};
}
async function createApp(storage: ReturnType<typeof createStorageService>) {
const { assetRoutes } = await vi.importActual<typeof import("../routes/assets.js")>("../routes/assets.js");
const app = express();
app.use((req, _res, next) => {
req.actor = {
type: "board",
source: "local_implicit",
userId: "user-1",
};
next();
});
app.use("/api", assetRoutes({} as any, storage));
return app;
}
async function requestApp(
app: express.Express,
buildRequest: (baseUrl: string) => request.Test,
) {
const { createServer } = await vi.importActual<typeof import("node:http")>("node:http");
const server = createServer(app);
try {
await new Promise<void>((resolve) => {
server.listen(0, "127.0.0.1", resolve);
});
const address = server.address();
if (!address || typeof address === "string") {
throw new Error("Expected HTTP server to listen on a TCP port");
}
return await buildRequest(`http://127.0.0.1:${address.port}`);
} finally {
if (server.listening) {
await new Promise<void>((resolve, reject) => {
server.close((error) => {
if (error) reject(error);
else resolve();
});
});
}
}
}
describe("POST /api/companies/:companyId/assets/images", () => {
beforeEach(() => {
vi.resetModules();
vi.doUnmock("../services/activity-log.js");
vi.doUnmock("../services/assets.js");
vi.doUnmock("../services/index.js");
vi.doUnmock("../routes/assets.js");
vi.doUnmock("../routes/authz.js");
vi.doUnmock("../middleware/index.js");
registerModuleMocks();
vi.clearAllMocks();
createAssetMock.mockReset();
getAssetByIdMock.mockReset();
logActivityMock.mockReset();
});
it("accepts PNG image uploads and returns an asset path", async () => {
const png = createStorageService("image/png");
const app = await createApp(png);
createAssetMock.mockResolvedValue(createAsset());
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/assets/images")
.field("namespace", "goals")
.attach("file", Buffer.from("png"), "logo.png"),
);
expect([200, 201], JSON.stringify(res.body)).toContain(res.status);
expect(res.body.contentPath).toBe("/api/assets/asset-1/content");
expect(createAssetMock).toHaveBeenCalledTimes(1);
expect(png.__calls.putFileInputs[0]).toMatchObject({
companyId: "company-1",
namespace: "assets/goals",
originalFilename: "logo.png",
contentType: "image/png",
body: expect.any(Buffer),
});
});
it("accepts namespaces that hold identity-provider user ids", async () => {
const png = createStorageService("image/png");
const app = await createApp(png);
createAssetMock.mockResolvedValue(createAsset());
const namespace = "profiles/oidc:example|jane.example@example.com";
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/assets/images")
.field("namespace", namespace)
.attach("file", Buffer.from("png"), "avatar.png"),
);
expect(res.status, JSON.stringify(res.body)).toBe(201);
expect(png.__calls.putFileInputs[0]).toMatchObject({
companyId: "company-1",
namespace: `assets/${namespace}`,
originalFilename: "avatar.png",
contentType: "image/png",
});
});
it("rejects namespaces with characters outside the accepted set", async () => {
const png = createStorageService("image/png");
const app = await createApp(png);
createAssetMock.mockResolvedValue(createAsset());
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/assets/images")
.field("namespace", "profiles/bad name!")
.attach("file", Buffer.from("png"), "avatar.png"),
);
expect(res.status).toBe(400);
expect(res.body.error).toContain("namespace");
expect(res.body.details?.[0]?.path).toEqual(["namespace"]);
expect(png.__calls.putFileInputs).toHaveLength(0);
expect(createAssetMock).not.toHaveBeenCalled();
});
it("rejects namespaces that hold a dot path segment", async () => {
const png = createStorageService("image/png");
const app = await createApp(png);
createAssetMock.mockResolvedValue(createAsset());
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/assets/images")
.field("namespace", "profiles/../secrets")
.attach("file", Buffer.from("png"), "avatar.png"),
);
expect(res.status).toBe(400);
expect(res.body.error).toContain("namespace");
expect(png.__calls.putFileInputs).toHaveLength(0);
expect(createAssetMock).not.toHaveBeenCalled();
});
it("allows supported non-image attachments outside the company logo flow", async () => {
const text = createStorageService("text/plain");
const app = await createApp(text);
createAssetMock.mockResolvedValue({
...createAsset(),
contentType: "text/plain",
originalFilename: "note.txt",
});
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/assets/images")
.field("namespace", "issues/drafts")
.attach("file", Buffer.from("hello"), { filename: "note.txt", contentType: "text/plain" }),
);
expect([200, 201]).toContain(res.status);
expect(res.body.contentPath).toBe("/api/assets/asset-1/content");
expect(res.body.contentType).toBe("text/plain");
});
it("names the limit in human units when a file exceeds the attachment cap", async () => {
const app = await createApp(createStorageService());
createAssetMock.mockResolvedValue(createAsset());
const file = Buffer.alloc(MAX_ATTACHMENT_BYTES + 1, "a");
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/assets/images")
.attach("file", file, "too-large.png"),
);
expect(res.status).toBe(422);
expect(res.body.error).toBe("File is larger than the 10 MB limit");
});
});
describe("POST /api/companies/:companyId/logo", () => {
beforeEach(() => {
vi.resetModules();
vi.doUnmock("../services/index.js");
vi.doUnmock("../routes/assets.js");
vi.doUnmock("../routes/authz.js");
vi.doUnmock("../middleware/index.js");
registerModuleMocks();
vi.clearAllMocks();
createAssetMock.mockReset();
getAssetByIdMock.mockReset();
logActivityMock.mockReset();
});
it("accepts PNG logo uploads and returns an asset path", async () => {
const png = createStorageService("image/png");
const app = await createApp(png);
createAssetMock.mockResolvedValue(createAsset());
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/logo")
.attach("file", Buffer.from("png"), "logo.png"),
);
expect(res.status, JSON.stringify({ body: res.body, text: res.text, createCalls: createAssetMock.mock.calls.length })).toBe(201);
expect(res.body.contentPath).toBe("/api/assets/asset-1/content");
expect(createAssetMock).toHaveBeenCalledTimes(1);
expect(png.__calls.putFileInputs[0]).toMatchObject({
companyId: "company-1",
namespace: "assets/companies",
originalFilename: "logo.png",
contentType: "image/png",
body: expect.any(Buffer),
});
});
it("sanitizes SVG logo uploads before storing them", async () => {
const svg = createStorageService("image/svg+xml");
const app = await createApp(svg);
createAssetMock.mockResolvedValue({
...createAsset(),
contentType: "image/svg+xml",
originalFilename: "logo.svg",
});
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/logo")
.attach(
"file",
Buffer.from(
"<svg xmlns='http://www.w3.org/2000/svg' onload='alert(1)'><script>alert(1)</script><a href='https://evil.example/'><circle cx='12' cy='12' r='10'/></a></svg>",
),
"logo.svg",
),
);
expect(res.status, JSON.stringify({ body: res.body, text: res.text, createCalls: createAssetMock.mock.calls.length })).toBe(201);
expect(svg.__calls.putFileInputs).toHaveLength(1);
const stored = svg.__calls.putFileInputs[0];
expect(stored.contentType).toBe("image/svg+xml");
expect(stored.originalFilename).toBe("logo.svg");
const body = stored.body.toString("utf8");
expect(body).toContain("<svg");
expect(body).toContain("<circle");
expect(body).not.toContain("<script");
expect(body).not.toContain("onload=");
expect(body).not.toContain("https://evil.example/");
});
it("allows logo uploads within the general attachment limit", async () => {
const png = createStorageService("image/png");
const app = await createApp(png);
createAssetMock.mockResolvedValue(createAsset());
const file = Buffer.alloc(150 * 1024, "a");
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/logo")
.attach("file", file, "within-limit.png"),
);
expect(res.status, JSON.stringify({ body: res.body, text: res.text, createCalls: createAssetMock.mock.calls.length })).toBe(201);
});
it("rejects logo files larger than the general attachment limit", async () => {
const app = await createApp(createStorageService());
createAssetMock.mockResolvedValue(createAsset());
const file = Buffer.alloc(MAX_ATTACHMENT_BYTES + 1, "a");
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/logo")
.attach("file", file, "too-large.png"),
);
expect(res.status).toBe(422);
expect(res.body.error).toBe("Image is larger than the 10 MB limit");
});
it("rejects unsupported image types", async () => {
const app = await createApp(createStorageService("text/plain"));
createAssetMock.mockResolvedValue(createAsset());
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/logo")
.attach("file", Buffer.from("not an image"), "note.txt"),
);
expect(res.status).toBe(422);
expect(res.body.error).toBe("Unsupported image type: text/plain");
expect(createAssetMock).not.toHaveBeenCalled();
});
it("rejects SVG image uploads that cannot be sanitized", async () => {
const app = await createApp(createStorageService("image/svg+xml"));
createAssetMock.mockResolvedValue(createAsset());
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/logo")
.attach("file", Buffer.from("not actually svg"), "logo.svg"),
);
expect(res.status).toBe(422);
expect(res.body.error).toBe("SVG could not be sanitized");
expect(createAssetMock).not.toHaveBeenCalled();
});
});
describe("GET /api/assets/:assetId/content", () => {
beforeEach(() => {
vi.resetModules();
vi.doUnmock("../services/index.js");
vi.doUnmock("../routes/assets.js");
vi.doUnmock("../routes/authz.js");
vi.doUnmock("../middleware/index.js");
registerModuleMocks();
vi.clearAllMocks();
getAssetByIdMock.mockReset();
});
it("reads only the requested storage range for a large asset", async () => {
const storage = createStorageService("text/plain");
getAssetByIdMock.mockResolvedValue({ ...createAsset(), contentType: "text/plain", byteSize: 10 * 1024 * 1024 });
vi.mocked(storage.getObject).mockImplementation(async (_company, _key, options) => {
expect(options).toEqual({ range: { start: 8192, end: 12288 } });
return { stream: Readable.from(Buffer.alloc(4097, 65)), contentLength: 4097 };
});
const res = await requestApp(await createApp(storage), baseUrl => request(baseUrl).get("/api/assets/asset-1/content").set("Range", "bytes=8192-12288"));
expect(res.status).toBe(206);
expect(res.headers["content-range"]).toBe("bytes 8192-12288/10485760");
expect(res.headers["content-length"]).toBe("4097");
expect(res.text).toBe("A".repeat(4097));
});
it.each(["bytes=999-1000", "bytes=0-1,3-4", "bytes=1x-3", "bytes=-", "items=0-3"])("rejects invalid asset ranges before storage reads: %s", async range => {
const storage = createStorageService();
getAssetByIdMock.mockResolvedValue(createAsset());
const res = await requestApp(await createApp(storage), baseUrl => request(baseUrl).get("/api/assets/asset-1/content").set("Range", range));
expect(res.status).toBe(416);
expect(res.headers["content-range"]).toBe("bytes */40");
expect(storage.getObject).not.toHaveBeenCalled();
});
it("downloads script-capable HTML with nosniff and a sandbox CSP", async () => {
const html = Buffer.from("<script>globalThis.__assetXss = true</script>");
const storage = createStorageService("text/html");
getAssetByIdMock.mockResolvedValue({
...createAsset(),
contentType: "text/html",
byteSize: html.byteLength,
originalFilename: "proof.html",
});
vi.mocked(storage.getObject).mockResolvedValue({
stream: Readable.from(html),
contentType: "text/html",
contentLength: html.byteLength,
});
const res = await requestApp(await createApp(storage), (baseUrl) =>
request(baseUrl).get("/api/assets/asset-1/content"),
);
expect(res.status).toBe(200);
expect(res.headers["content-disposition"]).toBe('attachment; filename="proof.html"');
expect(res.headers["x-content-type-options"]).toBe("nosniff");
expect(res.headers["content-security-policy"]).toBe("sandbox; default-src 'none'");
});
it("downloads SVG instead of rendering it on the application origin", async () => {
const svg = Buffer.from("<svg xmlns='http://www.w3.org/2000/svg'><circle r='4'/></svg>");
const storage = createStorageService("image/svg+xml; charset=utf-8");
getAssetByIdMock.mockResolvedValue({
...createAsset(),
contentType: "image/svg+xml; charset=utf-8",
byteSize: svg.byteLength,
originalFilename: "logo.svg",
});
vi.mocked(storage.getObject).mockResolvedValue({
stream: Readable.from(svg),
contentType: "image/svg+xml; charset=utf-8",
contentLength: svg.byteLength,
});
const res = await requestApp(await createApp(storage), (baseUrl) =>
request(baseUrl).get("/api/assets/asset-1/content"),
);
expect(res.status).toBe(200);
expect(res.headers["content-disposition"]).toBe('attachment; filename="logo.svg"');
expect(res.headers["x-content-type-options"]).toBe("nosniff");
expect(res.headers["content-security-policy"]).toBe("sandbox; default-src 'none'");
});
it("keeps curated image types inline", async () => {
const image = Buffer.from("png-bytes");
const storage = createStorageService("image/png");
getAssetByIdMock.mockResolvedValue({
...createAsset(),
byteSize: image.byteLength,
});
vi.mocked(storage.getObject).mockResolvedValue({
stream: Readable.from(image),
contentType: "image/png",
contentLength: image.byteLength,
});
const res = await requestApp(await createApp(storage), (baseUrl) =>
request(baseUrl).get("/api/assets/asset-1/content"),
);
expect(res.status).toBe(200);
expect(res.headers["content-disposition"]).toBe('inline; filename="logo.png"');
expect(res.headers["x-content-type-options"]).toBe("nosniff");
expect(res.headers).not.toHaveProperty("content-security-policy");
});
});