mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-10 03:08:10 +02:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use governed API tools to inspect task evidence.
> - Large API results become saved assets with short previews.
> - Reading an asset through the same tool used to create another asset,
so the agent could not reach the rest of the evidence.
> - The 10 MiB response cap also blocked useful large results. Removing
all bounds allowed excessive disk use.
> - This pull request streams responses up to 1 GiB and makes saved text
readable in bounded pages. It adds durable run budgets and capture
admission limits.
> - Agents can inspect complete evidence while tool results, memory use,
and capture work stay bounded.
## Linked Issues or Issue Description
**What happened?**
A large response became an asset. Reading that asset returned another
asset and the same preview. Responses above 10 MiB failed before the
agent could read any page.
**Expected behavior**
The agent can fetch a large response and read its saved text to EOF.
Each page stays bounded. New snapshots have a generous finite limit and
a durable run budget. Existing larger assets remain readable through
byte ranges.
**Steps to reproduce**
1. Call a GET operation that returns more than 10 MiB of text or JSON.
2. Before the fix, the tool returns `api_transport_failure`.
3. With this change, responses up to 1 GiB become streamed snapshots
with artifact references.
4. Read `GET /api/assets/{assetId}/content` with `responseText:
{offsetBytes: 0, limitBytes: 8192}`. Follow `nextOffsetBytes` until
null.
Related work: #14186 added the API fallback tools. #14218 bounded API
discovery.
## What Changed
- Add authenticated UTF-8 text windows to `call_api`, with byte offsets
and total size. Keep each page at or below 24 KiB.
- Stream new responses above 24 KiB through private temporary files into
company-owned assets. Bound each capture to 1 GiB of decoded bytes.
Reject oversized declared lengths before reading and count streamed
bytes before writing.
- Reserve capture budget in the run record before spilling. Allow 4 GiB
per run. Settle successful captures to their actual size. Failed or
interrupted captures retain their full 1 GiB reservation. Run restarts
do not reset the budget.
- Enforce a 20 GiB company snapshot quota with database reservations.
Count legacy snapshots and unfinished storage work across runs and
processes. Asset deletion frees quota.
- Limit large captures to two per company and four per server process.
Hold slots through storage upload and temporary-file cleanup. Use a
10-minute download deadline and 30-second connection/idle-read timeouts.
- Return explicit size, budget, busy, and timeout errors. Preserve
unknown outcomes for mutations whose response cannot be captured.
- Read saved assets through authenticated storage ranges, with at most
two extra bytes for UTF-8 and EOF handling. Unpaged reads return the
existing asset and digest with a bounded preview. Reads create no copies
and do not consume capture budget.
- Keep existing assets above 1 GiB readable in pages. Use safe integer
offsets and PostgreSQL `bigint` asset sizes.
- Stream large S3 uploads through ordered multipart requests. Abort
failed uploads and remove partial local files.
- Revalidate run authority during downloads. Keep company authorization,
GET-only text paging, redirect denial, and mutation replay receipts.
- Document the separate 10 MiB upload limits. This PR does not raise
memory-buffered attachment ingestion limits. Future large video uploads
need streamed ingestion and storage quotas.
## Verification
- Full workspace `pnpm -r typecheck` and `pnpm build` pass after
rebasing on master.
- Focused API and response tests: 1,761 pass. Cover declared and chunked
oversize responses, incorrect Content-Length, exact-limit success,
active-stream deadline, cancellation, cleanup, concurrency admission,
and mutation outcome handling.
- Real HTTP integration: 28 tests pass, including runnerd → PRP →
authority → HTTP, a 12 MiB snapshot, final-page/EOF reads, cross-company
denial, a persisted 3 GiB sparse asset, and large mutation receipt
replay.
- The HTTP suite verifies durable run-budget accounting, simultaneous
runs competing for company quota, legacy snapshot accounting, deletion
refunds, failed-storage reservations, cleaned-failure refunds,
metadata-rollback cleanup refunds, preservation after a lost commit
acknowledgement, and small/saved reads after capture-budget exhaustion.
- A standalone proof streams exactly 1 GiB through the production
capture helper, verifies the final bytes, and removes its temporary
file. It uses repeated 256 KiB chunks and records a peak process RSS of
191 MiB.
- Earlier storage verification covers exact S3 multipart boundaries,
cleanup/abort failures, and a 17 MiB transfer through the real AWS SDK
to a local HTTP S3 endpoint. No cloud S3 qualification was run for this
follow-up.
- The local full test run was interrupted for the company-quota changes.
A later targeted run hit exhausted macOS shared-memory slots before
tests started; two unattached PostgreSQL segments with dead owners were
reclaimed before retrying. All 55 current-head checks pass at
`aebb80ceeeee77d5a56b67bfffd835f2f846878c`, including the full CI test
suite, typecheck, build, browser suites, security scan, and Greptile
(5/5). There are no unresolved review threads. The combined rebased test
catalog also passes (48 tests).
- Earlier paging acceptance passed Daytona and separate staging at
`7739879e9`. Those runs predate the streaming and budget changes.
## Risks
- The 1 GiB response cap and 10-minute active-download deadline are
intentional product limits. Larger live results must use endpoint
pagination or a direct file workflow. Existing larger assets remain
readable through bounded ranges.
- A durable 20 GiB company snapshot quota counts stored runner-api
assets and active/orphan reservations across runs and processes. The
operator can set PAPERCLIP_RUNNER_API_COMPANY_CAPTURE_MAX_BYTES to a
finite value of at least 1 GiB. Deleting snapshots frees capacity;
possible orphan storage must be reconciled before releasing its
reservation.
- A failed capture uses its full reservation. A new large capture needs
a full 1 GiB available, even if it later completes at a smaller size.
Small reads and existing asset pages remain available.
- Concurrency limits apply per server process. The run byte budget is
shared through the database.
- The `integer` to `bigint` migration rewrites asset metadata and takes
an exclusive table lock. File bytes stay in storage.
- A live endpoint is fetched once before returning its snapshot.
Continue reading the saved artifact for stable pages. Mutations may
commit before any size or transport error; inspect state before
retrying.
- Attachment uploads and native file handoffs still default to 10 MiB.
Raising buffered ingestion paths to GiB sizes is separate work.
## Model Used
OpenAI Codex, based on GPT-6, with code execution and repository tools.
The runtime does not expose an exact serving model variant or
context-window size. The earlier paging work also used browser testing
and subagents.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
520 lines
18 KiB
TypeScript
520 lines
18 KiB
TypeScript
import { beforeEach, describe, expect, it, vi } from "vitest";
|
|
import express from "express";
|
|
import { Readable } from "node:stream";
|
|
import request from "supertest";
|
|
import { MAX_ATTACHMENT_BYTES } from "../attachment-types.js";
|
|
import type { StorageService } from "../storage/types.js";
|
|
|
|
const { createAssetMock, getAssetByIdMock, logActivityMock } = vi.hoisted(() => ({
|
|
createAssetMock: vi.fn(),
|
|
getAssetByIdMock: vi.fn(),
|
|
logActivityMock: vi.fn(),
|
|
}));
|
|
|
|
function registerModuleMocks() {
|
|
vi.doMock("../services/activity-log.js", () => ({
|
|
logActivity: logActivityMock,
|
|
}));
|
|
|
|
vi.doMock("../services/assets.js", () => ({
|
|
assetService: vi.fn(() => ({
|
|
create: createAssetMock,
|
|
getById: getAssetByIdMock,
|
|
})),
|
|
}));
|
|
|
|
vi.doMock("../services/index.js", () => ({
|
|
assetService: vi.fn(() => ({
|
|
create: createAssetMock,
|
|
getById: getAssetByIdMock,
|
|
})),
|
|
logActivity: logActivityMock,
|
|
}));
|
|
}
|
|
|
|
function createAsset() {
|
|
const now = new Date("2026-01-01T00:00:00.000Z");
|
|
return {
|
|
id: "asset-1",
|
|
companyId: "company-1",
|
|
provider: "local",
|
|
objectKey: "assets/abc",
|
|
contentType: "image/png",
|
|
byteSize: 40,
|
|
sha256: "sha256-sample",
|
|
originalFilename: "logo.png",
|
|
createdByAgentId: null,
|
|
createdByUserId: "user-1",
|
|
createdAt: now,
|
|
updatedAt: now,
|
|
};
|
|
}
|
|
|
|
type TestStorageService = StorageService & {
|
|
__calls: {
|
|
putFileInputs: Array<{
|
|
companyId: string;
|
|
namespace: string;
|
|
originalFilename: string | null;
|
|
contentType: string;
|
|
body: Buffer;
|
|
}>;
|
|
};
|
|
};
|
|
|
|
function createStorageService(contentType = "image/png"): TestStorageService {
|
|
const calls: TestStorageService["__calls"] = { putFileInputs: [] };
|
|
const putFile: StorageService["putFile"] = async (input: {
|
|
companyId: string;
|
|
namespace: string;
|
|
originalFilename: string | null;
|
|
contentType: string;
|
|
body: Buffer;
|
|
}) => {
|
|
calls.putFileInputs.push(input);
|
|
return {
|
|
provider: "local_disk" as const,
|
|
objectKey: `${input.namespace}/${input.originalFilename ?? "upload"}`,
|
|
contentType: contentType || input.contentType,
|
|
byteSize: input.body.length,
|
|
sha256: "sha256-sample",
|
|
originalFilename: input.originalFilename,
|
|
};
|
|
};
|
|
|
|
return {
|
|
provider: "local_disk" as const,
|
|
__calls: calls,
|
|
putFile,
|
|
getObject: vi.fn(),
|
|
headObject: vi.fn(),
|
|
deleteObject: vi.fn(),
|
|
};
|
|
}
|
|
|
|
async function createApp(storage: ReturnType<typeof createStorageService>) {
|
|
const { assetRoutes } = await vi.importActual<typeof import("../routes/assets.js")>("../routes/assets.js");
|
|
const app = express();
|
|
app.use((req, _res, next) => {
|
|
req.actor = {
|
|
type: "board",
|
|
source: "local_implicit",
|
|
userId: "user-1",
|
|
};
|
|
next();
|
|
});
|
|
app.use("/api", assetRoutes({} as any, storage));
|
|
return app;
|
|
}
|
|
|
|
async function requestApp(
|
|
app: express.Express,
|
|
buildRequest: (baseUrl: string) => request.Test,
|
|
) {
|
|
const { createServer } = await vi.importActual<typeof import("node:http")>("node:http");
|
|
const server = createServer(app);
|
|
try {
|
|
await new Promise<void>((resolve) => {
|
|
server.listen(0, "127.0.0.1", resolve);
|
|
});
|
|
const address = server.address();
|
|
if (!address || typeof address === "string") {
|
|
throw new Error("Expected HTTP server to listen on a TCP port");
|
|
}
|
|
return await buildRequest(`http://127.0.0.1:${address.port}`);
|
|
} finally {
|
|
if (server.listening) {
|
|
await new Promise<void>((resolve, reject) => {
|
|
server.close((error) => {
|
|
if (error) reject(error);
|
|
else resolve();
|
|
});
|
|
});
|
|
}
|
|
}
|
|
}
|
|
|
|
describe("POST /api/companies/:companyId/assets/images", () => {
|
|
beforeEach(() => {
|
|
vi.resetModules();
|
|
vi.doUnmock("../services/activity-log.js");
|
|
vi.doUnmock("../services/assets.js");
|
|
vi.doUnmock("../services/index.js");
|
|
vi.doUnmock("../routes/assets.js");
|
|
vi.doUnmock("../routes/authz.js");
|
|
vi.doUnmock("../middleware/index.js");
|
|
registerModuleMocks();
|
|
vi.clearAllMocks();
|
|
createAssetMock.mockReset();
|
|
getAssetByIdMock.mockReset();
|
|
logActivityMock.mockReset();
|
|
});
|
|
|
|
it("accepts PNG image uploads and returns an asset path", async () => {
|
|
const png = createStorageService("image/png");
|
|
const app = await createApp(png);
|
|
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/assets/images")
|
|
.field("namespace", "goals")
|
|
.attach("file", Buffer.from("png"), "logo.png"),
|
|
);
|
|
|
|
expect([200, 201], JSON.stringify(res.body)).toContain(res.status);
|
|
expect(res.body.contentPath).toBe("/api/assets/asset-1/content");
|
|
expect(createAssetMock).toHaveBeenCalledTimes(1);
|
|
expect(png.__calls.putFileInputs[0]).toMatchObject({
|
|
companyId: "company-1",
|
|
namespace: "assets/goals",
|
|
originalFilename: "logo.png",
|
|
contentType: "image/png",
|
|
body: expect.any(Buffer),
|
|
});
|
|
});
|
|
|
|
it("accepts namespaces that hold identity-provider user ids", async () => {
|
|
const png = createStorageService("image/png");
|
|
const app = await createApp(png);
|
|
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const namespace = "profiles/oidc:example|jane.example@example.com";
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/assets/images")
|
|
.field("namespace", namespace)
|
|
.attach("file", Buffer.from("png"), "avatar.png"),
|
|
);
|
|
|
|
expect(res.status, JSON.stringify(res.body)).toBe(201);
|
|
expect(png.__calls.putFileInputs[0]).toMatchObject({
|
|
companyId: "company-1",
|
|
namespace: `assets/${namespace}`,
|
|
originalFilename: "avatar.png",
|
|
contentType: "image/png",
|
|
});
|
|
});
|
|
|
|
it("rejects namespaces with characters outside the accepted set", async () => {
|
|
const png = createStorageService("image/png");
|
|
const app = await createApp(png);
|
|
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/assets/images")
|
|
.field("namespace", "profiles/bad name!")
|
|
.attach("file", Buffer.from("png"), "avatar.png"),
|
|
);
|
|
|
|
expect(res.status).toBe(400);
|
|
expect(res.body.error).toContain("namespace");
|
|
expect(res.body.details?.[0]?.path).toEqual(["namespace"]);
|
|
expect(png.__calls.putFileInputs).toHaveLength(0);
|
|
expect(createAssetMock).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects namespaces that hold a dot path segment", async () => {
|
|
const png = createStorageService("image/png");
|
|
const app = await createApp(png);
|
|
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/assets/images")
|
|
.field("namespace", "profiles/../secrets")
|
|
.attach("file", Buffer.from("png"), "avatar.png"),
|
|
);
|
|
|
|
expect(res.status).toBe(400);
|
|
expect(res.body.error).toContain("namespace");
|
|
expect(png.__calls.putFileInputs).toHaveLength(0);
|
|
expect(createAssetMock).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("allows supported non-image attachments outside the company logo flow", async () => {
|
|
const text = createStorageService("text/plain");
|
|
const app = await createApp(text);
|
|
|
|
createAssetMock.mockResolvedValue({
|
|
...createAsset(),
|
|
contentType: "text/plain",
|
|
originalFilename: "note.txt",
|
|
});
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/assets/images")
|
|
.field("namespace", "issues/drafts")
|
|
.attach("file", Buffer.from("hello"), { filename: "note.txt", contentType: "text/plain" }),
|
|
);
|
|
|
|
expect([200, 201]).toContain(res.status);
|
|
expect(res.body.contentPath).toBe("/api/assets/asset-1/content");
|
|
expect(res.body.contentType).toBe("text/plain");
|
|
});
|
|
|
|
it("names the limit in human units when a file exceeds the attachment cap", async () => {
|
|
const app = await createApp(createStorageService());
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const file = Buffer.alloc(MAX_ATTACHMENT_BYTES + 1, "a");
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/assets/images")
|
|
.attach("file", file, "too-large.png"),
|
|
);
|
|
|
|
expect(res.status).toBe(422);
|
|
expect(res.body.error).toBe("File is larger than the 10 MB limit");
|
|
});
|
|
});
|
|
|
|
describe("POST /api/companies/:companyId/logo", () => {
|
|
beforeEach(() => {
|
|
vi.resetModules();
|
|
vi.doUnmock("../services/index.js");
|
|
vi.doUnmock("../routes/assets.js");
|
|
vi.doUnmock("../routes/authz.js");
|
|
vi.doUnmock("../middleware/index.js");
|
|
registerModuleMocks();
|
|
vi.clearAllMocks();
|
|
createAssetMock.mockReset();
|
|
getAssetByIdMock.mockReset();
|
|
logActivityMock.mockReset();
|
|
});
|
|
|
|
it("accepts PNG logo uploads and returns an asset path", async () => {
|
|
const png = createStorageService("image/png");
|
|
const app = await createApp(png);
|
|
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/logo")
|
|
.attach("file", Buffer.from("png"), "logo.png"),
|
|
);
|
|
|
|
expect(res.status, JSON.stringify({ body: res.body, text: res.text, createCalls: createAssetMock.mock.calls.length })).toBe(201);
|
|
expect(res.body.contentPath).toBe("/api/assets/asset-1/content");
|
|
expect(createAssetMock).toHaveBeenCalledTimes(1);
|
|
expect(png.__calls.putFileInputs[0]).toMatchObject({
|
|
companyId: "company-1",
|
|
namespace: "assets/companies",
|
|
originalFilename: "logo.png",
|
|
contentType: "image/png",
|
|
body: expect.any(Buffer),
|
|
});
|
|
});
|
|
|
|
it("sanitizes SVG logo uploads before storing them", async () => {
|
|
const svg = createStorageService("image/svg+xml");
|
|
const app = await createApp(svg);
|
|
|
|
createAssetMock.mockResolvedValue({
|
|
...createAsset(),
|
|
contentType: "image/svg+xml",
|
|
originalFilename: "logo.svg",
|
|
});
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/logo")
|
|
.attach(
|
|
"file",
|
|
Buffer.from(
|
|
"<svg xmlns='http://www.w3.org/2000/svg' onload='alert(1)'><script>alert(1)</script><a href='https://evil.example/'><circle cx='12' cy='12' r='10'/></a></svg>",
|
|
),
|
|
"logo.svg",
|
|
),
|
|
);
|
|
|
|
expect(res.status, JSON.stringify({ body: res.body, text: res.text, createCalls: createAssetMock.mock.calls.length })).toBe(201);
|
|
expect(svg.__calls.putFileInputs).toHaveLength(1);
|
|
const stored = svg.__calls.putFileInputs[0];
|
|
expect(stored.contentType).toBe("image/svg+xml");
|
|
expect(stored.originalFilename).toBe("logo.svg");
|
|
const body = stored.body.toString("utf8");
|
|
expect(body).toContain("<svg");
|
|
expect(body).toContain("<circle");
|
|
expect(body).not.toContain("<script");
|
|
expect(body).not.toContain("onload=");
|
|
expect(body).not.toContain("https://evil.example/");
|
|
});
|
|
|
|
it("allows logo uploads within the general attachment limit", async () => {
|
|
const png = createStorageService("image/png");
|
|
const app = await createApp(png);
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const file = Buffer.alloc(150 * 1024, "a");
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/logo")
|
|
.attach("file", file, "within-limit.png"),
|
|
);
|
|
|
|
expect(res.status, JSON.stringify({ body: res.body, text: res.text, createCalls: createAssetMock.mock.calls.length })).toBe(201);
|
|
});
|
|
|
|
it("rejects logo files larger than the general attachment limit", async () => {
|
|
const app = await createApp(createStorageService());
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const file = Buffer.alloc(MAX_ATTACHMENT_BYTES + 1, "a");
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/logo")
|
|
.attach("file", file, "too-large.png"),
|
|
);
|
|
|
|
expect(res.status).toBe(422);
|
|
expect(res.body.error).toBe("Image is larger than the 10 MB limit");
|
|
});
|
|
|
|
it("rejects unsupported image types", async () => {
|
|
const app = await createApp(createStorageService("text/plain"));
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/logo")
|
|
.attach("file", Buffer.from("not an image"), "note.txt"),
|
|
);
|
|
|
|
expect(res.status).toBe(422);
|
|
expect(res.body.error).toBe("Unsupported image type: text/plain");
|
|
expect(createAssetMock).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects SVG image uploads that cannot be sanitized", async () => {
|
|
const app = await createApp(createStorageService("image/svg+xml"));
|
|
createAssetMock.mockResolvedValue(createAsset());
|
|
|
|
const res = await requestApp(app, (baseUrl) =>
|
|
request(baseUrl)
|
|
.post("/api/companies/company-1/logo")
|
|
.attach("file", Buffer.from("not actually svg"), "logo.svg"),
|
|
);
|
|
|
|
expect(res.status).toBe(422);
|
|
expect(res.body.error).toBe("SVG could not be sanitized");
|
|
expect(createAssetMock).not.toHaveBeenCalled();
|
|
});
|
|
});
|
|
|
|
describe("GET /api/assets/:assetId/content", () => {
|
|
beforeEach(() => {
|
|
vi.resetModules();
|
|
vi.doUnmock("../services/index.js");
|
|
vi.doUnmock("../routes/assets.js");
|
|
vi.doUnmock("../routes/authz.js");
|
|
vi.doUnmock("../middleware/index.js");
|
|
registerModuleMocks();
|
|
vi.clearAllMocks();
|
|
getAssetByIdMock.mockReset();
|
|
});
|
|
|
|
it("reads only the requested storage range for a large asset", async () => {
|
|
const storage = createStorageService("text/plain");
|
|
getAssetByIdMock.mockResolvedValue({ ...createAsset(), contentType: "text/plain", byteSize: 10 * 1024 * 1024 });
|
|
vi.mocked(storage.getObject).mockImplementation(async (_company, _key, options) => {
|
|
expect(options).toEqual({ range: { start: 8192, end: 12288 } });
|
|
return { stream: Readable.from(Buffer.alloc(4097, 65)), contentLength: 4097 };
|
|
});
|
|
const res = await requestApp(await createApp(storage), baseUrl => request(baseUrl).get("/api/assets/asset-1/content").set("Range", "bytes=8192-12288"));
|
|
expect(res.status).toBe(206);
|
|
expect(res.headers["content-range"]).toBe("bytes 8192-12288/10485760");
|
|
expect(res.headers["content-length"]).toBe("4097");
|
|
expect(res.text).toBe("A".repeat(4097));
|
|
});
|
|
|
|
it.each(["bytes=999-1000", "bytes=0-1,3-4", "bytes=1x-3", "bytes=-", "items=0-3"])("rejects invalid asset ranges before storage reads: %s", async range => {
|
|
const storage = createStorageService();
|
|
getAssetByIdMock.mockResolvedValue(createAsset());
|
|
const res = await requestApp(await createApp(storage), baseUrl => request(baseUrl).get("/api/assets/asset-1/content").set("Range", range));
|
|
expect(res.status).toBe(416);
|
|
expect(res.headers["content-range"]).toBe("bytes */40");
|
|
expect(storage.getObject).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("downloads script-capable HTML with nosniff and a sandbox CSP", async () => {
|
|
const html = Buffer.from("<script>globalThis.__assetXss = true</script>");
|
|
const storage = createStorageService("text/html");
|
|
getAssetByIdMock.mockResolvedValue({
|
|
...createAsset(),
|
|
contentType: "text/html",
|
|
byteSize: html.byteLength,
|
|
originalFilename: "proof.html",
|
|
});
|
|
vi.mocked(storage.getObject).mockResolvedValue({
|
|
stream: Readable.from(html),
|
|
contentType: "text/html",
|
|
contentLength: html.byteLength,
|
|
});
|
|
|
|
const res = await requestApp(await createApp(storage), (baseUrl) =>
|
|
request(baseUrl).get("/api/assets/asset-1/content"),
|
|
);
|
|
|
|
expect(res.status).toBe(200);
|
|
expect(res.headers["content-disposition"]).toBe('attachment; filename="proof.html"');
|
|
expect(res.headers["x-content-type-options"]).toBe("nosniff");
|
|
expect(res.headers["content-security-policy"]).toBe("sandbox; default-src 'none'");
|
|
});
|
|
|
|
it("downloads SVG instead of rendering it on the application origin", async () => {
|
|
const svg = Buffer.from("<svg xmlns='http://www.w3.org/2000/svg'><circle r='4'/></svg>");
|
|
const storage = createStorageService("image/svg+xml; charset=utf-8");
|
|
getAssetByIdMock.mockResolvedValue({
|
|
...createAsset(),
|
|
contentType: "image/svg+xml; charset=utf-8",
|
|
byteSize: svg.byteLength,
|
|
originalFilename: "logo.svg",
|
|
});
|
|
vi.mocked(storage.getObject).mockResolvedValue({
|
|
stream: Readable.from(svg),
|
|
contentType: "image/svg+xml; charset=utf-8",
|
|
contentLength: svg.byteLength,
|
|
});
|
|
|
|
const res = await requestApp(await createApp(storage), (baseUrl) =>
|
|
request(baseUrl).get("/api/assets/asset-1/content"),
|
|
);
|
|
|
|
expect(res.status).toBe(200);
|
|
expect(res.headers["content-disposition"]).toBe('attachment; filename="logo.svg"');
|
|
expect(res.headers["x-content-type-options"]).toBe("nosniff");
|
|
expect(res.headers["content-security-policy"]).toBe("sandbox; default-src 'none'");
|
|
});
|
|
|
|
it("keeps curated image types inline", async () => {
|
|
const image = Buffer.from("png-bytes");
|
|
const storage = createStorageService("image/png");
|
|
getAssetByIdMock.mockResolvedValue({
|
|
...createAsset(),
|
|
byteSize: image.byteLength,
|
|
});
|
|
vi.mocked(storage.getObject).mockResolvedValue({
|
|
stream: Readable.from(image),
|
|
contentType: "image/png",
|
|
contentLength: image.byteLength,
|
|
});
|
|
|
|
const res = await requestApp(await createApp(storage), (baseUrl) =>
|
|
request(baseUrl).get("/api/assets/asset-1/content"),
|
|
);
|
|
|
|
expect(res.status).toBe(200);
|
|
expect(res.headers["content-disposition"]).toBe('inline; filename="logo.png"');
|
|
expect(res.headers["x-content-type-options"]).toBe("nosniff");
|
|
expect(res.headers).not.toHaveProperty("content-security-policy");
|
|
});
|
|
});
|