Files
PaperClipAI/server/src/services/native-runtime/runner-api-artifacts.test.ts
T
DottaandPaperclip 84fc67317e feat(runner): export authorized read responses as artifacts (#20606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents use authorized API reads to get task evidence.
> - Small responses return inline JSON, so an agent must copy each
document into a local helper.
> - Copying changes bytes and uses model output for data transport.
> - This pull request adds an explicit artifact export for non-sensitive
GET reads.
> - Helpers can download the exact response bytes and check their
identity.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The native `call_api` response capture path.

**Current behavior**

Small JSON responses have no artifact reference. Local helpers need the
model to copy the JSON data.

**Proposed behavior**

`saveResponse: true` saves a complete authorized GET response below or
above the inline threshold. Zero-byte success returns an explicit empty
receipt without storage. The receipt returns an artifact ID, byte size
and SHA-256 for `download_artifact`. Other callers keep the current
behavior.

Related work: #17393 covers saved app-tool result size. #18105 covers
mutation response paging. This change adds an explicit read export.

## What Changed

- Add GET-only `saveResponse` validation and capture through the current
artifact store.
- Refuse sensitive operations, denied reads, partial responses and
combined text paging.
- Keep route authorization, actor visibility, storage budgets and
download integrity checks.
- Update the native tool contract, generated catalogs and API guide.
- Add boundary and real-route byte comparison tests.
- Preserve saved run state. This is an optional input field; old calls
and receipts keep their current shape. No schema migration or identity
reset is needed. The seeded catalog fixture is generated.

## Verification

- Final published head `d47b52e1bf412418e24768f865cc139097a6d068`:
[typecheck, build, all tests, Linux Rust and aggregate verify
passed](https://github.com/paperclipai/paperclip/actions/runs/42918883477);
[Windows Rust
passed](https://github.com/paperclipai/paperclip/actions/runs/42918883348);
[quality gate
passed](https://github.com/paperclipai/paperclip/actions/runs/42918883016).
Greptile completed successfully at this exact head, 5/5; all five review
threads resolved. Automated Paperclip review remains queued due to a
failed reviewer run; no final verdict is claimed for that service.
- Current head: `d47b52e1bf412418e24768f865cc139097a6d068`. Merged
current master to refresh GitHub's stale base snapshot; the PR now
contains exactly nine intended artifact transport files, with no
lockfile or UI changes. Fresh final-head CI/review is required.
- Previous head `986ad87490fff93cfec313ba6fd56d3dc0bb665b` passed
typecheck, build, Rust verification, artifact/native suites and
completed test shards; remaining shards were still running when the base
snapshot was refreshed. Greptile was 5/5, with one unrelated master UI
finding resolved as outside the corrected PR diff.
- Review fixes verified at `986ad87490fff93cfec313ba6fd56d3dc0bb665b`:
2,135 checks passed across the five focused suites; Runner TypeScript
build and regenerated contract/catalog checks passed. This head replays
the original implementation on current master and retains the original
PR commit in its history.
- Review red/green at replayed baseline `da3a1d4de9`: empty HTTP 200/204
tests failed with real storage's `File is empty`, then passed with zero
new assets or quota reservations. Error-body tests failed without
size/truncation fields, then passed.
- Empty successes now return `empty: true`, `byteSize: 0`, `data: null`
without storage. Nonempty JSON results such as `[]` export normally.
Error receipts report byte size and truncation. Docs and tool guidance
state the existing 10 MiB download limit and paged text alternative.
- Previous-head CI typecheck/build/tests passed except the Rust
process-lineage cleanup test (`unattributed_orphan`); final-head CI and
fresh review remain required.
- Red at `7368a0b15c`: the small-document export test failed with
`Invalid call_api arguments` before production edits. The same check
passed after the change.
- Final post-rebase run at `6e7a35ed6a59bb74f4218312d9f10557099433c3`:
all 2,124 checks passed across the API, capture, artifact boundary,
real-route export and artifact download suites.
- Real-route export test passed: HTTP bytes, stored bytes and downloaded
bytes were equal. Foreign-company access and forged download metadata
were rejected.
- Existing API integration and artifact suites passed. Nearby API
integration checks also passed (59 passed, 2 skipped).
- Disposable native helper scan passed: 12 existing documents, 15 GET
requests, zero mutations, 73,919 bytes, 17.985 seconds. All files passed
size/hash checks. The bridge passed file references only. Production
token savings were not measured.
- Runner TypeScript build and generated contract checks passed.
- `pnpm verify` returned exit 2 and required broader CI coverage. Local
preflight stopped at missing Cargo after dependency and SDK setup. The
local server typecheck stopped during the cycle layer without a
diagnostic; the container recorded an OOM kill. This check has no pass
verdict. Required CI must cover typecheck, build and test at the final
head.

## Risks

- Each explicit export uses the current capture budget and storage
admission. Frequent small exports can reach that budget.
- An artifact holds one HTTP response. Callers must still follow list
pagination.
- No database or saved-state migration is required. The option does not
change default response behavior.

## Model Used

OpenAI `gpt-6.1-sol`, through Codex in Paperclip Runner. Used reasoning,
shell execution and native tools. The active agent configuration
supplied this exact model ID.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-11 06:26:41 -06:00

86 lines
4.9 KiB
TypeScript

import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";
import { describe, expect, it, vi } from "vitest";
import { executeRunnerApi, RUNNER_API_INLINE_BYTES, type RunnerApiIo } from "./runner-api-client.js";
const context = { companyId: "company-a", issueId: "issue-a", runId: "run-a", workMode: "standard" };
const projects = "GET /api/companies/{companyId}/projects";
const io = (fetcher: typeof fetch): RunnerApiIo => ({
apiUrl: "http://127.0.0.1:3100", token: "private-agent-token", fetch: fetcher,
readFile: async () => { throw new Error("not an upload"); },
saveResponse: async body => {
const bytes = Buffer.isBuffer(body) ? body : await readFile(body.path);
return { artifactId: "saved", byteSize: bytes.length, sha256: createHash("sha256").update(bytes).digest("hex") };
},
});
describe("explicit read response artifacts", () => {
it("exports a small authorized document without JSON transcription or a second request", async () => {
const source = Buffer.from(`{ "body": ${JSON.stringify('café 🌍\n# Evidence\n"quoted"')}, "empty": [] }\n`);
const fetcher = vi.fn(async () => new Response(source, { headers: { "content-type": "application/json" } }));
let saved: Buffer | undefined;
const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, {
...io(fetcher),
saveResponse: async (body, contentType) => {
saved = Buffer.isBuffer(body) ? body : await readFile(body.path);
return { artifactId: "document", byteSize: saved.length, contentType };
},
});
expect(result).toMatchObject({ ok: true, artifact: { artifactId: "document", byteSize: source.length } });
expect(result).not.toHaveProperty("data");
expect(saved).toEqual(source);
expect(fetcher).toHaveBeenCalledTimes(1);
});
});
describe("complete authorized read export", () => {
it.each(["[]", "null", "{\n \"body\": \"é🧭\\n\\\"quoted\\\"\"\n}", ...[RUNNER_API_INLINE_BYTES - 1, RUNNER_API_INLINE_BYTES, RUNNER_API_INLINE_BYTES + 1, 100000].map(n => "x".repeat(n))])("preserves source bytes and integrity for response %#", async source => {
const bytes = Buffer.from(source);
const fetcher = vi.fn(async () => new Response(bytes, { headers: { "content-type": "text/plain", "x-paperclip-has-more": "true", "x-paperclip-next-offset": "2" } }));
const reserve = vi.fn(async () => vi.fn(async () => {}));
const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, { ...io(fetcher), reserveResponseCapture: reserve });
expect(result).toMatchObject({ ok: true, artifact: { byteSize: bytes.length, sha256: createHash("sha256").update(bytes).digest("hex") }, pagination: { hasMore: true, nextOffset: 2 } });
expect(result).not.toHaveProperty("data");
expect(result).not.toHaveProperty("preview");
expect(fetcher).toHaveBeenCalledTimes(1);
expect(reserve).toHaveBeenCalledTimes(1);
await expect(reserve.mock.results[0].value).resolves.toHaveBeenCalledWith(bytes.length, true);
});
it.each([403, 500])("marks a truncated HTTP %s error body without exporting it", async status => {
const source = "é".repeat(RUNNER_API_INLINE_BYTES);
const saveResponse = vi.fn(io(fetch).saveResponse);
const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, {
...io(async () => new Response(source, { status })), saveResponse,
});
expect(result).toMatchObject({ ok: false, status, byteSize: Buffer.byteLength(source), truncated: true });
expect(result).not.toHaveProperty("artifact");
expect(saveResponse).not.toHaveBeenCalled();
});
it.each([403, 404, 500, 206])("does not export an HTTP %s response", async status => {
const saveResponse = vi.fn(io(fetch).saveResponse);
const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, {
...io(async () => new Response("private", { status })), saveResponse,
});
expect(result).toMatchObject({ ok: false, status });
expect(result).not.toHaveProperty("artifact");
expect(saveResponse).not.toHaveBeenCalled();
});
it.each([
{ operationId: "POST /api/companies/{companyId}/projects" },
{ operationId: "GET /api/board-api-keys" },
{ operationId: projects, responseText: {} },
{ operationId: projects, pathParams: { companyId: "foreign" } },
])("rejects invalid export before dispatch: %j", async args => {
const fetcher = vi.fn(fetch);
await expect(executeRunnerApi({ ...args, saveResponse: true }, context, io(fetcher))).rejects.toThrow();
expect(fetcher).not.toHaveBeenCalled();
});
it("keeps the default small JSON response inline", async () => {
const saveResponse = vi.fn(io(fetch).saveResponse);
expect(await executeRunnerApi({ operationId: projects }, context, { ...io(async () => Response.json([])), saveResponse })).toMatchObject({ data: [] });
expect(saveResponse).not.toHaveBeenCalled();
});
});