feat(runner): export authorized read responses as artifacts (#20606)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents use authorized API reads to get task evidence.
> - Small responses return inline JSON, so an agent must copy each
document into a local helper.
> - Copying changes bytes and uses model output for data transport.
> - This pull request adds an explicit artifact export for non-sensitive
GET reads.
> - Helpers can download the exact response bytes and check their
identity.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The native `call_api` response capture path.

**Current behavior**

Small JSON responses have no artifact reference. Local helpers need the
model to copy the JSON data.

**Proposed behavior**

`saveResponse: true` saves a complete authorized GET response below or
above the inline threshold. Zero-byte success returns an explicit empty
receipt without storage. The receipt returns an artifact ID, byte size
and SHA-256 for `download_artifact`. Other callers keep the current
behavior.

Related work: #17393 covers saved app-tool result size. #18105 covers
mutation response paging. This change adds an explicit read export.

## What Changed

- Add GET-only `saveResponse` validation and capture through the current
artifact store.
- Refuse sensitive operations, denied reads, partial responses and
combined text paging.
- Keep route authorization, actor visibility, storage budgets and
download integrity checks.
- Update the native tool contract, generated catalogs and API guide.
- Add boundary and real-route byte comparison tests.
- Preserve saved run state. This is an optional input field; old calls
and receipts keep their current shape. No schema migration or identity
reset is needed. The seeded catalog fixture is generated.

## Verification

- Final published head `d47b52e1bf412418e24768f865cc139097a6d068`:
[typecheck, build, all tests, Linux Rust and aggregate verify
passed](https://github.com/paperclipai/paperclip/actions/runs/42918883477);
[Windows Rust
passed](https://github.com/paperclipai/paperclip/actions/runs/42918883348);
[quality gate
passed](https://github.com/paperclipai/paperclip/actions/runs/42918883016).
Greptile completed successfully at this exact head, 5/5; all five review
threads resolved. Automated Paperclip review remains queued due to a
failed reviewer run; no final verdict is claimed for that service.
- Current head: `d47b52e1bf412418e24768f865cc139097a6d068`. Merged
current master to refresh GitHub's stale base snapshot; the PR now
contains exactly nine intended artifact transport files, with no
lockfile or UI changes. Fresh final-head CI/review is required.
- Previous head `986ad87490fff93cfec313ba6fd56d3dc0bb665b` passed
typecheck, build, Rust verification, artifact/native suites and
completed test shards; remaining shards were still running when the base
snapshot was refreshed. Greptile was 5/5, with one unrelated master UI
finding resolved as outside the corrected PR diff.
- Review fixes verified at `986ad87490fff93cfec313ba6fd56d3dc0bb665b`:
2,135 checks passed across the five focused suites; Runner TypeScript
build and regenerated contract/catalog checks passed. This head replays
the original implementation on current master and retains the original
PR commit in its history.
- Review red/green at replayed baseline `da3a1d4de9`: empty HTTP 200/204
tests failed with real storage's `File is empty`, then passed with zero
new assets or quota reservations. Error-body tests failed without
size/truncation fields, then passed.
- Empty successes now return `empty: true`, `byteSize: 0`, `data: null`
without storage. Nonempty JSON results such as `[]` export normally.
Error receipts report byte size and truncation. Docs and tool guidance
state the existing 10 MiB download limit and paged text alternative.
- Previous-head CI typecheck/build/tests passed except the Rust
process-lineage cleanup test (`unattributed_orphan`); final-head CI and
fresh review remain required.
- Red at `7368a0b15c`: the small-document export test failed with
`Invalid call_api arguments` before production edits. The same check
passed after the change.
- Final post-rebase run at `6e7a35ed6a59bb74f4218312d9f10557099433c3`:
all 2,124 checks passed across the API, capture, artifact boundary,
real-route export and artifact download suites.
- Real-route export test passed: HTTP bytes, stored bytes and downloaded
bytes were equal. Foreign-company access and forged download metadata
were rejected.
- Existing API integration and artifact suites passed. Nearby API
integration checks also passed (59 passed, 2 skipped).
- Disposable native helper scan passed: 12 existing documents, 15 GET
requests, zero mutations, 73,919 bytes, 17.985 seconds. All files passed
size/hash checks. The bridge passed file references only. Production
token savings were not measured.
- Runner TypeScript build and generated contract checks passed.
- `pnpm verify` returned exit 2 and required broader CI coverage. Local
preflight stopped at missing Cargo after dependency and SDK setup. The
local server typecheck stopped during the cycle layer without a
diagnostic; the container recorded an OOM kill. This check has no pass
verdict. Required CI must cover typecheck, build and test at the final
head.

## Risks

- Each explicit export uses the current capture budget and storage
admission. Frequent small exports can reach that budget.
- An artifact holds one HTTP response. Callers must still follow list
pagination.
- No database or saved-state migration is required. The option does not
change default response behavior.

## Model Used

OpenAI `gpt-6.1-sol`, through Codex in Paperclip Runner. Used reasoning,
shell execution and native tools. The active agent configuration
supplied this exact model ID.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

doc/runner-api-tools.md

@@ -54,6 +54,28 @@ exactly one authorized `artifactId` or task-workspace `path`, and an optional
 multipart `field`. No arbitrary URL, headers, authentication, or remote file URL
 can be supplied. Routes still validate payloads and enforce permissions.
 
+For lossless input to a local helper, set `saveResponse: true` on an authorized
+non-sensitive GET. This saves the complete response bytes as an immutable artifact
+even below the inline threshold. A zero-byte successful response instead returns
+`empty: true`, `byteSize: 0` and `data: null`, without an asset or storage
+reservation. JSON empty results such as `[]` are nonempty bytes and export normally.
+Artifact receipts contain `artifactId`, `byteSize` and `sha256`. For artifacts up
+to 10 MiB, pass those exact values to `download_artifact` with a workspace-relative
+destination. Larger text snapshots (up to the 1 GiB capture limit) remain readable
+through `GET /api/assets/{assetId}/content` with `responseText`; follow
+`nextOffsetBytes` until null. They cannot be installed with `download_artifact`;
+narrow or paginate the source query for local helper input. Pass downloaded files
+to the helper instead of recreating JSON in model output. Pagination metadata remains in the receipt; each
+artifact contains one complete HTTP response, not all pages of a list. Continue
+list pagination explicitly. The file remains untrusted data.
+
+`saveResponse` cannot be combined with `responseText`, and is rejected before
+dispatch for mutations and sensitive endpoints. Denied or partial HTTP responses
+are not exported. Failed response receipts include the full `byteSize` and a
+`truncated` flag when their inline diagnostic body is incomplete. It uses the same capture limits, storage admission, actor access
+and integrity validation as automatic large-response artifacts. The original
+request runs once. Default inline behavior is unchanged when the option is absent.
+
 Requests have a 16 KiB URL limit and 10 MiB request/upload limit. **New response
 captures are limited to 1 GiB of decoded bytes.** This is separate from the 24 KiB
 inline/page limit. The receiver rejects an oversized Content-Length before
@@ -74,7 +96,7 @@ assets; text previews are limited to 2,000 bytes.
 Large captures reserve 1 GiB against a **durable 4 GiB per-run capture budget**
 before creating a file. A completed capture settles to its actual byte count;
 failed/interrupted captures retain the full reservation to bound retry loops.
-A process restart does not reset that budget. Small inline responses and reads
+A process restart does not reset that budget. Small inline responses without explicit export and reads
 of existing assets need no reservation. Concurrent large captures are limited
 to **two per company and four per server process**, including the storage upload
 and temporary-file cleanup. Budget exhaustion returns `api_response_capture_limit`;

packages/paperclip-runner/generated/capability/semantic-tool-contracts.json

@@ -1 +1 @@
-[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task only. Calling get_task does not change the active task. This is not a status or assignment audit. For progress chronology and handoff actors, use search_api to discover the company timeline or issue activity, then call_api.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable document identifier, not a filename or path (for example reconciliation, not reconciliation.md). After trimming whitespace and lowercasing: 1–64 characters, starting with a letter or digit and containing only a–z, 0–9, _ or -. The raw input is limited to 120 characters.","maxLength":120,"minLength":1,"pattern":"^[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*[A-Za-z0-9\\u212a][A-Za-z0-9_\\u212a-]{0,63}[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*$","type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable document identifier, not a filename or path (for example reconciliation, not reconciliation.md). After trimming whitespace and lowercasing: 1–64 characters, starting with a letter or digit and containing only a–z, 0–9, _ or -. The raw input is limited to 120 characters.","maxLength":120,"minLength":1,"pattern":"^[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*[A-Za-z0-9\\u212a][A-Za-z0-9_\\u212a-]{0,63}[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*$","type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.Line truncated
+[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task only. Calling get_task does not change the active task. This is not a status or assignment audit. For progress chronology and handoff actors, use search_api to discover the company timeline or issue activity, then call_api.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable document identifier, not a filename or path (for example reconciliation, not reconciliation.md). After trimming whitespace and lowercasing: 1–64 characters, starting with a letter or digit and containing only a–z, 0–9, _ or -. The raw input is limited to 120 characters.","maxLength":120,"minLength":1,"pattern":"^[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*[A-Za-z0-9\\u212a][A-Za-z0-9_\\u212a-]{0,63}[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*$","type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable document identifier, not a filename or path (for example reconciliation, not reconciliation.md). After trimming whitespace and lowercasing: 1–64 characters, starting with a letter or digit and containing only a–z, 0–9, _ or -. The raw input is limited to 120 characters.","maxLength":120,"minLength":1,"pattern":"^[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*[A-Za-z0-9\\u212a][A-Za-z0-9_\\u212a-]{0,63}[\\u0009-\\u000d\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]*$","type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.Line truncated

packages/paperclip-runner/generated/semantic-action-catalog.json

@@ -112,7 +112,7 @@
       "planning",
       "skill_test"
     ],
-    "description": "Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses the same route permissions as agent REST access. Prefer dedicated tools for common workflows; route denials still apply. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.",
+    "description": "Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses the same route permissions as agent REST access. Prefer dedicated tools for common workflows; route denials still apply. For lossless helper input, use saveResponse:true on a non-sensitive GET to save nonempty responses as artifacts. Empty responses return empty:true without an artifact. Use download_artifact for artifacts up to 10 MiB; larger text artifacts require paged content reads. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.",
     "effect": "write",
     "inputSchema": {
       "additionalProperties": false,
@@ -218,6 +218,10 @@
             }
           },
           "type": "object"
+        },
+        "saveResponse": {
+          "description": "GET only: save complete nonempty authorized, non-sensitive response bytes as an immutable artifact even below the inline threshold. Returns artifactId, byteSize and sha256 without inline body transcription. Zero-byte success returns empty:true and no artifact. download_artifact accepts at most 10 MiB; use paged content reads for larger text artifacts. Uses existing capture limits and budgets. Cannot combine with responseText. Denied, sensitive and partial responses are not exported. Never repeats the source request.",
+          "type": "boolean"
         }
       },
       "required": [

packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json

@@ -24,7 +24,7 @@
     "prpVersion": 1,
     "nativeExecutionVersion": 1,
     "catalogVersion": 1,
-    "catalogSha256": "sha256:bcbe20460b6f3cf3fc814b088d24c81fc2cfb337c42f2907b416c29b4410e76d",
+    "catalogSha256": "sha256:c5b8e95e8125d68467b8f29d86426fde5280f59e111bc4e48f8684daa174aaca",
     "driverContractVersion": 1,
     "driverKind": "paperclip-deterministic",
     "driverVersion": "1.0.0"

packages/paperclip-runner/protocol/manifest.json

@@ -185,7 +185,7 @@
     },
     {
       "path": "fixtures/evals/native-execution-seeded.json",
-      "sha256": "b4fd01c42444d812bd3d6335f65a9871a7615a1183313d27a17f9467c0656524",
+      "sha256": "2cd8450f79dc7ccd618072ba6da43fcb02c56f59cb83164760e066583f1357ba",
       "expectation": "accept",
       "compatibilityCase": "canonical"
     },

packages/paperclip-runner/src/protocol-actions/call-api.ts

@@ -28,7 +28,7 @@ export const callApiAction = {
   },
   "documentation": {
     "title": "Call the Paperclip API",
-    "description": "Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses the same route permissions as agent REST access. Prefer dedicated tools for common workflows; route denials still apply. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.",
+    "description": "Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses the same route permissions as agent REST access. Prefer dedicated tools for common workflows; route denials still apply. For lossless helper input, use saveResponse:true on a non-sensitive GET to save nonempty responses as artifacts. Empty responses return empty:true without an artifact. Use download_artifact for artifacts up to 10 MiB; larger text artifacts require paged content reads. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.",
     "note": "Production HTTP fallback; does not grant privileges or replace dedicated tools."
   },
   "examples": {
@@ -51,7 +51,7 @@ export const callApiAction = {
       "operationId": "call_api",
       "version": 1,
       "title": "Call the Paperclip API",
-      "description": "Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses the same route permissions as agent REST access. Prefer dedicated tools for common workflows; route denials still apply. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.",
+      "description": "Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses the same route permissions as agent REST access. Prefer dedicated tools for common workflows; route denials still apply. For lossless helper input, use saveResponse:true on a non-sensitive GET to save nonempty responses as artifacts. Empty responses return empty:true without an artifact. Use download_artifact for artifacts up to 10 MiB; larger text artifacts require paged content reads. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.",
       "exposure": "optional",
       "requiredClaims": [
         "api:call"
@@ -96,6 +96,10 @@ export const callApiAction = {
             "type": "string",
             "maxLength": 120
           },
+          "saveResponse": {
+            "description": "GET only: save complete nonempty authorized, non-sensitive response bytes as an immutable artifact even below the inline threshold. Returns artifactId, byteSize and sha256 without inline body transcription. Zero-byte success returns empty:true and no artifact. download_artifact accepts at most 10 MiB; use paged content reads for larger text artifacts. Uses existing capture limits and budgets. Cannot combine with responseText. Denied, sensitive and partial responses are not exported. Never repeats the source request.",
+            "type": "boolean"
+          },
           "responseText": {
             "description": "GET only: return a bounded UTF-8 text window inline, including JSON as text, without saving another artifact. Offsets and limits are bytes. Use the returned nextOffsetBytes to continue; null means complete. Prefer reading a saved artifact for a stable snapshot. New live responses have a 1 GiB capture limit and a 4 GiB per-run capture budget. Existing larger assets remain readable in bounded pages. If a live response returns an artifact, continue on its content operation for a stable snapshot.",
             "type": "object",

server/src/services/native-runtime/runner-api-artifacts.integration.test.ts

@@ -0,0 +1,87 @@
+import { createHash, randomUUID } from "node:crypto";
+import { readFile } from "node:fs/promises";
+import { join } from "node:path";
+import { eq } from "drizzle-orm";
+import { assets, runnerApiResponseReservations } from "@paperclipai/db";
+import { beforeAll, afterAll, describe, expect, it, vi } from "vitest";
+import { startRunnerApiTestServer } from "../../__tests__/helpers/runner-api-server.js";
+import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
+
+describe("authorized read artifacts through real routes and storage", () => {
+  let server: Awaited<ReturnType<typeof startRunnerApiTestServer>>;
+  const oldSecret = process.env.PAPERCLIP_AGENT_JWT_SECRET;
+  beforeAll(async () => {
+    process.env.PAPERCLIP_AGENT_JWT_SECRET = randomUUID();
+    server = await startRunnerApiTestServer();
+  }, 60000);
+  afterAll(async () => {
+    await server?.close();
+    if (oldSecret === undefined) delete process.env.PAPERCLIP_AGENT_JWT_SECRET;
+    else process.env.PAPERCLIP_AGENT_JWT_SECRET = oldSecret;
+  });
+
+  it.each([200, 204])("returns an explicit empty HTTP %s receipt without storage or quota reservations", async status => {
+    const fixture = await server.fixture();
+    const authority = new PaperclipRunnerToolAuthority(server.db, { ...fixture, storage: server.storage, workspaceRoot: fixture.workspace });
+    const before = await server.db.select().from(assets).where(eq(assets.companyId, fixture.companyId));
+    // Inject only the upstream empty response; storage and reservation writes
+    // remain real so production's empty-file rejection cannot be hidden.
+    const spy = vi.spyOn(globalThis, "fetch").mockResolvedValue(new Response(null, { status }));
+    try {
+      const receipt = await authority.execute({ tool: "call_api", callId: randomUUID(), arguments: {
+        operationId: "GET /api/issues/{id}", pathParams: { id: fixture.issueId }, saveResponse: true,
+      } });
+      expect(receipt).toMatchObject({ ok: true, status, empty: true, byteSize: 0, data: null });
+      expect(receipt).not.toHaveProperty("artifact");
+    } finally { spy.mockRestore(); }
+    expect(await server.db.select().from(assets).where(eq(assets.companyId, fixture.companyId))).toEqual(before);
+    expect(await server.db.select().from(runnerApiResponseReservations).where(eq(runnerApiResponseReservations.companyId, fixture.companyId))).toEqual([]);
+  });
+
+  it("preserves exact HTTP bytes in the immutable asset and downloaded helper input", async () => {
+    const fixture = await server.fixture();
+    const authority = new PaperclipRunnerToolAuthority(server.db, { ...fixture, workspaceRoot: fixture.workspace });
+    const execute = (tool: string, args: object) => authority.execute({ tool, callId: randomUUID(), arguments: args }) as Promise<any>;
+    const body = '# Evidence\n\né🧭 "quoted" \\escaped\n```json\n{"empty":[]}\n```';
+    await execute("call_api", { operationId: "PUT /api/issues/{id}/documents/{key}", pathParams: { id: fixture.issueId, key: "evidence" }, body: { title: "Evidence", format: "markdown", body, baseRevisionId: null } });
+    const originalFetch = globalThis.fetch;
+    let source: Buffer | undefined;
+    let requests = 0;
+    // Observe real authorized HTTP bytes without replacing the server or storage.
+    const spy = vi.spyOn(globalThis, "fetch").mockImplementation(async (...args) => {
+      const response = await originalFetch(...args);
+      requests++;
+      source = Buffer.from(await response.clone().arrayBuffer());
+      return response;
+    });
+    let receipt: any;
+    try {
+      receipt = await execute("call_api", { operationId: "GET /api/issues/{id}/documents/{key}", pathParams: { id: fixture.issueId, key: "evidence" }, saveResponse: true });
+    } finally { spy.mockRestore(); }
+    expect(requests).toBe(1);
+    expect(receipt).toMatchObject({ ok: true, artifact: { byteSize: source!.length, sha256: createHash("sha256").update(source!).digest("hex") } });
+    expect(receipt).not.toHaveProperty("data");
+    expect(receipt).not.toHaveProperty("preview");
+    const [asset] = await server.db.select().from(assets).where(eq(assets.id, receipt.artifact.artifactId));
+    const stored = await server.storage.getObject(fixture.companyId, asset.objectKey);
+    const chunks: Buffer[] = [];
+    for await (const chunk of stored.stream) chunks.push(Buffer.from(chunk));
+    expect(Buffer.concat(chunks)).toEqual(source);
+    await execute("download_artifact", { artifactId: receipt.artifact.artifactId, byteSize: receipt.artifact.byteSize, sha256: receipt.artifact.sha256, destination: "helper-response.json" });
+    const downloaded = await readFile(join(fixture.workspace, "helper-response.json"));
+    expect(downloaded).toEqual(source);
+    expect(JSON.parse(downloaded.toString()).body).toBe(body);
+
+    for (const args of [
+      { ...receipt.artifact, sha256: "0".repeat(64), destination: "forged.json" },
+      { ...receipt.artifact, destination: "../escape.json" },
+    ]) {
+      const { url, contentType, ...input } = args;
+      await expect(execute("download_artifact", input)).rejects.toThrow();
+    }
+    const foreign = await server.fixture();
+    const denied = await execute("call_api", { operationId: "GET /api/issues/{id}", pathParams: { id: foreign.issueId }, saveResponse: true });
+    expect(denied.ok).toBe(false);
+    expect(denied).not.toHaveProperty("artifact");
+  });
+});

server/src/services/native-runtime/runner-api-artifacts.test.ts

@@ -0,0 +1,85 @@
+import { createHash } from "node:crypto";
+import { readFile } from "node:fs/promises";
+import { describe, expect, it, vi } from "vitest";
+import { executeRunnerApi, RUNNER_API_INLINE_BYTES, type RunnerApiIo } from "./runner-api-client.js";
+const context = { companyId: "company-a", issueId: "issue-a", runId: "run-a", workMode: "standard" };
+const projects = "GET /api/companies/{companyId}/projects";
+const io = (fetcher: typeof fetch): RunnerApiIo => ({
+  apiUrl: "http://127.0.0.1:3100", token: "private-agent-token", fetch: fetcher,
+  readFile: async () => { throw new Error("not an upload"); },
+  saveResponse: async body => {
+    const bytes = Buffer.isBuffer(body) ? body : await readFile(body.path);
+    return { artifactId: "saved", byteSize: bytes.length, sha256: createHash("sha256").update(bytes).digest("hex") };
+  },
+});
+describe("explicit read response artifacts", () => {
+  it("exports a small authorized document without JSON transcription or a second request", async () => {
+    const source = Buffer.from(`{ "body": ${JSON.stringify('café 🌍\n# Evidence\n"quoted"')}, "empty": [] }\n`);
+    const fetcher = vi.fn(async () => new Response(source, { headers: { "content-type": "application/json" } }));
+    let saved: Buffer | undefined;
+    const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, {
+      ...io(fetcher),
+      saveResponse: async (body, contentType) => {
+        saved = Buffer.isBuffer(body) ? body : await readFile(body.path);
+        return { artifactId: "document", byteSize: saved.length, contentType };
+      },
+    });
+    expect(result).toMatchObject({ ok: true, artifact: { artifactId: "document", byteSize: source.length } });
+    expect(result).not.toHaveProperty("data");
+    expect(saved).toEqual(source);
+    expect(fetcher).toHaveBeenCalledTimes(1);
+  });
+});
+
+describe("complete authorized read export", () => {
+  it.each(["[]", "null", "{\n \"body\": \"é🧭\\n\\\"quoted\\\"\"\n}", ...[RUNNER_API_INLINE_BYTES - 1, RUNNER_API_INLINE_BYTES, RUNNER_API_INLINE_BYTES + 1, 100000].map(n => "x".repeat(n))])("preserves source bytes and integrity for response %#", async source => {
+    const bytes = Buffer.from(source);
+    const fetcher = vi.fn(async () => new Response(bytes, { headers: { "content-type": "text/plain", "x-paperclip-has-more": "true", "x-paperclip-next-offset": "2" } }));
+    const reserve = vi.fn(async () => vi.fn(async () => {}));
+    const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, { ...io(fetcher), reserveResponseCapture: reserve });
+    expect(result).toMatchObject({ ok: true, artifact: { byteSize: bytes.length, sha256: createHash("sha256").update(bytes).digest("hex") }, pagination: { hasMore: true, nextOffset: 2 } });
+    expect(result).not.toHaveProperty("data");
+    expect(result).not.toHaveProperty("preview");
+    expect(fetcher).toHaveBeenCalledTimes(1);
+    expect(reserve).toHaveBeenCalledTimes(1);
+    await expect(reserve.mock.results[0].value).resolves.toHaveBeenCalledWith(bytes.length, true);
+  });
+
+  it.each([403, 500])("marks a truncated HTTP %s error body without exporting it", async status => {
+    const source = "é".repeat(RUNNER_API_INLINE_BYTES);
+    const saveResponse = vi.fn(io(fetch).saveResponse);
+    const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, {
+      ...io(async () => new Response(source, { status })), saveResponse,
+    });
+    expect(result).toMatchObject({ ok: false, status, byteSize: Buffer.byteLength(source), truncated: true });
+    expect(result).not.toHaveProperty("artifact");
+    expect(saveResponse).not.toHaveBeenCalled();
+  });
+
+  it.each([403, 404, 500, 206])("does not export an HTTP %s response", async status => {
+    const saveResponse = vi.fn(io(fetch).saveResponse);
+    const result = await executeRunnerApi({ operationId: projects, saveResponse: true }, context, {
+      ...io(async () => new Response("private", { status })), saveResponse,
+    });
+    expect(result).toMatchObject({ ok: false, status });
+    expect(result).not.toHaveProperty("artifact");
+    expect(saveResponse).not.toHaveBeenCalled();
+  });
+
+  it.each([
+    { operationId: "POST /api/companies/{companyId}/projects" },
+    { operationId: "GET /api/board-api-keys" },
+    { operationId: projects, responseText: {} },
+    { operationId: projects, pathParams: { companyId: "foreign" } },
+  ])("rejects invalid export before dispatch: %j", async args => {
+    const fetcher = vi.fn(fetch);
+    await expect(executeRunnerApi({ ...args, saveResponse: true }, context, io(fetcher))).rejects.toThrow();
+    expect(fetcher).not.toHaveBeenCalled();
+  });
+
+  it("keeps the default small JSON response inline", async () => {
+    const saveResponse = vi.fn(io(fetch).saveResponse);
+    expect(await executeRunnerApi({ operationId: projects }, context, { ...io(async () => Response.json([])), saveResponse })).toMatchObject({ data: [] });
+    expect(saveResponse).not.toHaveBeenCalled();
+  });
+});

server/src/services/native-runtime/runner-api-client.ts

@@ -21,6 +21,7 @@ export const runnerApiCallSchema = z.object({
   body: z.unknown().optional(),
   contentType: z.string().max(120).optional(),
   files: z.array(fileSchema).max(10).optional(),
+  saveResponse: z.boolean().optional(),
   responseText: z.object({
     offsetBytes: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER).optional(),
     limitBytes: z.number().int().min(4).max(RUNNER_API_INLINE_BYTES).optional(),
@@ -55,6 +56,10 @@ export function validateRunnerApiCall(value: unknown, context: RunnerApiContext)
   let input = parsed.data;
   const operation = runnerApiOperation(input.operationId);
   if (operation.transport !== "rest") throw unprocessable(`This endpoint requires its existing protocol client; call_api supports REST only and no other API route substitutes for it.${operation.description.trim() ? " Read its catalog description for details." : ""}`);
+  if (input.saveResponse && (operation.method !== "GET" || runnerApiSensitiveResponse(operation.method, operation.path))) {
+    throw badRequest("This request was not sent: saveResponse supports non-sensitive GET reads only.");
+  }
+  if (input.saveResponse && input.responseText) throw badRequest("Use either saveResponse or responseText, not both.");
   if (input.responseText && operation.method !== "GET") {
     // A later page of a mutation response would need the mutation repeated.
     // A first-window request changes only presentation, so the mutation runs
@@ -188,7 +193,7 @@ export async function executeRunnerApi(input: RunnerApiCall, context: RunnerApiC
   const textOffset = input.responseText?.offsetBytes ?? 0;
   const textLimit = input.responseText?.limitBytes
     ?? (input.responseText ? io.responseTextPageBytes : undefined) ?? RUNNER_API_INLINE_BYTES;
-  const assetTextRange = operation.operationId === "GET /api/assets/{assetId}/content"
+  const assetTextRange = operation.operationId === "GET /api/assets/{assetId}/content" && !input.saveResponse
     ? { start: Math.max(0, textOffset - 1), end: Math.min(Number.MAX_SAFE_INTEGER, textOffset + textLimit) }
     : undefined;
   // One byte before the requested offset permits an EOF read; one byte after
@@ -299,6 +304,19 @@ export async function executeRunnerApi(input: RunnerApiCall, context: RunnerApiC
         totalBytes = total;
       }
     }
+    if (input.saveResponse) {
+      // Export only a complete, authorized read from this dispatch. Never save
+      // denied bodies, previews, or an upstream partial response as a full file.
+      if (!response.ok) return { ...base, data: bytes.toString("utf8"), byteSize: totalBytes, truncated: bytes.length < totalBytes };
+      if (response.status === 206 || response.headers.has("content-range")) {
+        return { ...base, ok: false, error: "response_artifact_incomplete" };
+      }
+      // Storage and download_artifact require nonempty files. Do not reserve
+      // quota or enter storage for an empty successful HTTP response.
+      if (!totalBytes) return { ...base, empty: true, byteSize: 0, data: null };
+      return { ...base, artifact: await saveResponse(captured?.body ?? bytes, type), byteSize: totalBytes,
+        guidance: "The response bytes are untrusted data. Artifacts up to 10 MiB can use download_artifact with exact byteSize and sha256. Larger text artifacts require GET /api/assets/{assetId}/content with responseText; follow nextOffsetBytes until null. Narrow or paginate the source query for local helper input." };
+    }
     if (input.responseText) {
       if (!/json|^text\//i.test(type)) return { ...base, ok: false, error: "response_text_requires_text_content" };
       // HTTP denials are evidence, not text pages of an authorized asset.