mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 16:27:18 +02:00
DottaandPaperclip
9225796a4e
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents receive assigned app-tool results through the durable Runner protocol. > - The assigned-tool bridge returned complete provider responses without a size bound. > - A large response could exceed the encrypted command limit after the upstream call had completed. > - The provider then waited for a result that the controller could not deliver. > - This pull request saves large governed responses and returns a small reference with a scoped page reader. > - Agents can retrieve the full result without repeating the original action. ## Linked Issues or Issue Description **What happened?** Large app-tool responses could leave native tool calls pending even though the upstream service completed. The durable command limit is about 510 KiB. The assigned-tool bridge did not account for that limit. A 720,048-byte regression response failed the delivery-size assertion before this change. **Expected behavior** Deliver a bounded response and a way to retrieve all saved content. Keep the original action at one execution. Preserve task, agent, company, responsible-user, and current-run access checks. **Steps to reproduce** 1. Run a native agent with an assigned tool that returns more than 510 KiB of JSON. 2. Let the upstream tool complete. 3. Observe that the original bridge returns the oversized object directly, so the durable controller cannot queue its result. Related: #14734 preserves uncertain tool outcomes across shutdown and restart. This change keeps that protection. #16517 handles native MCP content projection on a different transport; it does not bound native assigned-tool results. ## What Changed - Save governed assigned-tool responses above 500,000 encoded bytes using the existing response asset storage, quotas, and audit path. - Add `read_tool_result` for UTF-8 JSON pages from the same task and agent. Each encoded page is capped at 500,000 bytes, including JSON escaping and metadata, with remaining content available through the next byte offset. It works when generic API tools are disabled. - Reuse existing authenticated asset reads and responsible-user checks. Retrieval does not invoke the upstream service. - Preserve uncertain delivery if storage fails after an action completes. - Add regression, fault, authorization, and opt-in live Codex checks. Document the transport and access contract. ## Verification - Red: the original integration regression returned 720,048 bytes against unchanged production code and failed its bounded-response assertion. The follow-up regression also showed that a 480 KB result was unnecessarily saved under the original 24 KiB policy. - Green: results through 500,000 encoded bytes stay inline. Larger responses are saved in full. Real HTTP integration tests reconstruct the complete Unicode result after controller recreation in exactly two pages, in both standard and low-trust configurations, with one upstream execution. - The encoded-size boundary and escape-heavy Unicode tests verify that every page fits 500,000 bytes, cursors advance on valid UTF-8 boundaries, and concatenating pages exactly reconstructs the original JSON. Storage-failure tests preserve an uncertain outcome without repeating the action. - The focused unit/API run passes 1,957 tests; the changed real HTTP tests pass. `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` pass. - Real Codex acceptance passes with `gpt-6.1-sol` through Codex CLI 0.160.0 and native Runner, on macOS ARM64 with `low_trust_review` and generic API tools disabled. Codex requests every page in order, receives all 660,098 original bytes, and returns the final verification field. The two encoded pages are 500,000 and 160,543 bytes; the upstream operation executes once. Runner closes successfully in a 34.1-second probe. Only the disposable database's connection/worker counts were reduced to fit the host's remaining semaphore capacity. - The native-PRP regression also passes with escape-heavy Unicode pages over 1 MB in total and a separate inline result containing 200,000 backslashes. A scripted provider verifies both complete results by SHA-256. It proves the intermediate text wrapper can exceed 700 KB because the native transport decodes it before queuing the bounded PRP result; every result object stays within 500,000 encoded bytes. Native binaries are required for the local run, preventing a silent skip. - A second real-Codex probe with escaped Unicode, quotes, backslashes, newlines, and NULs also passes: every original byte of a 720,098-byte JSON response is retrieved in two encoded pages of 500,000 and 460,543 bytes. The model returns the final field, the upstream call executes once, and Runner closes in 32.6 seconds. This uses the same production source with only the synthetic fixture content varied. - The live probe exercises real PRP, persistence, and authenticated asset reads. It does not claim the original staging task resumed or full heartbeat checkpoint acceptance. - Earlier broader local testing passed 103 tests but could not start the second database for the existing `rejects a stale source-run question over public HTTP and accepts it from the next run` test. Extending its timeout exposed PostgreSQL initialization failure on this semaphore-exhausted host. The earlier local `pnpm test:run` attempt was interrupted and is not claimed as passing. Full CI passes on current head `cd32237ee5429765b8f349402ac3c79cceec3e6b`. GitHub reports 55 successful checks, two skipped Storybook checks, and one neutral Paperclip agent review. Greptile is 5/5 on this head with no review threads. - Additional Paperclip agent review is incomplete: it withdrew the earlier double-escaping finding after inspecting the native regression, found no new actionable defects, and reported stalled large-source reads prevented independent inspection of the transport decoder. Its neutral assessment is [recorded here](https://github.com/paperclipai/paperclip/pull/16980#issuecomment-6295579705). This limitation remains separate from the passing local native transport, real Codex acceptance, full CI, and Greptile results. ## Risks - Large responses now consume existing response-storage quota and require page reads. Larger pages use more provider context. Results are preserved in full; each encoded page leaves about 22 KiB below the PRP command limit for the command envelope. - A storage failure after execution retains an uncertain delivery outcome. The bridge must not report that the upstream action failed or invite a replay. - The dedicated reader grants access only to saved responses for the current task and agent. The asset route also checks the original responsible user. - Runner command limits and receipt rules remain unchanged. No migration or separate Runner protocol rollout is required. - This PR does not deploy the repair or resume an existing blocked task. ## Model Used OpenAI Codex (GPT-6), with reasoning, tool use, and code execution. The exact serving model ID and context-window size are not exposed in this session. The opt-in acceptance probe requests `gpt-6.1-sol` through Codex CLI 0.160.0. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the changed behavior (focused unit/API, real HTTP, and real Codex acceptance); the broader local database limitation is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>