Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-29 21:07:40 +01:00
Code Issues Packages Projects Releases Wiki Activity
5,776 Commits 1,320 Branches 2,151 Tags
c08aa4585e74813751b6041b9d7cd76f7bfdc5bc
Commit Graph
4 Commits
Author SHA1 Message Date
DottaandPaperclip 2d0c138122 Expand direct assistant MCP tools for work and configuration (#15380) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People also use assistants in Codex, Claude, and other MCP clients.
> - The existing assistant connection can read work and create tasks or
comments.
> - It cannot edit tasks, exchange files, or manage normal agent and
project settings.
> - These operations must retain the person's permissions and
Paperclip's execution rules.
> - This pull request adds an explicit operation registry and separately
consented configuration access.
> - Assistants can manage work without receiving credentials or runner
authority.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: assistant MCP, domain routes, consent UI, storage, and
Product E2E.

**Problem or motivation**

A connected assistant cannot update tasks, maintain documents, attach
files, or configure existing agents, projects, and skills. Users must
leave the assistant for these routine actions.

**Proposed solution**

Add named tools and a restricted API registry to direct connections.
Require separate configuration consent. Reuse domain routes and retry
receipts. File uploads save the attachment when the byte transfer
succeeds.

**Alternatives considered**

Arbitrary REST forwarding would expose administration and credential
operations. Runner impersonation would bypass execution ownership.
Separate upload completion calls add unnecessary client state.

**Roadmap alignment**

Checked ROADMAP.md and related MCP pull requests. This extends the
human-authorized connection from #14933. It does not replace the runner
or introduce agent impersonation.

Companion Cloud routing and directory isolation:
https://github.com/paperclipai/paperclip-cloud/pull/678.

## What Changed

- Add task editing, finish/block, documents/revisions, deliverables,
agent settings/instructions, projects/repositories, and skills/files.
- Add an allowlisted API search/call registry with identical field
restrictions, scopes, and retry identities.
- Add unchecked configuration consent. Existing write grants retain
their current authority.
- Add hashed, expiring file transfer tickets and atomic upload receipts.
No completion call is required.
- Preserve company boundaries, human attribution, active native
execution ownership, and execution review gates.
- Add protocol/domain tests, consent stories, and eight paid Product E2E
workflows.
- Repair two CI fixture races: await cold route setup before assertions,
and wait for asynchronously loaded connection copy. Both fixture suites
pass (24 + 48 tests).

## Verification

- Consent revision: one write-access checkbox controls requested work
and configuration permissions in browser and device flows. All 16
consent tests, UI typecheck/build and token gates pass. Updated
interactive stories cover default approval, opt-out and viewer
restrictions. The paid browser helper uses the new exact label. Real
GPT-5.4 Mini Product E2E passes 2/2 at
`64f96373118eb190f8cba1c2ab17cb979555f3ad` (configuration + permission
denial), campaign `local-2026-10-07T00-51-14-337Z`, no automatic
retries, cleanup passed; $0.04149375 estimated assistant cost plus
unpriced worker usage. Raw results, usage and source fingerprints are
retained in the worktree. UI and Product E2E typechecks pass.

- Prior head `2f246d4b74f1f98c75ebcb37ae6753a748237fac`: all 52 checks
pass; two optional Storybook checks skip. Greptile 5/5 on that head, no
unresolved review threads. Final consent head
`64f96373118eb190f8cba1c2ab17cb979555f3ad` also has all checks passing
and Greptile 5/5 with no unresolved threads. The unchanged Cursor
sandbox test had one 10-second timeout, passed in local isolation, and
passed its single CI rerun; the failed attempt remains in [the CI
run](https://github.com/paperclipai/paperclip/actions/runs/37554106934).
The existing chat retry-denial browser test had one visibility failure;
its single rerun passes, and the failed attempt remains in [the CI
run](https://github.com/paperclipai/paperclip/actions/runs/37542735691).

- Full workspace `pnpm -r typecheck` and `pnpm build` pass at final
runtime source `b2196fae1`. UI token gates pass.
- 139 MCP/OAuth/transfer/privacy tests and 76 grader calibration tests
pass, including one-connection PostgreSQL OAuth and concurrent upload
retries.
- Paid Product E2E: all eight expanded cases qualified across Mini,
Haiku and Sonnet. A merged-source repeat passed 23/24; one Haiku cell
timed out before application startup. Final affected-case qualification
passes 9/9 on all three models with grader v16, including the failed
cell. Automatic retries disabled; failures, costs, source hashes and
independent durable-state/file assertions are retained in [the
verification
record](doc/plans/2026-10-06-expanded-assistant-mcp-verification.md).
- Actual Codex CLI, Claude Code and OpenCode clients completed local
reads/mutations. Codex wrote a report, Claude updated it in a later
conversation, and OpenCode uploaded/downloaded a file with matching
SHA-256 and registered the attachment. Revoking the CLI grant rejects
subsequent bridge initialization.
- Butter staging is verified on final runtime `b2196fae1`
([deployment](https://github.com/paperclipai/paperclip-cloud/actions/runs/37538432138)).
A fresh OpenCode workspace fetched the copied invitation, configured
remote MCP, started OAuth and reached real consent with configuration
unchecked. Invalid transfer tickets return 403 through Cloud. Human
approval for the new persistent staging grant is pending; hosted
task/file success is not yet claimed. The final transaction fix is
deployed.
- Full local `pnpm test:run` passed 15,614 general-server tests but
stopped on two macOS timeouts. The heartbeat test passed in isolation;
the existing 40,000-file Git stress fixture timed out again. Its Linux
CI lane passes. Later local full-suite phases did not run after the
timeout; this is not an all-green local full-suite claim.
- Instructions and security limits are in `doc/public-mcp.md`; the saved
plan is `doc/plans/2026-10-06-expanded-assistant-mcp-tools.md`.

## Risks

- This expands the experimental direct MCP surface. Explicit schemas and
domain permissions must stay synchronized.
- Migration 0311 adds transfer tickets and upload receipts. Expired
orphan cleanup must not remove committed attachments.
- Configuration requires a new consent request containing that scope;
the single write-access choice controls it alongside work mutations.
Refreshing an old grant does not add it.
- The public directory keeps its original ten tools through the
companion Cloud change.
- Hosted consent/work proof remains the final delivery gate. The PR
stays draft while approval of the new staging grant is pending; code
checks and review are green. Merging is a separate action.

## Model Used

OpenAI Codex (GPT-6, tool use and code execution). The exact serving
model ID and context window are not exposed in this session. Paid
evaluation models: gpt-5.4-mini, claude-haiku-4-5-20251001;
claude-sonnet-4-6.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused suites; full-suite
macOS limitation disclosed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-10-06 20:29:58 -05:00
DottaandPaperclip 3447609d22 fix(runner): stream and page large API responses within capture budgets (#14301) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use governed API tools to inspect task evidence.
> - Large API results become saved assets with short previews.
> - Reading an asset through the same tool used to create another asset,
so the agent could not reach the rest of the evidence.
> - The 10 MiB response cap also blocked useful large results. Removing
all bounds allowed excessive disk use.
> - This pull request streams responses up to 1 GiB and makes saved text
readable in bounded pages. It adds durable run budgets and capture
admission limits.
> - Agents can inspect complete evidence while tool results, memory use,
and capture work stay bounded.

## Linked Issues or Issue Description

**What happened?**
A large response became an asset. Reading that asset returned another
asset and the same preview. Responses above 10 MiB failed before the
agent could read any page.

**Expected behavior**
The agent can fetch a large response and read its saved text to EOF.
Each page stays bounded. New snapshots have a generous finite limit and
a durable run budget. Existing larger assets remain readable through
byte ranges.

**Steps to reproduce**
1. Call a GET operation that returns more than 10 MiB of text or JSON.
2. Before the fix, the tool returns `api_transport_failure`.
3. With this change, responses up to 1 GiB become streamed snapshots
with artifact references.
4. Read `GET /api/assets/{assetId}/content` with `responseText:
{offsetBytes: 0, limitBytes: 8192}`. Follow `nextOffsetBytes` until
null.

Related work: #14186 added the API fallback tools. #14218 bounded API
discovery.

## What Changed

- Add authenticated UTF-8 text windows to `call_api`, with byte offsets
and total size. Keep each page at or below 24 KiB.
- Stream new responses above 24 KiB through private temporary files into
company-owned assets. Bound each capture to 1 GiB of decoded bytes.
Reject oversized declared lengths before reading and count streamed
bytes before writing.
- Reserve capture budget in the run record before spilling. Allow 4 GiB
per run. Settle successful captures to their actual size. Failed or
interrupted captures retain their full 1 GiB reservation. Run restarts
do not reset the budget.
- Enforce a 20 GiB company snapshot quota with database reservations.
Count legacy snapshots and unfinished storage work across runs and
processes. Asset deletion frees quota.
- Limit large captures to two per company and four per server process.
Hold slots through storage upload and temporary-file cleanup. Use a
10-minute download deadline and 30-second connection/idle-read timeouts.
- Return explicit size, budget, busy, and timeout errors. Preserve
unknown outcomes for mutations whose response cannot be captured.
- Read saved assets through authenticated storage ranges, with at most
two extra bytes for UTF-8 and EOF handling. Unpaged reads return the
existing asset and digest with a bounded preview. Reads create no copies
and do not consume capture budget.
- Keep existing assets above 1 GiB readable in pages. Use safe integer
offsets and PostgreSQL `bigint` asset sizes.
- Stream large S3 uploads through ordered multipart requests. Abort
failed uploads and remove partial local files.
- Revalidate run authority during downloads. Keep company authorization,
GET-only text paging, redirect denial, and mutation replay receipts.
- Document the separate 10 MiB upload limits. This PR does not raise
memory-buffered attachment ingestion limits. Future large video uploads
need streamed ingestion and storage quotas.

## Verification

- Full workspace `pnpm -r typecheck` and `pnpm build` pass after
rebasing on master.
- Focused API and response tests: 1,761 pass. Cover declared and chunked
oversize responses, incorrect Content-Length, exact-limit success,
active-stream deadline, cancellation, cleanup, concurrency admission,
and mutation outcome handling.
- Real HTTP integration: 28 tests pass, including runnerd → PRP →
authority → HTTP, a 12 MiB snapshot, final-page/EOF reads, cross-company
denial, a persisted 3 GiB sparse asset, and large mutation receipt
replay.
- The HTTP suite verifies durable run-budget accounting, simultaneous
runs competing for company quota, legacy snapshot accounting, deletion
refunds, failed-storage reservations, cleaned-failure refunds,
metadata-rollback cleanup refunds, preservation after a lost commit
acknowledgement, and small/saved reads after capture-budget exhaustion.
- A standalone proof streams exactly 1 GiB through the production
capture helper, verifies the final bytes, and removes its temporary
file. It uses repeated 256 KiB chunks and records a peak process RSS of
191 MiB.
- Earlier storage verification covers exact S3 multipart boundaries,
cleanup/abort failures, and a 17 MiB transfer through the real AWS SDK
to a local HTTP S3 endpoint. No cloud S3 qualification was run for this
follow-up.
- The local full test run was interrupted for the company-quota changes.
A later targeted run hit exhausted macOS shared-memory slots before
tests started; two unattached PostgreSQL segments with dead owners were
reclaimed before retrying. All 55 current-head checks pass at
`aebb80ceeeee77d5a56b67bfffd835f2f846878c`, including the full CI test
suite, typecheck, build, browser suites, security scan, and Greptile
(5/5). There are no unresolved review threads. The combined rebased test
catalog also passes (48 tests).
- Earlier paging acceptance passed Daytona and separate staging at
`7739879e9`. Those runs predate the streaming and budget changes.

## Risks

- The 1 GiB response cap and 10-minute active-download deadline are
intentional product limits. Larger live results must use endpoint
pagination or a direct file workflow. Existing larger assets remain
readable through bounded ranges.
- A durable 20 GiB company snapshot quota counts stored runner-api
assets and active/orphan reservations across runs and processes. The
operator can set PAPERCLIP_RUNNER_API_COMPANY_CAPTURE_MAX_BYTES to a
finite value of at least 1 GiB. Deleting snapshots frees capacity;
possible orphan storage must be reconciled before releasing its
reservation.
- A failed capture uses its full reservation. A new large capture needs
a full 1 GiB available, even if it later completes at a smaller size.
Small reads and existing asset pages remain available.
- Concurrency limits apply per server process. The run byte budget is
shared through the database.
- The `integer` to `bigint` migration rewrites asset metadata and takes
an exclusive table lock. File bytes stay in storage.
- A live endpoint is fetched once before returning its snapshot.
Continue reading the saved artifact for stable pages. Mutations may
commit before any size or transport error; inspect state before
retrying.
- Attachment uploads and native file handoffs still default to 10 MiB.
Raising buffered ingestion paths to GiB sizes is separate work.

## Model Used

OpenAI Codex, based on GPT-6, with code execution and repository tools.
The runtime does not expose an exact serving model variant or
context-window size. The earlier paging work also used browser testing
and subagents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-09-28 10:36:32 -05:00
Dotta 04a19cbc6e Address artifact PR review feedback 2026-05-30 20:51:46 +00:00
ForgottenandClaude Opus 4.6 fdd2ea6157 feat: add storage system with local disk and S3 providers 
Introduces a provider-agnostic storage subsystem for file attachments.
Includes local disk and S3 backends, asset/attachment DB schemas, issue
attachment CRUD routes with multer upload, CLI configure/doctor/env
integration, and enriched issue ancestors with project/goal resolution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 10:31:56 -06:00