mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-21 19:02:37 +01:00
30c718e04bcb7507596d5489e5eb91c3d0aa0a40
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
50b79d711a |
fix(runner): keep call_api writes carrying their body (#21002)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native agents use the `call_api` tool when no dedicated tool covers an API operation > - #20606 added an optional `saveResponse` flag to the shared `call_api` schema for GET reads > - After that change, OpenCode agents with DeepSeek v4.1 flash often sent write calls without `body` > - Body-less POST calls reached the routes and failed with 400; agents then repeated the same call many times > - This pull request moves the save option to its own read-only tool and refuses body-less writes before they are sent > - The benefit is that the write schema returns to its earlier shape, and a missed `body` gets one clear, actionable error ## Linked Issues or Issue Description Refs #20606 **What happened?** In a full chat-eval cohort on master (OpenCode, DeepSeek v4.1 flash), write calls through `call_api` arrived without `body`. The model's own reasoning planned a body, but the emitted calls had only `operationId`, `pathParams`, `query` and `saveResponse`. About 100 refused PATCH calls listed exactly those arguments (the cohort before #20606 had none with `saveResponse`). 72 POST calls reached the server with no body and got 400 (0 in the earlier cohort). One case sent the same body-less resolve call 21 times. **Expected behavior** Write calls carry their request fields in `body`. When a body is missing, the agent gets one clear error before anything is sent, and it can correct the call. **Steps to reproduce** Run the OpenCode chat cases that resolve approvals, edit queued messages, delegate with context, edit task properties or start preview services on master after #20606. Count `call_api` writes without `body` and POST 400 responses. **Additional context** The tool schema that OpenCode sends to the model is intact: `body` is present. A short-context replay with the same schema did not reproduce the omission (direct provider calls and local OpenCode runs: every write carried `body`, with or without `saveResponse`). The omission appears in long sessions, and it correlates with the added read-only flag. This change removes that flag from the write schema and adds a safety net for the remaining cases. ## What Changed - Remove `saveResponse` from the advertised `call_api` schema and description. The description now says that request fields go only in `body`, and the `body` description says when it is required. - Add a separate read-only `save_api_response` tool (`operationId`, `pathParams`, `query`). It accepts only GET operation IDs, has no body field, and uses the existing `saveResponse` export path with the same access, sensitivity, budget and integrity rules. - Refuse a write before dispatch when its body is missing: PATCH with a required body (as before), and POST or PUT whose JSON request schema names required fields. The error names the operation, the allowed fields, the received arguments, the shape to resend, and that `query`, `responseText` and `saveResponse` never carry request fields. - Keep server-side acceptance of `saveResponse` on `call_api`, so saved calls and receipts from earlier runs stay valid for recovery. - Regenerate the runner contract, catalog, seeded fixture and manifest. Update `doc/runner-api-tools.md`. ## Verification - Red at master sources: the 3 replayed recorded writes (POST resolve-from-comment, PATCH issue, POST comment, each with `query: {}` and `saveResponse: false` and no body) were sent to the route instead of being refused, and the schema test found no `save_api_response` tool. 4 tests failed. - Green with this change: `server/src/services/native-runtime/runner-api.test.ts` 2105/2105 passed; `runner-api.integration.test.ts` 61/61 passed; `runner-bridge-tools.test.ts`, `task-document-key-contract.test.ts` and `runner-api-artifacts.test.ts` passed; runner `protocol-action-contracts.test.ts` 70/70 passed; server `tsc --noEmit` passed. - `runner-api-artifacts.integration.test.ts` now saves through `save_api_response` and checks that a write operation ID or a body is refused. Its download step needs Linux; it fails on macOS at master too, so CI covers it. - Paid E2E, OpenCode + DeepSeek v4.1 flash, [run 43138806990](https://github.com/paperclipai/paperclip/actions/runs/43138806990) on this fix: 5/5 affected cases passed (questions-and-approvals.reject-and-revise, steering.edit-queued-message, task-coordination.delegate-with-context, tasks.edit-properties, workspaces.preview-service). Body-less write refusals went from 30 to 0, and body-less POST 400s went from 48 to 0, compared with the earlier cohort on the same cases. The approval resolve call was sent once (it was sent 24 times before). Every logged write carried a body; the remaining non-2xx writes are the cases' intended stale-edit and failing-fixture responses. - Run-history redaction also covers `save_api_response` arguments; a regression test fails on the earlier source and passes now. ## Risks - Agents that learned `call_api` + `saveResponse: true` in a running session will see a schema refusal and must use `save_api_response`. Saved calls keep working for recovery. - The new POST/PUT refusal applies only when the JSON request schema names required fields, so writes whose body is optional are still sent. - One more small read tool is advertised when API tools are enabled. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use, shell execution and code editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
faae6b919f |
fix(runner): explain task scope on missing tool inputs (#19774)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents read saved task documents through native tools.
> - Active-task tools use the task bound to the run. Other task tools
require a task UUID.
> - Agents sometimes call the second set with only a document key.
> - The validator rejects the call, but gives no instruction about the
active-task tool.
> - This pull request adds scope instructions to the tool schemas and
returns them for missing required fields.
> - Agents can correct a rejected call. Task selection and permission
checks stay in place.
## Linked Issues or Issue Description
**What happened?**
Two observed document reads sent only `{"key":"sentry-case"}` to
`read_task_document`. Each returned `Invalid tool input: / must have
required property 'taskId'`. Each later retry with a task UUID returned
without an error.
Pattern: [Task reads fail when agents omit
taskId](https://www.braintrust.dev/app/Paperclip/p/Paperclip%20Staging/patterns/703d8c29-6a40-4aa2-b2c6-e2c5c36b5657).
Related PR: #18824 changed document key validation. It does not address
task scope.
**Expected behavior**
The tool contract must state which task it reads. A missing-field error
must include the schema owner's recovery instructions. A retry must
still pass validation and task permission checks.
**Steps to reproduce**
1. Call `read_task_document` through the MCP bridge with only a document
key.
2. Inspect the error sent to the caller.
3. Retry with an authorized task UUID. Confirm the returned document
belongs to that task.
### Hypothesis and thesis for a fix
Hypothesis: generic tool descriptions contribute to confusion between
active-task and explicit-task tools. An agent can also ignore a correct
schema. These tests do not measure the rate of model mistakes.
Thesis: Paperclip owns task scope instructions. Its five explicit-task
tools now describe the required UUID and the active-task alternative.
The two MCP transports return direct missing-property descriptions from
the admitted schema. The transports do not infer task IDs or apply
Paperclip task policy.
Proof: the same HTTP MCP regression fails on baseline
`f673bd3cb5b5b65ec51781e37355e168c3fe8ae4` and passes with the fix. It
still rejects before handler dispatch. An explicit retry reaches the
requested task. An isolated real API and PostgreSQL test checks the
saved revision, active-task selection, malformed UUIDs, and
cross-company denial through both transports.
## What Changed
- Add UUID and active-task guidance to `get_task`, `comment_on_task`,
and the three explicit-task document tools.
- Return bounded, distinct descriptions for up to three direct missing
required fields. Keep ordinary validation errors unchanged.
- Share this formatter between the Runner and OpenCode MCP bridges.
- Add transport regressions, real route and database checks, and tool
documentation.
## Verification
- Baseline red: both transport regressions return only the
required-property error and fail the recovery instruction assertion.
- Candidate green: 58 Runner and OpenCode transport and error formatter
tests pass on Linux with Node 24.22.2.
- Final-head server verification passed all 115 tests: 113 document-key
checks and two real API integration cases.
- The two local integration cases use real MCP HTTP, agent authority,
API routes, and disposable PostgreSQL. They read a selected other task
and its saved revision, read the active task separately, reject missing
IDs for all five tools, reject malformed UUIDs, and return 404 for a
foreign company task.
- Runner TypeScript typecheck and build pass.
- The full server typecheck passed on head
`03a37e9e50896416cfd45be33bf6dbe3e13169df` with
`PAPERCLIP_TYPECHECK_LOW_MEMORY=1 node scripts/typecheck-layers.mjs`.
The first run ended without a compiler diagnostic. Final-head CI passes
and Greptile is 5/5.
- Full preflight stops because cargo is absent. Rust checks were not run
locally.
- One existing delayed-body fetch test failed with a Node 25 fetch
runtime error. All transport tests pass on Node 24. The first
integration attempt used the wrong fixture return shape; the test was
corrected before the passing run.
- CI follow-up: the Runner test program reported four TS18046 errors
because `Response.json()` returns `unknown` under CI typings. Both tests
now declare the response type. `pnpm exec tsc -p tsconfig.test.json
--noEmit` passes, and both transport suites pass again (56 tests). Final
head `ffccebca19fe9e21a097829adc020679ffa7d86c` passes CI and Greptile
5/5.
- Review follow-up: the foreign task now has a saved document with a
distinct body. Both integration cases passed again after this test-only
change. The read must return 404 and must not return that body.
Current-head CI passes and Greptile is 5/5.
- Final CI:
https://github.com/paperclipai/paperclip/actions/runs/42352822739 . All
required jobs passed for `ffccebca19fe9e21a097829adc020679ffa7d86c`.
Greptile is 5/5 with no unresolved review thread.
- Real local API behavior is verified. External model behavior and
deployed recovery are not verified. No merge or deployment is included.
## Risks
- Missing-field errors now contain schema descriptions. The text is
limited to direct properties in the admitted schema, at most three
distinct descriptions and 1,000 characters each.
- Nested schemas and references do not get recovery text. Their
validation errors remain intact.
- Clearer instructions can help callers recover. They do not guarantee
that a model will choose the correct tool.
## Model Used
- OpenAI GPT-6 via Codex, with tool use and code execution. The exact
deployment model ID and context window are not exposed to this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
c8822ffd4b |
fix(runner): deliver a lead's queued requirement to a busy task's next turn (#20028)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - In chat, a lead agent routes new requirements to tasks that other
agents own
> - The lead uses the native `comment_on_task` tool for this
> - The tool description only says the comment does not reopen or
interrupt the task
> - A lead read this as "the busy assignee never receives it" and tried
other routes, such as the assignee's workspace files
> - The comment API also keeps another agent's comment as a note unless
it sends `resume: true`, and the tool could not send it, so the queued
change was dropped when the worker completed the task first
> - This pull request states the real queued-delivery behavior and adds
an optional `resume` flag to the tool
> - The benefit is that the lead routes the change once, through the
product, and reports it honestly as saved for later
## Linked Issues or Issue Description
**What happened**
The person asked the chat lead to change the headline of an existing
website task. The website assignee was in an active run. The lead read
`comment_on_task` ("without reopening or interrupting it") and concluded
in its reasoning that the running worker would not see a comment. It
then listed the assignee's workspace directory and considered writing
the worker's input file directly. It used its full reply window without
routing the change.
When the lead did post the change with `comment_on_task`, Paperclip
queued it behind the active run. The worker then completed the task. The
wake queue cancelled the deferred wake ("no longer applies to a terminal
task") because an agent comment carries follow-up intent only with
`resume: true`, and `comment_on_task` sent only `{ body }`. The
requirement never reached the worker.
**Expected behavior**
The lead posts the change as a comment on the existing task with
follow-up intent. Paperclip queues it and delivers it to the assignee's
next turn, also when the assignee completes the task first. The lead
tells the person that the change is saved for that next turn, not
already applied.
**Steps to reproduce**
1. Start a task for a worker agent and keep the worker in an active run.
2. In the lead's chat, ask for a change to that task's requirements.
3. Read the lead's reasoning: it cannot tell from the tool description
that a comment reaches a busy assignee later.
4. When the lead comments, let the worker finish the task as done: no
continuation run starts for the queued comment.
## What Changed
- `server/src/services/native-runtime/runner-bridge-tools.ts`: the
`comment_on_task` description now says to use it for a new or changed
requirement on an existing task, instead of a duplicate task or the
task's workspace files. It says that a comment on a running task is
queued and delivered at the start of the assignee's next turn on that
task, and that the agent reports it as saved, not applied.
- The same tool now accepts an optional `resume` boolean. The
description tells agents to set it for a requirement change, and
explains that without it a comment on finished work is only a note.
-
`server/src/services/native-runtime/paperclip-runner-tool-authority.ts`:
`taskBridgeApiRequest` forwards `resume: true` to `POST
/api/issues/{id}/comments` and omits it otherwise. The existing
comment-route permission check for follow-up intent on another agent's
task still applies.
- `server/src/services/native-runtime/runner-bridge-tools.test.ts`:
regression tests assert the advertised description and schema, and the
forwarded request body.
## Verification
- `cd server && pnpm exec vitest run
src/services/native-runtime/runner-bridge-tools.test.ts` passes (11
passed, 6 platform-skipped).
- The wake-queue rule that keeps a resume-intent comment alive after
completion is already covered in
`server/src/modules/wake-queue/application/use-cases.test.ts`
(`done_no_resume` against resume).
- End to end: the `agent-chat.multiple-handoffs` chat eval, run with
this commit cherry-picked onto its case branch. Results are added as a
comment.
## Risks
- Low risk. No route or permission changes. The new `resume` field uses
the existing comment API flag and its existing authorization (assignee,
manager override, or open-write access).
- An agent that sets `resume: true` on a comment to finished work
reopens that work. This is the existing API behavior; the description
limits it to requirement changes.
- A runner with an active session goal can steer a comment into the live
turn. The description describes the normal queued path. It does not
promise the live path.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with
tool use (shell, file edits).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
86bc3db693 |
feat(runner): search and load unassigned company skills (#19676)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents use tools to read reusable company skills. > - The existing native skill tools read only skills assigned to the agent. > - An agent can stop when a task needs another skill that is already in the company library. > - This pull request adds tools to search and read accessible company skills on demand. > - Agents can use the saved instructions without a persistent assignment change. ## Linked Issues or Issue Description **Problem or motivation** An agent cannot use an unassigned company skill through the standard native skill tools. A generic API workaround needs extra discovery and can return the full library. **Proposed solution** Add `search_skills` and `load_skill` to the native tool catalog. Search returns small pages of stored metadata. Load returns text from a saved version and lists its files. Both tools use the current run identity and existing company and project access checks. **Alternatives considered** Assign each skill before a read, or use generic API discovery. Both add work to a read that the company already permits. **Roadmap alignment** This extends the existing skills discovery path. It does not add a store, permission model or installation flow. Related: #15409 adds an assigned-skill loader to a different adapter. ## What Changed - Add bounded company skill search. Search does not synchronize sources or fetch repositories. - Add versioned text reads for SKILL.md and supporting files. Page text and file lists. Reject unsafe paths and binary files. - Recheck access after storage reads. Keep the existing closed Muse, skill-test and completion-review tool scopes. - Add integration tests with isolated PostgreSQL and real local package storage. Update the native tool count assertion and skill documentation. - Limit text files to 16 MiB before body reads. Accept saved file paths up to 4,096 characters plus the repository prefix. Return the response digest only on the first page. - Add optional byte limits to the package readers. Select only the requested legacy file in PostgreSQL. Cast the size comparison to `bigint` so existing callers can omit the limit. - Cache decoded text and its digest for one run. Limit the cache to 64 MiB and 64 files. Check access on each page and clear the cache when the run closes. - Read repository source files and wrapper inputs through verified warm reads or isolated recovery. These reads do not hold the company startup lock. ## Verification - Local verification tested tree `7ba5828387ca0a9c2ee99655818a28de1235a854`, published as commit `c9d04cba601541cca506f5b72aa6f7272a25379f`. The published tree matches that tested checkout. Current head `33fa3d3d4c7988288685ce7c6499d064e0cb9079` (tree `8daaa2f3264d1cd65a9b6d85dbb9d3729ac4a29d`) removes six comment lines from three source files; its executable code is unchanged from the locally tested revision. - Four native skill and authority Vitest suites: 72 passed and 7 existing sandbox checks skipped. All 20 skill-library integration tests passed. - Skill-library and package-store suites: 65 passed. This includes the 45 package-store cases and the same 20 integration cases. Ran with umask `022` for executable-file tests. - Server source and test typecheck layers passed in low-memory mode. The affected source cycle also passed a final separate check. - `git diff --check` and `node scripts/check-module-boundaries.mjs` passed. - Red/green evidence: the new default inline-read case reproduced PostgreSQL error `22003`. Two max-size paging cases reproduced extra body reads. Warm and cold repository reads timed out while another connection held the company startup lock. All five cases pass after the fixes. - Both 16 MiB paging cases check that the second page performs no additional body read. They also check that source-access revocation blocks a cached page. Warm and cold lock cases load both the generated wrapper and a supporting file while the startup lock remains held. - The first cold-read test attempt had an incorrect helper import. Corrected the test import, reproduced the lock timeout, then verified the fix. - [CI on the previous revision](https://github.com/paperclipai/paperclip/actions/runs/42319652795) passed typecheck, build and policy. Serialized shard 2 failed in the unchanged test `schedules bounded retries for failed accepted interaction continuation wakes`. The same case passed locally: 1 passed; 416 cases were excluded by the filter. The cause of the earlier CI failure remains unknown. [CI for |
||
|
|
da77c1bcfa |
fix(runner): align ordinary document key contracts (#18824)
## Thinking Path
> - Paperclip helps people manage AI agents that do work.
> - Native agents use tools to read and write task documents.
> - A document key is a stable identifier. The service trims it and
converts it to lowercase before it checks the key.
> - The tool schemas accepted filenames and long keys that the service
rejects.
> - This pull request makes ordinary document tool schemas match the
existing service contract.
> - Agents can now see the key rules before they call the tools.
## Linked Issues or Issue Description
**What happened?**
The schemas for `read_document`, `write_document`, and
`list_document_revisions` accepted `reconciliation.md` and keys longer
than 64 characters after normalization. The task document service
rejects these values. The descriptions did not explain that the key is
an identifier rather than a filename.
**Expected behavior**
The model catalog and the live and scenario tool definitions must
describe the existing document key rules. Values accepted after trimming
and conversion to lowercase must remain compatible. Historical
case-document keys use a separate contract.
**Steps to reproduce**
Validate `{ "key": "reconciliation.md" }` against the previous
`read_document` schema. It passes. Call the document service with that
key. It rejects the key before it reads the database. The added
regression repeats this comparison for all three tools.
## What Changed
- Apply the same document-key pattern to `read_task_document` and
`write_task_document` at the API bridge. Keep their existing raw-input
limit of 100 characters; ordinary tools keep 120. Export the shared
schema through the existing Runner package and server vendor boundary.
Route permissions, normalization and document locks stay with their
current owners.
- Add one shared schema for ordinary task-document keys. Keep the
existing 120-character raw input cap and the 64-character normalized
identifier limit.
- Preserve uppercase input, ECMAScript trim whitespace, and the Kelvin
sign, which converts to ASCII `k`.
- Apply the schema to the model catalog and the three live and scenario
definitions. Regenerate the catalogs, seeded fixture identity, and
fixture manifest.
- Test schema parity against the real task document service. Keep
historical case keys and the backend validator unchanged.
## Verification
- Current-head required CI is green at
`e67d3f1f49393660f0c70b3d1d136e6f86af821d`: 56 successful checks, four
intentional skips, and two neutral checks. [CI attempt
2](https://github.com/paperclipai/paperclip/actions/runs/41715924252)
passed after one targeted rerun of five jobs interrupted by the same
runner shutdown; the original interruption is retained. Fresh Greptile
and Paperclip reviews are both 5/5 with no unresolved review threads.
Hosted recovery remains unverified until automatic deployment and
replay.
- Normal master refresh to `e67d3f1f49393660f0c70b3d1d136e6f86af821d`
includes the already merged external-object startup fixture correction.
The previous required canary failed downloading
`mdast-util-gfm-table@2.0.1` with npm E404; later registry metadata and
tarball HEAD both returned 200. A separate general shard failed the
ten-thousand-future-objects startup assertion. Both original failures
remain recorded; neither was bypassed or retried. At the refreshed
baseline, 122 document/bridge cases and the exact large-object startup
case pass locally. Six document cases skip by platform. Fresh required
CI and reviews must pass on this head.
- The current-head Paperclip review found the same mismatch in the
task-scoped bridge. Before the correction, the real advertised schemas
failed 22 parity cases while 87 controls passed. After the correction
and a normal master refresh, the document-key and bridge suites pass 122
cases, with six platform-specific skips. The tests compare advertised
JSON Schema, runtime Zod validation, and the actual normalized document
service. They retain the bridge raw 100/101 boundary and historical
case-key controls.
- Runner TypeScript build, all server typecheck layers, and the actual
production Runner and server build pass. The final server build stamp
binds commit `80c880709c5f5630799fb8a48d54ff11ebc16610`. Full
current-head CI and fresh review are required after the correction. The
earlier CI run was externally cancelled and rerun; its cancellation
remains recorded.
- The regression failed in 33 of 64 cases before the fix. The remaining
cases passed as controls.
- The fixed regression passes all 67 cases, including every ECMAScript
trim character.
- The four focused Runner contract suites pass: 116 tests. They cover
all three tool surfaces and existing case behavior.
- Generated catalogs and manifest checks pass. Module-boundary and diff
checks pass.
- All server production typecheck layers pass. The test layer passes
after correcting the new test’s Ajv import. The full production server
and Runner build passes at
|
||
|
|
09ad9adce2 |
feat(server): adopt running remote agents through durable controller checkpoints (#17439)
## Thinking Path > - Paperclip must keep admitted remote work alive while its main server is replaced. > - Runner owns recovery mechanics and exposes opaque checkpoints through #17097. > - The server owns company scope, encryption, Stop, tool authority and execution ownership. > - A successor must prove current ownership before it adopts the original execution. > - Checkpoint bursts must also leave capacity for normal requests and lease renewal. > - This PR persists and adopts that authority through the existing runtime boundaries. ## Linked Issues or Issue Description **What existing behavior does this improve?** Remote agents retain the original execution during a bounded main-server outage. **Current behavior** A server restart can lose the controller's local state and connection. An unsafe replacement could repeat a turn or an uncertain mutation. Checkpoint writes in the request pool can also delay ordinary requests during an aligned burst. **Proposed behavior** Restore encrypted control records from Postgres and adopt the original runner under a fenced lease. Recover known tool receipts through current tool authority. Give control checkpoints a shared pool capped at two connections per database owner. Depends on #17097. Successors are #17455 and #17456. Qualification and presentation remain separate review slices. ## What Changed - Add company/run-bound encrypted checkpoint records and migration 0394. Keep the SQL idempotent after renumbering against current master. - Recover startup and expired ownership through native runtime and heartbeat. Preserve the sandbox lease, admitted runner artifact, harness configuration, native session and active turn. - Keep Stop, finalization, preparation ownership and stale-writer fencing transactional. Preserve graceful detach and natural crash takeover. - Recover committed tool receipts through the existing authority. Reauthorize reads and hold uncertain mutations. - Bound startup retention with the existing coordinator and preserve zero idle scans. - Share at most two checkpoint connections per database client. Keep application and liveness capacity separate. Close the checkpoint pool during shutdown; late writes cannot reopen it. - Count SQL across both application and checkpoint clients. Prove request/liveness progress, command recovery and failure after pool shutdown with real Postgres. ## Verification Project creation recovery now uses the existing project activity receipt owner. It retains authenticated input normalization, fingerprint checks and current run authority after the receipt lock. An absent receipt returns no result before mutation. The new lost-HTTP-response and no-effect regression uses real API routes and Postgres. The seven transport cases pass locally. This Mac skips all 20 database cases. On current source `5de803c2cd`, Linux [job 122593415890](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415890) executes all 20 project-tool cases successfully (9.480 seconds), including the committed-project lost-response regression. Its shard passes 1,627 cases with three unrelated skips. Server typecheck passes. The transport regression initially has two failures; a subsequent incorrect expectation about schema defaults was corrected before the final pass. Ordinary execution adds no checkpoint writes or recurring work. Current candidate `5de803c2cd` includes the project-creation receipt recovery fix and master `2cd747d1b6`. Migration 0394 retains the prior idempotent checkpoint DDL. Every master SQL file and journal entry is preserved. Snapshots 0390–0394, migration checks and server typecheck pass. The ordinary warm-turn regression fails on its second turn before `a219ab51ad`. After correction, three completed warm turns select the correct prior checkpoint and preserve the original session. All 288 provider-wiring cases pass. The control plane, external persistence callback, identity rotation and filesystem are real; the backend and database-load seam are scripted. The 13 warm admission/configuration variants also pass after final master integration. This does not qualify an underlying harness. At current source `5de803c2cd`, Linux [job 122593415907](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415907) passes all 940 native-session-executor cases (61.431 seconds), including the warm external-checkpoint behavior; the shard passes 2,358 cases with one skip. Current [Runner job 122593415475](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415475) passes the native recovery lane: 115 passes and five container-only skips, including the encrypted restore guard, fresh preparation (63.189 seconds), blocked claim lock (1.226 seconds) and recovery beyond 101 blocked owners (3.897 seconds). The first lane passes 2,215 cases with one skip. The SQL cost test reports zero idle and streaming SQL for both output volumes; this is component evidence, not a production capacity claim. At prior candidate `1ff7bb44f4`, Linux job 122549307215 passes all 938 native-executor cases, including the three-turn external-checkpoint regression. Job 122549306469 passes the native recovery lane with 115 passes and five container-only skips, including the encrypted-restore session guard, fresh preparation (63.664 seconds), claim-lock contention (3.961 seconds), and 101-owner recovery (3.824 seconds). This Mac skips the real-Postgres suite. The earlier full local executor attempt has 921 passes and 16 failures during embedded-Postgres initialization before assertions. The original fixture collisions and their corrections remain recorded. At earlier candidate `8fbe62b41d`, server shard job 122535673841 has 3,876 passes, one failure and one skip in an unchanged expiry-worker retry/drain timing test. The same server/database tree on qualification source `b64cac6951` passes all 20 expiry cases and all 3,877 shard cases. The single diagnostic retry, job 122541891915, passes all 20 expiry cases and all 3,877 shard cases (one skip). No assertion was weakened. The original failure remains recorded and its exact timing race is not proven. All Actions checks and legacy status passed on `8fbe62b41d`, with completed Greptile 5/5 and no open feedback. The next candidate `1ff7bb44f4` also passed all Actions checks and Greptile 5/5. Its Paperclip Review remained queued before the new master conflict required integration. The new candidate needs fresh exact-head CI and reviews; old checks do not qualify it. Migration generation is checkpoint-only, all master SQL and journal entries remain unchanged, and migration checks, server typecheck and all 13 warm-owner variants pass after integration. The first migration check was blocked by TSX sandbox IPC; its permitted repeat passes. The first warm-test command used an incorrect path and ran no tests; the corrected command supplies the 13-case receipt. All 16 database pool-lifetime/options/teardown cases pass after granting loopback permission; the first batch had two EPERM failures and 14 passes. Earlier semantic-tool/HTTP recovery, adoption, encryption-batch, zero-streaming-write and idle-SQL receipts remain source-attributed in the [qualification record](https://github.com/paperclipai/paperclip/blob/2ff66f57da0da35bac1c19a2366ea4b84d87a986/doc/plans/2026-10-22-hot-upgrade-qualification.md). That record retains earlier assertion, setup and cleanup failures and their corrections. No failed gate is waived. The historical 300-session component campaign passes four after phases: steady read p99 is 107–108 ms at 500 reads/s and 100 writes/s; aligned-burst read p99 is 103–105 ms after the bounded pool, versus 584–689 ms before. This comparison isolates the persistence corrections. It does not measure the final stack's hosted active-agent capacity. The hosted zero-agent API pair leaves SQL and CPU essentially unchanged, but both arms fail absolute latency gates and candidate peak cgroup memory fails its gate. The active-agent/viewer matrix, different-artifact upgrade and rollback remain open. Run `PAPERCLIP_TEST_DATABASE_URL=<owned-disposable-postgres> pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-runner-restart-recovery.integration.test.ts`. Previous head `f2db8a6568` had one failed workspace setup gate: [job 122555672441](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122555672441) passes all 4,169 UI tests and 285 CLI cases (one skip), but the company import/export suite reaches its 30-second health deadline during doctor startup. Its output does not reach the server-start step. The CLI, server, database, shared, adapter and plugin trees are identical on qualification source `a866619d7b`, whose [job 122555886663](https://github.com/paperclipai/paperclip/actions/runs/40609057937/job/122555886663) passes that fixture and all 286 CLI cases. The one diagnostic retry, [job 122561057225](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122561057225), passes the formerly failing fixture and all 286 CLI cases. All Actions checks and legacy status on `f2db8a6568` pass. Its exact-head Greptile completed at 5/5 with no open feedback. The current integrated head requires fresh checks and a completed Paperclip Review before merge. The original failure remains recorded; its exact timing cause is unproven. A repeated failure requires investigation and repair rather than another retry. All current-head CI, legacy status, exact-head completed Greptile 5/5, resolved feedback and ordinary GitHub merge eligibility remain required before merge. The latest integration preserves master’s published Copilot v70 profiles and every published migration. Checkpoint migration `0394_conscious_whizzer.sql` was generated with `pnpm db:generate` and retains the prior idempotent DDL. The Runner vendor rebuild and replay parity checks pass. Server typecheck and all 13 warm-owner variants pass after that rebuild. The imported report-lock patch is byte-identical to master; its intentional patch-context whitespace is retained, and the feature delta passes whitespace checks. No hosted experiment was run. ## Risks The pool adds up to two active connections per instance, shared across agents. Checkpoint delivery can wait behind that bound; it still requires its commit. Steady component-test Postgres CPU rises by about 1.7 percentage points of a two-CPU allocation. These corrections do not remove the stack's control and renewal writes. The migration and key must survive replacement. Fresh-database tests and local load do not establish different-artifact hosted upgrade or rollback continuity. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
dd777f4b73 |
feat(dot): complete onboarding and expand governed Runner capabilities (#15414)
## Thinking Path
> - Paperclip manages AI agents, their work, and their permissions.
> - Paperclip Runner supplies the same admitted tool authority to each
provider.
> - The Dot provider in #15402 needs reliable onboarding and useful
agent capabilities.
> - An idle Dot could not start work, assign a human, read skills, or
produce a workspace artifact.
> - Pairing also relied on a second configuration save before ordinary
admission could work.
> - This pull request adds governed idle admission and shared Runner
tools, and completes pairing atomically.
> - Operators can test Dot with fresh data while keeping normal company,
approval, budget, and run ownership checks.
## Linked Issues or Issue Description
**Subsystem affected**
Paperclip Runner, dedicated Dot MCP access, OAuth onboarding,
experimental settings, and empty worktree startup. This PR builds on
merged provider PR #15402. It reuses the merged MCP gateway from #14846
and assistant connection work from #14933 and #15380.
**Problem or motivation**
An idle Dot could see assignments but could not act on a conversation
request until someone created a task first. Its Runner catalog could not
assign tasks to humans or read pinned skills and workspace files.
First-time OAuth discovery and pairing also needed browser fixes, and a
completed pairing did not persist its binding reference on the agent.
**Proposed solution**
Keep Dot within the existing Runner. Admit a visible agent-authored
intake task for idle requests. Add people, human assignment, cross-task,
skill, and optional sandbox workspace tools through shared authority.
Relay assigned app calls through the configured MCP gateway. Add lease
renewal and follow-up references. Save pairing and its configuration
revision atomically. Show prerequisites and provide a complete copy
prompt.
**Roadmap alignment**
This extends the experimental provider in #15402. It uses the existing
governed gateway, task model, skills, artifact path, and native Runner.
It adds no separate execution subsystem.
## What Changed
- Add a standalone OpenAI Dot agent choice with an independent
experimental opt-in. It works with the general Runner option off.
Require Assistant connections (MCP), authenticated sign-in, and public
HTTPS for pairing.
- Use Dot’s own option for company package import. Allow an unpaired Dot
configuration to save after external billing acknowledgement; task
admission still requires pairing. Prepare the shared dev binary for
Dot-only opt-in.
- Label saved Dot agents as OpenAI Dot. Use prerequisite-check copy in
setup and runtime configuration.
- Route Dot creation directly to pairing after explicit external billing
acknowledgement. Hide local CLI, model, and harness setup for Dot.
Preserve the shared Runner implementation and canonical API type. Other
Runner providers still require their general opt-in.
- Add an empty worktree option with fresh signing keys and no production
data copy.
- Fix public-client OAuth negotiation, discovery compatibility, and
optional separate browser authorization origin.
- Add one-use pairing consent preview, clear copied setup instructions,
and atomic binding persistence and cleanup.
- Add idle request admission with stable request IDs and normal
scheduling, permissions, budgets, and task ownership.
- Add identity and people discovery, human task assignment and
reassignment, and authorized cross-task comments and documents.
- Read assigned skill files from pinned manifests. Relay assigned app
calls through the merged gateway without exposing credentials.
- Add an off-by-default workspace bridge. Constrain paths and writes.
Run commands in a deny-by-default OS sandbox with no network or injected
credentials. Reserve mutations before effects and never blindly repeat
uncertain work.
- Add rolling lease renewal, task pagination, bounded operation limits,
and deduplicated follow-up references without comment bodies in
webhooks.
- Add an off-by-default attachment reading setting. Restrict reads to
files on the current assigned task. Verify size and hash, cache bounded
verified copies per run, paginate text or binary bytes, and recheck live
authority before returning.
- Keep file grants operator-owned. Reject agent self-grants across
configuration routes. Preserve attachment consent in create/import
forms. Close generic API file bypasses while retaining current-run
response snapshots and permitted uploads.
- Keep provider limits explicit. Do not inherit a Dot binding,
attachment permission, or workspace permission when hiring another
agent.
- Include the required Markdown format in cross-task document writes and
validate the API title limit. Verify real creation and revision
persistence.
- Restrict command execution to Linux bubblewrap with descendant
containment. macOS retains workspace file tools and artifact publishing,
while refusing command calls. Explain the platform limit in setup.
- Exclude Paperclip instance state from workspace files, uploads,
artifact publication, and sandbox commands. Protect nested directories
and case variants. Fail closed when the directory protection scan
exceeds 4,096 directories.
- Add static UI compression for slow public tunnels. Document setup, the
complete tool inventory, and qualification limits.
## Verification
- Merge preparation on `00ca2c75b` integrates merged base #15402 and
master `fc6304dfe`. The ancestry commit preserves the reviewed follow-up
source tree. The subsequent security fix excludes instance state from
file tools, uploads, artifact publication, and sandbox commands. It
preserves private task authorization and task monitors. It regenerates
the combined tool catalog, seeded catalog digest, and protocol manifest.
Workspace typecheck, full build, and UI token gates pass. All sixteen
real Dot broker cases and 93 company import cases pass after the review
fixes and creator-attribution test correction. The exported
human-assignment catalog and Unicode page boundaries are also fixed. All
ten catalog tests and nine workspace/skill bridge tests pass. All 35
workspace bridge and authority tests passed after the instance-state
fix, including real macOS commands in that intermediate version. The
subsequent document and descendant-containment fixes pass 44 focused
tests across bridge, authority, and setup UI, with six Linux command
cases skipped on macOS. Real cross-task documents pass the route
validator and persist two revisions. macOS refuses command execution and
does not advertise the tool. Full workspace typecheck and build, changed
server/UI typechecks, and token gates pass again on the final commit.
Greptile rates final head
|