mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 09:07:46 +02:00
9db3a5726e70fe00579e7bf7dfdd04ff48cb4b35
5505
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
9db3a5726e |
refactor(tools): isolate GitHub access discovery (#17144)
## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Tool connections discover repositories through GitHub account grants. > - The tool-access service mixes GitHub response parsing with grant authorization and persistence. > - These responsibilities have different inputs and failure rules. > - This PR puts GitHub discovery in one provider module. > - The service keeps current authority checks and stored grant updates. ## Linked Issues or Issue Description **What existing behavior does this improve?** The ownership and review of GitHub access discovery in the tool-access service. **Current behavior** The 21,525-line service contains GitHub pagination, account metadata validation, and grant lifecycle code. **Proposed behavior** A GitHub module returns normalized access metadata. Existing service exports remain compatible. This is the first PR in a three-part refactor stack. No behavior change is intended. A search found no existing tool-access refactor PR or issue. Stack order: this PR, #17146, then #17147. Each PR targets its predecessor. ## What Changed - Move six functions into `tool-access/github-access-discovery.ts`. Their bodies are unchanged. - Keep grant selection, credential resolution, refresh retry, membership locks, revision checks, and webhook updates in the service. - Add public-entry characterization tests for HTTP error classification and invalid pagination. - Record five candidate boundaries and the initial three-PR scope in the dated plan. - Reduce `tool-access.ts` from 21,525 lines / 829,519 bytes to 21,237 lines / 820,116 bytes. The new module is 294 lines / 9,722 bytes. ## Verification - Before and after: `pnpm exec vitest run server/src/__tests__/github-grant-metadata.test.ts server/src/__tests__/project-repositories.test.ts` — 13 tests pass. - Before and after: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts -t 'GitHub|github'` — 11 selected tests pass with disposable PostgreSQL; 455 cases are outside this selection. - `pnpm --filter @paperclipai/server typecheck` passes. - Exact text comparison confirms all six moved bodies match the base. `git diff --check` passes. - The initial baseline attempt failed because the plugin SDK had not been built. The normal workspace prerequisite build resolved it before behavioral validation. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. CI is green at `07e78dd1b1c26e0bd4234006766444c71e234451`. Greptile is 5/5 with no actionable findings. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - Import wiring could break existing consumers. Existing imports remain supported and all four production call sites are unchanged. - Provider requests retain their ordering, timeout, fixed GitHub origin, and response redaction. The new module has no database dependency or shared mutable state. - Tests simulate GitHub responses. They do not qualify live GitHub access. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
0081863463 |
Fix isolated ACPX admission for assigned PRP tools (#17149)
## Thinking Path - Paperclip uses Runner to execute agent tasks under Core's selected policy. - The ACPX sidecar can receive assigned MCP tools in its PRP catalog. - Isolated admission still requires a separate native MCP endpoint for those assignments. - This check rejects a valid PRP route before the provider can start. - This PR identifies the sidecar's PRP transport at the runtime-host boundary. - The benefit is that assigned tools work with the existing isolated permission policy. ## Linked Issues or Issue Description **What happened?** An isolated Claude task with an assigned MCP binding fails with `assigned native MCP launch binding is unavailable` when Core delivers its tools through PRP. **Expected behavior** The sidecar must accept its authenticated PRP tool bridge as the assigned-tool route. Callers that use the native gateway must still supply its complete binding. **Steps to reproduce** 1. Open an isolated Claude sidecar session with `deny-all` permissions and a read-only policy. 2. Supply an assigned MCP binding in the runtime context and its tools in the PRP catalog. 3. Omit native gateway environment variables. Admission fails before provider startup. **Paperclip version or commit** Reproduced against `92cfd701c39bb1ea394a796023481a21decb2532`. **Deployment mode** Runner with an ACPX sidecar and controller-owned tool delivery. Related contract work: #16978. The search found no existing PR or issue for this failure. ## What Changed - Let runtime-host callers select PRP delivery when they supply the semantic tool bridge. - Select that route in the sidecar. Keep native gateway validation and direct callers' default behavior. - Preserve exact controller operation names in PRP mode, including the compact assigned-MCP catalog. Keep existing aliases for other bridge callers. - Preserve the selected permission and read-only policies. - Add one process-level sidecar regression and one host admission regression. The host test calls the real authenticated MCP bridge. - Document the route and refresh the generated Copilot source hashes. The provider and profile versions stay the same. ## Verification - Passed 11 focused tests after the rebase across the runtime host, sidecar, tool bridge, and Copilot identity tests. They cover both compact catalog operations, restricted and approve-all policies, legacy aliases, missing and invalid gateway bindings, and the existing direct gateway path. - The process-level test crosses the real sidecar dispatcher and stops at workspace validation before provider startup. The host test uses real ownership sockets and MCP HTTP, with a fake provider port. - The generated identity tests match the current source hashes. The historical fixture stays unchanged. - The standard review passed on the rebased commit. [Apex passed the required 5/5 threshold](https://github.com/paperclipai/paperclip/runs/119513362214) on `e63a223`, with 11 files reviewed and no findings. - Rebased onto master `734c265`. The conflicts affected generated source identities and their test expectations. All reviewed behavior files remain byte-for-byte unchanged. The user authorized retaining the Apex result for this rebase. [CI passed on `3facef5`](https://github.com/paperclipai/paperclip/actions/runs/39685163239). The final PR checks contain 56 passes and four expected skips. The branch is current with master, has no conflicts, and has no unresolved review comments. - `node packages/paperclip-runner/scripts/copilot-profile-identity.mjs --check` passed. - Initial local runs exposed missing test dependencies and sandbox denial of loopback sockets. After the scoped dependency install and socket permission, the selected tests passed. - Broad typecheck, build, and test verification runs in GitHub CI, as requested. No live provider or model request was made. ## Risks - PRP callers must supply the assigned tools in the controller-owned catalog. The existing bridge still enforces its closed catalog and authentication. - An explicit PRP selection without a semantic bridge still fails. Invalid native gateway bindings still fail. - No database, protocol-frame, provider-version, or credential-storage change is required. ## Model Used OpenAI GPT-6 through Codex, with source inspection, editing, and command execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.17 |
||
|
DottaandPaperclip
|
734c265339 |
Default new Pi agents to Paperclip Runner (#17142)
## Thinking Path > - Paperclip manages AI agents and their work. > - Users choose a harness during agent setup. > - Claude, Codex, and OpenCode already receive their qualified Runner default. > - Pi has a qualified native integration but new Pi agents still use the direct adapter. > - This change gives new Pi agents the native default on supported targets. > - Saved agents keep their runner and compatible settings. Legacy remains available in Advanced. ## Linked Issues or Issue Description Refs #17120, #14921, #14922, #14923, #14924, and #14956. Related generic default work: #15422. This PR is the separate Pi follow-up to the merged OpenCode change. **Current behavior** Choosing Pi for a new agent saves the direct Pi adapter. The native Pi integration remains behind the generic experimental flag. **Proposed behavior** Default new Pi agents to ACPX Pi on macOS ARM64/x64 and GNU/Linux x64 with supported glibc. Keep unsupported targets and explicit Legacy selections on the direct adapter. Require an explicit provider/model before test or hire. Permit model-free inactive setup drafts. **Reason and benefit** Users choose Pi once and receive its qualified runner. Saved agents retain their identity and settings. Conversions reject unsupported legacy thinking and launch settings instead of dropping them. ## What Changed - Add Pi to the shared platform/default contract and server creation selection. - Admit native Pi without the generic experimental flag. - Preserve provider models, credentials, workspace settings, exact thinking, and saved native policies. - Add Pi runner controls, branding, setup resume, onboarding, built-in configuration, and import handling. - Add managed Pi OpenRouter routing and matching native provider API-key accounts. Keep subscription and custom gateway exclusions. Clear alternate credentials and reject custom Pi provider overrides. - Keep Pi draft models empty instead of applying Claude's default model. - Verify setup against the pinned Pi runtime. For native Pi on the official OpenRouter route, reverify the API key live without requiring a global Legacy Pi CLI. - Match Pi’s bounded controller close window to its native snapshot cleanup and lifetime verification, preserving explicit caller deadlines and the mandatory drain/suspension proof. - Bind Pi sidecars to the package that owns the pinned native assets, including source and vendored server layouts. Other providers retain their existing package authority. - Keep protocol-owned finish/block tools out of the ACPX forwarded catalog. The trusted bridge installs these tools itself; the complete Rust authorization catalog and terminal receipt ledger remain intact. This fixes actual Pi session startup and applies to other ACPX consumers. - Advance Copilot’s shared-source attestation to v58 because it pins the shared files touched by Pi. Preserve immutable v57 bytes and historical decoding; Pi’s own release identity is unchanged. - Add regression coverage and update the paired maps and developer documentation. ## Verification - Passed: full repository typecheck and full production build on final headcanary/v2026.1021.0-canary.16 |
||
|
DottaandPaperclip
|
92cfd701c3 |
feat: add governed Braintrust app connections (#17141)
Add hosted US and EU Braintrust OAuth and API-key connections through the existing governed MCP system. Keep reviewed tool risks and catalog review policy in connector modules and a narrow registry. Preserve credential ownership, operator decisions, regional setup and changed-tool quarantine. Include official artwork, setup guidance, focused regressions and sanitized qualification evidence. US OAuth has bounded live read/write, gateway, reconnect and cleanup proof. Remaining account-method and token-lifecycle qualification gaps are documented. Co-Authored-By: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.15 |
||
|
DottaandPaperclip
|
efc2ddc7bf |
fix: stop idle login cleanup polling (#17106)
**Dependency landed:** PR #17129 is merged. This branch is rebased onto master `0f15f82b19` and preserves its provider-owned completion contract. The shared save path calls `notifyLoginSessionChanged(tx)`; the cleanup module invalidates both existing sandbox cleanup workers without selecting a provider. ## Thinking Path > - Paperclip manages work for AI agents. > - Subscription login can allocate a temporary sandbox. > - Existing cleanup workers scan the database every heartbeat tick, even with no login work. > - Empty scans keep idle databases active. > - This PR uses committed session changes and saved expiry times to schedule cleanup. > - Existing rows still drive recovery after failure and restart. ## Linked Issues or Issue Description **What happened?** On master `898144f4b3`, the enabled heartbeat scheduler runs both login reapers every 30 seconds by default. An instance with no pending logins still queries sessions and leases. **What did you expect to happen?** After startup reconciliation, idle login cleanup should make no recurring SQL or provider calls. New sessions, expiry, failed cleanup and restart must still recover. **Steps to reproduce** Start a claimed instance with heartbeat scheduling enabled and no pending login sessions. Observe repeated adapter-login and setup-token cleanup queries at the heartbeat interval. Related sleep migration: #17094. Searched open issues and PRs. Related setup-token cleanup work: #15774, #14236 and #16648. This PR does not change their release-result or null-lease behavior. ## What Changed - Register separate adapter-login and setup-token workers with the existing delivery coordinator and scheduler. - Notify before writes in the owning stores, the lease-tag write, and AI connection save. Shared save uses a provider-neutral notification; each cleanup worker checks its own durable predicates. The shared transaction protocol retains intent until the outer transaction settles. - Reconstruct session expiry, promotion expiry and concrete cleanup retries from existing rows. Preserve configured heartbeat spacing to bound scan bursts. - Recover tagged orphan leases even with terminal or missing sessions. Protect live acquisitions that have not attached their lease yet. - Exclude local-subscription rows from sandbox reapers. Their existing local-login worker owns cleanup. - Preserve deferred work across drain and shutdown. Add no tables, Cloud alarms or host-sleep exemptions. - Update the inventory and document producer coverage, writer scope and recovery boundaries. ## Verification - Updated head: `1d57e13606`. Seven focused suites passed (282 tests): AI connections, login cleanup, device-login service/reaper, login-session runtime, and setup-token session/reaper. `pnpm -r typecheck` and `pnpm build` passed. The duplicate local `pnpm test:run` was interrupted after CI passed all server shards; it is not counted as a complete local pass. Full-suite evidence comes from the exact-head CI run. Greptile reviewed this exact head at 5/5 with no findings or unresolved threads. [Full CI run 39669285577](https://github.com/paperclipai/paperclip/actions/runs/39669285577) passed on attempt 2. Both aggregate gates passed; no checks remain failed or pending. - Extended the real PostgreSQL AI connection tests for both Anthropic and OpenAI. The tests check that neither cleanup observer wakes inside the completion callback, both wake after commit, and an idempotent save does not emit another wake. - Existing cleanup tests cover idle/history states, producer wake, expiry, promotion, cancellation, retry/restart, orphan recovery, nested commit/rollback, drain, concurrency, and shutdown. - Current-head CI attempt 1: the unchanged `sandbox-callback-bridge.test.ts` HTTP/2 preface test failed because its child process could not bind `127.0.0.1:34033` (`EADDRINUSE`). Its single unchanged rerun passed. The exact test also passed locally without changing code or timeouts. This establishes recovery from the bind collision; it does not establish which process owned the port. No login code ran in that package test. - Previous-head failures remain recorded: the login-runtime transaction fixture was corrected; a workspace-runtime readiness port-bind failure passed in isolation and fresh CI, but its first failure cause remains unconfirmed. An earlier combined AI suite timed out in local-login isolation and passed alone; its timing cause remains unconfirmed. An earlier local full suite was interrupted and was not counted as passing. ## Risks - Process-local signals cover one active server with the instrumented shared database handle. Independent SQL writers or replicas need their own signal integration. - Provider resources that never became persisted tagged leases remain the responsibility of the unchanged general lease/spool recovery. Setup-token acquisition retains its required provider expiry. - Cleanup starts are bounded by the configured heartbeat interval. Writes during a sweep can cause a final follow-up scan before quiet. - Existing login and resource blockers still prevent unsafe host sleep. This is worker polling removal, not provider sleep qualification. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and test execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal/instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.14 |
||
|
DottaandPaperclip
|
0f15f82b19 |
feat: offer existing Claude and Codex agents a runner upgrade (#17122)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each agent stores its harness, model, account and runtime configuration. > - New Claude and Codex agents now use Paperclip Runner on qualified targets. > - Existing agents keep their legacy runner and need a clear upgrade action. > - The existing configuration API already translates settings and records revisions. > - This pull request adds an explicit upgrade action that uses that API. > - Operators can retain supported settings and correct rejected settings in place. ## Linked Issues or Issue Description **What existing behavior does this improve?** Existing Claude and Codex agent configuration. Related defaults: #15683 and #15993. **Current behavior** Existing agents retain their legacy runner. The operator must find the runner selector in Advanced to change it. **Proposed behavior** Show Upgrade runner when Paperclip Runner is the qualified default for the saved environment. One click saves pending edits and requests the existing server conversion. Show any rejected settings without changing saved configuration. **Reason and benefit** Operators can adopt the current runner while retaining their harness, model, account bindings, instructions and history. Unsupported settings remain visible and editable. **Breaking changes** None. Visiting configuration and saving unrelated changes do not upgrade an agent. The server API, authorization and conversion rules are unchanged. ## What Changed - Add an upgrade notice, pending state, saved result and retry feedback to the production configuration form. - Include pending profile, runtime and environment-variable edits in the upgrade request. - Bind asynchronous upgrade feedback and save receipts to their original agent. Navigating away cannot show success on another agent or redirect back. - Keep the notice absent for native agents, other harnesses, unavailable targets and failed discovery. - Add configuration/API regressions and database session-reset checks for Claude and Codex. - Add 13 production-page Storybook states with browser checks in both themes and on mobile. - Update the product contract and feature/access maps. ## Verification - Configuration form and original-agent save receipts: 135 tests passed. Existing agent route/rename helpers also pass. - Runner selection, configuration API and database migration regressions: 220 tests passed. - Storybook browser checks: 26 passed. Interactive cases wait for Storybook's visible Pass report. Static cases check the rendered production form. These use simulated HTTP boundaries. - Full typecheck, build, map checks and token gates passed on the current head. Storybook build and public preview verification passed. - Published fixture preview: https://storybook.paperclip.page/storybook/branches/codex~2Fagent-runner-upgrade/?path=/story/agents-runner-upgrade--legacy-codex - All 58 current-head PR checks passed, including the full test shards, browser suites, build and aggregate gates. The duplicate local pnpm test:run was stopped after 34 minutes once the same-head full CI suite passed; it had not finished its general-server phase. No local full-suite pass is claimed. - Real disposable local test drive: Codex used gpt-5.4 and Claude used claude-sonnet-4-6. Both completed one task on Legacy, upgraded through the browser button, and completed another task on Paperclip Runner with the same model and API credentials. Independent file checks confirmed saved instructions and environment bindings before and after conversion. - Reopened both configurations and compared identity, model, credential references, instruction hashes, working directories, run policies and configuration revisions. - A Claude Chrome setting produced a field-specific rejection. The saved agent stayed Legacy. Clearing Chrome in the form and clicking Upgrade runner saved the pending correction and completed conversion. ## Risks - The existing translator rejects settings that the new runner cannot honor. The notice shows that error and leaves the form available for correction. - Converted agents start fresh sessions. Prior runs remain available. - Real task execution was checked on local macOS with API credentials. SSH, sandbox targets and subscription credentials were not exercised in this change. - Storybook responses simulate persistence. The real test drive and database tests provide separate persistence evidence. ## Model Used - OpenAI GPT-6 through Codex. The exact serving variant and context-window size were not exposed in this session. Used reasoning, code editing, shell tools and browser interaction. No subagents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
13a83dd68a |
docs: require sleep-aware background work in reviews (#17138)
## Thinking Path
> - Paperclip manages agents in instances that can be idle between
tasks.
> - Recurring empty database and network checks can keep idle instances
and databases awake.
> - Worker migrations already use events, transaction-aware signals, and
shared deadlines.
> - New features can still add idle scans unless contributors and code
reviewers share an explicit rule.
> - This PR makes that rule discoverable through the repository rule
loader and Greptile.
> - The rule applies to added or expanded behavior without requiring
unrelated legacy polling to be removed.
## Linked Issues or Issue Description
Refs #15625, #17000, #17094, #17106, and #17129.
**Current behavior**
The repository has module ownership and runtime dependency rules, but no
explicit review requirement against new idle polling. A new duty in a
heartbeat tick can undo the benefits of previous worker migrations.
**Proposed behavior**
Require events for changed state, transaction-aware signals for
database-produced work, and shared scheduling for real deadlines. Reject
recurring empty scans, including scans disguised as scheduler callbacks.
Review narrow polling exceptions against concrete obligations and sleep
impact.
**Reason and benefit**
Make sleep-aware design a default review expectation. Preserve delivery,
ambiguous-commit recovery, drain/resume, and external wake correctness
while preventing new permanent idle traffic. Existing unrelated polling
stays outside the requested change.
## What Changed
- Add rules/sleep-aware-background-work.mdc for server, database,
adapter, plugin, and runner background paths, including
server/src/index.ts.
- Require concrete triggers, pending-work predicates, quiet conditions,
actual deadlines, and lifecycle evidence.
- Describe narrow exceptions for active operations and external sources,
with required scope, stop conditions, and sleep limitations.
- Distinguish in-process scheduling from durable Cloud wake coverage.
- Register the matching scopes in .greptile/files.json and link the rule
from rules/README.md.
- Reuse the existing module-boundary rule for ownership instead of
duplicating provider policy.
## Verification
- Passed: JSON parsing, required rule metadata, exact parity between
frontmatter globs and Greptile scopes, and all repository links.
- Passed: 15 representative path matches, including server startup,
nested service workers, producer routes, tests, database signals,
adapters, plugins, and both runner source trees. UI, CLI, docs, and
workflow paths remain outside this rule's glob scope.
- Passed: git diff --check.
- Manually checked contrasting review cases: reject a new empty queue
interval; reject an always-rescheduled empty scan; allow an actual retry
deadline; allow a bounded active readiness probe with lifecycle cleanup;
leave untouched legacy loops alone; reject sleep approval based only on
an internal timer.
- No runtime code changed. Application builds and tests were not rerun
for this rules-only change. This validates rule structure and intended
interpretation, not future reviewer compliance or deployed sleep.
- Greptile: 5/5 on
|
||
|
DottaandPaperclip
|
39e44a9d9b |
Default new OpenCode agents to Paperclip Runner (#17120)
Default new OpenCode agents to Paperclip Runner on qualified macOS and Linux x64 targets. Preserve saved runner choices, explicit Legacy selection, and unsupported-target fallback. Pi remains a separate change. Verified the complete browser path with a real OpenRouter connection: create the new agent, assign a task, observe its response and Done status, and reload to confirm persistence. CI passes and Greptile is 5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0a5f1546ee |
feat(cases): add reusable prompt workflows (#17114)
## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.13 |
||
|
DottaandPaperclip
|
159d006bb4 |
refactor: let login flows own session completion (#17129)
## Thinking Path > - Paperclip manages AI agents and their connected accounts. > - Login flows bind verified credentials to an owned connection. > - The shared save service currently uses the provider name to choose a session ID column and completion state. > - A provider can support more than one login flow. Provider identity does not define session identity or state. > - This PR gives every flow the same internal session ID contract and lets the setup-token owner complete its own claim. > - Credential writes, session binding, and completion remain in one transaction. ## Linked Issues or Issue Description Refs #17106. This is an independent prerequisite for that sleep cleanup PR. It contains no scheduler or cleanup-worker changes. **Current behavior** The connection save service looks up Anthropic sessions by public ID and all other sessions by internal ID. It also marks every Anthropic session as stored. Local Anthropic login works because its public and internal IDs happen to match, and its caller overwrites that intermediate state. **Proposed behavior** Every caller passes an internal session ID. A small setup-token completion service resolves the public ID within its owner scope and supplies a transaction callback. That callback uses the existing conditional stored transition. Device and local login flows keep their own final states. **Reason and benefit** Adding or changing a login flow no longer requires provider branches in the shared session binding path. The service still owns membership checks, row locking, target validation, credential writes, and idempotent binding. The flow owns only public-ID resolution and its completion transition. **Breaking changes** The internal TypeScript save signature changes. All production callers are updated. Public HTTP payloads, database schema, and local session URLs stay unchanged. ## What Changed - Replace the ambiguous session ID argument with an internal session reference and an optional completion callback. - Run the callback inside the existing save transaction, only for the first successful binding. - Move public-ID resolution and the stored transition to setup-token-ai-connection.ts. - Update setup-token, device, and local callers. - Add real PostgreSQL tests for ID selection, owner and target checks, one-time completion, and rollback after a rejected completion. - Exercise local Anthropic and OpenAI HTTP completion with distinct public and internal IDs. - Document the ownership contract in AI-CONNECTIONS.md. ## Verification - Passed: pnpm -r typecheck. - Passed: pnpm exec vitest run server/src/__tests__/ai-connections.test.ts server/src/__tests__/agent-device-login-routes.test.ts server/src/__tests__/agents-claude-oauth-binding.test.ts — 233 tests. - Passed: pnpm build. - Passed: full CI, including test shards, typecheck, build, and E2E, on |
||
|
Devin Foley
|
a1e2173ceb |
feat(apps): add Ditto connection (#16472)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents reach external services through the Apps catalog. Each catalog entry is a reviewed `AppDefinition` that connects a provider's hosted MCP server to Paperclip's shared vault, grants, policies, gateway, and audit trail. > - Ditto (`dittowords.com`) is a single source of truth for product copy. Its hosted MCP server lets agents read style guide rules, search and reuse approved text, and create, edit, translate and delete text, components, variants and style guides. > - Ditto is not in the catalog. Operators must use the generic "Connect your own MCP server" flow. That flow has no branding, no guidance about Ditto's `token` API-key scheme, and no warning about permanent deletes. > - The generic risk classifier also reads several Ditto mutations as reads: `translate_*` replaces variant text, and `link_*` and `unlink_*` replace developer IDs. > - This pull request adds the Ditto definition, its artwork, the research and permission-review ledger rows, documentation, deterministic tests, and one reviewed risk-classification rule. > - The benefit is a branded, governed Ditto connection with browser sign-in, an API-key option that uses the correct header, and correct risk levels for every documented tool. ## Linked Issues or Issue Description **Problem or motivation** Product and engineering teams use Ditto to keep UI copy consistent. They want their Paperclip agents to follow the Ditto style guide and reuse approved strings when they write code. Ditto is not in the Apps catalog. Operators must paste the MCP URL into the generic remote-MCP flow. That flow gives no branding and no method guidance. It also does not tell the operator that Ditto API keys use `Authorization: token <key>`, not `Bearer`, and it does not warn that delete tools are permanent. The generic classifier also marks five Ditto mutations as reads, so the Permissions screen shows them with the wrong risk level. **Proposed solution** Add a catalog-only Ditto connection built from the connector playbook. Browser sign-in uses dynamic client registration on Ditto's Auth0 authorization server and requests the reviewed scopes `openid`, `profile`, `email` and `offline_access`. An optional method sends a customer-created workspace API key with the `token` scheme. Both methods share one `write` capability profile, so the delete and link warning shows before sign-in. Add a reviewed `ditto` rule to `classifyRisk`: `list_*`, `get_*` and `search_*` tools, `suggest_edit`, and tools that Ditto marks read-only are reads; `delete_*` tools are destructive; all other tools are writes. **Alternatives considered** A plugin was not needed because no custom UI, tables, workers, or webhooks are involved. Requesting no scopes was not chosen, because Auth0 issues a refresh token only for `offline_access`, so the connection would expire with its first access token. Ditto's separate documentation MCP server (`developer.dittowords.com/mcp`) was not added, because it only searches Ditto's public docs. **Roadmap alignment** This extends the existing self-serve remote-MCP connection catalog and does not overlap planned core work. ## What Changed - Added the `ditto` provider to `scripts/ingest-app-definitions.mjs` (category `content`, `token` API-key placement, guidance, warnings, shared capability profile, description) and regenerated `packages/shared/src/app-definitions/ditto.json` and the generated registry. - Added the Ditto row to the self-serve MCP research ledger with `dcr_or_api_key` auth and risk tier S4, because the normal tool catalog includes permanent deletes. Widened `SelfServeMcpResearchEntry.wave` to include `4`, which existing entries already use. - Added permission reviews for `ditto/mcp-oauth` (explicit scopes `openid`, `profile`, `email`, `offline_access`) and `ditto/mcp-api-key` (provider key), with evidence links. - Added a reviewed `ditto` rule to `classifyRisk` in `server/src/services/tool-access.ts`. - Added Ditto's mark (`ui/public/brands/apps/ditto.png`, the avatar of Ditto's official GitHub organization) and the brand manifest entry. - Added gallery copy for the Ditto card. - Added `doc/connections/DITTO.md` (sign-in sequence diagram, provider and Paperclip endpoints including `/api/tools/oauth/callback`, scopes, administrator setup, capabilities and policy, manifest, brand provenance, validation hook) and linked it from the connections README and the permission audit. - Tests: Ditto definition shape, store visibility and artwork, URL recognition, reviewed scopes, `token` header projection with the key kept out of connection config, risk classification of every read-looking Ditto mutation, the connect form's sign-in default, the warning before sign-in, API-key gating, and the pinned catalog counts. ## Verification - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts packages/shared/src/app-definitions-url.test.ts ui/src/pages/apps/AppsConnect.test.tsx ui/src/pages/apps/Browse.test.tsx ui/src/lib/app-brand-assets.test.ts ui/src/pages/apps/AppLogo.brand-assets.test.tsx server/src/__tests__/tool-access-service.test.ts` — 765 passed. - `node scripts/check-app-brand-assets.mjs` and `node --test scripts/app-brand-validation.test.mjs` — passed. - `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter @paperclipai/server typecheck`, `pnpm --filter @paperclipai/ui typecheck` — clean. - Manual: on a local instance, open Apps → Browse and confirm the Ditto card and icon. Open `/apps/connect?source=ditto` and confirm that sign-in is the default, that the warning shows before sign-in, and that the API-key method is under Advanced. - Live metadata probe: an unauthenticated `initialize` on `https://api.dittowords.com/v2/mcp` returns 401 with `resource_metadata`. The protected-resource and authorization-server documents return the recorded endpoints. Dynamic client registration succeeds, and the authorize endpoint redirects to Ditto's sign-in page with the reviewed scopes and the `resource` parameter. - Each of the 41 documented tool names was run through `classifyRisk` with and without the new rule. Without the rule, `translate_text_items`, `translate_library_components`, `link_figma_nodes_to_text_item`, `link_text_items_to_component` and `unlink_text_items_from_component` were reads. ## Risks - Low risk to existing providers: the catalog change is additive, and the new `classifyRisk` branch runs only for the `ditto` source template. - Ditto delete tools are permanent, and linking a text item to a component replaces its developer ID. Both methods warn the operator before connecting, and the API-key helper text tells operators to set delete and link actions to Ask first. - Ditto API keys are not scoped, reach the whole workspace, and act as the user who created them. Paperclip cannot narrow an issued key. - The permission-review ledger records live proof for both methods as not run. The lifecycle checklist in `doc/connections/DITTO.md` still needs a documented pass with a Ditto workspace, including confirmation that Auth0 issues a refresh token that `/v2/mcp` accepts. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`, 1M context) in Claude Code, with tool use (shell, file editing, web research). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
6dda8cc2b7 |
fix: preserve Stop authority and qualify complete recovery journeys (#17105)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final headcanary/v2026.1021.0-canary.12 |
||
|
Nicky LeachandPaperclip
|
42715d459f |
Support authenticated WebSocket routes and agent views in plugins (#17111)
## Thinking Path > - Paperclip manages agents and their work. > - Plugins provide optional views and connect to external runtimes. > - A live runtime view needs a binary browser connection. > - The current plugin API supports JSON routes but cannot proxy that connection. > - Agent detail tabs are declared in the plugin contract but are not mounted on agent pages. > - This PR completes those two host capabilities and keeps upstream credentials on the server. ## Linked Issues or Issue Description **Subsystem affected** Plugin scoped API routes and agent detail views. **Problem or motivation** Plugins cannot expose a binary or text stream through a browser session without exposing an upstream credential or adding a separate browser gateway. Agent-scoped detail contributions also have no mount on the agent page. **Proposed solution** Allow declared board GET routes to select a server-only WSS upstream. Run upgrades through the existing HTTP authentication pipeline and trusted Origin check. Add agent company resolution with the existing `agent:read` decision. Mount contributed detail views in agent navigation. **Alternatives considered** A separately hosted viewer needs its own browser credentials and cross-origin policy. JSON or SSE transport adds encoding and buffering to a binary protocol. **Roadmap alignment** This extends the existing plugin API and detail-tab contract. It adds no desktop provider to Core. I checked ROADMAP.md and searched related open PRs; I found no equivalent transport and agent view change. ## What Changed - Add declared WebSocket routes requiring board authentication, GET, and `http.outbound`. Reject body-based company resolution because browser upgrades cannot send a body. - Keep worker-selected upstream URLs and headers out of browser JSON responses. Strip browser cookies and Origin. Preserve binary frames and socket backpressure. - Close both sockets on disconnect or worker shutdown. Reauthorize on reconnect after a five-minute connection lifetime. Preserve HTTP idle-work accounting for the stream lifetime. - Resolve agent scope through `agent:read` and mount generic agent plugin detail views, with current agent/view breadcrumbs and browser titles. ## Verification - Final head `9d8d269040acf9dc7bb65524876406af522dacb5`: CI passes (including Runner Rust, build, and browser E2E); Greptile 5/5; no unresolved review threads; no merge conflicts. Verified 2026-10-21. - Core server TypeScript check: `node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit`. - Focused server suites: 17 passing tests (`server/src/__tests__/plugin-scoped-api-routes.test.ts` and `server/src/http/server.test.ts`), including arbitrary build-output and metrics channels with real local text and binary WebSocket round trips, disconnect cleanup, denied actors and origins, credential exclusion, agent-read denial, and cross-company 404 equivalence. The test replaces TLS with local HTTP while retaining the real upgrade and binary stream. - UI suites: 9 passing tests for independent Insights and Notes contributions, removed contributions, built-in route preservation, sidebar links, direct plugin entry and heading updates after tab navigation. These contributions do not require the WebSocket transport. - Shared validator suite: 42 passing tests, including rejecting body-based WebSocket resolution while retaining ordinary HTTP body resolution and accepting query/entity scopes. - UI token gates pass. - UI TypeScript check: `node ../scripts/storybook-catalog.mjs` then `tsc -b`. - Shared and plugin SDK builds pass. - Full Runner build reached its unrelated Rust step and stopped because cargo is unavailable. Runner TypeScript build was run separately. No live remote VM was used. ## Risks - This adds a browser ingress path for trusted installed plugins. The plugin must select its upstream from a trusted binding. HTTP requests cannot retrieve the upstream descriptor. - Membership or binding changes are rechecked on reconnect. Connections last at most five minutes, and worker shutdown is checked every five seconds. - Initial CI caught a missing cross-company 404 guard and an old test that assumed exactly one upgrade listener. Both were corrected. Greptile's body-resolution and stale-heading findings are fixed with regression coverage; latest-head CI passes and Greptile scored 5/5 with no unresolved findings. - No schema migration. Rollback restores the previous Core image; plugins using this transport must be disabled or rolled back first. ## Model Used OpenAI Codex, based on GPT-6, with reasoning and code/tool execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] Thinking path and issue description included - [x] Model described - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] No internal issues, private links, or runtime credentials included - [x] Descriptive branch name - [x] Focused tests and relevant typechecks run - [x] Tests and documentation updated - [x] Risks documented - [x] All CI gates green - [x] Greptile 5/5 with no unresolved findings - [x] I will address review findings before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
507a0d7098 |
test: verify repository delivery after real runner conversion (#17089)
## Thinking Path > - Paperclip manages agents that perform useful work. > - Repository setup and Runner selection must preserve that work context. > - A successful provider turn cannot prove configuration retention or usable delivery. > - The repaired setup chain needs independent product checks with the recorded models. > - This PR adds explicit repository and conversion journeys to the existing Product E2E harness. > - The observer checks source bytes, Git identity, saved configuration, and downloaded output. > - Development and staging qualification remain separate. ## Linked Issues or Issue Description Refs #17088 and #17047. The complete repository setup repair in #17088 is merged. No competing conversion journey was found in the open PR search. **What happened?** Existing checks do not prove that a real legacy agent retains its configured context when switched to Paperclip Runner and then delivers verified repository work. The projectless repair check also uses different default models from the recorded staging models. **Expected behavior** A configured repository task delivers correct, accessible output. A real legacy-to-native conversion preserves configuration, instructions, environment, and equivalent permission policy before doing subsequent work. A projectless task continues through setup without supervision. **Steps to reproduce** List the configured-repository, legacy-conversion, and automatic-repository-configured-models suites. Run one exact ID from the README. These are explicit paid development checks, excluded from the default selection. ## What Changed - Add four configured-repository cells and four real legacy-conversion cells for Codex gpt-6.1-sol and Claude claude-opus-5-5 on local and Daytona environments. - Seed a fresh reference only in an owned disposable Git checkout. Compare the delivered JSON with its original bytes, hash, and commit. - Switch the same real legacy agent through the public Runner selection API. Compare allowlisted configuration and instruction digests before and after conversion. - Require one committed native run with the configured provider, model, and environment, a visible registered download after reload, Done, and no pending question. - Add four projectless replay cells with the recorded staging models and the unchanged strict continuation oracle. - Use production hire instructions and existing sandbox profiles, budgets, report pipeline, and cleanup. Record API-key and per-turn lifecycle differences explicitly. - Retain the required allowlisted API snapshot after successful automatic setup. Preserve lifecycle IDs, state, ordering, and receipt references without raw provider content. Verify both admissions retain the configured provider and model. ## Verification - Product E2E support after integrating the setup repair: 3,353 passed, two existing skips; source prerequisites: 128 passed. - Product E2E typecheck passes. Discovery lists all twelve new explicit cells. - Independent oracle calibration rejects wrong configuration, instructions, provider, model, source bytes, hash, commit, downloaded output, delivery, and pending questions. - The first full support run exposed a catalog count mismatch and an unprovisioned new profile ID. Both are fixed; the successful run uses the existing trusted sandbox profile IDs without changing workflow authority. - Review found fixture prerequisite defects: optional CLI-only Codex arguments and platform qualification after Runner selection. Both are corrected; the disposable Claude baseline also excludes its unsupported optional CLI turn limit. Seven new setup checks cover probe ordering, absent qualification, immutable images, local conversion, and uncertain probe accounting. The final focused cohort passes 86 checks and E2E typecheck passes. - The first qualification-regression run also had an incorrect test assumption about the legacy timeout; the corrected red reaches the unsupported extraArgs setting. No production conversion restriction was relaxed. - A retained four-cell development campaign delivered independently verified ordinary Codex output and real Codex/Claude conversion results with retained configuration. All three attempts nevertheless failed the required evidence contract because api-state.json was missing. The flow now saves allowlisted task/run lifecycle fields; it does not export prompts, comments, interactions, configuration contents, or provider outputs. The projectless cell delivered valid bytes but used a monitor fallback and failed the unchanged continuation oracle; its driver correction belongs to #17088. All four attempts remain failures; fresh exact-candidate acceptance is pending. These API-key/per-turn checks do not establish subscription authentication, warm reuse, Slack delivery, or the complete milestone. - Four fresh local ordinary-repository and conversion cells passed at `c4e409d819cc0a703736d3f1ed3552064ca690b8`. Independent downloaded artifacts and saved configuration evidence agree. - The repaired automatic setup chain passed all six journey checks for both models, but first failed because the success path omitted api-state.json. The separate capture repair now passes the fresh two-cell [campaign](https://github.com/paperclipai/paperclip/actions/runs/39633381139) at `7a7c26f6688ec6b0e0454562a43fd48ab1a81c97`. Both required API files and their run/intent/dispatch bindings were independently checked. Earlier failures remain failures. - After merging current master, 114 workspace/admission/continuation checks across six suites pass. The provider/model and evidence privacy cohort passes 65 checks, and E2E typecheck passes. The final model-binding assertion has unit calibration; fresh actual-model acceptance of that assertion is still pending. - Exact-head CI/review and complete staging acceptance remain pending. Local development passes do not establish the 24-cell milestone. ## Risks The real provider may expose additional conversion or setup defects. Failures remain evidence and cannot be replaced by catalog registration or a later retry. The conversion uses the production API and does not inject a native result. Only synthetic data is seeded in a resolved fixture-owned directory. Configuration evidence contains digests, not raw environment values. This PR changes evaluation code, not production behavior. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>canary/v2026.1021.0-canary.11 |
||
|
|
eece07a747 |
fix: repair the complete projectless repository setup chain (#17088)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
3ce8d976a9 |
Add experimental Paperclip Notebook MCP connection (#16984)
## Thinking Path > - Paperclip connects agents to external tools through saved accounts and per-action permissions. > - Notebook exposes artifact management through an ordinary remote MCP server with OAuth. > - A private pilot needs a hidden, default-off catalog and execution gate. > - Local file publishing belongs to Notebook, which can accept bounded file bytes through ordinary MCP tools. > - Paperclip therefore needs the connector registration and generic availability checks, with no file-transfer architecture. ## Linked Issues or Issue Description **Subsystem affected** Apps catalog, experimental feature metadata, connection setup, and connector availability. **Problem or motivation** Users need to connect Notebook as their Cloud account, assign its tools to agents, and retain Paperclip's normal Allowed / Ask first / Off rules. The private pilot must remain hidden until an operator enables it. Notebook should own its upload protocol and local publishing script. **Proposed solution** Register a personal remote MCP OAuth connection at Notebook's canonical endpoint. Reuse existing OAuth discovery, credential storage, refresh, reconnect, and removal. Enforce availability through generic catalog feature metadata and saved application identity. Keep Notebook's standalone uploader in its own service repository; it calls the ordinary gateway and receives no saved OAuth credential. **Alternatives considered** A gallery-only filter permits retained tools and deep links while disabled. A shared workspace publisher and private upload-ticket transport add runtime behavior that a normal MCP connector does not require. Notebook can receive file bytes directly through its own tools instead. **Roadmap alignment** This extends Connected Apps and supports Artifacts and Work Products. ROADMAP.md and related PRs were checked. This PR is standalone on master. The separate OAuth lifecycle and file-transfer proposals (#17085 and #17086) are closed and are not dependencies. ## What Changed - Register Notebook's personal MCP OAuth method, canonical endpoint, scopes, catalog metadata, and branding. - Declare managed `enablePaperclipNotebook` with false defaults and no user-facing settings toggle or staging opt-in. - Use generic availability metadata for gallery/setup, retained connection identity, OAuth completion, health, tool/context discovery, and dispatch. Resources/prompts recheck before credentials, including retained handles. Removal remains available while disabled. - Resolve catalog application identity from the saved application key, so editable connection labels cannot bypass the hidden gate. - Select a method's sole supported credential kind when no kind is supplied. - Keep UI changes to metadata-driven availability and existing setup/reconnect controls. - Document Notebook's service-owned upload script, inline publication, ordinary action policies, and credential behavior. - Add ordinary MCP OAuth, assignment, action-policy, disabled-state, settings, and catalog tests. No workspace contribution registry, private transfer endpoint, native publisher, Runner file-reader change, or Paperclip CLI upload command is included. The entire native-runtime and CLI trees match master. Upload code lives in `paperclip-cloud/notebook/`. ## Verification Source head: `65ef741e4f476c67425b57d2fb2d6d4c00e99b06`, directly on master `53732c811bb69655eb43d3d12f2063ae4ca5ad67`. The diff is 42 files. Shared execution changes only enforce generic catalog availability; tool access also resolves saved application identity and the sole supported credential kind. Neither module contains Notebook upload logic. - Catalog/shared/settings/UI suites: 592 tests passed across 12 files. Shared/server/UI typechecks passed using installed compilers. - Fresh real PostgreSQL generic MCP suite: 84/84 passed at final source; server typecheck and module boundaries pass. Coverage includes personal OAuth, ordinary refresh/reconnect, assignments, Allowed / Ask first / Off, hidden feature enforcement, and local credential removal. Four new resources/prompts cases failed against the prior code and pass after filtering saved application identity and rechecking before credentials. They prove missing/relabelled setup markers and stale eligibility cannot dispatch or read secrets while disabled; an ordinary provider remains available with the flag off. - The full tool-access service suite passed 463/464 cases. One unchanged Google Workspace `slides.write` case timed out at 15 seconds; it passed on one isolated retry without source or timeout changes. Its exact test block matches master. The cause is not established; final-head CI must qualify the complete suite. - Maps, module boundaries, token gates, and whitespace checks passed. Byte comparisons confirm the former transfer/runtime/CLI surfaces match master. - Fresh exact-head Greptile is 5/5 with zero unresolved findings after an exhaustive feedback audit. Current-head CI workflow [39625691137](https://github.com/paperclipai/paperclip/actions/runs/39625691137) succeeded, including both required checks (`ci / verify`, `ci / e2e`), Build, and browser E2E. Supplementary security/advisory scan status is recorded in the PR Checks panel and qualification receipt; prior-head scans are not counted. The superseded `4bd7af9b2e` completed CI successfully, but Greptile found the valid resources/prompts availability gap described above. It was fixed, replied to, and resolved; earlier-head results do not qualify this revision. Preserved environment failures: the first socket-restricted UI/route run had 93 listener setup failures, and the same 93 cases passed with local-socket access. The package-manager launcher could not fetch its signature metadata; installed compilers passed without dependency or configuration changes. The full service timeout and isolated retry remain recorded separately. The superseded `585e870e73` CI also failed an unchanged agent-chat project-plan revision assertion after task creation; the complete spec and relevant runtime sources match master, and a same-master prior run passed. Its cause is not established. Logs are preserved; no test or timeout was weakened and no old-head rerun was requested. The subsequent heads run full CI independently; no prior-head pass substitutes for current qualification. The Notebook uploader requires a separate service release. This public PR adds no uploader runtime to Paperclip and claims no newly deployed browser/agent acceptance. The Notebook-owned upload path is qualified separately. Code-owner review remains required before merge. ## Risks - The feature flag gates the curated connector; it does not create a Notebook service-side team allowlist or block arbitrary standard MCP URLs. - Publishing, sharing, and deletion retain the existing assignment and action-policy boundaries. - The hidden feature must be checked for retained tools and saved identity, including when connection labels change. - Disconnect removes local credentials. It does not revoke the remote Notebook authorization; the owner can revoke it in Notebook. - Ordinary refresh uses the existing client. A lost response after refresh-token rotation can require reconnect. - The standalone Notebook script and its bounded upload tools must ship together in the Notebook service before that documented convenience path is available. - No database schema, workflow, lockfile, shared transfer, or Runner change is included. No merge, deployment, or rollout is performed by this update. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository inspection, code execution, and tool use. The runtime did not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.10 |
||
|
Nicky LeachandPaperclip
|
00331e0254 |
Connect native Runner admission to plugin-owned workspaces (#16978)
## Thinking Path > - Paperclip manages work for AI agents. > - A plugin can own an execution environment and its files. > - #17029 supplies the task driver and bounded setup verification. > - Heartbeat still prepares workspace files on Core and expects a shell transport. > - This PR gives the selected plugin ownership of workspace preparation and native task execution. > - Core keeps admission, tool authority, and recovery checks. ## Linked Issues or Issue Description **Subsystem affected** Heartbeat workspace admission and native Runner execution. **Problem or motivation** A task-only plugin cannot run through Core filesystem staging or shell execution. Native admission and recovery must use the same lease and workspace ownership. **Proposed solution** Use the task driver from #17029. Resolve workspace and project identities without Core file operations. Connect native dispatch and restart recovery to that driver. **Alternatives considered** Keep workspace admission and dispatch in one change. A workspace change alone would leave dispatch on the previous transport. An execution change alone would leave Core staging files for the wrong host. ## What Changed - Enable plugin environment selection for native workspace sessions. - Resolve workspace metadata and authorized project references without Core checkout, copy, or provision operations. - Preserve agent-default and primary workspace policies. Use the pinned workspace provider for plugin-backed project storage. Reject unsupported Core service commands and cross-owner reuse. - Connect native execution to the prepared task driver. Keep assigned MCP authority on Core. - Save exact checkpoint identity, transfer selected account refreshes, and require termination proof before replacement or cleanup. - Complete managed plugin workspaces through their storage provider. Use the same check after a restart. Keep branch validation and leave task-owned files on the plugin. - Include default instruction content in runtime synchronization when no instruction directory exists. - Defer new-agent Copilot verification until the saved-agent lifecycle can acquire the agent-owned computer. ## Verification - Rebased onto `origin/master` at `898144f4b3`. Master advanced once during verification. Both manual conflicts joined import blocks: one in `native-session-executor.ts`, and one in `heartbeat/runtime-execution.ts`. The seven patches retain their logic. The first rebased head passed repository typecheck, build, all 862 focused execution/finalizer cases, and all CI gates. The latest head `93e5a72892` passes server typecheck and all 880 focused execution, finalization, and execution-control notification tests. All 54 CI/review checks pass, with two expected skips. Greptile ran automatically and scored this head 5/5 with no unresolved findings. No new review or Apex request was made for these import-only resolutions. - The successful CI workflow initially left three stale job-check reports. Targeted policy and summary reruns cleared the reports. No source change or test fix was needed. - Repository typecheck and build pass: `pnpm -r typecheck` and `pnpm build`. - Focused native execution and restart tests pass: 795 cases. The account and harness suites pass: 159 cases. Workspace admission passes: 48 cases. The SDK task lifecycle, workspace API, runtime-hook, and instruction checks also pass. - The heartbeat integration case uses the merged workspace provider contract and reaches native dispatch with no project files on Core. The new-agent case confirms that creation leaves the agent in `preparing` until lifecycle verification. - Map validation and the Copilot source-identity check pass. No Runner implementation or source-identity bump is in this PR. - The first rebased workspace run found two stale path assertions and a recovery guard that did not recognize task-owned targets. These are fixed. The storage integration fixture also needed its persisted source binding; the production authorization check stays in place. - Two prior review findings required rejection of Core service startup and cross-owner workspace reuse. The existing admission fixtures now check both failures. The installed-account authentication finding was fixed in merged #17029. - Tests reuse the current fixtures. No new provider test matrix or live Runner process is added for these fixes. - The prior head `2db273ec12` passed CI on its first attempt and reached standard Greptile 5/5. Apex scored 4/5 and found local-folder checks during plugin completion and recovery. - The completion fix reproduces both failures. It passes all 110 cases in the runtime execution, workspace finalizer, and workspace provider integration suites. Server typecheck and map checks pass. CI passed on `62d8335b4e` on its first attempt, and standard Greptile scored 5/5. The next Apex review scored 4/5. It found that a failed plugin inspection could abort restart recovery. - The restart fix records a failed workspace receipt and uses the current run retry policy. A regression first reproduced the aborted pass. It now checks retry state, retained results, and completion of another workspace in the same pass. All 68 finalizer cases and server typecheck pass. CI passed on `7d4016323b` on its first attempt. Standard Greptile and Apex both scored 5/5. Apex left one P2 about lost failure codes. - The diagnostic fix keeps the known branch and owner mismatch codes in receipts and run recovery state. It gives repair guidance for those codes and keeps unknown plugin errors generic. All 96 affected finalizer cases and server typecheck pass. Before the rebase, required CI passed on `c2fa61c6c8` on its first attempt. Standard Greptile and Apex both scored 5/5 on that head, with no unresolved findings. The new head retains these reviewed patches. - The earlier Superagent scan on `c2fa61c6c8` was inconclusive after a pinned merge-base checkout failed or timed out. The current scan on `93e5a72892` passes. - The exact rebased head passes all 48 workspace cases and server typecheck. The first local test launch could not bind PostgreSQL inside the sandbox. The authorized local-server run passed. - The prior head reached standard Greptile 5/5. Its first CI run hit an unchanged Runner cleanup timeout and an external build-runner shutdown. Both jobs passed on the single retry. The timeout cause is not established. - A local regression check found that a retained Core workspace could also enter a plugin task without file transfer. Core now rejects this move before provider work. The existing ledger test covers both directions and confirms that the source files stay intact. - The first active standard review scored 4/5. It found that a fresh plugin retry could enter Core Git preparation. This mode now uses the existing unsupported-storage error before workspace resolution or acquisition. Both unsupported-mode cases pass, and server typecheck passes. - No live Cloud or provider validation was run. ## Risks - The prerequisite #17029 is merged. This PR targets master. - Native plugin tasks support per-turn workspace sessions. Core service commands and fresh retries that import retained files remain unsupported. A workspace storage plugin can use the modes and strategies it declares. - Core-managed credentials support Codex and direct Claude accounts. Other managed credential routes fail closed. Other harnesses can use an installed account. - A provider must preserve its prepared configuration and report process termination correctly. Uncertain termination keeps the task pending cleanup. - Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.9 |
||
|
DottaandPaperclip
|
898144f4b3 |
fix: stop idle execution-control recovery polling (#17094)
Replace idle execution-control polling with workers driven by committed writes and persisted deadlines. Preserve startup recovery, bounded retries, and shutdown safety. Stabilize the Slack setup browser regression by waiting for the restored setup screen before simulating its webhook. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
53732c811b |
fix(evals): classify latency from observation bounds (#16829)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E probes measure when acknowledgements and replies become visible. > - A poll gives a time interval for an event. > - The first positive poll can occur after a target that the event could still meet. > - This pull request compares the full interval with a strict or inclusive target. > - Reports retain the raw observations and the original verdict when the timing verdict changes. ## Linked Issues or Issue Description **What happened?** A historical acknowledgement was absent at 1901 ms and first present at 2427 ms. A report marked the 2000 ms target as failed. The interval crosses that target, so the available evidence cannot give a pass or fail verdict. **Expected behavior** Return pass when the full interval meets the target. Return fail when the full interval rules out the target. Return inconclusive for overlap, invalid evidence, or insufficient evidence. Respect strict and inclusive targets and open and closed endpoints. **Steps to reproduce** Run the deterministic timing tests. Use the acknowledgement interval `(1901, 2427] ms` with `<= 2000 ms`. Compare the late reply interval `(52965, 53334] ms` with `<= 30000 ms`. The first is inconclusive. The second fails. **Paperclip version or commit** The source includes master at `14afc12124017d4ec30d0f4bfd6892f8ce475d21`. Related PR: [#13710](https://github.com/paperclipai/paperclip/pull/13710) changed chat latency and session reuse. This change corrects probe evidence and report verdicts. ## What Changed - Add a shared interval classifier with explicit endpoint and target operators. - Start observation before dispatch. Use a monotonic clock. Bound asynchronous reads by request start and completion. - Measure the first visible response separately from message creation and final settlement. Share browser reads with an optional acknowledgement probe. - Keep raw bounds, clock resolution, message metadata, failed reads, and original verdicts in retained report evidence. - Add deterministic coverage for threshold equality, missing data, edited messages, asynchronous reads, teardown, report correction, and a finite-domain oracle. - Bound observer shutdown to one second across pending and final reads. Retain timeout counts. Ignore late completions and start no more reads after shutdown. - Document the timing contract in the existing Product E2E guide. - Keep current startup reuse evidence after the master merge. Update four stale test expectations for qualified Copilot admission. Keep the explicit suite and provider restrictions. ## Verification - Current head: `c42e87e6d3b4b347e7cb176f56819c1d0abe106d`. - Passed: `pnpm test:e2e:runner:unit` (3,287 Vitest tests and 128 native completion tests; two explicit skips). - Passed: `pnpm test:e2e:runner:typecheck`, `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, and `git diff origin/master --check`. - Passed: [exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39607140418), including both required checks (`ci / verify` and `ci / e2e`), all general and serialized test jobs, all eight browser shards, build, canary dry run, and the clean public npm-install smoke. - Greptile reviewed the complete current diff and returned [5/5 with no actionable issues](https://github.com/paperclipai/paperclip/pull/16829#issuecomment-6266704814). All review threads are resolved. - The redundant local `pnpm test:run` was stopped after the exact-head CI test jobs passed. It did not produce a complete local verdict; the full repository test evidence is the CI run above. - The [earlier broad run](https://github.com/paperclipai/paperclip/actions/runs/39249282597) failed on the old head. Its full test command timed out after 15 minutes. Five E2E support failures came from ambient `NODE_PATH` and a missing historical Git object. Current master supplies the closed-environment launcher and retained source fixture. The full support suite passes with these fixes. - After the current master merge, four stale Copilot assertions and the nested unit launcher failed. Copilot is now qualified. Those assertions now require no qualification override and retain the rejected suite/provider cases. The full documented unit command passes. - The first local typecheck after the merge used stale generated package declarations and failed. The typechecks pass after `pnpm build`. - These checks use synthetic timing evidence. They make no live provider calls and claim no new live latency measurement. ## Risks - The browser diagnostic depends on the existing message and receipt DOM markers. Controlled tests cover visibility, receipt separation, concurrent reads, and teardown. - Corrected timing verdicts can become inconclusive. The report keeps the original verdict and does not change the overall workflow grade. - This change adds no database migration or product surface. ## Model Used - OpenAI GPT-6 Codex. Model family: GPT-6. This session does not expose a more specific model ID or context window size. The agent used reasoning, shell tools, code inspection, and deterministic tests. The earlier implementation session did not retain its exact model ID. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
14afc12124 |
docs: define module ownership and shared change review (#17087)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Coding agents extend connectors and shared control-plane services. > - Provider details can leak into shared services during these changes. > - Moving code into a new file does not explain why shared behavior must change. > - This pull request adds module ownership and shared-change review rules. > - The rules help agents justify the change before implementation and review. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** Repository architecture rules and the connector authoring playbook. **What's wrong?** Subsystem rules describe some ownership boundaries. The repository lacks a general rule for deciding which behavior belongs in a feature module, what shared contracts must change, and how to review the resulting scope. An integration can therefore add provider-specific logic to shared code or bundle large platform changes into one connector PR. **Suggested fix** Add a repository-wide rule and load it in Greptile. Apply the rule during connector planning and PR preparation. Keep provider protocol behavior with its owner. Require a reason for shared changes and checks through affected consumers. Related public context: [Notebook connector PR #16984](https://github.com/paperclipai/paperclip/pull/16984). This PR changes instructions only. ## What Changed - Add a concise module-boundary rule for ownership, narrow contracts, shared controls, delivery scope, and verification. - Register the rule as repository-wide Greptile context. - Add planning and PR review steps to the connector playbook. Keep tests and docs with their behavior. ## Verification - Passed rule frontmatter, Greptile registration, referenced-file, and playbook-link checks. - Passed `git diff --check origin/master...HEAD`. - Reviewed the rule against the relevant connector failure and allowed cases: small dispatch branches, one-consumer shared capabilities, and unrelated fixes. - Fresh Greptile and Paperclip reviews both scored commit `759572fe503e1b6dfa9527f53de6c326a2c3bff6` 5/5. No actionable findings or unresolved review threads remain. - Final current-commit GitHub checks: 55 successful, 1 neutral, and 2 skipped; no pending or failed checks. Both required `ci / verify` and `ci / e2e` gates passed. - Initial CI attempt: the build and runtime boot smoke test passed. The spot runner received a shutdown signal during the clean public npm install check, canceling that step and failing the build job. [Interrupted job log](https://github.com/paperclipai/paperclip/actions/runs/39601811838/job/119228042468). Retried failed jobs once. Attempt 2 passed the complete build, runtime boot, and clean-install verification. The initial runner interruption remains recorded here. - Full local application checks are unavailable: this checkout has no workspace dependencies, and `pnpm -r typecheck` stopped during pnpm bootstrap because registry fetches failed and the requested pnpm version could not be verified. Local application tests and build were not run. GitHub checks must pass before merge. ## Risks - The rule could cause unnecessary abstractions or extra PRs if applied mechanically. It permits small dispatch branches and useful capabilities with one consumer. It rejects arbitrary file counts and unrelated restructuring. - Prose and metadata checks do not prove that future agents will follow the rule. Hosted review receives it when this branch is reviewed. ## Model Used - OpenAI Codex: `gpt-6-astra` authored the guidance; `gpt-6.1-sol` prepared delivery. Both used reasoning, filesystem tools, and GitHub tooling. Context window size was not recorded in the available model settings. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused documentation and metadata validation; application checks unavailable as stated above) - [x] I have added or updated tests where applicable (no runtime change; targeted structural checks cover the changed instructions) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c6ee96a3df |
fix: stop idle MCP event polling (#17036)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - MCP task subscriptions and Dot mailboxes notify external clients when work changes. > - Their delivery workers queried SQL every two seconds, including when no work existed. > - Those queries prevent idle databases from becoming quiet across many hosted instances. > - Existing activity, mailbox, subscription, and delivery records already preserve the work. > - This pull request uses committed-write signals and saved retry deadlines to drive delivery. > - Idle subscriptions become quiet while startup, retry, drain, and shutdown recovery remain supported. ## Linked Issues or Issue Description Related sleep migrations: #15625, #17000, #17008. Searched open PRs and issues for MCP event sleep/polling work; no duplicate found. **What happened?** Each public MCP and Dot event service started a two-second interval. It checked settings, scanned subscriptions, created delivery receipts, cleaned expired records, and attempted delivery even without changes. **Expected behavior** An empty or subscribed-but-idle service produces no recurring SQL or network traffic. Committed events trigger prompt delivery. Only unfinished work retains a retry deadline. **Steps to reproduce** Configure a public MCP origin and start the server. Observe recurring settings queries with MCP disabled, or subscription/queue queries with it enabled. No incoming task changes are required. **Paperclip version or commit** Initially inspected master at `8b4ab65b1c3d863d282752e83307943720a5c85c`; rebased onto `c2daf906d9` after master advanced. The only conflict was overlapping imports in the heartbeat queue; both were preserved. ## What Changed - Register separate MCP and Dot workers with the existing delivery coordinator. Remove both unconditional intervals. - Notify from committed activity, comment, mailbox, subscription, and experimental-settings writes. Centralize comment insertion in one small helper so normal callers do not manage recovery state. - Preserve outer transaction ownership, uncertainty reconciliation, and requests deferred during drain or standby. - Restore delivery retry/abandoned-claim deadlines on startup. Scan subscriptions in pages of 20; drain full source and delivery batches without another external event. Scope subscription refreshes to their subscription or shared Dot binding, issue activity to its task, comment discovery to its company, and mailbox/assignment changes to their binding. Retry deadlines do not rescan unchanged sources. - Run expired-record cleanup opportunistically. Preserve on-use expiry checks and the existing one-time automatic Dot challenge behavior. - Add real PostgreSQL lifecycle coverage and guarded external test database options for existing MCP/Dot integrations. Document producers and the supported writer model. ## Verification - Final head `e54fb34485921016ef988f7df9cca173a9ebdfc8`: all CI checks passed (55 successful checks/statuses, one neutral review, two intentionally skipped Storybook checks). Mergeable with no unresolved review threads. Greptile: 5/5 on this exact commit; security scan passed. - All 284 affected tests passed in one local run after rebasing: 16 real-PostgreSQL lifecycle tests, 24 coordinator/scheduler/standby tests, 84 public MCP tests, 54 Dot runner tests, and 106 interaction-service tests. - Server TypeScript passed after refreshing generated Runner/plugin SDK dependencies from master. Full workspace typecheck and build passed before the rebase; final CI repeated typecheck and build successfully on the rebased head. - Coverage includes SQL/network quiet with configuration and history, committed writes, outer commit/rollback, drain/standby, retry restoration, reenable/unsubscribe, concurrent writes, shutdown, source/delivery/subscription batch boundaries, Dot challenges and reconnects, native broker fences, comment discovery, and company-isolated scans. - CI exposed an existing lock-order test whose insert proxy missed nested transactions and waited indefinitely. The fixture now observes nested inserts, preserves the lock assertion, and fails explicitly if interception is missed. The complete 106-test interaction suite and the formerly hanging CI shard pass. The warm-standby/settings test doubles were also updated for the new coordinator/transaction boundaries. - Earlier fixture failures (required timestamps, cleanup, external database selection, async waits) were corrected. One existing OAuth first-poll timing assertion and one E2E company-cleanup race passed on single reruns; the final full CI run passed without a rerun. No OAuth or company-deletion behavior was changed. - The local full `pnpm test:run` attempt could not initialize embedded PostgreSQL after five attempts. It was stopped; affected integrations were verified with separate disposable PostgreSQL 17 databases, and final CI supplied the full-suite result. Embedded-only local skips are not counted as passes. ## Risks - Notifications use one live server/database scope. Independent SQL writers and separate live replicas do not trigger these process-local workers. Startup recovers durable state; continuous multi-writer discovery needs a notification/ingress contract. - Idle physical cleanup waits for startup or another enabled sweep. Subscription and authorization expiry remain enforced on use. - Existing delivery claims, retry bounds, signatures, and receipt deduplication remain in place. No new tables or migrations. - Host sleep-safety checks remain conservative. MCP/Dot retry deadlines are not yet covered by Paperclip Cloud's scheduled-wake contract. This proves worker quiet, not deployed host suspension or fleet savings. ## Model Used OpenAI Codex (GPT-6), with reasoning, local code execution, and tool use. The runtime does not expose a more specific model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
eaacadeba3 |
fix(recovery): let preservation owners reopen retained sources (#17053)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Recovery must save retained workspace files before ordinary execution can resume. > - The environment runtime blocks access to retained sources to prevent unsafe reuse or deletion. > - The recovery worker also used that ordinary resume path. > - The guard returned no lease, so recovery reported an invalid source binding before it could save files. > - This pull request gives the physical preservation owner a separate, source-bound resume path. > - Ordinary execution and destructive cleanup keep their existing guards. ## Linked Issues or Issue Description Refs #16965. **What happened?** A retained workspace with a workspaceRestoreRecovery marker could not recover through the real environment runtime. The runtime returned null before calling the provider. Recovery classified this as source_binding_invalid. Earlier recovery fixtures replaced the runtime facade, so they did not exercise this guard. **Expected behavior** The verified preservation owner can reopen its exact stopped allocation to save files. Other callers remain blocked. Recovery preserves newer local files and does not start another agent or repeat a completed external action. **Steps to reproduce** 1. Seed a terminal source run with a retained sandbox, a recovery action, and the workspaceRestoreRecovery lease marker. 2. Give the source a verified stop receipt. 3. Call recoverRetainedWorkspace through environmentRuntimeService. 4. Before this change, the recovery action returns to blocked with source_binding_invalid. No files are copied. **Paperclip version or commit** Reproduced on master at |
||
|
DottaandPaperclip
|
30b753f614 |
perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path > - Paperclip manages AI agents and their paid work. > - Budgets must prevent new work and stop reported overruns. > - Cost settlement also ran serial lifecycle reconciliation for every eligible company agent. > - That made request latency grow with unrelated idle agents and delayed results on stop delivery. > - This change separates atomic admission and accounting from durable asynchronous enforcement. > - Scoped live usage detects overruns without writing provisional ledger charges. > - The result is constant query scope for ordinary settlement and recoverable stop delivery. ## Linked Issues or Issue Description **What happened?** Settlement and budget checks entered company-wide lifecycle reconciliation. A company with 1,000 unrelated idle agents performed 1,001 accounting transactions for one ordinary cost evaluation. User requests could also wait for provider termination. Running usage checkpoints did not enforce reported monetary exposure. **Expected behavior** Keep the final reservation gate authoritative. Commit charges, scope blocks, and enforcement intent before returning. Deliver stop requests after commit. Read only applicable policies and execution scopes for ordinary charges. Stop long-running work when supported reported or priced usage establishes an overrun. **Steps to reproduce** 1. Create a disposable company with one company budget and idle agents. 2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated idle agents. 3. Compare query counts, company accounting transactions, and duration. 4. Use the controlled partial-usage and delayed-stop fixtures to check enforcement and request responsiveness. **Paperclip version or commit** The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from `master`. Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853, #13075. These cover the affected service charge, cancellation, and accounting paths. This change removes their shared reconciliation overhead. ## What Changed - Batch applicable policy exposure under the existing company accounting lock. Preserve exact fractional money and zero-reservation dispatch fences. - Commit a durable final budget dispatch claim before provider I/O. Retain a separate final issue/run ownership transaction through synchronous handoff, without holding the company accounting lock or awaiting provider work. Preserve native recovery lease checks and duplicate handoff fencing. Browser handoff rechecks current policy; it reuses a consumed rate slot only for the identical policy generation. - Separate ledger spend, live observed spend, and effective held capacity. Use the maximum of reservation and live observation for each unfinished operation. - Deliver budget cancellation through the existing coordinator. Revalidate generations and persist queued/running stop fences before external termination. Bound stop concurrency to four. - Apply agent holds only to the affected agent. Project budgets leave other projects running. Project real company transitions in bounded lifecycle batches. - Replace periodic broad reconciliation with durable pending work, startup repair, receipt replay, and concrete retry/window deadlines. Retain drain and idle-safety checks. - Enforce accepted cumulative usage checkpoints without provisional charges. Final accounting atomically replaces live exposure and settles reservations. - Add operator-gated budget spans, regression coverage, map updates, and a real-PostgreSQL comparison harness. No schema or first-party telemetry change. ## Verification - Full workspace `pnpm -r typecheck` passes on the final ownership fix, rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head `4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks pass. - Focused real-PostgreSQL coverage includes the 292-case rebased accounting/dispatch run, 63 browser cases, 52 run-control/cancellation cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and the exact Muse confirmation regression. Native executor unit coverage passes all 768 cases. Final ownership coverage passes: 74 adapter/use-case cases, nine public regression cases, and the additional deleted-project race. These exercise competing ownership, budget changes and charges between phases, zero/nonzero duplicate claims, rollback, recovery contention, read-only capacity rechecks, and controlled HTTP startup after commit. Another 74 pure/startup/tracing cases pass. The complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its external security scan identified a stale-owner handoff race. The final fix restores synchronous issue/run ownership fencing after the accounting transaction commits. Final head `8e84f55ce8` has fresh Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing security scan. The complete latest-head [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808) passes, including budget, browser, Runner, build, and canary jobs. The unchanged Runner close fixture passed on its sole same-head job retry. - Broad local `pnpm test:run` invocations were stopped before required source updates (rebase and review fixes). They are not claimed as full passes. Final full-suite coverage comes from the complete latest-head CI matrix; local focused coverage and the incomplete broad invocations are reported separately. - Reproduce performance with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-enforcement.ts --baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with `--reverse`. The harness uses disposable real PostgreSQL, 20 warm samples per variant/count, and records raw timings, SQL/lock counts, agent reads, unchanged agent state, and query plans. Final production-source measurement revision: `eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is committed in `4c5f4de8b1`. | Unrelated idle agents | Baseline median / p95 | New median / p95 | Baseline SQL / locks | New SQL / locks | New agent reads | | --- | ---: | ---: | ---: | ---: | ---: | | 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 | | 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 | | 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1 | 0 | - These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64. They isolate the cost-evaluation core and complete admission path, without hosted-region or provider latency. Prior-revision reversed-order controls also preserved the constant query counts; the final measurements above use the final production source. - Complete reserve → final ownership gate → immediate offline provider result is measured separately, with one responsible agent and one active operation at each unrelated-agent count. Admission p95 baseline/new: 0 agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents **24.51 / 74.05 ms**. Baseline admission was already scoped; the stronger final fence adds bounded overhead. New counts stay constant at 53 Drizzle statements, three transactions, two company-accounting locks, and five agent-reading statements bound to the responsible agent. Zero unrelated-agent reads, exactly one provider callback per sample, and no leftover runs or holds. All admission scenarios stay below 250 ms. No external provider calls are included. - The partial-usage/delayed-termination fixture passed 20/20: receipt persistence p95 13 ms, receipt return to observed durable Stop fence p95 28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database request p95 1 ms while termination was blocked. After releasing the controlled termination fixture, acknowledgement p95 was 14 ms. These are separately measured phases; fixture acknowledgement is not live-provider termination latency. - Models and provider deadlines are unchanged. No deployment, live inference, or staging acceptance is claimed. A quiet 30-minute follow-up waits for the merged revision on staging, then repeats classifier and controlled budget-stop acceptance in staging, recording hosted bookkeeping, queue delay, and actual provider acknowledgement separately. - Development failures retained: finalizer notification regressions (fixed; 27/27 affected cases passed), invalid browser admission fixture assumptions (fixed; browser coverage now passes 63/63), and first-head lifecycle/dispatch CI regressions (reproduced and repaired). A prior-head Cases E2E test hit its unchanged announcement-dismiss handler race while opening a filter; the same journey passed on `1ac9b58`. The external security review then found a stale-owner race between accounting commit and provider entry; final verification covers a competing run in that gap while retaining provider I/O outside the accounting lock. Early PostgreSQL startup attempts hit local semaphore pressure; serial reruns succeeded. The final-head Runner lane had one failure among 1,823 tests: the unchanged `timeout-attachment` fixture failed at its first process close after 6.28 seconds. The sole same-head job retry passed all 1,819 active tests (four skipped), including that fixture in 1.85 seconds. Three isolated local probes could not run correctly because the Rust fake-provider fixture binary was absent; cleanup masked the original startup failure. CI builds that fixture through its normal lane. These probes are inconclusive and are not counted as passes. ## Risks - This changes accounting, lifecycle, and cancellation ordering. Durable generations, source/sequence checks, ownership leases, and company locking remain required for correctness. - Provider stop acknowledgement can be slow or fail. Stop intent remains pending until existing termination proof succeeds; cancelled row status alone does not acknowledge delivery. - Mid-run limits depend on provider usage reports or supported price calculations. Providers that report only at completion cannot provide exact mid-call detection. - Signals are process-local wake hints for the supported single-active-writer model. Durable state and startup recovery remain authoritative. - There are no migrations, new queue tables, or cached balances. Repeat the classifier and controlled budget-stop acceptance after normal rollout. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and parallel coding agents. The host does not expose the exact backend model revision or context-window size, so those values are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dff63551fe |
fix: resume unfinished work after repository preparation (#17047)
## Thinking Path
> - Paperclip manages agents that perform work for a company.
> - Task repositories are prepared at normal run admission.
> - A running native task can stage a repository while its work remains
unfinished.
> - The controller did not offer that saved setup as a continuation.
> - The agent could add a timer or wait for a user restart.
> - This change uses the existing same-agent continuation and status
ledger.
> - The accepted explanation and proven conversation remain available
after setup.
## Linked Issues or Issue Description
Refs #17045. That prerequisite has merged. This PR targets master.
**What happened?**
A projectless native task can request a repository for the next
admission. An unfinished task has no server-selected repository
continuation. An explicit same-agent continuation also loses chat
provenance.
**Expected behavior**
An unfinished standard task can yield and resume through the existing
admission queue. Its explanation and final result retain the authorized
conversation. A task that only stages a repository can complete without
another run.
**Steps to reproduce**
1. Start a native standard task in a task directory.
2. Request an authorized repository through prepare_repository.
3. Yield with unfinished work and no explicit continuation.
4. Observe that the former driver rejects the yield, although the
pending repository is saved.
## What Changed
- Offer a saved same-agent continuation for the current task's pending
repository.
- Accept that current server continuation in the shared provider driver.
- Resolve chat provenance through applied status decisions, delivered
effects, and admitted wakes.
- Preserve accepted setup explanations without repeating an owned
question card.
- Document optional reproduction details and configured repository
discovery.
- Extend database regressions and workspace access maps.
- Preserve Copilot profile v52 and bind the shared continuation source
in v53. Include the active identity in the QA image fingerprint.
## Verification
- Five focused server suites: 100 checks pass.
- Shared continuation driver: 12 checks pass.
- Server typecheck passes.
- Map contract and whitespace checks pass.
- Existing API and driver assertions fail on the former implementation.
- Real dispatcher and queue cohort: 43 checks pass. The original
reply-loss regression fails before the dispatch receipt fix.
- Continuation driver and installation identity cohort: 62 checks pass.
- Regenerated capability inventory and contract checks pass.
- Final response precedence cohort: 109 checks pass. The stale-comment
case fails on the former implementation.
- All required CI checks, build, and both source reviews pass on
canary/v2026.1021.0-canary.6
|
||
|
|
8b873e0112 |
fix(adapter-utils): skip the materialize lock for an already-current skill (#17061)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A remote Claude ACP run stages the selected runtime skills into the sandbox. The host archives the materialized skills directory with tar and uploads it. > - Concurrent runs of one agent share that skills directory. The directory key is the selected skill set and its content. > - Each run calls `materializePaperclipSkillCopy` for each skill. The function always created and removed a sibling `<skill>.lock` directory, even when the copy was already current. > - The archive step lists the directory entries first and then gives the names to tar. When a concurrent run removed its lock between these two steps, tar failed with `Cannot stat` and the whole run failed as `adapter_failed`. > - This pull request checks the materialized fingerprint before it takes the lock. An up-to-date copy now leaves no transient entries in the shared directory. > - The benefit is that concurrent runs of one agent no longer fail each other's skill staging. ## Linked Issues or Issue Description **What happened?** A remote `claude_local` run failed before the agent started. The skill staging step ran `tar -c ... -T entries.nul` on the shared runtime-skills directory. Tar stopped with `tar: <skill>.lock: Cannot stat: No such file or directory` and exit status 2. Another run of the same agent was materializing the same skills at the same time. **Expected behavior** A run that finds its skills already materialized does not change the shared skills directory. Concurrent runs can archive that directory safely. **Steps to reproduce** 1. Materialize a skill into a skills directory. 2. Call `materializePaperclipSkillCopy` again in a loop for the same skill. 3. At the same time, call `createTarballFromDirectory` on the skills directory in a loop. 4. Before this change, most archives fail with `Cannot stat` on `<skill>.lock`. **Agent adapter(s) involved** `claude_local` (ACP, remote sandbox). The function is shared, so `grok_local` and the Codex and Gemini ACP skill paths also benefit. ## What Changed - `packages/adapter-utils/src/server-utils.ts`: `materializePaperclipSkillCopy` hashes the source and checks the materialized sentinel before it takes the lock. When the copy is current, it returns without creating the lock directory or a temporary copy. The existing check inside the lock stays, so a copy that is missing or out of date still uses the lock as before. The source is hashed once per call, the same as before. - `packages/adapter-utils/src/server-utils.test.ts`: a new test holds a live lock from another owner, then materializes an already-current skill. The call must return at once and must not change the lock or the directory. ## Verification - `cd packages/adapter-utils && npx vitest run src/server-utils.test.ts`: 132 passed. - Red/green: without the fix, the new test times out because the call waits on the live lock. With the fix, it passes. - `npx vitest run src/acpx-engine/execute.test.ts -t skill`: 15 passed. - `npx tsc --noEmit -p tsconfig.json` in `packages/adapter-utils`: passes. - Real-dependency race check (a temporary script, not committed). The script used real files and the system tar. It ran `materializePaperclipSkillCopy` for 5 skills in a loop while `createTarballFromDirectory` archived the same directory for 8 seconds: - Before the fix: 1994 of 2307 archives failed with `tar: skill-1.lock: Cannot stat: No such file or directory`. - After the fix: 0 of 2021 archives failed. - Assumption under test: the sentinel check outside the lock is safe. The sentinel is written into the temporary copy before the atomic rename, so a matching sentinel means a complete copy. The lock never protected readers: the caller already reads `SKILL.md` after the lock is released. - Remaining limit: the first materialization of a new skill set still uses the lock. A concurrent archive of the same directory is unlikely then, because a run archives the directory only after it has materialized every skill in it. ## Risks - Low risk. The change only skips the lock when the result is already correct. A missing, damaged, or out-of-date copy uses the same locked path as before. - No change to the materialized layout, the sentinel format, or the lock format. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). The exact context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>nightly/v2026.1021.0-nightly.0 canary/v2026.1021.0-canary.5 |
||
|
|
f3af9887ca |
fix(chat): keep each chat conversation in its own wake batch (#17054)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat channels let a linked user talk to an agent from Slack, GitHub and other platforms. Each external thread is a chat conversation that is bound to one issue. > - One issue can have many conversations. A GitHub pull request review makes one conversation for each inline finding thread. > - When replies arrive behind an active run, the wake queue merges them into one deferred wake. The merge did not look at the conversation. > - A reviewed chat run can attest only one conversation. A merged batch from two threads always failed attestation with `setup_failed` / `reviewed_chat_execution_binding_not_authorized`, so no reply got an answer. > - This pull request keeps each chat conversation in its own wake batch. The single-conversation attestation check stays the same. > - The benefit is that replies in different review threads each get their own attested run, one after the other. ## Linked Issues or Issue Description Refs #16793 (related: chat origin during coalescing). No public issue exists, so the bug report follows. **What happened?** A linked user replied in three different inline review-finding threads of one pull request while the agent's review run was still active. The three wakes merged into one deferred wake. The promoted run failed in about 2 seconds with `setup_failed` / `reviewed_chat_execution_binding_not_authorized` and `providerWorkStarted=false`. None of the three replies got an answer. The failed-chat retry cannot recover this batch, because a retry source holds one conversation. **Steps to reproduce** 1. Bind two chat conversations (two GitHub finding threads) to one `in_review` issue. 2. Start a run for the assigned agent on that issue. 3. As the linked user, reply once in each thread. 4. Finish the active run. 5. See one promoted run that fails with `reviewed_chat_execution_binding_not_authorized`. **Expected behavior** Each thread gets its own run. Each run attests its one conversation and answers its reply. **Cause** `findExistingDeferredWake` matched on company, agent, issue and actor only. The active-run coalesce path had the same gap. `authorizeChatConversationForBoundRun` correctly denies a batch with more than one conversation. ## What Changed - `run-dispatch/domain/wake-context.ts`: add the `chatConversationId` wake-context key, a reader, and `wakeChatConversationsConflict`. A conflict means that both wakes have a conversation and the two conversations are different. - `issue-assignment-wakeup.ts` and `chat-channels/inbound/wakeups.ts`: stamp the conversation ID on each inbound chat wake. - `chat-channels/failed-run-retry.ts`: stamp the conversation ID on failed-chat retry wakes, so later replies do not merge into a retry from another conversation. - `wake-queue` admission: do not coalesce into an active run of another conversation. The deferred-wake lookup now matches only wakes of the same conversation or wakes without a conversation. - `heartbeat/queue.ts`: the same-scope queued or running coalesce target is skipped when the conversations conflict. - `doc/execution-semantics.md`: describe per-conversation chat wake batching. - Tests: an integration regression in the GitHub review block of `chat-channels.integration.test.ts`, and unit cases for the active-run branch in `wake-queue/application/use-cases.test.ts`. ## Verification - Red before the fix: `pnpm --dir server exec vitest run src/__tests__/chat-channels.integration.test.ts -t "replies in review finding threads"`. The main case fails with one run: `{ status: "failed", errorCode: "setup_failed", error: "reviewed_chat_execution_binding_not_authorized" }`. - Green after the fix, same command, 3 of 3 pass: - Replies in two finding threads give two successive runs. Each run succeeds, has `paperclipExternalChatExecutionBound: true`, and holds only its own comment and conversation. - Two replies in the same thread still give one run with both comments. - After the linked identity is revoked, each separated run is still denied (`setup_failed`) and the adapter does not execute. - `vitest run src/modules/wake-queue src/modules/run-dispatch`: all pass, with the new cases (different conversation, same conversation, run without a conversation, non-chat wake). - Related suites pass: `heartbeat-comment-wake-batching`, `durable-chat-wakeup`, `chat-failed-run-retry-boundary`, `issue-queued-comments-routes`, `heartbeat-stale-queue-invalidation`. - Full `chat-channels.integration.test.ts`: 823 passed, 1 timed out. The timed-out test (`GitHub terminal receipt composes pinned HTTP and durable source authority (held_token)`) passes alone with and without this change. The sandbox has a 4 GB memory cap. - Not run locally: `pnpm -r typecheck` and `pnpm build`. The server `tsc` process was killed at the 4 GB sandbox memory cap. CI runs both. - Runtime dependency note: the deferred-wake lookup adds one JSONB predicate to the existing deferred-wake query. It reads the same rows. The query count does not change. The integration regression runs it against embedded Postgres. ## Risks - Low risk. No schema change and no migration. - Behavior change: chat replies from different conversations on one issue now run one after the other, not in one batch. Before this change, such a batch always failed, so no working flow becomes slower. - Wakes without a conversation (non-chat wakes, and wakes queued before this deploy) keep the old coalescing. A deferred wake from before the deploy has no conversation key, so one later reply can still merge into it. That window closes when the queue drains. - The check keeps the single-conversation attestation in `authorizeChatConversationForBoundRun`. Revoked identities are still denied. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code through the Paperclip agent harness. It used tool use (shell, file editing, test runs) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>canary/v2026.1021.0-canary.4 |
||
|
Nicky Leach
|
65f59e339a | Extend native instance secrets to project lifecycle plugins (#17034) canary/v2026.1021.0-canary.3 | ||
|
DottaandPaperclip
|
c2daf906d9 |
fix: preserve task continuation and question delivery (#17045)
## Thinking Path
> - Paperclip manages agent work and delivers its results.
> - One task can need several runs and a saved workspace.
> - A run can succeed while the task waits for its next run.
> - Chat fallback treated that success as completion and dropped useful
status.
> - Monitor runs also lost their conversation origin, and text-question
cards lost their prompt.
> - This change preserves the task outcome and exact delivery authority
across those boundaries.
## Linked Issues or Issue Description
**What happened?**
A Slack task requested a repository, yielded to a scheduled
continuation, and showed only a generic completion message. The next run
could not publish through the original conversation. A later text
question showed its title and Respond button without the question
context. Workspace recovery could commit the saved monitor result
without restoring its explanation.
**Expected behavior**
The original conversation shows the accepted continuation explanation,
receives the resumed result, and displays the full question when input
is needed. A successful provider run does not claim that unfinished work
is complete. Recovery delivers saved output once without another
provider result.
**Steps to reproduce**
Start an authorized chat task, prepare a repository for a later
admission, save a monitor wait, and let that monitor resume the task.
Repeat with workspace finalization interrupted and with a free-text
question. Inspect the originating conversation and the persisted output.
Related merged work: #16806 and #16975. The open monitor and recovery
changes found during duplicate search address different status and
liveness policies.
## What Changed
- Keep committed monitor summaries visible through the shared board and
chat response resolver. Preserve interaction ownership and comment
reuse.
- Recover a saved monitor explanation after finalization, with no
repeated provider result.
- Prove monitor publication origin from the admitted wake and matching
scheduling receipt. Match company, task, agent, requester, and schedule;
reject ambiguous sources.
- Include monitor continuations in the existing publication scan and use
the canonical task binding. Keep current transport authorization.
- Suppress completion fallback for yielded results. Explain missing
final output without claiming completion.
- Include question summaries, prompts, and help text in modal-backed
cards. Keep answered text private.
- Update execution semantics, capability maps, and the bounded
reliability plan.
## Verification
- Original source fails four composed PostgreSQL regressions: monitor
origin, yielded completion, question body, and saved-summary recovery.
The shared response resolver also fails before the repair.
- Repaired focused cohort: 148 tests pass across publication, question
privacy, scheduler, and native monitor suites. These use real disposable
PostgreSQL; they do not qualify live Slack.
- Full workspace typecheck and production build pass locally. Map checks
and whitespace checks pass.
- All CI gates pass at
canary/v2026.1021.0-canary.2
|
||
|
|
bbcf5dfe98 |
fix(heartbeat): renew legacy controller leases on a private pool (#17032)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A legacy (non-native) agent run keeps a controller lease on its `heartbeat_runs` row. The server renews the lease every 10 seconds. If no renewal succeeds in 60 seconds, the watchdog stops the run with `renewal_deadline`. > - #16359 added timings to lease-loss reports. The first staging event with these timings shows `deadlineLateMs=0`, `renewalPendingMs=50000`, and 108 renewals in about 22.5 minutes, where about 135 are expected. Thus the event loop was not blocked. One renewal UPDATE was in flight for 50 seconds, and renewals were slow for a long time before the loss. > - The agent had exited with code 0. The run was in post-execution `sync_back`. Other runs on the same host completed. No code path held a lock on the run row across slow work. A later `instruction_save` UPDATE on the same row completed while the earlier renewal was still pending, so a row-lock wait does not explain the stall. > - postgres.js 3.4.9 sends a new query to a busy connection when no connection is idle (`handler` → `go(busy.shift(), query)` in `src/index.js`). The query waits behind the statement that is in flight on that connection. The renewal had no timeout, and the `pending` flag stopped all later renewals. Thus one slow unrelated statement on the shared pool can stop a healthy run. > - This pull request moves lease renewal to a small private pool, bounds each attempt with a statement timeout, and retries a timed-out attempt until the existing lease deadline. > - The benefit is that slow application queries on the shared pool can no longer cancel a healthy legacy run. ## Linked Issues or Issue Description Refs #16359 (lease-loss timings that identified this cause) Refs #16358 (pre-dispatch lease loss becomes retryable; different path) Refs #16917 (open: keeps a clean adapter result after a lease lapsed; it does not prevent the loss) Refs #13389 (open: host sleep / late timers; different cause) **What happened?** A staging legacy `claude_local` run was cancelled with `Legacy controller lease lost (renewal_deadline)` after the agent exited with code 0. The timings show that the deadline timer ran on time, and one renewal UPDATE was pending for 50 seconds. **Expected behavior** A live controller keeps its lease. Slow unrelated database work does not cancel the run. **Steps to reproduce** 1. Create a database client with one pooled connection. 2. Start `select pg_sleep(5)` on that client. 3. Call `watchLegacyControllerLease(...).assertOwned()` with the same client. 4. Before this change, the renewal waits about 5 seconds behind `pg_sleep`. With a long enough slow statement, the lease deadline stops the run. ## What Changed - `packages/db`: Add `livenessDb(db)`. It gives each `createDb` client a private pool of at most 2 connections (`application_name=paperclip-controller-lease`). The pool is created on first use. Test doubles and transactions get the original client back. - `packages/db`: Add `closeLivenessDb(db, timeoutSeconds)`. The server calls it in `closeDatabaseClients`, so shutdown and failed startup end the private pool too. - `legacy-controller-lease.ts`: The watchdog renews in a transaction on the private pool. Each attempt sets a transaction-local `statement_timeout` of 10 seconds. A timed-out attempt (`57014`) is retried. After a successful retry, the local deadline is measured from that attempt's start, because that attempt's UPDATE wrote the fresh 60 second lease. The existing 60 second deadline, the abort race, and the `renewal_rejected` / `renewal_error` attribution do not change. `renewLegacyControllerLease` itself does not change, so the runtime-selection caller inside its own transaction is not affected. - Tests: a saturated shared pool no longer delays renewal; a renewal that times out on a locked run row retries and keeps the lease; the private pool is cached per client, has its own `application_name`, and closes; after a 45 second timed-out attempt and a successful retry, the next deadline is a full 60 seconds after the retry. The existing test doubles now model the transaction. - `docs/deploy/database.md`: Count the up to 2 extra connections per server process when you size `max_connections`. ## Verification - `cd server && npx vitest run src/services/legacy-controller-lease.test.ts`: 20 passed, on embedded Postgres. - Red/green: with the old `legacy-controller-lease.ts` and the new tests, the saturated-pool test fails with `expected 5018 to be less than 2000`. With the fix, it passes. - Red/green for the retry deadline: with the deadline measured from the first attempt, the retry-deadline test fails (`expected true to be false`) because the run is aborted 15 seconds after the retry. With the fix, it passes. - Mutation check: when the timeout retry is removed, the locked-row test fails with `canceling statement due to statement timeout`. - `npx vitest run src/__tests__/heartbeat-process-recovery.test.ts -t "lease|dispatch|controller|interaction"`: 43 passed. - `npx vitest run src/services/heartbeat/run-completion.test.ts src/__tests__/heartbeat-orphaned-active-lease-sweep.test.ts`: 53 passed. - `npx vitest run src/__tests__/server-startup-feedback-export.test.ts`: passed. - `cd packages/db && npx vitest run src/work-signals.test.ts src/client-options.test.ts src/client-teardown-registry.test.ts`: 24 passed. `npx tsc --noEmit` for `packages/db` passed. - Not run locally: the full server `tsc` and the full server suite. The sandbox has a 4 GB memory limit, and `tsc` was killed (exit 137). CI is the gate for the server typecheck, the full tests, and the build. ## Risks - Each server process can open up to 2 more Postgres connections. Deployments that are at their `max_connections` limit must allow for them. The docs now say this. - A renewal attempt is now 4 statements (BEGIN, `set_config`, UPDATE, COMMIT) instead of 1. Each statement is a short primary-key operation on a dedicated connection. - If many runs wait on locked run rows at the same time, renewals for other runs can queue on the 2 private connections. The 10 second statement timeout limits each wait, which is less than the 60 second lease. - Behavior for a real lease loss does not change. A rejected renewal and a non-timeout error still stop the run at once, and the 60 second deadline still applies. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code in agent mode with tool use (shell, file edit, and test execution). A read-only subagent helped with code search. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
e5351b240a |
Verify native Runner setup through plugin tasks (#17029)
## Thinking Path > - Paperclip manages work for AI agents. > - An environment plugin can own a Runner task. > - The merged launch-preparation contract lets the execution owner build the provider configuration. > - Core still needs a task process owner and a setup test that uses that configuration. > - This PR connects the task API to the Runner process-owner contract. > - Operators can test a plugin environment before Core sends it task work. ## Linked Issues or Issue Description **Subsystem affected** Runner, plugin SDK, and agent environment tests. **Problem or motivation** A task-only environment cannot use the shell-based setup path. Core must verify the Runner hello exchange and process cleanup through dedicated task operations. **Proposed solution** Add a lease-bound task driver and use it for a bounded setup probe. Keep heartbeat admission and task execution in #16978. This PR follows the merged launch-preparation contract in #16997. **Alternatives considered** A provider declaration alone cannot prove that Runner and the provider connect. A shell bridge would give the plugin an unnecessary command interface. ## What Changed - Pass structured Runner launch data to process owners. Support an external connection and explicit detach. - Add typed launch authentication checks and checkpoint operations. Check lease, run, project, and retained-task authority. - Connect launch preparation, PRP, termination proof, and credential handoff to the task driver. Reject local environment overrides during installed-account setup. Pass saved-account identity and credential home directly from Core. - Run a short-lived setup probe through the existing agent environment test API. Require a verified provider hello and cleanup. Keep saved Copilot account checks metadata-only through a dedicated task operation. - Update the Copilot source identity because the shared transport source changed. Keep provider behavior and the earlier identity fixture. ## Verification - CI passes on `f987adec7`: 55 successful checks and 5 expected skips. Standard Greptile and Apex are both 5/5 with no open findings. The branch merges cleanly with current master. - Repository typecheck and build pass after the rebase onto the workspace-provider contract. Server typecheck and build also pass after the final setup fix. - The 110 focused lease, SDK, setup, Copilot, and route cases pass after the rebase. All 14 setup cases pass after the final change. These include a real local Runner with a test provider. The saved-agent model-discovery case also passes. - Apex found that Core's relative workspace marker failed the driver path check, and non-Codex sessions omitted the observed model. Both failures were reproduced and fixed. The probe leaves physical workspace selection to plugin preparation. The transport reads each provider's observed model and exposes it to the driver. All 14 setup cases, 6 transport peer cases, and both existing real Runner/OpenCode cases pass. Runner build and server typecheck pass. - The coverage audit removed two redundant cases, repeated fixture code, and two extra real Runner launches. It kept authorization, credential selection, cancellation, lost replies, termination proof, checkpoint identity, and provider setup checks. - The Copilot source-identity check passes. The added-line secret and PII scan passes. The unmerged v52 attestation was regenerated after the shared transport fix. - CI on `13faeff02` caught one stale Copilot digest assertion after attestation regeneration. The literal now matches the candidate attestation, and all 49 installation cases pass. That CI lane otherwise passed 1,818 cases. - An earlier CI run hit an unchanged UI toast timing test. The separate fix is tracked in #16952. Current CI passes that shard. CI also caught a stale model-discovery assertion in this change; the assertion is fixed. - The local full-suite run was stopped when master changed. The final CI run supplies exact-head full-suite validation. No live Cloud or provider test was run. ## Risks - Native hello tests require a saved Core account or the installed account. Local environment overrides are unsupported for installed-account probes and fail before the provider runs. - A native task provider must declare launch preparation and authentication-check version 1. It must keep the checked configuration fixed through submission. Core rejects providers that cannot make this guarantee. - A lost task reply does not prove process termination. Core retains grants until the provider proves termination and Core saves required state. - This PR does not enable native heartbeat dispatch through plugin environments. Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
79b2bf78dd |
Create agent records when onboarding names are submitted (#17005)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core onboarding selects an agent and connects its provider account. > - The current flow creates the agent record after authentication. > - Setup needs a stable agent identity as soon as the user submits its name. > - This PR saves a paused setup record on name submission and finishes the hire on that same record. > - The existing sandbox authentication and approval paths continue to apply. ## Linked Issues or Issue Description **Subsystem affected** Core agent lifecycle, shared validators, and onboarding UI. **Current behavior** Onboarding creates the database record at the final hire step. Earlier authentication and saved setup have no agent record. **Proposed behavior** Create a paused agent with a setup hold when the user submits its name. Do this before adapter selection or authentication. Update that same record after adapter selection. Reuse its ID on retries and saved setup. Finish that record through the existing hire path. **Reason and benefit** Give setup a stable identity without coupling this refactor to computer provisioning or a new execution backend. **Breaking changes** Incomplete setup now leaves an inactive agent record. Existing clients can still create and hire agents without the new optional fields. The sandbox authentication flow and API key support remain in place. **Additional context** Requested Core-only refactor. Related work: #15422 changes runner defaults. This PR preserves those choices. No duplicate early-record refactor was found in the issue and PR search. ## What Changed - Save the agent when the user submits its name in both onboarding entry points. Use a provisional adapter until the user chooses one. Reopen name-only drafts at adapter selection. - Add retry-safe setup creation and same-record hire completion. Keep approval checks at final hire. - Save and restore setup configuration in both onboarding entry points. Show Finish setup for paused setup agents. - Restrict setup changes to authorized board users. Block generic resume until setup finishes. - Add lifecycle, authorization, connection, and UI coverage. Update the specification and feature maps. ## Verification - Latest focused server run: 126 lifecycle and AI-connection tests passed after the Apex fixes. An earlier broader run passed 301 lifecycle, instruction, adapter-validation, and AI-connection tests. Earlier authentication, retry, cross-company, and direct-create regressions also passed. - Name-save verification: 336 tests across the wizard, naming dialog, New agent, action-button, and wizard-step suites passed. Coverage includes retry identity, repeated submission, closing during a save, saved-model restoration, and a disabled process adapter. - Shared, server, and UI TypeScript checks passed. The latest UI package check and direct server TypeScript check passed. The server package preparation command could not finish locally because Cargo is unavailable; full build preparation runs in CI. Map checks, token gates, and `git diff --check` passed; each pushed diff was scanned for credentials and PII. - All six onboarding browser tests passed together with installed Chrome against a fresh throwaway instance after the Apex fixes. They include numbered duplicate names, first-time Copilot without a connected account, and Grok’s saved model. They verify early persistence in both entry points, closing and reopening a name-only draft, same-record adapter selection, sandbox sign-in, login reload, and failed-connection gating. Provider responses are simulated; Core persistence is real. - Regression coverage includes incomplete Copilot creation versus final validation, Kimi settings through redacted responses, stale local drafts, company switches during creation, saved environments, and provider changes with personal/shared old bindings. - Apex found five name-only setup gaps: numbered names, early Copilot account validation, the Grok model default, same-name draft ownership, and provisional runner restoration. The fixes preserve server names, defer implicit account binding until hire, set Grok’s model, avoid name-based reuse of unfinished records, and restore Auto for pending adapter choices. UI, database, and browser regressions cover these cases. - Standard Greptile found a Grok provider mismatch in the new adapter save. The save now uses ACPX with the Grok profile. Both the 129-test dialog/setup run and a real-API browser regression passed after the fix. The first browser attempt stopped on an ambiguous empty-roster button selector; it now scopes the click to the toolbar. - Moving the save point exposed a saved-model reset and timing assumptions in the same-name and portal fixtures. The source choice now preserves the restored model, and tests cover same-name recovery without a second hire. - Initial CI exposed direct-create compatibility, concurrent setup effects, fixture assumptions, and a missing-fields assumption in the provider-validation guard. These were fixed and verified locally. Review findings were addressed with authorization, recovery, credential-persistence, and restored-configuration regressions. - The final desktop capture passed after host resource cleanup. A repeat first failed before Core started because macOS had exhausted shared-memory IDs. Standalone PostgreSQL initialization reproduced the same error. Releasing only unattached control segments from this task’s stopped test databases allowed the unchanged revision to pass. No Core or system setting changed. - Local database tests required port access outside the sandbox. Missing declared development dependencies were supplied outside the branch. No live provider sign-in was tested locally. - [Full monorepo CI](https://github.com/paperclipai/paperclip/actions/runs/39522434236) is green on `82b82fec189a172a885d4dd565ae2f4a18c73efc`. Standard Greptile scored 5/5 with zero new comments (check `118969424903`). The subsequent Apex review also scored 5/5 with zero new comments (check `118972866331`) on that same commit. All review threads are resolved. - The separate Superagent Security scan returned neutral/inconclusive because its pinned merge-base checkout failed or timed out. GitHub returned 404 for check and suite rerun requests. This is not a successful security scan; the CI workflow and other security checks passed. ## Risks - Users can leave inactive setup records behind, including after name submission alone. These records cannot run until hire completes. - Setup completion touches hiring and approval boundaries. Tests cover retries, company isolation, creator permissions, and approval snapshots. - An uncertain credential-save result retains its new secret reference for retry. A confirmed client rejection removes only credentials staged by that attempt. - No schema migration or VM provisioning change is required. Agent-computer integration is out of scope. ## Model Used OpenAI Codex, based on GPT-6, with repository search, code editing, and tool execution. The exact deployment ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Apex Greptile is also 5/5 on the same commit as standard review and green CI - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d33b40a5c3 |
fix: recover saved work and provide reliable task Retry (#16965)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description **What happened?** A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. **Expected behavior** Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. **Steps to reproduce** 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6c9b87d092 |
trace(server): expose repository admission waits (#17040)
## Thinking Path > - Paperclip manages work performed by AI agents. > - The controller prepares a task workspace before it starts a provider turn. > - Repository admission also runs when a checkout already exists. > - One observed reuse took 16.877 seconds. Its child spans explained only 3.556 seconds. > - The missing detail covered repeated authority checks, catalog work, and publication waits. > - This change exposes those operations under the existing execution attempt. > - Operators can expand the slow phase before choosing a performance fix. ## Linked Issues or Issue Description Refs #16963, #16330, and #16515. The provider-routing work in #17019 has a different scope. **What existing behavior does this improve?** Repository-admission traces for new and reused checkouts. **Subsystem affected** server/ orchestration and operator-configured OpenTelemetry tracing. **Current behavior** The admission span can contain a long interval without child spans. Existing project checks, HTTP requests, and checkout lookup do not explain current-task checks, database awaits, credential refresh, or publication locks. **Proposed behavior** Expose nested spans for current binding and task access, individual identity reads, catalog construction, credentials, grant transactions, filesystem validation, Git inspection, and directory-lock acquisition, work, and release. **Reason and benefit** The trace can distinguish repeated checks, remote calls, database awaits, and directory contention. A database-await span includes driver, network, pool, and server time. It is not isolated SQL execution time. **Breaking changes** None. No API, schema, permission, retry, lock, or repository behavior changes. ## What Changed - Add fixed admission phases beneath the existing execution attempt. Keep the existing project, checkout, and HTTP detail. - Trace current-task authorization stages only under workspace access. Unrelated decisions cannot consume that detail budget before admission. - Trace GitHub catalog reads, credentials, provider metadata, grant row locking and writes, transaction settlement, and webhook work. - Separate directory-lock acquisition, the protected operation, and release without changing the adapter lock contract. - Document the hierarchy and timing limits. Keep endpoint gating, span caps, and hashed run correlation. Add no span attributes or exception text. ## Verification - Nine new tests passed with real PostgreSQL, real Git, and SDK-exported spans. They cover directory contention, grant-row blocking, all three binding checks, reuse without cloning, dirty files, private-task denial, receipt repair, conflict retry, endpoint-off behavior, and parent IDs. - Twenty earlier unrelated authorization decisions do not exhaust the admission detail budget. - Existing authorization and trust suites: 105 tests passed. Existing timing/context suites: 26 tests passed. Existing repository-access tests: 6 tests passed. - `pnpm -r typecheck` and `pnpm build` passed on commit `5a986d8c86644dfb7ee39b3b832e2d7c86532598`. - `pnpm test:run` could not complete locally. The temporary five-connection cluster returned `sorry, too many clients already` in chat fixtures, and the run was interrupted. It is not a pass. The complete standard CI workflow passed on the same commit, including all 12 server shards and all three chat shards. - Final current-head checks: 54 success, two neutral, two skipped, zero pending or failing. The [CI aggregate](https://github.com/paperclipai/paperclip/actions/runs/39522187940/job/118971261973) passed. [Greptile](https://github.com/paperclipai/paperclip/pull/17040#issuecomment-6305066520) and [Paperclip Review](https://github.com/paperclipai/paperclip/pull/17040#issuecomment-6305017182) both assessed this exact commit at 5/5 with no actionable findings or unresolved threads. - `node scripts/check-module-boundaries.mjs` and `git diff --check` passed. - This macOS host exhausted its SysV semaphore quota. Database tests used a temporary command-scoped initdb preload with five connections, fewer workers, and a larger lock table. No repository test infrastructure or shared host settings changed. Initial skipped tests, focused-fixture failures, and clock-order assertion failures were corrected and rerun; they are not counted as passes. The complete local run remained connection-limited and was interrupted. - Local SDK export proves parentage and privacy. Hosted Honeycomb ingestion remains unverified until deployment. No deployment occurred. ## Risks Span recording requires an operator-configured OTLP endpoint. Each fixed detail name remains capped at 16 occurrences per attempt. Later operations still execute under their enclosing phase. Timing wrappers preserve the existing operations and their errors. No migration is required. The change does not itself reduce admission latency. ## Model Used OpenAI Codex, based on GPT-6, with tool use, code execution, and delegated agents. The exact hosted model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.1 |
||
|
Nicky LeachandPaperclip
|
8b4ab65b1c |
Add plugin-owned project storage and task workspaces (#17001)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks can use isolated repository workspaces. > - Core currently prepares those repositories on its own filesystem. > - A storage provider may only expose repositories inside remote agent computers. > - Project storage and task workspace preparation must have one provider owner. > - This pull request adds that contract and keeps execution as a separate plugin interface. > - Core keeps authorization, durable task bindings, and lifecycle fences. ## Linked Issues or Issue Description **Subsystem affected** Plugin SDK, project workspace policy, heartbeat preparation, execution leases, and workspace lifecycle. **Problem or motivation** Core cannot prepare a task workspace when its repository storage is unavailable on the Core host. An execution lease must not be required just to prepare persistent storage. **Proposed solution** Let one plugin provide project resource lifecycle and workspace prepare, inspect, restore, and remove behavior. Core reserves a task workspace before it calls the plugin. The plugin returns an opaque durable reference. The matching execution plugin receives that reference on its task RPC. **Alternatives considered** Preparing repositories on Core would require a provider mount on Core. Preparing them only during task launch would tie persistent workspace lifecycle to execution leases and agent availability. **Roadmap alignment** This extends the remote provider model in ROADMAP.md. Related execution work is in #16978. This PR does not replace that task bridge. Native execution of managed storage requires the bridge; this change fails closed when it is absent. ## What Changed - Add one storage provider declaration with supported modes/strategies, a capability, a validated RPC contract, and a test harness. - Persist provider ownership, repository revision, and execution-workspace identity before external effects. Reconcile retries with a stable operation ID. - Prepare provider storage before environment acquisition. Keep Core filesystem preparation out of this path. - Pin the durable reference to the matching execution plugin and validate the lease binding before task dispatch. - Route inspection, reopen, and removal through the existing lifecycle process. Keep local data when remote recovery is uncertain. - Keep the application database pool free during lifecycle locks. Preserve pending workspace selections, validate source edits before retirement, and retry unfinished closed storage cleanup. - Protect provider metadata and live repository sources from caller edits. Storage edits retire only closed records with confirmed removal; retired records cannot reopen. ## Verification - Current head `8214b3ca5`: all 25 provider lifecycle tests pass with `maxConnections=1`. They cover preparation, inspection, reopening, removal, a paused provider with responsive ordinary SQL and a competing source edit, pending selection preservation, rejected source edits, cleanup retry scheduling, and providers that cache operation results, and committed cleanup notifications for storage intent and recovered state. - Final frozen-tree lifecycle run on `8214b3ca5`: all 318 tests passed in 10 files, covering workspace services, reopen, compaction, task binding, project sources, heartbeat preparation, display caching, and source locks. - Direct server `tsc --noEmit` passes. The SDK build and all 128 SDK tests passed during initial verification; later SDK changes are documentation only. - Full CI on the previous head `ec5c7209a` passed all 55 applicable checks; 2 conditional checks skipped. Standard Greptile reviewed current head `342e6d9c3` at 5/5 with no unresolved threads. All 55 applicable CI checks passed on `342e6d9c3`; 2 conditional checks skipped. The Slack browser and CLI startup jobs passed on their single unchanged-head rerun. Its Apex review returned 4/5 with one pending-restoration issue. Commit `1b58943b1` reconciles that durable intent before cleanup, advances the generation without reopening the Core row, and blocks direct removal or retirement while intent remains unresolved. Standard Greptile reviewed `1b58943b1` at 5/5 with no new findings. All 54 applicable CI checks passed on `1b58943b1`; 3 conditional checks skipped. The chat job passed on its single unchanged-head rerun. Apex then returned 4/5 for a missing cleanup wake after recovery. Commit `8214b3ca5` atomically registers cleanup intent before storage intent and recovery-result writes. Its provider test observes real committed notification hints and the visible advanced generation. Standard Greptile reviewed `8214b3ca5` at 5/5 with no new findings. All 55 applicable current-head CI checks passed on the first run; 2 conditional checks skipped. Final Apex verification completed at 5/5 on this exact head: 38 files reviewed, no new comments, and no unresolved threads. The PR is ready for review and mergeable. - The first Apex review found five issues after a standard 5/5: application-pool starvation, stale pending selections, retirement during rejected source edits, lost cleanup retries, and reused inspection identities. Commits `6d72e7fd4` and `342e6d9c3` fix each with regression coverage. Archive lock waits use dedicated connections, and source edits fail promptly if the lifecycle is busy. Retirement compares normalized values. The normalized-source regression initially failed with the raw-value comparison; after moving the check after normalization, a clean provider-suite rerun passed all 24 tests at that stage. Those threads are resolved; the earlier Apex 0/5 is not a passing acceptance result. - Earlier local integration attempts could not bind localhost inside the sandbox. The same suites passed with permission to start local PostgreSQL. - The earlier broad local `pnpm test:run` overlapped edits and reported stale-revision and checkout-path/Git-fixture failures. It was not a final-tree result. Full CI subsequently passed; the original local failure remains part of verification history. - Local repository-wide typecheck/build cannot complete because `cargo` is unavailable. Direct server TypeScript checks and affected tests pass; full Rust and build validation comes from CI. - An earlier browser run on `ff918c5a8` failed in the unchanged case-filter journey: expected “Done,” received “All statuses,” with an announcement handler recorded during the assertion. The cause is not confirmed. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39500611857/job/118896954508) remains verification history. The unchanged shard passed in the next full CI run on `ec5c7209a`; no UI or fixture change was made. - Current-head browser shard 2 failed waiting five seconds for the mocked Slack setup Success heading. This PR changes neither that UI nor its fixture. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021232) remains verification evidence. The cause is not confirmed; the single unchanged-shard rerun passed; no UI or fixture change was made, and the original cause remains unconfirmed. - The workspace test shard passed its 3,978 general tests, then its CLI company-import/export fixture timed out after 30 seconds waiting for startup health before creating any company or workspace. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021855) remains verification history. The child reached external-PostgreSQL startup without exiting or reporting an exception. Its cause is not confirmed; the single unchanged-job rerun passed, including the CLI fixture. No startup or fixture change was made, and the original cause remains unconfirmed. - The new cached-removal regression initially failed because its fake worker advertised availability for unrelated fixture plugins, so another workspace changed its fake checkout state. Worker availability now matches the fixture plugin; the complete 25-test provider suite passes. - On `1b58943b1`, the chat shard timed out polling for one conversation after an authorized GitHub description edit. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39510521657/job/118930193418) remains evidence. The exact case passes locally against PostgreSQL (1 passed, 1,380 unrelated cases skipped), and this PR changes neither that test nor its handler. The original cause is unconfirmed; the single unchanged-job CI rerun passed without a code or fixture change. The original cause remains unconfirmed. - The diff was checked for credentials, private references, and generated artifacts. ## Risks This is an additive provider contract with no database migration. Existing projects keep their local behavior unless their policy selects a storage plugin. A selected provider must be ready and enabled. Unknown or unsafe cleanup fails closed. Managed native execution depends on #16978. Core refuses a host Runner fallback. Providers must fence lifecycle generations, preserve ambiguous retries, and report remote recovery honestly. Provider-specific live storage and guest mount qualification remain separate from these Core contract tests. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and tests. The exact model variant was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass (affected suites; broad-run limits are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.0 |
||
|
Devin FoleyandPaperclip
|
572bef4ba7 |
fix: omit re-downloadable Codex remote plugins from failover backups (#17026)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native Codex sessions in remote sandboxes copy the sandbox's Codex
home into a host-side failover backup after every turn
> - Ephemeral sandboxes restore that backup at the start of every turn,
so its size affects both disk use and turn latency
> - Most of each backup is remote plugin data that Codex itself treats
as a cache and re-downloads on startup
> - This pull request leaves that data out of Codex checkpoints, keeping
everything a resumed session needs
> - The benefit is backups roughly 80–90% smaller, less data moved per
turn, and much slower disk growth
## Linked Issues or Issue Description
Bug report described in-PR (bug report template). Related: #16866
(expires failover backups after their issues close).
**What happened?**
Failover backups grow quickly because each Codex checkpoint includes
`$CODEX_HOME` remote plugin data. On one instance, 204 backups held 22.1
GB:
- `codex-home/plugins/cache/openai-curated-remote`: 8.3 GB (about 29 MB
per backup);
- `codex-home/cache/remote_plugin_catalog`: 6.1 GB (one 38 MB catalog
file per backup);
- `codex-home/sessions` (the conversation history): 0.31 GB.
About 80 new backups (9.8 GB) were written per day. Ephemeral sandboxes
also copy all of this into a fresh sandbox before each turn.
**Expected behavior**
Checkpoints keep what a resumed session needs and omit data that Codex
re-downloads itself.
**Steps to reproduce**
1. Run a Codex agent with ChatGPT auth in a remote sandbox environment.
2. Inspect
`<instance>/runtime/paperclip-runner/durable-sessions/*/failover-backups/current/codex-home`.
3. Most bytes are under `plugins/cache/openai-curated-remote` and
`cache/remote_plugin_catalog`.
**Paperclip version or commit:** `master` at
canary/v2026.1020.0-canary.15
|
||
|
DottaandPaperclip
|
f89c37791c |
Add campaign sidebar and case navigation to runner E2E reports (#17010)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E campaigns test runner behavior across agent profiles and environments. > - The retained reports contain many suites, cases, and execution previews. > - A long report makes it hard to find a case or return to the campaign context. > - This pull request adds campaign navigation, case breadcrumbs, and separate global history pages. > - Users can scan all execution previews or inspect one case without losing their place. ## Linked Issues or Issue Description **What existing behavior does this improve?** Navigation in the retained Runner E2E campaign reports and history homepage. **Subsystem affected** Test report infrastructure in `tests/runner-e2e/`. **Current behavior** Reports show a long configuration matrix. The suite index and campaign history share the same page. Case links do not provide a focused inspection view. **Proposed behavior** Global overview, history, and attention pages show global navigation. A specific campaign adds a folding suite tree. All results keeps the full scrollable report. A case opens with breadcrumbs and an execution picker. **Reason and benefit** Users can find a case, compare its executions, and return to their reading position. Global pages do not show suites from an unrelated campaign. **Breaking changes** The report HTML layout changes. Retained result schemas, execution identities, and suite and execution anchors remain unchanged. **Additional context** Refs: #16996. This extends the merged report navigation and local preview work. The duplicate PR and issue search found no open PR for this change. ## What Changed - Add shared report navigation with the Paperclip brand mark and wordmark. - Scope the suite tree to the current campaign. Keep global overview, history, and attention pages separate. - Group all execution previews by suite and case. Reuse the existing evidence cards and image gallery. - Add focused case inspection, configuration selection, breadcrumbs, filter handling, and scroll restoration. - Move historical trends to the global history page. Keep billing details, missing results, and failure evidence available. - Update report documentation and browser regressions for filters, status dots, configuration reloads, breadcrumbs, and scroll restoration. ## Verification - [Final CI](https://github.com/paperclipai/paperclip/actions/runs/39500401411) passes on `89a8b5c290de3cfb9bdc061be3c88a29fd160a62`. Typecheck, build, runner checks, server suites, and all browser shards pass. - All four report browser regressions pass. They cover filters, gallery navigation, repeated suite jumps, the folding sidebar, breadcrumbs, scroll restoration, configuration selection and reload, mobile navigation, and aggregate case status. - The 63 report, history, preview-boundary, and navigation tests pass. Runner E2E typecheck passes with `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Manual browser checks covered desktop and a 390-pixel viewport. Retained campaigns covered 209 executions in eight suites, 179 executions in seven suites, and 15 executions in one suite. Browser fixture checks also confirmed filtered inspection and aggregate status. No paid evaluations ran. - The initial report browser tests still targeted the old layout and routes. They were updated for the new navigation. Review also exposed filtered-inspection and aggregate-status defects; both were fixed before the final passing browser run. - The unchanged Cases UI grouping test [failed once on the final head](https://github.com/paperclipai/paperclip/actions/runs/39500401411/job/118898559197). Its full file also failed locally, while the isolated test passed. The same UI code passed in earlier CI. A [single CI retry](https://github.com/paperclipai/paperclip/actions/runs/39500401411/job/118903333430) passed all 3,978 UI tests and 286 CLI tests without code changes. The cause remains unresolved. - Two jobs on an intermediate head were interrupted by runner shutdown signals. Those remain interrupted results. The complete final-head CI run supersedes them for merge verification. - Workspace `pnpm -r typecheck` and `pnpm build` also passed locally before the rebase. The full local `pnpm test:run` attempt encountered repeated embedded PostgreSQL startup failures and was stopped. It is not a passing local full-suite result; final CI provides the full verification. ## Risks - Navigation and hash handling can select the wrong case or hide results. Focused tests and browser checks cover these paths. - Historical campaign links must resolve from both the preview and published report locations. - The report layout changes. Graders, prompts, retained outcomes, and billing calculations do not change. ## Model Used - OpenAI Codex with `gpt-6-astra` and `gpt-6.1-sol`. Used reasoning, repository tools, code execution, and browser verification. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.14 |
||
|
Nicky LeachandPaperclip
|
2a5a323e57 |
Add execution-owned Runner launch preparation (#16997)
## Thinking Path > - Paperclip manages work for AI agents. > - Runner can execute in a different environment from Core. > - Core owns provider selection, policy, and resource assignments. > - The execution environment must prepare its own files and paths. > - A shared contract must keep these responsibilities separate. > - This PR adds that contract to the Runner execution driver. ## Linked Issues or Issue Description **Problem or motivation** Runner transport prepares provider files on the controller before launch. An execution owner cannot prepare those files in its own environment through the existing driver. **Proposed solution** Add `prepareLaunch` to the execution driver. Supply versioned Core requirements. Accept a configuration ID, requirements digest, and ordinary provider descriptor. Validate Core policy and assignments before launch. Keep existing provider session checks. **Alternatives considered** Provider-specific ownership flags would spread the boundary across backends. Shell preparation would expose process details in the plugin API. Refs #16993, #16977. Both prerequisites are merged. This PR now targets master. Native plugin integration and execution-host preparation follow separately. PR #16978 contains an earlier, unmerged prototype. Its `persistentAgentHome` mode and skill path-map call will be replaced by this preparation contract before activation. This PR preserves current master callers of the skill helper. ## What Changed - Add the optional preparation callback and an explicit Runner artifact identity. - Keep the built-in preparation path as the default. - Validate provider policy and resource assignment identity. Preserve instruction text when resource locations change. - Persist the attachment seed after an authority epoch rotation, so a later controller restart can recover the same run. - Preserve explicit context removal on a new run. Only an omitted context can reuse the saved context. Remove its registered instruction framing when context is removed. - Save exact requirements and configurations in a private file outside Runner commands. Retain the active record across failed attachments. Use current grants for new runs. Keep incompatible session changes rejected. - Join delayed preparation during startup and warm attachment cleanup. Retain an unsettled checkpoint after a timeout. Prevent work after cancellation. - Use the supplied Runner artifact for interrupted handoff recovery. Restore the exact target configuration at each handoff crash boundary. Check the Runner state when the final ACK may be lost. - Bind the changed shared sources to Copilot profile v51. Preserve the v50 receipt and current integrity checks. ## Verification - Workspace typecheck passed. - All 487 affected Runner transport, control-plane, preparation store, builder, policy, and profile tests passed on commit `eaf8ae8d1`. - Workspace build and typecheck passed again on commit `eaf8ae8d12c7ef2d22d226e4b3f05fdbf79ef4cb`. - Both warm attachment cleanup regressions fail before the fix. They pass after the fix. The tests cover delayed preparation, timeout retention, and concurrent attachment rejection. - The routed handoff regression fails with a missing controller binary before the fix. It passes after the fix. The execution owner launches the real fixture Runner. - The 200,000-byte instruction regression fails before the fix and passes after it. Four prepared handoff crash cases verify exact request and configuration recovery. The store tests verify private file mode, conflicting records, corrupt data, and active-run retention. - A real Runner with a controlled fake provider verifies execution-host paths, exact recovery, warm reuse, incompatible profile rejection, and cancellation. It does not contact an external provider. - The regression found undefined settings in strict PRP command hashing. Requirements now contain only JSON values. The warm transition test passes without changing PRP validation. - The broader root suite on the prerequisite finished with 22,420 passing tests, seven failed tests in unchanged Git and diagnostic code, and two suite setup errors because Cargo was absent from its PATH. This PR does not claim a passing full root suite. - The CI Build job passed compilation and production runtime boot, then its runner received a shutdown signal during the clean public install check. Other jobs reported lost communication with their self-hosted runners. The failed jobs passed on retry on the same commit; successful jobs kept their results. - The current Superagent security scan is inconclusive because its pinned merge-base checkout failed or timed out. GitHub rejected the check rerun request through the available API. This is not a passing security scan. - Standard Greptile and Greptile Apex each returned 5/5 on `eaf8ae8d1`. All review threads are resolved. CI passed on attempt 2. Current checks have 55 successes, four optional skips, and the inconclusive security scan described above. The branch is ready, mergeable, and current with master. - No live provider qualification or deployment was run. ## Risks - A preparation owner must store configurations and keep retries bound to the same task and requirements. - Copilot source binding requires a new profile identity. This PR does not claim fresh live qualification. - The callback is an SDK extension. This PR does not connect it to a plugin task process owner or add a Launcher endpoint. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
15176963b1 |
fix: stop idle skill-source polling (#17008)
## Thinking Path > - Paperclip manages agent work and the skills that agents use. > - Project repositories create durable import jobs, and installed sources can refresh automatically. > - The server queried these records every five seconds, even when no work existed. > - Idle tenant databases need to stop receiving those discovery queries. > - This change uses the existing transaction notifications and delivery coordinator. > - Saved refresh, retry, and lease deadlines determine when the worker runs next. > - Cloud does not yet wake these deadlines, so outstanding skill work retains the instance. ## Linked Issues or Issue Description **What existing behavior does this improve?** Skill-source background synchronization. This continues the idle-work migrations in #15625 and #17000. A search found no overlapping open polling-removal PR. **Current behavior** On master `b11010b0d4`, startup installs a five-second skill-source interval and also calls it from the external-object scheduler. Each sweep queries active companies, import capacity, pending imports, and automatic refreshes. **Proposed behavior** Run at startup, after a relevant transaction settles, or at a saved deadline. Stay SQL-quiet when no eligible work remains. Preserve import admission, authorization, atomic claims, retry backoff, and the existing five-minute upstream check. **Reason and benefit** Remove recurring database activity from inactive tenant instances without requiring new tables or a new scheduler. ## What Changed - Replace both recurring skill-sync calls with one delivery-coordinator registration. - Notify from attachment/detachment, source creation/settings/manual refresh/disconnection, project/company deletion, and company status changes, including budget resume. - Restore the earliest eligible `nextSyncAt` or lease-expiry deadline. Due backlogs and conflicts retry after five seconds; no idle recovery cadence remains. - Admit deferred imports again after a batch frees capacity. Preserve requests through drain, standby, restart, and commits during a sweep. - Add sleep-safety predicates for imports and eligible automatic sources until Cloud supports these deadlines. - Document the lifecycle and process-local writer contract. No schema or API changes. - Guard external test database URLs before migrations or cleanup: require a `_test` database name and reject database/options overrides. ## Verification - 147 targeted tests pass: 39 server-startup tests, 14 new real-PostgreSQL lifecycle tests, 20 shared coordinator/scheduler tests, 63 existing import/refresh/retry regression tests, and 11 test-database guard cases. - CI exposed a stale startup mock without the new `start` method. The fixture is updated and all 39 startup tests pass locally. - Tests exercise real enqueue/settings/company-reactivation producers, nested commit, rollback, saved retry/lease restart, manual refresh failure, capacity admission, concurrent commit, shutdown cancellation, and SQL quiet after startup or pause. - Sleep-safety tests reject sleep for future imports and automatic refreshes, and accept inactive retained state. - `pnpm -r typecheck` and `pnpm build` pass. - `pnpm test:run` was attempted and stopped after repeated embedded-PostgreSQL startup failures in unrelated suites. The two existing import/refresh suites initially skipped for the same host limitation; after adding explicit disposable-database options, all 63 regression tests passed against PostgreSQL 17. Full CI passed on `ddb3ee51369baa8c7153bae9fd88efb8b903f2bc`: 55 successful checks, one neutral review check, and two intentional skips. ## Risks - Notifications are process-local. Independent SQL writers and separate live replicas do not wake this worker; startup reconstructs persisted work. - Enabled automatic sources still contact GitHub every five minutes. Their upstream changes have no event ingress here. - Cloud scheduled wakes currently cover routines and issue monitors, not skill sync. This change retains instances with outstanding skill obligations. It does not claim measured host/database suspension or fleet savings. - No change to source authorization, installed content on failure, or existing claim semantics. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The host does not expose an exact model revision or context-window size to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.13 |
||
|
DottaandPaperclip
|
b11010b0d4 |
Simplify failure banners and show only the latest error (#16982)
Remove dashed failure dividers and show only the latest adjacent run or recovery error. Preserve permitted retries, pending state and unrelated notice boundaries. Add production-component Storybook examples and browser regression coverage. Co-Authored-By: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.12 |
||
|
DottaandPaperclip
|
cb2d6e80d3 |
perf: stop polling idle Muse maintenance (#17000)
## Thinking Path > - Paperclip manages AI agents and their work. > - Muse connects an external worker to a Paperclip agent. > - Its maintenance loop scans the database every five seconds, even without bindings. > - These empty scans prevent an otherwise idle database from sleeping. > - The shared delivery coordinator can react to committed writes and saved deadlines. > - This change uses that coordinator and retains recovery for unfinished assignments. > - Idle Muse maintenance can now stop querying the database. ## Linked Issues or Issue Description **What existing behavior does this improve?** Muse qualification, OAuth authority, and terminal assignment maintenance. **Current behavior** A claimed instance runs two empty scans every five seconds even when Muse has no bindings. This produces about 34,560 unnecessary queries per day. Warm standby suppression from #16974 does not cover claimed instances. **Proposed behavior** Recover once at startup. Wake after relevant transactions settle. Schedule qualification expiry at its saved deadline. Retain five-second recovery only for failures and unfinished assignments. **Reason and benefit** Remove recurring database traffic from idle Muse maintenance while preserving authority checks and recovery. Reuse the database work signal foundation from #15625 without adding tables. ## What Changed - Register Muse maintenance with the shared delivery coordinator and its idle admission rules. - Signal pairing, OAuth attachment and revocation, qualification changes, and new assignments before transaction writes. The existing transaction layer waits for outer commit. - Restore qualification deadlines from persisted bindings. Retry failed sweeps and unfinished assignments. - Add PostgreSQL tests for idle quiet, nested commit, rollback, deadlines, standby, drain, and failure recovery. Verify signals from actual Muse producers. Preserve startup, newly committed work, and failed sweep retries across idle drain without SQL polling. - Document the remaining recovery cadence and sleep limitations. ## Verification - `pnpm -r typecheck`: passed before rebase. Server typechecking also passed after rebase and the drain fix. - `pnpm build`: passed before rebase. - Final change: 33 maintenance, delivery coordinator, and scheduler tests passed. Maintenance used a fresh disposable PostgreSQL 17 database. - Muse control-plane tests: 254 passed against the same disposable database. - `pnpm test:run`: attempted, then stopped after repeated embedded PostgreSQL startup failures from this host's semaphore limit. Native-session, Muse, and AI-connection suites could not create local clusters. The complete suite runs in CI. - `git diff --check`: passed. - Final commit `e106cbf20d`: all CI gates passed, including complete test suites, E2E, build, production bundle boot, public-install smoke test, and canary dry run. Greptile and Paperclip source reviews both rated 5/5, with no unresolved findings. The focused suites accept `PAPERCLIP_MUSE_MAINTENANCE_TEST_DATABASE_URL` and `PAPERCLIP_MUSE_CONTROL_TEST_DATABASE_URL`. Use a disposable test database; fixtures remove test data. ## Risks - Unfinished assignments retain five-second recovery. This change removes idle polling, not all Muse polling. - Signals are process-local. Independent database writers need explicit integration. Startup recovers persisted state. - Qualification deadlines depend on the running process. Existing durable Muse sleep blockers remain unchanged. - The receiver keeps its batching timer. An empty buffer already makes no database calls. - No schema migration or API change. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact model version is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run has the environment limitation above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2b16262280 |
Improve runner E2E report navigation and local previews (#16996)
## Thinking Path > - Paperclip is the open source app that people use to manage AI agents for work. > - Product E2E campaigns check user journeys across agent profiles and environments. > - Readers use the campaign history and reports to inspect results and retained evidence. > - The reports showed image summaries, many unselected suites, and a long sticky index. > - Individual case cards also showed routine diagnostics before the visual evidence. > - This pull request adds text summaries, feature filters, folding case indexes, and simpler cards. > - A local preview uses retained public data so designers can change reports without another paid campaign. ## Linked Issues or Issue Description **Current behavior** The report is hard to scan as the journey catalog grows. Suite links can point into hidden results. Case indexes do not show whether a case passed across its configurations. Design changes also need a realistic local preview. **Proposed behavior** Show the selected campaign as text. Link journeys to current feature-map capabilities. Add folding case indexes with status indicators, reliable section links, and a back-to-top control. Keep failed outcomes visible and put routine diagnostics under Test details. Render retained public campaign data with local templates. **Reason and benefit** Readers can find a feature or failed case faster. Designers can iterate against real retained data without starting an application instance or using provider credentials. **Breaking changes** No result schema or grading change. The report stops embedding the generated summary PNG. The publisher still produces that artifact for existing consumers. Related work: #12889 added the existing matrix browser. #12846 concerns the public image publication boundary. This PR changes report presentation and local preview behavior. ## What Changed - Replace image summaries with shared HTML suite summaries on the history and campaign pages. - Show selected executions and preserve missing results for selected cells. - Add explicit journey-to-capability associations, feature filters, and a coverage view with unmapped journeys. - Add folding case indexes, section case lists, and red, green, or amber status indicators across matching configurations. - Fix anchor navigation through filters and add a floating back-to-top control. - Simplify case cards. Keep failure reasons visible. Fold routine checks, billing, and context into Test details. - Add an original-image link to the gallery. - Add a local preview utility with public data caching and offline mode. Document the direct Node command. - Test manifest-listed evidence, rejection of unlisted files and encoded traversal, cache reuse, and offline misses with a real loopback server and a stub downloader. ## Verification - Focused report, history, and preview tests: 75 passed in 6 files. - Report Playwright spec: 4 passed in Chrome with a standalone config that does not start the application server. - `node scripts/check-maps.mjs`: passed. - `git diff --check`: passed. - Manually checked retained campaign reports, feature filters, folding indexes, failure links, card details, gallery links, mobile layout, and keyboard navigation. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm test:run`: stopped after 15 native session database cases failed. The local Postgres package first lacked dynamic-library links. After link hydration, the same cases still failed. A direct `initdb` check then reported `could not create semaphores: No space left on device` and identified the macOS semaphore limit. The complete local suite has not passed. - `pnpm test:e2e:runner:unit`: 3,250 passed, 6 failed, 2 skipped. All six failures reproduced against unchanged `origin/master` report sources with the same installed dependencies. They concern Copilot admission and its Daytona image input, plus the launcher test that runs those assertions. No report or history test failed. - No live provider evaluation was run for this presentation change. - CI on `b1a9c06823` first failed browser shard 3/8. The agent-chat split-handoff test reached cleanup, where cancelling a run returned HTTP 500 from a Postgres deadlock. Cleanup did not restore the experiment setting, and a later test failed its initial setting check. This shard does not contain the report spec. One targeted rerun passed, and that full workflow then passed. This shows recovery; no production fix was made for the deadlock. The later preview-test commit requires its own fresh checks. To review the design, run `node --watch --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/preview.ts`. Open the history address printed by the command, then open a campaign. The README covers offline mode and cache refresh. ## Risks - Feature associations use the current map. They describe linked scenarios, not proof that a whole capability is covered. Historical grades remain unchanged. - New or unfamiliar cases remain unmapped until their assertions are inspected. - The preview needs the public history host on first access. Offline mode requires cached files. It binds to the loopback interface and serves manifest-listed public evidence. - Hiding routine diagnostics adds one disclosure step. Failed or incomplete outcomes remain visible. This is report polish. It does not duplicate planned core work in `ROADMAP.md`. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, code execution, and browser automation. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4145b5048a |
docs: deepen task, approval, secret, and recovery maps (#16994)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its feature map describes user journeys. Its access map describes the matching agent paths. > - The recent index and capability additions improved discovery and breadth. > - Broad baseline records still combined operations with different actors, permissions, and retry behavior. > - This pull request reviews the existing task, approval, secret, and recovery capabilities at operation level. > - Each operation now has its own access boundary, source evidence, and verification recipe. Source review remains separate from live proof. ## Linked Issues or Issue Description Refs: #16987, #16990, #16025. These related pull requests are merged. The search found no open duplicate or matching issue. **Issue type** Missing documentation. **Where is the issue?** The paired task, questions-and-approvals, secrets, and recovery files under `maps/feature-map/` and `maps/agent-access-map/`. **What's wrong?** Baseline records do not show enough detail to verify who can perform each action, which state must persist, or what happens after a denial, race, or retry. A declared scenario tool can also be mistaken for an admitted production tool. **Suggested fix** Split the four areas into source-reviewed operation records. Add compact checkpoints to their user recipes. Preserve existing capability IDs and the map schema. ## What Changed - Promote 27 existing capability records to `operation-reviewed`, with 125 operation records: tasks 32, questions and approvals 39, secrets 28, recovery 26. - Give each operation its own API, agent surface, policy, source evidence, verification recipe, and remaining gap. - Separate human, legacy agent, run JWT, native binding, mode, and resource authority. Split operations where those rules differ. - Correct production completion paths, plan acceptance and handoff, question dismissal and withdrawal, secret proposal authority, runtime recovery grants, and explicit duplicate-risk decisions. - Preserve access gaps found in source, including the binding-proposal dependency cascade and narrower native/CLI fields. - Add one checkpoint table per product recipe for saved results, denial, races, and retries. - Keep all 191 existing capability IDs. Change only eight Markdown files. Keep every operation's evidence `source-only` and verification `not-run`; the subset below does not prove an entire operation across its actors and surfaces. ## Verification Source review uses revision `3bbc2104b65caf9f7763938a01260a5673e12f7a`. Read-only area reviews and a second source review covered the changed pairs. Passed: - `node scripts/check-maps.mjs`: 35 paired areas, 191 capability IDs, valid references and fields. This is structural verification. - `node --test scripts/check-maps.test.mjs scripts/storybook-feature-index.test.mjs`: 17 passed. - `node scripts/storybook-catalog.mjs`: 308 files, 2,899 stories. Generated outputs remain ignored. - Capability comparison with the base: all 191 IDs unchanged; the four reviewed areas contain 27 capabilities and 125 source-only operation records. - `pnpm -r typecheck` and `pnpm build`. - `git diff --check`. - Focused existing interaction and proposal route tests: 80 passed. The two selected DB-backed suites skipped 119 tests because local embedded PostgreSQL could not start. These skipped tests are not passes. Live subset on a fresh local instance with a disposable PostgreSQL container: - Created the company and an unassigned task through the browser. Added a comment and completed the task. The saved status and comment survived reload. - Created a managed secret with inert dummy data through the browser. Its metadata and version survived reload. The metadata response did not contain the dummy value. - Created a question fixture through board REST. Invalid input returned 422 and left it pending. Browser dismissal kept it pending and preserved the draft. Reopening restored the choice. Submission survived reload. A duplicate response returned 409 and left the saved answer unchanged. - Used a real issued legacy agent API key. Task read and safe company-secret catalog returned 200. Full secret listing and run-only secret list/value routes returned 403. Revoked the key after the checks. - Read an empty recovery projection. Resolving a nonexistent action returned 404 and left the task unchanged. This did not execute recovery. - Closed the owned browser tab and stopped only the verified test runtime and labelled database container. Evidence survived cleanup. No provider or model ran. Limits and retained failures: - Native embedded PostgreSQL launch failed because the host could not allocate semaphores. The container-backed app was the successful alternate environment. - Full `pnpm test:run` was attempted. Fifteen DB-dependent native-session cases failed at database startup, and several DB suites skipped. The run was stopped after repeated failures of that prerequisite. The full local suite did not pass. - The sandboxed build attempt failed at the local tsx IPC socket. The required build passed outside the sandbox. - The first CI build job passed compilation and production boot, then its npm-install check was interrupted when the runner received a shutdown signal. The corrected head passed the full CI run, including the build and public-install checks. - Initial Greptile review found the recovery note field and native search/history limits were inaccurate. Corrected `resolutionNote` and the production cap of 100, then reran the map checks. Greptile reviewed the corrected head at 5/5; both earlier threads are resolved. - An initial harness used a nonexistent generic secret-value URL and received 404. The actual mapped agent routes were then checked separately. An initial recovery assertion expected 409; the actual missing-action contract returned 404. These were harness corrections, not product fixes. - Native run bindings, two-person ownership, secret delivery/rotation, external vaults, provider-stop proof, workspace repair, and actual recovery execution remain unexecuted. The map does not claim those journeys passed. The separate external **Paperclip Review** check is still in progress on `904509df80cc6de16d36906a410a66786f619b20`. The reviewer reports that its full changed-file request is pending. CI is green and Greptile is 5/5, but this external review has not published a final assessment. Its review service must finish that request and assessment. ## Risks Documentation can become stale as policy and runtime bindings change. Each operation cites source and retains its verification gaps. The records describe observed enforcement; they do not authorize broader access or certify live behavior. This pull request changes no product code, schema, or permission enforcement. ## Model Used OpenAI Codex, `gpt-6.1-sol`; 475,000-token context window; xhigh reasoning; tool use and code execution. Read-only source agents assisted the area reviews. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused map and route checks; full-suite host blocker recorded above) - [x] I have added or updated tests where applicable (existing map regression tests cover this documentation-only change) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.11 |
||
|
Devin FoleyandPaperclip
|
a2d26a83fe |
fix(inbox): keep automatic work out of Mine by default
Keep channel and routine attribution out of Mine unless the person participates directly, is assigned the task, or receives a current addressed request. Apply the same source rules to unread state, archives, and task notifications. Preserve direct comments and current human action requests. Restore the two expressions damaged by redaction during the earlier master merge. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
dc6dd23ea4 |
Add plugin Runner preparation task contract (#16993)
## Thinking Path > - Paperclip manages work for AI agents. > - Environment plugins can own Runner tasks. > - Core selects provider policy and runtime assignments. > - The execution environment must resolve its own paths and configuration. > - The task API needs a preparation exchange before execution. > - This PR adds that exchange under the existing lease authority. ## Linked Issues or Issue Description **Problem or motivation** The environment task API can start a Runner. It cannot return a prepared provider configuration from Core requirements. **Proposed solution** Add a versioned `prepare` request and `prepared` result. Keep the provider configuration bound to its task and requirements. Check the same company, run, project, and retained-session authority as submission. **Alternatives considered** A shell preparation command would expose execution details in the API. Provider-specific ownership flags would spread this contract across backends. **Roadmap alignment** This extends the existing plugin and remote execution APIs. It does not add a separate execution system. Refs #16977, #15406. The prerequisite #16977 is merged. This PR now targets master. Runner transport and provider implementations follow separately. ## What Changed - Add strict launch requirements and prepared result schemas. - Add the preparation capability version and saved configuration ID. - Apply existing lease and resource checks to preparation. - Document idempotency, cancellation, recovery, and cleanup requirements. ## Verification - `pnpm -r typecheck` passed. - Plugin SDK build passed. - All 55 Dot Runner and native Codex server integration tests passed with Cargo available on PATH. - All 18 environment task runtime tests passed. These include preparation dispatch, inactive leases, wrong run identity, unauthorized projects, result identity, and unknown command rejection. - The broader root suite on the parent refactor finished with 22,420 passing tests, seven failed tests in unchanged Git and diagnostic code, and two suite setup errors because Cargo was absent from its PATH. This PR does not claim a passing full root suite. - No live provider calls or deployment were run. - Standard Greptile and Apex each scored commit `d3ac15906240b7066f5ff505ed2ffd9ef5dd43c1` 5/5. There are no unresolved review threads. - All 55 CI checks passed. Two optional checks were skipped. The branch is ready and mergeable against master. - A fresh master fetch and merge-base check passed immediately before the Apex request. ## Risks - This defines a new optional worker operation. Existing providers keep their current operations. - A provider must validate policy and resource bindings. Schema validation alone does not validate a concrete provider descriptor. - This PR defines the exchange. It does not connect native execution to a preparation provider. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
05f7079ecb |
Extract Runner provider launch preparation (#16977)
## Thinking Path > - Paperclip manages work for AI agents. > - Runner starts providers in local and remote environments. > - Core selects the provider and task policy. > - The execution environment supplies paths and installed files. > - The transport currently combines these inputs in one large method. > - This PR extracts provider descriptor construction before a separate plugin preparation change. ## Linked Issues or Issue Description **What existing behavior does this improve?** Runner provider launch preparation. **Current behavior** The PRP transport constructs each provider descriptor inside startup. This couples provider configuration to transport lifecycle code. **Proposed behavior** The transport calls one built-in descriptor builder. The builder takes Core provider requirements and execution-environment locations as separate inputs. File staging, provider validation logic, and session rules keep their current behavior. The required Copilot source-identity update changes profile admission metadata. **Reason and benefit** This gives launch preparation a separate unit of code. A later PR can add plugin-owned preparation without adding ownership flags to each provider backend. Refs #15406, #16486. This replaces the earlier installed-home implementation in this PR. #16978 remains a draft. ## What Changed - Extract provider descriptor construction and its existing permission helpers. - Keep helper exports at their previous import path. - Add focused tests for selection, permissions, locations, and managed profile rejection. - Update the Copilot source identity. Bind the extracted file and retain the v49 history. The source attestation requires v50 after this move. This changes the admitted profile identity even though no execution behavior changes are intended. - Document the boundary and its current limits. ## Verification - 338 focused builder, transport, Codex security, and Copilot identity tests passed on the rebased head. - `pnpm -r typecheck` passed. `pnpm build` passed. - The full root `pnpm test:run` on the pre-rebase head is still in progress. It has reported five failures in unchanged server tests. One isolated failure rejects the word `private` from this checkout's `/private/tmp` stack path. The full root suite is not reported as passing. - All 56 CI checks passed on the rebased head. Four optional checks were skipped. The branch is mergeable. Workspace typecheck and build passed on this head. - Standard Greptile and Apex each scored commit `47f631883d8f417f6a818500ebe4e77e22cc70b8` 5/5. The rebase onto master changed only Copilot identity metadata, history handling, and related tests. `git range-diff` confirms that the launch-preparation implementation is unchanged. The existing Apex result remains the review evidence; it did not review the new commit hash. - No live provider calls or Cloud deployment were run. ## Risks - A moved field could change provider startup. Existing transport tests and the new boundary tests check the projection. - Copilot source binding requires a new identity after the move. Existing artifacts and retained sessions must match the new admitted profile. This PR does not claim fresh live qualification. - This PR does not enable plugin-owned preparation. File materialization remains in the existing SDK path. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bbc2104b6 |
docs: map missing product and operator capabilities (#16990)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The feature map records product entry points and verification recipes. > - The paired agent access map records APIs and permission boundaries for the same capability IDs. > - The new index in #16987 made several missing capabilities easier to find. > - Those gaps hid existing assistant, privacy, cost, and operator workflows. > - This pull request adds 15 capabilities with source references, test scope, and manual checks. > - Readers can find these workflows without treating documentation or structural checks as live proof. ## Linked Issues or Issue Description Refs: #16987. Related map work: #16025 and #16214. A search of related pull requests found no duplicate of these additions. **Issue type** Missing documentation. **Where is the issue?** `maps/feature-map/` and `maps/agent-access-map/`. **What's wrong?** The maps omit public assistant MCP connections, private task and project sharing, subscription fees, cost-recording recovery, agent API keys, keyboard entry points, and several operator CLI workflows. **Suggested fix** Add product recipes and matching access records. Show the full capability IDs in the README index and each page. Keep the existing ID format and all 175 existing IDs. Exclude AI account pools. ## What Changed - Add the `assistant-connections` family with setup, consent, device login, client use, and revocation. - Add `tasks/privacy`, `projects/privacy`, `budgets-costs/subscriptions`, and `budgets-costs/accounting-recovery`. - Add `agents/api-keys`, `agents/managed-agent-setup`, and `navigation-preferences/command-palette`. - Add `cli-operations/feedback-export`, `chat-channels/outbound-email`, and `instance-operations/runtime-setup`. - Pair all 15 additions with access rules, source evidence, manual checks, and explicit coverage gaps. Update the visible index to 35 families and 190 capability IDs. - Add cross-links from existing access, connection setup, search, and CLI recipes. Do not change the schema or add AI account pools. - Clarify that the task Share sheet displays direct and inherited access. Manage inherited access at its ancestor or project source. ## Verification - `node scripts/check-maps.mjs` passes: 35 areas and 190 paired capability IDs, with valid references and contract fields. - `node --test scripts/check-maps.test.mjs scripts/storybook-feature-index.test.mjs` passes: 17 tests. - `node scripts/storybook-feature-index.mjs` and `git diff --check` pass. - A comparison with the base confirms all 175 existing IDs remain and exactly the intended 15 IDs were added. The README index matches the discovered feature metadata. - `node scripts/storybook-catalog.mjs` cannot run because this checkout lacks `storybook/internal/csf-tools` and installed workspace dependencies. - `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were attempted. All stopped before their task ran: the pnpm 9.15.4 bootstrap could not fetch and verify its registry packages in this environment. - Source review covers the documented UI, CLI, routes, services, and existing tests. No live browser, provider, or authorization journey was run. New access records remain `baseline`, `source-only`, and `not-run`. - Greptile and the Paperclip reviewer both rate current commit `bb8eb797b0a1dc26256faf2f0e57cbe7abc9adcf` 5/5. The inherited-access review thread is resolved. - [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/39466772379) passes for that commit, including typecheck, build, unit tests, Runner checks, and browser tests. Browser shard 7 passed after one retry for a runner shutdown. All PR checks are successful, neutral, or skipped. ## Risks - Low runtime risk: this change only edits maps. No application behavior, API, dependency, or schema changes. - Source-reviewed recipes can drift or miss deployment-specific behavior. The records state their limits and require live checks before a capability is marked verified. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, model ID `gpt-6.1-sol`, 475,000-token context window, xhigh reasoning effort, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.10 |
||
|
DottaandPaperclip
|
505e1a5582 |
feat(cases): add persistent workflows with same-task agent handoffs (#16959)
## Thinking Path > - Paperclip manages AI agents and their work. > - An agent workflow needs persistent facts and a safe way to transfer work. > - Creating a task for each phase splits the objective, discussion, workspace, and review policy. > - A case can retain the facts while one primary task moves between agents. > - Transfers must survive process failure and preserve private access, budgets, and completion gates. > - This pull request adds durable case handoffs, native case tools, and read-only case pages. > - Prompts describe the workflow. Existing task controls decide when work can run and finish. ## Linked Issues or Issue Description **Problem or motivation** Prompt-driven agent teams need a persistent record that can move through triage, implementation, verification, and correction. Ordinary reassignment does not provide a durable instruction, safe release barrier, or recoverable delivery receipt for an agent handing off its own running task. **Proposed solution** Extend standalone cases with one primary task, a descriptive phase, retained access sources, and a handoff ledger. Apply assignments after source execution and workspace settlement. Deliver each transfer through the existing durable wake queue. Provide `create_case`, `read_case`, `update_case`, and `handoff_case` in the native runner. Creation requires kickoff and explicit workflow text. An explicit handoff to `done` closes the case when normal task completion is accepted. **Alternatives considered** Do not add a workflow engine, filesystem queue, configured state graph, or one task per phase. The workflow document combines transition prose with an optional Mermaid `stateDiagram-v2` map. The server does not execute the diagram. The Pipelines experiment removed on master stays removed. **Roadmap alignment** This extends Agent Reviews and Approvals, Enforced Outcomes, Bounded Run Recovery, and Activity Log attribution. Existing approval, review, budget, privacy, workspace, and responsible-user rules remain authoritative. Related work: [ordinary handoff wake recovery](https://github.com/paperclipai/paperclip/pull/15019). This feature adds a separate persisted case-handoff receipt and release barrier. ## What Changed - Add case routing fields, retained access sources, durable handoff history, delivery leases, and replay-safe migrations. - Keep the same primary task across agent phases. Preserve each returning agent's own session and the task's workspace. - Fence outgoing mutations, wait for provider and workspace release, and recover delivery without duplicate task comments or runs. Recheck current assignment permissions as the original caller before applying a delayed transfer. Scope a run Stop to that run’s own handoff intents, so repeated old Stop requests preserve newer work. Cancelling terminal intent without a source run also requires fenced current task ownership, including native retries and preparation with an old checkout. For Cloud callers, revalidate current stack membership with an uncached Cloud RPC and derive owner elevation from the current managed setting. - Keep case reads, documents, assets, Audit, and live events inside retained task/project permissions. - Add native case creation/read/update/handoff tools. Store required workflow text and initial documents on the first task. Page oversized case facts and referenced document bodies through the dedicated tool with bounded UTF-8 JSON, current routing versions, snapshot conflicts, and current source authorization on every page. - Add read-only case lists and detail pages, task-style list controls, compact dotted field tables, linked handoff tasks, and case filters in Audit. - Add native/legacy integration tests, migration/restart tests, browser coverage, and bounded live receipt fixtures. Extend the trusted sandbox condition to cover the new subscription fixture profile. - Hide obsolete paused-agent warnings on completed or cancelled task threads. - Synchronize the company-switch settings test with React effects after a CI failure. Preserve its security assertions and production remount behavior. ## Verification The branch directly includes master `9ef434fe79`. The required `ci / verify` and `ci / e2e` gates pass on head `765fccd67c`. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/39464348614) passes typecheck, build, tests, and all eight browser shards. Security checks pass. [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16959#issuecomment-6291156549), with no unresolved review threads. The optional Paperclip Review check remains queued because its assigned agent is paused. It is not a required branch check. No check result was overridden. Local workspace typecheck and build pass after the master merge, along with database migration checks, token gates, map checks, Storybook catalog checks, and the Copilot profile identity check. Standard builds refresh Runner vendor output. Focused checks pass 2,008 API/paging/context tests and 7 strict case protocol tests. The final commit changes one test fixture only; its ten tests, UI typecheck, and token gates pass. No production source changed in that commit. On current head `765fccd67c`, all **194 critical workflow tests** executed and passed with no skipped tests in those files: - 38 Stop/ownership tests, including board completion intent, native retries, stale Stop, and preparation with an old checkout. - 2 actual legacy process/HTTP handoff tests, including scratch failure, a closed-stdio child that continues writing, and recovery through a fresh worker after the process group stops. - 5 migration tests, including SQL column metadata, preserved null authority context, and all six prior-numbering receipts. - 3 process-settlement and 5 scratch tests. - 88 handoff-service, 18 heartbeat handoff, and 21 native case-tool integration tests. - 7 bounded case/document-read projection tests and 7 strict protocol-contract tests. The current CI checkout was `fed65d49`, merging `765fccd67c` into master `9ef434fe79`. Its new document regressions reconstruct exact Unicode/escaped bodies and revisions through real native authority and authenticated HTTP with generic API tools disabled. They cover case-owned, primary-task, and linked-task documents; routing on every page; stale document snapshots; access loss and unlinking; unrelated/foreign/private sources; system documents; and quarantined content. Active-task document tools retain their existing behavior. These are current-head test results from the first CI attempt; the browser-only retry did not rerun these suites. The full `64fdd3403f` CI failed one separate settings test: after a company switch, it observed an unsettled API-key draft. The production component and test were unchanged from prior green source. Production switches remount both keyed settings layers; the test directly reused an unkeyed view and used a timer tick instead of waiting for React effects. The final commit replaces that fixture synchronization with React `act`. The exact secret-clearing assertions and all timeouts remain unchanged. The original assertion failure remains recorded; no unchanged retry was used. The narrow unchanged baseline and fixed fixture each pass 10 tests. The upstream result-paging merge preserves case capture namespaces and current asset privacy. Its API integration suite executed 39 of 41 tests successfully, with two skipped; the optional assigned-result PRP integration test was skipped. Neither skipped suite is counted in the 194 critical tests or as live provider proof. All 361 upstream migration entries and SQL bodies remain unchanged. Six case migrations follow master as 0363–0368 with identical SQL hashes. Official Drizzle generators produced the combined metadata and five retained snapshots. Generation reports no schema changes, and the snapshot drift test passes. The earlier `ce361ae5d0` CI passed on attempt 1, including its then-current 179 critical tests. Fresh review later identified unusable document references with generic API tools disabled; the bounded document reads and ten additional authenticated HTTP tests address that finding. The full local `pnpm test:run` attempt was cancelled after this Mac exhausted PostgreSQL semaphore capacity; it is **not** counted as a pass. Later database attempts also stopped before assertions or skipped, including one sandbox socket denial. Two original stale-Stop regressions did reproduce the bug against the old source. A REST serialization assertion passed after two preserved database-startup failures. No test timeout was increased. All 194 assertions listed above executed and passed in current-head CI. The complete 385-file, 3,963-test UI shard also passed after the fixture repair. A configured-provider receipt workflow passed 13 backend checks across triage → implementation → verification → correction → done, using one task and four work runs. The saved receipt is revision 2 with total 6 USD; case and task both closed. The original canonical attempt remains failed because its browser assertion confused the terminal destination with completed-task tooltips. The corrected full browser verifier passed on the same retained database and original runs, with no additional provider calls. The last clean-build browser readback is on `6a86c54a53`; later changes affect recovery, authorization, bounded reads, and settlement. Accounted work cost is a $1.010574 rate-card estimate; provider billing and setup-probe usage are unavailable. Copilot profile 49 passed the unchanged seven-fixture private protocol pack (46/46 checks, no retries) and the product file-edit canary after master's completion-replay change. All 60 bound implementation hashes remain unchanged on the current source. Historical profile-47/48 identities and results remain intact. See the [profile-49 compatibility record](https://github.com/paperclipai/paperclip/blob/codex/persistent-case-handoffs/doc/plans/2026-10-20-copilot-completion-recovery-compatibility-v49.json). This is bounded compatibility evidence, not a fresh full matrix, installed-package product test, or Daytona rollout. Preserved CI failures and retries: - On `20662930ac`, Slack managed setup failed a five-second success assertion after reload. Its test/application path was unchanged from a prior green head. One unchanged-source retry passed all 25 tests in that shard; no source or timeout changed. - On `9402b4f17a`, two CI workers received shutdown signals at 15:24:19 UTC. Neither log had an assertion failure before interruption. One unchanged-source retry of each passed all Runner static checks and all 32 tests in browser shard 6; both aggregates passed. The next head, `7b4374edaf`, passed all CI without retries. - On `295f289b5a`, serialized shard 9 timed out during its first device-login test, then the next test found an active login lease. Direct login/test paths were unchanged from a prior green head. Cold import plus a timed-out async body using the next test's mutable actor/store is the supported fixture-race explanation; no request trace proves that sequence. The original failure remains recorded. One unchanged-source retry passed all 294 tests in 19 files, including all 41 device-login tests. No login source or timeout changed. The subsequent `ce361ae5d0` CI passed without retries. - On `64fdd3403f`, one company-switch settings assertion observed an unsettled draft. The final test-only commit uses React `act` without changing assertions, timeouts, or production code. On `765fccd67c`, all ten settings tests and the complete 385-file, 3,963-test UI shard passed. - On `765fccd67c`, browser shard 3 completed the split-handoff assertions, then cleanup received HTTP 500 from run cancellation after a one-second PostgreSQL row-lock timeout. The extra chat run was reporting completed work while teardown began. Cleanup stopped before restoring a global experiment flag, causing a later assertion to fail. Relevant production and test paths are unchanged from an earlier green commit. Stop's lock order is consistent, and no nested self-lock was found; the exact PostgreSQL blocker was not recorded. One unchanged-shard retry passed. The original failure, trace, and downstream flag failure remain recorded. No Stop implementation, cleanup assertion, or timeout changed. The retry supports intermittent cleanup contention; it does not identify the original blocking transaction. ## Risks - Local process transfers require host-owned PID/group absence. Missing or uncertain termination evidence keeps delivery pending, independently of scratch setup. - Handoff admission and task cancellation share execution ownership. New regression tests cover temporary holds, late source writes, and uncertain provider termination. - Retained private sources use an intersection of existing permissions. This intentionally fails closed when a source is missing or an existing task cannot preserve that boundary. - Migrations add tables and indexes. They preserve master history and can replay over prior feature installs. Migration 0368 adds nullable internal authorization provenance with no default or authority backfill. Historical transfers still awaiting release without this context fail closed; an operator must cancel them and request a new handoff. - Delayed Cloud handoffs depend on the current Cloud membership RPC. An unavailable, ambiguous, expired, or mismatched response holds the handoff for retry; it does not reuse cached authority or grant access. - Bounded reads reload and reauthorize the selected projection per page. Underlying work grows with the data size and number of pages. Changed data or routing requires restarting from offset zero without the old snapshot. - Phase names and workflow diagrams remain advisory. They do not satisfy review or approval gates. - This is a large, cross-cutting PR. The maintainer requested one PR for the whole feature; the commits separate data, runtime, UI, and qualification work. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and delegated code review. The exact deployment model ID and context-window size were not exposed in the session. Live fixture agents used the explicitly configured provider model recorded in their test evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8ee851a0d8 |
fix(ui): keep runner turns stable through startup and completion (#16989)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task chat shows each agent turn from message submission through completion. > - Run admission can publish adapter metadata before native runtime metadata. > - The UI changed layouts during that gap and removed the agent identity. > - Completion reports can also arrive before the provider writes its final reply. > - The event reader treated those reports as final replies and then replaced them. > - This pull request keeps the runner layout stable and delays report fallbacks until the final history fetch completes. > - Readers see one provider reply, while report-only runs keep a useful answer. ## Linked Issues or Issue Description **What happened?** Sending a task message briefly replaced the runner avatar, name, and Thinking row with a generic Working row. At completion, a full report summary appeared as the answer. A shorter provider reply then replaced it while streaming. **Expected behavior** Keep the runner identity and header spacing stable during admission. Stream the provider's final reply once. Use a report summary as a fallback only after the run ends and its final history fetch completes without a provider reply. **Steps to reproduce** 1. Send a message to a native Paperclip Runner agent. 2. Observe the initial run record before native runtime metadata arrives. 3. Let the agent call its completion tool before writing its final response. 4. Observe the report summary, final response stream, and saved reply handoff. **Paperclip version or commit** Reproduced from source at `6b804bfeba`. The event-reader behavior was also present in earlier revisions. **Deployment mode** Self-hosted server. Local component tests and browser fixtures reproduce the UI transitions. Related work: Refs #12617, which added native runner turns to task chat. The duplicate search found no open PR for this specific defect. ## What Changed - Keep Paperclip Runner turns in the runner layout while runtime metadata resolves. Use the same header spacing for queued and running states. - Keep structured reports as report entries. Do not create synthetic provider final messages from them. - Track successful event fetches for each run status. Permit a terminal report fallback after the final fetch, or when the transcript contains its terminal receipt. - Preserve report-only answers and saved reply deduplication. - Add desktop and mobile regressions for streamed replies, late final events, report-only runs, pagination, and fetch failures. Delay missing-answer notices until terminal history is ready. Document the presentation rules in `DESIGN.md`. ## Verification - Passed all 8,750 UI tests across 769 files. This includes 456 focused tests for the thread, runner turn, native projection, native transcript hook, transcript adapter, and boundary fixtures. - The six completion regressions fail against the old native event reader and pass with the fix. Three empty-history regressions fail before the review fix and pass afterward. The startup regressions also fail against the old layout condition. - Passed UI TypeScript checking with `node node_modules/typescript/bin/tsc -b ui/tsconfig.json`. - Passed the UI production build with `cd ui && node node_modules/vite/bin/vite.js build`. - Passed `pnpm check:token-gates`, `git diff --check`, and the Storybook catalog generator. - Browser check: mounted the production turn with native event snapshots at desktop and 390-pixel mobile widths. The report stayed hidden while running. The actual reply stayed unchanged at completion. The report-only fallback appeared after completion. The identity row kept the same position and size. No browser errors or horizontal overflow occurred. The temporary fixture was removed. - Passed `pnpm -r typecheck` and `pnpm build`. Initial attempts failed with `COPILOT_INSTALLATION_INVALID`. Reinstalling the generated Copilot assets resolved that local packaging failure without source changes. - The full repository test run was attempted locally and stopped after repeated embedded PostgreSQL bootstrap failures in server suites. The complete UI suite passes independently. In CI, serialized shard 5/9 had one 15-second timeout in the unchanged `agent-live-run-routes.test.ts` history-limit case; its other 387 tests passed. All five history-limit variants pass locally. The single CI job retry passed. The original timeout cause is not established. ## Risks - Report-only answers appear later because the UI waits for terminal history. Failed event fetches keep that fallback pending until a retry succeeds; provider replies remain visible. - Historical replies and accepted response-wake behavior retain their existing resolution rules. The focused tests cover these paths. - The change does not alter provider execution, completion authority, database schema, or run timing. ## Model Used OpenAI GPT-6 through Codex. The exact serving model ID and context window are not exposed in this session. Used reasoning, code execution, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9ef434fe79 |
docs: expose feature map IDs and add a recipe index (#16987)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its feature map describes user journeys and their verification paths. > - The map already defines feature IDs and full capability IDs. > - The README and child pages did not show those IDs consistently. > - Readers need those IDs to connect issues, tests, stories, and evidence. > - This pull request adds a recipe index and displays the existing IDs. > - The benefit is clear references across the product and agent access maps. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** `maps/feature-map/README.md`, the child recipes, and their agent access companions. **What's wrong?** The README sends readers to Storybook to browse features. Child pages show local keys, such as `installation`, without the full `plugins/installation` ID. Readers must inspect the generator to find the full reference. **Suggested fix** List recipe links and existing IDs in the README. Show each feature ID and full capability ID in its child pages. Explain capability IDs and entry-point references. Keep the current keys and anchors. Related work: Refs #15288. Refs #16052. Refs #15983. No duplicate open PR or issue was found by title search. ## What Changed - Add an index grouped by feature area. It lists every recipe, feature ID, and full capability ID. - Explain how IDs are formed and used in issues, tests, Storybook, and evidence. - Add feature IDs to all product recipes and agent access companions. - Show full capability IDs beside recipe keys and below access and Storybook guidance headings. - Preserve existing keys, heading anchors, and generated ID values. Keep the latest upstream company capability in the index. ## Verification - `node scripts/check-maps.mjs` passes for 34 feature families and 175 capability IDs. - `node --test scripts/check-maps.test.mjs scripts/storybook-feature-index.test.mjs` passes all 17 tests. - `node scripts/storybook-feature-index.mjs` passes on this checkout and generates the feature indexes and overview inputs. - `git diff --check` passes. - A source comparison confirms that the README matches the recipe index. After removal of the added ID labels, all 68 child pages match the upstream prose and headings. - Full workspace checks were attempted: `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build`. They could not start because the local pnpm wrapper could not fetch and verify the pinned pnpm 9.15.4 release. This checkout also has no workspace dependencies installed. - `node scripts/storybook-catalog.mjs` could not run because `storybook/internal/csf-tools` is not installed. The direct map and index checks above pass without that dependency. - Remote typecheck, release-registry, and full build checks passed. The first attempts of [server shard 3/12](https://github.com/paperclipai/paperclip/actions/runs/39460473910/job/118758430448) and [server shard 12/12](https://github.com/paperclipai/paperclip/actions/runs/39460473910/job/118758429932) stopped after their runners received a shutdown signal. The test steps were canceled. GitHub rejects the rerun request with HTTP 403: "The workflow run containing this job is already running". Five other CI jobs remain active. The next action is to rerun the two interrupted shards after workflow 39460473910 ends. No application changes were made to address these runner interruptions. - To inspect the result, open the feature-map README, follow a recipe link, and compare its full capability IDs with its access companion. ## Risks - Low risk. The change updates map documentation only. It changes no application code or ID schema. - The readable README index and child labels repeat source IDs. Keep them aligned when a capability changes. - Generated capability descriptions now include the visible ID prefix. Generated IDs, labels, navigation order, and verification fields remain the same. - Full local workspace checks and catalog generation remain unverified for the environment reasons above. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, model `gpt-6.1-sol`, with a 475,000-token context window and `xhigh` reasoning effort. The model used local file tools, shell commands, and GitHub CLI access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.9 |
||
|
DottaandPaperclip
|
9225796a4e |
fix(runner): page oversized assigned tool results (#16980)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents receive assigned app-tool results through the durable Runner protocol. > - The assigned-tool bridge returned complete provider responses without a size bound. > - A large response could exceed the encrypted command limit after the upstream call had completed. > - The provider then waited for a result that the controller could not deliver. > - This pull request saves large governed responses and returns a small reference with a scoped page reader. > - Agents can retrieve the full result without repeating the original action. ## Linked Issues or Issue Description **What happened?** Large app-tool responses could leave native tool calls pending even though the upstream service completed. The durable command limit is about 510 KiB. The assigned-tool bridge did not account for that limit. A 720,048-byte regression response failed the delivery-size assertion before this change. **Expected behavior** Deliver a bounded response and a way to retrieve all saved content. Keep the original action at one execution. Preserve task, agent, company, responsible-user, and current-run access checks. **Steps to reproduce** 1. Run a native agent with an assigned tool that returns more than 510 KiB of JSON. 2. Let the upstream tool complete. 3. Observe that the original bridge returns the oversized object directly, so the durable controller cannot queue its result. Related: #14734 preserves uncertain tool outcomes across shutdown and restart. This change keeps that protection. #16517 handles native MCP content projection on a different transport; it does not bound native assigned-tool results. ## What Changed - Save governed assigned-tool responses above 500,000 encoded bytes using the existing response asset storage, quotas, and audit path. - Add `read_tool_result` for UTF-8 JSON pages from the same task and agent. Each encoded page is capped at 500,000 bytes, including JSON escaping and metadata, with remaining content available through the next byte offset. It works when generic API tools are disabled. - Reuse existing authenticated asset reads and responsible-user checks. Retrieval does not invoke the upstream service. - Preserve uncertain delivery if storage fails after an action completes. - Add regression, fault, authorization, and opt-in live Codex checks. Document the transport and access contract. ## Verification - Red: the original integration regression returned 720,048 bytes against unchanged production code and failed its bounded-response assertion. The follow-up regression also showed that a 480 KB result was unnecessarily saved under the original 24 KiB policy. - Green: results through 500,000 encoded bytes stay inline. Larger responses are saved in full. Real HTTP integration tests reconstruct the complete Unicode result after controller recreation in exactly two pages, in both standard and low-trust configurations, with one upstream execution. - The encoded-size boundary and escape-heavy Unicode tests verify that every page fits 500,000 bytes, cursors advance on valid UTF-8 boundaries, and concatenating pages exactly reconstructs the original JSON. Storage-failure tests preserve an uncertain outcome without repeating the action. - The focused unit/API run passes 1,957 tests; the changed real HTTP tests pass. `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` pass. - Real Codex acceptance passes with `gpt-6.1-sol` through Codex CLI 0.160.0 and native Runner, on macOS ARM64 with `low_trust_review` and generic API tools disabled. Codex requests every page in order, receives all 660,098 original bytes, and returns the final verification field. The two encoded pages are 500,000 and 160,543 bytes; the upstream operation executes once. Runner closes successfully in a 34.1-second probe. Only the disposable database's connection/worker counts were reduced to fit the host's remaining semaphore capacity. - The native-PRP regression also passes with escape-heavy Unicode pages over 1 MB in total and a separate inline result containing 200,000 backslashes. A scripted provider verifies both complete results by SHA-256. It proves the intermediate text wrapper can exceed 700 KB because the native transport decodes it before queuing the bounded PRP result; every result object stays within 500,000 encoded bytes. Native binaries are required for the local run, preventing a silent skip. - A second real-Codex probe with escaped Unicode, quotes, backslashes, newlines, and NULs also passes: every original byte of a 720,098-byte JSON response is retrieved in two encoded pages of 500,000 and 460,543 bytes. The model returns the final field, the upstream call executes once, and Runner closes in 32.6 seconds. This uses the same production source with only the synthetic fixture content varied. - The live probe exercises real PRP, persistence, and authenticated asset reads. It does not claim the original staging task resumed or full heartbeat checkpoint acceptance. - Earlier broader local testing passed 103 tests but could not start the second database for the existing `rejects a stale source-run question over public HTTP and accepts it from the next run` test. Extending its timeout exposed PostgreSQL initialization failure on this semaphore-exhausted host. The earlier local `pnpm test:run` attempt was interrupted and is not claimed as passing. Full CI passes on current head `cd32237ee5429765b8f349402ac3c79cceec3e6b`. GitHub reports 55 successful checks, two skipped Storybook checks, and one neutral Paperclip agent review. Greptile is 5/5 on this head with no review threads. - Additional Paperclip agent review is incomplete: it withdrew the earlier double-escaping finding after inspecting the native regression, found no new actionable defects, and reported stalled large-source reads prevented independent inspection of the transport decoder. Its neutral assessment is [recorded here](https://github.com/paperclipai/paperclip/pull/16980#issuecomment-6295579705). This limitation remains separate from the passing local native transport, real Codex acceptance, full CI, and Greptile results. ## Risks - Large responses now consume existing response-storage quota and require page reads. Larger pages use more provider context. Results are preserved in full; each encoded page leaves about 22 KiB below the PRP command limit for the command envelope. - A storage failure after execution retains an uncertain delivery outcome. The bridge must not report that the upstream action failed or invite a replay. - The dedicated reader grants access only to saved responses for the current task and agent. The asset route also checks the original responsible user. - Runner command limits and receipt rules remain unchanged. No migration or separate Runner protocol rollout is required. - This PR does not deploy the repair or resume an existing blocked task. ## Model Used OpenAI Codex (GPT-6), with reasoning, tool use, and code execution. The exact serving model ID and context-window size are not exposed in this session. The opt-in acceptance probe requests `gpt-6.1-sol` through Codex CLI 0.160.0. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the changed behavior (focused unit/API, real HTTP, and real Codex acceptance); the broader local database limitation is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |