8 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	adb69e8f09	feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683) ... ## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto | paperclip | legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. **Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass.** The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board **Try again** and **Resume** recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal **Approve completion** reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 11:33:02 -05:00
Nicky LeachandPaperclip	2dd9811e8b	Own agent hiring and termination in a lifecycle module (#15631) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Agent creation, approval, pause, and termination change whether an agent can work. > - These changes currently use separate service and database paths. > - Some agents need setup after their record exists and before their harness can run. > - This PR adds one module that owns those state changes and their transactions. > - The module records progress and retries incomplete work after a server restart. ## Linked Issues or Issue Description **Problem or motivation** A hire can become active before its execution environment is ready. Direct agent state writes also bypass one common lifecycle boundary. Termination does not have a durable step for resource cleanup. **Proposed solution** Use agent lifecycle commands for hiring, approval, pause, resume, and termination. Keep proposed hires pending until approval. Prepare each approved hire, test its saved harness configuration, and then permit work. Wait for required plugin work before completing a step. **Alternatives considered** A general database commit hook would expose transaction details to callers. A timer alone would delay each new hire. This module owns each command transaction and starts its worker after commit. A periodic scan recovers work after a process stops. **Roadmap alignment** This change adds the lifecycle boundary for the existing agent and plugin systems. It does not add a user interface or a provider implementation. It replaces the lifecycle delivery approach in #15391. Related proposals include #11475 and #13099; this change does not add their external executor protocols. ## What Changed - Add durable approval, preparation, verification, pause, resume, stop, and cleanup states. - Keep agent creation, lifecycle state, configuration writes, and agent record deletion behind `modules/agent-lifecycle`. Keep validation, credential rules, reads, onboarding, invitations, general approvals, and company data deletion in their services. - Commit a hire before the remaining onboarding or invitation work. Serialize requests with the existing dedicated database connection helper. Reuse the hire after a failed attempt. - Keep accepted work in the run queue during setup. Require execution and cleanup evidence before completing pause or termination. - Retain the original credential owner when approving migrated hires. - Start lifecycle work after each command commits. Use database leases and revision checks for retries and late results. - Add a generic required plugin method and capability. Store the required plugin IDs for each agent. Track host and plugin completion separately. - Keep harness verification and credential rules in services. Supply their results to lifecycle through explicit ports. Reject failed logins and incomplete probes. - Remove reverse imports from lifecycle adapters into services. Share record queries and validation below both layers. - Use one hire decision path for agent IDs and approval IDs. Resolve the approval and agent change in one transaction. Expose policy reconciliation on the command object. - Save configuration and verification invalidation through one module operation. Ordinary commands own their transaction. Restrict the shared transaction integration to its listed callers. Reject identity, company, accounting, timestamp, and unknown fields. - Require completed termination before deleting agent records. The company deletion service calls the module through a separate deletion interface. - Save an approved secret before applying its agent binding. A failed binding leaves the secret approved. A retry reuses it. - Remove old permission and metadata export files. Move hire configuration rules out of lifecycle persistence. - Report failed worker steps and scans with operation context and known error codes. Omit provider messages, stacks, and configuration values. Document ordinary application use and extension rules. - Recover expired native owners only after execution stops. Retry cleanup of failed retained leases. Wait for setup in the live smoke script. - Keep manual, budget, and company pause reasons separate. Guard execution status writes with the ready state. - Add a migration, boundary checks, regression tests, and a lifecycle guide. Keep the current UI. - Limit periodic policy checks to 100 eligible agents per minute. Continue from the last agent ID. Exclude pending hires and agents in termination or a final state. Keep explicit company and budget updates immediate. ## Verification - Latest head `ccc3c86b0` aligns the shared status update schema with the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2 CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile comment is fixed. CI is running; no new Apex review was requested for this small fix. - Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test uses the shared 90-second timeout from master; the older timeout commit was removed. The affected backup test passes. - Agent budget edits now pass the target agent ID through enforcement. All 50 affected tests, server TypeScript, and 11 boundary tests pass. Both configuration update paths have regression coverage. CI is running on the latest head. No new Apex review was requested for these small fixes. - Rebase `9b568666a` preserves the run-control extraction and delivery-work startup changes from master. All 559 affected tests, server TypeScript, and 11 boundary tests pass. CI and Apex review are running on the final head. - The policy scan fix passes 56 lifecycle and diagnostic tests, 24 policy and caller tests, and 11 boundary tests. A regression test checks the page limit, interval, excluded states, and cursor wraparound. - Rebase `fd4698750` passes all 611 tests in eight affected lifecycle, route, and heartbeat suites. Server TypeScript and all 11 module boundary tests pass. The company deletion rollback and concurrent hire tests are included. - Follow-up `00aac7646` fixes the failed decision deletion test and the public update schema. All 126 affected server tests, 8 shared tests, 2 CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI TypeScript checks pass. - Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc --noEmit`. - All 11 boundary tests pass: `node --test scripts/check-module-boundaries.test.mjs`. The repository boundary scan also passes. - All 68 lifecycle, failure diagnostic, and harness verification tests pass. Tests reject forbidden fields through the root command, combined transition command, and restricted transaction integration. They also check invalid fields returned by configuration preparation. - Failure tests cover host work, plugin work, claims, renewals, deferrals, and scans. Log tests check known error codes and exclusion of private error contents. - All 355 tests in 10 existing caller suites pass. These tests cover secret approval, pairing, pending hires, credentials, company import, built-in agents, managed agents, instructions, and skills. - The branch is rebased onto `origin/master` at `2eb5187e4`. The queue guards and cancellation lock fix use the extracted heartbeat queue. Agent cancellation uses the extracted run-control file. The source diff scan found no secrets or private URLs. - Earlier checks cover approval retries, rollback, late verification results, policy holds, deletion rollback, concurrent company deletion, and queue admission during setup. - Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it 5/5 with no open comments. CI and a new Apex review are pending for the rebased commit. The branch has no merge conflicts. - The full local build and native Dot suite were not run. The host has no Cargo. No live provider test was run. ## Risks - Background policy recovery takes more than one minute when more than 100 agents are eligible. Explicit company and budget updates still reconcile immediately. - The restricted configuration integration keeps an agent reference and its dependent records in one transaction. It cannot change a budget or invoke a lifecycle transition. A periodic scan starts pending setup after that transaction commits. - Secret approval and binding use separate transactions. A failed binding can leave an approved secret with no agent binding. - New approved hires remain unavailable until setup and verification complete. Existing clients see `paused` during these steps. This PR does not change the UI. - Migration `0320_daily_onslaught.sql` maps existing statuses and pause reasons. It does not create external resources or claim that an existing harness passed verification. - Use a coordinated server update for this migration. Older server versions can write legacy status without the new lifecycle checks. - A failed onboarding or invitation request can leave a committed hire. A retry reuses that hire and completes the remaining records. - Deletion requires completed termination. Company deletion can require another request while cleanup is pending. - Required plugins must make repeated calls safe and reject older revisions. A disabled or removed required plugin blocks completion until it is restored. - A new hire can run a second harness test after the existing UI test. Some adapters can charge for this test. - No live provider test was run. Provider implementation and resource backfill remain separate work. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code tools, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR with the feature request field labels - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] Targeted local tests pass - [x] All Paperclip CI gates are green on the new commit - [x] Greptile Apex is 5/5 with no open comments on the new commit - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 09:05:12 -07:00
Devin FoleyandPaperclip	799e4d556f	fix: make accounting durable and synchronize cost reporting (#14997) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-06 22:26:48 -07:00
DottaandFry	3ca196b0a6	feat(agents): persist agent files across tasks without revision history (#14420) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent needs personal files across tasks and sessions. > - AGENTS.md is one file in that directory. Supporting files need the same persistence. > - The Instructions Editor and agent runs must share one current directory. > - Concurrent runs should apply only the files they change. The last sync of the same file wins. > - This pull request uses existing file transport and removes temporary copies after sync. > - Old instruction-only sessions keep their restore contract. New saves do not create revision history. ## Linked Issues or Issue Description Refs #14325. This replaces its revision-oriented design with persistent agent files. Keep #14325 unmerged. Transport prerequisite #14416 merged first at `d172197117a14b80a1eb2d2835a0e7cce2679656`. This PR now targets master and remains below 100 changed files. Related work: #4513 and #8798 cover instruction tooling. This change handles run synchronization, cross-task personal files, browser editing, and old-session restoration. ## What Changed - Keep one current directory per company and agent. Point AGENT_HOME at a temporary working copy for each active run. Keep task files and provider HOME separate. - Restore text, binary files, and nested folders through workspace transport. Exclude remote agent files from task Git snapshots with a self-ignoring file inside the reserved runtime directory; never write through repository-controlled Git metadata. - Collect after the provider and child processes have stopped. Keep resumable conversation state. - Apply changed and deleted files under the agent lock. The last sync wins for the same file. Unrelated concurrent changes survive. - Remove temporary copies after successful sync, rejected sync, and staging failure. Register ownership before copying so restart recovery can remove interrupted preparation. Retry transient synchronization up to three times. Preserve the original remote lease reference until deletion succeeds; restart cleanup never acquires a replacement sandbox. Do not create captured directories or a conflict-review queue for new runs. - Keep browser editing, stale-draft protection, and streaming binary downloads. Keep the instruction entry and text editor limited to 1 MiB. - Keep historical agent-folder sync failures on their affected runs instead of repeating them above current saved instructions. Preserve legacy candidate review and current browser-save errors. Avoid duplicate quota warnings while retaining separate sync failures when they describe a different problem. - Require target-scoped caller grants for peer instruction access, while preserving self edits, responsible-user checks, and protected-change consent. - Treat full storage as a nonblocking run warning, never an agent pause or run-admission failure. Restore already-over-quota saved folders so ordinary agent cleanup can recover; warn on each run until cleanup. The run detail view shows the warning. - Allow 256 MiB per file, 2 GiB per directory, and 100,000 entries. Hash large files as streams. Check editor-save quotas with metadata instead of hashing unrelated files. - Preserve old native inputs, instruction-only copies, paths, digests, and pending legacy candidates. Adopt old revision heads once. New writes do not append history rows. - Add idempotent migration 0287 and verify upgrades from the preview tables and receipts. - Add nine interactive stories under **Agents / Persistent files**, including automatic incoming edits, stale browser drafts, and storage-limit diagnostics. ## Verification - Merge candidate: `4f5390107ec6ffd80a76d1d2e85530e66f21d079`, after merging current master and the landed transport prerequisite. Integration required no manual conflict resolution; the feature remains 99 changed files. Full workspace typecheck, production build, token gates, and 715 focused tests passed on this merge candidate. Fresh Greptile review is 5/5 with no unresolved findings. All 55 checks passed, with four conditional skips, including the build, typecheck, browser E2E, and canary dry run. A single retry recovered four jobs interrupted by runner shutdowns; no source changes were required. - Historical-warning UI fix: all 6,834 UI tests across 640 files passed, including regression coverage for three old failures, legacy preserved edits, and warnings scoped to the affected run. Full workspace typecheck, production build, Storybook build, and token gates passed. Browser-verified Storybook playtests passed for Historical Failures After Successful Save, Storage Limit, and Full Storage Run Warning. - Review follow-ups at `4e20c9fb2`: all 18 focused tests passed, including external Git directories, linked worktrees, symlinks, hardlinks, and distinct I/O failures alongside storage warnings. Server and UI typechecks, token gates, and the production build passed. - Storage warning regressions at `0724f3012`: all 33 directory tests and all five heartbeat-list tests passed, with no skips in their successful runs. They cover repeated runs while full, an already-over-quota saved folder, cleanup, warnings retained after unrelated save failures, and bounded warnings in large result JSON. Server typecheck passed after the final warning fixes. - Full workspace typecheck, production build, and token gates passed during this follow-up. Product E2E harness: 631 tests passed across 52 files; harness typecheck passed. Earlier native session/context and directory/legacy collection suites passed 537 tests; Runner unit/transport suites passed 329 tests. - **Real E2E at `0724f3012` (before this follow-up):** legacy local Codex and native Daytona Codex each passed six tasks, one server restart, seven independent assertions, and cleanup verification. Both prove browser-to-agent edits, agent-to-browser edits, nested/binary restoration, per-file last-sync-wins, a successful run after an oversized save rejection, and cleanup clearing the warning. - Native local Codex also passed the six-task quota flow before the final warning-retention fixes. That pass began at `918d1ed02` while the bounded-result warning fix was being edited, so it is not claimed as exact-final-head evidence. Its final-head rerun failed during embedded PostgreSQL bootstrap before any provider run: the macOS host had 87,365 of 87,381 SysV semaphores occupied. No unrelated services or kernel limits were changed. - The final-source report intentionally records **2/3 cells passed**, preserving the blocked native-local attempt: `tests/runner-e2e/results/agent-files-quota-final-20260928-report/`. Earlier failed attempts and provenance notes remain under `tests/runner-e2e/results/agent-files-quota-final-20260928-input/` and the original campaign directories. - Daytona used immutable image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:5643f0d801417cae3581833a1a3bc6715b325e028602738d2652c44cac5dc6bf` and its exact Linux runner binary. Controller source is `0724f3012`; image source is recorded separately. - Legacy-session compatibility and all three ACP Stop/resume browser regressions passed on the prior validated feature head `169fab46d5af21caa2269b4c1b29b69c933a6951`. They assert the same provider session is retained and interrupted writes are not replayed. Migration upgrade tests also passed earlier. - Nine interactive stories are under **Agents / Persistent files**, including **Full Storage Run Warning**. Its playtest and visual browser inspection passed; the warning states that runs continue and the editor remains available. - Prior-head checks on `4e20c9fb2`: 55 passed, two conditional jobs skipped, no failures or pending checks. All eight browser E2E shards and their aggregate passed. Fresh Greptile review is 5/5 with no findings; all review threads are resolved, the security scan passed, and GitHub reports no merge conflicts. - The broad local follow-up test run was interrupted after host semaphore exhaustion affected isolated PostgreSQL instances. It also encountered the existing macOS long-path fixture failure and two timeout failures. This is not a claim that the full local suite passed. Logs are retained; focused storage/warning tests passed. ## Risks - A later sync can overwrite an earlier edit to the same file, including a saved browser edit. There is no text merge or retained version. This is the intended last-sync-wins policy. - A save that exceeds a storage limit is rejected and its temporary copy is discarded. The run itself continues normally, and later runs restore the last saved files with a warning until cleanup. Transient sync failures get bounded retries. An I/O failure partway through a sync can leave some files updated; a failed receipt does not claim whole-folder success. - Larger folders increase copy time, network traffic, and temporary disk usage. Active runs still need working copies. Terminal runs do not accumulate archives. Operators must provision disk for agents and configured concurrency; these limits are not company-wide quotas. - A restored old native session remains instruction-only until a fresh session starts. Its original conflict fence and existing pending candidates remain compatible. - Provider processes close at the collection boundary. Conversation resume remains available, but warm process reuse is lost. - Backups must include the instance filesystem and database. External bundles keep their existing behavior until explicitly moved to managed storage. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific model ID or context-window size. Reasoning, code execution, and browser tools assisted this change. Real provider E2E uses `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Fry (Paperclip) <noreply@paperclip.ing>	2026-09-29 08:25:56 -05:00
DottaandPaperclip	9dd6526b47	fix(security): harden privileged server boundaries (#12776) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server controls secrets, host files, outbound requests, and workspace commands > - A red-team review found cases where restricted callers could cross these trust boundaries > - These cases could expose credentials or let untrusted input reach privileged resources > - This pull request applies least-privilege checks at each affected server boundary > - The benefit is safer agent execution without changing the private-instance bootstrap contract ## Linked Issues or Issue Description **What happened?** Several server paths used authorization, redaction, or content-delivery rules that were too broad. Restricted agent keys could obtain company-level operational data. Some adapter and instruction paths could reach server-owned network or file resources without the required owner approval. **Expected behavior** Paperclip must redact credential values, enforce restricted-key scopes, guard outbound network access, prevent same-origin script execution, and reserve host-level file and command controls for authorized operators. **Steps to reproduce** 1. Configure an authenticated development instance at the parent commit. 2. Exercise the affected APIs with a restricted agent key or a non-instance-admin company user. 3. Observe that the parent commit returns privileged data or accepts a privileged operation. 4. Repeat on this branch and observe a redacted response, a safe download, or an HTTP 403 response. **Paperclip version or commit** The findings reproduce from commit `39898ab22` and are fixed by this pull request. **Deployment mode** Authenticated self-hosted server and local development modes. **Installation method** Built from source with pnpm. ## What Changed - Redact generic secret `value` and `token` fields recursively in structured logs. - Classify exact and separator-suffixed `KEY` environment names as secrets in company exports. - Limit restricted self-identity responses and protect company run, log, and secret catalog APIs. - Route HTTP adapter requests through DNS-pinned SSRF protection with exact private-origin allowlisting. - Download HTML, SVG, and other script-capable assets with `nosniff` and a sandbox CSP. - Require instance-admin access for external instruction roots and exports that read them. - Block agent-authenticated host command persistence across supported workspace runtime shapes. - Apply the central runtime-management decision before workspace command controls. - Keep the documented first-user instance-admin claim contract unchanged. - Add regression tests and server-owner configuration documentation. ## Verification - `pnpm -r typecheck` passes. - The Node 24 remediation suite passes with 365 tests. It skips 25 environment-gated tests. - `pnpm build` passes under Node 24. - `git diff --check` passes. - The full local runner reaches known macOS-only general-server harness failures before the serialized route lane. The Linux PR matrix is the authoritative full-suite gate. ## Risks - Restricted agent keys now receive HTTP 403 responses from company-wide run, log, and secret catalog endpoints. - Script-capable assets now download instead of rendering inline. - External instruction roots now require instance-admin access. - Private HTTP adapter endpoints now require an exact origin in `PAPERCLIP_HTTP_ADAPTER_PRIVATE_ENDPOINT_ALLOWLIST`. - Public HTTP adapter endpoints remain enabled. Redirects and metadata or link-local targets remain blocked. - No database migration is required. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5. The exact serving snapshot and context-window size are not exposed. The model used tool-enabled reasoning, repository access, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-03 14:15:32 -05:00
DottaandPaperclip	ae6761e2b0	fix(server): authorize agent resume through direct grants (#12047) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip keeps agent lifecycle changes behind control-plane authorization > - Plugins can create agents in a paused state until an operator activates them > - An agent with a direct configuration grant could not resume these agents > - A paused plugin-managed agent also had no stable provenance in its pause reason > - This pull request adds one protected resume path and preserves every other lifecycle gate > - The benefit is safe recovery from plugin provisioning without a broad permission change ## Linked Issues or Issue Description Refs #8168. That pull request uses a role capability and also opens clear-error. This change uses the current grant system and keeps clear-error closed. **What happened?** A plugin can create a paused managed agent. An agent actor cannot resume that agent, even when the actor has a direct `agents:configure` grant. The paused agent can also have a null pause reason. **Expected behavior** An agent with a direct `agents:configure` grant can resume an accessible paused agent. An agent without that grant cannot resume it. Plugin-managed paused agents show stable plugin provenance. A completed resume stays in effect after reconcile. **Steps to reproduce** 1. Install a plugin that declares a managed agent with `status: paused`. 2. Give a same-company agent a direct `agents:configure` grant. 3. Call `POST /api/agents/{id}/resume` with the granted agent key. 4. On the base revision, observe a board-only authorization error. **Paperclip version or commit** `master` at `63df7ad2b3`. **Deployment mode** All deployment modes. This is a server authorization and reconcile behavior. ## What Changed - The resume route now uses the protected `agent_config:update` decision with `requiresChangeGrant: true` for agent actors. - The route keeps board access, tenant non-disclosure, and invalid organization-chain protection. - Resume activity now records the real user or agent actor, run, and API key. - Plugin-managed paused agents now receive a stable provenance reason and pause time at creation. - Reconcile backfills only a null reason on an agent that is still declared and stored as paused. - Reconcile preserves manual, budget, system, and other pause reasons. It does not pause a resumed agent again. - The implementation specification now records the narrow resume exception. ## Verification - `pnpm exec vitest run server/src/__tests__/agent-cross-tenant-authz-routes.test.ts server/src/__tests__/plugin-managed-agents.test.ts` passed: 2 files and 26 tests. - `pnpm --filter @paperclipai/server typecheck` passed. - `pnpm -r typecheck` passed. - `pnpm build` passed. - GitHub CI passed all policy, typecheck, build, test, e2e, canary, and security gates on commit `306edf469c`. - Greptile reviewed all 5 changed files. Its check passed with 0 comments and 0 unresolved threads. - The host uses Node 22.22.2. The repository requests Node 24.11 or newer, so pnpm printed engine warnings. - A broad `pnpm test:run` attempt did not complete its general-server group. Runtime port fixtures failed because host port `52000` was already bound. The isolated failing fixture reproduced the same port conflict. The focused feature tests passed before and after the final commit. ## Risks The main risk is an unintended lifecycle permission increase. The change limits agent access to resume only. It requires a protected direct-change decision. It does not open pause, clear-error, terminate, approval, or key-management routes. Tests cover denial, self-denial, tenant isolation, organization-chain checks, and activity attribution. There is no database migration. > This change fixes a narrow gap in the completed plugin, approval, and activity-log roadmap areas. It does not add a new roadmap feature. ## Model Used OpenAI Codex `gpt-5.6-sol`, with xhigh reasoning, tool use, and code execution. The runtime did not expose its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-23 16:15:32 -05:00
DottaandPaperclip	0096b56a1c	[codex] Add LLM Wiki plugin host support (#5597) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system needs host contracts and runtime support before large plugins can integrate cleanly. > - The source branch mixed the LLM Wiki package with supporting host/runtime work, managed plugin skills, root-level storage spaces, and a bookmarks reference plugin. > - [PAP-9173](/PAP/issues/PAP-9173) asked for the current branch to be split by file boundary: plugin package separately from everything else. > - [PAP-9188](/PAP/issues/PAP-9188) clarified that LLM Wiki may have plugin-local spaces, but Paperclip core should not reorganize top-level local storage into spaces. > - Follow-up review clarified that the bookmarks example should not ship in this PR either. > - This pull request contains the non-`packages/plugins/plugin-llm-wiki/` host/runtime work, keeps runtime state under the selected Paperclip instance root, and no longer includes the bookmarks example. ## What Changed - Added/updated plugin host contracts, SDK types, worker RPC plumbing, managed plugin skill support, and related server tests. - Removed the bookmarks example plugin package and its bundled-example/workspace references. - Removed the root-level local spaces CLI/migration surface and restored instance-root runtime defaults for config, db, logs, storage, secrets, workspaces, projects, and adapter homes. - Replaced shared root `space-paths` helpers with `home-paths` helpers for core runtime storage. - Tightened stranded recovery unique-conflict detection so concurrent recovery scans reuse the raced recovery issue when Postgres errors are wrapped. - Kept `packages/plugins/plugin-llm-wiki/` out of this PR diff; plugin-local spaces remain in the stacked plugin-only PR. ## Verification - `pnpm exec vitest run cli/src/__tests__/data-dir.test.ts cli/src/__tests__/home-paths.test.ts cli/src/__tests__/onboard.test.ts packages/shared/src/home-paths.test.ts packages/db/src/runtime-config.test.ts server/src/__tests__/agent-instructions-service.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts` - `pnpm exec vitest run packages/db/src/runtime-config.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "reuses the raced stranded recovery issue"` skipped locally because embedded Postgres did not initialize on this macOS temp host; the code path was typechecked and is covered by Linux CI. - Boundary check: no core references remain for `PAPERCLIP_SPACE_ID`, `spaces migrate-default`, `@paperclipai/shared/space-paths`, `registerSpacesCommands`, or the removed bookmarks example. - Previous PR head `4f23e034` had green GitHub checks: `verify`, all four serialized server shards, `e2e`, `Canary Dry Run`, `policy`, Snyk, and `Greptile Review`. Current head `582f466d` is re-running checks after the bookmarks deletion. ## Risks - Plugin host changes touch shared runtime paths, so regressions would most likely appear in adapter startup, plugin loading, or local dev path defaults. - Removing the bookmarks example also removes one demonstration of plugin database namespaces plus local-folder persistence; remaining plugin examples still cover bundled example discovery and plugin host flows. - The plugin package itself is intentionally deferred to the stacked plugin-only PR, where LLM Wiki plugin-local spaces live. - Existing installs that tested the transient root-level spaces CLI should stop using it; this PR intentionally removes that unsupported migration surface before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution enabled; context window not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass, except where noted above for host-specific embedded Postgres initialization - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Stacked follow-up: PR #5592 contains only `packages/plugins/plugin-llm-wiki/` and targets this branch. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-05-10 07:34:12 -05:00
DottaandPaperclip	3c73ed26b5	Expand plugin host surface (#5205) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The plugin system is the extension boundary for optional product capabilities > - Rich plugins need more than a worker entrypoint: they need scoped database storage, local project folders, managed agents/routines, host navigation, and reusable UI components > - The LLM Wiki work exposed those missing host surfaces while keeping plugin code outside the core control plane > - This pull request expands the core plugin host, SDK, server APIs, and UI bridge so plugins can declare and use those surfaces > - The benefit is that future plugins can integrate with Paperclip through documented, validated contracts instead of bespoke server or UI imports ## What Changed - Added plugin-managed database namespaces and migration tracking, including Drizzle schema/migration files and SQL validation for namespace isolation. - Added server support for plugin local folders, managed agents, managed routines, scoped plugin APIs, and plugin operation visibility. - Expanded shared plugin manifest/types/validators and SDK host/testing/UI exports for richer plugin surfaces. - Added reusable UI pieces for file trees, managed routines, resizable sidebars, route sidebars, and plugin bridge initialization. - Updated plugin docs and example plugins to use the expanded host and SDK surface. ## Verification - `pnpm install --frozen-lockfile` - `pnpm run preflight:workspace-links && pnpm exec vitest run packages/shared/src/validators/plugin.test.ts server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-local-folders.test.ts server/src/__tests__/plugin-managed-agents.test.ts server/src/__tests__/plugin-managed-routines.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts ui/src/api/plugins.test.ts ui/src/components/FileTree.test.tsx ui/src/components/ResizableSidebarPane.test.tsx ui/src/pages/PluginPage.test.tsx ui/src/plugins/bridge.test.ts` passed: 11 files, 67 tests. - Confirmed this PR changes 89 files and does not include `pnpm-lock.yaml` or `.github/workflows/*`. ## Risks - Medium: this expands plugin host contracts across db/shared/server/ui and includes a new core migration (`0076_useful_elektra.sql`). - The plugin database namespace validator is intentionally restrictive; plugin authors may need follow-up affordances for SQL patterns that remain blocked. - Merge this before the LLM Wiki plugin PR so the plugin can resolve the new SDK and host APIs. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled shell/git/GitHub workflow. Context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-05-05 07:42:57 -05:00