mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-10 12:07:09 +02:00
51bb41c7e334440566ecf63cfc96fc36da05d7d2
395
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b4a7a12985 |
feat: make recovery updates quieter (#10542)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The recovery subsystem restores work after an agent run stops or loses state > - Recovery notices currently use the same visual weight as normal work comments > - Recovery agents can also post long narratives that obscure the useful hand-off > - The server must identify recovery output because agents cannot set presentation controls > - This pull request adds compact recovery notices, structured action references, and brief recovery prompts > - The benefit is a quieter issue thread that still keeps recovery state inspectable ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: `server/`, `packages/shared`, and `packages/adapter-utils`. **Problem or motivation** Recovery notices and recovery-run comments can dominate an issue thread. Operators must scan routine recovery narration before they find the work hand-off. **Proposed solution** Give routine recovery output a compact system-notice presentation. Derive the presentation on the server so agents cannot hide arbitrary comments. Keep the successful missing-state summary fully visible because that comment is the recovery deliverable. **Alternatives considered** The UI could detect recovery text. That approach is fragile and does not provide structured action references. Agents could also set presentation directly, but that would weaken the current board-only security boundary. **Roadmap alignment** This change refines the completed “Self-healing runs & automatic recovery” and “Enforced Outcomes” roadmap areas. It does not add a competing roadmap capability. **Additional context** The scope covers shared comment validation, server recovery notices, agent-comment derivation, and recovery prompt text. No database migration is needed because presentation data already uses JSON. ## What Changed - Add the `compact` issue-comment presentation density to shared constants, types, and validation. - Give recovery escalation, waiting, and in-place notices compact titles and structured recovery-action metadata. - Use recovery-action metadata for notice deduplication, with the legacy text marker as a compatibility fallback. - Derive compact presentation for comments from recovery-scoped runs while preserving the board-only presentation boundary. - Keep successful missing-state recovery summaries fully visible. - Ask recovery participants to record outcomes in `resolutionNote` and keep source-issue comments brief. - Add shared, route, service, and prompt tests for the new behavior and exceptions. ## Verification - `pnpm -r typecheck` - Focused Vitest coverage: 320 tests passed across shared validators, adapter prompts, issue comments, recovery actions, and heartbeat recovery. - Full server phase: 292 files passed, 3,094 tests passed, and 2 tests skipped. - Full UI phase: 386 files passed and 3,182 tests passed. - `pnpm build` - Known master baseline: `cli/src/__tests__/secrets.test.ts` expects `pass`, but the current implementation returns `warn` when strict secret mode is disabled for Postgres. This branch does not change CLI secrets code. ## Risks - Low migration risk. The presentation column is JSON and needs no database migration. - Recovery-run detection depends on the persisted run context snapshot. - Structured metadata becomes the primary deduplication key. The existing body marker remains as a fallback for older comments. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with `gpt-5.6-sol`. The runtime did not expose the context-window size. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
9f7565f4ce |
feat: activate OpenTelemetry spans on the sandbox start path (#10536)
## Thinking Path > - Paperclip moves agent work through sandboxed execution and control-plane services. > - The sandbox start path now has a no-op span seam. > - This change turns that seam on when OTLP export is configured. > - It keeps the default path unchanged when export is off. > - The result is structured startup traces with low-cardinality attributes and explicit parent links. > - The benefit is better observability without changing normal behavior. ## Linked Issues or Issue Description No public GitHub issue exists for this change. ### Problem The sandbox start path has a tracer seam, but it stays a no-op unless the OTLP export path is active. ### Proposed solution Enable the server tracer on sandbox bring-up, open a root span, parent each startup boundary to that root, and keep the export path opt-in behind `OTEL_EXPORTER_OTLP_ENDPOINT`. ### Alternatives considered - Keep the start path as a no-op. I rejected that path because it leaves sandbox start opaque when OTLP export is already configured. - Add broad attributes for commands and paths. I rejected that path because the span allowlist must stay low-cardinality. ### Roadmap alignment This follows the current OTel sandbox-start work and keeps the default path unchanged. ## What Changed - Add a root sandbox startup span and child spans for each named startup boundary. - Keep concurrent bridge spans parented to the root span. - Inject the server tracer through the adapter deps without OpenTelemetry imports in the engine. - Attach host-received provider duration attributes only when the values are finite. - Keep span attributes inside the allowlist and keep command, path, id, and error text out of span data. ## Verification - The pushed branch already passed `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`. - The pushed branch already passed `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/`. - The pushed branch already passed `pnpm --filter @paperclipai/server exec vitest run src/__tests__/environment-execution-target.test.ts src/__tests__/instrumentation.test.ts`. - The pushed branch already passed `pnpm --filter @paperclipai/server exec tsc --noEmit`. ## Risks - OTel export changes trace volume when the endpoint is set. - The allowlist limits trace detail, so new fields need care. - The change stays no-op when OTLP export is off. ## Model Used - OpenAI GPT-5, tool use enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
5ec7ce76e5 |
Upload company import packages as compressed zip uploads (fix large-company imports through Cloud) (#10531)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company Import (#10507, hardened in #10523) lets a user upload a company package on the Import page > - The page expanded the user's `.zip` into a files map and POSTed it as ONE inline JSON body — ~40MB for a real company because attachment blobs get base64-inflated > - On Paperclip Cloud that body travels browser → harness proxy → tenant, where it truncated in transit → body-parser 400 → the browser saw "Failed to fetch", and nothing imported > - Two compounding causes: the giant inline body itself, and the board async opt-in riding an `x-paperclip-cloud-*` header that the Cloud harness strips as anti-spoofing (so async never engaged and the import held one fragile synchronous connection) > - This pull request uploads the raw compressed `.zip` as a multipart request (about a third the size, already compressed) parsed server-side into the same bundle the importer consumes, and moves the async opt-in to a proxy-safe `?async=1` > - The benefit is that a large-company import actually completes through Cloud: a small compressed upload, a real async job that survives dropped connections ## Linked Issues or Issue Description - Refs #10507 / #10523 (Import/Export and its hardening). No open issue; problem described above (large-company browser import through a proxy: inline JSON body truncates → 400 → "Failed to fetch"; async opt-in header stripped by the front door → async never engages). ## What Changed - **Multipart zip transport.** The Import page uploads the raw `File` as `multipart/form-data` (field `package`, import options in a JSON `meta` field); the server unzips it into `{ rootPath, files }` and runs the exact existing preview/import logic. The `application/json` inline path is byte-identical for CLI/programmatic callers. Bare `application/zip` (meta via `?meta=`) is also accepted for programmatic use. - **Shared node zip reader.** `packages/shared/src/portability-zip.ts` (node-only subpath, not re-exported to the browser bundle — same pattern as `portability-hash.ts`); the CLI's `zip.ts` becomes a thin re-export. Identical codec (STORE + DEFLATE via `inflateRawSync`, rejects data descriptors/zip64). - **Proxy-safe async signal.** `wantsAsyncImport` = `?async=1` (board browsers, survives the harness) OR the existing `x-paperclip-cloud-async-import` header (cloud tenants, set server-side). The UI async client now uses `?async=1`. Backward compatible. - **Size + preflight.** New `PORTABLE_ZIP_UPLOAD_LIMIT_BYTES = 128MB`; the inline 56MB preflight no longer gates the zip path (it shows the compressed size instead). Async submit/poll/resume, the duplicate-guard fingerprint (now over the resolved bundle), pause-on-import, progress/error panels, and activation all apply to the multipart path. - OpenAPI documents json + multipart + zip bodies and the `async` query param. ## Verification - Full typecheck chain (shared, server, ui, cli) clean. - 152 tests across 8 files: new `portability-zip.test.ts` (STORE/DEFLATE/base64-blob byte-exact round-trip, truncation throws, data-descriptor rejection); `company-portability-routes.test.ts` +7 (multipart import+preview equals the inline bundle; async multipart 202→poll→success; board async via `?async=1` with no cloud header; cloud-tenant async via header; sync fallback with neither; truncated-zip 400, nothing imported); `CompanyImport.test.tsx` asserts the local zip sends the raw File and the inline preflight no longer blocks; `openapi-routes.test.ts` green. - NOT yet measured: the end-to-end browser upload through the live Cloud harness — verified on staging after deploy before closing out. ## Risks - Import semantics unchanged — only transport changed; the JSON inline path is byte-identical, the cloud-tenant header async path untouched. Multipart parsing is server-side (memory-bound: a ~13MB zip → ~30MB files map, fine on the server). - The bare `application/zip` path is programmatic-only and covered by content-type dispatch but not a dedicated route test (the multipart path is). ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI, extended thinking + tool use; root-caused against live logs/DB and the harness proxy source. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
276ae3a75d |
Harden company import: durable UI, async jobs, integrity guard, batched inserts (#10523)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company Import/Export (#10507) moves whole companies between instances as portability bundles > - Real-world use on a large company (1,418 issues, ~10.6k comments) surfaced a cluster of related failures: the import took hours and the browser connection died while the server kept running, a retry silently produced a second partial import, the progress/error UI gave no durable signal, and a cloud-tenant user couldn't even open the companies afterward > - Root cause of the slowness: importBundle inserted every issue, comment, and document as a separate round-trip to a network Postgres — an N+1-over-network pattern > - This pull request hardens the whole import path: durable progress/error UI, an async server-side job so imports survive dropped connections (with a duplicate-submit guard), a fail-closed guard against incomplete payloads, and batched inserts that cut a large import from hours to minutes > - The benefit is that migrating a real, large company actually completes, is legible while it runs, and can't half-import twice ## Linked Issues or Issue Description - Refs #10507 (the Import/Export feature this hardens). Supersedes #10513 (the progress/error-UI piece, folded in here). No open issue; problem described above (large-company import: slow, connection-fragile, silently duplicable, opaque UI). ## What Changed - **Batched inserts (perf):** importBundle pre-generates entity ids in JS and inserts in chunked multi-row statements, so children no longer wait on parents' generated ids. A 1,418-issue import drops from ~15,600 insert statements to **82** (190×); benchmark below. Import semantics — collision handling, pause-on-import, label/blocker/monitor/attachment/embedded-asset handling, blob sha verification — are unchanged (full portability suite green). - **Async import jobs for board sessions:** the existing cloud-tenant async job path opens to board sessions with per-actor job keys; the import page submits, polls, and resumes watching after a reload or dropped connection instead of holding one fragile request. A non-terminal job blocks a duplicate submit (409 returns the running job), preventing the double-import. - **Fail-closed completeness guard:** an optional `expectedFileCount` on inline imports; the server rejects (422 `import_payload_incomplete`) a body carrying fewer files than declared, so a re-framed/short payload fails loudly instead of half-importing. - **Durable progress/error UI (was #10513):** persistent progress panels with size-aware copy, persistent error panels with retry guidance, and inline explanation when the preview button is disabled; request-lifecycle guards so stale previews/imports can't publish or detach. ## Verification - `pnpm -r` typechecks (shared, server, ui) clean. - `company-portability.test.ts` (76) + `company-portability-routes.test.ts` (30) green — the import correctness net — plus new `CompanyImport.test.tsx` async/resume/409 coverage and a new batching regression test (a 50-issue import issues <50 issue-insert statements; rows land unchanged). - **Batching benchmark (embedded Postgres):** at 1,418 issues × 7 comments × 1 doc — 82 insert statements vs ~15,598 one-per-row (190×), ~1s wall-clock; a row-verifying run at that scale imports all 1,418 issues / 9,926 comments / 1,418 documents with unique identifiers and no warnings (no rows dropped by chunking). Over a network DB the round-trip reduction is the hours→minutes lever. - What is NOT directly measured here: wall-clock against a real network Postgres (that happens on a staging deploy); the local timing is network-free. ## Risks - Batching is the load-bearing change: it rewrites the import write path. Mitigated by the unchanged 106-test correctness suite, a new scale/row-integrity test, and per-writer transactions (a failure rolls back its table group; not a single outer transaction across writers — noted, correctness preserved). - Async jobs are in-memory (lost on server restart → pollers 404 and can resubmit); matches the pre-existing cloud-tenant job semantics. - `expectedFileCount` is optional (older callers unaffected); over-count is allowed, only under-count fails closed. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI, extended thinking + tool use; implementation across Fable 5 subagents with live diagnosis against a running instance. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
fcf66f3a91 |
feat(skills): add managed skill rename API (#9688)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Company skills are reusable capabilities that operators install, edit, assign, and materialize for agents. > - Managed local skills currently lack a safe backend operation for changing their display name and canonical slug/key together. > - Treating rename as an ordinary save can leave duplicate records, stale runtime materializations, or agent assignments pointing at the old key. > - This pull request adds a company-scoped managed-skill rename contract, service operation, and REST endpoint with focused authorization and activity logging. > - The benefit is an atomic-enough, recoverable rename path that keeps disk state, database identity, and agent skill assignments synchronized. ## Linked Issues or Issue Description - Refs #2121 - Problem: managed company skills need a dedicated rename operation rather than save-time duplication behavior. - Expected behavior: renaming a managed skill updates its name, slug, key, source directory, frontmatter, runtime materialization, and assigned-agent references while preserving version pins. ## What Changed - Added shared request/result types and Zod validation for managed skill rename requests. - Added `POST /api/companies/:companyId/skills/:skillId/rename` with `skills.edit` policy checks and `company.skill_renamed` activity logging. - Restricted renames to Paperclip-managed local skills and added slug, key, and target-directory conflict handling. - Moved the managed directory, rewrote only the `SKILL.md` frontmatter name, updated the database row, and rolled filesystem changes back when persistence fails. - Rewrote assigned agents' desired-skill keys while preserving pinned version IDs and removed stale runtime materialization. - Added focused route and service coverage for success, no-op, name-only changes, conflicts, unsupported sources, assignment rewrites, rollback-sensitive behavior, and runtime cleanup. - Rejected multiline rename names before they can inject extra `SKILL.md` frontmatter fields. ## Verification - `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts` — 106 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. ## Risks - Filesystem and database updates cannot share one native transaction; the service stages filesystem changes and explicitly restores the original directory and markdown when the database transaction fails. - Renames intentionally reject catalog, remote, project-scanned, and unmanaged local skills to avoid changing identities owned by external sources. - No database migration is required; the endpoint updates existing company-skill and agent configuration fields. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent (exact underlying model ID and context-window size were not exposed to this runtime), with reasoning, repository tool use, code execution, and test execution. The rescued source commit also records assistance from Claude Opus 4.8. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
916c13501f |
Replace host-to-host Cloud Sync with full-fidelity company Import/Export (#10507)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A company accumulates real state — issues, labels, blockers, documents, work products, monitors, attachments, agents, routines — and people need to move that state between instances: self-hosted to cloud, cloud back to self-hosted, or plain backups > - The experimental, flag-gated Cloud Sync transport (#6548) tried to solve this host-to-host: the source pushed into a receiver over HTTPS with a cross-instance consent/token handshake, which required the destination to be publicly reachable and broke for common self-hosted topologies (plain-HTTP LAN/VPN origins); the receiver half never landed upstream at all > - Meanwhile the portability bundle and the existing export/import pages already move companies offline with none of those networking constraints — but silently dropped labels, blockers, issue documents, work products, monitors, and every attachment > - This pull request removes the host-to-host transport and makes Import/Export the single data-movement path: the pages become first-class company-settings destinations, exports declare exactly what they do not carry, and bundle schemaVersion 6 now carries all of the above, with attachments as content-addressed sha256 blobs verified before a single row is written > - The benefit is a migration and backup flow that works between any two instances with no reachability requirements, no cross-instance auth, and no silent data loss ## Linked Issues or Issue Description - Refs #6548 — the original Cloud Sync sender this PR supersedes and removes. - Related, not duplicates: #1697 (goals in the portability manifest — orthogonal field addition), #954 (an earlier import/export + skill-visibility proposal predating the current portability bundle). - No open issue describes this directly, so in brief (feature-request shape): **Problem** — moving a company between instances silently lost labels (imports with label references actually hard-failed), blocker relations, issue documents, work products, monitor state, and all attachments, and the alternative Cloud Sync transport required the destination to be publicly reachable over HTTPS plus a consent handshake, which failed for typical self-hosted setups. **Desired behavior** — one Import/Export flow in company settings that produces a portable bundle carrying all of that data, tells the operator up front what it cannot carry, imports with automations paused, and offers real one-click activation afterwards. ## What Changed - New export fidelity report (`GET /api/companies/:companyId/export/fidelity`) + an "Export fidelity" panel on the Export page listing anything a bundle will not include (now only: approvals, cost history, activity history) - Imports accept `pauseAutomations`; imported agents and routines land paused, the import result reports created routines, and the Import page ends in an activation panel that actually resumes selected agents/activates routines - Export and Import pages promoted into the company-settings nav; the Cloud Upstream wizard, ux-lab page, and API client removed; the old settings route redirects to Export - Host-to-host transport removed: upstream-sync/receiver-client routes and services, CLI `cloud connect`/`cloud push` + keypair store, the shared upstream transfer contract, and the `enableCloudSync` flag; migration `0196` drops the two experimental `cloud_upstream_*` sender tables - Bundle schemaVersion 6: labels (definitions + per-task names, remapped by name on import), blocker relations (`blockedBy` slugs, cycle-tolerant), issue documents (`tasks/<slug>/documents/<key>.md`), work products (system refs nulled), monitors (notes/scheduledBy restored, imported un-armed) - Attachments travel as content-addressed `blobs/<sha256>` entries (deduped; comment-scoped attachments re-link via comment index); every blob is hash-verified **before any write**, so a corrupted bundle cannot leave a partially imported company; both zip codecs now round-trip extensionless/binary entries byte-exactly; the Import page preflights the inline body limit and offers continue-without-attachments - v5 (and older) bundles still import, with an informational warning; bundles newer than v6 are rejected cleanly - Docs: board-operator import/export guide, CLI README, README/ROADMAP updated ## Verification - `pnpm -r` typechecks (shared, db incl. migration numbering/safety checks, server, ui, cli) and `pnpm check:token-gates` — clean - Vitest: full server + shared sweep 4,888 passed / 1 skipped, with the only 3 failures being pre-existing on `master` (2× heartbeat-workspace-branch-containment, 1× workspace-runtime auto-port; reproduced identically with this change stashed); ui + cli suites green; the embedded-Postgres export-fidelity suite applies the full migration chain including the new `0196` against a fresh database - Live end-to-end on a scratch instance: seeded a company with labels, a blocker pair, an issue document, a work product, a monitor, an agent, a routine, and two binary attachments (one comment-scoped) → export → import into a fresh company → labels remapped to new ids, blocker edge and document restored, monitor un-armed with notes intact, attachments byte-identical (sha256-compared through the API), agents/routines paused → activation panel resumed them; a v5-shaped bundle imported with only the info warning; flipping one byte in a blob made the import 422 with **zero** rows created - Reviewer repro: create a company with a labeled issue + attachment → Settings → Export → download → Settings → Import on another company/instance → watch the preview, apply with "start paused", then activate ## Risks - Migration `0196` drops `cloud_upstream_connections`/`cloud_upstream_runs` — experimental tables behind a default-off flag; their connection/run history is intentionally discarded - Breaking removals are all of experimental, flag-gated surface: `/api/upstream-sync/*` + `/api/cloud-upstreams/*` routes, `paperclipai cloud connect|push`, and the `enableCloudSync` flag (stale keys in stored instance settings parse harmlessly) - Import remains non-atomic on mid-apply errors generally (pre-existing behavior); the new blob verification specifically moved ahead of all writes so tampered bundles cannot create partial state - GitHub-sourced imports do not fetch `blobs/*` and skip attachments with a warning ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), via Claude Code CLI with extended thinking, tool use, and subagent orchestration; implementation and review split across Fable 5 subagents, with live end-to-end verification against a running instance ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Nicky LeachandPaperclip
|
15ce70dc18 |
feat(server): thread plural referenced-project workspaces through run prep (#10448)
## Thinking Path > - Paperclip coordinates work for autonomous companies. > - A run needs a workspace view before execution starts. > - That view now needs to cover one anchor project and more referenced projects. > - Those extra workspaces must stay separate and must not change the anchor path when the feature stays off. > - This PR threads plural workspace data through run prep behind a default-off kill switch. > - It also keeps extra project workspaces isolated and makes the realization contract round-trip the new shape. > - The benefit is safer run prep for referenced projects without changing the current default path. ## Linked Issues or Issue Description This PR does not link a public GitHub issue. It follows the internal run-prep task for plural referenced-project workspaces. Problem: - Run prep resolves the anchor project today, but it does not yet carry each referenced project into the run workspace view. - That gap blocks runs that need a second repo or sibling project during preparation. Proposed solution: - Thread a plural workspace result through run prep. - Keep the anchor path unchanged when the kill switch is off. - Resolve each referenced project into its own managed checkout directory when the flag is on. Alternatives considered: - Keep one shared workspace and layer the extra repos into it. Rejected because it would blur isolation and make failures harder to bound. - Upload the extra workspaces immediately. Rejected because this PR only prepares the data path. Roadmap alignment: - This change sits in the workspace and sandbox path. - It matches the roadmap work on workspace strategy and cloud or sandbox agents. ## What Changed - Added `additionalWorkspaces[]` to the run workspace result. - Split workspace resolution into an anchor path and an optional referenced-project path behind `PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC`. - Kept per-project failure isolation so one bad clone does not stop the run. - Keyed managed workspace directories by `projectId` so sibling workspaces stay separate. - Added `additionalSources[]` to the workspace realization request and kept read and write paths backward compatible. - Added tests for the anchor-only path, the new workspace shape, and the per-project directory rule. ## Verification - `pnpm --filter @paperclipai/server run typecheck` - `pnpm --filter @paperclipai/shared run typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-project-env.test.ts` 21/21 - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/workspace-runtime.test.ts` 98/98 ## Risks Low risk. The new path stays behind a default-off kill switch, so the anchor flow does not change when the flag is off. The main risk is a bad referenced project clone. That case now drops only the affected project and keeps the run alive. The shared type change also needs every consumer to use the new array field where extra workspaces matter. ## Model Used OpenAI Codex (GPT-5; exact internal model ID not exposed in this environment; tool use enabled) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
c274f10abc |
feat(server): computed owner instance-admin elevation for cloud-managed instances, behind platform floors (#10343)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Cloud-managed instances authenticate tenant users through a trusted-header path (`resolveCloudTenantActor`) that deliberately never grants `instance_admin`, so every tenant user is company-scoped > - On a dedicated (single-owner) managed instance that leaves the paying owner unable to administer their own instance: instance settings, the environments admin surface, and the custom sandbox image flow are all instance-admin gated (the environments UI can't even show the provider/image of the platform sandbox because the restricted read view blanks `config` entirely) > - Re-granting the old blanket `instance_user_roles` row would repeat the mistake the shared-pool hardening fixed: DB rows go stale, resurrect via restores, and elevate through every auth path > - This pull request elevates only the stack `owner`, computed per request at the trusted-header boundary behind a new managed-tier feature flag, and ships that elevation together with code floors on the platform-owned surfaces an instance admin must not control on a managed instance > - The benefit is that dedicated-stack owners can administer their own instance while platform credentials, execution policy, backups, and runtime code-install stay platform-owned, and self-hosted behavior is unchanged ## Linked Issues or Issue Description No public GitHub issue exists for this change; the underlying issue is described here following the feature-request template. ### Problem or motivation - On cloud-managed instances, tenant users resolved from trusted headers are always company-scoped. For dedicated instances with a single paying owner, the owner cannot reach any instance-admin surface of their own instance (instance settings, environments administration, custom image setup), and the restricted environment read view hides even structural fields like the sandbox provider and image. - The previous hardening intentionally removed blanket elevation (and purges stale `instance_user_roles` rows on every trusted-header authentication). That protection must not regress for shared multi-tenant pools. ### Proposed solution - Owner-only, computed, flag-gated elevation plus code floors on platform-owned surfaces, in one PR so the elevation can never ship without the floors. ### Alternatives considered - Re-inserting an `instance_user_roles` row for owners (the pre-hardening model): rejected — DB rows go stale, survive restores, and elevate through every auth path; #7525 removed exactly this. - Widening only the environments read view without any elevation: rejected — it fixes one screen but still leaves a dedicated-instance owner unable to administer instance settings or custom images. - Elevating additional stack roles (`member`/`admin`/`support`): rejected — only the owner has an ownership claim over the whole instance; other roles stay company-scoped. ### Roadmap alignment - Extends the shipped "Cloud deployments" roadmap work (multi-tenant isolation, company-scoped cloud tenants, managed-instance bootstrap) without overlapping planned core items, and leaves self-hosted behavior unchanged. ## What Changed - **New feature key** `enableOwnerInstanceAdmin` (`packages/shared`): boolean flag in `instanceExperimentalSettingsSchema`, catalog tier `managed`, `cloudDefault: true`, `selfHostedDefault: false`. Inert on self-hosted instances — the elevation path only exists behind the cloud tenant trust token. - **Computed elevation** (`server/src/middleware/auth.ts`): `resolveCloudTenantActor` now returns `isInstanceAdmin: true` only when the trusted-header stack role is `owner` **and** the flag is enabled. The flag is resolved through the instance-settings service so the managed-config overlay applies (the control plane can disable elevation fleet-wide without touching tenant databases; a DB row edit or restore cannot resurrect it). Resolution fails closed on settings read errors. The `instance_user_roles` never-insert and the per-request stale-row purge are byte-identical. `member`/`admin`/`support` stack roles stay company-scoped. - **Authorization guard split** (`server/src/services/authorization.ts`): the blanket-allow now trusts the actor's *computed* `isInstanceAdmin` flag (only the attested resolver can set it for `cloud_tenant` actors) while keeping the `instance_user_roles` DB lookup excluded for `cloud_tenant` — a stale or hand-inserted role row still elevates nothing. - **Floor F1 — platform environment credentials** (`server/src/routes/environments.ts`): on cloud-managed instances, platform-provisioned environment rows (`managedByPaperclip` marker, plus the legacy managed-Kubernetes marker) use a single floored view for every reader on all environment routes (list, get, create, update, delete responses): `envVars` are never echoed and credential-shaped `config` keys (reusing the managed-config `SECRET_LIKE_CONFIG_KEY_PATTERN`) are dropped — for **all** actors including instance admins — while structural config (provider, image, template, region, …) and the managed markers stay visible. This also fixes the environments UI for managed sandboxes, which previously lost the provider/image entirely in the restricted view. The floor also covers writes: `PATCH /environments/:id` and `DELETE /environments/:id` on a platform-provisioned row are rejected (403, `environment_platform_managed`) for every actor including instance admins, and the guard binds to the persisted row's markers so a patch cannot strip the managed marker to lift the floor. The one recovery path is a metadata-only PATCH that solely clears the marker keys (null/false), for rows stamped through the old unrestricted API before the markers became reserved — and it never applies to a row whose slot markers are live platform state: the single local row (`environments_local_driver_idx`), which `ensureLocalEnvironment` adopts and stamps on cloud-managed instances from every caller (company creation, the heartbeat, run orchestration), and the single marked sandbox row (`environments_managed_sandbox_idx`) while a managed-sandbox bootstrap path is configured (managed-config `environments` section or `PAPERCLIP_EXECUTION_MODE=kubernetes`) and the provisioner therefore adopts and refreshes it on every boot. Clearing a live slot row's markers would let the next write reclassify it as tenant-managed and bypass the floor; conversely, when no sandbox provisioning path is configured the platform holds no claim on any sandbox row, so a platform marker there is stale by definition and the recovery patch applies. Every marker outside a live slot is clearable, so no legacy row is ever locked permanently. Custom-image setup and probes on the platform sandbox stay available to instance admins — those are the owner-facing flows this elevation exists for. The marker keys themselves are reserved: client create/update payloads that set `managedByPaperclip` or `managedKubernetesSandbox` are rejected (422, `environment_platform_marker_reserved`) on cloud-managed instances, so a tenant row can never be stamped platform-provisioned through the API and self-locked behind the write floor (the provisioner writes markers at the service layer, not through these routes). Tenant-created environments are otherwise unaffected. - **Floor F2 — executionMode** (`server/src/routes/instance-settings.ts`): on cloud-managed instances, `PATCH /instance/settings/general` rejects writes that would change `executionMode` (403, `execution_mode_platform_managed`). Same-value echoes pass so settings forms that submit the full general-settings object keep working. The boot-time execution-policy bootstrap path is untouched (it calls the service directly). - **Floor F3 — manual database backups** (`server/src/routes/instance-database-backups.ts`): floored off on cloud-managed instances (403, `database_backups_platform_managed`); backups are platform-owned there, and the result would also echo a server-side filesystem path. - **Floor F4 — adapter code install** (`server/src/routes/adapters.ts`): `POST /adapters/install` and `POST /adapters/:type/reinstall` are floored off on cloud-managed instances (403, `adapter_install_platform_managed`). Adapter packages execute in the server process, so a runtime install would let an instance admin read the platform trust anchors out of the process environment. This mirrors the existing bundled-only plugin install floor; adapter code on managed instances comes bundled with the platform image. ## Instance-admin surface audit Before widening who can hold `isInstanceAdmin`, every instance-admin-gated surface in `server/src` was enumerated and reviewed for whether its response or side effects could echo process environment values or platform credentials (tenant trust token, JWT signing keys, database connection strings, provider API keys): 29 distinct gate definitions covering ~90+ call sites, in four groups — sole instance-admin gates (12), instance-admin-or-company-permission gates (10), response-shaping/scope-widening sites (6), and the central `allow_instance_admin` short-circuit in the authorization service (58 `decide()` call sites). Findings and dispositions: - **Environment read/write responses** exposed platform sandbox `envVars`/credential-shaped config to instance admins → closed by floor F1. - **Manual backup trigger** echoed a server filesystem path and triggers a platform-owned operation → closed by floor F3. - **Adapter install/reinstall** loads externally fetched code into the server process (indirect, complete env exposure) → closed by floor F4. The sibling plugin-install path already had a bundled-only floor on managed instances and needed no change. - **Token-minting surfaces** (gateway tokens, custom-image terminal/connection tokens) mint credentials scoped to the instance's own resources, not platform trust anchors → acceptable for an owner-admin of a dedicated instance; unchanged. - All remaining gated surfaces return ordinary instance-scoped business data; none echo `process.env` or platform secrets directly. OAuth client secrets are referenced by env-var *name* only; SSH private keys are stored as secret refs before persistence and are not echoed. Operational note for managed platforms: this model assumes the process environment of a managed instance holds only that instance's own credentials. Platform operators should keep provider credentials per-instance (never fleet-shared) since an instance admin ultimately controls in-process code on their own instance. ## Verification - `pnpm vitest run server/src/middleware/cloud-tenant-actor.test.ts` — resolver matrix: owner × flag on/off, flag via managed overlay (on-over-DB-off and off-over-DB-on), member/admin/support × flag on, no-token self-hosted, fail-closed settings read, purge still runs and no role row is ever inserted (14 tests). - `pnpm vitest run server/src/__tests__/authorization-service.test.ts` — computed flag elevates a `cloud_tenant` actor; a stale `instance_user_roles` row still never does; `session` actors unchanged (full suite, embedded Postgres). - `pnpm vitest run server/src/__tests__/environment-routes.test.ts` — F1: no secret echo to admins on get/list, structural config visible to restricted readers, platform-row PATCH/DELETE rejected for admins (including a marker-stripping patch), marker-clear recovery allowed for stale legacy rows and for a marked sandbox row when no provisioning path is configured, but refused on the managed local row and on the sandbox slot row under a managed-config `environments` entry or the forced kubernetes execution mode, client marker-stamping creates/patches rejected, tenant rows still readable and writable, self-hosted read+write regression (60 tests). - `pnpm vitest run server/src/__tests__/environment-service.test.ts` — `ensureLocalEnvironment` adopts a pre-existing local row on cloud-managed instances (marker stamped, other metadata preserved, idempotent — no rewrite on re-ensure) and leaves self-hosted rows untouched (22 tests, embedded Postgres). - `pnpm vitest run server/src/__tests__/instance-settings-routes.test.ts server/src/__tests__/instance-database-backups-routes.test.ts` — F2 change-vs-echo matrix incl. self-hosted regression; F3 floor for both admin shapes (32 tests). - `pnpm vitest run server/src/__tests__/adapter-routes-authz.test.ts` — F4 floor; self-hosted install/reinstall behavior unchanged (existing cases). - `pnpm vitest run server/src/__tests__/first-admin-claim.test.ts server/src/__tests__/bootstrap-claim-routes.test.ts server/src/__tests__/managed-config.test.ts server/src/__tests__/health.test.ts server/src/__tests__/instance-settings-managed-overlay.test.ts server/src/services/managed-environments.test.ts server/src/services/execution-policy-bootstrap.test.ts` — first-admin bootstrap gate and managed-config behavior unchanged (91 tests). - `pnpm vitest run packages/shared/src/feature-catalog.test.ts` — catalog/schema sync tests cover the new key (selfHostedDefault must equal the schema default). - `pnpm run typecheck` — all 31 workspace projects clean. ## Risks - Self-hosted behavior is unchanged: every floor binds to `isCloudManagedInstance()` (tenant trust token present), the new flag defaults off with no elevation path, and regression tests pin the self-hosted branches. - The elevation is fail-closed and stateless: turning the flag off (managed overlay or DB) de-elevates on the next request; there is no role row to clean up and restores cannot resurrect elevation. - On a cloud-managed instance a pre-existing unmarked local row is adopted (stamped `managedByPaperclip`) by the next ensure and becomes platform-owned — the intended managed-product semantic: the platform owns the single local slot. Self-hosted instances are untouched. - F1 widens restricted readers' view of platform-provisioned rows from fully blanked `config`/`metadata` to structural-only `config` plus markers. Platform-delivered config is guaranteed secret-free by the managed-config contract (secret-shaped keys fail startup), and the floor re-drops secret-shaped keys defensively. - One extra instance-settings read per trusted-header request for owner-role actors (the resolver already performs several queries per request). ## Model Used Claude Fable 5 (Anthropic) — model id `claude-fable-5`, extended thinking enabled, agentic tool use via Claude Code; read-only explore subagents on the same model were used for the surface audit sweep. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c3bd0c5d50 |
feat(skills): add beta releases for the core Paperclip skill (#10228)
## Thinking Path > - Paperclip is the open source control plane people use to organize and operate AI-agent companies. > - Agent behavior depends partly on the bundled Paperclip core skill synchronized into each runtime. > - The existing database and runtime plumbing already supports immutable skill-version snapshots and per-agent version selections, but no product workflow exposed that capability. > - Replacing the live bundled skill globally would make champion adoption risky and difficult to compare across agents. > - This pull request adds an experimental, instance-level beta-skills gate plus a repository release registry, immutable seeded releases, enforcement, and a per-agent release picker. > - The benefit is controlled per-agent evaluation of frozen core-skill releases while the default-off path remains behaviorally unchanged. ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting: `server/`, `ui/`, `packages/db`, and `packages/shared`. ### Problem or motivation Paperclip needs a safe way to evaluate improved versions of its core operating skill without globally replacing the live default. Today the version-snapshot and per-agent pin plumbing exists, but operators cannot use it. A global replacement would make regressions difficult to contain and would prevent controlled comparisons across agents. ### Proposed solution Add a default-off instance experiment that exposes immutable, named core-skill releases. When enabled, operators can pin each agent to a seeded release; when disabled, every agent resolves the live default while saved pins remain intact. Validate pinned writes at the API boundary, gate reads at runtime, and expose the selection in the agent Skills tab. ### Alternatives considered - **Replace the bundled core skill globally:** rejected because it changes every agent at once and provides no rollback/isolation boundary. - **Ship releases as separate skills:** rejected because releases are versions of one core capability, not independently enabled skills. - **Store release snapshots only outside the repository:** rejected because repository provenance and hashes make builds reproducible and reviewable. ### Roadmap alignment This extends the Skills Manager / Skill Studio direction in `ROADMAP.md` by making core-skill versions operable per agent. It does not duplicate another open implementation PR; GitHub searches found no related `enableBetaSkills` change. ### Additional context The feature remains experimental and default off. The V7 champion was selected through a multi-model evaluation process, and the frozen release contents are verified by SHA-256 below. ## What Changed - Added the default-off instance-level `enableBetaSkills` experimental flag. - Added `skills-releases/paperclip/` with the ordered release registry and frozen `v0` plus `v7-roster` snapshots. - Added release metadata to `company_skill_versions` and idempotent release seeding. The migration was planned as `0191`, then renumbered to `0192` because current `master` claimed `0191` before final rebase. - Added read-time gating and write-time validation so disabled instances always resolve the live default and reject pinned-version writes. - Added the per-agent Release picker in the agent Skills tab, including responsive layout and beta-pin state. - Kept `EDITS.md` out of the release registry and PR diff. ### V7 Adoption Evidence - Paid roster: 6 models, 94-case suite. - Result: 553/564 pass-within-2, mean 92.17/94, versus the P2 baseline of 544/564. - Reference model improved 84→91; maximin improved 84→90. - Final report: https://pages.paperclip.ing/skills/optimization/paperclip/pap-14624-p3-final-20260721/ ### Provenance - `v7-roster` is the Phase 1 champion plus additions-only edits E107–E112. Per-edit rationale remains in the evals repository at `source/v7-roster/EDITS.md` and is deliberately excluded from this PR. - `v0` is the `skills/paperclip` tree from commit `ea66ea81`. - Champion selection was accepted on July 21, 2026 via board card `9c304fc2` (PAP-14624 G3). - This delivery mechanism was accepted on July 24, 2026 via plan revision `2367abd2` (PAP-14858). ### QA Evidence - P4 QA matrix comment `b7f40522-4e9b-4a3a-9821-28e86fe1a987`: all 6 acceptance criteria passed. - Automated QA matrix: 166 tests passed with 0 failures, including real filesystem materialization and full SHA-256 assertions. - UI QA exercised the real agent Skills tab at desktop and mobile widths with the experimental flag both on and off. ## Verification - `pnpm check:token-gates` - Focused beta-release matrix: 169 tests passed across shared validators, server services/routes/heartbeat behavior, instance settings UI, and release picker UI. - `pnpm -r typecheck` - `pnpm build` - `pnpm test:run`: server and UI partitions passed; one CLI doctor test inherited temporary AWS credentials from the agent heartbeat and expected no static credentials. The isolated rerun with `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` unset passed 8/8. - V7 SHA-256: - `SKILL.md`: `53ab290489684cbf116fdd1406a95f6b6f53c9c36358b1bf8bfeae481e253575` - `references/cases.md`: `3b821f59064a7761091020a14819a8d787131f24029748563d6c0e1be7e6eaec` - `references/workflows.md`: `69747bd6e05f7e3673d1e67b07ff295df1869c05e1fd029804d5fa9177db92cd` - Confirmed 49 changed files, no `pnpm-lock.yaml`, no workflow changes, and no `EDITS.md`. ## Risks - **Migration:** low-to-moderate risk. Three nullable columns and one partial unique index are added idempotently; existing rows remain valid. - **Behavior:** low risk while the flag is off because read-time resolution forces the live default and saved pins are preserved but inactive. - **Frozen content:** release snapshots intentionally diverge from future live skill edits; provenance and hashes make that divergence explicit and reproducible. - **UI:** low risk. The picker only renders for the bundled core skill when the experimental flag is enabled and seeded releases exist. > This extends the existing Skills Manager / Skill Studio direction described in `ROADMAP.md`; it does not duplicate another open implementation PR. The GitHub PR search found no related `enableBetaSkills` change. ## Model Used - OpenAI Codex using `gpt-5.5` with reasoning and terminal/code-execution tools; context-window size is not exposed by this runtime. Earlier implementation commits also record Claude Opus 4.8 assistance where applicable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [ ] I have not referenced internal/instance-local Paperclip issues or links (required governance identifiers are included above; no internal URL is included) - [ ] My branch name describes the change and contains no internal Paperclip ticket id (the approved delivery plan mandated this shared branch name) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c111ee4cb3 |
feat(server): add per-user document stars (#9952)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work. > - Artifacts and documents are first-class outputs, but users need a personal way to keep important documents easy to find. > - Existing resource memberships already model per-user starred projects and agents with company scoping and activity logging. > - Documents lacked the equivalent membership model, route, and artifact filtering behavior. > - The shared membership contract also needs to remain safe for existing UI project/agent mutation helpers when documents become a recognized resource type. > - This pull request extends the existing resource-membership system with per-user document stars and a starred artifacts view. > - The benefit is a company-scoped, idempotent server foundation for a dedicated starred-documents experience without weakening authorization or artifact filtering semantics. ## Linked Issues or Issue Description ### Problem / Motivation Board users cannot star individual documents, and the company artifacts API cannot return only the current user's starred documents. ### Proposed Solution Add company/user-scoped document memberships, a board-only document star route, document membership data in the shared contract, and a `starred=true` artifacts filter. ### Alternatives Considered A document column was rejected because stars are per-user; a separate star API was rejected because projects and agents already use resource memberships. ### Roadmap Alignment This extends the existing Artifacts & Work Products roadmap area and does not duplicate another open pull request found in the repository search. ## What Changed - Added the `document_memberships` schema and migration with company/user/document uniqueness and starred ordering. - Extended shared resource-membership and artifact-query contracts for documents and `starred=true`. - Added company-scoped document star/unstar service and board-only route behavior with activity logging. - Added starred document artifact filtering, including user-authored documents, document kinds, cursor ordering, and incompatible-kind handling. - Preserved idempotency under concurrent star requests and synchronized UI membership defaults/helpers with the expanded contract. - Added focused shared, route, service, and UI regression coverage. ## Verification - `pnpm exec vitest run packages/shared/src/resource-memberships.test.ts server/src/__tests__/company-artifacts-service.test.ts server/src/__tests__/resource-memberships-routes.test.ts` - `pnpm exec vitest run ui/src/components/SidebarAgents.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarStarredProjects.test.tsx` - `pnpm --filter @paperclipai/db check:migrations` - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` ## Risks - The migration adds a new membership table and non-concurrent indexes; migration safety gates pass with the repository's established policy. - The starred artifacts query intentionally returns only documents and relaxes the normal agent-authored/system-kind predicates for documents the current user explicitly starred. - Document membership mutations remain board-user-only; agent callers receive no document-star capability. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI; runtime model ID and context-window size were not exposed to this session. Reasoning, repository tool use, code execution, and test execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
0cf64d36a5 |
feat(secrets): write through external values and deep-link details (#10196)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Its secrets subsystem can resolve external references such as AWS Secrets Manager values without copying those values into Paperclip custody. > - Operators also need to rotate a referenced secret's value while preserving the same provider reference for consumers inside and outside Paperclip. > - Previously, external-reference rotation could only retarget metadata, and secret detail sheets were driven by local component state rather than shareable navigation state. > - This pull request adds an optional provider write capability, implements AWS Secrets Manager write-through rotation, and exposes capability-aware rotate modes in the UI. > - It also makes secret and each-user definition detail sheets URL-driven and adds a copy-link action. > - The benefit is that operators can update the canonical external value safely while keeping AWS rotation tracking intact, and they can share or navigate directly to secret details. ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`server/`, `ui/`, and `packages/shared`). ### Problem or motivation External-reference secrets can follow a provider-managed value, but Paperclip could not write a replacement value back to providers that support it. Operators had to leave Paperclip, update the value separately, and then return without an auditable Paperclip rotation record. Secret detail sheets also could not be shared or restored through browser history because their selection lived only in component state. ### Proposed solution Add an optional `updateExternalSecretValue` provider capability and surface it as `supportsExternalValueWrites`. Implement AWS writes with `PutSecretValue` while leaving the resolution `versionId` unset so future reads continue following `AWSCURRENT`. Add write-value and retarget modes to the rotate dialog for capable providers. Drive secret detail selection from `?secret=` / `?definition=` query parameters and provide a copy-link action. ### Alternatives considered Converting an external reference into a Paperclip-managed secret would break consumers that depend on the existing provider reference. Pinning reads to the newly written AWS version would prevent later out-of-band rotations from flowing through. Keeping sheet selection only in React state would not support browser Back or shareable links. ### Roadmap alignment This extends the completed “Secrets Manager with per-agent access” roadmap capability; it does not duplicate a separate planned roadmap item. Public GitHub searches found no duplicate or closely related issue or PR. ### Additional context The PR includes focused provider, service, and UI render coverage. Cutter also generated previews for the deep-linked detail sheet and capability-aware rotate modes. ## What Changed - Added optional external-value write support to the secret provider contract and provider descriptors. - Implemented AWS Secrets Manager write-through with `PutSecretValue`, audit material, and compensation when persistence fails after the provider write. - Allowed `secretService.rotate()` value updates for external references while rejecting ambiguous value-plus-retarget combinations and unsupported providers. - Added capability-aware “Write new value” and “Change reference” rotate modes with updated custody and action copy. - Made secret and each-user definition detail sheets source their selection from URL query parameters, compose with folder paths, close through browser history, and expose a copy-link action. - Added provider, service, and UI render coverage for write-through, rollback, capability messaging, dialog modes, and deep links. ## Verification - `pnpm vitest run server/src/__tests__/aws-secrets-manager-provider.test.ts` — 18 passed. - `pnpm vitest run server/src/__tests__/secrets-service.test.ts` — 75 passed. - `pnpm vitest run ui/src/pages/Secrets.render.test.tsx` — 31 passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm check:token-gates` — passed with all gates clean. ## Risks - External value writes affect the canonical provider secret and therefore all consumers of that AWS secret; the UI explicitly labels this custody behavior. - A provider write can succeed before Paperclip persistence fails. The service records the written version and AWS support includes compensation coverage to restore the prior value where possible; unrecoverable failures return explicit audit-safe error details. - URL-driven sheet state changes navigation behavior; render tests cover deep links, Back/close behavior, and composition with folder query state. - No database migration or breaking API requirement is introduced; providers without the optional capability retain reference-only behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using exact model ID `gpt-5.6-sol`, high reasoning mode, Codex CLI `0.142.5`, with repository, shell, Git, GitHub CLI, and code-execution tools. The runtime did not expose a context-window size. - Earlier implementation commits were assisted by Anthropic `Claude Fable 5` as recorded in their commit trailers; the exact backend model ID and context-window size were not preserved in the workspace metadata. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes, or confirmed no documentation change is required - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f6ab82d490 |
feat(interactions): add interaction withdrawal and terminal-issue expiry (#10251)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents and boards coordinate through issue-thread interactions (request_confirmation, ask_user_questions, suggest_tasks, …) that wait as `pending` cards until someone resolves them > - Two lifecycle gaps existed: an interaction's creator could not take back a card it no longer stands behind, and interactions left `pending` on issues that reached a terminal status lingered forever as live-looking approval requests > - Stale pending cards mislead humans (they look actionable), distort attention/liveness signals, and in the worst case invite acting on a proposal whose issue is already closed or cancelled > - This pull request adds an explicit withdraw route for pending interactions and automatically expires pending interactions when their issue reaches a terminal status (including a catch-up sweep for issues closed before this change) > - The benefit is that interaction cards now faithfully reflect reality: only genuinely actionable requests stay pending, and creators can retract requests that events have overtaken ## Linked Issues or Issue Description Fixes #5787 Refs #7403 Related prior PRs found while searching for duplicates (all overlap partially; none combine both lifecycle paths or the route-level authorization used here): #6709 and #7312 (creator-withdraw attempts), #8169 (terminal expiry), #8081 and #5137 (generalized cancel/expire endpoints), #6094 (stale confirmation auto-resolve). Related merged context: #9568 (agent cancel for ask_user_questions), #10119 (tolerating legacy `withdrawn_by_creator` result outcomes — the reader side of the outcome this PR writes). ## What Changed - New route `POST /issues/:id/interactions/:interactionId/withdraw` that resolves a `pending` interaction to status `withdrawn` with a structured result (`outcome: "withdrawn"`, optional trimmed `reason`), stamps `resolvedBy*`/`resolvedAt`, touches the issue, logs activity, and emits resolved-interaction telemetry - Withdrawal authorization: board users, the interaction's creator agent, or the issue's current assignee agent (assignees additionally pass the standard issue-mutation gate); task-watchdog runs are explicitly rejected, and authorization-boundary plus low-trust control-plane checks apply - Withdrawing an already-resolved interaction returns `409`; unknown/cross-issue/cross-company interaction ids return `404` - New service method `expirePendingInteractionsForTerminalIssue`: when an issue transitions to a terminal status, all of its `pending` interactions are resolved to `expired` with `outcome: "issue_closed"`, guarded by a `status = 'pending'` predicate so concurrent resolutions are not overwritten - The same expiry runs as a catch-up when interactions are listed on an already-terminal issue, so cards stranded by issues closed before this change also get cleaned up; expired request_confirmations are logged with a distinguishing source - Shared package: new `withdrawIssueThreadInteractionSchema` validator, `WithdrawIssueThreadInteraction` type, and `withdrawn` / `issue_closed` result-outcome support for all interaction kinds (kind-aware result shapes for `ask_user_questions` and `request_item_verdicts`) - UI helper `ui/src/lib/issue-thread-interactions.ts` recognizes the new outcomes for card rendering - Docs: bundled skill API reference updated with the withdraw endpoint - Review follow-up: terminal expiry moved from the HTTP route hooks into `issueService.update`'s status-transition block, so direct service callers (tree control, recovery, pipelines, status cards) expire pending cards too; the list-endpoint catch-up remains for issues closed before this change - Review follow-up: withdrawing or issue-close-expiring a `request_confirmation` also settles its linked `tool_action_requests` row (withdraw -> `cancelled`, issue closed -> `expired`), so a parked tool call cannot stay approvable after its card is gone - Review follow-up: interaction cards render dedicated copy for the new outcomes ("Withdrawn" with the reason, "Expired when issue closed") instead of falling through to superseded-by-comment / stale-target variants; withdrawn plan reviews badge as "Withdrawn" rather than "Changes requested" ## Screenshots Card states rendered from a local ux-lab harness with mocked data ([full gallery](https://pages.paperclip.ing/pr-10251-interaction-withdrawal-cards/)): | Light | Dark | | --- | --- | |  |  | ## Verification - `pnpm --filter @paperclipai/shared build` — clean tsc - `cd server && npx vitest run src/__tests__/issue-thread-interaction-routes.test.ts` — 22 tests pass, including new coverage for: creator-agent withdraw success, non-creator/non-assignee agent 403, watchdog-run 403, double-withdraw 409, and board-user withdraw - `cd server && npx vitest run src/services/issue-thread-interactions.test.ts` — 4 tests pass, including terminal-issue expiry writing `issue_closed` results and leaving already-resolved interactions untouched - `cd ui && pnpm typecheck` — clean - `cd server && npx vitest run src/__tests__/issues-service.test.ts` — includes a new embedded-Postgres test proving a direct `issueService.update` terminal transition expires pending interactions and writes the activity-log entry - `cd ui && npx vitest run src/components/IssueThreadInteractionCard.test.tsx` — 32 tests, including new coverage for withdrawn / issue-closed confirmation and question cards - `cd server && npx tsc --noEmit` — matches the pre-existing repo error baseline exactly (no new errors) - Manual: `POST /issues/:id/interactions/:interactionId/withdraw` with `{"reason":"superseded"}` as the creator agent resolves the card to `withdrawn`; closing an issue with a pending confirmation flips it to `expired` with `outcome: "issue_closed"` ## Risks - Interactions on terminal issues now auto-expire (including retroactively via the list-time catch-up), so consumers that expected to resolve a pending interaction on a closed issue will get `409`; this is the intended semantics and matches how the attention feed already wants to treat dead cards - New result outcomes (`withdrawn`, `issue_closed`) are written to stored results; readers were already made tolerant of these outcome strings in #10119, so mixed-version reads are safe - No schema/migration changes; per-row conditional updates (`status = 'pending'`) avoid clobbering concurrent resolutions - Withdrawal is a new mutation surface, but it is strictly narrower than existing resolve paths (board, creator, or assignee only; watchdog runs blocked) ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic Mythos-class tier) with extended thinking and agentic tool use (Claude Code harness); commit authored in a Paperclip-managed engineering session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4e00818574 |
fix(runtime): support in-place workspace realization (#10230)
## Thinking Path > - Paperclip is the control plane people use to coordinate AI agents and their execution environments. > - Environment realization decides where an agent runs and which filesystem and toolchain are authoritative. > - Copy-based realization is unsafe for container-anchored tasks because absolute paths such as `/app` can point outside the synchronized tree and task-specific binaries may be absent. > - That mismatch can let an agent successfully verify work in a phantom writable path while sync-back silently discards the result. > - Existing task environments already provide the authoritative filesystem and toolchain, so they should be executed in place rather than copied. > - Copy mode still needs explicit confinement rules so aliases target the synchronized workspace and unsynchronized writable paths fail visibly. > - This pull request adds typed realization metadata, propagates the authoritative root through orchestration, and teaches Codex to honor it. > - The benefit is that container-anchored tasks operate on verifier-visible state with the intended tools, while copy mode remains safe and backward compatible. ## Linked Issues or Issue Description No public GitHub issue exists for this defect. GitHub duplicate searches for in-place execution, workspace realization, and authoritative workspace roots found no related pull request to link. ### What happened? Environment-backed agent runs were always realized through a copied workspace. Tasks anchored to absolute container paths could therefore write outside the synchronized tree, and task-provided toolchains were unavailable in the copy. A run could report success even though sync-back discarded its output. ### Expected behavior Existing task environments should run against their real authoritative root and toolchain. Copy-mode runs should map declared absolute aliases into the synchronized tree and reject writable paths that cannot be restored. ### Steps to reproduce 1. Run a Codex task environment whose required files live under `/app` or `/workspace` and whose required binary exists only in the task container. 2. Observe that copy realization changes the effective filesystem/toolchain or permits writes outside the synchronized root. 3. Complete and verify the task inside the agent sandbox. 4. Observe that the verifier cannot see out-of-tree artifacts or that task-specific commands were unavailable. ### Reproduction context - Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924` - Deployment: built from source in a task-container execution environment - Adapter: Codex local - Database: not database-related - Access context: agent execution ## What Changed - Added typed `copy | in_place` workspace-realization metadata, authoritative roots, confined aliases, and outbound restore paths to shared execution-target contracts. - Selected in-place realization for existing task environments and skipped archive prepare/restore when the authoritative environment is used directly. - Propagated the authoritative root into adapter context so Codex uses it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP execution. - Bound copy-mode aliases such as `/app` to the synchronized workspace and rejected writable out-of-tree paths without explicit restore mappings. - Added focused regression coverage while preserving existing copy-mode archive restore behavior. ## Verification - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm exec vitest run packages/adapter-utils/src/local-process-sandbox.test.ts packages/adapters/codex-local/src/server/acp.test.ts packages/adapters/codex-local/src/server/execute.remote.test.ts server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed, 4 skipped. - `pnpm -r typecheck` — passed. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u AWS_SESSION_TOKEN pnpm test:run` — passed across all general and serialized Vitest shards. - `pnpm build` — passed. - Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`, `openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate` each received terminal grade 1.0 against real task-environment paths and toolchains. ## Risks - In-place mode deliberately exposes the authoritative task root to the adapter; incorrect environment metadata could point execution at the wrong root. Typed metadata and focused orchestration tests cover selection and propagation. - Copy-mode writable-path validation is stricter and may reject previously accepted unsafe configurations. The rejection is intentional and produces a visible error instead of silently losing output. - The acceptance run is focused on four Codex task-environment workloads, not a broad cross-adapter benchmark. Existing copy-mode archive tests and the full repository suite remain green. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI coding agent; exact model ID and context-window size were not exposed to this runtime. Capabilities used: extended reasoning, repository editing, shell execution, test/build execution, Git, GitHub CLI, and Paperclip API tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
30ff3d7c58 |
feat(routines): expose activity gate API (#9438)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Scheduled routines provide recurring control-plane work without manual intervention > - The new activity gate can suppress scheduled runs when no external work occurred > - The core scheduler and database support landed without a public create/update contract > - Agents, operators, and managed plugins need validated fields plus discoverable semantics to opt in safely > - This pull request exposes the activity gate through routine APIs, revisions, plugin contracts, tests, and skill documentation > - The benefit is backward-compatible control over idle scheduled work without losing activity-triggered follow-up ## Linked Issues or Issue Description - Refs #8534 ## What Changed - Added shared activity-gate policy and scope enums with create/PATCH validation. - Persisted activity-gate fields through routine creation, updates, revision snapshots, pipeline snapshots, and revision restores. - Defaulted legacy revision snapshots during restore and added regression coverage for pre-field snapshots. - Extended managed-plugin routine declarations, production reconciliation, and the SDK test harness to preserve non-default gate settings. - Added end-to-end API coverage for create/PATCH/list/detail round-trips, defaults, and invalid enum rejection. - Documented schedule-only semantics, activity windows, own-run/read-action exclusions, scopes, and an hourly quiet-night watcher example. ## Verification - `pnpm exec vitest run packages/shared/src/validators/routine.test.ts server/src/__tests__/routines-service.test.ts server/src/__tests__/routines-e2e.test.ts` - `pnpm exec vitest run packages/shared/src/validators/plugin.test.ts packages/plugins/sdk/tests/testing-actions.test.ts server/src/__tests__/plugin-managed-routines.test.ts server/src/__tests__/routines-service.test.ts -t 'activity gate|preserves declared activity gate settings|resolves routine agent and project refs'` - `pnpm exec vitest run ui/src/lib/workspace-routines.test.ts ui/src/pages/Routines.test.tsx` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/plugin-sdk typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm check:token-gates` - GitHub CI: all final-head checks green; Storybook visual regression skipped by path rules. - Greptile: 5/5 with no unresolved review threads. ## Risks - Low risk: defaults remain `always` and `company`, preserving existing routine behavior and old revision snapshots. - Managed plugin manifests can now declare the same validated gate settings as the public routine API; omitted values retain core defaults. - Revision snapshots now include the new fields so policy changes are not lost or treated as no-ops during restore. > For core feature work, checked `ROADMAP.md`: this extends the existing Scheduled Routines roadmap item and does not duplicate a separate planned capability. ## Model Used - OpenAI GPT-5.5 via Codex CLI, with repository tool use and code execution; context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8f08ec5ce6 |
feat(status-cards): join summary-mentioned issues to watched set (#10205)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Status cards summarize changing company work and watch issues so later changes can produce useful deltas > - A summary can explicitly reference issues that are important to the update even when those issues do not match the card's configured queries > - Previously, those referenced issues were not retained in the watched set, so their later status, assignee, or comment changes could be missed > - The watched snapshot must avoid artificial additions or removals caused only by a summary changing which issues it references > - This pull request resolves issue references when a summary is written, persists them, and joins them to the watched snapshot with stable delta semantics > - The benefit is that status cards continue tracking the exact issues their latest update called out while keeping follow-up updates relevant and non-duplicative ## Linked Issues or Issue Description ### Pre-submission checklist - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip (or can reproduce on `master`). - [x] I have confirmed the error originates in Paperclip itself — not in my agent adapter, API provider, or local configuration. ### What happened? When a status-card summary explicitly referenced an issue by identifier or `/issues/<uuid>` URL, that issue was not automatically retained in the card's watched set unless it independently matched a configured query. Later status, assignee, or comment changes to an issue highlighted by the latest update could therefore be omitted. ### Expected behavior References in the latest summary should resolve only within the card's company, appear in dry runs and the watched-issues UI, count and fingerprint like query matches, and enter or leave the watched set without artificial added/removed deltas already represented by the summary change. ### Steps to reproduce 1. Create a status card whose query does not match a second issue in the same company. 2. Write a summary that references the second issue by identifier or issue URL. 3. Inspect the card's watched count or Watched issues tab. 4. Change the referenced issue's status, assignee, or comments and run the next update. 5. Before this change, the referenced issue is absent from the watched snapshot and its later change does not produce the expected delta. ### Paperclip version or commit - Reproduced on `master` before this PR (base commit `762ce5b4ef`). ### Deployment mode - Local dev (`pnpm dev`), built from source. ### Agent adapter(s) involved - Not adapter-specific (core bug). ### Database mode - Embedded Postgres test environment; the schema change uses standard PostgreSQL JSONB. ### Access context - Board (human operator). ## What Changed - Added migration `0191` and schema support for persisted `status_cards.mentioned_issue_ids`. - Resolved summary references by issue identifier or `/issues/<uuid>` URL within the status card's company when summaries are written. - Joined mentioned issues into watched counts and fingerprints so later status, assignee, and comment changes generate normal update deltas. - Suppressed artificial added/removed deltas when the latest summary starts or stops mentioning an issue. - Added `mentionedIssues` to dry-run responses and a “Mentioned in the latest update” group in the Watched issues tab. - Updated the summarizer prompt to explain that referenced issues automatically join the watched set. - Added focused server and UI coverage for reference resolution, snapshot behavior, deltas, API responses, and rendering. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/status-cards.test.ts src/__tests__/status-card-update-engine.test.ts` — 31 tests passed. - `pnpm --filter @paperclipai/ui exec vitest run src/pages/StatusCards/StatusCardTile.test.tsx` — 11 tests passed. - Earlier implementation verification also passed database/shared/server typechecks, UI `tsc -b`, the broader StatusCards UI test set, and embedded-Postgres migration application. ### Visual Verification - Greptile T-Rex ran Playwright browser checks successfully and captured the Status Card drawer Watched tab showing the new “Mentioned in the latest update” grouping: https://app.greptile.com/trex/runs/15796101/artifacts ## Risks - The migration adds a nullable JSONB column and is backward-compatible; existing cards have no mentioned issues until their next summary write. - Reference extraction is company-scoped to prevent cross-company issue association. - Watched counts and future fingerprints change for cards whose latest summaries reference issues; tests cover additions, removals, and suppression of spurious deltas. - This targeted status-card fix does not introduce a new roadmap subsystem or external integration. ## Model Used - Anthropic Claude Fable 5 (Paperclip model alias; exact underlying provider model ID and context window were not recorded in the implementation task metadata), with extended reasoning, tool use, and code execution. - OpenAI Codex coding agent (runtime model identifier and context window not exposed to this task) prepared the PR, rebased the branch, and ran focused verification with terminal tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Fable 5
|
3a16b91217 |
feat(status-cards): single-message setup drives query and update prompt (#10202)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their ongoing work. > - Status cards turn a standing question into recurring, agent-generated summaries on the board. > - The existing setup split intent across a watch prompt and separate update instructions, which made creation and later behavior harder to understand. > - A status card should have one durable source of truth for both deciding what to watch and telling the summarizer what each update must contain. > - This pull request makes the card prompt that source of truth, simplifies creation to one step, and lets operators choose the running agent immediately. > - The benefit is a smaller mental model, fewer configuration modes, and consistent update instructions throughout the card lifecycle. ## Linked Issues or Issue Description Status cards currently require operators to express the same intent in two places: the watch prompt and optional update instructions with append/replace/none modes. This feature simplifies the experimental status-card workflow so a single prompt defines both the watch query and every generated update. The create flow must also support selecting the responsible agent without a second setup step. Related prior status-card work: #10101. ## What Changed - Use the status card's single prompt to compile the watch query and directly instruct every summary update. - Add migration `0190_status_card_single_prompt` to remove `status_cards.instructions_mode` and `status_cards.instructions`. - Add `agentId` to `createStatusCardSchema`, validate company membership, and default new cards to the built-in Summarizer. - Replace the two-step create flow with one prompt-and-agent dialog and extract a shared `SummarizerAgentSelect` for create/settings surfaces. - Remove the extra-instructions settings section, reset incremental history when the prompt changes, and rename the board page to "Status". - Update the bundled `status-card-query` skill and board-operator documentation, then regenerate the skills catalog manifest. ## Verification - Server status-card suites: 29/29 passing. - UI `StatusCards` suites: 22/22 passing. - Skills catalog suite: 20/20 passing. - `tsc -b` passes for server, UI, shared, and database packages. - `pnpm check:migrations` passes. - Light and dark mode screenshots cover the new create dialog and settings tab. ## Risks - Migration `0190` intentionally drops existing separate instruction text. Existing card prompts remain and become the update instructions under the new model; status cards are experimental and feature-flagged. - Prompt edits now reset the incremental summary chain and trigger a full rebuild, which is intentional because the prompt is also the update contract. - Agent selection is company-scoped; invalid agent ids return a validation error rather than creating a misrouted card. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Implementation: Anthropic Claude via the `claude_local` adapter, agent label "Claude Fable 5"; extended reasoning, tool use, and code execution. The exact provider model id and context-window value were not retained in the task metadata. - PR preparation: OpenAI GPT-5.4 through Codex CLI, with reasoning, repository inspection, GitHub CLI, and Paperclip API tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7e40ed8c43 |
feat(status-cards): add experimental status card update view (#10101)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Operators need a board-level way to monitor a changing slice of company work without repeatedly rebuilding filters or reading raw task threads. > - Existing summaries are useful snapshots, but they do not provide a dedicated query-backed card with refresh policy, change tracking, update history, and per-update cost visibility. > - The capability needs to be safe to evaluate before it becomes part of the default product surface. > - This pull request adds end-to-end experimental Status Cards, from schema and query compilation through update orchestration and operator UI. > - The entire feature is gated behind the `enableStatusCards` experimental toggle, including its route and sidebar entry. > - The benefit is a governed, inspectable way to keep focused operational rollups current while preserving explicit controls over refresh frequency and spend. ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`packages/db`, `packages/shared`, `server/`, `ui/`, and bundled skills/docs). ### Problem or motivation Operators cannot currently define a reusable natural-language view of company work, compile it into an inspectable query, and keep its summary current as matching issues change. Rebuilding filters and rereading task threads makes board-level monitoring repetitive and hides the relationship between source changes, refresh cost, and the resulting summary. ### Proposed solution Add experimental Status Cards that compile operator intent into a query, summarize matched work, record each update, expose manual/interval/reactive refresh policies and costs, and preserve the last good result across stale, updating, paused, and error states. The capability is off by default and fully gated behind `enableStatusCards`, including its route and navigation entry. ### Alternatives considered - Extend existing one-off summaries: rejected because status cards require persistent query provenance, refresh policy, update history, and card-specific cost controls. - Add a dashboard-only filter widget: rejected because it would not provide governed background refresh, an update ledger, or an inspectable compile pipeline. - Ship the surface by default: rejected in favor of an experimental toggle while behavior and operator value are evaluated. ### Roadmap alignment This advances Paperclip’s board-level execution visibility and output-first product goals. `ROADMAP.md` was checked and no duplicate status-card initiative was found. ### Additional context No related open PR was found in the public GitHub search for status cards. The PR-only design wireframes were removed from the repository after review; the published prototype remains external to the production source tree. ## What Changed - Added company-scoped status-card schema, CRUD APIs, compile provenance, update ledger, shared contracts, validators, and OpenAPI coverage. - Added the text-to-query compile pipeline, bundled `status-card-query` agent skill, query versioning, and authorized write-back flow. - Added the experimental board, create flow, lifecycle tiles, detail/settings/debug drawers, archived view, routing, navigation, and instance setting. - Added a change-gated update engine with manual, interval, and reactive refresh policies, trigger selection, active hours, and daily token caps. - Added per-update token/cost recording, today and lifetime rollups, and policy-derived cost previews. - Added operator documentation and agent-authoring hardening for compile and update behavior. - Added PR-prep integration coverage for settings/startup wiring and replaced raw UI values with design-system tokens. - Removed the PR-only `design/pap-15023-status-cards` wireframe artifacts so the repository contains only production feature assets. ## Verification - `pnpm -r typecheck` — passes on the PR head; includes `ui` `tsc -b` passing. The UI compile gate was also independently recorded as passing at `6d7f3cf96b` on July 23, 2026. - `pnpm build` — passes. - `pnpm check:token-gates` — passes with all three gates clean. - `pnpm test:run` — 2,880 tests passed and 1 skipped; the sole failure was an unrelated 10-second `afterAll` database-cleanup timeout in `execution-workspaces-service.test.ts`. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/execution-workspaces-service.test.ts` — passes on immediate focused rerun (25/25). - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/instance-settings-service.test.ts src/__tests__/server-startup-feedback-export.test.ts` — passes (31/31). - `pnpm --filter @paperclipai/ui exec vitest run src/pages/StatusCards/StatusCardSettingsForm.test.tsx src/pages/StatusCards/StatusCardTile.test.tsx src/pages/StatusCards/format.test.ts src/lib/status-card-state.test.ts` — passes (26/26). - Recorded pre-PR QA: compile-pipeline e2e PASS; full lifecycle and cost QA PASS; security re-review PASS after write-back hardening; UX approved. - `pnpm exec vitest run packages/db/src/status-card-migrations.test.ts` — passes; reapplies migrations `0185`–`0189` against an already-migrated embedded Postgres database. - `pnpm --filter /db check:migrations` — passes migration numbering and safety checks. - `pnpm --filter /db typecheck` — passes. - Merged current `origin/master` on July 24, 2026 with no conflicts; migrations `0185`–`0189` remain unclaimed on master. ## Risks - The feature introduces five database migrations and a new background update path; all new DDL is repeat-safe after partial application, migration numbering/safety checks pass, and update execution is company-scoped and change-gated. - Natural-language compilation can produce invalid or overly broad queries; compile provenance, query validation, debug visibility, and version history make failures inspectable and recoverable. - Reactive or interval refresh could increase spend; active hours, max refresh frequency, daily token caps, per-update cost records, and budget-paused states bound and expose that risk. - The branch name contains an internal execution identifier because it is a fixed handoff branch; it was intentionally not renamed or rebased per the release handoff instructions. - Overall rollout risk is limited because the route, navigation, services, and UI are disabled by default behind `enableStatusCards`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.5 with reasoning, repository tool use, shell execution, GitHub CLI, and test/build execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change; the fixed execution-workspace identifier is documented as an authorized handoff exception - [x] I have run tests locally and they pass, with the one cleanup timeout passing on focused rerun - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7f766526a6 |
feat(sandbox): add task-scoped egress grants (#10155)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Confinement providers protect agent runs with default-deny network policies > - Kubernetes environments currently apply only provider-level, namespace-wide egress allowances > - Tasks that legitimately need GitHub or package registries therefore cannot request narrow access, while network failures do not explain the governing policy or how to request a grant > - This pull request adds issue-scoped egress grants that become workload-owned, run-label-selected policies and carries the effective grant through lease audit metadata > - The benefit is that internet-dependent work can run without enabling broad egress for every concurrent task, and denied requests point operators to the exact grant path ## Linked Issues or Issue Description No public issue exists. Related but distinct: Refs #9944, which adds a provider-wide open-internet posture; this PR keeps provider defaults narrow and adds per-task grants. **Problem / motivation** Kubernetes sandbox egress is configured at the provider/tenant level. A task that needs to clone from GitHub or install from PyPI cannot request those destinations without changing the policy for every run in the tenant namespace. DNS/connectivity failures also surface as generic tool errors with no policy name or remediation path. **Proposed solution** Accept `executionWorkspaceSettings.networkEgress.allowFqdns` and `allowCidrs`, forward the setting through heartbeat environment acquisition, and create a workload-owned NetworkPolicy or CiliumNetworkPolicy selected by `paperclip.io/run-id`. Record the effective grant in lease activity/metadata, expose policy context through `PAPERCLIP_NETWORK_EGRESS_*`, and append the grant path to likely policy-related stderr failures. **Alternatives considered** A provider-wide open-internet switch is broader than required and is already covered by #9944. Mutating the existing namespace policy would leak each task's destinations to other concurrent runs. Standard Kubernetes NetworkPolicy cannot enforce FQDNs exactly, so standard mode uses the existing hardened public-IPv4 TCP 80/443 fallback only for the selected run; Cilium mode remains exact. **Roadmap alignment** This extends the existing cloud/sandbox agent roadmap capability with task-level control-plane policy and does not duplicate a planned roadmap item. ## What Changed - Added validated `networkEgress` grants to issue execution workspace settings and forwarded them through environment lease acquisition. - Added workload-owned, run-label-scoped NetworkPolicy/CiliumNetworkPolicy resources for task FQDN/CIDR grants. - Added lease audit metadata, sandbox policy environment variables, and actionable network-denial stderr guidance. - Added focused parser, manifest, policy creation, and denial-message tests plus Kubernetes provider documentation. ## Verification - `pnpm -C packages/shared exec vitest run src/validators/issue.test.ts` — 27 passed. - `pnpm -C packages/plugins/sandbox-providers/kubernetes test -- --run test/unit/network-policy.test.ts test/unit/cilium-network-policy.test.ts test/unit/scoped-network-egress.test.ts` — 21 passed. - `pnpm -C server exec vitest run src/__tests__/execution-workspace-policy.test.ts` — 15 passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-plugin-environment.test.ts server/src/__tests__/environment-runtime.test.ts` — 26 passed. - `pnpm --dir packages/db build && pnpm --dir packages/shared build && pnpm --dir packages/plugins/sdk build` — passed, including migration safety checks. - `pnpm --dir packages/plugins/sandbox-providers/kubernetes typecheck && pnpm --dir server typecheck` — passed after refreshing the worktree's frozen offline dependencies. - End-to-end cluster validation of the `build-cython-ext` benchmark remains for CI/maintainer Kubernetes infrastructure; the focused tests assert `github.com` and `pypi.org` produce a policy selected only by the granted run. ## Risks - Standard NetworkPolicy cannot express FQDNs, so an FQDN grant allows hardened public IPv4 TCP 80/443 for that run; use Cilium mode for exact hostname enforcement. - The new field is additive and absent by default, so existing runs keep the current provider-level policy. - Workload owner references garbage-collect scoped policies with the Job/Sandbox; a cluster/controller that ignores owner references could temporarily strand a policy that still selects no future run ID. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, tool use and code execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
148a5b11f5 |
Route blocked transitions to explicit unblock owners (#10112)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies. > - Issue status transitions determine whether work keeps moving or silently stalls. > - A blocked issue previously could rely on prose alone, leaving the intended unblock owner unstructured and unnotified. > - Existing blocker-attention classification could identify stalled chains, but the signal was not delivered to the board attention feed. > - Blocked transitions also need rollout-safe deduplication so upgrades do not notify for historical issues and repeated processing does not create notification storms. > - This pull request adds structured unblock descriptors, prospective transition timestamps, owner delivery, and board attention routing with focused authorization controls. > - The benefit is that newly blocked work has an explicit, routable next action without weakening company boundaries or allowing agents to inject arbitrary human attention items. ## Linked Issues or Issue Description Related documentation PR: #10094. ### Subsystem affected Cross-cutting: `server/`, `packages/db`, and `packages/shared`. ### Problem or motivation An issue can enter `blocked` without a machine-readable unblock path. Prose-only ownership does not reliably wake the responsible agent or surface human-owned work, while the existing `blockerAttention` classifier is not delivered to an operator-facing attention feed. ### Proposed solution Require new transitions into `blocked` to have unresolved blockers, a pending interaction/approval, or a structured `{ owner, action }` descriptor. Notify an allowed owner once per prospective transition, route human-owned cases to board attention, and leave pre-rollout blocked issues untouched. ### Alternatives considered - Keep prose-only blockers: rejected because ownership remains unroutable. - Backfill all historical blocked issues: rejected because upgrades would create notification storms. - Let agents target arbitrary users or the board: rejected after security review because it creates an attention-injection channel. ### Roadmap alignment Aligns with `ROADMAP.md` → “Enforced Outcomes (watchdogs, recovery actions, review gates)” by making blocked work carry an explicit continuation path. ### Additional context The implementation is prospective-only and deduplicated per blocked transition. Agent-authored descriptors are limited to the acting agent; board actors retain human-owner routing. ## What Changed - Added persisted unblock descriptors and prospective blocked-transition delivery timestamps with an idempotent migration. - Added shared types and validation for board, user, and agent unblock owners. - Enforced valid blocked transitions and same-company owner validation in the issue update route. - Restricted agent-authored descriptors to the acting agent itself, preventing board/user attention injection by compromised agents. - Added one-per-transition agent wake delivery and prospective-only rollout gating. - Routed human-owned blocker attention into the board attention feed. - Added focused tests for validation, prospective delivery, flap deduplication, attention routing, route authorization, and stop-relay compatibility. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts server/src/__tests__/routable-blocked.test.ts server/src/__tests__/attention-service.test.ts packages/shared/src/validators/issue.test.ts` - `AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= pnpm test:run` - `pnpm build` - `pnpm --filter @paperclipai/db check:migrations` ## Risks - Behavioral shift: new `blocked` transitions without a real blocker, pending governed action, or structured descriptor now return `422`. - Notification abuse is constrained by same-company validation, agent self-only routing, prospective rollout gating, and transition-scoped deduplication. - Migration risk is low: columns are additive, nullable, and use `IF NOT EXISTS`; historical blocked issues are not backfilled or notified. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI with GPT-5.4, reasoning-enabled tool use and code execution. The runtime did not expose a context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Michael NguyenandPaperclip
|
5a00040c4c |
feat(plugin-sdk): interactions/approvals respond + attachment read capabilities for the chat gateway (#10066)
Adds the remaining 5 plugin capabilities + 7 worker→host RPC methods (interactions read/respond, approvals read/respond, attachment read) needed by the Slack chat gateway plugin (v0.5.0) to pass manifest capability validation and load. - Security review: PASS (LOOA-642) after the viewer-role privilege-escalation blocker (LOOA-648) was fixed on this branch (requireActiveHumanMember now rejects viewer on impersonation write-paths, matching assertCompanyAccess). - CI: Build, Typecheck, all server suites (3/3 + serialized 4/4), workspaces, e2e shard 2/2, and all security scanners (Snyk/Socket/Superagent/Greptile/security-review) green. - One e2e flake (signoff-policy 'non-participant cannot advance stage') is unrelated: it exercises execution-policy stage advancement (routes/issues.ts, untouched by this PR) and failed on a heartbeat_run_events FK race + 409 checkout conflict. Unblocks LOOA-629 (Slack gateway go-live) and the interview-ask feature. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
216d3d2680 |
Managed-instance config: fail-closed PAPERCLIP_MANAGED_CONFIG parsing and read-time settings overlay (#10058)
**Builds on.** #10055 — the `catalogVersion` this config document pins is the feature-catalog artifact #10055 emits. **Summary.** Instances operated by a managed hosting control plane can now receive instance configuration through a single environment variable, `PAPERCLIP_MANAGED_CONFIG` (versioned JSON: `mode`, `catalogVersion`, `features`, `plugins.autoInstall`). When the variable is absent the instance is self-hosted and nothing changes. When present, parsing is strict and **fail-closed**: blank value, malformed JSON, unknown feature key, a feature key this build's feature catalog does not mark tier `managed`, missing required section, or unsupported version refuses startup with a precise error — a typo that silently does nothing is how a security control quietly fails. Managed feature values are overlaid **at read time** inside the instance settings service (never persisted), so a DB restore or manual row edit cannot resurrect a disabled capability; responses expose per-key `managedKeys` metadata (`managed: true`, `managedBy`) so clients can render locked state. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs both self-hosted and under managed hosting, where an operator's control plane owns instance configuration > - Today instance feature settings live only in the tenant database; a hosting control plane has no way to enforce a configuration that tenant-side writes or restores cannot undo > - Managed configuration will carry security posture, so delivery must be atomic and parsing must fail closed — a typo that silently does nothing is how a security control quietly fails > - This pull request adds strict parsing of one `PAPERCLIP_MANAGED_CONFIG` env var and overlays its feature values at read time inside the settings service, never persisting them > - The benefit is a minimal, auditable managed-hosting contract: absent var ⇒ self-hosted instances are byte-for-byte unchanged; present ⇒ deterministic, locked configuration surfaced to clients via per-key managed metadata ## Linked Issues or Issue Description Refs #966 — this PR delivers that issue's "managed config injection" hook, via a strict env-var contract rather than the config-file path it sketches; the issue's other hooks (identity header, health, usage webhook, lifecycle, external secrets, IAM auth) are out of scope, so the PR refs rather than closes it. *Mechanism differs from #966's proposal, so the `feature_request` fields are also filled in:* - **Problem or motivation:** managed hosting deployments need to centrally enable/disable instance features; DB-stored settings can be edited, restored, or migrated back to permissive values, and nothing marks a value as operator-enforced. - **Proposed solution:** one versioned JSON env var; fail-closed parse at startup; read-time overlay in the settings service (precedence: managed value over stored value over schema default); `managedKeys` metadata in settings responses so clients can render locked state. - **Alternatives considered:** per-feature env vars (non-atomic across a half-updated env set, unbounded env surface); seeding the DB at boot (persisted values can be edited or restored over, and cannot express "forced"); lenient warn-and-drop parsing (fails open — unacceptable for a security-bearing control). - **Roadmap alignment:** supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/managed-config.ts` (pure parser over the env record) - Startup parse ordered before the first `instanceSettingsService` construction in `server/src/index.ts` - Read-time merge + `managedKeys` in the settings service - Shared validator updates ## Verification - 29 parser/overlay tests (fail-closed matrix incl. blank/whitespace env, missing sections, catalog-tier mismatch, empty-section happy path): `pnpm vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts` (from `server/`) - 40 existing settings route/service tests green: `pnpm vitest run src/__tests__/instance-settings-routes.test.ts src/__tests__/instance-settings-service.test.ts` (from `server/`) - 15 shared validator tests: `pnpm vitest run src/validators/instance.test.ts` (from `packages/shared/`) - Server `tsc --noEmit` clean: `pnpm typecheck` (from `server/`) ## Risks - Self-hosted instances (no `PAPERCLIP_MANAGED_CONFIG` set) are byte-for-byte unchanged — the parser only runs when the variable is present. - For managed instances, a malformed document now refuses startup by design (fail-closed). This is an intentional behavioral guarantee, not a regression: the control plane owns the variable and a precise startup error is the contract. - Overlay values are never persisted, so no migration or data-shape risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
ad74fb5450 |
Add a feature catalog build artifact derived from the experimental settings schema (#10055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Instances expose ~23 experimental feature settings, all declared in one shared zod schema and toggled per instance > - Deployment tooling and hosting control planes have no machine-readable list of those feature keys for a given release — the schema is only reachable from code that imports the package > - Any external system that references feature keys therefore does so as free text, and typos drift silently > - This pull request derives a versioned `feature-catalog.json` build artifact from the schema, with a compiler-checked metadata map so the schema stays the single source of truth > - The benefit is a stable contract external tooling can validate feature-key references against, with zero runtime behavior change ## Linked Issues or Issue Description No public issue exists; `feature_request` template fields: **Problem or motivation:** External deployment tooling cannot enumerate or validate an instance's feature keys per release; free-text references fail silently when keys are renamed or removed. **Proposed solution:** A metadata map keyed by the settings schema's own keys (compiler flags drift) plus a build step emitting `feature-catalog.json` (keys, tiers, defaults, `catalogVersion`) as a release artifact. **Alternatives considered:** A hand-maintained catalog file (drifts from the schema); serving the schema from a runtime API (requires a running instance at validation time — a build artifact works offline and pins to a release). **Roadmap alignment:** Supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed Adds a metadata map (title, description, tier, cloud/self-hosted defaults) keyed by the keys of `instanceExperimentalSettingsSchema`, so the schema stays the single source of truth and the compiler flags any drift. A new build step (`build:feature-catalog --version <v>`) emits `feature-catalog.json` — all 23 feature keys, their tiers, and a `catalogVersion` — as a release artifact that managed-hosting control planes can validate feature-flag writes against. No runtime behavior changes. - New `packages/shared/src/feature-catalog.ts`: per-flag metadata map keyed by a type derived from the settings schema (adding/removing/renaming a flag without updating the map is a compile error), plus `featureCatalogArtifactSchema` and `buildFeatureCatalogArtifact`/`renderFeatureCatalogArtifact` for the artifact - New `scripts/generate-feature-catalog.ts` wired as `pnpm build:feature-catalog --version <v>` - `scripts/create-github-release.sh` generates the artifact and uploads it as a GitHub Release asset (with a dry-run preview line) - Tests in `packages/shared/src/feature-catalog.test.ts` ## Verification - `vitest run packages/shared/src/feature-catalog.test.ts` — 9 tests: schema-key coverage, drift detection, artifact shape - `pnpm --filter @paperclipai/shared typecheck` - Artifact generation run end-to-end: `pnpm build:feature-catalog --version 0.0.0-test` emits 23 keys with `catalogVersion` ## Risks Low risk — no runtime behavior changes; the change is metadata, a build script, and a release-artifact emission step only. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Michael NguyenandPaperclip
|
e55d702916 |
feat(plugin-sdk): human-attributed issue comments for chat gateway plugins (#10050)
Adds the `issue.comments.create_human_attributed` capability and `ctx.issues.createComment` `actorUserId` option, with host-side active-human-member verification. LOOA-627. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
0b496c9c03 |
feat(secrets): add run-bound agent secret access (#9921)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Agents already receive selected company secrets through `env.*` bindings at run launch, but environment injection is ambient, long-lived, and not suitable for every secret consumer. > - The existing binding and secret-access-event models already provide company-scoped authorization and per-resolution audit seams. > - Agents need an explicit way to discover only the secrets granted to them and fetch a value on demand without exposing the wider company catalog. > - That capability must remain run-bound, preserve low-trust token carve-outs, and make every value read visible in both security and operator audit trails. > - This pull request adds an `access.*` delivery namespace, two run-bound agent routes, dual audit logging, documentation, and an operator grants editor. > - The benefit is least-privilege, revocable, auditable secret access while preserving existing env injection behavior. ## Linked Issues or Issue Description No pre-existing public issue. Related work: - Refs #9797 — existing in-sheet agent access UI that this PR extends to distinguish env and API delivery. - Refs #9918 — complementary searchable-agent picker improvement for the same secrets sheet. - Refs #9530 — related company-wide metadata catalog proposal; this PR intentionally exposes only the authenticated run's granted aliases and values. **Problem / motivation:** Agents can currently consume secrets only through process environment injection. This keeps values resident for the run, does not support on-demand consumers, and cannot provide a discrete operator-visible activity event for each agent-initiated read. **Proposed solution:** Treat `company_secret_bindings` as the source of truth for agent secret grants. Keep `env.KEY` as env delivery and add `access.ALIAS` for API-only delivery; an env binding also implies read access because the value is already present in the agent process. Add run-bound list/fetch endpoints that derive scope from the authenticated heartbeat run and never accept caller-selected overlays. **Alternatives considered:** A company-wide agent-readable catalog was rejected for this value path because it increases reconnaissance and does not prove a per-secret grant. Reusing the ephemeral environment-probe resolver was rejected because it lacks binding enforcement. Approval-gated reads and user-scoped secrets remain deferred beyond v1. **Roadmap alignment:** This extends the completed **Secrets Manager with per-agent access** roadmap capability from launch-time env injection to explicit run-bound API delivery without duplicating a separate planned initiative. ## What Changed - Added `access.*` agent binding validation and a dedicated run-bound resolver that combines `secrets:read` authorization with binding-context enforcement. - Added `GET /api/agents/me/secrets` for minimal granted metadata and `POST /api/agents/me/secrets/:key/value` for on-demand value fetches with `Cache-Control: no-store`. - Preserved the existing denials for low-trust review agents, task-bridge credentials, and skill-test tokens; standard long-lived agent API keys cannot call the run-bound routes. - Added dual audit behavior: value attempts write `secret_access_events` and `activity_log` (`secret.value.read`), while metadata listing writes the lighter `secret.access.listed` activity event. - Kept env compatibility: `env.*` remains injected at launch and also implies API read for the same bound agent; `access.*` never becomes an environment variable. - Added the agent-settings **Secret access** editor plus delivery-mode/alias surfacing on the Secrets page, with focused UI tests and tokenized layout styles. - Updated OpenAPI, shared types, agent-facing skill documentation, and API reference documentation. ### UI Screenshots P3 produced and reviewed three screenshots using mock data; images are intentionally not committed to the repository: - `secret-access-editor.png` — agent settings grant editor. - `secret-access-light.png` — Secrets-page delivery surfacing in light mode. - `secret-access-dark.png` — Secrets-page delivery surfacing in dark mode. The source attachments are retained with the implementation task and linked in the internal handoff; the public page publisher was unavailable in the PR-prep runtime. ## Verification - `pnpm exec vitest run server/src/__tests__/agent-secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts server/src/__tests__/secrets-routes.test.ts ui/src/lib/secret-delivery.test.ts ui/src/components/AgentSecretAccessEditor.test.tsx` — 5 files, 122 tests passed. - Security follow-up: `pnpm exec vitest run server/src/__tests__/agent-secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts` — 2 files, 73 tests passed after active-run and version-consistency fixes. - Final-head CI: all feature, typecheck, build, e2e, security, and review gates pass; `General tests (server (1/3))` remains red after one rerun because unrelated `heartbeat-retry-scheduling.test.ts` cleanup deletes `heartbeat_runs` before referenced `activity_log` rows. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm check:token-gates` — feature-local arbitrary-value violations fixed; command still reports five unchanged `#9627` literals outside this PR. - End-to-end QA passed all eight acceptance criteria: grant/list, fetch, dual audit, env-implies-read, denial matrix, revocation, UI rendering, and env-injection regression. Evidence: https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455492 - Security review returned PASS-with-required-changes; the implementation uses the required dedicated binding-enforcing resolver, run-bound JWT restriction, run-derived overlays, minimal metadata, and a resolver redaction-registration hook. Evidence: https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455382 ## Risks - A compromised agent can exfiltrate any secret explicitly granted to it; explicit company-scoped/run-scoped grants, revocation, and audit reduce but cannot remove that inherent capability risk. - The resolver invokes a redaction-registration hook before returning values, but the current route has no persistent cross-request per-run redaction registry. Paperclip-owned later comments/events therefore cannot yet guarantee automatic scrubbing of a deliberately copied fetched value; QA classified this as non-blocking residual hardening. - Audit-event insertion currently fails open if the security-event insert itself fails; the operator activity event provides partial redundancy, but a future hardening change should define fail-closed behavior for value delivery. - This PR overlaps `ui/src/pages/Secrets.tsx` with #9918 and may require a straightforward rebase after that PR moves. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.3-codex`, with reasoning, repository tool use, terminal execution, Paperclip API access, and GitHub CLI capabilities. Context-window size is not exposed by the runtime. - Anthropic Claude Opus 4.8 with 1M context and tool use assisted with the UI implementation commit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
cac3c0fa1a |
feat(connections): add runtime subjects and grants (#9982)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that lets operators connect external apps and govern which subjects may use those credentials > - #9958 established the v3 schema foundation and #9981 adds the AppDefinition catalog layer > - The runtime still needs subject-aware authorization state, scoped key handling, and API/OpenAPI routes so connected apps can actually be granted and used safely > - This pull request adds the runtime grants/authorization behavior on top of the catalog branch, while keeping unrelated dependency and workflow sync commits out of the stack > - The benefit is a reviewable runtime layer that can land after the catalog PR, then unblock the wizard and orchestrator cutover work ## Linked Issues or Issue Description Refs #9958 and #9981. Refs #9981. No public GitHub issue exists for this branch. This is the runtime layer for the Connections v3 stack and is rebased onto `master` after #9981 landed. ## What Changed - Adds the connection user authorization state migration and schema wiring. - Adds shared runtime subject/grant types and validators. - Adds runtime grant and scoped key behavior in the tool-access service. - Adds runtime route coverage and registers the routes in OpenAPI. - Replays only the Connections runtime commits on top of the catalog branch, dropping unrelated sync/dependency history from the prior closed runtime PR. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/validators/tool-access.test.ts server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: runtime grant enforcement is security-sensitive and must fail closed for unknown key scopes. - Migration ordering depends on the schema and catalog layers already merged through #9958 and #9981. - This PR is rebased and retargeted to `master` with runtime-only commits. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d23fbf8ae4 |
feat(connections): add AppDefinition Wave 1 catalog (#9981)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that defines which external apps and MCP-style integrations operators can browse, configure, and run > - The v3 schema core in #9958 added stable connection identities, auth metadata, and grant-aware contracts, but the app catalog still used the older gallery shape > - The product needs a richer, typed AppDefinition catalog so browsing and setup can render provider-specific auth and configuration requirements consistently > - This pull request moves the Wave 1 app catalog onto generated AppDefinition data and carries that shape through shared types, server lookup paths, and app connection UI > - The benefit is that follow-up runtime and wizard work can build against one catalog contract instead of local-only mock/gallery data ## Linked Issues or Issue Description Refs #9958. No public GitHub issue exists for this branch. This is the catalog layer for the Connections v3 stack after the schema-core foundation in #9958. ## What Changed - Adds generated AppDefinition data for the Wave 1 catalog and ingestion reporting. - Replaces the legacy tool app gallery exports with AppDefinition-centered shared contracts, validators, and tests. - Updates server tool-access lookup behavior to use the AppDefinition catalog. - Updates app connection UI surfaces and tests to consume AppDefinition-backed catalog data. - Documents the catalog ingestion workflow in the connector playbook. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts packages/shared/src/app-definitions-url.test.ts ui/src/pages/apps/AppsConnect.test.tsx server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: this changes the catalog contract used by shared, server, and UI app connection surfaces. - Catalog data quality matters because generated definitions now drive browse/setup display. - Follow-up runtime and wizard PRs must rebase on this branch or on master after this lands. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7e00f67138 |
feat(connections): add v3 schema core (#9958)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their governed access to external systems. > - Connected Apps build on the existing Apps and MCP gateway substrate so companies can configure reusable, auditable integrations. > - The current connection record does not yet have a stable public address, explicit ownership/auth method fields, or subject-specific credential grants. > - Without that schema core, later OAuth, per-user authorization, token brokering, triggers, and connector-service phases cannot enforce tenant and subject boundaries consistently. > - This pull request adds the forward-compatible Connections v3 schema core while preserving the existing connection lifecycle and directly migrating the remote MCP transport name. > - The benefit is a company-scoped, least-privilege foundation for one-click integrations without bypassing Paperclip secrets, profiles, rules, or audit controls. ## Linked Issues or Issue Description No matching public issue was found. **Problem** Paperclip's current app connections need a durable identity and authorization substrate before Connected Apps can safely support multiple setup methods, per-user credentials, provider tenants, and managed connector services. The existing schema only models a single connection-level credential set and uses legacy transport terminology. **Proposed solution** Add a stable company-scoped connection UID, explicit ownership/auth/transport fields, a subject-aware `connection_grants` table, and multi-key credential annotations. Backfill existing connections and workspace grants in a reversible migration, then update shared/server/UI contracts to the new `mcp_remote` transport name. **Related work** - Related foundation: #9534 - Roadmap: Connected Apps (one-click integrations) ## What Changed - Added company-scoped connection `uid`, `ownership`, `authKind`, and canonical transport fields across database, shared contracts, validators, services, and UI fixtures. - Added `connection_grants` with workspace/user subject rules, provider tenant metadata, credential secret refs, revocation state, company scoping, and uniqueness constraints. - Added migration `0182_connections_v3_schema_core` to backfill stable UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create default workspace grants, and support rollback coverage. - Added multi-key credential annotations and updated gateway/access services without changing the existing lifecycle behavior. - Updated the connection glossary, connector playbook, and security threat model for the new identity, grant, and relay boundaries. - Added explicit test UIDs to direct database fixtures so the new non-null invariant is exercised across affected server suites. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts server/src/__tests__/heartbeat-runtime-skills.test.ts server/src/__tests__/tool-oauth-legacy-backfill.test.ts server/src/__tests__/tool-access-policy-service.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts packages/db/src/connections-v3-schema-core-migration.test.ts packages/shared/src/validators/tool-access.test.ts --config vitest.config.ts` — 9 files, 218 tests passed. - Latest-head GitHub Actions: build, typecheck, general/serialized suites, backup/worktree restore coverage, both e2e shards, canary, policy, and security scans pass. - Greptile: 5/5 with zero unresolved threads. - `pnpm check:token-gates` remains red only on five pre-existing `#9627` color literals outside this change. ## Risks - **Migration risk:** UID backfill and default-grant creation touch every existing connection. The migration uses company-scoped uniqueness, deterministic legacy UIDs with ID suffixes, and seeded up/rollback coverage. - **Authorization risk:** Grant rows carry credential references. Constraints enforce workspace-vs-user subject shape, company/connection lookup indexes, one default grant per connection, and one user grant per connection/subject. Security review is requested specifically for this design. - **Compatibility risk:** `remote_http` is renamed directly to `mcp_remote`; all repository call sites and fixtures are updated in the same change. - **Future-phase risk:** Subject-bound token issuance, triggers, and connector-service relay verification remain fail-closed requirements documented for later phases; this PR does not expose those capabilities. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI coding agent. The runtime did not expose an exact underlying model ID or context-window size; capabilities used include repository inspection, code editing, shell execution, test execution, Git/GitHub CLI operations, and structured reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
c81a089c12 | feat(telemetry): align client with wire contract — chunking, deterministic batchId, batched retry, bounded store (#9946) | ||
|
|
bf6365637a |
feat(telemetry): pin silent-drop baseline and add caps/backoff config surface (Impl-1) (#9906)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The telemetry subsystem (`packages/shared/src/telemetry`) flushes event batches to an ingest endpoint; today the client drops batches silently on 429 / 413 / 400 / network error with no re-queue and no retry > - The current behaviour is uncharacterised — there are no regression tests pinning it, so any future retry work could accidentally change the drop semantics without a test failing > - Before adding retry logic it is critical to lock the current baseline as an explicit, documented contract so regressions are immediately visible > - Additionally, the retry/cap design needs a config surface (`maxEventsPerBatch`, `maxBodyBytes`, `maxPendingRetryBatches`, `backoff`) so operators can tune behaviour without forking; that surface should be defined and defaulted before the first consumer lands, not after > - This pull request adds characterisation tests that pin today's silent-drop baseline and adds an optional, additive config surface with resolved defaults — `client.ts` is untouched so no behaviour changes > - The benefit is that the retry work in the follow-up PR can build on a verified baseline, safe defaults, and a pre-wired config contract rather than specifying everything at once ## Linked Issues or Issue Description No pre-existing public GitHub issue. Underlying problem described below (feature-request template). **Problem or motivation** The telemetry client in `packages/shared/src/telemetry/client.ts` has no regression tests covering its error-path semantics. On 429 / 413 / 400 / network failure the current code drains the whole queue via `splice(0)` and discards the batch — silent drop, no retry. That behaviour is correct for a best-effort client today, but nothing verifies it. When retry logic lands, there is no safety net to confirm it does not accidentally re-queue already-dropped batches. In parallel, the retry design requires a typed config surface (`maxEventsPerBatch`, `maxBodyBytes`, `maxPendingRetryBatches`, `backoff`) that the client and its callers need to agree on before the first consumer is wired up. **Proposed solution** Two-phase additive scaffolding: 1. Lock the silent-drop baseline as pinned tests (temporary; Impl-2 replaces them when retry lands). 2. Add the config surface and defaults now so Impl-2 can consume them immediately. **Alternatives considered** Writing the tests and config surface inside the retry PR — rejected because it makes the retry diff much harder to review and loses the regression-safety benefit of a committed baseline. **Roadmap alignment** Maintenance / client-correctness improvement; not a core roadmap feature. ## What Changed - **`client.test.ts`** — 5 new Phase-1 characterisation tests pinning silent-drop on 429, 413, 400, and network-error. A `stubFetchStatus(status)` helper DRYs the four drop cases. These are temporary regression anchors: marked `TODO(impl-2): replace when retry lands`. - **`config.ts`** — `TELEMETRY_DEFAULTS` constant (frozen) + `resolveCaps()` + `resolveTelemetryConfig()`. Provides a single exported source of truth for defaults; nothing reads these yet. - **`types.ts`** — 4 optional, additive `TelemetryConfig` fields: `maxEventsPerBatch` (default 50), `maxBodyBytes` (default 524 288), `maxPendingRetryBatches` (default 20), `backoff` (exp-with-jitter shape). All optional and backward-compatible. - **`index.ts`** — re-exports `TELEMETRY_DEFAULTS` and `resolveTelemetryConfig` from the public package surface. - **`client.ts`** — no changes (verified via `git diff --stat`). ## Verification ```bash # Unit tests — 16 passed (5 new Phase-1 pins + 2 new Phase-2 config tests + 9 existing) pnpm --filter @paperclipai/shared exec vitest run src/telemetry # Type check — clean pnpm --filter @paperclipai/shared typecheck # Confirm client.ts untouched (zero output expected) git diff HEAD~1 -- packages/shared/src/telemetry/client.ts ``` All three commands verified locally before push. ## Risks **Low risk.** `client.ts` is untouched — no behaviour change. All four modified files are additive: - New tests cannot regress production behaviour. - New config fields are `optional` and their defaults match the current implicit constants in `client.ts`, so any future reader gets identical semantics until it overrides them. - `resolveTelemetryConfig` replaces no existing function; it is new. - No envelope / wire change; no new PII / crypto / sink. The `TODO(impl-2)` markers on the Phase-1 pins make their temporary nature explicit in code. ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200k tokens - **Capabilities:** Tool use, code execution, agentic task completion via Paperclip agent SDK - **Mode:** Standard (no extended thinking) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
051ae4d102 |
feat: restore decision training library and inspector (#9779)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and supervise governed work. > - Decisions capture high-value operator judgment, and the decision-training foundation merged in #9702 freezes that evidence for later evaluation and learning. > - Operators still need the UI from the closed stacked PR #9718 to intentionally capture examples and inspect the resulting dataset. > - GitHub automatically closed #9718 when its stacked base branch was deleted after #9702 merged, leaving the server foundation on `master` without the corresponding UI. > - This pull request restores the final UI and its still-required supporting API fields directly on current `master`, while excluding the obsolete migration and duplicated server-foundation diffs. > - The benefit is a reviewable replacement PR that preserves the completed decision-training workflow without replaying stale stack history. ## Linked Issues or Issue Description - Refs #9718 - Refs #9702 ## What Changed - Restored the top-level `/training` library and record inspector with search, filters, JSONL export, notes editing, and evidence tabs. - Restored the Decisions-row training affordance and capture drawer, including preview, provenance, deletion, cache refresh, and approval consistency behavior. - Restored the shared types and focused server support needed by the UI without reintroducing decision-training migrations or the already-merged server foundation. - Restored focused UI and attention-service tests from the final #9718 state. - Credit to the authors and reviewers of #9718; this recovery transplants their final reviewed delta after the stacked base deletion. ## Verification - `pnpm exec vitest run ui/src/pages/Training.test.tsx ui/src/components/DecisionTrainingDrawer.test.tsx ui/src/components/AttentionQueueRow.test.tsx server/src/__tests__/attention-service.test.ts server/src/__tests__/decision-training.test.ts` — 5 files, 48 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts` — 3 tests passed; confirms exact route/OpenAPI parity for the restored preview endpoint. - `pnpm check:token-gates` — the restored files are clean; the repository-wide command currently reports five pre-existing false positives where comments reference GitHub issue `#9627` as if it were a color literal. ## Risks - Low migration risk: this PR contains no database migrations and is based directly on current `master`. - The main behavioral risk is cache invalidation across Decisions and Training views; focused tests cover capture, update, deletion, row state, and approval refresh behavior. - The token-gate baseline remains red on unrelated `#9627` comment references; this PR does not modify those files. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.3-Codex, reasoning with repository/tool access and code execution. Context window not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a090c09ee5 |
feat: add decision training snapshot foundation (#9702)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - Human approvals, issue interactions, and execution decisions already capture high-value decision moments > - Those moments are currently transient and cannot be reused as stable evaluation or training examples > - Reusable examples need a server-owned, immutable snapshot so later comments or runs cannot leak into the recorded state > - Human notes need to remain editable and auditable without changing the captured state > - This pull request adds the database model, snapshot capture service, API, export format, and attention-feed enrichment for decision training > - The benefit is a durable, inspectable foundation for evaluating whether agents can reproduce good human decisions from only the context available at decision time ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`server/`, `packages/db`, and `packages/shared`). ### Problem or motivation Paperclip has no durable dataset for converting human decisions into evaluation-ready examples. Teams need to capture pending or resolved decisions with the exact issue context, comments, runs, and repository evidence available at a cutoff, while preventing future context from leaking into the example. ### Proposed solution Store immutable, schema-versioned snapshots anchored to durable interaction, approval, or execution-decision records; keep notes separately editable with history; expose human-only CRUD, list, and JSONL export APIs. ### Alternatives considered Client-generated snapshots were rejected because they duplicate cutoff logic and cannot reliably enforce no-leakage boundaries. Automatic outcome backfill was deferred so captured examples remain faithful to what was known at capture time. ### Roadmap alignment Supports the roadmap direction of turning completed work and decision patterns into reusable organizational knowledge. ### Additional context The implementation records explicit commit-resolution confidence (`exact`, `nearest_run`, `workspace`, or `none`) so downstream evaluation can distinguish evidence quality. ## What Changed - Added the `decision_training_examples` schema and idempotent migration with company, issue, and source/author indexes. - Added shared types for decision-training records, notes history, and versioned snapshots. - Added a single server-side snapshot capture path with inclusive comment cutoffs, pre-cutoff run capture, durable decision payloads, and explicit commit-resolution confidence. - Added create, list, detail, notes-only update, delete, and JSONL export routes with human-only write authorization and activity logging that skips no-op note submissions. - Added per-user `trainingExampleId` enrichment to attention items. - Added focused embedded-Postgres tests for cutoff boundaries, post-cutoff leakage, immutable snapshots, human-only writes, duplicate prevention, notes history, attention enrichment, and export shape. - Updated UI test and Storybook attention-item factories for the new required `trainingExampleId` contract. ## Verification - `pnpm exec vitest run server/src/__tests__/decision-training.test.ts` — 10 tests passed. - `pnpm --filter @paperclipai/db typecheck` — passed, including migration numbering and safety checks. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. ## Risks - The migration adds a new table and indexes only; it does not rewrite existing rows or install resolve-time hooks. - Snapshot JSON can grow with long comment threads and run histories; v1 intentionally favors complete, inspectable examples over aggressive truncation. - Commit SHA resolution is evidence-based and records `exact`, `nearest_run`, or `none` so downstream consumers can account for confidence. - The API is additive, but future UI work must continue to treat the snapshot as immutable and use notes-only updates. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.3-codex`, with repository tool use, terminal execution, and code-editing capabilities; context-window size is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1f1f545238 |
feat: add built-in summarizer and summary slots (#9713)
## Thinking Path > - Paperclip is the open source control plane people use to organize, govern, and understand AI-agent work > - Operators need concise, current status views across projects and execution workspaces without manually reading every issue and run > - Paperclip already has auditable issues, documents, built-in agents, routines, and live run events, but no first-class summary-slot workflow connecting those systems > - A built-in Summarizer can generate status prose through ordinary governed tasks while summary slots provide stable, revisioned destinations for that output > - The UI needs to show current summaries, generation progress, failures, revisions, and streaming draft status in the places operators already work > - This pull request adds the end-to-end summary-slot data, API, agent, orchestration, and UI surfaces behind an experimental setting > - The benefit is decision-oriented status context that remains company-scoped, auditable, retryable, and inexpensive by default ## Linked Issues or Issue Description No public GitHub issue exists for this feature. **Problem** Operators currently have to reconstruct project and workspace status by reading many issues, runs, and comments. This makes it hard to identify decisions, review queues, recent work, and the next event worth watching. **Proposed capability** Add an experimental summary system with revisioned summary slots for projects and workspaces, a paused-by-default built-in Summarizer agent, governed generation tasks, live draft status, and reusable UI cards. **Expected behavior** - Summary data remains company-scoped and revisions remain auditable. - Generation runs through normal issue/agent orchestration and deduplicates active requests. - Only the linked built-in Summarizer generation task can author a slot revision. - Operators can generate, retry, inspect revisions, and follow draft progress from project and workspace views. - The feature remains opt-in and background generation remains paused by default. ## What Changed - Added summary-slot schema, idempotent migrations, shared contracts, validators, API paths, and service tests. - Added company-scoped summary-slot routes for reading revisions, requesting generation, and guarded Summarizer writes with activity logging. - Added terminal generation finalization, failure reasons, assignment wakeups, and orchestration integration. - Added the paused-by-default built-in Summarizer bundle, low-cost runtime defaults, status-summarization skill, and stale-summary routine. - Added summary cards, revision selection, retry/configuration states, live draft streaming, transcript chunk handling, and project/workspace integrations. - Updated Claude local parsing for streamed status output and expanded server, adapter, shared, database, catalog, and UI coverage. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run packages/db/src/summary-slots-schema.test.ts packages/shared/src/summary-slot.test.ts server/src/__tests__/summary-slot-routes.test.ts server/src/__tests__/summary-slots.test.ts server/src/__tests__/built-in-agents.test.ts ui/src/components/SummarySlotCard.test.tsx ui/src/components/SummarySlotCard.status.test.tsx ui/src/components/useSummaryDraftStream.test.tsx ui/src/lib/summary-draft-stream.test.ts ui/src/lib/run-log-chunks.test.ts ui/src/context/LiveUpdatesProvider.hook.test.tsx` — 113 tests passed - `pnpm test:run` — server and UI suites passed; one CLI AWS doctor test was affected by inherited `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`, and passed when those host credentials were removed - `pnpm exec vitest run cli/src/__tests__/secrets.test.ts` with inherited AWS credential variables removed — 8 tests passed - `pnpm build` - `pnpm check:token-gates` currently reports nine `#9627` comment references introduced by current `master`; none are in this PR diff ## Risks - Database risk is limited by incrementally ordered, idempotent migrations and migration safety checks. - Summary generation creates normal issues/runs, so misconfiguration can produce failed slots; the UI exposes retryable failure reasons and agent configuration entry points. - Streaming draft parsing depends on the documented `STATUS:` protocol; final persisted revisions remain the source of truth. - The feature is experimental, opt-in, and its built-in routine is paused with no background token spend by default. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5.3 Codex with reasoning, repository tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Earlier branch commits also record Claude model co-authorship where applicable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
59fb27ff79 |
feat(inbox): let agents safely tidy user inboxes (#9724)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - The inbox is a per-user attention view, so archiving an item must not alter the underlying issue, assignment, or status > - Agents can help responsible users tidy resolved work only when the action is company-scoped, reversible, policy-controlled, and fully attributable > - The database and authorization foundations landed in #9654 and #9658, but the end-to-end archive routes, audit details, agent workflow guidance, and operator UI still need to ship together > - Separate stacked PRs #9659 and #9661 made the complete behavior harder to review and land as one coherent capability > - This pull request consolidates the remaining server, shared-contract, documentation, skill, and UI work on top of current master > - The benefit is a single reviewable change that lets agents safely archive responsible-user inbox items and lets users control or undo that behavior ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting inbox management across shared contracts, server authorization/routes/services, shipped agent skills, and the board UI. ### Problem or motivation Agents may complete work whose issue remains in the responsible user's Mine inbox. Existing board-user archive behavior does not provide the agent-facing policy endpoints, target resolution, heartbeat-run attribution, typed denials, conservative workflow guidance, or UI needed for safe agent-managed cleanup. ### Proposed solution Allow authorized agents to archive or unarchive responsible-user inbox items under the user's open, allowlist, or disabled policy; preserve actor/agent/run attribution in issue detail and activity records; expose policy controls and agent archive attribution in the UI; and document conservative cleanup rules for agents and PR gardening. ### Alternatives considered - Reuse generic issue mutation permissions: rejected because inbox state belongs to a target user and requires user-scoped authorization. - Automatically archive every completed or closed item: rejected because completion signals can still require human review or a decision. - Keep the backend and UI as separate stacked PRs: superseded by this consolidated PR so the complete user-visible behavior can be reviewed and verified together. ### Related work - Builds on merged foundations #9654 and #9658. - Supersedes the remaining stacked changes in #9659 and #9661. - `ROADMAP.md` has no overlapping inbox archive or inbox authorization initiative. ## What Changed - Added shared inbox-agent policy types and validators plus company-scoped self-service policy routes and OpenAPI coverage. - Enabled agent archive/unarchive mutations with responsible-user targeting, policy enforcement, typed failures, attribution, idempotency, and detailed activity auditing. - Returned agent archive attribution in issue detail and documented reversible inbox cleanup semantics in the implementation spec and Paperclip skill. - Added conservative PR-gardening inbox tidy guidance that keeps GitHub access read-only and avoids archiving work that still needs human action. - Added the Profile settings policy control and Issue Properties attribution/unarchive UI with focused component coverage and narrow-pane handling. ## Verification - `pnpm exec vitest run server/src/__tests__/inbox-archive-routes.test.ts server/src/__tests__/inbox-agent-policy-routes.test.ts server/src/__tests__/authorization-service.test.ts server/src/__tests__/openapi-routes.test.ts ui/src/components/InboxAgentPolicyControl.test.tsx ui/src/components/IssueProperties.test.tsx` — 110 passed. - `pnpm --filter @paperclipai/db exec vitest run src/inbox-archive-agent-policies-migration.test.ts` — 1 passed. - `node --test .agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` — 9 passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm check:token-gates` — changed files are clean; the repository-wide command currently reports nine unrelated pre-existing `#9627` literals outside this PR's diff. ## Risks - Agent inbox mutations broaden an existing endpoint path, so authorization and target resolution must remain fail-closed; focused route and authorization tests cover allowed and denied paths. - Archive state affects only the responsible user's inbox presentation and remains reversible; it does not mutate issue status, assignment, or visibility. - The UI policy defaults to the existing open behavior, while allowlist and disabled modes can reduce agent access. - This PR intentionally builds on #9654 and #9658 and contains no new migration number or modification to an already-applied migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.4, medium reasoning, repository tool use, shell execution, code review, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (`feat/inbox-agent-archive-complete`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
52aea90263 |
feat: organize skills with nested folders and My Skills (#9633)
## Thinking Path > - Paperclip is the open source control plane people use to organize and govern AI-agent companies > - Company skills are durable resources that users browse, import, assign, and maintain over time > - A flat skill list plus tags does not provide a stable location or hierarchy for personal, company, project-imported, and bundled skills > - Folder paths need to be canonical, company-scoped, safe to move, and preserved across re-imports without changing skill IDs > - The `/skills` UI also needs traversal, breadcrumbs, move/create flows, and a dedicated My Skills namespace that work on desktop and mobile > - This pull request adds the folder data model and APIs, reserved-root lifecycle, project import behavior, and the folder-first skills experience > - The benefit is a predictable filesystem-like organization model while tags remain available for cross-cutting classification ## Linked Issues or Issue Description Refs #9619 — the reviewed folder foundation was intentionally closed and folded into this combined feature PR. Refs #9026 — earlier flat-folder attempt superseded by this integrated implementation. Refs #3281 — related skill organization proposal; this PR uses canonical persisted folders rather than deriving groups from skill keys, and does not add hidden-skill behavior. **Feature request** - **Problem:** Skills currently lack a canonical hierarchical location, making personal skills, project imports, bundled skills, and company-authored skills difficult to traverse and manage at scale. - **Proposed behavior:** Add nested company-scoped folders with stable paths, reserved My/Projects/Bundled roots, subtree queries, safe move/create operations, and a folder-first `/skills` library UI. - **Import behavior:** New project scans file skills under `projects/<project-slug>`; later imports update content without overriding a user-selected folder. - **Alternatives considered:** Tags alone remain useful for cross-cutting classification, but they do not provide canonical location, nesting, reserved namespaces, or stable import placement. - **Roadmap alignment:** Extends the completed Skills Manager and Scheduled Routines capabilities without duplicating an active roadmap item. ## What Changed - Adds `folders` persistence for routine and skill folders, nested canonical paths, parent/slug/system-key fields, migration backfills, and reapply-safe migrations `0174`–`0175` after current master migrations. - Adds company-scoped folder CRUD, cycle/depth/namespace validation, reserved My/Projects/Bundled lifecycle, item moves, subtree filtering, and folder paths on skill results. - Preserves project-import placement: first import files into the project folder, while re-import keeps user-owned placement and stable skill IDs. - Adds the `/skills` folder tree rail, tags facet, breadcrumbs, subfolder browser, move/new-folder dialog, canonical detail location, inline tag editing, and folder-aware Studio creation. - Keeps bundled skills read-only even when their source metadata is incomplete by detecting the reserved Bundled folder and hiding selection/move actions. - Extends routine folder UI and OpenAPI coverage, and adds regression tests across migrations, services, routes, tree helpers, pages, and Studio creation. ## Verification - `pnpm exec vitest run packages/db/src/nested-skill-folders-migration.test.ts server/src/__tests__/folders-routes.test.ts server/src/__tests__/folders-service.test.ts server/src/__tests__/company-skills-service.test.ts server/src/__tests__/routines-service.test.ts ui/src/components/folders/FolderControls.test.tsx ui/src/components/folders/SkillFolderTree.test.tsx ui/src/components/folders/skill-folder-tree.test.ts ui/src/pages/CompanySkills.test.tsx ui/src/pages/Routines.test.tsx ui/src/pages/SkillStudio.test.tsx ui/src/lib/company-skill-routes.test.ts ui/src/lib/skill-create.test.ts` — 13 files, 192 tests passed. - `pnpm exec vitest run ui/src/pages/CompanySkills.test.tsx ui/src/components/folders/SkillFolderTree.test.tsx` — 2 files, 20 tests passed after preserving the existing PR's bundled-skill fixes. - `pnpm -r typecheck` — passed for all workspace packages. - `pnpm test:run` — passed in an isolated CI-like environment with inherited Paperclip runtime identity and static AWS credential variables removed. - `pnpm build` — production build passed for all workspace packages. - Greptile iteration 2 — 5/5 confidence with zero unresolved threads on commit `ff2d67aa71`. - Latest-head GitHub checks — all success, neutral, or skipped; PR is mergeable with a clean merge state. - `pnpm check:token-gates` — reports nine existing `#9627` comment false positives already present on `master`; this PR introduces no new token violation. ## Risks - **Migration/backfill:** `0174` creates the foundation and `0175` adds nested/reserved semantics. Both are ordered after current master migration `0173`, are covered by numbering/safety checks, and are designed to be reapply-safe. - **Reserved namespaces:** My, Projects, and Bundled roots are service-managed. Regression coverage prevents namespace squatting, cross-company folder use, bundled writes, cycles, and excessive depth. - **Behavioral change:** Project scans choose a project folder only on initial creation; existing skills deliberately retain their current folder during refresh. - **UI scope:** The folder rail applies to the Installed library; Catalog retains the discovery-oriented category sidebar. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI `gpt-5.5` in Codex CLI, medium reasoning mode; runtime did not expose a context-window value. Used repository/file tools, terminal execution, Git/GitHub operations, test execution, and code editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6ec059ab4e |
fix(server): suppress stale handoff alarms during live continuation (#9695)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane records a successful-run handoff when productive work ends without a durable next-step disposition > - That handoff state was derived only from the latest activity event, without checking whether a corrective run or wake was currently alive > - As a result, actively progressing issues could still show a high-severity missing-disposition alarm and blocked-inbox row > - The same stale required event could also remain indefinitely when a later successful run correctly skipped recovery because another valid continuation path already existed > - This pull request makes the derived state liveness-aware, suppresses attention only while the live path exists, and resolves stale required events on valid-path skips > - The benefit is that productive work stays calm while genuine stalls still resurface automatically when liveness disappears ## Linked Issues or Issue Description - **Bug:** An issue whose latest successful-run handoff event is `required` continues to report a missing disposition even while a heartbeat run, scheduled retry, or queued/deferred/claimed wake is actively targeting that issue. - **Expected behavior:** The API should expose current continuation liveness, the blocked inbox should suppress the alarm only while that path remains live, and a later successful run that skips recovery because a valid path exists should durably resolve the stale event. - **Related but distinct:** #9370 changes disposition freshness at detection time; #8748 adds an explicit policy opt-out. This PR preserves detection/escalation policy and fixes read-time/current-liveness state. ## What Changed - Extended `SuccessfulRunHandoffState` with `hasLiveContinuation` and optional `liveRunId` evidence. - Added bounded liveness hydration for required handoff states using active heartbeat-run and wake-request signals. - Suppressed `missing_disposition` blocked-inbox rows only while a run, scheduled retry, or live wake targets the issue. - Added durable `issue.successful_run_handoff_resolved` logging when handoff detection skips because another valid continuation path owns the next action. - Added focused regressions for live/absent derived state, self-healing attention suppression, valid-path skip classification, and resolved-event logging. - Updated UI normalization and fixtures for the shared contract without changing rendering behavior. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm vitest run server/src/services/recovery/successful-run-handoff.test.ts server/src/__tests__/issue-list-assignee-filter-routes.test.ts server/src/__tests__/issue-blocker-attention.test.ts` — 56 passed - `pnpm vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "queues one finish-handoff wake when a successful run leaves in-progress work without a next action"` — 1 passed - `git diff --check` ## Risks - Low risk: no schema or migration changes, and detection, bounded correction attempts, and escalation behavior are unchanged. - Liveness lookups are limited to issues whose latest handoff state is `required`; blocked-inbox suppression reuses rows already loaded by that query path. - Suppression is read-time and self-healing: when the run or wake stops, the alarm returns on the next fetch. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.4`, tool-enabled software-engineering workflow with repository, shell, test, Git, GitHub, and Paperclip control-plane access. Context-window size is not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a04a77c9d3 |
feat(authz): govern agent inbox archive access (#9658)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - The inbox subsystem must let agents act for a responsible user without silently granting access to every company user's tasks > - Existing authorization had no inbox-specific action, target-user scope, or per-user agent policy > - Inbox archive data also needs company-safe ownership and replay-safe schema changes before API mutations can rely on it > - This pull request adds the database policy foundation and a fail-closed `inbox:manage` authorization decision > - The benefit is a least-privilege core for later inbox archive endpoints, including explicit cross-user grants and low-trust denial ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`packages/db`, `packages/shared`, and `server`). ### Problem or motivation Agents need to manage inbox state for the user responsible for their run, but the control plane lacks an inbox-specific permission model and user-targeted grant scope. A generic mutation path would risk cross-user access or inconsistent policy enforcement. ### Proposed solution Add inbox archive ownership and per-user agent policies, introduce `inbox:manage`, and evaluate responsible-user defaults, disabled/allowlist policies, active membership, low-trust presets, and scoped cross-user grants in one authorization decision. ### Alternatives considered Reusing generic issue mutation permissions was rejected because it cannot express user-targeted inbox scope. Requiring grants for all self-user access was rejected because it would make the responsible-user path closed by default instead of using the requested per-user policy model. ### Roadmap alignment `ROADMAP.md` contains no overlapping inbox archive or inbox authorization item; this is incremental control-plane authorization work. ### Additional context This PR provides the authorization and schema foundation. Route and UI behavior can build on this decision without duplicating access-control rules. ## What Changed - Builds on the merged migration `0172_inbox_archive_agent_policies` (#9654) for company/user-scoped inbox archives and per-user agent policy rows. - Added replay-safe migration `0173_inbox_policy_agent_cleanup` with a GIN allowlist index and GIN-backed database cleanup that removes deleted agent IDs from policy allowlists. - Added Drizzle schema exports for inbox agent policies and responsible-user ownership on inbox archives. - Added the shared `inbox:manage` permission key and `scope.userIds` evaluation for user-targeted grants. - Added fail-closed inbox authorization for unresolved targets, inactive memberships, low-trust agents, disabled policies, allowlist misses, and ungranted cross-user access. - Added migration replay coverage and the full inbox authorization decision matrix. ## Verification - `pnpm exec vitest run packages/db/src/inbox-archive-agent-policies-migration.test.ts server/src/__tests__/authorization-service.test.ts` — 50 tests passed. - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check origin/master...HEAD` ## Risks - The merged `0172` migration changed inbox archive uniqueness from agent-owned to responsible-user-owned rows; `0173` is additive (index + cleanup trigger) and idempotent, and replay coverage verifies both remain safe for databases that already applied an earlier form. - `scope.userIds` uses the existing JSON grant-scope parser, so malformed privileged grant payloads continue to fail through the shared parsing behavior rather than a dedicated schema. - Cross-user grants intentionally act as board-admin overrides; responsible-user default access remains bounded by disabled and allowlist policies. - The authorization action is not yet wired to public mutation routes, limiting immediate behavioral impact while establishing the contract those routes must use. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with `gpt-5.6-sol`, high reasoning effort, CLI tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Context window size is not exposed by the configured adapter. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c65ab09d9f |
fix(recovery): wait for provider quota resets (#9635)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and keep assigned work moving safely. > - The recovery subsystem decides whether a failed agent run should retry, wait, block for configuration, or escalate to another owner. > - Provider usage-limit failures currently arrive as generic `adapter_failed` results, so stranded-work reconciliation can create takeover recovery even when the provider states that capacity will reset later. > - Credential and model lookup failures are also configuration problems, not evidence that another agent should take over the task. > - This pull request classifies those failure families at recovery time and persists the classification on the run. > - Quota failures now schedule a monitor for the original assignee at the parsed reset time, or after a bounded default backoff when no reset time is available. > - The benefit is that transient provider capacity waits no longer wake recovery owners, while configuration failures stop with an actionable classification. ## Linked Issues or Issue Description No public GitHub issue exists for this exact change. **What happened?** When an assigned issue's latest run failed with a provider usage-limit message such as "try again at 12:00 AM (UTC)," recovery treated the run as generic `adapter_failed` work and could create a takeover action. Missing credentials and `model_not_found` failures followed the same generic path. **Expected behavior:** Provider quota failures should keep the original assignee and schedule a monitor for the reset time, without creating recovery work or immediately waking another owner. Missing credentials and model lookup failures should be classified as `configuration_incomplete` and blocked with the configuration fix recorded. **Steps to reproduce:** 1. Assign and start an issue for an agent. 2. Record a failed heartbeat run with `errorCode: adapter_failed` and a provider quota/reset message. 3. Run stranded assigned-issue reconciliation. 4. Observe that the old behavior routes the issue through generic recovery instead of waiting for provider capacity. Reproduced on `master` at `9af96461d`. This is a core recovery bug, not adapter-specific, and applies to built-from-source deployments with either embedded PGlite or Postgres. Related work checked: #9288 adds adapter-side Claude provider-limit classification; #5392 suppresses some recovery creation for quota-class errors; #9634 is a broader recovery-routing change with overlapping provider-quota behavior. This PR is the narrow recovery-service fix with focused parsed-reset, fallback-backoff, zero-takeover, and configuration-failure coverage. ## What Changed - Added conservative recovery-time classification for provider quota, missing-credential, and model-not-found adapter failures. - Parsed provider reset timestamps with a default one-hour backoff when no usable reset time is present. - Persisted `provider_quota` or `configuration_incomplete` metadata on the failed heartbeat run. - Scheduled quota monitors for the active issue owner, including the current review participant, without creating recovery actions or enqueueing takeover wakes. - Routed configuration failures to blocked recovery with actionable evidence instead of a takeover. - Added unit and embedded-database regression coverage for parsed/fallback quota timing, zero CTO/recovery wake behavior, and configuration classification. ## Verification - `pnpm exec vitest run server/src/services/recovery/provider-failure-classification.test.ts server/src/__tests__/issue-recovery-actions.test.ts server/src/__tests__/issue-monitor-scheduler.test.ts server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts` — 4 files passed, 66 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. ## Risks - Recovery behavior changes for text-matched adapter failures; matching is intentionally conservative, and unmatched failures retain the existing generic recovery path. - Provider reset strings do not always include a date or timezone; parsing chooses the next future matching time and falls back to a one-hour wait when the timestamp is unusable. - This overlaps the provider-quota portion of broader recovery-routing PR #9634, so only one implementation should land if both remain open. - No schema, migration, API contract, or UI changes are included. No documentation update is needed because this corrects internal recovery behavior without changing operator commands or configuration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with model `gpt-5.4`, medium reasoning, tool use, and code execution. The runtime does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd7c0d5f83 |
fix(issues): deduplicate repeated creates (#9650)
## Thinking Path > Paperclip already treats issue creation as a company-scoped mutation, but retries and parallel agent heartbeats can submit the same create more than once. Client instructions cannot provide at-most-once behavior under concurrency, so the guard belongs in the server transaction. This change adds an explicit company-scoped idempotency contract, a conservative fallback for recent open same-parent titles, and run attribution for auditability. Advisory transaction locks serialize competing requests before lookup/insert, avoiding the race that affected the prior attempt. ## Linked Issues or Issue Description Fixes #6529. This is a clean replacement for #6936, which was closed because it mixed unrelated changes and its check-then-insert implementation was not concurrency-safe. Unlike that attempt, this PR is scoped to eight files, uses a dedicated idempotency-key table, and serializes duplicate candidates inside the create transaction. ## What Changed - Accept optional `idempotencyKey` and `allowDuplicate` fields on issue creation. - Replay the existing issue with HTTP 200 and deduplication metadata for a repeated company/key pair. - Deduplicate recent open issues with the same company, parent, and normalized title for 48 hours unless `allowDuplicate: true` is supplied. - Persist idempotency mappings in a company-scoped table and serialize competing creates with transaction advisory locks. - Populate `originRunId` from `X-Paperclip-Run-Id` for agent/manual creates when the body does not provide an origin run. - Add route integration coverage for key replay, title fallback, bypass, closed/old recreation, company scoping, and run attribution. ## Verification - `pnpm exec vitest run server/src/__tests__/issue-create-deduplication-routes.test.ts` — 7 tests passed. - `pnpm --filter @paperclipai/db typecheck` — passed, including migration numbering and safety checks. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check origin/master...HEAD` — passed. - `pnpm exec vitest run server/src/__tests__/issue-assigned-backlog-contract-routes.test.ts server/src/__tests__/issue-create-deduplication-routes.test.ts` — 10 tests passed after the service-contract compatibility fix. ## Risks - The title fallback intentionally treats normalized same-parent titles as duplicates for 48 hours; callers creating intentionally repeated titles must send `allowDuplicate: true`. - Advisory locks use hashed duplicate keys, so an extremely unlikely hash collision can serialize unrelated creates but cannot merge their lookup results. - Deleting an issue cascades its idempotency mapping, allowing the same key to create a replacement later. ## Model Used - OpenAI `gpt-5.6-sol`, high reasoning effort, Codex CLI with repository, shell, GitHub CLI, and Paperclip API tool access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ea0e899905 |
fix(search): honor extract match limits + harden pr-gardening candidate discovery (#9652)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The `/pr-gardening` skill drives a bundled agent that scans a
company's issues for those linked to open GitHub PRs, then reports on
their state; it relies on the server's company-search **extract**
endpoint to pull PR references out of issue bodies
> - Two gaps surfaced during end-to-end QA of the gardening workflow:
the extract service silently ignored a per-issue match cap, so callers
could not bound how many matches came back per issue, and the skill's
candidate-discovery scripts fell over on large repos and on issues that
referenced deleted PRs
> - Left unaddressed, the gardener either truncated its scan
unpredictably or aborted outright, so it could not reliably enumerate PR
candidates
> - This pull request honors an explicit `matchesPerIssue` limit in the
extract search API and hardens the skill's candidate discovery against
missing/unavailable PRs and oversized `gh` output
> - The benefit is a PR-gardening workflow that scans deterministically
and finishes cleanly on real-world companies
## Linked Issues or Issue Description
No pre-existing public GitHub issue — describing the bug in-PR following
the bug report template (`.github/ISSUE_TEMPLATE/bug_report.yml`).
### What happened?
The company-search extract endpoint accepted a per-issue match limit but
did not apply it, returning matches capped only by the old hardcoded
constant regardless of the caller's request. Separately, the
`/pr-gardening` skill's candidate-discovery scripts crashed when a
scanned issue referenced a deleted PR (GitHub `Not Found (HTTP 404)` /
GraphQL `Could not resolve to a PullRequest`) and could exceed the
default `gh` output buffer on large result sets, aborting the whole
scan.
### Expected behavior
The extract API bounds matches per issue when a caller passes
`matchesPerIssue` (default 20, max 200), and omitting it preserves the
previous default. The gardening scripts skip PRs that are
deleted/unavailable and tolerate large `gh` responses without aborting
the scan.
### Steps to reproduce
1. Call the company-search extract endpoint with a `matchesPerIssue`
value against an issue containing many PR references — previously the
value was ignored.
2. Run the pr-gardening candidate scan against a company whose issues
reference a since-deleted PR — previously the scan threw instead of
skipping that PR.
### Paperclip version or commit
`master` at the base of this PR (branch cut from current
`origin/master`).
### Deployment mode
Local Paperclip instance / self-hosted.
## What Changed
- **Extract search honors `matchesPerIssue`**: added the
`matchesPerIssue` field to the shared search validator/types and applied
the cap in `company-search-extract` so results are bounded per issue
(`packages/shared`, `server/src/services/company-search-extract.ts`,
`doc/SPEC-implementation.md`).
- **Hardened pr-gardening candidate discovery**: `find-candidates.mjs` /
`lib.mjs` now request `matchesPerIssue=200`, treat missing/unavailable
PRs (deleted PR → `isMissingPullRequestError` / `unavailable`) as skips
instead of fatal errors, and raise the `gh` `maxBuffer` to 50 MB for
large repos.
- **Tests**: expanded `company-search-extract-{routes,service}.test.ts`
for the new limit and added coverage in `pr-gardening.test.mjs`.
## Verification
Re-run on a fresh worktree cherry-picked onto current `master`:
- `node --test
.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` → 8/8 pass
- `pnpm vitest run
server/src/__tests__/company-search-extract-routes.test.ts
server/src/__tests__/company-search-extract-service.test.ts` → 10/10
pass
## Risks
Low risk. `matchesPerIssue` is optional and backward-compatible
(omitting it preserves prior behavior). The skill changes only add
skip/tolerance paths and a larger buffer; no schema or migration
changes.
## Model Used
Claude — Opus 4.8 (`claude-opus-4-8`), extended thinking, tool use /
code execution via the Claude Agent SDK.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
DottaandPaperclip
|
d32ed88443 |
fix(recovery): route recovery by failure cause (#9634)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies and their work. > - Its recovery subsystem detects stranded issue execution and decides whether to retry, escalate, or request operator intervention. > - The existing recovery path used a mostly generic owner ladder and generic execution contract, so transient failures could wake a manager who then performed the deliverable instead of repairing and returning the task. > - Provider quota failures also entered the same takeover path even when the correct action was to wait for capacity and retry the original assignee. > - Recovery actions already retain the source owner and evidence needed to choose a cause-specific route, render a scoped contract, and measure whether work was handed back. > - This pull request adds a cause-keyed recovery playbook, propagates its contract through every built-in adapter, and makes resolved recovery actions return work to the original owner by default. > - The benefit is bounded self-recovery that preserves task ownership, avoids needless management takeover, and makes recovery outcomes observable. ## Linked Issues or Issue Description No matching public GitHub issue was found. Related recovery work was reviewed but is not duplicated here: #9630 restores bounded recovery continuations, #8807 changes one assignee-ranking case, and #9404 records runtime-failure transition evidence. This change instead introduces cause-specific routing and recovery contracts across the recovery lifecycle. ### What happened? When an issue became stranded, recovery generally selected an owner through the same fallback ladder and rendered the normal execution contract. That made the recovery wake look like ordinary deliverable work, even when the correct action was to retry the original agent, repair its runtime, or wait for a provider quota reset. ### Expected behavior Recovery should select a response by failure cause, tell the recipient to recover rather than complete the deliverable, suppress takeover wakes for provider quota waits, and return repaired work to its original assignee unless the recovery owner explicitly completes it. ### Actual behavior Recovery could escalate transient failures to management, omit the cause-specific next action from the wake, and leave the recovery owner assigned after the runtime problem was resolved. ### Impact The generic path creates avoidable management work, ownership churn, and budget consumption while obscuring whether recovery successfully returned work to the responsible agent. ## What Changed - Added cause-keyed routing for process loss, missing disposition, provider quota limits, Codex output inactivity, workspace validation failures, and fallback recovery causes. - Added recovery-scoped wake rendering that replaces the generic execution contract with the failure summary, original assignee, attempt count, next action, and cause-specific playbook instruction. - Propagated the structured recovery contract through all built-in adapter execution paths, including Hermes local and gateway adapters. - Added provider-quota wait monitoring so capacity failures schedule the original assignee instead of enqueueing a takeover wake. - Added hand-back behavior and `handed_back` / `owner_completed` outcome accounting when recovery actions are resolved. - Added focused routing, renderer, quota-monitor, and hand-back regression coverage plus implementation-spec documentation. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/heartbeat-workspace-branch-containment.test.ts server/src/__tests__/issue-recovery-actions.test.ts` - 4 test files passed; 194 tests passed. - Targeted `pnpm --filter ... typecheck` across `@paperclipai/adapter-utils`, `@paperclipai/shared`, `@paperclipai/server`, `@paperclipai/ui`, and all nine changed adapter packages. - 13 affected workspace packages passed typecheck. - `pnpm check:token-gates` - All UI token gates passed. ## Risks - Recovery routing behavior changes for stranded work, so an incorrectly classified cause could select a different recipient than before; fallback causes retain the existing management ladder. - Provider quota detection depends on structured failure evidence and conservative text matching; unmatched failures continue through fallback recovery. - Adapter prompt plumbing changes across built-ins, covered by shared renderer tests and compile-time call signatures. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with exact model ID `gpt-5.6-sol`, using reasoning, tool use, and code execution. The runtime does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ae77908618 |
feat(search): add bulk extract endpoint (#9507)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies > - Agents and operators need company-scoped search to discover relevant issue history safely > - The interactive search endpoint intentionally returns compact excerpts and low pagination caps for UI use > - Automation that inventories repeated references, such as pull-request URLs, needs exhaustive distinct matches without loading full issue objects into an LLM context > - Client-provided regular expressions would create an unsafe and expensive query surface, so extraction must remain literal with server-owned expansion modes > - This pull request adds a bounded agent-oriented extraction endpoint with explicit truncation > - The benefit is deterministic, compact bulk discovery across issues, comments, and documents while preserving company authorization and rate limits ## Linked Issues or Issue Description ### Subsystem affected `server/` REST API and `packages/shared/` contracts. ### Problem or motivation The existing interactive company search caps issue pagination and snippets, so automation cannot reliably enumerate every distinct literal or pull-request URL across issue descriptions, comments, and linked documents without fetching large full issue payloads. ### Proposed solution Add `GET /api/companies/:companyId/search/extract` with escaped literal matching, optional server-owned URL token expansion, issue/comment/document scopes, status/date filters, higher issue-level pagination caps, compact source references, and explicit pagination/match truncation flags. ### Alternatives considered Reusing `GET /issues?q=` would return unnecessarily large issue objects; increasing interactive-search snippet limits would make the UI API heavier; accepting arbitrary client regex would expose avoidable database cost and ReDoS risk. ### Roadmap alignment `ROADMAP.md` does not currently list a conflicting company-search or bulk-extraction initiative. GitHub searches found no directly duplicative open issue or pull request. ## What Changed - Added shared query validation and response contracts for literal and URL extraction. - Added a company-scoped extraction service that pages issues, gathers matching issue/comment/document sources, expands URL tokens, deduplicates values, and reports truncation explicitly. - Added the authenticated route using the existing company-search authorization decision and rate limiter. - Added targeted Vitest coverage for URL extraction, multi-source dedupe, date/status filters, match caps, cross-company denial, and rate limiting. - Documented the extraction surface in the implementation specification. ## Verification - `pnpm exec vitest run server/src/__tests__/company-search-extract-service.test.ts server/src/__tests__/company-search-extract-routes.test.ts server/src/__tests__/company-search-rate-limit-routes.test.ts server/src/__tests__/company-search-service.test.ts` — 30 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check` — passed. ## Risks - Bulk substring search can scan large text columns. The endpoint mitigates this with a minimum literal length, bounded issue pagination, a 20-distinct-match cap per issue, explicit truncation, existing company-search rate limiting, and no client-provided regex. - URL expansion uses a fixed server-owned pattern plus an escaped literal. A security review is requested as part of PR review to confirm the pattern and abuse controls. - No database migration or existing API response shape changes are included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI coding agent; exact runtime model ID and context-window size were not exposed to the session. Tool-enabled code execution and repository editing were used with medium reasoning effort. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3ae2c30f2f |
feat(skills): import skills from projects (#9620)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Company skills make reusable agent behavior discoverable and editable from one place. > - Projects already contain skill directories, but operators had to import each skill path manually. > - Copying those skills would break the desired write-through workflow between Skill Studio and the source project. > - The server therefore needs a safe preview/select/import contract that only accepts rediscovered, workspace-contained candidates. > - The UI needs a guided project picker that explains reference semantics, handles conflicts, and remains usable on mobile. > - This pull request adds that end-to-end project skill import flow with authorization, tenant-scope, traversal, and symlink regression coverage. > - The benefit is faster bulk onboarding while keeping project files as the single source of truth. ## Linked Issues or Issue Description **Feature request** **Problem:** Importing several skills already stored in a Paperclip project requires operators to discover and submit each local path individually. This is slow, hides which well-known directories were searched, and makes conflict/already-imported states difficult to evaluate before mutation. **Proposed solution:** Add an “Import skills from project” flow that previews skills from well-known directories, lets operators selectively import eligible candidates, and stores local-path references so Skill Studio edits write through to the project files. **Alternatives considered:** Copying files into company-managed skill storage was rejected because it creates divergent copies. Trusting client-supplied paths was rejected because imports must be constrained to server-rediscovered, workspace-contained candidates. **Additional context:** GitHub duplicate search found no existing issue or PR for this exact workflow. Refs #3799 for related skill-import inventory behavior; this PR does not claim to close that issue. ## What Changed - Extend `scan-projects` with backward-compatible preview and selective-import modes, typed validation, candidate statuses, and OpenAPI coverage. - Discover project skills under `skills`, `.agents/skills`, `.claude/skills`, `.codex/skills`, `.cursor/skills`, `.opencode/skills`, and `.gemini/skills`. - Re-discover selections server-side, enforce company/project/workspace scope, and reject traversal or symlink escapes before creating `local_path` references. - Add the Skills-page menu entry and responsive project import dialog with project selection, grouped candidates, select all/deselect all, conflicts, empty/error/403 states, and import results. - Add route, service, and component regressions for preview authorization, cross-tenant selections, traversal/symlink safety, selection counts, grouping, and result semantics. ### Screenshots **Choose a project**  **Review discovered skills**  **Mobile selection footer**  **Import result**  ## Verification - `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts ui/src/pages/skills/ImportSkillsFromProjectDialog.test.tsx` — 3 files, 81 tests passed. - `pnpm check:token-gates` — all token gates clean. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - Security review passed after adding tenant-scope and unauthorized-preview regressions; UX re-review approved desktop/mobile surfaces; QA passed all seven acceptance areas including write-through editing, deduplication, conflicts, empty state, and permission denial. ## Risks - Files remain referenced in project workspaces, so moving or deleting a source directory can make an imported skill unavailable; the UI explicitly communicates the reference behavior. - New well-known directory scans may discover more candidates than older versions, but preview mode prevents mutation until the operator confirms a selection. - The endpoint remains backward compatible: omitting `mode` preserves the prior full-import behavior. - No schema migration or telemetry event changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 4.8 with tool use/code execution assisted with the UI implementation and UX polish. OpenAI Codex CLI with tool use/code execution assisted with server implementation, security fixes, regression coverage, integration, and PR preparation; the runtime did not expose Codex's exact backing model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
89ce36d7af |
feat(skills): open-by-default company skill policy and core UX (#9564)
## Thinking Path > - Paperclip uses company skills to make agent capabilities reusable across an organization. > - Skill operations currently mix capability availability with permission checks, which creates avoidable setup friction and inconsistent denial handling. > - The policy contract needs to remain open by default while allowing company-scoped restrictions for governed deployments. > - Core owns the canonical policy actions, persistence, evaluation, API behavior, safe import boundaries, and generic denial/read-only UI. > - Enterprise policy-editor implementation belongs in the separate `paperclip-ee` repository and is intentionally excluded from this PR. ### Problem or motivation Company skill operations can encounter permission dead ends even when no explicit restriction has been configured, and import-source classification can drift between policy evaluation and execution. ### Proposed solution Define eight canonical skill policy actions, default all actions to allowed, persist company-scoped restrictions, expose policy evaluation APIs, normalize import sources at the boundary, and update Skill Studio to present actionable restriction states without embedding Enterprise Edition implementation in the core repository. ### Alternatives considered Keeping capability checks distributed across routes and UI surfaces was rejected because it duplicates policy logic and makes denial behavior inconsistent. Shipping the Enterprise policy editor in this repository was rejected because `paperclip-ee` is a separate repository and must receive its own PR. ### Roadmap alignment Extends the completed **Skills Manager** roadmap area by adding coherent governance and removing workflow dead ends. ### Additional context The core API contract remains suitable for a separate Enterprise Edition editor, but this PR contains no `paperclip-ee` package or EE-specific UI integration code. ## What Changed - Added the company skill policy contract to product and implementation documentation, including the open-by-default rule, eight canonical actions, decision shape, and core/EE ownership boundary. - Added the company-scoped policy schema, migration `0170`, shared validators, policy service, REST routes, OpenAPI coverage, and focused server tests. - Hardened import policy enforcement by normalizing import sources and keeping source classification consistent between policy evaluation and execution. - Updated core Skill Studio behavior to remove generic permission dead ends and show actionable policy/platform denial states only when an operation is actually denied. - Removed the `plugin-paperclip-ee` package, Docker wiring, EE discovery/deep-link helpers, and EE-specific UI tests/stories from this PR so that implementation can be submitted separately to the EE repository. - Preserved open-by-default behavior when no explicit company restriction exists. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/components/skill-studio/SkillPolicySurfaces.test.tsx src/lib/skill-policy-denial.test.ts` — 20/20 passed. - `pnpm --filter @paperclipai/ui exec tsc --noEmit` — passed. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/worktree-config.test.ts` — 12/12 passed. - `pnpm check:token-gates` — passed with all gates clean. - `git diff --check` — passed. - `git diff --name-only origin/master | rg 'paperclip-ee|ee-skill-policy'` — no matches. ## Risks - Migration `0170` introduces company policy persistence; rollout depends on the migration applying before policy routes are exercised. - Open-by-default is an intentional behavioral policy: deployments expecting implicit denials must configure explicit restrictions. - Import normalization is security-sensitive and should retain focused review. - The separate EE editor must stay contract-compatible with the core policy API as policy actions evolve. ## Model Used - OpenAI Codex CLI, runtime model identifier and context-window size not exposed by this execution environment; reasoning, repository tool use, shell execution, and code review capabilities enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available to this runtime) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked or described the result above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the latest head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Evyatar Bluzer <bluzername@users.noreply.github.com> |
||
|
Nicky LeachandPaperclip
|
7947308276 |
fix(codex): classify refresh auth failures (#9598)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip supports the Codex local adapter, which runs OpenAI Codex CLI sessions on behalf of agents > - Codex uses OAuth refresh tokens to maintain long-running authenticated sessions > - When a refresh fails, the failure has distinct root causes: a refresh token was already reused in a parallel request, the token expired by TTL, or the token was invalidated/revoked by the provider > - Without classifying these failure modes, all refresh auth errors surface identically — operators cannot distinguish retryable transient collisions from permanent invalidations, and run logs carry no actionable diagnosis > - This pull request adds structured classification (`refresh_token_reused`, `refresh_token_expired`, `refresh_token_invalidated`) of Codex refresh-token auth failures across the CLI quota-probe, ACP auth path, and execute path > - The benefit is that these distinct failure modes can be surfaced in run logs and acted on appropriately — transient reuse can be retried; true invalidations require re-auth ## Linked Issues or Issue Description <!-- Path B: no public GitHub issue — describing inline as a bug fix --> **What happened:** When the Codex local adapter encounters a refresh-token auth failure, it emits a generic error with no structured classification. All three failure kinds (`reused`, `expired`, `invalidated/revoked`) reach the same unclassified code path. **Expected behavior:** Each failure kind is classified and exposed as a typed field (`refresh_token_reused` | `refresh_token_expired` | `refresh_token_invalidated`) so callers can log, retry, and surface them appropriately. **Steps to reproduce:** 1. Run a Codex agent session with a reused or expired OAuth refresh token. 2. Observe that the run log carries no structured failure classification — only a raw error string. **Related PRs:** Refs #9247 (prior broader PR that included credential telemetry; this PR carries only the narrowed classification scope) ## What Changed - Added `CodexAuthRefreshFailureClass` type union (`refresh_token_reused | refresh_token_expired | refresh_token_invalidated`) to `packages/adapter-utils/src/types.ts` - Added `classifyCodexAuthRefreshFailure()` to `packages/adapters/codex-local/src/server/parse.ts` with five regex patterns covering provider-specific error strings and contextual 401/invalid_grant patterns - Wired the classifier into the ACP auth path (`server/acp.ts`), execute path (`server/execute.ts`), and CLI quota-probe (`cli/quota-probe.ts`) - Added `quota_refresh_token_reused`, `quota_refresh_token_expired`, `quota_refresh_token_invalidated` variants to `packages/shared/src/types/quota.ts` - Added classification unit tests (`parse.test.ts`, `quota-spawn-error.test.ts`, `acp.test.ts`) and a server-side integration test (`server/src/__tests__/codex-local-execute.test.ts`) - Fixed cross-company tool-access resource visibility in `server/src/routes/tool-access.ts` - Stabilized `heartbeat-retry-scheduling.test.ts` (CASCADE cleanup), `heartbeat-run-log.test.ts`, and `quota-windows.test.ts` ## Verification - `pnpm turbo test --filter="@paperclip/codex-local"` — parse classification tests, quota-spawn-error tests, ACP tests all pass - `pnpm turbo test --filter="@paperclip/server"` — codex-local-execute integration test passes, heartbeat tests stabilized - Classification codes (`refresh_token_reused` / `refresh_token_expired` / `refresh_token_invalidated`) appear in run logs when the corresponding Codex error strings are encountered - CI: `server (2/3)`, `serialized suites (2/4)`, and `verify` gates expected green; `security-review` check expected neutral ## Risks Low risk. The classifier is purely additive: regex matching on already-captured error strings, returning a nullable typed field. Callers that do not inspect the classification field are unaffected. No execution paths, retry logic, or existing error surfaces changed. ## Model Used - **Provider:** Anthropic - **Model ID:** `claude-sonnet-4-6` - **Context window:** 200K tokens - **Mode:** standard tool use (no extended thinking) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes StubbemannandClaude Fable 5
|
543de323f6 |
fix(shared): tolerate empty-string user name in profile/session parse (#8986)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Humans sign in through the auth layer; every authenticated request
parses the session/user profile with `currentUserProfileSchema` in
`packages/shared`
> - The schema requires `name` to be `null` or a non-empty string, but
some identity providers hand back `name: ""` for users who never set a
display name
> - For those users the session payload fails validation on every
request, so the app treats them as unauthenticated and bounces them to
`/auth` in a loop — they can never get in
> - This pull request preprocesses empty/whitespace-only names to `null`
before validation, so the existing `min(1).max(120).nullable()` rule
still holds for real names
> - Review found the sibling `email` field has the same failure mode
(the DB `auth` schema declares `email` as `notNull`, so a provider that
supplies no email stores `""`, which `z.string().email()` rejects); the
same preprocess is applied there
> - The benefit is that users whose provider reports an empty name (or
email) can sign in normally instead of being locked out, with no change
in behavior for anyone else
## Linked Issues or Issue Description
No existing issue; described in-PR following the bug report template:
**What happened?**
Users whose auth provider returns `name: ""` (empty string) in the
profile payload fail `currentUserProfileSchema` / `authSessionSchema`
parsing (`name: z.string().min(1)...`). The parse failure makes the
session look invalid and the UI redirects to `/auth` on every attempt —
an endless sign-in loop. The `email` field has the same failure mode
(`z.string().email()` rejects `""`).
**Expected behavior:**
An empty display name (or email) should be treated the same as a missing
one (`null`); the user should be signed in normally.
**Steps to reproduce:**
Sign in with an account whose upstream identity record has an
empty-string name (or set a user's `name` column to `''` directly), then
load the app: session parse fails and you are bounced back to `/auth`.
**Adapter(s) involved:**
Not adapter-specific (core bug).
**Deployment mode / version:**
Any; reproduces on current `master`.
## What Changed
- `packages/shared/src/validators/access.ts`:
`currentUserProfileSchema.name` now runs through `z.preprocess` that
coerces empty or whitespace-only strings to `null` before the existing
`z.string().min(1).max(120).nullable()` validation.
- `packages/shared/src/validators/access.ts`: the same preprocess is
applied to `email` (review follow-up): `users.email` is `notNull` in the
DB schema, so a provider without an email stores `""`, which
`z.string().email()` rejects — the identical lockout loop.
Empty/whitespace-only emails now coerce to `null` (the field was already
nullable); malformed non-empty emails are still rejected.
- `packages/shared/src/validators/access.test.ts` (new): covers
empty-string → `null`, whitespace-only → `null`, real values preserved,
`null` preserved, and malformed non-empty email still rejected — for
both `name` and `email`, and the same cases through `authSessionSchema`.
## Verification
- `vitest run src/validators/access.test.ts` in `packages/shared` — 13
tests pass.
- `tsc --noEmit -p packages/shared` passes.
- Manual: parse `{ id, email: "", name: "", image: null }` with
`currentUserProfileSchema` — succeeds with `name: null` and `email:
null` instead of failing validation.
## Risks
- Low risk. The change only widens accepted input (empty/whitespace
string → `null` for `name` and `email`); every previously valid payload
parses identically. `updateCurrentUserProfileSchema` (user-initiated
rename) is untouched and still rejects empty names.
## Model Used
Claude Fable 5 (Anthropic, `claude-fable-5`, agentic coding harness via
Claude Code, extended reasoning enabled). Original fix drafted with
Claude Sonnet 4.6.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
DottaandPaperclip
|
1de0a3bb1e |
feat(mcp) [split 2/8]: add governed access contracts (#9557)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Governed MCP access spans contracts, runtime enforcement, adapters, UI surfaces, and operator verification > - The parity reference PR #9534 is too large for effective automated or human review > - The feature therefore needs a linear stack whose individual diffs stay below the 100-file review limit > - This pull request is split 2/8 and focuses on database schema and shared governance contracts > - The benefit is a standalone, testable review boundary while preserving byte-for-byte parity at the top of the stack ## Linked Issues or Issue Description - Related parity reference: #9534 - Problem: The governed access model needs additive persistence and synchronized shared types before server enforcement can compile. - Proposed solution: Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared types/validators/gallery helpers, and the minimal compile-required contract consumers identified by boundary testing. - Alternatives considered: keeping #9534 as one 403-file review, or rewriting the feature to manufacture seams; both were rejected in favor of path extraction plus compile-driven boundary moves. - Roadmap alignment: this advances the existing governed MCP/tool-access work already represented by #9534; it does not introduce a separate roadmap initiative. - Stack position: base branch is `pap10341-split/01-demo-servers`. - Merge policy: merge bottom-up, in order, only after the complete eight-PR stack has been reviewed and the top-of-stack parity gate remains empty. - Requested review: QA for migrations/validators; Greptile on every PR. ## What Changed - Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared types/validators/gallery helpers, and the minimal compile-required contract consumers identified by boundary testing. - Keeps this PR below 100 changed files and independently typecheckable. - Preserves the final tree from #9534 when combined with the other seven stack levels. ## Verification - `pnpm typecheck` — passed, including migration numbering and safety checks - `pnpm --filter @paperclipai/db test` — passed - `pnpm --filter @paperclipai/shared test` — passed ## Risks - Migration or contract mistakes could affect every upper layer; all migrations are additive/idempotent and compile consumers are included in this boundary. - Stack risk: merging out of order can expose incomplete layers; mitigate by following the documented bottom-up merge policy. - Parity risk: later edits to an intermediate branch can drift from #9534; mitigate by re-running the empty top-of-stack diff before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context window; medium reasoning with repository, shell, Git, GitHub CLI, and code-execution tools enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] Internal references are omitted except the execution-plan link explicitly required for this coordinated split stack - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Stack Coordination - Internal execution plan: [PAP-13874](/PAP/issues/PAP-13874#document-plan) - Parity reference: #9534 - Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563 - Merge bottom-up only after full-stack review and an empty parity diff at #9563. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
24bbc0f57e |
docs: add telemetry marker workflow (#9543)
## Thinking Path
> - Paperclip is an open-source agentic AI management platform that
ships anonymous usage telemetry to understand product health and
adoption.
> - The telemetry system has a generated contract
(`packages/shared/src/telemetry/generated/paperclip-telemetry.ts`) that
types every first-party event the product emits.
> - When a product change needs a new first-party event that is not yet
in the generated contract, contributors had no public workflow
explaining how to propose an event or later promote it into the contract
once accepted.
> - The gap leads to confusion at call sites: contributors either skip
tracking entirely or emit untyped events that bypass the privacy and
governance safeguards built into the contract.
> - This PR fills that gap by adding `doc/TELEMETRY_WORKFLOW.md` — a
public contributor guide that covers the full propose → promote
lifecycle: the `@ts-expect-error -- proposed-telemetry(...)` marker, the
canonical multi-line `track()` shape, the TS2578 expiry signal, and the
look-up-by-event-name promotion step.
> - `packages/shared/src/telemetry/README.md` gains a cross-reference so
readers of the data-contract doc can find the workflow guide without
searching.
> - `README.md` gains a one-line pointer in the telemetry section so the
workflow is discoverable from the project entry point.
## Linked Issues or Issue Description
No pre-existing public GitHub issue covers this doc gap. Inline
description:
**Problem:** Contributors adding product telemetry for events not yet in
the generated contract have no documented workflow. The
`@ts-expect-error -- proposed-telemetry(...)` pattern exists in the
codebase but is undocumented, leading to inconsistent usage and missing
adoption signals.
**Solution:** A new public guide (`doc/TELEMETRY_WORKFLOW.md`) documents
the marker format, the recommended multi-line `track()` shape that
preserves the TS2578 expiry signal, the dimension rules, and the
promotion checklist. Cross-references are added to
`packages/shared/src/telemetry/README.md` and the top-level `README.md`.
No related open PRs found.
## What Changed
- **New file `doc/TELEMETRY_WORKFLOW.md`** — public contributor guide
for the propose/promote lifecycle: marker format, canonical multi-line
example, TS2578 single-line trap, dimension rules, and step-by-step
promotion checklist.
- **`packages/shared/src/telemetry/README.md`** — added one-line
cross-reference pointing at `doc/TELEMETRY_WORKFLOW.md` for proposed
events not yet in the generated contract.
- **`README.md`** — added one-line pointer in the telemetry section so
the new workflow guide is reachable from the top-level project entry.
## Verification
This is a docs-only change. Verification steps:
1. Open `doc/TELEMETRY_WORKFLOW.md` and confirm it contains:
- The `^[a-z0-9][a-z0-9._:-]{1,63}$` event-name grammar.
- The exact `// @ts-expect-error -- proposed-telemetry(<issue>):
<rationale>` marker.
- The multi-line `client.track()` copy-paste example (directive on line
before event-name string).
- The explanation of the TS2578 single-line trap and the
look-up-by-event-name step.
- The promotion checklist in the "Promote An Event" section.
2. Confirm `packages/shared/src/telemetry/README.md` cross-references
`doc/TELEMETRY_WORKFLOW.md`.
3. Confirm the `README.md` telemetry section links to
`doc/TELEMETRY_WORKFLOW.md`.
## Risks
Low risk — docs-only change. No runtime behavior, schema, or existing
telemetry emission is affected. The risk is that the guidance could
diverge from the actual enforcement in the codebase over time; mitigated
by linking to the generated contract and keeping the doc in the same
repo.
## Model Used
Claude — `claude-sonnet-4-6` (Anthropic Claude Sonnet 4.6). Tool use
enabled. Extended context. Produced via the Paperclip agentic workflow
with `Co-authored-by: Paperclip <noreply@paperclip.ing>`.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
Nicky LeachandPaperclip
|
90f85a7d11 |
Add telemetry proposal extractor (#9544)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - It emits telemetry events to understand product usage — registered event names are gated by a generated `PAPERCLIP_EVENTS` registry so the client only enqueues known, schema-approved events > - When product teams want to instrument a new behaviour, they must first register the event name — but schema registration is a commit-and-release cycle, which creates friction in fast-moving product iterations > - A proposal lane is needed: let developers mark a `track()` call with a typed `@ts-expect-error` proposal marker so the event name can be reviewed and tracked in CI before the schema is formally registered > - The existing client had no guard against unregistered event names, so any call with an out-of-registry name (or a prototype-inherited key) would silently enter the queue, state, and network flush path > - This PR adds an `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the entry point of `track()` to swallow unregistered calls before any side effects, adds `scripts/extract-proposed-events.mjs` to scan source for proposal markers and emit a v2 JSON manifest with provenance and rationale, and documents the complete proposal workflow > - The benefit is that new instrumentation can be proposed and reviewed in code without touching the registered schema, and tooling can surface missing rationale before events graduate to stable ## Linked Issues or Issue Description No existing GitHub issue covers this change. This PR introduces a new feature. **Feature motivation:** Paperclip's telemetry schema is intentionally stable — registered event names are code-generated and gated. Product engineers who want to instrument a new behaviour today must land a schema change first, creating a two-step process that slows iteration. A proposal lane lets developers write the instrumentation call ahead of schema registration, protected by a compile-time `@ts-expect-error` marker that an extractor script can surface for review. This PR implements both the client-side safety gate and the extraction tooling. Refs: #9518 (closed predecessor — docs-only; this PR supersedes it with the full implementation) ## What Changed - Added `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the top of `TelemetryClient.track()`: unregistered event names (including prototype-inherited keys) are now swallowed before any state, queue, or network operation - Added `scripts/extract-proposed-events.mjs`: scans TypeScript source for `@ts-expect-error -- proposed-telemetry(<issue>): <rationale>` markers; emits a v2 JSON manifest per proposed event including name, rationale, provenance (repo-relative file + line), and a `rationale_missing` flag for CI enforcement - Added `scripts/extract-proposed-events.test.mjs`: test suite covering marker parsing, multi-line markers, path validation, out-of-repo rejection, and the v2 schema output contract - Added `doc/TELEMETRY_WORKFLOW.md`: documents the proposal workflow, the canonical multi-line marker example, rationale requirements, and how to graduate a proposed event to stable schema - Updated `packages/shared/src/telemetry/README.md`: added "Proposed Events" section to the Telemetry Data Contract per the contributing guide requirement for telemetry changes ## Verification Run all of the following from the repo root: ```sh # Extractor unit tests node --test scripts/extract-proposed-events.test.mjs # Telemetry client + types tests pnpm exec vitest run --config vitest.config.ts \ src/telemetry/client.test.ts src/telemetry/client-types.test.ts \ --reporter=verbose # (run from packages/shared) # Type-check pnpm --filter @paperclipai/shared typecheck # Smoke-run the extractor in local-test mode node scripts/extract-proposed-events.mjs --ref local-test ``` All four commands pass locally. ## Risks - **Silent drop on unregistered events:** The `Object.hasOwn` guard fails closed — any event name not in `PAPERCLIP_EVENTS` is silently dropped. If the generated registry is missing an event that was previously tracked, those calls will be silently lost. Mitigation: the extractor script surfaces proposed events that need registration; the TypeScript type system already enforces `TelemetryEventName ⊆ PAPERCLIP_EVENTS` at compile time. - **Extractor is read-only:** `extract-proposed-events.mjs` reads source and emits JSON; it does not modify any files. No runtime or schema risk. - Overall risk: **low**. The guard is additive and defensive; the extractor and docs are additive only. ## Model Used - Provider: Anthropic - Model ID: `claude-sonnet-4-6` - Context window: 200 K tokens - Capabilities: tool use, extended context, code generation ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6f204605ad |
fix(ui): show source SHA for unreleased builds (#9508)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Operators need to identify the exact build running from the persistent account menu > - Formal releases already have a concise public version, but source builds include a long derived version string > - The derived version identifies a commit but does not expose the source branch or a direct path to inspect the code > - Server Git metadata is auth-sensitive, so the UI must also refresh it when the current session changes > - This pull request shows linked branch and commit metadata for source builds while preserving `v<version>` for formal releases > - The benefit is faster build diagnosis with correct metadata across sign-in and sign-out transitions ## Linked Issues or Issue Description ### Pre-submission checklist - [x] I searched existing open and closed issues and found no duplicate implementing this exact account-menu behavior. - [x] The behavior reproduces on `master`. - [x] The behavior originates in Paperclip's core UI, not an adapter, provider, or local configuration. ### What happened? Source builds displayed the full derived version, such as `2026.626.0+58.git.518fc71ce`, without linking the operator to the corresponding source branch or commit. ### Expected behavior Source builds should show the concise branch and short commit SHA with links to GitHub, while formal releases should continue showing their public version. Auth transitions should refresh the health metadata that supplies those Git details. ### Steps to reproduce 1. Run Paperclip from a commit after a release tag. 2. Open the account menu. 3. Inspect the build label beneath the user identity. 4. Sign in or out and reopen the menu. ### Paperclip version or commit Any source build whose server version uses the `<version>+<count>.git.<sha>[.dirty]` format. ### Deployment mode Local dev (`pnpm dev`) or authenticated deployments. ### Installation method Built from source. ## What Changed - Detect source-derived version strings and render the source branch plus seven-character commit SHA in `SidebarAccountMenu`. - Link source branches and commits to the canonical `paperclipai/paperclip` GitHub repository. - Extend server Git metadata with the full SHA and expose it through health/OpenAPI contracts. - Refresh auth-sensitive health metadata after sign-in and every sign-out entry point. - Preserve the existing `v<version>` label for formal releases and add focused regression coverage. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/pages/Auth.test.tsx src/components/SidebarAccountMenu.test.tsx src/components/SidebarServerInfo.test.tsx` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/health.test.ts src/__tests__/server-info.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm check:token-gates` - `git diff --check public/master...HEAD` ## Risks - Low risk: formal release rendering retains the existing fallback behavior when the source-version pattern does not match. - Source links assume the build came from the canonical public repository; fork-only branches or commits may not resolve there. - Health metadata is invalidated after auth transitions, adding one bounded refetch so the displayed Git details match the new session. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.4 with medium reasoning, repository/tool access, shell execution, and code editing; context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
efcce9cc8e |
fix(adapters): record unpriced CLI usage (#9505)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Budgets and spend telemetry are control-plane safety features, not just reporting > - Local Codex and Claude adapters can execute through either ACP or their native CLI engines > - The ACP lane records usage and reported cost, but CLI JSON output often reports tokens without a price > - The CLI lane was either losing per-run usage semantics or coercing missing cost to zero, making real usage indistinguishable from a genuinely free run > - This pull request preserves CLI usage as per-run totals and records token-bearing runs without a reported price as explicitly unpriced ledger events > - The benefit is accurate usage accounting and a visible pricing gap instead of silently misleading zero-cost telemetry ## Linked Issues or Issue Description Refs #9471 Refs #9230 **Bug description** A `codex_local` run using the CLI engine can emit a final `turn.completed` event with millions of input tokens and tens of thousands of output tokens while the agent's spend ledger remains indistinguishable from a true zero-usage, zero-cost run. Claude CLI output has the same missing-price edge case. **Expected behavior** Token-bearing CLI runs should persist their usage. If the adapter reports a price, the ledger should record it as reported; if the CLI reports usage but no price, the ledger should explicitly mark the event as unpriced rather than silently treating missing price data as a reported `$0` cost. **Reproduction shape** 1. Configure `codex_local` with `engine: cli`. 2. Run a task that produces a `turn.completed` usage payload. 3. Observe token usage in the run stream. 4. Before this change, missing price data is represented as ordinary zero-cost spend and the CLI usage basis is not consistently propagated. ## What Changed - Mark Codex and Claude native CLI usage totals as `per_run` and propagate that basis through success and failure results. - Stop coercing missing Claude CLI cost to `0`. - Add `cost_status` to cost events with `reported` and `unpriced` values, including an idempotent migration and shared validation/types. - Persist token-bearing runs without a reported price as `unpriced` ledger events while retaining zero cents until an authoritative price exists. - Add parser, execute-path, heartbeat-accounting, and cost-service regression coverage for both local CLI adapters. - Document the cost-status invariant and CLI accounting behavior. ## Verification - `pnpm exec vitest run packages/adapters/codex-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/parse.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/heartbeat-cost-accounting.test.ts server/src/__tests__/costs-service.test.ts` — 6 files / 102 tests passed. - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` — includes migration numbering and safety checks. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/server typecheck` ## Risks - Existing cost rows default to `reported`, preserving current interpretation; only new token-bearing events with absent cost are marked `unpriced`. - This change does not invent model pricing. Budget hard stops still cannot charge an unknown amount, but operators and evals can now distinguish missing pricing from a genuinely reported zero cost. - Consumers that enumerate cost-event fields should tolerate the additive `costStatus` field. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model `gpt-5.3-codex`, with repository tool use and code execution; default reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
