mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 15:35:12 +01:00
7eea9d8214fec00dfde32bda7fcf9b20a428e330
1436
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin Foley
|
429792f1f3 |
fix(interactions): stop wedging confirmation accept on a terminal workspace_finalize (#10099)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - When an agent finishes work in an execution workspace, the board can confirm the result through an issue-thread interaction (e.g. the "Merged" / mark-done confirmation button on a `request_confirmation`). > - That accept action is gated: it must not race a worktree sync-back (`workspace_finalize`) that is still copying the agent's commits out of the sandbox, or the board could act on a base that hasn't received them yet. > - The gate (`runWorkspaceIsFinalized`) treated the sync-back as "settled" only when the latest `workspace_finalize` op was `succeeded` — so a run whose finalize reached a terminal `failed` state, or died leaving a stale `running` op, was treated as "still syncing" forever. > - Users hit a permanent, misleading `... has not finished syncing its workspace` error and could never click "Merged", even though nothing was syncing and the run had long since ended. > - This PR fixes the settle semantics so the gate blocks only while a sync-back is genuinely pending or in flight, and treats any terminal (or stale-orphaned) finalize as done. > - The benefit is that a failed or abandoned sync-back no longer wedges the human confirmation, while a genuinely in-flight sync-back on a live run still blocks correctly. ## Linked Issues or Issue Description No public GitHub issue exists for this. Describing the bug in-PR (bug report): **What happened** Clicking the "Merged" / mark-done confirmation at the bottom of an issue thread returns an error that the workspace "has not finished syncing its workspace" — but nothing is actually syncing, and the run that created the interaction has already ended. The confirmation is permanently stuck; the only workaround is to merge and mark the task done manually. **Expected behavior** Once the source run's worktree sync-back has finished — whether it succeeded, failed, or was skipped — the confirmation should be acceptable. The gate should block only while a sync-back is genuinely still running on a live run. **Steps to reproduce** Have an agent run reach `workspace_finalize` and end without a `succeeded` finalize (e.g. the sync-back fails, or the run process dies mid-finalize leaving a `running` op). Then attempt to accept the `request_confirmation` interaction it created → 409 "... has not finished syncing its workspace" with no way to proceed. **Paperclip version or commit** Reproduced on the current `master` line (server service); root cause is in `runWorkspaceIsFinalized` in `server/src/services/issues.ts`. **Deployment mode** Local / self-hosted instance (server service). **Root cause** `runWorkspaceIsFinalized` returned `true` only when the latest `workspace_finalize` operation was `succeeded`. A terminal `failed` finalize (the sync-back ran and failed; it will not retry within that run) and a `running` finalize left behind by a dead run both left the gate closed forever. ## What Changed - `runWorkspaceIsFinalized` (server/src/services/issues.ts) now treats a sync-back as **settled** when the latest `workspace_finalize` op reached any terminal status (`succeeded`, `failed`, or `skipped`), instead of only `succeeded`. - A `workspace_finalize` still marked `running` blocks only while its owning run is alive; a `running` record left behind by a terminal/missing run is treated as stale (settled), so a dead run can no longer wedge the gate. - Preserved existing behavior for the other cases: no operations recorded at all → settled; earlier phases recorded but no `workspace_finalize` yet → still blocks (the sync-back hasn't been attempted). - Extracted the run-liveness check into a shared exported helper `heartbeatRunIsTerminalOrMissing` and reused it from the existing `isTerminalOrMissingHeartbeatRun` closure (no behavior change there). - Added a short comment at the confirmation-accept gate (server/src/services/issue-thread-interactions.ts) documenting the relaxed settle semantics. - The dependency-readiness / blocker barrier (`listPendingFinalizeBlockerIssueIds`) is deliberately left unchanged: an automated dependent must not proceed onto a base that never received a blocker's synced-back commits, so a failed finalize keeps that gate closed. Only the human-driven confirmation accept is relaxed. - Added regression tests for: failed finalize, stale `running` finalize on a dead run, and a genuinely `running` finalize on a live run (must still block). ## Verification - `cd server && node_modules/.bin/vitest run src/__tests__/issue-thread-interactions-service.test.ts -t "accept"` → 17 passed (includes the 3 new regression tests), 21 unrelated tests skipped by the name filter. - Manual reasoning walkthrough of `runWorkspaceIsFinalized` for each op-history shape (no ops / earlier-phase-only / terminal finalize / running-on-dead-run / running-on-live-run) confirms the intended block-vs-settle outcome. ## Risks - Low risk and narrowly scoped to the human confirmation-accept gate. The only behavioral change is that a terminal (`failed`/`skipped`) or stale-orphaned `running` finalize now settles the gate instead of blocking forever. - A genuinely in-flight sync-back on a live run still blocks (covered by a regression test), so the accept cannot race commits that are actively being synced back. - The blocker/dependency barrier for automated dependents is unchanged, so no dependent will be advanced onto a base missing a failed blocker's commits. ## Model Used - Provider/model: Claude (Anthropic), **Opus 4.8**, model ID `claude-opus-4-8`, 1M context window. - Capabilities used: extended thinking, tool use (repo inspection, local test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
a7186dce4b |
fix(plugin-sdk): thread companyId through configChanged + fail-closed cross-tenant guard (LOOA-687) (#10096)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - First-party **plugins** run as isolated workers spawned by the host
`plugin-loader`, reading company-scoped config through a governed
`ctx.config.get(companyId)` channel.
> - The host→worker `configChanged` RPC carries `{ config, companyId }`,
but the SDK dispatch dropped the scope — `onConfigChanged(newConfig)`
was companyId-blind by design — so a **proactive** worker kept a single
worker-global config.
> - #10092 added a startup replay that fans out **every** stored
company's config through `configChanged`. With no deterministic
ordering, a plugin configured for more than one distinct company ends up
running as whichever DB row was delivered last.
> - That is a latent cross-tenant identity/secret confusion bug: one
company's bot token could be applied to another company's traffic.
> - This pull request threads `companyId` through `onConfigChanged` and
adds a fail-closed cross-tenant guard at the SDK layer, so a
single-tenant worker can never silently collapse to a second company's
config.
> - The benefit is that the config-delivery class is fixed at the SDK
boundary — before any genuinely multi-company proactive plugin ships —
without changing today's single-tenant behavior.
## Linked Issues or Issue Description
No public GitHub issue — describing in-PR (hardening / latent security):
**Latent cross-tenant config collapse.** The worker-side `configChanged`
dispatch forwarded only `config` and dropped `companyId`, so a proactive
plugin kept a single worker-global config. #10092's startup replay
delivers every configured company's config sequentially with no `ORDER
BY`, so a plugin with configs for more than one distinct company would
apply a nondeterministic last-write-wins global config (one tenant's
credential applied to another's traffic).
- Builds on and must merge after #10092.
- Not exploitable today: the only proactive consumer (the chat gateway)
has single-tenant config rows, so last-write-wins is a no-op. This is a
hardening pre-condition before any multi-company proactive plugin ships.
## What Changed
- **Thread scope through:** `onConfigChanged(newConfig, context)` with a
new exported `PluginConfigChangeContext { companyId }`. Backward
compatible — the second arg is optional; existing single-arg
implementations are unaffected.
- **Fail-closed cross-tenant guard** (`worker-rpc-host.ts`): a
single-tenant plugin that receives `configChanged` for a second,
distinct company with a *different* config is rejected with the new
`PLUGIN_RPC_ERROR_CODES.CROSS_TENANT_CONFIG` instead of silently
overwriting the applied tenant's config. Idempotent replays of the
*same* config under a different scope row remain allowed.
- **Opt-in `multiCompanyConfig: true`** on the plugin definition for
plugins that genuinely serve multiple companies from one worker (keying
per-company state on `context.companyId`); the guard is bypassed for
those.
- **Deterministic `ORDER BY companyId`** on `registry.listConfigs`, so
the startup replay binds a single-tenant worker to a stable company
across restarts.
- **Loader visibility:** a `CROSS_TENANT_CONFIG` rejection is logged at
`warn` (was best-effort `debug`) so the misconfiguration is surfaced.
- **Regression test**
(`packages/plugins/sdk/tests/worker-rpc-host.test.ts`): two distinct
companies delivered via the startup-replay path fail closed and stay
bound to the first company; an idempotent same-config replay under a
different scope row is allowed; a `multiCompanyConfig` plugin receives
per-company config with the correct `context.companyId`.
## Verification
- SDK `tsc --noEmit`: clean.
- SDK vitest `worker-rpc-host.test.ts`: 7/7 pass (incl. 3 new). The
two-distinct-company case **fails against pre-fix code** and passes
after the fix.
- #10092 embedded-postgres `plugin-config-startup-delivery.test.ts`: 3/3
pass (unaffected by the new `ORDER BY`).
- Full server `tsc --noEmit` against this SDK: clean.
## Risks
- **Low functional risk.** The second `onConfigChanged` arg is optional
and existing implementations are unchanged. Today's single-tenant
gateway keeps working — idempotent same-config replays are explicitly
allowed, so the go-live is preserved.
- **Behavioral shift on misconfig:** a genuinely multi-company plugin
that has NOT opted into `multiCompanyConfig` now fails closed
(`CROSS_TENANT_CONFIG`) rather than silently collapsing to one tenant.
This is the intended safer default; opt in with `multiCompanyConfig:
true` to serve multiple companies from one worker.
- **Not in scope (residual).** Per-company workers/connections for a
genuinely multi-company gateway increase resource use and are tracked
separately (ties into the #10092 fan-out/timeout follow-up). This PR
fixes the class and fails closed; it does not build multi-tenant
connection management.
## Model Used
Claude — Anthropic `claude-opus-4-8` (Opus 4.8), extended thinking, with
tool use / code execution via Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes`
/ `Refs` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change and contains no internal
Paperclip ticket id — branch predates this rule; not renaming an open PR
mid-review
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes — no
doc surface; internal SDK/host behavior only
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: anicca <annica@Michaels-Mac-Studio.local>
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
Michael NguyenandPaperclip
|
1ebf5254b6 |
fix(plugin-loader): deliver stored config to freshly-started plugin workers (#10092)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - One capability is first-party **plugins** that run as isolated workers spawned by the host `plugin-loader`, reading company-scoped config through a governed `ctx.config.get(companyId)` channel. > - A **proactive** plugin (e.g. a chat gateway that opens a Slack Socket Mode connection at startup) does its company work from `setup()`, where there is **no company-scoped invocation** — so `ctx.config.get()` is rejected with `company context is required`. > - The worker swallows that error and falls back to its default (feature-off) config, so the plugin comes up **inert** even though correct config exists in the database. > - This is a regression from #9557 ("governed access contracts"), which changed `plugin-loader.ts` `activatePlugin` from loading stored config into the worker bootstrap to `const config = {}`. > - This pull request replays each configured company's stored config to the freshly-started worker over the **same `configChanged` host→worker path an operator config-save already uses**. > - The benefit is that proactive plugins receive their config on worker start (both server boot and operator enable) without weakening the governed-access surface. ## Linked Issues or Issue Description No public GitHub issue — describing in-PR (bug): **Bug.** After a proactive plugin's worker spawns, it never receives its stored config. Governed access (`packages/plugins/sdk/src/host-client-factory.ts`) only resolves `config.get` inside a company-scoped invocation (event/action/tool, or explicit `params.companyId`). Proactive plugins operate from `setup()` where no such scope exists, so `config.get()` fails with `company context is required`, the worker falls back to defaults, and the feature stays disabled despite valid DB config. - Regression introduced by #9557. - Related follow-up (latent multi-company hardening): #10096. ## What Changed - `plugin-registry.ts`: add read-only `listConfigs(pluginId)` returning all stored company config rows for a plugin (scoped `where eq(pluginConfig.pluginId, pluginId)`). - `plugin-loader.ts`: after the worker starts in `activatePlugin`, replay each company's stored config through the existing `configChanged` host→worker RPC — one `{ config, companyId }` per row, the same payload shape as the operator config-save path in `routes/plugins.ts`. Best-effort and idempotent; covers both server-boot `loadAll` and operator enable. - test: DB-backed `plugin-config-startup-delivery.test.ts` covering `registry.listConfigs` completeness and cross-plugin isolation. ## Verification - `tsc --noEmit` on `@paperclipai/server` — clean. - New `plugin-config-startup-delivery.test.ts` (embedded-postgres, 3 cases) — pass. - Full PR CI green: typecheck, all server/e2e/serialized test shards, build, canary dry-run, verify, and the security scanners (Snyk, Socket, Superagent, Greptile). ## Risks - **Low functional risk.** Adds an outbound host→worker push that mirrors the already-shipped operator-save path. A worker without an `onConfigChanged` handler (or momentarily unavailable) simply keeps the runtime `ctx.config.get(companyId)` model. - **Startup fan-out.** One `configChanged` per configured company at activation (sequential, default RPC timeout). `plugin_config` rows are writable only by instance-admins, so fan-out size is operator-controlled — not a remote surface. - **No secret-handling change.** `configJson` is delivered as-is, exactly as `config.get`/operator-save already deliver it. No new secret sink; catch-blocks log only ids + `err.message` at debug, never `configJson`. - **Latent multi-company behavior (pre-existing, not introduced here).** The worker-side `configChanged` dispatch forwards only `config` (drops `companyId`), and `listConfigs` has no `ORDER BY`, so a plugin configured for **more than one** company would apply a nondeterministic last-write-wins global config. This is existing SDK behavior — operator-save already pushes into the same handler — and is **not reachable by the single-company consumer this fix targets**. Greptile flagged this shape (4/5). It is tracked and fixed as a separate, non-blocking hardening PR (#10096): thread `companyId` through `onConfigChanged`, deterministic ordering, bounded fan-out. ## Model Used Claude — Anthropic `claude-opus-4-8` (Opus 4.8), extended thinking, with tool use / code execution via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change and contains no internal Paperclip ticket id — branch predates this rule; not renaming an open PR mid-review - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes — no doc surface; internal SDK/host behavior only - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — 4/5; two latent multi-company items triaged as non-blocking and fixed in follow-up #10096 (see Risks) - [x] I will address all Greptile and reviewer comments before requesting merge — addressed: triaged as non-blocking follow-up in #10096 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Michael NguyenandPaperclip
|
5a00040c4c |
feat(plugin-sdk): interactions/approvals respond + attachment read capabilities for the chat gateway (#10066)
Adds the remaining 5 plugin capabilities + 7 worker→host RPC methods (interactions read/respond, approvals read/respond, attachment read) needed by the Slack chat gateway plugin (v0.5.0) to pass manifest capability validation and load. - Security review: PASS (LOOA-642) after the viewer-role privilege-escalation blocker (LOOA-648) was fixed on this branch (requireActiveHumanMember now rejects viewer on impersonation write-paths, matching assertCompanyAccess). - CI: Build, Typecheck, all server suites (3/3 + serialized 4/4), workspaces, e2e shard 2/2, and all security scanners (Snyk/Socket/Superagent/Greptile/security-review) green. - One e2e flake (signoff-policy 'non-participant cannot advance stage') is unrelated: it exercises execution-policy stage advancement (routes/issues.ts, untouched by this PR) and failed on a heartbeat_run_events FK race + 409 checkout conflict. Unblocks LOOA-629 (Slack gateway go-live) and the interview-ask feature. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
0ef3b320c7 |
Harden POST /plugins/install: canonicalize localPath for all instances; bundled-only floor for managed instances (#10067)
**Builds on** #10058 — managed detection keys off the *presence* of the `PAPERCLIP_MANAGED_CONFIG` env var that PR introduces, deliberately never its parsed body. **Summary.** Two layered hardenings of the plugin install route. (1) For **all** instances: `localPath` installs previously skipped the package-name validation entirely; the path is now null-byte-checked, resolved absolute, `realpath`'d (collapsing `..` traversal and symlinks), and required to be an existing directory before the loader ever sees it. (2) For instances running under a managed hosting control plane (detected by the *presence* of `PAPERCLIP_MANAGED_CONFIG` — deliberately never its body, so a corrupted document cannot widen the surface): registry/npm installs return 403, and `localPath` installs must canonicalize to inside the bundled plugin catalog root (`packages/plugins`) — a positive allowlist enforced in code at the route, independent of any flag value. Self-hosted behavior is otherwise unchanged. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The plugin system lets instance admins install plugins from a registry or from a local filesystem path, and plugin installation is code execution on the host > - The `localPath` branch of `POST /plugins/install` skips the validation applied to registry installs; the raw path reaches the plugin loader without canonicalization > - Separately, instances operated by a managed hosting control plane must constrain installs to the bundled plugin catalog, because there the host belongs to the operator, not the tenant > - This pull request canonicalizes and validates `localPath` for all instances, and adds a bundled-only install floor for managed instances > - The benefit is a smaller install-route attack surface everywhere, and a positive code-enforced allowlist where the operator owns the machine ## Linked Issues or Issue Description No public issue exists; `bug_report` template fields for the validation gap this PR fixes: - **What happened:** `POST /plugins/install` with `localPath` set bypasses the package-name validation entirely; the un-canonicalized path (relative segments, symlinks, no existence check) is handed straight to the plugin loader. - **Expected behavior:** path installs are validated like registry installs — null-byte-checked, resolved absolute, `realpath`'d, and required to be an existing directory before the loader sees them. - **Steps to reproduce:** as an instance admin, call `POST /plugins/install` with a `localPath` containing `..` traversal or a symlink pointing outside any plugin directory; observe the loader receives the raw path. Exploitability is bounded (the route already requires instance admin), so this is hardening of an admin-only surface rather than an open exploit. - **Version:** current `master`. The managed-instance bundled-only floor layered on top is new behavior (motivation: on managed hosting, arbitrary plugin install is arbitrary code execution on operator infrastructure), aligned with the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/plugin-install-guard.ts` — three pure primitives: managed detection (presence-based), path canonicalization (null-byte check → absolute resolve → `realpath` → must be an existing directory), and segment-based containment in the bundled plugin catalog root. - Route enforcement in `server/src/routes/plugins.ts`: npm/registry installs return 403 on managed instances; `localPath` installs are canonicalized on every instance and, on managed instances, must land inside the bundled catalog root. - The plugin loader now receives the canonical path instead of the raw request string. ## Verification - 15 guard unit tests (`server/src/__tests__/plugin-install-guard.test.ts`): traversal, symlink escape, null byte, file-vs-directory, string-prefix sibling root. - 13 route security tests (`server/src/__tests__/plugin-install-route-security.test.ts`): 403 matrix on managed instances + self-hosted happy paths. - 36 existing plugin route authz tests green (`server/src/__tests__/plugin-routes-authz.test.ts`). - Server `tsc --noEmit` clean. ```bash cd server pnpm vitest run src/__tests__/plugin-install-guard.test.ts src/__tests__/plugin-install-route-security.test.ts src/__tests__/plugin-routes-authz.test.ts pnpm exec tsc --noEmit ``` ## Risks - Managed instances: npm/registry installs and out-of-catalog `localPath` installs now return 403 — intended new behavior, enforced in code rather than configuration. - All instances: `localPath` installs that previously pointed at nonexistent paths or non-directories now fail with 400 before reaching the loader (previously the loader failed later, less safely). Symlinked deployment layouts are handled by canonicalizing both sides of the containment check. - Self-hosted npm install path is unchanged. Low residual risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
c4cdcc4826 |
Generalize bundled plugin provisioning: ensureBundledKubernetesPlugin → ensureBundledPlugins (#10063)
**Builds on** #10058 — reads `plugins.autoInstall` from the parsed managed-config contract #10058 introduces (the interim `readManagedPluginAutoInstall` shim is retired at rebase). **Summary.** Boot-time bundled-plugin provisioning becomes catalog-driven. A new bundled-plugin catalog lists the sandbox providers shipped in-tree (keys like `kubernetes`, `daytona` → plugin key + path under the catalog root). Managed instances read `plugins.autoInstall` from `PAPERCLIP_MANAGED_CONFIG`; unknown keys or paths escaping the catalog root (symlinks resolved) **throw before listen** — a managed instance refuses to start rather than boot half-provisioned. Installation keeps today's mechanism: an in-process, fail-safe `loader.installPlugin({ localPath })` under a system actor — no HTTP route, no user, no role widening. Self-hosted boot is unchanged (kubernetes bundle only, existing env override honored, install failures still log-and-continue). **Semantics.** A plugin already present in any non-uninstalled state is skipped, so an operator-disabled plugin is never silently re-enabled; managed mode reinstalls soft-uninstalled bundles (the control plane owns provisioning); removal from the autoInstall list never auto-uninstalls. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox-provider plugins ship in-tree, but boot-time provisioning is hard-coded to exactly one of them (Kubernetes) via a bespoke function > - On managed hosting, tenant users have no install privileges, so any bundled plugin that is not provisioned at boot is unusable > - Widening install routes or granting roles to fix that would trade a provisioning gap for a security regression > - This pull request generalizes the existing boot installer into a catalog-driven `ensureBundledPlugins`, fed by `plugins.autoInstall` from `PAPERCLIP_MANAGED_CONFIG` > - The benefit is that managed tenants get working bundled plugins out of the box, through the same in-process, role-free mechanism the codebase already trusts, while self-hosted boot is unchanged ## Linked Issues or Issue Description No public issue exists; `feature_request` template fields: - **Problem or motivation:** on managed instances tenant users cannot install plugins (by design they never hold instance admin), so even plugins shipped with the product are unusable; boot provisioning currently knows only the Kubernetes bundle. - **Proposed solution:** a bundled-plugin catalog plus `ensureBundledPlugins(keys)` driven by the managed config; same in-process `loader.installPlugin({ localPath })` under a system actor; unknown keys or catalog-escaping paths fail startup; already-present plugins are skipped so operator-disabled plugins are never silently re-enabled. - **Alternatives considered:** granting tenant users install privileges (widens secrets/adapters/settings access to solve a one-button problem); a separate non-admin install route for bundled plugins (new authz surface; provisioning removes the need for any install action at all). - **Roadmap alignment:** supports the in-progress "Cloud deployments" milestone and builds on the shipped sandbox-provider milestone in `ROADMAP.md`. Refs #10058. ## What Changed - New `server/src/services/bundled-plugins.ts`: the bundled-plugin catalog, the fail-to-start resolver (`resolveBundledPluginInstalls`, positive allowlist + catalog-root containment with symlinks resolved), and the fail-safe installer (`ensureBundledPlugins`). - `server/src/app.ts`: replaces the hard-coded `ensureBundledKubernetesPlugin` boot hook with resolver + installer wiring, with test hooks (`managedPluginAutoInstall`, `bundledPluginCatalogRoot` options). - `server/src/index.ts`: passes `plugins.autoInstall` from the single fail-closed `PAPERCLIP_MANAGED_CONFIG` startup parse (#10058) into `createApp`; absent env means self-hosted and changes nothing. ## Verification - 24 new tests in `server/src/__tests__/bundled-plugins.test.ts` (catalog resolution, containment incl. symlink and `..` escapes, skip/reinstall matrix, self-hosted invariants, installer error paths) — all green. - 85 adjacent startup/plugin-route/auto-build/managed-config tests green (`managed-config`, `instance-settings-managed-overlay`, `plugin-install-autobuild`, `plugin-routes-authz`, `server-startup-feedback-export`). - Server `tsc --noEmit` clean. ```bash cd server npx vitest run src/__tests__/bundled-plugins.test.ts npx vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts src/__tests__/plugin-install-autobuild.test.ts src/__tests__/plugin-routes-authz.test.ts src/__tests__/server-startup-feedback-export.test.ts npx tsc --noEmit ``` ## Risks - Managed instances with a malformed or unknown `plugins.autoInstall` entry now **refuse to start** (fail closed, by design) instead of booting half-provisioned; harness misconfiguration surfaces as a precise startup error. - Self-hosted behavior is unchanged (kubernetes bundle only, `PAPERCLIP_KUBERNETES_PLUGIN_PATH` honored without containment, install failures log-and-continue), so the default deployment path carries low risk. - No uninstall path exists in this module; removal from the autoInstall list can leave a previously provisioned plugin installed (intentional v1 semantics, documented in code). ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
216d3d2680 |
Managed-instance config: fail-closed PAPERCLIP_MANAGED_CONFIG parsing and read-time settings overlay (#10058)
**Builds on.** #10055 — the `catalogVersion` this config document pins is the feature-catalog artifact #10055 emits. **Summary.** Instances operated by a managed hosting control plane can now receive instance configuration through a single environment variable, `PAPERCLIP_MANAGED_CONFIG` (versioned JSON: `mode`, `catalogVersion`, `features`, `plugins.autoInstall`). When the variable is absent the instance is self-hosted and nothing changes. When present, parsing is strict and **fail-closed**: blank value, malformed JSON, unknown feature key, a feature key this build's feature catalog does not mark tier `managed`, missing required section, or unsupported version refuses startup with a precise error — a typo that silently does nothing is how a security control quietly fails. Managed feature values are overlaid **at read time** inside the instance settings service (never persisted), so a DB restore or manual row edit cannot resurrect a disabled capability; responses expose per-key `managedKeys` metadata (`managed: true`, `managedBy`) so clients can render locked state. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs both self-hosted and under managed hosting, where an operator's control plane owns instance configuration > - Today instance feature settings live only in the tenant database; a hosting control plane has no way to enforce a configuration that tenant-side writes or restores cannot undo > - Managed configuration will carry security posture, so delivery must be atomic and parsing must fail closed — a typo that silently does nothing is how a security control quietly fails > - This pull request adds strict parsing of one `PAPERCLIP_MANAGED_CONFIG` env var and overlays its feature values at read time inside the settings service, never persisting them > - The benefit is a minimal, auditable managed-hosting contract: absent var ⇒ self-hosted instances are byte-for-byte unchanged; present ⇒ deterministic, locked configuration surfaced to clients via per-key managed metadata ## Linked Issues or Issue Description Refs #966 — this PR delivers that issue's "managed config injection" hook, via a strict env-var contract rather than the config-file path it sketches; the issue's other hooks (identity header, health, usage webhook, lifecycle, external secrets, IAM auth) are out of scope, so the PR refs rather than closes it. *Mechanism differs from #966's proposal, so the `feature_request` fields are also filled in:* - **Problem or motivation:** managed hosting deployments need to centrally enable/disable instance features; DB-stored settings can be edited, restored, or migrated back to permissive values, and nothing marks a value as operator-enforced. - **Proposed solution:** one versioned JSON env var; fail-closed parse at startup; read-time overlay in the settings service (precedence: managed value over stored value over schema default); `managedKeys` metadata in settings responses so clients can render locked state. - **Alternatives considered:** per-feature env vars (non-atomic across a half-updated env set, unbounded env surface); seeding the DB at boot (persisted values can be edited or restored over, and cannot express "forced"); lenient warn-and-drop parsing (fails open — unacceptable for a security-bearing control). - **Roadmap alignment:** supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/managed-config.ts` (pure parser over the env record) - Startup parse ordered before the first `instanceSettingsService` construction in `server/src/index.ts` - Read-time merge + `managedKeys` in the settings service - Shared validator updates ## Verification - 29 parser/overlay tests (fail-closed matrix incl. blank/whitespace env, missing sections, catalog-tier mismatch, empty-section happy path): `pnpm vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts` (from `server/`) - 40 existing settings route/service tests green: `pnpm vitest run src/__tests__/instance-settings-routes.test.ts src/__tests__/instance-settings-service.test.ts` (from `server/`) - 15 shared validator tests: `pnpm vitest run src/validators/instance.test.ts` (from `packages/shared/`) - Server `tsc --noEmit` clean: `pnpm typecheck` (from `server/`) ## Risks - Self-hosted instances (no `PAPERCLIP_MANAGED_CONFIG` set) are byte-for-byte unchanged — the parser only runs when the variable is present. - For managed instances, a malformed document now refuses startup by design (fail-closed). This is an intentional behavioral guarantee, not a regression: the control plane owns the variable and a precise startup error is the contract. - Overlay values are never persisted, so no migration or data-shape risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9edde68373 |
feat(kubernetes): native file-sync lifecycle hooks over pod exec (#10053)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - AI agents run in sandboxed execution environments (Kubernetes pods, Daytona workspaces, etc.) and need to sync files between the host and those environments — for workspace setup, asset delivery, and output retrieval > - The existing sync path for Kubernetes uses a base64-over-exec chunk loop: each ~4 MB chunk requires its own `execInPod` round-trip, so large syncs balloon into many exec calls with corresponding overhead > - `execInPod` supports piped stdin/stdout, meaning the full transfer can be done as a single exec that streams a raw `tar` archive over the data channel — one round-trip regardless of file size, with nothing base64-encoded and nothing buffered whole in memory on either side > - PR-1 (#10013, merged) added the `onEnvironmentSyncIn`/`onEnvironmentSyncOut` opt-in hook API to the sandbox provider interface and documented the protocol; PR-2 (#10028, merged) implemented these hooks for the Daytona provider > - This pull request implements the same two lifecycle hooks in the Kubernetes sandbox provider, so workspace/asset file sync streams through one `execInPod` per operation instead of the chunk loop > - The benefit is significantly fewer exec round-trips for large syncs and flat memory use on both host and pod, with security properties preserved: atomic replace, secret-mode enforcement, path confinement, TOCTOU-safe snapshot, and member-confinement on host-assembled archives from sandbox-authored tar output ## Linked Issues or Issue Description This is the third and final PR in a sequential series: - Refs #10013 — PR-1: opt-in sync hook API + provider docs (merged) - Refs #10028 — PR-2: native file-sync lifecycle hooks for Daytona provider (merged) **Feature:** Native single-exec file-sync lifecycle hooks for the Kubernetes sandbox provider. *Motivation:* The existing Kubernetes sync path encodes files as base64 and loops over `execInPod` one chunk at a time (~4 MB per exec). For large workspaces or asset sets this is slow and resource-intensive. The Kubernetes `execInPod` API supports piped stdin/stdout, enabling a raw-`tar` streaming transfer that needs only one exec regardless of file count or size and never buffers the whole payload in memory. *Proposed solution:* Implement `onEnvironmentSyncIn` and `onEnvironmentSyncOut` in the Kubernetes provider using a streaming `execInPod` with a tar pipeline — for syncIn the host builds the archive on disk and streams its raw bytes into the pod's stdin (`head -c <exact-size> | tar -x`, no base64); for syncOut in-pod `tar` writes to the exec's stdout and the host streams those bytes straight to a file. Path confinement, atomic replace, secret-mode enforcement, TOCTOU protection, and a streamed-bytes fail-closed guard are all enforced. ## What Changed - **New `src/file-sync.ts`** in `packages/plugins/sandbox-providers/kubernetes/` — `performSyncIn` and `performSyncOut` over an injected pod-exec closure, keeping transfer logic hermetically unit-testable - **New `execInPodStreaming` in `src/pod-exec.ts`** — a streaming exec primitive that binds a caller-supplied stdin readable and a stdout writable to the exec WebSocket data channel, added alongside the existing `execInPod` (which is unchanged). This lets a transfer stream raw bytes to/from disk instead of buffering the payload as a single string - **Updated `src/plugin.ts`** — registers `onEnvironmentSyncIn`/`onEnvironmentSyncOut`; resolves the `sandbox-cr` pod exactly like `onEnvironmentExecute` and delegates; `job` backend rejects file-sync calls explicitly (out of scope) - **syncIn path:** host builds the tarball to a temp file → streams its raw bytes over exec stdin, bounded in-pod by `head -c <exact-archive-size> | tar -x` (no base64 anywhere) → extract into a `/proc/self/fd`-pinned reserved `0700` staging dir → `chmod`-before-`mv -f` atomic replace per file (directory mappings use `followSymlinks`→`-h`) - **syncOut path:** in-pod validate + realpath-snapshot each source (closes the validation→copy TOCTOU window) → single-exec `tar -c` streamed over exec stdout → host streams that stdout straight to a temp file through a byte-counting transform → member-confined extraction of the sandbox-authored archive - **Security properties:** secret files land at requested mode with no widened window; every interpolated path is shell-quoted and confined lexically plus via in-pod `realpath`; the outbound stream is bounded by a **streamed-bytes disk guard** (`MAX_SYNC_OUTPUT_BYTES`, 8 GiB default, per-call overridable) that fails the transfer closed — writing no target file — if an untrusted pod emits more bytes than allowed. Neither host nor pod buffers the whole payload, so there is no in-memory size cap on the transfer - **No changes** to `execInPod`, `wrapCommandWithEnv`, or `FastUploadInterceptor` (the `environmentExecute` path is untouched) - **No dependency or lockfile changes** - **New tests** in `test/unit/file-sync.test.ts` (atomic-replace, `0600` secret mode, symlink preserve/deref, dir-mapping, exclude, path-confinement rejection, streamed-output guard fail-closed) and `test/unit/pod-exec.test.ts` (streaming stdin/stdout, caller-sink error fail-closed), plus extended `test/unit/plugin.test.ts` ## Follow-up: Legacy Job-Lease Base64 Fallback Fix Addresses the Greptile 4/5 blocking finding ("Handle existing job leases", `server/src/services/environment-runtime.ts`). Job leases provisioned before the `nativeFileSyncUnsupported` metadata flag existed carry `backend: "job"` but no flag, so `supportsSync()` treated them as native-capable and routed their sync to the pod-exec hook — which the job backend rejects (it has no exec channel) instead of using the byte-identical base64 fallback. The fix adds a belt-and-suspenders gate on the persisted `backend === "job"` field alongside the existing `nativeFileSyncUnsupported` flag check, so pre-existing job leases continue syncing via the base64 fallback after deployment. No behaviour change for `sandbox-cr` leases. ## Verification - `pnpm --filter @paperclipai/sandbox-provider-kubernetes test` — 19 files / 182 tests green, including the existing `upload-interceptor` and `pod-exec` suites - `tsc --noEmit` in the kubernetes package — 0 errors - The sync hooks are opt-in; existing `environmentExecute` behaviour is unaffected and tested by the unchanged existing suites ## Risks - **Opt-in only:** `onEnvironmentSyncIn`/`onEnvironmentSyncOut` are registered conditionally; providers that do not register them fall back to the existing chunk loop. No regression risk on the existing path. - **Shell-injection surface:** all path interpolation uses shell-quoting; paths are additionally confined lexically and via in-pod `realpath` before use. - **TOCTOU on syncOut:** the in-pod snapshot validates and records file metadata before the tar call, closing the window between validation and copy. - **Archive member confinement:** host-side reassembly rejects any tar member whose resolved path escapes the target directory, preventing a malicious in-pod tar from writing outside the intended destination. - **Untrusted-output volume:** an over-large outbound stream trips the streamed-bytes disk guard and fails closed (no target written and the temp sink is swept) rather than filling host disk or memory; the guard bounds disk unconditionally and bounds memory insofar as WebSocket write-backpressure holds. ## Model Used Anthropic Claude Sonnet 4.6 (`claude-sonnet-4-6`) — produced by a Claude-based AI agent using agentic tool use and multi-step code generation. 200K context window, extended reasoning, code execution and verification capabilities. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Michael NguyenandPaperclip
|
e55d702916 |
feat(plugin-sdk): human-attributed issue comments for chat gateway plugins (#10050)
Adds the `issue.comments.create_human_attributed` capability and `ctx.issues.createComment` `actorUserId` option, with host-side active-human-member verification. LOOA-627. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
8af70b9fae |
fix(test): drain in-flight heartbeat runs before liveness teardown (#10040)
## Thinking Path > - Paperclip runs AI agent heartbeats to manage work; each heartbeat dispatches `executeRun` fire-and-forget, which is intentional for concurrency > - The server escalation test suite (`heartbeat-issue-liveness-escalation.test.ts`) exercises `reconcileIssueGraphLiveness`, which heals a resolved-dependency wake by enqueuing an on-demand heartbeat run > - `enqueueWakeup` → `startNextQueuedRunForAgent` dispatches the run fire-and-forget (`void executeRun(...)`), so the background run outlives the awaited reconcile call > - The test's `afterEach` polled `heartbeat_runs.status` to wait for idle, but that flips to `completed` while `executeRun`'s finally block is still flushing events — the escaping `heartbeat_run_events` insert could land between the events delete and the runs delete, tripping the FK constraint > - This PR fixes the race deterministically by tracking in-flight `executeRun` promises and exposing `heartbeatService.drainActiveRunExecutions()`, which the suite awaits before clearing tables > - The benefit is a permanently reliable escalation test suite with no sleeps, no retry bumps, and no production behavior change ## Linked Issues or Issue Description **What happened?** The `heartbeat-issue-liveness-escalation.test.ts` suite intermittently failed in CI with: ``` delete on table "heartbeat_runs" violates foreign key constraint "heartbeat_run_events_run_id_heartbeat_runs_id_fk" ``` **Expected behavior** `afterEach` cleanup should complete without FK violations. **Steps to reproduce** The race is timing-dependent but surfaces reliably when the teardown window is artificially widened. `reconcileIssueGraphLiveness()` heals resolved-dependency wakes by dispatching a heartbeat run fire-and-forget (`void executeRun(...)`). The old `afterEach` polled `heartbeat_runs.status` — but that flips to `completed` while `executeRun`'s finally block still has pending `heartbeat_run_events` row writes. The escaping insert can land between the events delete and the runs delete. **Paperclip version or commit** Reproducible on current `master` (commit `b57aa9950c707a024156c34b79326a82b2dcca31`) ## What Changed - **`server/src/services/heartbeat.ts`** — tracks all in-flight `executeRun` promises in a module-level `Set`; exposes `heartbeatService(db).drainActiveRunExecutions()`, which loops until the set drains (a completing run can enqueue the next queued run in its finally, so a single `await` is not enough) - **`server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts`** — replaces the poll-on-`heartbeat_runs.status` teardown with `await heartbeatService(db).drainActiveRunExecutions()` before clearing tables; removes the now-unnecessary `waitForHeartbeatRunToComplete` helper ## Verification ```bash # Full file (22 tests) npx vitest run server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts # 12x stress loop (264 test-runs, 0 failures) for i in $(seq 1 12); do npx vitest run server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts || break done # Type check the changed files npx tsc --noEmit ``` - 22/22 tests green locally - 12/12 full-file loop iterations: 264 test-runs / 264 afterEach cycles, 0 failures - Widened-teardown stress variant (failed deterministically before the fix) now passes with the drain ## Risks Low risk. The drain mechanism is additive — it only affects test teardown and could also be wired into graceful shutdown. The fire-and-forget dispatch in production is unchanged. The `Set`-based tracking adds negligible overhead per run dispatch (insert on dispatch, delete on completion). ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200K tokens - **Mode:** Tool use, code execution, extended reasoning ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
e1e35881af |
feat(cost-events): propagate issue.billing_code at heartbeat record time (#6821)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs report progress through the heartbeat service, which writes the cost ledger (`cost_events`) as usage accrues > - `cost_events` already has a `billing_code` column, but nothing populates it — the heartbeat writes `issueId`/`projectId` and leaves `billing_code` NULL > - Issues carry a `billing_code`, so the attribution data sits one join away but never reaches the ledger rows > - Reporting therefore has to reconstruct attribution by joining back to `issues` at query time, which reflects the issue's *current* billing code rather than the one in effect when the cost was incurred > - This pull request threads `billingCode` through `resolveLedgerScopeForRun` so the heartbeat stamps it onto each `cost_events` row at record time > - The benefit is that attribution is captured at write time and stays correct if an issue's billing code later changes ## Linked Issues or Issue Description No existing public GitHub issue. Describing the problem in-PR: **Problem.** `cost_events` has a `billing_code` column that is never written. The heartbeat's cost-ledger insert records `issueId` and `projectId` but not the billing code of the issue the run belongs to, so every row lands with `billing_code` NULL. **Impact.** Cost-per-billing-code reporting has to derive attribution by joining `cost_events` back to `issues` at query time. That join returns the issue's billing code *as of the query*, not as of when the cost was incurred, so historical cost reports shift retroactively whenever an issue is re-coded. **Desired behaviour.** The billing code in effect at record time is stored on the `cost_events` row itself. **Related PRs.** #6820 — same change to the same file by the same author, opened separately. These are duplicates; only one should land. ## What Changed - `resolveLedgerScopeForRun` now selects `issues.billingCode` alongside `id` and `projectId`. - The scope object it returns gained a `billingCode` field, populated with `issue?.billingCode ?? null`. - The early-return path for runs with no issue in context returns `billingCode: null`. - The `costs.createEvent` call in `heartbeatService` passes `billingCode: ledgerScope.billingCode` alongside `issueId`/`projectId`. No schema migration: `cost_events.billing_code` already exists. ## Verification **No automated test accompanies this change.** There is currently no test asserting that a `cost_events` row carries the issue's billing code when an issue is in scope, or `null` when there is not. A reviewer should treat the checks below as manual verification only. Manual verification against a running instance: ```sql -- Non-NULL billing_code for recent runs on billed issues SELECT billing_code, COUNT(*) FROM cost_events WHERE created_at > NOW() - INTERVAL '1 hour' GROUP BY billing_code; -- Cost attribution query this change is intended to enable SELECT billing_code, SUM(cost_cents) FROM cost_events GROUP BY billing_code; ``` Expected: rows for runs attached to an issue with a billing code now carry that code; runs with no issue in context remain NULL. ## Risks Low risk in blast radius, with two things worth a reviewer's attention: - **Behavioural shift for consumers.** `cost_events.billing_code` was uniformly NULL and now starts arriving populated. Anything downstream that groups, filters, or dedupes on that column will see new values and new cardinality. Existing rows are not backfilled, so the column is mixed NULL/non-NULL across the historical boundary. - **No test coverage.** The null-fallback behaviour on both paths is asserted only by reading the code, not by a test. - **Migration safety:** not applicable — no schema change; the column already exists. - **Failure mode:** if `billingCode` were absent from the `issues` selection the value would silently be `undefined` rather than erroring, so the field is worth confirming in review. ## Model Used **TODO (author):** this section is required and cannot be completed on your behalf. Please state the provider and model name, the exact model ID/version, and the reasoning/thinking mode used — or "None — human-authored" if no AI model was involved. Per the template, the "Generated with Claude Code" footer is not a substitute for this section. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [ ] I have specified the model used (with version and capability details) — **pending author input, see above** - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above — #6820 is a duplicate of this PR - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable — **no test added for the new field** - [x] I have updated relevant documentation to reflect my changes — not applicable, no user-facing or documented behaviour changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green — **`e2e` did not complete on `5ca5fde` (Playwright install timed out at 30m and the run was cancelled); all other checks pass** - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — **currently 4/5, sole finding being this description** - [ ] I will address all Greptile and reviewer comments before requesting merge --- <sub>This description was reformatted to `.github/PULL_REQUEST_TEMPLATE.md` by the Paperclip PR triage bot. The code was not modified. Checklist boxes reflect the PR's verifiable state at commit `5ca5fde`; unchecked items are genuinely outstanding, not oversights. The **Model Used** section requires input from the author. The previous description's `LEG-` reference was removed as an internal, instance-local identifier that the template prohibits.</sub> --------- Co-authored-by: Lead Backend Engineer Agent <backend1@legacykeeper.io> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com> |
||
|
|
b247bf7150 |
feat(runtime): opt-in sandbox file-sync lifecycle hooks (API + provider docs) (#10013)
## Thinking Path > - Paperclip is an open-source AI-agent management platform; agents run tasks inside sandboxed environments (Daytona, Kubernetes, E2B, etc.) > - The control-plane ↔ sandbox file-transfer path flows through the `environmentExecute` seam in `protocol.ts` — the only verb available to plugins — which forces a base64-over-exec chunked loop for every file move: workspace files, assets, Codex home sync > - This transport is correct and safe, but it bypasses provider-native bulk/streaming APIs (Daytona `uploadFiles`, K8s `FastUploadInterceptor` / volume mounts), leaving significant throughput on the table for large workspaces > - The right fix is an opt-in seam extension: providers with faster native transfer declare two optional verbs; providers that do not opt in stay on the existing fallback with zero code or behavior change required > - This PR adds the first layer of that extension — two optional verbs (`environmentSyncIn` / `environmentSyncOut`) in the plugin SDK, the runtime plumbing to prefer the native path for the two clean destroy-then-replace cases, and a doc for the contract > - The core correctness invariant is byte-identical fallback: if no provider opts in, execution is exactly what ships today; `assertSyncOperationsConfined` enforces host-side path confinement for providers that do opt in > - No provider advertises the verbs yet → zero production behavior change; future PRs wire up Daytona and K8s providers against this contract ## Linked Issues or Issue Description No public GitHub issue exists for this feature. Description follows the `feature_request` issue template: **Subsystem affected:** packages/plugins — plugin system; packages/adapter-utils — adapter runtime; server/ — EnvironmentRuntimeService **Problem or motivation:** Sandbox file transfers currently always use a base64-over-exec chunked loop regardless of what the underlying provider supports. For workspaces larger than a few MB this becomes the dominant wall-clock cost of every sandbox run, and it bypasses bulk/stream APIs that providers like Daytona already expose natively. **Proposed solution:** Add two optional, opt-in plugin hooks — `onEnvironmentSyncIn` / `onEnvironmentSyncOut` — to the plugin SDK. When a provider defines both hooks and both are advertised via the existing `supportedMethods` negotiation, the runtime prefers the native path for the two clean destroy-then-replace transfer cases; all other cases fall back to the existing byte-identical base64 transport. **Alternatives considered:** An unconditional verb would require every provider to implement or stub the verb. The opt-in / `METHOD_NOT_IMPLEMENTED` pattern (already used by `environmentExecute`) preserves backward compatibility with zero provider changes required. **Roadmap alignment:** Consistent with the ✅ "Cloud / Sandbox agents" and ✅ "Plugin system" milestones; extends the plugin seam rather than adding control-plane-level logic. **Additional context:** Searched open pull requests and issues for duplicate sandbox file-sync / native-transfer work; none found. ## What Changed - **`packages/plugins/sdk`** - `protocol.ts`: two new optional `HostToWorkerMethods` — `environmentSyncIn` / `environmentSyncOut` — plus generic `SyncOperation`, `SyncFileMapping`, and `SyncOutcome` types - `define-plugin.ts`: optional `onEnvironmentSyncIn` / `onEnvironmentSyncOut` fields on `PluginDefinition`; worker advertises each verb only when its hook is defined (else `METHOD_NOT_IMPLEMENTED`, mirroring `environmentExecute`) - `worker-rpc-host.ts`: route new verbs to plugin hooks - `index.ts`: re-export new public types - **`packages/adapter-utils`** - `command-managed-runtime.ts`: expose optional `syncIn` / `syncOut` on `CommandManagedRuntimeRunner` (available only when both verbs are advertised); add `assertSyncOperationsConfined` host-side path-confinement guard - `sandbox-managed-runtime.ts`: `SandboxManagedRuntimeClient` gains optional `syncIn` / `syncOut`; orchestrator prefers native path for default-provision asset inbound and workspace-download-into-fresh-dir outbound; all other paths keep the existing base64 fallback - `sandbox-file-sync.test.ts` (new): 234-line characterization suite — native-opt-in branch, fallback branch, `assertSyncOperationsConfined` escape-path rejection, `followSymlinks` → tar `-h` - `command-managed-runtime.test.ts`: negotiation + native-sync + confinement tests - **`server/src/services/environment-runtime.ts`**: `EnvironmentRuntimeService` delegates to `syncIn` / `syncOut`, gated on advertised support - **`server/src/services/environment-execution-target.ts`**: minor typing fix alongside the new verbs - **`doc/plugins/SANDBOX_FILE_SYNC_HOOKS.md`** (new): documents the full contract — opt-in / no-op guarantee, operation ordering, provider-may-tar, atomicity, `followSymlinks`, secret modes (0600, no window), path confinement, `operationId` opacity, resource bounds, shell-quoting ## Verification ```bash # SDK suite pnpm --filter packages/plugins/sdk test # Adapter-utils suite (includes new sandbox-file-sync characterization tests) pnpm --filter packages/adapter-utils test # Expected: 255 pass / 4 skip # Type-check across affected packages pnpm --filter packages/plugins/sdk typecheck pnpm --filter packages/adapter-utils typecheck # Server changed-file spot check: cd server && npx tsc --noEmit --skipLibCheck 2>&1 | grep -E "environment-(runtime|execution-target)" | head -20 ``` Key behavioral invariant to spot-check: with no provider opting in (the current state), run any sandbox task and confirm file-transfer behavior is byte-for-byte identical to what the pre-PR code produces. The characterization tests assert this at the unit level. ## Risks - **Zero production risk today**: no provider advertises `environmentSyncIn` / `environmentSyncOut`, so the new code paths are unreachable in production; all real traffic stays on the existing base64 fallback - **Path confinement**: `assertSyncOperationsConfined` rejects any `targetPath` that escapes the declared root — this is the primary security boundary for future providers. The test suite covers escape-path rejection - **Atomicity**: the contract delegates atomicity to providers; the doc explicitly calls out that directory-level ops are not guaranteed atomic - **Secret transport**: credential assets (e.g., Codex `auth.json`, directory mappings) continue to use the existing tar path — they do not go through the new verbs in any current provider > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Provider: Anthropic Model: `claude-sonnet-4-6` (Claude Sonnet 4.6) Context window: 200 K tokens Capabilities: extended tool use, multi-file code generation, agentic reasoning via the Paperclip agent framework ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
0b496c9c03 |
feat(secrets): add run-bound agent secret access (#9921)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Agents already receive selected company secrets through `env.*` bindings at run launch, but environment injection is ambient, long-lived, and not suitable for every secret consumer. > - The existing binding and secret-access-event models already provide company-scoped authorization and per-resolution audit seams. > - Agents need an explicit way to discover only the secrets granted to them and fetch a value on demand without exposing the wider company catalog. > - That capability must remain run-bound, preserve low-trust token carve-outs, and make every value read visible in both security and operator audit trails. > - This pull request adds an `access.*` delivery namespace, two run-bound agent routes, dual audit logging, documentation, and an operator grants editor. > - The benefit is least-privilege, revocable, auditable secret access while preserving existing env injection behavior. ## Linked Issues or Issue Description No pre-existing public issue. Related work: - Refs #9797 — existing in-sheet agent access UI that this PR extends to distinguish env and API delivery. - Refs #9918 — complementary searchable-agent picker improvement for the same secrets sheet. - Refs #9530 — related company-wide metadata catalog proposal; this PR intentionally exposes only the authenticated run's granted aliases and values. **Problem / motivation:** Agents can currently consume secrets only through process environment injection. This keeps values resident for the run, does not support on-demand consumers, and cannot provide a discrete operator-visible activity event for each agent-initiated read. **Proposed solution:** Treat `company_secret_bindings` as the source of truth for agent secret grants. Keep `env.KEY` as env delivery and add `access.ALIAS` for API-only delivery; an env binding also implies read access because the value is already present in the agent process. Add run-bound list/fetch endpoints that derive scope from the authenticated heartbeat run and never accept caller-selected overlays. **Alternatives considered:** A company-wide agent-readable catalog was rejected for this value path because it increases reconnaissance and does not prove a per-secret grant. Reusing the ephemeral environment-probe resolver was rejected because it lacks binding enforcement. Approval-gated reads and user-scoped secrets remain deferred beyond v1. **Roadmap alignment:** This extends the completed **Secrets Manager with per-agent access** roadmap capability from launch-time env injection to explicit run-bound API delivery without duplicating a separate planned initiative. ## What Changed - Added `access.*` agent binding validation and a dedicated run-bound resolver that combines `secrets:read` authorization with binding-context enforcement. - Added `GET /api/agents/me/secrets` for minimal granted metadata and `POST /api/agents/me/secrets/:key/value` for on-demand value fetches with `Cache-Control: no-store`. - Preserved the existing denials for low-trust review agents, task-bridge credentials, and skill-test tokens; standard long-lived agent API keys cannot call the run-bound routes. - Added dual audit behavior: value attempts write `secret_access_events` and `activity_log` (`secret.value.read`), while metadata listing writes the lighter `secret.access.listed` activity event. - Kept env compatibility: `env.*` remains injected at launch and also implies API read for the same bound agent; `access.*` never becomes an environment variable. - Added the agent-settings **Secret access** editor plus delivery-mode/alias surfacing on the Secrets page, with focused UI tests and tokenized layout styles. - Updated OpenAPI, shared types, agent-facing skill documentation, and API reference documentation. ### UI Screenshots P3 produced and reviewed three screenshots using mock data; images are intentionally not committed to the repository: - `secret-access-editor.png` — agent settings grant editor. - `secret-access-light.png` — Secrets-page delivery surfacing in light mode. - `secret-access-dark.png` — Secrets-page delivery surfacing in dark mode. The source attachments are retained with the implementation task and linked in the internal handoff; the public page publisher was unavailable in the PR-prep runtime. ## Verification - `pnpm exec vitest run server/src/__tests__/agent-secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts server/src/__tests__/secrets-routes.test.ts ui/src/lib/secret-delivery.test.ts ui/src/components/AgentSecretAccessEditor.test.tsx` — 5 files, 122 tests passed. - Security follow-up: `pnpm exec vitest run server/src/__tests__/agent-secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts` — 2 files, 73 tests passed after active-run and version-consistency fixes. - Final-head CI: all feature, typecheck, build, e2e, security, and review gates pass; `General tests (server (1/3))` remains red after one rerun because unrelated `heartbeat-retry-scheduling.test.ts` cleanup deletes `heartbeat_runs` before referenced `activity_log` rows. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm check:token-gates` — feature-local arbitrary-value violations fixed; command still reports five unchanged `#9627` literals outside this PR. - End-to-end QA passed all eight acceptance criteria: grant/list, fetch, dual audit, env-implies-read, denial matrix, revocation, UI rendering, and env-injection regression. Evidence: https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455492 - Security review returned PASS-with-required-changes; the implementation uses the required dedicated binding-enforcing resolver, run-bound JWT restriction, run-derived overlays, minimal metadata, and a resolver redaction-registration hook. Evidence: https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455382 ## Risks - A compromised agent can exfiltrate any secret explicitly granted to it; explicit company-scoped/run-scoped grants, revocation, and audit reduce but cannot remove that inherent capability risk. - The resolver invokes a redaction-registration hook before returning values, but the current route has no persistent cross-request per-run redaction registry. Paperclip-owned later comments/events therefore cannot yet guarantee automatic scrubbing of a deliberately copied fetched value; QA classified this as non-blocking residual hardening. - Audit-event insertion currently fails open if the security-event insert itself fails; the operator activity event provides partial redundancy, but a future hardening change should define fail-closed behavior for value delivery. - This PR overlaps `ui/src/pages/Secrets.tsx` with #9918 and may require a straightforward rebase after that PR moves. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.3-codex`, with reasoning, repository tool use, terminal execution, Paperclip API access, and GitHub CLI capabilities. Context-window size is not exposed by the runtime. - Anthropic Claude Opus 4.8 with 1M context and tool use assisted with the UI implementation commit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
5ed0b74b34 |
fix(runtime): scope PAPERCLIP_ env-binding strip to reserved keys (#9974)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs get their environment from user/adapter/project/routine env bindings resolved by the server heartbeat, plus `PAPERCLIP_*` runtime vars (identity, wake, workspace, API access) injected by the harness > - The heartbeat stripped **every** `PAPERCLIP_`-prefixed binding before resolution, so legitimately user-named keys (e.g. cloud provider token bindings like `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_*`) were silently dropped and never reached the run env > - At the same time, several adapters honored an explicitly configured `PAPERCLIP_API_KEY` over the harness-minted run token, which is exactly the one key config must never control > - This pull request replaces the blanket prefix strip with a precise three-rule policy: never accept `PAPERCLIP_API_KEY` from config, always let harness-assigned runtime vars win, and let every other `PAPERCLIP_*`-named user binding flow through > - The benefit is that user secrets with a `PAPERCLIP_`-style name work like any other binding, while runtime identity and API credentials stay fully harness-controlled ## Linked Issues or Issue Description **Bug description** (no public issue exists): - **What happened:** Env bindings whose key starts with `PAPERCLIP_` (e.g. a cloud provider token a user deliberately named `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_TOKEN`) were silently stripped by the server before secret resolution, so the spawned agent never received them. No error, no access event — the variable just never appeared. - **Expected behavior:** A user-named `PAPERCLIP_*` binding should reach the run env unless the harness itself uses that key. Only `PAPERCLIP_API_KEY` should be categorically rejected, and harness-assigned runtime vars (`PAPERCLIP_RUN_ID`, `PAPERCLIP_AGENT_ID`, wake/workspace vars, …) should always win over config. - **Steps to reproduce:** Configure an agent/project env binding named `PAPERCLIP_<ANYTHING>` (plain or secret_ref), run a heartbeat, and inspect the spawned process env — the key is absent. - **Deployment mode:** local server, any local adapter. Related prior PRs (different, save-time/API-layer blanket-ban approach; this PR supersedes that direction with a runtime allow-except-reserved policy): Refs #8239, Refs #8439. ## What Changed - `server/src/services/heartbeat.ts`: the pre-resolution strip now removes only `PAPERCLIP_API_KEY` (hard denylist) instead of every `PAPERCLIP_`-prefixed binding; other `PAPERCLIP_*` keys flow into binding resolution. Low-trust inline-sensitive-env checks now also cover those keys. - `packages/adapter-utils/src/server-utils.ts`: new `isForbiddenConfigEnvKey()` helper; the shared `refreshPaperclipWorkspaceEnvForExecution` merge drops `PAPERCLIP_API_KEY` from config and keeps harness-assigned `PAPERCLIP_*` keys authoritative. - `packages/adapter-utils/src/acpx-engine/execute.ts`: removed the explicit-`PAPERCLIP_API_KEY`-from-config allowance; the run token (`authToken`) is now always applied; config `PAPERCLIP_API_KEY` is ignored. - All local adapters (`claude-local`, `codex-local`, `cursor-local`, `gemini-local`, `grok-local`, `opencode-local`, `pi-local`) plus `cursor-cloud`, `hermes`, and the server `process` adapter: removed `hasExplicitApiKey`-style allowances so the harness token always wins, and guarded the remaining unguarded env-merge loops (claude-local inline loop, process adapter) with the same policy. - Tests updated/added: heartbeat binding-strip test now asserts the three-rule policy; adapter-utils merge tests assert the `PAPERCLIP_API_KEY` ban and `PAPERCLIP_*` pass-through; acpx engine tests moved credential fixtures to `authToken` and assert config `PAPERCLIP_API_KEY` is ignored while other `PAPERCLIP_*` config keys forward and still bust the session fingerprint on rotation. ## Verification - `pnpm vitest run packages/adapter-utils/src/server-utils.test.ts packages/adapter-utils/src/acpx-engine/execute.test.ts` — 127 passed - `pnpm vitest run server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-local-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/cursor-local-execute.test.ts server/src/__tests__/gemini-local-execute.test.ts` — 68 passed - Adapter package execute suites and the server tests touching API-key fixtures (`heartbeat-run-log`, `redaction`, `effective-run-config-fingerprints`, `agent-permissions-routes`) — green. Three pre-existing sandbox/SSH fixture failures reproduce identically on clean `master` on this host and are unrelated. - `pnpm --filter <pkg> typecheck` for server, adapter-utils, and all nine touched adapter packages — all pass. ## Risks - Behavioral change: a deployment that relied on configuring a static `PAPERCLIP_API_KEY` in adapter config env loses that override — by design; the harness-minted run token is now the only source. When no run token exists, no API key is injected at all. - `PAPERCLIP_*`-named user bindings now reach binding resolution and run envs; a key that collides with a harness runtime var is still discarded at merge time, so runtime identity/wake/workspace vars cannot be spoofed. - Low risk otherwise: no migrations, no API surface changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic Claude 5 family, Mythos-class tier), extended thinking enabled, agentic tool use (file edits, shell, test runner) via Claude Agent SDK. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
cac3c0fa1a |
feat(connections): add runtime subjects and grants (#9982)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that lets operators connect external apps and govern which subjects may use those credentials > - #9958 established the v3 schema foundation and #9981 adds the AppDefinition catalog layer > - The runtime still needs subject-aware authorization state, scoped key handling, and API/OpenAPI routes so connected apps can actually be granted and used safely > - This pull request adds the runtime grants/authorization behavior on top of the catalog branch, while keeping unrelated dependency and workflow sync commits out of the stack > - The benefit is a reviewable runtime layer that can land after the catalog PR, then unblock the wizard and orchestrator cutover work ## Linked Issues or Issue Description Refs #9958 and #9981. Refs #9981. No public GitHub issue exists for this branch. This is the runtime layer for the Connections v3 stack and is rebased onto `master` after #9981 landed. ## What Changed - Adds the connection user authorization state migration and schema wiring. - Adds shared runtime subject/grant types and validators. - Adds runtime grant and scoped key behavior in the tool-access service. - Adds runtime route coverage and registers the routes in OpenAPI. - Replays only the Connections runtime commits on top of the catalog branch, dropping unrelated sync/dependency history from the prior closed runtime PR. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/validators/tool-access.test.ts server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: runtime grant enforcement is security-sensitive and must fail closed for unknown key scopes. - Migration ordering depends on the schema and catalog layers already merged through #9958 and #9981. - This PR is rebased and retargeted to `master` with runtime-only commits. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d23fbf8ae4 |
feat(connections): add AppDefinition Wave 1 catalog (#9981)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that defines which external apps and MCP-style integrations operators can browse, configure, and run > - The v3 schema core in #9958 added stable connection identities, auth metadata, and grant-aware contracts, but the app catalog still used the older gallery shape > - The product needs a richer, typed AppDefinition catalog so browsing and setup can render provider-specific auth and configuration requirements consistently > - This pull request moves the Wave 1 app catalog onto generated AppDefinition data and carries that shape through shared types, server lookup paths, and app connection UI > - The benefit is that follow-up runtime and wizard work can build against one catalog contract instead of local-only mock/gallery data ## Linked Issues or Issue Description Refs #9958. No public GitHub issue exists for this branch. This is the catalog layer for the Connections v3 stack after the schema-core foundation in #9958. ## What Changed - Adds generated AppDefinition data for the Wave 1 catalog and ingestion reporting. - Replaces the legacy tool app gallery exports with AppDefinition-centered shared contracts, validators, and tests. - Updates server tool-access lookup behavior to use the AppDefinition catalog. - Updates app connection UI surfaces and tests to consume AppDefinition-backed catalog data. - Documents the catalog ingestion workflow in the connector playbook. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts packages/shared/src/app-definitions-url.test.ts ui/src/pages/apps/AppsConnect.test.tsx server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: this changes the catalog contract used by shared, server, and UI app connection surfaces. - Catalog data quality matters because generated definitions now drive browse/setup display. - Follow-up runtime and wizard PRs must rebase on this branch or on master after this lands. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7e00f67138 |
feat(connections): add v3 schema core (#9958)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their governed access to external systems. > - Connected Apps build on the existing Apps and MCP gateway substrate so companies can configure reusable, auditable integrations. > - The current connection record does not yet have a stable public address, explicit ownership/auth method fields, or subject-specific credential grants. > - Without that schema core, later OAuth, per-user authorization, token brokering, triggers, and connector-service phases cannot enforce tenant and subject boundaries consistently. > - This pull request adds the forward-compatible Connections v3 schema core while preserving the existing connection lifecycle and directly migrating the remote MCP transport name. > - The benefit is a company-scoped, least-privilege foundation for one-click integrations without bypassing Paperclip secrets, profiles, rules, or audit controls. ## Linked Issues or Issue Description No matching public issue was found. **Problem** Paperclip's current app connections need a durable identity and authorization substrate before Connected Apps can safely support multiple setup methods, per-user credentials, provider tenants, and managed connector services. The existing schema only models a single connection-level credential set and uses legacy transport terminology. **Proposed solution** Add a stable company-scoped connection UID, explicit ownership/auth/transport fields, a subject-aware `connection_grants` table, and multi-key credential annotations. Backfill existing connections and workspace grants in a reversible migration, then update shared/server/UI contracts to the new `mcp_remote` transport name. **Related work** - Related foundation: #9534 - Roadmap: Connected Apps (one-click integrations) ## What Changed - Added company-scoped connection `uid`, `ownership`, `authKind`, and canonical transport fields across database, shared contracts, validators, services, and UI fixtures. - Added `connection_grants` with workspace/user subject rules, provider tenant metadata, credential secret refs, revocation state, company scoping, and uniqueness constraints. - Added migration `0182_connections_v3_schema_core` to backfill stable UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create default workspace grants, and support rollback coverage. - Added multi-key credential annotations and updated gateway/access services without changing the existing lifecycle behavior. - Updated the connection glossary, connector playbook, and security threat model for the new identity, grant, and relay boundaries. - Added explicit test UIDs to direct database fixtures so the new non-null invariant is exercised across affected server suites. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts server/src/__tests__/heartbeat-runtime-skills.test.ts server/src/__tests__/tool-oauth-legacy-backfill.test.ts server/src/__tests__/tool-access-policy-service.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts packages/db/src/connections-v3-schema-core-migration.test.ts packages/shared/src/validators/tool-access.test.ts --config vitest.config.ts` — 9 files, 218 tests passed. - Latest-head GitHub Actions: build, typecheck, general/serialized suites, backup/worktree restore coverage, both e2e shards, canary, policy, and security scans pass. - Greptile: 5/5 with zero unresolved threads. - `pnpm check:token-gates` remains red only on five pre-existing `#9627` color literals outside this change. ## Risks - **Migration risk:** UID backfill and default-grant creation touch every existing connection. The migration uses company-scoped uniqueness, deterministic legacy UIDs with ID suffixes, and seeded up/rollback coverage. - **Authorization risk:** Grant rows carry credential references. Constraints enforce workspace-vs-user subject shape, company/connection lookup indexes, one default grant per connection, and one user grant per connection/subject. Security review is requested specifically for this design. - **Compatibility risk:** `remote_http` is renamed directly to `mcp_remote`; all repository call sites and fixtures are updated in the same change. - **Future-phase risk:** Subject-bound token issuance, triggers, and connector-service relay verification remain fail-closed requirements documented for later phases; this PR does not expose those capabilities. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI coding agent. The runtime did not expose an exact underlying model ID or context-window size; capabilities used include repository inspection, code editing, shell execution, test execution, Git/GitHub CLI operations, and structured reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Harshit KhemaniandClaude Fable 5
|
3e1dc90bf2 |
fix(execution-policy): final-stage approval terminates the policy instead of rewinding to stage 1 (#7893) (#7936)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Issues can carry an embedded multi-stage `executionPolicy` (e.g. QA → CodeReviewer → CodePusher) driven by `applyIssueExecutionStageTransition` in `server/src/services/issue-execution-policy.ts` > - On approval, the next stage was picked with `nextPendingStage()`, which scans the **whole** stage list from index 0 for the first id not in `completedStageIds` > - Stage ids are regenerated whenever the embedded policy is re-sent or edited mid-flow (a supported operation — the existing "reassigns the active stage when the current participant is removed" test depends on it), so earlier `completedStageIds` can stop matching the current policy; a final-stage approve then "finds" stage 1 pending again and rebuilds a first-stage review (#7893) — an endless re-review loop that can recycle indefinitely against a moving main tip > - This pull request makes approvals advance with a forward-only scan (only stages *after* the one being approved), so approving the last stage always terminates the policy, and adds a guard so an already-completed execution state is terminal for `status=done` > - The benefit is final-stage approvals close the issue as the policy intends, with no behavior change for non-final advancement or reject/changes_requested verdicts ## Linked Issues or Issue Description Fixes #7893 ## What Changed - `server/src/services/issue-execution-policy.ts`: - New `nextPendingStageAfter(policy, completedStage, state)` helper — forward-only scan from the approved stage's index; the approval path uses it instead of `nextPendingStage()`. Approving the final stage therefore always yields `nextStage === null` → completed state → the caller's `done` flows through. - New guard: `requestedStatus === "done"` with an already-`completed` execution state returns without restarting the chain at stage 1 (closes the same loop when a stale completed state lingers). - Reject/`changes_requested` verdicts and intact-state forward advancement are untouched. - `server/src/__tests__/issue-execution-policy.test.ts`: 4 regression tests, including one that reproduces the exact rewind (regenerated stage ids + final-stage approve → previously reassigned QA at `currentStageIndex 0`; now terminal completed) and an explicit final-stage rejection test pinning the unchanged path. ## Verification - `npx vitest run server/src/__tests__/issue-execution-policy.test.ts` → 54 passed (50 pre-existing + 4 new). - `pnpm --filter @paperclipai/server typecheck` → clean. - The rewind was confirmed empirically against unmodified code first (a test asserting the buggy output passed pre-fix and flips post-fix), plus brute-forced realistic operation sequences (checkout dances, status round-trips, interim comments per the agent flow documented around #4889) to verify intact-state flows are unaffected. - Related suites (`issue-execution-policy-routes`, `issue-comment-reopen-routes`, `issues-service`, `issue-thread-interaction-routes`, `issue-agent-mutation-ownership-routes`) also pass locally. ## Risks - Behavior deliberately preserved: non-final approvals (forward scan is identical when state is intact), rejections at any stage, reopen-from-done (state cleared on reopen, fresh chain still starts at stage 1), and explicit `in_review` restarts. - The policy schema has no terminal-state field, so per the issue's Ask the policy simply terminates and the requested `done` status flows through. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code, agentic mode with tool use (subagent implementation + independent adversarial review subagent), extended thinking enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (none found for #7893) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A — server-only change) - [x] I have updated relevant documentation to reflect my changes (N/A — internal stage-advance semantics) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (will confirm once CI runs on this PR) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending first review) - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
1ba79d82a5 |
fix(server): preserve terminal status on issue release (#7524)
Fixes #4206 ## Thinking Path > - Paperclip orchestrates AI agents on issues with checkout/release semantics for execution locks > - `POST /api/issues/:id/release` clears checkout and execution locks when a heartbeat ends without finishing the issue > - `issues.release()` unconditionally set `status: "todo"`, undoing terminal and waiting states (`done`, `cancelled`, `in_review`, `blocked`) set during the session > - Agents reported status drift after release (e.g. `in_review` → `todo`, `done` → `todo`), forcing manual PATCH recovery and risking silent stalls > - This pull request gates the `todo` re-queue to `in_progress` issues only and preserves all other statuses on release > - The benefit is lock cleanup without destroying workflow state agents already recorded ## Linked Issues or Issue Description - Fixes #4206 — `issues.release()` must not downgrade terminal/waiting statuses - Related internal incident: AIT-114 status drift on terminal issue release (AI Trading Council) ## What Changed - `server/src/services/issues.ts` — `releaseStatus` is `todo` only when `existing.status === "in_progress"`; otherwise preserves `existing.status` - `server/src/__tests__/issue-stale-execution-lock-routes.test.ts` — regression tests: release preserves done, cancelled, in_review, blocked keeps `done` and clears lock fields - `server/package.json` — patch bump `0.3.1` → `0.3.2` - `server/CHANGELOG.md` — documents the fix ## Verification ```sh pnpm --filter @paperclipai/server test issue-stale-execution-lock-routes ``` - 7/7 tests pass (parametrized done, cancelled, in_review, blocked) (includes new `preserves terminal status when releasing a done issue` and existing `in_progress` → `todo` on release) - CI: Build, Typecheck, serialized server suites, e2e, Canary Dry Run green on latest head `f31b55f` ## Risks Low risk. Behaviour change is intentional: non-`in_progress` releases no longer force `todo`. Agents that relied on release to re-queue `in_review`/`blocked` work must PATCH status explicitly (documented in agent lifecycle guidance). Rollback: revert this commit and redeploy `@paperclipai/server` 0.3.1. ## Model Used Anthropic Claude Opus 4.6 (extended thinking mode) — 200K context window, tool use enabled. Assisted implementation and PR packaging for AI Trading Council upstream port from local hotfix. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A) - [x] I have updated relevant documentation to reflect my changes (CHANGELOG) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (re-review requested on head `f31b55f`) - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: brandon <brandonburr@gmail.com> |
||
|
DottaandPaperclip
|
59eee4829c |
fix(inbox): stop archived items from resurfacing (#9931)
## Thinking Path > - Paperclip is the control plane operators use to coordinate AI-agent companies and review work needing attention. > - The Inbox is the operator-facing surface that aggregates tasks requiring attention across server state and shared client polling. > - Archiving a task optimistically removed it, but ordinary background activity and stale polling responses could make it reappear seconds later. > - The server therefore needs to distinguish genuine user-attention events from routine agent/system activity. > - The client also needs a bounded local archive guard across every Inbox query path while the server mutation and in-flight polls settle. > - This pull request fixes both resurrection paths and adds race-focused regression coverage. > - The benefit is stable archive behavior without hiding a genuine archive failure after reconciliation or reload. ## Linked Issues or Issue Description ### Pre-submission checklist - [x] Searched existing open and closed issues and pull requests; no duplicate implementation was found. - [x] Reproduced on `master` before this branch. - [x] Confirmed this is a Paperclip core bug, not adapter or provider behavior. ### What happened? Archiving an Inbox task hid it optimistically, then background refresh activity could insert it back into the list seconds later. ### Expected behavior A successfully archived task remains hidden during normal polling. A genuine failed archive may become visible again after reconciliation or reload. ### Steps to reproduce 1. Open Inbox with a visible task. 2. Archive the task. 3. Wait for shared polling or routine agent activity to refresh task data. 4. Observe the archived task reappear without a hard page refresh. ### Paperclip version or commit `master` before this branch. ### Deployment mode Built from source using the local development application. ### Installation method Built from source (`pnpm`). ### Agent adapter(s) involved Not adapter-specific; this is a core Inbox bug. ### Database mode Not database-mode-specific. ### Access context Board (human operator). ### Additional context The failure had independent server and client causes: routine activity could resurface archived rows server-side, while stale shared-poll responses could bypass optimistic client removal. ## What Changed - Restrict server-side Inbox resurfacing to explicit user-attention events rather than any issue activity write. - Add a bounded client-side archive guard with confirmation, failure restoration, and cache reconciliation behavior. - Apply the guard to Inbox rendering, badge counts, optimistic cache updates, and shared-poll result application. - Classify the generic compact Inbox query so stale shared-poll data cannot bypass the guard. - Add server visibility-matrix tests and UI race-condition regression tests. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/hooks/useSharedPolling.test.ts src/lib/inboxArchiveCache.test.ts src/pages/Inbox.test.tsx` — 25 passed. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/issues-service.test.ts` — 107 passed. - Branch rebased cleanly onto current `origin/master` before push. ## Risks - Low-to-moderate behavioral risk: resurfacing is intentionally narrower, so the server tests cover human comments, mentions, interactions, and status transitions that must still regain attention. - The client guard is bounded and cleared on mutation failure, limiting the risk of hiding a task whose archive did not persist. - No schema, migration, public API, workflow, dependency-lock, or visual-token changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude via Claude Code (`claude_local`; prior implementation/review run, exact underlying model ID and context window were not retained in the handoff metadata), with repository tool use and test execution. - OpenAI `gpt-5.5` via Codex CLI for final review repair and PR preparation, with reasoning, repository editing, GitHub tooling, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked/described the result above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip task identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation where needed; no documentation change is required for this bug fix - [x] I have considered and documented risks above - [x] All Paperclip-authored commits include the required co-author trailer --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
a59aa128a3 |
fix(api): sanitize createdByRunId on comment insert to prevent 500s (#9489)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The issue-comments API (`POST /api/issues/:id/comments`) attributes each comment to the run that created it via `created_by_run_id`, a foreign key into `heartbeat_runs` > - In multi-agent local control-plane usage, board/session clients sometimes forward an `X-Paperclip-Run-Id` that is not a real run row — a non-UUID client request id, a synthetic string, or a since-deleted run > - That value was written straight to the FK column, so the insert died with a Postgres FK violation and the endpoint returned HTTP 500, breaking agent coordination > - This PR resolves the run id defensively before insert: reject non-UUID shapes, verify the row exists for the company, and null out anything unresolvable while logging a warning > - The benefit is that a bad run-id header degrades gracefully to an unattributed comment (201) instead of a 500, so comment creation stays up ## Linked Issues or Issue Description No public issue exists; describing inline (bug): **What happened:** `POST /api/issues/:id/comments` returns HTTP 500 when the request carries an `X-Paperclip-Run-Id` that does not correspond to a row in `heartbeat_runs` (non-UUID value, synthetic client id, or deleted run). The value is written to the `created_by_run_id` FK, and Postgres rejects the insert with a foreign-key violation (SQLSTATE 23503). **Expected:** the comment is created (HTTP 201); an unresolvable run id is dropped to `null` rather than failing the request. **Impact:** in multi-agent usage, comment creation — and the agent coordination that depends on it — fails whenever a client forwards a run id that isn't a live run. ## What Changed - Add `resolveCommentCreatedByRunId(dbOrTx, companyId, runId)` — trims and validates UUID shape, then checks existence in `heartbeat_runs` scoped to the company; returns `null` for missing/invalid ids. - `addComment` now resolves the run id through that helper before insert and logs a warning when a supplied run id is dropped. - Add embedded-Postgres regression tests for the three cases (non-UUID header, unknown UUID, valid run id). ## Verification - `pnpm --filter server test issues-service` — the new `issueService.addComment createdByRunId` block passes. - Cases covered: non-UUID header → 201, `createdByRunId: null`; UUID absent from `heartbeat_runs` → 201, `null`; valid run id present for the company → preserved. ## Risks Low. Purely defensive — valid run ids are still preserved, only unresolvable ones are nulled. Adds one indexed, tenant-scoped `SELECT` per comment insert. ## Model Used Claude Opus 4.8 (extended thinking), via the Paperclip PR-triage cockpit, produced the added regression tests and this description. The original implementation is by @digitalflanker-ux; the author's model is unspecified. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (related: #4795 same fix; #8065 sibling FK-guard on the activity-log path) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [ ] My branch name describes the change and contains no internal Paperclip ticket id - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com> |
||
|
DottaandPaperclip
|
051ae4d102 |
feat: restore decision training library and inspector (#9779)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and supervise governed work. > - Decisions capture high-value operator judgment, and the decision-training foundation merged in #9702 freezes that evidence for later evaluation and learning. > - Operators still need the UI from the closed stacked PR #9718 to intentionally capture examples and inspect the resulting dataset. > - GitHub automatically closed #9718 when its stacked base branch was deleted after #9702 merged, leaving the server foundation on `master` without the corresponding UI. > - This pull request restores the final UI and its still-required supporting API fields directly on current `master`, while excluding the obsolete migration and duplicated server-foundation diffs. > - The benefit is a reviewable replacement PR that preserves the completed decision-training workflow without replaying stale stack history. ## Linked Issues or Issue Description - Refs #9718 - Refs #9702 ## What Changed - Restored the top-level `/training` library and record inspector with search, filters, JSONL export, notes editing, and evidence tabs. - Restored the Decisions-row training affordance and capture drawer, including preview, provenance, deletion, cache refresh, and approval consistency behavior. - Restored the shared types and focused server support needed by the UI without reintroducing decision-training migrations or the already-merged server foundation. - Restored focused UI and attention-service tests from the final #9718 state. - Credit to the authors and reviewers of #9718; this recovery transplants their final reviewed delta after the stacked base deletion. ## Verification - `pnpm exec vitest run ui/src/pages/Training.test.tsx ui/src/components/DecisionTrainingDrawer.test.tsx ui/src/components/AttentionQueueRow.test.tsx server/src/__tests__/attention-service.test.ts server/src/__tests__/decision-training.test.ts` — 5 files, 48 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts` — 3 tests passed; confirms exact route/OpenAPI parity for the restored preview endpoint. - `pnpm check:token-gates` — the restored files are clean; the repository-wide command currently reports five pre-existing false positives where comments reference GitHub issue `#9627` as if it were a color literal. ## Risks - Low migration risk: this PR contains no database migrations and is based directly on current `master`. - The main behavioral risk is cache invalidation across Decisions and Training views; focused tests cover capture, update, deletion, row state, and approval refresh behavior. - The token-gate baseline remains red on unrelated `#9627` comment references; this PR does not modify those files. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.3-Codex, reasoning with repository/tool access and code execution. Context window not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a090c09ee5 |
feat: add decision training snapshot foundation (#9702)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - Human approvals, issue interactions, and execution decisions already capture high-value decision moments > - Those moments are currently transient and cannot be reused as stable evaluation or training examples > - Reusable examples need a server-owned, immutable snapshot so later comments or runs cannot leak into the recorded state > - Human notes need to remain editable and auditable without changing the captured state > - This pull request adds the database model, snapshot capture service, API, export format, and attention-feed enrichment for decision training > - The benefit is a durable, inspectable foundation for evaluating whether agents can reproduce good human decisions from only the context available at decision time ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`server/`, `packages/db`, and `packages/shared`). ### Problem or motivation Paperclip has no durable dataset for converting human decisions into evaluation-ready examples. Teams need to capture pending or resolved decisions with the exact issue context, comments, runs, and repository evidence available at a cutoff, while preventing future context from leaking into the example. ### Proposed solution Store immutable, schema-versioned snapshots anchored to durable interaction, approval, or execution-decision records; keep notes separately editable with history; expose human-only CRUD, list, and JSONL export APIs. ### Alternatives considered Client-generated snapshots were rejected because they duplicate cutoff logic and cannot reliably enforce no-leakage boundaries. Automatic outcome backfill was deferred so captured examples remain faithful to what was known at capture time. ### Roadmap alignment Supports the roadmap direction of turning completed work and decision patterns into reusable organizational knowledge. ### Additional context The implementation records explicit commit-resolution confidence (`exact`, `nearest_run`, `workspace`, or `none`) so downstream evaluation can distinguish evidence quality. ## What Changed - Added the `decision_training_examples` schema and idempotent migration with company, issue, and source/author indexes. - Added shared types for decision-training records, notes history, and versioned snapshots. - Added a single server-side snapshot capture path with inclusive comment cutoffs, pre-cutoff run capture, durable decision payloads, and explicit commit-resolution confidence. - Added create, list, detail, notes-only update, delete, and JSONL export routes with human-only write authorization and activity logging that skips no-op note submissions. - Added per-user `trainingExampleId` enrichment to attention items. - Added focused embedded-Postgres tests for cutoff boundaries, post-cutoff leakage, immutable snapshots, human-only writes, duplicate prevention, notes history, attention enrichment, and export shape. - Updated UI test and Storybook attention-item factories for the new required `trainingExampleId` contract. ## Verification - `pnpm exec vitest run server/src/__tests__/decision-training.test.ts` — 10 tests passed. - `pnpm --filter @paperclipai/db typecheck` — passed, including migration numbering and safety checks. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. ## Risks - The migration adds a new table and indexes only; it does not rewrite existing rows or install resolve-time hooks. - Snapshot JSON can grow with long comment threads and run histories; v1 intentionally favors complete, inspectable examples over aggressive truncation. - Commit SHA resolution is evidence-based and records `exact`, `nearest_run`, or `none` so downstream consumers can account for confidence. - The API is additive, but future UI work must continue to treat the snapshot as immutable and use notes-only updates. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.3-codex`, with repository tool use, terminal execution, and code-editing capabilities; context-window size is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1f1f545238 |
feat: add built-in summarizer and summary slots (#9713)
## Thinking Path > - Paperclip is the open source control plane people use to organize, govern, and understand AI-agent work > - Operators need concise, current status views across projects and execution workspaces without manually reading every issue and run > - Paperclip already has auditable issues, documents, built-in agents, routines, and live run events, but no first-class summary-slot workflow connecting those systems > - A built-in Summarizer can generate status prose through ordinary governed tasks while summary slots provide stable, revisioned destinations for that output > - The UI needs to show current summaries, generation progress, failures, revisions, and streaming draft status in the places operators already work > - This pull request adds the end-to-end summary-slot data, API, agent, orchestration, and UI surfaces behind an experimental setting > - The benefit is decision-oriented status context that remains company-scoped, auditable, retryable, and inexpensive by default ## Linked Issues or Issue Description No public GitHub issue exists for this feature. **Problem** Operators currently have to reconstruct project and workspace status by reading many issues, runs, and comments. This makes it hard to identify decisions, review queues, recent work, and the next event worth watching. **Proposed capability** Add an experimental summary system with revisioned summary slots for projects and workspaces, a paused-by-default built-in Summarizer agent, governed generation tasks, live draft status, and reusable UI cards. **Expected behavior** - Summary data remains company-scoped and revisions remain auditable. - Generation runs through normal issue/agent orchestration and deduplicates active requests. - Only the linked built-in Summarizer generation task can author a slot revision. - Operators can generate, retry, inspect revisions, and follow draft progress from project and workspace views. - The feature remains opt-in and background generation remains paused by default. ## What Changed - Added summary-slot schema, idempotent migrations, shared contracts, validators, API paths, and service tests. - Added company-scoped summary-slot routes for reading revisions, requesting generation, and guarded Summarizer writes with activity logging. - Added terminal generation finalization, failure reasons, assignment wakeups, and orchestration integration. - Added the paused-by-default built-in Summarizer bundle, low-cost runtime defaults, status-summarization skill, and stale-summary routine. - Added summary cards, revision selection, retry/configuration states, live draft streaming, transcript chunk handling, and project/workspace integrations. - Updated Claude local parsing for streamed status output and expanded server, adapter, shared, database, catalog, and UI coverage. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run packages/db/src/summary-slots-schema.test.ts packages/shared/src/summary-slot.test.ts server/src/__tests__/summary-slot-routes.test.ts server/src/__tests__/summary-slots.test.ts server/src/__tests__/built-in-agents.test.ts ui/src/components/SummarySlotCard.test.tsx ui/src/components/SummarySlotCard.status.test.tsx ui/src/components/useSummaryDraftStream.test.tsx ui/src/lib/summary-draft-stream.test.ts ui/src/lib/run-log-chunks.test.ts ui/src/context/LiveUpdatesProvider.hook.test.tsx` — 113 tests passed - `pnpm test:run` — server and UI suites passed; one CLI AWS doctor test was affected by inherited `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`, and passed when those host credentials were removed - `pnpm exec vitest run cli/src/__tests__/secrets.test.ts` with inherited AWS credential variables removed — 8 tests passed - `pnpm build` - `pnpm check:token-gates` currently reports nine `#9627` comment references introduced by current `master`; none are in this PR diff ## Risks - Database risk is limited by incrementally ordered, idempotent migrations and migration safety checks. - Summary generation creates normal issues/runs, so misconfiguration can produce failed slots; the UI exposes retryable failure reasons and agent configuration entry points. - Streaming draft parsing depends on the documented `STATUS:` protocol; final persisted revisions remain the source of truth. - The feature is experimental, opt-in, and its built-in routine is paused with no background token spend by default. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5.3 Codex with reasoning, repository tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Earlier branch commits also record Claude model co-authorship where applicable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
b07b2994cc |
feat: stamp responsible users on activity logs (#9731)
## Thinking Path > - Paperclip is the control plane people use to manage AI-agent companies and their work > - The activity log is the generic audit spine for mutations across the control plane > - Activity rows identify agents and runs, but they do not persist the responsible human upstream > - Relying only on run joins loses attribution after run pruning and misses agent API-key actions outside a run > - This pull request resolves responsible-user attribution when each activity row is written and stores it directly > - The benefit is durable, queryable agent audit feeds without rewriting historical provenance ## Linked Issues or Issue Description ### Problem or motivation Agent activity records do not persist the responsible user, so attribution can disappear when runs are pruned and no-run API-key mutations cannot be attributed correctly. ### Proposed solution Resolve attribution for each new activity row from the run, related issue, active agent API key, or company default, in that order, and persist the result directly. ### Alternatives considered Read-time joins alone were rejected because pruned runs lose durable attribution and out-of-run agent-key actions have no run to join. Historical backfill was rejected because it would invent provenance. ### Roadmap alignment This strengthens the durable audit-trail direction described in `ROADMAP.md` without adding a new product surface. ## What Changed - Added nullable `activity_log.responsible_user_id` plus company/agent/time and company/responsible-user/time indexes. - Added an idempotent forward-only migration with no historical backfill. - Added centralized write-time resolution: heartbeat run → issue attribution → active agent API key → company default. - Propagated authenticated API-key IDs through existing request-backed `logActivity()` calls. - Added unit coverage for every fallback and an embedded-Postgres assertion for the no-run API-key stamping path. ## Verification - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/activity-log-responsible-user.test.ts src/__tests__/authz-company-access.test.ts` - Initial focused verification: 25 tests passed; migration safety passed. - Follow-up regression verification: 54 focused attribution/company-skill/environment/issue-tree/tool-gateway tests passed; server typecheck passed. - GitHub: full build, typecheck, server shards, serialized suites, e2e, security, and policy checks passed. ## Risks - Adding two indexes to an existing large table can hold a write lock while the transactional migration runs. The migration safety suppressions document why `CONCURRENTLY` is unavailable under the current Drizzle migration runner. - Historical rows remain nullable by design; this avoids inventing provenance and keeps the migration forward-only. - API-key attribution requires request-backed activity call sites to pass the authenticated key ID; this PR mechanically updates the existing actor-based activity calls and covers the no-run path with integration testing. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.4`; context-window size was not exposed by the runtime. Medium reasoning with repository editing, terminal execution, and test execution capabilities was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
53f09cb818 |
fix: prevent duplicate task creation and recovery loops (#9648)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies > - Agents, routines, productivity review, and recovery services can all create or re-trigger work > - Repeated heartbeats or catch-up cycles can produce duplicate tasks or repeat recovery actions before prior work is visible > - The base duplicate-create and routine catch-up coalescing work has now landed on `master` via related PRs while this PR was being prepared > - This pull request carries the remaining hardening: bounded idempotency retention, recovery cooldown/throttle fixes, productivity-review query batching and ordering fixes, and regression coverage > - The benefit is fewer duplicate tasks, safer retries, and enough provenance to diagnose any future recurrence ## Linked Issues or Issue Description Agents can retry issue creation after ambiguous responses or independently recreate the same child title, while recovery and short-interval routine catch-up paths can repeat before prior work settles. This can produce visible duplicate tasks and makes the originating heartbeat difficult to identify. Related work: Refs #8356 for caller-supplied issue-create idempotency and Refs #9224 for plugin-scoped issue-create idempotency. Prior related PR: #6936. The base issue-create deduplication and routine catch-up coalescing pieces have since landed on `master` via #9650 and #9649; this PR remains as the follow-up hardening stack on top of those changes. ## What Changed - Add 7-day retention for issue-create idempotency claims with indexed, batch-limited cleanup so the claim table does not grow forever. - Preserve recovery cooldown intent after terminal recovery actions are closed, and throttle repeated source-scoped recovery work. - Batch productivity-review source-activity checks to avoid repeated per-source queries while keeping the no-action suppression behavior. - Order productivity-review no-action streak windows by review creation time, matching the window semantics even when completion timestamps are out of order. - Preserve generated issue IDs in route mocks used by backlog/assignment contract tests. - Document PR-gardening task deduplication expectations in the company skill. - Add focused regression tests for idempotency retention, liveness recovery cooldowns, and productivity-review batching/suppression/ordering behavior. ## Verification - `pnpm --filter @paperclipai/server typecheck` — passed on latest head. - `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/productivity-review-service.test.ts server/src/__tests__/issue-create-deduplication-routes.test.ts --reporter verbose` — 2 files, 23 tests passed on latest head. - `pnpm --filter @paperclipai/adapter-utils build && pnpm exec vitest run --project @paperclipai/adapter-utils --reporter dot` — 30 files passed, 476 tests passed, 8 skipped. - `pnpm build` — passed on latest head. - `pnpm -r typecheck` — passed on the rebased head before the final productivity-review ordering commit; the latest touched server code is covered by the server typecheck above. - `pnpm check:token-gates` — passed. - `pnpm test:run` — progressed through server, UI, CLI, shared, skills-catalog, and DB sections, then exposed an adapter-utils compiled-test fingerprint mismatch before rebuilding adapter-utils; the adapter-utils project passed after rebuild, and the GitHub split PR checks passed on the pushed head. ## Risks - Caller-supplied idempotency replay is now bounded to 7 days; reusing an old key after retention can create new work, which matches retry-oriented idempotency semantics. - Recovery and productivity-review timing changes may suppress redundant follow-up work; focused tests cover the intended boundaries. - Advisory locking and idempotency cleanup rely on PostgreSQL-compatible transaction semantics already used by the production data layer. - The migration extends the private claim table indexes without rewriting existing issue rows. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex based on GPT-5, with reasoning, repository editing, shell/tool execution, and test execution. Exact model ID and context-window size are not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
59fb27ff79 |
feat(inbox): let agents safely tidy user inboxes (#9724)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - The inbox is a per-user attention view, so archiving an item must not alter the underlying issue, assignment, or status > - Agents can help responsible users tidy resolved work only when the action is company-scoped, reversible, policy-controlled, and fully attributable > - The database and authorization foundations landed in #9654 and #9658, but the end-to-end archive routes, audit details, agent workflow guidance, and operator UI still need to ship together > - Separate stacked PRs #9659 and #9661 made the complete behavior harder to review and land as one coherent capability > - This pull request consolidates the remaining server, shared-contract, documentation, skill, and UI work on top of current master > - The benefit is a single reviewable change that lets agents safely archive responsible-user inbox items and lets users control or undo that behavior ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting inbox management across shared contracts, server authorization/routes/services, shipped agent skills, and the board UI. ### Problem or motivation Agents may complete work whose issue remains in the responsible user's Mine inbox. Existing board-user archive behavior does not provide the agent-facing policy endpoints, target resolution, heartbeat-run attribution, typed denials, conservative workflow guidance, or UI needed for safe agent-managed cleanup. ### Proposed solution Allow authorized agents to archive or unarchive responsible-user inbox items under the user's open, allowlist, or disabled policy; preserve actor/agent/run attribution in issue detail and activity records; expose policy controls and agent archive attribution in the UI; and document conservative cleanup rules for agents and PR gardening. ### Alternatives considered - Reuse generic issue mutation permissions: rejected because inbox state belongs to a target user and requires user-scoped authorization. - Automatically archive every completed or closed item: rejected because completion signals can still require human review or a decision. - Keep the backend and UI as separate stacked PRs: superseded by this consolidated PR so the complete user-visible behavior can be reviewed and verified together. ### Related work - Builds on merged foundations #9654 and #9658. - Supersedes the remaining stacked changes in #9659 and #9661. - `ROADMAP.md` has no overlapping inbox archive or inbox authorization initiative. ## What Changed - Added shared inbox-agent policy types and validators plus company-scoped self-service policy routes and OpenAPI coverage. - Enabled agent archive/unarchive mutations with responsible-user targeting, policy enforcement, typed failures, attribution, idempotency, and detailed activity auditing. - Returned agent archive attribution in issue detail and documented reversible inbox cleanup semantics in the implementation spec and Paperclip skill. - Added conservative PR-gardening inbox tidy guidance that keeps GitHub access read-only and avoids archiving work that still needs human action. - Added the Profile settings policy control and Issue Properties attribution/unarchive UI with focused component coverage and narrow-pane handling. ## Verification - `pnpm exec vitest run server/src/__tests__/inbox-archive-routes.test.ts server/src/__tests__/inbox-agent-policy-routes.test.ts server/src/__tests__/authorization-service.test.ts server/src/__tests__/openapi-routes.test.ts ui/src/components/InboxAgentPolicyControl.test.tsx ui/src/components/IssueProperties.test.tsx` — 110 passed. - `pnpm --filter @paperclipai/db exec vitest run src/inbox-archive-agent-policies-migration.test.ts` — 1 passed. - `node --test .agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` — 9 passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm check:token-gates` — changed files are clean; the repository-wide command currently reports nine unrelated pre-existing `#9627` literals outside this PR's diff. ## Risks - Agent inbox mutations broaden an existing endpoint path, so authorization and target resolution must remain fail-closed; focused route and authorization tests cover allowed and denied paths. - Archive state affects only the responsible user's inbox presentation and remains reversible; it does not mutate issue status, assignment, or visibility. - The UI policy defaults to the existing open behavior, while allowlist and disabled modes can reduce agent access. - This PR intentionally builds on #9654 and #9658 and contains no new migration number or modification to an already-applied migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.4, medium reasoning, repository tool use, shell execution, code review, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (`feat/inbox-agent-archive-complete`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
52aea90263 |
feat: organize skills with nested folders and My Skills (#9633)
## Thinking Path > - Paperclip is the open source control plane people use to organize and govern AI-agent companies > - Company skills are durable resources that users browse, import, assign, and maintain over time > - A flat skill list plus tags does not provide a stable location or hierarchy for personal, company, project-imported, and bundled skills > - Folder paths need to be canonical, company-scoped, safe to move, and preserved across re-imports without changing skill IDs > - The `/skills` UI also needs traversal, breadcrumbs, move/create flows, and a dedicated My Skills namespace that work on desktop and mobile > - This pull request adds the folder data model and APIs, reserved-root lifecycle, project import behavior, and the folder-first skills experience > - The benefit is a predictable filesystem-like organization model while tags remain available for cross-cutting classification ## Linked Issues or Issue Description Refs #9619 — the reviewed folder foundation was intentionally closed and folded into this combined feature PR. Refs #9026 — earlier flat-folder attempt superseded by this integrated implementation. Refs #3281 — related skill organization proposal; this PR uses canonical persisted folders rather than deriving groups from skill keys, and does not add hidden-skill behavior. **Feature request** - **Problem:** Skills currently lack a canonical hierarchical location, making personal skills, project imports, bundled skills, and company-authored skills difficult to traverse and manage at scale. - **Proposed behavior:** Add nested company-scoped folders with stable paths, reserved My/Projects/Bundled roots, subtree queries, safe move/create operations, and a folder-first `/skills` library UI. - **Import behavior:** New project scans file skills under `projects/<project-slug>`; later imports update content without overriding a user-selected folder. - **Alternatives considered:** Tags alone remain useful for cross-cutting classification, but they do not provide canonical location, nesting, reserved namespaces, or stable import placement. - **Roadmap alignment:** Extends the completed Skills Manager and Scheduled Routines capabilities without duplicating an active roadmap item. ## What Changed - Adds `folders` persistence for routine and skill folders, nested canonical paths, parent/slug/system-key fields, migration backfills, and reapply-safe migrations `0174`–`0175` after current master migrations. - Adds company-scoped folder CRUD, cycle/depth/namespace validation, reserved My/Projects/Bundled lifecycle, item moves, subtree filtering, and folder paths on skill results. - Preserves project-import placement: first import files into the project folder, while re-import keeps user-owned placement and stable skill IDs. - Adds the `/skills` folder tree rail, tags facet, breadcrumbs, subfolder browser, move/new-folder dialog, canonical detail location, inline tag editing, and folder-aware Studio creation. - Keeps bundled skills read-only even when their source metadata is incomplete by detecting the reserved Bundled folder and hiding selection/move actions. - Extends routine folder UI and OpenAPI coverage, and adds regression tests across migrations, services, routes, tree helpers, pages, and Studio creation. ## Verification - `pnpm exec vitest run packages/db/src/nested-skill-folders-migration.test.ts server/src/__tests__/folders-routes.test.ts server/src/__tests__/folders-service.test.ts server/src/__tests__/company-skills-service.test.ts server/src/__tests__/routines-service.test.ts ui/src/components/folders/FolderControls.test.tsx ui/src/components/folders/SkillFolderTree.test.tsx ui/src/components/folders/skill-folder-tree.test.ts ui/src/pages/CompanySkills.test.tsx ui/src/pages/Routines.test.tsx ui/src/pages/SkillStudio.test.tsx ui/src/lib/company-skill-routes.test.ts ui/src/lib/skill-create.test.ts` — 13 files, 192 tests passed. - `pnpm exec vitest run ui/src/pages/CompanySkills.test.tsx ui/src/components/folders/SkillFolderTree.test.tsx` — 2 files, 20 tests passed after preserving the existing PR's bundled-skill fixes. - `pnpm -r typecheck` — passed for all workspace packages. - `pnpm test:run` — passed in an isolated CI-like environment with inherited Paperclip runtime identity and static AWS credential variables removed. - `pnpm build` — production build passed for all workspace packages. - Greptile iteration 2 — 5/5 confidence with zero unresolved threads on commit `ff2d67aa71`. - Latest-head GitHub checks — all success, neutral, or skipped; PR is mergeable with a clean merge state. - `pnpm check:token-gates` — reports nine existing `#9627` comment false positives already present on `master`; this PR introduces no new token violation. ## Risks - **Migration/backfill:** `0174` creates the foundation and `0175` adds nested/reserved semantics. Both are ordered after current master migration `0173`, are covered by numbering/safety checks, and are designed to be reapply-safe. - **Reserved namespaces:** My, Projects, and Bundled roots are service-managed. Regression coverage prevents namespace squatting, cross-company folder use, bundled writes, cycles, and excessive depth. - **Behavioral change:** Project scans choose a project folder only on initial creation; existing skills deliberately retain their current folder during refresh. - **UI scope:** The folder rail applies to the Installed library; Catalog retains the discovery-oriented category sidebar. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI `gpt-5.5` in Codex CLI, medium reasoning mode; runtime did not expose a context-window value. Used repository/file tools, terminal execution, Git/GitHub operations, test execution, and code editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6ec059ab4e |
fix(server): suppress stale handoff alarms during live continuation (#9695)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane records a successful-run handoff when productive work ends without a durable next-step disposition > - That handoff state was derived only from the latest activity event, without checking whether a corrective run or wake was currently alive > - As a result, actively progressing issues could still show a high-severity missing-disposition alarm and blocked-inbox row > - The same stale required event could also remain indefinitely when a later successful run correctly skipped recovery because another valid continuation path already existed > - This pull request makes the derived state liveness-aware, suppresses attention only while the live path exists, and resolves stale required events on valid-path skips > - The benefit is that productive work stays calm while genuine stalls still resurface automatically when liveness disappears ## Linked Issues or Issue Description - **Bug:** An issue whose latest successful-run handoff event is `required` continues to report a missing disposition even while a heartbeat run, scheduled retry, or queued/deferred/claimed wake is actively targeting that issue. - **Expected behavior:** The API should expose current continuation liveness, the blocked inbox should suppress the alarm only while that path remains live, and a later successful run that skips recovery because a valid path exists should durably resolve the stale event. - **Related but distinct:** #9370 changes disposition freshness at detection time; #8748 adds an explicit policy opt-out. This PR preserves detection/escalation policy and fixes read-time/current-liveness state. ## What Changed - Extended `SuccessfulRunHandoffState` with `hasLiveContinuation` and optional `liveRunId` evidence. - Added bounded liveness hydration for required handoff states using active heartbeat-run and wake-request signals. - Suppressed `missing_disposition` blocked-inbox rows only while a run, scheduled retry, or live wake targets the issue. - Added durable `issue.successful_run_handoff_resolved` logging when handoff detection skips because another valid continuation path owns the next action. - Added focused regressions for live/absent derived state, self-healing attention suppression, valid-path skip classification, and resolved-event logging. - Updated UI normalization and fixtures for the shared contract without changing rendering behavior. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm vitest run server/src/services/recovery/successful-run-handoff.test.ts server/src/__tests__/issue-list-assignee-filter-routes.test.ts server/src/__tests__/issue-blocker-attention.test.ts` — 56 passed - `pnpm vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "queues one finish-handoff wake when a successful run leaves in-progress work without a next action"` — 1 passed - `git diff --check` ## Risks - Low risk: no schema or migration changes, and detection, bounded correction attempts, and escalation behavior are unchanged. - Liveness lookups are limited to issues whose latest handoff state is `required`; blocked-inbox suppression reuses rows already loaded by that query path. - Suppression is read-time and self-healing: when the run or wake stops, the alarm returns on the next fetch. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.4`, tool-enabled software-engineering workflow with repository, shell, test, Git, GitHub, and Paperclip control-plane access. Context-window size is not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a04a77c9d3 |
feat(authz): govern agent inbox archive access (#9658)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - The inbox subsystem must let agents act for a responsible user without silently granting access to every company user's tasks > - Existing authorization had no inbox-specific action, target-user scope, or per-user agent policy > - Inbox archive data also needs company-safe ownership and replay-safe schema changes before API mutations can rely on it > - This pull request adds the database policy foundation and a fail-closed `inbox:manage` authorization decision > - The benefit is a least-privilege core for later inbox archive endpoints, including explicit cross-user grants and low-trust denial ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`packages/db`, `packages/shared`, and `server`). ### Problem or motivation Agents need to manage inbox state for the user responsible for their run, but the control plane lacks an inbox-specific permission model and user-targeted grant scope. A generic mutation path would risk cross-user access or inconsistent policy enforcement. ### Proposed solution Add inbox archive ownership and per-user agent policies, introduce `inbox:manage`, and evaluate responsible-user defaults, disabled/allowlist policies, active membership, low-trust presets, and scoped cross-user grants in one authorization decision. ### Alternatives considered Reusing generic issue mutation permissions was rejected because it cannot express user-targeted inbox scope. Requiring grants for all self-user access was rejected because it would make the responsible-user path closed by default instead of using the requested per-user policy model. ### Roadmap alignment `ROADMAP.md` contains no overlapping inbox archive or inbox authorization item; this is incremental control-plane authorization work. ### Additional context This PR provides the authorization and schema foundation. Route and UI behavior can build on this decision without duplicating access-control rules. ## What Changed - Builds on the merged migration `0172_inbox_archive_agent_policies` (#9654) for company/user-scoped inbox archives and per-user agent policy rows. - Added replay-safe migration `0173_inbox_policy_agent_cleanup` with a GIN allowlist index and GIN-backed database cleanup that removes deleted agent IDs from policy allowlists. - Added Drizzle schema exports for inbox agent policies and responsible-user ownership on inbox archives. - Added the shared `inbox:manage` permission key and `scope.userIds` evaluation for user-targeted grants. - Added fail-closed inbox authorization for unresolved targets, inactive memberships, low-trust agents, disabled policies, allowlist misses, and ungranted cross-user access. - Added migration replay coverage and the full inbox authorization decision matrix. ## Verification - `pnpm exec vitest run packages/db/src/inbox-archive-agent-policies-migration.test.ts server/src/__tests__/authorization-service.test.ts` — 50 tests passed. - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check origin/master...HEAD` ## Risks - The merged `0172` migration changed inbox archive uniqueness from agent-owned to responsible-user-owned rows; `0173` is additive (index + cleanup trigger) and idempotent, and replay coverage verifies both remain safe for databases that already applied an earlier form. - `scope.userIds` uses the existing JSON grant-scope parser, so malformed privileged grant payloads continue to fail through the shared parsing behavior rather than a dedicated schema. - Cross-user grants intentionally act as board-admin overrides; responsible-user default access remains bounded by disabled and allowlist policies. - The authorization action is not yet wired to public mutation routes, limiting immediate behavioral impact while establishing the contract those routes must use. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with `gpt-5.6-sol`, high reasoning effort, CLI tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Context window size is not exposed by the configured adapter. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c65ab09d9f |
fix(recovery): wait for provider quota resets (#9635)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and keep assigned work moving safely. > - The recovery subsystem decides whether a failed agent run should retry, wait, block for configuration, or escalate to another owner. > - Provider usage-limit failures currently arrive as generic `adapter_failed` results, so stranded-work reconciliation can create takeover recovery even when the provider states that capacity will reset later. > - Credential and model lookup failures are also configuration problems, not evidence that another agent should take over the task. > - This pull request classifies those failure families at recovery time and persists the classification on the run. > - Quota failures now schedule a monitor for the original assignee at the parsed reset time, or after a bounded default backoff when no reset time is available. > - The benefit is that transient provider capacity waits no longer wake recovery owners, while configuration failures stop with an actionable classification. ## Linked Issues or Issue Description No public GitHub issue exists for this exact change. **What happened?** When an assigned issue's latest run failed with a provider usage-limit message such as "try again at 12:00 AM (UTC)," recovery treated the run as generic `adapter_failed` work and could create a takeover action. Missing credentials and `model_not_found` failures followed the same generic path. **Expected behavior:** Provider quota failures should keep the original assignee and schedule a monitor for the reset time, without creating recovery work or immediately waking another owner. Missing credentials and model lookup failures should be classified as `configuration_incomplete` and blocked with the configuration fix recorded. **Steps to reproduce:** 1. Assign and start an issue for an agent. 2. Record a failed heartbeat run with `errorCode: adapter_failed` and a provider quota/reset message. 3. Run stranded assigned-issue reconciliation. 4. Observe that the old behavior routes the issue through generic recovery instead of waiting for provider capacity. Reproduced on `master` at `9af96461d`. This is a core recovery bug, not adapter-specific, and applies to built-from-source deployments with either embedded PGlite or Postgres. Related work checked: #9288 adds adapter-side Claude provider-limit classification; #5392 suppresses some recovery creation for quota-class errors; #9634 is a broader recovery-routing change with overlapping provider-quota behavior. This PR is the narrow recovery-service fix with focused parsed-reset, fallback-backoff, zero-takeover, and configuration-failure coverage. ## What Changed - Added conservative recovery-time classification for provider quota, missing-credential, and model-not-found adapter failures. - Parsed provider reset timestamps with a default one-hour backoff when no usable reset time is present. - Persisted `provider_quota` or `configuration_incomplete` metadata on the failed heartbeat run. - Scheduled quota monitors for the active issue owner, including the current review participant, without creating recovery actions or enqueueing takeover wakes. - Routed configuration failures to blocked recovery with actionable evidence instead of a takeover. - Added unit and embedded-database regression coverage for parsed/fallback quota timing, zero CTO/recovery wake behavior, and configuration classification. ## Verification - `pnpm exec vitest run server/src/services/recovery/provider-failure-classification.test.ts server/src/__tests__/issue-recovery-actions.test.ts server/src/__tests__/issue-monitor-scheduler.test.ts server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts` — 4 files passed, 66 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. ## Risks - Recovery behavior changes for text-matched adapter failures; matching is intentionally conservative, and unmatched failures retain the existing generic recovery path. - Provider reset strings do not always include a date or timezone; parsing chooses the next future matching time and falls back to a one-hour wait when the timestamp is unusable. - This overlaps the provider-quota portion of broader recovery-routing PR #9634, so only one implementation should land if both remain open. - No schema, migration, API contract, or UI changes are included. No documentation update is needed because this corrects internal recovery behavior without changing operator commands or configuration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with model `gpt-5.4`, medium reasoning, tool use, and code execution. The runtime does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
85404b46c5 |
fix(server): throttle serial recovery repeats (#9651)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies. > - Its recovery services create productivity reviews and liveness escalations when work stops making progress. > - Existing uniqueness guards prevent concurrent duplicates, but terminal recovery tasks can still be recreated serially without enough time for conditions to change. > - That creates noisy review churn for persistently stalled issues and immediate liveness re-escalation after a recovery task closes. > - This pull request adds bounded, configurable cooldown and no-action suppression behavior to those two recovery paths. > - The benefit is quieter recovery automation that still resumes automatically after source activity or cooldown expiry. ## Linked Issues or Issue Description ### Pre-submission checklist - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I can reproduce this behavior on `master`. - [x] I have confirmed the behavior originates in Paperclip core recovery orchestration, not an adapter, provider, or local configuration. ### What happened? Recovery reconciliation can serially recreate equivalent system-origin tasks after previous tasks become terminal. Productivity reviews allowed multiple creations for the same source issue within a rolling day, and a closed liveness escalation could be recreated immediately for the same incident or recovery leaf. ### Expected behavior Productivity review creation should be limited to once per rolling 24 hours, repeated completed reviews that produced no source action should eventually suppress further creation until activity resumes, and recently terminal liveness escalations should receive a short cooldown before recreation. ### Steps to reproduce 1. Create a stalled assigned issue that meets productivity-review eligibility. 2. Complete repeated productivity-review tasks without adding source-issue activity, then reconcile again within 24 hours. 3. Create and close a liveness escalation for a blocked issue graph, then immediately reconcile the same graph. 4. Observe that equivalent system tasks can be recreated serially without a meaningful state change. ### Paperclip version or commit `5588ddf68175eea448f9d19677b97d7393c38c3d` (`master` when reproduced) ### Deployment mode Local dev (`pnpm dev`) ### Installation method Built from source (`pnpm dev` / `pnpm build`) ### Agent adapter(s) involved - [x] Not adapter-specific (core bug) ### Database mode Embedded PGlite (default — `DATABASE_URL` unset) ### Access context Unclear / not applicable ### Node.js version Current repository-supported Node.js runtime. ### Operating system Linux development environment. ### Relevant logs or output No error is emitted; the bug is repeated task creation visible in persisted issue history. ### Relevant config (if applicable) No special configuration is required. ### Additional context The concurrent/open-task uniqueness guards work as designed; this change targets serial repeats after matching tasks become terminal. ### Privacy checklist - [x] I have reviewed all pasted output for PII and redacted where necessary. ## What Changed - Tightened the productivity-review creation cap to one review per source issue in a rolling 24-hour window. - Added configurable suppression after three consecutive completed reviews with no source-issue activity, with automatic reset when source activity occurs. - Added a configurable one-hour default cooldown for matching terminal liveness escalations. - Exposed the liveness reconciliation clock/cooldown inputs for deterministic orchestration tests. - Added focused tests for daily enforcement, no-action suppression and reset, and cooldown expiry. ## Verification - `pnpm exec vitest run server/src/__tests__/productivity-review-service.test.ts server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts` — 36 tests passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - Confirm the focused tests demonstrate creation after source activity and after the liveness cooldown expires. ## Risks - Low-to-moderate behavioral risk: recovery tasks intentionally appear less often, so overly aggressive thresholds could delay intervention for a persistently stalled issue. - Thresholds are configurable through reconciliation inputs, and source activity resets productivity-review suppression. - No database migration, public API change, telemetry contract change, or UI behavior change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.3-Codex, with reasoning, repository/terminal tool use, code execution, and test execution. The runtime did not expose a reliable context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
992389480a |
fix(server): restore hot-restart run adoption (#9647)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The local heartbeat/runtime subsystem starts long-running local agent processes and records their run state. > - Operators sometimes need to rebuild and restart the Paperclip server while local agent processes are still alive. > - A normal restart should remain conservative, but a guarded production hot restart needs an explicit marker, startup reconciliation, and an inspectable report. > - The broader hot-restart PR is currently merge-conflicted, so this pull request lands the minimal server-side recovery path on current `master`. > - The benefit is that deploy operators can restart from a current branch without reverting production changes and without marking adopted live runs as `process_lost`. ## Linked Issues or Issue Description No public GitHub issue exists for this deploy-safety fix. Bug fix: - What happened: the current deployable `master` branch did not include the hot-restart marker CLI, startup adoption report path, or health version proof needed by guarded service restarts. - Expected behavior: a deploy operator can write a one-shot marker before restarting, the old server snapshots eligible running child processes, the new server reports adopted/finalized/lost runs, and adopted live runs are not reaped as `process_lost`. - Steps to reproduce: restart a server with running local child-process heartbeat runs without the marker/adoption path; startup orphan reaping has no adoption metadata and treats live detached children as lost. - Paperclip version/commit: fixed on top of `master` at `b606869a6`. - Deployment mode: production/local-service style deployments that rebuild and restart the primary `paperclip.service`. - Related PR: Refs #9628. This PR intentionally lands a smaller deploy-safe subset because #9628 is currently merge-conflicted. - Duplicate search: searched public PRs/issues for `hot restart` and `process_lost adoption`; #9628 is the directly related prior implementation. ## What Changed - Added `scripts/request-hot-restart.ts` to write a one-shot hot-restart intent marker under `PAPERCLIP_HOME`. - Added `server/src/services/hot-restart.ts` for intent/report path resolution, parsing, atomic writes, shutdown snapshots, and marker cleanup. - Wired server shutdown/startup so explicit hot restarts snapshot active runs, skip the normal heartbeat drain, reconcile live child processes on boot, and write `hot-restart-report.json`. - Preserved adopted run metadata so normal orphan reaping does not regress adopted live runs to `process_lost`. - Added `serverVersion` health proof alongside existing `version`, plus docs and regression coverage. ## Verification - `pnpm vitest run server/src/__tests__/health.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts` — 2 files passed, 100 tests passed. - `pnpm --filter @paperclipai/server typecheck` - `env PAPERCLIP_HOME="$PAPERCLIP_RUN_SCRATCH_DIR/hot-restart-cli-smoke" pnpm --filter @paperclipai/server exec tsx ../scripts/request-hot-restart.ts --server-pid 12345` - Branch ancestry checked after `git fetch origin master`: `origin/master` was `b606869a6`, and `HEAD..origin/master` was empty. ## Risks - Medium risk: process adoption depends on PID/PGID metadata and the service manager leaving child processes alive for the guarded restart. - Normal restarts remain conservative, but an incorrect marker PID intentionally falls back to graceful drain instead of adoption. - The PR is server-only and does not include the broader UI/experimental-setting work from #9628. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5 via Codex coding agent in a Paperclip execution workspace; tool use and shell/code execution enabled; context window not surfaced by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4f9894df44 |
fix(server): bound accepted-interaction continuation recovery (#9656)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Heartbeat recovery keeps assigned issues moving when a run or continuation path disappears > - Accepted issue-thread interactions can create a continuation wake after an agent previously parked for review > - The recovery sweep could requeue that accepted-interaction wake while the queued-run gate cancelled it using the older pre-acceptance park summary > - That cancellation path had no bound, so recovery could repeat the same wake and cancellation indefinitely > - This pull request makes accepted-interaction evidence supersede the older park and caps repeated recovery cancellations at three attempts > - The benefit is that accepted work resumes normally, while genuine repeated failures become a visible dependency wait or escalation instead of a cancel loop ## Linked Issues or Issue Description Refs #9331 The accepted-interaction continuation recovery added by #9331 can encounter a stale continuation summary written before approval. The sweep requeues a continuation carrying the accepted interaction timestamp, but queued-run invalidation cancels it because the older summary says to wait for review. Recovery then sees the accepted interaction without a successful run and requeues again. This PR prevents that stale-summary cancellation and adds a bounded fallback if three equivalent cancellations have already occurred. ## What Changed - Let queued continuation wakes with a parseable `interactionResolvedAt` bypass a pre-acceptance waiting-for-review park summary. - Count consecutive unsuccessful continuation runs for the same issue and agent since interaction acceptance; after three review-park cancellations, convert a real dependency wait or use the existing visible escalation path. - Add focused regression coverage for the park bypass, unchanged non-interaction park behavior, below-cap requeue, cap escalation, and successful-run skip. - Document the accepted-interaction precedence and bounded requeue contract in execution semantics §9.2. ## Verification - `pnpm vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "accepted interaction continuation recovery|accepted interaction recovery after its continuation succeeds|requeues accepted interaction continuations stranded"` - `pnpm vitest run server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts -t "pre-acceptance review park|continuation summary parks executor work"` - `pnpm --filter @paperclipai/server typecheck` ## Risks - Low risk: the park bypass only applies when the queued context contains a parseable interaction resolution timestamp. - The retry bound is scoped to unsuccessful `issue_continuation_needed` runs for the same company, issue, agent, error code, and post-acceptance time window. - No schema, migration, API, or UI changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.5`, high-reasoning coding mode with repository tool use and command execution; context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3124dd0f1e |
feat(server): recovery observability report and rate alert (#9644)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - When a run is stranded (process lost, adapter failure, a finished
run with no disposition, an over-eager inactivity kill), the harness
opens a *recovery action* and wakes an owner to recover it
> - Recovery volume regressed sharply in one week — 3.26% of all runs vs
a ~1.2% monthly norm, 5–8x the prior volume — and nobody noticed until
it was ~194 actions deep, because there was no way to *see* the recovery
rate
> - We also could not see which causes drive recovery, nor how often a
manager ends up doing the deliverable work themselves instead of handing
it back to the original owner (the product goal is that managers doing
the work stays rare)
> - This pull request adds a recovery-observability report + API
endpoint: weekly rate normalized per run, a threshold alert, the cause
taxonomy live from the ledger, and the handed-back vs owner-completed
ratio and per-cause routing outcomes
> - The benefit is that a recovery regression like that week is caught
by a threshold instead of by a human noticing it by feel, and each
recovery playbook row can be verified in production
## Linked Issues or Issue Description
**Feature.**
**Problem or motivation**
Recovery takeovers are a first-class exception path
(`issue_recovery_actions`), but there is no aggregate view of them. A
week where the recovery rate tripled went unnoticed until it was deep.
There is no signal for (a) the per-run recovery rate over time, (b)
which cause + run error code drives it, or (c) whether the recovery
owner hands the task back to the original assignee or ends up doing the
deliverable work themselves.
**Proposed solution**
A read-only report service and `GET
/companies/:companyId/recovery-observability` endpoint that surfaces the
weekly rate, a threshold alert, the cause taxonomy, the hand-back ratio,
and per-cause routing outcomes.
**Alternatives considered**
Adding `handed_back` / `owner_completed` to the recovery-action outcome
vocabulary and writing them at resolution time. Rejected for this
change: the distinction is derivable from the recovery owner, the
recorded return owner, and where the source issue actually landed, so
the report works against all historical data without a backfill.
**Roadmap alignment**
Implements the recovery-observability line of the approved
recovery-takeover plan (make regressions visible via a threshold rather
than by human feel); no schema or write-path change.
## What Changed
- Add `server/src/services/recovery-observability.ts`:
- `recoveryObservabilityService(db).report(companyId, { weeks,
thresholdPercent, now })` returns weekly rates (recovery actions / runs,
Monday-anchored to match the retrospective), a `cause` +
`latestRunErrorCode` breakdown, a handed-back vs owner-completed
summary, and per-cause routing outcomes.
- `evaluateRecoveryRateAlert(weekly, thresholdPercent)` — a pure
function (default threshold 2% of runs) returning the breached weeks and
whether the latest week regressed.
- `classifyRecoveryHandoff(...)` — a pure classifier deriving
`self_recovery` / `handed_back` / `owner_completed` from the recovery
owner, return owner, and final issue landing.
- Add `GET /companies/:companyId/recovery-observability` (optional
`weeks` and `threshold` query params) to the existing dashboard router.
- The `weeks` window is bounded (`MAX_WINDOW_WEEKS = 104`,
service-authoritative and re-clamped at the route) so a large query
value can't over-allocate the per-week array.
- Add tests: unit coverage for the alert and the classifier, plus an
embedded-Postgres integration test that seeds synthetic runs and
recovery actions crossing 2% and asserts the alert fires and the
hand-back ratio is computed.
## Verification
- `CI=1 NODE_ENV=development npx vitest run
server/src/__tests__/recovery-observability.test.ts` — 10/10 pass
(includes the synthetic 2%-crossing alert case and the hand-back ratio
case).
- Rendered against a live database of 300+ recovery actions: the weekly
rates reproduce the retrospective (e.g. 1.37% / 1.53% / 0.65% / 0.86% /
1.37% for early-June weeks), the alert fires on the two most recent
weeks (3.15% and 3.05%, both over 2%), and the hand-back summary shows
owner-completed ≈ 73% vs handed-back ≈ 27% — matching the observed
"managers keep ~80% of takeovers".
## Risks
- Low risk. Read-only: adds one GET endpoint and a service; no schema,
migration, or write-path changes. The hand-back classification reads the
source issue's current assignee/status, so a much-later reassignment
could reclassify a historical action — acceptable for an aggregate trend
view.
## Model Used
- Claude, `claude-opus-4-8` (Opus 4.8), extended thinking, tool use /
code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
DottaandPaperclip
|
8368fb30b0 |
fix(routines): coalesce sub-hourly catch-up runs (#9649)
## Thinking Path > - Paperclip is the open source control plane people use to run AI-agent companies. > - Scheduled routines support catch-up policies when the server resumes after missed cron ticks. > - The existing capped replay policy dispatched once per missed tick, which can flood the board after downtime for frequent schedules. > - Sub-hourly routines usually need one prompt catch-up execution rather than historical per-tick replay, while hourly-or-slower schedules may rely on the existing behavior. > - This pull request coalesces missed sub-hourly ticks into one execution and keeps the slower-schedule behavior unchanged. > - The benefit is bounded recovery work without changing the semantics of lower-frequency scheduled routines. ## Linked Issues or Issue Description ### What happened? When a scheduled routine using `enqueue_missed_with_cap` resumes after several missed sub-hourly cron ticks, Paperclip dispatches one catch-up execution for every missed tick. Those executions arrive in a same-second burst and can flood the board with duplicate-looking work. ### Expected behavior Sub-hourly schedules should advance past all missed ticks but dispatch exactly one catch-up execution. Hourly-or-slower schedules should retain capped per-tick replay. ### Steps to reproduce 1. Build Paperclip from `master` and create a routine with a sub-hourly cron schedule and `catchUpPolicy: enqueue_missed_with_cap`. 2. Set its persisted `nextRunAt` far enough in the past to cover several scheduled occurrences. 3. Run routine catch-up processing. 4. Observe multiple catch-up dispatches instead of one coalesced execution. ### Paperclip version or commit Reproduced on `master` before this PR. ### Deployment mode Built from source in local development with embedded PGlite. ## What Changed - Classify sub-hourly cadence from timezone-aware scheduled occurrences, avoiding daily multi-minute false positives while supporting schedules restricted to active days. - Coalesce all missed sub-hourly ticks into one catch-up dispatch while advancing `nextRunAt` to the next future occurrence. - Preserve capped per-tick replay for hourly-or-slower schedules. - Clarify the catch-up policy labels in both routine editing surfaces. - Add regression coverage for both the coalesced and preserved behaviors. ## Verification - `pnpm exec vitest run server/src/__tests__/routines-service.test.ts --testNamePattern='coalesces multiple missed sub-hourly ticks|continues replaying each missed hourly tick|continues replaying missed ticks for daily schedules with multiple minute values|coalesces sub-hourly schedules restricted to weekdays'` — 4 passed. - `pnpm check:token-gates` — all gates clean. - `git diff --check origin/master...HEAD` — clean. ## Risks - Low-to-moderate behavioral risk: sub-hourly routines using `enqueue_missed_with_cap` now intentionally receive one recovery execution instead of one per missed tick. - Hourly-or-slower schedules retain their previous capped replay behavior, limiting the compatibility surface. - No schema, migration, workflow, or lockfile changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI with GPT-5.5, medium reasoning, code execution and repository tool use; the runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd7c0d5f83 |
fix(issues): deduplicate repeated creates (#9650)
## Thinking Path > Paperclip already treats issue creation as a company-scoped mutation, but retries and parallel agent heartbeats can submit the same create more than once. Client instructions cannot provide at-most-once behavior under concurrency, so the guard belongs in the server transaction. This change adds an explicit company-scoped idempotency contract, a conservative fallback for recent open same-parent titles, and run attribution for auditability. Advisory transaction locks serialize competing requests before lookup/insert, avoiding the race that affected the prior attempt. ## Linked Issues or Issue Description Fixes #6529. This is a clean replacement for #6936, which was closed because it mixed unrelated changes and its check-then-insert implementation was not concurrency-safe. Unlike that attempt, this PR is scoped to eight files, uses a dedicated idempotency-key table, and serializes duplicate candidates inside the create transaction. ## What Changed - Accept optional `idempotencyKey` and `allowDuplicate` fields on issue creation. - Replay the existing issue with HTTP 200 and deduplication metadata for a repeated company/key pair. - Deduplicate recent open issues with the same company, parent, and normalized title for 48 hours unless `allowDuplicate: true` is supplied. - Persist idempotency mappings in a company-scoped table and serialize competing creates with transaction advisory locks. - Populate `originRunId` from `X-Paperclip-Run-Id` for agent/manual creates when the body does not provide an origin run. - Add route integration coverage for key replay, title fallback, bypass, closed/old recreation, company scoping, and run attribution. ## Verification - `pnpm exec vitest run server/src/__tests__/issue-create-deduplication-routes.test.ts` — 7 tests passed. - `pnpm --filter @paperclipai/db typecheck` — passed, including migration numbering and safety checks. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check origin/master...HEAD` — passed. - `pnpm exec vitest run server/src/__tests__/issue-assigned-backlog-contract-routes.test.ts server/src/__tests__/issue-create-deduplication-routes.test.ts` — 10 tests passed after the service-contract compatibility fix. ## Risks - The title fallback intentionally treats normalized same-parent titles as duplicates for 48 hours; callers creating intentionally repeated titles must send `allowDuplicate: true`. - Advisory locks use hashed duplicate keys, so an extremely unlikely hash collision can serialize unrelated creates but cannot merge their lookup results. - Deleting an issue cascades its idempotency mapping, allowing the same key to create a replacement later. ## Model Used - OpenAI `gpt-5.6-sol`, high reasoning effort, Codex CLI with repository, shell, GitHub CLI, and Paperclip API tool access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ea0e899905 |
fix(search): honor extract match limits + harden pr-gardening candidate discovery (#9652)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The `/pr-gardening` skill drives a bundled agent that scans a
company's issues for those linked to open GitHub PRs, then reports on
their state; it relies on the server's company-search **extract**
endpoint to pull PR references out of issue bodies
> - Two gaps surfaced during end-to-end QA of the gardening workflow:
the extract service silently ignored a per-issue match cap, so callers
could not bound how many matches came back per issue, and the skill's
candidate-discovery scripts fell over on large repos and on issues that
referenced deleted PRs
> - Left unaddressed, the gardener either truncated its scan
unpredictably or aborted outright, so it could not reliably enumerate PR
candidates
> - This pull request honors an explicit `matchesPerIssue` limit in the
extract search API and hardens the skill's candidate discovery against
missing/unavailable PRs and oversized `gh` output
> - The benefit is a PR-gardening workflow that scans deterministically
and finishes cleanly on real-world companies
## Linked Issues or Issue Description
No pre-existing public GitHub issue — describing the bug in-PR following
the bug report template (`.github/ISSUE_TEMPLATE/bug_report.yml`).
### What happened?
The company-search extract endpoint accepted a per-issue match limit but
did not apply it, returning matches capped only by the old hardcoded
constant regardless of the caller's request. Separately, the
`/pr-gardening` skill's candidate-discovery scripts crashed when a
scanned issue referenced a deleted PR (GitHub `Not Found (HTTP 404)` /
GraphQL `Could not resolve to a PullRequest`) and could exceed the
default `gh` output buffer on large result sets, aborting the whole
scan.
### Expected behavior
The extract API bounds matches per issue when a caller passes
`matchesPerIssue` (default 20, max 200), and omitting it preserves the
previous default. The gardening scripts skip PRs that are
deleted/unavailable and tolerate large `gh` responses without aborting
the scan.
### Steps to reproduce
1. Call the company-search extract endpoint with a `matchesPerIssue`
value against an issue containing many PR references — previously the
value was ignored.
2. Run the pr-gardening candidate scan against a company whose issues
reference a since-deleted PR — previously the scan threw instead of
skipping that PR.
### Paperclip version or commit
`master` at the base of this PR (branch cut from current
`origin/master`).
### Deployment mode
Local Paperclip instance / self-hosted.
## What Changed
- **Extract search honors `matchesPerIssue`**: added the
`matchesPerIssue` field to the shared search validator/types and applied
the cap in `company-search-extract` so results are bounded per issue
(`packages/shared`, `server/src/services/company-search-extract.ts`,
`doc/SPEC-implementation.md`).
- **Hardened pr-gardening candidate discovery**: `find-candidates.mjs` /
`lib.mjs` now request `matchesPerIssue=200`, treat missing/unavailable
PRs (deleted PR → `isMissingPullRequestError` / `unavailable`) as skips
instead of fatal errors, and raise the `gh` `maxBuffer` to 50 MB for
large repos.
- **Tests**: expanded `company-search-extract-{routes,service}.test.ts`
for the new limit and added coverage in `pr-gardening.test.mjs`.
## Verification
Re-run on a fresh worktree cherry-picked onto current `master`:
- `node --test
.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` → 8/8 pass
- `pnpm vitest run
server/src/__tests__/company-search-extract-routes.test.ts
server/src/__tests__/company-search-extract-service.test.ts` → 10/10
pass
## Risks
Low risk. `matchesPerIssue` is optional and backward-compatible
(omitting it preserves prior behavior). The skill changes only add
skip/tolerance paths and a larger buffer; no schema or migration
changes.
## Model Used
Claude — Opus 4.8 (`claude-opus-4-8`), extended thinking, tool use /
code execution via the Claude Agent SDK.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
DottaandPaperclip
|
d32ed88443 |
fix(recovery): route recovery by failure cause (#9634)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies and their work. > - Its recovery subsystem detects stranded issue execution and decides whether to retry, escalate, or request operator intervention. > - The existing recovery path used a mostly generic owner ladder and generic execution contract, so transient failures could wake a manager who then performed the deliverable instead of repairing and returning the task. > - Provider quota failures also entered the same takeover path even when the correct action was to wait for capacity and retry the original assignee. > - Recovery actions already retain the source owner and evidence needed to choose a cause-specific route, render a scoped contract, and measure whether work was handed back. > - This pull request adds a cause-keyed recovery playbook, propagates its contract through every built-in adapter, and makes resolved recovery actions return work to the original owner by default. > - The benefit is bounded self-recovery that preserves task ownership, avoids needless management takeover, and makes recovery outcomes observable. ## Linked Issues or Issue Description No matching public GitHub issue was found. Related recovery work was reviewed but is not duplicated here: #9630 restores bounded recovery continuations, #8807 changes one assignee-ranking case, and #9404 records runtime-failure transition evidence. This change instead introduces cause-specific routing and recovery contracts across the recovery lifecycle. ### What happened? When an issue became stranded, recovery generally selected an owner through the same fallback ladder and rendered the normal execution contract. That made the recovery wake look like ordinary deliverable work, even when the correct action was to retry the original agent, repair its runtime, or wait for a provider quota reset. ### Expected behavior Recovery should select a response by failure cause, tell the recipient to recover rather than complete the deliverable, suppress takeover wakes for provider quota waits, and return repaired work to its original assignee unless the recovery owner explicitly completes it. ### Actual behavior Recovery could escalate transient failures to management, omit the cause-specific next action from the wake, and leave the recovery owner assigned after the runtime problem was resolved. ### Impact The generic path creates avoidable management work, ownership churn, and budget consumption while obscuring whether recovery successfully returned work to the responsible agent. ## What Changed - Added cause-keyed routing for process loss, missing disposition, provider quota limits, Codex output inactivity, workspace validation failures, and fallback recovery causes. - Added recovery-scoped wake rendering that replaces the generic execution contract with the failure summary, original assignee, attempt count, next action, and cause-specific playbook instruction. - Propagated the structured recovery contract through all built-in adapter execution paths, including Hermes local and gateway adapters. - Added provider-quota wait monitoring so capacity failures schedule the original assignee instead of enqueueing a takeover wake. - Added hand-back behavior and `handed_back` / `owner_completed` outcome accounting when recovery actions are resolved. - Added focused routing, renderer, quota-monitor, and hand-back regression coverage plus implementation-spec documentation. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/heartbeat-workspace-branch-containment.test.ts server/src/__tests__/issue-recovery-actions.test.ts` - 4 test files passed; 194 tests passed. - Targeted `pnpm --filter ... typecheck` across `@paperclipai/adapter-utils`, `@paperclipai/shared`, `@paperclipai/server`, `@paperclipai/ui`, and all nine changed adapter packages. - 13 affected workspace packages passed typecheck. - `pnpm check:token-gates` - All UI token gates passed. ## Risks - Recovery routing behavior changes for stranded work, so an incorrectly classified cause could select a different recipient than before; fallback causes retain the existing management ladder. - Provider quota detection depends on structured failure evidence and conservative text matching; unmatched failures continue through fallback recovery. - Adapter prompt plumbing changes across built-ins, covered by shared renderer tests and compile-time call signatures. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with exact model ID `gpt-5.6-sol`, using reasoning, tool use, and code execution. The runtime does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ae77908618 |
feat(search): add bulk extract endpoint (#9507)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies > - Agents and operators need company-scoped search to discover relevant issue history safely > - The interactive search endpoint intentionally returns compact excerpts and low pagination caps for UI use > - Automation that inventories repeated references, such as pull-request URLs, needs exhaustive distinct matches without loading full issue objects into an LLM context > - Client-provided regular expressions would create an unsafe and expensive query surface, so extraction must remain literal with server-owned expansion modes > - This pull request adds a bounded agent-oriented extraction endpoint with explicit truncation > - The benefit is deterministic, compact bulk discovery across issues, comments, and documents while preserving company authorization and rate limits ## Linked Issues or Issue Description ### Subsystem affected `server/` REST API and `packages/shared/` contracts. ### Problem or motivation The existing interactive company search caps issue pagination and snippets, so automation cannot reliably enumerate every distinct literal or pull-request URL across issue descriptions, comments, and linked documents without fetching large full issue payloads. ### Proposed solution Add `GET /api/companies/:companyId/search/extract` with escaped literal matching, optional server-owned URL token expansion, issue/comment/document scopes, status/date filters, higher issue-level pagination caps, compact source references, and explicit pagination/match truncation flags. ### Alternatives considered Reusing `GET /issues?q=` would return unnecessarily large issue objects; increasing interactive-search snippet limits would make the UI API heavier; accepting arbitrary client regex would expose avoidable database cost and ReDoS risk. ### Roadmap alignment `ROADMAP.md` does not currently list a conflicting company-search or bulk-extraction initiative. GitHub searches found no directly duplicative open issue or pull request. ## What Changed - Added shared query validation and response contracts for literal and URL extraction. - Added a company-scoped extraction service that pages issues, gathers matching issue/comment/document sources, expands URL tokens, deduplicates values, and reports truncation explicitly. - Added the authenticated route using the existing company-search authorization decision and rate limiter. - Added targeted Vitest coverage for URL extraction, multi-source dedupe, date/status filters, match caps, cross-company denial, and rate limiting. - Documented the extraction surface in the implementation specification. ## Verification - `pnpm exec vitest run server/src/__tests__/company-search-extract-service.test.ts server/src/__tests__/company-search-extract-routes.test.ts server/src/__tests__/company-search-rate-limit-routes.test.ts server/src/__tests__/company-search-service.test.ts` — 30 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check` — passed. ## Risks - Bulk substring search can scan large text columns. The endpoint mitigates this with a minimum literal length, bounded issue pagination, a 20-distinct-match cap per issue, explicit truncation, existing company-search rate limiting, and no client-provided regex. - URL expansion uses a fixed server-owned pattern plus an escaped literal. A security review is requested as part of PR review to confirm the pattern and abuse controls. - No database migration or existing API response shape changes are included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI coding agent; exact runtime model ID and context-window size were not exposed to the session. Tool-enabled code execution and repository editing were used with medium reasoning effort. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3ae2c30f2f |
feat(skills): import skills from projects (#9620)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Company skills make reusable agent behavior discoverable and editable from one place. > - Projects already contain skill directories, but operators had to import each skill path manually. > - Copying those skills would break the desired write-through workflow between Skill Studio and the source project. > - The server therefore needs a safe preview/select/import contract that only accepts rediscovered, workspace-contained candidates. > - The UI needs a guided project picker that explains reference semantics, handles conflicts, and remains usable on mobile. > - This pull request adds that end-to-end project skill import flow with authorization, tenant-scope, traversal, and symlink regression coverage. > - The benefit is faster bulk onboarding while keeping project files as the single source of truth. ## Linked Issues or Issue Description **Feature request** **Problem:** Importing several skills already stored in a Paperclip project requires operators to discover and submit each local path individually. This is slow, hides which well-known directories were searched, and makes conflict/already-imported states difficult to evaluate before mutation. **Proposed solution:** Add an “Import skills from project” flow that previews skills from well-known directories, lets operators selectively import eligible candidates, and stores local-path references so Skill Studio edits write through to the project files. **Alternatives considered:** Copying files into company-managed skill storage was rejected because it creates divergent copies. Trusting client-supplied paths was rejected because imports must be constrained to server-rediscovered, workspace-contained candidates. **Additional context:** GitHub duplicate search found no existing issue or PR for this exact workflow. Refs #3799 for related skill-import inventory behavior; this PR does not claim to close that issue. ## What Changed - Extend `scan-projects` with backward-compatible preview and selective-import modes, typed validation, candidate statuses, and OpenAPI coverage. - Discover project skills under `skills`, `.agents/skills`, `.claude/skills`, `.codex/skills`, `.cursor/skills`, `.opencode/skills`, and `.gemini/skills`. - Re-discover selections server-side, enforce company/project/workspace scope, and reject traversal or symlink escapes before creating `local_path` references. - Add the Skills-page menu entry and responsive project import dialog with project selection, grouped candidates, select all/deselect all, conflicts, empty/error/403 states, and import results. - Add route, service, and component regressions for preview authorization, cross-tenant selections, traversal/symlink safety, selection counts, grouping, and result semantics. ### Screenshots **Choose a project**  **Review discovered skills**  **Mobile selection footer**  **Import result**  ## Verification - `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts ui/src/pages/skills/ImportSkillsFromProjectDialog.test.tsx` — 3 files, 81 tests passed. - `pnpm check:token-gates` — all token gates clean. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - Security review passed after adding tenant-scope and unauthorized-preview regressions; UX re-review approved desktop/mobile surfaces; QA passed all seven acceptance areas including write-through editing, deduplication, conflicts, empty state, and permission denial. ## Risks - Files remain referenced in project workspaces, so moving or deleting a source directory can make an imported skill unavailable; the UI explicitly communicates the reference behavior. - New well-known directory scans may discover more candidates than older versions, but preview mode prevents mutation until the operator confirms a selection. - The endpoint remains backward compatible: omitting `mode` preserves the prior full-import behavior. - No schema migration or telemetry event changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 4.8 with tool use/code execution assisted with the UI implementation and UX polish. OpenAI Codex CLI with tool use/code execution assisted with server implementation, security fixes, regression coverage, integration, and PR preparation; the runtime did not expose Codex's exact backing model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
9af96461d5 |
fix(server): restore stranded recovery continuations (#9630)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI agents and their work. > - Its server recovery layer classifies blocked issue graphs and restores interrupted heartbeat execution. > - A dependent issue could remain dispatch-suppressed by a cancelled blocker without producing operator-visible attention when the dependent still displayed as todo or backlog. > - Separately, a monitor-triggered run that lost its process before disposition could consume the monitor's one-shot wake without scheduling the existing bounded continuation. > - Both gaps strand useful work even though Paperclip already has the relevant blocker-attention and process-loss recovery mechanisms. > - This pull request widens the existing classification path and reuses the single process-loss retry for monitor dispatches with no future wake. > - The benefit is visible, routable recovery without weakening dependency checkout rules or introducing an unbounded retry loop. ## Linked Issues or Issue Description No matching public GitHub issue or pull request was found. ### What happened? Two server recovery cases could leave work stranded: 1. A non-terminal, agent-assigned issue with an unresolved cancelled blocker remained ineligible for checkout, but blocked-chain liveness classification only inspected issues already displaying `blocked` or `in_review`, so the existing `blocked_by_cancelled_issue` attention was not surfaced. 2. A one-shot issue monitor cleared its next check when dispatched. If that monitor-triggered run ended as `process_lost` without a tracked local child, the existing bounded retry gate rejected it and no future monitor wake remained. ### Expected behavior - Cancelled blockers continue to be unresolved dependencies, and their dependents receive blocker attention regardless of whether the dependent currently displays as backlog, todo, blocked, or in review. - A monitor-triggered run lost before disposition receives exactly one bounded continuation when no future monitor check exists; a second loss follows the normal recovery-action escalation path. ### Steps to reproduce 1. Create an agent-assigned todo issue blocked by a cancelled issue and run issue-graph liveness classification. 2. Observe that no cancelled-blocker finding appears before this change. 3. Dispatch a due issue monitor, clear its one-shot `monitorNextCheckAt`, and mark the resulting untracked run `process_lost`. 4. Observe that no retry is queued before this change. ### Environment - Paperclip commit: `3e348b96b` - Deployment: built from source / local test environment - Adapter: not adapter-specific; core server recovery - Database: embedded test database ## What Changed - Inspect non-terminal, agent-assigned issues with unresolved blocker edges during blocked-chain liveness classification. - Include cancelled dependents in the existing blocked-inbox attention query while preserving company-scoped relation checks. - Allow monitor-triggered `process_lost` runs with no future monitor wake to use the existing single bounded retry. - Mark monitor recovery retries as continuation-needed context and retain the existing second-loss escalation behavior. - Document cancelled-blocker and monitor-dispatch recovery semantics. - Add focused regressions for liveness findings, attention propagation, one retry, and second-loss escalation. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/issue-blocker-attention.test.ts server/src/__tests__/issue-liveness.test.ts` — 3 files, 126 tests passed. - `pnpm --filter @paperclipai/server typecheck` — passed. ## Risks - Low risk and server-only. The liveness scan inspects more unresolved dependency shapes, which can produce additional existing attention entries for previously invisible cancelled blockers. - Monitor recovery remains bounded by `processLossRetryCount < 1`, and the extra path only applies when the dispatch was monitor-triggered and no future monitor check exists. - No schema, migration, authorization, API-contract, or UI changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI `gpt-5.4` through Codex CLI, with reasoning, repository tool use, command execution, and test execution capabilities. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
3a727bf780 | fix(codex): warn when sandbox auth is shadowed (#9259) | ||
|
|
89ce36d7af |
feat(skills): open-by-default company skill policy and core UX (#9564)
## Thinking Path > - Paperclip uses company skills to make agent capabilities reusable across an organization. > - Skill operations currently mix capability availability with permission checks, which creates avoidable setup friction and inconsistent denial handling. > - The policy contract needs to remain open by default while allowing company-scoped restrictions for governed deployments. > - Core owns the canonical policy actions, persistence, evaluation, API behavior, safe import boundaries, and generic denial/read-only UI. > - Enterprise policy-editor implementation belongs in the separate `paperclip-ee` repository and is intentionally excluded from this PR. ### Problem or motivation Company skill operations can encounter permission dead ends even when no explicit restriction has been configured, and import-source classification can drift between policy evaluation and execution. ### Proposed solution Define eight canonical skill policy actions, default all actions to allowed, persist company-scoped restrictions, expose policy evaluation APIs, normalize import sources at the boundary, and update Skill Studio to present actionable restriction states without embedding Enterprise Edition implementation in the core repository. ### Alternatives considered Keeping capability checks distributed across routes and UI surfaces was rejected because it duplicates policy logic and makes denial behavior inconsistent. Shipping the Enterprise policy editor in this repository was rejected because `paperclip-ee` is a separate repository and must receive its own PR. ### Roadmap alignment Extends the completed **Skills Manager** roadmap area by adding coherent governance and removing workflow dead ends. ### Additional context The core API contract remains suitable for a separate Enterprise Edition editor, but this PR contains no `paperclip-ee` package or EE-specific UI integration code. ## What Changed - Added the company skill policy contract to product and implementation documentation, including the open-by-default rule, eight canonical actions, decision shape, and core/EE ownership boundary. - Added the company-scoped policy schema, migration `0170`, shared validators, policy service, REST routes, OpenAPI coverage, and focused server tests. - Hardened import policy enforcement by normalizing import sources and keeping source classification consistent between policy evaluation and execution. - Updated core Skill Studio behavior to remove generic permission dead ends and show actionable policy/platform denial states only when an operation is actually denied. - Removed the `plugin-paperclip-ee` package, Docker wiring, EE discovery/deep-link helpers, and EE-specific UI tests/stories from this PR so that implementation can be submitted separately to the EE repository. - Preserved open-by-default behavior when no explicit company restriction exists. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/components/skill-studio/SkillPolicySurfaces.test.tsx src/lib/skill-policy-denial.test.ts` — 20/20 passed. - `pnpm --filter @paperclipai/ui exec tsc --noEmit` — passed. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/worktree-config.test.ts` — 12/12 passed. - `pnpm check:token-gates` — passed with all gates clean. - `git diff --check` — passed. - `git diff --name-only origin/master | rg 'paperclip-ee|ee-skill-policy'` — no matches. ## Risks - Migration `0170` introduces company policy persistence; rollout depends on the migration applying before policy routes are exercised. - Open-by-default is an intentional behavioral policy: deployments expecting implicit denials must configure explicit restrictions. - Import normalization is security-sensitive and should retain focused review. - The separate EE editor must stay contract-compatible with the core policy API as policy actions evolve. ## Model Used - OpenAI Codex CLI, runtime model identifier and context-window size not exposed by this execution environment; reasoning, repository tool use, shell execution, and code review capabilities enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available to this runtime) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked or described the result above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the latest head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Evyatar Bluzer <bluzername@users.noreply.github.com> |
||
|
|
24bd860280 |
Stop cancelled productivity review loops (#5210)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Productivity review reconciliation creates manager-owned review issues when assigned work shows no-comment, long-active, or high-churn patterns. > - SplatImmo hit a loop because productivity-review issues were auto-cancelled while the source issue still matched the same trigger. > - The service already snoozed recently completed reviews, but cancelled reviews were ignored for that snooze check. > - This pull request treats recently cancelled productivity reviews as terminal snooze evidence. > - The benefit is that cancelling a review now suppresses immediate recreation without disabling useful future productivity reviews. ## What Changed - Renamed the recent-review lookup to terminal-review semantics and included `cancelled` alongside `done`. - Added a regression test proving a recently cancelled productivity review produces `snoozed` instead of creating another review. ## Verification - `pnpm exec vitest run server/src/__tests__/productivity-review-service.test.ts` passes: 1 file, 12 tests. - Queried the SplatImmo Paperclip instance for existing `Review productivity` issues: 500 `issue_productivity_review` issues found, all already `cancelled`, 0 active. ## Risks - Low risk: this only affects the reconciliation branch after a terminal productivity-review issue exists. - Operators who cancel a productivity review now get the same default 6-hour quiet window as completed reviews; after that window, persistent evidence can still create a fresh review. ## Model Used - OpenAI Codex coding agent, GPT-5 class model, tool-enabled code editing and local command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Yanis Ismail <yanis.ismail@emissive.fr> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ea66ea81e6 |
fix(auth): honor responsible-user grants for company skills (#9571)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Company skills are governed resources, so board users and agents acting for responsible users must be authorized consistently before mutating skill configuration > - The responsible-user authorization intersection handled several task permissions but did not map company-skill mutation actions to the corresponding `skills:create`, `skills:update`, and `skills:delete` grants > - That gap caused valid skill import and mutation requests to be rejected even when the responsible user held the exact direct permission required by the route > - The branch also introduces the repo-sourced `prepare-paperclip-pr` skill so the standard PR preparation process is versioned and reviewable alongside the code > - This pull request adds the missing authorization mappings, covers board, agent, JWT-route, and denial behavior with regression tests, and adds the renamed PR-preparation skill > - The benefit is that governed company-skill workflows honor explicit grants without weakening the responsible-user permission intersection ## Linked Issues or Issue Description No public issue exists. Bug-report shape: - **Affected area**: company skill authorization and skill import routes - **Observed behavior**: agents acting under a responsible user could receive `403` responses for company-skill mutations even when that user had the matching direct `skills:create`, `skills:update`, or `skills:delete` grant - **Expected behavior**: the responsible-user authorization intersection should accept exact company-skill grants while preserving denials for missing or unrelated grants - **Reproduction**: authenticate as an agent with a responsible user, grant that user the relevant company-skill permission, then import or mutate a company skill - **Additional repository change**: adds the renamed `prepare-paperclip-pr` skill as the versioned source of truth for PR preparation Supersedes #9324, which added the PR-preparation skill under the old `prepare-pr` name. ## What Changed - Added `.agents/skills/prepare-paperclip-pr/SKILL.md` with the standard worktree, commit, rebase, guardrail, review-loop, and handoff procedure - Mapped company `skill_config:create`, `skill_config:update`, and `skill_config:delete` actions to direct `skills:create`, `skills:update`, and `skills:delete` responsible-user grants - Preserved restrictive behavior for unsupported resources, missing grants, and unrelated permissions - Added authorization-service regression coverage for board actors and responsible-user agent intersections - Added route-level JWT regression coverage for company skill imports, including allowed and denied cases ## Verification - `pnpm exec vitest run server/src/__tests__/authorization-service.test.ts server/src/__tests__/company-skills-import-authz-routes.test.ts` — 42 tests passed - `pnpm -r typecheck` — passed - `pnpm build` — passed - `pnpm test:run` — server and UI groups passed; one unrelated CLI doctor assertion failed because the execution environment injects static `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`, which intentionally changes the result from `pass` to `warn` - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY NODE_ENV=development pnpm exec vitest run cli/src/__tests__/secrets.test.ts -t 'passes AWS doctor checks when non-secret provider config is present'` — passed, confirming the full-suite failure is environment-specific - GitHub CI — all required checks passed on head `0758393c`; one unrelated `packages/db/src/client.test.ts` 5-second timing timeout passed on the single allowed failed-job rerun after three consecutive local passes (42/42 tests) ## Risks - Low-to-moderate authorization risk: the change expands accepted responsible-user grants only for company-scoped skill configuration actions and is protected by explicit allow/deny regression cases - No database migrations, workflow changes, lockfile changes, or UI changes - The added skill is documentation consumed by agent tooling and does not alter runtime application behavior > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent; exact runtime model ID and context-window size were not exposed to the session. Used reasoning, terminal execution, Git/GitHub tooling, and test/build execution. - Earlier commits were assisted by Claude Fable 5 (`claude-fable-5`) and an OpenAI Codex coding agent, as recorded in the branch history/task workflow. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and documented the one environment-specific full-suite failure - GitHub CI — all required checks passed on head `0758393c`; one unrelated `packages/db/src/client.test.ts` 5-second timing timeout passed on the single allowed failed-job rerun after three consecutive local passes (42/42 tests) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes StubbemannandClaude Fable 5
|
1cfed0c0ff |
security(invites): widen invite-token entropy and rate-limit public invite endpoints (#8979)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Companies onboard human members through shareable invite links; the `/api/invites/:token` endpoints are deliberately public so a recipient can view the invite and accept it without being logged in > - That publicness makes the invite token itself the only secret guarding company membership — and it was guessable: the token suffix carried only ~41 bits of entropy, and the endpoints had no rate limiting > - An attacker could therefore enumerate the token space online and accept an invite into someone else's company, gaining member access to its onboarding data, skills, and workspace > - This pull request widens invite tokens to 256 bits of entropy and puts a per-IP rate limit in front of every public `/invites/:token` sub-route > - The benefit is that invite links stop being brute-forceable while their shape, storage scheme, and UX stay exactly the same — existing links keep working ## Linked Issues or Issue Description No public issue exists; describing the problem in-PR (security/bug): **What happens:** Company invite tokens are **public**: anyone with the link can `GET /api/invites/:token`, fetch onboarding/logo/skills, and `POST /api/invites/:token/accept`. Two weaknesses combined to make them brute-forceable: 1. **Token entropy ~41 bits.** The token suffix was 8 chars over a 36-char alphabet (`8 * log2(36) ≈ 41.4` bits). That is online-enumerable. 2. **No rate limit on `/invites/:token*`.** The public endpoints had no throttling, so the ~41-bit space could be enumerated online. **Impact:** an attacker who guesses a live token can accept the invite and join the company as a member — unauthenticated, from any IP. **Expected:** invite tokens should be computationally infeasible to guess, and the public endpoints should throttle guessing attempts anyway (defense in depth). ## What Changed **Entropy** - `createInviteToken` now uses `crypto.randomBytes(32)` (256 bits) base64url-encoded, keeping the human-readable `pcp_invite_` prefix so link shape and UX are unchanged. The duplicate generator in `plugin-host-services.ts` is updated to match. - Tokens are stored **hashed** (sha256) in `invites.tokenHash`; the raw value is only returned once on creation. Storage scheme is unchanged. - **Backward compatible**: only newly minted tokens are affected; lookup is by hash of the presented value, so existing invite links keep working. **Rate limit** - New generic in-memory per-IP sliding-window limiter (`server/src/services/invite-rate-limit.ts`, 20 req/min/IP), applied as a router-level middleware on `/invites/:token` so every current and future sub-route is covered (summary, logo, onboarding, onboarding.txt, skills/index, skills/:name, test-resolution, and POST accept). - Returns `429` with `Retry-After` and `X-RateLimit-*` headers. In-memory ⇒ per-process, which bounds enumeration per replica. Mirrors the existing `company-search-rate-limit` pattern; no new dependency. - Adds a `tooManyRequests(429)` error helper in `server/src/errors.ts`. ## Verification - `invite-token-entropy.test.ts`: prefix preserved, suffix ≥ 128 bits / 22 chars, charset, 1000 unique tokens. - `invite-rate-limit.test.ts`: allows up to limit then 429s with retry-after; per-IP isolation; forgets hits after the window. - `invite-rate-limit-route.test.ts`: `GET /invites/:token` and `POST /invites/:token/accept` return 429 once the per-IP threshold is exceeded. - Manual: create an invite, open the link (works once per token as before), then hammer `GET /api/invites/<token>` >20 times within a minute from one IP → `429` with `Retry-After`. - Server package typechecks clean for all touched files. ## Risks - Low risk. Token change affects only newly minted tokens; existing links resolve via the same sha256-hash lookup. - The limiter is in-memory and per-process: in multi-replica deployments each replica enforces its own 20 req/min/IP budget. That still bounds enumeration (per-replica) and matches the existing `company-search-rate-limit` approach; a shared store can be layered later if needed. - Legitimate users behind a single NAT/proxy IP share the 20 req/min budget for invite endpoints; the invite flow makes only a handful of requests, so headroom is ample. - No DB migration, no API shape change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) — Claude Fable 5 (`claude-fable-5`), extended thinking enabled, agentic tool use (code search, editing, local typecheck) via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Supersedes #8147. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
b4e7ba5143 |
feat(run-logs): durable run-log store via object-storage mirror (#8984)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every agent run streams its stdout/stderr/system output into the run-log store (`server/src/services/run-log-store.ts`), and the run-log API serves those logs back for review and debugging > - The only store implementation is `local_file`: logs live on the server pod's filesystem under `PAPERCLIP_HOME` > - In hardened / ephemeral deployments, `PAPERCLIP_HOME` is an `emptyDir` with no persistent volume, so every pod restart wipes the log files while the DB row still references them — the run-log API then returns "Run log not found" for every completed run after any redeploy > - Run logs are the primary audit/debugging trail for agent work; losing them on routine redeploys undermines trust in the platform > - This pull request adds transparent durability: when `RUN_LOG_S3_BUCKET` is set, the store mirrors each completed log to object storage on `finalize` (same `logRef` key) and falls back to it on `read` when the local file is gone; live append/tail stays on the fast pod-local file > - The benefit is that completed run logs survive pod restarts and redeploys with zero changes for existing deployments (unset bucket = today's behaviour) and zero downstream changes (store id stays `local_file`) ## Linked Issues or Issue Description No existing public issue — inline description following the bug report template: **What happened?** After any server pod restart/redeploy, the run-log API returns "Run log not found" for all previously completed runs. The DB still references the log file, but the file is gone because run logs are written only to the pod-local filesystem. **Expected behavior:** Completed run logs remain readable across pod restarts and redeploys. **Steps to reproduce:** 1. Deploy the server with `PAPERCLIP_HOME` on an `emptyDir` (no persistent volume — common in hardened/ephemeral Kubernetes deployments). 2. Complete an agent run and confirm its log is readable via the run-log API. 3. Restart or redeploy the server pod. 4. Request the same run's log — the API throws "Run log not found". **Paperclip version or commit:** reproducible on current `master`. **Deployment mode:** Kubernetes (server pod without persistent volume). **Agent adapter(s) involved:** Not adapter-specific (core bug). Supersedes #8795. ## What Changed - `server/src/services/run-log-store.ts`: the local-file store becomes a durable store with an optional object-storage mirror - `finalize` mirrors the completed NDJSON log to S3-compatible object storage (keyed by the same `logRef`), best-effort so a failed upload can never break run finalization; upload failures are logged via `console.warn` so operators can detect a persistently broken mirror before a pod roll makes logs unreadable - `read` serves the pod-local file when present and falls back to a ranged object-storage read (with correct `nextOffset`) when the local file is gone - Live `append`/tail stays on the pod-local file — fast path unchanged, no per-chunk PUT - Store id stays `local_file`, so nothing downstream changes (feedback pipeline, read casts, fixtures untouched) - New optional config, all read at store construction: `RUN_LOG_S3_BUCKET`, `RUN_LOG_S3_ENDPOINT`, `RUN_LOG_S3_REGION` (default `us-east-1`), `RUN_LOG_S3_PREFIX` (default `run-logs`), `RUN_LOG_S3_FORCE_PATH_STYLE` (default `true`); credentials via the standard AWS env chain; works with any S3-compatible endpoint - Reuses the existing `createS3StorageProvider`; deliberately independent from `PAPERCLIP_STORAGE_PROVIDER` so enabling durable logs does not redirect workspace/file storage - `server/src/services/run-log-store.test.ts` (new): 7 tests with an in-memory `StorageProvider` mock ## Verification - `npx vitest run src/services/run-log-store.test.ts` in `server/` — 7/7 pass locally: - store id stays `local_file` - live read served from the local file (no S3 round-trip) - `finalize` uploads the completed log to the mirror - read falls back to S3 after a simulated pod roll (local file deleted) - ranged S3 read returns correct slice + `nextOffset` - not-found when neither local nor mirror has the log - local-only safe degrade when no bucket is configured - `npx tsc --noEmit -p server` — clean for the touched files - Manual: set `RUN_LOG_S3_*` against any S3-compatible endpoint (e.g. MinIO), complete a run, delete the local `.ndjson` file, and re-request the log via the run-log API — it is served from the mirror ## Risks - Low risk: with `RUN_LOG_S3_BUCKET` unset (the default), behaviour is byte-for-byte today's local-only store - Mirror upload is best-effort by design — a misconfigured bucket loses durability (not correctness) for affected runs; failures are now surfaced via a `console.warn` per failed upload - No DB migration, no API shape change, no change to the persisted `store`/`logRef` handle format ## Model Used - Claude (Anthropic), model ID `claude-fable-5` (Fable 5), via Claude Code with extended thinking and tool use (code execution, file editing). Original implementation TDD-authored with the same tooling. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
df0e5bd021 |
fix(interactions): don't supersede decision cards on machine-authored comments (#9015)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents and humans coordinate on issue threads, where `request_confirmation` cards capture pending decisions; a genuine human comment on the thread is meant to supersede (cancel) a card. > - Supersession is keyed on `!comment.authorUserId` — the guard assumes only real human comments carry a user id. > - But local-CLI agent heartbeats post comments under user auth, so a machine comment's `authorUserId` is populated **nondeterministically per run** (the same agent resolves as `agent` on one run and `user` on another). > - As a result an agent's own on-thread comment — or a teammate's, from a different run — can carry `authorUserId` and silently expire a pending decision card. A card was observed expiring 7ms after its own automated comment landed, stranding the decision with no live approval path. > - This PR switches the discriminator to a durable, deterministic signal already persisted on every comment — `created_by_run_id` — so only comments with **no run context** (genuine board-UI comments) supersede. > - The benefit: machine-authored comments can never again expire decision cards, while real human supersession is preserved exactly. ## Linked Issues or Issue Description No public GitHub issue — describing the bug in-PR. - **What happened:** A pending `request_confirmation` decision card was expired by an automated, machine-authored comment on the same thread. Supersession is keyed on `!comment.authorUserId`, but local-CLI agent heartbeats post under user auth, so a machine comment's `authorUserId` is set nondeterministically per run. An agent's own comment (or a teammate's, from a different run) can therefore carry a user id and expire a pending card — one was observed expiring 7ms after its own automated comment landed. - **Expected behavior:** Only genuine interactive human (board-UI) comments should supersede pending decision cards. Machine-authored comments must never expire them, regardless of how the adapter's auth resolves. - **Steps to reproduce:** With a pending `request_confirmation` card (`supersedeOnUserComment: true`), post a comment via a local-CLI agent run whose actor resolves to `user`; the card expires with outcome `superseded_by_comment`. - **Deployment mode:** server (self-hosted), reproduced against `master`. Related PRs (same lifecycle area, not duplicates): #6094 (auto-resolve stale `request_confirmation` interactions) and #8799 (expire ask-user questions superseded by comments, merged). ## What Changed - Supersession now fires **only on comments with no run context** (`created_by_run_id` is null), in both paths: - `expireRequestConfirmationsSupersededByComment` (live post path) — early-return when `comment.createdByRunId` is set. - `expireRequestConfirmationsSupersededByHistoricalComments` (repair sweep) — query filters `isNull(created_by_run_id)`. - Mirrors the existing `shouldImplicitlyMoveCommentedIssueToTodo` reopen guard, which already uses run context to solve the same nondeterministic-identity problem. - Adds live + historical regression tests asserting a run-originated comment under user auth does not supersede a pending card. ## Verification - Interactions service suite: **27 tests pass (1 file)**, including the two new regression tests. - CI: all substantive gates green (Build, General tests, serialized server suites, Typecheck, e2e, verify, security-review, policy). - Manual: with a pending card, a comment carrying `created_by_run_id` leaves it `pending`; a comment with null run context still supersedes it. ## Risks - Low risk, narrowly scoped to the supersession discriminator. Human supersession is preserved (comments with no run context still cancel cards); only the machine-authored case is closed. - No schema migration — `created_by_run_id` is already persisted by `addComment`. - Alternatives considered: (a) ignore only the assignee's own run — misses cross-run machine comments; (b) default `supersedeOnUserComment: false` for agent-created cards — would drop the legitimate "human comment redirects → cancel the card" behavior. The run-context guard covers all machine comments while preserving human supersession. ## Model Used Claude Opus 4.8 (Anthropic), model id `claude-opus-4-8`, extended reasoning + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change and contains no internal Paperclip ticket id — **not yet met**; renaming an open PR's branch risks closing this PR, so it's flagged for a maintainer to rename safely (or via the GitHub rename-branch API). - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes — N/A (internal behavior fix, no user-facing docs) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (the only red check is the automated PR-review template gate this revision addresses) - [ ] Greptile is 5/5 with no open P2s — re-review requested after this revision - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com> |
||
|
DottaandPaperclip
|
931eec3fbf |
feat(mcp) [split 4/8]: wire gateway runtime and Smoke Lab (#9559)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Governed MCP access spans contracts, runtime enforcement, adapters, UI surfaces, and operator verification > - The parity reference PR #9534 is too large for effective automated or human review > - The feature therefore needs a linear stack whose individual diffs stay below the 100-file review limit > - This pull request is split 4/8 and focuses on gateway runtime, Smoke Lab, plugins, and server wiring > - The benefit is a standalone, testable review boundary while preserving byte-for-byte parity at the top of the stack ## Linked Issues or Issue Description - Related parity reference: #9534 - Problem: The policy core needs runtime execution, endpoint guards, route registration, heartbeat integration, and adapter MCP injection to become operational. - Proposed solution: Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Alternatives considered: keeping #9534 as one 403-file review, or rewriting the feature to manufacture seams; both were rejected in favor of path extraction plus compile-driven boundary moves. - Roadmap alignment: this advances the existing governed MCP/tool-access work already represented by #9534; it does not introduce a separate roadmap initiative. - Stack position: base branch is `pap10341-split/03-server-tool-access`. - Merge policy: merge bottom-up, in order, only after the complete eight-PR stack has been reviewed and the top-of-stack parity gate remains empty. - Requested review: SecurityEngineer for gateway, endpoint guard, token issuance, and runtime wiring; Greptile on every PR. ## What Changed - Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Keeps this PR below 100 changed files and independently typecheckable. - Preserves the final tree from #9534 when combined with the other seven stack levels. ## Verification - `pnpm typecheck` - Changed server test set — 26 files, 382 tests passed - Affected server adapter tests — 38 tests passed after concrete adapter boundary move - Adapter-utils and Codex focused tests — 76 tests passed ## Risks - Remote endpoint validation, token handling, and runtime supervision are security-sensitive and can fail closed or deny legitimate access if misconfigured. - Stack risk: merging out of order can expose incomplete layers; mitigate by following the documented bottom-up merge policy. - Parity risk: later edits to an intermediate branch can drift from #9534; mitigate by re-running the empty top-of-stack diff before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context window; medium reasoning with repository, shell, Git, GitHub CLI, and code-execution tools enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] Internal references are omitted except the execution-plan link explicitly required for this coordinated split stack - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Stack Coordination - Internal execution plan: [PAP-13874](/PAP/issues/PAP-13874#document-plan) - Parity reference: #9534 - Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563 - Merge bottom-up only after full-stack review and an empty parity diff at #9563. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
