mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-26 18:31:34 +01:00
57a63e44e5b5cdff3e32d854810a5674fd6399a7
3238
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
57a63e44e5 |
fix(installer): require explicit noninteractive consent
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
70e0b05026 |
docs(cli): align update options with implementation
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e386d647d8 |
chore: retrigger PR policy checks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
74f0197bfd |
chore: retrigger PR policy checks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0d966b614a |
fix(release): add npm publish crash diagnostics (#10041)
## Thinking Path > - Paperclip publishes a coordinated set of packages through its release workflows > - Bundled packages use a pinned npm CLI so trusted publishing works consistently > - The canary publisher now crashes deterministically inside npm before useful output reaches the workflow log > - The npm debug log that contains the underlying failure disappears with the hosted runner > - This pull request upgrades the pinned publish CLI and preserves both verbose HTTP activity and npm debug logs on failure > - The benefit is that the plausible HTTP-layer fix ships immediately, while any remaining CI-only failure becomes diagnosable ## Linked Issues or Issue Description - **Problem:** The canary release workflow fails on the first bundled package with `npm error Exit handler never called!` and no preceding diagnostic output. - **Expected behavior:** Bundled packages publish through trusted publishing, or the workflow retains enough npm diagnostics to identify the actual failure. - **Reproduction:** Run the canary release workflow in GitHub Actions; the failure reproduced on both attempts of run 29948506814. - **Version/commit:** Current `master` after #10024 and #10030. - **Deployment mode:** GitHub-hosted release workflow using Node.js 24 and npm trusted publishing. - Related: #10024, #10030. ## What Changed - Bumped the bundled publish CLI from npm 11.16.0 to npm 11.18.0. - Added `--loglevel verbose` to bundled npm publish invocations. - Dumped the last 300 lines of every npm debug log after failed canary or stable publishes, with common registry credential forms redacted. - Updated release assertions to pin npm 11.18.0 and verify verbose logging. ## Verification - `pnpm test:release-registry` — 66 tests passed. - `bash -n scripts/release-lib.sh`. - Parsed `.github/workflows/release.yml` with Python/PyYAML. - Smoke-tested npm log redaction with representative Authorization, `_authToken`, and token environment values. - Smoke-tested npm debug-log redaction against Authorization, `_authToken`, and `npm_token` examples. - `git diff --check origin/master...HEAD`. - The merge-triggered canary workflow remains the live trusted-publishing verification. ## Risks - Low code risk: changes are isolated to the release publisher and its workflow diagnostics. - npm 11.18.0 could expose a different registry/runtime regression; failure-time debug log dumping makes that actionable. - Verbose npm output increases release log volume but does not change package contents or dist-tags; common credential forms are redacted before debug logs are printed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.3 Codex, tool-enabled coding agent with repository and shell execution; context window size is not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3e9214cd6d |
fix(cli): harden managed lifecycle coordination
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b45f2c0b9e |
fix(doctor): ignore empty managed install stores
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aff4e184cb |
fix(cli): ignore update cache in managed install doctor
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d851c2e6c0 |
test(installer): make fixture logging atomic
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3e7fcb5e7b |
test(cli): cover final installer hardening
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a1662d921b |
fix(installer): pin npm registry trust boundary
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f82ecd92bb |
fix(cli): allow cached managed reinstalls
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
512d0dddf6 |
feat(cli): add managed GitHub ref installs
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
148fa819a6 |
chore: retrigger PR policy checks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4eaa0c510a |
fix(cli): avoid guessed restart versions
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7484df71f4 |
feat(cli): add managed update and rollback
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2229aac205 |
fix(cli): preserve installs when service state is unknown
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3eafbb13a3 |
fix(installer): harden dry runs and unit escaping
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
108895421f |
fix(cli): fail closed on install locks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
72f8f84884 |
fix(cli): serialize managed uninstall
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
535b1b7cd9 |
fix(cli): harden managed install store
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
68b7a76211 |
fix(installer): verify third-party bootstrap scripts
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ceee604228 |
fix(cli): make bootstrap options explicit
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ed4eaf6f46 |
fix(cli): harden service lifecycle edge cases
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c18d1d378f |
fix(cli): align installer flags and restart health
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5aa6080ca8 |
chore: retrigger PR policy checks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
256629a414 |
chore(cli): clean installer smoke script whitespace
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4410c4a9c1 |
fix(cli): harden managed install filesystem writes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6c8d141f0b |
fix(cli): harden install integration checks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fb260f0ed7 |
feat(cli): integrate managed install workflows
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
550a0fe29c |
feat(cli): add managed npm install store
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ac9688f7c6 |
docs: document installer raw fallback
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a942c7b3e7 |
feat(cli): add managed service lifecycle
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
95656189d5 |
feat(cli): add curl bootstrap installer
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
8af70b9fae |
fix(test): drain in-flight heartbeat runs before liveness teardown (#10040)
## Thinking Path > - Paperclip runs AI agent heartbeats to manage work; each heartbeat dispatches `executeRun` fire-and-forget, which is intentional for concurrency > - The server escalation test suite (`heartbeat-issue-liveness-escalation.test.ts`) exercises `reconcileIssueGraphLiveness`, which heals a resolved-dependency wake by enqueuing an on-demand heartbeat run > - `enqueueWakeup` → `startNextQueuedRunForAgent` dispatches the run fire-and-forget (`void executeRun(...)`), so the background run outlives the awaited reconcile call > - The test's `afterEach` polled `heartbeat_runs.status` to wait for idle, but that flips to `completed` while `executeRun`'s finally block is still flushing events — the escaping `heartbeat_run_events` insert could land between the events delete and the runs delete, tripping the FK constraint > - This PR fixes the race deterministically by tracking in-flight `executeRun` promises and exposing `heartbeatService.drainActiveRunExecutions()`, which the suite awaits before clearing tables > - The benefit is a permanently reliable escalation test suite with no sleeps, no retry bumps, and no production behavior change ## Linked Issues or Issue Description **What happened?** The `heartbeat-issue-liveness-escalation.test.ts` suite intermittently failed in CI with: ``` delete on table "heartbeat_runs" violates foreign key constraint "heartbeat_run_events_run_id_heartbeat_runs_id_fk" ``` **Expected behavior** `afterEach` cleanup should complete without FK violations. **Steps to reproduce** The race is timing-dependent but surfaces reliably when the teardown window is artificially widened. `reconcileIssueGraphLiveness()` heals resolved-dependency wakes by dispatching a heartbeat run fire-and-forget (`void executeRun(...)`). The old `afterEach` polled `heartbeat_runs.status` — but that flips to `completed` while `executeRun`'s finally block still has pending `heartbeat_run_events` row writes. The escaping insert can land between the events delete and the runs delete. **Paperclip version or commit** Reproducible on current `master` (commit `b57aa9950c707a024156c34b79326a82b2dcca31`) ## What Changed - **`server/src/services/heartbeat.ts`** — tracks all in-flight `executeRun` promises in a module-level `Set`; exposes `heartbeatService(db).drainActiveRunExecutions()`, which loops until the set drains (a completing run can enqueue the next queued run in its finally, so a single `await` is not enough) - **`server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts`** — replaces the poll-on-`heartbeat_runs.status` teardown with `await heartbeatService(db).drainActiveRunExecutions()` before clearing tables; removes the now-unnecessary `waitForHeartbeatRunToComplete` helper ## Verification ```bash # Full file (22 tests) npx vitest run server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts # 12x stress loop (264 test-runs, 0 failures) for i in $(seq 1 12); do npx vitest run server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts || break done # Type check the changed files npx tsc --noEmit ``` - 22/22 tests green locally - 12/12 full-file loop iterations: 264 test-runs / 264 afterEach cycles, 0 failures - Widened-teardown stress variant (failed deterministically before the fix) now passes with the drain ## Risks Low risk. The drain mechanism is additive — it only affects test teardown and could also be wired into graceful shutdown. The fire-and-forget dispatch in production is unchanged. The `Set`-based tracking adds negligible overhead per run dispatch (insert on dispatch, delete on completion). ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200K tokens - **Mode:** Tool use, code execution, extended reasoning ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b57aa9950c |
fix(test): stop flaky server-suite afterAll hook timeouts (#10024)
## Thinking Path > - Paperclip is an open-source AI agent management platform; its test suite spans a `server` package that mounts real embedded Postgres databases in `beforeAll`/`afterAll` hooks > - The `server` package CI shard runs all ~93 suites serially (`maxWorkers=1`) on a loaded CI host; each suite boots and tears down its own embedded Postgres in hook callbacks > - vitest's default `hookTimeout` is 10 seconds; under load, graceful embedded-Postgres shutdown occasionally crosses that threshold > - This produces intermittent `Error: Hook timed out in 10000ms` failures in `afterAll` hooks — not test assertion failures — and the suites pass on re-run, making them textbook flaky tests > - Inspecting `embedded-postgres@18.1.0-beta.16` shows that `stop()` takes no argument (no fast-shutdown mode), SIGINTs postgres (already PostgreSQL "fast shutdown"), and resolves only on the child's `exit` event with no internal time bound > - Two targeted fixes: (1) raise `hookTimeout` and `teardownTimeout` to 30 s in `server/vitest.config.ts` — one config change that eliminates the flake for all ~93 suites at once; (2) wrap `stop()` in a 5 s bounded `Promise.race` in the test helper so a slow shutdown can never hang the hook regardless of OS scheduling variance > - This PR changes only test-infra and test-config; no production-code behavior changes ## Linked Issues or Issue Description No public GitHub issue exists for this flake. Inline bug description (bug report template): **What happened?** The `General tests (server (N/3))` CI shards intermittently fail with `Error: Hook timed out in 10000ms` in `afterAll` hooks and pass on re-run. Every test assertion passes; only the teardown hook exceeds vitest's default timeout. **Expected behavior** CI passes reliably. Teardown timeouts should not be a source of flake. **Steps to reproduce** Run the server test suite repeatedly on a loaded host or in CI with `maxWorkers=1` — the shard occasionally crosses 10 s in `afterAll` during embedded-Postgres shutdown. **Paperclip version** `master`, any build that includes `server/vitest.config.ts` without an explicit `hookTimeout`. **Deployment mode** Self-hosted (CI). ## What Changed - **`server/vitest.config.ts`** — added `hookTimeout: 30000` and `teardownTimeout: 30000`. Removes flake across all ~93 server suites at once. 30 s gives generous headroom over observed worst-case teardown while still catching a genuinely hung hook. - **`packages/db/src/test-embedded-postgres.ts`** — added `stopEmbeddedPostgresBounded()`, a 5 s `Promise.race` wrapper around `stop()`. Applied at all three call sites inside `cleanup()`. Data dir is still removed unconditionally; errors are still swallowed; the null-instance guard is preserved. Existing behavior unchanged except the shutdown can no longer block indefinitely. ## Verification - `tsc --noEmit` clean on `packages/db` (built against worktree-local `shared`) - `packages/db` `client.test.ts` passes 14/14 — boots embedded Postgres and exercises the bounded teardown via `cleanup()` in `afterEach` - Standalone bounded-race semantics verified: hang resolves at the 5 s bound; late or immediate `stop()` rejection swallowed; no unhandled rejection; null-instance path safe - CI: all 3 server shards + split-verify lane (Async-Verification Gate) expected green after this PR ```bash # Reproduce the teardown test locally: cd packages/db && npx vitest run src/client.test.ts # Type-check packages/db: npx tsc --noEmit -p packages/db/tsconfig.json ``` ## Risks Low risk. No product-code changes — test-infra and test-config only. The vitest timeout increase is additive (raises the ceiling; never lowers it). The bounded race wrapper preserves prior teardown behavior exactly: data dir always removed, errors always swallowed, stop is still attempted. A worst-case outcome is that a genuinely hung `stop()` now surfaces as a test timeout at 30 s instead of 10 s — still caught, just later. ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200 k tokens - **Capabilities:** tool use, code execution, extended reasoning - **Mode:** Paperclip agent heartbeat (autonomous execution with human board oversight) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
47c38777d8 |
fix(release): use trusted publishing npm for bundled packages (#10030)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip publishes canary and stable packages through a shared release script > - GitHub Actions authenticates those publishes through npm trusted publishing and an OIDC identity token > - Bundled-dependency packages recently moved from pnpm publish to a pinned npm CLI to preserve their bundled files > - That pin selected npm 10, which cannot use trusted publishing, so the first bundled package failed with `ENEEDAUTH` > - This pull request keeps bundled-package packing on npm 10 while routing actual publishing through the trusted-publishing-capable npm 11 version > - The benefit is bundled packages keep their required npm packaging behavior while canary and stable releases authenticate successfully ## Linked Issues or Issue Description **What happened** The canary release job failed while publishing `@paperclipai/adapter-utils` with `ENEEDAUTH`. The package has bundled dependencies, so the release helper selected pinned `npm@10.9.7`; the workflow provides OIDC trusted publishing rather than an npm token, and npm 10 cannot use that authentication path. Because this is the first package attempted, the release exited before trying the remaining packages. **Expected behavior** Bundled-dependency packages should publish with an npm CLI that both preserves bundled dependencies and supports GitHub Actions trusted publishing. **Steps to reproduce** Run the canary release workflow from master after PR #9980. The `publish_canary` job reaches `@paperclipai/adapter-utils`, invokes `npx npm@10.9.7 publish`, and fails with `ENEEDAUTH`. **Deployment mode** GitHub Actions canary and stable npm release workflows. Refs #9980. ## What Changed - Kept bundled-package dry-run packing on npm `10.9.7`, which successfully produces the staged tarball. - Routed bundled-package publishing through npm `11.16.0`, which supports GitHub Actions trusted publishing. - Split the pack and publish helpers so future npm changes cannot silently couple the two compatibility requirements. - Updated focused release and ACPX packaging tests to enforce both versions and call paths. ## Verification - `pnpm test:release-registry` — 67 passed locally. - Initial all-npm-11 PR head: Canary Dry Run reproduced an npm-internal crash during bundled `pack`. - Current head `5b3961ed13dd26ed2d6b1096ea23fd91b32e4353`: Canary Dry Run passed with split npm pack/publish helpers. - All PR checks passed, including build, typecheck + release registry, server/workspace suites, both e2e shards, security gates, and Greptile. - Greptile reviewed the current head at 5/5 confidence with no blocking issues. ## Risks - Low risk: the change only separates the npm CLI used for bundled-package packing from the CLI used for publishing. - The versions remain explicitly pinned because npm 11.16.0 currently crashes on the bundled pack payload, while npm 10.9.7 cannot perform trusted publishing. - Focused tests assert both pins and both helper call paths, and the full Canary Dry Run passes on the current head. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with repository, shell, GitHub CLI, and code-execution tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
e1e35881af |
feat(cost-events): propagate issue.billing_code at heartbeat record time (#6821)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs report progress through the heartbeat service, which writes the cost ledger (`cost_events`) as usage accrues > - `cost_events` already has a `billing_code` column, but nothing populates it — the heartbeat writes `issueId`/`projectId` and leaves `billing_code` NULL > - Issues carry a `billing_code`, so the attribution data sits one join away but never reaches the ledger rows > - Reporting therefore has to reconstruct attribution by joining back to `issues` at query time, which reflects the issue's *current* billing code rather than the one in effect when the cost was incurred > - This pull request threads `billingCode` through `resolveLedgerScopeForRun` so the heartbeat stamps it onto each `cost_events` row at record time > - The benefit is that attribution is captured at write time and stays correct if an issue's billing code later changes ## Linked Issues or Issue Description No existing public GitHub issue. Describing the problem in-PR: **Problem.** `cost_events` has a `billing_code` column that is never written. The heartbeat's cost-ledger insert records `issueId` and `projectId` but not the billing code of the issue the run belongs to, so every row lands with `billing_code` NULL. **Impact.** Cost-per-billing-code reporting has to derive attribution by joining `cost_events` back to `issues` at query time. That join returns the issue's billing code *as of the query*, not as of when the cost was incurred, so historical cost reports shift retroactively whenever an issue is re-coded. **Desired behaviour.** The billing code in effect at record time is stored on the `cost_events` row itself. **Related PRs.** #6820 — same change to the same file by the same author, opened separately. These are duplicates; only one should land. ## What Changed - `resolveLedgerScopeForRun` now selects `issues.billingCode` alongside `id` and `projectId`. - The scope object it returns gained a `billingCode` field, populated with `issue?.billingCode ?? null`. - The early-return path for runs with no issue in context returns `billingCode: null`. - The `costs.createEvent` call in `heartbeatService` passes `billingCode: ledgerScope.billingCode` alongside `issueId`/`projectId`. No schema migration: `cost_events.billing_code` already exists. ## Verification **No automated test accompanies this change.** There is currently no test asserting that a `cost_events` row carries the issue's billing code when an issue is in scope, or `null` when there is not. A reviewer should treat the checks below as manual verification only. Manual verification against a running instance: ```sql -- Non-NULL billing_code for recent runs on billed issues SELECT billing_code, COUNT(*) FROM cost_events WHERE created_at > NOW() - INTERVAL '1 hour' GROUP BY billing_code; -- Cost attribution query this change is intended to enable SELECT billing_code, SUM(cost_cents) FROM cost_events GROUP BY billing_code; ``` Expected: rows for runs attached to an issue with a billing code now carry that code; runs with no issue in context remain NULL. ## Risks Low risk in blast radius, with two things worth a reviewer's attention: - **Behavioural shift for consumers.** `cost_events.billing_code` was uniformly NULL and now starts arriving populated. Anything downstream that groups, filters, or dedupes on that column will see new values and new cardinality. Existing rows are not backfilled, so the column is mixed NULL/non-NULL across the historical boundary. - **No test coverage.** The null-fallback behaviour on both paths is asserted only by reading the code, not by a test. - **Migration safety:** not applicable — no schema change; the column already exists. - **Failure mode:** if `billingCode` were absent from the `issues` selection the value would silently be `undefined` rather than erroring, so the field is worth confirming in review. ## Model Used **TODO (author):** this section is required and cannot be completed on your behalf. Please state the provider and model name, the exact model ID/version, and the reasoning/thinking mode used — or "None — human-authored" if no AI model was involved. Per the template, the "Generated with Claude Code" footer is not a substitute for this section. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [ ] I have specified the model used (with version and capability details) — **pending author input, see above** - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above — #6820 is a duplicate of this PR - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable — **no test added for the new field** - [x] I have updated relevant documentation to reflect my changes — not applicable, no user-facing or documented behaviour changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green — **`e2e` did not complete on `5ca5fde` (Playwright install timed out at 30m and the run was cancelled); all other checks pass** - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — **currently 4/5, sole finding being this description** - [ ] I will address all Greptile and reviewer comments before requesting merge --- <sub>This description was reformatted to `.github/PULL_REQUEST_TEMPLATE.md` by the Paperclip PR triage bot. The code was not modified. Checklist boxes reflect the PR's verifiable state at commit `5ca5fde`; unchecked items are genuinely outstanding, not oversights. The **Model Used** section requires input from the author. The previous description's `LEG-` reference was removed as an internal, instance-local identifier that the template prohibits.</sub> --------- Co-authored-by: Lead Backend Engineer Agent <backend1@legacykeeper.io> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com> |
||
|
|
b247bf7150 |
feat(runtime): opt-in sandbox file-sync lifecycle hooks (API + provider docs) (#10013)
## Thinking Path > - Paperclip is an open-source AI-agent management platform; agents run tasks inside sandboxed environments (Daytona, Kubernetes, E2B, etc.) > - The control-plane ↔ sandbox file-transfer path flows through the `environmentExecute` seam in `protocol.ts` — the only verb available to plugins — which forces a base64-over-exec chunked loop for every file move: workspace files, assets, Codex home sync > - This transport is correct and safe, but it bypasses provider-native bulk/streaming APIs (Daytona `uploadFiles`, K8s `FastUploadInterceptor` / volume mounts), leaving significant throughput on the table for large workspaces > - The right fix is an opt-in seam extension: providers with faster native transfer declare two optional verbs; providers that do not opt in stay on the existing fallback with zero code or behavior change required > - This PR adds the first layer of that extension — two optional verbs (`environmentSyncIn` / `environmentSyncOut`) in the plugin SDK, the runtime plumbing to prefer the native path for the two clean destroy-then-replace cases, and a doc for the contract > - The core correctness invariant is byte-identical fallback: if no provider opts in, execution is exactly what ships today; `assertSyncOperationsConfined` enforces host-side path confinement for providers that do opt in > - No provider advertises the verbs yet → zero production behavior change; future PRs wire up Daytona and K8s providers against this contract ## Linked Issues or Issue Description No public GitHub issue exists for this feature. Description follows the `feature_request` issue template: **Subsystem affected:** packages/plugins — plugin system; packages/adapter-utils — adapter runtime; server/ — EnvironmentRuntimeService **Problem or motivation:** Sandbox file transfers currently always use a base64-over-exec chunked loop regardless of what the underlying provider supports. For workspaces larger than a few MB this becomes the dominant wall-clock cost of every sandbox run, and it bypasses bulk/stream APIs that providers like Daytona already expose natively. **Proposed solution:** Add two optional, opt-in plugin hooks — `onEnvironmentSyncIn` / `onEnvironmentSyncOut` — to the plugin SDK. When a provider defines both hooks and both are advertised via the existing `supportedMethods` negotiation, the runtime prefers the native path for the two clean destroy-then-replace transfer cases; all other cases fall back to the existing byte-identical base64 transport. **Alternatives considered:** An unconditional verb would require every provider to implement or stub the verb. The opt-in / `METHOD_NOT_IMPLEMENTED` pattern (already used by `environmentExecute`) preserves backward compatibility with zero provider changes required. **Roadmap alignment:** Consistent with the ✅ "Cloud / Sandbox agents" and ✅ "Plugin system" milestones; extends the plugin seam rather than adding control-plane-level logic. **Additional context:** Searched open pull requests and issues for duplicate sandbox file-sync / native-transfer work; none found. ## What Changed - **`packages/plugins/sdk`** - `protocol.ts`: two new optional `HostToWorkerMethods` — `environmentSyncIn` / `environmentSyncOut` — plus generic `SyncOperation`, `SyncFileMapping`, and `SyncOutcome` types - `define-plugin.ts`: optional `onEnvironmentSyncIn` / `onEnvironmentSyncOut` fields on `PluginDefinition`; worker advertises each verb only when its hook is defined (else `METHOD_NOT_IMPLEMENTED`, mirroring `environmentExecute`) - `worker-rpc-host.ts`: route new verbs to plugin hooks - `index.ts`: re-export new public types - **`packages/adapter-utils`** - `command-managed-runtime.ts`: expose optional `syncIn` / `syncOut` on `CommandManagedRuntimeRunner` (available only when both verbs are advertised); add `assertSyncOperationsConfined` host-side path-confinement guard - `sandbox-managed-runtime.ts`: `SandboxManagedRuntimeClient` gains optional `syncIn` / `syncOut`; orchestrator prefers native path for default-provision asset inbound and workspace-download-into-fresh-dir outbound; all other paths keep the existing base64 fallback - `sandbox-file-sync.test.ts` (new): 234-line characterization suite — native-opt-in branch, fallback branch, `assertSyncOperationsConfined` escape-path rejection, `followSymlinks` → tar `-h` - `command-managed-runtime.test.ts`: negotiation + native-sync + confinement tests - **`server/src/services/environment-runtime.ts`**: `EnvironmentRuntimeService` delegates to `syncIn` / `syncOut`, gated on advertised support - **`server/src/services/environment-execution-target.ts`**: minor typing fix alongside the new verbs - **`doc/plugins/SANDBOX_FILE_SYNC_HOOKS.md`** (new): documents the full contract — opt-in / no-op guarantee, operation ordering, provider-may-tar, atomicity, `followSymlinks`, secret modes (0600, no window), path confinement, `operationId` opacity, resource bounds, shell-quoting ## Verification ```bash # SDK suite pnpm --filter packages/plugins/sdk test # Adapter-utils suite (includes new sandbox-file-sync characterization tests) pnpm --filter packages/adapter-utils test # Expected: 255 pass / 4 skip # Type-check across affected packages pnpm --filter packages/plugins/sdk typecheck pnpm --filter packages/adapter-utils typecheck # Server changed-file spot check: cd server && npx tsc --noEmit --skipLibCheck 2>&1 | grep -E "environment-(runtime|execution-target)" | head -20 ``` Key behavioral invariant to spot-check: with no provider opting in (the current state), run any sandbox task and confirm file-transfer behavior is byte-for-byte identical to what the pre-PR code produces. The characterization tests assert this at the unit level. ## Risks - **Zero production risk today**: no provider advertises `environmentSyncIn` / `environmentSyncOut`, so the new code paths are unreachable in production; all real traffic stays on the existing base64 fallback - **Path confinement**: `assertSyncOperationsConfined` rejects any `targetPath` that escapes the declared root — this is the primary security boundary for future providers. The test suite covers escape-path rejection - **Atomicity**: the contract delegates atomicity to providers; the doc explicitly calls out that directory-level ops are not guaranteed atomic - **Secret transport**: credential assets (e.g., Codex `auth.json`, directory mappings) continue to use the existing tar path — they do not go through the new verbs in any current provider > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Provider: Anthropic Model: `claude-sonnet-4-6` (Claude Sonnet 4.6) Context window: 200 K tokens Capabilities: extended tool use, multi-file code generation, agentic reasoning via the Paperclip agent framework ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d54ff52fc3 |
test(heartbeat): await execution drain before cleanup (#10023)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI agents and their work > - Heartbeat scheduling tests protect the orchestration rules that serialize an agent's runs > - The dependency scheduling suite waits for run rows to become terminal before deleting shared database fixtures > - A terminal row is persisted before asynchronous execution finalization and successful-run handoff work fully drain > - The test then clears process tracking and deletes heartbeat events while finalization can still append another event > - This pull request waits for each tracked run's execution promise to drain before resetting mocks or deleting fixtures > - The benefit is deterministic cleanup that preserves the production lifecycle ordering and prevents release CI flakes ## Linked Issues or Issue Description ### What happened? Release run `29936031931` failed in `heartbeat-dependency-scheduling.test.ts` while deleting `heartbeat_runs`. Asynchronous heartbeat finalization inserted a new `heartbeat_run_events` row after the test had already deleted existing events, causing the run-row delete to violate the event foreign key. ### Expected behavior The serialized heartbeat test suite should finish all asynchronous run execution work before destructive fixture cleanup. ### Steps to reproduce 1. Check out commit `2aef4641b48e88f5ce7e75ce69fbe3bf6bbfc60d`. 2. Run `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/heartbeat-dependency-scheduling.test.ts --pool=forks --isolate` repeatedly with PostgreSQL test support enabled. 3. Observe that teardown can delete heartbeat events while execution finalization is still able to append another event, causing a foreign-key failure when heartbeat runs are deleted. ### Paperclip version or commit `2aef4641b48e88f5ce7e75ce69fbe3bf6bbfc60d` ### Deployment mode Other — GitHub Actions release verification. ### Installation method Built from source with pnpm. ### Agent adapter(s) involved Not adapter-specific (core heartbeat test lifecycle). ### Database mode External PostgreSQL test database. ### Relevant logs or output `delete from "heartbeat_runs"` failed because the run remained referenced by `heartbeat_run_events_run_id_heartbeat_runs_id_fk`. ## What Changed - Collect heartbeat run IDs after queued/running rows settle and await `heartbeat.waitForRunExecutionDrain()` for each run. - Reset the adapter mock and clear process tracking only after asynchronous heartbeat finalization has completed. ## Verification - Ran `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/heartbeat-dependency-scheduling.test.ts --pool=forks --isolate` 10 consecutive times; all 10 runs passed with 6/6 tests. ## Risks - Low risk: test-only cleanup ordering change using an existing heartbeat service drain API. Production behavior is unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with exact model IDs `gpt-5.5` for this heartbeat and `gpt-5.6-sol` for the recovered initial implementation run; tool-enabled code inspection, GitHub diagnostics, and shell test execution. Runtime context-window sizes were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
2aef4641b4 |
docs(release): add v2026.722.0 changelog (#10008)
## Release changelog: Paperclip v2026.722.0 Adds `releases/v2026.722.0.md`, the user-facing stable changelog for the `v2026.722.0` release, generated per `.agents/skills/release-changelog/SKILL.md`. - **Range:** `v2026.720.0..origin/master` — 44 commits from 13 contributors. - **Version:** confirmed via `scripts/release.sh stable --date 2026-07-22 --print-version` → `2026.722.0`. - **Breaking changes:** none. Two additive DB migrations (Connections v3 schema core + connection user authorization state); no destructive migrations. - **Sections:** Highlights (run-bound agent secret access; Windows local-agent spawning), Improvements, Experimental (Connections v3 foundation, gated behind the Apps experimental setting), Fixes, Upgrade Guide, Contributors. Contributor count/names follow the skill rules (bots and Paperclip founders excluded from the listed names but counted in the total). Related issue: PAP-14983. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
github-actions[bot]andlockfile-bot
|
486ecac5e9 |
chore(lockfile): refresh pnpm-lock.yaml (#9964)
Auto-generated lockfile refresh after dependencies changed on master. This PR only updates pnpm-lock.yaml. Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
d31a28828b |
fix(acpx): support Windows agent spawning (#9980)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Local Claude, Codex, Gemini, and custom ACP adapters run through the shared embedded ACPX engine > - That engine wrapped every local agent command in a generated Bash script to inject environment variables and filter child stderr > - Windows cannot directly spawn that Bash wrapper, and npm/pnpm ACP binaries are exposed through `.cmd` shims there > - ACPX 0.12 already supports per-session child environment variables, so the wrapper is unnecessary > - This pull request registers agent commands directly, injects env through ACPX session options, captures child stderr in-process, and adds a real Node ACP spawn smoke on Ubuntu and Windows > - The benefit is one cross-platform spawn path with a reusable smoke test instead of parallel shell-wrapper implementations ## Linked Issues or Issue Description Fixes #9941. Refs #9428 and #9771. **What happened** ACPX-backed local agents failed to start on Windows because Paperclip registered a generated POSIX `.sh` wrapper as the agent command. Windows also needs the `.cmd` npm/pnpm shim when resolving built-in ACP binaries, and symlink creation can fail with `EPERM` for seeded auth/skill files. **Expected behavior** The same ACPX engine path should spawn a real ACP agent on Windows and Linux, forward Paperclip/runtime env without mutating `process.env`, preserve filtered/unfiltered child stderr behavior, and fall back to copies where Windows symlinks are unavailable. **Steps to reproduce** Run a local ACPX adapter on Windows with the prior wrapper path. ACPX attempts to spawn the generated `.sh` file and the agent never initializes. **Deployment mode** Local Paperclip adapters using `packages/adapter-utils/src/acpx-engine/`. ## What Changed - Removed generated Bash agent/env wrappers and registered local commands directly with ACPX. - Passed the resolved child environment through ACPX `sessionOptions.env`, including resume retry paths. - Added a minimal `acpx@0.12.0` package patch exposing child stderr callbacks and allowing documented uppercase env-map keys in persisted session options. - Moved stderr tee/filter behavior in-process: raw stderr remains in the per-run file while benign `nes/close` noise is omitted from live stderr. - Preferred `.cmd` ancestor binaries on Windows and added `EPERM` copy fallbacks for Codex auth seeding and Gemini skill materialization. - Added a real Node ACP echo-agent spawn smoke that can run directly on any supported platform. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts packages/adapter-utils/src/acpx-engine/spawn-smoke.test.ts` — 57 passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `node --test scripts/acpx-patch-packaging.test.mjs scripts/release-lib.test.mjs` — 10 passed. - Full canary release dry run under Node 24.18.0 / npm 11.16.0 — passed in an isolated scratch clone. - `git diff --check` — passed during implementation verification. - One-time GitHub Actions proof: [Ubuntu ACPX spawn smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774579), [Windows ACPX spawn smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774558), and [Canary Dry Run](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774497) passed on head `f345ac69f2`; the dedicated smoke jobs are intentionally not retained in the recurring PR workflow. ## Risks - The ACPX stderr callback and env persistence exemption are carried as a pnpm dependency patch until ACPX exposes/fixes those behaviors upstream. - Child stderr is synchronously appended to preserve ordering and failure diagnostics; unusually high-volume agent stderr could briefly block the Node event loop. - The Windows-specific `.cmd` resolution and symlink `EPERM` branches are proven by the standalone smoke test and the linked one-time `windows-latest` run rather than a permanent CI gate. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5.4 via Codex CLI, medium reasoning, repository/tool execution enabled; context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
0b496c9c03 |
feat(secrets): add run-bound agent secret access (#9921)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Agents already receive selected company secrets through `env.*` bindings at run launch, but environment injection is ambient, long-lived, and not suitable for every secret consumer. > - The existing binding and secret-access-event models already provide company-scoped authorization and per-resolution audit seams. > - Agents need an explicit way to discover only the secrets granted to them and fetch a value on demand without exposing the wider company catalog. > - That capability must remain run-bound, preserve low-trust token carve-outs, and make every value read visible in both security and operator audit trails. > - This pull request adds an `access.*` delivery namespace, two run-bound agent routes, dual audit logging, documentation, and an operator grants editor. > - The benefit is least-privilege, revocable, auditable secret access while preserving existing env injection behavior. ## Linked Issues or Issue Description No pre-existing public issue. Related work: - Refs #9797 — existing in-sheet agent access UI that this PR extends to distinguish env and API delivery. - Refs #9918 — complementary searchable-agent picker improvement for the same secrets sheet. - Refs #9530 — related company-wide metadata catalog proposal; this PR intentionally exposes only the authenticated run's granted aliases and values. **Problem / motivation:** Agents can currently consume secrets only through process environment injection. This keeps values resident for the run, does not support on-demand consumers, and cannot provide a discrete operator-visible activity event for each agent-initiated read. **Proposed solution:** Treat `company_secret_bindings` as the source of truth for agent secret grants. Keep `env.KEY` as env delivery and add `access.ALIAS` for API-only delivery; an env binding also implies read access because the value is already present in the agent process. Add run-bound list/fetch endpoints that derive scope from the authenticated heartbeat run and never accept caller-selected overlays. **Alternatives considered:** A company-wide agent-readable catalog was rejected for this value path because it increases reconnaissance and does not prove a per-secret grant. Reusing the ephemeral environment-probe resolver was rejected because it lacks binding enforcement. Approval-gated reads and user-scoped secrets remain deferred beyond v1. **Roadmap alignment:** This extends the completed **Secrets Manager with per-agent access** roadmap capability from launch-time env injection to explicit run-bound API delivery without duplicating a separate planned initiative. ## What Changed - Added `access.*` agent binding validation and a dedicated run-bound resolver that combines `secrets:read` authorization with binding-context enforcement. - Added `GET /api/agents/me/secrets` for minimal granted metadata and `POST /api/agents/me/secrets/:key/value` for on-demand value fetches with `Cache-Control: no-store`. - Preserved the existing denials for low-trust review agents, task-bridge credentials, and skill-test tokens; standard long-lived agent API keys cannot call the run-bound routes. - Added dual audit behavior: value attempts write `secret_access_events` and `activity_log` (`secret.value.read`), while metadata listing writes the lighter `secret.access.listed` activity event. - Kept env compatibility: `env.*` remains injected at launch and also implies API read for the same bound agent; `access.*` never becomes an environment variable. - Added the agent-settings **Secret access** editor plus delivery-mode/alias surfacing on the Secrets page, with focused UI tests and tokenized layout styles. - Updated OpenAPI, shared types, agent-facing skill documentation, and API reference documentation. ### UI Screenshots P3 produced and reviewed three screenshots using mock data; images are intentionally not committed to the repository: - `secret-access-editor.png` — agent settings grant editor. - `secret-access-light.png` — Secrets-page delivery surfacing in light mode. - `secret-access-dark.png` — Secrets-page delivery surfacing in dark mode. The source attachments are retained with the implementation task and linked in the internal handoff; the public page publisher was unavailable in the PR-prep runtime. ## Verification - `pnpm exec vitest run server/src/__tests__/agent-secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts server/src/__tests__/secrets-routes.test.ts ui/src/lib/secret-delivery.test.ts ui/src/components/AgentSecretAccessEditor.test.tsx` — 5 files, 122 tests passed. - Security follow-up: `pnpm exec vitest run server/src/__tests__/agent-secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts` — 2 files, 73 tests passed after active-run and version-consistency fixes. - Final-head CI: all feature, typecheck, build, e2e, security, and review gates pass; `General tests (server (1/3))` remains red after one rerun because unrelated `heartbeat-retry-scheduling.test.ts` cleanup deletes `heartbeat_runs` before referenced `activity_log` rows. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm check:token-gates` — feature-local arbitrary-value violations fixed; command still reports five unchanged `#9627` literals outside this PR. - End-to-end QA passed all eight acceptance criteria: grant/list, fetch, dual audit, env-implies-read, denial matrix, revocation, UI rendering, and env-injection regression. Evidence: https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455492 - Security review returned PASS-with-required-changes; the implementation uses the required dedicated binding-enforcing resolver, run-bound JWT restriction, run-derived overlays, minimal metadata, and a resolver redaction-registration hook. Evidence: https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455382 ## Risks - A compromised agent can exfiltrate any secret explicitly granted to it; explicit company-scoped/run-scoped grants, revocation, and audit reduce but cannot remove that inherent capability risk. - The resolver invokes a redaction-registration hook before returning values, but the current route has no persistent cross-request per-run redaction registry. Paperclip-owned later comments/events therefore cannot yet guarantee automatic scrubbing of a deliberately copied fetched value; QA classified this as non-blocking residual hardening. - Audit-event insertion currently fails open if the security-event insert itself fails; the operator activity event provides partial redundancy, but a future hardening change should define fail-closed behavior for value delivery. - This PR overlaps `ui/src/pages/Secrets.tsx` with #9918 and may require a straightforward rebase after that PR moves. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.3-codex`, with reasoning, repository tool use, terminal execution, Paperclip API access, and GitHub CLI capabilities. Context-window size is not exposed by the runtime. - Anthropic Claude Opus 4.8 with 1M context and tool use assisted with the UI implementation commit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>canary/v2026.722.0-canary.1 |
||
|
|
5ed0b74b34 |
fix(runtime): scope PAPERCLIP_ env-binding strip to reserved keys (#9974)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs get their environment from user/adapter/project/routine env bindings resolved by the server heartbeat, plus `PAPERCLIP_*` runtime vars (identity, wake, workspace, API access) injected by the harness > - The heartbeat stripped **every** `PAPERCLIP_`-prefixed binding before resolution, so legitimately user-named keys (e.g. cloud provider token bindings like `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_*`) were silently dropped and never reached the run env > - At the same time, several adapters honored an explicitly configured `PAPERCLIP_API_KEY` over the harness-minted run token, which is exactly the one key config must never control > - This pull request replaces the blanket prefix strip with a precise three-rule policy: never accept `PAPERCLIP_API_KEY` from config, always let harness-assigned runtime vars win, and let every other `PAPERCLIP_*`-named user binding flow through > - The benefit is that user secrets with a `PAPERCLIP_`-style name work like any other binding, while runtime identity and API credentials stay fully harness-controlled ## Linked Issues or Issue Description **Bug description** (no public issue exists): - **What happened:** Env bindings whose key starts with `PAPERCLIP_` (e.g. a cloud provider token a user deliberately named `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_TOKEN`) were silently stripped by the server before secret resolution, so the spawned agent never received them. No error, no access event — the variable just never appeared. - **Expected behavior:** A user-named `PAPERCLIP_*` binding should reach the run env unless the harness itself uses that key. Only `PAPERCLIP_API_KEY` should be categorically rejected, and harness-assigned runtime vars (`PAPERCLIP_RUN_ID`, `PAPERCLIP_AGENT_ID`, wake/workspace vars, …) should always win over config. - **Steps to reproduce:** Configure an agent/project env binding named `PAPERCLIP_<ANYTHING>` (plain or secret_ref), run a heartbeat, and inspect the spawned process env — the key is absent. - **Deployment mode:** local server, any local adapter. Related prior PRs (different, save-time/API-layer blanket-ban approach; this PR supersedes that direction with a runtime allow-except-reserved policy): Refs #8239, Refs #8439. ## What Changed - `server/src/services/heartbeat.ts`: the pre-resolution strip now removes only `PAPERCLIP_API_KEY` (hard denylist) instead of every `PAPERCLIP_`-prefixed binding; other `PAPERCLIP_*` keys flow into binding resolution. Low-trust inline-sensitive-env checks now also cover those keys. - `packages/adapter-utils/src/server-utils.ts`: new `isForbiddenConfigEnvKey()` helper; the shared `refreshPaperclipWorkspaceEnvForExecution` merge drops `PAPERCLIP_API_KEY` from config and keeps harness-assigned `PAPERCLIP_*` keys authoritative. - `packages/adapter-utils/src/acpx-engine/execute.ts`: removed the explicit-`PAPERCLIP_API_KEY`-from-config allowance; the run token (`authToken`) is now always applied; config `PAPERCLIP_API_KEY` is ignored. - All local adapters (`claude-local`, `codex-local`, `cursor-local`, `gemini-local`, `grok-local`, `opencode-local`, `pi-local`) plus `cursor-cloud`, `hermes`, and the server `process` adapter: removed `hasExplicitApiKey`-style allowances so the harness token always wins, and guarded the remaining unguarded env-merge loops (claude-local inline loop, process adapter) with the same policy. - Tests updated/added: heartbeat binding-strip test now asserts the three-rule policy; adapter-utils merge tests assert the `PAPERCLIP_API_KEY` ban and `PAPERCLIP_*` pass-through; acpx engine tests moved credential fixtures to `authToken` and assert config `PAPERCLIP_API_KEY` is ignored while other `PAPERCLIP_*` config keys forward and still bust the session fingerprint on rotation. ## Verification - `pnpm vitest run packages/adapter-utils/src/server-utils.test.ts packages/adapter-utils/src/acpx-engine/execute.test.ts` — 127 passed - `pnpm vitest run server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-local-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/cursor-local-execute.test.ts server/src/__tests__/gemini-local-execute.test.ts` — 68 passed - Adapter package execute suites and the server tests touching API-key fixtures (`heartbeat-run-log`, `redaction`, `effective-run-config-fingerprints`, `agent-permissions-routes`) — green. Three pre-existing sandbox/SSH fixture failures reproduce identically on clean `master` on this host and are unrelated. - `pnpm --filter <pkg> typecheck` for server, adapter-utils, and all nine touched adapter packages — all pass. ## Risks - Behavioral change: a deployment that relied on configuring a static `PAPERCLIP_API_KEY` in adapter config env loses that override — by design; the harness-minted run token is now the only source. When no run token exists, no API key is injected at all. - `PAPERCLIP_*`-named user bindings now reach binding resolution and run envs; a key that collides with a harness runtime var is still discarded at merge time, so runtime identity/wake/workspace vars cannot be spoofed. - Low risk otherwise: no migrations, no API surface changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic Claude 5 family, Mythos-class tier), extended thinking enabled, agentic tool use (file edits, shell, test runner) via Claude Agent SDK. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7985c1e000 |
refactor(codex-local): stage an allowlist for sandbox CODEX_HOME sync (#9972)
## Thinking Path
> - Paperclip manages AI agents for work; agents can run in remote
sandboxes via the codex-local adapter
> - When a remote sandbox run starts, the codex-local adapter syncs the
host CODEX_HOME directory into the sandbox so Codex can authenticate and
load its configuration
> - The old approach staged the entire CODEX_HOME and relied on a 4-name
denylist (`tmp`, `.tmp`, `sessions`, `shell_snapshots`) to exclude
unnecessary state
> - The denylist was too narrow: large host-local runtime files
(`logs_2.sqlite`, `plugins/`, etc.) still uploaded on every remote run —
state the sandbox never uses
> - Additionally, `config.toml` (which embeds the managed MCP
`Authorization: Bearer …` header) was staged at 0644, relying solely on
the staging directory's 0700 permission for protection — a
credential-bearing file without per-file least-privilege
> - This pull request replaces the denylist with an explicit allowlist:
only `{auth.json, config.toml, config.json, instructions.md, skills/}`
are staged into a private `0700` temp dir; all staged regular files are
written `0600` (fail-closed on any staging error)
> - The fix is adapter-local to `packages/adapters/codex-local` with
zero changes to the shared `packages/adapter-utils` seam; inbound
auth-merge and outbound copy-back are preserved unchanged
## Linked Issues or Issue Description
No public GitHub issue for this specific bug. Describing the problem
directly:
**Bug:** The codex-local remote sandbox sync uploads more CODEX_HOME
content than the sandbox uses, and stages a credential-bearing file with
an over-permissive mode.
- **What happened:** The codex-local adapter synced the entire
CODEX_HOME directory minus four directory names (`tmp`, `.tmp`,
`sessions`, `shell_snapshots`). Large host-local runtime artifacts — a
SQLite write-ahead log that can reach 420 MB, plugin directories, crash
logs — uploaded on every remote run. `config.toml` (which embeds the
managed MCP `Authorization: Bearer …` header) was staged at `0644`; only
the staging temp dir's `0700` mode prevented broader exposure.
- **Expected behavior:** Only the files Codex needs inside the sandbox
should sync: `auth.json`, `config.toml`, `config.json`,
`instructions.md`, and `skills/`. All staged regular files should use
mode `0600` (single-layer credential protection is insufficient for
secret-bearing files).
- **Steps to reproduce:** Run a remote codex sandbox from a host with a
mature CODEX_HOME. Observe the sync payload includes `sessions/`,
`*.sqlite`, `plugins/`, etc.
Related work: Refs #9621 (outbound auth sync-back in the same
codex-local adapter area; this PR covers the inbound staging direction)
## What Changed
- New `stageCodexHomeForSync(codexHomeDir)` helper in
`packages/adapters/codex-local/src/server/codex-home.ts` stages only
allowlisted files into a fresh `fs.mkdtemp`-created `0700` temp dir,
then registers that dir as the `home` asset (dropping the old `exclude`
field entirely)
- All staged regular files written with mode `0600` — auth.json holds
single-use OAuth tokens; config.toml embeds MCP bearer headers; least
privilege applies throughout
- Symlinks dereferenced to resolved file bytes (`auth.json` single-use
token, each `skills/` entry land as real files)
- Missing-but-optional allowlist entries skipped silently (keyring-only
mode with no `auth.json`; absent `config.json`)
- Staged temp dir removed on teardown AND on error; staging is
fail-closed — a staging error aborts the run rather than shipping a
partial CODEX_HOME
- `execute.ts` updated to call `stageCodexHomeForSync` instead of the
old whole-dir + denylist path; the `exclude` field is removed from the
`home` asset
- Adapter-local fix: zero diff to `packages/adapter-utils` (shared
managed-runtime seam untouched); inbound auth-merge (`provision`) and
outbound copy-back (`restore`) preserved unchanged
## Verification
```sh
# Full codex-local test suite
npx vitest run packages/adapters/codex-local
# 174/174 pass
# Shared seam regression — must be zero diff to packages/adapter-utils
npx vitest run packages/adapter-utils
# 14/14 pass; zero changes to packages/adapter-utils source
# TypeScript (both packages)
npx tsc --noEmit -p packages/adapters/codex-local/tsconfig.json
npx tsc --noEmit -p packages/adapter-utils/tsconfig.json
# Both clean
```
Key new test coverage
(`packages/adapters/codex-local/src/server/codex-home.test.ts`):
- Stager copies exactly the allowlisted entries and nothing else
- Staged `auth.json` dereferenced from symlink and written mode `0600`
- Staged temp dir created with mode `0700`
- ALL staged regular files mode `0600`, including `config.toml` carrying
an MCP bearer token (regression test fails against the pre-fix `0644`
path)
- Fail-closed: I/O error during staging removes the temp dir and
re-throws
- Missing-but-optional entries (`config.json`) skipped without error
- Dangling symlink entries skipped without error
Execute-path integration test (`execute.remote.test.ts`): home asset
uses staged dir; staged config preserves provider block; skills survive;
staged temp dir removed after run teardown.
## Risks
- **Allowlist narrower than denylist:** Any CODEX_HOME file not in
`{auth.json, config.toml, config.json, instructions.md, skills/}` is now
excluded. If a codex-local configuration stores additional files in
CODEX_HOME that the sandbox requires, those runs will miss that state.
This tradeoff is intentional — only known-necessary files sync.
- **Permission change:** `config.toml` previously staged at `0644`, now
staged at `0600`. Strictly more restrictive; no known behavioral impact
(the sandbox reads as the file owner). This was the security gap that
motivated the fix.
- **Fail-closed staging:** A staging error (disk full, temp dir creation
failed) now aborts the run rather than falling back to the raw
CODEX_HOME. This is the safer behavior, but it changes the failure mode
from "run with potentially wrong state" to "run aborted cleanly."
## Model Used
Claude Opus 4.8 (claude-opus-4-8), Anthropic. Reasoning/agentic tool-use
mode (extended thinking, multi-step tool calls).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
canary/v2026.722.0-canary.0
|
||
|
DottaandPaperclip
|
cac3c0fa1a |
feat(connections): add runtime subjects and grants (#9982)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that lets operators connect external apps and govern which subjects may use those credentials > - #9958 established the v3 schema foundation and #9981 adds the AppDefinition catalog layer > - The runtime still needs subject-aware authorization state, scoped key handling, and API/OpenAPI routes so connected apps can actually be granted and used safely > - This pull request adds the runtime grants/authorization behavior on top of the catalog branch, while keeping unrelated dependency and workflow sync commits out of the stack > - The benefit is a reviewable runtime layer that can land after the catalog PR, then unblock the wizard and orchestrator cutover work ## Linked Issues or Issue Description Refs #9958 and #9981. Refs #9981. No public GitHub issue exists for this branch. This is the runtime layer for the Connections v3 stack and is rebased onto `master` after #9981 landed. ## What Changed - Adds the connection user authorization state migration and schema wiring. - Adds shared runtime subject/grant types and validators. - Adds runtime grant and scoped key behavior in the tool-access service. - Adds runtime route coverage and registers the routes in OpenAPI. - Replays only the Connections runtime commits on top of the catalog branch, dropping unrelated sync/dependency history from the prior closed runtime PR. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/validators/tool-access.test.ts server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: runtime grant enforcement is security-sensitive and must fail closed for unknown key scopes. - Migration ordering depends on the schema and catalog layers already merged through #9958 and #9981. - This PR is rebased and retargeted to `master` with runtime-only commits. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.721.0-canary.14 |
||
|
DottaandPaperclip
|
d23fbf8ae4 |
feat(connections): add AppDefinition Wave 1 catalog (#9981)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that defines which external apps and MCP-style integrations operators can browse, configure, and run > - The v3 schema core in #9958 added stable connection identities, auth metadata, and grant-aware contracts, but the app catalog still used the older gallery shape > - The product needs a richer, typed AppDefinition catalog so browsing and setup can render provider-specific auth and configuration requirements consistently > - This pull request moves the Wave 1 app catalog onto generated AppDefinition data and carries that shape through shared types, server lookup paths, and app connection UI > - The benefit is that follow-up runtime and wizard work can build against one catalog contract instead of local-only mock/gallery data ## Linked Issues or Issue Description Refs #9958. No public GitHub issue exists for this branch. This is the catalog layer for the Connections v3 stack after the schema-core foundation in #9958. ## What Changed - Adds generated AppDefinition data for the Wave 1 catalog and ingestion reporting. - Replaces the legacy tool app gallery exports with AppDefinition-centered shared contracts, validators, and tests. - Updates server tool-access lookup behavior to use the AppDefinition catalog. - Updates app connection UI surfaces and tests to consume AppDefinition-backed catalog data. - Documents the catalog ingestion workflow in the connector playbook. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts packages/shared/src/app-definitions-url.test.ts ui/src/pages/apps/AppsConnect.test.tsx server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: this changes the catalog contract used by shared, server, and UI app connection surfaces. - Catalog data quality matters because generated definitions now drive browse/setup display. - Follow-up runtime and wizard PRs must rebase on this branch or on master after this lands. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.721.0-canary.13 |
||
|
DottaandPaperclip
|
7e00f67138 |
feat(connections): add v3 schema core (#9958)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their governed access to external systems. > - Connected Apps build on the existing Apps and MCP gateway substrate so companies can configure reusable, auditable integrations. > - The current connection record does not yet have a stable public address, explicit ownership/auth method fields, or subject-specific credential grants. > - Without that schema core, later OAuth, per-user authorization, token brokering, triggers, and connector-service phases cannot enforce tenant and subject boundaries consistently. > - This pull request adds the forward-compatible Connections v3 schema core while preserving the existing connection lifecycle and directly migrating the remote MCP transport name. > - The benefit is a company-scoped, least-privilege foundation for one-click integrations without bypassing Paperclip secrets, profiles, rules, or audit controls. ## Linked Issues or Issue Description No matching public issue was found. **Problem** Paperclip's current app connections need a durable identity and authorization substrate before Connected Apps can safely support multiple setup methods, per-user credentials, provider tenants, and managed connector services. The existing schema only models a single connection-level credential set and uses legacy transport terminology. **Proposed solution** Add a stable company-scoped connection UID, explicit ownership/auth/transport fields, a subject-aware `connection_grants` table, and multi-key credential annotations. Backfill existing connections and workspace grants in a reversible migration, then update shared/server/UI contracts to the new `mcp_remote` transport name. **Related work** - Related foundation: #9534 - Roadmap: Connected Apps (one-click integrations) ## What Changed - Added company-scoped connection `uid`, `ownership`, `authKind`, and canonical transport fields across database, shared contracts, validators, services, and UI fixtures. - Added `connection_grants` with workspace/user subject rules, provider tenant metadata, credential secret refs, revocation state, company scoping, and uniqueness constraints. - Added migration `0182_connections_v3_schema_core` to backfill stable UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create default workspace grants, and support rollback coverage. - Added multi-key credential annotations and updated gateway/access services without changing the existing lifecycle behavior. - Updated the connection glossary, connector playbook, and security threat model for the new identity, grant, and relay boundaries. - Added explicit test UIDs to direct database fixtures so the new non-null invariant is exercised across affected server suites. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts server/src/__tests__/heartbeat-runtime-skills.test.ts server/src/__tests__/tool-oauth-legacy-backfill.test.ts server/src/__tests__/tool-access-policy-service.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts packages/db/src/connections-v3-schema-core-migration.test.ts packages/shared/src/validators/tool-access.test.ts --config vitest.config.ts` — 9 files, 218 tests passed. - Latest-head GitHub Actions: build, typecheck, general/serialized suites, backup/worktree restore coverage, both e2e shards, canary, policy, and security scans pass. - Greptile: 5/5 with zero unresolved threads. - `pnpm check:token-gates` remains red only on five pre-existing `#9627` color literals outside this change. ## Risks - **Migration risk:** UID backfill and default-grant creation touch every existing connection. The migration uses company-scoped uniqueness, deterministic legacy UIDs with ID suffixes, and seeded up/rollback coverage. - **Authorization risk:** Grant rows carry credential references. Constraints enforce workspace-vs-user subject shape, company/connection lookup indexes, one default grant per connection, and one user grant per connection/subject. Security review is requested specifically for this design. - **Compatibility risk:** `remote_http` is renamed directly to `mcp_remote`; all repository call sites and fixtures are updated in the same change. - **Future-phase risk:** Subject-bound token issuance, triggers, and connector-service relay verification remain fail-closed requirements documented for later phases; this PR does not expose those capabilities. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI coding agent. The runtime did not expose an exact underlying model ID or context-window size; capabilities used include repository inspection, code editing, shell execution, test execution, Git/GitHub CLI operations, and structured reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.721.0-canary.12 |
||
|
Andrew AymelogluandClaude Fable 5
|
98608dd71c |
fix(cli): add missing company manifest fields to import selection test fixture (#9978)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The CLI's company import/export (portability) subsystem has a typed manifest, `CompanyPortabilityCompanyManifestEntry`, that test fixtures must satisfy > - PR #2379 was authored in April and merged on 2026-07-21 without a rebase; in the interim the manifest type gained five required fields > - Its new test fixture predates those fields, so `tsc --noEmit` now fails on master (TS2739), which fails the Release workflow's `verify_canary / Typecheck` job and blocks canary publishing > - This pull request adds the five missing fields to that one fixture, matching the sibling fixtures in the same file > - The benefit is a green typecheck on master, unblocking the Release pipeline ## Linked Issues or Issue Description No open issue exists; the problem is described here (bug path): - **What happens:** `pnpm typecheck` fails on master at `cli/src/__tests__/company.test.ts:744` with `error TS2739: Type '{...}' is missing the following properties from type 'CompanyPortabilityCompanyManifestEntry': attachmentMaxBytes, feedbackDataSharingEnabled, feedbackDataSharingConsentAt, feedbackDataSharingConsentByUserId, feedbackDataSharingTermsVersion`. See the failing Release run: https://github.com/paperclipai/paperclip/actions/runs/29855224743/job/88718044096 - **Expected:** master typechecks cleanly and the Release workflow publishes the canary - **Cause:** semantic conflict — Refs #2379 (merged with pre-existing green checks from April, before the manifest type gained the five required fields) ## What Changed - Added `attachmentMaxBytes: null`, `feedbackDataSharingEnabled: false`, `feedbackDataSharingConsentAt: null`, `feedbackDataSharingConsentByUserId: null`, and `feedbackDataSharingTermsVersion: null` to the company manifest fixture in the test "includes extension file even when all entities are deselected" (`cli/src/__tests__/company.test.ts`), using the same values and field order as the two sibling fixtures in the same file ## Verification - `cd cli && pnpm typecheck` — fails on master with TS2739 at `src/__tests__/company.test.ts:744`, passes with this change - `cd cli && pnpm vitest run src/__tests__/company.test.ts` — all tests pass (runtime behavior unchanged; the fixture only gains fields the code under test does not read) - CI: the `verify / Typecheck` job on this PR exercises the same gate that is currently red on master ## Risks - Low risk — a 5-line, test-only fixture change; no runtime code touched. Worst case is a still-failing typecheck, which CI on this PR verifies before merge. ## Model Used - Claude Fable 5 (Anthropic, model ID `claude-fable-5`), via Claude Code with extended thinking and tool use (GitHub CLI/API for investigation, diff authored by the model) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01T5W8yjizAewpLtmsBHKPuA Co-authored-by: Claude Fable 5 <noreply@anthropic.com>canary/v2026.721.0-canary.11 |