mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
974949a39bf87644e1579adb545ab485a083316c
35
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jannes StubbemannandClaude Fable 5.1
|
5752d6bd93 |
fix(heartbeat): block runs on a stuck sandbox plugin and re-enable errored bundled plugins at boot (#12957)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents run inside environments. A sandbox environment gets its
sandbox from a provider plugin (for example the bundled
`paperclip.kubernetes-sandbox-provider`), and every run starts by
acquiring a lease through that plugin.
> - When a plugin activation fails once (on a hosted deployment: one
`RPC call "initialize" timed out after 15000ms`), the loader calls
`markError`. That persists `status = error` on the plugin row and
switches off worker auto-restart. Boot activation (`loadAll`), the
bundled-plugin bootstrap and the lazy worker recovery all consider only
`ready` plugins, so the plugin stays in `error` across restarts until an
operator enables it by hand.
> - Every run that needs the provider then fails before dispatch with
`Sandbox provider "kubernetes" is installed via plugin "...", but that
plugin is currently error.` That message matches neither the retryable
classifier (`... but its worker is not running`) nor any configuration
classifier, so the run is recorded as a plain `setup_failed`, the issue
is released, and the scheduler dispatches the same failing run again on
the next tick. On the hosted deployment one company produced about
11,300 identical failed runs, one every 30 seconds, for a week (#12953
is a customer's report of the same condition).
> - Two gaps cause this: the heartbeat treats a condition that only an
operator can change as a transient setup failure, and the bundled-plugin
bootstrap never gives a plugin in `error` another chance even though the
bundle ships with the release image.
> - This pull request classifies the "installed but not ready" lease
failure as `configuration_incomplete`, so the existing recovery path
moves the issue to `blocked` with one recovery action and an actionable
notice; and it re-enables a bundled plugin found in `error` once per
boot, so the next server restart heals the plugin.
> - The benefit is that a stuck provider plugin surfaces as one blocked
issue per task with clear next steps, instead of an endless stream of
identical failed runs, and a restart repairs the plugin without an
operator having to know the plugin API.
## Linked Issues or Issue Description
- Refs #12953 — hosted report: "that plugin is currently error" on every
run for six days, including runs that were retried by hand. This PR
stops the retry loop (issue goes to `blocked`) and makes a server
restart re-activate the bundled plugin. It does not change how a managed
Kubernetes environment is provisioned for a company, which the same
report also mentions.
- Related PR: #9760 pauses the agent for the permanent `Adapter "..." is
not in the configured adapter registry` setup failure. This PR handles a
different permanent condition (plugin not `ready`) and routes it through
the existing `configuration_incomplete` recovery path (issue-level block
with a recovery action) rather than an agent-level pause, because the
gap is on the plugin, not on the agent. The two do not overlap in code
paths.
- No existing issue covers the bundled-plugin re-enable. Bug
description:
**What happened**
A bundled sandbox provider plugin went to `status = error` after one
failed activation. It stayed in `error` across every later server
restart. Every run for every agent on that provider failed lease
acquisition in under a second with `... but that plugin is currently
error.` (`setup_failed`), and the heartbeat kept dispatching new runs
that failed the same way.
**Expected behavior**
A run that fails because its provider plugin is not `ready` is recorded
as a configuration gap and the issue is moved to `blocked` with a notice
that names the plugin and its status, so no further runs are dispatched
until an operator acts. A bundled plugin left in `error` gets a fresh
activation attempt on the next boot.
**Steps to reproduce**
1. Install a sandbox provider plugin and create a sandbox environment
that uses it; make it an agent's default environment.
2. Set the plugin row's status to `error` (or make its worker fail
`initialize` once so the loader does it).
3. Assign an issue to the agent and let the heartbeat run it.
4. Observe: the run fails with `... but that plugin is currently error.`
as `setup_failed`, the issue is released, and the next tick dispatches
another run that fails the same way. Restart the server: the plugin is
still `error`.
**Paperclip version**
master at
|
||
|
Jannes StubbemannandClaude Fable 5.1
|
023e640a7e |
fix(db): reap idle pool connections, name the pool, and end it on shutdown (#12956)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server keeps one postgres.js pool (`packages/db/src/client.ts`, `createDb`) for every query it runs. #10795 made the pool tunable from the environment, but the defaults stayed at the driver defaults: an idle connection never closes, the pool reports itself as `postgres.js`, and no code path ever calls `sql.end()`. > - On a hosted Paperclip deployment the server entered a restart loop (a bundled plugin failure that #12953 describes made every run fail, and the pool saturated). Each generation opened its ten connections, died, and left the backends open on the PostgreSQL side until TCP keepalive reaped them hours later. After about 20 generations the backends exceeded `max_connections`, and every later boot died on its first bootstrap query with `sorry, too many clients already`, before `server.listen()`. The loop could not heal itself. #9555 describes the same shape on a launchd-supervised self-hosted install. > - Three properties of the pool combine to make this possible: idle connections are never reaped, the pool is never ended on any exit path, and an operator cannot even find the leaked backends in `pg_stat_activity` because they carry the generic driver name. > - This pull request gives the pool a 60 second idle timeout and the `paperclip` application name by default, exposes `max_lifetime` and `application_name` through the same `DATABASE_*` environment contract that #10795 introduced, and ends the pool on the orderly SIGINT/SIGTERM path and on the fail-loud startup path. > - The benefit is that a restarting or crash-looping server releases its backends instead of accumulating them, and an operator can see and count Paperclip's connections. ## Linked Issues or Issue Description - Refs #9555 — database connection pool leak causes an infinite restart loop under load. This PR closes the "pool never ends, idle connections never close" part of that report. - Refs #12953 — hosted outage report. The pool exhaustion is the second half of that incident; the first half (a stuck sandbox provider plugin) has its own PR. - Related prior PRs: #9597 and #8780 both propose hard-coded `idle_timeout` / `max_lifetime` values in `createDb`. Both predate #10795 (merged), which made these options environment-driven; this PR builds on the merged shape and adds the shutdown `end()` that neither covers. #4006 and #7481 are closed earlier attempts in the same area. ## What Changed - `packages/db/src/client.ts` - New `resolveDatabaseClientOptions()` applies Paperclip defaults on top of the environment: `idleTimeoutSeconds` defaults to 60 (`DEFAULT_DATABASE_IDLE_TIMEOUT_SECONDS`) and `applicationName` to `paperclip` (`DEFAULT_DATABASE_APPLICATION_NAME`). `createDb` uses it for both the environment path and explicit options. - `DATABASE_IDLE_TIMEOUT_SECONDS` now accepts `0` to restore the driver default (keep idle connections open). Negative or non-integer values still throw. - New environment variables: `DATABASE_MAX_LIFETIME_SECONDS` (positive integer, maps to `max_lifetime`) and `DATABASE_APPLICATION_NAME` (non-empty string, maps to `connection.application_name`). - `postgresJsOptions()` maps the two new options. - `server/src/shutdown.ts` - `finalizeServerShutdown` gains two optional ordered steps: `closeHttpListener` runs first, before the application services stop; `closeDatabase` runs after the application services and before the embedded PostgreSQL stop. A failure in either is logged and does not stop the teardown. Final order: listener → application services → database pool → embedded PostgreSQL → instrumentation → Sentry. - New `closeHttpListenerForShutdown()`: stops accepting requests, closes idle keep-alive sockets, waits up to 5 s for open connections, then closes whatever is left. Requests still in flight are drained while every service is available, and none can reach a route after `sql.end()`, on the signal path and the programmatic path alike (the programmatic path's later `server.close` finds the listener closed and skips). - `server/src/app.ts`: the app shutdown hook (`shutdownAppServices`) now stops the plugin job scheduler, whose tick queries the database, so a programmatic `shutdown()` leaves no timer running against the ended pool. - `server/src/index.ts` - `startServer()` is now a thin wrapper around the boot sequence. When the boot sequence throws after the pool exists, the wrapper ends the pool (and the separate migration pool, when configured) before it rethrows. This covers the `process.exit(1)` path in the main module and the CLI `paperclip run` path alike. - The orderly shutdown passes the same `closeDatabaseClients` to `finalizeServerShutdown`. - `endDatabaseClient` tolerates a client without `$client` (test doubles) and uses a 5 second end timeout. - Docs: `docs/deploy/database.md` gets a "Connection Pool Settings" table with every `DATABASE_*` pool variable, its default and its effect; `doc/DATABASE.md` lists the two new variables. - Tests - `packages/db/src/client-options.test.ts`: parsing of the new variables, `0` for the idle timeout, rejection of malformed values, driver option mapping, and the `resolveDatabaseClientOptions` defaults. - `packages/db/src/client.test.ts` (embedded PostgreSQL): `createDb(url)` reports `application_name = paperclip` for its own backend, and a pool with `idleTimeoutSeconds: 1` has zero backends in `pg_stat_activity` after the timeout. - `server/src/shutdown.test.ts`: the listener closes before the application services, and the database close runs between the application services and the embedded PostgreSQL stop; a failing database close is logged while the teardown still finishes; `closeHttpListenerForShutdown` closes idle sockets and resolves on close, force-closes after the grace period, and is a no-op when the listener was never bound. ## Verification - `pnpm --filter @paperclipai/db typecheck` — passes (`check:migrations` + `tsc --noEmit`). - `cd server && pnpm typecheck` — passes. - `cd packages/db && pnpm exec vitest run src/client-options.test.ts src/client.test.ts src/client-teardown-registry.test.ts` — 9 + 18 + 3 tests pass (the `client.test.ts` cases need embedded PostgreSQL; the new one waits up to 10 s for the idle reap and passed in about 3 s). - `cd server && pnpm exec vitest run src/shutdown.test.ts src/__tests__/server-startup-feedback-export.test.ts src/__tests__/bootstrap-claim-routes.test.ts` — 34 + 11 tests pass. The startup-feedback suite exercises `startServer()` with a mocked `createDb`, which is why `endDatabaseClient` tolerates a client without `$client`. - Manual check for a reviewer: start the server against any PostgreSQL, then run `SELECT application_name, state, count(*) FROM pg_stat_activity GROUP BY 1, 2;`. Paperclip's backends now show `paperclip`. Leave the server idle for more than 60 s and the idle backends disappear. Send SIGTERM and the backends close before the process exits. ## Risks - Behavior change with no environment set: idle pooled connections now close after 60 s. The next query after an idle period pays a reconnect (single-digit milliseconds on a local socket). postgres.js reconnects transparently. Set `DATABASE_IDLE_TIMEOUT_SECONDS=0` to keep the previous behavior. - `application_name` changes from `postgres.js` to `paperclip`. Anything that filtered `pg_stat_activity` on the old name would need an update; nothing in this repo does. - The HTTP listener now closes at the start of the final teardown (after the heartbeat run drain, which still needs the API for running agents). The pool close runs after the application services. A late query from a timer that survived the service shutdown would fail with a driver "connection ended" error instead of running; the known database-backed timer (the plugin job scheduler) is now stopped in the service shutdown. - The listener drain adds at most 5 s to a shutdown while long-lived connections (for example WebSocket clients) are open; after that they are closed forcibly. - `startServer()` is split into a wrapper and the boot sequence. The exported signature and return type are unchanged. - No migration, no schema change. ## Model Used - Claude Fable 5.1 (`claude-fable-5-1`) via Claude Code, extended thinking, tool use (file edits, shell, test runs). The change was produced with the model and reviewed by the submitting human. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014t3bi2beVNVVHAxK36dmXm --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
d77eeb8914 |
fix(sandbox-bridge): allow the agent-hire skill's routes through the callback bridge (#8978)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Managed agents run inside a sandbox and reach the Paperclip server only through the sandbox callback bridge, which forwards a fixed route allowlist (`DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST`) > - The `paperclip-create-agent` skill instructs an agent to call adapter/icon discovery endpoints, compare existing agent configurations, submit a hire request, and link the resulting approval to its source issue > - None of those routes were on the bridge allowlist, so a sandboxed agent following the skill correctly hit `Route not allowed` on every call — including the hire `POST` itself — making hiring impossible from inside a sandbox > - This pull request adds the six routes the skill uses to the bridge allowlist, while keeping direct agent creation (`POST /api/companies/:id/agents`) denied > - The benefit is that hiring works end-to-end for sandboxed agents through the approval-gated `agent-hires` path, without widening the bridge beyond what the skill needs ## Linked Issues or Issue Description No public issue exists; describing the bug in-PR (bug template fields): - **What happened:** A managed agent running in a sandbox followed the `paperclip-create-agent` skill and got `Route not allowed` from the callback bridge on every endpoint the skill documents — adapter discovery (`/llms/agent-configuration.txt`, `/llms/agent-configuration/:adapterType.txt`, `/llms/agent-icons.txt`), config comparison (`GET /api/companies/:id/agent-configurations`), the hire submission (`POST /api/companies/:id/agent-hires`), and approval linking (`POST /api/issues/:id/approvals`). - **Expected behavior:** An agent with hiring permission can complete the hire flow from inside a sandbox; the bridge forwards the skill's routes and the server enforces authorization (`canCreateAgents`). - **Impact:** Hiring by sandboxed agents was fully broken — the failure is in the transport allowlist, not permissions, so no configuration could work around it. Related: #8981 (companion fix making the `paperclip-create-agent` skill available to agents that can hire; supersedes #8823). The two changes serve the same end-to-end hire flow but are independently mergeable — this PR is purely the bridge transport allowlist. Supersedes #8853. ## What Changed - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: add six routes used by the `paperclip-create-agent` skill to `DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST` (three `GET /llms/...` discovery routes, `GET .../agent-configurations`, `POST .../agent-hires`, `POST /api/issues/:id/approvals`), with a comment documenting why direct agent creation stays denied - `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`: assert the six routes are allowed, and add negative cases proving the regexes do not over-match (no `POST .../agents`, no non-`.txt` or arbitrary `/llms` files, no `agent-hires` sub-resources) ## Verification - `npx vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts` — 13/13 tests pass locally - `npx tsc --noEmit -p packages/adapter-utils` — clean - Manual: run a managed agent in a sandbox, invoke the `paperclip-create-agent` skill, and confirm the discovery calls, hire `POST`, and approval linking all pass through the bridge; `POST /api/companies/:id/agents` still returns `Route not allowed` ## Risks - Low risk: additive allowlist entries only; anchored regexes with `[^/]+` segments prevent over-matching (covered by tests) - The bridge allowlist bounds surface area but does not replace server-side authorization — the hire `POST` remains approval-gated and permission-checked (`canCreateAgents`) on the server > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) — Fable 5 (`claude-fable-5`), extended thinking, agentic tool use via Claude Code; original diff authored with Claude Opus 4.8 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jannes Stubbemann
|
6d2eab742f |
fix(server): retry runs that hit a sandbox provider worker restart window instead of failing setup (#10212)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs execute in sandbox environments acquired through provider
plugins (e.g. the Kubernetes sandbox provider)
> - Lease acquisition happens during run setup, before the adapter
executes
> - When a provider plugin's worker is momentarily unavailable (a server
or plugin restart window), lease acquisition throws "Sandbox provider
... is installed via plugin ..., but its worker is not running."
> - The heartbeat setup path records that as a terminal `setup_failed`:
no retry classifier matches the message, so the run dies instantly even
though the worker returns seconds later
> - This PR classifies that transient condition as retryable
infrastructure so the run is retried instead of being lost to a restart
blip
> - The benefit is that routine restarts no longer produce spurious
instant run failures
## Linked Issues or Issue Description
No public GitHub issue exists; describing inline following the bug
report template.
**What happened**
During a brief sandbox-provider-worker restart window, several runs
failed instantly with `setup_failed` ("... but its worker is not
running."), while runs on the same agent moments earlier and later
succeeded.
**Expected behavior**
A transient, self-healing worker-unavailable condition should schedule a
bounded retry, not terminally fail the run.
**Steps to reproduce**
Trigger a run while the sandbox provider plugin worker is momentarily
unavailable (a server or plugin restart). Lease acquisition throws the
worker-not-running error and the run is finalized as `setup_failed` with
no retry. The recovery test added here reproduces the classification
path.
**Deployment mode**
Cloud multi-tenant execution (Kubernetes sandbox provider plugin).
## What Changed
- Added a dedicated, readable predicate that recognizes the transient
sandbox-provider-worker-unavailable lease failure and treats it as
retryable infrastructure, so the heartbeat schedules a bounded
continuation retry instead of finalizing terminally
- The predicate is anchored to the full lease-failure phrasing (`is
installed via plugin ... but its worker is not running`) so it cannot
match the permanent "provider not installed" message emitted by config
validation
- Added tests proving the readiness poll already waits the full deadline
while the worker handle is absent or `starting` (registered-late
coverage); no poll behavior change was needed
## Verification
- `cd server && npx vitest run
src/__tests__/environment-runtime.test.ts` — poll exhaustion +
registered-late cases
- `npx vitest run src/__tests__/heartbeat-process-recovery.test.ts` —
worker-unavailable message schedules a retry; a non-matching permanent
provider failure still escalates terminally (negative case)
## Risks
Low risk. The retry is bounded by the existing
infrastructure-continuation attempt cap (max 3), the message match is
narrow enough to exclude the permanent provider-not-installed failure
(covered by a negative test), and no readiness-poll or lease-acquisition
behavior changed.
## Model Used
Claude (Anthropic) via Claude Code. Implementation and tests authored by
a Claude Sonnet-class model (`claude-sonnet-5`) dispatched as isolated
per-task implementer agents under a multi-agent orchestration workflow;
root-cause investigation, planning, and two-stage adversarial code
review performed by additional Claude agents. Extended thinking and tool
use enabled throughout.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
Jannes StubbemannandClaude Opus 5
|
a0bdf388af |
fix(agents): refuse to hire onto an adapter this instance cannot run (#10256)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Hiring an agent means choosing a harness (adapter) for it, and an
instance can declare which harnesses it actually runs through
`PAPERCLIP_ADAPTERS`, which `reconcileAdapterAvailability` turns into a
disabled set at boot
> - The hire and create routes validate the adapter type with
`assertKnownAdapterType`, which only asks whether the adapter is
REGISTERED — a disabled adapter passes
> - So an agent can be created on a harness the instance cannot run, and
the failure only appears later, per run, at lease time: `Adapter "..."
is not in the configured adapter registry`
> - By then the error is in a run log, minutes after the choice, with
nothing tying it back to the harness the user picked; the agent also
keeps accepting work it can never do
> - This pull request validates the hire and create paths against the
ENABLED set and refuses with a message that names the adapters that are
available
> - The benefit is that an impossible choice fails at the moment it is
made, in the words of the choice itself, instead of as a run failure the
user cannot act on
## Linked Issues or Issue Description
No existing issue; describing it here per the bug report template.
**What happened**
On an instance with a curated registry, a company's Chief of Staff was
hired on `cursor_cloud`, which that instance had disabled. The API
accepted the hire. Its first assignment run then failed:
```
Failed to acquire lease for environment "Kubernetes Sandbox" (sandbox): Adapter "cursor_cloud" is not in the configured adapter registry
```
and its automation run sat in `queued` for hours afterwards. Nothing in
the hire response, the agent detail view, or the agent's status
explained that this harness could never run.
**Expected behavior**
hiring on an adapter the instance has disabled is refused at hire time,
with a message naming the adapters that can be chosen.
**Steps to reproduce**
1. Start the server with a registry that omits an otherwise-registered
adapter, e.g. `PAPERCLIP_ADAPTERS` listing `claude_local` but not
`cursor_cloud`.
2. `POST /api/companies/:companyId/agents` with
`{"name":"CoS","adapterType":"cursor_cloud"}`.
3. The agent is created (201). Every run it attempts fails at lease time
with the message above.
**Paperclip version or commit**
master (`4c55f0d8d`).
## What Changed
- `server/src/routes/agents.ts`: adds `assertSelectableAdapterType`,
which extends `assertKnownAdapterType` with an enabled-set check and
throws `422 Adapter "<type>" is not available on this instance.
Available adapters: <list>`. The hire (`POST .../agent-hires`) and
create (`POST .../agents`) paths now use it.
- Routes that operate on an EXISTING agent keep
`assertKnownAdapterType`, so an agent already running on a
since-disabled adapter is unaffected — the same rule
`listEnabledServerAdapters` already documents ("hidden from selection,
still functional for agents that already use them").
- `server/src/__tests__/agent-adapter-validation-routes.test.ts`: mocks
the adapter-plugin store's disabled set (so the test never writes to a
real `~/.paperclip/adapter-settings.json`), and covers
refuse-when-disabled (including that the message names the alternatives
and that no agent is created) plus create-still-works-when-enabled.
## Verification
```
pnpm vitest run server/src/__tests__/agent-adapter-validation-routes.test.ts
```
13 tests pass, including the two new cases and the existing
unknown-adapter-type test.
Manual: disable an adapter (`PATCH /api/adapters/:type {"disabled":
true}` as an instance admin, or omit it from `PAPERCLIP_ADAPTERS` and
restart), then POST an agent with that `adapterType` — 422 naming the
available adapters, and no agent row is created.
## Risks
Low, and scoped to new selections:
- Automation that creates agents on a disabled adapter now gets a 422
where it previously got a 201 followed by runs that always failed. That
is the intended behavior change, and the message names the valid
choices.
- Existing agents, and every route that acts on an existing agent, are
untouched.
- The enabled set comes from the same store `GET /api/adapters` already
reports, so the API and the picker cannot disagree.
## Model Used
Claude Opus 5 (Anthropic), model id `claude-opus-5`, 1M context window,
extended thinking, with tool use and code execution via Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change
(`upstream/adapter-selection-guard`) and contains no internal ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (the
new helper documents the selection-vs-existing-agent rule)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Related: #10254 makes the adapter inventory readable during onboarding,
which is what lets the picker hide these adapters in the first place.
This PR is the server-side backstop for the same failure.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jannes StubbemannandClaude Fable 5
|
8a5c0615f9 |
fix(adapter-utils): forward sandbox callback bridge traffic to the local listen origin (#10017)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can execute in remote sandboxes, where a callback bridge relays in-sandbox Paperclip API calls back to the host server process > - The bridge worker resolves its forward target from PAPERCLIP_RUNTIME_API_URL / PAPERCLIP_API_URL, which now prefer a configured public base URL and therefore mean "the origin browsers and external agents use" > - The bridge worker runs inside the same process that serves the API, so forwarding through the public origin routes an in-process loopback hop through the network edge > - On a deployment whose public origin sits behind a session-gated edge proxy, every forwarded agent API call is rejected at the edge, so agents in sandboxes cannot read their identity, comment, or hire > - This pull request resolves the bridge forward target from the explicit hostApiUrl override or the local listen host and port only, never the public URL exports > - The benefit is that sandbox agent API calls keep working regardless of how the public base URL is configured or gated ## Linked Issues or Issue Description No existing issue. Describing in-PR following the bug report template: **What happened?** On a cloud deployment with a session-gated public edge, setting a public base URL (PAPERCLIP_PUBLIC_URL) caused every in-sandbox agent API call through the sandbox callback bridge to fail with `403 text/plain "Access denied"` from the edge proxy. With PAPERCLIP_BRIDGE_DEBUG enabled, the bridge logs show the forward target is the public origin, and every proxied request (for example `GET /api/agents/me`) returns the edge proxy's 403 instead of reaching the API. **Expected behavior** The bridge worker runs in the same server process that serves the API, so forwarded calls should target the local listen origin and succeed regardless of how the public origin is configured or gated. **Steps to reproduce** 1. Run the server with a public base URL configured, fronted by a proxy that requires a browser session on API routes. 2. Start a sandbox-executed agent run (any adapter using the sandbox callback bridge). 3. Observe every in-sandbox call to the Paperclip API fail with the proxy's 403; with PAPERCLIP_BRIDGE_DEBUG the forward URL is the public origin. **Paperclip version or commit** Current `master`. **Deployment mode** Self-hosted server behind a reverse proxy. **Agent adapter(s) involved** All sandbox-executed adapters (the bridge is adapter-agnostic). ## What Changed - `packages/adapter-utils/src/execution-target.ts`: `startAdapterExecutionTargetPaperclipBridge` now resolves its forward target as `input.hostApiUrl?.trim() || resolveDefaultPaperclipApiUrl()`. It no longer consults `PAPERCLIP_RUNTIME_API_URL` / `PAPERCLIP_API_URL`, which now describe the public origin for browsers and external agents, exactly the wrong target for an in-process loopback hop. `resolveDefaultPaperclipApiUrl()` builds `http://<PAPERCLIP_LISTEN_HOST>:<PAPERCLIP_LISTEN_PORT>` (exported by server boot before any run executes) and maps wildcard listen hosts to the loopback address of the same family (`0.0.0.0` to `127.0.0.1`, `::` to `[::1]`), so the forward target always matches the address family the server is bound to. `input.hostApiUrl` remains the explicit override seam. A comment documents the reasoning. - `packages/adapter-utils/src/execution-target-sandbox.test.ts`: two new tests. One sets both public URL env vars to an unreachable public https origin and asserts the bridge forwards to the local listen origin (fails before this fix with a 502 because the worker targets the public origin). One asserts an explicit `hostApiUrl` input still overrides everything. - The acpx-engine bridge start (`packages/adapter-utils/src/acpx-engine/execute.ts`) passes no `hostApiUrl` and goes through the same resolution site, so it is covered by the same fix. The sandbox-facing env builder in `server-utils.ts` is intentionally untouched; the bridge env overrides `PAPERCLIP_API_URL` inside the sandbox separately. ## Verification - `npx vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts` (28 tests pass; the new local-origin test fails without the fix) - `pnpm --filter @paperclipai/adapter-utils typecheck` (clean) - Full adapter-utils suite run; the only failures are pre-existing environment-dependent tests (bubblewrap and shallow-clone tests on macOS) identical on a clean `master` checkout ## Risks - Low risk. Deployments where the bridge previously worked did so precisely because the forward target already resolved to the local origin (no public URL configured, so the chain fell through to the same `resolveDefaultPaperclipApiUrl()` result). The only behavioral shift is for deployments with a public URL configured, where forwarding through the edge was either wasteful (an unnecessary network round trip) or broken (session-gated edge). The explicit `hostApiUrl` override seam is preserved for callers that need a nonlocal target. ## Model Used - Claude Fable 5 (claude-fable-5), extended thinking, via Claude Code ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
1f7959bc69 |
fix(codex-local): skip benign stderr warnings when deriving the fallback run error (#10003)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents execute through adapters; the codex_local adapter runs the Codex CLI and reports each run's outcome, including an error message when the CLI exits nonzero > - When no error can be parsed from the CLI's JSONL output, `toResult` in `packages/adapters/codex-local/src/server/execute.ts` falls back to the first non-empty stderr line as the run error > - The adapter itself passes the approvals-bypass flag, so the CLI's first stderr line is always the benign startup warning "YOLO mode is enabled. All tool calls will be automatically approved." > - Failed runs therefore record that warning as their error, hiding the real cause (for example an OpenAI API error further down in stderr) and making failures hard to diagnose from the run record > - This pull request derives the fallback error from the first meaningful stderr line, skipping a conservative set of known benign lines, and keeps the existing behavior when every line is benign > - The benefit is that failed Codex runs surface the actual failure reason instead of a harmless startup warning, without ever producing an emptier message than before ## Linked Issues or Issue Description No public issue exists for the codex_local case. The same bug class was fixed for gemini-local in Refs #5099 and Refs #3476; this PR applies the equivalent fix to codex_local. **What happened?** On a multi-tenant cloud deployment of Paperclip, several codex_local runs failed and their run records showed `error_code=adapter_failed` with the error text "YOLO mode is enabled. All tool calls will be automatically approved." That is a benign Codex CLI startup warning, printed on every run because the adapter passes the approvals-bypass flag itself. The real failure (an OpenAI API error printed later in stderr) was never surfaced. **Expected behavior** When the Codex CLI exits nonzero and no error was parsed from its JSONL output, the run error should be the first stderr line that actually explains the failure, not a startup warning the adapter itself provoked. **Steps to reproduce** 1. Configure a codex_local agent and make the underlying Codex CLI invocation fail after startup (for example, configure a model id the active credentials cannot use). 2. Run the agent so the CLI exits nonzero with no parsed JSONL error. 3. Inspect the run's error message: it shows the YOLO approvals warning (the first stderr line) instead of the real error printed further down in stderr. ## What Changed - Added `firstMeaningfulStderrLine` next to `firstNonEmptyLine` in `packages/adapters/codex-local/src/server/execute.ts`, with a conservative benign-line predicate covering the YOLO approvals warning and `[paperclip] ...` diagnostic lines the adapter injected (for example ACP fallback notes). - Used it only in the `toResult` fallback error derivation. If every stderr line is benign, the existing chain still applies (first non-empty line, then `Codex exited with code N`), so the message never gets emptier than today. Logging is unchanged. - Added `packages/adapters/codex-local/src/server/execute.stderr-error.test.ts`: four end-to-end cases through `execute()` with a mocked CLI process, plus unit coverage for the new helper. Tests were written first and confirmed failing before the fix. ## Verification - `pnpm exec vitest run packages/adapters/codex-local/src/server/execute.stderr-error.test.ts` (7 tests pass; 5 failed before the fix as expected) - `pnpm exec vitest run packages/adapters/codex-local` (21 files, 188 tests pass) - `pnpm run typecheck` in `packages/adapters/codex-local` (clean) ## Risks Low risk. Only the derived fallback `errorMessage` changes, and only when a benign line would otherwise have been picked; parsed JSONL errors, logging, retry/quota/auth classification inputs, and the empty-stderr exit-code fallback are untouched. The benign-line list is deliberately conservative (exact prefixes) so real errors are never skipped. ## Model Used Claude Fable 5 (claude-fable-5), extended thinking, via Claude Code ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.7
|
7f2ed0ad90 |
security(server): close cross-tenant existence oracle (404 instead of 403) (#3967)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - In a multi-tenant deployment, route handlers that take a resource id (`issue`, `goal`, `project`, `approval`, etc.) look the resource up by id and then call `assertCompanyAccess` on its `companyId` — 404 if it doesn't exist, 403 if it exists in another tenant > - The split status codes are a classic *existence oracle*: any authenticated user can enumerate ids across tenants by probing for the 403/404 boundary, mapping out which issues, labels, approvals, etc. exist in other customers' tenants even when they cannot read the contents > - The right fix is a single uniform 404 for both "not found" and "found but cross-tenant", which collapses the oracle but still preserves write-path checks (active membership, viewer-readonly) for *authorized* tenants > - This pull request adds a non-throwing `hasCompanyAccess(req, companyId)` helper plus a `getAccessibleResource` wrapper that ~130 handlers across 14 route files now use, folding the access check into the existence check while still running `assertCompanyAccess` for authorized tenants so viewer-readonly / inactive-membership rejections fire unchanged on write paths > - The benefit is closing a multi-tenant information leak without breaking write-path security or single-tenant local-first behavior ## Linked Issues or Issue Description Refs #709 — asks for company-scope regression coverage across approval/activity/access routes, because a subtle route refactor could leak cross-tenant data; this PR hardens exactly those surfaces (uniform 404 across 14 route files including `approvals`, `activity`, `secrets`) and updates cross-tenant expectations in test files. It does not add the full coverage matrix #709 asks for — hence Refs, not Closes. No existing issue covers the oracle itself — described in-PR: - Route handlers returned 404 for "not found" but 403 for "exists in another tenant", a classic *existence oracle*: any authenticated user could enumerate ids across tenants by probing the 403/404 boundary. - That maps out which issues, labels, approvals, etc. exist in other customers' tenants even when their contents are unreadable. - Fix: a uniform 404 for both cases, while keeping write-path checks (active membership, viewer-readonly) for authorized tenants. ## What Changed - **`server/src/routes/authz.ts`** — new `hasCompanyAccess(req, companyId): boolean` helper alongside the existing `assertCompanyAccess`. Docstring spells out the two-step pattern (404 gate, then `assertCompanyAccess` for write-path checks). The helper mirrors `assertCompanyAccess`'s company-scope semantics exactly — in particular, signed-in instance admins do **not** get blanket access to companies they are not a member of (the repo's `authz-company-access` tests pin that behavior for `assertCompanyAccess`; an earlier draft of the helper accidentally widened it for reads). - **`getAccessibleResource(req, res, lookup, notFoundMessage)`** — the safe thing is now the easy thing. One helper wraps the whole pattern (uniform 404 for missing/cross-tenant, then `assertCompanyAccess` for write-path membership checks) and ~130 handlers across 14 route files use it: ```ts const goal = await getAccessibleResource(req, res, svc.getById(id), "Goal not found"); if (!goal) return; ``` Files: `activity`, `agents`, `approvals`, `assets`, `costs`, `environments`, `execution-workspaces`, `file-resources`, `goals`, `issue-tree-control`, `issues`, `projects`, `routines`, `secrets`. Handlers with bespoke not-found behavior (the legacy `200 []` contract, audit-logged denials in `file-resources`, null-returning authz helpers) compose `hasCompanyAccess` directly using the documented two-step pattern: ```ts // step 1: close the oracle (uniform 404 for both not-found and cross-tenant) if (!existing || !hasCompanyAccess(req, existing.companyId)) { res.status(404).json({ error: "Goal not found" }); return; } // step 2: enforce write-path membership checks for authorised tenants (no-op on GET) assertCompanyAccess(req, existing.companyId); ``` Routes where `companyId` comes from *request input* (`req.params.companyId`, `req.body.companyId`, e.g. in `companies.ts` and `plugins.ts`) deliberately retain plain `assertCompanyAccess` — there's no existence oracle to close because the companyId is an input, not a discovered value. - **Full-sweep coverage** — a scripted audit of every `assertCompanyAccess(req, <resource>.companyId)` call site in `server/src/routes/` found ~55 lookup-then-assert pairs the first pass missed; all are now gated. Notable ones: the `/secret-provider-configs/:id` CRUD routes, the agents instructions-bundle/config-revision/skills-sync routes (which check access via the `assertCanUpdateAgent` / `assertCanReadAgent` / `assertCanManageInstructionsPath` helpers), `POST /heartbeat-runs/:runId/watchdog-decisions`, `GET /issues/:id/cost-summary`, the environment + environment-lease GET routes, all six issue-tree-control routes, ~24 issue sub-resource routes (document annotations, interactions, approvals links, recovery actions, plan decompositions, lock/unlock), and the three workspace file-resource routes (these throw `notFound` instead of `forbidden` inside their audit-logging wrappers, so denied attempts are still activity-logged server-side while the client sees a uniform 404). - **Helpers made self-defending** — `assertCanUpdateAgent` / `assertCanReadAgent` / `assertCanManageInstructionsPath` (agents) and `assertCanManage{Project,Execution}WorkspaceRuntimeServices` throw `notFound` for cross-tenant resources before their `assertCompanyAccess` step, so a future caller that forgets the route-level gate still can't reopen the oracle. - **Pattern enforcement** — new `authz-existence-oracle-guard.test.ts` statically scans `server/src/routes/*.ts` and fails CI on any `assertCompanyAccess(req, <resource>.companyId)` call that is not preceded by a `hasCompanyAccess` gate, with an explicit allowlist (plus staleness check) for the request-input cases. New routes that regress to the 403/404 split fail the suite with a message pointing at the documented pattern. - **Tests** — cross-tenant expectations updated from 403→404 where routes are now gated; new `hasCompanyAccess` unit tests in `authz-company-access.test.ts` pin the instance-admin/local-implicit/agent/none semantics in lockstep with `assertCompanyAccess`; `write-path-membership.test.ts` (added in an earlier round) confirms viewer/inactive users are still rejected on writes. - **One legacy-contract preserve** — `GET /heartbeat-runs/:runId/issues` still returns `200 []` for both "doesn't exist" and "cross-tenant" so the legacy contract is preserved while the oracle stays closed. ## Verification - `pnpm run typecheck` — PASS. - `pnpm -F @paperclipai/server exec vitest run` — full server suite green locally apart from 4 pre-existing local-environment failures (`paperclip-skill-utils` ×2 and `workspace-runtime` ×1 are cwd/git-environment dependent — verified identical on a clean checkout of the base; `heartbeat-process-recovery` is the known macOS flake). - The new `authz-existence-oracle-guard` test sweeps `server/src/routes/*.ts` and confirms no remaining `assertCompanyAccess(resource.companyId)` site without a `hasCompanyAccess` gate; the only allowlisted holdouts take `companyId` from request input. ## Risks - **API contract narrowing.** Any client that specifically checked for `403` on cross-tenant access now sees `404`. This is a strict narrowing (one status instead of two for the same negative outcome) and matches what a client should expect for any id it can't access. - **Write-path checks preserved.** `assertCompanyAccess` still runs after the 404 gate on write routes, so viewer-readonly / inactive-membership rejections fire unchanged for legitimate users. - **Instance-admin scope unchanged.** `hasCompanyAccess` denies signed-in instance admins without an explicit membership, exactly like `assertCompanyAccess` (pinned by unit tests) — so the gate introduces no new read access for admins. - **Single-tenant local-first deploys** behave identically — the helper short-circuits to `true` for `local_implicit` sessions. - No new env vars, no deployment-mode switch. ## Model Used Claude Opus 4.7 (1M context), extended thinking mode; completeness sweep + instance-admin parity fix by Claude Fable 5 (1M context). ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Checked ROADMAP.md — part of the multi-tenant hardening initiative - [x] Tests run locally and pass - [x] Added/updated cross-tenant 404 expectations across test files - [x] No UI changes - [x] Documented risks above - [x] Will address all Greptile and reviewer comments before merge Part of the multi-tenant hardening initiative — see also #5864 (per-company JWT keys) and #5865 (plugin tables `company_id`). --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
1cfed0c0ff |
security(invites): widen invite-token entropy and rate-limit public invite endpoints (#8979)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Companies onboard human members through shareable invite links; the `/api/invites/:token` endpoints are deliberately public so a recipient can view the invite and accept it without being logged in > - That publicness makes the invite token itself the only secret guarding company membership — and it was guessable: the token suffix carried only ~41 bits of entropy, and the endpoints had no rate limiting > - An attacker could therefore enumerate the token space online and accept an invite into someone else's company, gaining member access to its onboarding data, skills, and workspace > - This pull request widens invite tokens to 256 bits of entropy and puts a per-IP rate limit in front of every public `/invites/:token` sub-route > - The benefit is that invite links stop being brute-forceable while their shape, storage scheme, and UX stay exactly the same — existing links keep working ## Linked Issues or Issue Description No public issue exists; describing the problem in-PR (security/bug): **What happens:** Company invite tokens are **public**: anyone with the link can `GET /api/invites/:token`, fetch onboarding/logo/skills, and `POST /api/invites/:token/accept`. Two weaknesses combined to make them brute-forceable: 1. **Token entropy ~41 bits.** The token suffix was 8 chars over a 36-char alphabet (`8 * log2(36) ≈ 41.4` bits). That is online-enumerable. 2. **No rate limit on `/invites/:token*`.** The public endpoints had no throttling, so the ~41-bit space could be enumerated online. **Impact:** an attacker who guesses a live token can accept the invite and join the company as a member — unauthenticated, from any IP. **Expected:** invite tokens should be computationally infeasible to guess, and the public endpoints should throttle guessing attempts anyway (defense in depth). ## What Changed **Entropy** - `createInviteToken` now uses `crypto.randomBytes(32)` (256 bits) base64url-encoded, keeping the human-readable `pcp_invite_` prefix so link shape and UX are unchanged. The duplicate generator in `plugin-host-services.ts` is updated to match. - Tokens are stored **hashed** (sha256) in `invites.tokenHash`; the raw value is only returned once on creation. Storage scheme is unchanged. - **Backward compatible**: only newly minted tokens are affected; lookup is by hash of the presented value, so existing invite links keep working. **Rate limit** - New generic in-memory per-IP sliding-window limiter (`server/src/services/invite-rate-limit.ts`, 20 req/min/IP), applied as a router-level middleware on `/invites/:token` so every current and future sub-route is covered (summary, logo, onboarding, onboarding.txt, skills/index, skills/:name, test-resolution, and POST accept). - Returns `429` with `Retry-After` and `X-RateLimit-*` headers. In-memory ⇒ per-process, which bounds enumeration per replica. Mirrors the existing `company-search-rate-limit` pattern; no new dependency. - Adds a `tooManyRequests(429)` error helper in `server/src/errors.ts`. ## Verification - `invite-token-entropy.test.ts`: prefix preserved, suffix ≥ 128 bits / 22 chars, charset, 1000 unique tokens. - `invite-rate-limit.test.ts`: allows up to limit then 429s with retry-after; per-IP isolation; forgets hits after the window. - `invite-rate-limit-route.test.ts`: `GET /invites/:token` and `POST /invites/:token/accept` return 429 once the per-IP threshold is exceeded. - Manual: create an invite, open the link (works once per token as before), then hammer `GET /api/invites/<token>` >20 times within a minute from one IP → `429` with `Retry-After`. - Server package typechecks clean for all touched files. ## Risks - Low risk. Token change affects only newly minted tokens; existing links resolve via the same sha256-hash lookup. - The limiter is in-memory and per-process: in multi-replica deployments each replica enforces its own 20 req/min/IP budget. That still bounds enumeration (per-replica) and matches the existing `company-search-rate-limit` approach; a shared store can be layered later if needed. - Legitimate users behind a single NAT/proxy IP share the 20 req/min budget for invite endpoints; the invite flow makes only a handful of requests, so headroom is ample. - No DB migration, no API shape change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) — Claude Fable 5 (`claude-fable-5`), extended thinking enabled, agentic tool use (code search, editing, local typecheck) via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Supersedes #8147. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
b4e7ba5143 |
feat(run-logs): durable run-log store via object-storage mirror (#8984)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every agent run streams its stdout/stderr/system output into the run-log store (`server/src/services/run-log-store.ts`), and the run-log API serves those logs back for review and debugging > - The only store implementation is `local_file`: logs live on the server pod's filesystem under `PAPERCLIP_HOME` > - In hardened / ephemeral deployments, `PAPERCLIP_HOME` is an `emptyDir` with no persistent volume, so every pod restart wipes the log files while the DB row still references them — the run-log API then returns "Run log not found" for every completed run after any redeploy > - Run logs are the primary audit/debugging trail for agent work; losing them on routine redeploys undermines trust in the platform > - This pull request adds transparent durability: when `RUN_LOG_S3_BUCKET` is set, the store mirrors each completed log to object storage on `finalize` (same `logRef` key) and falls back to it on `read` when the local file is gone; live append/tail stays on the fast pod-local file > - The benefit is that completed run logs survive pod restarts and redeploys with zero changes for existing deployments (unset bucket = today's behaviour) and zero downstream changes (store id stays `local_file`) ## Linked Issues or Issue Description No existing public issue — inline description following the bug report template: **What happened?** After any server pod restart/redeploy, the run-log API returns "Run log not found" for all previously completed runs. The DB still references the log file, but the file is gone because run logs are written only to the pod-local filesystem. **Expected behavior:** Completed run logs remain readable across pod restarts and redeploys. **Steps to reproduce:** 1. Deploy the server with `PAPERCLIP_HOME` on an `emptyDir` (no persistent volume — common in hardened/ephemeral Kubernetes deployments). 2. Complete an agent run and confirm its log is readable via the run-log API. 3. Restart or redeploy the server pod. 4. Request the same run's log — the API throws "Run log not found". **Paperclip version or commit:** reproducible on current `master`. **Deployment mode:** Kubernetes (server pod without persistent volume). **Agent adapter(s) involved:** Not adapter-specific (core bug). Supersedes #8795. ## What Changed - `server/src/services/run-log-store.ts`: the local-file store becomes a durable store with an optional object-storage mirror - `finalize` mirrors the completed NDJSON log to S3-compatible object storage (keyed by the same `logRef`), best-effort so a failed upload can never break run finalization; upload failures are logged via `console.warn` so operators can detect a persistently broken mirror before a pod roll makes logs unreadable - `read` serves the pod-local file when present and falls back to a ranged object-storage read (with correct `nextOffset`) when the local file is gone - Live `append`/tail stays on the pod-local file — fast path unchanged, no per-chunk PUT - Store id stays `local_file`, so nothing downstream changes (feedback pipeline, read casts, fixtures untouched) - New optional config, all read at store construction: `RUN_LOG_S3_BUCKET`, `RUN_LOG_S3_ENDPOINT`, `RUN_LOG_S3_REGION` (default `us-east-1`), `RUN_LOG_S3_PREFIX` (default `run-logs`), `RUN_LOG_S3_FORCE_PATH_STYLE` (default `true`); credentials via the standard AWS env chain; works with any S3-compatible endpoint - Reuses the existing `createS3StorageProvider`; deliberately independent from `PAPERCLIP_STORAGE_PROVIDER` so enabling durable logs does not redirect workspace/file storage - `server/src/services/run-log-store.test.ts` (new): 7 tests with an in-memory `StorageProvider` mock ## Verification - `npx vitest run src/services/run-log-store.test.ts` in `server/` — 7/7 pass locally: - store id stays `local_file` - live read served from the local file (no S3 round-trip) - `finalize` uploads the completed log to the mirror - read falls back to S3 after a simulated pod roll (local file deleted) - ranged S3 read returns correct slice + `nextOffset` - not-found when neither local nor mirror has the log - local-only safe degrade when no bucket is configured - `npx tsc --noEmit -p server` — clean for the touched files - Manual: set `RUN_LOG_S3_*` against any S3-compatible endpoint (e.g. MinIO), complete a run, delete the local `.ndjson` file, and re-request the log via the run-log API — it is served from the mirror ## Risks - Low risk: with `RUN_LOG_S3_BUCKET` unset (the default), behaviour is byte-for-byte today's local-only store - Mirror upload is best-effort by design — a misconfigured bucket loses durability (not correctness) for affected runs; failures are now surfaced via a `console.warn` per failed upload - No DB migration, no API shape change, no change to the persisted `store`/`logRef` handle format ## Model Used - Claude (Anthropic), model ID `claude-fable-5` (Fable 5), via Claude Code with extended thinking and tool use (code execution, file editing). Original implementation TDD-authored with the same tooling. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
543de323f6 |
fix(shared): tolerate empty-string user name in profile/session parse (#8986)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Humans sign in through the auth layer; every authenticated request
parses the session/user profile with `currentUserProfileSchema` in
`packages/shared`
> - The schema requires `name` to be `null` or a non-empty string, but
some identity providers hand back `name: ""` for users who never set a
display name
> - For those users the session payload fails validation on every
request, so the app treats them as unauthenticated and bounces them to
`/auth` in a loop — they can never get in
> - This pull request preprocesses empty/whitespace-only names to `null`
before validation, so the existing `min(1).max(120).nullable()` rule
still holds for real names
> - Review found the sibling `email` field has the same failure mode
(the DB `auth` schema declares `email` as `notNull`, so a provider that
supplies no email stores `""`, which `z.string().email()` rejects); the
same preprocess is applied there
> - The benefit is that users whose provider reports an empty name (or
email) can sign in normally instead of being locked out, with no change
in behavior for anyone else
## Linked Issues or Issue Description
No existing issue; described in-PR following the bug report template:
**What happened?**
Users whose auth provider returns `name: ""` (empty string) in the
profile payload fail `currentUserProfileSchema` / `authSessionSchema`
parsing (`name: z.string().min(1)...`). The parse failure makes the
session look invalid and the UI redirects to `/auth` on every attempt —
an endless sign-in loop. The `email` field has the same failure mode
(`z.string().email()` rejects `""`).
**Expected behavior:**
An empty display name (or email) should be treated the same as a missing
one (`null`); the user should be signed in normally.
**Steps to reproduce:**
Sign in with an account whose upstream identity record has an
empty-string name (or set a user's `name` column to `''` directly), then
load the app: session parse fails and you are bounced back to `/auth`.
**Adapter(s) involved:**
Not adapter-specific (core bug).
**Deployment mode / version:**
Any; reproduces on current `master`.
## What Changed
- `packages/shared/src/validators/access.ts`:
`currentUserProfileSchema.name` now runs through `z.preprocess` that
coerces empty or whitespace-only strings to `null` before the existing
`z.string().min(1).max(120).nullable()` validation.
- `packages/shared/src/validators/access.ts`: the same preprocess is
applied to `email` (review follow-up): `users.email` is `notNull` in the
DB schema, so a provider without an email stores `""`, which
`z.string().email()` rejects — the identical lockout loop.
Empty/whitespace-only emails now coerce to `null` (the field was already
nullable); malformed non-empty emails are still rejected.
- `packages/shared/src/validators/access.test.ts` (new): covers
empty-string → `null`, whitespace-only → `null`, real values preserved,
`null` preserved, and malformed non-empty email still rejected — for
both `name` and `email`, and the same cases through `authSessionSchema`.
## Verification
- `vitest run src/validators/access.test.ts` in `packages/shared` — 13
tests pass.
- `tsc --noEmit -p packages/shared` passes.
- Manual: parse `{ id, email: "", name: "", image: null }` with
`currentUserProfileSchema` — succeeds with `name: null` and `email:
null` instead of failing validation.
## Risks
- Low risk. The change only widens accepted input (empty/whitespace
string → `null` for `name` and `email`); every previously valid payload
parses identically. `updateCurrentUserProfileSchema` (user-initiated
rename) is untouched and still rejects empty names.
## Model Used
Claude Fable 5 (Anthropic, `claude-fable-5`, agentic coding harness via
Claude Code, extended reasoning enabled). Original fix drafted with
Claude Sonnet 4.6.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Jannes StubbemannandClaude Fable 5
|
4f539625f7 |
build(agent-runtime): ship ripgrep in the base image (#8976)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents execute inside sandboxed runtime containers built from `docker/agent-runtime/Dockerfile.base` > - OpenCode's skill tooling shells out to ripgrep; when `rg` is not on PATH it tries to download a pinned build from `github.com/BurntSushi/ripgrep/releases` at run time > - In a sandbox with locked-down egress that download hangs ~127s and then fails, burning run budget on every agent run before the agent reaches its actual work > - The root repo `Dockerfile` already installs ripgrep; the agent-runtime base image drifted without it > - This pull request adds `ripgrep` to the base image's apt install so OpenCode uses the system binary and never reaches for the network > - The benefit is that every sandboxed agent run stops wasting ~2 minutes on a doomed download and spends its budget on real work ## Linked Issues or Issue Description No existing issue — problem described here per the bug report template: - **What happened:** Sandboxed agent runs using OpenCode stall for ~127 seconds at startup, then log `Transport error ... BurntSushi/ripgrep/releases/download/...` before continuing degraded. - **Expected behavior:** The agent starts working immediately; skill tooling finds `rg` on PATH. - **Root cause:** The agent-runtime base image (`docker/agent-runtime/Dockerfile.base`) does not ship ripgrep, so OpenCode falls back to downloading a pinned build at run time, which egress-restricted sandboxes block. - **Reproduction:** Run any OpenCode-backed agent in a sandbox with locked-down egress using the current agent-runtime image; observe the startup hang and transport error. Supersedes #8859. ## What Changed - Added `ripgrep` to the existing `apt-get install --no-install-recommends` list in `docker/agent-runtime/Dockerfile.base` - Added an explanatory comment documenting why ripgrep must be present (run-time download fallback + egress-restricted sandboxes), restoring parity with the root repo `Dockerfile` ## Verification - `docker build -f docker/agent-runtime/Dockerfile.base .` then `docker run --rm <image> rg --version` — prints the ripgrep version from the system package - Run an OpenCode-backed agent in an egress-restricted sandbox on the new image: no `BurntSushi/ripgrep` download attempt, no ~127s startup stall ## Risks - Low risk: no behavior change beyond shipping one additional apt package in the base image; slightly larger image size > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic), model ID `claude-fable-5` (Fable 5), agentic coding via Claude Code with tool use; original change authored with Claude Opus 4.8 (1M context) and re-based/re-verified with Fable 5 ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
cd1b4f275d |
feat(ui): default to system prefers-color-scheme for first-time visitors (supersedes part of #3732) (#5873)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The UI ships a dark/light theme toggle and persists the user's explicit choice in `localStorage` > - For first-time visitors with no stored choice, the pre-React bootstrap script in `ui/index.html` hardcoded `"dark"` and never consulted the OS preference > - As a result, users on a light-themed OS were forced into dark mode until they clicked the toggle once — a first-impression friction with no compensating benefit > - This pull request makes the bootstrap respect `prefers-color-scheme` for first-time visitors and adds a `matchMedia` listener so the in-app theme auto-follows OS changes until the user makes an explicit choice > - The benefit is a friction-free first visit that matches every other modern web app, with zero impact on users who have already chosen a theme ## Linked Issues or Issue Description No existing issue covers this directly — problem described in-PR (bug shape): - For first-time visitors with no stored theme choice, the pre-React bootstrap script in `ui/index.html` hardcoded `"dark"` and never consulted the OS `prefers-color-scheme` preference. - Users on a light-themed OS were forced into dark mode until they clicked the toggle once — first-impression friction with no compensating benefit. - This PR supersedes part of PR #3732 (the `prefers-color-scheme` bootstrap slice); no standalone issue was filed for it. ## What Changed - **`ui/index.html`** — the pre-React bootstrap script now computes a `prefersDark` fallback via `matchMedia("(prefers-color-scheme: dark)")`, guarded by a `typeof window.matchMedia === "function"` feature-detection check, in place of the hardcoded `"dark"` default. A stored `localStorage` value still takes precedence. - **`ui/src/context/ThemeContext.tsx`** — tracks an `hasExplicitChoice` flag. While `false`, a `MediaQueryList` `change` listener keeps the in-app theme in sync with OS theme switches. Once `setTheme` / `toggleTheme` runs, the choice is persisted and the listener is removed. ## Verification - `pnpm --filter @paperclipai/ui run typecheck` — clean. - `npx vitest run src/components/SidebarAccountMenu.test.tsx` — 1/1 pass (the one test that exercises `ThemeContext`). - Manual: launched the Vite UI dev server with a mocked `/api/auth/get-session` 401, navigated to `/auth` with no `localStorage.theme` set, and toggled the browser's emulated `prefers-color-scheme` between `dark` and `light`. Bootstrap renders the matching theme without a flash. Screenshots committed. **Screenshots — first-visit (no stored theme) with system pref:** | System dark | System light | | --- | --- | <img width="1280" height="800" alt="first-visit-prefers-dark" src="https://github.com/user-attachments/assets/c30274d5-a2e0-4ed8-b7ef-b96b8caac5ca" /> <img width="1280" height="800" alt="first-visit-prefers-light" src="https://github.com/user-attachments/assets/ebd8e9cf-0be1-4b36-a6f8-eb38b00dff5c" /> (Without this PR, both screenshots would have rendered dark.) ## Risks Low. Purely additive: - A stored `localStorage` value still takes precedence over the OS preference, so users who have already picked a theme are unaffected. - SSR-safe: both new code paths are gated on `typeof window !== "undefined"` (the inline script only runs in the browser, and the React `matchMedia` listener is attached inside `useEffect`). - No new dependencies, no API surface change. ## Model Used Claude Opus 4.7 (1M context), extended thinking mode. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Checked ROADMAP.md — not in conflict with planned core work - [x] Tests run locally and pass - [x] No new test cases — the change is observable only via real `matchMedia` events, which jsdom does not faithfully implement; manual verification above - [x] UI change — before/after screenshots in `docs/pr-screenshots/pr-5873/` - [x] No documentation updates required - [x] Documented risks above - [x] Will address all Greptile and reviewer comments before merge Supersedes part of #3732 (the `prefers-color-scheme` bootstrap slice). --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Devin Foley <devin@paperclip.ing> |
||
|
|
8ddd735a7a |
feat(ui): theme toggle on unauthenticated auth page (supersedes part of #3732) (#5874)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Operators sign in via the `/auth` page, which renders before any session exists > - The signed-in app has a theme toggle inside `SidebarAccountMenu`, but the signed-out `/auth` page has none — first-time visitors are stuck in whichever theme was hardcoded at boot > - Master's existing toggle was inline inside `SidebarAccountMenu.tsx` as a `MenuAction` row, not exported as a reusable widget; round-1 of this PR added a standalone `ThemeToggle` but punted on unifying the two surfaces > - This pull request makes `ThemeToggle` the canonical theme widget (one source of truth for label, icon, and toggle behaviour), used both as a compact icon button on `/auth` and as a full-width menu row in `SidebarAccountMenu` > - The benefit is a working pre-auth theme switch and zero risk of the two call sites drifting out of sync as the theme model evolves ## Linked Issues or Issue Description No existing issue covers this directly — problem described in-PR (feature-gap shape): - The signed-in app has a theme toggle inside `SidebarAccountMenu`, but the signed-out `/auth` page has none — first-time visitors are stuck in whichever theme was hardcoded at boot. - The existing toggle lived inline in `SidebarAccountMenu.tsx` as a `MenuAction` row, not exported as a reusable widget, so the two surfaces could drift apart as the theme model evolves. - This PR supersedes part of PR #3732 (the auth-page toggle slice); no standalone issue was filed for it. Duplicate-PR search: related open theme PRs #2769 and #4666 add in-app three-state system-theme toggles — different surface from this PR (unauthenticated auth page); sibling PR in this series: #5873. ## What Changed - **`ui/src/components/ThemeToggle.tsx`** — accepts `variant: "icon" | "menu-action"` (default `"icon"`) and an `onAfterToggle` callback. Both variants share `useTheme` and the same label/icon derivation. The `menu-action` variant matches the existing `MenuAction` row styling. - **`ui/src/components/SidebarAccountMenu.tsx`** — drops its inline `useTheme()` + `MenuAction`-for-the-theme-row in favor of `<ThemeToggle variant="menu-action" onAfterToggle={() => setOpen(false)} />`. Sun/Moon icon imports and theme state move with it. - **`ui/src/pages/Auth.tsx`** — unchanged from round 0; renders `<ThemeToggle />` at top-right of the `/auth` page (already using the default `icon` variant). - **`ui/src/components/ThemeToggle.test.tsx`** (new) — covers both variants, the `onAfterToggle` callback, and the label/icon flip across themes. - **`ui/src/components/SidebarAccountMenu.test.tsx`** — unchanged; its `ThemeContext` mock still works because `ThemeToggle` uses the same hook. ## Verification - `pnpm --filter @paperclipai/ui run typecheck` — clean. - `npx vitest run src/components/ThemeToggle.test.tsx src/components/SidebarAccountMenu.test.tsx` — 5 passed (4 new + 1 existing). - Manual: launched `pnpm dev` for the UI, mocked `/api/auth/get-session` 401, navigated to `/auth` — toggle is visible top-right, click flips light/dark. Screenshots committed. ## Risks Low. The change is structural — both call sites render the same widget that already worked on each surface independently. `SidebarAccountMenu`'s popover behaviour is preserved via `onAfterToggle`, and `ThemeContext` is untouched. ## Model Used Claude Opus 4.7 (1M context), extended thinking mode. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Checked ROADMAP.md — not in conflict with planned core work - [x] Tests run locally and pass - [x] Added tests for the new ThemeToggle component (both variants) - [x] UI change — before/after screenshots in `docs/pr-screenshots/pr-5874/` - [x] No documentation updates required (purely internal refactor + new component) - [x] Documented risks above - [x] Will address all Greptile and reviewer comments before merge Supersedes part of #3732. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Devin Foley <devin@paperclip.ing> |
||
|
Jannes StubbemannandClaude Opus 4.7
|
362c30ccdc |
feat(server): opt-in OpenTelemetry auto-instrumentation (#3735)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Production self-hosters increasingly expect telemetry out of the box — Jaeger, Tempo, Honeycomb, Datadog, Grafana Cloud, Dynatrace all speak OTLP > - Today there is no OpenTelemetry bootstrap in the server, so operators who want traces have to patch their fork or run a sidecar that captures only HTTP-level info > - An opt-in bootstrap that costs nothing when disabled is the minimum-viable surface for this audience > - The OpenTelemetry packages are heavyweight enough that we don't want them in the default dependency graph — they should load only when the operator configures an OTLP endpoint > - This pull request adds a self-contained `server/src/instrumentation.ts` that dynamically imports the OTel SDK and starts it when `OTEL_EXPORTER_OTLP_ENDPOINT` is set, and is a complete no-op otherwise ## Linked Issues or Issue Description No existing issue covers this directly — feature-gap description following the feature-request template: **Problem or motivation** Production self-hosters increasingly expect telemetry out of the box — Jaeger, Tempo, Honeycomb, Datadog, Grafana Cloud, Dynatrace all speak OTLP — but the server has no OpenTelemetry bootstrap. Operators who want traces today must patch their fork or run a sidecar that captures only HTTP-level information. **Proposed solution** An opt-in OTel bootstrap gated on `OTEL_EXPORTER_OTLP_ENDPOINT`, loaded via dynamic `import()` only when configured, so the heavyweight OTel packages stay out of the default dependency graph. **Alternatives considered** Related open PRs found during the duplicate-PR search approach observability differently: #4894 adds OTLP instrumentation to Paperclip core unconditionally, and #3752 proposes an observability plugin. Not duplicates — different layering: this PR keeps the default install dependency-free via opt-in dynamic import. ## What Changed - New `server/src/instrumentation.ts` — opt-in OpenTelemetry auto-instrumentation. Gated on `OTEL_EXPORTER_OTLP_ENDPOINT`. Respects the standard OTel env vars (`OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`). Skips the fs/dns/net auto-instrumentations (too chatty). `sdk.start()` is wrapped in try/catch so a bad endpoint or missing native bindings doesn't crash the server. `process.once("SIGTERM" / "SIGINT", …)` for clean shutdown on the first signal only. OTel packages are loaded via dynamic `import()` so they are true optional runtime dependencies — no entries in `package.json`, no lockfile churn. - `server/src/index.ts` — import `./instrumentation.js` as the very first statement so auto-instrumentation can patch `http` / `express` / `pg` before they are evaluated by downstream modules. ## Verification - `OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 pnpm start` after `pnpm install @opentelemetry/{sdk-node,auto-instrumentations-node,exporter-trace-otlp-grpc,resources,semantic-conventions}` in `server/` — traces show up in the configured collector; HTTP, Express, and Postgres spans are populated. - `OTEL_EXPORTER_OTLP_ENDPOINT` unset — server starts with no OTel-shaped output in logs, no behavior change. - `OTEL_EXPORTER_OTLP_ENDPOINT=…` set but packages not installed — single `console.warn` at startup telling the operator which packages to install. ## Risks Low. No behavior change unless the env var is set. The bootstrap never throws into the caller; every failure path ends in `console.warn` / `console.error` and falls through to non-traced operation. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.7
|
937fe62d10 |
feat(server): TRUST_PROXY supports CIDR list + named subnets (supersedes #3729) (#5872)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Express looks at incoming `X-Forwarded-For` headers only when
`app.set("trust proxy", …)` says it should, and uses that resolved
client IP downstream for rate-limiting, audit logging, and any
auth/abuse signal that ties back to source IP
> - The original PR #3729 added `TRUST_PROXY` accepting only `"true"` or
a positive integer, which forces operators to pick between two unsafe
defaults: hop-count (brittle if topology changes) or boolean-true (any
client can spoof `X-Forwarded-For` and bypass rate-limits or pollute
audit logs)
> - `trust proxy: true` is one of the most common Express
misconfigurations and trivially exploitable for IP-spoofing-based
rate-limit bypass; the safest config — trust only the LB's actual CIDR
or only loopback — was unreachable with the previous parser
> - This pull request replaces the parser with full Express 5 support —
unset / `false` / `0` (Express default), positive integer hop count,
comma-separated CIDR list, named subnets (`loopback`, `linklocal`,
`uniquelocal`) — and emits a startup error naming the offending token on
invalid input
> - The benefit is that operators can now trust *only* their actual
ingress and close the spoofing window without leaking client-IP
integrity to downstream layers, while preserving every
previously-working config as a strict superset
## Linked Issues or Issue Description
Refs #1690 — login returns 500 behind a reverse proxy because Express
`trust proxy` is not enabled; this PR ships the configuration surface
(`TRUST_PROXY` with CIDR lists and named subnets) that lets operators
enable it safely. It does not change the default, so #1690 still
requires the operator to set `TRUST_PROXY` — hence Refs, not Fixes.
No other existing issue covers this directly — remaining problem
described in-PR:
- The original `TRUST_PROXY` parser (PR #3729, which this PR supersedes)
accepted only `"true"` or a hop count, forcing operators to choose
between brittle hop-counting and the spoofable `trust proxy: true`.
- `trust proxy: true` lets any client spoof `X-Forwarded-For` and bypass
rate limits or pollute audit logs; the safest config — trusting only the
LB's actual CIDR or only loopback — was unreachable with the previous
parser.
Duplicate-PR search: #1854 / #1714 are earlier minimal trust-proxy
enablement PRs; this PR supersedes #3729 and generalizes beyond a
boolean enable (CIDR lists + named subnets).
## What Changed
- **`server/src/middleware/trust-proxy.ts`** — new helper exposing
`parseTrustProxyEnv` (testable) and `applyTrustProxy(app)` (one-call
boot wiring). Surface:
- Unset / `""` / `false` / `0` → no `app.set("trust proxy", …)` (Express
default: trust nothing).
- `true` → `app.set("trust proxy", true)`. Documented as unsafe in
untrusted-LB deployments.
- Positive integer (e.g. `"2"`) → hop count. Strict parse: rejects
`"01"`, leading/trailing whitespace.
- Comma-separated list of CIDRs and/or named subnets (e.g.
`"loopback,uniquelocal,10.0.0.0/8,fd00::/8"`) → array passed to
`app.set("trust proxy", [...])`.
- Anything else → startup error naming the offending token.
- **`server/src/app.ts`** — one import + one call to
`applyTrustProxy(app)`.
- **`server/src/__tests__/trust-proxy.test.ts`** — 12 cases: unset,
`"true"`, `"0"`, `"2"`, `"01"` rejected, `" 2 "` rejected, `"loopback"`,
`"loopback,uniquelocal"`, `"10.0.0.0/8"`, `"10.0.0.0/8,fd00::/8"`,
`"bogus"` rejected (error names the bad token), mixed-list with one bad
token rejected (error names the offending token specifically).
## Verification
- `pnpm --filter @paperclipai/server run typecheck` — clean.
- `npx vitest run trust-proxy` — 12/12 pass.
## Risks
- **No new required env vars.** Unset means default Express behavior
(trust nothing). Pure superset of #3729's surface — anything that worked
under #3729 still works here.
- **Strict parse.** `"01"` and `" 2 "` are rejected on purpose so
configuration mistakes surface at startup, not as silently-degraded
auth/rate-limit behavior. The error message names the offending token.
- **No runtime cost** — the parse runs once at boot. The downstream
`trust proxy` setting is internal to Express.
- Single-tenant local-first deploys unaffected by default.
## Model Used
Claude Opus 4.7 (1M context), extended thinking mode.
## Checklist
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] Thinking path traces from project context to this change
- [x] Model used specified
- [x] Checked ROADMAP.md — not in conflict with planned core work
- [x] Tests run locally and pass (`trust-proxy` 12/12)
- [x] Added boundary cases (leading-zero, whitespace, unknown token,
mixed-list-with-bad-token)
- [x] No UI changes
- [x] Documented risks above
- [x] Will address all Greptile and reviewer comments before merge
Closes #3729.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Jannes StubbemannandClaude Opus 4.7
|
3701be76fa |
fix: read-only agent config/skill endpoints should not require agents:create (#3725)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Access to agents, skills, and configurations is governed by a per-company permission system > - `agents:create` is a mutation-tier permission that controls who can create or modify agents > - `assertCanReadConfigurations` delegates to `assertCanCreateAgentsForCompany`, effectively requiring `agents:create` just to *read* agent configs, skills, and config revisions > - That's a permission regression: any company member without `agents:create` hits 403 on the Skills tab, agent config pages, and revision history — but those responses are already secret-redacted > - This pull request loosens the read gate to company membership only, while keeping every mutation-adjacent gate at `agents:create` ## Linked Issues or Issue Description No existing issue covers this directly — problem described in-PR following the bug-report template: **What happened** `assertCanReadConfigurations` delegates to `assertCanCreateAgentsForCompany`, effectively requiring the mutation-tier `agents:create` permission just to *read* agent configs, skills, and config revisions. Any company member without `agents:create` hits 403 on the Skills tab, agent config pages, and revision history — even though those responses are already secret-redacted (`redactAgentConfiguration`, `redactConfigRevision`). **Expected behavior** Read-only configuration/skill/revision endpoints are readable by any company member; only mutation-adjacent endpoints require `agents:create`. **Steps to reproduce** As a company member without `agents:create`, open the Skills tab or an agent config page (or `GET` the config/skill/revision endpoints) — the request fails with 403. ## What Changed - `server/src/routes/agents.ts`: - `assertCanReadConfigurations` now requires company membership only (plus the existing agent-key cross-company check). Previously it required `agents:create`. - `actorCanReadConfigurationsForCompany` (the boolean twin, used by `GET /agents/:id` to decide whether to return a restricted detail) now uses the standard try/catch-around-`assertCompanyAccess` pattern. - `POST /companies/:companyId/adapters/:type/test-environment` is not a pure read (it exercises adapter secrets) and now calls `assertCanCreateAgentsForCompany` directly instead of going through `assertCanReadConfigurations`. Behavior for this endpoint is unchanged. ## Verification - Existing tests pass. - Manual: log in as a company member without an `agents:create` grant. Visit the Skills tab on an agent and the agent configuration panel — both load. Try to edit the agent — blocked, as before. - Manual: POST to `/companies/:companyId/adapters/:type/test-environment` as the same user — still 403. ## Risks Low. The only behavior change is on read endpoints whose responses were already redacted (\`redactAgentConfiguration\`, \`redactConfigRevision\`). No secret escapes anywhere. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.6
|
c21f70ef1c |
fix: skip gosu when already running as target user (#2908)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The reference container image must be deployable on both Docker Compose (where it starts as root and `gosu`'s a `USER_UID`/`USER_GID` switch) and Kubernetes (where the pod is typically constrained by PodSecurity) > - The Kubernetes operator (paperclipinc/paperclip-operator#45) sets `runAsNonRoot: true`, `runAsUser: 1000`, `allowPrivilegeEscalation: false`, and `drop: ALL` capabilities by default — the unconditional `usermod` + `gosu` flow in the entrypoint requires root + `CAP_SETUID` / `CAP_SETGID`, making the image undeployable on any cluster enforcing baseline or restricted PodSecurity > - Without root, neither the user remap nor `gosu` can ever succeed — so the fix is a runtime branch: non-root starts `exec` the command directly (warning if the runtime UID/GID differs from the requested one), while root starts keep the existing `usermod`+`gosu` flow > - This also covers platforms that assign arbitrary UIDs (OpenShift restricted SCC), which previously crashed with a cryptic `usermod: Permission denied` > - The benefit is one image that works for both deployment shapes with no operator-side workaround — pure superset, no breaking change ## Linked Issues or Issue Description Refs paperclipinc/paperclip-operator#45 (cross-repo) — the operator's default pod security context (`runAsNonRoot: true`, `runAsUser: 1000`, `allowPrivilegeEscalation: false`, `drop: ALL`) is blocked by this entrypoint behavior. The operator shipped a stopgap (paperclipinc/paperclip-operator#46 lets the CRD override the security context); this PR is the image-side fix that makes the secure defaults work out of the box. Supersedes #2904 (v1 of this branch). No in-repo issue covers this directly — problem described in-PR following the bug-report template: **What happened** The entrypoint unconditionally runs `usermod`/`groupmod`/`chown` + `exec gosu node`, which requires root plus `CAP_SETUID` / `CAP_SETGID` — any non-root start crashes (`gosu` cannot drop privileges; a mismatched UID dies earlier at `usermod: Permission denied`), making the reference image undeployable on clusters enforcing baseline or restricted PodSecurity. **Expected behavior** A non-root container `exec`s the command directly (with a clear warning if its UID/GID differs from the requested `USER_UID`/`USER_GID`, since a remap is impossible without root). The existing root + `usermod` + `gosu` flow is preserved for Docker Compose, where the container starts as root and switches to the requested UID/GID. **Deployment mode** Kubernetes with baseline/restricted PodSecurity and OpenShift-style arbitrary-UID platforms (failing cases); Docker Compose root-start (must keep working). ## What Changed - **`scripts/docker-entrypoint.sh`** — branch on the runtime UID: - **Non-root start** → `exec "$@"` directly. If the runtime UID/GID differs from `USER_UID`/`USER_GID`, print a one-line warning to stderr first (the remap is impossible without root; the warning keeps volume-permission mismatches diagnosable instead of failing cryptically inside `usermod`). - **Root start** → unchanged: `usermod`/`groupmod` remap when requested, `chown` of `/paperclip` when a remap happened, then `exec gosu node "$@"`. ## Verification **Automated:** `server/src/__tests__/docker-entrypoint.test.ts` runs the real entrypoint with `id`/`usermod`/`groupmod`/`chown`/`gosu` stubbed via PATH and asserts all five privilege branches (root+defaults, root+remap, non-root match, arbitrary non-root UID, GID mismatch) — runs in the regular server suite, no Docker needed. **Manual (Docker):** ran the entrypoint on `node:lts-trixie-slim` (the image's actual base) across the full matrix, with `gosu` stubbed to a marker: - [x] Root start, defaults → no remap, `gosu node` invoked (Docker Compose flow unchanged) - [x] Root start, `USER_UID=1001`/`USER_GID=1001` → `Updating node UID/GID to 1001` + `gosu node` (remap flow unchanged) - [x] Non-root `--user 1000:1000` (the operator's `runAsUser: 1000` shape) → silent direct exec, command runs as 1000:1000 - [x] Non-root `--user 1234:1234` (arbitrary UID) → warning `running unprivileged as 1234:1234; cannot remap to requested 1000:1000`, then direct exec (previously: crash) - [x] Non-root `--user 1000:1001` (GID mismatch) → warning, then direct exec - [x] Baseline check: master's entrypoint fails for any non-root start (gosu/usermod require root) End-to-end cluster verification under restricted PodSecurity exercises the same branch as the `--user 1000:1000` case above; the operator repo's deploy is the natural place for that smoke test once this ships in an image tag. ## Risks - **Backward-compatible.** Docker Compose / root-entrypoint path is byte-for-byte the same flow — `usermod`/`gosu` runs whenever the container starts as root. - **Behavior change only for previously-broken starts.** Non-root containers used to crash; they now run. The only observable difference for a *working* deployment is none. - **Mismatched non-root UID/GID warns instead of failing.** Deliberate: the remap is impossible without root, and arbitrary-UID platforms (OpenShift) rely on group-writable volumes; a hard fail would keep them broken. The stderr warning preserves diagnosability. - **No new env vars, no API surface.** Pure entrypoint behavior change gated on the runtime UID. - **Restricted PodSecurity ready.** The non-root branch needs no Linux capabilities — works under `drop: ALL`. ## Model Used Claude Opus 4.6; rebase, non-root generalization, and verification matrix by Claude Fable 5 (1M context). ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Checked ROADMAP.md — not in conflict with planned core work (agent-runtime sandbox images use `tini`, no gosu — unaffected) - [x] Tests run locally and pass (new `docker-entrypoint.test.ts` covering all five privilege branches, plus a manual Docker matrix on the real base image; see Verification) - [x] No UI changes - [x] Documented risks above - [x] Will address all Greptile and reviewer comments before merge Unblocks the default (non-overridden) security context of paperclipinc/paperclip-operator#45 / #46. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
05bcd3ce84 |
feat(security): plugin tables get company_id FK for tenant isolation (#5865)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The plugin subsystem persists state into four tables (`plugin_entities`, `plugin_job_runs`, `plugin_logs`, `plugin_webhook_deliveries`) and those rows currently have no notion of an owning tenant — so company-deletion doesn't cascade plugin state, and operators have no way to query "what does this plugin own for company X?" > - The fix is one thin slice of tenant-isolation hygiene that doesn't change any external API: add a nullable `company_id` FK with `ON DELETE CASCADE` to the four plugin tables, index it, and scope the `plugin_entities` external-id uniqueness per-tenant > - The benefit is plugin-row tenant attribution + cascade cleanup, with zero impact on single-tenant local-first deploys (`NULL` continues to mean instance-scope) > **Rebase note (scope narrowed):** This PR originally also hardened the schedulers (`heartbeat.tickTimers` / `resumeQueuedRuns` / `enqueueWakeup` and `routines.tickScheduledTriggers`) to skip archived companies. That half has since landed on `master` via #7478 (`93206f73`, "Stop archived companies from waking agents") with a stricter implementation (`status != 'active'` plus a skipped-request audit row). On rebase those changes were dropped as redundant — `heartbeat.ts` and `routines.ts` are now identical to `master`, and the scheduler-specific tests were removed. **This PR is now DB-only.** ## Linked Issues or Issue Description No existing issue covers this directly — problem described in-PR: - Four plugin tables (`plugin_entities`, `plugin_job_runs`, `plugin_logs`, `plugin_webhook_deliveries`) persist rows with no notion of an owning tenant. - Company deletion therefore does not cascade plugin state, and operators have no way to query "what does this plugin own for company X?" - One thin slice of tenant-isolation hygiene fixes this without changing any external API: a nullable `company_id` FK with `ON DELETE CASCADE`, an index, and per-tenant scoping of the `plugin_entities` external-id uniqueness. - Part of the multi-tenant hardening initiative alongside #3967 (cross-tenant 404 oracle) and #5864 (per-company JWT keys). ## What Changed **Schema (`packages/db/src/schema/plugin_*.ts`):** - Nullable `companyId` FK with `onDelete: "cascade"` added to `plugin_entities`, `plugin_job_runs`, `plugin_logs`, `plugin_webhook_deliveries`. - A btree index on each new `company_id` column (`<table>_company_idx`). - `plugin_entities_external_idx` rescoped from `(plugin_id, entity_type, external_id)` to `(company_id, plugin_id, entity_type, external_id)` and switched to `UNIQUE … NULLS NOT DISTINCT` so instance-scope rows (`company_id IS NULL`) keep their dedup guarantee while tenants get their own namespace. **Migration:** - `0095_plugin_company_id_tenant_isolation.sql` — 14 statements: 4 column adds + 4 FK constraints (`ON DELETE CASCADE`) + 4 indexes + drop/recreate of the external-id unique constraint. - Journal entry + regenerated `0095_snapshot.json`. **Tests:** - `server/src/__tests__/plugin-tenant-isolation.test.ts` — `NULL` preserves backward compat; `CASCADE` on company delete across all four tables; per-tenant external-id namespacing; NULL-NULL collision rejected (`NULLS NOT DISTINCT`). ## Verification - `pnpm --filter @paperclipai/db run check:migrations` — pass. - `pnpm --filter @paperclipai/db typecheck` (`tsc`) — pass. - `vitest run plugin-tenant-isolation` — **4/4 pass** (embedded Postgres applies `0095` and exercises cascade + NULLS NOT DISTINCT). ## Notes - **Clean snapshot, no drift.** The earlier revision of this PR shipped a ~17.6k-line meta snapshot that was almost entirely pre-existing drift. On rebase the migration was renumbered (the old `0090_brainy_darkhawk` collided with `master`'s `0090_resource_memberships … 0094`) and regenerated from the current `master` baseline via `drizzle-kit generate`. The result is a 14-statement migration containing **only** the plugin-table changes — no unrelated drift. - **Backward-compatible.** `NULL company_id` continues to mean instance-scope (cron jobs, public webhooks). No new env vars, no API surface change. Single-tenant local-first deploys unaffected. ## Risks - **Migration is additive and nullable** — `0095` adds nullable `company_id` columns, FK constraints, and indexes; existing rows stay valid (`NULL` keeps meaning instance-scope) and no backfill is required. - **`ON DELETE CASCADE` is a behavioral change**: deleting a company now removes its plugin rows across all four tables. Intended (it is the point of the PR), but operators relying on plugin rows surviving company deletion would be affected. Covered by the cascade tests. - **Uniqueness semantics change on `plugin_entities`**: the external-id constraint is rescoped per-tenant and switched to `UNIQUE … NULLS NOT DISTINCT`, so two instance-scope rows (`company_id IS NULL`) with the same external id are now rejected instead of coexisting. Covered by the NULL-NULL collision test. - **No API surface change, no new env vars.** Single-tenant local-first deploys unaffected. (Section added retroactively to match the PR template; distilled from the What Changed / Notes sections above.) ## Model Used Same authoring setup as #5864 (same series, same day): Claude Opus 4.7 (1M context), extended thinking mode. (Section added retroactively.) ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Tests run locally and pass (plugin-tenant-isolation 4/4) - [x] `check:migrations` + db typecheck pass - [x] No UI changes - [x] Migration carries only the intended changes (no snapshot drift) - [x] Scheduler half dropped as superseded by #7478 Part of the multi-tenant hardening initiative — see also #3967 (cross-tenant 404 oracle) and #5864 (per-company JWT keys). --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Devin Foley <devin@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes StubbemannandClaude Opus 4.7
|
70357b961f |
feat(security): per-company JWT signing keys for multi-tenant isolation (#5864)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents authenticate to the server with a JWT signed by the deployment's master secret > - In a multi-tenant deployment, all agents from every tenant are signed with the *same* key, so a leak (CI/staging dump, hostile contractor with infra access, supply-chain) lets the attacker mint tokens for *any* tenant > - The same master secret also issued tokens with a 48-hour TTL, giving any leaked token a two-day window of validity even after rotation > - This pull request derives a per-company signing key via `HMAC-SHA256(master, "jwt:<companyId>")` and reduces the default TTL to 1h; the verifier tries the per-company key first and falls back to the master secret only for tokens issued before this change so no agent gets locked out on deploy > - The benefit is multi-tenant key isolation (a leak of one company's derived key cannot forge tokens for another) and a tighter blast-radius on any leaked token, with zero local-first impact (single-tenant deploys derive their one company's key the same way and continue to work unchanged) ## Linked Issues or Issue Description Refs #5288 — a separate key-hygiene finding in the same module (`agent-auth-jwt.ts` falls back to `BETTER_AUTH_SECRET` as the JWT signing secret). Related agent-JWT trust-model concern, but not fixed by this PR — the master-secret fallback selection is unchanged here. No existing issue covers this PR's problem directly — described in-PR: - In a multi-tenant deployment, agents from every tenant get JWTs signed with the *same* master key, so a single leak (CI/staging dump, hostile contractor, supply chain) lets the attacker mint tokens for *any* tenant. - The same master secret issued tokens with a 48-hour TTL, giving any leaked token a two-day validity window even after rotation. - Fix: derive a per-company signing key via `HMAC-SHA256(master, "jwt:<companyId>")` and reduce the default TTL to 1h, with a master-secret verification fallback so pre-existing tokens are not locked out on deploy. ## What Changed - **`server/src/agent-auth-jwt.ts`** - New `deriveCompanySigningKey(masterSecret, companyId)` — `HMAC-SHA256` with domain-separated input (`jwt:<companyId>`) so the master secret can be safely reused for other HMAC purposes in the future without cross-protocol risk. - `signAgentJwt` always signs with the derived per-company key. - `verifyAgentJwt` reads `company_id` from the token's (untrusted) claim payload, looks up the candidate derived key, and verifies. If that fails AND a master secret is set, it falls back to verifying with the raw master secret — pre-existing tokens validate until they expire. Verification still fails if the signature doesn't bind. - Default TTL: `60 * 60 * 48` → `60 * 60`. Existing `PAPERCLIP_AGENT_JWT_TTL_SECONDS` override still wins. - **`PAPERCLIP_AGENT_JWT_DISABLE_LEGACY_FALLBACK`** (optional, default off) — operators set this ~one TTL after deploying to sunset the master-secret verification fallback entirely, closing the window in which a leaked master secret could forge arbitrary-`exp` tokens for any tenant. - **`server/src/__tests__/agent-auth-jwt.test.ts`** (6 new cases) - Per-company isolation via tamper: token for company A fails when verified for company B. - Legacy-token verification path: tokens signed with the raw master secret still verify. - Default TTL is 1h. - Legacy fallback toggle: master-secret tokens accepted when unset, rejected when enabled, and per-company tokens unaffected either way. ## Verification - `pnpm --filter @paperclipai/server run typecheck` — clean. - `npx vitest run agent-auth-jwt` — 11/11 pass (6 new + 5 existing). - Manual: token signed for company A under per-company key fails when verified against company B's derived key. ## Risks - **Backward-compatible verification**, so no agent gets locked out on deploy — but operators relying on hot-swapping the master secret should note that pre-existing tokens *will* keep validating against the master key until their TTL elapses, unless `PAPERCLIP_AGENT_JWT_DISABLE_LEGACY_FALLBACK=true` is set to end the fallback window explicitly. - **TTL reduction is a default, not a hard cap.** Operators who relied on the 48h window can override via env. If 1h is too aggressive for upstream taste, happy to gate the change behind an env var. - **No new required env vars.** Single-tenant local-first deploys derive one company's key the same way and behave identically to today. - **Domain-separated HMAC input** (`jwt:<companyId>`) means the master secret can be safely reused for other future HMAC purposes without cross-protocol risk. ## Model Used Claude Opus 4.7 (1M context), extended thinking mode; rebase + legacy-fallback sunset documentation by Claude Fable 5 (1M context). ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Checked ROADMAP.md — part of the multi-tenant hardening initiative - [x] Tests run locally and pass (`agent-auth-jwt` 11/11) - [x] Added per-company-isolation, legacy-fallback, and TTL-default tests - [x] No UI changes - [x] Documented risks above - [x] Will address all Greptile and reviewer comments before merge Part of the multi-tenant hardening initiative — see also #3967 (cross-tenant 404 oracle) and #5865 (plugin tables `company_id`). --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
606e74d11f |
cloud_tenant: company-scoped tenants, never instance-admin (#7525)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies, and a single server instance can host many companies. > - The auth middleware (`server/src/middleware/auth.ts`) supports a `cloud_tenant` mode where a trusted hosting proxy injects per-request identity headers, designed originally for one-deployment-per-tenant setups. > - In that original setup, granting every cloud tenant the `instance_admin` role was harmless; on a **shared, multi-tenant pool** it means any paying tenant is admin of the whole instance and can reach every other tenant's data. > - A tenant only needs to own its own company — which it already gets via the company membership the same code path upserts — so instance-level admin is never appropriate for `cloud_tenant` actors. > - This PR removes the `instance_admin` grant from the cloud-tenant path and pins `isInstanceAdmin: false` on the resolved actor. > - Greptile review then surfaced a follow-up gap: deployments that ran the pre-hardening build still have stale `instance_admin` rows in `instance_user_roles`, which other lookups (BetterAuth session path, board API keys, and the authorization service's own DB re-check) would still honor. > - The follow-up commit closes that gap by purging stale rows at the cloud-tenant auth boundary and by teaching the authorization service that `cloud_tenant` actors are never instance admins. > - The benefit is that shared-pool hosting becomes structurally safe: tenants are company-scoped owners, never instance admins — including on deployments upgrading from the older behavior. ## Linked Issues - Refs #966 — managed SaaS multi-tenant hosting is the deployment shape this hardening protects. - Refs #5015 — same problem space: instance-admin-scoped credentials are too broad for multi-company instances; tenants need company-scoped access. Neither issue is fully closed by this PR; it removes the instance-admin grant from the `cloud_tenant` trusted-header path specifically. ## What Changed - `server/src/middleware/auth.ts` - Removed the `instanceUserRoles` insert that granted every cloud tenant `instance_admin`; `resolveCloudTenantActor` now returns `isInstanceAdmin: false` (was `true`). - `resolveCloudTenantActor` now **deletes** any stale `instance_admin` row for the authenticated tenant user on every trusted-header request, so grants left behind by pre-hardening deployments are purged at the source (closes the Greptile P2: stale rows could otherwise re-elevate the user via the BetterAuth session path, board API keys, or the authorization service). - The function is `export`ed so it can be unit-tested directly. - `server/src/services/authorization.ts` - `authorizationService` previously re-checked `instanceUserRoles` from the DB regardless of the actor flag, which would have elevated even hardened `cloud_tenant` actors while a stale row lingered. Actors with `source === "cloud_tenant"` are now never elevated to instance admin; other board actors keep the existing lookup. - `server/src/services/authorization.ts` + `server/src/middleware/auth.ts` (follow-up commit `dc57a71c7`) - CI on the merge ref surfaced that elevation removal alone strands real cloud tenant users: board actors only ever reached `issue:read` / `issue:mutate` through instance-admin elevation (`permissionForAction` maps both to no grant key). `decide()` now grants `cloud_tenant` actors with an **active membership in the resource company** the same read surface as a same-company agent (`agent:read`, `company_scope:read`, `issue:read`, `project:read`) plus `issue:mutate` for non-viewer members — cross-company access stays denied (new `allow_company_member` reason). - `resolveCloudTenantActor` seeds the standard role-default permission grants (`ensureHumanRoleDefaultGrants`) so granted actions (e.g. `tasks:assign`, `agents:create` for owners) work without elevation. - Master-side route tests that stubbed cloud tenant actors with `isInstanceAdmin: true` now seed a real membership and assert under the hardened contract (`issue-identifier-routes`, `multilingual-issues-routes`, `issue-comment-redaction`). - Tests - `server/src/middleware/cloud-tenant-actor.test.ts` (new): cloud tenant is never instance-admin, is scoped to exactly the one company from its stack, still upserts user/company/membership, purges stale `instance_admin` rows, returns null without the server token, and maps non-owner stack roles without elevating. - `server/src/__tests__/auth-session-route.test.ts`: end-to-end middleware regression — a user with a stale `instance_admin` row stops being elevated via the session path once they authenticate through the cloud-tenant path (with a control assertion showing the pre-purge elevation). - `server/src/__tests__/authorization-service.test.ts` (embedded Postgres): a `cloud_tenant` actor with a stale `instance_admin` row in the real DB cannot cross company boundaries, while a `session` actor with the same row still resolves `allow_instance_admin`. ## Verification Run from the repo root after `pnpm install --frozen-lockfile`: ```bash cd server npx vitest run src/middleware/cloud-tenant-actor.test.ts src/__tests__/auth-session-route.test.ts # 9 tests passed npx vitest run src/__tests__/authorization-service.test.ts # 16 tests passed (embedded Postgres) pnpm typecheck # clean ``` Also ran the broader auth-related suites locally (`auth-routes`, `authz-company-access`, `better-auth`, `adapter-routes-authz`, `express5-auth-wildcard`): 8 files, 58 tests, all passing. ## Risks - **This touches authentication and authorization paths directly.** Mistakes here are security bugs in both directions; review accordingly. - **Behavioral change for existing `cloud_tenant` deployments:** tenants that previously (incorrectly) had instance-admin lose it — including the ability to see/manage other companies on the instance. This is the intended hardening, but any single-tenant deployment that relied on the cloud-tenant identity for instance administration must provision a separate admin identity. - **The purge is destructive by design:** if an operator's instance-admin identity is *also* provisioned through the cloud-tenant headers (same user id), its `instance_admin` row will be deleted on the next trusted-header request. Operators should hold admin through a non-cloud-tenant identity. - **Residual gap (documented, not fixed here):** a deployment that ran the old cloud_tenant build and then *disabled* cloud-tenant mode keeps stale rows until the affected user re-authenticates through the cloud path. A data migration was considered and deliberately avoided: there is no reliable SQL predicate for "cloud-tenant-provisioned user" (no source column), so a migration risks deleting legitimate admins. - No schema or migration changes; no UI changes. ## Model Used - Claude Fable 5 (claude-fable-5, 1M context), extended thinking + tool use, via Claude Code — this revision; original PR authored in an earlier Claude Code session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (none duplicate this; related issues Refs #966 / #5015 are linked in the issue section) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (no UI changes) - [x] I have updated relevant documentation to reflect my changes (no existing docs reference `cloud_tenant` mode) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes Stubbemann
|
482f64e343 |
fix(plugin-kubernetes): resolve sandbox pod by exact name (controller labels pods with sandbox-name-hash, not sandbox-name) (#7982)
## Thinking Path Production e2e on the merged #5790 plugin failed on every fresh lease with "Failed to install the adapter runtime command" for a harness that was present in the runtime image. Tracing the lease showed the first exec resolved no pod: the exact-label fallback added during the #5790 review queries `agents.x-k8s.io/sandbox-name=<name>`, but the kubernetes-sigs agent-sandbox controller labels pods only with `agents.x-k8s.io/sandbox-name-hash` (see `sandboxLabel` in its `controllers/sandbox_controller.go`) and NAMES the backing pod exactly after the Sandbox CR. The selector matches nothing, `findPodForSandbox` returns null, execute returns "podName could not be resolved", and adapter-utils misreports it as a missing runtime command. ## What Changed Between the `status.podName` read and the label fallback, try an exact-name pod GET (`readNamespacedPod({namespace, name})`). This is collision-free, so the original review concern (name-prefix matching execing into a concurrent sandbox's pod) stays honored. A 404 falls through to the existing full-name label selector for controller versions that do set such a label. Non-404 errors propagate unchanged. ## Verification - New unit test pins the controller reality: pod named exactly like the sandbox, only a `sandbox-name-hash` label, no full-name label; fails before the fix, passes after. - Review-feedback round: the primary-path test now asserts the exact-name GET is never called, and a new test covers non-404 error propagation (403 rejects, no fallback). 153/153 plugin tests green, tsc clean. - Production-verified on our deployment: agent runs were broken on every fresh lease before this patch and complete end-to-end after it (gVisor sandbox pool, agent-sandbox controller v0.4.6; verified run with cost event and agent reply on a fresh tenant). ## Risks Low: one additional pod GET per first-exec on a fresh lease, only when `status.podName` is unset. Non-404 errors from the GET propagate unchanged (now test-pinned). ## Issue No existing issue; the defect is described in full under Thinking Path (introduced by the review-round fallback change in #5790, first hit in production e2e on 2026-06-11). ## Model Used Claude Fable 5 (claude-fable-5, Claude Code CLI, extended reasoning, tool use) ## Duplicate search Searched open and closed PRs for `findPodForSandbox`, `sandbox-name-hash`, and pod-resolution fixes; no duplicate found. Related parent: #5790 (introduced the fallback this PR repairs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (no UI change) - [x] I have updated relevant documentation to reflect my changes (code comments; no doc surface affected) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jannes StubbemannandClaude Opus 4.8
|
69a368ed51 |
fix(gemini-local): pre-select gemini-api-key auth in managed-HOME settings.json for headless runs (#7918)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The gemini-local adapter runs gemini-cli headlessly, including on remote/sandboxed execution targets where the adapter manages a dedicated HOME under the runtime root > - gemini-cli hard-refuses headless runs with "Invalid auth method selected." unless `$HOME/.gemini/settings.json` persists an auth selection; setting `GEMINI_DEFAULT_AUTH_TYPE` alone does NOT satisfy it (proven in an isolated pod) > - With a managed HOME the runtime root replaces the image home, so any settings.json baked into the agent image (or the user's real home) is invisible to the CLI, and every sandboxed gemini run dies before doing any work > - This affects any sandbox provider that runs gemini with API-key auth through the managed-HOME path (SSH, E2B, Daytona, Kubernetes, or any other remote execution target); it is a headless-execution bug fix, not gateway- or deployment-specific behavior > - This pull request makes the adapter pre-select the `gemini-api-key` auth type in the managed `$HOME/.gemini/settings.json` whenever a Gemini/Google API key is present, writing both settings schema generations and never touching an existing settings.json > - The benefit is that gemini agents actually run headlessly on remote and sandboxed execution targets without any manual settings provisioning ## Linked Issues or Issue Description No existing issue; describing the bug in-PR (bug template fields): - **What happened:** Headless gemini-local runs on remote/sandboxed execution targets fail immediately with `Invalid auth method selected.` even though `GEMINI_API_KEY` is provided. - **Expected:** Providing the API key should be enough for a headless run to authenticate and proceed. - **Root cause:** gemini-cli requires an auth selection persisted in `$HOME/.gemini/settings.json` for non-interactive runs; the `GEMINI_DEFAULT_AUTH_TYPE` env var does not substitute for it (verified in an isolated pod with only the env var set). The adapter's managed-HOME execution path points HOME at the runtime root, so any pre-existing settings.json (image-baked or user home) is hidden and the CLI finds no auth selection. - **Reproduction:** Run the gemini-local adapter against a remote/sandboxed execution target with `GEMINI_API_KEY` set and no settings.json under the managed HOME; the run aborts with the error above. - Duplicate/related search: no existing PR or issue addresses this; closest related is #7693 (bundles gemini-cli in the Docker image), which makes the CLI available but does not fix headless auth selection. ## What Changed - `packages/adapters/gemini-local/src/server/execute.ts`: after provisioning the managed HOME, when a Gemini/Google API key is present, write `$HOME/.gemini/settings.json` pre-selecting `gemini-api-key` auth. Both settings schema generations are written (legacy top-level `selectedAuthType` and current `security.auth.selectedType`) so old and new gemini-cli versions are covered. - The write is strictly scoped to the managed HOME (the per-run runtime root on sandbox transports). On non-managed remote targets (SSH), where the remote home is the user's real home and existing settings remain visible to the CLI, the adapter creates nothing (review feedback, P1). - The write is guarded by `[ -f ... ] ||` so a user-shipped settings.json (e.g. via workspace) is never overwritten. - The key-presence gate checks the run env AND the host process env (`GEMINI_API_KEY` / `GOOGLE_API_KEY`): in sandboxed paths the key never enters the adapter's run env; it reaches the agent pod via the sandbox provider's per-run secret (env passthrough from the host env), so the host env is the correct signal there. - `packages/adapters/gemini-local/src/server/execute.remote.test.ts`: a new sandbox-transport test asserts the settings.json write lands under the per-run runtime root (path + `gemini-api-key` content), and the SSH test asserts no settings.json is created on a non-managed home. ## Verification - `npx vitest run packages/adapters/gemini-local`: 3 files, 17 tests, all pass. - `pnpm --filter @paperclipai/adapter-gemini-local typecheck` and `build`: clean (test file is covered by the package tsconfig `include`). - Negative control: in an isolated pod, gemini-cli with `GEMINI_API_KEY` + `GEMINI_DEFAULT_AUTH_TYPE` set but no settings.json still fails with `Invalid auth method selected.`; with the settings.json written by this change, the run proceeds. - Verified end-to-end: a gemini agent in a hardened Kubernetes (gVisor) sandbox completed a real task (with `GOOGLE_GEMINI_BASE_URL` pointing at a GenAI-compatible endpoint), producing a billed usage row. That deployment supplies the verification evidence; the fix applies to any sandbox provider running gemini with API-key auth. ## Risks - Low risk. The new write only fires on the managed-HOME path (per-run runtime root) when an API key is present, and only when no settings.json exists yet, so existing setups, real user homes on SSH targets, and user-provided settings are unaffected. - If a future gemini-cli changes the settings schema again, the file may need a third generation key; both current generations are written today. ## Model Used - Claude (Anthropic), Claude Opus 4.8, 1M context, extended thinking, with tool use (code execution / shell) via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (no UI change) - [x] I have updated relevant documentation to reflect my changes (code comments document the behavior; no doc pages cover managed-home auth) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (review requested) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
9e750d3e92 |
feat(codex-local): env-driven gateway routing via PAPERCLIP_CODEX_PROVIDERS config.toml (#7919)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `codex-local` adapter runs the OpenAI Codex CLI; Paperclip already maintains a managed `CODEX_HOME` per company and ships it to remote/sandboxed execution targets > - Deployments increasingly put an OpenAI-compatible LLM gateway between the harness and the model for cost, governance, or data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign endpoints. But Codex has no CLI flag or env var for a custom endpoint: its only mechanism is `[model_providers.<id>]` tables (with `base_url`, `env_key`, `wire_api`) in `$CODEX_HOME/config.toml`, selected by a root-level `model_provider` key > - Today there is no supported way to get such provider config into the managed `CODEX_HOME`, so gateway routing requires hand-editing files the adapter owns and regenerates > - This pull request adds the codex analogue of #7837's opencode mechanism: a `PAPERCLIP_CODEX_PROVIDERS` JSON env var whose shape maps 1:1 onto codex's TOML schema, merged into the managed `config.toml` so the existing asset-shipping + `env.CODEX_HOME` mechanics deliver it to local and sandboxed runs alike; nothing here is specific to one hosting setup > - The benefit is Codex works behind any OpenAI-compatible gateway with config only; with no env set, behavior is unchanged ## Linked Issues or Issue Description No existing issue; describing in-PR (feature / adapter enhancement). - **Gap:** there is no supported way to register a custom/gateway `[model_providers.*]` endpoint for `codex-local`. Codex's only custom-endpoint mechanism is `config.toml` (`base_url` + `env_key` + `wire_api`, selected via the root `model_provider` key), and the adapter owns/regenerates the managed `CODEX_HOME`, so operators cannot durably hand-edit it. - Related: #7837 (the opencode-local analogue of this change, same env-driven gateway-routing pattern). Searched for duplicate/related PRs: no existing codex-local gateway/provider-routing PR found. > Note on ROADMAP: this is adapter-level, opt-in config (defaults unchanged) that *enables* gateway routing for one harness; it is not the core "Cloud / Sandbox agents" platform work itself. ## What Changed - New `prepareCodexRuntimeConfig()` (`packages/adapters/codex-local/src/server/runtime-config.ts`): reads `PAPERCLIP_CODEX_PROVIDERS` (run env first, then `process.env`), shaped as `{"providers": {"<id>": {base_url, env_key, wire_api, ...}}, "model_provider": "<id>"}`, and merges it into the managed `CODEX_HOME`'s `config.toml`. No-op when unset or empty. - A malformed value (invalid JSON, not a JSON object, no `providers` object, no usable provider entries, or individual entries with empty names or non-object values, which are skipped by name) is never silently dropped: each case surfaces a distinct, user-visible note (via the prepare notes, which flow into command notes + `onLog`) and unusable input leaves `config.toml` untouched. - Merge is marker-delimited and TOML-correct: existing `config.toml` content is preserved between two managed blocks. Root keys (e.g. `model_provider`) are prepended **before the first table header** (TOML root-region rule), `[model_providers.*]` tables are appended. Pre-existing same-name provider sections and root `model_provider` keys are excised so the managed definitions win without duplicate-table parse errors. - `{env:VAR}` placeholders are expanded server-side for literal-credential fields; `env_key` indirection remains the preferred path. - Crash-safe restore: prepare writes a pre-run backup (`config.toml.paperclip-backup`) before the merged file; `cleanup()` restores the original in the execute `finally` and removes the backup. If a run never reaches `cleanup()` (a throw during the setup between prepare and execution, or SIGKILL), the next prepare restores the original from the backup with full fidelity, including user `[model_providers.*]` sections the merge excised (review feedback, P2); plain block-stripping remains the fallback for pre-backup state. - An explicit adapter-config `env.CODEX_HOME` override is treated as user-managed: no merge, surfaced as a command note. - Dependency-free hand-emitted TOML (strings/numbers/booleans, arrays of scalars, plain objects as inline tables); basic strings escape U+0000-U+001F and U+007F per TOML 1.0 (review feedback, P2). Merged output was additionally validated locally with python tomllib during development; the committed tests assert the structural invariants. - `execute.ts` wiring: `prepareCodexRuntimeConfig` runs after `prepareManagedCodexHome` (before the home ships to the remote target), notes surface via `onLog` + command notes, and the `finally` calls `cleanup()`. **Note for reviewers:** current codex removed `wire_api = "chat"` (openai/codex#10157, Feb 2026), so gateway provider configs must use `wire_api = "responses"`, i.e. the gateway must speak `/v1/responses`. The adapter passes the value through verbatim; this is a codex-side constraint worth knowing when configuring it. ## Verification - `pnpm --filter @paperclipai/adapter-codex-local build` and `typecheck`: tsc clean against current `master` - `pnpm exec vitest run packages/adapters/codex-local`: 45 passing (incl. 17 `runtime-config` tests: fresh-merge + cleanup restore, root-region placement, same-name provider override, inline tables/arrays, DEL escaping, `{env:}` expansion from run env + `process.env`, per-case malformed-input notes with `config.toml` untouched, skipped-entry notes alongside a successful merge, silent no-op when unset/empty, explicit-`CODEX_HOME` skip note, backup restore of excised user sections after an interrupted run, backup removal on cleanup, stale-block self-heal, re-run replacement) - Verified end-to-end: a codex agent in a hardened Kubernetes (gVisor) sandbox completed a real task routed through an OpenAI-compatible gateway's `/v1/responses`, with a billed usage row recorded on the gateway. That deployment supplies the verification evidence; the mechanism is gateway-agnostic. ## Risks Low. Entirely env-driven and opt-in; with `PAPERCLIP_CODEX_PROVIDERS` unset the adapter never touches `config.toml` and behavior is byte-identical to before. The merge preserves user content, restores the original file on cleanup, and survives interrupted runs via the pre-run backup; malformed input surfaces a visible note and is ignored without touching `config.toml`. No migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md (adapter-level opt-in config enabling gateway routing; not the core sandbox-platform work, noted above) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#7837 is the opencode analogue; no codex-local duplicate found) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (env var documented inline; no central doc references the adapter env yet) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (green on the previous head; re-running on the final note-copy polish commit) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (both review P2s are fixed at head: the interrupted-run restore via the pre-run backup and the U+007F escaping; a re-review is requested for the note-copy polish) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
6e4aca9c67 |
feat(pi-local): env-driven gateway routing via PAPERCLIP_PI_PROVIDERS models.json (#7920)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `pi-local` adapter runs the Pi coding agent, including inside remote/sandboxed execution targets; Pi resolves `--provider P --model M` by an exact (provider, id) match against its model registry, and it has no base-url CLI flag or env var: a `models.json` in its agent config dir (`$PI_CODING_AGENT_DIR`, falling back to `$HOME/.pi/agent`) is its only mechanism for custom or OpenAI/Anthropic-compatible endpoints > - Deployments increasingly put an LLM gateway between the harness and the model for cost, governance, or data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign endpoints. Today there is no supported way to get such provider config into Pi's registry for orchestrated runs > - The opencode adapter gained the equivalent capability in #7837 and codex in #7919; this pull request is the Pi analogue, so the harness layer stays gateway-agnostic regardless of which CLI an agent uses; nothing here is specific to one hosting setup > - This pull request reads `PAPERCLIP_PI_PROVIDERS` (Pi's `models.json` `providers` shape), materialises a managed `models.json` in a temp agent-config dir, points `PI_CODING_AGENT_DIR` at it, and ships it to remote execution targets with the run > - The benefit is Pi works behind any compatible gateway with config only; with no env set, behavior is unchanged ## Linked Issues or Issue Description No existing issue; describing in-PR (feature / adapter enhancement). - **Gap:** there is no supported way to register custom/gateway providers + models for `pi-local`. Pi's only custom-endpoint mechanism is a `models.json` in its agent config dir, and orchestrated (especially sandboxed) runs have no way to provision one declaratively. - Related: #7837 (the opencode-local analogue, same env-driven gateway-routing pattern) and #7919 (the codex-local analogue). Searched for duplicate or related PRs: no existing pi-local gateway/provider-routing PR found. > Note on ROADMAP: this is adapter-level, opt-in config (defaults unchanged) that *enables* gateway routing for one harness; it is not the core "Cloud / Sandbox agents" platform work itself. ## What Changed - New `packages/adapters/pi-local/src/server/runtime-config.ts`: `preparePiRuntimeConfig()` reads `PAPERCLIP_PI_PROVIDERS` (a JSON object in pi's `models.json` `providers` shape) from the run env, then `process.env`. When set, it expands `{env:VAR}` placeholders (run env first, then process env; unresolvable placeholders left intact), writes `{"providers": ...}` to a managed temp dir as `models.json`, and returns env with `PI_CODING_AGENT_DIR` pointing at it plus a cleanup handle. - `execute.ts`: the prepared dir ships to remote execution targets as the managed-runtime asset `agentConfig` (same mechanism as opencode's `xdgConfig`), and `PI_CODING_AGENT_DIR` is repointed to the in-target path; cleanup runs in `finally`. - Misconfiguration is visible, not silent: a set-but-unusable `PAPERCLIP_PI_PROVIDERS` (invalid JSON, not an object, no provider objects) surfaces an explanatory note instead of proceeding unconfigured into an opaque model-not-found failure later, and provider entries with non-object values are skipped with a note naming them. Unset/empty stays a silent no-op (feature off). - Defaults unchanged: with `PAPERCLIP_PI_PROVIDERS` unset, the adapter behaves byte-for-byte as before, for local runs and for every existing sandbox provider. ## Verification - All pi-local tests green against this base (new: providers written verbatim, `{env:VAR}` expansion from run env/process env/unresolvable, no-op when unset, `PI_CODING_AGENT_DIR` set and shipped, the misconfiguration notes incl. skipped non-object entries, remote asset sync + env repoint). Typecheck and build clean. - Production end-to-end evidence (our deployment, used as verification, not as the scope of the change): a pi agent in a Kubernetes gVisor sandbox resolved a custom provider from the shipped `models.json`, completed an assigned issue through an Anthropic-compatible gateway, and landed a billed usage row. ## Risks Low. The entire feature is opt-in behind one env var; the only behavior change when it is set is the intended one. The managed dir replaces the host agent dir for the run by design (credentials travel inside the provider config or via env-key indirection), which is the correct posture for orchestrated runs. No migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md (adapter-level opt-in config enabling gateway routing; not the core sandbox-platform work, noted above) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#7837 and #7919 are the opencode/codex analogues; no pi-local duplicate found) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (env var documented inline; no central doc references the adapter env yet) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (green on the previous head; re-running on the final note-copy polish commit) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (both prior review findings are fixed at head: the indirect notes-based guard is now an explicit `agentConfigDir` handle, and a failed `models.json` write no longer leaks the temp dir; a re-review is requested for the note-copy polish) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
1ac1ba5442 |
feat(opencode-local): env-driven gateway routing (custom providers, small/cheap model, remote allow-all) (#7837)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The `opencode-local` adapter runs the OpenCode harness; its
model/provider routing assumes built-in providers (anthropic/openai/...)
and their default models
> - Deployments increasingly put an OpenAI/Anthropic-compatible LLM
gateway between the harness and the model for cost, governance, or
data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate
proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign
endpoints. But OpenCode only resolves `--model provider/model` when the
model is registered in a provider's `models` map, and
`OPENCODE_ALLOW_ALL_MODELS` does NOT bypass its internal `getModel()`
> - Several lanes also fall back to built-in default models the gateway
may not serve: the auxiliary/title model (e.g. `claude-haiku-*`) and the
budget/recovery "cheap" lane (`openai/gpt-5.1-codex-mini`); these abort
runs with "no keys found that support model"
> - This pull request makes the adapter's provider/model wiring
declarative via env, so any such deployment can register gateway models
+ pin the auxiliary/budget lanes without code changes; nothing here is
specific to one hosting setup
> - The benefit is OpenCode works behind any compatible gateway with
config only; with no env set, behavior is unchanged
## Linked Issues or Issue Description
No existing issue; describing in-PR (feature / adapter enhancement).
- **Gap:** there is no supported way to register custom/gateway
providers + models for `opencode-local`, nor to pin the auxiliary
(title-gen) and budget (recovery) model lanes, so routing OpenCode
through a gateway fails at `getModel()` or on the default helper models.
- Related: #5737 (exe.dev sandbox installs for gemini/opencode local),
#5823 (unblock claude_local on remote sandbox providers).
> Note on ROADMAP: this is adapter-level, opt-in config (defaults
unchanged) that *enables* gateway routing for one harness; it is not the
core "Cloud / Sandbox agents" platform work itself. Happy to
redirect/discuss in #dev if preferred.
## What Changed
- `PAPERCLIP_OPENCODE_PROVIDERS`: merge custom/extended providers
(OpenCode `provider` shape) into the runtime `opencode.json`, so gateway
models are registered and `--model provider/model` resolves. `{env:VAR}`
placeholders are expanded server-side (so a key need not depend on the
sandbox run env).
- A malformed `PAPERCLIP_OPENCODE_PROVIDERS` is no longer silently
ignored: invalid JSON, a non-object value, and individual provider
entries with non-object values (which are skipped by name) each append a
visible note to the run notes so the misconfiguration is diagnosable
(addresses both review P1s).
- `PAPERCLIP_OPENCODE_SMALL_MODEL` / `PAPERCLIP_OPENCODE_CHEAP_MODEL`:
pin the auxiliary (title-generation) and budget (recovery-retry) lanes
to gateway-served models; defaults unchanged.
- Honour `OPENCODE_ALLOW_ALL_MODELS` on the **remote** execution path
too (was local-only, a parity gap).
- `PAPERCLIP_OPENCODE_PRINT_LOGS`: optional toggle adding `--print-logs`
so OpenCode logs surface on stderr for diagnosing remote/sandbox runs.
- `buildOpenCodeModelProfiles()` guards its `process.env` default with
`typeof process` so the shared client/server module stays browser-safe
(a bare `process.env` at module load threw ReferenceError in the browser
under Vite dev middleware and broke UI rendering in the e2e lane).
## Verification
- `pnpm --filter @paperclipai/adapter-opencode-local build` and
`typecheck` (tsc clean)
- `pnpm exec vitest run packages/adapters/opencode-local/src` shows 33
passing (incl. new tests for the provider merge, `{env:}` expansion, the
malformed/non-object/skipped-entry provider notes, small/cheap-model
resolution, and the remote allow-all bypass)
- Manually verified end-to-end against a real
OpenAI-/Anthropic-compatible gateway: with the providers + small/cheap
model set, both the title-gen and main task route to the configured
gateway model and the agent completes (a real completion is returned and
billed). That deployment supplies the verification evidence; the
mechanism is gateway-agnostic.
## Risks
Low. Everything is env-driven and opt-in; with no env set the generated
config output is unchanged, and the cheap model profile keeps its model
(the only difference is its updated human-readable description).
Defaults preserved: built-in providers, Codex-mini cheap lane with
`variant: low`, no `--print-logs`. No migration/UI impact.
## Model Used
Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking +
tool use, via Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md (adapter-level opt-in config enabling
gateway routing; not the core sandbox-platform work, noted above)
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (#5737, #5823)
- [x] I have either (a) linked existing issues OR (b) described the
issue in-PR following the relevant issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots (n/a, no UI)
- [ ] I have updated relevant documentation to reflect my changes (env
vars documented inline via comments; no central doc references the
adapter env yet)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(the P1 about silently dropped malformed providers JSON is addressed in
|
||
|
Jannes StubbemannandClaude Opus 4.8
|
398d746093 |
build(agent-runtime): harness runtime images for sandboxed execution (stage 3/3) (#7934)
> [!NOTE] > This is **stage 3 of 3** of the staged Kubernetes contribution: stage 1 is the kubernetes sandbox-provider plugin (#5790), stage 2 is the provider backend/hardening refresh filed separately, and this stage ships the runtime images those sandboxes run. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandboxed agent execution (Refs #248) runs each agent turn in an isolated environment; the kubernetes sandbox provider (stage 1, #5790) schedules those runs as hardened pods > - A sandbox pod needs a runtime image with the harness CLI preinstalled: installing CLIs at run start is slow, flaky, and needs network egress the sandbox should not have > - There is no first-party image family for this, so every deployer would have to hand-roll Ubuntu + Node + CLI images per harness and solve signal handling, non-root, and image chaining themselves > - This PR ships the agent-runtime image family: a hardened base (non-root uid 1000, tini, git, the agent shim) plus one derived image per harness, a buildx bake file that chains them, and a publish workflow with cosign keyless signing > - The benefit is that any sandbox infrastructure, the kubernetes provider or otherwise, gets ready-made, signed, security-hardened per-harness runtime images that are verified in production across five harnesses ## Linked Issues or Issue Description Refs #248 (sandboxed agent execution proposal) and #5790 (the kubernetes sandbox provider, stage 1 of this contribution, which consumes these images as per-run runtime images via its adapter defaults). No issue covers the image gap itself, described in-PR: sandbox providers reference `ghcr.io/paperclipai/agent-runtime-*` images, but the repository contains neither the Dockerfiles nor the workflow that builds and publishes them. Without this, self-deployers cannot reproduce or audit the images their agent runs execute in. ## What Changed - `docker/agent-runtime/Dockerfile.base`: foundation image. Ubuntu 22.04 + Node 22 + git + tini (PID 1, signal propagation) + non-root `paperclip` user (uid/gid 1000) + the agent shim compiled in a Go build stage. `WORKDIR /workspace`, entrypoint `tini -- paperclip-agent-shim`. - One derived Dockerfile per harness: `opencode` (opencode-ai), `pi` (@mariozechner/pi-coding-agent), `codex` (@openai/codex), `gemini` (@google/gemini-cli, plus headless auth-mode settings), `claude` (@anthropic-ai/claude-code, symlinked as `claude-code`). Each installs the CLI as root, returns to uid 1000, and asserts the binary is on PATH at build time. - `acpx` and `hermes` Dockerfiles are included in the bake group but are not in the default publish scope (hermes is a stub until a CLI package exists). - `docker/agent-runtime/buildx-bake.hcl`: builds the whole family in one pass. Derived targets chain off the `base` target through bake `contexts` (the literal registry in each `FROM` is overridden to `target:base` at build time, so no intermediate push is needed). `REGISTRY` (default `ghcr.io/paperclipai`) and `VERSION` are overridable variables. - `tools/agent-shim/`: a small Go shim that runs as the container command. It reads `/run/paperclip/runtime-command.json` (`{ "command", "args" }`), resolves the harness CLI on PATH, and `syscall.Exec`s it so SIGTERM from the kubelet reaches the harness directly. Harness-agnostic, with unit tests. - `.github/workflows/agent-runtime-images.yml`: builds and pushes the default scope (base, opencode, pi, codex, gemini, claude) for linux/amd64 on `workflow_dispatch` (explicit version tag) or pushes to `master` touching these paths, then signs every digest with cosign keyless OIDC. Uses only `GITHUB_TOKEN`; no extra secrets. - `docker/agent-runtime/README.md`: image lineup, base contents, local build instructions, the runtime-command contract, and the security model. Additive only: nothing in the product loads these images. Deployments opt in via their sandbox provider configuration (for example the kubernetes plugin's image settings). ## Verification - `cd tools/agent-shim && go build ./... && go test ./... && go vet ./...`: all passing. - `docker buildx bake -f docker/agent-runtime/buildx-bake.hcl --print base opencode pi codex gemini claude`: resolves cleanly; every tag and build context lands on `ghcr.io/paperclipai/agent-runtime-*` and derived targets map the base ref to `target:base`. - Workflow YAML validated (parses, single job, no org-specific secrets). - This exact image family (built from these Dockerfiles, bake file, and workflow) is what runs agent execution in production on paperclip.inc, verified end-to-end across five harnesses (opencode, pi, codex, gemini, claude): each as a full loop from assigned issue to per-run runtime image in a sandboxed pod to completed run. ## Risks - Low risk: purely additive, nothing in paperclip-server or the UI references these files. The workflow only triggers on its own paths. - Derived images install harness CLIs `@latest` at build time; a broken upstream CLI release would surface at image build, not at run time, and the PATH assertion fails the build rather than shipping a broken image. - The hermes image is an explicit stub (documented in its Dockerfile) until a hermes CLI package exists; it is outside the default publish scope. - cosign signing is keyless OIDC with the workflow identity; no long-lived signing keys are introduced. ## Model Used Claude Opus 4.8 (claude-opus-4-8, 1M context, extended thinking, tool use via Claude Code). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (no UI changes) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
4ad94d0bde |
feat(server): kubernetes execution integration for sandbox-provider plugins (stage 2/3) (#7938)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The execution subsystem runs those agents in environments (local, ssh, sandbox), and sandbox-provider plugins let an environment materialize per-run sandboxes > - Stage 1 (#5790) contributed a first-party Kubernetes sandbox-provider plugin, but the server core has no way to adopt it operationally: no per-run adapter selection, no way to force an instance onto sandboxed execution, no declarative adapter/model configuration, and the plugin must be installed by hand > - Without this, a multi-tenant or security-conscious deployment cannot guarantee that agent runs never execute on the host, and a single environment cannot serve agents with different harnesses > - This pull request adds the server + SDK integration: per-run adapterType on the lease protocol, an env-gated forced-Kubernetes execution policy with provisioning and a per-run allowlist guard, a declarative adapter registry and model list, in-cluster env passthrough for sandbox plugin workers, fail-safe auto-install of the bundled plugin, and the matching UI affordance > - The benefit is that sandbox-provider plugins become fully usable for Kubernetes execution: operators configure everything via environment variables and GitOps, while self-hosters who set none of the variables see exactly the behavior they have today ## Linked Issues or Issue Description Refs #5790 (stage 1 of 3: the Kubernetes sandbox-provider plugin package). No existing issue. Feature description: the server core lacks the integration seams to operate a sandbox-provider plugin as the mandatory execution path of an instance. This PR is stage 2 of 3 of the staged Kubernetes contribution; stage 3 will contribute the agent runtime images and their build pipeline. ## What Changed One line per piece: - `packages/plugins/sdk/protocol.ts`: optional `adapterType` on `PluginEnvironmentAcquireLeaseParams` so a provider can select the runtime image per run; existing providers simply ignore it - `server/services/environment-runtime.ts` + `environment-run-orchestrator.ts`: thread the agent's adapter type into both lease-acquiring drivers, including the heartbeat path (the two call sites have historically drifted, hence the pinned test) - `server/services/environments.ts`: `ensureKubernetesEnvironment` / `findKubernetesEnvironment`, an idempotent managed Kubernetes environment per company, identified by a metadata marker and refreshed (not recreated) on config change; `timeoutMs` rides on the config for slow cold-start leases - `server/services/execution-allowlist.ts`: pure (driver, provider, policy) -> allow/deny guard; `executionMode=kubernetes` only allows the kubernetes sandbox provider - `server/services/execution-policy-bootstrap.ts` + startup hook in `server/index.ts`: parse `PAPERCLIP_EXECUTION_MODE` / `PAPERCLIP_K8S_*`, persist `executionMode` into instance general settings, and provision the managed environment for every company; fails loud on misconfiguration - `server/services/heartbeat.ts`: when the policy forces Kubernetes, pin run selection to the managed environment (also overriding any persisted workspace environment id), refuse to fall back to local, and re-check the actually acquired environment against the allowlist as defense in depth - `server/services/adapter-registry-bootstrap.ts` + shared `AdapterRegistryEntry` type/validator: declarative `PAPERCLIP_ADAPTERS` registry (inline JSON or file) that reconciles adapter availability at startup and rides on the Kubernetes environment config - `server/services/adapter-models-env.ts` + `adapters/registry.ts`: `PAPERCLIP_ADAPTER_MODELS` lets an operator declare picker model lists the server cannot CLI-discover - `server/services/plugin-loader.ts`: pass `KUBERNETES_SERVICE_HOST/PORT(_HTTPS)` through to plugin workers that register environment drivers, so in-cluster API clients can be constructed; all other host env stays stripped - `server/app.ts`: fail-safe auto-install of the bundled kubernetes plugin at boot; no-ops when the bundle is absent and never blocks startup on error - `packages/shared` types/validators: `InstanceExecutionMode` on general settings (optional, strict schema) - `ui/lib/forced-kubernetes-environment.ts` + `AgentConfigForm`: when the policy is active, show a read-only Kubernetes environment instead of the environment picker and default new agents onto the managed environment - Tests for every new module plus the adapterType pin in `heartbeat-plugin-environment` and the managed-environment lifecycle in `environment-service` Everything is gated: with `PAPERCLIP_EXECUTION_MODE`, `PAPERCLIP_ADAPTERS`, and `PAPERCLIP_ADAPTER_MODELS` unset (and no bundled plugin present), every code path reduces to current behavior. The per-run `adapterType` is an optional SDK parameter that existing providers ignore. ## Verification - `cd server && npx tsc --noEmit`: clean (0 errors); `ui` typecheck also clean - Targeted suites all green (11 files, 90 tests): `npx vitest run server/src/__tests__/heartbeat-plugin-environment.test.ts server/src/__tests__/environment-service.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/environment-run-orchestrator.test.ts server/src/__tests__/plugin-database.test.ts server/src/services/execution-policy-bootstrap.test.ts server/src/services/execution-allowlist.test.ts server/src/services/adapter-registry-bootstrap.test.ts server/src/services/adapter-registry-bootstrap.reconcile.test.ts server/src/services/adapter-models-env.test.ts packages/shared/src/validators/adapter-registry.test.ts` - `npx vitest run ui/src/components/AgentConfigForm.test.ts`: green (6 tests) - Full `npx vitest run server/src/__tests__`: 2323 passed, 1 skipped; the only failures (heartbeat-process-recovery pid-retry, workspace-runtime symbolic-ref/git tests) reproduce identically on pristine `master` in the same environment, so they are machine-environment issues unrelated to this change; `server-startup-feedback-export` needed its `services/index.js` mock extended with the new export and is green - This integration has been running in production on a hosted multi-tenant deployment, where it executes agent runs across five different harnesses through the stage 1 plugin ## Risks - Low for existing deployments: every behavior is env-gated and the defaults preserve current semantics; the auto-install block is wrapped fail-safe and skips silently when the plugin bundle is absent - `executionMode` is a new optional field on a strict zod schema; absent input normalizes exactly as before - The forced policy intentionally fails runs loudly (rather than falling back to local) when no managed Kubernetes environment exists; this only affects instances that explicitly set `PAPERCLIP_EXECUTION_MODE=kubernetes` ## Model Used Claude Opus 4.8 (claude-opus-4-8, 1M context), extended thinking, agentic tool use via Claude Code. ## UI screenshots The UI change is a new read-only "Execution" section in `AgentConfigForm`, shown only when the instance execution policy forces Kubernetes (`executionMode=kubernetes`); there is no "before" state for it (the section did not exist, and instances without the forced policy render the existing picker unchanged). Captured from the new Storybook stories added in this PR (`Product/Agent Management`): Managed Kubernetes environment present (read-only display, no local/SSH picker):  No managed environment available yet (warning notice, no silent local fallback):  ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
05ab45225a |
feat(plugin-kubernetes): self-hostable Kubernetes sandbox provider (stage 1/3: plugin package) (#5790)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers are the seam that lets agent runs execute in isolated environments; today the only first-party remote provider is Daytona, a hosted third-party service > - Self-hosters running Paperclip on their own infrastructure (often Kubernetes already) have no first-party way to run agent sandboxes on a cluster they control > - That gap matters for teams with data-residency, sovereignty, or cost constraints who cannot or will not send workloads to a hosted sandbox service > - This pull request adds a Kubernetes sandbox-provider plugin as a standalone, workspace-excluded package: it implements every SandboxProvider hook the Daytona provider does, on infrastructure the operator owns > - The benefit is that any Paperclip deployment with a Kubernetes cluster gets multi-tenant, network-isolated, quota-bounded agent sandboxes with zero new external dependencies ## Linked Issues or Issue Description No existing issue. Following the feature template: - **Problem:** Paperclip's remote sandbox execution requires a hosted third-party provider. Self-hosters cannot run agent sandboxes on their own Kubernetes clusters with a first-party provider. - **Proposed solution:** A `@paperclipai/plugin-kubernetes` sandbox-provider plugin with two backends: long-lived sandboxes via the [kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) CRD (multi-command exec, adapter-install pattern) and one-shot `batch/v1` Jobs (stable APIs only, no extra controllers). - **Alternatives considered:** Driving kubectl from a generic shell provider (no lifecycle/lease semantics), or requiring a hosted provider (exactly the constraint this removes). ## What Changed This is **stage 1 of 3** of a staged contribution (direction agreed with maintainers): the plugin package alone. Stage 2 (server integration: lease params, provider registration) and stage 3 (agent runtime images + CI) are companion PRs that will be cross-linked from a comment here. - New package `packages/plugins/sandbox-providers/kubernetes` (workspace-excluded, like the path already carved out in `pnpm-workspace.yaml`): src, unit + kind integration tests, operator prerequisite manifests, README, smoke-test guide - Implements the full SandboxProvider hook surface the Daytona provider implements: `validateConfig`, `probe`, `acquireLease`, `resumeLease`, `releaseLease`, `destroyLease`, `realizeWorkspace`, `execute` - Two backends: `sandbox-cr` (default; long-lived pod via the agent-sandbox `Sandbox` CR, supports multi-command exec) and `job` (one-shot `batch/v1` Job; nothing beyond k8s 1.27+ required) - Per-run adapter resolution: one environment serves mixed harnesses; the per-run `adapterType` hint is read through a local optional type extension, so the plugin typechecks and builds against the current plugin SDK and simply falls back to the environment's configured default adapter until stage 2 lands - Exec-env wrapping: the Kubernetes exec API carries no environment, so commands are wrapped to receive the run's env - Fast-upload interception for workspace realization, scoped per lease - Per-tenant isolation: derived namespace per company, RBAC, ResourceQuota, restricted-PSS pod security (runAsNonRoot, drop ALL, seccomp RuntimeDefault, no SA token automount) - Network egress policy in two flavors: native `NetworkPolicy` and `CiliumNetworkPolicy` (FQDN allowlists) - Image allowlist with glob matching, registry override, and per-run image override validation - Per-run Kubernetes Secrets carrying agent credentials, ownerRef'd to the Job or Sandbox CR for cascade GC ## Verification - Standalone build, exactly as the README documents: ```bash cd packages/plugins/sandbox-providers/kubernetes pnpm install --ignore-workspace pnpm test # 147 unit tests, 17 files, all green pnpm typecheck # clean against the in-repo plugin SDK on master pnpm build # dist/ emitted, manifest + worker entrypoints present ``` - A kind-cluster end-to-end integration test is included (`RUN_K8S_INTEGRATION_TESTS=1 pnpm test test/integration/end-to-end-run.test.ts`) - Beyond CI: this provider has been verified in a production multi-tenant deployment against five harnesses (opencode, pi, codex, gemini, claude code) with real billed runs ## Risks - **Zero behavior change for any existing deployment.** The package is workspace-excluded; nothing in the server imports or loads it until stage 2's integration lands. No existing code paths are touched. - The default `sandbox-cr` backend depends on an alpha CRD (`agents.x-k8s.io/v1alpha1`); the README flags this and the `job` backend uses only stable APIs as a fallback. - Risk surface is confined to deployments that explicitly install and configure the plugin. - The default runtime images (`ghcr.io/paperclipai/agent-runtime-*`) are published by the stage 3 companion PR (#7934); until that lands, deployments must point `runtimeImage` at their own images. ## Model Used Claude Opus 4.8 (1M context), extended thinking, with tool use (Claude Code). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (no UI changes) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending this push) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
93cdc5c1ce |
fix(adapter-utils): tar sandbox workspace by entry, not '.', to avoid EPERM on unowned target dir (#7836)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can run in remote/sandboxed environments via the shared sandbox managed-runtime in `@paperclipai/adapter-utils` (used by SSH/E2B/Daytona and other sandbox providers), which syncs the workspace into the sandbox by tarring it up and extracting it inside the pod/host > - When the sandbox runs the harness as a non-root user whose home/workspace dir it does not own (for example a hardened, non-root, gVisor pod with an `emptyDir`-mounted workspace), the workspace upload aborts before the agent can start > - Root cause: `createTarballFromDirectory` archives `.`, embedding a `./` self-entry whose mode/mtime tar then tries to restore onto the **extraction target directory**; `chmod`/`utime` of `.` fails with `Operation not permitted` for a non-owner > - This is not specific to any one deployment: the `.` self-entry EPERM can bite every sandbox provider built on the shared managed runtime as soon as the extracting user does not own the target directory, which is the norm for hardened non-root sandboxes > - This pull request archives the directory's top-level entries by name instead of `.`, so there is no `./` self-entry and extraction never touches the target dir's metadata > - The benefit is that workspace sync works in any sandbox where the target dir is non-root or not owned by the extracting user, without GNU-only tar flags ## Linked Issues or Issue Description No existing issue; describing in-PR (bug). - **What happens:** managed sandbox runs that sync the workspace fail at upload with `tar: .: Cannot utime: Operation not permitted` / `tar: .: Cannot change mode to ... : Operation not permitted`, aborting the run before the harness starts. - **Where:** `packages/adapter-utils/src/sandbox-managed-runtime.ts`, in `createTarballFromDirectory` (archives `.`). - **When:** the extraction target directory is not owned by the (non-root) user extracting the tar inside the sandbox. - Closely related (different root cause): #6560 (E2B workspace upload + lease idle failures). ## What Changed - `createTarballFromDirectory` enumerates the directory's top-level entries with `fs.readdir` and passes them by name after `--` (guards flag-like filenames) instead of archiving `.`, eliminating the `./` self-entry that triggers the EPERM. - Empty workspaces (legitimate for blank-workspace runs) write a valid 1024-byte all-zero EOF tar instead of invoking `tar` with no paths. - `--exclude` patterns continue to apply (to nested matches and any named entry). ## Verification - `pnpm --filter @paperclipai/adapter-utils build` (tsc clean) - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` runs green - New tests: uploaded workspace/asset tarballs contain no `.`/`./` member yet still extract correctly; empty workspace produces a valid (no-op) tarball. Existing managed-runtime sync test unchanged. - Manually verified in a hardened (non-root, gVisor) sandbox pod: with the fix, the workspace upload that previously aborted with the EPERM now succeeds. That deployment is the reproduction and verification environment; the fix itself is provider-agnostic. ## Risks Low. Behavior is unchanged for owned/root targets; the archive contents are the same minus the `./` self-entry (which tar recreates implicitly on extract). Portable across GNU/BSD/busybox tar (no GNU-only `--no-overwrite-dir`). No API/migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work (bug fix in shared sandbox utils, not core feature work) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#6560) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (n/a, internal behavior, no docs reference this) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (the only finding was the description-template P2, resolved by this description; the latest review covers the current head with no code findings and all CI gates are green) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jannes Stubbemann
|
7463479fc8 |
fix: disable HTTP caching on run log endpoints (#3724)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Every run emits a streaming log that the web UI polls so humans can watch what the agent is doing > - Log responses go out without explicit cache directives, so Express adds an ETag > - If the first poll lands before any bytes have been written, the browser caches the empty / partial snapshot and keeps getting `304 Not Modified` on every subsequent poll > - The transcript pane then stays stuck on "Waiting for transcript…" even after the log has plenty of content > - This pull request sets `Cache-Control: no-cache, no-store` on both run-log endpoints so the conditional-request path is defeated ## What Changed - `server/src/routes/agents.ts` — `GET /heartbeat-runs/:runId/log` now sets `Cache-Control: no-cache, no-store` on the response. - Same change applied to `GET /workspace-operations/:operationId/log` (same structure, same bug). ## Verification - Reproduction: start a long-running agent, watch the transcript pane. Before the fix, open devtools and observe `304 Not Modified` on each poll after the initial 200 with an empty body; the UI never updates. After the fix, each poll is a 200 with fresh bytes. - Existing tests pass. ## Risks Low. Cache headers only affect whether the browser revalidates; the response body is unchanged. No API surface change. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed |
||
|
Jannes Stubbemann
|
390502736c |
chore(ui): drop console.* and legal comments in production builds (#3728)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - The web UI is a single-page app built with Vite and shipped as a
static bundle to every deployment
> - Production bundles carry `console.log` / `console.debug` calls from
dev code and `/*! … */` legal-comment banners from third-party packages
> - The console calls leak internals to anyone opening devtools and
waste bytes per call site; the legal banners accumulate throughout the
bundle
> - Both problems affect every self-hoster, since they all ship the same
UI bundle
> - This pull request configures esbuild (via `vite.config.ts`) to strip
`console` and `debugger` statements and drop inline legal comments from
production builds only
## What Changed
- `ui/vite.config.ts`:
- Switch to the functional `defineConfig(({ mode }) => …)` form.
- Add `build.minify: "esbuild"` (explicit — it's the existing default).
- Add `esbuild.drop: ["console", "debugger"]` and
`esbuild.legalComments: "none"`, gated on `mode === "production"` so
`vite dev` is unaffected.
## Verification
- `pnpm --filter @paperclipai/ui build` then grep the
`ui/dist/assets/*.js` bundle for `console.log` — no occurrences.
- `pnpm --filter @paperclipai/ui dev` — `console.log` calls in source
still reach the browser console.
- Bundle size: small reduction (varies with project but measurable on a
fresh build).
## Risks
Low. No API surface change. Production code should not depend on
`console.*` for side effects; any call that did is now a dead call,
which is the same behavior most minifiers apply.
## Model Used
Claude Opus 4.6 (1M context), extended thinking mode.
## Checklist
- [x] Thinking path traces from project context to this change
- [x] Model used specified
- [x] Tests run locally and pass
- [x] CI green
- [x] Greptile review addressed
|
||
|
Jannes Stubbemann
|
0d87fd9a11 |
fix: proper cache headers for static assets and SPA fallback (#3734)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Every deployment serves the same Vite-built UI bundle from the same express app > - Vite emits JS/CSS under `/assets/<name>.<hash>.<ext>` — the hash rolls whenever the content rolls, so these files are inherently immutable > - `index.html` references specific hashed filenames, so it has the opposite lifecycle: whenever we deploy, the file changes but the URL doesn't > - Today the static middleware sends neither with cache headers, and the SPA fallback serves `index.html` for any unmatched route — including paths under `/assets/` that no longer exist after a deploy > - That combination produces the familiar "blank screen after deploy" + `Failed to load module script: Expected a JavaScript MIME type but received 'text/html'` bug > - This pull request caches hashed assets immutably, forces `index.html` to `no-cache` everywhere it gets served, and returns 404 for missing `/assets/*` paths ## What Changed - `server/src/app.ts`: - Serve `/assets/*` with `Cache-Control: public, max-age=31536000, immutable`. - Serve the remaining static files (favicon, manifest, robots.txt) with a 1-hour cache, but override to `no-cache` specifically for `index.html` via the `setHeaders` hook — because `express.static` serves it directly for `/` and `/index.html`. - The SPA fallback (`app.get(/.*/, …)`) sets `Cache-Control: no-cache` on its `index.html` response. - The fallback returns 404 for paths under `/assets/` so browsers don't cache the HTML shell as a JavaScript module. ## Verification - `curl -i http://localhost:3100/assets/index-abc123.js` → `cache-control: public, max-age=31536000, immutable`. - `curl -i http://localhost:3100/` → `cache-control: no-cache`. - `curl -i http://localhost:3100/assets/missing.js` → `404`. - `curl -i http://localhost:3100/some/spa/route` → `200` HTML with `cache-control: no-cache`. ## Risks Low. Asset URLs and HTML content are unchanged; only response headers and the 404 behavior for missing asset paths change. No API surface affected. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed |
||
|
Jannes Stubbemann
|
6059c665d5 |
fix(a11y): remove maximum-scale and user-scalable=no from viewport (#3726)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Humans watch and oversee those agents through a web UI > - Accessibility matters for anyone who cannot read small text comfortably — they rely on browser zoom > - The app shell's viewport meta tag includes `maximum-scale=1.0, user-scalable=no` > - Those tokens disable pinch-zoom and are a WCAG 2.1 SC 1.4.4 (Resize Text) failure > - The original motivation — suppressing iOS Safari's auto-zoom on focused inputs — is actually a font-size issue, not a viewport issue, and modern Safari only auto-zooms when input font-size is below 16px > - This pull request drops the two tokens, restoring pinch-zoom while leaving the real fix (inputs at ≥16px) to CSS ## What Changed - `ui/index.html` — remove `maximum-scale=1.0, user-scalable=no` from the viewport meta tag. Keep `width=device-width, initial-scale=1.0, viewport-fit=cover`. ## Verification - Manual on iOS and Chrome mobile: pinch-to-zoom now works across the app. - Manual on desktop: Ctrl+/- zoom already worked via `initial-scale=1.0`; unchanged. ## Risks Low. Users who were relying on auto-zoom-suppression for text inputs will notice nothing (modern Safari only auto-zooms below 16px). No API surface change. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed |
||
|
Jannes Stubbemann
|
f460f744ef |
fix: trust PAPERCLIP_PUBLIC_URL in board mutation guard (#3731)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Humans interact with the system through a web UI that authenticates a session and then issues mutations against the board > - A CSRF-style guard (`boardMutationGuard`) protects those mutations by requiring the request origin match a trusted set built from the `Host` / `X-Forwarded-Host` header > - Behind certain reverse proxies, neither header matches the public URL — TLS terminates at the edge and the inbound `Host` carries an internal service name (cluster-local hostname, IP, or an Ingress backend reference) > - Mutations from legitimate browser sessions then fail with `403 Board mutation requires trusted browser origin` > - `PAPERCLIP_PUBLIC_URL` is already the canonical "what operators told us the public URL is" value — it's used by better-auth and `config.ts` > - This pull request adds it to the trusted-origin set when set, so browsers reaching the legit public URL aren't blocked ## What Changed - `server/src/middleware/board-mutation-guard.ts` — parse `PAPERCLIP_PUBLIC_URL` and add its origin to the trusted set in `trustedOriginsForRequest`. Additive only. ## Verification - `PAPERCLIP_PUBLIC_URL=https://example.com pnpm start` then issue a mutation from a browser pointed at `https://example.com`: 200, as before. From an unrecognized origin: 403, as before. - Without `PAPERCLIP_PUBLIC_URL` set: behavior is unchanged. ## Risks Low. Additive only. The default dev origins and the `Host`/`X-Forwarded-Host`-derived origins continue to be trusted; this just adds the operator-configured public URL on top. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed |
