mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 09:37:19 +01:00
4e97daafa06a54f02027335437c4f48e50b84236
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
ac9459bae3 |
feat: configure company agent observability (#16874)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Companies use several agent harnesses and execution environments. > - Operators need one place to select where native agent logs and traces go. > - An inference proxy would add a dependency to the agent request path. > - This pull request supplies native OpenTelemetry settings to managed Codex and Claude Code processes. > - Company secrets protect destination credentials. Export failure does not change the task result. ## Linked Issues or Issue Description **Subsystem affected** Company settings, shared contracts, database, server, adapters, and runner. **Problem or motivation** An operator cannot configure a company destination for native agent logs and traces. Manual harness configuration is hard to apply across local and remote execution. **Proposed solution** Add one company destination: Braintrust or a custom OTLP HTTP/protobuf endpoint. Apply a transient snapshot when execution preparation starts. Keep credentials out of API reads, task environment manifests, recovery files, and command arguments. Replace a warm process when its export settings change and preserve its conversation. **Alternatives considered** Multiple destinations can use a user-operated collector for fan-out. This change does not add a collector, an inference proxy, or transcript scraping. **Roadmap alignment** This extends company settings, secret management, and support for multiple harnesses. The maintainer requested this implementation. Searches for observability and Braintrust found no duplicate company destination feature. Related infrastructure tracing work in #16236 has a separate content-free contract. ## What Changed - Add strict schemas, revision checks, encrypted secret storage, activity records, and board-only settings routes. - Add explicit synthetic logs and traces tests with a bounded HTTP request and sanitized results. - Supply native Codex configuration and Claude Code environment values through local and remote launch paths. - Preserve Codex login refresh through the existing managed credential link. Retain unsaved drafts across background reads. Require complete custom header re-entry when the endpoint origin changes. - Separate process export compatibility from conversation identity. Isolate generated Codex configuration and preserve resumable sessions, including remote home refresh after export changes. - Advance the immutable Pi/Copilot source attestations for shared environment and sidecar code changes. Keep their provider payloads unchanged and retain prior identities for decoding. - Redact destination credentials throughout launch diagnostics and HTTP logs. - Add the production settings form, Storybook states and journey, documentation, and feature/access map entries. ## Verification - Real managed local Codex 0.160.0 and Claude Code 2.1.286 tasks exported to Braintrust. API readback and the Braintrust UI showed the test records. A real remote Daytona Codex task exported through the legacy `exec` path. - Live tests exercised content capture off, export disabled, warm conversation reuse, and a destination that accepted connections but never replied. Agent work succeeded and export requests stopped at the configured three-second timeout. The UI connection test returned “Test payload accepted”; the saved API key stayed masked after reload. - All current-head CI test lanes, typechecks, packaging/build checks, and required aggregates passed on `71e994b2be9c8fb6d6e938a4ad7cf4c95fdafe31` ([CI run](https://github.com/paperclipai/paperclip/actions/runs/39319194318)). Greptile completed on that exact commit at 5/5, with zero unresolved review threads. The additional optional Paperclip Review check remains queued waiting for its assigned agent. - Local workspace typecheck and build passed. Focused verification included 239 combined runner tests, 789 API/native-session tests, 107 Codex home/ACP/sandbox-site tests, 204 feature/privacy tests, and the affected Rust suites. Migration, token, map, Storybook catalog, and Storybook build checks passed during acceptance. These checks ran at the relevant implementation or merge boundary; the current-head CI is the complete final gate. - Managed AI credential projections now receive enabled and disabled company export settings; genuine caller-owned Codex homes remain untouched. Two regressions reproduced the skip before the repair. All 276 engine/lifecycle tests and adapter-utils typecheck pass afterward. The launched TOML retains provider/MCP settings, credential continuity, and redaction. - Settings saves, removal, and the real company-deletion owner acquire secret mutation ownership before database locks. PostgreSQL NOWAIT probes cover all three paths; concurrent save/rotation/removal/deletion requests settle correctly. The affected suites passed 137 tests, followed by all 32 service/connection cases after expanded deletion coverage. Server and Codex adapter typechecks pass. - Codex source profiles use the effective CLI argument list. A spawned fixture CLI reproduced both ignored-argument failures; all three precedence/fallback cases now pass. - Real tar staging reproduced remote conversation loss in both API-key and subscription modes before the repair. Both now pass: changing managed settings preserves sessions, archives, native indexes, and refreshed subscription credentials. A corrupt upload preserves the prior config and conversation. - Preserved failure evidence: initial CI caught stale Pi/Copilot source attestations, a missing test-harness helper, and packaging fixture/probe mismatches; those were corrected and covered by focused checks. Superseded heads had intermittent attachment-composer timeouts despite successful uploads; [the original trace and screenshot remain available](https://github.com/paperclipai/paperclip/actions/runs/39309701089/artifacts/12215109203). The same attachment case passed on the final head: [25/25 tests in shard 3](https://github.com/paperclipai/paperclip/actions/runs/39319194318/job/118288637051). - A connection-fault fixture also exposed a stale pooled socket after its hang case; closing fixture responses isolated scenarios and the full affected suite passed. - The broad local test run was not clean: database resource failures, timeouts, an older cached module graph, and a stale optional-resource manifest assumption were recorded. Focused checks passed after the documented fixes. This does not establish a clean full local pass. - Native remote Codex, remote Claude, and the full ACPX Codex bridge remain unqualified. The direct Codex resume test does not establish full bridge acceptance. ## Risks - Native exporters perform network requests from each execution environment. A server connection test does not prove remote reachability. - Prompts, assistant responses, and tool content are enabled by default when supported. Native caps and harness-specific controls still apply. - Caller-owned Codex homes are unsupported for automatic configuration and remain untouched. - Native harnesses and their tool processes can inspect export credentials. The settings form and documentation advise using ingestion-scoped credentials. Retained headers remain bound to their saved scheme, host, and port; changing origin requires fresh values or removal. - Migration 0357 adds a company-scoped row and secret reference. It is safe to reapply. Secret removal and company deletion use the existing secret lifecycle. - Slow or unavailable destinations must not block task settlement. Export settings take effect at the next execution preparation boundary. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and browser verification. The exact serving model ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
0819cac4c6 |
feat(secrets): add agent-readable /secrets/catalog endpoint (#9530)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents can be configured with env bindings that reference company
secrets — they specify which secret by UUID in `adapterConfig.env`
> - But there is no API endpoint agents can call to look up a secret
UUID by name — `GET /companies/:companyId/secrets` is board-only, and
the internal `secrets.resolve` handler only accepts UUIDs
> - So when an agent needs to wire a new secret (e.g. an API key for a
new skill), it has no way to discover the UUID from a known name like
`HOMEBOX_API_KEY` — the user must find it by inspecting browser network
traffic
> - The fix is a read-only catalog endpoint that agents can call to get
the `id`/`name`/`key`/`status` mapping — no values, no provider config —
just enough to resolve a name to a UUID
> - This PR adds `GET /companies/:companyId/secrets/catalog`, guarded by
`assertBoardOrAgent` + `assertCompanyAccess`, so agents can discover the
UUID they need without board-level access and without any secret value
being exposed
## Linked Issues or Issue Description
No pre-existing public issue. Describing inline per the feature request
template:
**Subsystem affected:** `server/` — REST API & orchestration services
**Problem or motivation:**
Agents that configure env bindings must reference secrets by UUID
(`secretId`). There is no agent-accessible API to resolve a secret name
to its UUID. `GET /companies/:companyId/secrets` requires board access;
the internal `secrets.resolve` handler rejects anything that is not
already a UUID. Agents and their operators are forced to find UUIDs by
inspecting browser network requests, which is friction that should not
exist.
**Proposed solution:**
Add a read-only catalog endpoint — `GET
/companies/:companyId/secrets/catalog` — that agents can call. It
returns only non-sensitive metadata (`id`, `name`, `key`, `status`) for
each active company secret, stripped of values, provider configuration,
and version history. Board callers get the same response. The existing
full-detail list endpoint (`GET /companies/:companyId/secrets`) remains
board-only and is unchanged.
**Alternatives considered:**
- Allow agents to call the existing `/secrets` list — rejected because
it returns full rows including provider metadata; narrowing the response
is safer.
- Add a name-to-UUID lookup by query param — simpler but less useful; a
full catalog means the agent can do the resolution locally without a
second round-trip.
**Roadmap alignment:** Does not duplicate anything in `ROADMAP.md`.
## What Changed
- `server/src/routes/secrets.ts` — new `GET
/companies/:companyId/secrets/catalog` route registered before the
board-only `GET /companies/:companyId/secrets` route. Uses
`assertBoardOrAgent` + `assertCompanyAccess`. Calls `svc.list()` then
projects each row to `{ id, name, key, status }` before responding.
- `server/src/__tests__/secrets-routes.test.ts` — adds `list` to the
shared mock service object (it was missing); adds a `describe` block
with four test cases: board caller receives stripped metadata, agent
caller in the same company receives stripped metadata, unauthenticated
request gets 401, agent from a different company gets 403.
## Verification
**Automated:**
```bash
pnpm --filter @paperclipai/server test --run secrets-routes
```
All four new test cases (board access, agent access, unauthed rejection,
cross-company rejection) should pass.
**Manual:**
1. Start the Paperclip server locally.
2. Create a company and a secret via the UI.
3. Call the endpoint as a board user:
```bash
curl http://localhost:3100/api/companies/<companyId>/secrets/catalog \
-H "Authorization: Bearer <board-session-token>"
```
Expect a JSON array with `id`, `name`, `key`, `status` fields — no
`provider`, no `referenceCount`, no version data.
4. Call the same endpoint with an agent API key:
```bash
curl http://localhost:3100/api/companies/<companyId>/secrets/catalog \
-H "Authorization: Bearer <agent-api-key>"
```
Expect the same response.
5. Call with an agent API key scoped to a *different* company — expect
403.
## Risks
Low risk. This is a purely additive, read-only endpoint. No existing
behavior changes. The only new capability is that agents can discover
the UUIDs of secrets in their own company — metadata they already need
to do their job. Secret values are never returned. Authorization reuses
the existing `assertBoardOrAgent` and `assertCompanyAccess` guards
already used throughout the codebase.
## Model Used
Claude Sonnet 4.6 (`claude-sonnet-4-6`) — Anthropic, extended context,
tool use enabled. The entire change (route, tests, PR description) was
produced by the model operating as a Paperclip CEO agent assigned to the
task.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Austin Pilz <austinpilz@users.noreply.github.com>
Co-authored-by: root <root@paperclip.pilz.dev>
Co-authored-by: Internet Historian <agent@paperclip.internal>
Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>
|
||
|
|
89ce36d7af |
feat(skills): open-by-default company skill policy and core UX (#9564)
## Thinking Path > - Paperclip uses company skills to make agent capabilities reusable across an organization. > - Skill operations currently mix capability availability with permission checks, which creates avoidable setup friction and inconsistent denial handling. > - The policy contract needs to remain open by default while allowing company-scoped restrictions for governed deployments. > - Core owns the canonical policy actions, persistence, evaluation, API behavior, safe import boundaries, and generic denial/read-only UI. > - Enterprise policy-editor implementation belongs in the separate `paperclip-ee` repository and is intentionally excluded from this PR. ### Problem or motivation Company skill operations can encounter permission dead ends even when no explicit restriction has been configured, and import-source classification can drift between policy evaluation and execution. ### Proposed solution Define eight canonical skill policy actions, default all actions to allowed, persist company-scoped restrictions, expose policy evaluation APIs, normalize import sources at the boundary, and update Skill Studio to present actionable restriction states without embedding Enterprise Edition implementation in the core repository. ### Alternatives considered Keeping capability checks distributed across routes and UI surfaces was rejected because it duplicates policy logic and makes denial behavior inconsistent. Shipping the Enterprise policy editor in this repository was rejected because `paperclip-ee` is a separate repository and must receive its own PR. ### Roadmap alignment Extends the completed **Skills Manager** roadmap area by adding coherent governance and removing workflow dead ends. ### Additional context The core API contract remains suitable for a separate Enterprise Edition editor, but this PR contains no `paperclip-ee` package or EE-specific UI integration code. ## What Changed - Added the company skill policy contract to product and implementation documentation, including the open-by-default rule, eight canonical actions, decision shape, and core/EE ownership boundary. - Added the company-scoped policy schema, migration `0170`, shared validators, policy service, REST routes, OpenAPI coverage, and focused server tests. - Hardened import policy enforcement by normalizing import sources and keeping source classification consistent between policy evaluation and execution. - Updated core Skill Studio behavior to remove generic permission dead ends and show actionable policy/platform denial states only when an operation is actually denied. - Removed the `plugin-paperclip-ee` package, Docker wiring, EE discovery/deep-link helpers, and EE-specific UI tests/stories from this PR so that implementation can be submitted separately to the EE repository. - Preserved open-by-default behavior when no explicit company restriction exists. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/components/skill-studio/SkillPolicySurfaces.test.tsx src/lib/skill-policy-denial.test.ts` — 20/20 passed. - `pnpm --filter @paperclipai/ui exec tsc --noEmit` — passed. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/worktree-config.test.ts` — 12/12 passed. - `pnpm check:token-gates` — passed with all gates clean. - `git diff --check` — passed. - `git diff --name-only origin/master | rg 'paperclip-ee|ee-skill-policy'` — no matches. ## Risks - Migration `0170` introduces company policy persistence; rollout depends on the migration applying before policy routes are exercised. - Open-by-default is an intentional behavioral policy: deployments expecting implicit denials must configure explicit restrictions. - Import normalization is security-sensitive and should retain focused review. - The separate EE editor must stay contract-compatible with the core policy API as policy actions evolve. ## Model Used - OpenAI Codex CLI, runtime model identifier and context-window size not exposed by this execution environment; reasoning, repository tool use, shell execution, and code review capabilities enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available to this runtime) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked or described the result above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the latest head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Evyatar Bluzer <bluzername@users.noreply.github.com> |
||
|
Jannes StubbemannandClaude Fable 5
|
1cfed0c0ff |
security(invites): widen invite-token entropy and rate-limit public invite endpoints (#8979)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Companies onboard human members through shareable invite links; the `/api/invites/:token` endpoints are deliberately public so a recipient can view the invite and accept it without being logged in > - That publicness makes the invite token itself the only secret guarding company membership — and it was guessable: the token suffix carried only ~41 bits of entropy, and the endpoints had no rate limiting > - An attacker could therefore enumerate the token space online and accept an invite into someone else's company, gaining member access to its onboarding data, skills, and workspace > - This pull request widens invite tokens to 256 bits of entropy and puts a per-IP rate limit in front of every public `/invites/:token` sub-route > - The benefit is that invite links stop being brute-forceable while their shape, storage scheme, and UX stay exactly the same — existing links keep working ## Linked Issues or Issue Description No public issue exists; describing the problem in-PR (security/bug): **What happens:** Company invite tokens are **public**: anyone with the link can `GET /api/invites/:token`, fetch onboarding/logo/skills, and `POST /api/invites/:token/accept`. Two weaknesses combined to make them brute-forceable: 1. **Token entropy ~41 bits.** The token suffix was 8 chars over a 36-char alphabet (`8 * log2(36) ≈ 41.4` bits). That is online-enumerable. 2. **No rate limit on `/invites/:token*`.** The public endpoints had no throttling, so the ~41-bit space could be enumerated online. **Impact:** an attacker who guesses a live token can accept the invite and join the company as a member — unauthenticated, from any IP. **Expected:** invite tokens should be computationally infeasible to guess, and the public endpoints should throttle guessing attempts anyway (defense in depth). ## What Changed **Entropy** - `createInviteToken` now uses `crypto.randomBytes(32)` (256 bits) base64url-encoded, keeping the human-readable `pcp_invite_` prefix so link shape and UX are unchanged. The duplicate generator in `plugin-host-services.ts` is updated to match. - Tokens are stored **hashed** (sha256) in `invites.tokenHash`; the raw value is only returned once on creation. Storage scheme is unchanged. - **Backward compatible**: only newly minted tokens are affected; lookup is by hash of the presented value, so existing invite links keep working. **Rate limit** - New generic in-memory per-IP sliding-window limiter (`server/src/services/invite-rate-limit.ts`, 20 req/min/IP), applied as a router-level middleware on `/invites/:token` so every current and future sub-route is covered (summary, logo, onboarding, onboarding.txt, skills/index, skills/:name, test-resolution, and POST accept). - Returns `429` with `Retry-After` and `X-RateLimit-*` headers. In-memory ⇒ per-process, which bounds enumeration per replica. Mirrors the existing `company-search-rate-limit` pattern; no new dependency. - Adds a `tooManyRequests(429)` error helper in `server/src/errors.ts`. ## Verification - `invite-token-entropy.test.ts`: prefix preserved, suffix ≥ 128 bits / 22 chars, charset, 1000 unique tokens. - `invite-rate-limit.test.ts`: allows up to limit then 429s with retry-after; per-IP isolation; forgets hits after the window. - `invite-rate-limit-route.test.ts`: `GET /invites/:token` and `POST /invites/:token/accept` return 429 once the per-IP threshold is exceeded. - Manual: create an invite, open the link (works once per token as before), then hammer `GET /api/invites/<token>` >20 times within a minute from one IP → `429` with `Retry-After`. - Server package typechecks clean for all touched files. ## Risks - Low risk. Token change affects only newly minted tokens; existing links resolve via the same sha256-hash lookup. - The limiter is in-memory and per-process: in multi-replica deployments each replica enforces its own 20 req/min/IP budget. That still bounds enumeration (per-replica) and matches the existing `company-search-rate-limit` approach; a shared store can be layered later if needed. - Legitimate users behind a single NAT/proxy IP share the 20 req/min budget for invite endpoints; the invite flow makes only a handful of requests, so headroom is ample. - No DB migration, no API shape change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) — Claude Fable 5 (`claude-fable-5`), extended thinking enabled, agentic tool use (code search, editing, local typecheck) via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Supersedes #8147. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |