Commit Graph
3 Commits
Author SHA1 Message Date
ad961227f5 feat(secrets): add user-specific runtime secrets (#8825)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs often need provider credentials, API tokens, and other
environment-bound secrets.
> - Company-level secrets work for shared credentials, but they do not
model values that should differ by human operator.
> - Without a user-scoped model, a run can dispatch without knowing
whether the responsible human has supplied the needed value.
> - Paperclip also needs run attribution to make those user-scoped
runtime checks deterministic and auditable.
> - This pull request adds user-specific secret definitions, per-user
values, environment bindings, responsible-user attribution, and runtime
resolution gates.
> - The benefit is that teams can define the secret once, let each user
provide their own value, and block runs before dispatch when required
user secrets or active definitions are unavailable.

## Linked Issues or Issue Description

Refs #224
Refs #6057

This PR implements user-specific secret support as a core
secret-management capability rather than a one-off adapter setting. It
is related to existing public work on company secrets UI and runtime
secret refs, but is distinct because the value is owned by the
responsible user and resolved at run dispatch time.

Related PR search before opening found existing secrets work such as
#1550, #8256, #8614, #8634, and #8647; none of those add the full
user-secret definition/value/runtime gate covered here.

## What Changed

- Added user-secret definitions and per-user "My secrets" values,
keeping stored values out of access metadata.
- Added `user_secret_ref` environment bindings and UI affordances to
pick them alongside existing secret refs.
- Added responsible-user runtime resolution so user-secret refs resolve
against the human responsible for the run.
- Added pre-dispatch missing-secret gates so runs fail before adapter
dispatch when required user values are absent or definitions are
inactive.
- Added low-trust allowlist hardening for user-secret runtime access.
- Added issue, routine, run, and agent API key responsible-user
attribution and fail-closed dispatch behavior when attribution cannot be
resolved.
- Added denial-copy mapping so responsible-user authorization failures
surface as actionable run outcomes instead of opaque setup failures.
- Added OpenAPI documentation for the user-secret routes.
- Rebases cleanly on current `master`; migrations were renumbered
incrementally as `0128_user_specific_secrets`,
`0129_agent_api_key_responsible_user`, and
`0130_run_responsible_user_invariant` after upstream `0126`/`0127`
migrations.
- Removed previously committed local design screenshots so the PR
contains code/docs/tests only.

## Verification

- PASS: PR head `2527febd106bcf3ca264ca0da7fca491084192d6` is based on
`paperclipai/paperclip:master`.
- PASS: `git diff --check`
- PASS: `git diff --name-only public/master...HEAD | rg
'^(pnpm-lock\\.yaml|\\.github/workflows/|screenshots/)' || true`
produced no files.
- PASS: migration journal audit confirmed unique indexes through `130`
with tail entries `0126_issue_comment_derived_attribution`,
`0127_environment_custom_images_instance_scoped`,
`0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and
`0130_run_responsible_user_invariant`.
- PASS: `pnpm --filter @paperclipai/ui typecheck`
- PASS: `pnpm --filter @paperclipai/server typecheck`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-responsible-user-invariant.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-active-run-output-watchdog.test.ts
src/__tests__/heartbeat-stale-queue-invalidation.test.ts
src/__tests__/heartbeat-workspace-finalize-branch.test.ts
src/__tests__/issue-monitor-scheduler.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-comment-wake-batching.test.ts
src/__tests__/heartbeat-retry-scheduling.test.ts
src/__tests__/heartbeat-accepted-plan-workspace-refresh.test.ts
src/__tests__/heartbeat-plugin-environment.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/low-trust-red-team-routes.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/secrets-service.test.ts` (55 tests)
- PASS: `pnpm vitest run server/src/__tests__/secrets-routes.test.ts
server/src/__tests__/secrets-service.test.ts` (89 tests after final
Greptile cleanup fixes)
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-issue-liveness-escalation.test.ts` (17 tests
after the final rebase CI fix)
- PASS: focused server Vitest batches covering heartbeat recovery,
project env, plugin env, routines, low-trust, pipelines, monitors,
watchdog, and stale queue paths.
- PASS: GitHub checks are green on
`2527febd106bcf3ca264ca0da7fca491084192d6`, including Typecheck +
Release Registry, Build, General tests, serialized server suites, e2e,
Canary Dry Run, verify, security checks, and Greptile Review.
- PASS: Greptile Review completed successfully on
`2527febd106bcf3ca264ca0da7fca491084192d6` with Confidence Score 5/5,
and GraphQL review-thread audit returned zero unresolved non-outdated
threads.

## Risks

- Runtime behavior now depends on a run having a correct responsible
user; missing or incorrect responsibility assignment can block runs
before adapter dispatch.
- `user_secret_ref` bindings intentionally expose metadata without
values, but UI/API callers may need to handle the new binding kind
explicitly.
- External secret providers and IAM policies are not automatically
provisioned by this PR; operators still need to configure provider-side
access for non-local vaults.
- The PR is broad across db/shared/server/UI/runtime paths, so release
validation should include both API and UI secret workflows before merge.
- The migration renumbering is intentionally incremental after upstream
migrations; the branch migrations use guarded
column/table/index/constraint creation so users who tested the older
draft numbering should not hit duplicate DDL for the existing objects.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5-based coding agent (`gpt-5`), Codex local adapter
with shell/tool use and code execution. Context window and internal
reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 05:58:20 -05:00
DottaandPaperclip fd2f82ac5b [codex] Add built-in Hermes adapters (#8543)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent adapters are the boundary between the control plane and the
runtimes that actually do work.
> - Hermes support needs to be available as first-class local and
gateway adapters while still preserving the adapter-manager override
path for external packages.
> - The adapter work touches runtime execution, UI adapter metadata,
onboarding prompts, scoped credentials, release packaging, and smoke
coverage, so the handoff needs concrete verification rather than only
unit tests.
> - This pull request adds built-in Hermes local and Hermes gateway
support, keeps external adapter overrides compatible, and
documents/tests the gateway flow end to end.
> - The benefit is that operators can hire Hermes-backed agents without
a manual plugin install, while self-hosted installs can still
override/shadow the built-ins through Adapter manager packages.

## Linked Issues or Issue Description

No public GitHub issue exists for this exact Hermes built-in adapter,
gateway onboarding, and release-source work.

Problem description:
- Hermes local and gateway adapters need a public, reviewable source
path in the monorepo so package artifacts and built-in adapter behavior
match the application source.
- Operators need built-in `hermes_local` and `hermes_gateway` adapter
choices without losing the ability to install external Hermes packages
as overrides.
- Gateway onboarding needs secure defaults for API server URLs, API
keys, and generated agent setup text.
- Hermes-originated task bridge credentials need narrower API-key scope
configuration.
- Related public PRs found during duplicate search include #3027, #2363,
#7544, #7950, #8095, and #8543.

## What Changed

- Added the unified Hermes adapter package with local and gateway
server/UI/CLI exports, config schemas, transcript parsing, model
detection, and package metadata.
- Registered `hermes_local` and `hermes_gateway` as built-in adapters
across shared constants, server registries, CLI packaging, and UI
adapter registries.
- Kept the external adapter override path compatible so installed Hermes
packages can shadow built-ins and restore the built-in parser when
disabled.
- Added Hermes gateway onboarding docs, board-operator docs, Docker
smoke assets, and shell smoke harnesses for join/e2e validation.
- Added scoped task-bridge API-key support, authorization checks,
issue-origin handling, and tests for Hermes-created Paperclip tasks.
- Hardened gateway transport and redaction behavior for API keys,
headers, session data, and smoke diagnostics.
- Updated release packaging/bootstrap checks for the Hermes packages
while leaving `pnpm-lock.yaml` out of the PR per repository policy.

## Verification

Targeted local verification recorded before PR handoff:
- `pnpm --filter @paperclipai/hermes-paperclip-adapter exec vitest run
src/gateway/server/execute.test.ts` — 14/14 passed.
- `pnpm test:hermes-gateway-smoke` — 6/6 passed.
- Hermes package typecheck/build checks passed.
- Focused server/UI adapter tests passed — 31/31.
- Release helper Node tests passed — 18/18.
- `git diff --check origin/master..HEAD` passed.

Fresh Docker E2E smoke evidence:
- Ran `pnpm smoke:hermes-gateway-e2e` on 2026-06-26 with a fresh state
directory and fresh Docker container against a live Paperclip dev
server.
- Hermes direct execution reached `completed`.
- Hermes stop/cancel path reached `cancelled`.
- Hermes gateway created a Paperclip task, Paperclip ran the Hermes
agent, and the task reached `done` with the expected marker response.
- Temporary board auth keys, token files, smoke state, and Docker
containers were cleaned up after the run.

PR checks on head `b5eae40ce`:
- GitHub Actions passed: `policy`, `review`, `Typecheck + Release
Registry`, all general test shards, all serialized server shards,
`Build`, `Canary Dry Run`, `e2e`, and aggregate `verify`.
- External checks passed: Snyk and Socket Project Report.
- External Socket Pull Request Alerts remained pending after the
first-party CI matrix completed.

## Risks

- Medium risk: this spans adapter registration, package publishing,
gateway execution, onboarding docs, API-key scoping, and UI adapter
metadata.
- Migration risk is low: the scope-config migration adds a nullable
column and does not rewrite existing keys.
- Gateway execution depends on operator-provided Hermes API
configuration; the smoke covers the Docker gateway path but real
deployments may differ by network/auth setup.
- Direct Greptile review on the latest expanded diff is file-count
limited, although the commitperclip review gate passed.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5 coding agent, tool use enabled in a local repository
workspace. Context window size is not exposed in this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Commitperclip review gate is green; direct Greptile review is
file-count limited on the latest expanded diff
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-26 16:04:58 -05:00
ForgottenandClaude Opus 4.6 8c830eae70 Expand data model with companies, approvals, costs, and heartbeats
Add new DB schemas: companies, agent_api_keys, approvals, cost_events,
heartbeat_runs, issue_comments. Add corresponding shared types and
validators. Update existing schemas (agents, goals, issues, projects)
with new fields for company association, budgets, and richer metadata.
Generate initial Drizzle migration. Update seed data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 09:07:22 -06:00