Commit Graph
8 Commits
Author SHA1 Message Date
DottaandPaperclip 77295faebc fix(server): recover approved tool actions from committed changes (#17454)
## Thinking Path

> - Paperclip manages agent work and its governed tool calls.
> - A human approval can persist before the approving process dispatches
the tool.
> - Recovery currently scans approved requests on every maintenance
tick.
> - Completed and deliberately inert approvals do not owe another
execution.
> - Test-tab approval also needs a shared execution claim before
recovery can safely race the foreground path.
> - This PR wakes recovery from committed changes and retries only
unresolved approvals.
> - Idle history becomes quiet while existing approval and execution
checks remain with the gateway.

## Linked Issues or Issue Description

Refs #17453. This is the fifth slice of the recurring-server sleep
migration, stacked on `codex/sleep-04-tool-review-expiry`.

**Problem**
Approved tool requests are scanned periodically even when no executable
work remains. The Test-tab foreground path lacks the request claim
needed to admit concurrent recovery safely.

**Proposed Solution**
Use the existing transaction-aware review notification boundary and
shared deadline worker. Keep signed-payload interpretation, current
policy and target checks, and provider dispatch in the gateway. Share a
claim between Test-tab foreground and recovery, then persist both
authoritative outcome rows atomically.

**Alternatives Considered**
A longer interval still scans idle history. A new durable queue would
duplicate existing request/invocation state. Dispatching Test-tab
approvals without a claim risks duplicate external calls.

## What Changed

- Recheck both principals for delegated Test-tab approvals. Save their
rate receipts in the same claim transaction, share only the same bucket,
and roll back new charges on denial.

- Preserve delegated gateway authorization and both issuer/target rate
receipts inside the existing claimed-invocation transaction; retain the
durable actor during approved replay.

- Expire stale managed connector payloads before consuming approved-tool
admission.

- Persist current rate-policy charges and their invocation receipts
atomically before dispatch, preserving exact approval claim fencing and
read-only preflight.

- Recover approved requests at startup and after committed review
changes; retry only unresolved approvals.
- Leave legacy agent approvals without execute-on-approve intent inert.
Repair completed Test-tab history without another provider call.
- Fence Test-tab execution and outcome settlement with an atomic claim
shared by both callers. Revalidate the approved target and current
policy before dispatch.
- Keep provider calls outside database transactions. Retain quarantine
for interrupted execution and unknown outcomes.
- Add a concurrent, idempotent partial index for approved-request
pagination. Reuse the parent's company-scoped review signals and shared
scoped deadline worker.
- Remove approved scanning from the aggregate tick. Preserve the public
combined sweep and scheduler-disabled startup behavior.
- Document producer ownership, quiet states, lifecycle behavior, and
remaining recovery duties.

## Verification

Latest published head: `23868c1751c100a27f58087f7ce16b93c52d4eba`,
targeting master. The review-found delegated gateway pool deadlock is
repaired by passing the admission transaction through token and
issuer-authority reads. Foreground and recovery each reproduced the
deadlock with a one-connection pool; all three constrained-pool cases,
including revoked authority, now pass. All 81 HTTP integration cases
pass. The historical foreground rate-slot regression also reproduced
before its repair; verified preview hashes are accepted only by
read-only preflight, while consuming admission still requires the
complete identity under the exact claim. Five full suites passed 329
cases; the gateway service then passed all 141 cases after two
additional mixed historical hash controls. Production types/build passed
before the final normal integration of master `35820cf398` (including
independent Grok/Runner changes); integration validation is running.
Initial fixture/assertion errors and the prior E2E
restoration/cancellation deadlock remain recorded in [the owning
audit](doc/idle-approved-tool-recovery.md#gateway-admission-uses-its-transaction-connection).
Fresh-head CI, Greptile 5/5, and passing Ponytail are still required
before merge.


- Master landing reconciliation at
`8dfe3958db6e4e8a0b7499f3f5be0a396ca77361` after #17453 merged:
production and server test files are byte-identical to the accepted
320-case repair. The landed migration journal prefix is unchanged, and
#17928's independent eval reporting changes are preserved. Fresh CI and
reviews on this exact head are required before merge.

- Historical-identity repair published at
`096b7c7198c3ca864aca9377e261cbaa0a8aad6f` (production/test source
identical to integrated acceptance
`81231824d773a79fc50c835eccf7d9b3977e79fc`): all 320 accepted gateway,
policy, startup, preparation-timing and authenticated HTTP management
cases pass. Server types and the production server build pass. Four real
historical sensitive-argument regressions failed before repair;
invalid-signature, foreign-hash and stale-claim controls remain denied
without dispatch or identity changes. Both agent and Test-tab calls
deliver the original signed arguments once, and new rate policies still
apply.
- The first integrated combined run passed 319 cases and hit a Slack bot
ownership collision left in a reused HTTP test database. The guard
correctly returned 409. The full 78-case HTTP suite passed on a fresh
dedicated fixture without product or assertion changes. Earlier
fixture-construction errors are distinguished from the four real
regressions in the audit document. Fresh remote CI and reviews remain
required.

- Parent reconciliation at `736e7f56097055e21ce13b4e45d9033391483af8`:
299 cases pass (221 full gateway/policy/startup cases and 78
assembled-server connection management cases), with no skipped cases.
Server types pass. This includes real delegated Test-tab approval
admission for newly added shared and caller rate rules. Fresh remote CI
and Greptile review remain required.

Approved delegated Test-tab calls now evaluate caller and target
policies in the existing claim transaction. Real HTTP regressions first
showed a missing caller charge and shared-limit self-rejection. All 78
connection-management cases and 92 gateway cases now pass on the owning
branch, along with server production typechecking. All 78
connection-management cases also pass at combined tip
`d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover
shared charges, separate charges, rollback on denial, duplicate
approval, and rejection of a later authorized call. Intermediate
counter/identity fixture failures and a reused-database Slack collision
are recorded in [the owning
audit](https://github.com/paperclipai/paperclip/blob/bcca5b006976a471f4afd1f70c4031aa40a2b3f7/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes).
The earlier combined checks below remain evidence for their stated
revisions. Current-head CI and review remain required.

Current head: `bcca5b006976a471f4afd1f70c4031aa40a2b3f7`. This remains a
single-duty PR stacked on #17453.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/a021467f04d22be42ee5b0c87b6583b2300a62a9/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Notifications cover the existing database-owner, single-active-writer
contract. Independent SQL writers and other replicas need an explicit
wake path.
- Unknown provider outcomes cannot be replayed safely. Failed result
persistence retains an executing claim and the existing stale-execution
quarantine.
- Interrupted execution and terminal interaction projection repair
remain periodic until their separate slices. This PR changes only
approved recovery.
- A pending-only review write can trigger one empty approved-recovery
pass. Settled idle work schedules no further scan.
- The partial index uses the existing top-level concurrent migration
executor. Snapshot pruning follows repository policy.
- Cloud sleep eligibility and external alarm coverage remain unchanged.
Worker quiet is not proof of deployed instance suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 16:10:44 -05:00
DottaandPaperclip 13709b4c7a feat(connections): let agents configure connections within delegated authority (#17737)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.

## Linked Issues or Issue Description

**Subsystem affected**

Server authorization, connection services, shared contracts, native API
discovery, database schema.

**Problem or motivation**

Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.

**Proposed solution**

Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.

**Roadmap alignment**

This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.

## What Changed

- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.

## Verification

- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.

- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.

- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.

- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on bf760912 failed automatic Slack provisioning because vault
persistence cleared an already verified manifest-update challenge. The
same regression was reproduced locally. The provider-owned helper now
preserves proof only for the current signing fingerprint and callback
URL. Eight existing compatibility cases pass, including this case and
initial GitHub secret rejection. The first uncommitted GitHub-field
variant also failed that initial boundary and was narrowed to reconnect;
both failures remain in qualification.
- The upstream merge conflicted only in generated migration metadata.
Upstream 0379 is preserved, normal generation/pruning produced 0380, and
two repeated applications of the idempotent additive SQL passed. The
final combined stack passed **128 tests** (55 actual-auth, 27 OpenAPI
and 46 wizard cases) in fresh PostgreSQL. Eight Google UI cases also
pass; the two protocol cases could not initialize embedded PostgreSQL in
this fresh local attempt. Their earlier passing evidence remains
historical; fresh Linux CI must execute them. Vercel personal credential
attribution is preserved in companion #17739 with its other
ownership/consent follow-ups, keeping this PR at 99 files.
- The suite covers fresh defaults, restricted users/agents, scopes,
personal isolation, token revocation, human handoffs, callback replay,
execution approvals, authorized self-targeted policy changes,
self-approval denial, and current authority before commit. Reproduce API
checks with `pnpm exec vitest run
server/src/__tests__/connection-management.integration.test.ts
server/src/__tests__/openapi-routes.test.ts`, using a fresh database
through `PAPERCLIP_CONNECTION_MANAGEMENT_TEST_DATABASE_URL` or the
default embedded test database.
- Companion #17739 also records consent separately from token refresh so
saved Google choices survive refresh. Its eight service-picker and two
local-HTTP OAuth tests pass; both new UI cases were red before the fix.
Normal migration generation confirms no SQL change is needed for the
additive JSON metadata.
- The integration module shares one assembled server, authentication
factory, and provider lifecycle to preserve the
HTTP-to-vault-to-task-continuation proof. A reused external database
caused a Slack fixture identity conflict; the fresh final run passed.
Earlier fixture/transport and contract-update failures remain in the
companion qualification record.
- The broad local `pnpm test:run` attempt remains blocked by macOS
embedded PostgreSQL's host semaphore limit. It is not a clean pass. The
fresh Linux CI matrix must complete. Provider fixtures do not prove live
setup or an empirical LLM success rate.
- A current-head completed Paperclip assessment is not yet recorded. Its
earlier external recovery boundary has not been bypassed, and no
incomplete assessment is reported as passing. The user authorized both
merges when checks pass; merge this implementation first, then replay,
retarget, and requalify companion #17739 against the squash-merged
master.

- After the master refresh, local workspace typecheck/build, **97
actual-auth/OpenAPI cases** and **141 gateway/policy cases** pass. The
original fixed database port was occupied; Docker assigned a free
localhost port. The first neighbor setup command could not resolve root
`tsx`, leaving its schema uninitialized; the built database package
migrated the separate fixture and all unchanged cases passed. Both setup
failures are retained in qualification. Fresh Linux CI remains required;
no full local test-sweep pass is claimed.

- The f9 Greptile review scored **2/5** for two reproduced findings:
Slack test sends skipped the caller’s tool policy, and healthy App
credentials completed an unconfirmed identity handoff. Both are fixed in
this head and have resolved evidence threads
([Slack](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927765),
[identity](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927895)).
Actual key/JWT cases cover twelve Slack caller/target/recipient
combinations; the human-session GitHub OAuth callback alone remains
pending until explicit confirmation. All **99 API/OpenAPI tests** (72
actual-auth + 27 contracts), workspace typecheck and build pass. On the
combined companion, all four focused cases (including the twelve Slack
combinations), typecheck/build and maps also pass. The continuation
suite’s 50 embedded-PG cases were skipped because its database could not
start locally; fresh Linux CI must run them. Initial fixture
credential-collision and audit-projection assertion failures are
retained with the red reproductions in the companion qualification
record. No full local sweep or live consent is claimed.

## Risks

- The maintainer explicitly confirmed authorized policy administration,
including self-targeted changes that loosen controls. These writes
require management grants and the responsible user’s current authority.
Agents cannot approve their own governed action requests or edit their
own principal permission grants. The actual-auth regression exercises
both permitted administration and those denials.
- This is a broad authorization change. Incorrect scope or owner
resolution could deny permitted setup or expose another user's
resources. Integration cases cover both permitted and denied requests
with actual authentication.
- The additive nullable gateway-token column does not infer identities
for legacy clients or backfill agent grants. Legacy agent tokens without
a responsible user fail closed.
- Provider fixtures test supported setup and callback contracts. They do
not prove live provider availability.

## Model Used

- OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code
execution. The runtime does not expose a more specific deployment ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — focused suites above;
full local sweep remains blocked as documented
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — exact `2ecd6451d0` Linux CI
completed SUCCESS; external Paperclip Review is still queued
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
exact `2ecd6451d0` review completed; both f9 findings have resolved
evidence replies
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 08:20:24 -05:00
DottaandPaperclip 9c442cf48e feat(connections): deploy and configure Supabase and Vercel (#17217)
Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations.

Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior.

Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-23 11:37:29 -07:00
DottaandPaperclip 30b753f614 perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path

> - Paperclip manages AI agents and their paid work.
> - Budgets must prevent new work and stop reported overruns.
> - Cost settlement also ran serial lifecycle reconciliation for every
eligible company agent.
> - That made request latency grow with unrelated idle agents and
delayed results on stop delivery.
> - This change separates atomic admission and accounting from durable
asynchronous enforcement.
> - Scoped live usage detects overruns without writing provisional
ledger charges.
> - The result is constant query scope for ordinary settlement and
recoverable stop delivery.

## Linked Issues or Issue Description

**What happened?**

Settlement and budget checks entered company-wide lifecycle
reconciliation. A company with 1,000 unrelated idle agents performed
1,001 accounting transactions for one ordinary cost evaluation. User
requests could also wait for provider termination. Running usage
checkpoints did not enforce reported monetary exposure.

**Expected behavior**

Keep the final reservation gate authoritative. Commit charges, scope
blocks, and enforcement intent before returning. Deliver stop requests
after commit. Read only applicable policies and execution scopes for
ordinary charges. Stop long-running work when supported reported or
priced usage establishes an overrun.

**Steps to reproduce**

1. Create a disposable company with one company budget and idle agents.
2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated
idle agents.
3. Compare query counts, company accounting transactions, and duration.
4. Use the controlled partial-usage and delayed-stop fixtures to check
enforcement and request responsiveness.

**Paperclip version or commit**

The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from
`master`.

Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853,
#13075. These cover the affected service charge, cancellation, and
accounting paths. This change removes their shared reconciliation
overhead.

## What Changed

- Batch applicable policy exposure under the existing company accounting
lock. Preserve exact fractional money and zero-reservation dispatch
fences.
- Commit a durable final budget dispatch claim before provider I/O.
Retain a separate final issue/run ownership transaction through
synchronous handoff, without holding the company accounting lock or
awaiting provider work. Preserve native recovery lease checks and
duplicate handoff fencing. Browser handoff rechecks current policy; it
reuses a consumed rate slot only for the identical policy generation.
- Separate ledger spend, live observed spend, and effective held
capacity. Use the maximum of reservation and live observation for each
unfinished operation.
- Deliver budget cancellation through the existing coordinator.
Revalidate generations and persist queued/running stop fences before
external termination. Bound stop concurrency to four.
- Apply agent holds only to the affected agent. Project budgets leave
other projects running. Project real company transitions in bounded
lifecycle batches.
- Replace periodic broad reconciliation with durable pending work,
startup repair, receipt replay, and concrete retry/window deadlines.
Retain drain and idle-safety checks.
- Enforce accepted cumulative usage checkpoints without provisional
charges. Final accounting atomically replaces live exposure and settles
reservations.
- Add operator-gated budget spans, regression coverage, map updates, and
a real-PostgreSQL comparison harness. No schema or first-party telemetry
change.

## Verification

- Full workspace `pnpm -r typecheck` passes on the final ownership fix,
rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head
`4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks
pass.
- Focused real-PostgreSQL coverage includes the 292-case rebased
accounting/dispatch run, 63 browser cases, 52 run-control/cancellation
cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and
the exact Muse confirmation regression. Native executor unit coverage
passes all 768 cases. Final ownership coverage passes: 74
adapter/use-case cases, nine public regression cases, and the additional
deleted-project race. These exercise competing ownership, budget changes
and charges between phases, zero/nonzero duplicate claims, rollback,
recovery contention, read-only capacity rechecks, and controlled HTTP
startup after commit. Another 74 pure/startup/tracing cases pass. The
complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its
external security scan identified a stale-owner handoff race. The final
fix restores synchronous issue/run ownership fencing after the
accounting transaction commits. Final head `8e84f55ce8` has fresh
Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing
security scan. The complete latest-head [CI
matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808)
passes, including budget, browser, Runner, build, and canary jobs. The
unchanged Runner close fixture passed on its sole same-head job retry.
- Broad local `pnpm test:run` invocations were stopped before required
source updates (rebase and review fixes). They are not claimed as full
passes. Final full-suite coverage comes from the complete latest-head CI
matrix; local focused coverage and the incomplete broad invocations are
reported separately.
- Reproduce performance with `pnpm --filter @paperclipai/server exec tsx
../scripts/benchmark-budget-enforcement.ts
--baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with
`--reverse`. The harness uses disposable real PostgreSQL, 20 warm
samples per variant/count, and records raw timings, SQL/lock counts,
agent reads, unchanged agent state, and query plans. Final
production-source measurement revision:
`eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is
committed in `4c5f4de8b1`.

| Unrelated idle agents | Baseline median / p95 | New median / p95 |
Baseline SQL / locks | New SQL / locks | New agent reads |
| --- | ---: | ---: | ---: | ---: | ---: |
| 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 |
| 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 |
| 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1
| 0 |

- These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64.
They isolate the cost-evaluation core and complete admission path,
without hosted-region or provider latency. Prior-revision reversed-order
controls also preserved the constant query counts; the final
measurements above use the final production source.
- Complete reserve → final ownership gate → immediate offline provider
result is measured separately, with one responsible agent and one active
operation at each unrelated-agent count. Admission p95 baseline/new: 0
agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents
**24.51 / 74.05 ms**. Baseline admission was already scoped; the
stronger final fence adds bounded overhead. New counts stay constant at
53 Drizzle statements, three transactions, two company-accounting locks,
and five agent-reading statements bound to the responsible agent. Zero
unrelated-agent reads, exactly one provider callback per sample, and no
leftover runs or holds. All admission scenarios stay below 250 ms. No
external provider calls are included.
- The partial-usage/delayed-termination fixture passed 20/20: receipt
persistence p95 13 ms, receipt return to observed durable Stop fence p95
28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database
request p95 1 ms while termination was blocked. After releasing the
controlled termination fixture, acknowledgement p95 was 14 ms. These are
separately measured phases; fixture acknowledgement is not live-provider
termination latency.
- Models and provider deadlines are unchanged. No deployment, live
inference, or staging acceptance is claimed. A quiet 30-minute follow-up
waits for the merged revision on staging, then repeats classifier and
controlled budget-stop acceptance in staging, recording hosted
bookkeeping, queue delay, and actual provider acknowledgement
separately.
- Development failures retained: finalizer notification regressions
(fixed; 27/27 affected cases passed), invalid browser admission fixture
assumptions (fixed; browser coverage now passes 63/63), and first-head
lifecycle/dispatch CI regressions (reproduced and repaired). A
prior-head Cases E2E test hit its unchanged announcement-dismiss handler
race while opening a filter; the same journey passed on `1ac9b58`. The
external security review then found a stale-owner race between
accounting commit and provider entry; final verification covers a
competing run in that gap while retaining provider I/O outside the
accounting lock. Early PostgreSQL startup attempts hit local semaphore
pressure; serial reruns succeeded. The final-head Runner lane had one
failure among 1,823 tests: the unchanged `timeout-attachment` fixture
failed at its first process close after 6.28 seconds. The sole same-head
job retry passed all 1,819 active tests (four skipped), including that
fixture in 1.85 seconds. Three isolated local probes could not run
correctly because the Rust fake-provider fixture binary was absent;
cleanup masked the original startup failure. CI builds that fixture
through its normal lane. These probes are inconclusive and are not
counted as passes.

## Risks

- This changes accounting, lifecycle, and cancellation ordering. Durable
generations, source/sequence checks, ownership leases, and company
locking remain required for correctness.
- Provider stop acknowledgement can be slow or fail. Stop intent remains
pending until existing termination proof succeeds; cancelled row status
alone does not acknowledge delivery.
- Mid-run limits depend on provider usage reports or supported price
calculations. Providers that report only at completion cannot provide
exact mid-call detection.
- Signals are process-local wake hints for the supported
single-active-writer model. Durable state and startup recovery remain
authoritative.
- There are no migrations, new queue tables, or cached balances. Repeat
the classifier and controlled budget-stop acceptance after normal
rollout.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution,
and parallel coding agents. The host does not expose the exact backend
model revision or context-window size, so those values are not asserted.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 07:23:00 -05:00
DottaandPaperclip dabb8c5daf test: remove repeated setup and strengthen regression proof (#16022)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its tests protect company boundaries, approvals, durable tasks,
credentials and the board UI.
> - Several owner suites repeat large database fixtures and render
frames.
> - Some tests repeat stronger proof or pass when the behavior they
describe is broken.
> - This pull request shares exact setup, removes proven duplicate cases
and repairs weak observations.
> - It preserves distinct security, lifecycle, concurrency and
user-facing contracts.
> - The benefit is 5,027 fewer test lines and stronger regression
checks.

## Linked Issues or Issue Description

Refs #15978. This continues the merged test-audit cleanup in [PR
#15978](https://github.com/paperclipai/paperclip/pull/15978), which
removed 11,976 test/support lines. The savings in this pull request are
additional.

Related open work: [PR
#13460](https://github.com/paperclipai/paperclip/pull/13460) changes
legacy OAuth parent/version lock ordering. This pull request retains
those production paths and the existing lock-order test. Its secrets
changes compact fixtures and repair the two AWS wrapper exclusion
observations.

Scoped title searches found no open duplicate audit or
fixture-consolidation PR and no matching audit issue. The problem is
described here using the enhancement template.

**What existing behavior does this improve?**

The maintenance and regression value of the existing server, UI, CLI,
shared and wiki-plugin test suites.

**Subsystem affected**

Cross-cutting test code. Two UI modules also lose unused test inspection
exports.

**Current behavior**

Owner tests repeat database rows, service inputs, HTTP setup and render
frames. Some cases repeat stronger assertions. Two AWS lock tests record
only cleanup events, so an AWS provider write can happen inside the held
lock without failing those observations. Several other negative tests
accept or do not observe the wrong result. Worktree fixtures also use a
temporary path alias that conflicts with the existing canonical-path
checks on macOS.

**Proposed behavior**

Use small local helpers that preserve exact inputs, fresh allocations,
callbacks, query order and await positions. Remove only cases with named
stronger keepers. Tighten negative assertions and observe actual AWS
provider entry while a confirmed lock holder remains parked. Resolve
temporary fixture roots to their canonical paths.

**Reason and benefit**

Remove 5,027 net test lines and 15 redundant registered executions.
Remove 11 lines of unused UI inspection exports. Keep every distinct
company, actor, approval, credential, persistence, stale-write and
concurrency contract. No wall-time speedup is claimed for setup
compaction.

**Breaking changes**

None for application behavior, APIs or stored data. The removed exports
expose private UI test state and have no remaining callers.

## What Changed

- Consolidate repeated runtime, workspace, portability, route,
tool-access, policy, gateway and state-service fixtures.
- Share repeated UI render frames and CLI/wiki-plugin setup while
retaining the original actions and expected results.
- Remove 15 repeated case executions only where the audit records an
exact stronger keeper.
- Repair weak HTTP/UI negative assertions and two AWS account-home lock
observations. Keep the existing durable results and cleanup behavior.
- Remove the unused `__inflightGetCount` and `__secondTickInternals`
inspection exports. Keep observable request coalescing and ticker
lifecycle checks.
- Canonicalize temporary worktree fixture paths without weakening
production path checks.
- Record the owner decisions, savings, independent preservation reviews
and validation in [the dated audit
report](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-rounds.md)
and [the compact control
evidence](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-controls.json).

## Verification

- Complete owner and retained-sibling baselines passed before each audit
cutover. Independent reviewers checked the full original tests, actual
helper bodies, exact inputs, receipt provenance, async order, fresh
allocations and case registrations.
- All 14 UI/route/wiki mutations failed at their intended assertions:
ten UI/API/ticker controls, one route control and three wiki
persistence/receipt controls. Each mutated production source was
restored to its exact original bytes.
- Both old AWS lock tests passed with their respective AWS-only lock
bypass. Both repaired tests failed at the intended assertions with the
same bypasses. These give 16 intended failures and two passing weak
controls across 18 executions. Each source was restored exactly.
- Final complete state-owner/sibling run: 329/329 passed at code
revision `64370831ebe191e5ce092f85f79b6fb661df7e16`.
- Final `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build`:
passed at the same code revision.
- Delivery head `58c5c4aa241c85f4665a3159c192888bbb2a5cfa` includes the
evidence commit and master `4b766b872993045a2e787404f0475503683e5ded`.
All 27 audited test files and both application files match the locally
verified code revision byte-for-byte. Current-head CI validates the
integrated branch, including the incoming master changes.
- All eight production-owner SHA guards for the 16 deliberate failing
mutation controls still match the integrated delivery checkout. The
mutation receipts therefore apply to those current owners.
- Full default `pnpm test:run`: **failed** with exit 1 at frozen code
revision `64370831ebe191e5ce092f85f79b6fb661df7e16`; HEAD and tracked
source bytes remained unchanged. The first general-server group reported
878 passing files, four failing files and six skipped files; 17,846
passing cases, 24 failing cases and 105 skipped cases. Later groups did
not run. Failures were `http-log-redaction` (socket hangup),
`public-mcp` (initial 15-second timeout followed by hosted token
“Refresh before subscribing” failures), `remove-test-home` (macOS
`ENOTEMPTY` where permission errors were expected, plus cleanup
failure), and `workspace-git-snapshot-streaming` (300-second timeout).
All four failing test files and the relevant cleanup/MCP production
owners are unchanged by this audit. Full local success is not claimed.
- Focused complete-owner diagnosis at the same frozen revision: 149/152
passed, three failed, zero skipped. `public-mcp` passed 82/82 and
`http-log-redaction` passed 66/66 unchanged. The two `remove-test-home`
cases reproduced macOS `ENOTEMPTY` against an expected `EACCES`/`EPERM`
contract; the 32 MiB workspace snapshot case reproduced its existing
300,000 ms timeout. HEAD and tracked files stayed clean. Exact blob
comparison confirmed all four test files and the relevant cleanup/MCP
owners match both inspected master
`4b766b872993045a2e787404f0475503683e5ded` and the frozen local
revision. The three remaining local failures are unresolved; no timeout,
retry or assertion was weakened.
- Current-head CI: [workflow
38299262149](https://github.com/paperclipai/paperclip/actions/runs/38299262149)
has passed runner selection and created all downstream
test/typecheck/build gates; those gates are queued, with no failures
reported. Greptile completed successfully at
`58c5c4aa241c85f4665a3159c192888bbb2a5cfa` with [5/5 and zero
comments](https://github.com/paperclipai/paperclip/pull/16022#issuecomment-6131457592).
All review threads are resolved or absent. This pull request will remain
open after verification.

Reviewers can reproduce the required workspace checks:

```sh
pnpm -r typecheck
pnpm check:token-gates
pnpm test:run
pnpm build
```

Browser suites were not run. Their sources and user journeys are
unchanged by this audit.

## Risks

- A fixture helper can hide a specialized input or change object
freshness. Full-program expansion checks and independent reviews
preserve the exact old inputs and execution order.
- Removing duplicate proof can remove a distinct contract by mistake.
The audit records the keeper for each deletion; all distinct security
and lifecycle cases remain.
- Concurrency checks can miss progress during a fixed delay. The AWS
repairs record actual provider entry, confirm holder entry, use measured
entry windows, require exact final event order and drain both operations
in `finally`.
- The full local default run failed. Focused diagnosis reproduced two
macOS cleanup failures and one existing snapshot timeout in unchanged
owners; these remain unresolved. Current-head CI is pending. No test
deadline, retry policy, skip or CI shard was weakened to obtain a pass.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-6, reasoning, repository inspection, code
execution and delegated audit/review. The runtime does not expose the
exact served model identifier or context-window size; those values are
unavailable and are not inferred from the model catalog.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 22:23:11 +00:00
DottaandPaperclip 20ccf3f476 feat(apps): add connection grants and delegated identities (#12341)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - External tools need explicit identity and access boundaries.
> - Shared connection credentials cannot represent every user-scoped use
case.
> - Grants must stay company-scoped and support safe delegation.
> - This pull request adds connection grants, identity rules, and their
database contract.
> - The benefit is durable control over which identity an agent may use.

## Linked Issues or Issue Description

Refs #11965

This is stack 3 of 11. It depends on stack 2 and replaces another
reviewable part of #11965.

## What Changed

- Add company and user connection grants.
- Add delegated identity and membership rules.
- Synchronize database, shared, server, and UI contracts.
- Register the grant-member replacement route in the OpenAPI surface in
the same layer that mounts it.
- Add migration 0231 with replay-safe guards and coverage.

## Verification

- `pnpm -r typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/openapi-routes.test.ts` (5 passed)
- `pnpm --filter @paperclipai/db check:migrations`
- `pnpm build`

## Risks

- Incorrect grant selection could expose the wrong credential scope.
- The service enforces company and subject boundaries before credential
use.
- Migration 0231 is generated, ordered after 0230, and safe to replay.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked a public issue or pull request with `Refs #`
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:33 -05:00
DottaandPaperclip 7e00f67138 feat(connections): add v3 schema core (#9958)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies and their governed access to external systems.
> - Connected Apps build on the existing Apps and MCP gateway substrate
so companies can configure reusable, auditable integrations.
> - The current connection record does not yet have a stable public
address, explicit ownership/auth method fields, or subject-specific
credential grants.
> - Without that schema core, later OAuth, per-user authorization, token
brokering, triggers, and connector-service phases cannot enforce tenant
and subject boundaries consistently.
> - This pull request adds the forward-compatible Connections v3 schema
core while preserving the existing connection lifecycle and directly
migrating the remote MCP transport name.
> - The benefit is a company-scoped, least-privilege foundation for
one-click integrations without bypassing Paperclip secrets, profiles,
rules, or audit controls.

## Linked Issues or Issue Description

No matching public issue was found.

**Problem**

Paperclip's current app connections need a durable identity and
authorization substrate before Connected Apps can safely support
multiple setup methods, per-user credentials, provider tenants, and
managed connector services. The existing schema only models a single
connection-level credential set and uses legacy transport terminology.

**Proposed solution**

Add a stable company-scoped connection UID, explicit
ownership/auth/transport fields, a subject-aware `connection_grants`
table, and multi-key credential annotations. Backfill existing
connections and workspace grants in a reversible migration, then update
shared/server/UI contracts to the new `mcp_remote` transport name.

**Related work**

- Related foundation: #9534
- Roadmap: Connected Apps (one-click integrations)

## What Changed

- Added company-scoped connection `uid`, `ownership`, `authKind`, and
canonical transport fields across database, shared contracts,
validators, services, and UI fixtures.
- Added `connection_grants` with workspace/user subject rules, provider
tenant metadata, credential secret refs, revocation state, company
scoping, and uniqueness constraints.
- Added migration `0182_connections_v3_schema_core` to backfill stable
UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create
default workspace grants, and support rollback coverage.
- Added multi-key credential annotations and updated gateway/access
services without changing the existing lifecycle behavior.
- Updated the connection glossary, connector playbook, and security
threat model for the new identity, grant, and relay boundaries.
- Added explicit test UIDs to direct database fixtures so the new
non-null invariant is exercised across affected server suites.

## Verification

- `pnpm --filter @paperclipai/shared typecheck`
- `pnpm --filter @paperclipai/db typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/ui typecheck`
- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/tool-gateway-service.test.ts
server/src/__tests__/tool-gateway.test.ts
server/src/__tests__/heartbeat-runtime-skills.test.ts
server/src/__tests__/tool-oauth-legacy-backfill.test.ts
server/src/__tests__/tool-access-policy-service.test.ts
server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts
packages/db/src/connections-v3-schema-core-migration.test.ts
packages/shared/src/validators/tool-access.test.ts --config
vitest.config.ts` — 9 files, 218 tests passed.
- Latest-head GitHub Actions: build, typecheck, general/serialized
suites, backup/worktree restore coverage, both e2e shards, canary,
policy, and security scans pass.
- Greptile: 5/5 with zero unresolved threads.
- `pnpm check:token-gates` remains red only on five pre-existing `#9627`
color literals outside this change.

## Risks

- **Migration risk:** UID backfill and default-grant creation touch
every existing connection. The migration uses company-scoped uniqueness,
deterministic legacy UIDs with ID suffixes, and seeded up/rollback
coverage.
- **Authorization risk:** Grant rows carry credential references.
Constraints enforce workspace-vs-user subject shape, company/connection
lookup indexes, one default grant per connection, and one user grant per
connection/subject. Security review is requested specifically for this
design.
- **Compatibility risk:** `remote_http` is renamed directly to
`mcp_remote`; all repository call sites and fixtures are updated in the
same change.
- **Future-phase risk:** Subject-bound token issuance, triggers, and
connector-service relay verification remain fail-closed requirements
documented for later phases; this PR does not expose those capabilities.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex CLI coding agent. The runtime did not expose an exact
underlying model ID or context-window size; capabilities used include
repository inspection, code editing, shell execution, test execution,
Git/GitHub CLI operations, and structured reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-21 15:16:26 -05:00
DottaandPaperclip cfa5e0704e feat(mcp) [split 3/8]: add tool access policy core (#9558)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 3/8 and focuses on tool-access policy and
authorization core
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: Authorization, OAuth binding, secret projection, content
guards, and policy evaluation need a security-reviewable server
boundary.
- Proposed solution: Adds tool-access services/routes/tests plus the
runtime service dependencies directly imported by the core, without
registering the routes in the application.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `pap10341-split/02-schema-shared`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: SecurityEngineer for authz, OAuth, secrets, and
content guards; Greptile on every PR.

## What Changed

- Adds tool-access services/routes/tests plus the runtime service
dependencies directly imported by the core, without registering the
routes in the application.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck`
- Focused server Vitest run — 4 files, 143 tests passed

## Risks

- Authorization bugs could permit cross-company or over-broad tool
access; the PR remains inert until PR 4 wiring and requires dedicated
security review.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 14:22:39 -05:00