mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 05:16:50 +01:00
f03522ea8fba2dfd3b08229bfc33a753e6cbd82c
31
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
71b448b9cb |
fix(server): schedule inbound voice cleanup from durable obligations (#17580)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls need approval, expiry, and safe provider cleanup. > - The remaining voice maintenance tick scans admissions every second, including idle endpoints. > - Admission status does not prove that the provider call has ended. > - Existing admission rows can retain the cleanup owner, confirmation, and retry deadline. > - This pull request replaces the cleanup scan with committed changes and actual deadlines. > - Idle cleanup stops querying while unfinished calls keep a recoverable path to termination. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound voice admission cleanup. Its reply-delivery parent #17578 has landed; this PR targets master. Live-session recovery is in #17576. It preserves separate ownership for those services. **Current behavior** The voice maintenance tick scans expired admissions even when no cleanup is due. A session can commit before the separate approved-admission write. Expiry can then terminate that authorized call. A rejected guest call can be labeled ended before provider termination succeeds. Seven-day retention can delete that unfinished cleanup. **Proposed behavior** Wake cleanup from committed admission and session changes. Schedule the original expiry or saved retry. Claim current state under the admission lock before provider I/O. Commit approval with session reservation. Keep unconfirmed termination durable and transfer bound calls to live-session recovery. **Reason and benefit** Configured idle endpoints and confirmed history cause no recurring admission cleanup SQL or provider requests. Approval, retries, crash recovery, and cleanup remain supported. History retention and call reports stay separate migration items. ## What Changed - Recover terminal and rejected legacy admissions whose session binding was lost. Copy known termination or bind a proven matching live session for cleanup without granting approval. - Register inbound cleanup with the shared coordinator and scoped deadline worker. Yield after each 100-row scope page and reset cursors on registration. Remove only cleanup from periodic voice maintenance. Keep complete manual reconciliation. - Add commit-aware notifications at admission, approval, rejection, guest binding, pending-tool expiry, lifecycle confirmation, and session termination writes. - Commit authorized session binding and admission approval together. Recheck cleanup ownership under the same admission lock. Recover older interrupted bindings only from matching session evidence. - Add nullable cleanup confirmation, next-attempt, and provider-account fields to the existing admission table. No new queue table is added. Generated migration 0396 backfills only provable historical termination in committed 1,000-row keyset batches and adds a concurrent partial index. - Save the retry before provider contact. An unconfirmed end receipt retains cleanup. Bound calls remain owned by live-session recovery. - Verify the signed provider account before using a rotated key for unbound cleanup. A different account cannot inspect or end the saved call. Legacy admissions require their original credential fingerprint. - Prevent history pruning and company deletion from erasing unfinished cleanup. Company deletion locks endpoints before the voice owner's check, matching ingress serialization. - Document producer coverage, ownership, deadlines, verification, and remaining sleep limits. ## Verification Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply parent #17578 has landed. A normal merge integrates actual master `e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated changes, and regenerates this slice’s migration after landed migration 0395. Fresh exact-head reviews and CI are required. - The previous Ponytail 3/5 finding is repaired: legacy terminal admissions no longer enter approval-only repair, and rejected admissions with valid matching live sessions retain their rejection and persist the binding needed for session-scoped cleanup confirmation. Five regressions failed with ownership conflicts before the repair. Five conflicting-session regressions prevent an unrelated leg from being claimed. No cleanup operation grants approval or starts a provider call. - **24 complete cleanup lifecycle cases and 122 complete voice-store integration cases pass** on the repair. After actual-master integration and migration regeneration, the complete 24-case cleanup suite passes again on a new empty PostgreSQL 17 database, and the database migration integration test passes separately. The cleanup owner, voice-store test and admission schema are byte-identical to that tested repair; the cleanup fixture only changes the migration filename. Migration numbering/safety, database build, production server typecheck and build pass. - Migration `0396_tan_paladin.sql` was generated with `pnpm db:generate`. Every landed SQL migration and the complete parent journal prefix are unchanged. Generated snapshots retain exactly 0392–0396, pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and committed 1,000-row historical backfill are retained byte for byte under the new generated filename, registered for top-level execution. The real application migrator boots a fresh database, repairs an interrupted concurrent index, serializes replays and verifies eligible-history backfill across actual transactions. - The preceding published head `48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but failed CI server shard 9/12: one existing connected MCP rate-limit test expected two real-clock calls to share a fixed 60-second bucket. A controlled window rollover reproduced its exact failure. That existing fixture now freezes only `Date` around the same-window 429 assertion, verifies acceptance after advancing one window, and restores the real clock in `finally`. All 92 gateway acceptance cases pass. Production rate limiting, timeouts and assertions are unchanged. This adds one existing CI fixture to the slice; the original failed run remains recorded. - The first regression invocation skipped the suite because it used the wrong database variable; that is not a pass. The corrected invocation reproduced five ownership conflicts. The first fixed cleanup run passed 22 and failed two new low-level deletion test calls on a retained-session foreign key; the regressions now check the cleanup deletion guard at its owning boundary, with production deletion ordering unchanged. The final full suite passes all 24. - The first complete voice-store invocation passed 121 and timed out on one transcript case at the unchanged 15-second limit. That case passed in isolation on a fresh database. One subsequent complete invocation, with builds finished, passed all 122 without source, assertion or timeout changes. The underlying timeout cause is unproven; the original failure remains recorded. A new PostgreSQL startup also failed from exhausted host semaphore capacity; fresh databases on the already-owned server were used without removing shared IPC resources. - Prior source `b8b12199b9` passed 266 distinct cases and carried three guarded mutation proofs; those older results are not added to this repair’s case count. Its CI run 40641168596 failed E2E shard 3 when legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub initially rejected a retry while the run was active, then accepted one targeted retry after completion; attempt 2 passed on that same old head. The lock holder and underlying cause remain unproven. The unchanged Node policy checker’s older failure on ignored generated Copilot manifests and passing tracked-input mirror are also retained in the audit; no workflow, checker, vendor manifest or ignore rule was modified. - [Current repair and failure record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28). Current-head CI and Greptile 5/5 gate merge. A completed failing Ponytail verdict must be fixed; missing or interrupted reviews do not block, and active pending review waits are capped at 30 minutes under the maintainer’s explicit instruction. No new full local repository suite, real provider, Cloud alarm, host/database suspension or savings claim is made. ## Risks - The migration distinguishes admission status from physical termination. Legacy ended guest rejections without proof remain cleanup obligations. Missing original credentials can keep those old obligations unresolved. - Future writers must use the existing notification boundary. Independent SQL writers and multiple processes need their own wake contract. - Pending inbound cleanup now blocks company deletion instead of allowing cascade deletion to lose the provider obligation. Existing voice-history deletion restrictions remain. This deliberately includes an active anonymous inbound ring and an old call whose termination cannot be verified. An operator must stop ingress and settle the call before deletion; this slice adds no abandonment override and never treats missing credentials as proof of termination. This is the retained cleanup ownership policy, not a claim that every deletion attempt can complete immediately. - History retention and report processing still run in the remaining maintenance lane. This PR does not extend Cloud wake coverage or claim measured host/database suspension. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5398275d2a |
fix(server): schedule gateway credential expiry from committed changes (#17450)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The tool gateway stores credentials for sessions and named gateways. > - Those credentials already contain expiry dates. > - The server repeatedly scans their tables even when no expiry work exists. > - Creation signals and saved deadlines can drive cleanup without idle scans. > - This PR registers that work with the existing coordinator. > - Authentication continues to reject expired credentials independently of physical cleanup. ## Linked Issues or Issue Description Refs #17446, #17221, #17274. This is the third sleep-migration slice, stacked on #17446. **What problem does this solve?** Gateway credential cleanup queries PostgreSQL on every maintenance tick, including empty tables and retained non-expiring tokens. **What is the proposed solution?** Signal expiring token/session creation from its transaction. Restore the earliest saved expiry at startup. Retry only while cleanup or reconciliation remains outstanding. **Why is this useful?** Configured idle gateways no longer create recurring expiry traffic. Existing credential data remains the durable source of truth. Related work: #17221 and #17274 change run-token lifetime and cleanup eligibility. They remain separate. This PR preserves current master authorization and deletion rules. When those PRs are combined, update the deadline query and signals to match their new operator-token expiry predicate; run-token idle markers must not keep this worker scanning. Retain the separate run-liveness reaper and run both regression suites. Text mergeability is insufficient evidence for that integration. ## What Changed - Make the credential test clock follow real scheduled work without overshooting future-expiry assertions; production behavior is unchanged by this fixture follow-up. - Coalesce sequential credential notifications within the former cleanup cadence, preserve loaded expiry deadlines and immediate authentication rejection, and bound scheduled deletion to 500 rows per pass. - Added transaction-aware intent before the two existing credential inserts. Non-expiring named tokens do not signal expiry work. - Added earliest-expiry selection in the gateway service and a narrow registration wrapper for the shared deadline helper. - Removed gateway credential cleanup from the aggregate maintenance tick. Startup reconstructs saved deadlines. Heartbeat-disabled instances retain their startup-only cleanup behavior. - Preserved expiry/revocation checks, run authorization, audit behavior, and the existing 500-token cleanup batch limit. - Added real PostgreSQL tests and a lifecycle/producer document. Tool-action review recovery remains a separate slice. - Drain all expired session/token pages in explicit and scheduler-disabled startup cleanup, while retaining one-page scheduled passes and authentication enforcement. ## Verification - At `907098cd3656bac59c60220c73b085d0349d4b5f`, all 290 Runner transport tests pass locally (including all seven prepared-environment variants). CI run `40570815360` attempt 1 had one provider-drain settlement failure in the `close-attachment` case; the identical source passed the preceding CI lane. A single failed-job rerun is in progress. The failure remains recorded; no shutdown assertion or production behavior was changed to suppress it. - Master landing reconciliation at `907098cd3656bac59c60220c73b085d0349d4b5f` after #17446 merged: all production and test files are byte-identical to `24318d9b99ee7f044a864ff3981fc1929bf3e509`, where the 150-case acceptance and server types passed. Only commit ancestry and the plan's merged status changed. Fresh CI/review is pending on this head. - Parent reconciliation at `24318d9b99ee7f044a864ff3981fc1929bf3e509`: 150 affected tests pass across gateway credential expiry, the complete tool gateway suite, custom-image worker and startup. The accepted PostgreSQL runs skip no cases. Server types pass. Existing direct tests cover sequential-write coalescing and saved-deadline precedence; the older Ponytail finding needs a fresh review. Remote CI is pending. Current head: `10c85a5eaa288867aa2997a27aee5f675c2a713d`. This remains a single-duty PR stacked on #17446. - Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5` with normal merges. Every branch passed generated-schema checks and its target-branch migration-order gate. Existing SQL is unchanged; the 25 stack migrations follow master’s new migration. - Duty-specific producer coverage, lifecycle tests, and earlier failure records: [2026-10-08-instance-sleep-and-scheduler-inventory.md](https://github.com/paperclipai/paperclip/blob/10c85a5eaa288867aa2997a27aee5f675c2a713d/doc/plans/2026-10-08-instance-sleep-and-scheduler-inventory.md), [2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/10c85a5eaa288867aa2997a27aee5f675c2a713d/doc/plans/2026-10-24-recurring-server-sleep-stack.md). - Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355 startup/cancellation/recovery/dispatch/preservation tests**, **75 connection-management tests**, and production server typechecking pass. These combined checks do not mean every intermediate PR independently ran the full repository suite. - Real master-to-stack database upgrade passes: **379 unchanged prior migration receipts plus 25 new migrations**, with the gateway responsibility fixture preserved. - Owning-branch checks: 84 runner installation/settlement cases plus runner typechecking and generated TypeScript/Rust profile agreement; 253 GitHub review workflow cases. Stop-attribution and held-worker readiness regressions failed before their fixes. - Failure accounting: the initial combined run passed 349 cases and failed six idle assertions in a reused preservation database containing old cancelled runs. Repeating the combined command with a fresh preservation database passed. Earlier invalid project selection, merge import/type errors, and the interrupted full local suite are not counted as passing checks. - **Latest-head CI and review remain required.** Prior CI findings include GitHub idle observation, route teardown, attention cleanup, touch-picker E2E, workspace health, and voice-cleanup timing; local passes do not erase those failures. No full local repository-suite, deployed sleep, fleet cost, or live Copilot qualification claim is made. ## Risks - Shared-owner signals cover supported local producers. Independent SQL writers or another replica need a separate ingress contract; startup recovery is not live discovery. - This is scheduling only. The open run-token lifetime PRs need semantic integration as described above before combining their behavior. - Expiry cleanup can wait during drain; authentication still enforces expiry on use. Existing Cloud safety gates are preserved. Local deadlines do not prove external wake coverage or deployed suspension. - Revocation does not move expiry. Non-expiring tokens remain retained by design; this PR does not invent a retention deadline. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact serving model identifier and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Latest-head CI is now green after one evidence-backed retry. Greptile is 5/5 and all review threads are resolved. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
13709b4c7a |
feat(connections): let agents configure connections within delegated authority (#17737)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.
## Linked Issues or Issue Description
**Subsystem affected**
Server authorization, connection services, shared contracts, native API
discovery, database schema.
**Problem or motivation**
Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.
**Proposed solution**
Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.
**Roadmap alignment**
This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.
## What Changed
- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.
## Verification
- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.
- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.
- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.
- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on
|
||
|
DottaandPaperclip
|
9c442cf48e |
feat(connections): deploy and configure Supabase and Vercel (#17217)
Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations. Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior. Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d33b40a5c3 |
fix: recover saved work and provide reliable task Retry (#16965)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description **What happened?** A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. **Expected behavior** Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. **Steps to reproduce** 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
370fb50ad4 |
fix(mcp): keep sibling tools available after transient calls (#16890)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use remote MCP tools through a governed gateway. > - The gateway lists tools only while their shared connection is healthy. > - One failed call currently marks the whole connection as unhealthy. > - Later calls to healthy sibling tools then fail with `tool_not_found`. > - This pull request keeps non-authentication call failures on their invocation. > - Agents can continue their work while credential and access gates remain in force. ## Linked Issues or Issue Description Refs #10147. Related work: #15462 fixed oversized and malformed response handling. #15448 covers timeout and JSON-RPC health handling together with separate SSE and catalog discovery changes. Its patch still marks fetch failures and HTTP failures as connection errors. #15325 proposed a transport failure counter; it has merge conflicts and has not changed since October 6. #11910 is an older timeout fix with merge conflicts. This PR covers the remaining fetch and HTTP failure paths on current master, with the same narrow call-health rule for timeouts and JSON-RPC failures. It does not include the SSE or discovery changes from #15448. The timeout session-reset check follows the recovery case discussed in #15448. **What happened?** A shared remote MCP connection works until one call has a network failure, timeout, HTTP 503, or JSON-RPC error. Every later call on that connection returns `tool_not_found`, including healthy sibling tools. Connection setup then asks for user attention even though the failure did not prove that credentials need repair. **Expected behavior** The original invocation retains its failure. Authorized sibling tools remain callable. The gateway does not replay an ambiguous action. HTTP 401/403, missing credentials, and explicit insufficient scope keep their access gates. **Steps to reproduce** 1. Give an agent access to two read tools on one remote MCP connection. 2. Make one tool fail with a network error or HTTP 503. 3. Call the other tool in the same gateway session. 4. Before this fix, the second call fails with `tool_not_found` before provider dispatch. **Paperclip version or commit** `master` at `e6dcd1e8e9`. The regression runs against source with synthetic data. **Deployment mode** Any deployment that uses a shared remote MCP connection. This is a core gateway issue. ## What Changed - Preserve connection health after a network failure, timeout, non-authentication HTTP error, or JSON-RPC error from one tool call. - Retain HTTP 401/403 and insufficient-scope health gates. - Retain HTTP 401/403 gates during MCP initialization and its initialized notification, with HTTP 503 controls for both stages. - Discard only the upstream session used by a network failure or timeout. Let the next explicit call initialize again. - Add sibling-tool regression cases, authentication controls, and real HTTP header/body timeout checks. - Update runtime recovery guidance and the feature and access maps. ## Verification - Red: the four new sibling-tool cases failed on the original implementation. Each second call returned HTTP 404 with `tool_not_found`. HTTP 401/403 control cases passed. - Green: the same six-case selection passed after the production change: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-gateway-service.test.ts -t 'keeps sibling MCP tools usable|keeps the connection blocked' --maxWorkers=1`. - The regression checks the actual gateway and embedded PostgreSQL. The injected provider fault preserves the call boundary. It checks failure recording, no automatic replay, catalog retention, and successful sibling dispatch. - The real local HTTP fixture checks cancellation before headers and during a body, then a successful explicit call with a fresh MCP session. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Complete nearby suites pass: 166 tests across `tool-gateway-service.test.ts`, `tool-gateway.test.ts`, `remote-mcp-protocol.test.ts`, and `mcp-http.test.ts`, with one test worker. - `pnpm --filter @paperclipai/server exec tsc --noEmit` passes after refreshing Runner TypeScript declarations. The first package typecheck used declarations from before the rebase and failed two unchanged native-session call sites; rebuilding that dependency resolved both errors. - The rebased head also passes all 11 selected sibling/authentication/session regressions. - Review regression red: setup-stage HTTP 401/403 controls failed because connection health stayed `ok`; the two setup-stage HTTP 503 controls passed. Green: all six setup controls and the complete 166-test nearby run pass. The fix separately handles typed initialization authentication failures and preserves the existing caller error. The server compiler also passes after this change. - Full repository CI passes on `256a65ccc302c1c31a80e87f1f91411d955028de`: typecheck, build, test shards, Runner checks, browser checks, Canary dry run, and aggregate gates. All checks are terminal: 55 successful, 2 expected skips, and the separate Paperclip Review check is neutral. Greptile is 5/5 on this exact head, with no open findings. - Full repository checks were run in CI. Local verification in this worktree used the focused suites and server compiler to keep parallel work bounded. - This change has no live provider or staging deployment proof. ## Risks - A provider that remains unavailable can stay listed until a health check detects the outage. Each call still fails with its original controlled reason code. - No automatic retry occurs because a failed remote action may already have run. - Already unhealthy connections still need a successful health check after an upgrade. - No schema, credential policy, or permission contract changes. ## Model Used - OpenAI Codex, GPT-6 family, with tool use and code execution. The runtime does not expose the exact serving model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1e731520bb |
fix(mcp): deduplicate discovered tool descriptors (#16281)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive governed app tools through the MCP gateway. > - GitHub review tasks can contain a PR discussion and inline finding threads. > - Discovery repeated each tool for every conversation linked to the task. > - Native tool registration treated these repeated descriptors as a fatal collision. > - This pull request lists GitHub tools once and deduplicates identical descriptors across providers. > - Agents can continue work while conflicting definitions and normal tool permissions remain enforced. ## Linked Issues or Issue Description **What happened?** A follow-up run on a GitHub review task failed with `assigned_mcp_tool_name_collision` before the provider started. The task had a main conversation and an inline finding thread. Discovery returned the same seven tools twice. Repeated descriptors from other providers could cause the same failure. **Expected behavior** Discovery exposes each identical tool once. Separate connection names remain separate tools. Conflicting definitions remain an error. Every call still passes the gateway's policy, approval, revocation, and audit checks. **Steps to reproduce** 1. Run a GitHub review and publish an inline finding. 2. Send a follow-up request on the same PR task. 3. Inspect the task-bound bot catalog and the next native startup. 4. Separately, return two identical descriptors from an assigned app or plugin catalog. Related: #13717 added the GitHub review workflow. ## What Changed - Use a distinct GitHub discovery query across conversations that share an endpoint and task. Keep both thread mappings. - Deduplicate identical gateway descriptors by their full gateway name across providers. Use this rule for MCP listings, plugin listings, assigned-tool startup, and fresh assigned-tool discovery. - Reject conflicting descriptors with `tool_name_collision`. Keep the projected-name collision guard. - Cover GitHub inline-thread discovery, repeated plugin/app tools, direct and on-demand catalogs, revocation, conflicting bindings, and separate connection names. - Update the execution contract, GitHub documentation, and agent access map. ## Verification - Regression baseline: six new assertions failed on the original implementation. GitHub returned fourteen entries, plugin discovery repeated a tool, and native startup failed on repeated app tools. - Focused verification passed: 272 GitHub cases, all 29 assigned-tool cases, and all 88 gateway acceptance cases. - `pnpm -r typecheck` passed. - `pnpm build` passed. - `node scripts/check-maps.mjs` passed. - Current-head CI passed: 55 checks passed, two inapplicable checks skipped. Greptile scored the current head 5/5 with no unresolved threads. - Full local `pnpm test:run` encountered a 15-second timeout in the existing bearer-auth throttling test, before tool discovery. That test passed in the separate 88-case gateway run and on one isolated rerun. The duplicate broad local run was then stopped; the complete current-head CI matrix passed. ## Risks - Deduplication keeps the first identical descriptor and preserves listing order. Same-name descriptors with different definitions fail with HTTP 409 rather than selecting an ambiguous binding. - The GitHub query changes result cardinality. It retains the existing task/company/agent filters and leaves conversation rows intact. - No schema migration or credential change. Live staging recovery requires deployment and a new authorized run. ## Model Used OpenAI Codex, based on GPT-6, with code inspection, editing, browser investigation, and test execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1e0cfcb4fc |
test: retire redundant suites and consolidate owner coverage (#15978)
Retire duplicate ACPX and Codex proof after migrating unique assertions to their behavioral owners. Consolidate repeated fixtures and exact case matrices while preserving lifecycle, protocol, company, credential, approval, and ordering contracts. Remove the unused runtime-controls compatibility path and document test ownership and preservation evidence. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
50b1f95e79 |
fix(tool-gateway): keep MCP connection healthy on oversized/malformed responses (#15462)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call remote MCP tools through the Paperclip tool gateway > - The gateway keeps a health status for each remote MCP connection, and it hides the tools of a connection that has `healthStatus = "error"` > - One oversized, malformed, or invalid-JSON `tools/call` reply set the whole connection to `error` > - Health goes back to `ok` only after a successful call, but the hidden tools prevent that call, so the connection stayed locked until a person reconnected it > - This pull request makes these reply errors fail only the one call, and it starts a new MCP session for the caller on the next call > - The benefit is that one large or bad reply no longer disconnects a working connection for every agent ## Linked Issues or Issue Description No public issue exists. Related open PRs fix other health downgrades in the same function. They do not overlap with this change: - Refs #11910 (timeouts and JSON-RPC errors) - Refs #15325 (transport errors) **What happened?** An agent called a remote MCP tool that returned more than `MAX_REMOTE_MCP_RESPONSE_BYTES` (1 MB). The gateway returned 502 `mcp_remote_response_too_large`. It also set the connection to `healthStatus = "error"`. Then `connectedMcpConnectionFilter` hid all tools of the connection (except `per_user` connections), and `connections_search` showed the connection as `needs_user_action`. The `malformed_response` and `invalid_json` errors from `readMcpHttpResponse` did the same thing. `readMcpHttpResponse` can also cancel the reply stream before the end. The remote server can then close its MCP session. The gateway kept the cached `mcp-session-id` for up to 30 minutes and cleared it only after a 404. The next calls then failed with "Remote MCP session expired". **Expected behavior** A reply that is too large or not correct fails only that call. The connection stays healthy, and its tools stay visible. The next call uses a new MCP session. **Steps to reproduce** 1. Add a remote MCP connection with `mcpSessionRequired: true`. 2. Call a tool that returns a reply larger than 1 MB. 3. Look at the connection: `healthStatus` is `error`. 4. Start a new gateway session: the tools of the connection are not in the list. **Paperclip version or commit** `master` at `99a9de9940bf5974352d9dbfbb2f21e62e89689f` **Deployment mode** All modes. The fault is in the server tool gateway. ## What Changed - `server/src/services/tool-gateway.ts`: `too_large`, `malformed_response`, and `invalid_json` from `readMcpHttpResponse` now fail only the call. The caller gets the same 502 reason code as before. The gateway does not call `markRemoteConnectionHealth(…, "error")` for these errors. The `invalid_json` branch after `JSON.parse(body)` also does not change health now, so all `invalid_json` paths are the same. - The `mcp_remote_response_too_large` message now tells the caller to request a smaller result, for example a narrower query or a smaller page size. The error details now include `maxBytes`. - `server/src/services/mcp-http.ts`: new `forgetMcpHttpSession()`. It removes only the cached session for one scope and one credential set, and only while that entry still holds the session ID that failed. It does not touch other agents, sessions that are initializing, or a newer session that replaced the failed one. - The gateway calls `forgetMcpHttpSession()` after these reply errors, so the next call from that caller initializes a new session. The existing 404 "session expired" path now uses the same function. Before, both paths cleared all sessions and all pending initializations on the connection. That made a concurrent initialization by another agent fail with "MCP connection changed while initializing", which the gateway reported as a fetch failure and marked as a connection `error`. - `forgetMcpHttpSessions(connectionId)` is not changed. Disconnect and revocation still clear the full connection. Why `malformed_response` and `invalid_json` are also per-call errors: - Each error is about one reply body. The server was reachable, accepted the credentials, and sent HTTP 2xx. A reply can be too large or bad because of the tool and its arguments. That is not a fault of the connection. - The other malformed-reply checks in the same function (payload is not an object, or has no `result`) already throw `remote_mcp_malformed_response` and do not change health. Only the reader-level errors changed health. - Health recovers only after a successful call. An `error` status for one bad reply therefore locks the connection until a person reconnects it. - Other failures (HTTP errors, fetch failures, timeouts, JSON-RPC errors) still change health. This PR does not change them. #11910 and #15325 address some of them. ## Verification - `server/src/__tests__/tool-gateway.test.ts`: the recovery test now runs for three replies: oversized, invalid JSON, and a reply without the requested message ID. Each run uses a fake HTTP MCP server with `mcpSessionRequired: true` that gives `session-N` for each `initialize`. Each run checks that: - the call fails with the correct 502 reason code (and, for the oversized reply, the smaller-page hint and `maxBytes: 1000000`) - `healthStatus` stays `ok` - a new gateway session still lists the tool and can call it - the second `tools/call` uses `session-2`, not `session-1` - New test: agent A gets an oversized reply while agent B initializes a session on the same connection. Agent B's call completes, health stays `ok`, and the two calls use `session-1` and `session-2`. With the old connection-wide reset, this test fails: agent B gets 502 `mcp_remote_fetch_failed`. - `server/src/__tests__/remote-mcp-protocol.test.ts`: new unit test. `forgetMcpHttpSession()` keeps the session of a different identity, and a late failure from an old session does not remove the newer session. - Each regression check fails without its fix. Without the health change, the test fails on `healthStatus: 'error'`. Without the session reset, it fails with `['session-1', 'session-1']`. ``` cd server npx vitest run src/__tests__/tool-gateway.test.ts src/__tests__/tool-gateway-service.test.ts src/__tests__/remote-mcp-protocol.test.ts Test Files 3 passed (3) Tests 132 passed (132) ``` - I did not run the full server `tsc --noEmit` locally because the sandbox does not have sufficient memory. The CI typecheck covers it. ## Risks - Low risk. The change affects only the error path of remote MCP `tools/call`. - A remote server that always sends bad replies now keeps `healthStatus = "ok"`. Each call still fails with a clear 502 reason code and an audit record, so the failure stays visible. Only the connection-wide hiding of tools stops. - After one of these errors, the next call from the same caller sends one more `initialize` request. This adds one round trip. - A 404 "session expired" now clears only the session of the caller that got the 404. Before, it cleared the cached sessions of all agents on the connection. If the remote server restarts, each agent now gets its own "session expired" error one time and then initializes again. A 404 is about one session, and the old connection-wide clear also cancelled other agents' initializations. - #11910 and #15325 change the same catch block. The PR that merges last can have a small merge conflict. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, 1M-token context window. - Run as an agent in Claude Code with tool use (shell, file edit, and local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no user-facing documentation changes are necessary) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge This PR replaces #15418. It keeps the same commits on a branch name without an internal ticket ID, and adds a fix for the Greptile review on #15418. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
9786f6df56 |
fix(runner): preserve credential content in document saves (#14937)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Runner sends authorized tool calls to the control plane. > - Agents use these calls to save plans and instruction files. > - The Runner used diagnostic secret detection to reject execution arguments. > - Ordinary credential-related prose could reject a document save before persistence. > - This pull request forwards the original arguments and leaves credential policy to the provider harness. > - The benefit is reliable saves with useful diagnostic records. ## Linked Issues or Issue Description Related foundation: Refs #12415 and #14430. No duplicate save-policy fix was found. **What happened?** A `write_document` call failed before the server saved its plan. The Runner reported `semantic tool input contains credential material; refusing to execute altered arguments`. The detector also masked ordinary phrases such as `secret manager` and `credential handling` in diagnostics. Both TypeScript dispatchers had equivalent execution gates. One dispatcher also rewrote structured approval and question payloads before execution. **Expected behavior** Paperclip forwards authorized arguments unchanged. The provider harness decides credential-content policy. Log and audit redaction does not reject or rewrite save input. **Steps to reproduce** 1. Send an authorized `write_document` call with a plan that discusses credential handling. 2. Include an intentional credential value in the body to exercise harness-owned policy. 3. The old Runner rejects the call. With this change, the document service stores the exact body. 4. Diagnostic records still mask explicit credential values. Qualified credential fields, short bearer values, opaque diagnostic pairs, and valid encoded JSON token headers have regression coverage. **Paperclip version or commit** Reproduced at `c46e41e81c03cd3c8b64cf993615b604d7fe8c62`. The branch is based on current `master`. **Deployment mode** Server deployment with the native Paperclip Runner. Local regression tests use the real document service and an embedded test database. ## What Changed - Remove credential-content vetoes from Rust admission and both TypeScript semantic dispatchers. - Preserve original structured approval and question arguments during execution. - Keep transport bounds, schema checks, authorization, idempotency, and audit masking. - Require explicit credential syntax or recognized formats for diagnostic masking. Preserve ordinary prose, metadata, and dotted identifiers. - Test exact document persistence, replay, nested argument identities, and masked audit copies. - Remove obsolete retry guidance and document harness-owned credential policy. ## Verification - `cargo test --manifest-path packages/paperclip-runner/runner/Cargo.toml --locked -p paperclip-runner-core --lib --test acpx_event_payload --test acpx_provider_state --test acpx_provider_turns`: 355 tests passed. - Focused server and adapter tests: 189 tests passed after rebase. These include the real document save and the complete tool-gateway suite. - Semantic dispatcher and conformance tests: 34 tests passed. - Diagnostic redaction and MCP tests: 46 tests passed, including all six review examples. - `pnpm -r typecheck` and `pnpm build` passed on the repaired branch. - The broad local root suite was interrupted after database fixture setup failures. The focused database suites passed. CI runs the complete configured test lanes. ## Risks - Authorized tool arguments can intentionally contain credentials. The harness must enforce its content policy. - Diagnostic detection is narrower. Explicit assignments, credential fields, and recognized credential formats remain masked. - The change does not add a database migration or change company authorization. ## Model Used - OpenAI GPT-6 through Codex. The session exposes the GPT-6 model family; its exact runtime model identifier and context window size are not exposed. Used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7a52dcdc74 |
fix: repair MCP validation and cancelled execution recovery (#14951)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The tool gateway gives agents access to connected services. Recovery controls what happens when a run stops. > - Generated tool names can exceed the provider limit after the MCP client adds its prefix. > - The same invalid definition can fail each automatic retry. A cancelled run can also hold saved messages without showing its cause. > - This pull request bounds tool names, stops configuration retries, and retains cancellation evidence. > - It shows the stopped run and admits saved input only after the existing safety checks pass. > - The benefit is a clear recovery path that preserves operator Stop and prevents duplicate message delivery. ## Linked Issues or Issue Description **What happened?** A long connected MCP tool name makes the provider reject the entire request. Automatic recovery repeats the invalid request. Separately, unexpected legacy cancellations can leave saved input behind a recovery hold. The notice does not identify the stopped run or its cause. **Expected behavior** Complete MCP names fit the provider limit. Tool-definition errors require configuration repair. Cancelled runs retain their source and reason. The recovery notice shows the cause and saved-message count. Verified unexpected cancellations can start a fresh turn through the existing admission checks. **Steps to reproduce** 1. Assign an App gallery connection with a long application key and tool name to a Claude agent. 2. Start a run. The provider rejects a name over 128 characters, including its MCP prefix. 3. For cancellation recovery, stop a legacy provider turn without an operator Stop request and send a user message while the recovery hold is active. 4. Inspect the recovery notice and the deferred message queue. **Paperclip version or commit** Rebased onto master at `cf8ad63c806685bfd7c48e3ed4a919d61a7c55f1`. **Deployment mode** Hosted or self-hosted server with legacy Claude or Codex execution. Related public work: - Refs #14017. That PR caps name segments. This PR preserves existing short names and uses stable hash aliases for long complete names. It also covers classification and recovery. - Refs #4510. That PR adds a cancellation-source column. This PR records bounded evidence in the existing run result, without a migration. - Refs #12552 and #4506. Those PRs suppress recovery after operator cancellation. This PR preserves operator intent and uses the existing continuation gates. ## What Changed - Bound gateway names with the full provider prefix in the 128-character budget. Retain the original upstream tool name for dispatch and permissions. - Classify invalid tool definitions as configuration failures before diagnostic redaction. Stop automatic retries and continuation attempts for that error code. - Persist cancellation source, expectedness, initiator, reason, and time. Preserve recorded Stop intent when adapter results arrive. Report unexpected started cancellations with closed diagnostic labels. - Show the run cause, saved-message count, and Inspect run link. Offer Continue for eligible unexpected cancellations. Require verified provider stop, empty tool inventory, ownership, and the existing pause, budget, approval, and dependency gates. Use the existing queue for single delivery. - Add regression coverage and update the execution, MCP gateway, and run-log documentation. ## Verification - `pnpm -r typecheck` and `pnpm build` passed. - `pnpm check:token-gates` passed. - Ran `pnpm test:run` and completed its workspace and serialized groups. Initial resource and timing failures passed on isolated reruns. All 149 serialized route suites passed. - Reran the changed server, adapter, and UI suites after the rebase. Coverage includes long-name upstream dispatch, configuration retry suppression, cancellation evidence retention, privacy labels, oversized run projection, and concurrent saved-message delivery. - `pnpm test:e2e tests/e2e/legacy-failure-continuation.spec.ts` passed all six browser scenarios. The recovery notice shows the run cause and inspection link, and each recovery entry point reaches one new response. - Added database-backed checks for active, removed, paused, unavailable, and disabled chat connections. The final continuation and recovery-notice suites passed 167 tests. Externally bound chats hide board Continue and show a usable next action. - All 55 GitHub checks passed on `42afbf1371dcaeb72646e3d8f65c19ff7cddf8de`. Two unrelated Storybook jobs were skipped by their normal conditions. Greptile reviewed that commit at 5/5 with no findings and no open review threads. ## Risks - Long tool names change to aliases. Existing short names stay compatible. The original connection and upstream name remain the dispatch authority. - Invalid tool definitions no longer get automatic retries. An operator must repair the configuration before a new attempt. - Continuation changes apply only to positively identified unexpected legacy cancellations with complete empty tool inventory. Operator Stop, unknown historical cancellations, outstanding tools, and unverified provider termination keep their holds. - No database migration. The added projection fields are optional. Cancellation reason and initiator IDs remain local run evidence; Sentry receives only closed source and initiator-type labels and expectedness. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository editing, shell execution, and GitHub tool use. The runtime does not expose the exact model variant or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ad55d0a281 |
fix(connections): repair personal credentials and request write access (#14739)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use Apps through a gateway that checks identity, company access, and action policies. > - Personal pasted credentials can point to company secrets. Setup can show success while the gateway rejects every call. > - Several OAuth methods also omit the scopes needed for their supported write actions. > - This pull request gives setup, health checks, and invocation the same credential rules. Owners repair existing connections by reconnecting. > - New connections request reviewed permissions for their supported actions. Read-only choices remain available under Advanced. > - Agents can use the connections people give them, while existing consent, identity boundaries, and action restrictions remain enforced. ## Linked Issues or Issue Description Refs #14009 and #14008. This addresses the personal-credential defect. The separate GitHub organization-identity selection defect is outside this change. Related work: #13942 fixed part of new personal-key setup. #14200 independently fixes legacy personal reconnect and protects managed-agent profile credentials during removal. This PR covers that ownership invariant across key and secret-URL setup, reconnect, health, discovery, and invocation, and keeps owner reconnect as the repair path. #14059 tracks requested versus provider-asserted OAuth scopes; it remains separate work. I searched open PRs and issues for Zapier, Airtable scopes, connector writes, and personal credential failures. **What happened?** A Zapier secret URL saved through personal setup can become a company secret referenced by a user grant. Health checks bypass the gateway's ownership check, so the connection appears healthy but calls fail with `grant_credential_invalid`. Custom-header paths can also receive a duplicate `credentials.` prefix. Omitted OAuth scopes make write access depend on provider defaults. **Expected behavior** Personal invocation credentials belong to the selected user. Setup, health, and actual calls enforce the same rule. New connections request documented permissions for supported read and write actions. Existing tokens gain no permissions without provider consent. **Steps to reproduce** 1. Connect Zapier or a generic secret URL with the personal identity. 2. Allow an agent to use the connection and complete setup. 3. Invoke a tool through a run-scoped gateway. The legacy layout fails ownership validation despite successful setup. **Paperclip version or commit** The implementation started from `44736c9c7c67b7b646ead9d51721db10f5b83835` and was rebased onto master at `94e8dec56`. **Deployment mode** Built from source. Regression tests use isolated PostgreSQL fixtures and controlled MCP transports. ## What Changed - Share credential writing, ownership validation, and canonical paths across initial setup, resume, reconnect, rotation, health, discovery, and gateway calls. Keep OAuth client-registration secrets separate from invocation credentials. - Existing personal connections with company-scoped credentials require owner reconnect with a fresh key or secret URL. Reconnect creates a correctly owned value and updates the existing grant and declarations. There is no automatic ownership backfill or new startup hook. - Preserve PostgreSQL timestamp precision when reconnect checks whether a grant changed. Previously, converting the timestamp to a JavaScript Date could reject reconnect with a false concurrent-change error. - Protect credentials used by other grants, connections, bindings, managed-agent profiles, routine triggers, or secret proposals from connection removal. - Review all 117 tool methods, including 84 OAuth methods. Record explicit scopes or documented provider-default exceptions with official evidence. Add Airtable's seven scopes, Hugging Face repository/job scopes, and other documented MCP permissions. - Prefer available write-capable methods. Put explicit read-only choices under Advanced. Explain pasted-key permissions and offer reconnect for missing OAuth consent. Preserve existing grants, policies, Google availability gates, and curated scope allowlists. - Reconnect generic secret URLs and custom headers using their stored credential fields. Refresh the catalog after setup, correct reconnect feedback and error guidance, and let Cancel exit invalid setup while Save & exit retains draft-saving behavior. - Apply ownership checks to the new GitHub repository/skill connection picker. Align the permission audit with the Google scope reductions merged on master. - Add run-scoped gateway, ownership, owner-reconnect, OAuth URL, insufficient-scope, UI, and catalog-wide regression coverage. Update the connector playbook and permission audit. ## Verification Latest commit `97bc0b86e0eae0ec892e4ac44beff1a66164b20e` passes all CI/status gates (55 completed check runs, no failures or pending checks) and has a completed Greptile review at **5/5 with no outstanding findings**. GitHub reports the PR as mergeable/CLEAN. - **Embedded browser:** used the actual server and built UI from this worktree, a fresh isolated database, and local HTTP MCP fixtures. Completed personal bearer-key, secret-URL, and custom-header setup; reproduced the legacy ownership failure; reconnected through the owner’s form; and completed writes afterward. Read-back was verified for bearer-key and secret-URL connections. Public organization-wide setup appeared immediately in Browse without reload. Zapier URL validation/Cancel and Google’s enrollment gate were also exercised. - **Persistence and invocation:** verified user ownership, canonical `credentials.authorization` / `remote.url` / `headers.X-Api-Key` declarations, and unchanged connection/grant identity. The old company secrets retain their ownership. Separate HTTP calls through an actual run-scoped gateway session completed a write and read-back. - **Backend coverage:** the final gateway suite passes all 82 cases, including catalog Zapier and generic inline reconnect. It checks company/user isolation, canonical declarations, same-endpoint URL validation, fresh credentials, retained restrictions, and real gateway read/write execution using fixture transport. A timestamp with PostgreSQL microseconds covers the former false reconnect conflict. - **Local checks:** 368 catalog, gateway, repository, and UI tests passed before the final extra Zapier case; 49 GitHub skill access tests also passed. All three Apps browser regressions pass, including reconnect through the actual form and catalog visibility without reload. Full `pnpm -r typecheck`, `pnpm build`, server typecheck after the final patch, and token gates passed. Full tool-access service runs hit varying 15-second Google fixture timeouts; both affected cases and the updated reconnect assertion pass in isolation (3 tests). The complete test matrix passes in CI on this head. - **Verification limits:** no live provider account was available for Zapier/Airtable/OAuth consent or account-bound write proof. Public metadata and local fixtures do not establish provider consent. The original development database clone failed on a pre-existing missing `tool_connections_transport_check` constraint; browser acceptance used a fresh isolated database created by the normal CLI onboarding flow. ## Risks - Existing broken personal connections stay unusable until their owner reconnects. Health, discovery, and invocation return an actionable ownership error; startup does not rewrite credential ownership. - Scope changes affect new authorization requests. Providers may still require resource selection, account roles, paid plans, or app verification. Existing consent and action restrictions remain unchanged. - Shared credentials are retained rather than reassigned or revoked. Provider-default exceptions and unavailable live checks are documented in `doc/connections/CONNECTOR-PERMISSION-AUDIT.md`. - No new endpoint, database table, lockfile change, or CI workflow change is included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code editing, shell execution, web research, and browser tools. The exact deployment model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2de43fc909 |
fix(issues): keep agent mentions as context and defer personal app authorization (#14577)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each task has one assignee. Explicit assignment and review requests select who should act. > - An agent mention started another agent on a task it did not own. Native attachment staging then rejected that run. > - Allowing that run through startup could also let two agents work on the same task. > - Mentions should identify relevant context. They should not start work or forward comments to other tasks. > - A personal app installed on a shared agent must also wait until tool use to resolve the current user's grant. > - This pull request removes mention dispatch and keeps missing personal app credentials from blocking startup. ## Linked Issues or Issue Description **What happened?** A native agent mentioned on another agent's task failed with `paperclip_runner_attachment_staging_not_authorized`. The source task could already be complete. A nearby optional-app warning was a separate problem: personal app tools were excluded when their shared health state required attention. **Expected behavior** An agent mention is context only. It does not wake the agent, take ownership, or copy a comment onto another task. Normal feedback still reaches the assignee. Assignment and explicit review requests still dispatch work. An unavailable personal app does not block startup or produce a startup warning. Tool use requests the current user's authorization and never uses another user's grant. **Steps to reproduce** 1. Assign a task to agent A. Post a comment that mentions agent B, including a comment that closes A's task or references B's child task. 2. Confirm the comment retains its agent link and B receives no run or deferred wake. A can still receive normal feedback. 3. Install an active personal MCP connection on B. Give only Alice a grant and leave shared health at `error`. 4. Explicitly assign work to B for another user. Confirm it can finish without using the app. 5. Ask B to use the app. Confirm its tool call shows an inline connection request for the current user. Related work: Refs #11144. This change uses the existing execution-time personal grant resolution. ## What Changed - Remove mention dispatch from standalone comments and issue updates. Remove implicit forwarding of parent comments to a mentioned worker's child task. - Ignore new requests with the legacy mention wake reason before creating a run or deferred request. Preserve already accepted queue entries, which can combine assignments and feedback with a later mention. - Remove the native mention admission, staging, and finalization exceptions from this PR. Native task ownership checks remain intact. - Keep active, installed personal app tools available despite shared health errors. Remove optional-app startup warnings. Tool execution retains the current user's grant and policy checks. - Update agent instructions and product/API docs. Refresh generated capability source anchors. ## Verification - Red: comment-route regressions reproduced extra agent wakes and child comment forwarding. A separate regression proved that cancelling by the last coalesced reason could drop an accepted assignment. - Green: the targeted route, wake queue, heartbeat, workspace, responsible-user, MCP discovery, and HTTP gateway suites passed. The final queue and heartbeat rerun passed 104 tests, the restored queue adapter passed 56, and both comment-route suites passed 135. These include accepted assignment preservation, rejection of new mention requests, and normal assignee feedback. - `pnpm -r typecheck` and `pnpm build` passed locally. The full local `pnpm test:run` attempt was interrupted for review/CI fixes, so it is not claimed as a completed local pass. It exposed a cleanup timing race in the concurrent-mention assertion, now fixed and verified across 10 repetitions. CI also exposed an obsolete test waiting for the removed mention lookup; it was reproduced and fixed, then both comment suites passed. Final full-suite verification is through CI. - Final head `bd9ea4cb05a8f081c54e017760a8999f9ea6ef44`: 54 checks passed, 2 Storybook checks intentionally skipped; no pending or failing checks. Full CI includes general and serialized suites, all 8 browser shards, runner verification, typecheck, build, and canary dry run. Greptile is 5/5 on this exact commit, with no unresolved findings. - One unchanged Cursor adapter test hit its 10-second CI timeout. All 5 tests in that file passed locally; one retry of its CI shard passed all 674 tests (3 skipped). The aggregate verification gate then passed. No code or timeout was changed for that retry. - Live browser check: inserted a structured mention with the picker on a human-owned task. The saved link remained visible. Database checks found zero new runs and zero wake requests. - Live Codex runner check: explicitly assigned that task with the unavailable personal app attached. The run succeeded and committed completion without using the app or creating a connection card. - Live browser follow-up: asked the assignee to call PostHog and mentioned another enabled agent as context. Only the assignee ran. It succeeded and displayed the existing inline connection card. Only Alice's grant existed; the run belonged to a different user. - The HTTP regression covers tool discovery with no provider calls or connection cards, first use returning the current user's authorization request, and successful retry after that user's grant exists. - App checks use an isolated local fixture and a fake MCP provider. They do not use production app credentials. ## Risks - Intentional behavior change: workflows that used mentions to wake agents must use assignment, a bounded child task, or an explicit review request. - Already accepted queue entries retain their prior rules. An old entry can combine assignment or feedback with a later mention; its last reason cannot safely identify mention-only work. New mention requests create no run or deferred wake. - Personal apps with a shared health error remain discoverable. Actual tool use still requires the responsible user's grant and existing policy gates. - No database migration or public API schema change. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser testing. The exact serving model ID and context-window size are not exposed in this session. - Live native-run verification used `gpt-6-astra` through the Codex provider. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
18dac1e1ef |
feat(connections): add experimental memory providers and remote MCP access (#13942)
## Thinking Path > - Paperclip manages AI agents and their work. > - Connections give agents governed access to external tools. > - Agents need durable memory across tasks and execution environments. > - Mem0, Zep, Supermemory, Cognee, and Honcho provide hosted memory tools. > - This pull request adds their setup flows behind an experimental toggle. > - It also delivers assigned MCP tools through the native remote Codex runner. > - Operators can connect a provider once and use the same governed tools locally or in Daytona. ## Linked Issues or Issue Description **Problem or motivation** The Apps catalog lacks a complete set of memory providers. Remote native Codex agents also need access to assigned managed MCP tools without receiving provider credentials. **Proposed solution** Add five memory connectors behind the disabled-by-default Experimental memory connectors setting. Use the existing connection setup and permissions UI. Default their tools to Allowed. Preserve company boundaries, operator permission changes, provider scopes, and audit attribution. **Alternatives considered** Direct provider credentials in each sandbox would duplicate setup and bypass the managed gateway. The remote runner instead uses its existing protocol channel to call the gateway on the server. **Roadmap alignment** This maintainer-requested experiment supports the Memory / Knowledge and Connected Apps roadmap areas. It adds provider connections without introducing a separate memory UI. Related connector authoring documentation is tracked in #13692; no duplicate memory-provider implementation was found. ## What Changed - Add provider definitions, official branding, and the experimental setting for all five providers. - Use OAuth for Zep and Supermemory, API credentials for Mem0 and Honcho, and a bundled Cloud API bridge for Cognee with no runtime downloads or subprocesses. - Default memory tools to Allowed and classify destructive actions explicitly. - Fix personal remote credential resolution and propagate provider tool errors. - Relay assigned managed MCP tools to remote native Codex through the runner protocol. Recheck current authority for each call and rotate stale tool contracts. - Add 21 Storybook states and complete OAuth walkthrough fixtures. - Document provider research, sanitized tool inventory, and live local and Daytona proof. ## Verification - Passed workspace typecheck: `pnpm -r typecheck`. - Passed production build: `pnpm build`. - Full local `pnpm test:run`: 13,273 passed, with failures from process/readiness timeouts under parallel load. Reran all 16 affected suites with one worker: 456 passed, leaving two macOS `/var` versus `/private/var` path assertions. Both passed with `TMPDIR=/private/tmp`. No test failures remain unverified. Latest-head remote CI passes all 54 checks (two optional Storybook jobs skipped). Greptile is 5/5 with no unresolved findings. - Latest Cognee gateway regression: 71 passed, including public deployment without a runtime host and immediate recovery after a provider error. Bundled bridge tests: 25 passed. - Browser setup and real agent tasks exercised all five providers. Mem0, Cognee, Zep, and Honcho have successful store/retrieve proof. - Supermemory now has scoped read/write consent. Local storage and real Daytona write, document read, and semantic recall passed; indexing completion was verified before claiming success. - All five providers were exercised through a real Daytona sandbox and its native runner MCP relay. Provider credentials remained on the server. The final bundled Cognee bridge also passed a fresh Daytona store/recall run. All disposable sandboxes were removed and verified absent after testing. - Storybook is rebuilt and contains the experimental toggle, catalog, setup, permissions, and error states. The Zep and Supermemory access steps advance correctly. ## Risks - Provider OAuth scopes and plan limits remain independent of Paperclip tool permissions. An Allowed tool can still be rejected by the provider. - Providers can queue memory indexing; save acceptance does not prove that semantic recall is ready. - Remote tool contracts must stay synchronized with current connection authority. Regression tests cover revocation and stale contracts. - This change has no database migration. Existing connections remain usable when the experimental catalog toggle is disabled. ## Model Used OpenAI Codex, GPT-6 Astra (`gpt-6-astra`), with reasoning, repository edits, code execution, and browser automation. The exact context window size is not exposed in this session. Live acceptance agents used `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b82661b561 |
refactor(connections): retire the legacy Composio broker (#13758)
## Thinking Path > - Paperclip manages agents and their access to external tools. > - Connectors expose these tools through a governed MCP gateway. > - PR #13755 added a direct Composio MCP connection behind the experimental MCP aggregators flag. > - The old project API-key broker still created toolkit child connections and showed a separate Services tab. > - Keeping both paths leaves obsolete setup and session code in the product. > - This change removes the broker and preserves direct MCP setup, credentials, permissions, and execution. > - Saved legacy records fail closed and remain available for explicit removal. ## Linked Issues or Issue Description Related: #13755. This retirement supersedes the legacy-path fixes proposed in #12630, #12632, #12634, and #12906. It does not close those PRs. **What existing behavior does this improve?** Composio connector setup, management, and runtime dispatch. **Current behavior** Composio offers both direct MCP and a project API-key broker. The broker mints sessions and creates one child connection per toolkit. **Proposed behavior** Offer only direct MCP. Remove the toolkit Services UI, REST routes, API client, and session broker. Block saved legacy parent and child records from discovery, execution, health checks, reconnect, and OAuth. Preserve their records and credentials until the operator removes each connection. **Reason and benefit** The direct MCP connector becomes the single supported Composio workflow. Provider accounts remain managed in Composio. ## What Changed - Remove the API-key catalog method and its generated-source definition. - Delete Composio broker clients, session creation, account synchronization, child lifecycle, and toolkit routes. - Remove the Services tab, service rows, child provenance, and cascade-removal controls. Keep Vercel provenance intact. - Retain a shared retirement guard for stored legacy records. Show Retired status and replacement/removal guidance in the connection list and details; hide obsolete runtime controls. - Preserve the experimental MCP aggregators flag and direct MCP infrastructure. - Replace broker fixtures with retirement tests and extend direct Composio catalog/reconnect coverage. ## Verification - Focused shared, server, and UI tests passed with one worker. Server retirement tests use a name filter; no full local test suite was run, as requested. - Server and UI TypeScript checks passed. - Token gates and UI build passed. - Real browser: opened the saved Composio connection, refreshed all 11 tools, and ran the provider's read-only GitHub account-list operation through the standard Test dialog as an agent. The provider returned success using the existing OAuth credentials. - See `doc/connections/COMPOSIO-BROKER-RETIREMENT.md` for scope and live evidence. - Storybook build passed. A fresh real agent used `COMPOSIO_SEARCH_TOOLS` and `COMPOSIO_MULTI_EXECUTE_TOOL` to return the actual Paperclip DeepWiki hierarchy: one success, zero errors. Gateway audit records confirm both calls succeeded. - Browser retirement check: a credential-free legacy fixture showed the guidance, opened the direct MCP replacement flow, and was removed through the standard confirmation. - Focused regressions for the experimental settings copy and exact OpenAPI route coverage passed. All latest-head CI checks passed (54 successful, two intentionally skipped); Greptile scored 5/5 with no unresolved review threads. The PR has no merge conflicts. ## Risks This intentionally breaks the old Composio project API-key and child-connection workflow. Existing legacy records cannot run, even if their stored status is active. Operators must create a new direct MCP connection and choose access rules; credentials and grants are not migrated. Remove each old record separately to delete its credentials. No schema migration or data deletion runs automatically. Direct MCP connections keep their existing grants and secrets. ## Model Used OpenAI GPT-6 via Codex, with reasoning, code execution, and browser tools. The exact runtime variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e8c8ba3c19 |
feat(apps): add experimental MCP aggregator connectors (#13755)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its tool gateway applies company access rules and approval controls to connected apps. > - MCP aggregators expose many apps through one provider endpoint. > - Each aggregator needs its own credential, catalog, grants, and lifecycle in Paperclip. > - This pull request adds independent Zapier, Arcade, Composio Connect, and Executor setup with a common Access → Connect layout. > - A default-off MCP aggregators flag lets operators opt in while we complete provider acceptance tests. > - Agents use the normal Paperclip permissions, Test screen, and gateway after setup. ## Linked Issues or Issue Description **Subsystem affected** Apps, connection setup, shared contracts, and the remote MCP gateway. **Problem or motivation** Aggregator endpoints need clear provider setup and correct MCP sessions. Generic setup does not explain each provider's authentication or broad execution tools. Provider approval must preserve the original execution instead of replaying a write. **Proposed solution** Add four separate connectors behind Settings → Experimental → MCP aggregators. Start with human and agent access, then connect the endpoint and read its tools. Enable tools by default. Use the existing Permissions and Test screens after setup. Keep legacy Composio API-key and child connections intact. **Alternatives considered** A shared connection for all providers would mix credentials and access rules. Separate provider-specific permission and test screens would duplicate existing controls. Vercel Connect is outside this change. **Roadmap alignment** Extends the existing MCP Tool Gateway & Apps capability and the Connected Apps roadmap area. This work was requested and reviewed by the maintainer. Related work: #11894, #12630, #12632, #12634, and #12906 concern the legacy Composio broker. #13102 also covers remote MCP pagination. This change preserves the broker path and adds initialized sessions, response matching, and provider resume handling alongside pagination. ## What Changed - Add branded setup and interactive Storybooks for Zapier, Arcade, Composio Connect, and Executor. Use the existing access controls and normal action tests. Do not request a connection name or action choices during setup. - Add the default-off `enableMcpAggregators` flag to settings, managed feature metadata, the catalog, and setup guards. Hidden connections keep running. Legacy Composio connections remain unchanged. - Reuse the vault, grants, policy, and catalog models. Support OAuth discovery, bearer tokens, custom headers, and credential-bearing URLs. Add no database tables or migrations. - Initialize and retain Streamable HTTP sessions by connection and effective credentials. Read paginated catalogs and match streaming responses to request IDs. - Classify unfamiliar aggregator tools as writes despite upstream read-only hints; only exact reviewed read capabilities enter the read-only allowlist. Legacy Composio child behavior is preserved. - Preserve provider authorization links and execution IDs. Support Executor approve/resume, decline, and cancel without automatic replay of uncertain writes. - Preserve Off and Ask first choices during refresh and reconnect. Allow new tools and retire removed tools. Keep agent access updates atomic and preserve an empty agent selection. - Document connector UX rules, provider branding sources, and live acceptance results. - Stabilize the existing Sentry release fixture after its repeated CI failure by reusing one module mock; production Sentry behavior is unchanged. ## Verification - Final head `d11781970`: [CI run](https://github.com/paperclipai/paperclip/actions/runs/35633534900) passed, including broad typecheck, test shards, build, and E2E. All 54 checks pass; 2 optional checks are skipped. Greptile is 5/5, Security Scan passes, and all review threads are resolved. - Passed 27 focused connector Vitest checks and 18 connector-only Storybook browser checks before the flag change. All 85 stories rendered at desktop and narrow widths. - Passed 5 connector lifecycle/server checks and 7 selected flag checks after adding the flag. The latter cover settings, managed defaults, cached catalog visibility, and all four setup routes. - Review fixes passed 13 risk/handoff/lifecycle checks, dedicated session-expiration and transport regressions, 13 selected connector/gateway CI cases, and 10 selected setup/reconnect UI cases. A real Composio connection-list call also succeeded through the refreshed UI on `9ab115f71`. - UI and server TypeScript checks passed. UI build, Storybook build, token gates, and diff whitespace checks passed during implementation. - Real browser and real Paperclip agent tests passed for Arcade, Composio, and Executor. Tested action permissions, denied agent access, reconnect, disconnect, and isolation. Tested Arcade catalog additions/removal and Executor provider approve/resume, decline, and cancel. - Zapier live acceptance is incomplete. Its dedicated provider server is configured, but its credential-copy dialog returned an empty clipboard through browser automation. No live Zapier action is claimed. - The three isolated Sentry release cases pass after the CI fixture fix. - Local verification is deliberately narrow at the maintainer's request. The full local suite, recursive typecheck, and repository-wide build were not run. CI provides the broader checks. ## Risks - Shared MCP transport changes affect other remote MCP servers. Protocol fixtures cover initialized sessions, streaming response matching, pagination, and isolation. - Broad execution tools remain broad permissions. The provider governs actions inside those tools. - Provider handoff links are retained briefly in memory. After a server restart, a one-time link may require reopening the provider dashboard. Paperclip does not replay the original call. - Zapier remains unproven live. Custom-header imports and self-hosted endpoints have fixture coverage rather than a separate live account for every variant. - Turning the experimental flag off hides setup; it does not revoke existing credentials or stop existing connections. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, shell execution, and browser automation. The exact runtime model ID and context-window size are not exposed in this session. A separate Anthropic-backed Paperclip agent performed live gateway acceptance tasks. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
165b10bd98 |
fix: enable GitHub Actions MCP toolset (#13553)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The GitHub connector lets agents use repository tools through MCP. > - GitHub excludes Actions from its default MCP toolsets. > - Approval of Actions permissions therefore does not make workflow tools appear in Paperclip. > - This pull request adds Actions to the requested toolsets for discovery and execution. > - Users can refresh existing connections and use workflow tools under the existing access rules. ## Linked Issues or Issue Description **What happened?** GitHub Actions tools remain absent after the GitHub App receives Actions read/write access and the user refreshes actions in Paperclip. Paperclip does not request the Actions MCP toolset. **Expected behavior** Authorized GitHub connections expose workflow tools, including `actions_run_trigger` with `method: "run_workflow"`, so agents can dispatch an existing release workflow. **Steps to reproduce** 1. Connect GitHub to Paperclip with access to a repository that has a dispatchable workflow. 2. Grant the GitHub App Actions read/write permission and approve the installation update. 3. Refresh the connection's actions in Paperclip. 4. Observe that the workflow tools are absent. **Paperclip version or commit** Base commit: `fae698031`. **Deployment mode** Hosted instance with a managed GitHub connection. The same missing header affects PAT connections. No matching public issue or pull request was found in the duplicate search. ## What Changed - Send `X-MCP-Toolsets: default,actions` through the shared GitHub MCP header helper. This covers discovery, refresh, and execution for existing and new managed or PAT connections, including legacy rows identified through `transportConfig`. - Test catalog refresh, tool risk classification, and workflow dispatch through a mock MCP server. - Document tool names, workflow arguments, required GitHub permissions, and the refresh step. ## Verification - Passed both affected test suites: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway.test.ts` (391 tests). - Passed `pnpm check:token-gates` and `git diff --check`. - Live provider check: the default catalog returned 45 tools. `default,actions` returned 49 tools, with no tools removed. The four added tools were `actions_get`, `actions_list`, `actions_run_trigger`, and `get_job_logs`. - Live `actions_get` / `get_workflow` call succeeded. No workflow was dispatched during live verification. - Passed `pnpm -r typecheck` and `pnpm build` with the existing Rust toolchain added to PATH. - Rechecked server typecheck and build after the legacy-connection fix; both passed. - The full local test run has reported three skills-cache failures in `company-skills-service.test.ts`. All three reproduce on the untouched base commit (`fae698031`) on this macOS host: runtime-cache directory renames fail with `EACCES`. The full run remains in progress. - Greptile: 5/5 on `7d391e3c7`, with no unresolved review threads. - After deployment, use **Refresh actions** on an existing GitHub connection and verify the workflow tools appear. ## Risks - Refreshed GitHub catalogs expose more tools. Existing access, approval, and quarantine rules still apply. `actions_run_trigger` keeps GitHub's destructive classification because it also supports cancellation and log deletion. - GitHub still enforces token and installation permissions. Dispatch requires Actions write permission and a workflow with `workflow_dispatch`. - No database migration or saved connection edit is required. ## Model Used OpenAI GPT-6 through Codex, with code execution and tool use. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e200104727 |
feat: review connection actions from tasks (#13063)
Bring governed connection reviews into task history and composer approvals. Share resolution with Connections, add scoped remembered permissions, and resume agents through durable outcome receipts. Keep cards compact, collapse raw results, isolate untrusted provider output, bound continuation payloads, and reconcile missed live events. Add Storybook coverage, browser journeys, and service regression tests. Verification: all PR CI gates passed, Greptile 5/5, security scans passed, five connection-review browser journeys passed, and real native Codex approval/continuation was verified against the local MCP fixture. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0ffc091473 |
feat(connections): add durable GitHub identities and webhooks (#12843)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents need source control access for repository work > - A shared token cannot preserve the responsible person's identity or an agent's dedicated identity > - GitHub App tokens also need durable refresh, repository access checks, and webhook delivery > - Paperclip already has managed connections, encrypted grants, run secret leases, and merge-confirmation behavior > - This pull request extends those systems with GitHub identities instead of adding a parallel credential system > - The benefit is durable GitHub access with explicit identity, repository, runtime, and webhook boundaries ## Linked Issues or Issue Description No public GitHub issue describes this connection change. This description follows the feature request template. **Subsystem affected** Connected Apps, connection grants, secret resolution, native Git runtime setup, webhook processing, and the Apps UI. **Problem or motivation** Users need to connect GitHub once and let agents use the correct GitHub identity. A run should use a dedicated agent account when one exists. Otherwise, it should use the responsible person's account. The connection must survive token expiry, repository access changes, and temporary instance downtime. **Proposed solution** Add user-owned and agent-owned GitHub grants to the existing connection model. Resolve one identity for MCP, Git, `gh`, health checks, and webhook bindings. Store provider tokens in the existing encrypted secret system. Refresh expiring token pairs under the existing lease and compare-and-swap path. Register signed Cloud webhook bindings and process normalized pull request and installation events through a durable local inbox. **Alternatives considered** An organization-wide GitHub token would lose person and agent attribution. Environment variables alone would bypass the managed connection and grant model. A new GitHub-only credential store would duplicate the existing secret and access systems. GitHub App installation tokens and private-key custody remain outside this first version. **Roadmap alignment** This change implements the Connected Apps direction. It also extends the shipped MCP Tool Gateway, per-agent secret access, and action-attribution systems. It does not add a repository catalog. The open repository catalog work in [#11234](https://github.com/paperclipai/paperclip/pull/11234) is related and complementary. ## What Changed - Added agent-owned connection grants and a per-agent credential policy with company and subject constraints. - Added a managed GitHub App method while keeping the personal access token method as an advanced fallback. - Added durable access-token and refresh-token handling with proactive rotation and one automatic recovery after a provider `401`. - Added GitHub identity and installation summaries without storing repository-name lists. - Added signed Cloud webhook binding, event lease, acknowledgement, local idempotency, pull request merge processing, and installation access handling. - Added one identity resolver for MCP, native Git, `gh`, checkout, health checks, and webhook bindings. - Added a class-3 run projection for `GH_TOKEN`, `GITHUB_TOKEN`, a `github.com`-only credential helper, SSH-to-HTTPS rewrite, and GitHub noreply commit attribution. - Added personal and dedicated-agent setup choices plus identity, repository, continuity, and webhook status in the Apps UI. - Added schema migrations, tests, and connection documentation. ## Verification - The current head is fully green in GitHub CI, including build, typecheck, all serialized/general server shards, all browser shards, policy, canary dry run, review, and security checks. - Live staging proof completed with a non-expiring GitHub App user token, selected-repository installation, repository add/remove refresh, managed MCP, native `gh`, HTTPS clone/push/delete, GitHub noreply commit attribution, signed merged-PR webhook acceptance, durable Cloud-to-instance delivery, and installation-access event processing. Temporary branches and temporary repository access were removed afterward. - `pnpm check:token-gates` passed. - `pnpm -r typecheck` passed before and after the rebase onto `origin/master`. - `pnpm build` passed. - The focused connector suite passed 285 tests after the rebase. - The full stable suite passed 5,790 tests and failed 22 tests across 8 general server files. The failures reproduced as shared-runner environment issues. They included `/tmp` versus `/private/tmp`, closed database connections, and invalid high ephemeral ports. The focused connection tests pass in isolation. ## Risks - Migrations add agent grant subjects and a durable connection-event inbox. Migration numbering and safety checks pass. - A raw GitHub user token enters the agent process for Git and `gh`. Per-tool Ask-first controls cannot limit those shell operations. The UI warns users about this boundary. - GitHub App user tokens can be non-expiring. Paperclip performs a continuity check every 30 days, but provider revocation still requires a reconnect. - The webhook path accepts only signed and bounded payloads. It stores a minimal normalized record and no raw provider payload. - GitHub repository permissions remain authoritative. Removed access can make a cached repository count temporarily stale, but runtime access fails immediately. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, `gpt-5.6-sol`, extended reasoning, tool use, code execution, browser control, and multi-file repository editing. The context window size was not provided. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandDev Agent
|
25cf079ec5 |
feat(runner): add Codex-native application integration (#12591)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner package is useful only when the application can start, observe, and recover a native Codex run safely. > - Existing direct adapters must keep their current execution and finalization paths. > - The application boundary therefore needs additive persistence, authorization, coordination, and recovery behind an explicit experimental adapter. > - This pull request adds that Codex-only boundary without activating generalized providers, remote environments, or the later task/SDK surfaces. ## Linked Issues or Issue Description **Subsystem affected** Shared contracts, database persistence, adapter utilities, server native-runtime services, and the experimental Paperclip Runner adapter. **Problem or motivation** The already-landed runner package has a qualified Codex path, but the application needs durable native-run state, guarded runtime selection, authenticated coordination, tool security, finalization, and recovery before the experimental adapter can be exercised safely. **Proposed solution** Add a Codex-only `paperclip_runner` application path behind the existing default-off native-runner setting. Bind native state and coordination to company/run identity, preserve persisted-run recovery, and leave every direct adapter on its existing legacy execution path. **Alternatives considered** The earlier stack boundary introduced a generalized executor and remote-environment lifecycle here. That made this PR depend on implementations in higher PRs and changed reusable sandbox behavior globally. Those pieces are now deferred together to #12592. **Roadmap alignment** ROADMAP.md does not list a conflicting native-runner integration project. This change adds the application boundary for the existing Runner architecture. ## What Changed - Added native run/result/finalization/provider-trace persistence, shared validators, and idempotent migration/replay coverage. - Added guarded Codex-only runtime selection, authenticated PRP coordination, recovery, finalization, and interaction services. - Added run/company-bound tool-gateway authorization, credential redaction, SSRF protections, and replay-safe behavior. - Added the explicit `paperclip_runner` adapter behind the default-off rollout setting. - Preserved legacy answered-question wake projection and direct-adapter execution/finalization paths. - Hardened cancellation so only owned in-memory child processes are signaled; persisted recycled PIDs/process groups are never trusted. - Retained the narrow Claude ACPX isolated-context security follow-up discovered after #12590. - Deferred the generalized executor, provider ingress, remote lifecycle, SDK/lab/eval work, release-process changes, and lockfile. ## Verification - Changed-file delta against `master`: 133 files. - GitHub Actions is the authoritative verification environment for this PR. - Full CI, security, and Greptile review will run on this lowest unmerged stack PR. - Local tests/build/typecheck were not run because this checkout is resource constrained. - Static diff/reference checks pass, and `pnpm-lock.yaml` is unchanged. ## Risks - This touches central heartbeat and agent-route code, so legacy compatibility is the primary risk. - Runtime selection remains Codex-only and explicit; direct Codex, Claude, OpenCode, process, HTTP, and plugin adapters remain on their existing paths. - Fresh native starts fail closed while the rollout flag is off; persisted native records remain readable and recoverable. - Cancellation, company/run binding, tool calls, status decisions, and completion writes are guarded or replay-safe. > For core feature work, check [ROADMAP.md](ROADMAP.md) first and discuss it in #dev before opening the PR. Feature PRs that overlap with planned core work may need to be redirected. ## Model Used OpenAI Codex, GPT-5.6, with repository tools, code execution, and parallel agent review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [ ] I have run tests locally and they pass — GitHub Actions is authoritative for this resource-constrained checkout - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [ ] All Paperclip CI and security gates are green - [ ] Greptile is 5/5 with no open actionable findings - [x] I will address all Greptile and reviewer comments before merge ## Stack - Position: 3 of 5 overall; lowest of 3 currently unmerged - Base: `master` - Previous: [#12590](https://github.com/paperclipai/paperclip/pull/12590), qualified Claude ACPX runtime — merged - Next: [#12592](https://github.com/paperclipai/paperclip/pull/12592), generalized Codex executor, task experience, and developer SDKs --------- Co-authored-by: Dev Agent <dev@paperclip.ing> |
||
|
Dotta
|
a560b48d6d |
feat(apps): refine Postman and Shopify setup (#12357)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps give those agents governed access to external tools. > - Provider catalogs must match each provider's current protocol and credential contract. > - Postman method labels and API-key placement were outdated. > - Shopify now offers a UCP commerce endpoint that needs a managed agent-profile argument. > - This pull request updates both providers and documents the complete connection-authoring workflow. > - The benefit is accurate setup, safer runtime defaults, and a repeatable provider review process. ## Linked Issues or Issue Description Refs #11965 This is stack 10 of 11. It depends on stack 9 and preserves the final catalog work recovered from #11965. Related: #5904 covers Shopify skill routing. This pull request covers the Apps connection contract instead. ## What Changed - Update Postman hosted MCP methods, capability choices, default selection, and bearer-token placement. - Add Shopify UCP commerce and Storefront compatibility methods with public-store prerequisites. - Inject the reviewed Shopify UCP agent profile at runtime and remove that managed field from user input schemas. - Classify Shopify checkout completion and cancellation as destructive actions. - Expand the connection authoring runbook from provider research through verification and pull request handoff. - Add focused shared, server, and UI coverage. - Make the approved-execution waiter phase-aware so slow preparation cannot consume the provider execution timeout and grace period. - Settle legacy pre-execute-on-approve requests and invocations as failed, clear their stale idempotency key, and allow a fresh governed approval instead of leaving work stuck in `executing`. ## Verification - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts server/src/__tests__/tool-access-service.test.ts ui/src/pages/apps/AppsConnect.test.tsx -t "Postman|Shopify|normalizeConnectionMethodConfig|classifyRisk"` (16 passed) - `pnpm exec vitest run server/src/services/approved-execution-wait.test.ts` (4 passed) - `pnpm exec vitest run server/src/__tests__/tool-gateway.test.ts -t "enforces policy, approvals, retries, rate limits, and company boundaries for connected remote MCP calls"` (1 passed) - `pnpm exec vitest run server/src/__tests__/tool-gateway-service.test.ts` (21 passed; includes legacy approval settlement and fresh-approval recovery) - `pnpm --filter @paperclipai/server typecheck` - `pnpm check:token-gates` - `pnpm -r typecheck` - `pnpm build` ## Risks - Shopify UCP calls now include a Paperclip-managed agent profile that overrides caller input at the same path. - Postman EU credentials now use the hosted MCP server's bearer-token contract instead of the general REST API header. - The catalog generator and checked-in definitions change together to prevent regeneration drift. - Approved execution preparation has an explicit two-minute bound; provider execution retains its own 65-second timeout and persistence grace starting from durable provider start. - Legacy approvals created before execute-on-approve are intentionally terminalized and must be requested again under the current signed contract. > I checked `ROADMAP.md`. This provider update does not duplicate planned core work. The related open Shopify PR addresses skill routing, not Apps connections. ## Model Used OpenAI Codex, GPT-5. The runtime exact model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
b3343dbd64 |
feat(connections): add self-serve intent runtime (#12345)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents need a governed way to request app connections during issue work. > - The catalog now describes the available providers and setup methods. > - A request must become a durable, company-scoped intent before an operator acts on it. > - This pull request adds that intent runtime across server, agent, CLI, and shared contracts. > - The benefit is a safe bridge from agent need to operator-approved setup. ## Linked Issues or Issue Description Refs #11965 This is stack 7 of 11. It depends on stack 6 and replaces another reviewable part of #11965. ## What Changed - Add connection intent types, validation, service logic, and routes. - Add agent runtime tools and CLI support for connection requests. - Add issue-thread interaction support for connection intents. - Add runtime, route, adapter, and contract tests. - Hold the final resolved-continuation row lock through asynchronous adapter preparation until an actual process spawn, so parking or reassignment cannot cross that boundary. - Report Hermes Gateway's first remote run request through the shared dispatch hook so the resolved-intent lock is released at the true dispatch boundary. - Revalidate the addressed user's live non-viewer membership and connection-management authority for every intent mutation, including OAuth completion. ## Verification - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - Result: 176 tests passed. - `pnpm build` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-stale-queue-invalidation.test.ts` (32 passed; includes non-process dispatch lock-release coverage) - `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/connection-intents-service.test.ts -t "addressed-user mutation"` (1 passed) - `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/tool-access-service.test.ts -t "binds OAuth callback completion to the initiating board session"` (1 passed) - `pnpm --filter @paperclipai/hermes-paperclip-adapter test -- src/gateway/server/execute.test.ts` (23 passed; includes dispatch-hook ordering and exactly-once coverage) - `pnpm --filter @paperclipai/hermes-paperclip-adapter typecheck` ## Risks - A malformed intent could create an unusable operator request. - Validators and company checks reject invalid or cross-company requests. - The final continuation gate holds the issue row lock through adapter preparation until process or remote dispatch; later operator changes use the normal active-run interruption path. - The change does not add a database migration. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked the public source pull request with `Refs #` - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6244e4cf32 |
feat(apps): add Composio and Gmail connectors (#12342)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - App connections need both direct providers and managed provider hubs. > - The grant layer now defines safe credential ownership. > - Composio needs parent and child connection lifecycle rules, and Gmail needs governed setup. > - This pull request adds both connector families on the grant foundation. > - The benefit is broader app access without weakening credential isolation. ## Linked Issues or Issue Description Refs #11965 This is stack 4 of 11. It depends on stack 3 and replaces another reviewable part of #11965. ## What Changed - Add Composio parent and child connection support. - Add Gmail connection setup and governance. - Preserve credential paths and remove duplicate binding declarations. - Cascade Composio pause and restore actions to child connections. ## Verification - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - Result: 164 tests passed. - `pnpm build` ## Risks - Parent lifecycle changes can affect every Composio child. - The service restores only children whose provider accounts remain active. - Credential binding paths are normalized before secret resolution. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
20ccf3f476 |
feat(apps): add connection grants and delegated identities (#12341)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - External tools need explicit identity and access boundaries. > - Shared connection credentials cannot represent every user-scoped use case. > - Grants must stay company-scoped and support safe delegation. > - This pull request adds connection grants, identity rules, and their database contract. > - The benefit is durable control over which identity an agent may use. ## Linked Issues or Issue Description Refs #11965 This is stack 3 of 11. It depends on stack 2 and replaces another reviewable part of #11965. ## What Changed - Add company and user connection grants. - Add delegated identity and membership rules. - Synchronize database, shared, server, and UI contracts. - Register the grant-member replacement route in the OpenAPI surface in the same layer that mounts it. - Add migration 0231 with replay-safe guards and coverage. ## Verification - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/openapi-routes.test.ts` (5 passed) - `pnpm --filter @paperclipai/db check:migrations` - `pnpm build` ## Risks - Incorrect grant selection could expose the wrong credential scope. - The service enforces company and subject boundaries before credential use. - Migration 0231 is generated, ordered after 0230, and safe to replay. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked a public issue or pull request with `Refs #` - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b51112798f |
feat(apps): improve gateway and workspace connection UX (#12340)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - App connections must work in both the operator UI and agent tool gateway. > - The first stack layer adds secure remote connections. > - Operators still need clear setup, test, and recovery states. > - This pull request adds the gateway behavior and the workspace connection experience. > - The benefit is a connection flow that is easier to understand and recover. ## Linked Issues or Issue Description Refs #11965 This is stack 2 of 11. It depends on stack 1 and replaces another reviewable part of #11965. ## What Changed - Improve remote tool gateway connection behavior. - Add clearer app setup, test, and recovery states. - Add focused server and UI tests for the new paths. - Keep the diff isolated from later identity and catalog work. - Stabilize DNS-pinned remote HTTP protocol fixtures and the managed-runtime public-origin fixture for this independently tested layer. ## Verification - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` (150 passed) - `pnpm test:run` - `pnpm check:token-gates` - `pnpm build` ## Risks - Gateway errors now surface through new user-facing states. - A stale connection can require a new setup attempt. - The change does not add a database migration. - The injected HTTP transport and public URL are test-only fixtures; production DNS pinning and runtime behavior are unchanged. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cabc9146d0 |
feat(apps): add secure remote MCP and PostHog setup (#12339)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps give those agents governed access to external tools. > - Remote MCP setup needs secure endpoint validation and durable credentials. > - PostHog needs both browser sign-in and personal API key setup paths. > - This pull request adds the shared remote MCP foundation and the PostHog definition. > - The benefit is a secure and reusable base for later app connection work. ## Linked Issues or Issue Description Refs #11965 This is stack 1 of 11. It replaces the first reviewable part of #11965. ## What Changed - Add guarded remote MCP setup and credential handling. - Add PostHog OAuth and API key connection methods. - Add focused server, shared contract, and UI coverage. - Keep the migration replay-safe and idempotent. - Give the late-close security regression the same 10-second CI headroom as the adjacent real-timer handshake test. - Synchronize fake-timer handshake tests at the exact ensure-session boundary so real filesystem setup cannot race the fake deadline. - Drive PTY overflow coverage only after listener registration so scheduling cannot reorder the test fixture. ## Verification - pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts server/src/__tests__/plugin-worker-manager.test.ts (220 passed; affected cases also passed five focused stress repetitions) - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never leaks a sandbox-provided value from a late close rejection into logs or the result"` (1 passed) - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never promotes a late ensureSession resolution|closes a late-resolving real handle exactly once"` (2 passed) - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - `pnpm --filter @paperclipai/db check:migrations` - `pnpm build` ## Risks - Remote endpoint validation can reject configurations that previously passed without checks. - OAuth configuration errors can block setup until the operator corrects the provider settings. - The migration uses guarded statements so repeated execution is safe. - The test-only synchronization changes do not affect runtime behavior; they remove filesystem/fake-clock and listener-registration races observed under parallel CI load. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
7c8064da1b |
test(runtime): assert order-independent port invariants for concurrent siblings (#11754)
## Thinking Path > - Paperclip manages work for AI agents. > - The workspace runtime starts isolated services for concurrent workspaces. > - A runtime test assumed that one concurrent lane always received the base port. > - The allocator guarantees distinct ports, but scheduling decides which lane receives the base port. > - This pull request changes the test to assert allocator guarantees without lane-order assumptions. > - The benefit is a stable test that still checks the complete bounded port range. ## Linked Issues or Issue Description **What happened?** The concurrent sibling workspace runtime test failed intermittently because it assumed array index 0 received the base port. **Expected behavior** The test must accept either lane as the base-port owner while it checks the allocator invariants. **Steps to reproduce** 1. Start two isolated workspace runtimes with `Promise.all`. 2. Force the second lane to start first. 3. Run the old assertions. 4. Observe that the test expects the wrong lane to receive the base port. **Paperclip version or commit** This change targets the current `master` branch. **Deployment mode** Built from source test suite. **Installation method** Built from source with pnpm. **Database mode** Not database-related. ## What Changed - Replace lane-order assertions with order-independent port invariants. - Assert distinct ports, the base lower port, and the bounded upper port. - Keep concurrent startup, service URL checks, and persisted-row checks. ## Verification - The target test passed 12 consecutive runs. - The full test file passed 128 of 128 tests. - Both forced lane orderings passed with the new invariants. - TypeScript reported no errors in the changed file. - CI will run after this pull request opens. ## Risks Low risk. This pull request changes one test file and does not change runtime code. ## Model Used OpenAI Codex, GPT-5, tool use and code execution enabled. The model reviewed and prepared the pull request metadata. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
05d58cd884 |
fix(tool-gateway): keep unsigned ask-first requests out of the review queue without cancelling them (#11338)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The tool gateway creates approval requests and the review queue reads them > - The gateway creates a request row before it adds the signature > - A review-queue read can see the row during that short unsigned state > - The old read path cancels the unsigned row, so approval returns `409 action_not_pending` > - This pull request hides unsigned in-flight rows and keeps them pending until signing finishes > - The benefit is that approval succeeds while invalid signed requests remain cancelled ## Linked Issues or Issue Description **What happened?** A review-queue read cancelled a pending tool action request when the request had no signature yet. The next approval call returned `409 action_not_pending`. **Expected behavior** The review queue must hide an unsigned in-flight request and keep its state as `pending`. A request with an invalid signature must remain cancelled. **Steps to reproduce** 1. Create a require-approval tool action request. 2. Read the review queue while the request signature is still null. 3. Approve the request after the creator adds the signature. 4. Observe that the old code cancels the request and the approval call fails. **Paperclip version or commit** Commit `720aa0a494bbaa1711bc7a3d795f810765915bfe`. **Deployment mode** Local dev with the embedded PGlite database. **Installation method** Built from source with pnpm. **Agent adapter(s) involved** Not adapter-specific. This is a core tool access service bug. **Database mode** Embedded PGlite. **Access context** Board and agent tool approval flow. ## What Changed - Keep a pending request with a null signature out of `listActionRequests` results. - Cancel a request when its non-null signature fails verification. - Add a permanent regression test for the unsigned request transition. - Update the contract test for unsigned and invalid-signature requests. ## Verification - Run the tool access service, tool gateway service, tool gateway, and tool access policy service tests. - Confirm 227 tests pass. - Run the `@mcp-runnable` Playwright end-to-end suite in CI. - Run the US-9 loop 30 times in CI. ## Risks The change alters review-queue filtering for unsigned requests. A null signature now means that signing remains in progress. Invalid signed requests keep the existing cancellation behavior. The change has no database migration. ## Model Used OpenAI Codex, GPT-5, with tool use and code execution. The model reviewed the handoff, repository rules, and pull request state. The implementation author supplied the code and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
cca2806e57 |
test(tool-gateway): make idle-down slot test deterministic via injected clock (#10226)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server has a gateway layer that coordinates tool execution and runtime slots > - The idle-down test for the local stdio fixture slot was relying on real wall-clock timing > - On slower runners, that made the test nondeterministic because the slot could be reaped before the presence assertion ran > - This pull request switches the test to use the existing injectable clock seam so time only advances when the test says it should > - The benefit is that the idle-down behavior stays covered while the test becomes deterministic and no longer flakes under load ## Linked Issues or Issue Description This PR fixes a flaky gateway test in the server test suite. The `tool-gateway` idle-down scenario was asserting slot presence while also depending on a very short real-time idle TTL and a later sleep-based reap. On loaded runners, the intervening work could exceed the TTL, which caused the slot to disappear early and the assertion to see an empty list. The fix keeps the production code path unchanged and drives the test from the supervisor's existing injectable clock. The test now holds time steady through the presence check, then advances the clock past the idle deadline to trigger the reap deterministically. The original behavioral assertions stay intact: slot reuse, counter increments, metadata, and stop status still get verified. ## What Changed - Replaced the real-time idle-down wait in the `tool-gateway` test with the runtime supervisor's injectable clock seam. - Kept the existing assertions for slot reuse, slot identity, counters, metadata, and stop behavior. - Removed the test's dependency on wall-clock timing so the idle-down path is deterministic under load. ## Verification - Targeted server typecheck passed with `tsc --noEmit`. - `tool-gateway.test.ts` passed in full: 49/49. - The targeted idle-down scenario passed 50/50 in a tight loop with 0 failures after the clock injection change. ## Risks - Low risk: this is a test-only change and does not modify production gateway logic. - The test now exercises the idle-down logic through a controlled clock rather than real elapsed time, which is the point of the fix but does slightly reduce wall-clock realism in the test itself. ## Model Used OpenAI Codex (GPT-5), tool-using coding agent; context window not surfaced in the workspace. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7e00f67138 |
feat(connections): add v3 schema core (#9958)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their governed access to external systems. > - Connected Apps build on the existing Apps and MCP gateway substrate so companies can configure reusable, auditable integrations. > - The current connection record does not yet have a stable public address, explicit ownership/auth method fields, or subject-specific credential grants. > - Without that schema core, later OAuth, per-user authorization, token brokering, triggers, and connector-service phases cannot enforce tenant and subject boundaries consistently. > - This pull request adds the forward-compatible Connections v3 schema core while preserving the existing connection lifecycle and directly migrating the remote MCP transport name. > - The benefit is a company-scoped, least-privilege foundation for one-click integrations without bypassing Paperclip secrets, profiles, rules, or audit controls. ## Linked Issues or Issue Description No matching public issue was found. **Problem** Paperclip's current app connections need a durable identity and authorization substrate before Connected Apps can safely support multiple setup methods, per-user credentials, provider tenants, and managed connector services. The existing schema only models a single connection-level credential set and uses legacy transport terminology. **Proposed solution** Add a stable company-scoped connection UID, explicit ownership/auth/transport fields, a subject-aware `connection_grants` table, and multi-key credential annotations. Backfill existing connections and workspace grants in a reversible migration, then update shared/server/UI contracts to the new `mcp_remote` transport name. **Related work** - Related foundation: #9534 - Roadmap: Connected Apps (one-click integrations) ## What Changed - Added company-scoped connection `uid`, `ownership`, `authKind`, and canonical transport fields across database, shared contracts, validators, services, and UI fixtures. - Added `connection_grants` with workspace/user subject rules, provider tenant metadata, credential secret refs, revocation state, company scoping, and uniqueness constraints. - Added migration `0182_connections_v3_schema_core` to backfill stable UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create default workspace grants, and support rollback coverage. - Added multi-key credential annotations and updated gateway/access services without changing the existing lifecycle behavior. - Updated the connection glossary, connector playbook, and security threat model for the new identity, grant, and relay boundaries. - Added explicit test UIDs to direct database fixtures so the new non-null invariant is exercised across affected server suites. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts server/src/__tests__/heartbeat-runtime-skills.test.ts server/src/__tests__/tool-oauth-legacy-backfill.test.ts server/src/__tests__/tool-access-policy-service.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts packages/db/src/connections-v3-schema-core-migration.test.ts packages/shared/src/validators/tool-access.test.ts --config vitest.config.ts` — 9 files, 218 tests passed. - Latest-head GitHub Actions: build, typecheck, general/serialized suites, backup/worktree restore coverage, both e2e shards, canary, policy, and security scans pass. - Greptile: 5/5 with zero unresolved threads. - `pnpm check:token-gates` remains red only on five pre-existing `#9627` color literals outside this change. ## Risks - **Migration risk:** UID backfill and default-grant creation touch every existing connection. The migration uses company-scoped uniqueness, deterministic legacy UIDs with ID suffixes, and seeded up/rollback coverage. - **Authorization risk:** Grant rows carry credential references. Constraints enforce workspace-vs-user subject shape, company/connection lookup indexes, one default grant per connection, and one user grant per connection/subject. Security review is requested specifically for this design. - **Compatibility risk:** `remote_http` is renamed directly to `mcp_remote`; all repository call sites and fixtures are updated in the same change. - **Future-phase risk:** Subject-bound token issuance, triggers, and connector-service relay verification remain fail-closed requirements documented for later phases; this PR does not expose those capabilities. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI coding agent. The runtime did not expose an exact underlying model ID or context-window size; capabilities used include repository inspection, code editing, shell execution, test execution, Git/GitHub CLI operations, and structured reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
931eec3fbf |
feat(mcp) [split 4/8]: wire gateway runtime and Smoke Lab (#9559)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Governed MCP access spans contracts, runtime enforcement, adapters, UI surfaces, and operator verification > - The parity reference PR #9534 is too large for effective automated or human review > - The feature therefore needs a linear stack whose individual diffs stay below the 100-file review limit > - This pull request is split 4/8 and focuses on gateway runtime, Smoke Lab, plugins, and server wiring > - The benefit is a standalone, testable review boundary while preserving byte-for-byte parity at the top of the stack ## Linked Issues or Issue Description - Related parity reference: #9534 - Problem: The policy core needs runtime execution, endpoint guards, route registration, heartbeat integration, and adapter MCP injection to become operational. - Proposed solution: Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Alternatives considered: keeping #9534 as one 403-file review, or rewriting the feature to manufacture seams; both were rejected in favor of path extraction plus compile-driven boundary moves. - Roadmap alignment: this advances the existing governed MCP/tool-access work already represented by #9534; it does not introduce a separate roadmap initiative. - Stack position: base branch is `pap10341-split/03-server-tool-access`. - Merge policy: merge bottom-up, in order, only after the complete eight-PR stack has been reviewed and the top-of-stack parity gate remains empty. - Requested review: SecurityEngineer for gateway, endpoint guard, token issuance, and runtime wiring; Greptile on every PR. ## What Changed - Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Keeps this PR below 100 changed files and independently typecheckable. - Preserves the final tree from #9534 when combined with the other seven stack levels. ## Verification - `pnpm typecheck` - Changed server test set — 26 files, 382 tests passed - Affected server adapter tests — 38 tests passed after concrete adapter boundary move - Adapter-utils and Codex focused tests — 76 tests passed ## Risks - Remote endpoint validation, token handling, and runtime supervision are security-sensitive and can fail closed or deny legitimate access if misconfigured. - Stack risk: merging out of order can expose incomplete layers; mitigate by following the documented bottom-up merge policy. - Parity risk: later edits to an intermediate branch can drift from #9534; mitigate by re-running the empty top-of-stack diff before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context window; medium reasoning with repository, shell, Git, GitHub CLI, and code-execution tools enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] Internal references are omitted except the execution-plan link explicitly required for this coordinated split stack - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Stack Coordination - Internal execution plan: [PAP-13874](/PAP/issues/PAP-13874#document-plan) - Parity reference: #9534 - Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563 - Merge bottom-up only after full-stack review and an empty parity diff at #9563. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |