mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 05:16:50 +01:00
57902152ac3218096fca669d33ff18400a2a86ae
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
13709b4c7a |
feat(connections): let agents configure connections within delegated authority (#17737)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.
## Linked Issues or Issue Description
**Subsystem affected**
Server authorization, connection services, shared contracts, native API
discovery, database schema.
**Problem or motivation**
Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.
**Proposed solution**
Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.
**Roadmap alignment**
This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.
## What Changed
- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.
## Verification
- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.
- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.
- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.
- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on
|
||
|
DottaandPaperclip
|
8415f4a679 |
fix(github): drain queued pings during active App setup (#17551)
## Thinking Path > - Paperclip connects external work to an agent and its ordinary tasks. > - A dedicated GitHub App must prove signed webhook delivery during setup. > - Cloud queues the original bytes until the tenant can authenticate them. > - An older personal connection can leave a long empty-queue backoff in that tenant. > - Active App setup can then time out before the queued ping is read. > - This change lets active setup request the existing event drain with a short rate limit. > - The same webhook handler still owns signature verification and setup state. ## Linked Issues or Issue Description Refs: #17457, #17513. **What happened?** Cloud accepted a new App's ping, but the tenant read it only after the setup wait timed out and cleanup archived the draft. An existing personal GitHub connection had left an empty-queue backoff. The poller checked that deadline before it read the new App state. **Expected behavior** An active manager setup request must be able to read the queued ping promptly. It must not bypass signature checks or create another App. **Steps to reproduce** 1. Let a personal GitHub connection build its empty-queue backoff. 2. Start a dedicated App and complete its installation. 3. Return to the waiting setup screen while the signed ping is queued. **Paperclip version or commit** This behavior is present on the base commit of this PR. ## What Changed - Let the authenticated wizard request a queue drain only while its Cloud App still lacks signed-delivery verification. Existing manager, credential, and installation checks run first. - Register a narrow callback to the server-owned GitHub poller. Keep provider handling in the GitHub services. Remove the callback on server close. - Share one in-flight attempt between setup and background requests. Limit setup attempts to one per five seconds, including after errors. - Retain the existing passive backoff, sealed queue, acknowledgement rules, and canonical webhook signature handler. - Add regressions for the old backoff, concurrent requests, failed requests, and wizard signature verification. Update the setup documentation and map. ## Verification - Observed the backoff regression fail against the old service: active setup made no additional Cloud lease request. It passes with this fix. - Three focused event-poller tests pass. They use the real admission/backoff code with stubbed SQL and Cloud boundaries. - All 39 startup tests pass. Server TypeScript, map checks, and diff checks pass. - The Postgres integration suites could not start locally because this host exhausted its semaphore limit. They remain required CI checks. This is not a passing local integration result. - Full current-head CI passes, including the Postgres and chat integration suites. Greptile scored this exact head 5/5 with no unresolved findings. - The additional Paperclip review check ended neutral because its execution did not complete. It did not publish a validated assessment. - A fresh deployed setup attempt remains required before live qualification. ## Risks A waiting setup screen now makes a bounded readiness request to Cloud. The existing connector request timeout still applies. Concurrent calls share the same drain, and failures release it for a later retry. The request stops after persisted verification or when the browser stops advancing setup. No timer, durable queue, fleet setting, or sleep blocker is added or removed. The existing background reconciliation path still has its existing sleep restrictions. Live setup-to-review behavior is not established by these local tests. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime model ID and context-window size are not exposed in this session. Used for code execution and regression analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
ed2c22ea8c |
feat(connections): simplify dedicated GitHub App setup (#15416)
## Thinking Path > - Paperclip manages agent work and the tools each agent can use. > - GitHub bots need their own App identity and restricted repository access. > - Manual App configuration and repeated setup gates make that connection hard to finish. > - GitHub can create an App from a manifest, but people must approve creation and repository access. > - Each Paperclip stack should own configuration, credentials, signature verification, and task admission. > - This change uses Cloud as an encrypted callback and event gateway, then completes setup in the stack. > - GitHub review bots have their own experimental setting. Chat connector opt-in does not control them. ## Linked Issues or Issue Description **What existing behavior does this improve?** Dedicated GitHub App setup, bot settings, and task-scoped GitHub replies. **Current behavior** Users must configure an App, copy credentials, repeat repository selection, and complete manual verification gates. Review settings are hard to scan. Mentions in descriptions can be missed, and automatic completion text can produce an extra GitHub comment. **Proposed behavior** Choose an agent and connect GitHub under a personal account or organization. GitHub asks for creation and installation consent. Paperclip resumes the same draft, stores secrets locally, imports access, verifies delivery, and opens Settings. Each accepted request has one editable App-owned comment for progress and results. Review assessments produce deterministic checks for the reviewed commit. **Breaking changes** GitHub bots require their own default-off experimental opt-in. The setup-method and delegated-agent screens are removed. Per-person automatic-run flags remain readable but are replaced by author filters. Existing connections, direct webhooks, saved restrictions, and manual existing-App recovery remain supported. Companion gateway: https://github.com/paperclipai/paperclip-cloud/pull/688 (merged). The roadmap and related PRs were checked; no duplicate dedicated-App wizard was found. ## What Changed - Add an independent **GitHub review bots** setting across the catalog, routes, identity admission, and task controls. Existing bots continue running when the UI setting is off. - Replace setup with Choose agent and Connect GitHub. Support personal and organization owners, inline low-trust conversion, saved drafts, known organization suggestions, automatic completion, and same-App recovery. - Keep manifests, OAuth PKCE, credential exchange, vault storage, installation verification, and original webhook-signature validation in the stack. Use the Cloud sealed gateway for public callbacks and outbound local delivery. - Add resumable owner metadata in migration 0323. Retry a failed registration renewal through the same durable replacement rather than creating another App route. - Support authorized mentions in issue and PR bodies, comments, inline comments, and review summaries. Preserve attachments, edited-source identity, duplicate protection, and lifecycle invalidation. Filter requests explicitly addressed to another connected bot. - Give agents task-scoped review, comment, and update tools. Update one working comment through progress and final output. Close coalesced requests and recover delayed acknowledgements, including deleted earlier working comments. Preserve automatic replies for legacy connections without saved review configuration. Preserve partial publication evidence and quarantine uncertain automatic fallbacks without blocking Board messages or questions. - Reorganize Settings and Access with rich slash-skill instructions, numeric passing scores, clear automation controls, formal-review tooltips, member and contributor sections, and verified App branding links. Preserve invalid score drafts and automation choices across tabs. - Paginate repositories in 20-row batches with search and a true connection-wide toggle. Batch verification at GitHub's 500-ID token limit. Keep saved repository overrides while hiding their editor. - Give each review a detail URL that remains usable beyond the newest 100 records. Synchronize the manager OpenAPI contracts and add recovery, authorization, UI, and browser regressions. - Reconcile the current master branch and retain its agent avatar, feature catalog, and extracted heartbeat preparation code. Preserve a saved test-drive port and recover from an occupied high port. ## Verification - Focused UI, independent experimental-setting, and CLI tests: 199 passed. - `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates` passed. The broad local Vitest attempt was interrupted after resource timeouts and stale expectations; the affected regressions passed focused reruns. The latest-head full CI matrix passed, including all browser shards, server/chat suites, builds, typechecking, canary packaging, and Docker context integrity. A Docker Hub 429 required a CI retry; it passed without changing the code. - Final review repairs: 127 attachment/event tests and server typecheck passed. All 13 final integration regressions passed under the normal CI timeout, covering legacy reply compatibility, deleted coalesced comments, inline-only review completion, progress edits, and native final recovery. Restart-safe description/review attachments passed the attachment suite. - Final regressions: four reply/lifecycle tests, both native GitHub wait tests, 171 feature-gate/reply-contract tests, the routed review preview, and five scheduler/reply regressions passed. The final fixture repair also passed all six setup-continuation cases and all three complete chat-test shards in CI, covering terminal receipts and retries under the normal CI timeout. One loaded-host receipt run exceeded its local timeout; the exact case passed in an isolated rerun and the complete CI shard. - GitHub integration tests cover personal and organization setup, retry recovery, edited mentions, attachment provenance, coalesced requests, delayed acknowledgements, partial review writes, old review URLs, and company boundaries. - Browser qualification: both GitHub journeys, all five provider setup/management journeys, and the GitHub uncertain-send recovery journey passed. GitHub setup, reconnect, and task replies run with chat disabled; chat enabled alone cannot expose GitHub bots. The disposable qualification database was removed after testing. - Earlier live qualification verified App-owned issue replies, native slash-skill invocation, progress comment edits, and a 5/5 PR review with a passing current-head check in the dedicated permissions-smoke repository. These live checks did not use personal credentials as bot execution authority. ## Risks - Deploy the merged Cloud gateway protocol before the stack update. Each callback and encrypted event must retain the original instance and tenant binding. - Migration 0323 uses idempotent column additions for test installations that already applied the earlier draft migration number. - A verified connection does not prove runtime isolation or a completed review. Runtime checks remain separate. - Required merge checks must be configured in GitHub. A passing numeric score does not formally approve a PR; approval and request-changes tools remain opt-in. - Existing-App credentials cannot silently renew an uncertain manifest exchange. Recovery preserves the existing App, draft, connection, and agent. - Repository changes save immediately. Global toggles affect all currently available repositories, including unloaded rows, without granting new installation access. - A failed or uncertain tool write suppresses automatic fallback text; the governed outbox retains its evidence. Deleted or replaced response comments are not recreated through a personal credential fallback. - GitHub logo upload remains an optional human action in GitHub App settings. Runtime provisioning, delegated setup, screenshot publication, and production push delivery remain deferred. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; complete test matrix is checked in CI) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |