mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 18:19:05 +02:00
9dcccc9e4610c8bc9a516fadf03033fbb2000cda
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
c6ee96a3df |
fix: stop idle MCP event polling (#17036)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - MCP task subscriptions and Dot mailboxes notify external clients when work changes. > - Their delivery workers queried SQL every two seconds, including when no work existed. > - Those queries prevent idle databases from becoming quiet across many hosted instances. > - Existing activity, mailbox, subscription, and delivery records already preserve the work. > - This pull request uses committed-write signals and saved retry deadlines to drive delivery. > - Idle subscriptions become quiet while startup, retry, drain, and shutdown recovery remain supported. ## Linked Issues or Issue Description Related sleep migrations: #15625, #17000, #17008. Searched open PRs and issues for MCP event sleep/polling work; no duplicate found. **What happened?** Each public MCP and Dot event service started a two-second interval. It checked settings, scanned subscriptions, created delivery receipts, cleaned expired records, and attempted delivery even without changes. **Expected behavior** An empty or subscribed-but-idle service produces no recurring SQL or network traffic. Committed events trigger prompt delivery. Only unfinished work retains a retry deadline. **Steps to reproduce** Configure a public MCP origin and start the server. Observe recurring settings queries with MCP disabled, or subscription/queue queries with it enabled. No incoming task changes are required. **Paperclip version or commit** Initially inspected master at `8b4ab65b1c3d863d282752e83307943720a5c85c`; rebased onto `c2daf906d9` after master advanced. The only conflict was overlapping imports in the heartbeat queue; both were preserved. ## What Changed - Register separate MCP and Dot workers with the existing delivery coordinator. Remove both unconditional intervals. - Notify from committed activity, comment, mailbox, subscription, and experimental-settings writes. Centralize comment insertion in one small helper so normal callers do not manage recovery state. - Preserve outer transaction ownership, uncertainty reconciliation, and requests deferred during drain or standby. - Restore delivery retry/abandoned-claim deadlines on startup. Scan subscriptions in pages of 20; drain full source and delivery batches without another external event. Scope subscription refreshes to their subscription or shared Dot binding, issue activity to its task, comment discovery to its company, and mailbox/assignment changes to their binding. Retry deadlines do not rescan unchanged sources. - Run expired-record cleanup opportunistically. Preserve on-use expiry checks and the existing one-time automatic Dot challenge behavior. - Add real PostgreSQL lifecycle coverage and guarded external test database options for existing MCP/Dot integrations. Document producers and the supported writer model. ## Verification - Final head `e54fb34485921016ef988f7df9cca173a9ebdfc8`: all CI checks passed (55 successful checks/statuses, one neutral review, two intentionally skipped Storybook checks). Mergeable with no unresolved review threads. Greptile: 5/5 on this exact commit; security scan passed. - All 284 affected tests passed in one local run after rebasing: 16 real-PostgreSQL lifecycle tests, 24 coordinator/scheduler/standby tests, 84 public MCP tests, 54 Dot runner tests, and 106 interaction-service tests. - Server TypeScript passed after refreshing generated Runner/plugin SDK dependencies from master. Full workspace typecheck and build passed before the rebase; final CI repeated typecheck and build successfully on the rebased head. - Coverage includes SQL/network quiet with configuration and history, committed writes, outer commit/rollback, drain/standby, retry restoration, reenable/unsubscribe, concurrent writes, shutdown, source/delivery/subscription batch boundaries, Dot challenges and reconnects, native broker fences, comment discovery, and company-isolated scans. - CI exposed an existing lock-order test whose insert proxy missed nested transactions and waited indefinitely. The fixture now observes nested inserts, preserves the lock assertion, and fails explicitly if interception is missed. The complete 106-test interaction suite and the formerly hanging CI shard pass. The warm-standby/settings test doubles were also updated for the new coordinator/transaction boundaries. - Earlier fixture failures (required timestamps, cleanup, external database selection, async waits) were corrected. One existing OAuth first-poll timing assertion and one E2E company-cleanup race passed on single reruns; the final full CI run passed without a rerun. No OAuth or company-deletion behavior was changed. - The local full `pnpm test:run` attempt could not initialize embedded PostgreSQL after five attempts. It was stopped; affected integrations were verified with separate disposable PostgreSQL 17 databases, and final CI supplied the full-suite result. Embedded-only local skips are not counted as passes. ## Risks - Notifications use one live server/database scope. Independent SQL writers and separate live replicas do not trigger these process-local workers. Startup recovers durable state; continuous multi-writer discovery needs a notification/ingress contract. - Idle physical cleanup waits for startup or another enabled sweep. Subscription and authorization expiry remain enforced on use. - Existing delivery claims, retry bounds, signatures, and receipt deduplication remain in place. No new tables or migrations. - Host sleep-safety checks remain conservative. MCP/Dot retry deadlines are not yet covered by Paperclip Cloud's scheduled-wake contract. This proves worker quiet, not deployed host suspension or fleet savings. ## Model Used OpenAI Codex (GPT-6), with reasoning, local code execution, and tool use. The runtime does not expose a more specific model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fc0d390d98 |
fix: enforce Cloud-enabled features on every managed stack (#16953)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud sets managed feature values for each tenant stack. > - Staging lets admins change experimental features. > - The staging exception also let old tenant false values override Cloud true values. > - This could disable required features such as isolated workspaces. > - This pull request keeps every Cloud-enabled feature on and rejects attempts to disable it. > - The same rule applies to staging, production, and managed stacks without a hostname. ## Linked Issues or Issue Description Refs: #16876 **What happened?** The staging override added in #16876 treats managed true values as editable defaults. A stored tenant false value can disable a feature that Cloud enables. Settings writes also accept attempts to disable Cloud-enabled features. **Expected behavior** If Cloud sets a feature to true, the tenant reads true and cannot turn it off. This applies to every managed stack. Staging can still change allowed experiments that Cloud has not enabled. **Steps to reproduce** 1. Set a managed staging primary hostname and enable isolated workspaces in `PAPERCLIP_MANAGED_CONFIG`. 2. Start with `enableIsolatedWorkspaces: false` in the tenant settings row. 3. Read `/api/instance/settings/experimental`. Before this fix, staging returns false. 4. Try to PATCH that flag to false. After this fix, the API returns 403 without saving, and reads return true with managed control metadata. **Paperclip version or commit** Reproduced against master at `cde944ed75648adfb569e86133adc8b118b55dd6`. ## What Changed - Keep managed true values in the staging read overlay. They override old tenant values and lock the existing UI controls. - Reject attempts to set any Cloud-enabled feature to false before writing settings. - Cover every managed feature in service tests across staging, production, and an absent hostname. Add real-service HTTP tests for isolation on staging and production. - Update operator documentation and the feature and access maps. ## Verification - Confirmed the new service regression fails before the fix: staging returns stored false, and writes can attempt to disable enabled flags. - Passed all six focused settings suites: 168 tests. Passed the final route and overlay suites after adding the HTTP regression: 109 tests. - Passed existing experimental settings UI tests: 54 tests. These cover the existing managed-control behavior. - Passed `node scripts/check-maps.mjs` and `git diff --check`. - Passed `pnpm -r typecheck` and `pnpm build`. The first typecheck used an install that did not honor locked versions and hit an unrelated `@pierre/diffs` API mismatch. Reinstalling from the lockfile resolved it; no lockfile changes are included. - Full CI passed on `f43136812a92ec838a24f0c423270e4fbd997c28`: 55 successful checks, one neutral review check, and two expected skips. This includes typecheck, build, general and serialized tests, runner checks, and E2E shards. [CI run](https://github.com/paperclipai/paperclip/actions/runs/39411907756). - The duplicate local `pnpm test:run` was stopped after the complete CI matrix passed. It is not counted as a completed local full-suite pass. - Greptile reviewed `f43136812a92ec838a24f0c423270e4fbd997c28`: 5/5 with no outstanding findings. The only finding on the first review was the repository rule against new code comments; those comments were removed. ## Risks - Staging users can no longer disable a feature that Cloud enables. This is the intended behavior. - Clients that send a stale full settings form containing false for a Cloud-enabled feature receive 403 and must reload. - No database migration is needed. Stored false values remain in the database, but cannot override an active Cloud true value. - Cloud-disabled staging experiments remain editable. Production restrictions and self-hosted settings remain unchanged. ## Model Used - OpenAI Codex, `gpt-6.1-sol`, with reasoning, repository tools, and test execution. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c5226daf7a |
fix(cloud): enable experimental toggles on staging (#16876)
Allow instance admins to change approved experimental feature controls on managed Cloud staging. Preserve production restrictions and managed access and execution policy. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9077748894 |
Remove the Pipelines experiment and preserve its data (#16834)
## Thinking Path > - Paperclip manages AI agents and their work. > - Pipelines was an unused experiment with its own UI, API, CLI, and automation. > - Keeping it leaves unsupported product surfaces and extra task-detail queries. > - Removing the experiment must preserve data that users already saved. > - This pull request removes the feature and keeps its database schema and migration history. > - Cases and Routines remain available as separate features. ## Linked Issues or Issue Description **What existing behavior does this improve?** Remove the unused Pipelines experiment across the application. **Current behavior** The experiment includes pipeline boards, stage settings, case automation, review queues, learnings, permissions, and CLI commands. Task detail also queries pipeline links. **Proposed behavior** Remove all these product surfaces and their implementation. Preserve existing tables, records, and documents. Ignore the retired boolean in older managed configurations so upgrades still start. **Breaking changes** Pipeline URLs, API endpoints, CLI commands, and `pipelines:write` are removed. Task detail no longer includes the old `linkedCases` field. The independent Cases API remains available. No database migration or cleanup is included. Related feature work: #7907, #9110, #10905, #11375, #13004, #14671, #16133. Related issues: #9663, #10310, #10312. This removal supersedes the feature; it does not implement those pending enhancements or delete their saved data. ## What Changed - Remove Pipeline UI pages, navigation, gates, clients, query keys, helpers, and unused design tokens. - Remove Pipeline REST routes, services, automation dispatch, task-detail queries, and OpenAPI discovery. - Remove the CLI commands, shared contracts, experimental flag, catalog entry, and permission defaults. - Remove feature-specific tests, tutorials, smoke scripts, maps, and Storybook examples. - Preserve database schema, migration history, and the reserved document key used by old records. - Keep Cases routes independent and cover obsolete stored and managed settings. ## Verification - Passed: `pnpm build` and `pnpm -r typecheck`. - Passed: targeted settings, managed configuration, sidebar, settings UI, routing, and feature-catalog tests. A first sidebar run caught a shared Workspaces icon import; the import was restored and the sidebar rerun passed. - Passed: 40 tests covering Cases routes, OpenAPI discovery, retained Pipeline table constraints, and schema snapshot drift. - Passed: token gates, map validation, generated API metadata check, Storybook catalog validation, and 28 map/catalog tests. - Passed: isolated local browser verification of navigation and Experimental settings with Pipelines absent; Cases and Routines still open normally. Live GET/POST requests to retired Pipeline endpoints return 404, and the retired setting is absent. CLI help has no Pipeline commands. - Passed: 22 access-service and agent-default-grant tests after restoring unrelated permission assertions flagged in review. - Local full run: `pnpm test:run` reported one failure in the unchanged workspace-runtime port-allocation test. That test passed in an isolated rerun. The long local run was stopped after full CI passed; it is not reported as a complete local pass. - Passed: all 56 checks on `def9181f3a4491d95fca1d38e9739f5b821aae3e` (2 conditional checks skipped), including build, typecheck, test shards, and browser suites. Eleven interrupted CI jobs reported “The operation was canceled”; their one-time retry passed. [CI run](https://github.com/paperclipai/paperclip/actions/runs/39255211006). - Greptile: 5/5 on the final commit, with the only review finding fixed and resolved. - Sandbox socket restrictions affected the first build/API test attempts. The same commands ran with local sockets available. ## Risks - Existing Pipeline clients must stop using the removed endpoints and commands. - Existing data remains in the database and backups, but the removed UI no longer displays it. - Historical boolean `enablePipelines` values are ignored. They cannot restore the feature. - Cases and Routines must retain their separate behavior. Targeted route tests cover this boundary. ## Model Used OpenAI Codex, GPT-6, with repository inspection, editing, shell execution, and browser verification. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8781f06a87 |
feat(connections): enable MCP aggregators by default (#13964)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Connections let those agents use external services with explicit access rules. > - Zapier, Arcade, Composio Connect, and Executor already have setup and runtime support. > - Their experimental switch still blocks discovery and setup by default. > - This pull request removes those gates and the Settings toggle. > - Users can connect these providers without enabling an experiment. ## Linked Issues or Issue Description Refs #13755. Refs #13941. **What existing behavior does this improve?** Apps browsing, inline setup, and agent connection search for the four MCP aggregators. **Current behavior** An instance must enable the MCP aggregators experiment before users or agents can start setup. **Proposed behavior** All four providers are available by default on local and managed instances. Old stored and managed values still parse but cannot disable them. ## What Changed - Remove the aggregator gates from Apps, inline setup, server setup, and agent search. - Remove the Settings toggle and its UI hook. - Retain the old setting key only for upgrade compatibility. Normalize it to true and ignore managed overrides, as Apps already does. - Replace opt-in fixtures with default-on coverage. Test old false values, all four setup flows, provider choice, and the removed toggle. - Update current connector guidance and remove the opt-in from the runner acceptance fixture. ## Verification - 306 focused tests passed across eight files: shared remote MCP contracts; server remote MCP lifecycle, aggregator fallback, settings normalization, and managed overlay; UI Apps browsing, setup, and experimental settings. - Server and UI TypeScript checks passed. - UI token gates and `git diff --check` passed. - The full local suite was not run, per the maintainer's instruction. All 54 CI checks passed; two checks were skipped. One unrelated workspace-preview readiness timeout passed on one failed-shard retry. - The setup fixtures use simulated MCP responses. This change does not claim new live provider acceptance. ## Risks - Existing instances now show all four providers, even if the old flag was false. This is intentional. - External provider choice, credentials, company isolation, agent grants, and tool policies still apply. Showing a connector does not authorize an external account. - No data migration is required. The compatibility key keeps old managed configuration documents valid. - Historical Zapier live acceptance remains incomplete in the existing evidence report. The maintainer explicitly requested the default-on rollout for all four existing providers; the report records that scoped exception. ## Model Used OpenAI GPT-6 (`gpt-6-astra`) through Codex, with reasoning, repository tools, and test execution. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
889947c238 |
feat: add experimental native chat connectors (#13038)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People also ask agents for work in their existing chat tools. > - Each external conversation needs one task and a current authorized source. > - Retries, Stop, and provider failures must not duplicate work or expose private data. > - The first chat PR establishes the opt-in provider and data contracts. > - This PR adds experimental channel integration and its durable control plane. > - Users can request work from connected channels and inspect delivery in Paperclip. ## Linked Issues or Issue Description Refs #13100 and #13092. This is the second of exactly two chat PRs. Foundation #13100 is merged and changed 143 files. Runner prerequisite #13092 is also merged. This PR changes 400 files against master, below the 500-file review limit. It contains no wireframe images or HTML galleries. ## What Changed - Add native Slack, GitHub, Microsoft Teams, Telegram, and Discord chat connections. Keep chat disabled unless the operator enables experimental chat connectors. Preserve the production GitHub tool connection and its normal setup path. - Bind each provider bot identity to one immutable Paperclip agent. Bind each admitted external conversation to one task. Paperclip owns tasks, runs, permissions, and audit records. - Add durable admission, per-conversation queues, questions, task controls, progress, final replies, images, files, and delivery receipts. Board comments remain internal unless explicitly sent to the channel. - Check current identity, provider reach, resource access, credentials, runtime generation, and exact source before provider effects. Keep private responses private. Never send raw reasoning, private logs, credentials, or tool arguments. - Hold uncertain sends for explicit audited resolution. Make Board Send-to-channel atomic and idempotent. Keep reconnect and setup credentials in Paperclip secret storage. - Preserve current native-runner authority across retries, lost acknowledgements, and recovery. Keep immutable input and completion contracts separate from newer user input. Receipt reconciliation cannot launch a provider. - Reconcile chat close/new ordering and provider-effect lock order. Audit resource access changes in the same transaction. Submit only the selected resource from each UI toggle so stale pages cannot undo unrelated access changes. - Drain Codex stdout before certifying process exit. Bound the drain with the existing shutdown grace. Preserve observed terminal authority without treating an undrained process as successful or reusable. - Incorporate master `018ca5da` with its ACP Stop, mobile task layout, runner packaging, and official lock changes. Preserve dedicated chat-answer continuations in both directions when ordinary queued comments are adopted after Stop. - Fence late adapter readiness behind an earlier Stop for the same run. Preserve verified cleanup for registered adapters. Handle single Stop, agent pause, duplicate Stops, and failure release without creating a false cancellation receipt. - Incorporate master's `6dd48cad4` wake-queue extraction. Preserve exact failed-chat retry authorization and lineage, retired question-source suppression, and the block on generic recovery that would discard the admitted source. Fresh deferred input retains its separate promotion path. - Incorporate master `2a05b5ed3` and its queue-admission extraction, simplified transaction ports, and separate runner CI job. Preserve exact durable receipts, actor separation, and dedicated-answer isolation through the new module. A failed receipt insert rolls back the accompanying deferred-wake merge. ## Verification Current head: `afe19299d06253cb628eb398e91d1200ea9f412a`, incorporating master `2a05b5ed3457ea33efd6895520447d1d97fe98d8`. The conflicts are resolved. This successor fixes two test-harness boundaries exposed by CI: per-case route-module preparation and actual durable-save completion before intentional runner termination. Production code and all existing test/turn deadlines are unchanged. [Exact-head Greptile review](https://github.com/paperclipai/paperclip/pull/13038#issuecomment-5587250594) is **5/5**, completed September 10 at 13:20:55 UTC, with no actionable findings or open review threads. [Fresh exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/34481724341) passes **all 24 jobs**, including Build and both required aggregates. Normal exact-head guarded merge was attempted and rejected by the remaining branch approval policy: CODEOWNER review is required and no human approval is present. Normal **squash auto-merge is enabled** as of September 10 at 13:36:26 UTC. Requested CODEOWNERS have been notified; no approval bypass or self-approval was used. Earlier-head results below remain historical evidence, not qualification of this successor. - Final exact-head Linux evidence: 995/995 chat integration cases; 36/36 agent-skills routes; 35/35 runner live-session cases, including real process kill/resume; 1948 runner Vitest cases with three existing benchmark/platform guards; 870/870 API-authority cases; and 104 browser cases with four existing optional skips. Rust, conformance/replay, full repository build, typecheck, canary, all server/workspace shards, and both required aggregates pass with normal CI concurrency. Earlier failed attempts remain recorded below. - Latest test-only qualification: 141/141 route/permissions/authentication cases pass in separate cold forks, with plain server types and independent review clear. The real-runner suite passes 35/35, with plain runner types and independent review clear. A controlled premature-save acknowledgement fails as expected; matching ownership/effect/process evidence, rejected saves, real turn outcome, test abort, and pre-kill liveness are covered. No local reproduction of the original CI scheduling failure is claimed. The preceding [CI run](https://github.com/paperclipai/paperclip/actions/runs/34479680858) passes 21/24 jobs, including all 995 Linux chat cases and browser aggregate (104 passed, four existing optional skips); only Build, the skills serialized shard, and the required verification aggregate fail. Its exact-head Greptile review was 5/5. Both failed job logs are retained. - Final fixture qualification: all eight focused Discord cases and all 995 chat integration cases pass. The exact modal statement/PID is observed before taking the real connection lock; the test then proves its actual blocking relationship before mutation. Original SQL execution, provider behavior, negative assertions, and 1s/15s timeouts remain unchanged. Independent review is clear and test/production hashes remain frozen. The preceding [CI attempt](https://github.com/paperclipai/paperclip/actions/runs/34477184777) passed 22 jobs, including Build/runner, typecheck, canary, all other test shards, and browser aggregate (104 passed, four existing optional skips); the two fixture failures and failed verification aggregate remain recorded, not relabeled as a pass. - Current queue-module composition: 308/308 recovery/batching/queue/Stop tests; 995/995 full chat integration; 89/89 module tests, including real PostgreSQL receipt-insert rollback; 24/24 workflow/module-boundary tests; plain server and UI types. All four actual local process/ACP browser paths pass in 1.4 minutes. Fresh databases, no skips or retries, stable reviewed source hashes. The initial boundary failure is retained; its no-op service wrapper was removed without changing recovery context or weakening the check. An exploratory standalone test-directory typecheck fails because its new upstream transformation config is not a standalone typechecking project; standard CI/build does not invoke it, and no configuration was weakened to suppress those diagnostics. - The preceding head `e02a63d462ce5d47433b0aeb632bb6fd20aab1ba` passed [all 24 CI jobs](https://github.com/paperclipai/paperclip/actions/runs/34436462958) and exact-head Greptile review at 5/5. Required CODEOWNER review prevented its normal merge before master advanced again. - Final extracted-module composition: 307/307 recovery, batching, queue and Stop-control tests; 995/995 full chat integration; 49/49 module tests including eight PostgreSQL adapter cases; and 19/19 issue-update tests. Plain server types pass. All four actual local process/ACP browser paths pass in 1.3 minutes. Fresh databases, no skips or retries in these cohorts, frozen source hashes, and independent review clear. - The preceding head `3e4e1c1c` passes [all PR CI jobs](https://github.com/paperclipai/paperclip/actions/runs/34415826820), including Build and required `ci / verify` and `ci / e2e`. Both the original Rust failure and the previously load-sensitive lineage fixture pass with unchanged Linux concurrency. Master advanced afterward and required this reconciliation. - Final master composition: 448/448 focused UI tests, 186/186 adapter tests, 24/24 queue/control tests, and 11/11 packaging tests. Plain UI, server, shared, and adapter types pass. Token gates and diff checks pass. Independent server and UI reviews are clear. - Stop-registration regression: both real-service cases fail against exact `a95` source and pass with the fix. The full corrected recovery/control suite passes 265/265. Duplicate-owner and failed-Stop controls also pass. Plain server types pass. The readiness barrier prevents provider startup without adding an acknowledgment to an already terminal run. - Final qualification strengthens terminal-field equality and repeats both affected cases successfully on a fresh database. All four actual local process/ACP browser paths pass again in 1.3 minutes, without skips or retries. The final screenshot shows Cancelled, a paused subtree, retained input, and no error toast. - Two new actual-service regressions fail before the merge fix. They prove that queued-comment adoption could consume a dedicated chat answer or add unrelated input to that answer. The fixed four-case cohort passes, including ordinary upstream continuation and adapter Stop controls. Full recovery passes 257/257. All four actual local process/ACP Stop browser flows pass in 1.4 minutes, without skips or retries, on a fresh database. - The unchanged runner artifact was qualified with 171/171 transport tests, 870/870 API-authority tests, conformance 1/1, and replay 11/11. Six controlled reader tests prove the exit/drain repair. Its local serial Rust workspace passed 546 top-level cases plus two invoked helpers; the later passing Linux CI supplies default-concurrency evidence. - Prior exact-source full chat integration passes 995/995. Settings regressions cover concurrent stale pages, 501 destinations, pending state, rejected updates, and explicit retry. These deterministic tests do not prove live provider behavior. - Retained failed attempts and their causes are in the [qualification log](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-chat-queue-and-webhook-repair.md). The first merge adapter run timed out while macOS slept for 290 seconds. Its unchanged repeat passed with a temporary sleep guard. No assertion, deadline, or CI gate was weakened. Review commands include `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-process-recovery.test.ts src/__tests__/issue-queued-comments-routes.test.ts` and `pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/acp-stop-continuation.spec.ts`. Database suites require fresh disposable databases. See the [browser runbook](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-04-chat-adapters-browser-e2e-runbook.md) for provider setup and separate live acceptance steps. ## Risks - This remains experimental. Deterministic tests and bounded live evidence do not establish every provider feature, tenant, permission layout, or media shape. Teams work-tenant qualification is still open. - Failed and uncertain provider effects remain visible and can require operator action. A transport receipt does not prove recipient visibility. - Native controller and runner artifacts must remain compatible. Preserve lease ownership, terminal authority, source binding, and quarantine during future changes. - Access and audit rows commit together, but activity notifications remain best-effort. This is not a new durable event outbox. - The PR operation does not deploy a live server, replace its runner, or change provider permissions. Remaining live qualification is documented in the [temporary handoff](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-open-qualification-followups.md). ## Model Used OpenAI Codex assisted with implementation, tool execution, testing, and review. The work records `gpt-6-astra` assistance. The environment does not report a context-window size. No private reasoning traces are included. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8c3b8c432a |
Simplify app connections and enable managed Google access (#12728)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Apps subsystem gives humans and agents governed access to external tools. > - The current connection flow hides Apps behind an experimental gate and repeats setup text. > - Google sharing choices and generic MCP permissions do not use one consistent opening model. > - Self-hosted installs also need a safe default origin for managed OAuth without a manual config file. > - This pull request makes Apps available, simplifies connection setup, and applies one governed permissions model. > - The benefit is a shorter connection flow that works on a clean self-hosted install. ## Linked Issues or Issue Description **What existing behavior does this improve?** This improves the Apps connection setup flow, managed Google connection flow, generic MCP connection flow, navigation, and runtime origin discovery. **Subsystem affected** Cross-cutting. This changes `ui/`, `server/`, `packages/shared/`, connector documentation, and browser tests. **Current behavior** Apps require an experimental switch. Setup pages repeat titles and explanatory copy. Connection names require manual input. Google credential sharing does not always offer both personal and organization access. Generic MCP providers do not start with the same permission choices. Managed OAuth needs a public URL setting even when the request already has a safe HTTPS origin. **Proposed behavior** Apps are available by default. Setup asks only for required permissions and sharing choices. Paperclip creates conflict-free connection names. Google apps and generic MCP providers use the same human and agent access model. Managed OAuth derives a validated same-origin HTTPS URL when no explicit public URL is set. **Reason and benefit** A clean self-hosted install can connect a managed Google app without hidden setup. Humans can share a service account with their organization. The shorter flow reduces duplicated choices and setup errors. **Breaking changes** The Apps experimental switch is removed. Existing connection APIs remain compatible. New connections can receive a numeric suffix when a name already exists. No duplicate or related public issue was found. ## What Changed - Removed the Apps experimental gate and the breadcrumb that leaves the Apps section. - Simplified all connection setup pages and moved optional provider requirements into one small link. - Added consistent human and agent access choices for Google apps, Zapier, and generic MCP connections. - Added organization sharing to Google Workspace credentials while keeping personal access available. - Generated connection names automatically and resolved name conflicts with numeric suffixes. - Derived a validated public HTTPS origin from the request for config-free managed OAuth. - Updated connector contracts, tests, browser coverage, and authoring documentation. ## Verification - `pnpm check:token-gates` - `pnpm -r typecheck` - `pnpm build` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/generic-mcp-connection.test.ts` (273 passed) - Targeted UI/service regression suite (308 passed) - Six targeted Playwright connection journeys on a fresh onboarding instance (6 passed) - Fresh-install browser proof through Tailscale HTTPS: enrolled with Paperclip Cloud, connected managed Google Drive, and completed a real read operation. - [Exact-head CI run](https://github.com/paperclipai/paperclip/actions/runs/33669760711): all 23 matrix jobs passed, including build, typecheck, server, serialized, canary, and all browser shards. - Greptile 5/5 on `0ae2a859f269984ee950d0af231a5b09a06f3dfd`, with no unresolved review threads. ## Risks Apps are now visible to all operators. The removed experimental flag no longer hides unfinished app definitions. Managed Google availability still depends on the Cloud profile rollout and active instance enrollment. Automatic conflict handling changes only the display name of a newly conflicting connection. > I checked [`ROADMAP.md`](ROADMAP.md). MCP Tool Gateway and Apps are shipped. Connected Apps is planned, and this change improves the existing shipped connection flow. ## Model Used OpenAI Codex, GPT-5, with reasoning, browser control, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
216d3d2680 |
Managed-instance config: fail-closed PAPERCLIP_MANAGED_CONFIG parsing and read-time settings overlay (#10058)
**Builds on.** #10055 — the `catalogVersion` this config document pins is the feature-catalog artifact #10055 emits. **Summary.** Instances operated by a managed hosting control plane can now receive instance configuration through a single environment variable, `PAPERCLIP_MANAGED_CONFIG` (versioned JSON: `mode`, `catalogVersion`, `features`, `plugins.autoInstall`). When the variable is absent the instance is self-hosted and nothing changes. When present, parsing is strict and **fail-closed**: blank value, malformed JSON, unknown feature key, a feature key this build's feature catalog does not mark tier `managed`, missing required section, or unsupported version refuses startup with a precise error — a typo that silently does nothing is how a security control quietly fails. Managed feature values are overlaid **at read time** inside the instance settings service (never persisted), so a DB restore or manual row edit cannot resurrect a disabled capability; responses expose per-key `managedKeys` metadata (`managed: true`, `managedBy`) so clients can render locked state. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs both self-hosted and under managed hosting, where an operator's control plane owns instance configuration > - Today instance feature settings live only in the tenant database; a hosting control plane has no way to enforce a configuration that tenant-side writes or restores cannot undo > - Managed configuration will carry security posture, so delivery must be atomic and parsing must fail closed — a typo that silently does nothing is how a security control quietly fails > - This pull request adds strict parsing of one `PAPERCLIP_MANAGED_CONFIG` env var and overlays its feature values at read time inside the settings service, never persisting them > - The benefit is a minimal, auditable managed-hosting contract: absent var ⇒ self-hosted instances are byte-for-byte unchanged; present ⇒ deterministic, locked configuration surfaced to clients via per-key managed metadata ## Linked Issues or Issue Description Refs #966 — this PR delivers that issue's "managed config injection" hook, via a strict env-var contract rather than the config-file path it sketches; the issue's other hooks (identity header, health, usage webhook, lifecycle, external secrets, IAM auth) are out of scope, so the PR refs rather than closes it. *Mechanism differs from #966's proposal, so the `feature_request` fields are also filled in:* - **Problem or motivation:** managed hosting deployments need to centrally enable/disable instance features; DB-stored settings can be edited, restored, or migrated back to permissive values, and nothing marks a value as operator-enforced. - **Proposed solution:** one versioned JSON env var; fail-closed parse at startup; read-time overlay in the settings service (precedence: managed value over stored value over schema default); `managedKeys` metadata in settings responses so clients can render locked state. - **Alternatives considered:** per-feature env vars (non-atomic across a half-updated env set, unbounded env surface); seeding the DB at boot (persisted values can be edited or restored over, and cannot express "forced"); lenient warn-and-drop parsing (fails open — unacceptable for a security-bearing control). - **Roadmap alignment:** supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/managed-config.ts` (pure parser over the env record) - Startup parse ordered before the first `instanceSettingsService` construction in `server/src/index.ts` - Read-time merge + `managedKeys` in the settings service - Shared validator updates ## Verification - 29 parser/overlay tests (fail-closed matrix incl. blank/whitespace env, missing sections, catalog-tier mismatch, empty-section happy path): `pnpm vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts` (from `server/`) - 40 existing settings route/service tests green: `pnpm vitest run src/__tests__/instance-settings-routes.test.ts src/__tests__/instance-settings-service.test.ts` (from `server/`) - 15 shared validator tests: `pnpm vitest run src/validators/instance.test.ts` (from `packages/shared/`) - Server `tsc --noEmit` clean: `pnpm typecheck` (from `server/`) ## Risks - Self-hosted instances (no `PAPERCLIP_MANAGED_CONFIG` set) are byte-for-byte unchanged — the parser only runs when the variable is present. - For managed instances, a malformed document now refuses startup by design (fail-closed). This is an intentional behavioral guarantee, not a regression: the control plane owns the variable and a precise startup error is the contract. - Overlay values are never persisted, so no migration or data-shape risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |