6 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	f589660ec0	feat(routines): add safe webhook setup and in-routine run management (#13637) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines turn scheduled work and external events into tasks for an assigned agent. > - Webhook setup was disabled, and actor authentication rejected valid webhook bearer keys. > - Operators need to connect and test a sending app before events can start work. > - This pull request adds a guided setup with durable connection tests that cannot dispatch a task. > - It keeps trigger management, execution tasks, and activity within the routine. > - The benefit is a webhook that can be configured, verified, and operated from one place. ## Linked Issues or Issue Description Fixes #11937. Related: #13216 adds provider-specific Sentry support. This PR addresses general routine setup and ingress. #6841 addresses legacy secret bindings; this PR retains the existing secret service. **Current behavior** Webhook creation is disabled. Bearer deliveries can fail in agent authentication before the routine checks its key. Setup has no safe connection test. Runs and Activity send the operator away from the routine. **Proposed behavior** Choose a schedule or a webhook. Follow the setup steps, copy credentials or complete agent instructions, and test delivery without creating work. Finish setup to allow future events to start tasks. Edit or remove compact trigger cards, undo removal, and inspect tasks and activity inside the routine. **Reason and benefit** An operator can verify credentials and delivery before enabling automatic work. Durable setup state survives refreshes and restarts. Retry receipts prevent an old test event from starting work after activation. ## What Changed - Add a production trigger wizard using reusable Slack setup navigation and footer components. - Add schedule and webhook choices, one-time credentials, agent instructions, and live connection feedback. - Persist pending setup, test delivery receipts, connection status, and reversible trigger removal. - Keep setup checks free of routine runs, tasks, and agent wakeups. Preserve delivery idempotency after activation. - Add compact trigger cards, inline editing, key rotation, pause controls, removal, and Undo. - Keep Runs and Activity in the routine. Use the shared task list and compact activity rows. - Permit only exact public delivery POSTs through actor authentication. Retain webhook authentication, JSON-object validation, and log redaction. - Add production-backed Storybook states and focused server, database, and UI coverage. - Document signing modes, setup checks, retries, rotation, HTTPS ingress, and navigation. ## Verification - Full workspace typecheck, build, and token gates passed on the rebased branch. Storybook also builds. - Focused routine, middleware, logging, shared wizard, and UI coverage passes on the rebased branch: 195 tests across 14 files. The migration passed on a fresh PostgreSQL database and on two repeated applications. - Browser testing used the real app, database, and a deterministic process worker through Tailscale HTTPS and the current Cloud proxy code. - Verified rejected keys, safe setup deliveries, persisted state after restart, activation, retry deduplication, key rotation, schedule editing, removal, and Undo. - Fresh bearer and GitHub-signed deliveries created tasks that the worker checked out and completed. Runs and Activity stayed within the routine. - Current Cloud ingress tests passed. Public delivery POSTs passed through without a browser session; management routes remained gated. - All 54 current-head PR checks pass, including general and serialized tests, all eight browser E2E shards, typecheck, build, runner checks, security checks, and the canary dry run. Two optional Storybook jobs are skipped by workflow conditions. - Greptile is 5/5 on commit `7ea63a61e`, with no unresolved review threads. The stale connection-status finding is fixed and covered by a regression test. - No production deployment was performed. ## Risks - Migration 0281 adds three trigger columns and a test-receipt table. It is additive and safe to reapply. Apply it before running the new server. Existing triggers remain live by default. - Requests without delivery IDs are new events after activation. Senders must reuse an event's delivery ID for retries. - Completed webhooks keep normal dispatch behavior. Their management connection check can start work; the UI states this. - Removing a trigger archives it. Undo restores the URL and credentials. Permanent deletion remains available through the existing API. - Public ingress must remain restricted to the delivery POST route. The tenant verifies credentials. Cloud sleeping-stack behavior is unchanged. - Shared setup components also serve Slack. Existing setup contracts and navigation tests cover that integration. - Senders must use application/json with an object. Other media types receive 415. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, shell execution, and browser testing. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-19 10:49:05 -05:00
DottaandPaperclip	889947c238	feat: add experimental native chat connectors (#13038) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People also ask agents for work in their existing chat tools. > - Each external conversation needs one task and a current authorized source. > - Retries, Stop, and provider failures must not duplicate work or expose private data. > - The first chat PR establishes the opt-in provider and data contracts. > - This PR adds experimental channel integration and its durable control plane. > - Users can request work from connected channels and inspect delivery in Paperclip. ## Linked Issues or Issue Description Refs #13100 and #13092. This is the second of exactly two chat PRs. Foundation #13100 is merged and changed 143 files. Runner prerequisite #13092 is also merged. This PR changes 400 files against master, below the 500-file review limit. It contains no wireframe images or HTML galleries. ## What Changed - Add native Slack, GitHub, Microsoft Teams, Telegram, and Discord chat connections. Keep chat disabled unless the operator enables experimental chat connectors. Preserve the production GitHub tool connection and its normal setup path. - Bind each provider bot identity to one immutable Paperclip agent. Bind each admitted external conversation to one task. Paperclip owns tasks, runs, permissions, and audit records. - Add durable admission, per-conversation queues, questions, task controls, progress, final replies, images, files, and delivery receipts. Board comments remain internal unless explicitly sent to the channel. - Check current identity, provider reach, resource access, credentials, runtime generation, and exact source before provider effects. Keep private responses private. Never send raw reasoning, private logs, credentials, or tool arguments. - Hold uncertain sends for explicit audited resolution. Make Board Send-to-channel atomic and idempotent. Keep reconnect and setup credentials in Paperclip secret storage. - Preserve current native-runner authority across retries, lost acknowledgements, and recovery. Keep immutable input and completion contracts separate from newer user input. Receipt reconciliation cannot launch a provider. - Reconcile chat close/new ordering and provider-effect lock order. Audit resource access changes in the same transaction. Submit only the selected resource from each UI toggle so stale pages cannot undo unrelated access changes. - Drain Codex stdout before certifying process exit. Bound the drain with the existing shutdown grace. Preserve observed terminal authority without treating an undrained process as successful or reusable. - Incorporate master `018ca5da` with its ACP Stop, mobile task layout, runner packaging, and official lock changes. Preserve dedicated chat-answer continuations in both directions when ordinary queued comments are adopted after Stop. - Fence late adapter readiness behind an earlier Stop for the same run. Preserve verified cleanup for registered adapters. Handle single Stop, agent pause, duplicate Stops, and failure release without creating a false cancellation receipt. - Incorporate master's `6dd48cad4` wake-queue extraction. Preserve exact failed-chat retry authorization and lineage, retired question-source suppression, and the block on generic recovery that would discard the admitted source. Fresh deferred input retains its separate promotion path. - Incorporate master `2a05b5ed3` and its queue-admission extraction, simplified transaction ports, and separate runner CI job. Preserve exact durable receipts, actor separation, and dedicated-answer isolation through the new module. A failed receipt insert rolls back the accompanying deferred-wake merge. ## Verification Current head: `afe19299d06253cb628eb398e91d1200ea9f412a`, incorporating master `2a05b5ed3457ea33efd6895520447d1d97fe98d8`. The conflicts are resolved. This successor fixes two test-harness boundaries exposed by CI: per-case route-module preparation and actual durable-save completion before intentional runner termination. Production code and all existing test/turn deadlines are unchanged. [Exact-head Greptile review](https://github.com/paperclipai/paperclip/pull/13038#issuecomment-5587250594) is **5/5**, completed September 10 at 13:20:55 UTC, with no actionable findings or open review threads. [Fresh exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/34481724341) passes **all 24 jobs**, including Build and both required aggregates. Normal exact-head guarded merge was attempted and rejected by the remaining branch approval policy: CODEOWNER review is required and no human approval is present. Normal **squash auto-merge is enabled** as of September 10 at 13:36:26 UTC. Requested CODEOWNERS have been notified; no approval bypass or self-approval was used. Earlier-head results below remain historical evidence, not qualification of this successor. - Final exact-head Linux evidence: 995/995 chat integration cases; 36/36 agent-skills routes; 35/35 runner live-session cases, including real process kill/resume; 1948 runner Vitest cases with three existing benchmark/platform guards; 870/870 API-authority cases; and 104 browser cases with four existing optional skips. Rust, conformance/replay, full repository build, typecheck, canary, all server/workspace shards, and both required aggregates pass with normal CI concurrency. Earlier failed attempts remain recorded below. - Latest test-only qualification: 141/141 route/permissions/authentication cases pass in separate cold forks, with plain server types and independent review clear. The real-runner suite passes 35/35, with plain runner types and independent review clear. A controlled premature-save acknowledgement fails as expected; matching ownership/effect/process evidence, rejected saves, real turn outcome, test abort, and pre-kill liveness are covered. No local reproduction of the original CI scheduling failure is claimed. The preceding [CI run](https://github.com/paperclipai/paperclip/actions/runs/34479680858) passes 21/24 jobs, including all 995 Linux chat cases and browser aggregate (104 passed, four existing optional skips); only Build, the skills serialized shard, and the required verification aggregate fail. Its exact-head Greptile review was 5/5. Both failed job logs are retained. - Final fixture qualification: all eight focused Discord cases and all 995 chat integration cases pass. The exact modal statement/PID is observed before taking the real connection lock; the test then proves its actual blocking relationship before mutation. Original SQL execution, provider behavior, negative assertions, and 1s/15s timeouts remain unchanged. Independent review is clear and test/production hashes remain frozen. The preceding [CI attempt](https://github.com/paperclipai/paperclip/actions/runs/34477184777) passed 22 jobs, including Build/runner, typecheck, canary, all other test shards, and browser aggregate (104 passed, four existing optional skips); the two fixture failures and failed verification aggregate remain recorded, not relabeled as a pass. - Current queue-module composition: 308/308 recovery/batching/queue/Stop tests; 995/995 full chat integration; 89/89 module tests, including real PostgreSQL receipt-insert rollback; 24/24 workflow/module-boundary tests; plain server and UI types. All four actual local process/ACP browser paths pass in 1.4 minutes. Fresh databases, no skips or retries, stable reviewed source hashes. The initial boundary failure is retained; its no-op service wrapper was removed without changing recovery context or weakening the check. An exploratory standalone test-directory typecheck fails because its new upstream transformation config is not a standalone typechecking project; standard CI/build does not invoke it, and no configuration was weakened to suppress those diagnostics. - The preceding head `e02a63d462ce5d47433b0aeb632bb6fd20aab1ba` passed [all 24 CI jobs](https://github.com/paperclipai/paperclip/actions/runs/34436462958) and exact-head Greptile review at 5/5. Required CODEOWNER review prevented its normal merge before master advanced again. - Final extracted-module composition: 307/307 recovery, batching, queue and Stop-control tests; 995/995 full chat integration; 49/49 module tests including eight PostgreSQL adapter cases; and 19/19 issue-update tests. Plain server types pass. All four actual local process/ACP browser paths pass in 1.3 minutes. Fresh databases, no skips or retries in these cohorts, frozen source hashes, and independent review clear. - The preceding head `3e4e1c1c` passes [all PR CI jobs](https://github.com/paperclipai/paperclip/actions/runs/34415826820), including Build and required `ci / verify` and `ci / e2e`. Both the original Rust failure and the previously load-sensitive lineage fixture pass with unchanged Linux concurrency. Master advanced afterward and required this reconciliation. - Final master composition: 448/448 focused UI tests, 186/186 adapter tests, 24/24 queue/control tests, and 11/11 packaging tests. Plain UI, server, shared, and adapter types pass. Token gates and diff checks pass. Independent server and UI reviews are clear. - Stop-registration regression: both real-service cases fail against exact `a95` source and pass with the fix. The full corrected recovery/control suite passes 265/265. Duplicate-owner and failed-Stop controls also pass. Plain server types pass. The readiness barrier prevents provider startup without adding an acknowledgment to an already terminal run. - Final qualification strengthens terminal-field equality and repeats both affected cases successfully on a fresh database. All four actual local process/ACP browser paths pass again in 1.3 minutes, without skips or retries. The final screenshot shows Cancelled, a paused subtree, retained input, and no error toast. - Two new actual-service regressions fail before the merge fix. They prove that queued-comment adoption could consume a dedicated chat answer or add unrelated input to that answer. The fixed four-case cohort passes, including ordinary upstream continuation and adapter Stop controls. Full recovery passes 257/257. All four actual local process/ACP Stop browser flows pass in 1.4 minutes, without skips or retries, on a fresh database. - The unchanged runner artifact was qualified with 171/171 transport tests, 870/870 API-authority tests, conformance 1/1, and replay 11/11. Six controlled reader tests prove the exit/drain repair. Its local serial Rust workspace passed 546 top-level cases plus two invoked helpers; the later passing Linux CI supplies default-concurrency evidence. - Prior exact-source full chat integration passes 995/995. Settings regressions cover concurrent stale pages, 501 destinations, pending state, rejected updates, and explicit retry. These deterministic tests do not prove live provider behavior. - Retained failed attempts and their causes are in the [qualification log](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-chat-queue-and-webhook-repair.md). The first merge adapter run timed out while macOS slept for 290 seconds. Its unchanged repeat passed with a temporary sleep guard. No assertion, deadline, or CI gate was weakened. Review commands include `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-process-recovery.test.ts src/__tests__/issue-queued-comments-routes.test.ts` and `pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/acp-stop-continuation.spec.ts`. Database suites require fresh disposable databases. See the [browser runbook](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-04-chat-adapters-browser-e2e-runbook.md) for provider setup and separate live acceptance steps. ## Risks - This remains experimental. Deterministic tests and bounded live evidence do not establish every provider feature, tenant, permission layout, or media shape. Teams work-tenant qualification is still open. - Failed and uncertain provider effects remain visible and can require operator action. A transport receipt does not prove recipient visibility. - Native controller and runner artifacts must remain compatible. Preserve lease ownership, terminal authority, source binding, and quarantine during future changes. - Access and audit rows commit together, but activity notifications remain best-effort. This is not a new durable event outbox. - The PR operation does not deploy a live server, replace its runner, or change provider permissions. Remaining live qualification is documented in the [temporary handoff](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-open-qualification-followups.md). ## Model Used OpenAI Codex assisted with implementation, tool execution, testing, and review. The work records `gpt-6-astra` assistance. The environment does not report a context-window size. No private reasoning traces are included. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-10 10:06:45 -05:00
DottaandPaperclip	ebaeba40ee	feat: simplify agent onboarding and configuration (#13011) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators create agents and configure their runtimes in the board UI. > - The old creation flow presents several choices and a large form before an agent can start. > - The existing onboarding controls already provide clear provider connection steps. > - This pull request uses those controls in a new-agent wizard and organizes the full configuration pages. > - Operators can connect, test, save, and assign a first task while keeping the existing configuration tools. ## Linked Issues or Issue Description Related: #10974. That earlier open PR also reorganizes agent configuration. This PR follows the reviewed Storybook designs for agent creation and the current configuration tabs. **What existing behavior does this improve?** Agent creation, provider connection, runtime tests, and full agent configuration. **Current behavior** The creation dialog leads to a large manual configuration form. Provider login controls differ from onboarding. Environment variables and secret access appear in separate places. **Proposed behavior** Choose a name and adapter. Connect Claude or Codex through the existing onboarding controls. Configure and test the runtime, save the agent, and open a task dialog with that agent assigned. Use the same design on the existing configuration tabs. **Reason and benefit** The first setup asks for fewer decisions. The full editor keeps instructions, skills, runtime controls, secret access, permissions, keys, and revisions available in clear sections. **Breaking changes** The board creation and configuration layouts change. The test-environment API adds an optional, allowlisted `testCredentials` field for one-shot probes. Database contracts stay the same. Native ACPX tests now reject unsupported local platforms before a CLI login can mask the runtime restriction. ## What Changed - Added a new-agent wizard with numbered steps, adapter branding, provider connections, editable model choices, runtime tests, and confirmation. - Added Codex app-server, Claude ACPX, and OpenCode runner choices. - Stored API credentials through existing secret APIs and persisted references in agent configuration. New setup keys are isolated from credentials used by existing agents. - Preserved external-agent invitations beside the wizard, including optional messages, one-time prompts, and clipboard fallback. - Added OpenRouter provider and secret bindings for Pi and OpenCode. - Added adapter-specific prerequisite fields for Cursor, Gemini, Kimi, and Hermes. Cursor Cloud keys are saved as new organization secrets. - Fixed Cursor Cloud repository field mapping, omitted empty remote environment values, and added useful model and repository error messages. - Preserved complete MCP assignments when multiple valid profiles contain more than 250 tools in total. Generated profiles retain exact tool selectors. - Added service branding and deployment-aware adapter choices. Cloud setup offers Claude, Codex, and OpenCode; local native runners require the experimental setting. - Made the agent list responsive at intermediate widths. - Applied the reviewed design to the real agent configuration pages. Kept the instruction editor, skills, and existing mutations. - Combined secret access and environment variables under one Save and Discard action. - Added interactive Storybook screens for setup, configuration, confirmation, authentication, and test results. - Fixed Pi provider-error parsing and thinking-effort persistence. Native ACPX validates Linux x64 on the actual local, SSH, or sandbox target. - Redacted the complete transient probe-credential field from HTTP error logs, including rejected provider names. ## Verification - Current head `df0292fe6` has a fresh Greptile 5/5 review with no unresolved findings. All 31 executed CI checks passed, including the aggregate verification gate and all browser E2E shards. Storybook visual regression is skipped by its workflow; the local Storybook build passed. - Browser tests completed real assigned tasks with direct Codex, Claude, OpenCode, Pi, and native Codex. - Verified external-agent invitation generation and automatic prompt copying in the live browser. - Pi and OpenCode used an existing OpenRouter secret. Browser checks covered save and reload, instruction edits, skill selection, environment-variable Save and Discard, and assigned task creation. - Invalid Claude API credentials remained on the connection step with an error. A live Pi/OpenRouter invalid-key probe returned a provider failure and left the user-secret inventory unchanged (zero entries before and after). - Full workspace typecheck and build passed after rebasing onto current master. After review fixes, server and UI typechecks, token gates, and the full build passed again. Storybook built successfully. - All 5,542 local UI tests passed. The Cursor Cloud and Pi adapter regressions passed all 24 tests. Review regressions passed 69 server tests and all 18 agent-list tests. - The local full test command ran 6,971 general server tests successfully. Editing review fixes during that long run caused nine tests to use stale modules; fresh isolated runs passed. An unrelated embedded-Postgres fixture hit the host shared-memory limit; its 15 affected tests passed when the fixture groups ran separately. - Local workspace groups passed after rerunning 18 CLI tests sequentially to avoid host database limits and parallel-load timeouts. The local full command stopped at the general server phase, so serialized server verification comes from the five passing CI shards. - Browser testing at 390px confirmed that the agent action menu opens and the page has no horizontal overflow. CI browser E2E shards passed. - Review the `Onboarding / New agent` and `Agents / Configuration refresh` Storybook groups. In the real app, create an agent, run its connection test, save it, assign a task, and reload its configuration. ## Risks - This changes the main agent setup and configuration UI. Regression tests cover routing, persistence, secret bindings, and form actions. - Native Claude ACPX requires Linux x64. Direct Claude works on macOS. Remote checks execute a bounded platform probe and reject unsupported or unverified targets. - A native OpenCode task reached the provider context limit because of its tool payload. Its provider connection test passed. Direct OpenCode completed a task. This existing native execution limit is not fixed here. - Claude and Codex connection keys use the existing user-secret store. Other runtime setup keys use distinct organization secrets. Existing credentials are never rotated. Probes do not store entered keys. Failed agent creation removes newly staged credentials. - Cursor Cloud has not completed a live task. Its authenticated account still needs GitHub repository access. The live run passed MCP provisioning, remote environment validation, and explicit Auto model selection before the repository prerequisite blocked execution. - Generated runtime MCP profiles can exceed the public profile-edit request limit. They still contain exact catalog selectors and preserve permission boundaries. - No database migration, dependency, lockfile, or workflow changes are included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, shell execution, and browser automation. The runtime did not expose the exact model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-08 14:22:32 -05:00
DottaandPaperclip	9dd6526b47	fix(security): harden privileged server boundaries (#12776) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server controls secrets, host files, outbound requests, and workspace commands > - A red-team review found cases where restricted callers could cross these trust boundaries > - These cases could expose credentials or let untrusted input reach privileged resources > - This pull request applies least-privilege checks at each affected server boundary > - The benefit is safer agent execution without changing the private-instance bootstrap contract ## Linked Issues or Issue Description **What happened?** Several server paths used authorization, redaction, or content-delivery rules that were too broad. Restricted agent keys could obtain company-level operational data. Some adapter and instruction paths could reach server-owned network or file resources without the required owner approval. **Expected behavior** Paperclip must redact credential values, enforce restricted-key scopes, guard outbound network access, prevent same-origin script execution, and reserve host-level file and command controls for authorized operators. **Steps to reproduce** 1. Configure an authenticated development instance at the parent commit. 2. Exercise the affected APIs with a restricted agent key or a non-instance-admin company user. 3. Observe that the parent commit returns privileged data or accepts a privileged operation. 4. Repeat on this branch and observe a redacted response, a safe download, or an HTTP 403 response. **Paperclip version or commit** The findings reproduce from commit `39898ab22` and are fixed by this pull request. **Deployment mode** Authenticated self-hosted server and local development modes. **Installation method** Built from source with pnpm. ## What Changed - Redact generic secret `value` and `token` fields recursively in structured logs. - Classify exact and separator-suffixed `KEY` environment names as secrets in company exports. - Limit restricted self-identity responses and protect company run, log, and secret catalog APIs. - Route HTTP adapter requests through DNS-pinned SSRF protection with exact private-origin allowlisting. - Download HTML, SVG, and other script-capable assets with `nosniff` and a sandbox CSP. - Require instance-admin access for external instruction roots and exports that read them. - Block agent-authenticated host command persistence across supported workspace runtime shapes. - Apply the central runtime-management decision before workspace command controls. - Keep the documented first-user instance-admin claim contract unchanged. - Add regression tests and server-owner configuration documentation. ## Verification - `pnpm -r typecheck` passes. - The Node 24 remediation suite passes with 365 tests. It skips 25 environment-gated tests. - `pnpm build` passes under Node 24. - `git diff --check` passes. - The full local runner reaches known macOS-only general-server harness failures before the serialized route lane. The Linux PR matrix is the authoritative full-suite gate. ## Risks - Restricted agent keys now receive HTTP 403 responses from company-wide run, log, and secret catalog endpoints. - Script-capable assets now download instead of rendering inline. - External instruction roots now require instance-admin access. - Private HTTP adapter endpoints now require an exact origin in `PAPERCLIP_HTTP_ADAPTER_PRIVATE_ENDPOINT_ALLOWLIST`. - Public HTTP adapter endpoints remain enabled. Redirects and metadata or link-local targets remain blocked. - No database migration is required. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5. The exact serving snapshot and context-window size are not exposed. The model used tool-enabled reasoning, repository access, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-03 14:15:32 -05:00
DottaandPaperclip	cabc9146d0	feat(apps): add secure remote MCP and PostHog setup (#12339) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps give those agents governed access to external tools. > - Remote MCP setup needs secure endpoint validation and durable credentials. > - PostHog needs both browser sign-in and personal API key setup paths. > - This pull request adds the shared remote MCP foundation and the PostHog definition. > - The benefit is a secure and reusable base for later app connection work. ## Linked Issues or Issue Description Refs #11965 This is stack 1 of 11. It replaces the first reviewable part of #11965. ## What Changed - Add guarded remote MCP setup and credential handling. - Add PostHog OAuth and API key connection methods. - Add focused server, shared contract, and UI coverage. - Keep the migration replay-safe and idempotent. - Give the late-close security regression the same 10-second CI headroom as the adjacent real-timer handshake test. - Synchronize fake-timer handshake tests at the exact ensure-session boundary so real filesystem setup cannot race the fake deadline. - Drive PTY overflow coverage only after listener registration so scheduling cannot reorder the test fixture. ## Verification - pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts server/src/__tests__/plugin-worker-manager.test.ts (220 passed; affected cases also passed five focused stress repetitions) - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never leaks a sandbox-provided value from a late close rejection into logs or the result"` (1 passed) - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never promotes a late ensureSession resolution|closes a late-resolving real handle exactly once"` (2 passed) - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - `pnpm --filter @paperclipai/db check:migrations` - `pnpm build` ## Risks - Remote endpoint validation can reject configurations that previously passed without checks. - OAuth configuration errors can block setup until the operator corrects the provider settings. - The migration uses guarded statements so repeated execution is safe. - The test-only synchronization changes do not affect runtime behavior; they remove filesystem/fake-clock and listener-registration races observed under parallel CI load. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-29 12:08:32 -05:00
NoSoloSoftandAndrew Aymeloglu	156830006b	fix: redact HTTP cookies from server logs (#7977) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators rely on Paperclip server logs for maintenance, incident triage, and support handoffs. > - The HTTP logger persisted request metadata and only redacted authorization headers. > - Request cookies and set-cookie headers can contain active session material and should not be written to durable logs. > - This pull request keeps the fix intentionally narrow: centralize the HTTP log redaction path list and include cookie-bearing headers. > - The benefit is lower credential/session leakage risk from routine server.log collection or sharing. ## Linked Issues or Issue Description No GitHub issue exists for this exact local finding. Inline bug report: - Type: security/privacy bug. - Affected area: server HTTP logging middleware. - Observed problem: local Paperclip maintenance found raw cookies present in server.log. - Expected behavior: durable HTTP logs redact authorization and cookie-bearing request/response headers. - Impact: anyone with access to copied/exported logs could see session-bearing cookie values. - Related/open PRs found during dedup search: #7242, #7306, #7346. This PR is the minimal local fix branch created from the verified local maintenance patch; those PRs may be better upstream candidates if maintainers prefer their broader coverage. ## What Changed - Added `HTTP_LOG_REDACT_PATHS` for HTTP logger redaction paths. - Kept existing `req.headers.authorization` redaction. - Added redaction for `req.headers.cookie`, request `set-cookie`, and response `set-cookie` paths. - Added focused tests asserting the required redaction paths are present and that pino-http output redacts live request/response header secrets. ## Verification - `pnpm exec vitest run server/src/__tests__/http-log-redaction.test.ts` - `pnpm --filter @paperclipai/server typecheck` - Pre-commit TruffleHog scan: 0 verified/unverified secrets. - PR CI observed passing so far for policy, Typecheck + Release Registry, Build, e2e, Socket, Snyk, security-review, and serialized/workspace suites; remaining jobs may still be running. ## Risks - Low runtime risk: this only expands pino redaction paths. - Possible coverage risk: broader redaction helpers in related PRs may cover more serialized variants beyond the pino-http request/response header pipeline tested here. - No migrations, schema changes, or UI changes. ## Model Used - OpenAI Codex via Hermes Agent, model gpt-5.5, tool-using coding/ops session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>	2026-07-20 17:17:08 -05:00