PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-25 02:26:12 +02:00

Author	SHA1	Message	Date
DottaandPaperclip	d66acb7ac1	feat: automate Slack bot app setup and installation (#15413 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connectors give each agent a customer-owned bot and task-backed conversations. > - Manual Slack setup requires app creation and copying durable credentials. > - Operators need a shorter setup that an assisting agent can use safely. > - This pull request creates the app through Slack's Manifest API and installs it through OAuth. > - Durable registration state supports recovery without creating another app. > - A four-screen wizard, automatic avatar upload, and OAuth account linking reduce setup work. > - Connector settings and per-turn tool guidance support daily use after installation. ## Linked Issues or Issue Description Subsystem affected Native Slack bot setup, company secret storage, chat connector management, and agent tool guidance. Problem or motivation New Slack bots require manual app creation and copying a signing secret and bot token. Interrupted setup can create duplicate apps. The setup and management screens contain unnecessary controls. Agents also need guidance for native questions, files, thread replies, and governed Slack actions. Proposed solution Use a temporary app-configuration access token to create a customer-owned app. Save durable secrets in the vault. Bind OAuth to the initiating actor, company, endpoint, registration revision, scopes, and configured origins. Preserve manual and existing-app recovery. Link the installing user's account, send a welcome DM, and advance from saved server evidence. Keep request-URL recovery instructions available if automatic connection detection waits. Alternatives considered The Slack CLI adds installation requirements. Socket Mode changes transport. A shared Paperclip-owned app changes app ownership. These alternatives are outside this change. Roadmap alignment This extends existing chat connectors and secrets capabilities. Related public work: #14037 and #13954 cover Slack MCP prerequisites and user OAuth. No duplicate bot-registration PR was found. ## What Changed - Share one reviewed manifest builder between automatic registration and manual setup. - Add replay-safe migration 0318 and company-bound registration state with vault references and uncertain-creation recovery. - Add registration, installation, callback, and resume APIs with short-lived, single-use OAuth state. - Save installation credentials before downstream checks and preserve bot identity constraints. - Reduce automatic setup to four screens. Keep advanced app details, manual recovery, and existing-app setup. - Upload the agent avatar with the Paperclip dark background. Link the OAuth installer's account and send setup DMs. - Show agent and connector-owner avatars. Simplify settings, access, and conversation screens. - Discover joined Slack channels and enable them by default. Start a task from a bare mention and admit same-thread follow-ups. - Refresh Slack tool guidance each turn. Add native-form, file, approval, and delivery regressions plus manual model probe definitions and sanitized acceptance records. - Update deployment/database docs, OpenAPI, redaction, removal cleanup, production Storybook stories, and provider browser tests. - Merge current master and move the registration migration after its latest migration without rewriting published commits. The completed Slack success view intentionally has a single centered Done action and no Save & exit, as explicitly requested by the product owner. `DESIGN.md` records this exception; unfinished setup steps retain the aligned wizard footer. ## Verification - Passed after the master merge: repository typecheck, full build, Storybook build, design-token gates, module-boundary gates, and migration generation. - Passed: all 352 focused Slack deterministic tests and all 14 affected provider browser tests. Browser tests use controlled provider fixtures and a separate throwaway instance. - Passed on current head `c5d01e0e2`: the complete GitHub test matrix (general server, chat, all workspaces, serialized server, and Runner), all eight browser shards, typecheck/release registry, build, canary dry run, security checks, and policy gates. There are 52 passing checks and no pending or failing checks. - Greptile completed on the exact current head with 5/5 and no actionable findings or open review threads. - Local repair verification passed 93 focused tests, including same-app reinstall after revocation and rejection of consent started before revocation, the AgentMail browser journey, and repository typecheck. Local build and Storybook build also passed. The redundant local full-suite rerun was stopped after the complete current-head CI matrix passed. - Real Slack setup and agent replies were exercised in the authorized isolated test drive during the setup iteration. - The ten additional model probes were attempted with legacy `codex_local`, `gpt-5.6-sol`: five passed, two failed, and three were partly verified. Native runtime is not qualified. See `server/src/services/connectors/slack/evals/2026-10-08-acceptance.md` for evidence and limits. - Passing model probes cover native forms, downloaded file bytes, bare mentions with thread replies, explicit posts/reactions, and saved approval denial. - The controlled uncertain-write probe found wrong delivery-check IDs. The canvas fallback attempt used an invented tool name. Search pagination/native search, a private-source denied-tool receipt, and distinct board/webhook origins remain unqualified. Reviewer path: enable Chat connectors, start Slack chat setup, select an agent, enter an app-configuration access token, and approve Slack installation. Send a message to the bot and confirm that setup advances to success. Inspect settings and allowed channels. See `doc/connections/SLACK-AUTOMATIC-SETUP.md` for deployment and recovery. ## Risks - Slack app creation has no provider idempotency guarantee. A timeout after dispatch stays uncertain until the operator checks Slack. - OAuth needs a stable public HTTPS board origin. Webhook ingress may use a separate configured HTTPS origin. Workspace policy can delay installation. - Migration 0318 can replay safely on instances that applied the earlier development migration. - OAuth installation now links the installer to the initiating Paperclip user. Identity checks and company access rules still apply. - Joined channels now enable bot responses by default. Linked-user authorization and per-action approval rules still apply. - Model behavior has the documented delivery-check and canvas fallback failures. A passing CI run does not establish that every model probe passed. - Removing the connection does not delete the customer's Slack app. No new first-party telemetry is added. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and browser verification. The runtime does not expose a more specific authoring model ID or context-window size. The live bot probes used OpenAI `gpt-5.6-sol` through `codex_local` in legacy mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-08 06:33:31 -05:00
DottaandPaperclip	b508a05c43	feat: add internal agent complaints and suggestions (#15367 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use legacy skills or native runner tools to work on tasks. > - Those agents can encounter friction that does not belong in the task thread. > - A complaint should preserve the raw reaction. A suggestion should describe an improvement. > - This pull request adds attributed local storage and both submission paths. > - Agents can submit feedback once and continue their primary work. ## Linked Issues or Issue Description Subsystem affected Server, database, shared contracts, runtime skills, and native runner tools. Problem or motivation Agents have no default internal channel for incidental complaints and suggestions. Sending this feedback through task comments adds noise and can alter task workflows. Proposed solution Store free-form feedback in the current instance database. Derive agent, run, company, and task attribution from active authority. Provide default legacy skills and provider-neutral native actions. Keep the instructions close to Warp's MIT-licensed originals. Alternatives considered Task comments and external Slack delivery add unwanted side effects. Mandatory suggestion fields and short editorial limits would discard useful feedback. This release has no listing API, UI, read tool, automatic triage, or external forwarding. Roadmap alignment This is a maintainer-requested addition to the existing runtime skills and runner tool paths. It does not duplicate a listed roadmap milestone. Searches for complaint tooling, suggestion-box, and agent commentary found no overlapping public PR or issue. ## What Changed - Add the company-scoped `agent_commentary` table, shared validation, and idempotent migration `0310`. - Add one transactional service and the agent-only POST route. Validate active authority before writes or replay. Redact known credentials. Commit a content-free audit with each new record. - Add `submit_complaint` and `submit_suggestion` to standard, ask, and planning modes. Keep review, revocation, and completion restrictions. Store replay identity on the commentary row. - Mount `complain` and `suggestion-box` by default for legacy agents. Bundle a dependency-free Node.js stdin helper in the operational skill and allow its POST through the sandbox bridge. - Preserve Warp's complaint voice and suggestion guidance, with attribution and local transport adaptations. Keep source attribution and MIT notices in each skill's LICENSE, outside runtime instructions. - Document custom-runtime HTTP use and database inspection. Add real-database tests and a repeatable live Codex smoke for local and Daytona execution. - Pin the lagging-source migration fixture before the identity-repair migration so later migrations preserve its regression coverage. ## Verification - Personally ran real Codex submissions in all four environments on 2026-10-06. Local runs passed at 20:35 UTC. Daytona native passed at 20:31 UTC; Daytona legacy passed at 20:33 UTC. Each stored exactly two rows with company, agent, run, and task attribution, wrote the continuation marker, exited zero, created no task comments, and left task status unchanged. Each recorded two content-free activity entries. - Daytona used production provider hooks, real remote execution and file transfer, the legacy queue callback bridge, and native private WebSocket ingress. The current Linux runner was built from `abf47b595`, staged, and verified against controller contracts. Both sandboxes were confirmed deleted. This is a focused feedback transport smoke; it does not claim full Runner E2E catalog or browser qualification. - The immutable base image and Linux binary digest are recorded in [the verification documentation](https://github.com/paperclipai/paperclip/blob/codex/agent-commentary/doc/agent-commentary.md#verification). The smoke script can save content-free JSON evidence. No credentials or feedback bodies are in these reports. \| Environment \| Runner \| Complaint row \| Suggestion row \| \| --- \| --- \| --- \| --- \| \| local \| legacy Codex \| `59413a00-1de2-4bb1-bcc6-9c4b54c64aa6` \| `3db2364d-3e15-4f47-846f-875d3902999d` \| \| local \| native Codex \| `5da22b5f-41df-4de5-8ba0-d9345ab01267` \| `2d5abe17-dd41-403c-a5ee-4729f2d58921` \| \| daytona \| legacy Codex \| `27c9d0aa-8477-409f-9da0-e8ffa48dee50` \| `209681c9-d1e9-4ce1-999e-48fa07692389` \| \| daytona \| native Codex \| `6eb001bb-4bcf-43f7-8717-f662f53dc7c3` \| `77c383d8-a997-49e5-a33e-25c70e15c0b2` \| - Run the local check with `node cli/node_modules/tsx/dist/cli.mjs server/scripts/verify-agent-commentary-live.ts`. The documentation gives the Daytona invocation. Both use disposable instance databases and normal Codex provider usage. - Repository `pnpm -r typecheck` and `pnpm build` passed after the test extension. The build includes runner generation, contracts, and replay checks. The smoke scripts also passed a separate TypeScript check. The lagging-source migration regression passed. All equivalent current-head Vitest CI shards passed. The local monolithic `pnpm test:run` invocation was stopped after CI supplied that coverage; it did not complete locally. - Focused tests cover company isolation, spoofing, revoked credentials, stale ownership, post-finish rejection, concurrent replay, conflicting keys, atomic rollback, and deletion through existing services. Boundary tests cover empty text, Unicode, text beyond 8,000 characters, and the 524,288-character ceiling without truncation. Mounting tests cover Codex, Claude, and sandbox staging. Helper tests cover standalone Node execution, stdin, invalid UTF-8, redirects, HTTP failure, and its deadline. Privacy and bridge tests cover successful and rejected requests. - Instructions were compared with Warp's originals. MIT notices and source credits live only in LICENSE files. Native tools preserve truthful disclosure when asked, without routine announcements. - [Full CI](https://github.com/paperclipai/paperclip/actions/runs/37508559190) and Greptile 5/5 passed on the earlier feature commit `5209c3501`. The later head found the migration-fixture assumption fixed in this update. On `8a4965164`, all 55 check contexts passed after one browser shard rerun. Its initial reviewer signoff failure also passed an isolated local browser run (1 test). Greptile scored that head 5/5 and identified one smoke cleanup gap. `6ecbafb0b` fixes failed-acquisition cleanup with four passing tests and a passing smoke-script typecheck. Fresh CI is pending for this final test-only fix. No commentary production code changed during verification. ## Risks - Feedback is internally attributed. It is not anonymous. Existing redaction removes known credentials, but agents must still omit sensitive content. Normal provider transcripts can include their submitted arguments. - Default skill availability changes for existing legacy agents. Runtime policy filtering still applies. The helper uses the existing Node.js runtime with no extra dependencies; custom runtimes can call the HTTP endpoint. - Feedback is removed with its run, agent, or company. Task deletion clears only the issue pointer. Normal database backups include the table. - The migration is additive and has no backfill. Writes serialize on the active run for replay consistency. No server suggestion quota is imposed. ## Model Used OpenAI `gpt-6-astra` through Codex, with `xhigh` reasoning effort and a reported 258,400-token context window. Capabilities used: repository inspection, code execution, and live runtime verification. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-06 16:14:06 -05:00
DottaandPaperclip	e34abee670	feat(mcp): connect assistants to a team with user OAuth (#14846 ) ## Thinking Path > - Paperclip gives teams durable tasks, agent execution, budgets, and approvals. > - People also use assistants in Codex, Claude, and other MCP clients. > - Those assistants need a scoped connection that preserves the person’s permissions and attribution. > - Delegating a task must not turn the assistant into the assigned agent. > - This PR adds opt-in user OAuth, ten first-party tools, browser consent, and workflow packages. > - Paid product evals verify the resulting tasks, documents, attribution, retries, and access boundaries. > - The team keeps working after the assistant conversation ends. ## Linked Issues or Issue Description Problem or motivation A person cannot connect an external assistant to an existing team through browser consent and safely delegate durable work as themselves. Proposed solution Expose an opt-in `/mcp/paperclip` endpoint with individually described first-party operations. Bind every connection to a person, client, company, resource, and scopes. Reuse domain authorization and scheduling. Package shared team-review, delegation, and follow-up workflows for OpenAI/Codex and Claude. Alternatives considered Related PRs #9393 and #12549 cover earlier remote MCP and board-operator approaches. This change uses user OAuth and a bounded public catalog. It does not expose a generic executor, operator administration, static shared board credentials, or external agent execution. Registry listing work in #9851 is a separate distribution step. Roadmap alignment This maintainer-requested implementation extends the governed MCP gateway, activity attribution, durable work products, and hosted deployment direction in `ROADMAP.md`. It implements the first release of the saved design plan; external agent participation and granted third-party tools remain later releases. ## What Changed - Add MCP 2.0 discovery and task status/comment/document Events on the same authenticated endpoint. Persist subscriptions and delivery receipts, verify HTTPS callbacks, sign Standard Webhooks, encrypt callback material, recheck permissions/Cloud membership, and bound retries/expiry. Older MCP clients keep their existing tools. - Add discovery, dynamic client registration, S256 PKCE, resource validation, rotating refresh tokens, revocation, and company consent. Store credentials as hashes and recheck membership at execution. - Add tools for connection identity, agents/projects, task search/read/create, human comments, documents/deliverables, and pending-approval links. Preserve current domain permissions and scheduling. - Add durable mutation receipts across reconnects. Matching retries replay results; uncertain outcomes keep the same request ID and require inspection. - Add consent and connection-management pages, OAuth log redaction, shared plugin workflows, and separate OpenAI/Codex and Claude package outputs. - Add eight paid Product E2E cases across three models, independent durable-state grading, usage evidence, cleanup, and report integration. Add task-document guidance and regenerate the runner capability inventories. - Add migrations 0301 and 0302, the dated implementation plan, result notes, and direct-client setup instructions in `doc/public-mcp.md`. ## Verification - Merge integration `e180b1948`: resolved conflicts with current master, preserved both eval registries, regenerated capability catalogs, and regenerated migrations as 0301/0302 while keeping the original replay-safe SQL byte-identical. Local migration safety/snapshot tests (26), MCP/OAuth tests (38), redaction/OpenAPI tests (71), and eval catalog/grading tests (198) pass. Token and capability gates pass. Full recursive typecheck passed. Fresh Greptile review is 5/5 with no unresolved findings. CI is green on this exact head (55 successes, two intentional skips, one neutral result): one unchanged Cursor sandbox test timed out at 10 seconds, then passed locally in 856 ms. A single retry of that failed shard and the aggregate workflow passed. Merge remains blocked on the repository code-owner approval rule. Earlier checks passed at `6aa0962d4fb715f2190bb7bb22efacab2e58495d`: 55 successes, two intentional skips and one neutral result. [The earlier CI run](https://github.com/paperclipai/paperclip/actions/runs/36901592350) includes all test shards, browser tests, typecheck, build and canary dry run. Greptile was 5/5 on that commit with no unresolved review threads. GitHub still requires code-owner review under the repository merge rules; passing checks do not bypass that approval. Paid source fingerprints remain separate below and in the dated result note. - Paid Events qualification passes 3/3: GPT-5.4 Mini, Claude Haiku 4.5 and Claude Sonnet 4.6. Each uses a real public HTTPS callback, signature verification and report retrieval in a fresh conversation. A final Mini regression passes after the quota/status fixes. All evidence validates. Bounded tunnel startup retries occur before provider calls and remain visible; failed earlier attempts retain their original grades. - The earlier complete seven-case matrix passes 21/21, with a separate 3/3 delegation regression. Two preceding matrices also passed 21/21 each. A complete 24-cell matrix including Events has not been run. [The dated results](doc/plans/2026-10-01-public-mcp-paid-eval-results.md) retain exact source fingerprints, failures, model IDs and partial costs. - Node 24: repository-wide `pnpm -r typecheck` and `pnpm build` pass after merging master. Server typecheck passes after the final quota/status changes. Eval typecheck and all 892 eval-support tests pass. - All 33 real MCP/OAuth tests pass. The preceding combined MCP, redaction, private-address and DNS-rebinding run passed 129 tests; two later MCP regressions cover quota reuse and unchanged-status suppression. All 28 adjacent issue-tree/stale-lock route tests pass. CI then found a null checkout result in the existing concurrent-workspace path; logging now uses optional status access. All 12 closed-workspace tests and all 33 MCP tests pass after that correction. The exact-start event calibration exposed a timestamp gap; scanning now includes the subscription start, with all 33 MCP tests and server typecheck passing. These two narrow corrections follow the paid regression. - A real Core → Cloud → Core authority round trip passes OAuth, MCP 2.0 subscription/delivery, current membership loss, unsubscribe, legacy SDK tools, refresh and revocation. Its callback transport is a fixture with independent HMAC verification. The paid Events campaigns separately prove public HTTPS delivery. - Earlier component qualification passed UI 7,117 tests, CLI 502, shared 832, skills catalog 20, database 160 and OpenAPI 10. Token gates, module boundaries, migration order and plugin regeneration passed. CI covers general/serialized suites, eight browser shards, runner checks, typecheck, build and canary dry run. - Local full-suite limitation: the earlier monolithic run was not clean. It encountered overlapping schema rebuilding, Mac database shared-memory limits and isolated CLI/fixture failures. Targeted reruns passed. The existing >32 MiB Git filename stress test still hit its 300-second Mac timeout. The additional serialized sweep stopped after 62 passing suites once CI passed. Original failures and partial logs remain; this PR does not claim a wholly green local monolithic run. - Local Codex CLI and Claude Code OAuth login and MCP SDK interoperability were verified. Public-store installation, actual ChatGPT Work Cloud Events UI, staging HTTPS client behavior and hosted newcomer provisioning remain release gates. Enablement is moving to Settings → Experimental → Assistant connections (MCP) in the stacked follow-up [#14933](https://github.com/paperclipai/paperclip/pull/14933). Merge both for the intended setup experience. This foundation branch alone still uses `PAPERCLIP_PUBLIC_MCP_ENABLED=true`. After deployment, set `PAPERCLIP_PUBLIC_URL` to the authenticated instance's HTTPS origin, and connect to `/mcp/paperclip`. Select a team and allow writes in browser consent. Configure an available agent and budget, then delegate and retrieve results later. For Events, rescan the deployed plugin catalog in ChatGPT Work Cloud; the host supplies its webhook credentials when the user asks to watch a task. See [the setup runbook](doc/public-mcp.md). ## Risks - Events are at-least-once and may arrive out of order. No replay cursor is advertised. Clients must refresh finite subscriptions, read current state and avoid comment feedback loops. Callback material uses the instance secrets master key; hosted subscriptions require the updated Cloud broker and are bounded to five minutes/the access proof expiry. - ChatGPT Work Cloud/dot event UI, plugin rescan and a hosted staging subscription remain deployment gates. Local signed-webhook and paid model evidence does not claim those surfaces have been exercised. - Disabled by default. Merging adds schema and opt-in code; it does not deploy a public endpoint, publish a store listing, create a team, or start paid agents. - Migrations 0301 and 0302 are additive and idempotent. Their SQL is unchanged from the earlier preview numbers, so hash-aware upgrade reconciliation preserves prior staging applications. Normal instance upgrades must apply it before enabling MCP. - Task creation and comments can schedule paid agent work. Consent and tool descriptions disclose that effect. Revocation blocks future calls but does not undo delegated work. - Public deployments need edge rate limits and credential-safe logging. Internal dispatch is restricted to the closed catalog and carries a request-local verified actor. - Hosted onboarding requires the companion Cloud broker, encryption-key configuration, and tenant rollout. Self-hosted direct connections can use this PR alone. - Store acceptance and agent-mode participation are not claimed. Checked-in plugin endpoints are development defaults; rebuild packages for a real deployment before installation. ## Model Used OpenAI GPT-6 in Codex, with reasoning, tool use, and code execution. A more specific serving version and context-window size were not exposed by the session. Paid eval models: `gpt-5.4-mini-2026-03-17`, `claude-haiku-4-5-20251001`, and `claude-sonnet-4-6`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted/component checks; full local-run limitations are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-06 11:48:53 -05:00
Devin FoleyandPaperclip	6f40e23536	fix(logging): redact cloud authentication headers (#14413 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server records HTTP requests to help operators diagnose failures. > - Cloud requests carry tenant credentials and signed assertions in headers. > - The HTTP logger did not redact four of these headers. > - This pull request adds those headers to the existing redaction list. > - Operators retain the route, method, and response status without recording these values. ## Linked Issues or Issue Description What happened? HTTP request logs could contain cloud tenant credentials, session identifiers, runtime identity assertions, and cloud control assertions. Expected behavior The logger must redact these header values for successful requests and failed requests. Steps to reproduce 1. Create an Express app with the production HTTP logger and redaction configuration. 2. Send a request with the four cloud headers and distinct test values. 3. Inspect the serialized request headers for responses with status 200, 403, and 500. Paperclip version or commit Reproduced on `0f14d26123` before this fix. Deployment mode Server with cloud proxy authentication. The regression test uses an in-process Express server. ## What Changed - Redact `x-paperclip-cloud-tenant-token`, `x-paperclip-cloud-session-id`, `x-paperclip-cloud-runtime-identity`, and `x-paperclip-cloud-control` in HTTP request logs. - Test real logger output for HTTP 200, 403, and 500 with mixed-case request header names. Route 403 and 500 through the production error handler. Check response bodies, log levels, and server error context. - Check that the method, route, and status remain available. ## Verification The `server/src/__tests__/http-log-redaction.test.ts` suite passed, including all three new cloud-header cases. - Rebased onto master at `14795136f5`. - `pnpm exec vitest run server/src/__tests__/http-log-redaction.test.ts` passed: 59 tests, including all three new response-status cases. The suite and server typecheck also passed after the error-handler coverage update. - `pnpm build` and `pnpm -r typecheck` passed. The full local `pnpm test:run` was attempted and stopped after the failures listed below. Greptile is 5/5 with zero unresolved review threads on the latest head. All checks for head `373d29e2f1` passed (53 successful, two intentional skips). - Full local validation did not pass. The attempt reproduced company-skill cache permission failures, the terminal-workspace cleanup assertion, a heartbeat feedback timeout, and one process-conversation timing failure. It was stopped during the general-server stage after these failures. Remaining general-server tests, other workspace groups, and serialized-server stages did not complete locally. Earlier clean-master checks reproduced the cache failures and isolated cleanup retries passed. The latest-head GitHub suite passed all of these groups. - No browser suites ran locally. This change does not affect browser behavior. ## Risks - These four values will no longer be available in HTTP logs. Route, method, and status remain available. - This change applies to new log entries. It does not remove old entries or rotate credentials. - No schema, API, or authentication behavior changes. ## Model Used OpenAI Codex, GPT-6, with repository inspection, code editing, and test execution. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local-run failures and incomplete stages are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 09:59:15 -07:00
Devin FoleyandPaperclip	92d4868e79	fix(server): isolate run errors and redact runtime capability headers (#13826 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - Operators use Sentry to investigate failed runs and server errors. > - Run reports attach a task ID, run ID, error code, and adapter fingerprint. > - The server skips Sentry's OpenTelemetry setup to preserve its separate tracing and privacy settings. > - Without an async context manager, a scope mutation can attach old run data to later errors. > - The HTTP logger also retains a runtime credential capability header. > - This change isolates run metadata and redacts that header so diagnostics identify failures without leaking credentials. ## Linked Issues or Issue Description Refs #13446 and #13719. What happened? After a terminal run failure, an unrelated server exception can inherit that run's tags, context, and fingerprint. Sentry then groups a database error with an earlier adapter failure. The real SDK reproduces this with the application's `skipOpenTelemetrySetup: true` setting. HTTP request logs also retain the `x-paperclip-github-capability` header, which must be treated as a credential. Expected behavior Run metadata belongs to the terminal run event. Later exceptions must not inherit it. Every genuine error must still be captured. Runtime capability headers must be redacted on success and failure logs. Steps to reproduce 1. Initialize the optional Sentry SDK with the application's options and an in-memory transport. 2. Capture a terminal run failure. 3. Capture an unrelated exception. 4. Inspect the second event. Before this fix, it contains the first run's identity and fingerprint. 5. Send a request with a fixture runtime GitHub capability header. Before this fix, HTTP logs retain the fixture value. ## What Changed - Pass tags, context, and fingerprint directly to `captureException` instead of mutating the ambient scope. - Preserve the existing run fields, grouping keys, ordinary exception capture, and privacy settings. - Test two run identities interleaved with unrelated exceptions against the real optional SDK. - Update the capture contract tests and document event-local run metadata. - Redact the runtime GitHub capability header through the existing HTTP logger policy. Test successful, denied, and failed requests. - Add a dedicated GitHub-hosted CI check that installs the exact optional SDK version declared in `server/package.json`. It fails if the real-SDK regression would be skipped. The SDK stays outside the workspace and production dependency graph. ## Verification - The real-SDK regression failed before the fix because the unrelated event contained `contexts.run_failure`. - Five focused suites passed: 123 tests, including all optional SDK tests. Suites: `run-failure-sentry-real-sdk.test.ts`, `run-failure-sentry.test.ts`, `sentry.test.ts`, `run-failure-report.test.ts`, and `http-log-redaction.test.ts`. A custom in-memory transport prevented outbound Sentry delivery. - All three new header-redaction cases failed before the policy fix and passed afterward. - The dedicated CI command passed locally with `PAPERCLIP_REQUIRE_SENTRY_TEST_SDK=1` and the audited SDK available through `NODE_PATH`. - Server TypeScript check passed with a scratch configuration that resolves this checkout's workspace packages. The existing dependency links point to another checkout. - `node scripts/check-module-boundaries.mjs` and `git diff --check` passed. - Gitleaks and a separate private-data scan passed before push. - Full local workspace typecheck, test, and build were not run. The machine has less than 2 GiB free and those commands include Rust builds. Full PR CI must pass before merge. - The dedicated real-SDK GitHub check passed with 1 test executed and no skips: https://github.com/paperclipai/paperclip/actions/runs/35774449002 - Greptile reviewed `c9db03bcab` at 5/5. Its only thread is resolved. Full PR CI passed on that same head: https://github.com/paperclipai/paperclip/actions/runs/35774449020 ## Risks Small change to error attribution. Unrelated errors may now form their correct Sentry groups instead of reopening a prior run group. No errors are filtered or suppressed. No tracing is enabled and no new event fields are added. No schema or runtime-execution changes. HTTP logs retain their request and status diagnostics while masking the capability value. The new SDK job has read-only permissions, no secrets, and an in-memory Sentry transport. ## Model Used OpenAI GPT-6 (Codex), with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-22 12:53:31 -07:00
DottaandPaperclip	f589660ec0	feat(routines): add safe webhook setup and in-routine run management (#13637 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines turn scheduled work and external events into tasks for an assigned agent. > - Webhook setup was disabled, and actor authentication rejected valid webhook bearer keys. > - Operators need to connect and test a sending app before events can start work. > - This pull request adds a guided setup with durable connection tests that cannot dispatch a task. > - It keeps trigger management, execution tasks, and activity within the routine. > - The benefit is a webhook that can be configured, verified, and operated from one place. ## Linked Issues or Issue Description Fixes #11937. Related: #13216 adds provider-specific Sentry support. This PR addresses general routine setup and ingress. #6841 addresses legacy secret bindings; this PR retains the existing secret service. Current behavior Webhook creation is disabled. Bearer deliveries can fail in agent authentication before the routine checks its key. Setup has no safe connection test. Runs and Activity send the operator away from the routine. Proposed behavior Choose a schedule or a webhook. Follow the setup steps, copy credentials or complete agent instructions, and test delivery without creating work. Finish setup to allow future events to start tasks. Edit or remove compact trigger cards, undo removal, and inspect tasks and activity inside the routine. Reason and benefit An operator can verify credentials and delivery before enabling automatic work. Durable setup state survives refreshes and restarts. Retry receipts prevent an old test event from starting work after activation. ## What Changed - Add a production trigger wizard using reusable Slack setup navigation and footer components. - Add schedule and webhook choices, one-time credentials, agent instructions, and live connection feedback. - Persist pending setup, test delivery receipts, connection status, and reversible trigger removal. - Keep setup checks free of routine runs, tasks, and agent wakeups. Preserve delivery idempotency after activation. - Add compact trigger cards, inline editing, key rotation, pause controls, removal, and Undo. - Keep Runs and Activity in the routine. Use the shared task list and compact activity rows. - Permit only exact public delivery POSTs through actor authentication. Retain webhook authentication, JSON-object validation, and log redaction. - Add production-backed Storybook states and focused server, database, and UI coverage. - Document signing modes, setup checks, retries, rotation, HTTPS ingress, and navigation. ## Verification - Full workspace typecheck, build, and token gates passed on the rebased branch. Storybook also builds. - Focused routine, middleware, logging, shared wizard, and UI coverage passes on the rebased branch: 195 tests across 14 files. The migration passed on a fresh PostgreSQL database and on two repeated applications. - Browser testing used the real app, database, and a deterministic process worker through Tailscale HTTPS and the current Cloud proxy code. - Verified rejected keys, safe setup deliveries, persisted state after restart, activation, retry deduplication, key rotation, schedule editing, removal, and Undo. - Fresh bearer and GitHub-signed deliveries created tasks that the worker checked out and completed. Runs and Activity stayed within the routine. - Current Cloud ingress tests passed. Public delivery POSTs passed through without a browser session; management routes remained gated. - All 54 current-head PR checks pass, including general and serialized tests, all eight browser E2E shards, typecheck, build, runner checks, security checks, and the canary dry run. Two optional Storybook jobs are skipped by workflow conditions. - Greptile is 5/5 on commit `7ea63a61e`, with no unresolved review threads. The stale connection-status finding is fixed and covered by a regression test. - No production deployment was performed. ## Risks - Migration 0281 adds three trigger columns and a test-receipt table. It is additive and safe to reapply. Apply it before running the new server. Existing triggers remain live by default. - Requests without delivery IDs are new events after activation. Senders must reuse an event's delivery ID for retries. - Completed webhooks keep normal dispatch behavior. Their management connection check can start work; the UI states this. - Removing a trigger archives it. Undo restores the URL and credentials. Permanent deletion remains available through the existing API. - Public ingress must remain restricted to the delivery POST route. The tenant verifies credentials. Cloud sleeping-stack behavior is unchanged. - Shared setup components also serve Slack. Existing setup contracts and navigation tests cover that integration. - Senders must use application/json with an object. Other media types receive 415. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, shell execution, and browser testing. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-19 10:49:05 -05:00
DottaandPaperclip	889947c238	feat: add experimental native chat connectors (#13038 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People also ask agents for work in their existing chat tools. > - Each external conversation needs one task and a current authorized source. > - Retries, Stop, and provider failures must not duplicate work or expose private data. > - The first chat PR establishes the opt-in provider and data contracts. > - This PR adds experimental channel integration and its durable control plane. > - Users can request work from connected channels and inspect delivery in Paperclip. ## Linked Issues or Issue Description Refs #13100 and #13092. This is the second of exactly two chat PRs. Foundation #13100 is merged and changed 143 files. Runner prerequisite #13092 is also merged. This PR changes 400 files against master, below the 500-file review limit. It contains no wireframe images or HTML galleries. ## What Changed - Add native Slack, GitHub, Microsoft Teams, Telegram, and Discord chat connections. Keep chat disabled unless the operator enables experimental chat connectors. Preserve the production GitHub tool connection and its normal setup path. - Bind each provider bot identity to one immutable Paperclip agent. Bind each admitted external conversation to one task. Paperclip owns tasks, runs, permissions, and audit records. - Add durable admission, per-conversation queues, questions, task controls, progress, final replies, images, files, and delivery receipts. Board comments remain internal unless explicitly sent to the channel. - Check current identity, provider reach, resource access, credentials, runtime generation, and exact source before provider effects. Keep private responses private. Never send raw reasoning, private logs, credentials, or tool arguments. - Hold uncertain sends for explicit audited resolution. Make Board Send-to-channel atomic and idempotent. Keep reconnect and setup credentials in Paperclip secret storage. - Preserve current native-runner authority across retries, lost acknowledgements, and recovery. Keep immutable input and completion contracts separate from newer user input. Receipt reconciliation cannot launch a provider. - Reconcile chat close/new ordering and provider-effect lock order. Audit resource access changes in the same transaction. Submit only the selected resource from each UI toggle so stale pages cannot undo unrelated access changes. - Drain Codex stdout before certifying process exit. Bound the drain with the existing shutdown grace. Preserve observed terminal authority without treating an undrained process as successful or reusable. - Incorporate master `018ca5da` with its ACP Stop, mobile task layout, runner packaging, and official lock changes. Preserve dedicated chat-answer continuations in both directions when ordinary queued comments are adopted after Stop. - Fence late adapter readiness behind an earlier Stop for the same run. Preserve verified cleanup for registered adapters. Handle single Stop, agent pause, duplicate Stops, and failure release without creating a false cancellation receipt. - Incorporate master's `6dd48cad4` wake-queue extraction. Preserve exact failed-chat retry authorization and lineage, retired question-source suppression, and the block on generic recovery that would discard the admitted source. Fresh deferred input retains its separate promotion path. - Incorporate master `2a05b5ed3` and its queue-admission extraction, simplified transaction ports, and separate runner CI job. Preserve exact durable receipts, actor separation, and dedicated-answer isolation through the new module. A failed receipt insert rolls back the accompanying deferred-wake merge. ## Verification Current head: `afe19299d06253cb628eb398e91d1200ea9f412a`, incorporating master `2a05b5ed3457ea33efd6895520447d1d97fe98d8`. The conflicts are resolved. This successor fixes two test-harness boundaries exposed by CI: per-case route-module preparation and actual durable-save completion before intentional runner termination. Production code and all existing test/turn deadlines are unchanged. [Exact-head Greptile review](https://github.com/paperclipai/paperclip/pull/13038#issuecomment-5587250594) is 5/5, completed September 10 at 13:20:55 UTC, with no actionable findings or open review threads. [Fresh exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/34481724341) passes all 24 jobs, including Build and both required aggregates. Normal exact-head guarded merge was attempted and rejected by the remaining branch approval policy: CODEOWNER review is required and no human approval is present. Normal squash auto-merge is enabled as of September 10 at 13:36:26 UTC. Requested CODEOWNERS have been notified; no approval bypass or self-approval was used. Earlier-head results below remain historical evidence, not qualification of this successor. - Final exact-head Linux evidence: 995/995 chat integration cases; 36/36 agent-skills routes; 35/35 runner live-session cases, including real process kill/resume; 1948 runner Vitest cases with three existing benchmark/platform guards; 870/870 API-authority cases; and 104 browser cases with four existing optional skips. Rust, conformance/replay, full repository build, typecheck, canary, all server/workspace shards, and both required aggregates pass with normal CI concurrency. Earlier failed attempts remain recorded below. - Latest test-only qualification: 141/141 route/permissions/authentication cases pass in separate cold forks, with plain server types and independent review clear. The real-runner suite passes 35/35, with plain runner types and independent review clear. A controlled premature-save acknowledgement fails as expected; matching ownership/effect/process evidence, rejected saves, real turn outcome, test abort, and pre-kill liveness are covered. No local reproduction of the original CI scheduling failure is claimed. The preceding [CI run](https://github.com/paperclipai/paperclip/actions/runs/34479680858) passes 21/24 jobs, including all 995 Linux chat cases and browser aggregate (104 passed, four existing optional skips); only Build, the skills serialized shard, and the required verification aggregate fail. Its exact-head Greptile review was 5/5. Both failed job logs are retained. - Final fixture qualification: all eight focused Discord cases and all 995 chat integration cases pass. The exact modal statement/PID is observed before taking the real connection lock; the test then proves its actual blocking relationship before mutation. Original SQL execution, provider behavior, negative assertions, and 1s/15s timeouts remain unchanged. Independent review is clear and test/production hashes remain frozen. The preceding [CI attempt](https://github.com/paperclipai/paperclip/actions/runs/34477184777) passed 22 jobs, including Build/runner, typecheck, canary, all other test shards, and browser aggregate (104 passed, four existing optional skips); the two fixture failures and failed verification aggregate remain recorded, not relabeled as a pass. - Current queue-module composition: 308/308 recovery/batching/queue/Stop tests; 995/995 full chat integration; 89/89 module tests, including real PostgreSQL receipt-insert rollback; 24/24 workflow/module-boundary tests; plain server and UI types. All four actual local process/ACP browser paths pass in 1.4 minutes. Fresh databases, no skips or retries, stable reviewed source hashes. The initial boundary failure is retained; its no-op service wrapper was removed without changing recovery context or weakening the check. An exploratory standalone test-directory typecheck fails because its new upstream transformation config is not a standalone typechecking project; standard CI/build does not invoke it, and no configuration was weakened to suppress those diagnostics. - The preceding head `e02a63d462ce5d47433b0aeb632bb6fd20aab1ba` passed [all 24 CI jobs](https://github.com/paperclipai/paperclip/actions/runs/34436462958) and exact-head Greptile review at 5/5. Required CODEOWNER review prevented its normal merge before master advanced again. - Final extracted-module composition: 307/307 recovery, batching, queue and Stop-control tests; 995/995 full chat integration; 49/49 module tests including eight PostgreSQL adapter cases; and 19/19 issue-update tests. Plain server types pass. All four actual local process/ACP browser paths pass in 1.3 minutes. Fresh databases, no skips or retries in these cohorts, frozen source hashes, and independent review clear. - The preceding head `3e4e1c1c` passes [all PR CI jobs](https://github.com/paperclipai/paperclip/actions/runs/34415826820), including Build and required `ci / verify` and `ci / e2e`. Both the original Rust failure and the previously load-sensitive lineage fixture pass with unchanged Linux concurrency. Master advanced afterward and required this reconciliation. - Final master composition: 448/448 focused UI tests, 186/186 adapter tests, 24/24 queue/control tests, and 11/11 packaging tests. Plain UI, server, shared, and adapter types pass. Token gates and diff checks pass. Independent server and UI reviews are clear. - Stop-registration regression: both real-service cases fail against exact `a95` source and pass with the fix. The full corrected recovery/control suite passes 265/265. Duplicate-owner and failed-Stop controls also pass. Plain server types pass. The readiness barrier prevents provider startup without adding an acknowledgment to an already terminal run. - Final qualification strengthens terminal-field equality and repeats both affected cases successfully on a fresh database. All four actual local process/ACP browser paths pass again in 1.3 minutes, without skips or retries. The final screenshot shows Cancelled, a paused subtree, retained input, and no error toast. - Two new actual-service regressions fail before the merge fix. They prove that queued-comment adoption could consume a dedicated chat answer or add unrelated input to that answer. The fixed four-case cohort passes, including ordinary upstream continuation and adapter Stop controls. Full recovery passes 257/257. All four actual local process/ACP Stop browser flows pass in 1.4 minutes, without skips or retries, on a fresh database. - The unchanged runner artifact was qualified with 171/171 transport tests, 870/870 API-authority tests, conformance 1/1, and replay 11/11. Six controlled reader tests prove the exit/drain repair. Its local serial Rust workspace passed 546 top-level cases plus two invoked helpers; the later passing Linux CI supplies default-concurrency evidence. - Prior exact-source full chat integration passes 995/995. Settings regressions cover concurrent stale pages, 501 destinations, pending state, rejected updates, and explicit retry. These deterministic tests do not prove live provider behavior. - Retained failed attempts and their causes are in the [qualification log](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-chat-queue-and-webhook-repair.md). The first merge adapter run timed out while macOS slept for 290 seconds. Its unchanged repeat passed with a temporary sleep guard. No assertion, deadline, or CI gate was weakened. Review commands include `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-process-recovery.test.ts src/__tests__/issue-queued-comments-routes.test.ts` and `pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/acp-stop-continuation.spec.ts`. Database suites require fresh disposable databases. See the [browser runbook](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-04-chat-adapters-browser-e2e-runbook.md) for provider setup and separate live acceptance steps. ## Risks - This remains experimental. Deterministic tests and bounded live evidence do not establish every provider feature, tenant, permission layout, or media shape. Teams work-tenant qualification is still open. - Failed and uncertain provider effects remain visible and can require operator action. A transport receipt does not prove recipient visibility. - Native controller and runner artifacts must remain compatible. Preserve lease ownership, terminal authority, source binding, and quarantine during future changes. - Access and audit rows commit together, but activity notifications remain best-effort. This is not a new durable event outbox. - The PR operation does not deploy a live server, replace its runner, or change provider permissions. Remaining live qualification is documented in the [temporary handoff](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-open-qualification-followups.md). ## Model Used OpenAI Codex assisted with implementation, tool execution, testing, and review. The work records `gpt-6-astra` assistance. The environment does not report a context-window size. No private reasoning traces are included. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-10 10:06:45 -05:00
DottaandPaperclip	ebaeba40ee	feat: simplify agent onboarding and configuration (#13011 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators create agents and configure their runtimes in the board UI. > - The old creation flow presents several choices and a large form before an agent can start. > - The existing onboarding controls already provide clear provider connection steps. > - This pull request uses those controls in a new-agent wizard and organizes the full configuration pages. > - Operators can connect, test, save, and assign a first task while keeping the existing configuration tools. ## Linked Issues or Issue Description Related: #10974. That earlier open PR also reorganizes agent configuration. This PR follows the reviewed Storybook designs for agent creation and the current configuration tabs. What existing behavior does this improve? Agent creation, provider connection, runtime tests, and full agent configuration. Current behavior The creation dialog leads to a large manual configuration form. Provider login controls differ from onboarding. Environment variables and secret access appear in separate places. Proposed behavior Choose a name and adapter. Connect Claude or Codex through the existing onboarding controls. Configure and test the runtime, save the agent, and open a task dialog with that agent assigned. Use the same design on the existing configuration tabs. Reason and benefit The first setup asks for fewer decisions. The full editor keeps instructions, skills, runtime controls, secret access, permissions, keys, and revisions available in clear sections. Breaking changes The board creation and configuration layouts change. The test-environment API adds an optional, allowlisted `testCredentials` field for one-shot probes. Database contracts stay the same. Native ACPX tests now reject unsupported local platforms before a CLI login can mask the runtime restriction. ## What Changed - Added a new-agent wizard with numbered steps, adapter branding, provider connections, editable model choices, runtime tests, and confirmation. - Added Codex app-server, Claude ACPX, and OpenCode runner choices. - Stored API credentials through existing secret APIs and persisted references in agent configuration. New setup keys are isolated from credentials used by existing agents. - Preserved external-agent invitations beside the wizard, including optional messages, one-time prompts, and clipboard fallback. - Added OpenRouter provider and secret bindings for Pi and OpenCode. - Added adapter-specific prerequisite fields for Cursor, Gemini, Kimi, and Hermes. Cursor Cloud keys are saved as new organization secrets. - Fixed Cursor Cloud repository field mapping, omitted empty remote environment values, and added useful model and repository error messages. - Preserved complete MCP assignments when multiple valid profiles contain more than 250 tools in total. Generated profiles retain exact tool selectors. - Added service branding and deployment-aware adapter choices. Cloud setup offers Claude, Codex, and OpenCode; local native runners require the experimental setting. - Made the agent list responsive at intermediate widths. - Applied the reviewed design to the real agent configuration pages. Kept the instruction editor, skills, and existing mutations. - Combined secret access and environment variables under one Save and Discard action. - Added interactive Storybook screens for setup, configuration, confirmation, authentication, and test results. - Fixed Pi provider-error parsing and thinking-effort persistence. Native ACPX validates Linux x64 on the actual local, SSH, or sandbox target. - Redacted the complete transient probe-credential field from HTTP error logs, including rejected provider names. ## Verification - Current head `df0292fe6` has a fresh Greptile 5/5 review with no unresolved findings. All 31 executed CI checks passed, including the aggregate verification gate and all browser E2E shards. Storybook visual regression is skipped by its workflow; the local Storybook build passed. - Browser tests completed real assigned tasks with direct Codex, Claude, OpenCode, Pi, and native Codex. - Verified external-agent invitation generation and automatic prompt copying in the live browser. - Pi and OpenCode used an existing OpenRouter secret. Browser checks covered save and reload, instruction edits, skill selection, environment-variable Save and Discard, and assigned task creation. - Invalid Claude API credentials remained on the connection step with an error. A live Pi/OpenRouter invalid-key probe returned a provider failure and left the user-secret inventory unchanged (zero entries before and after). - Full workspace typecheck and build passed after rebasing onto current master. After review fixes, server and UI typechecks, token gates, and the full build passed again. Storybook built successfully. - All 5,542 local UI tests passed. The Cursor Cloud and Pi adapter regressions passed all 24 tests. Review regressions passed 69 server tests and all 18 agent-list tests. - The local full test command ran 6,971 general server tests successfully. Editing review fixes during that long run caused nine tests to use stale modules; fresh isolated runs passed. An unrelated embedded-Postgres fixture hit the host shared-memory limit; its 15 affected tests passed when the fixture groups ran separately. - Local workspace groups passed after rerunning 18 CLI tests sequentially to avoid host database limits and parallel-load timeouts. The local full command stopped at the general server phase, so serialized server verification comes from the five passing CI shards. - Browser testing at 390px confirmed that the agent action menu opens and the page has no horizontal overflow. CI browser E2E shards passed. - Review the `Onboarding / New agent` and `Agents / Configuration refresh` Storybook groups. In the real app, create an agent, run its connection test, save it, assign a task, and reload its configuration. ## Risks - This changes the main agent setup and configuration UI. Regression tests cover routing, persistence, secret bindings, and form actions. - Native Claude ACPX requires Linux x64. Direct Claude works on macOS. Remote checks execute a bounded platform probe and reject unsupported or unverified targets. - A native OpenCode task reached the provider context limit because of its tool payload. Its provider connection test passed. Direct OpenCode completed a task. This existing native execution limit is not fixed here. - Claude and Codex connection keys use the existing user-secret store. Other runtime setup keys use distinct organization secrets. Existing credentials are never rotated. Probes do not store entered keys. Failed agent creation removes newly staged credentials. - Cursor Cloud has not completed a live task. Its authenticated account still needs GitHub repository access. The live run passed MCP provisioning, remote environment validation, and explicit Auto model selection before the repository prerequisite blocked execution. - Generated runtime MCP profiles can exceed the public profile-edit request limit. They still contain exact catalog selectors and preserve permission boundaries. - No database migration, dependency, lockfile, or workflow changes are included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, shell execution, and browser automation. The runtime did not expose the exact model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-08 14:22:32 -05:00
DottaandPaperclip	9dd6526b47	fix(security): harden privileged server boundaries (#12776 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server controls secrets, host files, outbound requests, and workspace commands > - A red-team review found cases where restricted callers could cross these trust boundaries > - These cases could expose credentials or let untrusted input reach privileged resources > - This pull request applies least-privilege checks at each affected server boundary > - The benefit is safer agent execution without changing the private-instance bootstrap contract ## Linked Issues or Issue Description What happened? Several server paths used authorization, redaction, or content-delivery rules that were too broad. Restricted agent keys could obtain company-level operational data. Some adapter and instruction paths could reach server-owned network or file resources without the required owner approval. Expected behavior Paperclip must redact credential values, enforce restricted-key scopes, guard outbound network access, prevent same-origin script execution, and reserve host-level file and command controls for authorized operators. Steps to reproduce 1. Configure an authenticated development instance at the parent commit. 2. Exercise the affected APIs with a restricted agent key or a non-instance-admin company user. 3. Observe that the parent commit returns privileged data or accepts a privileged operation. 4. Repeat on this branch and observe a redacted response, a safe download, or an HTTP 403 response. Paperclip version or commit The findings reproduce from commit `39898ab22` and are fixed by this pull request. Deployment mode Authenticated self-hosted server and local development modes. Installation method Built from source with pnpm. ## What Changed - Redact generic secret `value` and `token` fields recursively in structured logs. - Classify exact and separator-suffixed `KEY` environment names as secrets in company exports. - Limit restricted self-identity responses and protect company run, log, and secret catalog APIs. - Route HTTP adapter requests through DNS-pinned SSRF protection with exact private-origin allowlisting. - Download HTML, SVG, and other script-capable assets with `nosniff` and a sandbox CSP. - Require instance-admin access for external instruction roots and exports that read them. - Block agent-authenticated host command persistence across supported workspace runtime shapes. - Apply the central runtime-management decision before workspace command controls. - Keep the documented first-user instance-admin claim contract unchanged. - Add regression tests and server-owner configuration documentation. ## Verification - `pnpm -r typecheck` passes. - The Node 24 remediation suite passes with 365 tests. It skips 25 environment-gated tests. - `pnpm build` passes under Node 24. - `git diff --check` passes. - The full local runner reaches known macOS-only general-server harness failures before the serialized route lane. The Linux PR matrix is the authoritative full-suite gate. ## Risks - Restricted agent keys now receive HTTP 403 responses from company-wide run, log, and secret catalog endpoints. - Script-capable assets now download instead of rendering inline. - External instruction roots now require instance-admin access. - Private HTTP adapter endpoints now require an exact origin in `PAPERCLIP_HTTP_ADAPTER_PRIVATE_ENDPOINT_ALLOWLIST`. - Public HTTP adapter endpoints remain enabled. Redirects and metadata or link-local targets remain blocked. - No database migration is required. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5. The exact serving snapshot and context-window size are not exposed. The model used tool-enabled reasoning, repository access, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-03 14:15:32 -05:00
DottaandPaperclip	cabc9146d0	feat(apps): add secure remote MCP and PostHog setup (#12339 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps give those agents governed access to external tools. > - Remote MCP setup needs secure endpoint validation and durable credentials. > - PostHog needs both browser sign-in and personal API key setup paths. > - This pull request adds the shared remote MCP foundation and the PostHog definition. > - The benefit is a secure and reusable base for later app connection work. ## Linked Issues or Issue Description Refs #11965 This is stack 1 of 11. It replaces the first reviewable part of #11965. ## What Changed - Add guarded remote MCP setup and credential handling. - Add PostHog OAuth and API key connection methods. - Add focused server, shared contract, and UI coverage. - Keep the migration replay-safe and idempotent. - Give the late-close security regression the same 10-second CI headroom as the adjacent real-timer handshake test. - Synchronize fake-timer handshake tests at the exact ensure-session boundary so real filesystem setup cannot race the fake deadline. - Drive PTY overflow coverage only after listener registration so scheduling cannot reorder the test fixture. ## Verification - pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts server/src/__tests__/plugin-worker-manager.test.ts (220 passed; affected cases also passed five focused stress repetitions) - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never leaks a sandbox-provided value from a late close rejection into logs or the result"` (1 passed) - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never promotes a late ensureSession resolution\|closes a late-resolving real handle exactly once"` (2 passed) - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - `pnpm --filter @paperclipai/db check:migrations` - `pnpm build` ## Risks - Remote endpoint validation can reject configurations that previously passed without checks. - OAuth configuration errors can block setup until the operator corrects the provider settings. - The migration uses guarded statements so repeated execution is safe. - The test-only synchronization changes do not affect runtime behavior; they remove filesystem/fake-clock and listener-registration races observed under parallel CI load. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-29 12:08:32 -05:00
NoSoloSoftandAndrew Aymeloglu	156830006b	fix: redact HTTP cookies from server logs (#7977 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators rely on Paperclip server logs for maintenance, incident triage, and support handoffs. > - The HTTP logger persisted request metadata and only redacted authorization headers. > - Request cookies and set-cookie headers can contain active session material and should not be written to durable logs. > - This pull request keeps the fix intentionally narrow: centralize the HTTP log redaction path list and include cookie-bearing headers. > - The benefit is lower credential/session leakage risk from routine server.log collection or sharing. ## Linked Issues or Issue Description No GitHub issue exists for this exact local finding. Inline bug report: - Type: security/privacy bug. - Affected area: server HTTP logging middleware. - Observed problem: local Paperclip maintenance found raw cookies present in server.log. - Expected behavior: durable HTTP logs redact authorization and cookie-bearing request/response headers. - Impact: anyone with access to copied/exported logs could see session-bearing cookie values. - Related/open PRs found during dedup search: #7242, #7306, #7346. This PR is the minimal local fix branch created from the verified local maintenance patch; those PRs may be better upstream candidates if maintainers prefer their broader coverage. ## What Changed - Added `HTTP_LOG_REDACT_PATHS` for HTTP logger redaction paths. - Kept existing `req.headers.authorization` redaction. - Added redaction for `req.headers.cookie`, request `set-cookie`, and response `set-cookie` paths. - Added focused tests asserting the required redaction paths are present and that pino-http output redacts live request/response header secrets. ## Verification - `pnpm exec vitest run server/src/__tests__/http-log-redaction.test.ts` - `pnpm --filter @paperclipai/server typecheck` - Pre-commit TruffleHog scan: 0 verified/unverified secrets. - PR CI observed passing so far for policy, Typecheck + Release Registry, Build, e2e, Socket, Snyk, security-review, and serialized/workspace suites; remaining jobs may still be running. ## Risks - Low runtime risk: this only expands pino redaction paths. - Possible coverage risk: broader redaction helpers in related PRs may cover more serialized variants beyond the pino-http request/response header pipeline tested here. - No migrations, schema changes, or UI changes. ## Model Used - OpenAI Codex via Hermes Agent, model gpt-5.5, tool-using coding/ops session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>	2026-07-20 17:17:08 -05:00