mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
f467a239f0b9d3df48596a77bb29496bae734e07
2286
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
66424362a8 |
perf(runtime): skip discovery without a Browser Use connection (#16345)
Skip full tool discovery when no enabled, active Browser Use connection exists for the run company. Configured connections retain existing authorization and discovery.
Verified exact head
|
||
|
DottaandPaperclip
|
c41e00bca4 |
fix(server): index workspace source task enumeration (#16342)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent startup checks access to every source retained by a workspace. > - Repository admission repeats these checks while it holds a company lock. > - PR #16334 indexed the workspace privacy query. The next source-task query still cast the task primary key to text. > - That second query scanned company history for every workspace and delayed other agent launches. > - This pull request keeps the task column indexed and guards the retained identifiers before conversion. > - The benefit is faster startup with the same source validation and access decisions. ## Linked Issues or Issue Description Refs #16334. **What happened?** Agent launches still spent minutes preparing repositories after the first authorization query fix. The source-task lookup scanned 14,191 task rows for one matching source. **Expected behavior** Source validation should use the task primary key and workspace-binding index. It must still reject missing, malformed, noncanonical, and foreign-company sources. **Steps to reproduce** 1. Seed company task history and a workspace with attached and retained source tasks. 2. Call `assertTaskWorkspaceAccess` and capture its source-task SQL. 3. Run `EXPLAIN (ANALYZE, FORMAT JSON)` on that query. The baseline scans unrelated task history. ## What Changed - Convert retained text identifiers with the existing canonical UUID guard. Keep the indexed task column as a UUID. - Exercise the real source-access service against PostgreSQL. Verify complete source results, malformed and uppercase input behavior, and rows examined by the plan. - Document the second source-validation query and its regression coverage. ## Verification - Red: the new regression failed because the source lookup examined 1,202 rows for two sources. - Green: four focused workspace suites passed, 147 tests total. The final regression also checks malformed and uppercase query inputs. - Live read-only comparison: two warmups, then five samples per variant in alternating order. Median client query time changed from 4.88 ms to 0.35 ms. Complete result rows matched. Shared buffer hits changed from 3,984 to 5. These measurements cover this SQL statement, not total agent startup. - The complete source-access helper also passed for the same retained reader in read-only transactions. Its median changed from 15.75 ms to 10.75 ms; repository admission still repeats other checks. - `pnpm -r typecheck` and `pnpm build` passed locally. The staged Linux build and all seven final regression cases passed on Node 24.20.0. - All 55 current-head checks/statuses passed; two Storybook checks were skipped. Greptile reviewed `6cfb755` at 5/5 with no findings. - The full local test run encountered a tracing-test failure in `native-session-executor.test.ts`. Both that test alone and the complete file passed on rerun (698 tests). The remaining full-suite run is still in progress. ## Risks - This changes SQL planning on an authorization path. The canonical guard preserves the former exact lowercase text match and avoids UUID parse errors. - Company filters, missing-source checks, project checks, and per-source access decisions remain in place. - No schema migration or new index is required. Repeated checks across large repository histories remain a separate source of startup work. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, shell tools, code execution, and browser verification. The exact model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
90a45ef254 |
perf(skills): skip inventory work for empty runtime selections (#16343)
Return before company skill inventory work for a known empty native or stored-key selection. Preserve nonempty stored-key precedence, current access/version checks and legacy automatic skills.
Verified head
|
||
|
Devin FoleyandPaperclip
|
cda77edc60 |
perf(server): stop readiness waiting on startup housekeeping (#16329)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On restart, `/api/health` reports `starting` until startup recovery finishes, and deploy tooling waits for `ok` > - After heartbeat recovery, readiness also waits for housekeeping that every scheduler tick repeats and that serving requests does not need > - #16330 added OpenTelemetry spans for the main startup phases; native recovery and orphan reaping are the slow ones, and both run the native finalization reconciliation pass, which has no inner breakdown > - This pull request stops readiness waiting on that housekeeping, and records the phases of slow reconciliation passes as spans > - The benefit is faster readiness on every restart, and traces that show where a slow reconciliation pass spends its time ## Linked Issues or Issue Description Builds on #16330, which records `controller.startup.<phase>` spans. Related to #16328. **What existing behavior does this improve?** Server readiness time after a restart, and diagnosis of slow native recovery. **Current behavior** - After heartbeat recovery, readiness waits for the custom-image setup cleanup, tool connection health (remote MCP checks), decision expiry, the adapter and setup-token login reapers, environment lease cleanup (provider calls) and the retention sweep. Each also runs on every scheduler tick. On one instance they added 11–28 s to every measured restart. A throw in any of them fails startup. - On the same instance, native recovery took from 33 s to 11.5 min and orphan reaping from 30 s to 4 min across restarts in one hour. Both run the native finalization reconciliation pass. The #16330 spans time each step as a whole, but not the phases inside that pass. **Proposed behavior** - The housekeeping steps run after readiness. The scheduler interval still starts after them. - With an OTLP endpoint, a reconciliation pass of at least one second records a span with one child span per phase. **Reason and benefit** Readiness depends only on what serving requests and run recovery need. Slow reconciliation passes become attributable to a phase in the existing traces, without adding trace volume for the periodic pass. **Breaking changes** None. A failed housekeeping step no longer fails startup; it is logged and the next step runs. ## What Changed - `server/src/index.ts`: moves the housekeeping steps into `runStartupHousekeeping`, which runs after readiness under `trackHeartbeatSchedulerWork` (idle sleep and shutdown wait for it). Each step catches and logs its own failure (`startup housekeeping step failed`). The heartbeat scheduler interval starts when housekeeping finishes, unless the scheduler was stopped, so the reapers still finish before the first timer tick. - New `server/src/services/native-runtime/native-reconciliation-timing.ts`: records the phase boundaries of one pass. When the pass took at least one second and an OTLP endpoint is configured, it emits a `controller.native_reconciliation` root span (`paperclip.native-reconciliation`) with child spans `legacy_workspace_exports`, `obsolete_policy_reviews`, `automatic_completion_reviews`, `candidate_scan` and `candidates`, using the recorded start and end times. It records `scoped` and the candidate count only. A failed pass marks the pass span and the running phase. Tracing failures are swallowed. - `native-finalization-reconciler.ts`: the exported function wraps the pass in that timing and finishes it in `finally`, so failed passes are recorded; the original error is rethrown. - `doc/execution-semantics.md`: what readiness waits for, and the housekeeping order. `doc/observability.md`: the reconciliation spans. ## Verification - `pnpm exec vitest run src/__tests__/server-startup-feedback-export.test.ts src/services/native-runtime/native-reconciliation-timing.test.ts src/services/native-runtime/native-finalization-reconciler-timing.test.ts src/startup-recovery-timing.test.ts` (in `server/`): 52 passed. New tests: - Readiness is `ready` while a housekeeping step hangs, and the heartbeat interval starts only after the step settles. - A failed housekeeping step is logged, and the scheduler still starts after the remaining steps. - Span timing: nothing for a fast pass or without an endpoint; a slow pass emits the pass span and one child per phase with the recorded times; a failed pass marks the running phase and the pass; a failing tracer does not throw. - The reconciler records its phases and a failed outcome when a pass throws, and rethrows. - Three existing startup tests settle housekeeping before they read the scheduler interval or the setup cleanup. - `pnpm exec vitest run src/services/native-runtime`: 4,399 passed. - `pnpm typecheck` in `server/`: passed. ## Risks - Housekeeping now runs concurrently with the first requests and with `waitForExternalAdapters`. None of these steps is needed to serve requests, and each one also runs on every scheduler tick. - Readiness is reported earlier; the heartbeat timer starts at the same point relative to housekeeping as before. - The new spans exist only with an OTLP endpoint and only for passes of at least one second. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aa04c5c2a7 |
fix(runtime): keep the copyback lock transaction active (#16340)
Keep the dedicated workspace-copyback transaction active with bounded SELECT 1 probes while filesystem work runs. Preserve existing connection, owner-token, takeover and publication fences.
Verified head
|
||
|
DottaandPaperclip
|
cc96c9f832 |
fix(server): probe libc asynchronously before serving requests (#16335)
Linux runner qualification generated a full Node diagnostic report to read one libc version field. Reverse DNS on open sockets could stop the event loop for minutes. Replace report generation with a one-time asynchronous getconf probe during module initialization. Bound it to one second and 1 KiB of output, cache the measured facts, and preserve unknown-libc failure behavior. Verification: 149 focused tests, repository typecheck and build, full CI, and fresh Greptile 5/5 with no comments. A timed-out local heartbeat fixture passed when rerun alone. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
35b3345033 |
fix(server): support issue monitor deadlines in idle sleep reports (#16325)
## Thinking Path > - Paperclip manages work for companies and AI agents. > - Hosted workspaces can sleep when their required work has a safe wake path. > - The current idle check treats every open issue as background work. > - Backlog, unassigned and human-waiting records can therefore keep an idle process running. > - Future automated monitors still need their saved check time and execution claims. > - This PR adds a signed host capability for passive issues and verified monitor deadlines. > - The host can save one alarm for the earliest required routine or monitor time. ## Linked Issues or Issue Description **What happened?** A quiet workspace with a backlog or unassigned todo issue reports background work as present. A future issue monitor also prevents sleep even when the host can save a durable wake alarm. **Expected behavior** Proven passive records can sleep. A validated future monitor can sleep only after the host saves its alarm and validates the same owned report. Active, unknown and unresolved work stays awake. **Steps to reproduce** 1. Create a quiet workspace with an unassigned todo issue and no other work. 2. Acquire a bounded, signed Cloud idle hold. 3. Request the routine-only safety report with `scheduledWake=1`. The issue blocks sleep. 4. Request the new capability with `scheduledWake=2`. A passive issue can now report no work. 5. Add a valid future monitor to an assigned in-progress issue. The second capability returns its saved deadline for host acknowledgement. **Paperclip version or commit** Base: `c30fb7d2b75ea30f72d63656b2a7cfff7fe0929d`. **Deployment mode** Cloud-managed authenticated instance. Local scheduling remains in the existing scheduler. Related protocol: [#16215](https://github.com/paperclipai/paperclip/pull/16215). Open scheduler and monitor PRs were checked for overlap. This change leaves monitor admission, dispatch and quota recovery with their existing owners. The roadmap already includes scheduled work; this is a focused idle-safety extension. ## What Changed - Add signed scheduling capability `2`; capability `1` retains the routine-only rules. - Allow untouched onboarding prompts with idle assignees to sleep using the same awaiting-user predicate as recovery; typing, answering the opening card or a prior run removes the exemption. - Inspect passive issue state and future monitors in the same bounded, read-only snapshot as other work. - Check ownership, company scope, known states, claims, policy/deadline agreement, attempts, timeout and dependency repair. - Combine routine and monitor deadlines into a known proof kind and catalog generation. - Preserve other sleep blockers, including task watchdogs, conversations, live runs, external credentials and recovery. - Update the protected route, OpenAPI contract, documentation and PostgreSQL regression tests. ## Verification - Full workspace typecheck and build passed. Final server typecheck and full build passed after the final guards. - Initial safety, scheduling, protected-route and control-authentication checks: 326 tests passed. - Final safety and heartbeat scheduling checks: 253 tests passed. Includes reporting a future deadline, restoring scheduling from PostgreSQL and one dispatch across competing schedulers. - Local general server checks: 20,260 passed; UI: 8,460 passed; CLI: 534 passed, including isolated reruns of two embedded-PostgreSQL startup failures. All remaining workspace groups and 158 serialized suites passed. The suite was completed in groups after the isolated infrastructure retries; the final onboarding change also passed the focused 674-test recovery/safety run. Earlier unchanged Git fixtures failed under Xcode's system `init.defaultBranch=main`, reproduced on the clean base and passed with a test-scoped fixture default; no source or machine configuration was changed. - Rebased issue, protected-route and scheduler checks: 115 tests passed; server typecheck passed. - The expanded change passed 674 idle-safety, scheduling and recovery regression tests, including six onboarding admission/invalidation cases. Final server typecheck and build passed. - Current-head Apex is 5/5 with no unresolved threads. All 54 current-head checks passed, with two expected skips. Four jobs were interrupted by runner shutdowns or lost communication on their first attempts; their targeted reruns passed. Live hosting qualification awaits the verified master release and matching host capability. ## Risks This intentionally covers a limited set of passive states and valid future monitors. Assigned active tasks without a proved waiting path, unknown states, claimed monitors and quota recovery still block sleep. It does not make every open issue safe to sleep. A matching host must recognize `routines_and_issue_monitors`, acknowledge the durable alarm and revalidate the exact owner, generation and deadline before stopping. The report alone grants no stop authority. Older hosts retain capability `1`. Independent database writers remain outside the admission contract. Existing scheduler claims and execution authority are unchanged. No schema migration is required. Roll back host negotiation to capability `1` to restore conservative admission, while retaining alarm delivery until outstanding alarms have been handled. ## Model Used - OpenAI GPT-6 (Codex), with repository editing, command execution and GitHub tools. The exact deployment ID and context-window metadata are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
aec80cbc8c |
fix(server): keep workspace compaction out of the startup sweep (#16328)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When the server starts, `/api/health` answers "starting" until startup recovery finishes, and deploy tooling waits for "ok" > - Startup recovery waits for the first sweep of the terminal-workspace worker > - #16304 added workspace compaction to that sweep. Compaction walks worktrees, runs Git, and deletes dependency trees, and it re-checks every eligible workspace after a restart > - Compaction is never needed to serve requests, so it should not delay readiness > - This pull request skips compaction in the startup sweep only; later sweeps compact as before > - The benefit is that a compaction backlog cannot make a restart slower ## Linked Issues or Issue Description Follow-up to #16304. **What existing behavior does this improve?** Server startup time after #16304. **Current behavior** `await terminalWorkspaceWorker.ready` blocks the readiness flip until the first terminal-workspace sweep ends. That sweep compacts every eligible workspace in its page. The per-cycle compaction memo is in memory, so after each restart every eligible workspace is walked and checked with Git again, and any directories found are deleted before the sweep ends. **Proposed behavior** The first sweep skips compaction. Pending compactions keep their deadlines, so the worker wakes and a later sweep compacts them. **Reason and benefit** Readiness depends only on work that is needed to serve requests. On one measured instance the first sweep took 5–8 s with compaction; with a larger backlog, compaction could add seconds per workspace. **Breaking changes** None. Compaction happens one sweep later after a restart. ## What Changed - `sweepTerminalWorkspaces(limit, signal, { compact })`: `compact: false` skips compaction for that sweep and keeps each pending compaction deadline. - `registerTerminalWorkspaceWork` passes `compact: false` to its first sweep (the one readiness waits for) and `compact: true` after that. - `server/src/index.ts` forwards the option through the logging wrapper. - `doc/DEVELOPING.md`: the compaction section says the startup sweep never compacts. ## Verification - `pnpm exec vitest run src/__tests__/terminal-workspace-work.test.ts src/__tests__/execution-workspace-compaction.test.ts src/__tests__/workspace-compaction.test.ts` (in `server/`): 35 passed. - New: the worker passes `compact: false` only to the startup sweep and `compact: true` to the next one. - New: a sweep with `compact: false` leaves the files in place, the compaction deadline stays due, and the next sweep compacts. - `pnpm typecheck` in `server/`: passed. ## Risks - Low risk. After a restart, compaction runs a few seconds later than before. Archive behavior does not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ebceb2a744 |
trace(server): time workspace, native setup and startup recovery (#16330)
## Thinking Path > - Paperclip manages agents and their work. > - The controller prepares workspaces before it starts a Runner. > - A slow run can spend 30–80 seconds inside the combined workspace phase. > - Startup recovery can also delay queued work for minutes. > - Native runs also have gaps before Runner startup and after the provider result. > - Existing traces do not identify the awaited steps inside those intervals. > - This change adds closed timing boundaries around the existing operations. > - Operators can locate the wait without changing execution or exposing content. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operator-configured OpenTelemetry diagnosis of controller preparation and startup recovery. Related: #16178, #16236 and #16285 introduced the existing preparation timing and sampling contract. #16306 and #16312 reduce separate configuration and copyback work. **Current behavior** The workspace interval combines source resolution, trust checks, provisioning, database persistence, binding and repository admission. Startup awaits accounting and native recovery before queued work resumes, but neither has a separate duration. Native backend setup and post-turn cleanup also contain unmeasured awaits. CPU profiles and a fast database probe cannot identify which await is slow. **Proposed behavior** Record workspace steps and native controller setup and cleanup with the existing hashed run ID. Record startup phases as independent, operator-sampled root spans without custom attributes. Keep the same operation order, results, failures and authorization. **Reason and benefit** These boundaries distinguish source work, database writes, repository admission and restart recovery. This is a diagnosis change. It does not claim a response-time improvement or a solved latency budget. ## What Changed - Add 14 closed workspace step names around 16 existing awaits. Repeated updates and bindings produce separate spans. - Time startup tool reviews, gateway sessions, delivery-worker readiness, accounting, plugin readiness, native recovery, hot restart, orphan reaping, retries, queued work and goal recovery. - Time native GitHub setup, wake binding, backend construction, connector assignments, MCP discovery, tool authority, companion selection and binary resolution. Time selected post-turn GitHub, coordinator and instruction cleanup awaits. - Move preparation timing to a pure shared module. Keep the existing heartbeat import as an alias. Both heartbeat and native timing skip tracer lookup when the endpoint is unset or blank, including when another library registered a provider. - Tracing lookup, callback, status and end faults cannot repeat work or replace its value or error. - Add real PostgreSQL workspace timing coverage and silent no-op/fault controls. Update the tracing contract and paired run maps. - Keep span attributes closed. Startup spans add none; workspace steps use only the existing run hash. No commands, paths, returned records or exception messages are recorded. ## Verification - Diagnostic-gap baseline: the new real workspace test fails against the unchanged workspace implementation because the expected step spans are absent. It passes with the wrappers. - Final affected coverage: **804 distinct cases pass** across seven complete suites: 698 native-executor cases, 39 real PostgreSQL workspace cases, 61 timing/context/startup cases, and six execution-configuration integration cases. The final native and unit invocations pass with no selected skips. The workspace suite passed in the initial six-file invocation on the same final workspace source. - Preserve the initial six-file attempt as failed: **7 failed and 789 passed**. All seven errors occurred during embedded PostgreSQL startup. The log reports local macOS shared-memory exhaustion. A fresh 15-case database group passes, followed by the complete 698-case native suite. These runs overlap; do not add their counts. - The new native timing assertion fails against unchanged native code because the expected spans are absent. It passes with the wrappers and retains the credential-isolation assertions. - Full `pnpm -r typecheck` and `pnpm build`: passed on the final production source. Earlier nullable-closure errors are fixed with erased assertions inside existing guards. The failed attempts remain recorded. - Normalized AST comparison: removing only timing wrappers, timing imports/local handles and erased assertions restores the original workspace, startup and native executor programs. This checks call order and policy expressions; it is not live acceptance. - The expanded head first failed two configuration integration assertions because their test providers had no endpoint. Configure the endpoint explicitly in those two tracing tests, and restore it after each test. All six cases then pass with the original credential cleanup, redaction and exact span assertions. Production is unchanged by this repair. - The first PR head failed one unchanged attachment test at its 15-second deadline in CI. The complete 32-case attachment suite passes locally with that deadline. Do not label the failed CI attempt green. The expanded head needs fresh full CI. - `node scripts/check-maps.mjs`: passed, 35 areas and 176 capability IDs. `node scripts/storybook-catalog.mjs`: 283 files and 2,728 stories. Structural checks only; no browser, play or Storybook build claim. - `git diff --check`: passed. No added prose comments, lockfile changes or new 1,000-line crossings. - A duplicate full local test run was not started. Fresh exact-head full CI and E2E pass on `15a683bb2419960f611584f8c07cdcfc9ce3af0b`. All 56 check runs are successful or optional skipped. Superagent and legacy Snyk pass. Completed Apex review is 5/5 on this same head, with zero unresolved review threads. Commands: ```sh pnpm exec vitest run --project @paperclipai/server --maxWorkers=1 --fileParallelism=false server/src/services/native-runtime/native-session-executor.test.ts server/src/services/execution-preparation-timing.test.ts server/src/startup-recovery-timing.test.ts server/src/services/heartbeat/preparation-timing.test.ts server/src/services/heartbeat/preparation-timing-context.test.ts server/src/services/heartbeat/workspace-preparation.test.ts pnpm -r typecheck pnpm build node scripts/check-maps.mjs node scripts/storybook-catalog.mjs ``` ## Risks - Configured operators receive more spans. They still follow the configured sampler and exporter. - Startup spans measure the awaited phase, including remaining readiness waits. They do not measure earlier worker construction or a provider turn. - Workspace and native setup intervals overlap their containing phases. Do not add them to the parent duration. Native timing covers named awaits, not every statement or every error path. - These spans are not yet deployed or tested against a live latency run. No first-party Telemetry event, run-log write, schema or API changes are included. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact served variant and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
445feaa7aa |
fix(server): index retained workspace task authorization (#16334)
## Thinking Path > - Paperclip manages AI agents and their work. > - Before an agent starts, repository admission checks access to retained workspace files. > - Admission holds a company provenance lock while it checks each source and reader. > - The retained-task query cast the indexed task ID to text. PostgreSQL could evaluate privacy across company history before it matched one source ID. > - Repeated slow checks kept the lock for hours and blocked other agent launches. > - This change looks up the canonical source UUID through the existing primary key before it checks privacy. > - The query does less work and keeps the same access decisions. ## Linked Issues or Issue Description **What happened?** Nine runs appeared active but had no provider process or log file. Read-only database inspection found a transaction holding the company repository-provenance lock for more than ten hours. Eight other transactions waited for that lock. The owner repeatedly ran workspace authorization queries. The company had about 13,400 tasks and 8,500 workspaces. **Expected behavior** A retained-source check should find the referenced task before it evaluates that task's privacy. Unrelated task history should not multiply the privacy work for every source and reader. **Steps to reproduce** 1. Run the new `workspace-authorization-query.test.ts` against authorization code from `5c7a025032`. 2. The real PostgreSQL fixture has 1,200 unrelated tasks and one retained source. The regression fails with 2,402 recursive privacy traversals, above the bound of 100. The other five access tests pass. 3. Apply this change and run the same test. All six tests pass. Related: Refs #16330 adds workspace timing evidence. This PR fixes a measured SQL plan. It does not change those tracing boundaries. ## What Changed - Compare the UUID task column with a guarded canonical UUID from the retained-source JSON key. - Reuse `canonicalExecutionUuid`. Invalid and noncanonical keys still match nothing and cannot cause a UUID parse error. - Add real PostgreSQL coverage for query work, missing and foreign sources, malformed and uppercase IDs, both agent and sponsor grants, and grant revocation. - Add a reproducible baseline comparison and document its scope in `doc/DATABASE.md`. ## Verification - Red: the unchanged query fails the new work-bound regression. Five access cases pass. - Green: 101 tests pass across the new query suite, workspace binding, and the production privacy review suite. - Full `pnpm -r typecheck` and `pnpm build`: pass. Dependencies use the repository's pinned pnpm 9.15.4. - All current-head CI checks pass, including the full server, chat, workspace, serialized, Runner, and eight E2E test lanes. Build, typecheck, canary, and security checks pass. Head: `ad907f6e14c2d773fa8c20692e5ccc10215c9297`. - Greptile: 5/5 on that head, with no actionable findings or unresolved review threads. The branch is mergeable. - The local full `pnpm test:run` was started, then stopped after a concurrent disposable-database check hit the macOS shared-memory limit. This is not counted as a local full-suite pass. The benchmark passed when rerun alone; the complete suite passed in CI. - `git diff --check`: pass. The branch has no lockfile or workflow changes. Synthetic SQL comparison on PostgreSQL 18.1, with 13,429 tasks and one retained source: | Query | Median | Five measured samples | Recursive privacy traversals | | --- | --- | --- | --- | | Baseline `5c7a025032` | 42.57 ms | 42.57, 47.71, 44.51, 41.29, 41.44 ms | 26,860 | | Fixed | 2.37 ms | 2.76, 2.35, 2.37, 2.11, 3.33 ms | 4 | Both versions return identical complete rows. The benchmark alternates variants after two warmup rounds. It loads the baseline authorization module with current dependencies; the UUID helper is unchanged. The measured gain is about 18x for this SQL fixture. The final committed benchmark also passed a second isolated run: 55.44 ms baseline and 3.26 ms fixed, with identical complete rows (about 17x). Two read-only queries on the affected database also returned identical complete rows. One changed from 129 ms to 7 ms; a workspace with 49 retained sources changed from 576 ms to 11 ms. These are single paired measurements, not endpoint or startup benchmarks. Reproduce: ```sh pnpm exec vitest run server/src/__tests__/workspace-authorization-query.test.ts server/src/__tests__/task-workspace-binding.test.ts server/src/__tests__/privacy-production-review.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-authorization.ts --baseline-ref=5c7a025032 pnpm -r typecheck pnpm test:run pnpm build ``` ## Risks - A direct UUID cast could broaden or break access for malformed keys. The existing canonical guard and regression cases preserve the former text-match behavior. - Planner choices can differ across PostgreSQL versions and data sets. The regression bounds real privacy work rather than elapsed time or one exact plan shape. - Admission still checks every retained source and reader under its existing lock. This removes a measured query cost; it does not claim a bound for the full admission process. - No schema migration, permission change, lock removal, or live deployment is included. Recovery of the affected runs remains unverified until rollout. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact served model variant and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
8ffc5748a2 |
fix(server): stop libc detection from reverse-resolving every socket (#16333)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #15683 made new Codex agents default to Paperclip Runner on Linux, which needs the host's glibc version > - `localAgentRunnerTarget()` reads that version from `process.report.getReport()` > - Unless `excludeNetwork` is set, Node's diagnostic report resolves the endpoints of every open socket with synchronous reverse DNS on the main thread > - `GET /adapters` calls it once per adapter on every request, so on a host whose proxy address has no fast PTR answer, the whole server stops responding > - This pull request excludes network data from the report and detects libc once per process > - The benefit is that runner selection costs a few milliseconds once, instead of blocking the server on every request ## Linked Issues or Issue Description Regression from #15683. **What happened?** After #15683, a server running behind a reverse proxy stopped responding. Requests, database replies, log output and health checks all stalled for minutes. The Node main thread was idle on CPU but blocked in a synchronous DNS read: reverse (PTR) lookups of the proxy's address, each waiting for the resolver timeout. **Expected behavior** Detecting the C library must not block the event loop on DNS, and must not repeat on every request. **Steps to reproduce** 1. Run the server on Linux where the reverse DNS lookup of a connected peer's address is slow (for example, a proxy address whose PTR lookup times out). 2. Open the board, which polls `GET /adapters`. 3. The server stops answering. Each `getReport()` call reverse-resolves every open socket synchronously. Measured on Node 24.21 with one open TCP connection to such a peer: `process.report.getReport()` took **9,774 ms** with network data and **4 ms** with `excludeNetwork`. A busy server has hundreds of open sockets. **Paperclip version/commit** `master` after #15683. ## What Changed - `server/src/lib/agent-runner-platform.ts`: - Generates the report with `process.report.excludeNetwork = true` and restores the previous value in `finally`. - Detects libc once per process and caches the result, because the C library cannot change while the process runs. A failed report also caches "unknown", so it is never retried per request. - Adds `resetLocalAgentRunnerTargetCacheForTests()`. - Reads `excludeNetwork` through a narrow typed view, because the installed Node type definitions do not declare it yet. - New `server/src/__tests__/agent-runner-platform.test.ts`. - `agent-runner-selection.test.ts` and `dev-runner-native-binary.test.ts` reset the cache between cases, because they change the mocked glibc version per case. ## Verification - `pnpm exec vitest run src/__tests__/agent-runner-platform.test.ts src/services/agent-runner-selection.test.ts src/__tests__/dev-runner-native-binary.test.ts src/services/local-ai-browser-login.test.ts` (in `server/`): 165 passed. New tests: - The report is generated with `excludeNetwork` set, and the previous setting is restored. - Five calls generate one report. - A failing report gives `libc: "unknown"` once and restores the setting. - No report is generated off Linux. - `pnpm typecheck` in `server/`: passed. - Measured on Node 24.21 (above): 9,774 ms → 4 ms for one report with one slow-PTR socket open. ## Risks - Low risk. The detected glibc version is the same; only the network section of the report is skipped, and nothing reads it. - The cached result does not change until restart. The C library cannot change while the process runs. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs, live diagnosis of the stalled process). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5c7a025032 |
fix(ui): clear consumed messages from the pending queue (#16317)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations keep follow-up messages pending while an agent works. > - The screen combines optimistic sends, queue snapshots, and run input records. > - A late send response can restore a local queue flag after a run takes the message. > - A stale queue snapshot can then keep the same message in pending. > - This pull request matches the input record to its source queue and clears stale pending flags. > - The benefit is that consumed messages leave pending while unsent messages stay visible. ## Linked Issues or Issue Description **What happened?** A follow-up can remain in the task composer's pending row after a run takes it as input. Local queue state can outlive the send request. The old projection cleared that state for steering, but not for ordinary run-start input. A stale queue read can also retain the consumed entry. **Expected behavior** A message that a started run takes as input appears once in history and leaves pending. Other pending messages remain available. A message assigned to a run that has not started remains pending. **Steps to reproduce** 1. Send a follow-up while a task has a live run. 2. Let a started run record that message as input before or after the send response arrives. 3. Let a queue read still include the consumed message. 4. Observe the message remain pending on the base revision. **Paperclip version or commit** Reproduced against master at `33d24e48ad` with controlled request ordering. **Deployment mode** Source checkout. Browser verification used the production task screen with a simulated native runner and delayed API responses. It did not run a live model or change a deployed instance. Related: [PR #15030](https://github.com/paperclipai/paperclip/pull/15030) addresses acknowledgement matching in the runner console. This change addresses the task-detail composer and does not modify that console. ## What Changed - Clear local pending flags when a started run records the comment as input. - Use live-run input IDs while the run-history query catches up. - Match stale queue entries by their primary or coalesced source wake ID, even when the snapshot targets the previous run or has no target. - Retain coalesced source queue IDs in run context and include receipt IDs in compact task run reads. - Keep unstarted run inputs and authoritative retries pending. Cover send ordering, same-run and successor delivery, queued snapshots, coalesced inputs, and retry consumption. - Add the delivery-race checks to the steering feature map. ## Verification - Before the fix, the first four request-order regressions failed with the consumed message still in pending. Coverage now includes 16 combinations of receipt source, send-response order, and stale snapshot kind. - Focused UI checks: 5 files and 233 tests passed. These cover task detail, queue projection, optimistic comments, queued-message controls, and the task-chat adapter. - `pnpm -r typecheck` passed. - Server receipt checks: 5 files and 139 tests passed with `PAPERCLIP_TEST_POSTGRES_TEMPLATE=0`. These exercise real database reads, coalesced heartbeat input, and polling routes. The default cached-template path timed out during setup on this host; the fresh-cluster path passed without longer timeouts. - Token gates and `node scripts/check-maps.mjs` passed. - Browser: sent a follow-up with a delayed response in the production task screen. The consumed message moved into history. A second unsent follow-up stayed queued. Reload preserved both states. The API and runner were simulated. - `pnpm build` passed. - The full local repository test run was started, then stopped to isolate database setup timeouts. All 55 GitHub CI checks passed on `4ec6820cf1f89880b801fd5f938d72f54591c5e2`, including the full test matrix. One serialized server shard passed on its single rerun after a 15-second route-test timeout; the same 97-test route suite also passed locally. - Greptile reviewed the final head at 5/5. All review threads are resolved. ## Risks - Queue display depends on matching comment IDs, source queue IDs, and a recorded start time. Rows without start evidence stay pending. Newer queues remain visible until their own input receipt arrives. - Run context gains additive receipt metadata. No schema migration is required. Dispatch order, stored queue state, company access, and steering authority stay unchanged. - Browser evidence covers controlled response ordering. It does not establish deployment or live-provider behavior. ## Model Used - OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The exact serving model ID and context-window limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
c30fb7d2b7 |
refactor(core): share Runner execution lifecycle (#16318)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core starts Runners in local, SSH, and sandbox environments. > - Coding runs, Dot runs, and setup probes repeat connection setup code. > - These paths must keep the same preparation, authentication, and cleanup order. > - This pull request gives them one execution lifecycle contract and one server connection driver. > - Core keeps the existing PRP control plane after connection. > - The benefit is one place to maintain environment routing before adding another execution provider. ## Linked Issues or Issue Description **What existing behavior does this improve?** Runner environment preparation and connection setup across local, SSH, and sandbox execution. **Current behavior** The coding transport owns the environment hook type. Dot borrows that type. The server repeats connection selection, registration, ingress activation, and release in task execution and setup probes. Changes to one path can miss another path. **Proposed behavior** Use a shared `RunnerExecutionDriver` contract. Route built-in task execution and remote setup probes through `createNativeRunnerExecutionDriver`. Keep provider session setup, process supervision, PRP commands, and recovery logic on their existing paths. **Reason and benefit** Environment code can prepare resources and provide a connection without adding another PRP execution path. The same driver preserves local loopback, remote outbound WSS, and authorized sandbox ingress. **Breaking changes** None intended. Existing flat SDK options remain compatible. This is an in-process contract. It does not add a plugin wire contract or Launcher APIs. **Additional context** Related: #15406 proposes plugin-provided Runner execution. This change starts from `master` and does not depend on that PR. It extracts the existing lifecycle before adding another provider. Related: #16125 covers remote environment support for external adapters; this PR does not implement that feature. ## What Changed - Extract and export `RunnerExecutionDriver` from the coding transport. Use it for coding and Dot options. - Share server connection negotiation across coding runs, Dot runs, and remote setup probes. - Preserve outbound validation before staging, capability checks before ingress, process dispatch before ingress connection, and verified process readiness. - Preserve warm run registration, restart guards, checkpoint hooks, and cleanup ownership. - Add 13 focused driver tests and document the lifecycle boundary. ## Verification - Targeted server execution and recovery tests: 728 passed, 35 skipped. These cover `native-session-executor`, `native-ssh-command-runner`, the new driver, warm restart transitions, and Runner restart recovery. - Targeted setup and driver tests: 109 passed after the review fix. Twelve driver tests overlap with the preceding count. The added test verifies that failed CA staging cannot leave a PRP route behind. - Runner connectivity tests: 11 passed. - Runner package lifecycle tests: 287 passed across coding transport, Dot, restart continuation, and settlement. These exercised the compiled Rust Runner. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `git diff --check`: passed. Local secret-pattern scan: clear. - The PRP transport runtime and remote process launcher remain unchanged. A byte comparison against the base confirmed this. - CI passed on `b10789dfce30a837ce890f44e352064c7a74366f`: all 55 active checks are green, including the full test shards, Runner checks, browser tests, typecheck, build, and aggregate gate. Greptile scored this commit 5/5 with no unresolved findings. - An earlier broad local shard run was not green: 34,888 passed, 12 failed, and 4,125 skipped. Failures were ten timeouts and two Git fixture assertions affected by local `init.defaultBranch=main`. Isolated retry passed all six non-route timeout suites (427 tests), two route suites, and both Git assertions with a test-process-only `init.defaultBranch=master`. The agent live-run route retry still timed out. No unrelated source changes were made. Further broad local runs were stopped at the user's request. Targeted commands: ```sh cd server node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-ssh-command-runner.test.ts src/services/native-runtime/native-runner-execution-driver.test.ts src/services/native-runtime/native-warm-transition-restart.integration.test.ts src/services/native-runtime/native-runner-restart-recovery.integration.test.ts --config vitest.config.ts node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/setup-readiness.test.ts src/services/native-runtime/native-runner-execution-driver.test.ts --config vitest.config.ts cd ../packages/paperclip-runner node ../../node_modules/vitest/vitest.mjs run src/live/runnerd-codex-transport.test.ts src/drivers/dot/runnerd-dot-driver.test.ts src/live/runnerd-restart-continuation.test.ts src/live/runnerd-codex-transport-settlement.test.ts --maxWorkers=2 cd ../.. node node_modules/vitest/vitest.mjs run packages/adapter-utils/src/runner-connectivity.test.ts ``` ## Risks - Startup ordering and connection cleanup are sensitive. Focused tests cover local registration, outbound validation, CA mapping, ingress ordering, release, and warm authority guards. - No live SSH host or sandbox provider was used. Existing mocked remote tests and real local Runner tests passed. - No database, PRP wire protocol, or Launcher API changes are included. ## Model Used OpenAI GPT-6 via Codex, with tool use and code execution. The served model variant and context window were not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the stated targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
51b56706fb |
fix(grok): verify file completion and recover failed sessions (#16305)
## Thinking Path > - Paperclip manages work and authority for AI agents. > - Grok Bot runs outside Paperclip and uses authenticated upload grants for files. > - Native completion requires a verified file publication receipt. > - The upload saved the file but did not create that receipt. Real Grok work could not finish. > - A failed run also left an assignment that the Stop action could not fence. > - This pull request connects verified uploads to normal completion and repairs that recovery action. > - The result preserves file integrity and current work-session authority. ## Linked Issues or Issue Description Refs: #16219, #16274. **What happened?** A real Grok Bot saved a task document and CSV. Semantic completion rejected the CSV because its upload had no verified publication receipt. After the managed run failed, Stop rejected its remaining assignment as a changed session. The driver also discarded the remote monitor's known failure reason. **Expected behavior** An authenticated upload of the exact requested bytes can satisfy normal completion. Stop can fence the remaining assignment of the selected failed run. An old Stop action cannot stop newer work. Known remote lease and reconnect errors remain visible without raw diagnostics or credentials. **Steps to reproduce** 1. Connect an experimental Grok Bot agent and verify its routine. 2. Assign a task that requires a document and a file. 3. Upload the file through its accepted assignment and cite the attachment in semantic completion. 4. Observe the missing publication receipt error on the previous implementation. 5. For recovery, select Stop after that run fails but its assignment remains active. **Paperclip version or commit** The live defects were observed after #16274. The current candidate merges master `739f9e40a4` and is `26c5b9601d1cb2b8f5bb7fcf730f019523763f5b`. ## What Changed - Record a server-owned publication receipt in the verified upload transaction. Bind it to the real native Grok run and exact stored file metadata. - Validate external publication against the actual upload and admitted assignment. Keep existing native and person upload rules. - Recover the committed receipt after a lost response. Reverify the same bytes when an older upload lacks proof. - Hold assignment authority through publication with grant-before-assignment and task-before-run lock order. Disconnect does not wait for remote file storage. - Allow Stop to fence the exact lingering assignment of a terminal run. Reject stale Stop calls for successor work. - Preserve only known lease/reconnect failure codes from the remote monitor. Do not expose raw stderr. - Extend real Rust, overlapping document/upload and disconnect/upload, file retry, completion and Stop regressions. Document setup, files and recovery. ## Verification - The real Rust Grok regression requests a file, checks an overlapping native document save, concurrent retries and lost-response recovery, rejects missing delivery evidence, then uses normal semantic completion and finalization. - Focused tests cover person MCP, native file handoff, completion, Stop and both external driver providers. - Local repository typecheck and production build pass on the current candidate. The five focused protocol suites pass (264 tests); all 17 external Runner driver tests pass, with the driver unchanged since the master merge. - Token gates and Storybook build pass. Nine existing Grok fixture journeys pass their browser interaction assertions; the static probe separately records five fixture API routes returning 404. This is UI fixture evidence only. - Full repository coverage passed in CI. The duplicate local `pnpm test:run` was interrupted after CI completed; its partial log is retained and is not reported as a local full-suite pass. The earlier aggregate invocation was interrupted for the second review fix. - All CI gates pass on `26c5b9601d1cb2b8f5bb7fcf730f019523763f5b`: 55 successful checks and two skipped Storybook jobs (no UI diff). This includes the complete sharded repository suites, native Runner, release checks, build and security scans. - Current-head Greptile is 5/5 with zero unresolved findings. The PR has no merge conflicts. - The prior live attempt is failed acceptance evidence. Real Grok requalification, proactive work, controller restart and active-session Stop remain release gates. This PR does not claim those passed. ## Risks - Upload and completion now share a durable run receipt. Lock order and current assignment authority must remain consistent with native mutations. - Publication fails closed if the file, run or grant changes. A retry must keep the same upload identity and bytes. - Person connections gain no agent authority. No database migration is required. - Known exit diagnostics do not prove global Grok termination. Grok spending remains unreported. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution and computer use. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ad0bd26bb7 |
perf(heartbeat): avoid Git inspection for stored workspace reads (#16306)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat selects and restores a saved execution workspace before dispatch. > - These reads used the workspace detail reader, which also scans Git and resolves delivery state. > - Configuration reads do not use those live display fields. Each extra process can delay other controller work. > - This pull request adds a stored-record reader and uses it in configuration, environment selection, and finalization. > - It also derives ancestry from the existing ahead/behind result and skips delivery HEAD reads for unfinished tasks. > - The benefit is fewer Git processes while live execution and cleanup guards remain in place. ## Linked Issues or Issue Description Related: #16282, #16302, #13619. Those changes reduce background or display scan demand. This change removes live display inspection from execution configuration reads. **What existing behavior does this improve?** Heartbeat workspace configuration reads and workspace close-readiness checks. **Current behavior** Reading saved workspace configuration also loads runtime health, Git status, ancestry, and delivery state. Close readiness walks history twice to determine ancestry. **Proposed behavior** Execution reads stored configuration through a company-checked method. Live branch validation and destructive cleanup still inspect current Git state. Close readiness uses its ahead/behind result to determine ancestry. **Reason and benefit** A synthetic benchmark of the actual service returned identical stored fields for 20 workspaces. Detail reads launched 60 Git commands and took 689 ms. Stored reads launched zero Git commands and took 8 ms. This was a local macOS measurement, not an end-to-end cloud latency claim. **Breaking changes** None to public APIs or saved data. Workspace detail responses keep their live fields. ## What Changed - Add `getStoredById(companyId, id)` and use it in three heartbeat reads. - Preserve the access-denied error for a foreign-company workspace. - Remove the extra `merge-base` process and the unused HEAD read for unfinished tasks. - Add recovery, company-boundary, ancestry, and cleanup regressions. - Add a repeatable database/Git benchmark and document the read boundary. ## Verification - The new environment-selection regression failed before the fix with `Live Git inspection is unavailable` and passes after it. All eight environment-selection tests pass. - On the final rebased commit, workspace service, environment selection, and branch finalization: 89 tests passed. Branch-containment, runtime-execution, and display-cache suites also passed during development. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: started locally, then stopped after all full-suite CI test shards passed on the final commit. The complete local run was not finished; CI provides the full-suite result. - Final commit `9a13ef0481b7bf6a60b2b7ac13f35f78a14e232c`: 55 successful checks and 2 expected skips. Greptile: 5/5, no actionable findings. - `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-record-reads.ts`: identical stored fields, 60 to zero Git commands, 689 ms to 8 ms for 20 reads. ## Risks - Execution callers must use saved fields only. Detail readers and live branch and cleanup guards remain separate. - A failed ancestry comparison stays unknown. Automatic cleanup still rejects unknown delivery state and rechecks Git before deletion. - This patch has not been deployed. It does not prove the full slow-run latency is resolved. ## Model Used OpenAI Codex, GPT-6. The exact model identifier and context window are not exposed in this session. Used reasoning, repository inspection, code editing, and local command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4460785534 |
fix(skills): preserve referenced files in large repository imports (#16308)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use skills imported from project repositories.
> - Large repositories fall back to isolated skill folders.
> - That fallback drops shared files and turns advisory links into
import failures.
> - This pull request retains bounded referenced files and keeps missing
references advisory.
> - Users can import valid skills such as test-audit without editing the
source repository.
## Linked Issues or Issue Description
Refs #16066. Refs #16256. Refs #16288. Related draft #15730 adds
optional reference selection to individual skill packages. This fix
applies automatic repository fallback and does not add that selection
UI.
**What happened?**
A project source showed 86 warnings. It rejected 23 skills solely
because references were absent from the isolated package. It also
treated web routes and inline Markdown examples as files. For
test-audit, repository scripts were resolved inside the skill folder.
**Expected behavior**
Retain referenced files that fit the package limits. Missing references
must not block an otherwise valid selected skill. Preserve content
validation and installed shared files.
**Steps to reproduce**
Import paperclipai/paperclip at commit
|
||
|
Devin FoleyandPaperclip
|
5b29b9062e |
fix(server): compact finished task worktrees and shorten the reaper cooldown (#16304)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each task can run in its own isolated Git worktree, and each worktree installs its own dependencies > - The terminal-workspace reaper removes a worktree only when the task tree is finished, the delivery is merged, the files are clean, and a 7-day cooldown has passed > - A recurring task that runs every hour keeps a week of worktrees, and each worktree holds gigabytes of `node_modules`. A finished task with no merged pull request, or with leftover agent files, keeps its worktree forever > - On a busy instance this fills the disk in days, and a larger volume only delays the next failure > - This pull request adds compaction: one hour after a task tree finishes, the reaper removes regenerable directories from the worktree and keeps the workspace, the branch, and all source changes > - The benefit is that finished tasks stop holding gigabytes of disk, and no work is lost because a resumed task regenerates the files when its workspace is provisioned again ## Linked Issues or Issue Description Refs #2022 (retention for workspace cleanup). Related: #11936 and #14608 add external reclaim scripts (a scheduled tool and a disk-guard step). This pull request does the reclaim inside the server's terminal-workspace reaper instead. That way it uses the reaper's lifecycle lock, issue-tree checks, and live-run checks, and needs no extra scheduler. **What existing behavior does this improve?** Disk retention of execution workspaces after their tasks finish. **Current behavior** A finished task keeps its full worktree, including installed dependencies and build output, for at least 7 days. If the delivery is not merged, or the worktree has uncommitted or untracked files, the reaper never removes it. On one instance, an hourly recurring task created about 22 worktrees per day at about 3 GB each (about 2.8 GB of that was `node_modules`). Together with finished tasks that had no merged pull request, they filled a 100 GB volume in about two days. **Proposed behavior** One hour after a task tree becomes terminal, the reaper removes regenerable directories (installed dependencies, build output, tool caches) from the worktree. It keeps the worktree, the branch, the commits, and every tracked, untracked, or other ignored file. When the task resumes, the existing reuse path runs the provision step, which reinstalls the dependencies. Clean, merged worktrees are archived after 1 day instead of 7. **Reason and benefit** Most of a finished worktree's size is regenerable. Compaction removes that part without the risk of deleting work, so it can apply to workspaces that the reaper must keep (no merged pull request, leftover files). The shorter cooldown limits the cost of the source checkouts that stay. **Breaking changes** None. The default archive cooldown changes from 7 days to 1 day. `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS` still overrides it. `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. ## What Changed - New `server/src/services/workspace-compaction.ts`: - Finds directories with a regenerable name (`node_modules`, `dist`, `build`, `coverage`, `storybook-static`, `target`, `.cache`, `.next`, `.nuxt`, `.parcel-cache`, `.svelte-kit`, `.turbo`, `.vite`). - Keeps only the directories that the repository's own `.gitignore` files ignore and that hold no tracked file. A negated pattern (`!build/`), a global excludes file, and `.git/info/exclude` do not count. Each candidate is checked per path, so dependencies below an ignored parent (`vendor/tool/node_modules`) are found. - Searches inside a regenerable-named directory that does not qualify (for example a tracked `build/`), so its ignored dependencies still go. - Never enters `.git` or `.paperclip` (the worktree instance data), and never follows a symlink. - Each Git read has a 60-second deadline and stops when the sweep is aborted. - Moves the directories into `.paperclip-compaction-trash` beside the worktree with a same-filesystem rename, and can restore them. - `sweepTerminalWorkspaces` compacts each terminal workspace once per terminal cycle, after the compaction delay and before the archive cooldown check: - Only isolated linked Git worktrees qualify. A shared workspace, a project's main checkout, or a row whose `.git` is a directory is never compacted. - The move runs inside a transaction that holds the workspace lifecycle lock. The guard checks a fresh row and the same lifecycle generation. It also checks that the issue tree is terminal, that no queued or running run works on the tree or the workspace (by checkout or execution run, or by run context), and that no runtime service runs from the workspace. - The guard runs before the move and again after it. If the second check fails (for example, the task reopened or a run started), the directories move back. Otherwise the trash is deleted. - The worker wakes at the compaction deadline, not only at the archive deadline. - Stale trash from an interrupted compaction or a failed delete is purged by later sweeps. The sweep checks each worktree parent at most every 30 minutes, whether or not anything is left to compact. - New counters `compacted` and `compactionFailed` in the sweep result. The activity log gets an `execution_workspace.regenerable_files_removed` entry with the removed paths. - Config: new `PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS` (default `1`) and `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED`. The default for `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS` changes from `7` to `1`. Both new keys are added to the run-failure diagnostics allowlists. - Docs: `doc/DEVELOPING.md` documents the new variables and has a new "Terminal workspace compaction" section. `doc/SPEC-implementation.md` describes compaction. - `execution-workspaces.ts` was already above 1,000 lines. The file-system logic is in the new module. Only the guarded orchestration, which uses the existing lifecycle lock and run predicates, is in the service. ## Verification - `pnpm exec vitest run src/__tests__/workspace-compaction.test.ts src/__tests__/execution-workspace-compaction.test.ts` (in `server/`): 27 passed. `workspace-reaper-cooldown-config.test.ts`: passed. - Unit tests: directory selection (ignored, untracked, regenerable name only, `.env` and local data kept, tracked `coverage` kept, `.paperclip` skipped, symlinks not followed), nested candidates under a source `build/` and under an ignored `vendor/`, a negated `!build/` kept, global and `info/exclude` rules ignored, glob characters in names, abort, main checkout vs linked worktree, move and restore, restore that keeps a recreated directory, and stale-trash purge. - Embedded Postgres tests: - A no-PR, unmerged, dirty workspace is compacted. HEAD, the branch ref, `git status --porcelain --untracked-files=all`, untracked notes, `.env`, local data, and `.paperclip` data stay the same, and the row stays active. - A delivered workspace inside the cooldown is compacted. - The delay is respected, and the worker wakes at the delay. - An active checkout run, a queued run found through its context, an unfinished child issue, and a running runtime service all block compaction. - The task reopens during the move: the files are restored and unchanged. - A run starts during the move: the files are restored. - A shared workspace and a main checkout are never compacted. - Compaction runs once per terminal cycle and again after a later cycle. - Disabled compaction does nothing. - End to end: `realizeExecutionWorkspace` creates a worktree, the sweep compacts it, and a second `realizeExecutionWorkspace` reuses the same worktree. The provision command regenerates `node_modules`, and the untracked notes and the unmerged commit are still present. - Leftover trash is purged when the workspace has nothing left to compact. - Mutation checks: when the post-move guard is disabled, both race tests fail. Without the negation fix or the repository-only source fix, their tests fail. - `pnpm exec vitest run` for `execution-workspaces-service.test.ts`, `terminal-workspace-work.test.ts`, `execution-workspace-reopen.test.ts`, `server-startup-feedback-export.test.ts`, and `workspace-runtime.test.ts`: 335 passed. Two `workspace-runtime.test.ts` cases fail locally with and without this change (`does not diagnose a missing remote branch as a rejected credential`, `rejects explicit main on a master-only remote ...`). They also fail on unmodified `master` on the same machine, so the cause is the local Git environment. - `pnpm typecheck` in `server/`: passed. `pnpm check:module-boundaries`: passed. ## Risks - A repository can keep something that is not regenerable in an ignored `dist`, `build`, `target`, or `.cache` directory. Compaction removes it. The fixed name list, the ignore check, and the tracked-file check limit this. `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. - A repository without a provision command does not get dependencies back automatically. This is the same state as a new worktree for that repository, where the agent already installs them. - A resumed task pays for a dependency install again. This happens only for tasks that finished more than an hour earlier. - The shorter default cooldown removes clean, merged worktrees after 1 day instead of 7. An operator who wants the old hold can set `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS=7`. - Self-hosted instances get the same behavior. There is no schema change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). A separate Claude agent read the reaper code during the diagnosis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
adb69e8f09 |
feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683)
## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto | paperclip | legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. **Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass.** The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board **Try again** and **Resume** recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal **Approve completion** reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
21ee5c9c85 |
feat(sentry): add opt-in run failure reporting with saved run references (#16300)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs retain execution results and output in the instance. > - Operators can report run failures to Sentry for investigation. > - A backend Sentry DSN currently enables those reports with no separate choice. > - A repair bot also needs the exact instance, company, agent, and saved run. > - This pull request adds an explicit reporting mode and authenticated run references. > - The benefit is that operators can investigate every failure on selected deployments while keeping other deployments quiet. ## Linked Issues or Issue Description **What existing behavior does this improve?** Optional server reporting of terminal agent run failures to Sentry. **Current behavior** A backend DSN enables run failure reports. Known account, quota, and connection failures are excluded. Reports omit company and agent references. Runs without a task are skipped. Tool-name collision errors omit the conflicting bindings. **Proposed behavior** Set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain the existing filters. Set it to `all` to report every failed or timed-out run, including known setup and quota failures. The default is `off`. Expected stops stay excluded. Each report supplies identifiers and API paths that a bot can use with its own authorized Paperclip connection. **Reason and benefit** Operators can enable broad failure reporting on one staging deployment. A bot can fetch the saved run, events, and logs instead of relying on a short error label or copying complete transcripts into Sentry. **Breaking changes** A backend DSN alone no longer enables run failure reports. Existing operators must set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain the previous policy. Ordinary server exception capture remains enabled by the DSN. Failure grouping gains the phase and original cause code when available. Refs #16281, #16297, #16298. These cover related tool discovery and failure diagnostics. This change retains their existing boundaries and does not depend on their open changes. ## What Changed - Add `off`, `unexpected`, and `all` modes. Missing and invalid values keep reporting off before diagnostic reads. - Keep the Sentry backend DSN and audited optional SDK requirements. - Include company, agent, run, and optional task identifiers. Include versioned paths for the saved run, events, log, and task. Add absolute run URLs only for a valid public HTTP(S) origin. - Report company runs without a task. Prefer the first-class issue binding when selecting the task reference. - Select saved native provider/model settings, timestamps, and original failure codes. Preserve gateway reason codes through redacted exception projection. - Group by error code, adapter, and available phase/cause. Keep run IDs and error text out of grouping. - Include the tool names and connection bindings in collision errors without copying tool schemas or provider metadata. - Document configuration, upgrade behavior, bot access, redaction, retention, and best-effort delivery. ## Verification - All 56 checks passed for `cccb47a721eaa1cc36e63278fc177b9101477541`. Two unrelated Storybook jobs were skipped by their path filters. No checks remain pending. - Complete [Paperclip CI run](https://github.com/paperclipai/paperclip/actions/runs/38591422284): passed. This covers general and serialized tests, Runner checks, browser E2E shards, typecheck, build, release-registry checks, canary dry run, and Docker checks. - Greptile reviewed that exact head: 5/5, with no unresolved threads or actionable findings. Superagent, Socket, and Snyk checks passed. - Local focused reporting, diagnostics, collision, and real SDK contract tests: 410 passed across five suites. This includes the external-wake filter in both reporting modes. - The audited `@sentry/node@10.71.0` tests use an in-memory transport. No events were sent to Sentry. - `pnpm -r typecheck` and `pnpm build`: passed locally after the final rebase. - `node scripts/check-module-boundaries.mjs` and `git diff --check`: passed. - The full local `pnpm test:run` command was stopped to avoid duplicating the complete CI shards. It did not finish locally. The full suite passed in CI. ## Risks - The new default disables existing run failure exports until the operator selects a mode. The upgrade setting is documented. - `all` can increase event volume and include known user-action failures. Secret redaction and diagnostic limits still apply. - Reporting remains asynchronous and best effort. This PR adds no durable delivery queue and does not install a bot or Sentry alert rule. - API references grant no authority. Existing company, private-task, and run-output checks still apply. Complete transcripts stay in Paperclip. - This changes optional Sentry error reporting. It changes neither first-party Telemetry nor OpenTelemetry span exports. It adds no database migration. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, shell, file editing, and test execution. The exact backend deployment ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
09a2f0b7b0 |
refactor(server): isolate provider message admission and runtime policies (#16289)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn external messages into shared task workflows. > - The main service still contains some provider decisions. > - These decisions belong with the existing channel modules. > - This pull request isolates provider message-admission and runtime policies. > - Shared authorization, persistence, and runtime ownership keep their current order. ## Linked Issues or Issue Description Refs: #16235. Stack part 3/3. Base: `master`. #16276 and #16283 are merged. **What existing behavior does this improve?** Provider ownership and readability in the chat channel service. **Current behavior** The entry point combines shared workflows with provider decisions. **Proposed behavior** Provider modules own their decisions. Shared modules own product state and authorization. **Reason and benefit** Maintainers can find provider behavior in its channel folder. **Breaking changes** None intended. Public service methods and durable data shapes stay compatible. ## What Changed - Move Discord SDK admission and its barrier to the Discord message module. - Move Photon live-message eviction and startup recovery policy to its runtime callbacks module. - Move Slack mention eligibility and Teams setup-channel eligibility to their message modules. - Keep Photon deferred ingress and Speko conversation policy with their provider modules. - Keep ordered SDK message batching in a shared runtime module with a live per-message ingress callback. - Keep receiver leases, cache lifetime, startup failure cleanup, authorization locks, and transaction ownership shared. - Reduce `chat-channels.ts` from 3,570 to 3,548 lines. ## Verification - The same 46 runtime/admission integration cases and all 18 Photon channel cases passed before and after in the owned Linux container. - The existing first-channel Teams orphan-admission case passed before and after moving its eligibility predicate. - The same 94 SDK adapter and lifecycle-normalization tests passed locally. - Server typecheck and module-boundary checks passed. - Token comparison confirms ordered SDK batching and every processor argument, plus Photon invalidation and its live runtime lookup. - The shared factory creates no cache or worker. It reads the ingress policy for each message and awaits processing in order. - The extraction rebased without conflicts onto master after #16276 and #16283 merged. Git range-diff reports the same patch. Final head `7b63b569e2a0dbcc959310f565f77f367fa64aa0` has the same patch by range-diff. The 94 SDK/lifecycle tests, server typecheck, and boundary check passed on the final head. All CI gates pass on that exact head. Greptile reviewed it at 5/5 with no inline findings or unresolved review threads. - One Runner CI test failed its original short timing window. The unchanged test passed earlier-head CI and local reproduction. The single failed-job retry passed. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321 channel integration cases passed. The normal UI, CLI, database, adapter, plugin, and serialized API test selections ran. - The first Linux `pnpm test:run` returned 1. Original-source runs reproduced the local failures. Container-only repairs supplied missing tools, process reaping, and sufficient Node heap. The existing mocked-fetch CLI telemetry fixture passed with the image opt-out flag overridden only for that fixture. - One local venue limitation remains: adapter-utils T24 fails unchanged on both original and final source in the Linux container. Its entire 56-case suite passes before and after on the host. All other failed or aborted suites passed in repaired reruns. No test assertion or timeout changed. - These fixtures prove application integration. They do not prove live provider execution. ## Risks - A captured option value or changed callback order could affect deferred ingress or stale-runtime handling. - The ingress callback and runtime getter remain live. Core maps and receiver lease operations keep one owner. - No schema migration or public API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7df55f3dcc |
refactor(server): isolate provider webhook lifecycle decisions (#16283)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn external messages into shared task workflows. > - The main service still contains some provider decisions. > - These decisions belong with the existing channel modules. > - This pull request moves webhook recovery and payload-scope decisions to their provider modules. > - Shared authorization, persistence, and runtime ownership keep their current order. ## Linked Issues or Issue Description Refs: #16235. Stack part 2/3. Base: `master`. #16276 is merged. **What existing behavior does this improve?** Provider ownership and readability in the chat channel service. **Current behavior** The entry point combines shared workflows with provider decisions. **Proposed behavior** Provider modules own their decisions. Shared modules own product state and authorization. **Reason and benefit** Maintainers can find provider behavior in its channel folder. **Breaking changes** None intended. Public service methods and durable data shapes stay compatible. ## What Changed - Move GitHub revoked-installation recovery classification to `github/webhook-scope.ts`. - Move Teams payload-scope validation to its message module. - Record normalized GitHub, Telegram, and Teams lifecycle events through one shared durable call. - Keep Slack callback ordering and persistence failures outside parser catches. - Reduce `chat-channels.ts` from 3,598 to 3,570 lines. ## Verification - The same 86 selected webhook/lifecycle integration cases passed before and after in an owned Linux container. - The same 17 selected Telegram edit and Teams edit/delete/restore cases passed before and after in that container. - The same 94 SDK adapter and lifecycle-normalization tests passed locally before and after. - Server typecheck and module-boundary checks passed. - Token comparison confirms the shared recording payload and runtime fence, plus the Teams scope and parser inputs. - The host database baseline failed before assertions because System V shared-memory IDs were exhausted. No other PostgreSQL server or host setting was changed. Database verification used an isolated container under its unprivileged user. - The extraction rebased without conflicts after #16276 merged. Git range-diff reports the same patch. The 94 SDK/lifecycle tests and server typecheck passed on this base. All CI gates pass on rebased head `d835d7c3815c168226fa646aec6370b6a8ac94c1`. Greptile reviewed that exact head at 5/5, with no inline findings or unresolved review threads. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321 channel integration cases passed. The normal UI, CLI, database, adapter, plugin, and serialized API test selections ran. - The first Linux `pnpm test:run` returned 1. Original-source runs reproduced the local failures. Container-only repairs supplied missing tools, process reaping, and sufficient Node heap. The existing mocked-fetch CLI telemetry fixture passed with the image opt-out flag overridden only for that fixture. - One local venue limitation remains: adapter-utils T24 fails unchanged on both original and final source in the Linux container. Its entire 56-case suite passes before and after on the host. All other failed or aborted suites passed in repaired reruns. No test assertion or timeout changed. - These fixtures exercise application state and the service entry point. They do not prove live provider execution. ## Risks - A scope or ordering error could record an event for the wrong provider context. - The selected parser returns normalized data. The shared service keeps durable recording and runtime fencing. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
74ba530b9e |
fix(workspaces): keep cleanup Git scans fresh and dedupe page reads (#16302)
## Thinking Path > - Paperclip manages execution workspaces for agent tasks. > - It must preserve work when it closes a workspace. > - The terminal-workspace worker checks Git before it decides whether a workspace can close. > - A request for fresh Git status could join a scan that started before a file changed. > - Several records for one directory can also repeat the same Git work in a bounded page. > - The worker already reads the task tree before Git, then reads it again for delivery assessment. > - This pull request fixes the fresh-read boundary and removes those repeated reads. ## Linked Issues or Issue Description - Refs #14759 and #14423 for the background Git scan load. - Refs #16282 for the merged event-driven terminal-workspace worker. - Refs #13619 for the original community investigation and tests. Thanks to @fayonation for identifying the shared-directory and in-flight scan cases. This implementation starts from current master and adapts those ideas to the new worker. ## What Changed - Give correctness-sensitive Git status requests their own scheduler scan. They skip completed cache entries and older scans in progress. - Share compatible Git readiness results across workspace records in one bounded worker page. The key includes company, path, branch, base ref, and runtime ownership. Each record keeps its own task and delivery assessment. - Reuse the task tree already read by the worker during delivery assessment. - Add regression tests for a file created during an older scan, compatible scan sharing, and company and branch separation. Update developer and workspace map documentation. ## Verification - `pnpm exec vitest run server/src/services/workspace-git-operation-scheduler.test.ts server/src/__tests__/execution-workspaces-service.test.ts` — 92 tests passed. - `pnpm -r typecheck` — passed with the repository's pnpm 9.15.4. - `node scripts/check-maps.mjs` — passed. - `pnpm build` — passed. - The full local `pnpm test:run` was interrupted when its original worktree disappeared. All PR test shards passed in GitHub CI. - Latest commit `e7622127c7d978a9a90857112675834725965d72`: 53 successful PR checks, 2 expected skips, and no failures. Greptile reviewed this commit at 5/5 with no open findings. ## Risks - A shared page inspection can become stale if the directory changes before the next record is assessed. The final cleanup still starts a separate Git scan and checks current task, ownership, and lifecycle state under the existing guards. - The new scan key changes request coalescing for correctness-sensitive callers. They still use the scheduler's concurrency and queue limits. There is no database migration or API change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code editing, and shell tools. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
bcfc03bc94 |
fix(sentry): keep provider-rejected Grok wakes out of run failure reports (#16297)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner can run an external Grok bot. Paperclip offers the bot an assignment and sends a webhook wake. The bot must accept the assignment in 10 minutes. > - When the Grok endpoint rejects every wake with HTTP 4xx, the bot never sees the work. The offer expires and the run fails with `Dot assignment acceptance or execution authority expired.` > - Paperclip already records each wake in `grok_bot_dispatches` with its HTTP status. The operator sees the rejection and can retry it. Thus Paperclip operates correctly in this case. > - But `reportRunFailure` sends this run to Sentry as an unexplained backend error. This makes noise and hides real defects. > - This pull request skips the Sentry report only when durable receipts prove that the Grok endpoint rejected every wake. > - The benefit is a cleaner Sentry signal. Local defects, uncertain delivery, and accepted work stay visible. ## Linked Issues or Issue Description Sentry issue `PAPERCLIP-BACKEND-3M` (staging, 1 event). **What happened?** A Grok bot run on staging failed after 10 minutes in the `preparing` stage. Sentry got the error `Dot assignment acceptance or execution authority expired.` with the code `native_provider_terminal_failed`. The stored receipts show two `mailbox_updated` wakes for the assignment. The Grok endpoint rejected both wakes with HTTP 400: the first wake and one operator retry. The assignment was never accepted. Wakes with the same shape got HTTP 200 before and after this run. **Expected behavior** The run fails, and the operator sees the rejected delivery. Sentry does not get a backend error for a proven external rejection. **Steps to reproduce** 1. Connect a Grok bot agent and verify its routine. 2. Make the Grok endpoint return HTTP 400 for work wakes. 3. Assign a task to the agent and wait 10 minutes. 4. See that the run fails and that a Sentry event is sent. **Paperclip version or commit** `f69c6a8` (master). ## What Changed - `server/src/services/run-failure-report.ts`: Add `isRejectedExternalBotWake`. It returns true only when all of these conditions are true: - The run failed in native mode with `native_provider_terminal_failed`. - The finalization `failureDetail.originalFailureCode` is `dot_assignment_expired`. - The run has exactly one Dot assignment, and that assignment was never accepted. - The assignment has at least one `mailbox_updated` wake, and each wake has the status `rejected` with an HTTP 4xx status. - `captureTerminalRunFailure` skips the Sentry capture when the check is true. If the receipt check itself fails, it logs a warning and the failure stays reportable. The run status, the task, and the operator recovery do not change. - `packages/db`: Add the index `grok_dispatch_assignment_kind_idx` on `grok_bot_dispatches (assignment_id, kind)` (migration `0346`). The receipt lookup and the dispatcher's per-assignment checks use this index. - `server/src/services/__tests__/run-failure-report.test.ts`: Add one test for the skipped case. Add nine tests for cases that must stay reportable: no wake, a local rejection with no HTTP status, a delivered wake, uncertain delivery, accepted work, a different provider failure code, no finalization, more than one offer, and a receipt read that fails. ## Verification - `pnpm --dir server exec vitest run src/services/__tests__/run-failure-report.test.ts`: 208 tests pass. - I removed the new catch and ran the tests again. The receipt-read-failure test failed. - `pnpm --filter @paperclipai/db run check:migrations` and `src/migration-snapshot-drift.test.ts` pass. - I removed the source change and ran the new tests again. The skipped-case test failed and the six reportable cases passed. This shows that the test proves the change. - I did not run the full `pnpm -r typecheck` locally. The server `typecheck` script needs `cargo` to build the Runner binary, and the sandbox stopped `tsc` with SIGKILL. CI runs the full typecheck. ## Risks - Low risk. The change only removes one Sentry report. It does not change run state, recovery, or what users see. - The check uses durable receipts, not message text. A wake with no HTTP status (local validation or credential failure) stays reportable, so a Paperclip defect cannot hide behind this rule. - For failed native runs with `native_provider_terminal_failed`, the check adds up to three indexed reads. Other runs do no extra reads. - Migration `0346` builds one index in a transaction. `grok_bot_dispatches` was created in `0344` and is small, so the build lock is short. The table is not on the migration gate's large-table list. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Agentic tool use with shell, code search, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
f16c0ad5c8 |
refactor(server): isolate chat conversation and control policies (#16276)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn external messages into shared task workflows. > - The main service still contains some provider decisions. > - These decisions belong with the existing channel modules. > - This pull request separates conversation binding and Telegram control policy. > - Shared authorization, persistence, and runtime ownership keep their current order. ## Linked Issues or Issue Description Refs: #16235. Stack part 1/3. Base: `master`. **What existing behavior does this improve?** Provider ownership and readability in the chat channel service. **Current behavior** The entry point combines shared workflows with provider decisions. **Proposed behavior** Provider modules own their decisions. Shared modules own product state and authorization. **Reason and benefit** Maintainers can find provider behavior in its channel folder. **Breaking changes** None intended. Public service methods and durable data shapes stay compatible. ## What Changed - Keep shared conversation lookup and completion in `inbound/conversation-binding.ts`. - Move Photon binding-completion policy to its messages module. - Move Telegram command classification, forum behavior, and chronology exemptions to its commands module. - Keep the original conversation generation separate from the retained task binding. - Reduce `chat-channels.ts` from 3,649 to 3,598 lines. ## Verification - The same 62 selected Telegram/Teams integration cases passed before and after extraction. - All 18 Photon channel integration cases passed before and after extraction in separate test processes. - The initial combined baseline command had a Photon database-startup hook timeout. No Photon assertion ran in that invocation. The unchanged Photon suite then passed in isolation with its existing timeouts. - Server typecheck and the module-boundary check passed. - A token comparison confirms the moved lookup, completion, and generation logic, after normalizing the explicit policy callbacks. - All current-head CI checks passed. Greptile reviewed this exact head at 5/5 with no findings. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321 channel integration cases passed. The normal UI, CLI, database, adapter, plugin, and serialized API test selections ran. - The first Linux `pnpm test:run` returned 1. Original-source runs reproduced the local failures. Container-only repairs supplied missing tools, process reaping, and sufficient Node heap. The existing mocked-fetch CLI telemetry fixture passed with the image opt-out flag overridden only for that fixture. - One local venue limitation remains: adapter-utils T24 fails unchanged on both original and final source in the Linux container. Its entire 56-case suite passes before and after on the host. All other failed or aborted suites passed in repaired reruns. No test assertion or timeout changed. - These fixtures exercise the assembled service and persisted state. They do not prove live third-party execution. ## Risks - Changing query order or losing the original generation could bind a message to the wrong task. - The workflow retains the original lookup result and all existing query and write conditions. - No schema migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd64eb27b0 |
perf(recovery): use indexed execution identity lookups (#16291)
## Thinking Path > - Paperclip manages AI agents and their execution. > - Recovery must keep old execution ownership visible before new work starts. > - Ownership reads filter large run contexts through a combined text expression. > - Recovery and status delivery also cast indexed UUID columns to text. > - These reads can scan history before finding the requested execution. > - This pull request preserves identity and ownership rules while using existing indexes. > - The benefit is less database work during recovery and task polling. ## Linked Issues or Issue Description Refs #14406. Related community report: #15275. This is an independent implementation. It does not use that pull request's code or commits. It covers the reported execution joins and the additional ownership query hotspot. **What happened?** Accumulated staging query statistics showed 11,782 ownership reads at 51 ms each. A paired recovery-action read averaged 22 ms. Both queried large run JSON. Text casts also prevented indexed UUID lookups in recovery and status delivery. **What did you expect to happen?** Recovery must find the same execution evidence through existing indexes. Malformed identifiers must match nothing without a cast error. Native issue bindings must retain precedence. Missing termination evidence must continue to block new work. **Steps to reproduce** Run the disposable benchmark below. It loads the original service modules from the baseline revision. It compares their exact ownership, blocker action and status-delivery SQL with the current modules on 1,400 synthetic runs, 350 historical recovery actions and 192 KiB contexts. ## What Changed - Convert saved text to UUID only after checking its canonical lowercase form. - Keep indexed run and task UUID columns unchanged in execution joins. - Split native and legacy issue predicates so existing indexes can filter ownership candidates. - Preserve company scope, native precedence, process and lease holds, recovery eligibility and delivery receipts. - Add real PostgreSQL identity and ownership regressions, a reproducible benchmark and database documentation. ## Verification - Local service SQL benchmark: ownership 78.30 ms to 1.14 ms, with the same two complete rows. Blocker action read 76.94 ms to 22.55 ms, with the same complete row. These are synthetic SQL timings, excluding process probes, provider calls and HTTP. - Status delivery returned the same 100 rows. Timing was neutral on the small-task fixture: 7.85 ms before and 8.30 ms after. No speedup is claimed for that case. - 231 local tests passed across six server suites. They cover malformed identifiers, canonical spelling, company scope, native precedence, retained processes, workspace restore, native replacement, conversations and delivery. - `pnpm -r typecheck` and `pnpm build` passed. - `git diff --check` passed. - All current-head CI and security checks passed, including the full general and serialized suites, browser shards, Runner verification and build. Greptile reviewed the exact commit at 5/5 with no actionable findings. - The older full local sweep in the companion worktree was stopped after final-commit complete CI passed. It is not claimed as a completed local pass. The 231 focused tests above passed locally on this branch. ```sh pnpm --filter @paperclipai/server exec vitest run src/services/conversation-continuation.test.ts src/services/legacy-execution-recovery.test.ts src/__tests__/legacy-workspace-restore-recovery.test.ts src/__tests__/chat-completion-delivery.test.ts src/__tests__/agent-conversations.test.ts src/services/native-runtime/native-safe-replacement.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-recovery-queries.ts --baseline-ref=1e731520bb ``` ## Risks - PostgreSQL accepts more UUID spellings than the old text joins. The explicit guard preserves the old lowercase spelling requirement. - Query plans depend on the data distribution. Small task sets can show neutral status-delivery timing. The benchmark is not a production latency claim. - Recovery ownership checks remain conservative. Process, lease, workspace and no-replay rules are unchanged. - No migration, new index, cache, cleanup change or lock removal is included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, shell tools and local code execution. The runtime does not expose a more specific backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ebeac833a6 |
perf(runs): bound history and reduce repeated context reads (#16286)
## Thinking Path > - Paperclip helps operators manage AI agents and inspect their work. > - Run history supplies recent execution records to the board and API clients. > - The list repeats eight reads of a large stored JSON context. > - A caller can also omit the limit and read the complete history. > - This pull request expands context fields once and bounds every list read. > - The benefit is lower database cost and bounded redaction work. ## Linked Issues or Issue Description Refs #958. Related community reports: #13962 and #10631. This is an independent implementation. It does not use their commits or code. It covers the history bounds reported in both. Process memory monitoring from #10631 is outside this change. **What happened?** The run-history query repeatedly extracts fields from large context JSON. Requests without a limit also load every historical run before authorization and redaction. Accumulated staging query statistics showed 271 ms for summary history and another 58 ms for its redaction lookup. **What did you expect to happen?** Normal reads must use a bounded recent history. Context projection must preserve the existing values and privacy behavior with less repeated database work. **Steps to reproduce** Run the disposable benchmark below against the baseline revision. It creates 1,400 synthetic runs with 192 KiB contexts. It compares the exact service SQL and every returned column for both summary and full history. ## What Changed - Expand context fields with one lateral record projection. Preserve non-object legacy contexts and the first-class issue binding. - Enforce a 200-row default and a 1,000-row maximum in the route and service. - Apply the default in the shared UI client, including both agent detail implementations. - Preserve response arrays, authorization, result projection and secret redaction. - Fetch older linked runs by ID in both agent pages. Check company and agent scope, navigation races and visible retry errors. - Update OpenAPI, data-path documentation and capability maps. - Add real PostgreSQL and HTTP regressions plus a reproducible benchmark. ## Verification - Local benchmark: summary history 37.72 ms to 19.34 ms; full history 37.84 ms to 19.01 ms. All 200 complete positional rows match in each comparison. These are synthetic local SQL timings, excluding HTTP and redaction. - History, route, privacy and redaction suites: 160 server tests passed across six files. Twenty-two UI API, selection, run-panel and log-visibility tests passed. Both agent-page variants render an older linked run with an empty history page on desktop and mobile. - `pnpm -r typecheck` and `pnpm build` passed. - Map validation and `git diff --check` passed. - All current-head CI and security checks passed, including the full general and serialized suites, browser shards, Runner verification and build. Greptile reviewed the exact commit at 5/5 with no remaining findings. - The full local sweep began before the final compatibility fixes. It was stopped after the final commits passed complete CI. The focused local suites listed above passed on the final code. Before interruption, the broader local sweep passed 31,743 tests across the general server, UI and several package groups, with no reported failures. The full local command is not claimed as a completed pass. ```sh pnpm --filter @paperclipai/server exec vitest run src/services/heartbeat/run-state.test.ts src/__tests__/heartbeat-list.test.ts src/__tests__/agent-live-run-routes.test.ts src/__tests__/run-secret-redaction.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts pnpm --filter @paperclipai/ui exec vitest run src/api/heartbeats.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-run-history.ts --baseline-ref=1e731520bb ``` ## Risks - Clients that relied on an unbounded array now receive 200 rows by default. Explicit limits allow up to 1,000. This change does not add pagination. - Legacy context conversion must stay identical. Real database regressions cover null, array, scalar, nested values and native issue precedence. - No schema migration or persistent cache is introduced. Redaction and access checks remain active. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, shell tools and local code execution. The runtime does not expose a more specific backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
03fcd3c1fb |
fix(native-runtime): report which close barrier failed (#16298)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agent runs use the Paperclip runner. When a run ends, the controller closes the session. Close must prove that the runner durably suspended before the controller releases its checkpoint. > - If close cannot prove this, the run fails with `provider_transport_failed: runner did not durably suspend before checkpoint`. > - The runner knows which barrier was not proven. It writes that settlement only to the instance run log. > - The Sentry report for the failed run has no settlement evidence. Operators cannot tell a stuck suspend command from an undrained provider, an unreadable provider state, or unsettled tool results. > - This pull request sends that settlement to the run failure report as closed labels and counts. > - The benefit is that the next occurrence shows which barrier failed, so the root cause can be fixed instead of guessed. ## Linked Issues or Issue Description No public issue exists. This describes the bug. **What happened?** Hosted staging native Codex runs failed with `provider_transport_failed: runner did not durably suspend before checkpoint`. This happened on two runs, on 2026-09-30 and 2026-10-14. The second run used a release from 2026-10-14. The Sentry events contain only the error code, run phase, driver kind, and duration. They contain no stack and no settlement data. The native executor calls `reportRunFailure(db, run)` without the caught error, and the runner settlement is never read. **Expected behavior** A failed close report should tell an operator which barrier was not proven: runner suspension, provider drain, provider state, or semantic tool settlement. It must not expose tool identities, digests, provider turn IDs, or unredacted error text. **Steps to reproduce** 1. Run a native Codex session whose close cannot prove durable runner suspension. For example, the runner suspend command does not complete before the close deadline. 2. Observe the Sentry run failure event. Before this change, `run_execution` has no close settlement fields. ## What Changed - `collectRunFailureDiagnostics` accepts a new `nativeCloseFailure` option. It reads only a `native_session_close_unrecoverable` error, directly or up to four `cause` levels deep. - It adds closed labels and counts to `run_execution`: `closeRunnerSuspended`, `closeProviderDrained`, `closeSuspendCommandStatus`, `closeRunnerLifecycle`, `closeRunnerIdentityMatches`, `closeProviderState`, `closeProviderPendingEventCount`, `closeProviderTurnActive`, `closeSemanticResultPersistenceFailed`, `closeSemanticToolsPending`, `closeSemanticToolDeliveriesOpen`, and `closeSemanticResultFailures`. - Values outside the known vocabularies are dropped. Getters that throw are contained. - The native session executor passes the caught error as `nativeCloseFailure` when it reports a terminal run failure. It does not pass the error as `error`, because the raw error text and stack bypass the executor's identity redaction. - `doc/observability.md` documents the new fields and the privacy boundary. ## Verification - `cd server && npx vitest run src/services/__tests__/run-failure-diagnostics.test.ts src/__tests__/run-failure-sentry.test.ts`: 130 tests pass. - New tests check the full label set, a wrapped cause, null suspension and provider states, unrelated error codes, hostile and unknown values, and a missing settlement. They also check that no tool identity, digest, turn ID, or error text leaves the module. - Negative control: with the reader disabled, the new main test fails. - A scoped `tsc` run on `run-failure-diagnostics.ts`, its test, and `run-failure-report.ts` passes. The full server `tsc` and `pnpm build` were not run locally. The sandbox has a 4 GiB memory limit and no `cargo` for the runner binary. CI runs the full checks. ## Risks - Low risk. The change only adds Sentry context fields for one error code. Close, quarantine, retry, and recovery behavior do not change. - Sentry grouping does not change. The fingerprint stays `[errorCode, agentAdapter]`, and no exception chain is added. - The data path is Observability (Sentry error reporting, gated by the operator DSN). It does not change Telemetry. The fields contain only booleans, fixed labels, and counts. The DSN gate is unchanged. - These fields are observations. They do not prove a root cause by themselves. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), run through Claude Code as an agent with tool use (shell, file edits, test execution, Sentry and GitHub connectors). Context window and reasoning mode are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
df4ee06e2f |
feat(skills): select and validate tracked GitHub branches (#16288)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Skills can track a branch in an authorized GitHub repository. > - Sources previously offered only a default-branch repair action. > - People need to select a listed branch or enter a branch that GitHub has not listed. > - The server must validate the selected ref with the caller's authorized connection. > - This pull request connects the approved branch picker to production APIs. > - Installed skills remain usable while the source changes its tracked ref. ## Linked Issues or Issue Description **Subsystem affected** Skills UI, shared API contracts, GitHub source services, and agent API discovery. **Problem or motivation** The default-branch repair action does not let people choose another branch. The settings service also validates the old ref when a caller supplies a new ref other than HEAD. A missing tracked branch can prevent connection repair even when the selected account has repository access. **Proposed solution** Load a bounded branch list with the authorized connection. Mark the default branch. Search listed branches or accept a typed ref. Validate the selected ref before saving. Keep the chosen value after a load error, invalid ref, or concurrent update. Save the selected connection and branch together during connection repair. **Alternatives considered** A default-branch button does not support a named branch. A text field alone does not help people find the repository default or available branches. **Roadmap alignment** This improves the existing Skills source manager. It does not add a new skill import system. **Additional context** Refs #16066 and #16256. This is the production implementation of the approved branch-selection Storybook journeys. ## What Changed - Add an authorized branch-list endpoint and shared response/query types. Register it in agent API discovery. - Validate the requested ref, including the returned commit SHA. Preserve source leases, revision checks, project ownership, skill identities, selections, and installed versions. - Add the searchable branch picker to Sources and connection repair. Allow typed refs during branch discovery and after discovery fails. - Keep save errors in the dialog. Retain the selected branch through a concurrent source update. - Use the production components and API client in Storybook. Add keyboard, touch, loading, retry, invalid-ref, connection-repair, and concurrent-update browser journeys. - Update product documentation and the feature/access maps. ## Verification - Red/green: two new database regressions fail against the previous service and pass against the implementation. - Linux database/service suite: 85 tests passed. This includes source visibility, human/agent authorization, selected connections, immutable versions, and invalid refs. - Rebased branch and review fixes: 94 focused UI, route, branch-validation, and fixture tests passed. Two conflict-message regressions and the stale-retry fixture regression fail before the fix and pass after it. - Storybook build passed. All 26 focused browser journeys passed. - Repository typecheck and build passed. Final UI typecheck passed after the review fixes. - The full local Vitest run was interrupted when the checkout and its dependencies disappeared. It reported missing working-directory/Postgres files and worker startup errors. Its two timed-out tests pass after the checkout was restored. The complete CI suite passed on the final head, `cf5d4c8f0a`: [CI run](https://github.com/paperclipai/paperclip/actions/runs/38571068332). One unchanged workspace timestamp test required a single rerun. Its release-minus-one-millisecond value can equal the original confirmation time. The rerun passed. - Live acceptance passed on implementation head `5dd935bf14` in an isolated macOS arm64 instance with Node 24.21.0 and an authorized private GitHub repository. The UI loaded the actual default branch, saved branch and selected connection together, and preserved the installed skill/version IDs after reload. An invalid typed ref retained the draft and showed the provider error. A valid typed HEAD ref then saved successfully. The owned runtime and database were stopped after the test. Focused unit, Linux database, and browser suites passed again after the error-handling review fixes. - Greptile reviewed final head `cf5d4c8f0a` at 5/5. Both review threads are resolved; the fresh review has no remaining findings. - Token gates and map/catalog checks passed. - Review entry points: `Skills / Project repository skills / 03 Branch selection components` and `04 Branch selection journeys`. Run `pnpm storybook` to inspect them. - Browser command: `pnpm exec playwright test --config tests/storybook-visual/playwright.config.ts tests/storybook-visual/skill-source-branch-selection.spec.ts tests/storybook-visual/project-repository-skills.spec.ts`. - Manual check: open an independent GitHub source, choose a new branch, select the default or type a ref, save, and reload. Try an invalid ref and confirm the selection remains in the dialog. For a project-bound source, change its ref in project settings. ## Risks - GitHub discovery is bounded to 100 listed branches. The default branch is always included. Typed refs remain available. - A source can change while the dialog is open. The API rejects a stale revision and the UI keeps the draft for retry. - A ref change schedules the existing refresh pipeline. Installed versions remain available until a successful refresh imports new content. - No database migration or credential storage change is required. Existing HEAD requests remain supported. ## Model Used OpenAI Codex, GPT-6 family. The serving model's exact identifier and context-window size are not exposed in this session. Used reasoning, repository tools, code execution, and browser tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d4892d4761 |
fix(observability): sample native runs independently of triggers (#16285)
Start native background run traces under the operator's root sampling policy and link them to valid trigger contexts. Preserve child parenting and silent unconfigured tracing. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f69c6a8a35 |
fix(github): retry and recover review publication (#16284)
## Thinking Path
> - Paperclip manages agent work and its published results.
> - GitHub review bots publish task-bound assessments through governed
tools.
> - A review can survive a caller timeout because its publication intent
is durable.
> - A transient provider failure still returned a failed receipt before
recovery.
> - Gateway replay could return an empty cached result, and publication
retries had no dedicated scheduler deadline.
> - This pull request adds bounded retries, receipt recovery, and an
independent durable worker.
> - A temporary GitHub failure can recover without another review or
duplicate publication.
## Linked Issues or Issue Description
Refs #16275. That change increased the GitHub tool deadline. This change
repairs retry and recovery behavior.
**What happened?**
A one-off GitHub HTTP 503 or 429 left the caller with a failed
publication receipt. The saved retry did not have its own scheduler
worker. An identical gateway call after a timeout could return an empty
cached result instead of the saved publication receipt.
**Expected behavior**
Temporary failures should retry under current authority. Pending work
should resume at its saved deadline after restart. Exact retries should
return the current receipt. Lost responses must not create duplicate
comments, findings, or checks.
**Steps to reproduce**
1. Admit an advisory PR review request and begin its assessment.
2. Return HTTP 503 or 429 for the first summary-comment PATCH, then let
GitHub succeed.
3. Submit the assessment through the run-scoped tool gateway.
4. On master, the result says publication failed. With this change, the
same call returns the processed receipt and neutral check.
5. For receipt recovery, use a shorter caller deadline and repeat the
exact call after publication finishes.
**Paperclip version or commit**
Reproduced on master at `6cb6ac3b8e40cb90d734ba115a2f22f850ff9477`.
## What Changed
- Classify transient transport, server, and rate-limit failures. Honor
Retry-After and primary reset deadlines. Stop retries for permanent
permission and validation errors.
- Retry safe reads and installation-token issuance within a bounded call
window. Retry publication writes through the existing outbox and
provider receipt checks.
- Wait for up to three publication attempts within a 45-second retry
window. Keep the durable eight-failure cap and enforce saved deadlines
on explicit retries.
- Register an independent GitHub publication worker with
transaction-owned wake notifications, startup recovery, and per-PR lease
deadlines.
- Recover exact comment, progress, and assessment gateway calls from
their stored publication. Require the same company, agent, task, run,
connection, tool, arguments, and current policy. Clear stale timeout
errors after recovery. Approved exact retries read the current receipt
through the existing signed replay path; the worker retains the verified
approval for deferred writes and still rejects revocation.
- Add regression coverage and update the connector guide and
feature/access maps.
## Verification
- Red: both one-off HTTP 503 and 429 regressions failed on master with a
failed publication receipt.
- Green: the same tests passed after the change.
- GitHub review workflows: 133 covered against isolated PostgreSQL 18.
The final combined run passed 132 and hit one fifteen-second timeout
under concurrent build load. All three variants of that workflow passed
in isolation. The earlier combined run passed all 131 workflows before
the two approval regressions were added.
- Retry policy and delivery coordinator tests: 35 passed.
- Additional focused checks passed for the actual eight-failure cap,
exact-argument conflicts, policy revocation, and timeout receipt
recovery. The approved deferred-publication and revocation regressions
failed before the review repair and passed after it, with one approval
request preserved.
- `pnpm -r typecheck`: passed. `pnpm build`: passed. `node
scripts/check-maps.mjs`: passed.
- The broad local `pnpm test:run` path remains blocked by the host's
exhausted PostgreSQL shared-memory slots. The changed database-backed
workflows use an isolated Docker database instead. All Paperclip CI jobs
passed at
|
||
|
DottaandPaperclip
|
7c016c049b |
perf(workspaces): wake terminal cleanup from committed changes (#16282)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use execution workspaces to retain their files and Git branches. > - A reaper closes eligible workspaces after their task tree is terminal. > - The reaper scanned every heartbeat tick and read Git before cheap eligibility checks. > - Permanent scans also query idle databases when no cleanup work exists. > - This pull request wakes cleanup after transaction settlement and restores bounded work at startup. > - The benefit is fewer scans while fresh cleanup guards continue to protect retained work. ## Linked Issues or Issue Description Related work: Refs #13619, Refs #11753. Those PRs bound scans or back off unchanged candidates. This PR uses the transaction work coordinator to remove the permanent reaper tick. It overlaps their early terminal-state filtering but does not change their runtime restart behavior or introduce a second Git scheduler cache. **What existing behavior does this improve?** Terminal workspace cleanup and Git inspection for workspace detail reads. **Subsystem affected** Server orchestration, execution workspaces, and idle sleep admission. **Current behavior** The heartbeat tick scans workspace candidates every thirty seconds. It runs Git before rejecting nonterminal trees, cooldowns, and active runs. Display inspections expire after five seconds. **Proposed behavior** Recover workspace rows in bounded startup pages. Wake after relevant issue, workspace, and pull-request work-product transactions settle. Schedule the terminal tree cooldown or a five-minute eligibility retry. Back off unchanged candidates across unrelated wakes, and invalidate their backoff after relevant committed changes. Park empty and nonterminal-only lanes. Reuse display inspections for thirty seconds. Read fresh Git before destructive cleanup. **Reason and benefit** Avoid recurring database and Git work when no cleanup is possible. Retain transaction admission holds, restart recovery, and fresh deletion checks. Retention eligibility can wait for the next host wake; active and failed recovery work still blocks sleep. **Breaking changes** No schema or API shape changes. Display Git state can lag by thirty seconds. Cleanup eligibility retries can take five minutes. ## What Changed - Add a terminal-workspace worker to the existing transaction work coordinator and scheduler. - Signal relevant writes before mutation on the owning transaction. Outer transaction settlement wakes cleanup. - Derive cooldown deadlines from the full company-scoped terminal task tree. Continue bounded recovery pages without a permanent sweep. - Bound unchanged candidate inspections to five minutes, retain at most 256 entries, and invalidate affected entries with scoped transaction hints. Repeated wakes preserve the earliest inspection deadline. - Collect deadlines from task trees already read by bounded pages. Restore the cursor after a failed page and rescan commits received during a rotation. - Check terminal state, cooldown, and active runs before Git. Preserve lifecycle locks, generation fences, and fresh final checks. - Keep failed or incomplete recovery visible to idle admission. Preserve the passive retention exception. - Extend the bounded display cache to thirty seconds and document the scheduling and wake limits. ## Verification - Focused workspace, coordinator, cache, and local sleep suites: 111 tests passed with real local PostgreSQL and Git worktrees where applicable. - Final worker, display cache, and local sleep suites: 28 tests passed. - The regressions prove three ineligible workspaces invoke zero Git scans, cached clean display state cannot remove new untracked files, and real producer notifications wait for outer transaction settlement. - `pnpm -r typecheck` and `pnpm build` passed locally. Map and module-boundary checks passed. - Work-product and sleep-safety suites: 221 tests passed. Reopen and issue-tree control suites: 46 tests passed. Final backoff, workspace, worker, and work-product suites: 87 tests passed. - Final workspace and worker suites: 81 tests passed. Deadline selection performs zero database reads. A failed full page retries its unread terminal workspace before recovery can park. - Full CI coverage passed on head `2c397970f7`: general tests, serialized server suites, runner verification, E2E shards, typecheck, build, and the final verification gate. The duplicate serial local `pnpm test:run` was stopped after full CI coverage passed; it is not reported as a completed local pass. - Greptile: 5/5 with both findings resolved. The security scan also passed. No failing or pending PR checks remain. ## Risks - Display state can be stale for thirty seconds. Destructive guards bypass the display cache. - Undelivered or otherwise ineligible terminal work retries after five minutes instead of each heartbeat tick. - Transaction signals cover the owning `createDb` scope. Direct SQL and independent server processes require startup recovery. - Local cooldown timers are maintenance hints. They do not establish an external wake alarm or whole-instance sleep safety. Existing blockers remain in force; failed bounded recovery keeps the host awake. ## Model Used OpenAI GPT-6, model ID `gpt-6`, through Codex. Reasoning, repository inspection, shell execution, and browser tools assisted the change. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1e731520bb |
fix(mcp): deduplicate discovered tool descriptors (#16281)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive governed app tools through the MCP gateway. > - GitHub review tasks can contain a PR discussion and inline finding threads. > - Discovery repeated each tool for every conversation linked to the task. > - Native tool registration treated these repeated descriptors as a fatal collision. > - This pull request lists GitHub tools once and deduplicates identical descriptors across providers. > - Agents can continue work while conflicting definitions and normal tool permissions remain enforced. ## Linked Issues or Issue Description **What happened?** A follow-up run on a GitHub review task failed with `assigned_mcp_tool_name_collision` before the provider started. The task had a main conversation and an inline finding thread. Discovery returned the same seven tools twice. Repeated descriptors from other providers could cause the same failure. **Expected behavior** Discovery exposes each identical tool once. Separate connection names remain separate tools. Conflicting definitions remain an error. Every call still passes the gateway's policy, approval, revocation, and audit checks. **Steps to reproduce** 1. Run a GitHub review and publish an inline finding. 2. Send a follow-up request on the same PR task. 3. Inspect the task-bound bot catalog and the next native startup. 4. Separately, return two identical descriptors from an assigned app or plugin catalog. Related: #13717 added the GitHub review workflow. ## What Changed - Use a distinct GitHub discovery query across conversations that share an endpoint and task. Keep both thread mappings. - Deduplicate identical gateway descriptors by their full gateway name across providers. Use this rule for MCP listings, plugin listings, assigned-tool startup, and fresh assigned-tool discovery. - Reject conflicting descriptors with `tool_name_collision`. Keep the projected-name collision guard. - Cover GitHub inline-thread discovery, repeated plugin/app tools, direct and on-demand catalogs, revocation, conflicting bindings, and separate connection names. - Update the execution contract, GitHub documentation, and agent access map. ## Verification - Regression baseline: six new assertions failed on the original implementation. GitHub returned fourteen entries, plugin discovery repeated a tool, and native startup failed on repeated app tools. - Focused verification passed: 272 GitHub cases, all 29 assigned-tool cases, and all 88 gateway acceptance cases. - `pnpm -r typecheck` passed. - `pnpm build` passed. - `node scripts/check-maps.mjs` passed. - Current-head CI passed: 55 checks passed, two inapplicable checks skipped. Greptile scored the current head 5/5 with no unresolved threads. - Full local `pnpm test:run` encountered a 15-second timeout in the existing bearer-auth throttling test, before tool discovery. That test passed in the separate 88-case gateway run and on one isolated rerun. The duplicate broad local run was then stopped; the complete current-head CI matrix passed. ## Risks - Deduplication keeps the first identical descriptor and preserves listing order. Same-name descriptors with different definitions fail with HTTP 409 rather than selecting an ambiguous binding. - The GitHub query changes result cardinality. It retains the existing task/company/agent filters and leaves conversation rows intact. - No schema migration or credential change. Live staging recovery requires deployment and a new authorized run. ## Model Used OpenAI Codex, based on GPT-6, with code inspection, editing, browser investigation, and test execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e9b6c4d44c |
perf: remove history scans from task and chat queries (#16278)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task reads and external chat notices must remain responsive as run history grows. > - The chat notice sweep casts indexed UUID columns to text. > - Run redaction and task provenance filter large JSON contexts with expressions that lack matching indexes. > - These queries scan history and read large stored contexts on common requests. > - This pull request adapts two existing fixes to current master and adds the missing provenance index. > - The benefit is less database work with the same query results and access rules. ## Linked Issues or Issue Description Refs #15927. This change adapts that chat sweep fix to the extracted inbound notice module and retains the contributor credit. Refs #12499. This change adds that missing nested-context index through a current, concurrent migration. It retains the contributor credit. This combined change replaces those older layouts for the measured query paths. It also covers task creation provenance, which those pull requests do not address. **What happened?** The chat notice sweep scans wakeup history. Issue redaction scans run contexts because only one branch of its JSON condition has an index. The task creation-source filter also scans contexts because its native and legacy ID expression lacks an index. **Expected behavior** The queries use indexed lookups and return the same rows. Company access, task origin precedence, secret redaction, and chat notice ordering remain the same. **Steps to reproduce** Run `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-slow-queries.ts --baseline-ref=f8af817f8d`. The benchmark creates disposable PostgreSQL data and compares the old query with the current query on the same fixture. ## What Changed - Compare chat UUID columns directly with safely parsed payload IDs. Reject malformed and noncanonical IDs without a cast failure. - Add the nested `paperclipIssue.id` run-context index and the exact native/legacy task-provenance expression index. - Build both indexes concurrently. Register the migration for top-level execution and invalid-index retry repair. - Extend chat payload and migration regression coverage. - Add a repeatable synthetic SQL benchmark and document its scope and command. - Keep the five newest schema snapshots. ## Verification - Focused local checks: 76 selected tests passed in five files. These cover FIFO notices, malformed IDs, conversation state, redaction, provenance, and company boundaries. - Disposable PostgreSQL 18.1 fixture: 335 tasks, 1,400 runs, 13,000 wakeups, 377 actions, and 192 KiB synthetic contexts. - Five warm samples per query, after two warmups. Every returned SQL column matched before and after, compared as positional tuples so duplicate unnamed projections are retained. | Query | Before, median | After, median | Improvement | | --- | ---: | ---: | ---: | | Chat notice candidates | 26.21 ms | 6.43 ms | 4.1× | | Task creation-source list | 159.69 ms | 6.51 ms | 24.5× | | Issue redaction contexts | 131.88 ms | 0.34 ms | 389.1× | The run-history context projection returned the same 200 rows and kept the same index plan and buffer counts. Its timings were noisy (37.30 ms before and 56.78 ms after in the last run; 34.25 ms and 34.14 ms in the previous run). No improvement is claimed for this control. Its repeated JSON projection remains a separate optimization opportunity. The populated plans use the wakeup primary key, the provenance expression index, and a BitmapOr over both redaction indexes. These are local SQL measurements. They do not prove an equivalent change in hosted API latency. - `pnpm -r typecheck` and `pnpm build` passed locally. - The full local `pnpm test:run` was started and then interrupted (exit 130) when the Mac reached its PostgreSQL shared-memory limit. It did not complete and is not reported as passed. The final SQL benchmark passed after its disposable database resources were released. The full general and serialized test suites passed in CI. - All CI checks for final head `193acca6abaa992acad12f4b631bd31dd3234641` passed, including full Vitest coverage, E2E checks, typecheck, build, migration order, and lifecycle boundaries. - Greptile reviewed that exact head at 5/5 with no actionable comments. ## Risks - Both indexes add storage and write cost. Initial builds read stored run contexts and can take time on large histories. - Concurrent index builds avoid holding a write-blocking table lock for the whole build. The existing migration retry path removes invalid partial indexes before retrying. - The chat parser accepts only canonical lowercase UUID text. This preserves the prior text comparison, including rejection of uppercase and malformed values. - Staging has not been migrated or deployed. Local PostgreSQL and synthetic data do not reproduce hosted load or network latency. ## Model Used - OpenAI Codex, based on GPT-6. The exact serving model ID and context window were not exposed in this session. Used reasoning, repository tools, browser inspection, and local code execution. The adapted fixes retain their original contributor credits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c80d93e14e |
fix(observability): time execution configuration steps (#16236)
Record optional execution-configuration timings before provider dispatch, with success, failure, sampling, cleanup, and silent-unconfigured controls. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6cb6ac3b8e |
fix(github): allow review publication to finish before tool timeout (#16275)
## Thinking Path > - Paperclip manages agent work and its published results. > - GitHub review bots publish task-scoped assessments through the tool gateway. > - A submission saves its assessment and publication intent before it waits for GitHub. > - The gateway's ten-second default can expire while those provider requests continue. > - GitHub can then show a completed review while the agent receives a timeout and blocks its task. > - This pull request gives GitHub bot tools the existing sixty-second maximum by default. > - The caller can receive the confirmed publication result without manual task repair for ordinary slower responses. ## Linked Issues or Issue Description **What happened?** A GitHub bot's `submit_review` call returned `504 tool_timeout`. Its saved assessment continued to publish a summary and completed check after the caller stopped waiting. The agent treated publication as unconfirmed and blocked the task. **Expected behavior** A valid GitHub response that takes longer than ten seconds should return its confirmed publication receipt within the gateway's supported deadline. **Steps to reproduce** 1. Configure a report-only GitHub review bot and admit a PR review request. 2. Delay the summary-comment PATCH response by eleven seconds. 3. Submit a complete assessment through a real run-scoped tool gateway. 4. Before this fix, the caller gets a timeout while the durable action publishes one summary and a neutral check. 5. With this fix, the caller gets `completed`, `processed`, and the assessment score. Retrying does not add another comment. **Paperclip version or commit** Reproduced from master at `2c17d3ca52a7840f280f1dd87975a2ad77de3b35`. Related changes: Refs #16259 and #16263. Those changes repaired connection recovery. This change addresses the subsequent submission deadline. ## What Changed - Use a sixty-second default for `paperclip_github_chat` tools in run execution and board tool tests. - Preserve explicit shorter deadlines and the existing maximum. - Add PostgreSQL-backed regressions through the real gateway for delayed publication, explicit deadlines, neutral checks, and duplicate prevention. - Document the deadline and the durable publication behavior. ## Verification - Red: the eleven-second publication regression failed with `ToolGatewayHttpError(504, tool_timeout)` after the summary and neutral check were published. - Green: the same regression passed after the repair. - GitHub review workflow suite: 122 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: running; results will be recorded before handoff. - Live local verification is running against a dedicated test App on [the disposable test PR](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/pull/9). A test-only preload delays one real GitHub PATCH response by eleven seconds. It is not part of this change. ## Risks GitHub bot calls can occupy their execution slot for up to sixty seconds by default. Calls that exceed that limit can still time out. Durable publication may continue after any caller timeout, so a timeout alone is not proof of a failed write. This change does not add automatic task reconciliation for arbitrarily slow or unavailable providers. No schema or credential changes are required. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2c17d3ca52 |
fix(github): recover webhook verification in attention state (#16263)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review bots require signed webhook delivery before activation. > - Cloud can deduplicate an old ping after a connection repair. > - A fresh event could verify active connections, but an attention-state connection discarded it before authentication. > - This pull request allows signed delivery verification while the connection needs attention. > - It discards ordinary work until the manager completes the existing reconnect and activation checks. ## Linked Issues or Issue Description Refs #16259. Related: #16138 addresses the separate classification of local authorization failures. **What happened?** Live verification of #16259 found that a GitHub App already marked as needing attention remained stuck waiting for a signed ping. The Cloud relay had already delivered that ping. New signed events were discarded by the attention-state ingress gate before they could verify delivery. **Expected behavior** A fresh valid event from the bound installation verifies delivery. It does not activate the connection, change repository access, or start work. The manager can then complete the normal reconnect flow. **Steps to reproduce** 1. Put an existing App connection into attention state with webhook verification cleared after repair. 2. Deliver a fresh signed comment event for its bound installation. 3. Continue setup. Before this fix, webhook verification remains missing and reconnect cannot proceed. ## What Changed - Use a content-free verification path for signed ordinary GitHub events while the connection needs attention. - Allow a fresh signed event from the same installation to verify webhook delivery. - Verify delivery even from disabled or unavailable repositories. Discard the event before content storage, resource reconciliation, or runtime creation. - Keep newly verified connections in the provider-setup step. Later installation events cannot activate them before reconnect. Preserve normal installation recovery for existing qualified connections. - Extend recovery regressions to cover attention state, foreign installations, invalid signatures, missing installation IDs, unchanged resources, no wakeups, and successful reconnect after verification. - Document the recovery behavior. ## Verification - The targeted regression reproduced the attention-state failure before the change. - All ten recovery cases pass after the change. The tests cover disabled and unavailable repositories, content-free verification, and installation events after verification. - Server typecheck passed. - All 260 GitHub integration tests passed after the review fixes. Final server typecheck also passed. - `pnpm -r typecheck` and `pnpm build` passed. - The full local suite is running. One unrelated workspace-runtime conflict-reference test failed and then passed on its isolated rerun. Final totals and CI/review outcomes will be updated before merge. - Live verification will use the existing advisory review bot after deployment. ## Risks - Attention-state ordinary events authenticate under the current credential lease without creating a durable content record. They return before resource reconciliation or runtime creation. - Verification is still bound to the current credential lease, runtime generation, valid signature, and installation ID. It does not activate the bot or replace normal setup checks. - Paused, revoked, archived, and unconfigured connections retain their existing ingress rules. No schema changes or permission expansion. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c36281d639 |
fix(skills): repair missing branches and show grouped import progress (#16256)
## Thinking Path > - Paperclip lets people manage AI agents and their reusable skills. > - GitHub skill sources must keep their installed content usable during updates. > - A source can track a branch that no longer exists. > - The UI treats that GitHub error as a connection problem, so saving a valid account does not resolve it. > - Project workspaces also show the same repository several times, and long imports give little feedback. > - This pull request adds explicit branch repair, groups repository imports, and shows import progress. > - People can repair the actual problem and see when their import is working. ## Linked Issues or Issue Description Refs #16066 and #14713. This fixes the Sources experience introduced by those changes. Related adapter-source PRs #14340 and #14341 cover runtime mounting, not this source-management flow. No overlapping source-management fix was found. **What happened?** An installed source tracked `main` while its GitHub repository used `master`. GitHub returned HTTP 422. The UI showed a general read error and asked for a new connection after a valid account was saved. The project repository list repeated the same repository/ref for each project. Import had no spinner or explanation of queued work. **Expected behavior** Identify an unavailable branch and offer an explicit repair. Retain the selected connection and installed versions. Show one repository/ref row with its projects. Give immediate feedback and show the queued and active import states. **Steps to reproduce** 1. Install a source with an immutable commit and a saved tracked branch that does not exist. 2. Refresh it, or try to save a valid GitHub connection. 3. Add the same repository/ref to two legacy project workspaces and open Sources. 4. Import that repository and observe the row until installation completes. ## What Changed - Distinguish a missing GitHub revision from access denial and temporary read errors. - Validate both repository identity and the tracked revision when saving a connection. - Offer Use default branch on independent sources and in connection repair. Keep `HEAD` as the default-branch alias after refresh. Preserve skill IDs, versions and selections. - Keep active project refs under project configuration. Reject source-level changes that would alter their tracking. Recheck the ref under the source lock when project attachment races with branch repair. - Group matching repository/ref workspaces. Adopt eligible workspaces and skip pending workspaces. Keep remaining project links inside the existing source card. Allow remaining imports while another workspace is pending. - Show a spinner, busy state and queued/importing text. Keep the saved authorized account selected in connection repair. - Update shared contracts, feature/access maps, database notes and production-component Storybook journeys. ## Verification - Red/green: three local regressions failed on the original code: missing-ref classification, duplicate repository rows, and branch repair instead of another connection prompt. The changed UI/reader tests pass after the fixes. - Live local app with a real authorized GitHub credential: reproduced HTTP 422 for the unavailable branch; used the UI to restore default-branch tracking; confirmed the same connection, skill ID and single immutable version remained. - Live local grouped Paperclip import: one source, two project associations and 52 installed skills. Sources survived reload. No staging data was changed. - All 16 project repository Storybook browser journeys pass on the final head, including keyboard focus recovery after branch repair. These include branch repair, grouped imports, connection selection, recovery, removal and responsive layouts. - Repository typecheck, production build, Storybook build, token gates and 28 map/catalog tests passed. Final UI typecheck and build also passed after grouping remaining project links. - Linux qualification: all 94 focused source, project import, GitHub reader and UI tests pass with supported Node 24. No database tests were skipped. Both review findings failed before their fixes and pass after them. - The full local `pnpm test:run` attempt was stopped after macOS database suites exhausted the host shared-memory limit. The complete Linux CI matrix is the full-suite gate. The complete Linux CI matrix passes for head `f886e3ac073ec1c6f051fd6429496234f5143694`. - Greptile rates head `f886e3ac073ec1c6f051fd6429496234f5143694` 5/5. Both findings are fixed and all review threads are resolved. - Storybook: `skills-project-repository-skills-02-journeys--missing-branch` and `--duplicate-projects`. ## Risks - Default-branch repair changes the tracked branch only after an explicit user action. Existing execution version pins stay intact. - A selected connection now needs access to the tracked revision, not only repository metadata. - Grouping is by repository and ref. Project access and provenance remain separate and are checked by the existing services. - Legacy source errors contain safe text. The repair-action classifier maps known errors and treats other failures as retryable. No credentials or provider response bodies are returned. - No schema migration, workflow change, lockfile change or staging deployment. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model identifier and context-window size are not exposed in this session. Tool-assisted implementation, code execution and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a8b652b085 |
fix(github): preserve review delivery through reconnect (#16259)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review bots turn authorized repository events into agent tasks and publish the result on the pull request. > - Cloud Apps send callbacks through a relay, but the direct callback recovery worker reported that URL as broken. > - Reconnecting the same App then prevented a new authorized publication from updating its existing working comment. > - Report-only runs could also produce an action-required check when no complete assessment was available. > - This pull request preserves comment identity across reconnects, verifies fresh signed delivery after repair, and keeps report-only results neutral. > - Users can receive the review without changing repository access or making an advisory bot block their pull request. ## Linked Issues or Issue Description **What happened?** A review could be saved in Paperclip while GitHub kept showing “Working on this…”. A same-App reconnect changed the runtime generation and the existing response receipt then failed with “GitHub response binding changed”. Cloud callback recovery also compared the relay URL with a direct tenant URL. An incomplete or failed report-only run could produce an action-required check. **Expected behavior** A fresh authorized attempt updates the same App-owned comment after reconnect. Stale attempts and revoked people remain denied. Cloud relay delivery does not produce a direct-callback mismatch. Report-only checks remain neutral and explain incomplete work honestly. **Steps to reproduce** 1. Accept a GitHub request and publish its working comment. 2. Reconnect the same App before the agent publishes its result. 3. Submit the result and inspect the working comment and check. 4. Separately, let a report-only run end without an assessment and inspect its check conclusion. Related: #16138 addresses the separate classification of local authorization errors as provider credential failures. This change fixes the response binding and report-only paths themselves. ## What Changed - Bind durable response comments to the App and agent. Keep per-attempt runtime generation checks and provider comment ownership checks, including legacy receipts. - Exclude Cloud-managed Apps from the direct webhook recovery scan and its stale callback-mismatch warning. The Cloud inbox owns relay retries. - Let a fresh authenticated event from the bound installation verify webhook delivery after repair. Keep signature, runtime epoch, installation, and credential lease checks. - Keep report-only assessment and terminal-run checks neutral. Preserve incomplete/error text and gated-review behavior. - Add integration regressions for reconnect races, legacy comments, failed runs, Cloud recovery, and foreign or forged webhook traffic. Update connector documentation. - Correct the assigned-tool search guidance to name the actual input fields (`offset` and `schemaOffset`) and show valid arguments. The failed automatic review had an input-schema rejection before tool execution; this clarifies recovery without expanding tool access. ## Verification - `pnpm -r typecheck` passed. - GitHub policy and integration suite: 291 tests passed (`vitest run server/src/services/chat-github-review-policy.test.ts server/src/__tests__/chat-channels.integration.test.ts -t 'GitHub|github'`). - The report-only policy regression failed before the fix and passed after it. - `pnpm build` passed. - Assigned tool discovery: 24 tests passed. Its guidance now maps returned pagination offsets to the correct input fields and tells the model to omit unused optional fields. - Full `pnpm test:run` is running. It encountered a 15-second timeout in the unrelated authenticated Workspace calendar-write test. That test passed on one isolated rerun. - After the review-requested module extraction, server typecheck and both direct/Cloud recovery activity integration checks passed. - All 55 non-skipped CI checks passed for `32e690279d847e44a6946b6c7cfb110fb11aff79`. Greptile scored that commit 5/5 with no unresolved findings. Live staging publication verification follows deployment. ## Risks - A durable response can now survive a same-App reconnect. Fresh authorization, runtime generation checks for each publication attempt, App ownership, and the publication lease still fence writes. - Report-only failures intentionally become neutral. The check still reports the failure; it does not claim a complete assessment. - Cloud-managed callbacks rely on their existing durable relay inbox. The direct callback recovery worker remains unchanged for direct installations. - No schema migration or permission expansion. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40ecacff4c |
fix(voice): accept authenticated Cloud callers (#16254)
## Thinking Path > - Paperclip manages agent work through company-scoped tasks. > - Speko voice connections bind a caller to that work. > - Private voice routes must recognize an authenticated human caller. > - Cloud sign-in uses a trusted Cloud actor, rather than a local session actor. > - The voice caller check rejected that existing authentication type. > - This pull request accepts authenticated Cloud callers and retains the current access checks. > - Cloud users can load call history and start private voice sessions. ## Linked Issues or Issue Description Refs #15727, #15728, #15729. **What happened?** A signed-in Cloud user sees “Sign in to start a private voice conversation” when the Speko connection loads call activity. The same check prevents the user from starting a private voice session. **Expected behavior** The trusted Cloud user identity should have the same voice caller access as a local authenticated session. The current company and user identity checks must still apply. **Steps to reproduce** 1. Sign in to a Cloud workspace with experimental chat connectors enabled. 2. Connect a Speko persona to an existing agent. 3. Open its call activity, or start a browser voice session. 4. Observe the sign-in rejection despite the authenticated workspace session. **Paperclip version or commit** Reproduced on the voice route in master at |
||
|
DottaandPaperclip
|
75189b8762 |
refactor(server): isolate channel runtime admission and maintenance (#16235)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn provider messages into shared task workflows. > - The main service still implements provider details beside common authorization and persistence. > - These details belong in the existing channel folders. > - This pull request moves Discord runtime command capture and root receipts, Speko runtime and authority, and GitHub maintenance scheduling behind explicit typed functions. > - The service keeps its shared workflow and selects the provider implementation directly. ## Linked Issues or Issue Description Refs: #16192. Stack part 5/5. Depends on #16233. Base: `master` (parent PR merged). **What existing behavior does this improve?** The ownership and readability of provider behavior in the chat channel service. **Current behavior** The service implements Discord runtime command capture and root receipts, Speko runtime and authority, and GitHub maintenance scheduling inline. **Proposed behavior** Channel modules own these details. The core calls their typed functions. Runtime and database effects retain their current order. **Reason and benefit** Maintainers can find channel behavior in its channel folder without following a generic provider framework. **Breaking changes** None intended. Public service entry points and durable data shapes stay compatible. ## What Changed - Extract Discord runtime command capture and root receipts, Speko runtime and authority, and GitHub maintenance scheduling into the existing channel folders. - Keep shared company scope, authorization, and transaction ownership in their current workflow. - Reduce `chat-channels.ts` from 3715 to 3649 lines. ## Verification - The same 167 selected chat-runtime cases and 87 voice ownership cases passed before and after extraction. Full workspace typecheck and build passed. Token parity checks passed for six moved control-flow blocks. - Server typecheck passed. - `node scripts/check-module-boundaries.mjs` passed. - Full workspace typecheck and build passed on the cumulative stack. All current-head CI gates passed. Greptile completed at 5/5 with no findings. - The full local `pnpm test:run` command exited with three database fixture-startup hook timeouts. It reported 19,529 passing tests and 250 skipped tests. No assertion failed. Its later test groups did not run because the driver stops on the first failing group. - The timed-out files were `environment-custom-images-service.test.ts`, `generic-mcp-connection.test.ts`, and `modules/wake-queue/adapters/postgres.test.ts`. All 166 tests in these files passed in isolation on both baseline `edcfcaf3f6` and stack head `faaca26d46`, with existing timeouts and assertions. The tests and database-startup helper are unchanged. - Provider fixtures prove application integration. They do not prove live third-party execution. ## Risks - Incorrect inputs or changed await order could affect provider handling and recovery. - Existing integration tests exercise the assembled service before and after extraction. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c1994056f9 |
refactor(server): isolate channel registration and setup services (#16233)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn provider messages into shared task workflows. > - The main service still implements provider details beside common authorization and persistence. > - These details belong in the existing channel folders. > - This pull request moves Slack registration, account linking, test-message setup, GitHub setup progress, and Telegram webhook URL restrictions behind explicit typed functions. > - The service keeps its shared workflow and selects the provider implementation directly. ## Linked Issues or Issue Description Refs: #16192. Stack part 4/5. Depends on #16232. Base: `master` (parent PR merged). **What existing behavior does this improve?** The ownership and readability of provider behavior in the chat channel service. **Current behavior** The service implements Slack registration, account linking, test-message setup, GitHub setup progress, and Telegram webhook URL restrictions inline. **Proposed behavior** Channel modules own these details. The core calls their typed functions. Runtime and database effects retain their current order. **Reason and benefit** Maintainers can find channel behavior in its channel folder without following a generic provider framework. **Breaking changes** None intended. Public service entry points and durable data shapes stay compatible. ## What Changed - Extract Slack registration, account linking, test-message setup, GitHub setup progress, and Telegram webhook URL restrictions into the existing channel folders. - Keep shared company scope, authorization, and transaction ownership in their current workflow. - Reduce `chat-channels.ts` from 3813 to 3715 lines. ## Verification - The same 97 selected setup integration cases passed before and after extraction. The cases cover Slack installer linking and welcome eligibility, GitHub registration, and Telegram webhook restrictions. - Server typecheck passed. - `node scripts/check-module-boundaries.mjs` passed. - Full workspace typecheck and build passed on the cumulative stack. All current-head CI gates passed. Greptile completed at 5/5 with no findings. - The full local `pnpm test:run` command exited with three database fixture-startup hook timeouts. It reported 19,529 passing tests and 250 skipped tests. No assertion failed. Its later test groups did not run because the driver stops on the first failing group. - The timed-out files were `environment-custom-images-service.test.ts`, `generic-mcp-connection.test.ts`, and `modules/wake-queue/adapters/postgres.test.ts`. All 166 tests in these files passed in isolation on both baseline `edcfcaf3f6` and stack head `faaca26d46`, with existing timeouts and assertions. The tests and database-startup helper are unchanged. - Provider fixtures prove application integration. They do not prove live third-party execution. ## Risks - Incorrect inputs or changed await order could affect provider handling and recovery. - Existing integration tests exercise the assembled service before and after extraction. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
875943ac83 |
refactor(server): isolate provider webhook lifecycle processing (#16232)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn provider messages into shared task workflows. > - The main service still implements provider details beside common authorization and persistence. > - These details belong in the existing channel folders. > - This pull request moves GitHub webhook runtime deadlines and Slack lifecycle processing behind explicit typed functions. > - The service keeps its shared workflow and selects the provider implementation directly. ## Linked Issues or Issue Description Refs: #16192. Stack part 3/5. Depends on #16231. Base: `master` (parent PR merged). **What existing behavior does this improve?** The ownership and readability of provider behavior in the chat channel service. **Current behavior** The service implements GitHub webhook runtime deadlines and Slack lifecycle processing inline. **Proposed behavior** Channel modules own these details. The core calls their typed functions. Runtime and database effects retain their current order. **Reason and benefit** Maintainers can find channel behavior in its channel folder without following a generic provider framework. **Breaking changes** None intended. Public service entry points and durable data shapes stay compatible. ## What Changed - Extract GitHub webhook runtime deadlines and Slack lifecycle processing into the existing channel folders. - Keep shared company scope, authorization, and transaction ownership in their current workflow. - Reduce `chat-channels.ts` from 3906 to 3813 lines. ## Verification - The same 61 selected webhook/lifecycle integration cases passed before and after extraction. They cover cold GitHub runtime startup, signed ingress, installation recovery, Slack lifecycle effects, and Telegram/Teams edits. - Server typecheck passed. - `node scripts/check-module-boundaries.mjs` passed. - Full workspace typecheck and build passed on the cumulative stack. All current-head CI gates passed. Greptile completed at 5/5 with no findings. - The full local `pnpm test:run` command exited with three database fixture-startup hook timeouts. It reported 19,529 passing tests and 250 skipped tests. No assertion failed. Its later test groups did not run because the driver stops on the first failing group. - The timed-out files were `environment-custom-images-service.test.ts`, `generic-mcp-connection.test.ts`, and `modules/wake-queue/adapters/postgres.test.ts`. All 166 tests in these files passed in isolation on both baseline `edcfcaf3f6` and stack head `faaca26d46`, with existing timeouts and assertions. The tests and database-startup helper are unchanged. - Provider fixtures prove application integration. They do not prove live third-party execution. ## Risks - Incorrect inputs or changed await order could affect provider handling and recovery. - Existing integration tests exercise the assembled service before and after extraction. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b8ffeb973a |
feat(runner): add experimental Grok Bot agent participation (#16219)
## Thinking Path > - Paperclip manages AI agents that perform work for a company. > - The shared Runner owns task admission, authority, receipts and finalization. > - An existing Grok Bot needs to receive assignments and return results through that Runner. > - The Bot also needs governed access when it starts work from its own conversation. > - This PR adds agent-purpose MCP access and routine webhook dispatch. > - It reuses the external-agent Rust Runner and keeps Grok capabilities explicit. > - The provider stays experimental until the real-client managed journeys pass. ## Linked Issues or Issue Description **Agent or provider** Grok Bot, distinct from the Grok Build coding runtime in #13882. **Why this adapter is useful** Users can assign work to an existing Bot, receive progress, questions, documents and files in Paperclip, and let the Bot initiate work under its agent permissions. **How the agent is invoked** A webhook-only routine delivers bounded references. Dedicated named-agent OAuth and MCP provide inbound calls. Setup follows [Grok routines](https://cursor.com/help/grok-bot/routines) and [custom plugins](https://cursor.com/help/grok-bot/connect-plugins). The current desktop app needs its Bot-generated Webhook URL navigation link when routine-row clicks fail. The GitHub search found related Grok Build runtime work, but no existing Grok Bot agent provider. This extends the roadmap’s existing agent, Runner, governance and artifact foundations. ## What Changed - Add default-off Grok Bot participation through the shared real Rust external backend, preserving native provider identity guarantees and distinguishing session closure from remote termination. - Bind agent OAuth to company, agent, client, gateway and authorizer; recheck permissions and task-bound execution authority without manufacturing a native identity. - Persist routine configuration, dispatch intents, authenticated acknowledgement and recovery holds. Store URL/key together in encrypted secret storage, use guarded public egress, and never automatically replay ambiguous HTTP delivery. - Reuse native tools, durable waiting, completion assessment and review. Add task-bound file grants through existing artifact storage; suppress proactive echo dispatch and synchronous result wake loops. - Enforce the current native deliverable work-mode policy at file-ticket issuance and use. Add operator retries for confirmed rejected wakes, with retained receipts, stable retry intent and current-authority checks. Recovery locks bindings before routine rows. Thread Stop resolves the selected run and its agent. - Cancel queued and preparing Stop targets before assignment creation, restore saved setup after a failed request and successful retry, and publish the Grok provider descriptor schema. Advance the versioned Copilot source declaration for the shared schema change; its qualification remains pending. - Filter task summaries with the shared privacy policy, preserve feedback admission when wake delivery is blocked, reconcile settled delivery receipts without restoring work authority, and bind every Stop control to its displayed run. Recovery records distinguish dispatch uncertainty from a revoked session. - Add resumable guided setup, credential replacement, consent, management, observed work status and stop/recovery controls, plus an interactive Storybook journey using the production task thread, question form and report card. Grok’s own apps remain available; Paperclip workspace and third-party tools are deferred. ## Verification - Five focused protocol, authorization, person MCP and Runner files: 192 tests pass. The Grok assigned-work case executes the actual Rust Runner, saves a document, uploads/deduplicates a file and reaches ordinary finalization. - Proactive intake/deduplication and renewed-session recovery pass. These use scripted clients and synthetic storage; they are not real-provider acceptance. - Repository typecheck, production build, token gates, map checks and Storybook build/catalog checks pass. The local full-suite campaign was stopped after macOS shared-memory exhaustion repeatedly prevented embedded PostgreSQL startup. No local aggregate pass is claimed. The first clean-worker CI run identified avatar fixture/configuration, flag defaults, feature-card ordering and OpenAPI route-coverage gaps; all four focused regressions pass after correction. Those regressions pass. The latest clean-worker CI run also found a Copilot source-hash declaration mismatch after the shared descriptor update; the corrected declaration passes all 28 integrity tests. The next review corrections pass 144 rendered UI tests, 21 OpenAPI tests and 82 Runner contract/integrity/history tests. The latest database fixtures were blocked at startup by host shared-memory exhaustion; clean-worker CI must execute their new regressions. Full current-head CI now passes: 54 successful checks and four intentional skips. The database regression uses the PostgreSQL driver result shape; file-policy, rejected-feedback and lock-order cases pass on clean workers. Current-head Greptile is 5/5 with no unresolved findings. - Rendered setup steps and outbound-only verification were walked through in Storybook. Simulation cannot enable Done without an authenticated-reply state. - Actual client OAuth, continued access, reconnect and one routine wake with an exact chat acknowledgement were observed separately. This candidate’s live authenticated challenge, assigned/proactive workflows, Linux managed artifact, restart and cleanup are not yet qualified. The provider stays disabled by default. - The separate Cloud ingress change forwards only exact task-bound upload/download paths and preserves host identity and header stripping. - Current Cloud policy reserves Test for pre-merge app/harness experiments; Staging accepts only master-derived revisions. Real-client qualification needs a supported, authorized deployment target and remains outstanding. ## Risks One append-only migration adds routine/dispatch state and extends shared bindings and file tickets. The unshipped migrations were regenerated after upstream consumed their original numbers. Rolling back code retains the durable rows; revoke grants and inspect unknown remote activity before resuming dispatch. Grok cannot confirm remote termination, stream its complete transcript or report spending. Plugins are account-wide and do not isolate credentials between Bots. Real-client managed acceptance remains required before enabling the experimental provider. ## Model Used OpenAI Codex based on GPT-6, with reasoning, code execution and browser/native UI tools. The exact deployment model ID and context-window size are not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f6608f382f |
refactor(server): isolate GitHub task preparation and review containment (#16231)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn provider messages into shared task workflows. > - The main service still implements provider details beside common authorization and persistence. > - These details belong in the existing channel folders. > - This pull request moves GitHub task prompts, review provenance, and shared guest containment behind explicit typed functions. > - The service keeps its shared workflow and selects the provider implementation directly. ## Linked Issues or Issue Description Refs: #16192. Stack part 2/5. Depends on #16230. Base: `codex/chat-channel-inbound-policies`. **What existing behavior does this improve?** The ownership and readability of provider behavior in the chat channel service. **Current behavior** The service implements GitHub task prompts, review provenance, and shared guest containment inline. **Proposed behavior** Channel modules own these details. The core calls their typed functions. Runtime and database effects retain their current order. **Reason and benefit** Maintainers can find channel behavior in its channel folder without following a generic provider framework. **Breaking changes** None intended. Public service entry points and durable data shapes stay compatible. ## What Changed - Extract GitHub task prompts, review provenance, and shared guest containment into the existing channel folders. - Keep shared company scope, authorization, and transaction ownership in their current workflow. - Reduce `chat-channels.ts` from 3970 to 3906 lines. ## Verification - The same 130 selected GitHub and low-trust integration cases passed before and after extraction. Existing transactions still own containment, authority receipts, and review rows. - Server typecheck passed. - `node scripts/check-module-boundaries.mjs` passed. - Full workspace typecheck and build passed on the cumulative stack. All current-head CI gates passed. Greptile completed at 5/5 with no findings. - The full local `pnpm test:run` command exited with three database fixture-startup hook timeouts. It reported 19,529 passing tests and 250 skipped tests. No assertion failed. Its later test groups did not run because the driver stops on the first failing group. - The timed-out files were `environment-custom-images-service.test.ts`, `generic-mcp-connection.test.ts`, and `modules/wake-queue/adapters/postgres.test.ts`. All 166 tests in these files passed in isolation on both baseline `edcfcaf3f6` and stack head `faaca26d46`, with existing timeouts and assertions. The tests and database-startup helper are unchanged. - Provider fixtures prove application integration. They do not prove live third-party execution. ## Risks - Incorrect inputs or changed await order could affect provider handling and recovery. - Existing integration tests exercise the assembled service before and after extraction. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3e8e1efb84 |
refactor(server): isolate inbound channel message policies (#16230)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn provider messages into shared task workflows. > - The main service still implements provider details beside common authorization and persistence. > - These details belong in the existing channel folders. > - This pull request moves inbound message filtering, command rendering, reply metadata, and setup-reply settlement behind explicit typed functions. > - The service keeps its shared workflow and selects the provider implementation directly. ## Linked Issues or Issue Description Refs: #16192. Stack part 1/5. Base: `master`. **What existing behavior does this improve?** The ownership and readability of provider behavior in the chat channel service. **Current behavior** The service implements inbound message filtering, command rendering, reply metadata, and setup-reply settlement inline. **Proposed behavior** Channel modules own these details. The core calls their typed functions. Runtime and database effects retain their current order. **Reason and benefit** Maintainers can find channel behavior in its channel folder without following a generic provider framework. **Breaking changes** None intended. Public service entry points and durable data shapes stay compatible. ## What Changed - Extract inbound message filtering, command rendering, reply metadata, and setup-reply settlement into the existing channel folders. - Keep shared company scope, authorization, and transaction ownership in their current workflow. - Reduce `chat-channels.ts` from 4082 to 3970 lines. ## Verification - The same 14 selected chat/Photon integration cases passed before and after extraction, including Telegram forum controls, guidance retries, Teams setup replies, and Photon conversation reuse. - Server typecheck passed. - `node scripts/check-module-boundaries.mjs` passed. - Full workspace typecheck and build passed on the cumulative stack. All current-head CI gates passed. Greptile completed at 5/5 with no findings. - The full local `pnpm test:run` command exited with three database fixture-startup hook timeouts. It reported 19,529 passing tests and 250 skipped tests. No assertion failed. Its later test groups did not run because the driver stops on the first failing group. - The timed-out files were `environment-custom-images-service.test.ts`, `generic-mcp-connection.test.ts`, and `modules/wake-queue/adapters/postgres.test.ts`. All 166 tests in these files passed in isolation on both baseline `edcfcaf3f6` and stack head `faaca26d46`, with existing timeouts and assertions. The tests and database-startup helper are unchanged. - Provider fixtures prove application integration. They do not prove live third-party execution. ## Risks - Incorrect inputs or changed await order could affect provider handling and recovery. - Existing integration tests exercise the assembled service before and after extraction. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
450d93d49d |
fix(auth): deduplicate run history permission checks (#16242)
Share matching run-history permission checks within one list operation while preserving current authorization, redaction, and fresh checks on subsequent requests. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
5ea3f905cf |
fix: preserve workspace finalization during watchdog recovery (#16247)
Keep task completion separate from runtime and workspace settlement. Permit verified board file-only repair of successful legacy retained-source runs without authorizing replay. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
9cc979116e |
fix(server): stop reporting queries that outlive the database shutdown (#16243)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server reports unexpected route errors to Sentry through the
shared error handler.
> - An orderly shutdown drains the HTTP listener, and then it ends the
database pool.
> - `server.close()` completes when the sockets close. A route handler
whose client disconnected, or whose socket the drain timeout closed, can
still run.
> - The next query of that handler fails with postgres.js
`CONNECTION_ENDED`, because this process ended the pool. The error
handler reports this as a 500 crash.
> - This pull request marks the pool as closing just before it ends. The
error handler then skips the error sinks for that one error code during
shutdown.
> - Public MCP calls run in their own internal Express app, so the flag
is also shared with that app.
> - The benefit is that a normal restart does not create false Sentry
crash reports. Real errors are still reported.
## Linked Issues or Issue Description
No public GitHub issue exists. This is the bug description.
**What happened?**
A staging server restarted. During the restart, `GET
/api/companies/:companyId/heartbeat-runs` failed in the per-row run
privacy check (`canActorReadHeartbeatRun`). The error was `Failed query:
select "id" from "issues" ...`. Its cause was `write CONNECTION_ENDED
<host>:5432`. postgres.js raises this code only for a query that starts
after the process called `sql.end()`. The error handler sent the error
to Sentry as a crash.
**Expected behavior**
A query that runs after the shutdown ends the pool is an expected part
of the shutdown. The server must not report it as a crash. If the
response is still open, the server answers `503`. The server must
continue to report all other errors.
**Steps to reproduce**
1. Start the server, and send a slow request (for example, a large
heartbeat run list).
2. Close the client connection, or let the 5 second drain timeout close
it, while the handler still runs.
3. Send SIGTERM. The shutdown drains HTTP and ends the pool.
4. The handler's next query fails with `CONNECTION_ENDED`. Before this
change, the error goes to Sentry.
## What Changed
- `server/src/database-retry.ts`: Add `isEndedDbPoolError`. It finds the
postgres.js `CONNECTION_ENDED` code anywhere in the error `cause` chain.
`isTransientDbConnectionError` now uses the same chain walk. Its
behavior does not change.
- `server/src/index.ts`: The orderly shutdown sets
`app.locals.paperclipDatabaseClosing = true` immediately before it ends
the database pools.
- `server/src/middleware/error-handler.ts`: When the flag is set and the
error is `CONNECTION_ENDED`, the handler answers `503 { error: "Server
is shutting down" }`, or it ends the response if the headers are already
sent. It does not call Sentry or crash telemetry. This follows the
existing client-abort (`ECONNRESET`) case.
- `server/src/services/public-mcp/capabilities.ts` and
`server/src/app.ts`: `createMcpApiDispatch` takes the main app's locals.
Each per-call dispatch app reads `paperclipDatabaseClosing` through a
live getter, so a public MCP call that outlives the pool is handled the
same way (Greptile review finding).
- `server/src/__tests__/error-handler.test.ts`: Add tests for the 503
path, the headers-sent path, `CONNECTION_ENDED` outside shutdown (still
reported), and `CONNECTION_CLOSED` during shutdown (still reported).
- `server/src/__tests__/public-mcp-dispatch-shutdown.test.ts`: Send an
ended-pool error through the real MCP dispatch, both during shutdown
(503, not reported) and outside shutdown (reported).
## Verification
- `cd server && vitest run
src/__tests__/public-mcp-dispatch-shutdown.test.ts
src/__tests__/error-handler.test.ts
src/__tests__/cloud-tenant-transient-db-retry.test.ts`: 25 tests pass.
- With only the error-handler change reverted, the two new error-handler
shutdown tests fail. With only the dispatch change reverted, the MCP
shutdown test fails. The negative-control tests pass in both cases.
- I could not run the full `pnpm -r typecheck`, `pnpm test:run`, and
`pnpm build` locally, or `public-mcp.test.ts`, which needs built
workspace packages. The sandbox has a 4 GB memory limit. CI runs the
full gates.
## Risks
- Low risk. The skip applies only to `CONNECTION_ENDED`, and only after
the shutdown starts to end the pools. postgres.js raises that code only
for a pool that this process ended. Network and database failures
(`CONNECTION_CLOSED`, `CONNECTION_DESTROYED`, and others) are still
reported.
- `CONNECTION_ENDED` outside shutdown (for example, a dedicated client
used after it ended) is still reported as a crash.
- The MCP dispatch app gets only the shutdown flag from the main app's
locals. It does not get any other locals.
- No schema, API contract, or UI change.
## Model Used
Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool
use (repository inspection, shell, test execution) and the Sentry and
GitHub connectors.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
ba4fac585d |
feat(agents): show who created an agent in shared companies (#16055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A company in Paperclip can have more than one human member, and each member can create agents > - The app does not record which person created an agent > - When two or more people share a company, members cannot see who added an agent or who to ask about it > - This pull request records the creator of each agent and shows the creator on the agent Overview tab > - The benefit is clear ownership of agents in shared companies, with no change for single-user companies ## Linked Issues or Issue Description No public issue exists. This description follows `feature_request.yml`. **Subsystem affected** Agents: the agent data model, the agent create path on the server, and the agent Overview tab in the board UI. **Problem or motivation** Paperclip does not record which human user created an agent. When more than one human user works in a company, the agent page does not tell members who created the agent. **Proposed solution** Record the user ID of the creator when a human creates an agent. Show "Created by <name>" on the Identity card of the agent Overview tab. Show this row only when the company has two or more human members. Do not count the synthetic local Board principal as a human member. Do not show the local Board as a creator. **Alternatives considered** Count human users across the full instance. We did not do this. An instance-wide count tells a member of one company that other companies have users. The creator is always a member of the agent's company, so the company count is sufficient. **Roadmap alignment** `ROADMAP.md` has no item for agent ownership. This change is small and additive. ## What Changed - Add the nullable column `agents.created_by_user_id` (`packages/db/src/schema/agents.ts`, migration `0343_new_mentor.sql`). - Set the column in the single agent insert path (`server/src/modules/agent-lifecycle/adapters/records.ts`). The value comes from the authenticated user actor. The server ignores a creator value in the request body. Agents that agents or the system create keep `null`. - Backfill existing agents in the migration. The backfill uses the earliest `agent.created` or `agent.hire_created` activity row that a user actor wrote in the same company. - Add `createdByUserId` to the shared `Agent` type (`packages/shared/src/types/agent.ts`). - Add `ui/src/lib/agent-creator.ts`. It decides if the UI shows the creator row and it resolves the creator name. If the creator left the company, the row shows "Former member". - Add the "Created by" row to the Identity card in `ui/src/pages/AgentDetail.tsx`. If the company user directory request fails, the row shows "Could not load creator". The row does not disappear silently. - Add the Storybook story `Pages/Agent Detail → Shared Company Shows Creator`. - Document the column in `doc/SPEC-implementation.md` §7.2. ## Verification - `pnpm --filter @paperclipai/db typecheck` passes. This includes the migration numbering and migration safety checks. - `pnpm --filter @paperclipai/shared typecheck` passes. - `ui`: `tsc --noEmit` passes. - `server`: `vitest run src/modules/agent-lifecycle/__tests__/lifecycle.test.ts` passes (49/49). Two new tests: - The server records the creator from the actor and ignores a creator value in the request body. - The backfill takes the earliest user activity row. It skips agent actors and rows from other companies. - `server`: `vitest run src/__tests__/built-in-agents.test.ts` passes (40/40). A new test checks that a built-in agent enabled by a human records that human, and that automatic provisioning stays unattributed. The gated (pending-approval) provisioning test also asserts the creator. `src/__tests__/built-in-agent-routes.test.ts` also passes. - `ui`: `vitest run src/lib/agent-creator.test.ts src/pages/AgentOverview.test.tsx` passes (10/10). The tests cover these cases: the row shows with two or more humans, the row stays hidden with one human plus the Board, the UI never shows the Board as the creator, the row shows an error when the directory request fails, and the row stays hidden (no error) for agents with no creator or the Board as creator. The last case fails without the fix. - The current Storybook catalog uses source annotations and generated indexes. `node scripts/storybook-catalog.mjs --check` passes (264 story files, 2,514 stories). The merge keeps the authored catalog deletion from master. - `pnpm check:token-gates` is clean. - Screenshots from the Storybook story (fake data: two human members, Alice Admin and Bob Builder, plus the local Board). The red "Could not load public key" text is a Storybook artifact: the story has no fixture for the public key request. The existing `Dashboard` story shows the same text. **Two or more human members: the Identity card shows "Created by".**  **One human member plus the local Board: the row stays hidden.**  Full Overview tab: [overview-created-by.png](https://raw.githubusercontent.com/paperclipai/paperclip/9b62bd2ea6cf984ce6745f08cc45f44ddf00fb51/pr-16055/overview-created-by.png). The images are on the separate branch `pr-assets/agent-created-by`. They are not in this PR and do not go to `master`. - Manual check: open Storybook at `Pages/Agent Detail → Shared Company Shows Creator`. The Identity card shows "Created by Alice Admin". The `Dashboard` story has one human, so it does not show the row. - Final focused verification on `5a02350cf16050a3cf3cb3f8dc061fe61758c60d`: all 99 tests pass across the two UI suites, built-in-agent suite, and lifecycle suite. The backfill test applies the complete migration twice. The migration uses `ADD COLUMN IF NOT EXISTS` and preserves the company-scoped backfill. Migration numbering and token checks pass. - Full repository typecheck, build, and Storybook build pass. The generated creator story ID is present. A Chromium check renders the production agent page fixture and verifies the creator row and Alice Admin name. Map checks and their 21 tests pass. A fresh Apex review is 5/5 on the current head with no unresolved findings. All 54 successful CI checks pass on this head, with two intentional skips, including all general and serialized test shards, E2E, typecheck, and build. The workspaces-b job passed after rerunning a runner shutdown; no source change was needed. The separate, unsharded local `pnpm test:run` invocation has not completed; complete suite verification comes from CI. ## Risks - Migration: the migration adds one nullable column and runs one `UPDATE` on `agents` with a correlated subquery on `activity_log`. On an instance with a very large activity log, the backfill can take some time. The migration does not lock other tables and does not delete data. - Backfill accuracy: agents with no user-authored create activity row keep `null`. The UI then does not show a creator. This is the same as the current behavior. - Privacy: the row shows only names from the company user directory, and only to users who can already see that directory. - Low risk for single-user companies: the UI does not change for them. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: Anthropic, Claude Opus 5.5 (model ID `claude-opus-5-5`), run as a coding agent through Claude Code with tool use (shell, file edit, and test execution), standard context window. OpenAI GPT-6 (`gpt-6`) through Codex assisted conflict resolution, regression tests, and repository verification. This session does not expose an exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Amy (Opus) <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> |
||
|
|
a64969b288 |
fix(server): attribute legacy runs stopped by controller lease loss (#16084)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat runner executes legacy agent runs. A controller lease watchdog keeps durable ownership of each legacy run. > - When the watchdog loses the lease, it aborts the adapter. The abort reason was a plain `Error`, and nothing recorded why the run stopped. > - Completion then saved the cancellation as source `unknown`. Sentry got an unexpected cancel that it could not tell apart from other unrecorded stops. > - We saw this on a sandboxed Grok run. The system cancelled it after about 22 minutes with `cancellationSource: unknown`. Workspace copy-back then failed. Every `cancelRun` path records a stop request, so an unrecorded stop this late points to the lease watchdog. > - This pull request records a lease loss as its own cancellation source, with a closed cause label. > - The benefit is that the next event shows whether the lease loss caused the stop, and which renewal failure caused it. That is the missing fact to fix the root cause. ## Linked Issues or Issue Description Refs #15790 Refs #15920 **What happened?** A legacy sandbox run was stopped by the system while it was working. The run record and the Sentry diagnostics show `cancellationSource: unknown`, with no initiator. The controller lease watchdog aborts the adapter with a plain `Error("Legacy controller lease lost")`. Completion does not read the abort reason, so it records the stop as `unknown`. **Expected behavior** When the host stops its own adapter because it lost the controller lease, the run records that cause. Diagnostics name the cause with a closed label. A stop that a user or the control plane requested earlier keeps its recorded source. **Steps to reproduce** 1. Start a legacy run that has a controller lease. 2. Make lease renewal fail: the renewal query errors, the update matches no row, or the lease deadline passes. 3. Let the adapter return its cancelled result. 4. Read `resultJson.cancellation` on the run. Before this change, `source` is `unknown`. **Agent adapter(s) involved** Not adapter-specific (core bug). We saw it with `grok_local` in a sandbox. **Additional context** This change adds diagnostics. It does not prevent the lease loss. #15920 retries a transient copy-back failure. It does not help here, because the stopped sandbox rejects every retry. ## What Changed - `legacy-controller-lease.ts`: The watchdog aborts with a new `LegacyControllerLeaseLostError`. The error carries a closed cause: `renewal_rejected`, `renewal_deadline`, or `renewal_error`. - `run-cancellation.ts`: Add the `controller_lease` cancellation source. Add `controllerLeaseLossCancellation()` and `readControllerLeaseLossCause()`. The reader accepts only the server-written reason text. - `heartbeat/run-completion.ts`: When the outcome is `cancelled` and the abort reason is a lease loss, record a `controller_lease` cancellation (`expected: false`). This covers both an adapter that returns a cancelled result (`completeRun`) and one that throws after the abort, for example through `signal.throwIfAborted()` (`failRun`). A stop request recorded earlier still takes precedence. - `run-failure-diagnostics.ts`: Add `controllerLeaseLossCause` to the execution diagnostics. Only the closed label goes to Sentry, never the reason text. - `heartbeat/run-state.ts`: preserve `controller_lease` in the bounded SQL projection when saved results exceed 64 KB. A database regression verifies the default run-detail read. - `canContinueCancelledRun` stays unchanged. It does not accept `controller_lease`, so a lease loss does not authorize automatic continuation. - `doc/run-log-events.md`: Document the new source and cause labels. - Tests: unit tests for the cause on each watchdog path, completion tests that record the source when the adapter returns and when it throws the abort reason (both fail without the change), tests that a recorded stop request takes precedence, and a test that the Sentry report contains the closed label and not the reason text. ## Verification - `cd server && npx vitest run src/services/run-cancellation.test.ts src/services/legacy-controller-lease.test.ts src/services/__tests__/run-failure-report.test.ts src/services/heartbeat/run-completion.test.ts`: 238 passed, including the embedded Postgres suites. - I reverted only `run-completion.ts` and ran the new completion test. It failed with `source: "unknown"`. With the change, it records `controller_lease`. - I removed only the `failRun` attribution and ran the new thrown-error test. It failed without the `controller_lease` record, and passes with it. - Final full repository `pnpm -r typecheck` and `pnpm build` pass. - Final local regression verification: all 247 tests pass across run-state, cancellation, watchdog, failure reporting, and completion suites. The large-result test fails before the SQL allowlist change. All latest-head CI checks pass (54 successes and two intentional skips). Fresh Apex is 5/5 on `708856c568d07d7b446a4a4af2edc6d9eec7d155` with no unresolved findings. ## Risks - Low risk. The change adds a new source value to server-owned JSON. Old rows keep their values. The readers accept the new value, and no UI code switches on this field. - A lease-loss cancel stays `expected: false`, so it is still reported as unexpected, as before. Now it has a more specific source. - The Sentry payload gets one new closed label (`controllerLeaseLossCause`). It does not include reason text or IDs. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). OpenAI GPT-6 (`gpt-6`) through Codex added the large-result regression and SQL projection fix, with shell, file-edit, and test tools. The exact context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
