mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-21 02:43:25 +01:00
b4cea5b5a4e9b307ef325299b11eda63f54e286a
831
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
ae2afd525b |
Make harness choices deployment-configurable
Managed deployments need to control harness availability without changing Core policy. Add a generic environment contract and apply it across harness and model selectors, preserving saved supported profiles and validating import, team, and invite selections. Document deployment ownership in agent guidance. Validation: focused local tests and typechecks passed; all applicable CI checks passed on the reviewed head. Greptile scored 5/5 with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
425a30139f |
fix(ui): attribute a stackless unhandled fetch rejection to the code that started it (#22396)
Chromium creates `TypeError: Failed to fetch` with no stack frames, and the browser error monitor removes the SDK Breadcrumbs integration, so nothing wraps fetch to add a call site. An unhandled network rejection reached Sentry with no caller. Once monitoring starts, wrap window.fetch once. Only on rejection, classify the call-site stack as app, extension, other or unknown, keyed by the rejected value in a WeakMap. Tag unhandled object rejections with browser_fetch_initiator and browser_fetch_wrapper. No URL, path or stack text is sent. The returned promise rethrows the identical value, so a dropped promise is still reported. Diagnostics only; this does not fix the underlying rejection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
48025caaca |
fix(ui): mark the connectivity probe's timeout abort as intentional (#22390)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The browser UI shows a connection banner. It checks the server with a health probe (`probeServerHealth` in `ui/src/lib/connectivity.ts`) > - The probe stops its own `fetch` after 10 seconds with a reasonless `controller.abort()`. The app catches that rejection > - A `fetch` wrapper outside the app, such as a browser extension, can drop the rejection. The browser then reports `AbortError: signal is aborted without reason` as an unhandled rejection, and Sentry records an error that has no user effect > - #22090 added a marked abort reason for the shared GET and a Sentry filter that drops only that marked object. The probe timeout did not use it > - This pull request makes the probe timeout abort with the same marked reason > - The benefit is that the probe's own timeout no longer shows as a production error, and every other `AbortError` is still reported ## Linked Issues or Issue Description No public issue exists. This is the problem, in the bug template fields: **What happened?** A hosted browser reported one unhandled rejection: `AbortError: signal is aborted without reason` (DOMException code 20, mechanism `onunhandledrejection`, native trusted promise). The stack is a `setTimeout` callback in `probeServerHealth` that calls `controller.abort()` on the `/api/health` probe. The tab was hidden and had just resumed. **Expected behavior** The probe's own timeout does not appear in error monitoring. A real abort failure still appears. **Steps to reproduce** 1. Replace `window.fetch` with a wrapper that calls the real `fetch` and attaches only `.then(() => {})` to its promise. 2. Make `/api/health` hang for more than 10 seconds. 3. The probe timeout aborts the request. The app's `try/catch` handles its own promise, but the wrapper's derived promise rejects with no handler. **Paperclip version or commit** Reported on release `651749bdb4`. `probeServerHealth` is unchanged on master up to this branch's base (`201abacd6e`). **Deployment mode** Hosted, browser UI (Chrome-family message text). Refs #22090 (same filter, shared GET call site) and #22380 (same approach, announcement card). I searched open PRs for `probeServerHealth` and `intentionalAbortReason` and found no duplicate for this call site. ## What Changed - `ui/src/lib/connectivity.ts`: the probe timeout calls `controller.abort(intentionalAbortReason())` instead of `controller.abort()`. - `ui/src/lib/connectivity.test.ts`: a new test uses a `fetch` that settles only when its signal aborts. It checks that the probe returns `{ reachable: false }` after 10 seconds and that the signal reason is the marked intentional reason. ## Verification - `cd ui && npx vitest run src/lib/connectivity.test.ts src/lib/sentry.test.ts src/api/client.test.ts` → 3 files, 80 tests pass. - Red check: without the `connectivity.ts` change, the new test fails (`isIntentionalAbortReason(signal.reason)` is `false`). - `cd ui && npx vitest related src/lib/connectivity.ts --run` → 346 of 347 files pass. The 2 failures in `src/components/Layout.viewport.test.tsx` also fail on unmodified `master` in this sandbox, so this change does not cause them. - `tsc --noEmit` for `ui` reports no errors in the changed files. - The existing real-SDK tests in `ui/src/lib/sentry.test.ts` (from #22090) prove that `beforeSend` drops a marked rejection and still sends an unmarked `AbortError`. ## Risks - Low risk. The filter matches by object identity, and only `intentionalAbortReason()` makes these objects. An `AbortError` from any other source is still reported. - The probe's abort message changes from the browser's text to "The operation was aborted.". The probe catches every rejection and does not read the message. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
434e574ec6 |
fix(ui): stop reporting posthog-js request timeouts as unhandled rejections (#22381)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted Paperclip Cloud loads PostHog's browser script (`posthog-js`)
through a same-origin proxy for product analytics, and reports browser
errors to Sentry
> - posthog-js stops its own requests at a client-side timeout. For
`/flags` this timeout is 3000 ms by default. It makes `AbortError:
PostHog request timed out after 3000ms`, logs it as a warning, and tries
again
> - On a slow visitor network, this abort reason can escape as an
unhandled promise rejection inside PostHog's script. Sentry's global
handler then reports it as a board error. The board has no handler to
attach there, and the visitor sees no effect, because analytics never
gates the UI
> - This pull request makes the browser Sentry `beforeSend` drop exactly
that one error, and only when it comes through the unhandled-rejection
handler
> - The benefit is that Sentry shows board failures, not third-party
analytics timeouts
## Linked Issues or Issue Description
No public issue exists. This is the problem, in the bug template fields:
**What happened?**
A hosted browser reported one unhandled rejection: `AbortError: PostHog
request timed out after 3000ms` (mechanism `onunhandledrejection`,
native trusted promise). All stack frames are in `/_ph/static/array.js`
(posthog-js), below the SDK's `setTimeout` wrapper. posthog-js 1.453.0
makes this error in its fetch transport timeout (`abort(new
Error("PostHog request timed out after <n>ms"))` with `name =
"AbortError"`). `feature_flag_request_timeout_ms` is 3000 by default, so
this is the `/flags` request. posthog-js logs this as a warning and
tries again. The board has no handler that it can attach inside
PostHog's script.
**Expected behavior**
A timeout of an analytics request that posthog-js handles and retries
does not appear in error monitoring. Analytics never gates the board, so
the visitor sees no effect. Any other `AbortError` still appears.
**Steps to reproduce**
1. Open a Cloud board with browser analytics on, on a slow network.
2. Let a posthog-js `/flags` request take more than 3000 ms.
3. posthog-js aborts the request with its own `AbortError`. The reason
escapes as an unhandled rejection, and Sentry reports it.
**Paperclip version or commit**
Reported on release `651749bdb463aa09d1eb2ae6e61ac3da26161a74`. This
branch is based on `201abacd6e`.
**Deployment mode**
Hosted Paperclip Cloud, browser UI.
Related: #22090 (drops the shared GET's own cancellation through the
same `beforeSend` check). This change adds a second exact match to that
check. The PostHog proxy (`server/src/routes/posthog-proxy.ts`) refuses
at once at its limits and has a 30 s upstream timeout, so it does not
cause a 3 s client stall. I searched open and closed PRs for this error
message and found no duplicate.
## What Changed
- `ui/src/lib/sentry.ts`: add `isPostHogRequestTimeout`. It matches only
an `Error` with `name === "AbortError"` and the exact posthog-js message
`PostHog request timed out` (with an optional ` after <n>ms`).
- `ui/src/lib/sentry.ts`: the existing unhandled-rejection drop in
`beforeSend` (from #22090) now also matches `isPostHogRequestTimeout`.
As for the existing match, it consumes the rejection-event snapshot. All
other events keep the same enrichment and are sent.
- `ui/src/lib/sentry.test.ts`: add real `@sentry/browser` SDK tests that
count what reaches the transport.
## Verification
- `cd ui && npx vitest run src/lib/sentry.test.ts
src/api/client.test.ts`: 2 files, 56 tests pass.
- Red proof: with master's `sentry.ts`, the new test "drops posthog-js's
own timeout when it escapes as an unhandled rejection" fails (1 failed,
36 passed).
- The tests prove that these are still reported: another `AbortError`,
the same text on a plain `Error`, a string reason with the same text,
and the same error when Paperclip code captures it with
`captureException`.
- `npx tsc --noEmit -p ui/tsconfig.json` shows no errors in the changed
files. Other errors come from the unbuilt `@paperclipai/plugin-sdk` in
the local checkout.
## Risks
- Low risk. The filter matches one exact third-party error name and
message, and only on the unhandled-rejection path.
- If posthog-js changes the message, the filter stops matching, and
Sentry reports the error again. No board failure can be hidden.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
812e30d423 |
fix(ui): drop Firefox for iOS injected-script errors from browser Sentry (#22378)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The browser UI sends unhandled errors to Sentry through `ui/src/lib/sentry.ts` > - Firefox for iOS injects its own Reader-mode and helper scripts into each page. These scripts read a `__firefox__` global > - When one of these scripts runs before its setup is complete, the page's `window.onerror` reports an error. Sentry then records it as a Paperclip error > - No Paperclip code defines or reads `__firefox__`. These reports are browser noise and hide real errors > - This pull request drops errors whose message contains `__firefox__` with the built-in Sentry `ignoreErrors` option > - The benefit is fewer false error reports. All other errors from the same browser still go to Sentry ## Linked Issues or Issue Description **What happened?** Hosted browser Sentry recorded two errors from one page load on Firefox for iOS: `TypeError: undefined is not an object (evaluating 'window.__firefox__.reader')` and `ReferenceError: Can't find variable: __firefox__`. Both events have the same trace and the same timestamp. The only stack frame is the page document (line 1, column 19), not a Paperclip bundle file. **Steps to reproduce** 1. Open a Paperclip page in Firefox for iOS. 2. Let the browser's Reader-mode script run before its `__firefox__` setup is complete. 3. See the `__firefox__` error in the browser Sentry project. **Expected behavior** Sentry does not record errors that come from scripts the browser injects. **Actual behavior** Sentry records them as unhandled Paperclip errors. No duplicate or related public PRs were found (searched for `__firefox__` and "firefox ios sentry"). ## What Changed - `ui/src/lib/sentry.ts`: add `ignoreErrors: ["__firefox__"]` to the browser Sentry init options. A module comment tells why. - `ui/src/lib/sentry.test.ts`: add tests that use the real Sentry SDK. The tests send both `__firefox__` messages and make sure that Sentry drops them. Each test also sends a normal app error and makes sure that Sentry keeps it. - `doc/observability.md`: describe the filter. Correct an old line that said that the browser adds no `beforeSend` hook. ## Verification - `cd ui && pnpm exec vitest run src/lib/sentry.test.ts src/lib/browser-error-context.test.ts`: 55 passed. - Without the fix (tests kept), the 3 new tests fail. - `pnpm verify` was not run locally because of a partial dependency install. CI runs the full suite. ## Risks - Low risk. The filter drops only messages that contain the string `__firefox__`. Paperclip code does not use that name. All other errors are not changed. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in an agent harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
201abacd6e |
fix(ui): stop reporting the shared GET's own cancellation as an unhandled rejection (#22090)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The browser UI reads API data through a client that shares one
`fetch` between identical in-flight GETs (`ui/src/api/client.ts`)
> - When every caller of a shared GET leaves, the client aborts that
`fetch`. A live-update refetch that cancels an older read does this many
times in a busy tab
> - The app handles that rejection. But a `fetch` wrapper outside the
app, such as a browser extension, can drop it. The browser then reports
`AbortError: signal is aborted without reason` as an unhandled
rejection, and Sentry records an error that has no user effect
> - This pull request marks the abort reason that the client uses, and
makes the Sentry `beforeSend` hook drop an unhandled rejection only when
its value is that exact marked object
> - The benefit is that an intentional cancellation no longer appears as
a production error, and every other `AbortError` is still reported
## Linked Issues or Issue Description
No public issue exists. This is the problem, in the bug template fields:
**What happened?**
A hosted browser reported one unhandled rejection: `AbortError: signal
is aborted without reason` (DOMException code 20, mechanism
`onunhandledrejection`, native trusted promise). The stack is:
query-invalidation-batcher `flush` → `invalidateQueries` →
`refetchQueries` with `cancelRefetch` → React Query `Query.cancel` →
`onCancel` aborts the query signal → `coalescedGet` `onAbort` →
`releaseRef` → `controller.abort()` on the shared GET.
**Expected behavior**
An intentional cancellation of a shared GET does not appear in error
monitoring. A real abort failure still appears.
**Steps to reproduce**
1. Create a `QueryClient` and an observer whose `queryFn` calls
`api.get(path, { signal })`.
2. Replace `window.fetch` with a wrapper that calls the real `fetch` and
attaches only `.then(() => {})` to its promise. Some extensions and
injected scripts do this.
3. Start a refetch. While it is in flight, call `invalidateQueries` for
the same key through the batcher.
4. React Query cancels the first fetch, the shared GET is aborted, and
the wrapper's derived promise rejects with no handler. The app's own
promise chains do not cause the rejection: without the wrapper, the same
steps give zero unhandled rejections.
**Paperclip version or commit**
Reported on release `3a2f71db31`. `ui/src/api/client.ts` and
`ui/src/lib/sentry.ts` are unchanged on master up to this branch's base.
**Deployment mode**
Hosted, browser UI (Chrome-family message text).
Related: #21475 (groups failed-query Sentry reports) and #15940
(attributes stackless unhandled rejections). I searched open and closed
PRs for `releaseRef`, `inflightGets`, and this error message, and found
no duplicate.
## What Changed
- Added `ui/src/lib/intentional-abort.ts`. `intentionalAbortReason()`
makes an `AbortError` and records it in a `WeakSet`.
`isIntentionalAbortReason()` checks that identity without reading the
value.
- `ui/src/api/client.ts`: when the last caller leaves (abort or
timeout), the shared GET aborts with `intentionalAbortReason()` instead
of a reasonless `abort()`.
- `ui/src/lib/sentry.ts`: `beforeSend` returns `null` for an
unhandled-rejection event only when `hint.originalException` is a marked
reason. It also consumes the rejection-event snapshot, so the snapshot
cannot attach to a later report. The unhandled-rejection check is
computed once and reused by the existing diagnostics.
- `ui/src/lib/sentry.test.ts`: the real-SDK test harness now treats a
`null` return from `beforeSend` as a dropped event. Before, `?? event`
sent the event on.
- Tests: the shared signal reason is marked after an abort and after a
timeout, and a caller's own reason is not marked. Against the real
`@sentry/browser` SDK, the marked rejection is dropped and an unmarked
`AbortError` is still sent.
## Verification
- `cd ui && npx vitest run src/api/client.test.ts src/lib/sentry.test.ts
src/lib/browser-error-context.test.ts
src/lib/query-invalidation-batcher.test.ts` → 4 files, 100 tests pass.
- Red check: with the original `client.ts`, the new client test fails
(the reason is not marked). With the original `sentry.ts`, the new
Sentry test fails (the marked rejection is sent).
- A throwaway reproduction (not committed) used a real `QueryClient`,
the invalidation batcher, `api.get`, and a `fetch` stub that rejects
with `signal.reason`. Without a wrapper: 0 unhandled rejections. With
the `.then(() => {})` wrapper: exactly 1 unhandled `AbortError`, which
matches the report.
- `tsc --noEmit` for `ui` reports no errors in the changed files.
## Risks
- Low risk. The filter matches by object identity, and only the client
makes these objects. An `AbortError` from any other source is still
reported.
- The shared GET's rejection message changes from the browser's text to
"The operation was aborted.". Callers already receive their own
`AbortError`, and nothing reads the shared reason.
- If app code drops this rejection in the future, the filter also hides
that. The effect is the same: the request was cancelled on purpose.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (shell, file edits, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
c41145e72a |
fix: wait on pending questions and quiet provider holds (#22363)
## Thinking Path > - Paperclip manages agents and the tasks they work on. > - Recovery should continue unfinished work when it is eligible. > - An active unanswered question is a persisted reason to wait for a person. > - Treating a later Retry as an answer makes the recovery sweep repeatedly request continuations that admission refuses. > - Provider-limit holds also produce one cancellation toast per task, even though provider work never started. > - This PR aligns recovery with persisted interaction status and presents these waits accurately. ## Linked Issues or Issue Description Refs: #20466. This addresses the question blocker presentation and automatic recovery loop; it preserves the existing interaction-resolution policy. Related work: #19420 schedules provider quota retries, and #21570 handles notices and reply ownership in agent chats. This change concerns assigned-task reconciliation, task/inbox status, and per-run cancellation toasts. **What happened?** The assigned-work sweep ignored pending interactions whenever a newer run carried `explicitUserContinuation`. Continuation admission still refused the pending question, so repeated sweeps recorded unsuccessful continuation attempts. The task could meanwhile show an older provider error as its current blocker. Separately, each pre-dispatch provider-limit hold generated a cancellation toast, and the one-hour fallback was described as a known reset and promised start time. **Expected behavior** Wait quietly while the question remains active under the shared interaction eligibility rules, then react to the existing resolution notification. Dismissed or historical ordinary questions remain answerable without blocking work; governed requests keep their gates. Show the current question on task and inbox surfaces. Keep provider failure history, suppress cancellation toasts for expected provider holds, and describe scheduled availability checks without promising the provider will accept work. **Steps to reproduce** 1. Leave an `ask_user_questions` interaction pending with `supersedeOnUserComment: false`. 2. Create a later failed/interrupted explicit Retry and run assigned-work reconciliation repeatedly. 3. Observe that the question remains pending while the sweep attempts fresh continuations. 4. Separately, let an agent with recent provider rate-limit evidence receive automatic task wakes without a reported reset time. ## What Changed - Remove inferred question supersession from both assigned-work fact loading and candidate selection. Use the existing active-interaction predicate consistently in those reads, fresh-continuation admission, and the task projection. Cover dismissed/historical ordinary questions, retained governed gates, repeated checks, idle workers, and the existing resolution notification. - Add the `question` interruption action, load active pending question IDs alongside approvals in one company-scoped query, and show that blocker on tasks and inbox rows. Preserve a supported manual Retry after a failed attempt, suppress duplicate actions for saved input, and keep other current user gates authoritative. This preserves the existing issue-detail SQL statement budget. Historical run failures remain intact. Refresh the shared source SDK input pin. - Suppress cancellation toasts specifically for cancelled `provider_limit_wait` runs. Label provider retry deadlines as availability rechecks, retain relative-time formatting, and record successful scheduling as an informational lifecycle event. - Document the wait contract in the continuation architecture guide. ## Verification - Demonstrated failing regressions before the implementation: continuation loop, task projection, task notice, cancellation toast, provider deadline wording, and deadline formatting. - Initial focused Vitest qualification: 642 tests passed across 14 files, using: ```sh pnpm exec vitest run \ server/src/__tests__/{assigned-work-invariant,assigned-work-recheck-idle,heartbeat-provider-limit-wait,task-interruption-query}.test.ts \ server/src/services/{task-interruption-presentation,execution-projection,fresh-continuation-fallback,issue-thread-interactions}.test.ts \ server/src/services/recovery/assigned-work-invariant.test.ts \ ui/src/{components/ExecutionBlockerNotice.test.tsx,context/LiveUpdatesProvider.test.ts,context/LiveUpdatesProvider.hook.test.tsx,lib/provider-limit-message.test.ts,pages/Inbox.test.tsx} ``` - `pnpm --filter @paperclipai/shared typecheck` — passed. - Review follow-up: 157 tests passed in `assigned-work-invariant`, `task-interruption-query`, and `fresh-continuation-fallback`; 575 passed in `task-interruption-query`, `assigned-work-recheck-idle`, `task-interruption-presentation`, `issue-question-context`, `issue-thread-interactions`, and `explicit-native-continuation` (the query file overlaps). The strengthened final dismissed/historical/governed projection regression also passed separately. - Final query-consolidation verification: 157 tests passed across five files, including the unchanged 64-statement issue-detail census and mixed approval/question batches: ```sh pnpm exec vitest run \ server/src/__tests__/{issue-detail-statements,task-interruption-query,assigned-work-invariant,assigned-work-recheck-idle}.test.ts \ server/src/services/task-interruption-presentation.test.ts ``` - `pnpm --filter @paperclipai/server typecheck` — passed, including test layers. - Manual-recovery follow-up: 204 tests passed across seven projection/recovery/notice suites, and all 104 Inbox tests passed. The lost-runner integration still starts exactly one run from a manual Retry and leaves the question unanswered. The saved-decision census verifies one decision batch query for a 41-task list, with at most three statements referencing interactions; the overall 64-statement issue-detail limit remains unchanged. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm --filter @paperclipai/ui build` — passed. - `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --check` — passed. - `pnpm verify` — exit 2: broader CI is required for changed global inputs. Focused tests and package checks supply local evidence; broad CI remains required. - `git diff --check` and a local secret/instance-data scan — passed. - Final CI on `1c16fe7c0d8`: [trusted CI workflow](https://github.com/paperclipai/paperclip/actions/runs/44650155842) passed, including all server shards, native-server verification, build, canary, e2e, release registry, and serialized suites. All PR checks are complete, with no failures; the branch is conflict-free. - Greptile reviewed the latest revision, `1c16fe7c0d8`, at 5/5 with no actionable findings or unresolved review threads. The focused tests cover genuine provider failures, live work and terminal tasks, unchanged task locks/retry delivery, preservation of historical failures, and question resolution waking the affected task while other active pending questions remain quiet. Review regressions also cover dismissed/historical ordinary questions retaining Retry and governed requests retaining their gates. No live task was retried or answered for validation. ## Risks - An active pending question now consistently blocks automatic assigned-work recovery. Shared interaction eligibility determines when ordinary conversation input becomes deferred or historical; governed requests retain their authority. Automatic recovery no longer infers that a newer Retry resolved an active question. - Manual Retry keeps its existing admission policy and recovery checks. The question notice preserves that supported action after a failed attempt without changing approval or credential authority; pending saved input prevents duplicate actions. - The shared interruption action gains `question`; the task and inbox consumers are updated together. The pending-question query returns only task IDs and uses the existing company/task/status index. - Provider cooldown policy, quotas, credentials, routing, cancellation history, and durable retry timing are unchanged. The actual provider can still return a rate limit at the recheck. - No migration is required. Existing large owner modules receive small localized changes; no code file crosses 1,000 lines. ## Model Used OpenAI `gpt-6.1-sol` through the Codex harness in T3 Code, xhigh reasoning effort, with code execution and tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
19a7381398 |
feat: restore personal archiving in task views (#22358)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People use Tasks to see shared work and manage the work that needs their attention. > - The old inbox already let each person archive a task and undo that action. > - The combined Tasks surface needs the same controls in Unread and in task table rows. > - The existing inbox archive rules determine when new activity brings a task back. > - This pull request adds an Archived view so people can find and restore their archived tasks. ## Linked Issues or Issue Description **What existing behavior does this improve?** Personal task archiving in Tasks. Related work includes the combined Inbox and Tasks surface (#15100), the Coordinator workspace (#17983), moving Mine into Tasks (#19363), and the existing inbox archive rules (#9931). **Subsystem affected** Cross-cutting. The UI and task list API reuse the existing inbox archive state. **Current behavior** Mine has an archive action. Unread and the task table have no equivalent action. The Tasks menu has no dedicated view for restoring archived tasks. **Proposed behavior** Add Archive and Undo to the task views. Let people restore tasks from Archived. Keep archived tasks available in All tasks. **Reason and benefit** People can clear their personal lists and still find the original task. **Breaking changes** The task list API adds an optional archive filter and viewer archive field. Existing callers use the same response and filters unless they opt in. ## What Changed - Reuse the current inbox archive and restore actions, activity rules, and authorization. - Add Archive and Restore to task table rows, Archive and Undo to Unread, and a personal Archived view. - Give each archived task its own Undo control, including when people archive tasks quickly. - Add tests for the archive list, account isolation, new activity, row actions, and Undo. - Document the archive flow and API filter. Refresh the generated SDK source input registry. ## Verification - Passed 287 focused tests in 12 Vitest files. These cover the task views, row actions, archive cache, and PostgreSQL routes. - Passed local server and UI typechecks. The source input registry check also passed. - Reproduced a missing Undo action when archiving two tasks in a row. After the fix, the 158 task and inbox tests and the 2 toast tests passed. - On macOS with Chrome, archived a task from Mine and restored it using Undo. Archived another task from All tasks, reloaded Archived, restored it, and verified its Todo status and return to Mine. - A human tester also verified Archive, Undo, and Restore in the local browser. - On commit `c5526fd9b1`, GitHub's [verification](https://github.com/paperclipai/paperclip/actions/runs/44630156078/job/135950130662) and [browser tests](https://github.com/paperclipai/paperclip/actions/runs/44630156078/job/135946228919) passed. A targeted retry passed after the first attempt hit a watchdog self-test failure and two cancelled jobs. - [Greptile scored 5/5](https://github.com/paperclipai/paperclip/pull/22358#issuecomment-7024083610) on the same commit. Both review threads are resolved. ## Risks - An incorrect visibility rule could hide a task from a personal list. The archived filter uses the same activity predicate as Mine. The PostgreSQL test covers isolation between people and resurfacing after a reply. - The archive state is already stored per person and company. No database migration is required. ## Model Used - OpenAI Codex, GPT-6, with shell, code editing, and browser verification tools. The serving model ID and context size were unavailable in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Scott Tong <1504818+scotttong@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
c0aa7751f8 |
feat(responsibilities): let agents change areas and responsibilities for their person, with undo (#22261)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People keep their areas and responsibilities in Paperclip (experimental, `enableResponsibilities`). #22253 lets agents read them. > - Agents can still only propose. They cannot rename or archive an area, change an active responsibility, let one go, or move work between responsibilities in one step. > - A person who says "file my weekly routines under Sales" must then do it by hand. > - The maintainer chose this rule: an agent may make each change that its person could make, the person can undo it from the Dashboard history, and a permanent delete still needs the person's yes on a card. > - This pull request adds eight change tools and the server rules behind them. > - The benefit is that an agent maps work to responsibilities for the person, and the person keeps control through undo and the delete card. ## Linked Issues or Issue Description Refs #22253 (phase 1; this pull request is stacked on it). **Subsystem affected** Server (responsibility and area routes, Dashboard versions, native runner tool authority), shared validators and Dashboard types, the native runner tool catalog, the `paperclip` skill, and the Dashboard history UI. **Problem or motivation** An agent that works for a person can only propose areas and responsibilities. It cannot change active ones, archive or delete them, or move filed work in one step. Unfiling needs link IDs. There is no way for the person to undo an agent's change. **Proposed solution** When an agent acts for the person its run works for (`onBehalfOfUserId`) and that person directed the run (the run's own task is one they made, such as their chat, or their own message woke it; parent tasks do not count), apply that person's own rights: their own responsibilities, or anyone's for a company owner or admin; areas for any active member. Save each such change as a version of the person's Dashboard, so the existing history and Undo line show it under the agent's name. Make a permanent delete wait for an accepted confirmation card that the agent posted. Add `file` and `unfile` routes that take items by kind and ID, with an optional move. **Alternatives considered** One card for every change, or propose-only. The maintainer chose "act for me, with undo". A separate undo journal would need a new table and a new UI; Dashboard versions already have both. A new version cause needs a check-constraint migration; this change uses `change_set` with no `interactionId` instead. ## What Changed - Add tools `create_area`, `update_area`, `delete_area`, `create_responsibility`, `update_responsibility`, `delete_responsibility`, `file_under_responsibility` and `remove_from_responsibility`. They are standard-mode only and hidden while Responsibilities is off. - Add the writer rule for agents in `routes/agent-person-authz.ts` (`agentPersonInCompany`, `agentActsForResponsibilityOwner`), used by `routes/responsibilities.ts` and `routes/responsibility-areas.ts`. The person must have directed the run: a person can name anyone as a task's responsible person, and a run can also work for a routine's person or the company default, so those runs do not borrow that person's rights. An agent with no such person keeps the old propose-only rules. - The response names only the version of the responsibility's owner (for area changes, the acting person's), and `X-Paperclip-Dashboard-Owner` says whose history holds it; tools return `dashboardOwnerUserId`. A change Undo cannot reverse (the name, description, order or un-archiving of an area other people's proposed or active responsibilities share) saves no version and is not reported as undoable. - `/file` without `moveFrom` follows the `/links` rule, so `create_responsibility` with `fileItems` works for a proposal. - A title-only create is allowed for an acting agent. It is active by default and records the agent in `proposedByAgentId`. - Add `responsibility-agent-changes.ts`. In one transaction that holds each affected owner's Dashboard lock, it saves the dashboard before the change (when a person changed it by hand), runs the change on services bound to that transaction, and saves the result as a `change_set` version by that agent. The response names the version in `X-Paperclip-Dashboard-Version`. The tools say Undo only when a version was saved. - Dashboard snapshots now also hold each responsibility's filed items, description, "run by" agent, evidence, health override and note, and each area's description (optional fields). Restore puts them back. Older versions still restore as before. A matching older version is upgraded in place, so the first change after it compares and undoes in full. - Create and update accept `areaTitle`. The server finds or makes that area in the same transaction, so a refused change leaves no area. Agent creates reuse a live responsibility with the same title under a lock, so retries and parallel calls make no copies. - Add `POST /api/responsibilities/:id/file` (`items`, optional `moveFrom`: an ID or `all`) and `POST /api/responsibilities/:id/unfile` (`items`), each in one transaction. - Agent deletes need `confirmationId`: an accepted `request_confirmation` card that this agent posted, with target key `responsibility-delete:<id>` or `responsibility-area-delete:<id>`. Someone who may manage the owner must accept it, on the card or by a verified chat reply. The tools post the card (named after the item), return `awaiting_confirmation`, and delete on the next call after acceptance. - `DELETE /api/responsibility-areas/:id` accepts `moveResponsibilitiesTo`. Area changes and deletes save versions for everyone whose Dashboard shows the area. - Agent activity details now include `onBehalfOfUserId`. The Dashboard history shows "<agent>, for you" for these versions. `ResponsibilityDashboardVersionSummary` now has `interactionId`. - Update the OpenAPI registry and metadata, the `search_api` tool pointers, the skill, the API reference, the chat directive, the operation-groups spec, and the pinned counts and SDK source pins. ## Verification - Server: `vitest run` of the `responsibilit*` tests, `responsibility-tools.test.ts`, `openapi-routes`, `runner-api*`, `paperclip-runner-tool-authority` and `task-markdown` all pass. New tests cover these cases: acting-agent create, update and undo; file, move and undo; the rule for a person who may not manage the owner; delete only after an accepted card, including a verified chat reply; area delete with move and a version for the other owner; undo of a health override, an area description and an un-archived empty area; upgrade of an older version; an area by title that a refused change rolls back; parallel creates. - UI: `DashboardEditModule.test.tsx` and `responsibility-dashboard.test.ts` passed. `pnpm --filter @paperclipai/ui typecheck` passed. - Server `node scripts/typecheck-layers.mjs` passed. - Runner: catalog, semantic-tool, eval, protocol-action and permission-policy tests (284) passed. All generator checks pass. - I did not run the browser e2e suite locally. ## Risks - Medium. Agents that work for a person can now change and archive that person's responsibilities and areas without a card. Every change is a Dashboard version that the person can restore, and a delete still needs the card. - Undo restores titles, sentences, areas, order, status, filed items, description and the "run by" agent. It does not restore a bar or metric change; those stay in the metric history. - A restore of a version from before this change does not touch filed items, which matches the earlier behavior. - Undo of a rename or archive of an area that other people also use keeps its name, as the existing restore rule does for shared areas. - Manual edits do not take the Dashboard lock. A person's edit that commits while an agent's change is open can appear in that agent's version. - Changes to proposed or let-go responsibilities are not on a Dashboard, so they cannot be undone there. The tools say so. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
92dbf39e6f |
feat(analytics): attribute agent work and onboarding steps to a person on Cloud (#22260)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip Cloud sends product events straight to PostHog: server
events from `server/src/services/posthog-direct/` and browser events
from `ui/src/lib/analytics/`
> - A person-level funnel goes from sign-up to the first finished task
> - Agent-driven events (`agent.task_run`, `agent.task_completed`,
`agent.first_heartbeat`) used `workspace:<stackId>` as the distinct id,
so the funnel broke at those steps
> - The in-workspace onboarding wizard sent no step events, so we could
not see where people stop before their first agent
> - This pull request attributes agent work to the responsible person on
the Cloud direct path, and adds consent-gated onboarding step events in
the browser
> - The benefit is one PostHog person per customer across sign-up,
onboarding, agent runs, and task completion. Self-hosted output does not
change.
## Linked Issues or Issue Description
**Subsystem affected**
Cloud product analytics: Cloud direct PostHog delivery (server) and
browser product analytics (UI onboarding wizard).
**Problem or motivation**
A PostHog funnel from sign-up to the first finished task cannot follow
one person. Agent-driven events run in the background, with no signed-in
request, so they use the workspace id as the distinct id. The onboarding
wizard also sends no events, so we cannot see which step people leave.
**Proposed solution**
Give background emit sites a narrow scope that names the person
responsible for the work. Use the task creator first, then the task's
responsible user, then the run owner. Keep the signed-in request user
first. Add `attributed_via` to every Cloud direct event. Add four
browser events to the onboarding wizard. Send them through the existing
analytics module, so the consent rules stay the same.
**Alternatives considered**
A lookup in the transport at send time. We did not use it: it adds
database reads to the telemetry path and loses the emit site's context.
A new contract event for onboarding steps. We did not use it: the server
does not see the wizard steps, and the UI must never send
`PAPERCLIP_EVENTS` names.
**Additional context**
Self-hosted installs keep the collector path. The collector envelope
stays byte-identical (a test checks it).
## What Changed
- `server/src/telemetry-company-scope.ts`: add `withTelemetryActor({
companyId, person }, fn)`, `scopedTelemetryPerson()`, and
`resolveTelemetryPerson()`. The resolver picks the task creator, then
the task's responsible user, then the run owner. It ignores
`local-board`. `withTelemetryCompany` now calls `withTelemetryActor`.
- `server/src/services/posthog-direct/request-context.ts`: a signed-in
request user wins (`attributed_via: request`). Without one, the scoped
person is used.
- `server/src/services/posthog-direct/event-shape.ts`: every Cloud
direct event (including `cloud.workspace_started`) has `attributed_via`
(`request`, `task_creator`, `task_assigner`, `run_owner`, or `none`). An
event with a person keeps today's person-event shape (no
`$process_person_profile: false`).
- `server/src/services/agent-task-run-telemetry.ts`: `agent.task_run`
reads the task creator and responsible user in the existing agent query
(one left join, no new query), and the run's `responsibleUserId`.
- `server/src/routes/issues.ts`: `agent.task_completed` uses the issue
row that the route already has, and the agent actor's `onBehalfOfUserId`
as the run owner.
- `server/src/services/heartbeat/agent-status.ts`:
`agent.first_heartbeat` uses the agent's `createdByUserId` (already
loaded) as the run owner.
- `ui/src/lib/analytics/product-analytics.ts`: add
`captureProductAnalyticsEvent()`. It captures only with a browser
analytics config and granted consent. Events from before posthog-js
loads wait in a queue (at most 50) and keep their time. Withdrawn
consent or a change of signed-in person drops the queue.
- `ui/src/lib/analytics/onboarding-analytics.ts` and
`ui/src/components/OnboardingWizard.tsx`: capture `onboarding_opened`,
`onboarding_step_viewed`, `onboarding_step_completed` (with `step` and,
for `connect_model` and `launch`, `outcome`), and
`onboarding_dismissed`. `step` is `create_agent`, `connect_model`,
`review`, or `launch`. No event has free text.
- Docs: `packages/shared/src/telemetry/README.md` (Cloud Direct
Delivery) and `doc/cloud-browser-analytics.md` (new "Onboarding events"
section).
## Verification
- `server/src/__tests__/posthog-direct-telemetry.test.ts`: attribution
matrix (creator, assigner, run owner, none stays on
`workspace:<stackId>` with `$process_person_profile: false`), the
request user wins over the scope, and the self-hosted collector envelope
is byte-identical with a person scope.
- `server/src/__tests__/agent-task-run-telemetry.test.ts` (embedded
Postgres): `agent.task_run` scope for creator, assigner, run owner, and
nobody; another company's task is never read.
- `server/src/__tests__/issue-telemetry-routes.test.ts`:
`agent.task_completed` scope for creator and assigner.
- `server/src/__tests__/heartbeat-process-recovery.test.ts`:
`agent.first_heartbeat` scope is the agent owner.
- `ui/src/components/OnboardingWizard.test.tsx`: a launch for the old
company that settles after a switch to another company does not end the
new analytics session.
- `ui/src/lib/analytics/onboarding-analytics.test.tsx`: each transition
fires once, held events drop on a person change, a stale opening step is
not a view, dismissal fires once, nothing is captured when analytics is
off or consent is `denied` or `ask`, and queued events keep their time.
- `ui/src/lib/analytics/product-analytics-events.test.ts` still passes
(the UI never captures a `PAPERCLIP_EVENTS` name).
- Local Cloud-mode run: an isolated instance (own Postgres, trusted
Cloud tenant headers, `PAPERCLIP_POSTHOG_HOST` set to a local capture
server, never real PostHog). A stub stood in for posthog-js and sent
through the `/_ph` proxy.
- A user created a task and assigned it to the agent. `agent.task_run`
arrived with `distinct_id: paperclip-id:local-user-1`, `attributed_via:
task_creator`, the raw `task_id`, and no `$process_person_profile`. (The
run itself failed locally because the Cloud runner is not available. The
event still shows the attribution.)
- `agent.first_heartbeat` arrived with the same person and
`attributed_via: run_owner`. `server.started` kept
`workspace:stack-local` with `attributed_via: none`.
- Headless Chromium walked the wizard: `onboarding_opened`
(create_agent), `onboarding_step_viewed` (create_agent),
`onboarding_step_completed` (create_agent), `onboarding_step_viewed`
(connect_model), `onboarding_step_completed` (connect_model, `api_key`),
`onboarding_step_viewed` (review), `onboarding_step_completed` (review),
`onboarding_step_completed` (launch, `succeeded`). Each fired once, with
the signed-in user as the distinct id. Events from before posthog-js
loaded arrived with their original time.
- A second walk left the onboarding page on the connect step and sent
one `onboarding_dismissed` (connect_model).
## Risks
- Low risk for self-hosted: the collector path never reads the new
scope, and a test checks the envelope bytes.
- Cloud: agent events move from the workspace person to a real person.
Insights that count `workspace:` distinct ids for these events will
change. `attributed_via` lets a query filter or split them.
- `task_assigner` uses the task's responsible user. Paperclip does not
store who assigned a task, so this is the person who owns or delegated
the task.
- Privacy: the new data is Cloud-only and uses ids that Cloud events
already carry. No new free text, email, or name goes on an event.
**Privacy review requested** for the Cloud attribution change (per
`rules/data-paths.mdc`). The generated contract does not change:
`attributed_via` is a Cloud-only property, like `stack_id` and
`user_id`.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, headless browser).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
673742bfc6 |
fix(ui): keep artifact previews in agent chat (#22252)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat lets people discuss work and review the plans and documents it produces. > - A document link in Chat promises to open the sidebar. It currently moves the reader to Tasks. > - Readers need to see the document and the conversation together, even when the window is narrow. > - This pull request opens the document in Chat, frees space for reading, and gives readers a link to the source task. > - Readers can review the saved revision in context and open the task when they need its details. ## Linked Issues or Issue Description Refs #11226 for the document sidebar that this change builds on. **What happened?** Opening a saved plan from Agent Chat changed the route to Tasks. The document could show the same title twice. At narrow desktop widths, the chat list left too little space for the conversation beside the document. A task opened by UUID could still show a Chat header. **Expected behavior** The sidebar shows the saved revision beside the current conversation. It shows a matching title once. The chat list collapses when space is tight. The source link opens the standard task header with status, name, and ID. **Steps to reproduce** 1. Open an Agent Chat with a saved plan and select its sidebar link. 2. Use a narrow desktop window and inspect the conversation and plan. 3. Open the task that owns the plan and inspect the breadcrumb. **Paperclip version or commit** Reproduced on source commit `8285a81280`. **Deployment mode** Local trusted development instance with synthetic data. ## What Changed - Open sidebar document links on the current Chat route. Keep the requested revision and document owner, including links inside another task’s preview. Provide an **Open source task** link. - Show one title when the document title matches the opening Markdown heading. - Collapse the desktop chat list when an artifact tab would leave less than 420 px for the conversation. Keep **Browse chats** in the header with access to saved chat history. Restore the list when there is room. - Use the task breadcrumb on task URLs, with the short UUID for a conversation-backed task without a number. ## Verification - Ran 618 focused Vitest tests across 18 UI files on the published revision `2ff6c9e931`. All passed. - Ran `pnpm --filter @paperclipai/ui typecheck`. It passed. - Checked the synthetic demo in Chromium. Opened the plan in Chat, checked the title, resized the window, and followed **Open source task** to check its breadcrumb. Also verified a link inside a preview owned by another task. - Reproduced the chat navigation failure from the CI trace. The same Playwright test passed locally after limiting collapse to artifact tabs. - GitHub CI passed on `2ff6c9e931`, including build, typecheck, and all browser shards. The remaining jobs passed on a retry after runner shutdowns and a server handover test failure. The handover suite also passed locally (19 tests). [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/44423540581/attempts/2) - `pnpm verify` selected broader CI coverage because `doc/PRODUCT.md` changed. The full local suite was not run. ## Risks The chat list changes visibility when an artifact tab is active and the available width crosses the threshold. The list stays mounted to keep its search text. Document tabs use the existing read API and check the owning company. This change adds no database migration or API contract change. ## Model Used OpenAI Codex (GPT-6), with repository tools, shell execution, and Chromium. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have used public GitHub references for linked work - [x] My branch name describes the change and contains no internal Paperclip ticket ID - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Scott Tong <1504818+scotttong@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
4434714824 |
Fix sidebar Recents task order and archive visibility (#22251)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The sidebar gives people quick access to tasks, chats, and projects they recently opened. > - Expanding a parent or project in Recents can show tasks the person never opened. > - Archiving a task from the inbox leaves it in Recents. > - This change gives each recent task its own row and removes tasks archived for the current viewer. > - People can reopen an individual task and keep saved shortcuts above More. ## Linked Issues or Issue Description **What existing behavior does this improve?** Recents in the Standard and Classic sidebars. **Subsystem affected** Board navigation and task summaries. **Current behavior** Opening a parent or project can show other tasks beneath it. A recent subtask moves its parent group when reopened. Archiving a task leaves its recent link visible. **Proposed behavior** Show each visited task in its own row, in recent activity order. Keep canceled and completed tasks eligible. Remove tasks archived for the current viewer from Recents. Preserve saved pins. **Reason and benefit** People can find the task they opened by its own position in the list. Archiving removes its recent link without removing a saved shortcut. **Breaking changes** The Recents rows no longer expand into task lists. Project links still open the project. The navigation summary adds an optional personal archive flag. Related work: #22219, #22024, #21904, and #20450. ## What Changed - Render task, agent chat, and project rows directly in Standard Recents. Keep the existing controls to pin each kind of row. - Render standalone task rows with the existing Classic action menus. Remove sidebar child loading and refresh code. - Derive the viewer’s archive state using the existing inbox rule. Filter archived tasks out of Recents while retaining authorized pinned shortcuts. - Cover independent parent and child order, canceled and archived tasks, pin retention, and the Classic Archive action. ## Verification - Reproduced the empty Recents row after cancellation before the fix. - On the rebased branch, 424 focused UI and live-update tests and the UI typecheck passed on macOS. - Ran 77 issue-summary and OpenAPI integration tests with embedded PostgreSQL on macOS. The Runner source SDK registry check passed on the pushed revision. - The updated sidebar test renders 402 recent rows. A deliberate repeated task lookup made the input-read guard fail. The restored implementation passed. - The paused-assignee browser test checks the task detail heading in each interface. Both variants reproduced the CI locator failure locally and passed with the scoped locator. - All 17 focused Storybook browser checks passed in Chromium on macOS. - Run `pnpm verify` for local selection. The product documentation change requests broader GitHub CI coverage. - Open a parent task and a child task in either sidebar. Reopen the child and check that its row moves ahead of the parent. Cancel the child and check that it stays visible. Archive it and check that its recent row disappears. - On commit `8051038099`, [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/44421716811) passed, including the required verification and end-to-end checks. - [Greptile](https://github.com/paperclipai/paperclip/pull/22251#issuecomment-6994494812) rated the same commit 5/5. Its review thread is resolved. ## Risks - Recents uses the same viewer-specific archive rule as the inbox. Archiving prunes the local recent entry. A restored task can enter Recents again when opened. - Task summaries still return authorized tasks for saved pins. Child and unread counts remain in the existing API response for older clients. - The change requires no database migration. ## Model Used - OpenAI GPT-6 via Codex with shell, GitHub CLI, Vitest, and Chromium. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Scott Tong <1504818+scotttong@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3f263e4faf |
feat(ui): align routine controls and remember list views (#22182)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Routines define recurring work and create tasks. > - People need the same controls to find and inspect routines as they use for tasks. > - The routine list had different search controls and no detailed filters or column choices. > - This pull request aligns the controls and remembers each person's view. > - People can share a filtered link and see the responsible person beside the agent. ## Linked Issues or Issue Description **What existing behavior does this improve?** The Routines list controls and metadata in the current and classic interfaces. **Subsystem affected** ui/ — the React board. **Current behavior** Routines use different search controls from Tasks. The list has limited view settings and no filters for project, agent, enabled state, responsible person, or source. Routine metadata is fixed. **Proposed behavior** Show New routine on the left. Show Search, Sort, Filter, Group, Columns, and View all tasks on the right, in that order. Use the shared Tasks search and filter controls. Show the responsible person next to the agent. Let people choose metadata columns. Save filters, search, sort, and grouping in the URL. Remember view settings per person and company in local storage. Use a play icon for manual runs. **Reason and benefit** People can scan routine ownership, find a subset, share its link, and return to their preferred columns and view. **Breaking changes** The list uses metadata columns with horizontal scrolling on narrow screens. Existing company-level sort and group preferences are read as a fallback. No API or database change is required. Related work: Refs #22009, Refs #18606, Refs #18317. The open PR search found no duplicate of this list-control change. ROADMAP.md lists Routines as shipped and has no overlapping control work. ## What Changed - Align both Routines toolbars with the requested order and shared search and button styles. - Extract the existing Tasks filter group and checkbox row for reuse by Routines. - Add project, agent, enabled, responsible person, and source filters. - Keep linked filters ahead of saved filters. Preserve unrelated URL parameters. - Save preferences per person and company. Keep the controls usable when storage is unavailable. - Add selectable Project, Agent, Responsible user, Last run, and Enabled columns. - Use play-only manual run buttons with accessible names and pending indicators. - Correct the empty message when a status filter excludes every routine. - Add production-page Storybook examples and browser tests for both interfaces. ## Verification - Linux checkout, Node 25.9.0, system Chromium 148.0.7778.96. - The final change includes master through `07d9316218`. It uses the shared routine creation form and the upstream Runner test and SDK source-pin fixes. - 68 tests pass across eight focused UI suites at `440a9eb6a7`. UI typecheck, token gates, and whitespace checks pass. - Two new directory-error cases fail before the fix and pass after it. Both pages keep routine rows mounted, show an error with Retry, and restore responsible names after retry. - 59 checks pass across the SDK source-pin, public-type, and evidence suites. - The Runner test program typechecks after building its eval dependency. A local run of the native-session test file passed 300 tests. One additional test could not run because the local debug Runner binary was absent. CI must cover that case. - Storybook builds and all 13 browser checks pass on `64a87af7c085426d09eb8aa48bf9491487565990` after the latest master integration. They cover current and Classic controls, filters, columns, links, selection alignment, Recent Runs, mobile overflow, and shared toolbar behavior. - Storybook examples: Default, LinkedFilters, Classic, ClassicSelection, and ClassicRuns. They mount production pages with fictional API responses. Screenshots cover light, dark, filters, custom columns, and mobile. The baseline manifest has no snapshots; these are visual review captures. - Earlier preference, shared-link, title-link, compact-row, and filter-render-cost regressions have red/green evidence. The scheduled-routine journey has 17 passing unit checks from the prior revision. - The shared selector has three large-list cases with 1,200 routines and 120 folders. All filters are active. Each case checks row order and folder counts. Work counters throw if filtering repeatedly scans selected IDs or folder counts scan the routine list for each folder. Both deliberate regressions fail; the unchanged selector passes. Both Routines pages use this selector. The latest commit adds only these tests; production source is unchanged. - [CI at `64a87af7c0`](https://github.com/paperclipai/paperclip/actions/runs/44416091562) passed all jobs, including typecheck, Build, Runner, general tests, serialized tests, and E2E. [Final-head CI at `440a9eb6a7`](https://github.com/paperclipai/paperclip/actions/runs/44419144971) now passes on attempt 2. All 52 executed jobs pass; one file-selection job is skipped. Typecheck, Build, all test shards, E2E, Runner, and the aggregate verify gate pass. The first attempt failed in the timing-sensitive watchdog fixture; its retry passes. - `pnpm verify` returned exit 2 because this branch needs broad CI coverage. Final-head CI passes. Greptile and Ponytail both give 5/5 at `440a9eb6a75184a35001e99a17e6856b7cdb0bd5`. All 20 inline threads are resolved. - Prior CI at `10a5b0813b` passed all general and serialized test shards, all E2E shards, and Build. It failed on a circular test-mock type and a stale SDK source pin. Master now contains both repairs; the final branch uses those repairs. - The two Routines page bodies were already duplicated. This PR reduces them and shares list data, controls, rows, and view state. Codie owns the remaining page-shell consolidation as later work. - Live provider execution and scheduler behavior are outside this presentation change. Protected Storybook publication still requires a configured human reviewer; no deployment approval is inferred from merge authorization. ## Risks View settings now include filters. A person can return to a filtered list. The URL and active filter button show that state. Shared filter URLs override local filters. Column choices stay local. The shared filter extraction can affect Tasks, so its component and toolbar checks are included. No migration is needed. ## Model Used OpenAI GPT-6 through Codex. The runtime does not expose a more specific model revision. Used reasoning, source editing, shell tools, and browser automation. No sub-agents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
e4b119440e |
Show live provider reset countdowns in the viewer's timezone (#22225)
## Thinking Path > - Paperclip manages agents and their assigned work. > - Provider usage limits can hold that work until a reset deadline. > - Saved wait messages expose raw timestamps, and the usage popover shows a short weekday clock. > - People need the remaining wait and the exact deadline in their current timezone. > - This pull request formats both views in the browser and updates the countdown live. > - The browser uses its timezone while stored messages and retry deadlines stay unchanged. ## Linked Issues or Issue Description **What existing behavior does this improve?** Provider usage limit messages and the agent usage-limit property and popover. **Subsystem affected** UI presentation of provider usage limits, task notices, run details, log events, and recovery actions. **Current behavior** Wait messages print `2026-11-18T20:17:34.353Z`. The usage popover shows a short clock such as `Thu 23:00`. Neither gives a live relative countdown. If the provider did not report a reset, the scheduler uses the recorded failure time plus one hour. **Proposed behavior** A Pacific-time viewer sees `in 44 minutes (Nov 18, 2026, 12:17 PM PST)`. The relative part counts down while the tab is visible. The usage property and popover show the same local deadline. Expired usage windows clear through the existing summary rules. **Reason and benefit** People can see how long remains without converting a timestamp or refreshing the page. The date conversion handles daylight saving time and local date changes. **Breaking changes** Displayed text and popover layout change. Stored messages, API payloads, error codes, provider checks, and retry deadlines keep their contracts. ## What Changed - Add an optional timezone label to the existing `formatDateTime` helper. Existing callers keep their output. - Share relative and exact local time formatting between provider-limit messages and the usage popover. Unrelated text and invalid timestamps stay intact. - Reuse the existing shared one-second ticker. Countdown updates pause in hidden tabs, catch up when shown, and stop at the deadline or on unmount. No API polling is added. - Keep timer-driven message rendering separate from large parent pages. Hidden task-marker details do not subscribe until opened. - Update task notices, agent run errors in both UI shells, log events, and provider-limit next actions. - Update the usage-limit trigger and every window in its popover. Reset details wrap below the percentage row. The reported timestamp includes its timezone. Explicit clocks remain fixed for stories. ## Verification Tested source: `ef79ce40b170db89a6b831162c72d6446cd9492d`. Local checks ran with the same file contents on macOS arm64, Node 25.6.1, and pnpm 9.15.4. - From `ui/`: `pnpm exec vitest run src/lib/provider-limit-message.test.ts src/lib/utils.date-time.test.ts src/components/ProviderLimitMessage.test.tsx src/hooks/useSecondTick.test.tsx src/components/AgentRateLimitGauge.test.tsx src/components/task-chat/TaskChatMarker.test.tsx src/components/task-chat/TaskChatSystemNotice.test.tsx src/components/IssueRunLedger.test.tsx src/components/IssueRecoveryActionCard.test.tsx src/components/StalledDependencyNotice.test.tsx src/pages/AgentDetail.log-visibility.test.tsx src/pages/AgentDetail.log-resilience.test.tsx` — 176 tests passed across 12 files. - From `ui/`: `pnpm run typecheck` — passed, including Storybook catalog generation. - From the repository root: `pnpm check:token-gates` and `git diff --check` — passed. - Regression proof: four live-message cases failed against static rendering. Four usage-popover cases failed against the old clock. Both suites passed after the changes. - Fake-clock tests prove `44 minutes` becomes `43 minutes` without a page refresh or parent render. Other cases cover seconds near the deadline, one shared timer, hidden-tab pause and catch-up, expiry, and cleanup. - Timezone cases cover Pacific, India, and New Zealand timezones, a local date rollover, and daylight saving time. Task notice cases cover both presentations. Usage-popover cases cover Claude and Codex. - Local setup required a non-frozen install because the base lockfile lacked two AWS SDK dependencies from the server manifest. No dependency or lockfile change is included. - Final CI: [workflow attempt 4](https://github.com/paperclipai/paperclip/actions/runs/44384225521/attempts/4) succeeded on `ef79ce40b170db89a6b831162c72d6446cd9492d`. All 54 successful checks are green, including the applicable typecheck, build, server tests, Runner checks, and all eight end-to-end shards. Optional jobs skipped by the CI plan remain skipped. - Recovery record: [Canary attempt 1](https://github.com/paperclipai/paperclip/actions/runs/44384225521/job/135085343566), [attempt 2](https://github.com/paperclipai/paperclip/actions/runs/44384225521/job/135097010510), and [attempt 3](https://github.com/paperclipai/paperclip/actions/runs/44384225521/job/135100921795) ended with runner shutdown signals at 19:56:15, 20:12:55, and 20:19:18 UTC. The aggregate verification failures reflected that canary result. No source changed between attempts. The [fourth Canary Dry Run](https://github.com/paperclipai/paperclip/actions/runs/44384225521/job/135105826026) and [aggregate verification](https://github.com/paperclipai/paperclip/actions/runs/44384225521/job/135110708013) passed. - Greptile reviewed the final commit at 5/5 with no actionable findings and no unresolved review threads. GitHub reports the branch mergeable with a clean merge state. - Full local repository checks and manual browser acceptance were not run. CI supplied broad coverage on the final revision. The local popover tests exercise its content and clock with mocked popover primitives. ## Risks - Low risk. This changes UI presentation only. No migration or scheduler change is required. - The ticker updates visible future countdowns once per second. It makes no provider request and does not certify that provider capacity has returned. - Message recognition uses the existing provider-limit wording and ISO timestamps. Unrecognized messages retain their text. - Local timezone abbreviations can differ by browser locale data. Time conversion uses the browser's timezone rules. ## Model Used - OpenAI Codex, `gpt-6.1-sol`, with xhigh reasoning effort, repository inspection, code edits, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR with the enhancement template fields - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] The code, tests, and this description document the changed output - [x] I have considered and documented the risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
911fe5bb9a |
Fix Recents ordering and add sidebar pins (#22219)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The sidebar gives people quick access to tasks, chats, and projects. > - Recents should reflect the order in which a person opened their tasks. > - Opening a task only updated its saved details. It did not update the visit time. > - This pull request records task visits and adds Recents shortcuts to the existing sidebar pins. > - People can find their last task and keep frequently used items above More. ## Linked Issues or Issue Description **What happened?** Opening a personal task left it in its old position or added it at the bottom of Recents. The Recents rows also had no controls to pin their items. **Expected behavior** Opening a task moves it to the top of Recents. Opening a nested task moves its parent group to the top. A person can pin a task, chat, or project from Recents. **Steps to reproduce** 1. Open two personal tasks in the Coordinator workspace. 2. Reopen the first task and check its position in Recents. 3. Hover over a Recents row and look for a pin button. **Paperclip version or commit** Reproduced on source checkout `43b2d8be22`. **Deployment mode** Local dev with an isolated embedded PostgreSQL database. Related work: #22024, #21649, and #19685. ## What Changed - Record the first loaded task visit and subsequent reopens. Keep the timestamp during background refreshes. - Use the most recent task visit in each displayed group when ordering parent rows. - Add tasks, agent chats, and projects from Coordinator Recents to the existing sortable sidebar pins. Resolve saved shortcuts with the current company data and task read permissions. - Use the same highlight state for a Recents row and its pin button. This includes the current row, hover, and keyboard focus. ## Verification - The reopen test failed before the fix because the older task stayed below the newer task. The Chromium check also reproduced the missing pin on the current row before the highlight fix. - Ran the eight focused Vitest files for task detail, the two sidebars, recent task state, task warmup, and pinned navigation on macOS. Ran `pnpm --filter @paperclipai/ui typecheck`. - In Chromium on the isolated dev server, opened two tasks and reopened the first. Pinned a task and project and confirmed that both survived reload. Checked current, hovered, and keyboard-focused rows. Checked that a pointer click did not keep the pin visible after leaving an inactive row. - `pnpm verify` requested broader CI coverage because the product documentation changed. - GitHub CI passed on `b77fc06393`. The initial native-server and Canary jobs lost their runners. Both jobs passed on retry without a code change. [CI retry](https://github.com/paperclipai/paperclip/actions/runs/44373716458/attempts/2) - Greptile scored the same commit 5/5 and left no review threads. [Greptile review](https://github.com/paperclipai/paperclip/pull/22219#issuecomment-6988318333) ## Risks - Recent ordering keeps the existing participation filter and subtask groups. A task pin uses the existing summary batch to check read access independently of participation. - Pins are stored per browser user. Task and project keys include the company ID. Existing page and agent-chat keys keep their saved order. - The change uses the existing summaries API. It requires no schema migration. ## Model Used - OpenAI GPT-6 via Codex, with shell, GitHub CLI, and Chromium verification tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue using the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket ID - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Scott Tong <1504818+scotttong@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
33fac97cad |
feat(routines): ship the redesigned routine page and let live webhooks be tested again (#22205)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Routines start agent work on a schedule, from a webhook, or on demand > - The routine detail page split one routine across eight sections behind a section sidebar (overview, triggers, variables, secrets, delivery, runs, activity, history) > - People had to hunt through sections to see what a routine does and to change it, and a finished webhook could not be tested again without starting the routine > - This pull request makes the approved two-column routine page, one Edit dialog and one Webhook dialog the real routed page, and lets a live webhook go back into test mode > - The benefit is one page that answers what the routine does, when it runs, who runs it and what it made, with every change in one dialog ## Linked Issues or Issue Description **Problem** The routine detail page spreads one routine over eight sections and a section sidebar. Changing a routine means visiting several sections, each with its own save bar. A webhook that is already live cannot be tested again: the API only accepts `setupPending: false`, so testing a live webhook starts the routine. **Proposed solution** Ship the reviewed design (stories `routine-page-design`, `routine-edit-dialog-design`, `routine-webhook-dialog-design`) as the routed page in both UI shells, wired to the real API. Let `PATCH /api/routine-triggers/:id` accept `setupPending: true` for a live webhook so the next events are checked but never start the routine. **Alternatives considered** Keep the sections and restyle them. The owner rejected this: the section sidebar should go. ## What Changed - `/routines/:id` renders `RoutinePage` in both shells (`RoutineDetail.production.tsx` is removed). Status switch, Repeats rows, default agent, responsible user, project, Asks for, Secrets, Run rules, Managed by plugin, Instructions with Show all, copy and the comment count, and History. - History is paged: 20 tasks per page from `GET /companies/:id/issues?originKind=routine_execution&originId=…&sortField=created&sortDir=desc&limit=21&offset=…`, with a real Load more. Coalesced runs show as "folded into a task". - A failed refetch keeps the loaded page on screen (`useQueryView`); a History failure shows `QueryErrorState` in place. - Header: Edit, "…" (Version history, Audit log, Archive / Restore) and Run now. In the streamlined desktop shell they sit in the breadcrumb toolbar; elsewhere above the page. - Version history (`/routines/:id/history`, restore and restored-webhook keys kept) and Audit log (`/routines/:id/activity`, `RoutineAuditActivity`) open as sub-views with a way back. Old section addresses and `?tab=` redirect to `/routines/:id`. - Edit dialog saves title, instructions, variables, secrets, agent, project and responsible user in one request against the revision the dialog opened with (save conflict shows "changed while you were editing"). The first schedule is created, changed or removed after that. Priority and delivery settings are never sent, so saved values stay. - Webhooks are created, changed, tested and removed (with Undo) right away from the Edit dialog, with the Webhook dialog nested inside it. Closing the Webhook dialog ends setup or test mode. A webhook left in test mode shows "Testing · events don’t run" on the page and offers Finish testing. - Responsibility (only with `enableResponsibilities`): the page shows direct and through-project responsibilities from the link lookup; the dialog links or unlinks the direct one. With the flag off nothing is fetched or shown. - The routines list creates routines with the same dialog in create mode, filed in the open folder, and opens the new routine page. - Server: `setupPending: true` on a live webhook enters test mode (planned by a pure `planTriggerUpdate`, which also gives the activity action) (422 for other trigger kinds; 409 for a disabled or removed webhook or an archived routine). Test deliveries keep the last real delivery as `lastWebhookDelivery.previous`, and finishing testing restores it. Activity `routine.trigger_test_started`. - Additions to the reviewed design, from review, so no capability is lost: the schedule popover has Pause/Resume; standing-check and API rows open a small popover with Pause/Resume and Remove; removing a trigger from the page leaves a "… removed. Undo" line under Repeats. Stories: `StandingCheckOpened`, `ScheduleTriggerPaused(Opened)`, `RemovedWithUndo`. - Save is several requests (routine, schedule, responsibility). It changes only the schedule the form opened with, records each finished step, and a partial failure says what was saved and that saving again finishes it. The dialog cannot be dismissed while saving. Failed reads show a Retry state, never empty choices. - Removed: the routine section sidebar and contextual sidebar, the trigger wizard, trigger cards, save bar, section components and their tests. - Storybook: the three routine stories moved to `stories/routines` as current production stories, a new story mounts the routed page against a fixture API, and stories that used the old page were updated or removed. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/routines-e2e.test.ts` (17/17) and `src/__tests__/routines-service.test.ts` (105/105). - `pnpm exec vitest run` on the routine page, routines list, routine form, webhook instructions, Layout, shell navigation, audit and editor tests: 275 tests pass. - `cd ui && npx tsc -p tsconfig.json --noEmit`, `node scripts/check-token-gates.mjs`, `node scripts/check-maps.mjs`, `node scripts/storybook-catalog.mjs --check`: pass. - `pnpm verify` exits 2 (global CI input changed); required CI supplies the broad coverage. - Manual: an isolated worktree instance (`pnpm paperclipai worktree init --empty`, `pnpm dev`) with a seeded routine. Headless Playwright: an old `/triggers` address redirects, no section sidebar, History shows 20 then 23 after Load more, the schedule popover opens, pause and resume, Edit → Add webhook → created key → Finish, title save, Test webhook → Finish testing, Version history, Audit log shows "Started testing a webhook", phone width has no horizontal overflow, and Create routine from the list opens the new page. No console errors or failed API calls. ## Risks - Behaviour change: the section pages are gone. Old links redirect to the single page. - The delivery-setting editors (concurrency, catch-up, activity gate) and priority are removed from the UI. Saved values stay, and the API still accepts them. - The create dialog no longer offers a folder picker or delivery settings. A new routine goes into the folder the list is showing and takes the default settings. - Replace key in the Webhook dialog acts at once, as in the reviewed design. The old confirmation is gone. - No migration. `lastWebhookDelivery.previous` is an optional field inside the existing jsonb column. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file edits, headless browser) and subagents for the server change and the Storybook move. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
a9dc0df9da |
fix(ui): clarify dashboard proposal hierarchy (#22180)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent can propose dashboard areas and responsibilities in one card. > - The card put the area name above a separate “New area” row. > - Responsibility rows had no type label and no visible parent structure. > - This pull request puts area changes in the heading and nests responsibility changes below it. > - People can see what each change creates and where it belongs before they apply it. ## Linked Issues or Issue Description **What existing behavior does this improve?** The dashboard proposal card in task conversations and the dashboard edit module. **Subsystem affected** The board UI and its Storybook examples. **Current behavior** “New area” appears in a row below the area title. New responsibilities appear at the same level. Their type is not stated. **Proposed behavior** “New area” appears beside the area title. Each new responsibility has a label and sits below its parent area. Existing areas have a separate label. Area headings follow the heading level in the dashboard module. **Reason and benefit** The person can see the parent area, the change type and the text to review in that order. Long proposals are easier to scan. **Breaking changes** None. Proposal inputs, indexes, dependencies and apply/decline endpoints stay the same. ## What Changed - Put area changes and their controls in the area heading. - Nest responsibility rows under each area. Label additions, rewords, moves and removals. - Keep the drop/keep dependency behavior and the existing apply payload. - Keep area classification and group partitions in one helper. Expose whether each group has an existing parent area. - Add Storybook examples for the new hierarchy, a narrow layout and a mix of new and existing areas. - Add focused component and browser checks for the hierarchy, responsive layout and apply payload. ## Verification - 24 focused unit tests passed across the proposal card, dashboard module and dashboard helpers. - 25 Chromium checks passed against current-source Storybook after the review refactor, with one worker and no retries. They cover six proposal states in light and dark themes at widths of 390 and 960 pixels, plus the existing apply payload. - The UI typecheck passed in low-memory mode after the review refactor. The initial implementation also passed the static Storybook build; current-head CI supplies broad build coverage. - 30 map/catalog tests passed. Catalog checks, token gates and whitespace checks passed. - A hierarchy regression failed before the fix and passed after it. A later hosted-story check found a skipped heading level. The fix passed the expanded browser checks. - Initial concurrent browser, build and typecheck runs hit the workspace memory limit. The final browser pass used one worker and the typecheck used low-memory mode. - The first CI build stopped because its runner received a shutdown signal after the UI build completed. A new CI run checks the review fixes. - The local verification selector returned exit 2 because the new browser spec is a global CI input. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/44333497172) passed typecheck, build, all selected test suites, E2E shards and the final verify gate for `e71916cdde90ede313cc167ab8cfb8a5c3bb501a`. No visual baseline comparison was run because the baseline manifest has no published snapshots. - [Greptile reviewed the current head](https://github.com/paperclipai/paperclip/pull/22180#issuecomment-6982500386) at 5/5 with no findings. Both earlier maintainability threads are resolved; two Paperclip reviews remain pending. ## Risks Low risk. This changes UI structure and labels. Long text and drop/keep controls must continue to fit at narrow widths. The focused browser checks cover those states. Storybook uses synthetic API data and does not prove server persistence or permissions. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3c460f20cc |
feat(analytics): ask EU, UK and Swiss visitors before Cloud browser analytics (#22186)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip Cloud can send browser product analytics and session replays to PostHog. Self-hosted installs never do this. > - Today, browser analytics starts for every Cloud visitor after the page settles. > - Visitors in the EU, the EEA, the UK and Switzerland must agree before analytics cookies and session recordings start. > - This pull request adds a consent step for these visitors and a switch in the user's profile settings. > - The benefit is that Cloud analytics follows the owner decision: ask in Europe, keep the current behavior elsewhere, and let every user change the choice. ## Linked Issues or Issue Description **Subsystem affected** Paperclip Cloud browser product analytics (`ui/src/lib/analytics`, `ProductAnalyticsGate`, the session route, and Profile settings). **Problem or motivation** Browser analytics and session replay start for every Cloud visitor without a choice. EU, EEA, UK and Swiss visitors must consent first. The privacy policy draft also says that users can change the choice in settings. No such setting exists. **Proposed solution** Ask visitors in the EU, the EEA, the UK and Switzerland with a small bottom banner (Accept or Decline). Do not load posthog-js before they accept. Treat Do Not Track and Global Privacy Control as a decline for these visitors. Keep the choice in `localStorage`. Add a "Product analytics" switch to Profile settings for every user when browser analytics is on. **Alternatives considered** A server-side country from a trusted edge header. Paperclip Cloud has no such header today: Cloudflare records are DNS-only, so no `CF-IPCountry` arrives, and the Railway edge adds no country header. This PR supports an optional header (`PAPERCLIP_POSTHOG_COUNTRY_HEADER`) for a future edge, and uses the browser time zone until then. ## What Changed - `packages/shared`: `browserAnalytics` has a new optional `consentRequired: boolean | null`. - `packages/shared/src/analytics-consent.ts` (new): the one region policy. `countryRequiresAnalyticsConsent` (EU and EEA with outermost regions and Svalbard, GB with the Crown Dependencies and Gibraltar, CH) and `timeZoneRequiresAnalyticsConsent` (`Europe/*`, EU, EEA and UK zones outside Europe, and unknown zones such as `UTC` or `Etc/*`). The server and the UI both use it. - `server`: `resolveBrowserAnalyticsConfig` sets `consentRequired` from the header that `PAPERCLIP_POSTHOG_COUNTRY_HEADER` names. No header or an unknown code (`XX`) gives `null`. - `ui/src/lib/analytics/consent.ts` (new): decides `granted`, `denied` or `ask`. Order: for a visitor who must consent, Do Not Track or Global Privacy Control gives `denied` (also over a stored Accept); then the choice made on this page; then the stored choice; then the region (server value, else browser time zone). The choice is in `localStorage` key `paperclip.analyticsConsent`. All storage access is in `try/catch`. A cross-tab `storage` event updates the page. - `ProductAnalyticsGate`: one effect calls `syncProductAnalytics(config, consent)`. Shows `AnalyticsConsentBanner` when consent is `ask`. Accept loads posthog-js at once, without a reload. - `product-analytics.ts`: new `syncProductAnalytics`. It starts, opts in, opts out (`opt_out_capturing()` and `stopSessionRecording()`) or cancels a pending load. Any consent other than `granted` stops analytics. If consent is withdrawn while the script downloads, `init()` does not run until a later opt-in. - `AnalyticsConsentBanner` (new): a labelled bottom region with plain copy, a privacy policy link, and Decline and Accept buttons (shared `Button`). The copy says that recordings hide what you type and your messages, documents and connected accounts, but may show page names like task titles. It does not block the page. On mobile it sits above the bottom navigation. - `ProductAnalyticsSetting` (new) in Profile settings: a "Product analytics" switch (shared `ToggleSwitch`). It shows only when browser analytics is on for the instance. When GPC or DNT blocks analytics, it is off and disabled and says why. - `doc/cloud-browser-analytics.md`: new Consent section (signal, behavior table, storage key, how to reset) and the new variable. - `tests/runner-e2e/source-sdk-public-inputs.json`: refreshed for the changed shared validator. ## Verification - Red/green: the 7 new `ProductAnalyticsGate` consent tests fail against the master gate (7 failed, 3 passed). The 5 review-fix tests (failed write with working read, cross-tab change, decline during script download, granted -> ask from another tab, granted -> ask from a session refresh) fail on the first commit (5 failed, 16 passed). All pass now. - `cd ui && npx vitest run src/lib/analytics src/components/ProductAnalyticsGate.test.tsx src/components/AnalyticsConsentBanner.test.tsx src/components/ProductAnalyticsSetting.test.tsx src/pages/ProfileSettings.test.tsx`: 267 tests pass. - `cd server && npx vitest run src/__tests__/cloud-posthog-browser.test.ts src/__tests__/auth-routes.test.ts src/__tests__/auth-session-route.test.ts`: 87 tests pass. `packages/shared` `analytics-consent.test.ts` passes. `tsc --noEmit` for `ui`, `server` and `packages/shared` shows no new errors. - `pnpm verify` exits 2 (a doc is a global CI input). Required CI gives the broad coverage. - Real check in headless Chromium. I ran a local Cloud-managed instance (external local PostgreSQL, `PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN`, `PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`). `PAPERCLIP_POSTHOG_HOST` pointed at a local capture server, not PostHog. The capture server served a small stand-in for `array.js` that sets a `ph_*` cookie and `localStorage` key and posts `/e/` events. I set the visitor region with the browser time zone: | Visitor | Result | | --- | --- | | `Europe/Berlin`, no choice, 18 s wait | Banner shows. 0 `/_ph` requests. No `ph_*` cookie or storage key. | | Then Accept (keyboard Enter) | Banner closes. `GET /_ph/static/array.js` and `POST /_ph/e/` in 3 s. Key is `granted`. After reload: no banner, events continue. | | `Europe/London`, Decline, wait, reload, wait | No banner. 0 `/_ph` requests. No `ph_*` cookie or storage key. Key is `denied`. | | `Europe/Zurich` with Global Privacy Control | No banner. 0 `/_ph` requests. No `ph_*` cookie or storage key. | | `America/New_York` | No banner. posthog-js loads after the page settles, as before. | | Profile settings switch (US) | Shows on. Off: client is opted out at once. On: opt-in, events continue. | - I ran the browser check again after the review fixes, with the same results. - Screenshots (banner light, banner dark, mobile 375 px light and dark, settings switch): see the PR comment. - Cold load (local dev server, fresh browser context per run, median of 7, ms): | Build | Time zone | FCP | DOMContentLoaded | load | | --- | --- | --- | --- | --- | | master | New York | 728 / 724 | 687 / 684 | 689 / 686 | | this PR | New York | 748 / 724 | 708 / 684 | 710 / 686 | | master | Berlin | 740 | 694 | 697 | | this PR | Berlin | 756 / 724 | 716 / 684 | 718 / 686 | The second round gives the same medians for master and this PR. The first-round difference is noise. The consent code adds no request and no load-time wait. ## Risks - Region by time zone is approximate. It asks some visitors outside the EU (for example `Europe/Istanbul`, or `UTC` browsers). This is the safe direction. - A forged country header (only when `PAPERCLIP_POSTHOG_COUNTRY_HEADER` is set) changes only the default for that visitor's own browser. - The choice is per browser. A user on a new browser sees the banner again (in consent regions). - Visitors in consent regions now send no browser analytics until they accept. Expect fewer EU/UK sessions in PostHog. - Self-hosted installs: no change. They get no `browserAnalytics` config, so no banner, no switch and no load. ## Model Used - Claude Opus 5.5 (claude-opus-5-5), Anthropic, used through Claude Code with tool use (shell, file edits, headless browser). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
43b2d8be22 |
feat(dashboard): add an experimental areas and responsibilities dashboard (#22069)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The dashboard is the first page a person sees in a company > - The classic dashboard shows agents, runs, costs and recent tasks > - It does not show what the person is responsible for, or if that work goes well > - People think in areas, such as Fundraising or Customers, and in the things they own in each area > - This pull request adds a dashboard built on areas and responsibilities, behind an experimental setting that is off by default > - The benefit is that a person sees what they own, how it is going, and what agents do for it, on one page ## Linked Issues or Issue Description No public issue exists for this work. The description follows the feature request template. **Subsystem affected** Dashboard, navigation, the task conversation, routines, and the instance experimental settings. **Problem or motivation** The classic dashboard counts agents and tasks. It does not say what a person owns or if it goes well. People who run a company with agents want to see their areas, the responsibilities in each area, the bar each one must meet, and the agents and routines that serve it. **Proposed solution** A person owns responsibilities. Each responsibility belongs to an area and has a bar, a number to reach. Projects, tasks and routines link to a responsibility, and the page shows its status from those links. An agent can interview the person and draft the dashboard. The person accepts or declines each change, and any change can be undone from a saved version. **Alternatives considered** Add the same data to the classic dashboard. That page is a fixed set of cards for company activity, so a second, person-centred page is clearer. The setting lets a person pick one of the two. **Roadmap alignment** This is an experimental feature behind `enableResponsibilities`. It is off by default and changes nothing for a person who leaves it off. ## What Changed - Add the setting "Areas & Responsibilities dashboard" (`enableResponsibilities`, off by default). On, Dashboard in the sidebar opens the new page, and the classic dashboard link is not shown in the sidebars or the command palette. Off, nothing changes. - Add tables and migrations 0492 to 0497 for areas, responsibilities, links to work, metrics, status reports, saved dashboard versions, and model-call records for summaries. - Add the API: areas, responsibilities, links, bars, metrics, status reports, summaries, runs, automation, and the dashboard conversation with versions and undo. Each route checks company access and writes to the activity log. - Add four task conversations with an agent: about a bar, about an automation, about a routine, and about the whole dashboard. The agent proposes a change. Nothing applies until a person accepts it. - Add standing checks: routines that an agent carries out on its heartbeat and reports. - Add the Dashboard page: search, counts, areas that open and close, a primary New area button, and a message box at the bottom of the page. Add the responsibility page with its bar, metrics, work and routines. - Add the Paperclip skill sections and runner API entries that let an agent use these routes. The runner skill inventory is regenerated (166 to 174 rows). - Keep the new runner API tests in their own file so `runner-api.test.ts` stays at 985 lines. - Onboarding is not changed. The onboarding changes that build on this work are a separate pull request. ## Verification - Type checks pass for `packages/db`, `packages/shared`, `server` (all layers) and `ui`. - `pnpm --filter @paperclipai/db check:migrations` passes. A new `drizzle-kit generate` run produces no further change. - Tests for every changed file pass: 26 server test files (3,009 tests), 116 UI test files (1,991 tests) and 28 shared test files (432 tests). - The runner capability inventory checks pass. - I started the app on a local database that already held earlier data. Master's new migrations and these migrations applied without error. With the setting on, the Dashboard and a responsibility page loaded with no console or server errors. The More menu and the command palette show one Dashboard. - I did not run `pnpm verify`, the full local suite, or the browser suites. Required CI covers them. - The agent conversations are tested with simulated agents. No real model has run them. ## Risks - Medium. The change is large, and the setting is off by default. With it off, the new routes refuse requests and the UI is unchanged. - Migrations only add tables and columns. Two replace a CHECK constraint, and one builds an index `CONCURRENTLY`. These two have the `online_compatible` override with a reason in the file, and the second one is listed in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`. - Master can add migrations before this merges. The migrations then need new numbers. - `responsibilities.ts` (1,663 lines), `responsibility-summaries.ts` (1,239 lines) and one Storybook fixture file (1,305 lines) are new files over 1,000 lines. They each keep one cohesive unit together: the service for one table family, the model-call path for summaries, and the fixtures that the stories share. No existing file crosses 1,000 lines. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude: Claude Fable 5.2 (`claude-fable-5-2`) wrote most of the change, with Claude Sonnet 5.5 (`claude-sonnet-5-5`) for the final steps. Both used tool use, code execution and browser checks. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
ce27dde855 |
Start onboarding in chat and defer execution until the user chooses work (#22077)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Onboarding connects an agent and gives the user a place to discuss work. > - The previous flow created an execution task and project before the user chose a scope. > - A standing chat can gather the goal and clear exit criteria first. > - This pull request opens chat after onboarding and starts a live welcome after the chat is visible. > - The agent proposes work for acceptance, or creates a task when the user explicitly requests one. > - Subscription setup and shared question state must work for this first conversation to succeed. ## Linked Issues or Issue Description **Subsystem affected** Company onboarding, agent chat, local subscription setup, task numbering, and shared interaction views. **Problem or motivation** The first session creates work before the user has agreed to a goal. Standing chats consume task numbers. An answered chat question can remain pending in the inbox. Claude code entry requires an extra click, and switching from Claude to Codex can leave an incompatible model selected. **Proposed solution** Open an owner-bound chat for every new company. Start a live welcome after the chat paints in a visible tab. Gather scope in conversation and use the existing task acceptance flow. Keep chat records out of the task number sequence. Publish question responses to shared caches and cancel stale reads. Submit pasted Claude codes once and wait for verified sign-in before connecting. **Alternatives considered** A fixed welcome would not use the onboarding context. A hidden execution task would still create work and consume a task number. Keep the existing task proposal and acceptance contract instead. **Roadmap alignment** This improves the shipped Agent Chat and Personal & Shared AI Accounts flows in ROADMAP.md. It does not add a separate execution or approval system. Related prior work: Refs #19643 and #19512. Related route handoff work: Refs #12557. This change replaces the automatic first task with chat; it also repairs the subscription and shared question state paths used during that journey. ## What Changed - Finish onboarding in agent chat for default and team-template companies. Create no execution task or project. - Queue one live welcome after visible chat content paints. Pass company goals and the initial idea as context. Preserve access, budget, and hiring controls. - Instruct the agent to gather a goal and exit criteria, offer an interview, and propose tasks for acceptance. Allow direct creation on an explicit user request. - Leave new standing chats unnumbered. The first execution task receives number 1. Preserve existing identifiers. - Synchronize answered question cards across chat and inbox, including concurrent requests and inactive cached views. - Submit a pasted Claude authorization code once. Show connection progress and hide the restart link while sign-in is still preparing. Detect rejected OAuth codes promptly. Clear incompatible models when the provider changes. - Use `Work with [agent name]` as the default chat composer placeholder. - Add regression coverage and update onboarding documentation and API notes. ## Verification - Completed a real Claude subscription journey on macOS with an isolated local database. Authorized the account, pasted the real code, and reached the review step without clicking Submit code or Connect. Get started opened chat and the live welcome run succeeded. The company had zero execution tasks, zero projects, and task counter 0. - On rebased commit c664949858, 6 browser cases passed for paste-to-connect-to-chat, task-free onboarding, existing-company routing, the native connection probe gate, desktop/mobile planning, and two-tab chat/inbox question synchronization. - On the rebased head, 419 focused UI tests passed. The focused server and adapter run passed 160 tests and hit one `socket hang up` in the existing authenticated chat route test. The full 72-test conversation suite passed on one repeat. The cause of that first socket failure remains unconfirmed; the passing repeat does not establish its cause. - UI typecheck and server declaration typecheck pass after the rebase. The first server check found a missing local dependency from master; a frozen-lockfile install repaired it. No dependency manifest or lockfile change was needed. - The first Greptile review found two welcome edge cases. Four new tests reproduced both before the fix. On commit b380f14ef9, all 114 onboarding and queue tests pass, including timeout retries and queued welcomes replaced by a user message or chat reset. Server source and focused test type checks pass on this revision. - [Greptile review](https://github.com/paperclipai/paperclip/pull/22077#issuecomment-6961533641) gives rebased commit |
||
|
|
f05b90508a |
refactor(core): remove retired compatibility paths (#22055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core manages chat authorization, AI connection defaults, navigation, and the plugin host. > - Core still writes old default rows and checks an Agent Chat flag that always reads true. > - It also carries a desktop rail that cannot open and private aliases with canonical equivalents. > - This pull request removes those paths and moves their tests onto the paths Core uses. > - The result is fewer Core database operations and less code to maintain. ## Linked Issues or Issue Description **What existing behavior does this improve?** Core AI default writes, Agent Chat execution and delivery, private plugin host and Runner setup, and board navigation. **Subsystem affected** Cross-cutting. Core server, board UI, private Runner helpers, and a repository guard script. **Current behavior** The server writes provider-scoped and per-method AI defaults, then reads only the provider-scoped rows. It reads settings to decide whether to block Agent Chat, although every settings view enables it. The desktop sidebar renders expanded while still tracking requests to open and close a retired icon rail. The agent client retries conflicts that the server already resolves when there is exactly one active match. **Proposed behavior** Write provider-scoped defaults directly. Use the existing Agent Chat authorization and retry checks without the redundant settings reads. Render the current sidebar directly and let the API report true agent reference conflicts. **Reason and benefit** A personal non-custom connection save or explicit default selection sends one fewer SQL statement. It also avoids the provider-default write that the legacy trigger performed when the old statement inserted or updated a row. A successful task-to-chat send or receipt replay avoids two transaction settings SELECTs when the singleton row already exists. Conversation comment, wake release, and retry paths also avoid a transaction settings read when they previously entered the gate. Savings in other settings paths depend on cache hits. Core-to-plugin RPC and plugin-to-Launcher HTTP traffic stay the same. **Breaking changes** The public plugin SDK, host navigation, and organization-switcher fields retain their existing interfaces and behavior. The settings API still accepts and reports the Agent Chat compatibility key. This change needs no schema migration or rollout ordering. **Additional context** Refs #10610 for prior reporting on the retired sidebar rail. Refs #21756 for adjacent Coordinator plugin slots, which remain a separate change. ## What Changed - Stop writing per-method AI defaults. Preserve the provider-scope upsert and first-account behavior. Keep the existing tables and migration trigger. - Remove Agent Chat checks that read a constant true value. Keep conversation ownership, task scope, session, and request-retry checks. - Remove unused lock-owner fields and private plugin lifecycle, secret extraction, Gmail connector, PRP setup, heartbeat resolver, and ACPX aliases. - Use a single API request for agent references. Remove retired sidebar rail state, hooks, and branches across the active and production snapshot layouts. Preserve desktop resizing, mobile drawer controls, contextual navigation, and public plugin slot inputs. - Remove unused FeedCard and AgentDetail inputs and the test-only Git push matcher alias. - Keep the cleanup that erases historical plaintext chat retries from browser storage. Production source in the touched files is 106,800 → 105,633 nonblank lines, down 1,167. The production Git diff adds 251 physical lines and removes 1,480, down 1,229. Those figures exclude tests, fixtures, docs, and Storybook. The entire Git diff adds 482 physical lines and removes 2,325, down 1,843. ## Verification - Merged master at `b56f240f45` into the PR branch as `f73fea4009`. The five sidebar files retain the expanded navigation cleanup and master’s explicit session-replay masking contract. The merge also preserves master’s account-repair recovery guard and GitHub star-prompt behavior. - Focused UI Vitest run passed 361 tests across 11 files. This covers analytics privacy and replay masking, sidebar and layout, GitHub star prompt, and agent detail. - Focused server Vitest run passed 45 tests across three files. This covers restored configuration handling, conversation recovery, and Agent Chat authority and receipt replay. - `pnpm --filter @paperclipai/ui typecheck`, the Git push guard and its 14 tests, and `git diff --check origin/master...HEAD` passed. - `pnpm verify --base origin/master` classified the Git-push guard as a global CI input and returned exit 2. GitHub CI supplied the broader coverage. - GitHub CI and Runner image checks passed on `f73fea4009`. Both required checks, `ci / verify` and `ci / e2e`, passed. [CI workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261752) · [Runner image workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261413) - Standard Greptile reported 5/5 with no actionable findings or open review threads. [Greptile review](https://github.com/paperclipai/paperclip/pull/22055#issuecomment-6959078925) - The optional Superagent Security Scan finished neutral with an inconclusive workflow result and zero annotations. The final rollup had 61 successes, five skips, and one neutral, with no failed or pending checks. [PR checks](https://github.com/paperclipai/paperclip/pull/22055/checks) ## Risks Agent Chat authorization and retry are sensitive to small query changes. The final diff keeps the conversation agent field and all existing authorization and idempotency checks. The tests cover revoked defaults, legacy migration reads, bound-listener authority, and mobile and desktop navigation. Installations that still run software old enough to read the per-method AI table will no longer see new defaults in that table. Current Core reads the provider-scoped table. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with source inspection and shell-based tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b56f240f45 |
feat(analytics): show session replay text by default, mask only customer content (#22057)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted Paperclip Cloud records browser sessions with PostHog session replay > - Replay masked all text and showed only an allowlist of chrome, so most replays showed only stars > - The team cannot see how people use the board, and the allowlist needed a review for each new label > - The owner decided to show text by default and mask only marked customer content, the way PostHog normally works > - This pull request flips the default, removes the allowlist, and marks every surface that can show connected-account content > - The benefit is readable replays, while Google user data and Slack data stay masked ## Linked Issues or Issue Description **Subsystem affected** Hosted Cloud browser analytics: PostHog session replay in the board UI (`ui/src/lib/analytics/`). **Problem or motivation** Replay masked every text node. Only sidebar items and opt-in components showed text. Replays were almost unreadable. Each new label needed an `analyticsChrome` opt-in and an allowlist review. **Proposed solution** Record text as it is. Mask text only inside `ph-mask` (customer content), `ph-no-capture` (secrets, blocked) and editable content. Add `ANALYTICS_MASK_CLASS` to every surface that can show connected-account content (Gmail, Calendar, Drive and Docs, Slack) or other free-form text from agents or customers. **Alternatives considered** Keep "mask all" and grow the allowlist. This needs a review for each label and still hides most of the board. ## What Changed - `privacy.ts`: `ANALYTICS_MASK_TEXT_SELECTOR` is now `.ph-mask, .ph-no-capture, [contenteditable]:not([contenteditable="false"])`. rrweb masks a text node only when its parent or an ancestor matches the selector. Removed `maskReplayText`, `isReplayTextRevealed`, `ANALYTICS_UNMASK_ATTRIBUTE` and `ANALYTICS_UNMASK_PROPS`. - `product-analytics.ts`: removed `maskTextFn`, so rrweb uses its default star masking. `maskAllInputs`, `blockClass`, `maskAttributeFn`, `mask_all_text` and `mask_all_element_attributes` do not change. - Removed the `analyticsChrome` prop from `Button`, `Label`, `TabsTrigger`, `EmptyState` and `SidebarNavItem`, and from all sidebar call sites. - Added `ANALYTICS_MASK_CLASS` to the content element (not the chrome) of these surfaces: - Link preview cards, task quicklook description, external object pills and rows, related-work titles. - Email message cards and email send intents. - Connected account names and email addresses (Browse, Connections, app detail headers, app sidebar, connection choice list, connection access and repair cards), GitHub login and repositories, Slack workspace names, chat channel and linked user labels. - Inbox and decision rows (subject and detail), obligation evidence, Needs You notes and receipts, decision options and result summaries, and the interaction card roots (questions, options, answers, tool action requests, suggested tasks). - Approval payloads, action card input and reason, recovery card headline, evidence and options, run ledger retry detail, interrupt handoff previews. - Run transcripts, run failure details and run events on the agent page. - Search result snippets, toast bodies, voice and call transcripts, case summary, files and fields, attachment previews, artifact previews (text, tables, patches and documents). - Replaced `unmask-allowlist.test.ts` with `replay-masking.test.tsx`. It checks that unmarked chrome records as it is, that each marker masks, that a rendered voice transcript and run transcript mask, that each surface above has the class, and that autocapture records no element text. Updated `privacy.test.ts`, `shared-chrome.test.tsx` and `product-analytics.test.ts`. - Updated `doc/cloud-browser-analytics.md`: new default, marker classes, masked surfaces, and residual risk. ## Verification - `pnpm exec vitest run src/lib/analytics` in `ui/`: 83 tests pass. - Vitest for the 48 test files next to the changed components: 994 tests pass. - `tsc --noEmit -p ui/tsconfig.json`: no new errors (only the existing generated Storybook catalog errors). - `pnpm verify` exits 2 because a doc is a global CI input. Required CI supplies the broad coverage. - Manual: I ran an isolated local instance with a Cloud-like browser analytics config. `PAPERCLIP_POSTHOG_HOST` pointed to a local capture server, not PostHog. I opened a task page in headless Chromium, hovered a GitHub pull request link, and decoded the captured `/s/` replay payloads: - Clear text: sidebar (`New task`, `Chat`, `Tasks`, `Inbox`, `More`), breadcrumb, task title `Quarterly launch checklist`, `REP-1`, the Connectors promo heading and button. - Masked (stars only): the markdown task body, the chat comment, the link preview card (pull request title, author, URL, status), and the company menu. - Marker strings in the body, comment and preview never appeared in clear text in any payload. - The page `<title>` (task title and company name) is in the replay head. This is accepted residual risk. ## Risks - Text that is not marked is now recorded. Task titles, agent names, project names, company names, routine names, artifact and document titles, and activity rows are visible in replays. The owner accepted this. An agent can copy email or Slack text into a task title. - A new surface that shows connected-account content without `ANALYTICS_MASK_CLASS` is recorded until somebody marks it. `replay-masking.test.tsx` and the doc list the marked surfaces. - The masking options in `posthog.init` take precedence over the PostHog project masking settings (posthog-js 1.450), so a dashboard change cannot unmask content. Keep the project settings empty to avoid a console warning. - Some masked areas also mask nearby fixed labels (for example the interaction card buttons). This is safe. - Analytics is off on self-hosted and local installs, so they see no change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (claude-opus-5-5), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7f9867895c |
fix(ui): keep automated work out of personal recents (#22024)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The sidebar helps each person return to their own work. > - A task visit previously added any readable task to personal recents. > - Automated updates also moved old tasks above newer personal work. > - The Mine inbox already defines personal participation. > - This pull request applies that rule to recent task candidates and preserves their order during background updates. > - Users can see current task status without a stream of unrelated automated work. ## Linked Issues or Issue Description **What happened?** Routine tasks and imported GitHub review tasks appeared in personal recents after a read. Later agent updates moved these tasks up the list. A connected account or stored creator ID could make these tasks look like personal work. **Expected behavior** A task must pass the viewer's existing Mine participation rule before it appears in personal recents. Routine ownership, connected-account provenance and reads alone do not qualify. A viewer can qualify through manual creation, assignment, a direct comment, an allowed user action or a live request addressed to them. Inbox archive state does not remove personal history. Passive reads and machine updates must not advance its order. **Steps to reproduce** 1. Open an automated routine task or imported review task without participating in it. 2. Check the sidebar recents. 3. Let an agent update an older task while newer personal work is in the list. 4. Before this fix, the automated task appears and background updates can move older tasks up. ## What Changed - Add a viewer participation field to bounded navigation summaries. Reuse the canonical Mine query and existing access checks. The field is a read projection; no database migration is needed. - Show only confirmed personal tasks. Remove old nonparticipants after a successful read. Do not trust participation from local storage or full task caches. - Keep status and title snapshots current without changing personal order. Explicit local edits and comments can advance recency. - Migrate browser history once to a new storage version. Older tabs cannot restore machine-driven ordering. - Keep confirmed rows visible if a new batch request fails. Keep these facts within the current account and company. Hidden candidates do not bypass the one-second reorder delay. - Record sidebar renames as local activity. Place a first passive visit below existing history. - Use the saved comment time for comments sent with reassignment or custom run settings. Pending sends do not advance recency. - Keep readable subtasks as context beneath qualifying parents. Keep inbox archive independent of recents. - Update the design rules, UI behavior notes, regression tests and Runner source contract pin. ## Verification - Current head: `92ece0e61859a6c54b714b3611f062ff9b5633bb`. The PR merges cleanly with current master. - Latest red/green: both configured-comment cases failed before the fix and now pass. Reassignment and custom run settings advance recency only after the comment saves, using its saved time. - Latest local checks: 339 tests passed across IssueDetail and recent storage, hooks and sidebar. UI typecheck and diff checks passed. - Earlier local coverage: 592 targeted API/UI tests passed. After the master conflict resolution, all 29 overlapping Gmail tests and the 548-input source registry check passed. - Both Greptile and Paperclip Review report 5/5 on the current head. All four review threads are resolved. - Current-head [CI attempt 2](https://github.com/paperclipai/paperclip/actions/runs/44132164473) passed, including the final verify gate, build, typecheck, browser shards, serialized tests, Runner checks and all general-test shards. The workspace-a (1/2) retry passed after the first attempt hit an asynchronous jsdom/Radix event error in NewAgentDialog. - `pnpm verify` requires broader CI because DESIGN.md changed. No full local suite or deployed-browser acceptance was claimed. This change is not deployed. ## Risks - Unknown eligibility stays hidden until the summary succeeds. Failed requests preserve saved candidates and previously confirmed state within the current account and company, including after batch changes. - This intentionally removes automated tasks from old personal history when the viewer has no qualifying participation. - Readable subtasks under a personal parent remain context. They do not become independent recent entries from machine activity. - No task ownership, routine definition, inbox membership or permission rule changes. ## Model Used OpenAI GPT-6, running as Codex, with reasoning, repository tools and focused test execution. No additional model workers were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
8172d27089 |
feat(cloud): browser product analytics through a same-origin PostHog proxy (#22012)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Cloud runs the same app for hosted customers, and the same UI ships in the npm package. > - The PostHog project for Cloud gets only server events today. It has no page views, sessions, replays or people. > - So the Product and Web analytics pages are empty, and we cannot see how Cloud users move through the board. > - This pull request adds browser analytics for Cloud only. It is off until an operator sets two switches on a Cloud stack. > - Self-hosted installs do not change: they never download posthog-js and never serve the PostHog proxy. > - The benefit is real page views, sessions, heatmaps, Web Vitals and masked session replays for Cloud, linked to the user and the workspace. ## Linked Issues or Issue Description **Subsystem affected** Board UI (analytics bootstrap and privacy markers), server session payload and a new same-origin proxy route, shared validators. **Problem or motivation** Cloud sends only server events to PostHog. There is no browser SDK, so PostHog has no page views, sessions, replays or identified people for Cloud. We cannot see where Cloud users get stuck. **Proposed solution** The signed-in session gives the UI a PostHog config only on a Cloud-managed stack with `PAPERCLIP_POSTHOG_PROJECT_TOKEN` and `PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`. The UI loads posthog-js after the page settles. Events go through a strict same-origin proxy at `/_ph`, so ad blockers do not drop them. Replay masks all text and all inputs by default. **Alternatives considered** Bake a key into the UI build: refused, because the same build ships to self-hosted users. A generic reverse proxy: refused, because it would forward any path. Mask only content areas: refused, because a missed surface would leak; masking is deny by default. **Additional context** This is the browser part of the Cloud direct PostHog plan. The server direct sender is already on master (#22006). Fleet configuration is a separate change. PostHog person deletion on account deletion is a follow-up. ## What Changed - Uses the shared Cloud PostHog config (`resolveCloudPostHogConfig` in `server/src/services/cloud-posthog-config.ts`, already on master from the server direct-sending change). This PR does not change it. - `server/src/services/cloud-posthog-browser.ts`: `resolveBrowserAnalyticsConfig` returns `{ token, apiHost: "/_ph", uiHost, environment, stackId, replayEnabled }` only when `PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`. `environment` is always set: the explicit variable, else `staging` for a `*.staging.paperclip.app` host, else `production`. - `GET /api/auth/get-session` carries the new optional `browserAnalytics` field (the existing signed-in bootstrap request, so no new request on the cold path). `packages/shared` has `browserAnalyticsConfigSchema` and the session schema field. - `server/src/routes/posthog-proxy.ts`: a strict proxy, mounted before the body parsers and only when browser analytics is on. It forwards only these requests and answers 404 to all others without contacting PostHog: - `POST /e/`, `/i/v0/e/`, `/batch/`, `/s/`, `/flags/`, `/decide/` to the ingest host. - `GET /array/<token>/config(.js)` and `GET /static/[<version>/]<name>.js` to the assets host (derived from the ingest host). - `stream.pipeline` joins the streams, so an error on either side (PostHog closes the connection partway through a response, or the browser goes away) closes both and never goes unhandled. It builds a new header set from an allowlist plus `X-Forwarded-For`. It never forwards the inbound Host, Cookie or Authorization headers. It drops `Set-Cookie` from responses. Bodies stream; nothing is buffered. It refuses bodies over 64 MiB (413, also for chunked bodies that grow past the limit) and limits requests in flight to 16 per client address and 256 in total (429). - `ui/src/lib/analytics/product-analytics.ts` (the only module that talks to posthog-js) and `ui/src/components/ProductAnalyticsGate.tsx`: - posthog-js loads after the page settles (or at a 15 s deadline), only when the session has a config. It loads as PostHog's browser script `/_ph/static/array.js` through the proxy, the same way posthog-js itself loads its extensions (recorder, web vitals). It is not an npm dependency and is in no bundle chunk. - Init: page views on history change, page leave, autocapture, Web Vitals, heatmaps, `person_profiles: "identified_only"`, `tracing_headers` for our own host only, replay only when `replayEnabled`. Surveys, product tours, web experiments and conversations are off. - Super properties on every event: `deployment_source: "cloud"` and `environment`. - After sign-in: `identify(session.userId, { email, name })` (person properties only). `session.userId` is the authenticated actor's user id verbatim (for example `paperclip-id:...`): `GET /api/auth/get-session` copies `req.actor.userId` into it, and server events use that same actor id as `distinct_id`, so browser and server events land on one person. Tests: `auth-routes.test.ts` (the session returns the actor id verbatim) and `ProductAnalyticsGate.test.tsx` (identify uses it) and `group("workspace", stackId)`. On sign-out: `reset()`, then the super properties again. `useSignOut` calls it before it leaves the page, because Cloud sign-out navigates away while the session query is still populated. - Privacy (`ui/src/lib/analytics/privacy.ts`), deny by default: - Replay masks every text node (`maskTextSelector: "*"` with `maskTextFn`) and every input value (`maskAllInputs`). - Only elements with `ANALYTICS_UNMASK_PROPS` show their text: sidebar items whose label is a fixed string (`analyticsChrome`, added at the 99 call sites with literal labels), and a shared `Button`, `Label`, `TabsTrigger` or `EmptyState` only when its caller passes `analyticsChrome`. None of them opts in by default, because they also render customer text (agent names, document titles). Sidebar items with dynamic labels (recent tasks, agents, projects) stay masked. - Replay masks attribute values too (`maskAttributeFn`, deny by default). Only layout attributes (`class`, `style`, `role`, `data-state`, SVG geometry), rrweb metadata and stylesheet links keep their values. `title`, `aria-label`, `alt`, `placeholder`, `src`, `href`, `data-*` and element references (`id`, `for`, `aria-labelledby`, `aria-describedby`, `aria-controls`, which often embed names or slugs) are always masked, even on unmasked chrome. So the organization switcher's `aria-label`, task titles in `title` attributes, image sources and download links with file paths never reach a replay. - `before_send` (`scrubCapturedEvent`) drops the `title` property from every event, because posthog-js copies `document.title` (task and organization names) into page views. Every URL string in every event, including replay meta events, keeps only its origin and path: query and hash values never leave (they can hold tokens, `code`/`state`, file paths or search text), and the token segment of `/board-claim/<token>` and `/invite/<token>` is redacted. It recurses only into plain objects and arrays, so the `Date` timestamp stays intact. - `ANALYTICS_MASK_CLASS` keeps customer content masked even inside unmasked chrome: `MarkdownBody` (task and issue bodies, comments, chat messages, documents and artifact markdown), `MarkdownEditor`, the task chat thread, the issue chat thread, the comment timeline, the file viewer (source, markdown and CSV views) and the organization name in the organization switcher. Text in any `contenteditable` editor is always masked. - `ANALYTICS_NO_CAPTURE_CLASS` (`ph-no-capture`, a blank box in replay) on every password-type `Input` (about 40 secret, token and API key fields), environment variable values and new secret values. - Autocapture never records element text or attribute values (`mask_all_text`, `mask_all_element_attributes`). - Connected-account content (Google user data and Slack data: email bodies, calendar events, Slack messages in tasks, chat, tool results, previews) never appears unmasked in a replay. Replay masks all text by default; nothing that renders connected-account content is in the unmask allowlist; autocapture records no element text anywhere (`mask_all_text`). `ui/src/lib/analytics/unmask-allowlist.test.ts` pins the allowlist: only 7 files may spread the unmask attribute, sidebar opt-ins need a literal label, and any `analyticsChrome` opt-in on `Button`/`Label`/`TabsTrigger`/`EmptyState` fails until it is added to a reviewed list (empty today). The same test renders Gmail, Calendar and Slack-style tool results next to unmasked navigation and checks they are masked. - Guard test: UI code must never call `capture()` with a `PAPERCLIP_EVENTS` name, so each event has one sender. - Docs: `doc/cloud-browser-analytics.md` (switches, per-stack opt-out, proxy allowlist, identity, privacy) and a link from `doc/observability.md`. - No new npm dependency and no lockfile change: posthog-js comes from PostHog through the proxy at run time. (An earlier revision added `posthog-js` to `ui/package.json`; the Runner support suite requires every `ui` dependency in the reviewed lockfile, which a pull request cannot update, so the script approach replaced it.) ## Verification - `pnpm verify`: see the result below. - Focused tests: - `server/src/__tests__/cloud-posthog-browser.test.ts`: exposure matrix (self-hosted with every switch set, Cloud without token, without stack ID, without the browser switch, switch not `1`, enabled, replay off), environment fallback, proxy mount only when enabled, allowed routes, 404 for disallowed methods and paths (including `//host` and `..` paths) with no upstream contact, Host/Cookie/Authorization stripped, `Set-Cookie` dropped, 502 when PostHog is down, and no unhandled error when PostHog closes a response partway (red on the old `pipe()` code, green now). - `server/src/__tests__/auth-routes.test.ts`: the session carries `browserAnalytics` only on Cloud with the token and the switch. - `ui/src/lib/analytics/*.test.ts`, `ui/src/components/ProductAnalyticsGate.test.tsx`: no import of posthog-js without a config or when signed out; load after the page settles, once; identify, group, reset with a mocked posthog; email and name never become event properties; deny-by-default replay masking; guard test. - Real run (local, headless Chromium, production UI build, never the real PostHog project): - Self-hosted server, no env: 0 requests for PostHog's script and 0 requests to `/_ph` in 9 cold loads and in a 40 s session with navigation. `browserAnalytics` is `null`. `/_ph/e/` falls through to the UI. - Cloud config: a full Cloud boot needs S3, a managed database and the control plane. So a small local stand-in served the same production UI build, mounted the real proxy from `server/src/routes/posthog-proxy.ts`, added the real `resolveBrowserAnalyticsConfig()` output to the session, and forwarded other API calls to the local server. `PAPERCLIP_POSTHOG_HOST` pointed at a local capture server. - Captured through `/_ph` only (0 requests to any posthog.com host): `$pageview` on each client navigation (5), `$identify` with `$set: { email, name }`, `$groupidentify`, `$groups: { workspace: <stack id> }` on events, `environment` and `deployment_source` on every event, `$web_vitals`, `$autocapture` with no `$el_text`, `$$heatmap`, and 10 `$snapshot` replay events. A search of every decoded `/e/` and `/s/` payload (events, replay text and replay attributes) found the email only in `$identify`'s `$set`, and found none of: the task body, the typed comment, the task title, the organization name. No Cookie header reached the capture server. 96 of 113 API requests carried `X-POSTHOG-SESSION-ID` (the others ran before posthog-js loaded). - Client-side visits to `/invite/INVITESECRET123` and `/cli-auth/abc?token=CLISECRET456` reached PostHog as `/invite/REDACTED` and `/cli-auth/abc` (no query); neither secret was in any payload. - Disallowed proxy requests (`PUT /_ph/e/`, `GET /_ph/api/projects/`, `POST /_ph/static/x.js`) answered 404. - Sample replay payload (decoded rrweb text nodes from a captured `/s/` request). The task body, the typed comment, the task title, the organization name and the email were not in any replay payload: ```json {"text_nodes": [ {"parentTag": "title", "textContent": "****** ** * *********"}, {"parentTag": "a", "textContent": "**** ** **** *******"}, {"parentTag": "span", "textContent": "****** **"}, {"parentTag": "span", "textContent": "*** ****"}, {"parentTag": "span", "textContent": "Tasks"}, {"parentTag": "span", "textContent": "Inbox"}, {"parentTag": "h2", "textContent": "******* *******"}, {"parentTag": "p", "textContent": "****** *** * ******* ** ******"} ]} ``` - Cold load, 9 fresh-profile loads each, median, same stand-in with analytics off and on: | | DOMContentLoaded | Board content visible | | --- | --- | --- | | Analytics off | 128 ms | 336 ms | | Analytics on | 126 ms | 334 ms | Final head. The difference is inside run-to-run noise (earlier pairs: 332/335 ms and 335/348 ms). PostHog's script loads only after the content is visible. ## Risks - Inert by default. Without a Cloud-managed stack, a project token, a stack ID and `PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`, the session field is `null`, the proxy is not mounted and no analytics code runs. The OSS package does not change behavior. - Per-stack opt-out: `PAPERCLIP_POSTHOG_REPLAY_ENABLED=0` turns replay off; unsetting `PAPERCLIP_POSTHOG_BROWSER_ENABLED` turns all browser analytics off. Both apply on the next restart. - Replay text shown in the clear comes only from fixed sidebar labels and from shared components whose caller opts in. Today no `Button`, `Label`, `TabsTrigger` or `EmptyState` call site opts in, so replays show layout, clicks and navigation with most copy masked. Opting in fixed-label call sites is a follow-up. Review sample replays on staging before production. - Image `src` values are masked, so images record as empty boxes. Sandboxed iframes (HTML artifacts, MCP apps) also record as empty boxes. Replays therefore show layout and product copy, not customer media. - PostHog's script is not version-pinned: the assets host serves its current browser build, as for any PostHog snippet install, and posthog-js already loads its extensions from that host at run time. It runs only on Cloud stacks with the browser switch on. - The proxy needs no sign-in, like PostHog's documented proxies: posthog-js sends some events while a page unloads, and the proxy can only reach the two PostHog hosts, on the listed paths, with the public project token. Body size and in-flight limits bound the load one client can add. - posthog-js rejects bots (including headless browsers) on its own, which is the wanted production behavior. - `SidebarNavItem.production.tsx`, `Sidebar*.tsx` and the shared `ui/*` components changed by one attribute each; no visual change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, headless browser runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d1b5cfe038 |
feat(connections): let users delete data read through a connection (#22010)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents call connected apps, such as Google Workspace, through
Paperclip's tool gateway
> - Paperclip saves part of each tool result, approval previews, app
views and the full run transcripts that contain tool output
> - No user action deleted this data. Removing the connection left it in
place
> - Google's restricted-scope review requires that users can delete the
data an app obtained, and that the privacy policy says how
> - This pull request adds "Delete data read through this connection"
for any connection, on the connection's page and as an option when you
remove it
> - The benefit is that the picker text ("They stay there until you
delete them") is now true, and the privacy policy can link to a help
page
## Linked Issues or Issue Description
**Subsystem affected**
Connections (tool gateway audit tables, run logs, run events) and the
Connectors UI.
**Problem or motivation**
Paperclip stores content from a connection's tool calls in several
places and gives users no way to delete it:
- `tool_invocations.arguments_summary/result_summary` and
`tool_call_events.arguments_summary/request_summary/result_summary` keep
the first ~4,000 characters of each redacted value
(`summarizeToolValue`, `tool-access-policy.ts` `redactArguments`). No
retention limit.
- `tool_action_requests` keeps `canonical_arguments_summary`,
`preview_markdown` and `signed_arguments`. The linked approval card
(`issue_thread_interactions`) keeps copies in `payload.detailsMarkdown`,
`payload.toolAction.previewMarkdown`,
`payload.toolAction.argumentsSummaryJson` and
`result.toolAction.resultSummary`.
- `tool_mcp_app_views` keeps `tool_input` and `tool_result` for MCP app
views.
- Run transcripts hold full tool output: the run log (`local_file` with
an optional S3 mirror, or `s3_segments`/`s3_archive` object journals),
`heartbeat_run_events.message/payload`, and
`heartbeat_runs.stdout_excerpt/stderr_excerpt`. Neither run-log store
had a delete path.
- Connection removal (`removal.ts`) archives the connection and revokes
credentials. It deletes no content.
**Proposed solution**
A durable, resumable, company-scoped deletion request per connection.
See "Design" below.
**Alternatives considered**
- Delete synchronously in the request. Rejected: an interrupted request
leaves partial state that needs a person to retry it.
- Delete only the run events that mention the connection. Rejected: tool
output also reaches the model's later messages, so the safe unit is the
whole run transcript.
- Delete the audit rows. Rejected: counts, outcomes and hashes feed the
activity history and usage counts.
## Design
**Rule.** The job never drops a reference to content it has not
confirmed deleted. It is marked complete, and writes the
`tool_connection.content_deleted` activity entry, only after every step
succeeded. A step that cannot confirm its work throws. The request then
stays pending, the UI says "Paperclip keeps trying until it finishes",
and the worker retries it.
**Store inventory.** Every place that stores tool arguments, results or
previews. I found them from `summarizeToolValue`, `redactArguments`,
`writeAudit`, and every column or JSON field named like summary,
preview, arguments, result, toolInput or toolResult.
| Store | Content | Cleared by this PR | Proof |
| --- | --- | --- | --- |
| `tool_invocations.arguments_summary`, `result_summary`, `progress` |
Redacted request/result preview, provider progress | Yes: `summary`
becomes `""` with `contentDeletedAt`; `progress` null; size and sha256
kept | content-deletion test 1 |
| `tool_call_events.arguments_summary`, `request_summary`,
`result_summary` | Same previews | Yes, same | test 1 |
| `tool_access_audit_events.details.argumentsSummary` (`writeAudit`) |
Argument preview | Yes, same; decision, ids, size, hash kept | test 1 |
| `tool_action_requests.canonical_arguments_summary`,
`preview_markdown`, `signed_arguments` | Approval preview and arguments
| Yes, for settled requests | test 1, card retry test |
| Approval card in `issue_thread_interactions`:
`payload.detailsMarkdown`, `payload.toolAction.previewMarkdown`,
`payload.toolAction.argumentsSummaryJson`,
`result.toolAction.resultSummary` | Approval preview, arguments,
executed result | Yes. Cards are found from every settled request of a
batch, in the same transaction | test 1, card retry test |
| `heartbeat_runs.context_snapshot` and `agent_wakeup_requests.payload`
(`toolAction(s)[].resultSummary`,
`paperclipAgentMessage.untrustedToolResults[].resultSummary`) | Approved
action result delivered to the agent | Yes, for settled runs and settled
wakes | test 1 |
| `tool_mcp_app_views.tool_input`, `tool_result`, `model_context` | App
view input, result, shared context | Yes: rows deleted | test 1 |
| Run log: object journal, archive journal, original `local_file` log
and its legacy S3 mirror | Full transcript | Yes, every copy, for each
settled run that called the connection or received an approved result |
run-log-store tests, test 1 |
| `heartbeat_run_events.message`, `payload` | Transcript events | Yes,
except structural events (lifecycle, terminal, presentation, usage,
compaction, session/turn start, `native.*`) | test 1 |
| `heartbeat_runs.stdout_excerpt`, `stderr_excerpt`, and every string
`summary`, `result`, `message`, `text`, `output`, `content` or `body` at
any depth of `result_json` (including `nativeResult`, `acceptedResult`,
`semanticResult` and their `continuation`) | Output excerpts and final
answer | Yes; schema and control fields kept | test 1, canary |
| `approvals.payload` (formal board approval) | Tool name, risk and
hashes only | Not needed: no content | — |
| `tool_mcp_tasks` | Request hash and sealed provider task id; no result
| Not needed: no content | — |
| `tool_mcp_elicitations.questions` and their question cards | Questions
the provider asked a person | No. These are provider prompts, not tool
results, and the answered question stays in the task. Follow-up if
wanted | — |
| `tool_mcp_skills`, `tool_mcp_skill_files`, `tool_catalog_entries` |
Skills and tool descriptions the provider publishes | No. These are
provider definitions, not data read through tool calls | — |
| Transcripts of later runs that read an app view's `model_context` |
Shared app context | No. Nothing records which runs read it. The help
page says so. Follow-up | — |
| `issue_comments.body`, issue documents (including continuation
summaries), files, chat messages | Task content written by agents or
Paperclip | No, by design. The user removes it by editing or deleting it
or the task. The confirmation says in bold that anything agents wrote
into tasks is not removed, and so does the help page. The canary
excludes `issue_comments.body` by name for this reason | canary asserts
it is kept |
| Raw provider traces | Full provider traffic | No. Admin opt-in,
encrypted, expire within 24 hours | — |
| Feedback exports already sent | Run logs the user chose to share | No.
Already outside Paperclip | — |
| Content of runs of a deleted task or company | Same stores | No.
Follow-up | — |
**Durable request (schema change).** I added one table,
`tool_connection_content_deletions`, and migration `0490`. I followed
`doc/DATABASE.md`: edited the schema, exported it from
`packages/db/src/schema/index.ts`, ran `pnpm db:generate`, and ran the
migration drift and compatibility tests. The classifier marks it
`online_compatible` (a new table and its indexes only). No existing
table can hold a durable per-connection request with a cutoff time,
cursor, counts and retry state. Without it, an interrupted deletion
could resume only when a person pressed the button again.
**Resume after restart.** The worker uses the shared
`registerDeadlineWork` lifecycle on a new
`tool-connection-content-deletion` queue. The request insert signals the
queue in the same transaction. At server start the worker runs once,
`next()` finds any pending row, and processing continues from the saved
phase and cursor. Nobody presses anything. `next()` returns a deadline
only while a pending row exists, so an idle instance sends no recurring
query. A failure records `attempts` and `last_error` and retries after
30 seconds.
**One run selection.** `runsWithStoredContentSql` selects the runs whose
transcripts are deleted: runs that made a tool call or tool-call event
through the connection, and runs whose context lists one of the
connection's approved action requests, limited to runs that still hold a
log, an excerpt, result text or a content event. The dialog count, its
in-progress warning and the job's `runs` phase all use this one
function.
**Phases.** `invocations`, `events`, `auditEvents`, `appViews`,
`deliveredResults` (50 action requests per page; matching runs and wakes
are locked with `FOR UPDATE` before their documents are rewritten),
`runs` (the shared selection, 50 run ids per page, then one run at a
time), `done`.
**Canary test.** `tool-connection-content-deletion-canary.test.ts` puts
a unique marker in a tool call's arguments and result, an approved
action with a preview, a result delivered to a later run and its wake, a
native run that quotes the marker in its answer and events, a run log
migrated from a local file with an S3 mirror to an object journal, an
MCP App view and the audit tables. It checks that the marker is first
present in the database, the object store, the legacy mirror and the log
files. After the deletion it scans every `text`, `varchar`, `json` and
`jsonb` column of every table found through `information_schema` (no
hand-written table list), every object in both stores and every file
under the log directory. Any hit fails with the table and column or the
key. It runs three times: a complete deletion, a deletion interrupted
twice by storage failures and resumed, and two deletions for different
connections on one task at the same time. The only exclusion is
`issue_comments.body`, named with its reason.
**Batching and time budget.** Tool rows are read in pages of 200 with
keyset pagination on the existing `(company_id, created_at)` indexes,
bounded by the request time. Each page's writes and the saved progress
commit in one transaction. Counts are therefore exact, and a crash
repeats at most one page, idempotently. Runs advance one at a time from
the shared selection. Each 1,000-event page is its own step. Log removal
is its own step and runs before its progress transaction, so object
storage calls never run inside a database transaction. The 10-second
budget is checked after every step. Nothing changes on the agent boot or
tool-call path.
**In-flight work.** A run counts as settled only when its status is
terminal, `finished_at` is set, cost accounting is done, and no native
finalization is open. Calls, approvals, events, app views, delivered
results and transcripts tied to other runs or wakes are skipped and
counted. The confirmation says "N runs are still working and will be
skipped. Delete again after they finish."
**Permissions.** Board users with configure access (connection manager
or creator) can delete, like `DELETE /tool-connections/:id`. Agents,
viewers and other companies are refused. The activity entries
`tool_connection.content_deletion_requested` and
`tool_connection.content_deleted` hold counts only.
**UI reads.** The dialog loads counts once when it opens. While a
deletion runs, it polls `GET
/tool-connections/:id/stored-content/deletion`, a single indexed row
read, every 2 seconds. Polling stops when the job completes or the
dialog closes. The Delete button is not gated on counts.
## What Changed
- `packages/db`: new `tool_connection_content_deletions` table and
migration `0490` (online compatible: new table and its indexes only).
- `packages/shared`: `ToolConnectionStoredContent`,
`ToolConnectionContentDeletion` and counts types; optional
`contentDeletedAt` on `ToolRedactedValueSummary`.
- `server`: `RunLogStore.remove`, which deletes every copy of a run's
log (`composeRunLogStore`); `content-deletion.ts` service; deadline
worker wired into startup and pre-drain; `GET
/tool-connections/:id/stored-content`, `POST` and `GET
/tool-connections/:id/stored-content/deletion`, and `DELETE
/tool-connections/:id?deleteStoredContent=true`; OpenAPI entries.
- `ui`: "Delete data" row on the connection's Permissions tab with a
confirmation that shows counts, skipped runs, and "This cannot be
undone"; "Also delete data read through this connection" checkbox in the
Connectors removal dialog.
- Docs: user help
`docs/guides/board-operator/deleting-connection-data.md` (for the
privacy policy link) and an API note in
`doc/connections/CONNECTOR-PLAYBOOK.md`.
- `tests/runner-e2e/source-sdk-public-inputs.json` refreshed for the
shared type change.
`server/src/routes/tool-access.ts` (3,010 lines) and
`ui/src/pages/apps/Browse.tsx` (1,722 lines) grow by about 40 and 20
lines. The routes reuse that file's connection access checks; the dialog
reuses the existing removal confirmation. The new logic lives in its own
files.
## Verification
- `server/src/__tests__/tool-connection-content-deletion.test.ts`
(embedded PostgreSQL): content is gone from every "Yes" store in the
inventory; rows, hashes and sizes stay; another connection and another
company are untouched; a running run is skipped and is deleted after it
settles; a storage failure leaves the request pending with `retrying`,
and the next pass completes it; an approval-card update that fails once
is cleared on the retry; an unrecognised log backend keeps the pointer,
keeps the job pending and writes no completion entry; a zero time budget
yields after every step and still finishes; a repeat request returns the
pending one; agents and outsiders are refused; removal with
`deleteStoredContent=true` queues the deletion.
- `run-log-store.test.ts` and `object-run-log-store.test.ts`: `remove`
deletes local and mirrored logs and object journals, removes a migrated
run's original local file and mirror with its journal, refuses an object
log when object storage is not configured, and is idempotent.
- `ui/.../StoredContentDeletion.test.tsx`: confirmation text, counts,
skipped-run warning, and the request.
- Red: these tests fail on `master` because `remove`, the service and
the routes do not exist. Green on this branch.
- Ran: the tests above, `openapi-routes`,
`server-startup-feedback-export`, `tool-connection-removal`,
`packages/db` migration drift and compatibility tests, all
`ui/src/pages/apps` tests (950), `pnpm check:token-gates`, server and UI
`tsc --noEmit`. `pnpm verify` exits 2 (global CI input changed), so
required CI supplies broad coverage.
## Risks
- Deleting is permanent by design. The confirmation says so.
- A `local_file` log written on another replica's local disk stays on
that pod's disk until the pod is replaced; its S3 mirror is deleted. New
logs on S3-configured instances use object journals, which are deleted.
- On versioned buckets, earlier object versions remain until the
bucket's lifecycle policy removes them (the existing journal delete path
does not remove versions).
- Not in this PR (follow-ups): the "No" rows of the inventory (except
agent-written task content, which is out of scope by design), mainly the
same deletion when a task or a company is deleted, and later runs that
read an app view's shared context.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
a5da0f6032 | fix(ui): scope the inbox badge to the signed-in user (#21953) | ||
|
|
ee6922f705 |
perf(tasks): open recent tasks without waiting on a slow ten-task preload (#21939)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A person opens a task from the sidebar's Recent Tasks, the Inbox or the Tasks list, and the task chat shows its first frame from a `chat-bootstrap` read > - On staging, a task opened from Recent Tasks soon after a page load took 15.3 s to show its chat. The same task from the Inbox took 0.19 s > - Two defects cause this. Every task's bootstrap paid a fixed ~1.8 s for its latest runs. Recent Tasks preloaded ten tasks in one request (17.7 s), and an open waited for that whole request > - Recent Tasks also warmed its tasks only once per page load. Unread cache entries are dropped after five minutes, so its tasks were cold for the rest of the session > - This pull request makes the runs read use indexes, lets an open read its own task instead of waiting for a ten-task preload, and re-warms Recent Tasks on navigation > - The benefit is that a task from Recent Tasks opens as fast as from a task list ## Linked Issues or Issue Description Refs #19818, Refs #16713 (earlier proposals of the same `runsForIssue` rewrite; this PR adds a query-plan regression test and the client fixes). **What happened?** On staging, a click on a Recent Tasks link about 6 s after a page load took 15.3 s until the chat thread showed. The Inbox opened the same task in 0.19 s. **Steps to reproduce** 1. Load a page with a populated Recent Tasks sidebar. 2. Within a few seconds, click a recent task. 3. The chat stays on its loading state for many seconds. **Expected behavior** The task opens about as fast as from the Inbox or the Tasks list. **Root cause (measured on staging)** - `GET /issues/:id/runs?view=recent` took 1.7–1.9 s of server time for tasks with 1 to 24 runs. Every other first-frame read took 80–350 ms. The query filtered with `issue_id = $1 OR EXISTS (activity_log … run_id = heartbeat_runs.id)`, so Postgres walked every run of the company. - The bootstrap reads two tasks at a time, so the Recents preload of ten tasks took 17.7 s. Each task in a batch settles only with the whole request, and the page's reads wait for it (up to the 5 s share deadline, then read again under load). - `useRecentTaskOpenWarmup` remembered preloaded task ids for the life of the page and never preloaded them again. ## What Changed - `activityService.runsForIssue`: select the task's runs by `id IN (runs with issue_id = task UNION run ids from the task's activity)`. Each half uses its own index. The results are the same. - `startIssueChatBootstrap`: when the only request for a task is a background batch that is still loading, send a request for that task alone. All refs of the task point to the new request. The later batch answer only fills keys that are still empty or older. - `loadIssueChatBootstraps` (background): a task whose answer did not arrive is not held back for five minutes, so the next preload can retry it. - `useRecentTaskOpenWarmup`: use the background preload (five-minute repeat, the same as the cache GC time) and run again after each navigation (`navigationKey`). It no longer keeps a per-page-load set. `SidebarRecentTasks` and `CoordinatorSidebar` pass the pathname. ## Verification - `server/src/__tests__/issue-runs-read-cost.test.ts` (new): a company with 20,003 runs, and a task with 3 runs. EXPLAIN ANALYZE of the runs reads examined **40,011** run rows before the fix and **15** after (red/green). The test also checks the run order and the run that is linked only through activity. - `ui/src/lib/issue-chat-bootstrap.test.ts`: a new test opens a task while a ten-task preload is still loading. Before the fix it timed out waiting on the batch. After the fix it reads its own request. A second new test checks that a single-task request is still shared. - `ui/src/hooks/useRecentTaskOpenWarmup.test.tsx`: a new test re-warms Recents on a navigation after five minutes (red on the old hook). Another new test sends nothing for navigations inside five minutes. - Existing suites pass: activity service/routes, task cold open, agent conversations (197 tests); sidebar, Recents, navigation prefetch, issue detail cache, IssueDetail (341 tests). - `tsc --noEmit` is clean for `server` and `ui` (apart from the pre-existing Storybook index errors). - `pnpm verify` exits 2 (an opaque server setup dependency), so CI supplies the broad coverage. - After deploy: on staging, time a Recent Tasks click a few seconds after a cold load, and time `runs?view=recent`. ## Risks - Low risk. The SQL change keeps the same filter. It changes only how Postgres finds the candidate runs. The existing activity tests cover retry, credential and conversation cases. - An open during a background preload now sends one more bootstrap request for that task. This happens only on an open or a resting hover, never for each row. - Recents now sends at most one preload per five minutes while the person navigates. It sends nothing while the tab is idle or hidden. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, browser measurement on staging, code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f33c4cf6f7 |
feat(links): cache rich link status in the database and load it on idle (#21746)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task text contains links to pull requests and other work. > - A link has useful status that should be visible when a task opens. > - The current cache lives in the server process and the browser. > - A new browser or server restart has no stored status to show. > - This pull request stores link status in the database and reads it in batches. > - Idle page work updates stale status without a hover or a delay to task data. ## Linked Issues or Issue Description **Problem or motivation** PR status is unknown in a new browser until the user points to each link. The process cache cannot supply saved status after a server restart. Related work: #20593 and #20654. A search found no other PR for database link caching. **Proposed solution** Store status by company, viewer and canonical resource URL. Read saved status in one indexed batch query. After document load and core request settlement, use browser idle time to update missing or stale status. Use the same query for hover and idle work. **Alternatives considered** Adding provider calls to task reads would delay task loading. Company-wide reuse of personal GitHub results would cross viewer access boundaries. A periodic server worker would create work when no page needs the result. **Roadmap alignment** This extends the shipped rich-link feature. It does not duplicate a roadmap project or add a new provider integration. ## What Changed - Add an additive, repeat-safe `link_previews` migration. Store channel, resource kind, free-form status, JSON metadata and freshness. The table has no task relationship or GitHub status enum. - Bound each company/viewer cache to 1,000 rows across concurrent server instances. Remove rows expired for over 30 days during writes. Limit provider lookups to 600 requests per minute per scope/process. Keep cached batch reads available during backoff. - Add a database store owned by the link service. Reuse canonical results across tasks. Keep company and viewer isolation. Save hover and idle results. Preserve last known status during an outage and cache unavailable results. - Add a board-only batch GET endpoint, including viewer-role access. It reads at most 50 URLs with one indexed select and makes no provider calls. Both endpoints use private, no-store HTTP responses. - Add a page coordinator for mounted rich links. Reuse browser data, share pending reads, and defer provider work until document load, core request settlement and browser idle time. Limit automatic work to 200 unique links and three requests in flight. Remove queued work when links leave the page. Keep encoded batch URLs below 6 KB. - Keep exact navigation URLs, status styling, anchored cards and keyboard access. Add a page-load Storybook case and browser regression. The link service owns cache policy and provider calls. The browser coordinator owns page timing. The shared contract adds only the batch limit and response shape. The database adds storage because process and browser caches cannot serve a new browser session. Plugin resolver registration is outside this change; storage accepts future provider channels and status values. ## Verification - **All 93 affected tests pass:** 40 link service, real PostgreSQL and HTTP cases, plus 53 OpenAPI cases. Two SQL regressions and one HTTP case failed before the storage repair. Concurrent writers now retain at most 1,000 rows per company/viewer; cleanup removes long-expired rows and keeps recent stale status. Request backoff preserves cached reads and isolates companies/viewers. The earlier eight concurrent hydration regressions also pass. Focused store/service/unit types pass. Commands: `pnpm exec vitest run server/src/__tests__/link-previews.test.ts server/src/__tests__/link-preview-store.integration.test.ts server/src/__tests__/link-preview-routes.test.ts --maxWorkers=1` and `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts --maxWorkers=1`. - **All 50 affected server tests passed before the later server-only repairs** on the master-integrated source: 20 recovery/deadline cases, one authenticated review-counts HTTP case, 10 link-service cases, 13 real PostgreSQL cache cases and six link-route cases. Command: `pnpm exec vitest run server/src/__tests__/issue-recovery-deadlines.test.ts server/src/__tests__/tool-connection-review-counts-http.test.ts server/src/__tests__/link-previews.test.ts server/src/__tests__/link-preview-store.integration.test.ts server/src/__tests__/link-preview-routes.test.ts --maxWorkers=1`. - Reproduced the CI watchdog assertion locally twice before the test repair. Ordinary assignment recovery can win `wakeReason` while the watchdog wake coalesces into the same successful run with its full mandate. The test now verifies the persisted watchdog identity, watched issue and stopped fingerprint, plus exactly one review run. All 21 recovery/HTTP cases passed before integrating master; all 50 affected cases pass after integration. No runtime behavior was changed for this repair. - **Fresh static Storybook build and all 22 browser cases pass**, with two workers and retries disabled. They cover states, anchored hover, keyboard access, exact navigation, duplicate requests, reload persistence, expiry and saved-status hydration followed by one idle refresh. The first browser attempt could not launch because this session lacked Chromium; installing the expected browser resolved setup. That attempt had no behavioral verdict. - The cache repair remains covered by four PostgreSQL red/green regressions for expired memory and newer durable status during provider failure/timeout, plus check-time precedence and failed-read recovery. Earlier UI/database/focused server type and migration safety/compatibility/replay checks remain unchanged-input evidence. - Master `77ddf260bf9a3de3dcb8456a60f4f809d67401fd` is integrated cleanly. Link migration 0489 and its snapshot chain are unchanged. SDK source validation checks 545 inputs with no stale source identities or changed runtime/lock pins. Whitespace validation passes. - `pnpm verify --explain` returns exit 2 and requires broad CI. No full local suite or deployed acceptance is claimed. Earlier local full-server type checks exceeded memory; current-head CI supplies broad type/build/test coverage. - [Previous CI 43900336895](https://github.com/paperclipai/paperclip/actions/runs/43900336895) passed full types/build but failed on embedded PostgreSQL startup (`ECONNREFUSED`), the watchdog assertion above, and unrelated plan-task browser cleanup (activity-log foreign key on company deletion). One failed-jobs retry was requested. Its browser shard 7 now passes without code changes; the exact deletion-race cause remains unproven. The watchdog assertion is repaired in this revision. The previous failures remain recorded, not counted as an overall pass. - [Prior CI 43905482848](https://github.com/paperclipai/paperclip/actions/runs/43905482848) passed full types, build and all eight browser shards. Several test jobs were cancelled; the run has no overall pass. Its concurrent update finding is repaired. [Next CI 43909197530](https://github.com/paperclipai/paperclip/actions/runs/43909197530) also passed types, build and all browser shards, but several server jobs were cancelled. Greptile and Paperclip Review gave that revision 5/5. Its security scan found unbounded retained cache rows. The quota, retention and request-limit repair above addresses that finding. All ten review threads are resolved. - [Current CI 43912947278](https://github.com/paperclipai/paperclip/actions/runs/43912947278), fresh code reviews and a fresh security scan are pending on `335d89ef1668dcdeb66bb2059512a7c2da29e273`. Auto-merge is disabled until the current revision clears its gates. Published head: `335d89ef1668dcdeb66bb2059512a7c2da29e273`. Local head: `b37b87c704a2ba8a65233c689a6a743f4011ce14`. Both have tree `643ebd19b9729b1e6a6bb695442269c3db32be3a`; GitHub tree creation verified equality. ## Risks - Each company/viewer retains at most 1,000 rows. Writes remove excess rows and entries whose freshness expired over 30 days ago. Inactive scopes remain bounded and are cleaned on the next write. Company deletion removes all rows. No periodic worker runs. Scope writes use a transaction lock; the existing deadline and concurrency budget cover that work. - Last known status can be stale until idle or hover work completes. Provider failures keep that status through the negative-cache window. - Automatic enrichment adds bounded provider traffic. It runs only for mounted supported links, once per mount, with the existing timeout and server concurrency limit. - Cache storage is viewer-scoped because provider results can be private. New viewers load their own results. This prevents reuse across users with different repository access. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution and browser automation. The runtime does not expose the exact serving model ID or context-window size. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <Paperclip-Paperclip@users.noreply.github.com> |
||
|
|
29903bed59 |
fix(ui): add a space after composer autocomplete (#21754)
Preserve an ordinary space after composer autocomplete selections and place the caret after it. Cover all seven completion types and queries with trailing spaces. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
b5447432bd |
fix(ui): keep the active company on Cases and Status navigation links (#21794)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI builds company-prefixed links, such as `/PAP/cases`,
from short paths in the sidebar
> - The Cases and Status links use `/cases` and `/status`, but these
roots were not in the board route list
> - The router read `cases` as a company prefix and left the link
unprefixed, so the link only worked through the unprefixed redirect
> - When a person clicked Cases before the company list loaded, the
redirect picked an arbitrary company and showed its empty case list
> - The same race made `tests/e2e/cases-read-only.spec.ts` fail in "ci /
e2e shard (2/8)" on unrelated PRs (#21683, #21715, #21765)
> - This pull request adds both roots to the board route list and makes
the e2e test force the race
> - The benefit is that Cases and Status always open in the company the
person is in, and the flaky e2e test becomes deterministic
## Linked Issues or Issue Description
Refs #21683
Refs #21715
Refs #21765
**What happened?**
The read-only Cases e2e test failed about once per day with `Expected:
3, Received: 0` links in the case list (job 132875164249 on #21765, job
132680942843 on #21715). The failure screenshot shows the sidebar of a
different company ("Passive artifacts …", created by the previous spec)
and "No cases yet". The server log shows that the page asked for the
cases of the other company and never for the test company.
**Expected behavior**
A click on the Cases or Status link opens that page for the company in
the URL, also when the click happens before the company list has loaded.
**Steps to reproduce**
1. Have two or more companies.
2. Open `/<PREFIX>/issues` on a slow connection (the e2e test holds `GET
/api/companies`).
3. Before the company list loads, open More and click Cases.
4. The URL becomes `/cases`. When the list loads, the app redirects to
`/<OTHER>/cases` and shows another company's empty case list.
**Paperclip version or commit**
`master` at
|
||
|
|
5b750fdf25 |
feat(ui): add foldable sidebar subtasks with bounded refreshes (#21649)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - The sidebar helps people return to recent tasks. > - Recent parents need foldable child tasks. > - Busy companies must not refresh every child group for unrelated work. > - Exact unread counts must preserve personal reads and task access. > - This pull request adds child groups and bounds their refresh work. > - People can scan subtasks without repeated background requests. ## Linked Issues or Issue Description **Subsystem affected** Task navigation in the standard and Classic sidebars. **Problem or motivation** Recent parent tasks did not show their subtasks. The first implementation refreshed every previously viewed group. One unrelated update caused 52 child-page requests and one summary request after viewing 50 groups. Exact child counts also repeated privacy and unread checks per child. **Proposed solution** Show direct subtasks with independent folding and ten-row pages. Refresh affected groups and summary batches. Stop offscreen reads. Bound child-request concurrency. Aggregate viewer facts once and preserve the existing inbox unread rule. **Roadmap alignment** This extends existing task navigation. The existing feature PR is reused for its performance repair. No new PR duplicates it. ## What Changed - Add shared parent groups to both sidebar layouts. Preserve Classic row actions and active-child visibility. - Keep unread state personal to each task. Folding and paging do not mark tasks read. Counts include readable children beyond ten. - Target live refreshes with authorized current and previous parent IDs. Reuse parent metadata from the narrow run-binding read. Resolve fallback lookups once per delivery batch. Keep the previous summary fallback for older servers. - Limit child requests to three at a time. Cancel active and queued reads when groups leave the viewport. - Discard extra cached pages on Show less and collapse. - Compute exact navigation counts in one SQL statement. Materialize readable candidate IDs once. Use indexed participation sources and conservative unread candidates before the shared inbox rule. - Skip recursive privacy walks for standalone open tasks without a project. Keep the recursive rule for all restricted or inherited shapes and all delegated principals. - Add request-count, cancellation, reparenting, read-receipt, inbox-parity and privacy regressions. Include a disposable PostgreSQL benchmark. - Update product and API documentation. Retain the production Storybook states. ## Verification - Published head: `4148861437c1b6b635479ad92509dbc71155a232`. Its tree matches local `04be9d66d481d9bcd1c4d6158acf063a1e65ac7e`: `a23533e93d3d155918f0254d296a7fd2f0c7c12f`. - Rebased on master `ca23330c69b1f2b25bdadfcbda68317db97deaef`. Remote publication preserves prior feature commits. - The request regression failed before the fix and passes after it. An unrelated event now sends zero child requests. Current servers also avoid the unrelated summary request. Old-server events keep one summary fallback. - A relevant update after Show less reads one page. Reparenting refreshes two affected groups. Offscreen reads stay at zero and resume once on return. Active and queued cancellation tests pass. - The same synthetic authenticated HTTP workload with 50,000 read children improved from 767 ms to 92.84 ms median. Five warm samples per case. The all-unread case improved from 463.85 to 351.65 ms. Untouched children improved from 208.01 to 131.09 ms. The old-receipt case improved from 799.70 to 245.80 ms. These local measurements are not a production capacity guarantee. - Focused PostgreSQL routes, inbox-parity, privacy and WebSocket tests passed: 38 tests. - All 325 affected UI tests passed. All 38 focused PostgreSQL, privacy and WebSocket tests passed. UI typecheck and low-memory server declaration checks passed. - All 41 Chromium Storybook checks passed. They cover 20 states in both themes and keyboard, folding, paging and unread behavior. They use production components with HTTP fixtures. - Both new review findings failed as regressions before their fixes and pass afterward. Explicit child reads include waiting external conversations. Deleting a child publishes a separate parent-authorized update without deleted-child content. Both review threads are resolved. - `pnpm verify --explain` returned exit 2 because product documentation requires broader CI. [The prior CI run](https://github.com/paperclipai/paperclip/actions/runs/43711489306) found type errors in the new test fixtures. The same errors failed locally. The fixture fix passes `node server/scripts/typecheck-layers.mjs --layer test-tree-0` and all 21 issue summary route tests. Production code is unchanged. That CI run then found a live-delivery read regression. The existing test failed locally with eight statements against its six-statement budget. A new PostgreSQL regression also found three parent lookups for two same-batch calls. Run delivery now reuses parent metadata from its narrow run-binding query. Fallback lookups resolve to one promise. Both regressions pass without increasing the budgets. All 74 targeted route, WebSocket and privacy tests pass. The affected below, above and test-tree-0 typecheck layers pass. [CI attempt 2](https://github.com/paperclipai/paperclip/actions/runs/43715541480/attempts/2) passes on the published head. All 62 check runs are terminal and clean: 57 succeeded, two were neutral and three were skipped. Typecheck, build, policy, all general and serialized server suites, E2E and the [final verify gate](https://github.com/paperclipai/paperclip/actions/runs/43715541480/job/132829783231) pass. The separate security status passes. Attempt 1's server shard 7/12 reached its 20-minute job limit without a test verdict. One retry of that job passed without a source change. [Greptile completed successfully on this head](https://github.com/paperclipai/paperclip/runs/132816163906) with a [5/5 score](https://github.com/paperclipai/paperclip/pull/21649#issuecomment-6883916606). All five review threads are resolved. The final CI gate is complete. - A full local Storybook build exceeded available memory while server typechecking ran. Browser verification uses the current-source development server. Rebuilding stale Runner and SDK declarations resolved the initial server check errors. The first browser attempt lacked Chromium; all checks passed after installation. - Reproduce the database measurements with `pnpm --filter @paperclipai/server exec tsx scripts/benchmark-sidebar-subtasks.ts`. - Extra local shard diagnosis ran 164 suites. Under this shell's `0077` creation mask, 157 suites passed and three had fixture permission or timing failures. Rerunning those three with `0022` resolved the permission failures. The remaining ten cases passed with one worker: two workspace cases and eight archive cases. The eight archive cases also passed on the pre-performance-fix revision, whose archive source is identical. The broad local shard is not claimed as one passing execution. The same-head CI retry supplies complete final CI coverage. - Nothing is merged or deployed. ## Risks - Exact counts still grow with the readable hierarchy and comment history. The change reduces repeated work; it does not cap or approximate counts. - A relevant summary batch can contain several recent parents. Older servers retain a broader summary fallback until upgraded. - Live delivery adds bounded, permission-checked parent lookups. Short delivery-window reuse limits repeated reads. - The authorization fast path applies only to open tasks with no privacy parent and no project. Restricted cases retain the shared predicate. - No migration is required. ## Model Used OpenAI Codex, based on GPT-6. Used code execution, native GitHub tools, PostgreSQL and Chromium verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked a public issue or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open findings - [x] I will address all review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
11b09b7234 |
refactor: remove doc/execution-semantics.md and its references (#21699)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Contributors and agents read the docs under `doc/` for background on run execution > - `doc/execution-semantics.md` grew to 6,310 lines of per-PR notes > - It was hard to read and it drifted from the code > - The code and tests are the source of truth for execution behavior > - This pull request removes the file and every reference to it > - The benefit is one less stale document that readers can mistake for a requirement ## Linked Issues or Issue Description **Issue type** Outdated or unclear documentation. **Where is the issue?** `doc/execution-semantics.md`, and the docs, rules, skills, maps, code comments and log text that link to it. **What's wrong?** The file is 6,310 lines of notes added one PR at a time. Much of it repeats or contradicts the code. Readers and agents treat it as a contract, but the code and tests decide the behavior. **Suggested fix** Delete the file. Remove each link to it. Keep the documents that link to it. Point at code where a pointer is still useful. Do not move the content to another doc. Refs #10664 (an open docs PR that edits this file; it becomes obsolete). ## What Changed - Delete `doc/execution-semantics.md`. - Remove links and pointers in `doc/PRODUCT.md`, `doc/LOW-TRUST-PRESETS.md`, `doc/TASK-WATCHDOG.md`, `doc/composer-stop.md`, `doc/native-workspace-finalization-recovery.md`, `doc/run-log-events.md`, `doc/runner-internals.md` and `doc/workspace-repositories.md`. - `doc/TASK-WATCHDOG.md` now points at `server/src/modules/active-run-watchdog/` for the silent active-run watchdog. - Remove the link in `rules/upgrade-safe-run-state.mdc` and `maps/feature-map/recovery.md`. - Update the skills `diagnose-why-work-stopped`, `fix-failed-paperclip-run` and `terminal-bench-loop`. The `fix-failed-paperclip-run` skill now points at `server/src/services/heartbeat/` and `server/src/services/recovery/`. - Remove the file from the source anchors in `.devin/wiki.json`. - Remove the doc name from four code comments in `server/` and `ui/`. - The failed-attachment log message no longer ends with a link to the removed doc. Two tests now check the message text instead of the link. - Leave two dated plan files under `doc/plans/` unchanged. They are history. ## Verification - `git grep -i execution-semantics` finds no hits outside `doc/plans/`. - Every relative Markdown link in the changed files resolves. - `node scripts/check-maps.mjs` passes. - `node --test .github/scripts/tests/*.test.mjs` passes (808 tests). - `pnpm verify` was not run locally. The worktree has no `node_modules`. CI runs the changed Vitest files. ## Risks - Low risk. Most changes are docs and comments. - One runtime log line is shorter. It no longer has a link. No logic changes. - Open PRs that edit the deleted file, such as #10664, will conflict. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and shell access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fad8f051ca |
fix(ui): keep a saved task when its plan, run or work-product read answers 404 (#21684)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #21671 saves the first chat frame of recently opened tasks in the browser's read cache, so a hard load shows the chat before any response. > - It also removes a task's saved data at once when a read of the task or its chat is refused (401, 403 or 404). > - A task without a plan answers 404 on `GET /issues/:id/documents/plan`. That is a normal answer, but a reader that does not catch it records a query error. > - The cache treated that 404 as "the task is gone" and removed the task and its saved chat. On staging, a task without a plan was therefore never saved, so it never opened from the saved chat. > - This pull request revokes on a 404 only for the task record and its comments. 401 and 403 still revoke on every chat read. > - The benefit is that tasks without a plan (most tasks) open from the saved chat as intended. ## Linked Issues or Issue Description **Describe the bug** After #21671, a task without a plan is not kept in the saved read cache. The first chat frame is not shown from the saved cache for that task. **Steps to reproduce** 1. Open a task that has no plan document (for example PAP-4463 on staging). 2. Look at the saved read cache (`paperclip:query-cache:v1:*` in local storage) after the page settles. 3. The task's record and chat are missing. The page shows `GET /api/issues/:id/documents/plan` answering 404. **Expected behavior** A 404 for a missing plan, active run or work products keeps the task. Only a refusal of the task itself (its record or comments, or any 401/403) removes it. **Actual behavior** Any 404 on a saved chat read removes the task and its saved chat from memory and storage. Refs #21671. ## What Changed - `ui/src/lib/persisted-query-cache.ts`: a chat read revokes the task on 401 or 403, or on 404 only for the recent comments. The task record keeps revoking on 401, 403 and 404. - `ui/src/lib/saved-task-chat.test.ts`: a 404 for a missing plan, active run or work products keeps the task and its saved chat. - `doc/ui-internals.md`: the read-cache rules state which 404s revoke. ## Verification - `cd ui && npx vitest run src/lib/saved-task-chat.test.ts`: 24 passed. With the change in `persisted-query-cache.ts` reverted, the 3 new cases fail. The existing 401/403/404 revocation cases on recent comments still pass. - `src/lib/persisted-query-cache.test.ts` and `src/lib/restored-task-chat-bootstrap.test.ts` pass (90 tests across the three files). `pnpm --filter @paperclipai/ui typecheck` passes. - Staging evidence (build `fc9e4cbbb`, PAP-4463): the only 4xx during the load was `GET /api/issues/:id/documents/plan` 404, and the saved record did not hold the task. ## Risks - Low risk. A 404 on a plan, run or work-product read no longer revokes the task. A task that was deleted or made unreadable still answers 404 or 403 on its record and comments, and that removes it. Any 401 or 403 still removes it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser automation). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
97a167093d |
fix(queue): send a message into a starting run without a cancel or recovery errors (#21661)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A user can queue a message while an agent works, then send it into the running turn > - A run that is still starting takes queued messages into its first turn, and Interrupt hands off to them > - Both rules only accepted a queue that a user started > - A user's message can merge into an earlier automatic wake that waits behind the task, and that wake keeps its system requester > - Then the starting run did not take the message, Interrupt cancelled the boot, and the task showed "Couldn't finish this attempt" with Retry and "Waiting for execution recovery" under the composer until the cancelled boot settled > - This pull request makes both rules judge the messages and the recorded Interrupt intent, not the wake's requester, and shows an Interrupt hand-off as sent > - The benefit is that sending a queued message feels immediate and shows no error ## Linked Issues or Issue Description **What happened?** A run started from a resolved blocker. The user queued a message 20 seconds later, while the runner was still starting. The message merged into an older automatic deferred wake. The queue offered Interrupt instead of sending it. Interrupt cancelled the starting run. For about 7 seconds the task showed "Couldn't finish this attempt. Your message is saved." with Retry, and the composer showed "Waiting for execution recovery. Your message is saved." Then the message was delivered. **Expected behavior** The starting run takes the message into its first turn. When an Interrupt does stop a run, the message shows as sent, with no error banner, Retry, or recovery text. **Steps to reproduce** 1. Have an automatic deferred wake for a task (for example from an earlier blocked execution). 2. Start a run for the task with a Paperclip runner agent. 3. While the runner is still starting, queue a message and click the send control. ## What Changed - `deferredWakeJoinsFirstTurn`: a system or unattributed wake qualifies. Each message must still be the responsible user's own (`firstTurnEligibleComment`). A wake from another user or an agent keeps its own delivery. - Task interruption presentation: find the Interrupt queue by its recorded `queuedCommentInterrupt` intent, not by `requestedByActorType = user`. The hand-off now shows as working for a merged queue too. - Queued-comment read: while a deferred queue carries an Interrupt intent, report an `interrupt_handoff` wait instead of the recovery message. - UI (`TaskChatQueuedMessages`, `IssueDetail`): an `interrupt_handoff` response counts as sent. The messages stay out of the queue and no status or error line shows. ## Verification Red/green tests. Each failed before the fix and passes after it: - `server/src/__tests__/issue-queued-comments-routes.test.ts`: "gives a booting run's first turn the user's message merged into an earlier automatic wake"; the Interrupt fallback case now expects `interrupt_handoff` on the response and on a poll. 144/144 pass. - `server/src/__tests__/task-interruption-query.test.ts`: the hand-off case runs for a user-started and a system-started queue. 32/32 pass. - `ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx`: "shows an Interrupt as sent while the stopped turn hands off". 43/43 pass. - `first-turn-queued-input.test.ts` 8/8, `issue-queued-comment-queue` 7/7. Server and UI typecheck pass. Root cause from the staging wake diagnostics: the run started at 20:26:54, the message was coalesced at 20:27:17 into a deferred `issue_execution_deferred` automation wake from the previous day, the Interrupt at 20:27:20 recorded `execution_recovery`, and the false-positive hold resolved at 20:27:26. Not run: local runner E2E steering cells. The OpenCode chat-features cells require the trusted CI host. The Codex and Claude `revise-while-running` chat cells failed before steering on a fixture workspace path (`workspaces/` vs `isolated-workspaces/`), which this change does not touch. Staging verification follows the deploy. ## Risks - Low. A wider set of deferred wakes can join a starting run's first turn. Message eligibility, the claim receipt, and exactly-once delivery are unchanged. - An Interrupt intent that never delivers shows "sent" in the composer. The task notice still shows a failure with Retry after its existing grace period. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fc9e4cbbb3 |
perf(ui): show a returning task's saved chat before any server response (#21671)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI opens a task as a chat. A returning browser restores a saved read cache from local storage before the first render. > - #21650 made a returning hard load share the one chat bootstrap. But the saved cache held no chat, so the first chat frame still waited for that response. > - The owner approved saving recent comments in the browser on 2026-11-14, so that cached tasks show their chat at once. > - This pull request saves the first chat frame of the 20 most recently opened tasks, with size, age, revocation and isolation guardrails. The restored chat shows at once and revalidates through the bootstrap. > - It also adopts the hard-load bootstrap at app start and sends the other live-run readers through it, so readers that start before the task page mounts do not send their own requests. > - The benefit is that a hard load of a task the user opened before shows its conversation before any server response. ## Linked Issues or Issue Description **Describe the enhancement** A hard load of a task that the user opened before still waits for one server round trip before its chat appears. The saved read cache did not include comments. **Current behavior** With every API response delayed by 800 ms, a returning load shows the chat at 892 ms (median). The frame waits for the chat bootstrap response. **Proposed behavior** The saved read cache also keeps the first chat frame of recently opened tasks. A returning load shows that frame before any response, then revalidates through the bootstrap. **Why** The owner approved saving recent comments in the browser for this on 2026-11-14. Cold and cached loads must both be as fast as possible. Related: #21650 (returning load shares the chat bootstrap), #20594 and #20591 (returning inbox from the saved cache), #20592 and #21557 (cold task load from one bootstrap). ## What Changed - `ui/src/lib/persisted-query-cache.ts`: saves the reads the first chat frame waits for (`taskChatFirstFrameRef`): recent comments, their input facts, recent activity, the recent run window, interactions, attachment metadata, live and active runs, work products and the plan document. - Only for the 20 saved recent tasks, in the record of the task's own user and company. - At most 50 comments per task (the newest; `hasMore` is set), 192 KiB per task, 1 MiB in total, newest tasks first. - If storage refuses a record, the record is saved again without chat, so chat never displaces the rest of the cache. - All other field exclusions (tokens, credentials, environment values, embedded documents) still apply. Only `comments` is allowed, and only for these keys. - Expiry is the same as every other entry: 24 hours after the read. - A 401, 403 or 404 on the task or on any of its chat reads removes the task and its saved chat from memory and storage at once. - `ui/src/lib/issue-chat-bootstrap.ts` and `ui/src/main.tsx`: `adoptHardLoadChatBootstrap` adopts the bootstrap that the HTML started, at app start, under the task's restored id and identifier. Readers that start before the task page mounts share it. - `IssueDetail.tsx`, `LiveRunWidget.tsx`, `IssueRunLedger.tsx`, `useTaskPluginPanels.ts`: their `live-runs` reads share the loading bootstrap. Before, whichever observer mounted first could send its own request. - `doc/DEVELOPING.md`: the read-cache rules now state what is saved, the bounds, why, and the owner's approval on 2026-11-14. - Tests: `ui/src/lib/saved-task-chat.test.ts` (guardrails), a new case in `restored-task-chat-bootstrap.test.ts`, and an always-on perf test: "a returning task load shows its saved chat before any API response, then revalidates". ## Verification Local production build, local_trusted instance, CDP delay on every API request, fresh browser context per run, JavaScript in the HTTP cache. Time to the revealed history with the latest message and an editable composer. Saved read cache present against cleared (other storage kept), 5 alternating pairs each. | API delay | Build | Cleared | Present | Requests before frame (present) | | ---: | --- | ---: | ---: | ---: | | 800 ms | master `36101ebcb9` | 1,911 ms (1,895–1,920) | 892 ms (887–898), released by the bootstrap | 33 | | 800 ms | this PR | 1,922 ms (1,912–1,936) | 571 ms (556–588), before any response | 36 | | 300 ms | master | 916 ms (907–920) | 570 ms (564–576) | 37 | | 300 ms | this PR | 928 ms (910–998) | 586 ms (568–622) | 37 | - At 300 ms the bootstrap answers before the page mounts on both builds, so they are the same. At 800 ms the saved chat removes the wait for the bootstrap: −321 ms against master. - The new perf test holds every response for 2.5 s. It fails on master (frame at 2,595 ms, first response at 2,524 ms) and passes with this PR. - Guardrail tests, red/green: I disabled each guardrail in turn in `persisted-query-cache.ts` and ran `saved-task-chat.test.ts`. Each mutant made its test fail, and all 19 tests pass with the code. Mutants: no chat saved, no 50-comment cap, no 20-task cap, no per-task budget, no total budget, no retry without chat when storage refuses, no 24-hour expiry, no revocation on a chat read, chat stored in the wrong company record, and no field exclusions. - Sign-out and other-user isolation use the existing per-user records and their clearing (no new code). The tests cover them for saved chat. - "Never overwrites newer data" is the query cache's hydrate rule; the test covers it. "The bootstrap replaces saved data" uses the restored-entry rule from #21650; the test covers it. - App-start adoption test: it fails when the adoption call is removed, and passes with it. - Focused Vitest: 25 files that use the changed modules, including `IssueDetail.test.tsx` and `TaskChatThread.test.tsx`: 1,156 passed. `pnpm --filter @paperclipai/ui typecheck` passes. - Browser journeys against this build: `layout-stability.spec.ts`, `plan-startup.spec.ts`, `cold-bootstrap.perf.spec.ts`: 20 passed (2 opt-in benchmarks skipped). Leftover early reads on staging (one run after #21650 started `issues/:ref?view=core` and `issues/:ref/live-runs` before the frame): I could not reproduce them locally. That included sidebar Recents and a bootstrap delayed to 1.2 s with CDP initiator stacks. Both reads would go to the server only if no bootstrap was being tracked for the task, or if an observer used a direct read. This PR removes both openings: the bootstrap is adopted at app start, and every `live-runs` observer shares it. I will check the request timeline on staging after deploy. ## Risks - Privacy: recent comments of up to 20 recent tasks now stay in local storage for up to 24 hours, or until sign-out. The owner approved this on 2026-11-14. The bounds, the revocation and the field exclusions are tested. Saved data is per user and company, and sign-out clears it in every tab. - Storage: saved chat is limited to 1 MiB in total. A record that storage refuses is saved again without chat. - A saved chat can be up to 24 hours old when it shows. It is marked for revalidation, and the bootstrap answer replaces it within one round trip. A refused revalidation removes it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser automation through Playwright and Chrome). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ecdeed5e7e |
perf(ui): open a task restored from the saved read cache through the chat bootstrap (#21650)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI opens a task page as a chat. A hard load of a task URL starts one chat bootstrap request before the app's JavaScript runs. > - A returning browser also restores a saved read cache from local storage. This cache holds the core fields of recently opened tasks, but no chat history. > - The task page started the bootstrap only when the core task was not cached. A restored core task skipped it. The early bootstrap request ran, but nothing used its answer. > - So every first-frame read went to its own endpoint, and the comment input facts waited for the recent comments. A returning load was slower than a cold load: on staging, 3.6–3.8 s against 1.9–2.4 s. > - This pull request treats a task that only the saved cache holds as cold for the chat. The page adopts the bootstrap, the restored task's own revalidation shares it, and the answer replaces the restored data. > - The benefit is that a returning load is now faster than a cold load, not slower. ## Linked Issues or Issue Description **Describe the bug** A hard load of a task page is slower when the browser has a saved read cache than when it has none. Measured on staging on 2026-11-14 (hard load, JavaScript in the HTTP cache, 3 alternating pairs on one task): saved cache cleared 1.9–2.4 s, saved cache present 3.6–3.8 s, until the chat composer exists. **Steps to reproduce** 1. Open a task page so that the saved read cache (`paperclip:query-cache:v1:*`) holds it. 2. Close the tab and open the same task URL again. 3. Compare the time to the first chat frame with a load where the saved read cache is cleared. **Expected behavior** A load with a saved read cache is at least as fast as a load without one. The chat's first-frame reads share the one bootstrap request. **Actual behavior** With a saved read cache, the bootstrap request runs but its answer is not used. About 16 more per-endpoint reads run before the frame, and the frame waits for a second, serial wave of reads. Related: #20355 (chat bootstrap endpoints), #20592 (cold task open from one bootstrap), #20591 and #20594 (returning Mine inbox from the saved read cache), #21557 (cold-load perf spec). ## What Changed - `ui/src/lib/persisted-query-cache.ts`: records which queries a page restored from the saved read cache. `isRestoredSavedRead(client, key)` is true until a read or a local write in this page replaces that data. - `ui/src/pages/IssueDetail.tsx`: starts (or adopts) the chat bootstrap when the core task is missing or only restored. It passes the restored task's id and identifier as aliases, so reads keyed by the id share the request while it loads. - `ui/src/lib/issue-chat-bootstrap.ts`: `readFromIssueChatBootstrap` lets a key that holds only restored data share the bootstrap, like an empty key. `startIssueChatBootstrap` accepts aliases. - `ui/src/lib/issueChatBootstrapCache.ts` and `ui/src/lib/issueDetailCache.ts`: the bootstrap answer replaces restored data (core task under every ref). The restored core task's revalidation shares the bootstrap. A core task that was read in this page keeps its own small read. - `ui/src/hooks/useIssuePlanDocument.ts`: the first plan read shares the bootstrap's plan projection instead of reading `documents/plan` before the frame. - `tests/perf/issue-detail/cold-bootstrap.perf.spec.ts`: an always-on request-shape test for a returning load with a saved read cache, and an opt-in benchmark of alternating pairs (saved cache present and cleared). The harness now clears storage after the warm-up page closes, because closing the app saves its read cache again. - `ui/src/lib/restored-task-chat-bootstrap.test.ts`: unit tests for the restore-then-bootstrap path. - `tests/perf/issue-detail/README.md`: documents the returning-load test and benchmark. ## Verification Local production build (`PAPERCLIP_UI_DEV_MIDDLEWARE=false`), local_trusted instance, CDP delay of 300 ms on every API request, fresh browser context per run, JavaScript in the HTTP cache. Time from navigation start to the revealed history with the latest message and an editable composer. | Build | Saved read cache | Runs | Median | Spread | API requests before frame | Frame released by | | --- | --- | ---: | ---: | ---: | ---: | --- | | master `95a6ab31e6` | cleared | 5 | 913 ms | 908–957 | 33 | company list (shell stage 2) | | master `95a6ab31e6` | present | 5 | 1,204 ms | 1,178–1,215 | 49 | `activity?view=comment-inputs` (serial second wave) | | this PR | cleared | 7 | 942 ms | 925–997 | 35 | company list (shell stage 2) | | this PR | present | 7 | 573 ms | 566–658 | 37 | app boot and route mount (all task data is the bootstrap) | - Paired delta with this PR (present − cleared): median −376 ms over 7 pairs, all negative (−267 to −400). On master it was +292 ms over 5 pairs, all positive. - With an 800 ms API delay (this PR, 5 pairs): present 890 ms, released by the bootstrap response; cleared 1,910 ms. - Cold load with no app storage (existing benchmark, 7 runs): master 910 ms, this PR 939 ms. The host load average was about 20 during the PR run (other work on the machine). Requests before the frame are 35 in both. I make no claim of a cold change. - Request-shape tests (`cold-bootstrap.perf.spec.ts`): the new returning-load test fails on master (the timeline shows `?view=core`, `recent-comments`, `activity`, `runs`, `interactions`, `attachments`, `live-runs` before the frame) and passes with this PR. The existing cold test passes. - Unit tests: `restored-task-chat-bootstrap.test.ts` has 10 cases. With the behavior changes reverted (the restore marker kept so the file imports), 3 fail: the bootstrap refreshes the restored task and fills the chat, id-keyed reads share the loading bootstrap, and a partial answer. All 10 pass with this PR. The other 7 cases are guards that hold on both: a stale refresh reads the server, a failed bootstrap falls back, a 403 removes the restored task from memory and storage, another company's answer is not seeded, an answer is not used after another user signs in, and another user's saved task is not restored. - Focused Vitest: 27 files that use the changed modules, including `IssueDetail.test.tsx` and `TaskChatThread.test.tsx`: 1,208 passed. - Browser layout journeys against the PR build: `layout-stability.spec.ts` and `plan-startup.spec.ts`, 17 of 17 passed (reveal order, reading anchor, older history, Back, comment links, drafts during background refresh, failed and stalled reads with Retry, plan projection with stalled plan reads). - `pnpm verify` exit 2 (the README is a global CI input), so required CI supplies the broad coverage. Commands: ```sh cd ui && npx vitest run src/lib/restored-task-chat-bootstrap.test.ts PAPERCLIP_ISSUE_PERF_BASE_URL=http://127.0.0.1:<port> pnpm exec playwright test --config tests/perf/issue-detail/playwright.config.ts cold-bootstrap PAPERCLIP_ISSUE_PERF_BASE_URL=http://127.0.0.1:<port> PAPERCLIP_ISSUE_PERF_COLD_RUNS=7 pnpm exec playwright test --config tests/perf/issue-detail/playwright.config.ts cold-bootstrap -g returning ``` ## Risks - Low risk. The change only affects data that the saved read cache restored and that no read in this page has confirmed. Data read in this page keeps the old rule: a refresh asks the server. - The saved read cache content is unchanged. It still excludes comments and bootstrap payloads, so the first chat frame still needs one response (the bootstrap). It cannot appear before any task response without saving chat history, which the documented allowlist excludes. - If the bootstrap fails, stalls or omits the task, each read falls back to its own request, as on a cold load. A 403 or 404 on the restored task's fallback read removes it from memory and storage, as before. - Account changes: a bootstrap answer is bound to the session's user, as before. The saved cache restores data only for the same user. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser automation through Playwright). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
87d363a5c0 |
perf(ui): reveal a returning Mine visit from the saved cache (#20594)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Mine inbox is the board's main list of work for one person; the
app saves its query cache in `localStorage` so a returning visit can
paint from it
> - On a hard load the saved cache gave no benefit: Mine restored its
rows but still held them until the first approvals, join-request and
dismissal reads answered, because those items sort in among task rows by
time and a late one would push rows down (#20407)
> - Those three reads were not saved, so every returning visit waited on
the network for items it showed a minute before
> - This pull request saves what Mine showed from those reads, as one
record per user and company, and lets it answer the wait on a returning
visit while every read refreshes in the background
> - The benefit is that a returning visit shows its rows with no API
response at all; the first visit keeps its bounded wait unchanged
## Linked Issues or Issue Description
**What happened?**
A returning hard load of `/:companyPrefix/issues?view=mine` with the
saved cache present (`paperclip:query-cache:v1:<user>:<company>`) showed
rows no sooner than with the cache cleared. On the hosted staging
instance, rows appeared about 40 ms after `GET
/companies/:id/join-requests?status=pending_approval` returned in 7 of
10 runs.
**Steps to reproduce**
1. Open Mine once so the app saves its query cache.
2. Delay API responses (for example 300 ms) and reload.
3. Rows appear only after the approvals and join-request reads answer.
**Expected behavior**
Rows show from the saved cache at once. The reads refresh in the
background.
Related: #20407 (kept approvals and join requests in the reveal gate),
#20408 and #20419 (Mine snapshot and its perf guard), #20591 (removes
the second wave of reads; merged), #20592 (task chat preload; merged).
## What Changed
- `ui/src/lib/inbox-mine-reveal-sources.ts` (new): the record `["inbox",
companyId, "mine-reveal-sources", userId]` holds the approvals visible
in Mine, the pending join requests, and the dismissals of those items
and of Mine's failed runs. A record over 256 KB is replaced by an empty
`incomplete` record that never answers the wait. `readMineRevealSources`
accepts a record only for the same user and company.
- `ui/src/lib/persisted-query-cache.ts`: the saved cache keeps that
record per user and company, only when its data names the same user and
company as its key.
- `ui/src/pages/Inbox.tsx`:
- Mine writes the record once approvals, join requests and dismissals
all have server answers.
- On a returning visit with a restored Mine snapshot, the saved
approvals, join requests and dismissals stand in while their reads are
pending, and after a transient failure. A refusal (401/403/404) or a
server answer replaces them. The first-attempt wait is skipped only when
saved values stand in; the first visit is unchanged.
- Selection fix: the selected row's key is recorded only when the
selection moves. Before, every list change recorded the row now at the
old index before the re-anchor ran, so a row inserted above the
selection (a late approval, or any live insert) moved the selection to
another row.
- `ui/src/hooks/useInboxBadge.ts`: `useInboxDismissals` takes an
optional saved `fallback` and returns `hasServerData`. Other callers are
unchanged.
- Tests:
- `Inbox.test.tsx`: a returning visit reveals rows and the saved
approval while the reads are pending, then keeps selection on the same
row when a newer approval arrives above it; a transient failure keeps
the saved approval and a 403 drops it; a record stamped for another user
is ignored and the page waits as a first visit does.
- `inbox-mine-reveal-sources.test.ts`: projection, size limit,
user/company checks, persistence scope.
- `persisted-query-cache.test.ts`: the record round-trips for the same
user, restores as invalidated, and another user's record is not saved.
- `useInboxDismissals.test.tsx`: fallback while pending, after a
transient failure, dropped after a refusal, and no fallback.
- `tests/perf/collections/mine-returning-visit.spec.ts` +
`returning-visit-fixture.ts`: production-build browser checks and the
timing case; README updated.
## Verification
- `cd ui && pnpm exec vitest run src/pages src/lib src/hooks
src/components/coordinator`: 453 files, 5574 tests pass.
- Browser checks on a disposable local instance (production UI build,
`local_trusted`, embedded PostgreSQL, Chromium headless, 1440×900):
`PAPERCLIP_COLLECTION_PERF_BASE_URL=http://127.0.0.1:<port> pnpm exec
playwright test --config tests/perf/collections/playwright.config.ts
mine-returning-visit.spec.ts`
- With every read of the company held, task rows and saved approvals
show and no company read has answered. Fails on `master`, passes here.
- An approval dismissed after the cache was saved stands in, then leaves
when the reads answer. Fails on `master` (no saved approvals), passes
here.
- Another company's Mine never shows the saved company's rows. Passes on
both.
- Timing (5 fresh contexts per arm, alternating saved/cleared, every API
response +300 ms, same instance and fixture, host load average about
20). "Responses before rows" counts API responses that finished before
the first row:
| Build | Saved cache: median (range) | Responses before rows | Cleared
cache: median (range) |
| --- | --- | --- | --- |
| `master` (`99edcd28a7`) | 1028 ms (906–3011) | 16–19 | 1502 ms
(1423–1592) |
| This PR | 625 ms (589–672) | 0 | 1632 ms (1359–1692) |
| This PR + #20591 | 625 ms (604–671) | 0 | 1495 ms (1398–1585) |
Saved cache: difference of medians 403 ms. The remaining 625 ms is
JavaScript download and route-chunk loading in a fresh browser context;
no API response precedes the rows. Cleared cache: no change is claimed;
the spread overlaps.
- Existing collection specs on this build: `parent-context.spec.ts`
passes; `startup-regressions.spec.ts` has 4 failures (partial
failure/context error and two 429 cases) that fail the same way on the
`master` build. `mine-workspaces.spec.ts` was not run (needs a database
URL for its fixture).
- Review follow-up (
|
||
|
|
a726dce4f0 |
feat(mcp): show live progress for long connected-app tool calls in the task chat (#21449)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call tools from connected apps through the Paperclip MCP gateway > - Some tool calls take a long time: exports, builds, large searches > - The MCP progress utility lets an app report how far a call is, but Paperclip ignored it > - So the task chat showed only a spinner until the call finished, with no sign of life > - This pull request reads the app's progress and shows it on the tool's row while the call runs > - The benefit is that people can see that a long call is working, how far it is, and how long it has run ## Linked Issues or Issue Description **Subsystem affected** Connected apps (MCP gateway) and the task chat. **Problem or motivation** A long tool call from a connected MCP app shows only a spinner in the task chat. Paperclip does not send a `progressToken`, and it drops `notifications/progress` from the app. **Proposed solution** Send a `progressToken` on each `tools/call` that an agent run makes. Read the app's progress notifications from the call's response stream. Show a progress bar (or an indeterminate sweep), the latest message, and elapsed time on the running tool row. Keep the latest state so that a reload mid-call shows it. **Alternatives considered** A new transport for progress. The existing company live-events socket already carries per-run events with run read checks, so this change uses it. **Additional context** Spec: https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress ## What Changed - `mcp-http.ts`: `mcpHttpRequestInit` can add `_meta.progressToken`. `readMcpHttpResponse` gives each notification on the response stream to an observer before the result. The observer cannot fail or delay the call. New `mcpProgressNotificationOf` matches a notification to its token. - `tool-gateway.ts`: remote HTTP and local stdio `tools/call` from an agent run send a per-call token and forward progress to a reporter. Calls without a run send no token. - New `server/src/services/tool-call-progress.ts`: checks each update (finite, increasing, plain single-line text of at most 200 characters). It publishes a `heartbeat.run.tool_progress` live event at most every 250 ms (latest wins) and saves the latest update on the invocation at most every 1 s. Nothing is awaited by the call. A final `done` event goes out when the call ends. - DB: nullable `tool_invocations.progress` jsonb column (migration `0479`, `ADD COLUMN` only). A save never writes onto a finished call. - Live delivery: the new event follows its run like transcript output. A client gets it only for runs it follows, after the run read check, in the run's company only. - New route `GET /api/issues/:issueId/tool-call-progress` (task read check) returns the latest progress of the task's running calls, for a reload mid-call. Documented in `docs/api/issues.md`. - UI: `ToolCallProgressProvider` (one per chat) keeps the latest update per call. A running tool row subscribes and shows a bar or sweep, the message, the percent and elapsed time (design tokens, reduced motion honored). Finished rows drop it and never fetch or re-render for progress. `LiveUpdatesProvider` ignores the event for query invalidation. - `packages/kv-demo-mcp-server`: new `slow_with_progress` demo tool (`steps`, `stepMs`, `withTotal`) in its own file. - Storybook: `Tasks/Tool progress` stories (running with total, running without total, done). - Docs: new `doc/connections/MCP-PROGRESS.md`; `MCP-PROTOCOL-VERSIONS.md` no longer lists progress as unsupported. ## Verification - Red/green: on the original source, 5 of the new server tests fail (token in `_meta`, SSE with interleaved progress and result, notification matching, run-interest delivery, gateway end-to-end). With this change they pass. - `server/src/__tests__/mcp-http.test.ts` (13), `tool-call-progress.test.ts` (9: throttling, latest wins, bounded text, out-of-order drop, finish, failures never throw, company/run/tool scoping), `tool-call-progress-store.test.ts` (2, embedded Postgres: company and task scoping, no write onto a finished call), `realtime/live-event-delivery.test.ts` (run interest), `tool-gateway.test.ts` end-to-end against the real SDK kv demo server over Streamable HTTP: 6 provider updates over about 1.2 s give 3 to 7 throttled events, all for the right company, run, task and tool, the last one `done`; no event reaches another company; the invocation keeps the progress; the result is unchanged. - UI: `tool-call-progress.test.tsx` (10: name matching, row states, plain-text rendering, drop on complete, restore after reload, no fetch for finished rows). All 67 task-chat test files pass. - `packages/kv-demo-mcp-server`: 13 tests, including progress over HTTP with and without a total. - Storybook: checked the three stories in a browser. The bar fills to 40%, the sweep animates, the elapsed time ticks, and the done row shows no progress. - `pnpm verify`: see the CI checks for the final revision. ## Risks - Low. The migration adds one nullable column. - An SSE response is now bounded per message, not per stream, so handled notifications (such as progress) do not use up the result's size limit. Each message is still bounded. - Progress code runs only while a run's tool call is in flight. It adds a few bytes to the request and throttled, fire-and-forget work. Agent boot and time to first token do not change. - A row finds its call by run and tool name. If one run calls the same tool twice at the same time, both rows show the newer call's progress. - Progress from a call that became an MCP Task is not shown. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f58bbefc4e |
fix(ui): group failed-query Sentry reports by status and query name (#21475)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The browser app can report errors to Sentry. A final query error goes to Sentry when it is not an outage, a denial, or a missing item. > - Every failed API request throws an `ApiError` from the same frames in `api/client.ts`. > - Sentry groups errors by stack. Thus all unrelated server failures go into one issue, titled "ApiError: Internal server error". > - The report has no route, no query name, and no status. Breadcrumbs and URLs are removed for privacy. A person cannot find which endpoint failed. > - This pull request tags each failed-query report with its HTTP status and the static name of its query key. It also sets the Sentry fingerprint from these two values. > - The benefit: each failing endpoint gets its own Sentry issue, and a person can connect it to the server error. ## Linked Issues or Issue Description **What happened?** One browser Sentry issue collected unrelated server 500 errors from different pages and environments. The events had no route or query name. A person could not connect an event to a server error. **Steps to reproduce** 1. Turn on browser error monitoring with a Sentry DSN. 2. Make two different queries fail with a 500 response, for example the issues list and the agents list. 3. Look at Sentry. Both events are in one issue with the title "ApiError: Internal server error". Neither event shows which query failed. **Expected behavior** Each failing query has its own Sentry issue. The event shows the HTTP status and the static query name. It shows no IDs, refs, or filters. **Environment** The hosted browser app, staging and production builds. I searched open pull requests for duplicates. I found no pull request that changes browser query-error grouping. ## What Changed - `ui/src/lib/browser-error-context.ts`: add `queryKindFor`. It keeps only the first query-key part, and only when it is a lowercase word list such as `issues` or `agent-commentary`. Other values become `other`, so an ID, a ref, an email, or a typed value is never sent. - `ui/src/lib/browser-error-context.ts`: add `buildQueryErrorContext`. For an error with an HTTP status, it sets the `query_kind` and `api_status` tags and the fingerprint `["api-error", status, queryKind]`. For an error without a status, it sets only the `query_kind` tag and keeps default stack grouping. - `ui/src/lib/sentry.ts`: add `captureQueryException`. It uses the same serialized queue as `captureBrowserException`. Both functions now share one private capture step. - `ui/src/lib/query-client.ts`: `reportError` now also receives the query key. - `ui/src/main.tsx`: the query client reports through `captureQueryException`. - Tests for key sanitizing, the fingerprint, the reporter argument, and the gate call. ## Verification - Red before the fix: the 6 new assertions fail on the old source (`queryKindFor`, `buildQueryErrorContext` and `captureQueryException` do not exist, and `reportError` gets no key). - Green after the fix: `cd ui && npx vitest run src/lib/sentry.test.ts src/lib/browser-error-context.test.ts src/lib/query-client.test.ts` passes 3 files and 78 tests. - The privacy test makes sure that `company-uuid` and `PAP-1` from later key parts do not go into the Sentry call. It also makes sure that UUID, ref, email, and long heads become `other`. - `ui` typecheck: no errors in the changed files. The local install was filtered to the UI package, so unrelated packages (`plugin-sdk` build output, runner dependencies) could not resolve. Required CI supplies the full typecheck. - I did not run browser e2e suites. This change does not touch them. ## Risks - Low risk. Only the grouping and tags of browser query-error reports change. The app does not change what it shows to users. - Sentry opens new issues for future query errors, one for each status and query name. The old catch-all issue gets no new events. - A query key with a dynamic first part becomes `other`. Those errors still group together, but they keep the status tag. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code with tool use, repository inspection, and shell code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
37393f5dc5 |
fix(browser-tracing): send API spans only for measured page loads, without timing events (#21441)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted instances can send browser page-load traces to Honeycomb (#21156, #21392) > - On staging, one hour in one workspace sent about 64,000 Honeycomb events, and only about 15 were `page.load` > - Two sources make the volume: nine network timing span events on each API request span, and request spans for background polling in idle tabs > - The page-load metrics use neither of them > - This pull request drops the timing events and records API request spans only while a page load is measured > - The benefit is about 30x fewer events in a browser test, with the same page-load data and the same server trace joins ## Linked Issues or Issue Description Refs #21156, Refs #21392 **What existing behavior does this improve?** Browser tracing exports a span for every same-origin `/api/` fetch and nine network timing events on each of those spans. **Subsystem affected** UI browser tracing (`ui/src/lib/browser-tracing/`), on hosted instances with browser tracing turned on. **Current behavior** Honeycomb stores each span event as its own event. Each request span becomes about 10 events. Idle tabs poll (for example live runs every 3 s on a task page), so most spans are requests that no page load made. On staging: about 6,600 `GET` spans plus about 6,400 of each timing event per hour, against about 15 `page.load` spans. **Proposed behavior** Request spans carry no timing events. A request is recorded only when it starts while a `page.load` measurement is open. Other requests send no span and no `traceparent`. **Reason and benefit** Event volume drops by about 30x for an idle tab. `page.load`, Web Vitals, and the Resource Timing backfill do not change. No exported request span lacks its `page.load` parent, and no server span points at a browser span that was never sent. ## What Changed - `sdk.ts`: the fetch instrumentation runs with `ignoreNetworkEvents: true` (an option of the installed `@opentelemetry/instrumentation-fetch` 0.221.0). - `sdk.ts`: new `measuredRequestsOnly` wraps the tracer provider that only the fetch instrumentation uses. When no `page.load` is open, it returns a non-recording span with an invalid context. Nothing is exported. The W3C propagator sends no `traceparent` for an invalid context, so the server traces that request alone, as for an unsampled tab. The `page.load`, Web Vitals, and backfill spans use the normal tracer and do not change. - Failed idle requests are not kept. The server's own request span records server errors. To keep them, the browser must record every idle request and send its `traceparent` before it knows the status, and that brings back the dangling joins. - `span-sanitizer.ts`: comment update only. - `doc/browser-performance-tracing.md`: new "Live API spans" section with the rule, the note that timing events are not sent, and updated volume and trace-header text. ## Verification - Red then green (`ui/src/lib/browser-tracing/sdk.test.ts`): - `records no span and sends no traceparent for an API request made while no page load is measured`: failed on the old code (an idle request sent `traceparent`), passes now. - `sends request spans without per-request network timing events`: failed on the old code (9 timing events on the request span), passes now. - All 56 tests in `ui/src/lib/browser-tracing/` pass. `tsc --noEmit` for `ui` passes. - `pnpm verify` exits 2 because a doc is a global CI input. CI supplies the broad coverage. - Real browser: local dev server on an isolated empty instance with 8 tasks, a stand-in OTLP collector, headless Chromium, and the tracing config injected into the session response. Scenario: hard load of the all-tasks page, click a task, then 2 minutes idle. One Honeycomb event = one span or one span event. | Name | master | this branch | | --- | ---: | ---: | | `page.load` | 2 | 2 | | `web_vital.lcp` | 1 | 1 | | `HTTP` (backfill) | 33 | 32 | | `GET` | 165 | 18 | | `POST` | 1 | 1 | | each of `fetchStart`, `domainLookupStart`, `domainLookupEnd`, `connectStart`, `connectEnd`, `requestStart`, `responseStart`, `responseEnd` | 165 | 0 | | **Total events** | **1,522** | **54** | - `secureConnectionStart` does not appear locally because the dev server is plain HTTP. On staging it is the ninth event. - Branch: the hard `page.load` has 32 `HTTP` children and the client `page.load` of the task has 19 live children (`GET`, `POST`). 0 request spans without a `page.load` parent (master: 146). 19 requests sent `traceparent`, and all 19 point at an exported span (master: 172 sent, 6 point at spans never exported). - Idle traffic in this test (210 `/api/` requests in total): `GET /api/issues/:id/live-runs` 88 times in 2 minutes, and `GET /api/health` 44 times (a dev-server-only 2 s poll). Most of the rest are one-off refetches. - Bundle (`vite build`): entry 3,826,982 B raw both, gzip 1,156,148 → 1,156,173 B (+25 B, chunk hash). Lazy SDK chunk 85,423 → 85,825 B raw, 27,723 → 27,859 B gzip. ## Risks - Low risk. The change is limited to the operator-gated browser tracing path on hosted instances. - Behavior shift: Honeycomb no longer has browser spans for requests made outside a page load, such as polling and mutations in a loaded page. Their server spans are still recorded as their own traces. - A request that starts during a page load and finishes after it is still recorded, as a child of that load. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, headless Chromium). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ed671f57bf |
fix: Display Ollama billers with their product names in cost views (#20420)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #16573 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: The shared providerDisplayName map omitted ollama and ollama-cloud, causing all cost views using it to display the raw slugs. - This pull request: Display Ollama billers with their product names in cost views ## What Changed - `ui/src/lib/utils.ts`: Add Ollama and Ollama Cloud names in the shared formatter used by cost views. - `ui/src/lib/utils.provider-display-name.test.ts`: Reproduce the missing names and cover case-insensitive lookup and unchanged fallback behavior. Confirmed failure before the fix; all 20 selected tests and the workspace typecheck passed afterward. ## Verification - Reproducing test(s), added in this PR: `ui/src/lib/utils.provider-display-name.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 10 existing test file(s) near the change passed. - The full test suite passes except for the 16 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 3. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #16573 Other pull requests that name #16573: - #16609 (open): fix(ui): map Ollama and Ollama Cloud biller slugs to human-readable names - #16590 (open): fix(ui): format ollama and ollama-cloud provider names in cost views - #16574 (open): fix(ui): show Ollama and Ollama Cloud by name in cost views ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=16573 base=8e5959c759bb report=f738d3d291d2282c --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
b993702ae3 |
perf(ui): read the compact task index on the dashboard and refresh it once per window (#21413)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI keeps company lists fresh from live task activity events > - The dashboard, the activity feed and board chat read the full company task list, and every burst of task activity refetches it > - One full list request costs the server 60 statements and 61-90 MB of allocation for 500 tasks, and the staging server is saturated by allocation and garbage collection > - This pull request moves those three views to the compact task index under its own key, and task activity refreshes that key at most once per 30 seconds > - The benefit is about one tenth of the server allocation per open dashboard tab while agents work, with the same rows on screen ## Linked Issues or Issue Description **What existing behavior does this improve?** The dashboard, the activity feed and board chat load the full company task list. Every burst of task activity refetches that list. **Subsystem affected** Board UI: live updates, the dashboard, the activity feed and board chat. **Current behavior** The three views read the full company task list (`GET /api/companies/:id/issues`, no view) with the key `["issues", companyId]`. `LiveUpdatesProvider` invalidates that key on every task activity event. Each open tab refetches the list at most every 5 seconds while agents work. One request costs the server 60 statements and 61-90 MB of allocation for 500 tasks (local measurement). The staging server allocates 57.6 MB per second and is saturated by garbage collection. **Proposed behavior** The views read only the fields they show, from the compact task index. Task activity from other people refreshes the index at most once per window. The viewer's own task changes refresh it at once. **Reason and benefit** Less server allocation and fewer statements per open tab, with the same rows on screen. **Breaking changes** None. No API, schema or server change. ## What Changed - The dashboard, the activity feed and board chat read `view=index` under a new key `issues.listIndex(companyId)`. - Task activity from other people refreshes that key at most once per 30 seconds (`TASK_INDEX_REFRESH_WINDOW_MS`). The viewer's own task activity refreshes it at once. The first event after a quiet window refetches at once. Later events in the window share one trailing refetch. Nothing from the event goes into the cache. An unwatched index is marked stale and refetches on the next mount. The user's own task changes still invalidate the whole `["issues", companyId]` prefix immediately. - The dashboard takes its ten recent tasks from the index, sorted by updatedAt as before. It reads the status glyph fields for those ten tasks from `issues/summaries`, because the index omits displayStatus and blockerAttention. - Toast and route lookups that read the full list cache also read the index cache. - `server/scripts/query-cost/` is the per-caller query cost harness used for the measurements below. It records the caller of each query at construction, the bytes allocated to build it, the rows it returns and the statements sent. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/context/LiveUpdatesProvider src/pages/Dashboard.test.ts`: 261 pass. - Red/green: `LiveUpdatesProvider.task-index.test.ts` fails 6 of its first 10 cases against the provider on master and passes with this change. The tests cover a burst of events (4 index fetches in 90 seconds), the first event after a quiet window, a new task, a deleted task, a hidden task, a status/title/priority change, an event for a task outside the cache, an event for a task the viewer cannot read (nothing from the event goes into the cache), the viewer's own activity (refreshes at once), a task change made in this tab (refreshes at once through the company list invalidation), the trailing refresh (the last state shows when the window ends after activity stops), an unwatched index, and the live-event batcher path. - Other readers of `["issues", companyId]` (My Tasks, the task properties picker, the command palette, task detail cache writes, filtered and board-column lists) are not changed. The broad invalidation excludes only the new index key, so they refresh as before. A hidden tab keeps today's behavior: the live-event batcher still turns its refetches into stale marks. - Field parity (local, 1,500 tasks): the index returns the same 500 ids as the full list, with 0 differences in id, identifier, title, status, priority, createdAt, updatedAt and assigneeAgentId. It omits displayStatus and blockerAttention on all 500 rows, so the recent rows take those fields from summaries. - Real browser (headless Chromium, Playwright) against a local server from this branch, started with `test-drive --data-dir <scratch dir>`, which sets `PAPERCLIP_HOME` and `PAPERCLIP_CONFIG` to that directory: - Dashboard: the recent tasks and the "Tasks by Status" chart show the new tasks. The page sent 0 full-list requests, 2 index requests and 1 summaries request. - A task rename by this board user showed on the dashboard after 0.8 seconds, with 1 index refetch. - A second tab on the dashboard showed the same tasks. - Board chat loaded with 0 full-list requests and 3 index requests. Its activity feed showed the task names. - Not checked in the browser: activity from another actor. An agent key cannot edit a task without a heartbeat run context, so the window for other people's activity is covered by the unit tests only. - Nothing visual changed. The views keep their render code. Only their data source changed. - Cost per request (local, embedded PostgreSQL, Node 26, uninstrumented pass, 5 requests each), measured: `TSX_DISABLE_CACHE=1 UNITS=board ROUTES="issues list,issues list view=index,issues/summaries (10 ids)" REPEAT=5 NODE_OPTIONS=--expose-gc pnpm --filter @paperclipai/server exec tsx scripts/query-cost/drive.ts` | Request | Statements | Allocated | | --- | --- | --- | | full list | 60 | 61-90 MB | | view=index | 12 | 22-37 MB | | summaries, 10 ids | 23 | 2.8 MB | The ranges show the difference between runs. - Estimate, not measured: per open dashboard tab with continuous task activity, before about 13 MB/s (one full list per 5 seconds), after about 1.3 MB/s (one index per 30 seconds and one 10-id summaries request per 5 seconds). At the day-1 staging route rate (0.116 full lists per second) that is about 6.5 MB/s less. The real rate of task activity events on staging is not measured. ## Risks - Freshness: the dashboard's task charts and recent-task order, the activity feed's task names and board chat's task lookup can lag task activity from other people by up to 30 seconds. The status glyphs of the ten recent tasks stay on the 5-second lane. The viewer's own task changes refresh at once. The window is one constant, `TASK_INDEX_REFRESH_WINDOW_MS` in `LiveUpdatesProvider.tsx`, so a different value is a one-line change. - The index returns the same rows as the full list, so the charts count the same tasks. - The recent-task rows have no task description. The rows do not show it. - My Tasks, the task properties picker and the command palette still read the full list. They are not in this change. ## Model Used - Claude Opus 5.5 (claude-opus-5-5), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no documentation describes this behavior) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
84a4d345b3 |
refactor(docker): take the application image's Runner from the runner-artifacts stage (#21332)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner is the part that executes agent work. `master` publishes it alone as the image `ghcr.io/paperclipai/paperclip-runner`, built by the `runner-artifacts` stages of the `Dockerfile` > - The application image does not use those stages. It has five stages of its own that compile the daemon, it builds the Runner again inside the server build, and it builds a second provider pack > - So one `Dockerfile` holds two recipes for the Runner, and the two results differ: another base image and another daemon binary > - This pull request removes the second recipe. The application image copies the Runner out of the `runner-artifacts` stage, which the same build runs once > - The benefit is one Runner recipe, and an application image whose Runner is the Runner the contract describes. A later pull request can replace the stage with the published image ## Linked Issues or Issue Description This is the second part of a chain. The first part is #21331, which removes the cache check that cannot run on this `Dockerfile`. Refs #21204, #21229, #21230 (the Runner image), #21306 (the no-rebuild mode of the vendor step), #21307 (the pull-request check of the application image) and #21312. No issue exists for this change. **Problem or motivation** The stages `rust-toolchain`, `rust-chef`, `runner-plan`, `runner-deps` and `runner-build` compile the daemon on Debian trixie. The server build then runs the full build of the Runner package. The `runner-provider-pack` stage builds the provider pack. The `runner-artifacts` stages do all of that again for the Runner image, on Debian bookworm, with a stripped daemon. **Proposed solution** Read only the `runner-artifacts` stage from the application stages. Build it inline in every build. Do not pass a published image yet. **Alternatives considered** - Pass the published Runner image to the build in this pull request. That needs the resolver, labels and a rule for commits without an image. It is the next step, and this one must be safe alone. - Keep the old stages as a fallback. Two recipes is the problem. ## What Changed - `Dockerfile`: remove the five stages. `build` starts from `base`. It copies `/runner-package/dist` and the build stamp from `runner-artifacts`, sets `PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1` and runs the vendor check before the first application build. - `Dockerfile`: `runner-provider-pack` starts from `base`. One `RUN` with a bind mount of `runner-artifacts` copies the pack, the receipt and the archives, and moves the large files to their own tree. `build` and `runner-provider-pack` now come after the stages they read. - `Dockerfile`: the three `COPY` instructions of `production` that take the pack are byte-identical. A test pins all three. - `Dockerfile`: `build` declares the build argument `PAPERCLIP_RUNNER_EXPECT_INPUTS_DIGEST`, empty by default, and one step runs the new `scripts/check-runner-image-expectation.mjs` on `/runner-image.json` of `runner-artifacts`. **This pull request declares the argument. No build sets it. The next pull request, which gives a published Runner image to the build, is its first user.** With the argument set, the build fails unless `runner-artifacts` is the published image of exactly those inputs. `docker-app-check.yml` also runs when the script changes. - Comments: the `Dockerfile` has 220 comment lines, down from 247. The comments of the two moved stages moved with them, unchanged, so the diff shows them as added lines. Eight comments that named a removed stage or a step that moved get the smallest correction. Two comments in `prepare-runner-vendor.mjs` and `build-inputs.mjs` said that the `Dockerfile` relies on the standalone rebuild; that clause is removed. - Tests: replace the two cargo-chef tests of `docker-build-stamp.test.ts` with three tests of the new shape. Update the pack test, `docker-image-layer-reuse.test.ts`, `vite-build-commit.test.ts`, the stage rules of `runner-image-identity.test.ts`, and the include-layout test of `runner-release-crate-cache.test.mjs`, which now reads `runner-artifacts-runnerd`. - Docs: `doc/DOCKER.md`, `doc/DEVELOPING.md`, and a new section "The application image" in `doc/runner-image-contract.md`. ## Verification All builds: `linux/arm64`, local, `--build-arg PAPERCLIP_BUILD_COMMIT=<sha>` with one commit for every build. The builds ran before the rebase onto `master`, on the head of #21310 as it was before it merged. `master` changed neither the `Dockerfile` nor an input of the Runner in between. - `docker buildx build --target production`, cold, before and after: 506 s and 410 s. Compressed size: 2,971 MB and 2,900 MB. - The image config and the text of all 33 history rows are identical. The row `COPY /provider-pack-transport/ /opt/paperclip-runner/ # buildkit` matches the pattern of `retained-provider-pack-oci.ts` exactly once. - A comparison of every file of the two images (path, type, size, mode, link target, and SHA-256 under `/app` and `/opt`). The differences are in "Risks". All other files are byte-identical, the root `node_modules` included. - From the image after: the four `/app/packages/paperclip-runner/...` paths and the contents of `/opt/paperclip-runner/` exist. `paperclip-runnerd --build-metadata` runs. `provider-pack.json` has `runnerSourceRevision` equal to the build commit and no `runnerInputsDigest`. The entry points `index.js`, `live/index.js` and `testing.js` of the Runner import. - The vendor step logged `reusing the current Runner build` in both of its runs. Every `Compiling` line of the log is in `runner-artifacts-runnerd`. - `--target build`, `--target runner-provider-pack` and `--target cloud` built. The `build` stage has no `gcc`, `cc`, `cargo`, `rustc` or `pkg-config`. - `--target runner-artifacts` and `runner-artifacts-slim`, before and after: `runner-image-layout.mjs check` and `runner-image-oci.mjs check-layers` pass on both, in `node:24.22.3-bookworm`. The daemon, `provider-pack.json` and the two stable layers have the same digests before and after. - Without a build commit: `build` and `runner-provider-pack` built, and the pack was skipped with the same message. This run was on the next branch of this chain, after the rebase. The last steps of `production` were not run for this case. - The expectation argument, on this `Dockerfile`, `--target build --output type=cacheonly`, after the rebase: - No argument, inline Runner: the build passes (233 s). The step passes. - Argument, inline Runner: the build fails at the step. The message says that the `runner-artifacts` of the build are not a published Runner image. - Argument `d4df566f…` and the published image of those inputs as `--build-context runner-artifacts=docker-image://ghcr.io/paperclipai/paperclip-runner@sha256:0cd8250b…`: the step passes, and the build runs no Runner stage. The vendor step then fails with `inputs changed since the recorded build`, because this tree differs from the tree of that image by one comment line in `build-inputs.mjs`. With that line put back, the same build completes, and the vendor step reuses the published build. - Another digest with the same context: the build fails at the step. The message names the inputs the image has. - A tree whose lockfile is behind the manifests of the Runner fails at the frozen install of the Runner stages, after 169 s, with the message that stage already had. - `docker buildx build --check`: no warning, before and after. `scripts/check-docker-deps-stage.mjs` and `scripts/check-node-version-policy.mjs` pass. - Tests, on macOS and in `node:24.22.3-bookworm`, after the rebase: the changed server test files with the other tests that read the `Dockerfile` or the image workflows (231 pass in 10 files on Linux), `vite-build-commit.test.ts` (10 pass), `runner-release-crate-cache.test.mjs` (11 pass), `docker-deps-postinstall.test.mjs`. The changed server test files typecheck. - Not run: the full test suite, any `linux/amd64` build, `podman`, the workflows and the release pipeline. Only CI can prove amd64, `docker.yml`, `docker-merge-group.yml`, `release.yml`, `docker-app-check.yml` and `runner-image.yml`. ## Risks - **The digest of the Runner image changes.** Its identity covers the whole `Dockerfile`. The next `master` push publishes a new `inputs-<digest>`. The contents do not change: see the comparison above. - **A build that is given a Runner of other inputs fails; it does not ship that Runner.** With the argument, it fails at the expectation step. Without the argument, it fails at the vendor step when the build inputs of the Runner differ. One comment line in `build-inputs.mjs` changes those build inputs in this pull request, so the image published before this pull request cannot be vendored into a tree after it. - **What a consumer of the application image sees** (measured on `linux/arm64`): - The daemon is the stripped binary compiled on Debian bookworm: 23.2 MB, was 36.4 MB, at `/app/packages/paperclip-runner/dist/bin/paperclip-runnerd` and at the vendored copy in `server/dist`. Its hash changes. It has no symbols for a crash backtrace. - `/app/packages/paperclip-eval-kernel/dist` is absent. Only `dist/eval/workflow-harness.js` of the Runner imports that package. Only `eval/index.js` exports the harness again, and no module imports that file. The server runtime bundle does not name the package. - The provider pack: `provider-pack.json` is byte-identical. Two provider executables and one licence file are hard links to their other copy in the pack, where they were separate files. One empty directory is absent. The pack is 195 MB smaller, and the layer of its large files gets a new digest once. - The transport archives have other bytes, because the Runner build gives large files a fixed timestamp. The receipt names the same files with the same digests. - Paths, modes, owners, the image config and the layer order do not change. - **What a contributor and a maintainer of the image build see:** - **The image build does not run the drift checks of the Runner.** Before, the server build ran the full `build` script of the Runner, so a failed drift check failed the image build on `master`. Now the Runner stages run only the output steps of that script. Repository CI runs the checks on every pull request. `.github/docker-context-checks.Dockerfile` and three exceptions in `.dockerignore` exist for those checks in the image build; a separate follow-up removes them, and this chain does not edit them. - **The dependency-caching layer of the daemon is gone.** Before, cargo-chef compiled the Rust dependencies in a layer of their own, and a change to the Rust source compiled only the Runner's own crates. Now such a change compiles the dependencies again (146 s here for the whole daemon). A change to the server, the UI or the Runner's TypeScript still reuses the compiled daemon. - **A lockfile behind the manifests of the Runner fails the build.** Before, `docker build` resolved it inside the image. The image workflows refresh the lockfile first. A local build prints the command to run. The next pull request makes this failure early and clear. - **The build needs more disk**: about 25 GB of builder storage here, was about 20 GB, with the image export. - Rollback: revert this pull request. Images that exist do not change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), from Anthropic, through Claude Code. The model used tools: it edited files, ran the tests and ran the Docker builds that this description reports. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
e7e28ba241 |
feat(queue): show tasks waiting for an agent's slot as waiting, with their place in line (#21405)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each agent runs at most `runtimeConfig.heartbeat.maxConcurrentRuns` tasks at once. Other runs wait in the agent's queue with status `queued` > - On hosted staging, one agent had all 20 slots full and 295 queued runs. Some tasks waited 3 to 7 hours and never started > - The UI showed these tasks as live or working: a pulsing "Live" badge in the task header and in task rows, a spinning "Working … for 4 hours" in the task chat, and "N live" counts that included every queued run > - People could not tell that work was waiting, why it waited, or what to change > - This pull request makes a deep queue an honest, visible state: "Waiting for Codie · 20 slots max · 41 of 295 in line · Configure" > - The benefit is that operators see the real state and the one setting that changes it ## Linked Issues or Issue Description Refs #21391, #21349, #21242 **What happened?** A queued run that never started looked the same as a running run. The task header, the task rows in lists and the inbox, the task chat and the sidebar counts all showed queued runs as live or working. **Expected behavior** A run that waits for one of its agent's concurrency slots shows that it waits. It names the agent, the agent's limit and the run's place in line, and it links to where the limit is set. The place in line is the order in which the server starts the agent's queued runs. **Why does it matter?** With a deep queue, the UI said "working" for hours while nothing ran. Operators could not see the cause or the fix. ## What Changed - Server: a new module `server/src/services/heartbeat/queued-run-line.ts` now holds the facts that the queued-run dispatcher orders by (`queuedRunDispatchFacts`). The dispatch path in `queue.ts` and the new "place in line" read use this one builder, so the line the UI shows is the dispatch order. The logic moved without change, including the recovery-share fix from #21391. - Server: `executionProjectionsForRuns` adds `agentQueue` (agent, limit, running count, position, queued count) and the label "Waiting for <agent>" with cause `agent_at_capacity` to a queued, never-started run when its agent has no free slot. It reads one batch of agents and running counts. Then it builds one line per full agent (at most 8 agents per read), using only the three context keys that ordering needs. No new polling. - Server: the place in line uses `queuedRunDispatchLine`, an O(n log n) replay of the dispatcher. The plain replay rescans the queue after each step, which is quadratic (measured: 3.6 ms at 300 runs, 76 ms at 3,000, 600 ms at 10,000, all on the event loop). Dispatch itself is unchanged: it stops after the free slots. Each agent's line is memoized for 5 s and shared by concurrent reads. In a preview worktree, the line applies the same creation-time cutoff as dispatch. - Server: the task list summary adds `runningDescendantCount` (descendants whose run started) next to `liveDescendantCount`. `liveDescendantCount` still decides whether a blocked parent is covered. The "n live below" chip uses the new count. - Shared: the new `ExecutionAgentQueue` type and the optional field `ExecutionProjection.agentQueue`. - Server: `/companies/:companyId/live-runs` lists running runs before queued runs, so a deep queue cannot push running work out of the 50-row limit. - UI: a new `AgentQueueWait` component shows "Waiting for Codie · 20 slots max · 41 of 295 in line · Configure", with no spinner and no elapsed timer. Configure opens the agent's Advanced view, where the run policy (`maxConcurrentRuns`) is edited. The task chat (both renderers), the run ledger and the Storybook story use it. - UI: the task header and the task rows (task list, inbox, legacy views) show a quiet "Waiting" instead of the pulsing "Live" when every live run of the task has not started. The full sentence shows on hover. "n live below" counts only started sub-tasks. A waiting sub-task still covers its blocked parent. - UI: the sidebar "N live" counts (Dashboard, agents) count only started runs. - UI: the task chat pill now names other server holds on a queued run (for example "Waiting for maintenance") instead of "Working". - Docs: DESIGN.md records the waiting-for-slot hold. - Storybook: `runs-agent-queue-wait` stories (full line, task chat, task row, one slot, mobile), with play checks. ## Verification - Red/green server test: `server/src/services/execution-projection-agent-queue.test.ts` (embedded Postgres). Before this change, the projection has no `agentQueue`. Now it checks the agent name, limit, running count, positions 1 to 4 in dispatch order (a person's retry first, then priority, not creation order), no wait while a slot is free, and isolation from other companies' runs. It also checks that the positions equal the line that the dispatch builder produces. - `pnpm exec vitest run` on the new test, `queued-run-dispatch-order`, `recovery-wake-herd`, `heartbeat-queued-run-claim-isolation` and `agent-live-run-routes` in `server/`. - Runtime cost (rule `runtime-dependency-boundaries`): the assumption is that the line stays cheap for a deep queue. `queued-run-line.test.ts` checks that `queuedRunDispatchLine` equals the dispatcher's claim-every-run order over 400 random queues (ages around the aging boundary, recovery and person runs, blocked runs, deferred share runs, all share counters). It also orders 10,000 runs in under 250 ms (the quadratic replay takes about 600 ms). The DB test projects a 300-run queue end to end with correct positions 1 to 300. Remaining limit: the line is up to 5 s stale. - Red/green: `issue-list-assignee-filter-routes.test.ts` now asserts `runningDescendantCount: 0` for a parent whose only live descendant is queued and never started. - UI tests: `liveIssueIds.test.ts` (waiting tasks, started runs keep "Live", no "running below" count for waiting sub-tasks) and `TaskChatLiveRunPill.test.tsx` (waiting line with no spinner and the Configure link, other holds named). The suites for the run ledger, sidebar, sidebar agents, task detail, task columns and task chat also pass. - The Storybook stories render in headless Chromium and their play checks pass. - After deploy: open a queued task on staging and confirm the waiting state. ## Risks - Cost on read: a projection read that includes a queued run of a full agent now reads that agent's queued runs (only small columns) and the dispatch facts. This is at most one line per full agent every 5 s, for at most 8 agents per read. Reads with no queued, never-started runs do no more work. - The place in line is the server's prediction of dispatch order. A claim that is refused at start time (a gate, a blocker) can let the next run start first. The position updates on the next refresh. - The process-local "declined aged claim" hint that the dispatcher uses is not visible to other servers, so the projection does not use it. This can change only the aged-oldest share step. - Configure opens the agent's Advanced view (the YAML run policy). The agent's Runtime tab hides the run-policy fields today. - Low risk otherwise: no schema change and no migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, headless browser) and subagents for the UI survey. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
51c81c537b |
fix(browser-tracing): end hidden loads, cap timeouts, backfill hard-load API spans, record list view (#21392)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted instances send real-user page-load traces to Honeycomb (#21156), so we can set page-load SLOs > - Staging data showed four measurement gaps: 140 of 147 hard loads were hidden tabs that became 30 s timeouts, a few timeouts lasted about 2 hours after the computer slept, hard loads had no API child spans, and list views on the tasks page could not be told apart > - These gaps make every average and percentile wrong, so the data cannot back an SLO yet > - This pull request ends hidden loads as `hidden`, caps timeouts at exactly 30 s and marks suspends, backfills hard-load API spans from Resource Timing, and records the list view from an allowlist > - The benefit is page-load numbers that describe what users see, with no cost on the critical path ## Linked Issues or Issue Description Refs #21156 **What is the issue?** Browser page-load tracing records misleading data. A load in a background tab never paints, so it always ends as a 30 s `timeout`. A timeout that fires late (after sleep) reports hours. Hard loads have no API spans because the SDK loads after the page settles. `?view=` is stripped, so all task list views share one route. **Steps to reproduce** Open the all-tasks page in a background tab on a traced instance. The `page.load` span is `outcome=timeout`, `was_hidden=true`, about 30.4 s (reproduced locally on master: 30,359 ms and 30,988 ms). **Expected behavior** A hidden load is counted separately and ends quickly. A timeout lasts exactly 30 s. A hard load shows its API requests. The list view is a dimension. ## What Changed - `page-load-recorder.ts`: new outcome `hidden`. A load that starts hidden, or hides before it is ready, ends at the moment the tab hid (same rule `web-vitals` uses for LCP/FCP). For a hard load, the controller reads Chromium `visibility-state` entries, so a hide before tracing started is caught. A redirect while still hidden renames the hidden span instead of starting a second one. - `page-load-recorder.ts`: any end observed after start + 30 s is a `timeout` that ends at exactly start + 30 s. `paperclip.page.suspended=true` marks an end more than 5 s past the deadline or a gap of more than 5 s between frames. - `controller.ts`: a `PerformanceObserver` (`type: resource`, `buffered: true`) keeps only this origin's `/api/` fetch/XHR entries (max 500) until the SDK loads. It does not depend on the 250-entry resource buffer. - `sdk.ts`: when the SDK loads, it creates `HTTP` child spans under each `page.load` that started before the SDK, for requests that started in the load window and before live fetch tracing began (no duplicates). The name is `HTTP` because Resource Timing has no method (OpenTelemetry's name for an unknown method). Status comes only from `responseStatus` when the browser exposes it. The URL goes through the same sanitizer as live fetch spans and becomes `paperclip.api.area`. Spans carry `paperclip.api.timing_source=resource_timing`. - `page-view.ts`: `paperclip.page.view` for `/:companyPrefix/issues` (values from `TASK_VIEW_KEYS`, `default` when missing) and `/:companyPrefix/decisions` (values from `ATTENTION_PERSONAL_VIEWS`, `mine` when missing). Any other value is `other`. - `span-sanitizer.ts`: allowlist adds `paperclip.page.suspended`, `paperclip.page.view`, `paperclip.api.timing_source`. - `doc/browser-performance-tracing.md`: outcome list, the meaning of "loaded" for hidden tabs, suspend handling, page views, backfilled API spans, attribute table, and example queries that filter `outcome = ready`. ## Verification - Red then green: 17 new tests failed on the old code and `page-view.test.ts` failed to import; all 50 tests in `ui/src/lib/browser-tracing` pass after the change. `PageLoadTracker.test.tsx` passes. `tsc --noEmit` for `ui` passes. - `pnpm verify` exits 2 (a doc is a global CI input), so CI supplies broad coverage. - Real browser (local dev server on an empty isolated instance, stand-in OTLP collector, headless Chromium, tracing config injected into the session response): - Background tab (raw CDP, `Target.createTarget background:true`): `outcome=hidden`, duration 0 ms. Master: `outcome=timeout`, 30,359 ms. - Tab covered 400 ms into the load: `outcome=hidden`, 434 ms. Master: `timeout`, 30,988 ms. - Hard load of the all-tasks page: `page.load` with 13 to 38 `HTTP` child spans (areas `companies`, `plugins`, `cli-auth`, and others, status 200 where exposed), same trace, parent = `page.load`. - `?view=mine`: `paperclip.page.view=mine`. - Leak scan of every exported body: no URL, host, company id, company prefix, user id, or query text. - Entry bundle: 3,822,045 → 3,823,991 B raw (+1.9 KB), 1,154,280 → 1,155,097 B gzip (+0.8 KB, 0.07%). Lazy SDK chunk: 84,322 → 85,229 B raw, 27,304 → 27,630 B gzip. ## Risks - Low risk. The change is limited to the operator-gated Observability path on hosted instances; self-hosted installs never start the controller. - Behavior shift for dashboards: hidden tabs are now `outcome=hidden` instead of `timeout`, so timeout rates drop. Queries should filter `outcome = ready`. - The observer adds one callback per batch of finished requests while tracing is on, until the SDK loads. It keeps at most 500 entries. - Backfilled spans do not join server traces because no `traceparent` was sent. The docs say so. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, headless Chromium). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8af889f612 |
feat(mcp-apps): view messages with consent, app context, full screen and provider tool icons (#21266)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - MCP Apps (#21119) show interactive widgets from MCP servers, such as Circleback meeting cards, with the agent's reply in the task chat. > - The first version deferred three parts of the MCP Apps spec. A widget could not propose a chat message (`ui/message`). It could not give the agent context (`ui/update-model-context`). It could not go full screen. > - Paperclip also ignored the `icons` that MCP servers declare on tools (SEP-973). > - The security review of #21119 showed what these features need from day one: explicit consent, untrusted-data labels and strict icon sanitizing. > - This pull request adds the three host features and provider tool icons, with those protections built in. > - The benefit is that widgets can now work with the person and the agent, not only display data. ## Linked Issues or Issue Description Refs #21119 **Subsystem affected** MCP Apps host (task chat widgets), tool catalog, run wake prompt. **Problem or motivation** The MCP Apps host refused `ui/message` (-32601), dropped `ui/update-model-context` and kept every view inline. MCP tool `icons` were not parsed or shown. So widgets could not ask the agent a follow-up, the agent could not see what the person selected in a widget, large widgets were cramped, and tool rows used only generic icons. **Proposed solution** Add consent-gated view messages, per-view app context delivered to the task's next run as untrusted data, a full-screen dialog, and sanitized provider icons. Details are under "What Changed". **Alternatives considered** - Send view messages with a new server route that creates the comment itself. Rejected: comment creation has many rules (agent chat ownership, paused conversations, queued delivery, wakeups, receipts). The widget's Send button uses the same send path as the composer. A small route then records which view the comment came from. - Keep the iframe in place for full screen with CSS. Rejected: chat containers can create containing blocks that break `position: fixed`. A moved iframe always reloads, so the dialog hosts a new frame instead. **Roadmap alignment** Continues the MCP feature work after #21119, #21140 and #21219. It does not touch the MCP Tasks work (gateway `tools/call` polling and elicitation). ## What Changed - **`ui/message`.** The host shows "<App> wants to send: …" with **Send** and **Dismiss** under the widget (and inside the full-screen dialog). Nothing is sent before the click. Send posts the text as the viewer's own comment through the chat's normal send path (`handleThreadAdd`). `POST /api/issues/:issueId/mcp-app-views/:viewId/messages` then logs `issue.mcp_app_message_sent`. It accepts only the caller's own comment on the same task, matched by `clientRequestId`, from the last 10 minutes. One proposal per view can wait at a time. The view gets `{}` after the send, or -32000 "Message sending denied" after Dismiss. Only `user` text up to 8,000 characters is accepted. - **`ui/update-model-context`.** `PUT /api/issues/:issueId/mcp-app-views/:viewId/model-context` stores the latest context per view in `tool_mcp_app_views.model_context` (text blocks and `structuredContent`, 16 KB at most). Updates are applied in order and deduped in the browser and on the server. A change logs `issue.mcp_app_context_updated`, and an empty update clears the context. The task's next run gets the three most recently updated contexts as an `## App context (untrusted)` wake-prompt section. The section quotes the data as JSON and says it is third-party data, not instructions. The read runs in parallel with the other run preparation and never fails the run. CLI adapters and the native runner render it through the same `renderPaperclipWakePrompt`. - **Who may do this.** Messages and context need a signed-in person with `issue:comment` on the task. Agents and API keys get 403. Other tasks and companies get 404. `message` and `updateModelContext` are advertised in `hostCapabilities` only where the viewer can send messages. - **Full screen.** The host advertises `availableDisplayModes: ["inline","fullscreen"]` and handles `ui/request-display-mode`. It honors the view's declared modes and answers with the resulting mode. It sends `host-context-changed` with `displayMode` and the container (fixed `height`/`width` in full screen), and again when that container resizes. A new **Open full screen** control sits next to "Widget from …". Full screen is a Paperclip dialog that closes with Exit, Escape or the view's own request. - **Provider tool icons.** `icons` on MCP tool descriptors are sanitized into the new `tool_catalog_entries.icons` column, outside the version hash, so a new icon never quarantines a reviewed tool. Only `https:` URLs and `data:image/*` URIs are kept, and SVG only as `data:`. A `data:` icon is at most 32 KB, a URL at most 2,048 characters, with at most 4 icons per tool. `GET /api/companies/:companyId/tool-icons` lists them keyed by gateway tool name, in one cached request per chat. Task chat tool rows and the app's actions list show them through `ProviderToolIcon`. It renders `<img referrerPolicy="no-referrer">` at a fixed size and never inlines markup. It checks the source again before rendering and falls back to the taxonomy icon. - **Schema.** Migration `0477` adds `tool_catalog_entries.icons`, `tool_mcp_app_views.model_context` and `model_context_updated_at`, all nullable. - **Docs and stories.** `doc/connections/MCP-APPS.md` documents the new behavior, and the finished items are removed from "Limits and follow-ups". `Tasks/MCP App views` has new stories: "View asks to send a message", "Full screen" and "Tool rows with provider icons". The fixture view now uses `ui/message`, `ui/request-display-mode` and `ui/update-model-context`. ## Verification - `pnpm exec vitest run server/src/__tests__/mcp-apps.test.ts server/src/__tests__/tool-catalog-publication.test.ts server/src/__tests__/openapi-routes.test.ts packages/shared/src/mcp-icons.test.ts packages/shared/src/mcp-apps.test.ts`: pass (embedded Postgres). These tests cover: - Context: stored, deduped, logged once, 413 when too large, cleared, scoped to the task and company, refused for agents (403), and rendered into the wake prompt as untrusted data. - Messages: attributed only to the viewer's own recent comment on the same task. A teammate's comment, an unknown request id or an old comment gets 404. An agent gets 403. - Icons: stored sanitized, and the version hash and status do not change. Another company sees nothing. - `cd ui && pnpm exec vitest run src/components/mcp-app/ src/components/ProviderToolIcon.test.tsx src/pages/apps/app-detail/`: 69 pass. These tests cover: - The host protocol for each new method: capabilities, consent wait, one proposal at a time, invalid input, ordered and deduped context, the size limit, and the display-mode rules. - The consent card (nothing sent without Send). - The full-screen dialog (one frame at a time). - Icon rendering (no-referrer `img`, no inline SVG, fallback). - `cd ui && pnpm typecheck` (including Storybook) and `tsc --noEmit` for the server: clean. - Red/green: the new route and host tests fail on `master`, where the routes do not exist and the host answers -32601 or drops updates. The icon descriptor test failed when the publication write was removed. - Not run locally: the full suite, and `packages/adapter-utils/src/server-utils.test.ts › runChildProcess › does not crash when the child closes stdin before a delayed write`. That test times out on this Mac with or without this change, so it is a local platform issue. CI covers both. ## Risks - Migration risk is low: three nullable columns, no backfill. - Every run on a task now does one extra indexed read, in parallel with the other preparation. Errors are logged and the run continues without app context. - If the attribution request fails after the comment is sent, the comment stays without the `mcp_app_message_sent` activity row. This is documented. - Switching between inline and full screen restarts the view, so state inside the widget resets. This is documented as a limit. - Icon names shortened with a hash by the gateway (very long names) do not match, and those rows keep the taxonomy icon. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. One subagent (the same model) built the provider-icon part. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5ede64f247 |
fix(recovery): settle an older run's saved-work hold, fold a waiting Retry into the isolated continuation, and plain recovery wording (#21348)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - When a run is interrupted, a saved-work hold protects the files it
left. The fresh-continuation fallback then continues the task in a new
run, or asks a person to press Retry.
> - Staging found that an older run's saved-work hold made the dispatch
gate cancel every new run of the task. The hold was never settled, so
the task could not run again, and no person could act on the reason.
> - Recovery notices also showed internal codes and words such as
"board", "inspect", "repair" and "no live execution path". People cannot
act on these.
> - A person's Retry pressed while an isolated continuation was waiting
for delivery was saved, and then ran a second time after the
continuation.
> - This pull request settles an older run's hold through the fallback's
existing machinery, uses plain wording in every remaining recovery
notice (server and UI together), and folds a waiting Retry into the
isolated continuation that answers it.
> - The benefit is that a task held by an older run continues by itself,
or on one Retry, in a fresh isolated workspace. People read only what
happened and the one thing they can do, and a Retry never starts a
redundant second run.
## Linked Issues or Issue Description
**What happened?**
1. An earlier run's saved-work hold (for example "Automatic recovery of
saved work failed … Retry to try again") made the dispatch gate cancel
every new run of the task with `execution_reconciliation_required`. The
fresh-continuation fallback only took holds of the task's latest run.
Each refused run looked like a newer run, so the hold was never settled.
2. Recovery notices and recovery-task comments showed internal codes,
raw IDs, "board", "inspect", "repair" and "no live execution path". The
hold for remote files that could not be restored asked a person to
"verify its exact stop receipt" and to "record workspaceRepairEvidence".
3. On a projectless task whose old legacy writer might still be alive, a
person's Retry pressed after the isolated continuation was recorded, but
before it was delivered, stayed deferred. When the continuation run
ended, task release promoted that Retry into a second, sequential run.
Concurrent writers were already impossible on this path, because the
Retry waited behind the continuation. The second run was redundant: the
continuation was the answer to the person's Retry.
**Expected behavior**
An older run's hold does not block new runs indefinitely. The task
continues in an isolated workspace, by itself while the interruption is
recent, or on a person's Retry. Notices say what happened and the one
thing to do. A person's Retry answered by the continuation never starts
another run.
**Steps to reproduce**
1. Leave a hold from an older run whose saved-work recovery gave up,
then start a new run of the task. Before this change, the run is
cancelled at dispatch every time and the hold stays.
2. Open a task whose run was cut off and its retries exhausted. Before
this change, the notice reads "No live execution path … visible for
intervention".
3. On a projectless task with an isolated continuation recorded, press
Retry before delivery, then let the continuation end. Before this
change, a second run follows it.
## What Changed
- Older-run holds (`fresh-continuation-fallback.ts`,
`modules/run-dispatch/adapters/postgres.ts`):
- A run that was cancelled at dispatch (status `cancelled`, never
started, `execution_reconciliation_required`) never ran and follows no
run. The held-run listing and the fallback's successor check now ignore
it.
- The older run's hold goes through the fallback's existing machinery. A
saved-work recovery that is still in progress keeps its bounded wait.
Holds that a person owns stay. Past the bound, the hold is settled and
the task continues in a new isolated workspace. Held files are kept,
recorded as `kept_not_applied`, and never applied.
- One recency rule, `interruptionEligibility` (the #21222 rule), judges
recency by the newest run that the hold refused at dispatch. It no
longer uses the end time of the hold's own run. If the last attempt is
older than 24 hours, no automatic continuation starts. A person's Retry
always continues the task.
- After the dispatch gate refuses a run for another run's hold, it hands
that hold to the fallback in the same transaction. The normal dispatch
path is not changed.
- A person's Retry and the isolated continuation (`heartbeat/queue.ts`,
`execution-recovery-resolution.ts`, new
`fresh-continuation-retry-fold.ts`):
- The wake-admission transaction that inserts an isolated continuation
run (context `freshContinuationIsolation`) also coalesces the person's
waiting `task_retry` wake for that task, under the task row lock. This
applies only to a Retry with no message attached. The wake gets `status:
coalesced` and `runId` set to the continuation. A server stop after the
commit cannot leave the Retry unfolded.
- The delivery pass is idempotent about this. An action whose own
continuation is already queued is no longer hidden from the pass. When
the pass finds that continuation, it marks the action delivered and
folds any Retry that is still waiting.
- Before this change, the fold covered only a Retry that was already
waiting when the continuation was recorded. A Retry that carries a
person's message is not folded. Only a Retry whose source run is the
continuation's own source run is folded, and only while the continuation
is queued or running. A Retry for the continuation's own failure gets
its own run.
- Plain wording (server and UI together):
- "No live execution path" is now "Task could not continue". One seed
(`buildNoRunningExecutionNoticeSeed`) builds it for every producer.
- Exhausted saved-work recovery: "Paperclip could not recover the
agent's saved files … Press Retry to try again".
- The non-retryable continuation notice no longer shows the error code.
Recovery-task comments no longer show `stranded_issue_recovery`, raw IDs
or retry reason codes.
- `legacy-execution-recovery.ts` uses plain text for remote files that
could not be restored.
- `IssueRecoveryActionCard`: "Board decision required" is now "Waiting
for you".
- The UI still recognizes notices saved with the old text
(`adjacent-errors.ts`, `system-notice-humanizer.ts`).
- The precise codes stay in the evidence and in the notice details.
- Merge with current master: `recovery/service.ts` keeps master's
three-way choice (retry exhausted / continuation refused /
non-retryable). The non-retryable branch now uses the plain notice
without the code.
- Code comments that the original commits added are removed
(CONTRIBUTING).
## Verification
Red-to-green tests:
- `server/src/services/fresh-continuation-fallback.test.ts`:
- "continues a task held by an older run's saved-work hold in a new
isolated workspace…"
- "keeps the wait of a saved-work recovery that is still advancing…"
- "keeps a hold a person owns blocking…"
- "leaves an old hold for a person's Retry, and that Retry runs
isolated" (refused run and waiting Retry)
- "continues a recent restart-stopped run whose prepared remote
workspace could not be restored…" (red on the wording)
- "leaves an older restart-stopped remote restore failure for a person's
Retry, which then runs isolated"
- Same file, "consumes a person's Retry exactly once when the server
stops right after admitting the isolated continuation": the test runs
the admission and nothing after it. The Retry is already coalesced. The
test then resets it to unfolded, as an older release would leave it, and
runs the next delivery pass. The result is exactly one run and the Retry
consumed. Red without the in-transaction fold, and red without the
delivery-pass fold.
- Same file, "never folds a Retry for the admitted continuation's own
failure into that ended continuation": the continuation ends (real
`cancelRun`), then the person presses Retry on it. The Retry gets its
own run. Red without the source-run and liveness guard.
- Same file, "runs a person's Retry pressed before delivery as the
recorded isolated continuation, never in the shared directory": a
projectless task, an old legacy writer that may still be alive, and an
isolated continuation pending. Delivery starts exactly one run, which
carries `freshContinuationIsolation` in a `fresh_isolated_workspace`.
Extended here: after the continuation run ends (real `cancelRun` release
path, then `resumeQueuedRuns` and delivery), no run comes from the
person's Retry. The Retry wake stays `coalesced` into the continuation.
Red without the delivery fold (the Retry became a second run), green
with it.
Local runs on this branch (based on master
|
||
|
|
2483cb46c2 |
feat(plugins): deliver configured environment to plugin Runner tasks (#21280)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent gets environment variables from its execution environment, its own configuration, its project, and its routine. Secrets are resolved for each run. > - A plugin environment can run the agent as a native Runner task on its own host, through the environment task contract. > - That contract has no field for the agent's variables. Runs on a plugin environment get only the variables that the plugin sets itself. > - This pull request adds a versioned contract field for the values, sends them on a session handoff, lets Codex commands inherit the variables in an external sandbox, and lets a plugin protect the variables that it sets itself. > - The benefit is that plain and secret variables work on plugin environments in the same way as on local environments. ## Linked Issues or Issue Description Refs #21233. That pull request by @lucktastic finds the same gap and proposes the same capability and field names (`configuredEnvironmentVersion: 1`, `configuredEnvironment`). This pull request uses the same names, so providers implement one contract. It differs in these places: - Codex commands get the variables through an external-sandbox denylist, not through key names sent at `prepare`. A retained Runner does not need a new preparation when a key is added or removed. - A session handoff `resume` gets the values, because it starts a new Runner on another computer. Value changes on the same computer need no new field: Core's process reuse key includes the values, so Core starts a new task. - A driver can declare `protectedEnvironmentKeys`. Paperclip rejects or warns when one is saved. - The run log names skipped variables only for an older provider. Thank you to @lucktastic for the diagnosis and the contract shape. **What happened?** An agent has environment variables, plain and secret. Its runs on a plugin task environment do not get them. `PAPERCLIP_CONFIGURED_ENV_KEYS` in the run lists only the variables that the plugin sets. **Expected behavior** Each run gets the agent's configured variables. **Steps to reproduce** 1. Set a plain and a secret environment variable on an agent. 2. Assign the agent to a plugin environment that provides Runner tasks. 3. Start a run. Read the names in `PAPERCLIP_CONFIGURED_ENV_KEYS` in a command that the agent runs. 4. The agent's names are not in the list. ## What Changed - **SDK contract** (`packages/plugins/sdk/src/environment-tasks.ts`): - A provider advertises `configuredEnvironmentVersion: 1` on `describe`. - `submit` gets `configuredEnvironment`. `resume` gets it only for a session handoff. - Names, count, and sizes have the same limits as the Runner projection. - **Server**: - `createRunnerdBackendWithinSessionClaim` gives the run's configured projection to the plugin task owner. The projection has no provider credentials and no generated runtime keys. - `createPluginRunnerTaskOwner` sends the fields only to a provider with the capability. An older provider gets no new field, and the run log names the variables that were not set. The log never prints values. - **Codex** (`codex-security-config.ts`): - An externally sandboxed launch uses `inherit="all"`, `ignore_default_excludes=true`, and an `exclude` list of Codex's own credentials, instead of an `include_only` list fixed at preparation time. - A test reads the Runner's Codex environment keys, and fails if a new key is not classified. - Host launches keep the bounded allowlist. - **runnerd** (`codex_provider.rs`): `runtime_notification_codex_args` still grants the notification files to read-only profiles when no allowlist is present. Plan mode in an external sandbox needs these grants. - **Protected keys**: - An environment driver can declare `protectedEnvironmentKeys` in its manifest. - Paperclip rejects a protected key on an agent or environment that uses the driver. The 422 has code `protected_environment_variable`. - On a project, a routine, or another agent, Paperclip saves the key. The response has a `configurationWarnings` notice that names the environments. The project and routine pages show the notice as a toast. The agent pages already show configuration warnings. - **Docs**: `doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md` describes the provider obligations. - `tests/runner-e2e/source-sdk-public-inputs.json`: refresh the reviewed SDK source pin. ## Verification - `packages/plugins/sdk`: `vitest run tests/environment-tasks.test.ts` (38 pass). Accepts bounded values. Rejects bad names, NUL bytes, too many variables, and oversized values. - `server`: `vitest run src/services/native-runtime/plugin-runner-task.test.ts` (95 pass). Values go only to a provider with the capability, never in `prepare`. `resume` sends values only for a handoff. - `server`: `vitest run src/services/protected-environment-keys.test.ts src/__tests__/project-routes-env.test.ts` (10 pass). Rejection, notices, and the project route response. - `server`: all route suites that import the project, routine, environment, or agent routes (45 files, 1689 pass). - `packages/shared`: `vitest run src/validators/plugin.test.ts` (64 pass). - `packages/paperclip-runner`: `vitest run src/drivers/codex src/configured-environment.test.ts src/live/runner-provider-launch.test.ts` (444 pass). - `packages/paperclip-runner/runner`: `cargo test -p paperclip-runner-core --lib runtime_notification` (9 pass, 2 ignored as before). - `pnpm typecheck` in `packages/shared`, `packages/plugins/sdk`, `packages/paperclip-runner`, `server`, and `ui`. - Not run locally: `pnpm verify --full` and browser suites. Required CI covers the final revision. ## Risks - **Existing providers:** low risk. Core sends no new field to a provider without the capability. Their runs behave as before, with one added log line when variables were skipped. - **Codex in an external sandbox:** commands now see every variable in the Runner environment except Codex's own credentials. The execution environment builds that environment from an allowlist. If the Runner gives Codex a new credential, the drift test fails until the key is classified. - **Saving env:** new rejections happen only when an installed driver declares `protectedEnvironmentKeys`. No driver declares them today. - **Providers:** they must keep the values transient and never save or log them. The doc lists these obligations. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3b4c3265ed |
fix(recovery): offer Try again on a model-unavailable failure and recheck the model first (#21244)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a native run fails, the task thread shows a failure marker, and some markers offer Try again > - A `native_provider_model_rejected` failure blocks the task for the person and shows "Model unavailable", but it has no Try again control > - Recovery must just work: after the person fixes the model, they need one obvious control that continues the task, and that control must not start a run that is sure to fail > - This pull request shows Try again on the "Model unavailable" marker and makes the retry recheck the selected model before it starts a run > - The benefit is that the person fixes the model, presses Try again, and the task completes, and a too-early press explains which setting to change instead of failing again ## Linked Issue or Issue Description Refs #21210 **What happened?** Paid run 43321773136, `recovery.retry-exhausted` (OpenCode, `openrouter/deepseek/deepseek-v4.1-flash-1017`). The run failed with `native_provider_model_rejected`. The task stayed blocked and the thread showed "Model unavailable", but no Try again control appeared (`task-chat-run-failed-try-again` not found in 30 s). Only the "Run failed" and "Usage limit reached" markers offer Try again. **Expected behavior** The "Model unavailable" marker offers Try again. Pressing it checks the selected model first. If the provider-rejected model is still selected, no run starts, and the person sees which setting to change with a link to it. After the person changes the model, Try again starts one retry of the original run. **Steps to reproduce** 1. Set an agent's model to an id the provider rejects. 2. Assign it a task. The run fails with `native_provider_model_rejected`. 3. Open the task. Before this change, the marker has no Try again. ## What Changed - `server/src/services/native-runtime/model-rejected-retry.ts`: `rejectedModelRetryRefusal` reads the model the failed run sent (`runnerProfileJson.nativeExecutionInput.provider.model`) and the model a new run would use (the task's model override for its assignee, else the agent's model). If they are the same, it returns a 409 refusal with code `native_provider_model_still_unavailable`, the setting to change (`agent_model` or `task_model_override`) and the rejected model. - `server/src/routes/agents.ts`: the exact failed-run retry (`POST /agents/:id/wakeup`, `reason: retry_failed_run`) applies this check before it queues anything. - `ui/src/components/task-chat/adjacent-errors.ts`, `TaskChatThreadView.tsx`: one `isRetryableFailureMarker` rule, now including "Model unavailable". - `ui/src/lib/model-retry-refusal.ts`, `ui/src/pages/IssueDetail.tsx`: a refused retry shows "Model still unavailable" with the server's message and an "Open <agent> model settings" link to `/agents/<agent>/runtime`. A task override refusal names the override in the message. - Eval `recovery.retry-exhausted` (oracle `runtime.retry-exhausted.v7`): the rejected model stays selected after the failure. The person presses Try again (409, toast names the model, its link opens the agent's model settings, no run starts in 20 s, task stays blocked), restores the working model, presses Try again (202), and the one retry of the original run uses the working model, saves the brief and completes the task. 3 runs: failed 1, succeeded 2. The budget-hold and two follow-up chat phases are removed; the case now tests the model recheck. `recovery.yaml` context and expected outcome updated, recorded in `chat-full-catalog-testops-manifest.json` `changedFields`, pinned in its test, descriptors and inventory digest updated. ## Verification - RED on master: `TaskChatThread.test.tsx` "offers Try again on a model-unavailable failure and retries that run" fails (no control). `agent-live-run-routes.test.ts` "refuses a retry while the rejected model is still selected and starts no run" fails (the route queues the run). - GREEN: both pass. `agent-live-run-routes.test.ts` 101/101. - `server/src/services/native-runtime/model-rejected-retry.test.ts` (embedded PostgreSQL), 6/6: refuses while the agent still selects the rejected model (names the agent setting); names the task override; allows the retry when the agent or the override now selects another model, or the run failed for another reason; ignores another company's run. - `ui/src/lib/model-retry-refusal.test.ts`: names the setting and links to `/agents/morgan/runtime`; no link for a task override; other errors fall through to the generic toast. - Server `tsc --noEmit` clean. UI `tsc` clean apart from the pre-existing generated storybook catalog. `node scripts/check-token-gates.mjs`: clean. - Eval: `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog tests/runner-e2e/chat-catalog-domains tests/runner-e2e/chat-*.test.ts`: 339 files pass. New calibration rejects an accepted early retry, a run started while the model is rejected, a refusal without the model, setting link or blocked task, a retry that used the old model, and duplicate or missing retries. `pnpm test:e2e:runner:typecheck`: pass. Manifest test 10/10. `source-sdk-public-inputs --check`: `changed: false`. - Paid run: Runner Full-Stack E2E 43338695850 on throwaway branch `chat-eval/qa-mr-paid`. Result to follow in a comment. ## Risks - The recheck compares the selected model with the model the provider rejected. It does not call the provider. A model the provider rejected stays refused until the person selects another one; a different but also unsupported model starts a run that fails with the same marker again. - The refusal message is a toast (15 s). The marker detail still says to choose a supported model. - No persisted shape changes; runs saved by older builds carry the same `runnerProfileJson` field. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2d1c2db64a |
feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use connected apps through MCP. Paperclip already hosts MCP Apps views and speaks stateless MCP 2026-07-28. > - The MCP Skills extension (SEP-2640, Final) lets an app publish skills (instructions and files) next to its tools. Hugging Face already serves 25. > - Paperclip did not read them, so agents missed each app's own workflow guidance. > - Skills are a prompt-injection surface. The spec makes manifest binding, verification, origin tagging and approval revocation host requirements. > - This pull request makes Paperclip a Skills host: discover, review, verify, and deliver app skills to the agents that use the app. > - The benefit is that connected apps can teach agents their workflows, under board control. ## Linked Issues or Issue Description **Problem** Connected MCP servers can publish skills with the `io.modelcontextprotocol/skills` extension. Paperclip ignored them. A board could not see, review or approve them, and agents never received them. **Proposed solution** Implement the host side of SEP-2640. List skills during catalog refresh, record them per connection and company, bind board approval to the full file manifest, and load approved skills into agent runs with verified, lazily fetched content. **Why it matters** Apps ship workflow guidance with their tools. Without the extension, each team must copy that guidance into company skills by hand, and it goes stale. Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work this builds on). ## What Changed - **Client capability.** Every stateless request declares `io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The capabilities that `server/discover` advertises are remembered beside the protocol era. - **Discovery.** Catalog refresh calls `skills/list` (paged) only when the server declared the extension. A listing inside its `ttlMs` is reused. A skills failure never fails the tool refresh. Nothing runs while a run starts. - **Schema.** New `tool_mcp_skills` (company, connection, skill URI; frontmatter, sorted manifest, manifest digest, review state) and `tool_mcp_skill_files` (verified bytes keyed by connection, URI and digest). Both cascade with the connection. - **Governance.** New skills wait for review. Approval binds to the order-free digest of every `{uri, digest, size}`; a request for any other digest gets 409. A changed manifest or frontmatter revokes approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized and invalid skills are declined with a reason. Approve and decline use the company skill policy (`skills.install`, `external_package`, `mcp-connection:<id>`). - **Run time.** Approved skills of connections the agent can use load in parallel with other skill preparation, inside a 3 s budget. `SKILL.md` is read with the agent's grant and checked for size, SHA-256 and frontmatter. A failure calls `skills/get` and takes the skill out of use. Each skill becomes a per-run bundle on the connector skill path, named in its app's namespace, tagged with its origin. Paperclip writes the frontmatter: `allowed-tools` only if the board approved it. - **Supporting files.** `GET /api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed files to an agent in an active run, through the skill's own connection. The sandbox bridge allows the route. - **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…` (list, read instructions, approve, decline). The Skills page shows "Skills from connected apps" with app, state, size, requested tools and plain-language notes. It renders nothing when no app publishes skills. - **Docs.** `doc/connections/MCP-SKILLS.md`, linked from `doc/connections/README.md`. - **Shared frontmatter parser.** It now decodes YAML single-quoted scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and would fail the frontmatter check; now all 25 match. The generated runner copy is regenerated. File size notes: `server/src/services/tool-gateway.ts` grows by about 200 lines. The run-time reads must use its grant resolution and `callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting the gateway is a separate refactor. `CompanySkills.tsx` and `CompanySkills.production.tsx` gain 2–4 lines each; the section lives in its own component. ## Verification - `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake stateless skills server and embedded Postgres. It covers: discovery only after the server declares the extension; the capability in `_meta`; `ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md` at load; supporting files on read); a stale-digest approval refused; a reorder that keeps approval; a changed or added file that revokes it; dynamic skills declined; a same-size tampered `SKILL.md` rejected, refreshed with `skills/get`, and taken out of use; mismatched frontmatter rejected; reads outside the manifest refused; the same URI and name from two servers kept apart; company scoping; a slow app skipped within the budget and cached later; HTTP authorization (board only, agent needs an active run, run header must match) and activity logging. - Red/green: with the digest check disabled, the tamper test fails; with the manifest comparison disabled, the revocation test fails. Both pass with the code. - `server/src/__tests__/mcp-protocol-versions.test.ts`, `mcp-apps.test.ts`, `connection-instructions.test.ts`, `tool-gateway-discovery-http.test.ts`: 47 pass (two capability assertions updated for the new extension). - `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar case; the runner contract test passes (23 tests). - `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and `ui/src/pages/CompanySkills.test.tsx` (44) pass. `packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89) passes. - `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes except the known missing generated Storybook catalog files. - `pnpm verify` returned exit 2 (doc change is a global CI input); required CI supplies broad coverage. - Live check against `https://huggingface.co/mcp` without credentials, through Paperclip's own services and the real egress guard: 25 skills discovered as pending; three approved; cold load 135 ms, cached load 13 ms; a supporting file read through the manifest; the bundle tagged with its origin. - Storybook: `Agents and reusable capabilities/Skills and Skill Studio/Connected app skills` (needs review, all states, tool request, name collision, reading instructions). ## Risks - **Migration:** two new tables only. No changes to existing tables. Safe with the previous release running. - **Run start:** companies without an approved app skill pay one indexed read. With approved skills the wait is capped at 3 s; late or failing skills are left out with a `skills.mcp_unavailable` run warning and never fail the run. - **Agent harness enforcement:** Paperclip cannot gate each code-execution tool call an agent makes while following an app's skill. Approval is explicit and per skill, the approval note says agents may run commands the skill describes, and `allowed-tools` is never granted without a separate tick. - **Shared-home adapters** receive app skills like company skills. A revoked skill leaves on the next skill sync. - Not in this PR: per-connection opt-in for dynamic skills; serving Paperclip's own skills from its public MCP endpoint; `resources/directory/read`; app skills on an agent's skills tab. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, local test runs, live HTTP checks). Extended reasoning enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |