mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
9009fbdedb886c19185d5f55c2e3d7a8ec8fbc83
28
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
1e13c58a4e |
fix(heartbeat): name the cause of generic adapter failures and keep SQL text off the board (#20483)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat run settlement records why a run failed, and the board shows that record. > - Runs that fail without a named code settle as `adapter_failed` (or `setup_failed`) with no cause. > - A user, an operator, and the restart chaos harness cannot tell what went wrong from such a run. > - Some run errors also show raw SQL statement text to users. > - This pull request records a bounded `failureCause` code for generic failures, shows it on the board, and keeps SQL text out of run errors. > - The benefit is that a generic failure now names its cause, for example "adapter failed (40001)", without leaking messages, SQL, or secrets. ## Linked Issues or Issue Description Refs #20385 (the nested-cause extraction this change reuses). Related: #20421 (restart chaos universal checks U1-U4). **What happened?** The restart chaos audit of 79 saved waves found 30 failed runs with `errorCode: "adapter_failed"` and a null or generic error. Every U3 ("diagnosable") warning was `adapter_failed`. Nobody could tell what failed. Some board run errors also showed raw SQL statement text from wrapped database errors (Drizzle `Failed query: ... params: ...`). **Expected behavior** A run that fails with a generic code names a short, safe cause. The board shows the cause next to the stop reason. Run error text never shows SQL statements or their parameters. **Steps to reproduce** Make an adapter throw a wrapped database error (an `Error` whose nested `cause` has `code: "40001"` and whose message has a Drizzle `Failed query:` statement). Before this change, the run settles as `adapter_failed` with no cause, and `heartbeat_runs.error` contains the SQL statement and parameters. The new tests in `server/src/services/heartbeat/run-completion.test.ts` reproduce this. ## What Changed - Runner: move the #20385 event-commit cause extraction into `safeErrorCauseCode` (`packages/paperclip-runner/src/control-plane/safe-error-cause.ts`) and export it. The Runner's own diagnostics use the same function, unchanged in behavior. - Shared: add the `failureCause` contract (`packages/shared/src/run-failure-cause.ts`): one `[A-Za-z0-9_.:-]` token of at most 80 characters, `normalizeRunFailureCause`, `readRunFailureCause`, and `withoutSqlStatementText`. - Server: add `server/src/services/heartbeat/run-failure-cause.ts`. Settlement records `resultJson.failureCause` only when the code is generic (`adapter_failed`, `setup_failed`). `errorCode` stays the same. - Thrown errors (`failRun`, `failRunSetup`): `provider_http_<status>`, else `provider_<category>`, else the first safe code in the error and up to three nested causes (`40001`, `ECONNRESET`), else the error class name, else `unknown_error`. - Adapter results without a code (`completeRun`): the error family, `signal_<name>`, `process_cancellation_failed`, `exit_code_<n>`, `adapter_reported_error`, or `unknown_error`. - The failure log lines (`heartbeat execution failed`, `heartbeat execution setup failed`, and a new `heartbeat run failed` warning for generic adapter results) include `failureCause` and `errorCode`. - Server: replace SQL statement text in run error text with "Database query failed (<cause>)" at all three settlement paths, including the cancelled branch. The execution blocker's `runError`, the attention inbox (failed-run excerpt and metadata), and continuation summaries apply the same replacement on read, so older rows also stop showing SQL there. - Server: Retry admission and failed-preparation replay accept only a closed set of `resultJson` keys. `failureCause` is added to each of those lists (`task-retry.ts`, `chat-channels/failed-run-retry.ts`, `cancelled-native-startup.ts`, `fresh-workspace-continuation.ts`). It is a diagnostic code with no execution or replay authority, so a failed-preparation run stays retryable. - Server: `runsForIssue` returns a validated top-level `failureCause`. It is null for older rows, non-string JSON values, and malformed values. - UI: `RunForIssue.failureCause`; `IssueRunLedger` shows "adapter failed (40001)". Older runs keep "adapter failed". The Storybook run-ledger fixture has a failed run with a cause. - Restart chaos harness: run samples keep `failureCause`. U3 treats a generic code with a specific `failureCause` as diagnosable. A bare `Error`, `unknown_error`, or the no-code fallback `adapter_reported_error` still counts as vague. U3 stays a warning. - Docs: `doc/run-log-events.md` and the harness README describe the field and the SQL replacement. ## Verification - Red before the fix: the four new `run-completion.test.ts` cases failed against `origin/master`'s `run-completion.ts` (no `failureCause`, SQL text in `error`). - Green: - `server/src/services/heartbeat/run-completion.test.ts`: all tests passed (real embedded PostgreSQL). New cases cover a nested `40001` three levels deep for execution and setup failures, SQL and a secret parameter absent from `error` and `resultJson`, the bound (an 81-character code and a SQL "code" fall back to the class name), `exit_code_137` for a code-less adapter result, a cancelled result with SQL text, no cause for a named code, and the log line. On this loaded host the suite's `afterAll` database teardown hit its 30 s hook limit after every test passed (the known local teardown limit, also seen in #20385). CI runs the suite. - `server/src/services/heartbeat/run-failure-cause.test.ts` (4), `server/src/__tests__/activity-service.test.ts` (recorded, legacy, malformed, overlong, boolean, and number cause projections), `server/src/__tests__/issue-continuation-summary.test.ts` (legacy SQL row). - `packages/shared/src/run-failure-cause.test.ts` (6), `ui/src/components/IssueRunLedger.test.tsx` (21), restart chaos `universal.test.ts` and `observer.test.ts`. - `packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts`: 207 passed (the stall-diagnostic cases still read `40001` and `ECONNRESET`). - Typecheck: shared, runner, server (all layers), UI, restart chaos harness. `pnpm check:token-gates`: all gates clean. - `pnpm verify` returned exit 2 (global doc input changed); required CI supplies broad coverage. - CI on `6295f8d` found that the new key made failed-preparation runs ineligible for Retry (`chat-channels.integration.test.ts` saved board-message Retry got 409; `workspace-recovery-preparation-dispatch.test.ts` dispatched 0 Retries). Both reproduced locally and pass after the allowlist fix, with the other 459 Retry cases in `chat-channels.integration.test.ts` and all 25 in `workspace-recovery-preparation-dispatch.test.ts`. - No chaos waves were run. Storybook was not opened in a browser; the fixture typechecks and the label is covered by the UI test. ## Risks - Low risk. `errorCode` values do not change, so retry, recovery, and comparison logic is unaffected. - No schema change and no migration. `failureCause` is a new optional `resultJson` key. Rows saved by the previous release have no key and render as before; a new row read by an old server is ignored. - The SQL replacement only acts on a Drizzle `Failed query:` marker, or on `$n` placeholder text with a statement shape. Ordinary error text does not change. When it acts, it also drops the text after the statement. - Code that checks `resultJson` against a closed key list must allow `failureCause`. I found and updated the four lists on the Retry path; a list I missed would refuse a Retry for a run with a generic failure. - This is the run log / instance database path. It adds no Telemetry event. - The server failure log line still logs the raw error message (unchanged behavior); only the board-facing text is changed. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file editing) and code execution for tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
c985541a57 |
perf(db): index retry-exhaustion receipts for the attention scan (#20386)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The attention feed shows runs that failed after automatic retries stopped > - To find these runs, the feed reads every `heartbeat_run_events` row of the company and filters the rows by a message prefix > - Run output makes this table very large, so each feed request does a large scan > - On a hosted test database, this one statement used the most total database time: 923 calls at a mean of 5 s (4,709 s in total) > - Request authentication waits behind this background work, so all users feel the load > - This pull request adds a small partial index that holds only the retry-exhaustion receipts > - The benefit is that the same query reads a few index entries instead of all company events ## Linked Issues or Issue Description **What existing behavior does this improve?** The attention feed query that finds retry-exhausted runs (`listAttentionExhaustedRuns`). **Subsystem affected** Server attention feed and the `heartbeat_run_events` table schema. **Current behavior** The query reads all company rows in `heartbeat_run_events`, filters them by `event_type = 'lifecycle'` and the `Bounded retry exhausted` message prefix, and then joins the few matches to failed or timed-out runs. The cost grows with all run output. It does not grow with the number of exhausted runs. **Proposed behavior** A partial index holds only the matching receipts. The query plan reads the index and joins a few rows. The query text and the result do not change. **Reason and benefit** The scan was the most expensive statement by total time on a loaded hosted database. A local reproduction shows a 54x faster query and about 116x fewer buffer reads. **Breaking changes** None. The API, the feed rows, and their order do not change. ## What Changed - Add the partial index `heartbeat_run_events_retry_exhausted_idx` on `(company_id, run_id, id DESC)`. The index covers only rows where `event_type = 'lifecycle'` and `message like 'Bounded retry exhausted%'`. - Add migration `0454`. It uses `CREATE INDEX CONCURRENTLY IF NOT EXISTS`. The migration runner runs it as a top-level statement, the same as migrations 0432, 0436 and 0450. - Add a regression test. It pins the row order (newest run first, then the newest receipt on a tie) and the latest receipt per run. It also checks that the query plan can use the new index. If the query predicate changes so that the index no longer applies, the test fails. - Add `scripts/benchmark-exhausted-run-attention.ts`. It measures the same query with and without the index, and the time to build the index. - Document the index in `doc/DATABASE.md`. ## Verification Cause and thesis. The scan cost comes from reading all company events, not from the few exhausted runs. Thesis: an index that holds only the receipts makes the query cost depend on the number of receipts, and keeps the result identical. The benchmark below confirms the thesis. Rejected alternative without a migration. I first tried a LATERAL lookup that starts from the failed and timed-out runs. On the same fixture it was about 2x slower (median 379 ms against 194 ms). It reads the full event history of each failed run with random I/O, and that costs more than one sequential scan. Local SQL benchmark (the SQL that the service generates; same SQL with and without the index): ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-exhausted-run-attention.ts --runs=30000 --events-per-run=100 --samples=9 ``` Fixture (synthetic): one company with 30,000 runs (3,819 failed or timed out) and 3.0 M events, plus a decoy company with 0.5 M events. The table is 1.4 GB. About 5% of events are `lifecycle` events. Events of 20 concurrent runs are interleaved. 1 in 23 runs has a receipt, some runs have two receipts, and output decoys contain the prefix. Two agents are terminated. Embedded PostgreSQL 18.1 on an Apple laptop. The hosted database is Neon with different compute, so the absolute times are local SQL measurements only. The script runs two cycles (without index, then with index; then with index, then without index) to expose cache bias. Each measurement has 2 warmups and 9 samples. | Query | Without index | With index | Samples and spread | Output check | | --- | --- | --- | --- | --- | | `listAttentionExhaustedRuns` | median 157 ms (96–1,271 ms) | median 3.3 ms (2.2–5.7 ms) | 18 each | 133 rows, positional rows equal in all 4 measurements | | Buffers (EXPLAIN ANALYZE) | 127,743 blocks | 5,580 blocks | | plan uses the new index (asserted) | Speedup: 48x (median), on head `ae6037b2a0` (the same SQL and index as the final head). The machine was under heavy load from other work, so the spread without the index is wide. An earlier run on the first head gave 248 ms against 4.6 ms (54x). Index build (`CREATE INDEX CONCURRENTLY` from the migration file): 1.7 s and 2.5 s for the 1.4 GB table. Index size: 152 KiB. Review fix: the query now writes `event_type = 'lifecycle'` as a SQL literal, not a bound parameter. With prepared statements, PostgreSQL can reuse a generic plan. A generic plan cannot prove that a bound parameter matches the index predicate, so it would not use the index. Tests: - `cd server && npx vitest run src/__tests__/attention-service.test.ts`: 44 passed. The new test checks a fresh plan and a forced generic plan (`plan_cache_mode = force_generic_plan` with `PREPARE`). It failed with the bound `event_type` parameter and with a changed message prefix (confirmed red), and it passes with the fix. Locally, the file's `afterAll` database cleanup hook timed out on a heavily loaded machine (load average about 28). The timeout also occurs when the new test is excluded. - `pnpm --filter @paperclipai/db check:migrations`: passed. - `pnpm --filter @paperclipai/db typecheck` and `pnpm --filter @paperclipai/server typecheck`: passed. - `pnpm verify`: see the CI results for this head. ## Risks - Migration time. A concurrent index build reads the full `heartbeat_run_events` table and does not block writes. Locally, the build took 1.2–2.5 s for 1.4 GB. The time grows linearly with the table size. I did not time the build on a production-size clone. On a database with a 5 s full company scan, I estimate the build at tens of seconds, which is in the 1–2 minute budget. This estimate is not proven. - If the build stops, PostgreSQL can leave an invalid index. The migration runner repairs invalid concurrent indexes when it runs again, and `IF NOT EXISTS` makes a rerun safe. - The index predicate must stay textually equal to the query predicate. If someone changes the message prefix or binds a predicate value as a parameter, the planner cannot use the index. The new test catches both cases, for fresh and generic plans. - Each new receipt writes one more small index entry. Receipts are rare, so the write cost is very small. - The hosted improvement is not verified until deployment. After deployment, compare a new query-statistics window for this statement. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, local PostgreSQL). Standard context window and reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fca1f1be4f |
fix(attention): keep replaced confirmations out of Needs you after the newest is settled (#19520)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Needs you is the board inbox. It lists the decisions that wait for a person. > - Agents often ask for the same confirmation again on one task. Needs you shows only the newest pending confirmation for each task. > - When a person approved that newest card (for example in chat), or the card expired, the older card that it replaced came back into Needs you. > - Manual onboarding tests reported this: decision cards that were already approved in chat still showed in Needs you. > - This pull request makes the newest answered or expired confirmation also hide the older pending confirmations on the same task. > - The benefit is that Needs you does not show stale cards after a person answers the current one. ## Linked Issues or Issue Description **What happened?** A person approved a decision card in chat. Needs you still showed a decision card for the same task. Expired cards also caused an older card to show again. **Steps to reproduce** 1. An agent creates a confirmation on a task. Then it creates a second confirmation on the same task. 2. Needs you shows only the second card. 3. Approve the second card in chat (or let it expire). 4. The first card now shows in Needs you. **Expected behavior** Needs you shows no card for the task, because the newest ask is settled. **Root cause** `collapsePendingConfirmationsToNewest` compared only pending rows. Chat cards use the same `issue_thread_interactions` row and the same accept service, so there is no duplicate record. The server already drops answered rows. The problem was the older pending row that the collapse hid before. ## What Changed - `server/src/services/attention-source-eligibility.ts`: the collapse takes the newest settled (accepted, rejected, answered, expired) ordinary confirmation for each issue. Older pending ordinary confirmations stay hidden. A new `loadNewestSettledConfirmationAt` helper reads that value with one grouped query. - Cancelled and failed replacements do not settle the ask, so the older card returns (the existing retention test keeps this behavior). - Tool-action confirmations and quality-review confirmations are excluded from the settled check. A settled tool action never hides a different pending tool action. - `server/src/services/attention.ts` and `server/src/services/attention-retention.ts` use the new input, so the feed and the retention worker agree. - Review follow-up: the summary and task-index payload keeps `toolAction` and `secretProposal`. A settled card only hides older asks addressed to the same recipient. Tool actions, secret proposals and app connections (`connectionAuthorization`) form one governed set. They match the set the other confirmation readers use. A settled governed card never settles an ordinary ask, and a pending governed card is never hidden. - `server/src/__tests__/attention-service.test.ts`: new tests for "approved from chat", "rejected from chat", "expired", a re-ask after a settled card, another recipient, a settled secret proposal, tool actions in `list()`, and tool actions in `summary()`/`taskIndex()`. The tests use two agents, because the service already expires a same-agent re-ask at creation. ## Verification Red on `master` ( |
||
|
|
11ac90e31f |
fix(build): typecheck server/src/__tests__ and fix every type error (#18739)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server has about 1,100 test and helper files in `server/src/__tests__` > - `server/tsconfig.json` excludes that directory, Vitest does not check types, and no script runs the tsconfig in that directory > - So a test can call a function with the wrong arguments and still pass, and no tool reports it > - This gap hid 2,168 real type errors, and some of them hid tests that did not test what they claim > - This pull request fixes all of these errors and adds the directory to the server layers from paperclipai/paperclip#18737 > - The benefit is that a type error in a server test now fails the typecheck, with no baseline and no backlog ## Linked Issues or Issue Description Refs paperclipai/paperclip#18737, which is merged and added the server layers. Refs paperclipai/paperclip#18738, which is merged and checks the test projects outside `pnpm -r`, so it is not a duplicate. Refs paperclipai/paperclip#18137, which records that this directory has no check. **What happened?** No tool checked the types of 1,074 of the 1,101 files in `server/src/__tests__`. I searched `package.json`, the workflows, and all tsconfig files. 27 files are in the program of another tsconfig, and these 27 have 0 errors. 11 of the 27 are in `helpers/maintenance/tsconfig.json`. No script ran that project when I measured this. paperclipai/paperclip#18738 now runs it. **Expected behavior** The server typecheck checks these files, and a type error fails the check. **Steps to reproduce** 1. On the base commit, run `tsc -p server/src/__tests__/tsconfig.json --noEmit`. It reports 6,536 errors and needs 12.3 GiB. 2. Add a type error to any file in `server/src/__tests__`. 3. Run `pnpm --filter @paperclipai/server typecheck`. It passes. **Paperclip version or commit** I measured the error breakdown below on commit `8dfd632452`, with TypeScript 7.0.2. The branch now includes master up to paperclipai/paperclip#18865. **Deployment mode** Not applicable. This issue applies to development and CI. No production source file in the server changes. ### Error breakdown Each row adds one change to the row above. | Step | Total errors | Change | Cause | | --- | --- | --- | --- | | Config as committed | 6,536 | | | | Add `src/types/*.d.ts` | 4,447 | -2,089 | Config. The Express request types were missing. | | Set `rootDir` to the repository root | 2,436 | -2,011 | Config. | | Set `allowImportingTsExtensions` | 2,181 | -255 | Config. Vitest accepts `.ts` imports. | | Add `ES2024.Promise` to `lib` | 2,138 | -43 | Config. Node 24 has `Promise.withResolvers`. | | Set `allowJs` for `.mjs` imports | 2,148 | +10 | Config. It removes 8 `TS7016` errors and shows new real errors. | | Set `declaration` to false | 2,142 | -6 | Config. Tests emit nothing. | | Count only files in `server/src` | 2,058 | -84 | Other packages: 53 Daytona, 29 Runner test, 2 UI. | So 4,478 of the 6,536 errors (69%) came from config or scope, and 2,058 are real type errors in 314 files. The 29 Runner test errors are also real, which gives 2,087 errors. Tests that master added after this measurement had 81 more errors, so this pull request fixes 2,168 errors. This table classifies the first 2,058 errors. | Kind of real error | Errors | | --- | --- | | Wrong argument or assignment type | 1,023 | | Stale or partial fixture objects | 525 | | Property that the type does not have | 317 | | Missing narrowing of null or unknown | 101 | | Implicit any | 60 | | Missing name or module | 16 | | Other | 16 | ### What the type errors hid Most errors were stale fixtures and loose mocks. These errors hid real test problems, and this pull request fixes them: - **A test that never ran.** `instrumentation.test.ts` required `sdk-trace-base`, which is not installed. So 3 span tests were skipped. They now run and pass. - **Assertions that could not fail.** - `issue-thread-interactions-service` used a company ID as an issue ID. - `status-card-evaluation-work` compared `[]` with `[]` in a rollback check. - `slack-conversation-lifecycle` read `item.issueId`, which does not exist. - `email-channels` used a permission key that does not exist. - `heartbeat-project-env` imported `LOW_TRUST_REVIEW_PRESET`, which is not exported. The value was undefined. - **Postgres processes that did not stop.** 11 test files called `stop` on the test database helper. That method does not exist, and the helper method is `cleanup`. - **Arguments that the service ignored.** - `methodKey` is now `connectionMethodKey` in the Daytona, Unikraft, 1Password, and Railway tests. - 108 calls passed `"owner-user"` to a parameter that expects an options object. - 9 chat wizard fixtures passed `managementContext` as `source`, so they ran `existing_app` and not `manifest`. - 2 tests passed a `{ userId }` actor that the function ignores. - Inserts into `environments` passed a `companyId` column that the table does not have. - **Problems in tests that master added during this work.** - `sandbox-creation-operations` read `database.connectionString` when an external test database was set. In that mode `database` is undefined, so the fixture would stop with a TypeError. It now uses the external URL. - `idle-completed-history` set a Dot operation to `dispatched`. That is a Muse status, and the Dot type does not have it. The fixture now uses `admitted`, which is a real in-flight Dot status. ### Possible product problems, not changed These may be real problems in production code or types. I did not change them, because this pull request does not change production behavior: - `decisions-service` `service.create` does not validate the lenient `cancel_issue_tree` input. - `createConnection` ignores `input.connectionPurpose`. - The `createConnection` route schema default stores `{}` for `transportConfig`. - `prepareManagedAiRuntime` accepted the invalid value `mode: "connection"`. - `createBinding` accepts invalid `projectionClass` strings outside class 3. - `ensureManagedProjectWorkspace` forwards an untyped `repoRef` by spread. - `routineVariableSchema.label` is optional, but `RoutineVariable.label` is required. - `ToolRunContext.projectId` may be too strict. - `withdrawInteraction` reads `issue.status`, which its type omits. - `issueService.list` and `reconcileNativeFinalizations` omit fields that they return at runtime. - `createGitHubExternalObjectProvider` handles `tokenProvider: null`, but its type forbids this value. ## What Changed - Fixed the 2,168 type errors. The diff changes 322 test and helper files. - Corrected `server/src/__tests__/tsconfig.json`. Editors and the typecheck now read the same options and report the same errors. - Added three `test-tree` layers to `server/scripts/typecheck-layers.mjs`. They read the server layers as declarations, and any error fails them. A hash of the path selects the shard, so a new file does not move other files. - Added `server/scripts/typecheck-stubs/daytona-sdk.d.ts`. Server tests reach Daytona plugin source, and the root install does not have `@daytonaio/sdk`. - Changed two imports in `ui/src/lib/issue-execution-policy.ts` to use `.js` extensions, because server tests import this file. - Made `declarationsConfig` and `--declarations-only` skip the `test-tree` layers, because these layers emit nothing. Other projects read only the source layers. Added a test for this. - Updated `doc/DEVELOPING.md`. ## Verification Run these commands: ```sh pnpm --filter @paperclipai/server typecheck pnpm exec vitest run server/scripts/typecheck-layers.test.mjs ``` Results on macOS (arm64), Node 24, 16 cores. Each check ran after the merge of paperclipai/paperclip#18738 unless the row says otherwise: | Check | Commit | Result | | --- | --- | --- | | Server typecheck, `PAPERCLIP_TYPECHECK_LOW_MEMORY=1`, all 7 layers | `6e45a08573` | Exit 0, 0 errors, 62 seconds, peak 3.52 GiB | | `node scripts/typecheck-test-projects.mjs --low-memory` | `a3306bd13e` | Exit 0, 0 errors, 42 seconds, peak 3.25 GiB | | The same script with `--reuse-declarations` | `1b878aabc0` | Exit 0, 0 errors | | `typecheck-layers.test.mjs` and `live-recovery-work.test.ts` | `01db5634e4` | 90 passed. These include real compiler runs for a pass, a type error, and an invalid tsconfig. | | `tsconfig.declarations.json` after a full server typecheck | `0fb1e4fb0a` | It references only `below`, `cycle`, and `above`. No generated config names a `tsconfig.layers.json` file. | | Owned runtime test in `idle-completed-history` | `e8f2f448ac` | Passed, with the fix that master later replaced | | `idle-incoming-wake`, `plugin-idle-manager`, and the changed `chat-channels` test | `6e45a08573` | 30 passed | | Runner `native-execution.test.ts` | `eab4dcc1f4` | 114 passed | | The 19 server test files that changed since `eab4dcc1f4`, and `typecheck-layers.test.mjs` | `259737b700` | 4,275 of 4,277 passed. The 2 failures are in `chat-channels` and are in the list of 7 below. | The peak is the largest single process, from `/usr/bin/time -l`. Peaks for the server typecheck were between 3.42 GiB and 3.56 GiB across 8 runs. Peaks for the test projects were 3.25 GiB and 3.37 GiB across 2 runs. I did not measure a cgroup. **Runtime check of the fixes.** Types do not prove that a test still works. So I ran all 309 changed server test files, 11,929 tests, before the rebase onto paperclipai/paperclip#18737: | Result | Tests | Detail | | --- | --- | --- | | Passed | 11,913 | Includes the 2 tests and the 54 `dot-runner` tests that passed in a later run, as described below. | | Failed | 7 | The same 7 tests also fail on the commit before the fixes. | | Skipped | 9 | These tests have their own skip conditions: `sentry` 5, and 1 each in `bundled-plugin-registration`, `customer-success-cloud`, `onepassword-connection`, and `workspace-runtime-https-live-exercise`. | The 7 tests that fail with and without this change, on my machine: - 3 tests in `chat-channels` and `email-channels` stop with `ENOENT` on a `SKILL.md` file. The path is in the parent checkout, above the git worktree that I used, so I expect that the worktree is the cause. I did not prove this. - `workspace-instance-cleanup` compares `/private/var/...` with `/var/...`. This is a macOS symlink difference. - 2 tests in `git-reproducible-managed-checkouts` get `unknown_base_git_files` and expect `unpushed_managed_checkout`. I did not find the cause. - 1 more test in `chat-channels`, `materializes direct external-chat finals once`, gets `undefined`. I did not find the cause. CI is the check for these 7 tests. `ci / verify` passed on `eab4dcc1f4`. Failures during this work: - My fix to `agent-runtime-connector-access` changed an invalid permission key in the setup, but not in the assertion. The runtime check found this. I corrected the assertion, and the file passes. - `heartbeat-plugin-environment` failed one wait in the full run. It passed three times alone, so I think that load caused the failure. It can fail again under load. - `dot-runner` skipped all 54 tests in the full run because `cargo` was not on my `PATH`. It then passed 54 of 54 with the toolchain on the `PATH`. - My first `lib` value removed the default DOM types, and the error count went up by 375. I added the DOM libraries. - My Daytona stub with bare `any` left 48 errors, because generic helpers then inferred `unknown`. I added shapes that return promises. - One fix to `agent-secrets-routes` used `class_3_static_lease`, and the test failed. That class allows only specific paths. I reverted to `unclassified`. - `ci / General tests (server (8/12))` failed on `259737b700` because of the flaky port in `idle-completed-history`. paperclipai/paperclip#18853 fixed this test on master, so this pull request keeps that fix. **Audit of the diff.** I counted these patterns in added and removed lines, to find fixes that only hide an error: | Pattern | Added | Removed | | --- | --- | --- | | `@ts-ignore` | 0 | 0 | | `@ts-expect-error` | 0 | 1 | | `.skip` | 0 | 0 | | `as any` | 9 | 9 | | `: any` | 11 | 2 | | `as never` | 8 | 2 | | `as unknown as` | 35 | 4 | | `Promise<unknown>` | 16 | 5 | | `expect(` | 128 | 113 | No hunk removes more `expect(` calls than it adds. Three removals of test content were intentional, and I checked each one: - `muse-control` removed `signalToken`. Line 1012 already asserts that it is absent. - `merged-pr-confirmation-sweep` removed legacy aliases. Other tests cover them. - `heartbeat-workspace-branch-containment` removed a `finalize` branch that no test row can reach. Some partial fixtures stay unchecked through `as unknown as` and `Promise<unknown>`, mostly in `issue-agent-mutation-ownership-routes` and `issue-comment-reopen-routes`. **Accuracy of the check.** One program over the tree and the three shards reported the same errors at the same positions. Memory, as the peak of the largest process on one thread, was 3.93 GiB for one program and 2.43 GiB for the largest shard. I measured these two values before the fixes. Not run or not proven: - I did not run the full `pnpm test:run`, or a root `pnpm build`. I expect CI to run the full suites. - I did not run the 4 GiB container check, or the full `scripts/typecheck-low-memory.mjs`. The peaks above come from macOS, so they are not a Linux cgroup measurement. - I did not run the UI typecheck after the merges. - `workspace-runtime-https-live-exercise` is skipped, so its `cleanup` fix did not run. ## Risks - **A large diff in test files.** 328 files change. A pull request that edits the same tests can get merge conflicts. New tests on master can also add type errors before this merges. Any such error fails this check and needs a fix here. - **A higher cost for changes to source types.** A change to a type can add errors in test files that the author did not touch. The author must then fix them. This is the purpose of the check, but it is new work. - **Changed test behavior.** Some tests now run paths that they did not run before, such as the 9 chat wizard fixtures and the 3 span tests. They pass here, but they can be more sensitive to timing or to the environment. - **The Daytona stub can go stale.** The stub is written by hand. If the plugin uses a new SDK member, the typecheck reports an error until someone updates the stub. - **The release path.** `release-verify.yml` and `scripts/release.sh` run `pnpm -r typecheck`, so they now run this check. - **The tsconfig change.** The test transform reads `server/src/__tests__/tsconfig.json`. The options that I changed are for type checks, and the runs above passed. - **A gap that remains.** Tsconfigs in `packages/paperclip-runner` and in most sandbox provider plugins exclude `*.test.ts`. This check does not read most of those test files, so they can still drift. That work is separate from this pull request. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with reasoning, tool use, code execution, and subagents. The runtime does not show the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (7 tests fail locally, with and without this change. See Verification.) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1c9ca5829f |
perf(ci): overlap Runner workloads and cache debug workspace builds (#18794)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Required CI checks protect its server and Runner contracts. > - Recent full runs waited on general server files and the final Runner lane. > - That Runner lane ran package tests before native server tests. > - PR jobs also rebuilt unchanged debug workspace binaries. > - This pull request overlaps those workloads and reuses exact trusted build inputs. > - The benefit is a shorter critical path with the same test coverage and AWS authorization. ## Linked Issues or Issue Description Refs #18544, #18773, #18774, #18798, #18799. **What existing behavior does this improve?** The wall time of required PR tests and the precision of affected-test selection. **Current behavior** General server and native files use two workers. The second Runner Vitest lane runs its package shard and then the native server group. Each Runner job compiles debug workspace units again. A Vite raw import also retains its consumer for unrelated source edits. **Proposed behavior** Use three isolated workers for general server files and four for native files. Reuse the two Runner slots for a complete package suite and a complete native server suite. Restore all debug binaries only on an exact input key. Record raw imports as explicit file inputs. ## What Changed - Separate the Runner package and native server workloads without adding CI jobs. Preserve the old command arguments for master-pinned workflows, local commands, and other shard counts. - Increase file workers while retaining fork isolation, serial cases, and the shared-database guard. - Add an exact debug workspace cache. Derive binary targets from Cargo. Include source, fixtures, generated inputs, manifests, compiler identity, flags, Cargo configuration, and checkout path in its key. Discard incomplete restores. Only verified master pushes can save it. - Keep older PR checkouts working when they lack the new cache helper. - Track Vite `?raw` imports as repository file inputs. Keep unknown loaders, untracked files, and opaque effects conservative. - Add cache and coverage guards and document the behavior. AWS runner authorization is unchanged. - Keep paired test benchmarking available when install hooks correctly disable materialized dependency reuse. The old benchmark asserted a cache hit in that safe fallback. - Wait for active dispatch work to finish before asserting idle reads. Analyze the two 16,400-row database fixtures before their queries so they have current planner statistics. Suppress only PostgreSQL notices inside the cleanup transaction to avoid logging every cascaded table; warnings/errors remain visible. Row counts, real queries, assertions, database reset, and timeouts remain unchanged. ## Verification - 196 affected CI tests passed, including routing, coverage, selection failure fallbacks, shared-database serialization, and existing/new cache guards. Final worker/cache adjustments also passed their focused checks. - `pnpm build` and `pnpm -r typecheck` passed locally. Full test coverage is being verified by the Linux CI profile; `pnpm test:run` was not repeated as a full serial local run. The paired CI benchmark passed. Both paired benchmarks passed, and full CI passed again after integrating newer master commits. The external Paperclip Review app is the only pending status. - Actual debug artifacts remained fresh after simulating checkout timestamps: Cargo completed in 0.82 seconds; the content key was unchanged. This is local reuse evidence, not a measured GitHub cache transfer. - Same real UI inventory for a retention-test-only change: 861 files; retained selection fell from 413 to 401. All 12 omitted files are raw-input consumers; edits to their inputs remain covered by regression tests. - The first local worker comparison is invalid. Development dependency links disappeared during its second run. The two-worker baseline also had two macOS path assertions in company-skills. Neither result is counted as a successful speedup. Dependencies were restored. A local repeat was interrupted before producing results. The completed Linux comparison below supplies the valid paired evidence. - [Paired Linux benchmark](https://github.com/paperclipai/paperclip/actions/runs/41671244016): baseline and candidate ran the same 113 general-server files and 1,777 cases on one host. Wall time fell from 381.29s to 346.89s (9.0%). Both exited successfully and file/case inventories matched exactly. The fixture command took 31.77s versus 28.65s. This is one sequential pair, excluding setup, queueing and the full CI critical path. - [Repeat paired Linux benchmark](https://github.com/paperclipai/paperclip/actions/runs/41675496862): the same 113 files and 1,974 cases took 485.06s in the baseline versus 368.54s in the candidate (24.0%). File and case inventories again matched exactly. The baseline includes newer master changes than the first pair, so these are two independent comparisons, not repeated timings of an identical tree. - Initial full AWS profile: package suite passed 4,891 tests (289 files, one skipped); native suite passed 172 tests (9 files, 12 pre-existing platform/feature skips). Native execution took 239.06 seconds; the older sample ran 165 native tests in 391.29 seconds. The package lane now takes 11m01 including setup and debug compilation. This is an indicative cross-run comparison, not a paired full-CI speedup claim. - The initial run failed two UI fixtures, one serialized inbox settlement assertion, and one dispatch idle assertion. The two UI failures reproduced locally. The fixes preserve authorization, admission, and zero-idle-read assertions. All four affected files passed locally: 48/48 tests. Token gates passed. All four also passed in the next AWS profile. That profile passed every other workload but timed out the 16,400-row keyed attention fixture at its existing 180s limit; this is accounted for by the explicit fixture-statistics change. Local focused comparisons pass both large cases before and after; their roughly 33s test execution is unchanged locally, so this is not claimed as an additional speed gain. A full local file run passed both large cases (7.57s / 18.05s) but hit the unchanged 30s teardown deadline on its first test while emitting thousands of cascade notices. Cleanup now suppresses those notices within its transaction; the complete file then passed 34/34 in 107.72s. The latest-head AWS run passed the previously failing shard: the keyed bulk case took 40.25s (previously exceeded 180s), the keyless case took 8.02s, and the complete attention file took 89.61s. The 16,400-row fixtures and original timeouts remain intact. - [Full CI profile before master integration](https://github.com/paperclipai/paperclip/actions/runs/41675497699) passed on commit `5858956d4ac2c99c911ad2447b63e1f8fc84e31e`, including all tests, build, typecheck and final verification. Greptile reviewed that exact commit at 5/5; the only actionable review thread is resolved. The separate Paperclip Review app is still queued. - [Final merged-head CI](https://github.com/paperclipai/paperclip/actions/runs/41678956312) passed on `499aa206a6e03bf8f3249692ea731367a000591d`: 48 successful jobs and one intentionally skipped file-selection job. Backup checkpoint qualification and legacy security status passed. Greptile reviewed this exact commit at 5/5. The PR is conflict-free; only the separate Paperclip Review app remains queued. - Integrated current master after #18798 and #18799 merged the inbox and UI fixture fixes separately. Preserved master's inbox settlement fix and verified its entire file (6/6 tests). Those three files are no longer part of this PR's diff. Removed the temporary `profile-ci` label after both successful benchmark pairs; the normal planner still schedules every job because the CI configuration changes are global inputs. - Second AWS profile: native tests took 229.97s with the same 172 passes / 12 skips; package tests took 427.44s with 4,891 passes / 10 skips. The complete package job took 10m44, including setup and 79s debug compilation. - The PR uses the trusted workflow from master. Candidate worker/layout code runs now; workflow-level debug restores begin after merge and the first successful master cache writer. ## Risks - Extra workers can expose load-sensitive tests. The final full CI profile passed, but one run is not a long-term flake-rate measurement. - The cache is an optimization only. Misses rebuild, and test commands still execute. A new Cargo workspace member requires a cache-contract review. - Compiler alternatives remain experiments. Required TypeScript checks are unchanged. Typechecker experiment: three sequential checks per tool on identical built projects, with TypeScript 7.0.2, Bun 1.4.4, and tsrs 0.20.1-ts7.1.0-beta. Median seconds (TypeScript / Bun / tsrs): shared 0.429 / 0.083 / 0.157; UI 2.554 / 0.390 / 0.740; server 9.949 / 1.194 / 1.782. TypeScript and Bun passed all three projects. All tools caught the same four deliberate assignment, property, nullability, and generic-constraint errors. tsrs reported one additional server TS2345 diagnostic in the run-dispatch adapter, so it did not pass the project comparison. These local compiler times exclude generation and build steps; they are not full CI savings. ## Model Used OpenAI GPT-6 Codex. Tool use and code execution assisted this change. The exact deployment identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
a92f616cd0 |
fix(db): attribute bounded captures to execution owners (#18634)
## Thinking Path > - Paperclip manages agent work through shared server and database services. > - High query volume can spend main-thread time on SQL construction and result decoding. > - The existing bounded response capture counts rows and bytes, but it cannot name the operation that executed a query. > - Bundled callers can be missing, and measuring every parsed JSON value adds work to the diagnostic. > - This pull request carries a semantic execution owner into each measured query and bounds response sizing. > - Operators can rank query volume by owner before choosing a performance fix. ## Linked Issues or Issue Description Refs #17504, #17522, #17537, #17564. **What existing behavior does this improve?** The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic. **Subsystem affected** Database diagnostics and existing server execution boundaries. **Current behavior** The capture groups by table, operation, and builder caller. It cannot distinguish execution owners that reuse a prepared query. Bundled deployments need source maps for caller locations. The capture visits every returned value and serializes parsed JSON again to estimate its size. **Proposed behavior** Version 2 captures use fixed semantic owners for HTTP requests, delivery queues, execution phases, and budget work. They count completed calls, returned rows, errors, and completion wall time. They sample successful results 1, 17, 33, and so on, with fixed cell and string/byte limits. Complex values remain explicitly unmeasured. Semantic ownership works without source maps. **Reason and benefit** A short census can identify which operations issue the most queries or return the most rows. It preserves each query's execution owner through pooled connections, concurrent operations, and prepared-query reuse. It reduces diagnostic work without claiming per-query CPU attribution. **Breaking changes** Opt-in capture output moves from `entry.v1`/`summary.v1` to `entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace all-result byte estimates. Consumers must use the new fields and must not extrapolate them into a total. There is no database migration or application API change. ## What Changed - Added `withDatabaseOperation(owner, action)` to the DB package. The server owns semantic labels. Queries retain execution ownership in their completion closure. - Added fixed owners at existing HTTP, delivery, budget, and execution timing boundaries. Delivery work replaces the context of the producer that woke it. - Added complete call/row/error counters and completion wall durations within the existing bounded window. Durations include waits and are not CPU measurements. - Sampled and bounded primitive-value sizing. Complex and oversized values remain unmeasured. Known owners skip repeated stack extraction. - Preserved the deadline, group limit, paced output, query outcomes, and existing coverage exclusions. Updated the operator documentation. - Added real PostgreSQL owner-isolation and prepared-query tests, server scope tests, and plain bundled-runtime proof. - Moved projection-test and benchmark full-response sizing into an explicit test helper. Existing payload-size thresholds remain unchanged; these synthetic checks do not use the production sampler. ## Verification - Final-commit DB capture suites: 43 tests passed in 13.86 seconds against the actual PostgreSQL driver, including concurrent owners, prepared reuse, transaction failure, a dedicated client, sampling, and a query crossing the deadline. DB typecheck passed. - Server attribution and existing delivery/execution/budget suites: 53 tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5 tests passed. - Runtime bundle suite: 8 existing tests passed. The new no-source-map owner test passed after its fixture was corrected to use a driver thenable. An ordinary native Promise can bypass an overridden `.then`; the real PostgreSQL tests separately cover driver integration. - Observer-only experiment: three alternating pairs per case on Node 24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an already-materialized synthetic JSON result, median capture CPU fell from 82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell from 8.10 to 4.40 microseconds/call. Every result retained identity and all calls were counted. The new diagnostic intentionally measures less byte data; these are observer costs, not database latency or a hosted application speedup. Disabled-case timings were below one microsecond/call and too small for a performance conclusion. - Final lifecycle regression: 28 unit tests and DB typecheck passed. Closing the last capture disables and replaces owner storage; overlapping captures retain it. - Final-commit full recursive typecheck passed in 38.38 seconds and full build passed in 40.81 seconds. The build stamp matches `83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution tests also passed 5/5 before the test-helper-only migration. Full test acceptance will use the repository's parallel CI gates; the unsharded local command takes hours. - Verification failures retained: initial sandbox runs could not open local IPC/PostgreSQL listeners; permitted reruns exercised the real services. One old test expected JSON serialization and was updated to assert explicit unmeasured complex values. The first bundle fixture used the wrong promise boundary, as described above. A retained-callback regression also demonstrated that disabling and reusing the same AsyncLocalStorage instance could resurrect an old owner; replacing the instance after the last capture closes fixes that case. - The first CI run found older projection tests consuming the previous full-response fields. Those consumers now use the full-response test helper, preserving their byte thresholds. The two complete affected route suites plus helper tests passed 115/115 in 73.08 seconds. A temporary full-row query negative control then failed the unchanged 2,048-byte limit at 5,000,812 bytes and was restored; five other service suites passed 34/34 and the affected chat cases passed 3/3. All three previously failing CI shards passed on the corrected head. A local skill-list benchmark smoke check completed 255 captured queries with no in-flight queries or overflow; this was integration verification under concurrent test load, not a timing benchmark. - Two final-head runner jobs lost communication with GitHub before reporting a test result. One targeted retry of those jobs and their dependent aggregate passed. All 53 final-head checks passed, with 3 intentional skips; previously passing suites were preserved. - Hosted capture remains pending until the reviewed revision is deployed and a bounded window is explicitly configured. Reproduce the focused checks with Node 24 and the repository pnpm version: ```sh pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs pnpm -r typecheck pnpm build ``` ## Risks - Owner context is active while a valid future capture window is armed, up to 30 minutes before accounting starts. No-window and expired calls take the direct path. Diagnostics still add bounded work while enabled. - A nearest nested owner is reported. Broad `http.request` and unattributed groups remain visible; this does not identify every service callsite or SQL shape. - The existing census excludes tagged/raw explicit-execute/streaming paths and internal protocol commands. Its counts are not PostgreSQL transaction totals. - Completion wall time includes queue/database/network/scheduling waits. Concurrent durations overlap. Sampled bytes exclude complex values, large strings, and values beyond the sizing limits. - Activation is startup-only. Operators must choose the window after artifact preparation, allow rollout/warm-up time, and remove the expired setting afterward. ## Model Used OpenAI Codex, GPT-6 family, with code execution, repository inspection, and parallel agents. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
dce383c733 |
perf(server): stop recurring attention retention scans (#18388)
Reviewed and verified at
|
||
|
|
aa58fa1073 |
perf(ui): restore fast Coordinator task and inbox loading (#18323)
## Thinking Path > - Paperclip lets people manage work done by agents. > - Tasks, Inbox, and Needs You must stay usable as work history grows. > - The Coordinator views bypassed some existing collection optimizations. > - Navigation fetched complete decision cards for a count, and Tasks mounted every loaded row. > - This PR uses the canonical attention rules for small count and task-ID responses. It also restores bounded rendering and the optimized Inbox selection. > - The benefit is less data and less initial rendering work without a second definition of pending work. ## Linked Issues or Issue Description **What happened?** Coordinator navigation fetched the full attention feed on every cold collection visit. Tasks rendered the entire loaded collection. Coordinator could also select the old Inbox loader when Streamlined UI was off. **Expected behavior** Fetch only the projection needed by each surface. Start with a bounded set of rows. Keep complete task membership, accurate counts, current permissions, and stable loaded rows. **Steps to reproduce** 1. Enable Coordinator and open Tasks or Inbox with an empty browser query cache. 2. Observe a complete attention-card download for the navigation badge. 3. Open a large task list and observe that every loaded row mounts at once. 4. Disable Streamlined UI while keeping Coordinator enabled, then open Mine and observe three overlapping compact task reads. Related work: #17756, #3744, #17983, and #18312. ## What Changed - Add board-only attention count and complete task-ID projections. Reuse canonical company, privacy, addressee, deduplication, dismissal, snooze, and archive rules. Skip card bodies, plan/image reads, and presentation roster lookup. - Share the small count read across desktop and mobile navigation. Reuse the page-owned response on Needs You and Needs attention. Keep the newest known count through navigation, including zero. - Use the task-ID index for Needs attention and keep task row hydration in batches of at most 100. - Render 100 task rows initially, across the whole expanded tree. Reveal 150 more at a time through scrolling or a button. Keep existing rows through refresh and append. Defer offscreen leaf layout with shared tokens. - Keep Coordinator on the optimized Inbox when Streamlined UI is off. Use index reads for expanded project recents. - Add real-database, component, and production-build browser regression checks. Update the access and feature maps. ## Verification - All required CI checks pass on `1c68ac8957490a6c8585ba0939355f536c8120bf`: typecheck, build, server/workspace tests, serialized database suites, and eight E2E shards. Greptile is **5/5**, with its row-retention finding fixed and the review thread resolved. Storybook checks were intentionally skipped; Paperclip Review completed neutral. - Workspace typecheck and production build passed locally. The final rebased source also passed UI typecheck and production build. Token gates and map checks passed. - The attention service suite passed all 34 tests on disposable PostgreSQL. Final targeted checks cover compact projections, complete-set counts, quality approvals, snoozes, private tasks, and board/company access. Database response capture confirms compact projections do not read plans or images. All 31 OpenAPI tests passed after registering both endpoint contracts. - Six production-build browser journeys passed: desktop, mobile, Coordinator Inbox with Streamlined UI off, Needs You, Needs attention, and an older server page followed by task open/Back. They verify the initial 100-row budget, scrolling, retained DOM identity/height, one owner per attention read, complete task membership, 220 retained rows on Back, and less than 2 px of return-position difference. - The row-budget and Inbox regressions fail against original source and pass with this change. The final three affected UI suites passed 40 tests, and the broader Inbox/IssuesList suites passed 101 tests serially. ### Controlled production-build comparison Baseline `b31ec0a2dc079e32272a8f618425fe92b68bbaec` versus PR head `1c68ac8957490a6c8585ba0939355f536c8120bf`. Isolated baseline asset `index-CS9uPxMx.js`; candidate asset `index-BxJeTQ30.js`. Both UI builds used the same warm disposable API server and database, with 220 tasks and 110 pending board approvals. Chromium **151.0.7922.34**, 1440 × 900 viewport, **4× CPU throttle**, loopback with no added network latency, alternating baseline/candidate order. Cold cases use a new browser context with network cache disabled; warm cases navigate within the same context. Other host workloads are uncontrolled. Readiness requires the destination collection's first visible task row, then two animation frames. Inbox readiness waits for its canonical Mine route so previous-page rows cannot satisfy the measurement. Five paired samples per case; cold Inbox has a second five-pair confirmation wave because the first result was slower. Times below are **milliseconds**; parentheses show min–max. Paired savings are the median of individual before-minus-after pairs, not the difference between medians. | Journey | Pairs | Baseline median (range) | Candidate median (range) | Median paired saving | Faster pairs | | --- | ---: | ---: | ---: | ---: | ---: | | Cold Tasks | 5 | 2752 (2542–3025) | 2300 (2275–2524) | 328 | 5/5 | | Cold Inbox | 10 | 2536 (2283–2927) | 2582 (2117–2856) | 37 | 5/10 | | Warm Tasks navigation | 5 | 2054 (1826–2479) | 1448 (1419–1548) | 635 | 5/5 | | Warm Inbox navigation | 5 | 2020 (1485–2254) | 1568 (1289–1848) | 350 | 5/5 | | Task → Back to Tasks | 5 | 1747 (1609–1948) | 1327 (1238–1355) | 472 | 5/5 | Cold Tasks, warm Tasks, warm Inbox, and Back improved in every measured pair. **Cold Inbox latency is inconclusive:** the two waves pointed in opposite directions, with the candidate faster in 5/10 pairs. This PR claims less badge work there, not a proven cold Inbox latency improvement. The canonical badge response shrank from **200,039 to 359 decoded bytes** (encoded HTTP transfer: **8,125 to 598 bytes**). The compact response also skips card payloads, plans, images, and presentation-only roster work. These are measured mechanism reductions, separate from end-to-end latency. Needs You still owns its full card feed; Needs attention owns its complete task-ID projection. Reproduction setup and assertions are in `tests/perf/collections/README.md`. <details> <summary>Individual paired readiness measurements (ms)</summary> | Journey | Pair | Baseline | Candidate | | --- | ---: | ---: | ---: | | Cold Tasks | 1 | 3025 | 2283 | | Cold Tasks | 2 | 2542 | 2275 | | Cold Tasks | 3 | 2752 | 2424 | | Cold Tasks | 4 | 2783 | 2300 | | Cold Tasks | 5 | 2641 | 2524 | | Cold Inbox | 1 | 2927 | 2702 | | Cold Inbox | 2 | 2422 | 2629 | | Cold Inbox | 3 | 2409 | 2606 | | Cold Inbox | 4 | 2621 | 2769 | | Cold Inbox | 5 | 2539 | 2856 | | Cold Inbox | 6 | 2283 | 2117 | | Cold Inbox | 7 | 2466 | 2558 | | Cold Inbox | 8 | 2674 | 2286 | | Cold Inbox | 9 | 2696 | 2389 | | Cold Inbox | 10 | 2533 | 2183 | | Warm Tasks navigation | 1 | 2479 | 1423 | | Warm Tasks navigation | 2 | 2228 | 1548 | | Warm Tasks navigation | 3 | 2054 | 1419 | | Warm Tasks navigation | 4 | 1826 | 1448 | | Warm Tasks navigation | 5 | 1954 | 1518 | | Warm Inbox navigation | 1 | 2020 | 1661 | | Warm Inbox navigation | 2 | 2198 | 1848 | | Warm Inbox navigation | 3 | 1733 | 1497 | | Warm Inbox navigation | 4 | 2254 | 1568 | | Warm Inbox navigation | 5 | 1485 | 1289 | | Task → Back to Tasks | 1 | 1721 | 1330 | | Task → Back to Tasks | 2 | 1747 | 1275 | | Task → Back to Tasks | 3 | 1948 | 1355 | | Task → Back to Tasks | 4 | 1609 | 1327 | | Task → Back to Tasks | 5 | 1790 | 1238 | </details> ### Failed attempts and limits - The local full repository command hit embedded PostgreSQL startup failures and 16 failures in the untouched native session executor suite; it was stopped. The full local suite is not claimed green. CI on the final revision provides the broad test result. - A concurrent local UI run hit timing failures under host contention. Inbox/IssuesList subsequently passed all 101 tests serially. No tests were weakened to address these timeouts. - CI caught two integration gaps introduced during this work: missing OpenAPI entries for the new endpoints, and an attention fixture incompatible with the routine filters added on master. Both were fixed; the affected 31 OpenAPI and 40 UI tests pass. - The final revision's first CI attempt failed two untouched tests: scheduled-retry batch cancellation and the tool-access audience/archive race. Both failed jobs passed when rerun without changing the revision. Their original failed attempts remain visible in [the workflow history](https://github.com/paperclipai/paperclip/actions/runs/41083379875). - Browser evidence uses synthetic disposable data. This PR is not deployed staging or production verification. ## Risks - New attention sources must preserve canonical eligibility. Both projections call the existing collector, so they share its permissions and complete-set behavior. - The count and task-ID caches must not stand in for full decision cards. Each projection has its own key under the existing attention invalidation prefix. - A row budget can omit descendants if applied per parent. The tests cover one global budget across a large expanded tree, refresh, append, reset, and incremental loading. - Local browser evidence uses synthetic data and is not staging deployment proof. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact backend model deployment ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected suites; broad-run limitations are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
4c56702e06 |
fix(server): schedule retained native-session cleanup (#17717)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs can retain a local session after an interrupted durable suspend. > - The orphan reaper repeatedly scans completed history for that cleanup duty. > - Idle scans keep the database active. Removing them requires complete wake and ownership paths. > - This pull request moves retained-session cleanup to committed signals, local release events and saved deadlines. > - Existing physical proof checks still decide whether cleanup is safe. The shared scheduler does not grant execution authority. > - The benefit is quiet retained history with recoverable cleanup and owned shutdown. ## Linked Issues or Issue Description Refs #17709 and #18135. This is slice 51 of the existing recurring-server sleep stack. Both referenced dependencies have merged. The target is `master`. **What happened?** The periodic orphan pass scans retained native sessions even after their cleanup is complete or no automatic action is safe. Some eligible sessions also depend on a local session, environment lease or original maintenance callback releasing ownership. An earlier decision notification can be consumed before the run's later terminal projection commits. **Expected behavior** Reconstruct existing work at startup. Wake from committed writes and actual owner release. Register the saved claim expiry or a probe for a specific live process. Stop recurring discovery when no obligation remains. Preserve original callback ownership and all existing cleanup proof checks. **Steps to reproduce** 1. Retain completed native runs with a durable-suspend close failure. 2. Let the periodic orphan pass run with no new work. 3. Observe repeated cleanup discovery. Separately, delay terminal projection after its decision notification is consumed and observe the missing cleanup wake. **Paperclip version or commit** Parent #17709 at reviewed head `3b75f4bc1da1370304f069fc287af3d693575716`, landed as `df6156c2e37f542a69f393dde31e357b82b7cac1`. Current integrated master: `31b927e3013a8a4b3266854b740cf40a24753788`. ## What Changed - Move scoped, paged discovery to the retained cleanup owner. Preserve explicit manual recovery. - Register the duty with the shared scheduler and remove it from the periodic orphan pass. Keep concrete lease expiry and live-process probe obligations. - Use typed outcomes for settled/refused work and process, session, environment, maintenance or lease waits. Keep physical proof and quarantine decisions in the executor. - Wake from committed finalization and terminal projection writes, durable process/lease release, and local session or callback settlement. Routine company-only finalization hints leave dormant cleanup quiet. - Join duplicate cleanup and serialize inspections on one database lane. Keep original Runner callbacks owned until settlement and shutdown. - Preserve all master profile fixtures and the branch's historical identities. Save the colliding sleep profile 79 separately. Generate combined Copilot profile 80 from its source and retain master's other provider profiles. - Add the owner audit and regression coverage. No new table, migration or independent maintenance timer is added. ## Verification - Server acceptance: **2,342 distinct passing cases in 18 complete files**. The 14-file unaffected group contributes 1,532 passes. Full startup contributes 76, process recovery 410, environment service 51, and external-chat response 273. The 29-case cleanup worker suite and 977-case executor suite run in full. No case in these files is skipped. - Controlled scope regression: removing only the ordinary finalization issue/run guard wakes dormant retained cleanup; the same assertion passes after restoring it. A real local owner release still resumes the durable candidate. - Preserve the first broad failure: 1,607 passed and one startup shutdown case failed because the coordinator mock lacked `stop()`. Add only that mock method. All 76 unchanged startup cases then pass. Do not treat the first broad command as green. - Dedicated PostgreSQL 17 databases use the existing migration 413 schema. Embedded-only fixtures use the existing recovery helper with fresh per-call databases. All temporary imports are restored byte-exact. No assertion or timeout is relaxed. - Current-master integration preserves both complete source histories. The server, UI and database trees are identical to the server-tested source. All master historical profile fixtures remain byte-exact. The prepared sleep profile 79 remains independently inspectable. - Current runtime validation at `d08f764e3efe75e96929173afe7ee8739121ad9f` passes all 12 commands (22 installed ACPX/Grok contract cases and 26 native-provider Rust cases): `pnpm install --frozen-lockfile --ignore-scripts`; Copilot/Grok identity and generated profile checks; Runner and server typechecking; server build; Copilot/Pi installation, profile compatibility and transport settlement files (130 passed; one opt-in closed Pi snapshot case skipped because its distribution root is not configured); installed ACPX model/reconnect package contract and Grok identity tests; Rust native-provider backend tests; module boundaries and migration order. - Earlier 94-case installation/settlement acceptance applies to the prepared profile 79 source. It is retained as history and does not substitute for the current profile 80 validation above. - Failure history and duty-specific evidence remain in `doc/plans/2026-10-26-retained-native-session-cleanup-sleep-audit.md`. Exact-current-head acceptance: all 60 checks are clean, including protected CI and security; Greptile gives 5/5 for `281d579de0d08e907194b8b790a0597b07d16b7b` with no actionable findings. No Ponytail review is present under the authorized missing-review policy. - These checks do not claim a full local repository suite, live Copilot or Daytona qualification, deployed host/database suspension, or measured cost savings. ## Risks - Missing a supported producer could strand cleanup. Tests exercise real finalizer, heartbeat, environment and process writers. Process-local signals cover in-process writers; no multi-replica or external SQL notification transport is added. - A timeout can leave an original callback active. Cleanup and shutdown must keep waiting for it. The existing global maintenance drain is conservative across roots. - A retained live process still has a scoped 30-second absence probe. A saved lease keeps its actual deadline. Operational errors retain retry work. Immutable proof refusals and operator-required histories do not gain automatic retry authority. - Cleanup keeps all existing artifact, identity, claim, no-launch and quarantine checks. Terminal status alone does not authorize physical cleanup. - This PR does not expand Cloud alarm coverage or relax host sleep eligibility. It proves worker behavior, not deployed app/database suspension. - The Copilot source identity has changed; these checks do not claim new live provider qualification. - The parent has merged. This PR is merged only after its current-head protected CI and review gates pass under the existing campaign authorization. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
71f0c0c99e |
feat: add opt-in Coordinator workspace for chat, tasks, and decisions (#17983)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/147/changes/paperclip/147 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People coordinate work through conversations, tasks, and requests for input. > - These surfaces must share task state and preserve the context of each decision. > - The existing workspace makes people switch surfaces to follow that context. > - This pull request adds an optional Coordinator workspace with linked conversations, tasks, documents, and decisions. > - Each person can enable it in Settings > Experimental and return to the classic workspace. > - The benefit is a coherent path from a request to an assigned task and reviewed work. ## Linked Issues or Issue Description **Subsystem affected** UI, server, shared contracts, and database. **Problem or motivation** People need a consistent way to start tasks from agent conversations, answer questions, review work, and return to the source context. Decisions and task state must agree across these surfaces. **Proposed solution** Add a personal, default-off workspace experiment. Keep one conversation per agent. Add task proposals, compact task and decision cards, contextual document panels, and a Needs You queue with search, filters, sorting, grouping, and history. **Alternatives considered** Replacing the classic workspace for everyone would remove the opt-out path. Keeping separate decision state in each surface would create conflicting answers. This change uses the existing canonical task and interaction records instead. **Roadmap alignment** This supports the existing agent chat, task management, and human review capabilities. It is an experimental presentation choice. The user requested this isolated implementation after design review. No Discord discussion is claimed. ## What Changed - Add per-person workspace preferences, a default-off setting, and an additive migration. - Add Coordinator navigation, pinned destinations, task lists, agent switching, and contextual panels. - Link task suggestions and creation to the source conversation. Default new task ownership to the current agent unless the user specifies another owner. - Share pending questions, decision receipts, exact review targets, and history across tasks, chat, and Needs You. - Preserve task access, assignment authorization, revision binding, and required output review gates. - Add compact document links, relative timestamps, composer controls, and reduced-motion support for task and decision transitions. - Add component stories, regression coverage, and capability-map documentation. ## Verification - Final head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, based on master `8bc051245f77a47bee0f8ba45896b8bfa5922e0a`. Verified October 29, 2026 at 00:10 UTC: **MERGEABLE, 56 checks passed, four conditional skips, no pending or failed checks**. [Current CI run](https://github.com/paperclipai/paperclip/actions/runs/40790639688) passed without a rerun on this head. - Fresh Greptile **5/5**, zero new comments and zero open review threads; Superagent passed with no suspicious changes. [Checks](https://github.com/paperclipai/paperclip/pull/17983/checks). - Local workspace typecheck/build passed after the latest rebase. The final account-read patch passed UI typecheck/build and all 12 related preference/settings tests, including a delayed real-client request across account changes. Earlier scoped checks cover review deduplication/replay, decision deep links, cross-surface state, access boundaries, and migration replay. - Local database/browser rechecks after the latest rebase were limited by exhausted macOS shared-memory IDs: `initdb` failed before database creation. The local run passed 31 UI/snapshot tests and skipped 24 database tests; those skips and the blocked local browser start are not claimed as passes. The independent Linux CI database and browser checks passed. No running dev database or host setting was changed. - Upstream migration 0403 is preserved. Preferences and durable review retry aliases are generated as replay-safe migrations `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`; five newest snapshots are retained. Every detected conflict was rebased and pushed before waiting for CI/review, as requested. - The PR is unmerged. The active master ruleset still requires code-owner approval for the changed Paperclip skill; no approval or rule change was performed. <details> <summary>Earlier verification failures, integration history, and evidence</summary> These entries preserve prior checkpoints and failures; the current acceptance result is above. - Focused UI and server checks from the isolated build are recorded in the implementation plan. - Browser checks covered the opt-in shell, navigation, task/document links, and decision card states. No new live provider trial is claimed for this handoff. - Current acceptance head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, rebased onto master `8bc051245` immediately after its GitHub review-repair migration took 0403. Preferences are generated as idempotent `0404_tense_silver_sable.sql`, preserving upstream SQL/journal/snapshot history and five newest snapshots. GitHub reports MERGEABLE after the immediate lease-protected push. Scott's updated instruction is to rebase and push every conflict resolution first, without waiting for CI or Greptile; verification follows the push. - The final reviewed feature tree was consolidated for repeatable single-commit rebases. The complete previous commit history remains reachable through local recovery ref `codex/coordinator-recovery-before-rebase-20261028` (resolved snapshot `0e1bbdf4e`). The actual rebase completed, and its tree hash exactly matches that resolved snapshot. The force-with-lease required the known remote head `0e6ef7c44`, protecting against concurrent branch changes. No PR merge or permission/runner setting change occurred. Fresh CI/review are in progress for the current head. - `2001132fa` received Greptile 5/5 with one nonblocking account-switch read finding. Current head detaches path-coalesced preference GETs and disables HTTP caching, retaining React Query's account-scoped sharing. A delayed real-client GET regression proves account B can load its saved choice before account A responds, and A's late response only updates A's cache. Preference/settings tests 12/12 and UI typecheck passed; its review thread is resolved. Fresh current-head CI/reviews are pending. The prior CI chat shard 3 ended with “The operation was canceled” and no assertion failure ([job 123171595631](https://github.com/paperclipai/paperclip/actions/runs/40789688333/job/123171595631), `/tmp/pr17983-2001132-chat3.log`); no code/timeout change is attributed to that cancellation. - `843e823b4` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass. All browser tests including the repaired YAML shard passed, and 54 checks passed. General server shard 9 ended with “The operation was canceled” and no failed assertion; its test step was marked cancelled (job [123164350893](https://github.com/paperclipai/paperclip/actions/runs/40787501761/job/123164350893), log `/tmp/pr17983-843e-server9.log`). A single unchanged-head rerun was started after the workflow completed, but master moved while it ran. A local nested-clone check timed out in its 30-second database-startup hook and then failed cleanup before the 20 database cases ran; this is retained as a local verification limit (`/tmp/pr17983-843e-nested-clone-check.log`), not claimed as a pass or proof of the CI cancellation cause. - Following Scott's instruction, the new conflict was rebased and pushed immediately without waiting for that rerun. Upstream's `0403_natural_cerise.sql` is preserved; regenerated preferences and retry-alias migrations are `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`, replay-safe with five newest snapshots retained. The YAML helper conflict combines upstream's click and header behavior with the editor-selection guard and injected regression. Recovery ref `codex/coordinator-recovery-843e823b4` preserves pre-rebase work. Current head is MERGEABLE. Full workspace typecheck/build passed; UI and snapshot checks passed 31 tests, but 24 database tests were skipped by the local support probe, and the local browser server could not initialize PostgreSQL. A direct disposable `initdb` diagnostic established the host limitation: `could not create shared memory segment: No space left on device` (`shmget`, shared-memory IDs exhausted, not disk space). No dev database or host setting was changed. Diagnostics: `/tmp/pr17983-2001132-initdb-probe.log`, `/tmp/pr17983-2001132-db-probe.log`; checks: `/tmp/pr17983-2001132-tests.log`, `/tmp/pr17983-2001132-browser.log`, `/tmp/pr17983-2001132-types.log`, `/tmp/pr17983-2001132-build.log`. These local skipped/blocked checks are not claimed as passes; fresh CI on independent runners is in progress. - `d18f683cd` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass, but [E2E shard 7](https://github.com/paperclipai/paperclip/actions/runs/40785966243/job/123159719925) failed parsing an Advanced YAML clipboard read beginning with “Skip to Main Content.” The log is retained at `/tmp/pr17983-d18f-e2e7.log`. Current head hardens the test helper to verify editor focus and selection, clear stale clipboard content, and require the YAML header at the start of a fresh copy before parsing. Saved-state assertions and timeouts are unchanged. All six Advanced YAML browser journeys passed on a disposable local instance (41.0s); the exact lost-response journey also passed with a one-time injected surrounding-page copy, verifying rejection/reacquisition (17.6s). The precise browser focus/clipboard timing in CI remains unproven; the observed invalid-read path is now guarded and deterministically exercised. Logs: `/tmp/pr17983-yaml-clipboard-scope-browser.log`, `/tmp/pr17983-yaml-clipboard-injected-browser.log`. Fresh current-head CI and reviews are pending. - `19e6dcb0b` passed Superagent and Greptile 5/5, with one nonblocking deep-link finding. Current head preserves attention links using `?decisionId=`: the requested pending decision is rendered even beyond the first 40 cards, focused and scrolled into view, then only that parameter is consumed so later refreshes do not refocus. Other URL parameters remain. Needs You tests 10/10 and UI typecheck passed; its review thread is resolved. The preceding post-rebase regression run passed 45/45 tests. Fresh current-head CI/reviews are pending. - `6d772b7cc` reached Greptile 5/5 with zero comments/open threads and Superagent pass. While 32 checks had passed, master moved and caused a documentation conflict in `maps/feature-map/onboarding.md`. Rebased immediately, preserving both onboarding paragraphs. The complete four-commit branch was pushed as `19e6dcb0b` before waiting on CI/review, per Scott's instruction. A transient Git index lock interrupted the first continue; the command sequence briefly pushed intermediate `34e298c07`, then the completed rebase was immediately pushed. All feature and security/retry fixes are present; local recovery ref `codex/coordinator-recovery-6d772b7cc` preserves the pre-rebase history. The only tree differences from that ref are upstream's two smoke-test files and the combined documentation paragraph. GitHub reports MERGEABLE. Fresh current-head verification is in progress; prior-head review results are not current acceptance. - `21efa3120` passed Superagent but Greptile scored 4/5 for a valid retry-key persistence bug: an acknowledged duplicate key could reopen a resolved review. Current head adds an additive, replay-safe `0404_light_swarm.sql` migration for indexed, task-scoped aliases from acknowledged retry keys to the canonical interaction. Alias registration and retry lookup occur under the existing task lock; payload/actor mismatch still conflicts, and each canonical review permits at most 100 aliases. Original and acknowledged keys replay the terminal result rather than creating a new gate. Five newest migration snapshots are retained. Service/routes tests 165/165 passed; final quality review plus migration replay/snapshot checks 25/25 passed; server typecheck passed. The P1 thread is resolved in code; fresh current-head reviews and CI are pending. - `69c2c7543` passed 55 checks and Greptile 5/5 with no open Greptile findings, but Superagent check [123140675812](https://github.com/paperclipai/paperclip/pull/17983/checks?check_run_id=123140675812) correctly flagged unbounded pending quality-card creation. Current head fixes that under the existing task-row lock: equivalent requests with fresh idempotency keys reuse the pending card; conflicting requests for an unresolved deliverable return the existing interaction ID; pending cards are capped at ten per task across all reviewers. Distinct outputs remain independent, and already-answered items in a partially pending batch can receive a fresh review. Added real-database concurrency, cross-card duplicate, partial-answer, and hard-cap regressions. Quality review 22/22 tests and final server typecheck passed; earlier service/routes 163/163 tests passed before the final partial-answer refinement. One added assertion initially expected a service HTTP error, but duplicate keys are rejected earlier by the existing shared schema; the assertion now verifies that validation. Fresh current-head CI/reviews are pending; the action-required security result is not treated as passed. - `c9c575e5f` completed all 56 checks successfully with four conditional skips, Greptile 5/5, and Superagent pass. Its two P2 review findings are fixed in the current head: independently bound quality approvals now each remain visible in attention; changing/clearing URL-backed assignee/status filters updates the URL and survives task navigation. Both review threads are resolved. Regression evidence: attention service 34/34 tests passed, CoordinatorTasks 20/20 passed, UI and server typechecks and the full workspace build passed on `69c2c7543`. The first new UI test run caught an assertion racing React Router's transition; waiting for the navigation before asserting passes without weakening the navigation/remount checks. These older-head green checks are not claimed for the new fix commit. - Preceding `0e6ef7c44` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, and passed Superagent. A fifth migration-metadata collision arrived before the final Runner job finished. Those green results are preserved as prior-head evidence, not current acceptance. - Previous fully verified migration head: `0e6ef7c44696a377f35e69521b93587576374bd0`, integrating master `f8226b051` after the fourth migration-number collision. Master added terminal GitHub-response migration 0401 while the final native check ran; upstream 0401 SQL/journal/snapshot is preserved unchanged, and preferences are generated as idempotent `0402_yielding_blink.sql` on top. Exactly the five newest snapshots remain. GitHub reports MERGEABLE after the push. All 1,155 relevant migration/review/interaction/native-execution/remote-assets tests, workspace typecheck/build, migration guards and whitespace checks pass. Fresh exact-head CI/Greptile/Superagent are pending. - Immediately preceding head `36c67cf03` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, plus passed Superagent security. No retry was needed for that head's CI. Master advanced and introduced only a migration-metadata conflict while its final native check ran, so that green result does not establish acceptance of the current integration. Four consecutive collisions (0398, 0399, 0400, 0401) during verification are retained as concrete integration churn; a brief pause of competing migration merges was requested, without assuming it happened or changing any repo setting. The PR has not been merged. - Prior fully verified head: `36c67cf03`, on the clean `30059c0ed` integration of master `efd65c112`. The last Greptile P2 is fixed: Needs attention retains the participant-agent scope through authoritative server membership queries for each pending task, including historical participation when another agent currently owns the task. It does not approximate participation with current assignment or enumerate every company task. The actual view-switch regression and all 31 focused task/routing/collection tests, UI typecheck/build, token gates and whitespace checks pass. Greptile completed a fresh 5/5 review on this exact head (review 19) with zero new comments and no open review threads. CI and Superagent remain in progress; no current-head CI failure has been reported. - Head `30059c0ed` received Greptile 5/5 with that one nonblocking participant finding, now fixed. Superagent passed that head. Current-base integration restored the ephemeral RunsOn fleet after a superseded workflow retained queued aggregate jobs: normal cancellation was insufficient; the single force-cancel request was interrupted but readback confirmed that obsolete run cancelled and current CI began on the trusted fleet. No cancellation retry or runner/security/workflow changes were made. Its partial CI was superseded by the necessary participant fix and is not current acceptance. - Previous compatibility/base-sync head: `30059c0ededc0eab4e04e8299a884f36db8b292f`, integrating master `efd65c112` and the Inbox/workspace compatibility fixes. GitHub reports MERGEABLE after the push. Coordinator keeps Inbox standalone under More without rewriting the instance's Combined Inbox preference; opting out restores that preference. Saved merged-Inbox URLs resolve to their original owner, task status presets remain usable, and canonical repeated `workspace` parameters retain scoped, deduplicated pagination (including Needs attention). All 192 broader routing/Inbox/sidebar/privacy/review tests and the final 16 task-page tests pass, as do workspace typecheck/build, token gates and whitespace checks. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The preceding `d5b9be5bd` remained mergeable but its CI routing gate intentionally fell back to GitHub-hosted runners: “stacked base merge commit does not contain the current base branch.” All 45 downstream jobs stayed queued. Current master was merged cleanly to refresh the pinned base; no runner/security settings or workflow logic were changed. Greptile scored that preceding head 4/5 with the two compatibility findings fixed here. Its automatic Superagent scan was inconclusive at checkout; the documented retry was still pending when superseded. Those results do not establish acceptance of this current head. - Previous approval/migration head: `d5b9be5bd3b8d6b0437e42bf48138377f34fa67d`, integrating master `ba5bba86d` and the stale-output approval correction. GitHub reports MERGEABLE after the push. The fresh P1 finding is fixed: every submitted item now validates and locks its bound output inside the resolution transaction, whether or not it carries a required-quality-review binding. Real database regressions verify that an advanced document revision or deleted attachment refuses the entire batch without recording approvals, and current ordinary output reviews still resolve. All 272 combined migration/review/interaction tests pass; workspace typecheck/build passed. Fresh exact-head Greptile and CI are pending. The automatic Superagent scan ended inconclusive at checkout; one documented on-demand retry is pending. These are explicit verification limits, not passed security results. The P1 thread is resolved. - Master consumed the next migration slot three times during this verification pass: confirmation recovery took 0398, continuation recovery took 0399, and GitHub acknowledgement recovery took 0400. Each upstream SQL/journal/snapshot is preserved unchanged. Preferences are now generated as idempotent `0401_white_lenny_balinger.sql` from upstream 0400; exactly five snapshots remain, and replay coverage locates this migration by table content. This is recorded concurrent integration churn, not a claim that older green results apply to the latest base. - Head `ed3b68da9` fixed the two history/URL findings with 26 focused regression tests, server/UI typechecks, workspace build and token gates passing. Greptile then scored that head 4/5 with the stale-output P1 corrected in the current head. Its partial CI is superseded, not counted as current acceptance. - Previous filter-fix head: `ed3b68da9`, based on the `8458d39d9` merge of master `063cfd438`. The two fresh Greptile findings are fixed: history outcome filters now match recorded per-item verdicts in SQL before pagination (mixed reviews remain in both applicable outcomes), and same-page URL assignee changes/back/forward navigation update that filter without clearing independent filters. All 26 focused database/navigation/collection tests, server/UI typechecks, workspace build, token gates and whitespace checks pass. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The `8458d39d9` migration integration also passed all 328 targeted migration/interaction/confirmation tests, workspace typecheck/build and Superagent security scan. Greptile scored that head 4/5 with the two functional gaps fixed in the current head; those prior-head checks are not current acceptance. - Migration integration head: `8458d39d9721f4cb312c2390070a0609f8cddd1a`, integrating master `063cfd43813cabf18417605fb9eefb9aacf95c8d`. GitHub reports MERGEABLE after the push. Master advanced twice during this conflict-resolution pass: first its confirmation-recovery migration took 0398, then its continuation-recovery migration took 0399. Both upstream migrations and journal entries remain unchanged; Coordinator preferences are now generated as idempotent `0400_sad_guardian.sql` from upstream's latest snapshot. Exactly the five newest snapshots remain. The replay regression now locates the table migration by content instead of hardcoding a renumbered filename. - All 328 targeted question/review/confirmation/migration tests passed again after the second merge. Workspace typecheck/build and fresh exact-head CI/reviews are in progress. The preceding `0c46f46d6` passed workspace typecheck/build and Superagent security scan, but was superseded because master consumed migration 0399 while those checks ran. That prior-head result is not acceptance of the current head. No production settings or runner/security permissions were changed, and the PR has not been merged. - Previous integration head: `0c46f46d6f2b23417ab872ce3cb6ff2673647287`, integrating master `287b04d72f1b9fca1412e05435c75b8302ba0710`. GitHub reports MERGEABLE after the push. The renewed conflicts were `0398_snapshot.json`, `_journal.json`, and the top imports in `issue-thread-interactions.ts`. Upstream's `0398` confirmation-recovery index and journal history are preserved unchanged; Coordinator preferences were regenerated as idempotent `0399_bouncy_miek.sql`, with replay coverage updated and exactly the five newest snapshots retained. Both terminal confirmation notifications and Coordinator task assignment/required review behavior are preserved. - All 328 targeted interaction/review/confirmation/migration tests pass, as do workspace typecheck, map validation and whitespace checks. The workspace build and fresh exact-head CI/Greptile/Superagent checks are pending. The first migration-generation attempt hit the sandbox's local IPC restriction before generating files; the unchanged authorized local generation completed successfully. Prior-head green checks below do not establish acceptance of this merge. - Previous acceptance head: `8b7dcf739ed15afc51351fdfa0e0e5fbc75ab788`, with master `1aa3a21ac` integrated and GitHub reporting MERGEABLE. Greptile completed a fresh 5/5 review on this exact head with zero new actionable findings and no open review threads. The [fresh CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40752864319) completed SUCCESS on attempt 2 at 19:40:08 UTC, including all eight E2E shards. Final completed check set: 55 succeeded, four conditional checks skipped, zero pending/queued/running checks, and one neutral/inconclusive Superagent scan. The auxiliary commitperclip review and isolated native Runner compilation/cache-reuse check both passed. This is not a claim that Superagent produced a passed security verdict. GitHub still reports MERGEABLE against current master `60b8089a44b4eaa37e627adbfdf2f130bda13335`. - Current-head E2E: all eight shards and the aggregate passed, including the previously failing YAML browser journey. Serialized server shard 9/9 failed: the protected YAML projection test exceeded its original 15-second timeout; its late update request then reached the next test's reset mock (the captured payload is the prior test's model/label update). All 88 tests in that exact unchanged file passed locally in 38.03 seconds, with no timeout or assertion changes. The original slow-request cause remains unproven. CI failure log: `/tmp/pr17983-8b7-serialized9.log`; local recovery: `/tmp/pr17983-adapter-yaml-repro.log`. The single bounded unchanged shard rerun passed, and the workflow completed SUCCESS. Rerun log: `/tmp/pr17983-8b7-serialized9-rerun.log`. The initial failure is retained; a passing retry does not establish its original slow-request cause. - Greptile scored preceding head `4873704be` 5/5 with one valid P2: opened task decision records missed company-scoped live invalidation. The current head uses the existing company decision prefix for resolution and effect-completion events, and resets private record payloads on access changes. Its thread is resolved. All 107 focused record/live-update tests, UI typecheck/build, token checks and whitespace checks pass. - CI on `4873704be` failed E2E shard 7/8 in the Advanced YAML lost-creation-response journey: its clipboard helper selected page text starting with “Skip to Main Content” instead of YAML. The exact unchanged journey passed in a disposable local browser instance (1 test, 17.1 seconds). No assertion, timeout or production behavior was weakened; the original failure cause remains unproven. Failure log: `/tmp/pr17983-487-e2e7.log`; local check: `/tmp/pr17983-agent-yaml-repro.log`. The full unchanged Advanced YAML browser suite also passed locally (6 tests, 37.8 seconds). Fresh current-head CI subsequently passed this journey and all other E2E shards. - Both the automatic and one supported on-demand Superagent scan on `4873704be` ended neutral/inconclusive: “Pinned PR scan merge-base checkout failed or timed out.” These do not count as a passed security verdict. The current head's automatic scan and one documented on-demand retry also returned the identical neutral checkout failure, with zero annotations or further diagnostics. This is an external scanner blocker, not a passed security verdict. No security settings or permissions were changed. - Recorded decision links now render and focus the full source-task record in both Classic and Coordinator, including recorded choice, submitted inputs, and the existing effect-result resolver. Fresh terminal records take precedence over older open-decision cache entries. Context and account checks keep mismatched or inaccessible records out of the DOM; no decision-history tab was added to tasks or chat. The shared company live-run feed now drives live-only filtering, including ended runs and terminal tasks with stale run entries. All 213 focused tests, UI typecheck/build, token and whitespace checks pass; the final focus follow-up passed all 188 task-detail tests and UI typecheck. The first new navigation test failed because its test host omitted company context; the host was corrected before the passing run. - Previous integration head: `375c5e89516705050d94bab99336212841b2e6e1` merges master `1aa3a21ac` after four additional upstream commits. The sole conflict was competing imports in `server/src/services/issues.ts`; both quality-review policy and inbound wake-notice notifications are preserved. The PR-specific review import now sits beside execution-policy imports instead of the frequently changing first line. GitHub mergeability is rechecked after the push. - On the combined master tree, workspace typecheck/build, all 153 affected UI tests, map validation, and whitespace checks pass. Four server suites passed 77 tests; the fifth suite's module-import hook timed out while broad verification was running. Its unchanged isolated run passed all 41 tests in 7.2 seconds with the original timeout. This records recovery without claiming the original import-timeout cause is proven. CI on `375c5e895` passed every substantive job, including all eight E2E shards and Superagent; Greptile found the two functional gaps fixed in the current head. A short-lived `669d90876` commit added the main fixes before the focused navigation follow-up. - Immediately preceding head `9d9737b49` completed all 56 checks successfully with four conditional skips, Greptile 5/5 and no open findings, plus a successful Superagent scan. The signoff E2E shard passed on its single accepted rerun; its initial cancellation/timing failure is retained below. Master moved again while those checks ran, requiring this final integration rather than treating the prior head's green result as current acceptance. - Previous head `9d9737b496911a6054d0ea22d1b42b28b115e9ab` includes master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`; that head was subsequently conflicted by moving master. The master merge preserved durable account-bound writes and Coordinator UX. Workspace typecheck/build, 416 focused UI tests, and 142 affected comment-route tests passed. A stable full UI run on preceding head `938ad1665` passed all 9,833 tests across 836 files; its final full workspace build also passed. - Greptile found nested proposal project inheritance and Classic bound-revision links on `91be74952`. Both were fixed with 142 interaction-route tests, 113 focused UI tests, and UI/server typechecks. Its next review identified a valid draft-preservation bug in the new Classic loading UI. Head `9d9737b49` keeps loading/error notices outside editors and annotations while preserving dirty drafts; 115 focused review UI tests pass, including pending/failed revision loads and continued draft editing. All related threads are resolved. Greptile completed a fresh 5/5 review of `9d9737b496911a6054d0ea22d1b42b28b115e9ab` with no open findings. - CI on `938ad1665` exposed an integration regression with the newly merged durable startup saves: the personal preference gate blocked the task page before its first session lookup. The guarded title editor now mounts while that lookup is pending, with presentation-specific tools still gated. All 216 affected component tests pass. The unchanged five-test `disconnect-write-recovery.spec.ts` browser suite passes against a newly launched disposable local-trusted PostgreSQL instance on port 33219, including the exact failed startup-save journey. No live provider, hosted authentication or production proof is claimed. The helper-owned test server exited and the port has no listener. UI typecheck, built-browser assets, token and whitespace checks pass. - Initial durable-write merge tests found duplicate imports and unscoped draft fixtures, fixed before 416 focused tests passed. A sandbox typecheck could not create its IPC socket; the supported rerun passed. The first revision-link test used a wrong Return to latest button assertion, corrected before the 113-test pass. The startup E2E reproduction initially still failed because an outer shell gate also blocked the page; the subsequent fix passed all five cases without changing the test or its timeouts. A focused loading-header assertion initially used an incorrect fixture title; correcting it preceded the 216-test pass. These failures remain recorded. - Earlier routing gates fell back to GitHub-hosted runners while master was moving or conflicted. Superseded queued runs were cancelled, then the `938ad1665` routing gate explicitly selected the ephemeral RunsOn fleet. Its CI actively executed and completed most tests; only the diagnosed startup E2E shard and aggregate failed, alongside the valid Greptile finding. Auxiliary GitHub-hosted workflows also completed successfully on earlier heads, so queueing alone is not an administrative blocker. Fresh [CI on `9d9737b49`](https://github.com/paperclipai/paperclip/actions/runs/40744012678) is the acceptance target; the previously failing startup E2E shard now passes. A different signoff E2E shard failed during executor resubmission when its fixture run was cancelled as `issue_continuation_waiting_on_review`. The exact unchanged journey passed locally (1 test, 46 seconds), and the preceding head with identical server/signoff code passed that CI shard. This supports a bounded rerun but does not establish the timing failure's root cause. After the workflow finished, GitHub accepted one rerun of that failed shard and its dependent aggregate; both passed, completing all checks on that head. The failure log is retained at `/tmp/pr17983-9d9737-e2e7.log` and the local reproduction at `/tmp/pr17983-signoff-repro.log`. No assertion or timeout was weakened. - On the first merge, full workspace typecheck/build, all 9,622 UI tests, 267 tests across seven affected server suites, two migration checks, map validation, token gates, and the PR diff whitespace check passed. Greptile then found duplicate local task-search filtering and lost completed item-review detail. Both are fixed, with 42 passing focused UI tests and passing UI typecheck/build. - A subsequent Greptile review found attachment confirmations without working links and mobile history without a visible host. Confirmation layouts now derive the authenticated immutable-content endpoint from the attachment ID, and Needs You uses the existing full-screen mobile sheet. All 119 focused tests pass, including missing/stale display URLs, both review layouts, and both mobile history entry buttons. UI typecheck/build and token checks pass. Full workspace typecheck/build also pass after the second master merge. - Greptile gave head |
||
|
|
553a004926 |
fix(server): schedule approval retention from durable deadlines (#17497)
## Thinking Path > - Paperclip manages agent work and should become idle when it owes no background work. > - The attention feed archives old pending approvals under the existing 90-day policy. > - The server rebuilds the feed on each maintenance tick, even when approval state has not changed. > - Some approvals have never been read, so retention rows alone cannot define the work. > - Approval records and committed eligibility changes define the real deadline and wake triggers. > - This PR gives approval retention its own deadline worker and reuses the existing retention and archive-delivery contracts. > - The approval lane stops idle scans while other attention sources remain separate migration work. ## Linked Issues or Issue Description Stacked on #17490. This PR owns approval retention only. Archive notification delivery is already separated in #17475. Other attention-source retention and remaining recurring server duties remain in the stack inventory. The related-PR search found no overlapping approval-retention worker migration. **What existing behavior does this improve?** Automatic archival of old pending approvals from attention. **Current behavior** The shared maintenance tick builds the attention feed for each active company. That query discovers pending approvals and materializes retention records before checking their age. An unchanged or empty approval source is read repeatedly. **Proposed behavior** Reconstruct eligible approvals at startup. Schedule the earliest approval activity timestamp plus 90 days. Wake after committed changes to approvals, links, issue visibility, company activity, or retention controls. Stop scheduling when no eligible approval remains. Archive records through the existing version checks, audit writes, and notification outbox. **Reason and benefit** Idle approval retention should generate no recurring SQL. Unseen approvals still need eventual archival. A stale candidate must not archive an approval that was resubmitted while the worker waited for its source lock. **Breaking changes** None intended. The archive age, eligibility, keep controls, audit, and notification policy remain. Normal attention reads still include approvals. No Cloud sleep blocker is relaxed. ## What Changed - Restrict tool-review signals to their existing terminal transition owners (approve/reject/cancel). Creation/link/revision/resubmit signal retention only. - Coalesce retention notifications under one thirty-second aggregate read budget. Known deadlines still preempt it; no notification or durable deadline means no scan. - Add a bounded approval-retention worker. It uses the shared deadline coordinator and reads pending approvals directly. - Share first-linked-issue ordering and execution visibility through `approval-visibility.ts`, used by both the feed and worker. Preserve actor-specific read filtering and locked archival rechecks. - Materialize unseen approvals on demand inside manual keep/archive/revive transactions. Use the saved activity timestamp and existing versioned state; retain company scoping and rollback on failure. - Add awaited pre-write signals to approval, budget, tool, link, visibility, company, and retention mutation owners. Preserve outer transaction ownership; keep provider calls outside SQL wrappers. - Narrow retention inputs to the source fields the existing service uses. Signal materialization only for inserted or changed state so unchanged reads converge. - Exclude approval collection from the maintenance feed before its query. Other source kinds remain in that sweep. - Add the pending-only approval deadline index with concurrent, idempotent creation. Keep the five newest migration snapshots. - Add real-Postgres lifecycle tests and startup wiring tests. Enable validated external test databases for existing attention and budget suites. - Document producer coverage, quiet conditions, recovery and remaining scope in `doc/idle-approval-retention.md` and the stack ledger. ## Verification Current test revision: `dd844f4e8cbbf1db681a774dc31cd552301bef48`, followed by docs-only `3efda33812`, which received both 5/5 reviews and passing CI. This candidate normally integrates landed Grok `2adcde162e`. All runtime and acceptance-test sources are byte-identical to that tested/reviewed tree; only four independent Runner E2E accounting files and the ledger differ. Migration 0392 and the parent journal prefix are unchanged. Local checks are reused for their unchanged source; fresh final-head CI and reviews are required against master. This remains the approval-retention duty. - **315 cases pass** across eight complete suites: approval retention/service, attention, decision retention, budgets, tool gateway, server startup and scoped deadlines. All 30 real-PostgreSQL lifecycle cases pass. Server production typecheck and build pass. - **Four regressions failed before repair**: unseen young keep/archive/revive returned 404; the manual keep rollback case could not reach its update. They now pass. A failed keep update leaves no retention row or audit; retry succeeds with one audit. - Both linked-issue orders are checked against the feed: hidden first/visible later is excluded; visible first/hidden later remains eligible. The feed and worker use the same owner query, with actor read filtering preserved. - Generated concurrent/idempotent migration `0392_small_frightful_four.sql` adds only the pending approval index. Every historical parent SQL migration and all 390 parent journal entries are unchanged. Five generated snapshots remain (0388–0392). Schema drift, migration safety/numbering and actual fresh apply/replay on five isolated PostgreSQL databases pass. - Existing precision, transaction, drain/resume, failure fairness and shutdown tests remain. Earlier transaction-stub, budget-import, diagnostic-client and CI failures are recorded in the owning audit and stack ledger; later passes do not erase them. - Three additional real regressions failed before the wake repair: ordinary mutations emitted tool-review signals in both resolution cases, and sequential tenant writes caused repeated global reads. Tests now prove terminal approve/reject/cancel signals and one aggregate reconciliation per thirty-second window with a different company's pending approval retained. The aggregate budget is stricter than a separate per-company budget and matches the old default cadence; known deadlines preempt it. - The initial coalescing fixture expected the concrete pending-work hold to clear before reconciliation; it now preserves that hold and measures SQL. The first broad repair passed 313 cases but failed the existing parked-review cancel case. The original cancel producer is restored; all 315 now pass without weakening assertions. At old head `5991f257bb`, security was action-required and Greptile returned service error `8fb1835c-e564-476b-b802-d5be164ea915`. Fresh repaired-head results are required, not assumed. Commands: server Vitest for `approval-retention-work`, `approvals-service`, `attention-service`, `decision-retention-service`, `budgets-service`, `tool-gateway-service`, `server-startup-feedback-export`, `scoped-deadline-work`; server typecheck/build; DB snapshot drift, migration checks and fresh apply/replay. Fresh exact-head Greptile/Ponytail and CI remain merge gates. No full local repository suite or deployed host suspension/cost claim is made. Earlier stack verification below is historical evidence at the explicitly named revisions, not this head's acceptance: Approved delegated Test-tab calls now evaluate caller and target policies in the existing claim transaction. Real HTTP regressions first showed a missing caller charge and shared-limit self-rejection. All 78 connection-management cases and 92 gateway cases now pass on the owning branch, along with server production typechecking. All 78 connection-management cases also pass at combined tip `d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover shared charges, separate charges, rollback on denial, duplicate approval, and rejection of a later authorized call. Intermediate counter/identity fixture failures and a reused-database Slack collision are recorded in [the owning audit](https://github.com/paperclipai/paperclip/blob/20557d2c3dd0949c734eaa8095fd78b9b07d7fd5/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes). The earlier combined checks below remain evidence for their stated revisions. Current-head CI and review remain required. Earlier published head: `20557d2c3dd0949c734eaa8095fd78b9b07d7fd5`. This remains a single-duty PR stacked on #17490. - Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5` with normal merges. Every branch passed generated-schema checks and its target-branch migration-order gate. Existing SQL is unchanged; the 25 stack migrations follow master’s new migration. - Duty-specific producer coverage, lifecycle tests, and earlier failure records: [2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/76a49810c7ff403ae7720bef0d453747bd844731/doc/plans/2026-10-24-recurring-server-sleep-stack.md). - Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355 startup/cancellation/recovery/dispatch/preservation tests**, **75 connection-management tests**, and production server typechecking pass. These combined checks do not mean every intermediate PR independently ran the full repository suite. - Real master-to-stack database upgrade passes: **379 unchanged prior migration receipts plus 25 new migrations**, with the gateway responsibility fixture preserved. - Owning-branch checks: 84 runner installation/settlement cases plus runner typechecking and generated TypeScript/Rust profile agreement; 253 GitHub review workflow cases. Stop-attribution and held-worker readiness regressions failed before their fixes. - Failure accounting: the initial combined run passed 349 cases and failed six idle assertions in a reused preservation database containing old cancelled runs. Repeating the combined command with a fresh preservation database passed. Earlier invalid project selection, merge import/type errors, and the interrupted full local suite are not counted as passing checks. - **Latest-head CI and review remain required.** Prior CI findings include GitHub idle observation, route teardown, attention cleanup, touch-picker E2E, workspace health, and voice-cleanup timing; local passes do not erase those failures. No full local repository-suite, deployed sleep, fleet cost, or live Copilot qualification claim is made. ## Risks - Signals cover the application database owner and its child pools. Independent SQL writers and other active processes need an explicit wake integration. Startup reconstructs committed durable state. - The source row lock prevents stale approval-activity archival. It does not add cross-table serializability for concurrent link, visibility, or company-policy changes. - Newly changed eligibility can wait up to the thirty-second notification window, matching the former default maintenance cadence. Saved deadlines still preempt; no new idle polling is added. - Failed records retain recovery debt. Bounded cursor progress lets later rows proceed without dropping failures. - The shared retention transaction now groups materialization and activity-version updates. Existing attention and retention consumers are tested against PostgreSQL. - Kept, archived, resolved or inactive-company approvals can become quiet, but their supported mutations must retain their wake notifications. - The local scheduler cannot wake stopped compute. This PR neither adds approval deadlines to Cloud's external alarm contract nor claims measured host or database suspension. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact serving model identifier and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
2a45fd95df |
Reduce newer-run history reads for attention (#15697)
Replace broad newer-run responses with bounded company-scoped existence probes. Read keyless history once per unique agent and preserve existing matching and millisecond rules. Keep all stored run data. Verify with 31 real PostgreSQL attention tests, workspace typecheck, server build, response-budget counterfactuals, full green CI, and Apex 5/5 review. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
2dd9811e8b |
Own agent hiring and termination in a lifecycle module (#15631)
## Thinking Path > - Paperclip manages AI agents and their work. > - Agent creation, approval, pause, and termination change whether an agent can work. > - These changes currently use separate service and database paths. > - Some agents need setup after their record exists and before their harness can run. > - This PR adds one module that owns those state changes and their transactions. > - The module records progress and retries incomplete work after a server restart. ## Linked Issues or Issue Description **Problem or motivation** A hire can become active before its execution environment is ready. Direct agent state writes also bypass one common lifecycle boundary. Termination does not have a durable step for resource cleanup. **Proposed solution** Use agent lifecycle commands for hiring, approval, pause, resume, and termination. Keep proposed hires pending until approval. Prepare each approved hire, test its saved harness configuration, and then permit work. Wait for required plugin work before completing a step. **Alternatives considered** A general database commit hook would expose transaction details to callers. A timer alone would delay each new hire. This module owns each command transaction and starts its worker after commit. A periodic scan recovers work after a process stops. **Roadmap alignment** This change adds the lifecycle boundary for the existing agent and plugin systems. It does not add a user interface or a provider implementation. It replaces the lifecycle delivery approach in #15391. Related proposals include #11475 and #13099; this change does not add their external executor protocols. ## What Changed - Add durable approval, preparation, verification, pause, resume, stop, and cleanup states. - Keep agent creation, lifecycle state, configuration writes, and agent record deletion behind `modules/agent-lifecycle`. Keep validation, credential rules, reads, onboarding, invitations, general approvals, and company data deletion in their services. - Commit a hire before the remaining onboarding or invitation work. Serialize requests with the existing dedicated database connection helper. Reuse the hire after a failed attempt. - Keep accepted work in the run queue during setup. Require execution and cleanup evidence before completing pause or termination. - Retain the original credential owner when approving migrated hires. - Start lifecycle work after each command commits. Use database leases and revision checks for retries and late results. - Add a generic required plugin method and capability. Store the required plugin IDs for each agent. Track host and plugin completion separately. - Keep harness verification and credential rules in services. Supply their results to lifecycle through explicit ports. Reject failed logins and incomplete probes. - Remove reverse imports from lifecycle adapters into services. Share record queries and validation below both layers. - Use one hire decision path for agent IDs and approval IDs. Resolve the approval and agent change in one transaction. Expose policy reconciliation on the command object. - Save configuration and verification invalidation through one module operation. Ordinary commands own their transaction. Restrict the shared transaction integration to its listed callers. Reject identity, company, accounting, timestamp, and unknown fields. - Require completed termination before deleting agent records. The company deletion service calls the module through a separate deletion interface. - Save an approved secret before applying its agent binding. A failed binding leaves the secret approved. A retry reuses it. - Remove old permission and metadata export files. Move hire configuration rules out of lifecycle persistence. - Report failed worker steps and scans with operation context and known error codes. Omit provider messages, stacks, and configuration values. Document ordinary application use and extension rules. - Recover expired native owners only after execution stops. Retry cleanup of failed retained leases. Wait for setup in the live smoke script. - Keep manual, budget, and company pause reasons separate. Guard execution status writes with the ready state. - Add a migration, boundary checks, regression tests, and a lifecycle guide. Keep the current UI. - Limit periodic policy checks to 100 eligible agents per minute. Continue from the last agent ID. Exclude pending hires and agents in termination or a final state. Keep explicit company and budget updates immediate. ## Verification - Latest head `ccc3c86b0` aligns the shared status update schema with the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2 CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile comment is fixed. CI is running; no new Apex review was requested for this small fix. - Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test uses the shared 90-second timeout from master; the older timeout commit was removed. The affected backup test passes. - Agent budget edits now pass the target agent ID through enforcement. All 50 affected tests, server TypeScript, and 11 boundary tests pass. Both configuration update paths have regression coverage. CI is running on the latest head. No new Apex review was requested for these small fixes. - Rebase `9b568666a` preserves the run-control extraction and delivery-work startup changes from master. All 559 affected tests, server TypeScript, and 11 boundary tests pass. CI and Apex review are running on the final head. - The policy scan fix passes 56 lifecycle and diagnostic tests, 24 policy and caller tests, and 11 boundary tests. A regression test checks the page limit, interval, excluded states, and cursor wraparound. - Rebase `fd4698750` passes all 611 tests in eight affected lifecycle, route, and heartbeat suites. Server TypeScript and all 11 module boundary tests pass. The company deletion rollback and concurrent hire tests are included. - Follow-up `00aac7646` fixes the failed decision deletion test and the public update schema. All 126 affected server tests, 8 shared tests, 2 CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI TypeScript checks pass. - Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc --noEmit`. - All 11 boundary tests pass: `node --test scripts/check-module-boundaries.test.mjs`. The repository boundary scan also passes. - All 68 lifecycle, failure diagnostic, and harness verification tests pass. Tests reject forbidden fields through the root command, combined transition command, and restricted transaction integration. They also check invalid fields returned by configuration preparation. - Failure tests cover host work, plugin work, claims, renewals, deferrals, and scans. Log tests check known error codes and exclusion of private error contents. - All 355 tests in 10 existing caller suites pass. These tests cover secret approval, pairing, pending hires, credentials, company import, built-in agents, managed agents, instructions, and skills. - The branch is rebased onto `origin/master` at `2eb5187e4`. The queue guards and cancellation lock fix use the extracted heartbeat queue. Agent cancellation uses the extracted run-control file. The source diff scan found no secrets or private URLs. - Earlier checks cover approval retries, rollback, late verification results, policy holds, deletion rollback, concurrent company deletion, and queue admission during setup. - Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it 5/5 with no open comments. CI and a new Apex review are pending for the rebased commit. The branch has no merge conflicts. - The full local build and native Dot suite were not run. The host has no Cargo. No live provider test was run. ## Risks - Background policy recovery takes more than one minute when more than 100 agents are eligible. Explicit company and budget updates still reconcile immediately. - The restricted configuration integration keeps an agent reference and its dependent records in one transaction. It cannot change a budget or invoke a lifecycle transition. A periodic scan starts pending setup after that transaction commits. - Secret approval and binding use separate transactions. A failed binding can leave an approved secret with no agent binding. - New approved hires remain unavailable until setup and verification complete. Existing clients see `paused` during these steps. This PR does not change the UI. - Migration `0320_daily_onslaught.sql` maps existing statuses and pause reasons. It does not create external resources or claim that an existing harness passed verification. - Use a coordinated server update for this migration. Older server versions can write legacy status without the new lifecycle checks. - A failed onboarding or invitation request can leave a committed hire. A retry reuses that hire and completes the remaining records. - Deletion requires completed termination. Company deletion can require another request while cleanup is pending. - Required plugins must make repeated calls safe and reject older revisions. A disabled or removed required plugin blocks completion until it is restored. - A new hire can run a second harness test after the existing UI test. Some adapters can charge for this test. - No live provider test was run. Provider implementation and resource backfill remain separate work. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code tools, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR with the feature request field labels - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] Targeted local tests pass - [x] All Paperclip CI gates are green on the new commit - [x] Greptile Apex is 5/5 with no open comments on the new commit - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3a726e676f |
feat: enforce private task permissions across execution and data (#10633)
Enforce private task and project access across direct reads, search, execution, files, plugins, live delivery, and sharing mutations. Preserve downward-only sharing, current responsible-user authorization, and audited emergency access. Bind historical draft assets with migration 0314. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
667c79ded2 |
fix: prevent retry-exhaustion events from exhausting attention-feed memory (#13451)
## Thinking Path > - Paperclip manages AI agents and their work. > - Its attention feed shows failed runs whose retry budget is exhausted. > - Startup retention builds this feed before startup completes. > - The feed joins every exhaustion event to the full run context, then removes duplicate runs in JavaScript. > - Recovery can revisit an exhausted run and append the same event again. This multiplies the data loaded into memory. > - This pull request selects one small row per run in PostgreSQL and makes exhaustion writes idempotent. > - Existing duplicate events can stay in the database without multiplying run contexts in server memory. ## Linked Issues or Issue Description Refs #13367. This fixes the repeated-event allocation path in the retention feed. Other full-feed sources and sweep cadence remain separate concerns. **What happened?** The attention query loaded one full run context for every matching exhaustion event. Deduplication ran only after the driver had loaded those rows. A run with thousands of exhaustion events therefore produced thousands of context copies. The startup retention sweep can exhaust the server heap while reading this result. **Expected behavior** The query should return one row per exhausted run and only the context fields that the feed needs. Repeated checks of the same exhausted retry budget should reuse the original event. **Steps to reproduce** 1. Create a failed run with a 32 KB context and 2,500 matching exhaustion events. 2. Build the attention feed, including dismissed items, as startup retention does. 3. Inspect the database result before JavaScript feed processing. The old join returns 2,500 copies of the run context. 4. Call bounded retry scheduling repeatedly for a run at its retry limit. The old writer appends another exhaustion event on every call. **Paperclip version or commit** The attention query was introduced by #9380 and is present in stable `v2026.831.1`. The retention caller is also present in that stable release. The later recovery callback added by #13075 provides a repeated path into the exhausted-budget writer. This change is based on `c0fda8fac` after rebasing onto current master. Related PR: [#12162](https://github.com/paperclipai/paperclip/pull/12162) changes retention cadence and newer-run suppression queries. This change addresses the exhaustion-event join and duplicate event writes. ## What Changed - Select the newest company-scoped exhaustion event per run with a PostgreSQL `DISTINCT ON` subquery before joining run data. - Project only `issueId` and `taskId` from run context. Preserve JSON types, fallback behavior, run ordering, company filters, and run/agent status filters. - Reuse an exhaustion event for the same run, reason, attempt, and retry limit under the existing run-row lock. Recognize historical events without a migration. - Skip sequence allocation and live publication when a receipt already exists. - Document the run-log behavior and add PostgreSQL regression tests. ## Verification - Focused attention, retry-scheduling, and event-sequencing suites: 72 tests pass on the rebased head `39ce77158`. - The large-history fixture verifies two database result rows under 4 KB, newest-message selection, task-ID fallback, and company/status filtering. This assertion runs before feed deduplication. - Concurrent event tests verify one receipt across two database clients, reuse of historical receipts, distinct reason/attempt/budget keys, and stable event sequences. - Repeated scheduling through a new service instance produces no extra run-log or live events. - `pnpm --filter @paperclipai/server exec tsc --noEmit`: passes. - `pnpm -r typecheck` and `pnpm build`: pass on the rebased head, including native runner checks. - Greptile: 5/5 on `39ce77158`, with no review threads. - [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/34929002304): all 25 workflow jobs pass, including all server/workspace test shards, serialized server suites, browser tests, typecheck, build, native-runner verification, and the release dry run. Security checks also pass. The branch has no merge conflicts. - `pnpm test:run`: stopped with unrelated failures. Two chat integration cases passed when rerun separately (2 passed, 993 unselected). Three skill-cache cases failed on both this branch and the unmodified parent commit, with `EACCES` during directory rename. The full suite is not reported as green. ## Risks - No schema migration or data cleanup is required. The query still scans matching event history in PostgreSQL; its result size now scales with exhausted runs. - This does not bound every source in the attention feed or change retention scheduling. - An exhaustion receipt is emitted once per retry decision. Consumers that observed repeated copies will now receive one event. - Native source-event replay handling stays on its existing path. - A deployed application boot has not been verified. ## Model Used OpenAI GPT-6, used through Codex with reasoning, repository inspection, code editing, and test execution. The runtime does not expose a more specific model snapshot or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the focused tests locally and they pass (full-suite limitations are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3bafac12f7 |
refactor: remove automatic productivity reviews (#13263)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its recovery loop keeps assigned work moving after execution failures. > - Productivity review used run counts, comment counts, and elapsed time to create management tasks. > - Infrastructure failures could satisfy those rules and create more tasks without evidence that the source work needed management review. > - This pull request removes that detector and its continuation holds. > - Bounded recovery, budgets, explicit blockers, and normal review stages remain in place. > - Existing task records stay readable and unchanged. ## Linked Issues or Issue Description Refs #5897. That request describes unwanted automatic productivity reviews and asks to preserve existing tasks. This change retires the feature instead of adding another configuration switch. Related prior approaches: Refs #9191, Refs #12489. Those changes excluded infrastructure failures or bounded review creation. This removal replaces the detector rather than tuning its thresholds. ## What Changed - Delete the scheduled detector, automatic task creation, evidence refresh, and productivity continuation holds. - Remove computed productivity fields, special attention items, badges, and Storybook fixtures. - Retain historical origin values, decision compatibility, and recovery recursion exclusions. Add no migration and change no existing task data. - Update the execution contract. Replace feature tests with regressions for legacy task reads, ordinary attention, and bounded continuation in the presence of an old review. ## Verification - Targeted attention, issue-route, startup, and UI tests: 4 files and 101 tests passed. - Updated issue-route and UI tests: 2 files and 61 tests passed. - Bounded continuation regression: 2 cases passed, including a legacy review plus pre-dispatch cancellation churn. - `pnpm check:token-gates`: all four gates passed. - `git diff --check`: passed. - `pnpm build-storybook`: passed. - Greptile: 5/5 on `a5a612eea`, with no actionable findings. - Scheduler and historical recovery regressions: 2 files and 28 tests passed. - Repository `pnpm -r typecheck` and `pnpm build`: passed. - The complete `pnpm test:run` suite passed across the CI server, serialized-server, and workspace shards on `a5a612eea`. Stopped the duplicate local monolithic run after the full CI suite passed; no completed local full-suite result is claimed. The targeted local suites above passed. - CI serialized shard 5 initially hit a 10-second timeout in the first interaction-route test. The complete file passed locally (78 tests), then the single CI rerun passed. - All CI gates are green, including the build and end-to-end suites. - A local merge check against current `master` (`ce09ea40b`) completed without conflicts. ## Risks - API responses no longer include the computed `productivityReview` field. Consumers must stop using it. - The scheduler no longer creates management work from elapsed time, run counts, or missing comments. This is the intended behavior change. - Existing review tasks and explicit dependencies remain in place. Historical origins still prevent recursive recovery treatment. No task cleanup or data migration occurs. - The native review handoff repair is separate from this removal. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
20ccf3f476 |
feat(apps): add connection grants and delegated identities (#12341)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - External tools need explicit identity and access boundaries. > - Shared connection credentials cannot represent every user-scoped use case. > - Grants must stay company-scoped and support safe delegation. > - This pull request adds connection grants, identity rules, and their database contract. > - The benefit is durable control over which identity an agent may use. ## Linked Issues or Issue Description Refs #11965 This is stack 3 of 11. It depends on stack 2 and replaces another reviewable part of #11965. ## What Changed - Add company and user connection grants. - Add delegated identity and membership rules. - Synchronize database, shared, server, and UI contracts. - Register the grant-member replacement route in the OpenAPI surface in the same layer that mounts it. - Add migration 0231 with replay-safe guards and coverage. ## Verification - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/openapi-routes.test.ts` (5 passed) - `pnpm --filter @paperclipai/db check:migrations` - `pnpm build` ## Risks - Incorrect grant selection could expose the wrong credential scope. - The service enforces company and subject boundaries before credential use. - Migration 0231 is generated, ordered after 0230, and safe to replay. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked a public issue or pull request with `Refs #` - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
10d0555189 |
fix(interactions): authorize resolvers consistently (#11376)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Issue interactions give agents and people a structured decision record. > - Resolver routes used different authorization rules. > - Some routes blocked valid agents, including task watchdogs with normal issue access. > - The API did not show who could resolve a pending interaction. > - This pull request gives every interaction kind one resolver policy evaluator. > - The benefit is a clear decision path with consistent governance and company isolation. ## Linked Issues or Issue Description Fixes: #8087 Refs: #7403 Related PR: #11082 proposes board-only confirmation rules. This change keeps human-only review as an explicit policy. **What happened?** Agents could create issue interactions. Some resolver routes still required board access. This left valid agent confirmations pending. Task watchdogs could see the same problem without board identity. **Expected behavior** Every interaction kind must use one resolver policy contract. The contract must support `anyone`, `not_creator`, and `human_only`. It must also apply all normal governance controls. **Steps to reproduce** 1. Create a `request_confirmation` interaction as an agent. 2. Resolve it with another authorized agent. 3. Observe the board-only denial. **Paperclip version or commit** The problem exists on `master` before this change. **Deployment mode** Local development with `pnpm dev`. ## What Changed - Add canonical policies for `anyone`, `not_creator`, and `human_only`. - Use one server evaluator for every interaction kind. - Apply named addressees, company limits, review rules, and task watchdog scope. - Charge cross-issue resolutions to the existing per-run action limit. - Return the effective resolver audience in attention and interaction data. - Show the audience, governance choices, and denial reasons in the board UI. - Add telemetry, API documents, product documents, and regression fixtures. - Add migration provenance for safe legacy behavior. - Make migration `0218` safe for complete replays and partial prior runs. ## Product Rules - An interaction records a response. It does not grant authority for the next action. - `anyone` lets any authorized issue participant respond. - `not_creator` requires a responder other than the interaction creator. - `human_only` requires an authorized person. - A named addressee, company policy, or governed action can narrow the audience. - These controls cannot widen the audience. - A task watchdog uses the same rules as an ordinary agent. - A task watchdog does not receive board authority. - An agent resolution on another issue uses the shared cross-issue action limit. - Legacy pending interactions keep their earlier restrictions. - The UI shows the effective audience and a permanent denial reason. ## Verification - `pnpm --filter @paperclipai/db check:migrations` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run packages/db/src/issue-thread-interaction-resolver-policy-migration.test.ts` - The focused PostgreSQL test applies migration `0218` twice. - The test also completes a partial prior run and preserves existing provenance. - The latest GitHub head has 29 successful checks. - The opt-in Storybook visual check skipped as expected. - Greptile reports 5/5 with no open comments. ## Risks - New interaction writes use `anyone` by default. - Callers must select `not_creator` or `human_only` when they need stricter review. - Legacy pending interactions keep the old creator and human restrictions. - Migration `0218` fills only missing provenance fields during recovery. - Cross-issue resolutions can reach the existing action limit. - The shared evaluator affects every interaction kind. - Route, service, database, shared contract, and UI tests cover these rules. > This work matches the Agent Reviews and Approvals direction in `ROADMAP.md`. It does not duplicate a planned item. ## Model Used OpenAI Codex, GPT-5. The runtime does not expose the exact deployment ID or context window. The agent used reasoning, repository tools, shell commands, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked public issues or described the issue with the required labels - [x] I have not referenced internal Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open comments - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
8ee1fb21a6 |
feat(ui): badge the review policy when it constrains approval (#10938)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents move their work into review, and a reviewer must then give a verdict on it > - By default anyone with write access can give that verdict, including the agent that did the work > - The server can constrain that default per issue with a `reviewPolicy` column, but no screen showed the value > - A reviewer could therefore press Approve on a review that the server refuses, and get a 403 > - This pull request shows the policy as a badge on the two surfaces where a person gives a verdict > - It also makes an agent verdict read as a verdict in the activity timeline > - The benefit is that a reviewer sees who can approve before they try ## Linked Issues or Issue Description No public GitHub issue exists for this change. The description below follows `.github/ISSUE_TEMPLATE/enhancement.yml`. **What existing behavior does this improve?** The issue review flow. A reviewer cannot see the approval constraint on an issue before they give a verdict. **Subsystem affected** Web UI (`ui/`), with one supporting change in the server attention service. **Current behavior** The server stores an optional approval constraint for each issue in a `reviewPolicy` column. The column has three meaningful states: the default (`NULL` or `anyone`), `not_creator`, and `human_only`. The server enforces the constraint when it receives a verdict. No screen shows the value. Two problems follow: 1. A reviewer presses Approve on a review that the server refuses. The server answers 403, and the reason is not visible on the card. 2. An agent that accepts or rejects a review renders in the activity timeline as the raw action id, for example "issue thread interaction accepted". A person who reads the timeline cannot tell that a verdict was given. **Proposed behavior** Show the constraint as a read-only badge on the two surfaces where a person gives a verdict. Show no pixels for the default state, because the default is what every issue already does. Make an agent verdict read as a verdict in the timeline. Only agents set the column today, so this change adds no control to set it. **Reason and benefit** A reviewer sees the constraint before they act. This prevents the 403, and it removes the need to explain the 403 afterwards. The timeline also becomes complete, because it now shows agent verdicts and human verdicts in the same way. **Breaking changes** None. The change adds a badge and changes copy. It adds no column, no endpoint, and no request. **Additional context** The server-side column and the verdict enforcement landed earlier in #10931. This pull request is the user interface for that column. The default state stays unchanged on screen, so the badge appears on a small number of issues. ## What Changed - **A read-only "Approvals" row** in the issue Execution properties. The row renders *only* for a constrained policy: "Anyone else" (`not_creator`) or "Human only" (`human_only`). A `NULL` or `anyone` column adds no row, so the panel is untouched on the overwhelming majority of issues. - **The same badge on the stalled-review card** in `/decisions`, above the three review verbs. A reviewer now sees the constraint before they press Approve. The condition is the same, so the default card is unchanged. - **Agent verdicts read as verdicts in the activity timeline.** An agent that accepted or rejected a review request previously rendered the raw action id ("issue thread interaction accepted"). It now reads "approved the request". A stalled-review decision names the verb that the actor chose. - **A cleared policy reports as "anyone", not "none",** in the field-change receipt. The `reviewPolicy` column is nullable by default, so an absent value is a real setting rather than a missing one. - **All copy comes from `ui/src/lib/review-policy.ts`.** Its badge lookup returns `null` for the default. This makes "no pixels for the default" one enforced decision instead of a condition repeated at each call site. It also keeps the badge, the activity line, and the receipt reading alike. - **The server attention service carries the policy** on the review attention subject, so the stalled-review card can read it. ## Verification Automated tests: - `ui/src/lib/review-policy.test.ts` — the default returns no badge, however the column spells it (`null`, `undefined`, `"anyone"`). An unrecognised policy from the wire shows nothing rather than leaking an enum value. - `ui/src/components/AttentionQueueRow.test.tsx` — no badge on the default card, and the verbs still render. Suppression of the badge must not suppress the card. - `ui/src/components/IssueProperties.test.tsx` — no Approvals row on the default. The constrained row contains no `button`, so nothing there can PATCH. - `server/src/__tests__/attention-service.test.ts` — the review attention subject carries the policy, and subjects built from narrower selects do not claim one. Run them with: ```sh pnpm vitest run ui/src/lib/review-policy.test.ts \ ui/src/components/AttentionQueueRow.test.tsx \ ui/src/components/IssueProperties.test.tsx \ server/src/__tests__/attention-service.test.ts ``` Manual steps: 1. Open an issue that has no `reviewPolicy`. Confirm that the Execution properties panel shows no Approvals row. 2. Set the column to `not_creator`. Reload the issue. Confirm that the Approvals row reads "Anyone else", and that the row has no control. 3. Move that issue into review. Open `/decisions`. Confirm that the stalled review card shows the same badge above the review verbs. 4. Let an agent approve the review. Confirm that the activity timeline reads "approved the request" and not "issue thread interaction accepted". Screenshots were captured at 1440x900 and 390x844, in light mode and dark mode, with the three policy states side by side. The leftmost column in each capture is the default. It carries no badge and no extra row. ## Risks Low risk. - The change is additive on screen. Every new surface is behind a constrained policy, so the default path renders exactly as before. - The badge is read-only. It has no control and sends no request, and a test asserts that the row contains no `button`. - An unknown policy value from the wire renders nothing. It does not render the raw enum. - No migration, no schema change, and no endpoint change. ## Model Used Claude Opus 5 (Anthropic), model id `claude-opus-5[1m]`, 1M context window, with extended thinking and tool use enabled. Used through Claude Code for the implementation, the tests, and this description. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
678728f650 |
feat: maintained in_review review-path contract + stalled-review actions (#10675)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents move issues to `in_review` and rely on a "review path" (an interaction, an approval, a monitor, or a named reviewer) to tell them who decides next. > - That review path can silently disappear. A user comment supersedes the pending interaction, a monitor is exhausted, or a run ends without restoring a path. The issue then sits in `in_review` with nobody reviewing it and no visible action. > - Such issues become invisible zombies. Nobody knows a decision is owed, so the work stalls forever. > - This pull request makes the review path a maintained invariant, exposes a `reviewAttention` surface, and gives every stalled review three inline actions in the UI. > - The benefit is that an `in_review` issue always shows who reviews it, or shows an amber "nobody is reviewing this" notice with one-click Approve, Request changes, and Send back to work. ## Linked Issues or Issue Description This pull request describes the problem inline. The tracking issue is internal. **Subsystem affected** The review and attention loop that agents and humans share: the `in_review` status, the `reviewAttention` surface, the /decisions attention feed, and the issue-page review panel. **Problem or motivation** Agent-owned issues in `in_review` can lose their last review path. A user comment supersedes the pending interaction. A monitor is exhausted. A run ends without restoring a path. The issue then sits in `in_review` with no reviewer and no visible action. It becomes an invisible zombie and the work never progresses. **Proposed solution** Maintain the review path as a server invariant. Expose a `reviewAttention` field that says what is under review, who decides, and since when. Render a persistent review panel on the issue page and inline actions on the /decisions feed. Keep human PATCHes into `in_review` ungated, but record the requesting user so the panel never renders empty. **Alternatives considered** A pure background auto-recovery sweep. This stays opt-in and is not enough on its own, because it is invisible to the human. A bare status banner. This is rejected, because it gives no action to resolve the stall. **Roadmap alignment** This improves the core review and attention loop that both agents and humans use every day. ## What Changed - **Server — maintained review-path invariant:** when an issue enters or sits in `in_review`, the server derives and persists a review path (interaction, approval, monitor, or the requesting user) and recovers a stale path with one bounded wake instead of leaving the issue pathless. - **Server — `reviewAttention` surface:** a new field describes what is under review (bound target with links), who decides, since when, and whether the review is stalled. Stalled agent-assigned reviews are now included in the attention feed. - **Server — inline stalled-review decisions:** secured routes let a permitted responder Approve (→ `done`), Request changes (→ `todo` + wake carrying the note), or Send back to work (→ `todo` + wake) directly from the attention feed. - **Server — resume-intent wake:** an `in_review -> todo` transition now wakes the assigned agent so a resumed review is not dropped. - **Server — user-entry symmetry:** user PATCHes into `in_review` stay ungated (no 422 for humans) and record the requesting user, who becomes the named responder when no other path exists. - **UI — review panel:** a persistent `IssueReviewPanel` renders above the thread whenever status is `in_review`. The covered state shows the bound target, responder, and outcomes and hoists the pending interaction/approval card. The stalled state shows the amber notice plus the three actions. - **UI — decisions card actions:** the same three actions render inline on the /decisions `AttentionQueueRow`. - **UI — responsive fix:** the stalled action row stacks to full-width buttons at phone width and returns to a horizontal row at `sm` and up. New 390px stories capture the phone layout. ## Verification - `cd ui && npx vitest run src/components/IssueReviewPanel.test.tsx src/components/AttentionQueueRow.test.tsx src/lib/attention.test.ts src/api/issues.test.ts` — 91 tests pass. - Server suites added and updated: `issue-review-attention`, `issue-stalled-review-decision-routes`, `review-path-recovery`, `recovery-observability`, and related route/liveness tests (run by CI). - A designer reviewed the UI at 390px and desktop in light and dark themes on both the issue-page panel and the /decisions card. The stalled action row stacks cleanly at phone width with no overlap and keeps the horizontal row on desktop. ## Risks - **Migration:** adds migration `0200` (next after master `0199`, no renumber). It extends the agent-wakeup-requests schema and is additive. - **Behavioral shift:** `in_review -> todo` now dispatches a wake. This is intended (resume intent) and covered by tests. - **Authz:** the inline decision routes are permission-gated. Only a permitted responder sees and can trigger the actions. - Overall risk is moderate and contained to the review and attention loop. ## Model Used - Claude, Opus 4.8 (`claude-opus-4-8`), extended thinking, tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ded813ad6f |
feat(interactions): add governed agent addressees (#10252)
## Thinking Path > - Paperclip is the control plane that lets humans govern companies of AI agents. > - Issue-thread interactions are the structured handoff point for confirmations, questions, suggested tasks, and other governed decisions. > - Those interactions previously assumed that only board users could resolve them, preventing one agent from explicitly addressing another agent for a response. > - Agent resolution needs company-level governance, auditable resolver identity, safe terminal-state handling, and attention routing so authorization is enforced server-side rather than inferred from UI behavior. > - This pull request adds governed agent resolution, withdrawal and terminal expiry semantics, explicit agent addressees, lifecycle reconciliation, and attention-feed filtering. > - The benefit is that agents can participate in structured decisions without weakening board control, company isolation, wake behavior, or audit invariants. ## Linked Issues or Issue Description ### Subsystem affected Issue-thread interactions across database, shared contracts, server authorization/services, adapter callbacks, agent skill guidance, API docs, and UI governance surfaces. ### Problem or motivation Structured interactions were board-only, had no explicit agent addressee, and lacked durable withdrawal/terminal-expiry semantics. That made peer-agent decisions impossible to authorize and audit safely. ### Proposed solution Persist requested/effective resolver policy and addressee identity, enforce company governance and eligible agent resolution, reconcile addressee lifecycle changes, expose withdrawal and terminal expiry, and route attention to the intended active agent with board fallback. ### Alternatives considered Implicitly authorizing the issue assignee or mentioned agents was rejected as ambiguous and difficult to audit. Using comments alone was rejected because it loses structured outcomes and continuation behavior. ### Roadmap alignment Supports the ROADMAP direction for lightweight leadership-agent communication that still resolves into governed decisions and work objects. ### Additional context Public GitHub issue/PR search found no duplicate implementation; open PR search for interaction resolver governance and agent addressees only returned this PR. ## What Changed - Add company-scoped interaction resolver governance contracts and persistence. - Add requested/effective resolver policy, resolver identity, withdrawal, and terminal-expiry behavior. - Add explicit `addresseeAgentId` validation, authorization, persistence, lifecycle reconciliation, API documentation, and skill guidance. - Route pending addressed interactions to the intended invokable agent and fall back to board attention when that agent becomes ineligible or is deleted. - Preserve sandbox callback identity fields required by governed resolution paths. - Add migrations `0193` and `0194` plus route, service, attention, adapter, CLI, and UI coverage. - Add governance state and company settings UI, including responsive mobile behavior and distinct withdrawn/expired audit presentation. ## Verification - `pnpm check:token-gates` — passed. - `pnpm -r typecheck` — passed, including migration numbering and safety checks. - `pnpm test:run` — feature/server and UI workspace suites passed; one unrelated CLI AWS doctor test observed injected static AWS credentials and warned instead of passing. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts --project paperclipai` — 8 tests passed, confirming the failure was environment-sensitive. - `pnpm build` — passed. - Latest rebased head `e24cece6be9f1877bdbac7691bcb44fd583c0161` completed all GitHub CI jobs successfully. ## Risks - Migrations add interaction and company-governance fields; numbering is conflict-free on current `master`, additive statements are idempotent, and migration safety checks pass. - Agent authorization behavior expands beyond board-only resolution, but defaults remain board-only and coverage exercises company boundaries, resolver eligibility, lifecycle invalidation, wake behavior, withdrawal, expiry, and attention fallback. - Attention routing depends on current agent invokability; reconciliation and read-time filtering prevent stale addressees from retaining visibility or resolution authority. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.6-sol` with reasoning, terminal tool use, code execution, Git/GitHub integration, and Paperclip control-plane tools. Context-window metadata was not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
8e7f1c03eb |
feat(decisions): improve desk triage and queue parity (#10785)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The decisions desk and queue help operators find work that needs a human decision. > - The current views use different grouping, sorting, and labels. > - Repeated confirmation requests can also leave stale pending actions in the queue. > - Blocked-work attention can point at an intermediate issue instead of the terminal blocker. > - This pull request aligns the server contract and both user interfaces. > - The benefit is a smaller, clearer queue that ranks the decisions with the largest impact. ## Linked Issues or Issue Description Related PR: #10774 **What existing behavior does this improve?** The decisions desk and queue currently use different triage rules. They can show stale repeated confirmations and can rank blocked work by an intermediate issue. **Subsystem affected** This change affects attention aggregation, issue thread interactions, shared attention contracts, and the decisions user interface. **Current behavior** The desk uses a can-wait group that has no clear arrival meaning. The queue has fewer controls than the desk. Repeated pending confirmations remain actionable. Blocked-work rows do not always identify the terminal actionable blocker. **Proposed behavior** Group desk items by arrival date, and reserve Decide now for explicit due dates. Use one toolbar and shelf model on both pages. Supersede older repeated pending confirmations. Aggregate blocked work under the terminal actionable blocker and rank it by impact. **Reason and benefit** Operators get one consistent triage model. The badge reflects new and overdue work. High-impact blockers move to the top. Duplicate confirmation work no longer consumes attention. **Breaking changes** The attention summary field `decideNowCount` changes to `deskBadgeCount`. Consumers must use the new field. Older repeated confirmation interactions can now finish with the `superseded_by_newer_request` outcome. ## What Changed - Supersede older pending confirmation requests for the same issue and record the mutation in activity history. - Resolve blocked-work attention to actionable terminal blockers, suppress live blocker trees, and rank rows by blocked-work impact. - Group the decisions desk into New today and Earlier, and count new plus overdue work in the desk badge. - Share the decision toolbar and shelf components across the desk and queue. - Add queue grouping, sorting, filtering, aging, visible training controls, and clearer recommendation copy. - Add server, shared-contract, and user-interface tests for the new behavior. ## Verification - `pnpm check:token-gates` - `pnpm -r typecheck` - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm test:run` (all server, UI, CLI, shared, and catalog tests passed; one fixed five-second DB timeout flaked under full-suite load) - `NODE_ENV=test pnpm --filter @paperclipai/db exec vitest run src/status-card-migrations.test.ts` (passed in isolation) - `pnpm build` ## Risks - The attention summary field rename requires synchronized consumers. - Terminal-blocker traversal uses cycle and depth guards. A malformed dependency graph can stop at the last safe node. - The new arrival grouping changes which items contribute to the decisions badge. - Superseding repeated confirmations changes the terminal state of older pending interactions. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-5. The runtime did not expose a dated model snapshot or context-window size. The model used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
0a09e4d975 |
feat(decisions): add desk workflow and retention (#10672)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Decisions desk shows work that needs a human decision > - The queue foundation can group and rank decision work > - Operators also need a focused daily view and a safe way to handle old work > - This pull request adds the desk controls, the aging shelf, and reversible retention > - It also binds bulk archive decisions to the exact reviewed item set > - The benefit is a smaller daily queue without lost or orphaned work ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: server, UI, database, and shared contracts. **Problem or motivation** Decision work can grow into one large company list. Operators need quick queue and date controls. Old items also need a safe retention path that does not delete work. **Proposed solution** Add queue and date controls to the Decisions desk. Compute the aging shelf on the server. Archive idle items after 90 days unless an operator keeps them. Keep archived items searchable and revivable. Notify origin agents in one batch per sweep. Bind bulk archive proposals to a signed, exact item manifest. **Alternatives considered** Client-only aging can drift across browsers and source kinds. Deleting old rows removes audit and recovery paths. An unsigned dynamic bulk query can archive items that the reviewer did not inspect. **Roadmap alignment** This work supports the Work Queues and decision-memory directions in `ROADMAP.md`. It extends the Decisions and attention-feed foundation from #10651. Related earlier work includes #9380, #10010, and #10474. ## What Changed - Added the queue rail, date chips, decide split, triage strip, queue page, and aging shelf UI. - Added server-owned shelf state with per-queue retention overrides. - Added reversible retention state, archive history, and an idempotent notification outbox. - Added the 90-day archive sweeper, Keep exemption, archived feed query, and revive actions. - Added one origin-agent notification per agent and sweep. - Added signed bulk archive proposals with exact-set and version checks. - Persisted queue-exclusion reasons atomically and kept cross-domain source resolution per-item until it has an exact-set transaction contract. - Added API contracts, OpenAPI entries, migration coverage, focused tests, and Storybook screens. ## Verification - `pnpm -r typecheck` - `pnpm test:run` (server: 329 files and 3,453 tests passed; UI: 408 files and 3,362 tests passed) - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts` - `pnpm build` - `pnpm check:token-gates` - Focused retention, attention, decisions, migration replay, startup, and UI API tests. - Complete queue snapshot regression with 51 items across the normal 50-item page boundary. The unmodified CLI test reports one warning assertion in this runtime because the harness injects static AWS credential variables. The isolated test passes when those two variables are removed. ## Risks - The migration adds retention and notification outbox tables. It uses idempotent table, index, and foreign-key creation. - Retention runs on the heartbeat scheduler interval. Compare-and-set version checks prevent stale archive writes. - Bulk archive acceptance fails closed when authority, activity, version, or the reviewed set changes. - Archive is reversible and does not delete source records. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with `gpt-5.6-sol`. The model used tool calls, code execution, database migration generation, and test execution. The context-window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
30c49c8327 |
feat(decisions): add queues and prioritized attention feed (#10651)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source control plane for companies of AI agents. > - Operators use the attention feed to find decisions that need action. > - The feed has eleven source kinds, but it has no durable queue or triage state. > - The feed also returns every item and lacks decision deadlines, snooze state, and decision-focused ordering. > - This pull request adds secure queue sidecars and enriches the attention feed with triage data, filters, cursor pagination, and decide-now ranking. > - The benefit is a bounded feed that can show the most urgent decisions first without weakening source visibility rules. ## Linked Issues or Issue Description This pull request replaces the closed [#10634](https://github.com/paperclipai/paperclip/pull/10634). It combines that queue foundation with the dependent attention-feed change as one review unit. **Subsystem affected** Database schema, shared contracts, server authorization and REST APIs, and the UI attention client library. **Problem or motivation** The attention feed can contain hundreds of mixed decision items. Operators cannot group them into durable queues, set a decision deadline, snooze an item, or request a bounded page ordered by urgency. The current client must download the full feed on each refresh. **Proposed solution** Store queue membership and triage state by stable attention identity. Re-authorize each source during queue reads and writes. Enrich attention items with queue, deadline, snooze, expiry, rule, and origin data. Add activity and queue filters, opaque cursor pagination, decide-focused ordering, and a decide-now count. **Alternatives considered** Adding queue fields to every source would duplicate schema and authorization logic across eleven source kinds. Client-only filtering and sorting would still transfer the full feed and would make pagination unstable. **Roadmap alignment** This change improves the core decision-attention surface and operator oversight. It does not implement the separate general-purpose work queue milestone in `ROADMAP.md`. ## What Changed - Added company-scoped queue, membership, triage, and append-only event tables with actor and run provenance. - Added queue CRUD, item membership, starter-rule discovery, and decide-by and snooze endpoints. - Kept source authorization on each queue mutation, read, and count. - Added attention fields for expiry, rule, origin agent, queues, decide-by attribution, and snooze state. - Added activity date filters, queue filters, opaque cursor pagination, and configurable page limits. - Added decide-now ordering by deadline, expiry, severity, and activity. - Added `decideNowCount` and excluded actively snoozed items from the default feed. - Updated the shared and UI client contracts. - Added focused server, route, OpenAPI, and UI client tests. ## Verification - `pnpm exec vitest run server/src/__tests__/attention-service.test.ts server/src/__tests__/decision-queues-routes.test.ts server/src/__tests__/openapi-routes.test.ts ui/src/api/attention.test.ts ui/src/lib/attention.test.ts` (72 tests passed) - `pnpm --filter @paperclipai/db check:migrations` - `pnpm -r --filter @paperclipai/db --filter @paperclipai/shared --filter @paperclipai/server --filter @paperclipai/ui typecheck` - `git diff --check origin/master...HEAD` ## Risks - The migration adds four company-scoped tables and provenance foreign keys. Migration numbering and safety checks pass. - Attention reads can lazily create starter queues and memberships. Inserts are idempotent, audited, and transactional. - Cursor validity depends on the filtered feed. The API returns a clear validation error when the cursor item no longer exists in that feed. - Queue reads re-check source visibility. This favors correct authorization over fewer queries. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, model `gpt-5`. The runtime used agentic reasoning, repository tools, code execution, and test execution. The runtime did not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
492555aaf9 |
design(decisions): flatten decision cards to two task-borrowed types (#10474)
The Decisions queue ran five parallel colour/icon vocabularies chosen by source kind, plus a separate severity badge, so two rows needing the same response could look unrelated and none of it matched the task list. Every row now resolves to one of two kinds, each borrowing the task status it corresponds to: blocking renders as `blocked`, review as `in_review`, both through StatusGlyph and the existing --status-task-icon-* tokens. Source kinds keep their own wording; only colour and icon merge. Card anatomy follows the design mock: no left accent rail, rounded cards 16px apart, a "/"-separated meta breadcrumb, a named See more / See less control, and no separately tinted drawer when expanded. Verb order is fixed across both states. Severity moves from chrome to a toolbar filter. Four defects fixed along the way: - blocked rows reported themselves as their own blocker (server-side) - the task key was missing wherever the row's subject IS the task - the task quicklook stuck open, because closing handed focus back to a trigger that opens on focus - the card ring appeared on click, and only on cards with a toggle Also: the standard task preview is aligned to its trigger's text and scales out of it, the task eyebrow renders its project as a tile, and the first motion tokens land alongside the disclosure and crossfade. Supersedes #9574 and #9575. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
148a5b11f5 |
Route blocked transitions to explicit unblock owners (#10112)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies. > - Issue status transitions determine whether work keeps moving or silently stalls. > - A blocked issue previously could rely on prose alone, leaving the intended unblock owner unstructured and unnotified. > - Existing blocker-attention classification could identify stalled chains, but the signal was not delivered to the board attention feed. > - Blocked transitions also need rollout-safe deduplication so upgrades do not notify for historical issues and repeated processing does not create notification storms. > - This pull request adds structured unblock descriptors, prospective transition timestamps, owner delivery, and board attention routing with focused authorization controls. > - The benefit is that newly blocked work has an explicit, routable next action without weakening company boundaries or allowing agents to inject arbitrary human attention items. ## Linked Issues or Issue Description Related documentation PR: #10094. ### Subsystem affected Cross-cutting: `server/`, `packages/db`, and `packages/shared`. ### Problem or motivation An issue can enter `blocked` without a machine-readable unblock path. Prose-only ownership does not reliably wake the responsible agent or surface human-owned work, while the existing `blockerAttention` classifier is not delivered to an operator-facing attention feed. ### Proposed solution Require new transitions into `blocked` to have unresolved blockers, a pending interaction/approval, or a structured `{ owner, action }` descriptor. Notify an allowed owner once per prospective transition, route human-owned cases to board attention, and leave pre-rollout blocked issues untouched. ### Alternatives considered - Keep prose-only blockers: rejected because ownership remains unroutable. - Backfill all historical blocked issues: rejected because upgrades would create notification storms. - Let agents target arbitrary users or the board: rejected after security review because it creates an attention-injection channel. ### Roadmap alignment Aligns with `ROADMAP.md` → “Enforced Outcomes (watchdogs, recovery actions, review gates)” by making blocked work carry an explicit continuation path. ### Additional context The implementation is prospective-only and deduplicated per blocked transition. Agent-authored descriptors are limited to the acting agent; board actors retain human-owner routing. ## What Changed - Added persisted unblock descriptors and prospective blocked-transition delivery timestamps with an idempotent migration. - Added shared types and validation for board, user, and agent unblock owners. - Enforced valid blocked transitions and same-company owner validation in the issue update route. - Restricted agent-authored descriptors to the acting agent itself, preventing board/user attention injection by compromised agents. - Added one-per-transition agent wake delivery and prospective-only rollout gating. - Routed human-owned blocker attention into the board attention feed. - Added focused tests for validation, prospective delivery, flap deduplication, attention routing, route authorization, and stop-relay compatibility. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts server/src/__tests__/routable-blocked.test.ts server/src/__tests__/attention-service.test.ts packages/shared/src/validators/issue.test.ts` - `AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= pnpm test:run` - `pnpm build` - `pnpm --filter @paperclipai/db check:migrations` ## Risks - Behavioral shift: new `blocked` transitions without a real blocker, pending governed action, or structured descriptor now return `422`. - Notification abuse is constrained by same-company validation, agent self-only routing, prospective rollout gating, and transition-scoped deduplication. - Migration risk is low: columns are additive, nullable, and use `IF NOT EXISTS`; historical blocked issues are not backfilled or notified. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI with GPT-5.4, reasoning-enabled tool use and code execution. The runtime did not expose a context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
051ae4d102 |
feat: restore decision training library and inspector (#9779)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and supervise governed work. > - Decisions capture high-value operator judgment, and the decision-training foundation merged in #9702 freezes that evidence for later evaluation and learning. > - Operators still need the UI from the closed stacked PR #9718 to intentionally capture examples and inspect the resulting dataset. > - GitHub automatically closed #9718 when its stacked base branch was deleted after #9702 merged, leaving the server foundation on `master` without the corresponding UI. > - This pull request restores the final UI and its still-required supporting API fields directly on current `master`, while excluding the obsolete migration and duplicated server-foundation diffs. > - The benefit is a reviewable replacement PR that preserves the completed decision-training workflow without replaying stale stack history. ## Linked Issues or Issue Description - Refs #9718 - Refs #9702 ## What Changed - Restored the top-level `/training` library and record inspector with search, filters, JSONL export, notes editing, and evidence tabs. - Restored the Decisions-row training affordance and capture drawer, including preview, provenance, deletion, cache refresh, and approval consistency behavior. - Restored the shared types and focused server support needed by the UI without reintroducing decision-training migrations or the already-merged server foundation. - Restored focused UI and attention-service tests from the final #9718 state. - Credit to the authors and reviewers of #9718; this recovery transplants their final reviewed delta after the stacked base deletion. ## Verification - `pnpm exec vitest run ui/src/pages/Training.test.tsx ui/src/components/DecisionTrainingDrawer.test.tsx ui/src/components/AttentionQueueRow.test.tsx server/src/__tests__/attention-service.test.ts server/src/__tests__/decision-training.test.ts` — 5 files, 48 tests passed. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts` — 3 tests passed; confirms exact route/OpenAPI parity for the restored preview endpoint. - `pnpm check:token-gates` — the restored files are clean; the repository-wide command currently reports five pre-existing false positives where comments reference GitHub issue `#9627` as if it were a color literal. ## Risks - Low migration risk: this PR contains no database migrations and is based directly on current `master`. - The main behavioral risk is cache invalidation across Decisions and Training views; focused tests cover capture, update, deletion, row state, and approval refresh behavior. - The token-gate baseline remains red on unrelated `#9627` comment references; this PR does not modify those files. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.3-Codex, reasoning with repository/tool access and code execution. Context window not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
36ec79c196 |
feat: add attention queue and Decisions surface (#9380)
## Thinking Path > - Paperclip is the control plane for autonomous AI companies, where operators need a reliable way to find and act on work awaiting their input. > - The attention and issue-thread interaction subsystems expose those decision points across server APIs and the board UI. > - The previous navigation and interaction presentation left these actions fragmented and did not offer a controlled rollout for the Decisions surface. > - This branch adds the attention feed, richer interaction cards, grouping, dismiss/snooze behavior, and a gated Decisions sidebar entry. > - It also keeps experimental settings and API contracts synchronized, with an idempotent migration for the new dismissal state. > - This pull request delivers the complete, tested attention/Decisions experience as one reviewable unit. ## Linked Issues or Issue Description - Adds an operator-focused attention queue and Decisions experience: grouped decision cards, semantic interaction actions, dismiss/snooze handling, resilient interaction states, and an experimental flag to control the Decisions navigation entry. ## Feature Context ### Problem or Motivation Operators currently have to hunt across approvals, interactions, failed runs, and budget alerts to find decisions that need their action. ### Proposed Solution Provide a gated Decisions attention queue that groups actionable items, supports direct resolution, and preserves operator control through dismiss and snooze actions. ### Alternatives Considered Keep separate, source-specific views only; this leaves cross-cutting operator decisions fragmented and harder to prioritize. ### Roadmap Alignment This improves the V1 control-plane operator workflow by making pending governed actions discoverable in one company-scoped surface. ## What Changed - Added server attention-feed services, routes, interaction handling, dismiss/snooze support, and an idempotent `0145` inbox-dismissal migration. - Added shared attention, inbox-dismissal, and experimental-settings contracts. - Added Decisions/attention UI, interaction-card states, sidebar badge/navigation integration, grouping, keyboard support, and Storybook coverage. - Added tests for attention behavior, thread interactions, settings normalization, dismissals, and API behavior. - Removed generated screenshots from the final PR diff and rebased the branch onto current `master`. ## Verification - `pnpm check:token-gates` — passed. - `pnpm exec vitest run packages/shared/src/issue-thread-interactions.test.ts server/src/__tests__/attention-service.test.ts server/src/__tests__/inbox-dismissals.test.ts server/src/__tests__/issue-thread-interactions-service.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts ui/src/lib/attention.test.ts ui/src/components/AttentionQueueRow.test.tsx ui/src/components/IssueThreadInteractionCard.test.tsx ui/src/pages/InstanceExperimentalSettings.test.tsx` — passed: 158 tests across 9 focused files. - GitHub Actions for `ad636f560`: build and typecheck/release-registry have passed; remaining general-server and Greptile checks are in progress. ## Risks - Moderate: this is a cross-layer attention/interaction feature with a new migration and navigation behavior. - The `enableDecisions` experimental setting defaults to off, limiting rollout impact. - Existing dismissal data is backfilled to `dismiss`; the migration is idempotent and uses guarded constraint creation. > ROADMAP.md was checked; no duplicate planned core feature was identified. Related open pull requests were searched before opening this PR. ## Model Used - OpenAI GPT-5.5 via Codex CLI, with tool use and local code execution. Context-window size unavailable in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My public PR branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally; focused tests pass and the remaining unrelated AWS test failure is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |