mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-21 02:43:25 +01:00
f4a3f85b89b2dae7889b0a9c7fee07ed47b554cb
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
a097949864 |
fix(server): stop issue reference syncs from deadlocking on mutual mentions (#22389)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server keeps a table of issue reference mentions. It rebuilds the rows for an issue when its title or description changes. > - `syncIssue` locks the source issue row `FOR UPDATE` before it rebuilds the rows (added in #13898). > - Each mention insert does a foreign-key check. That check takes a `FOR KEY SHARE` lock on the target issue. `FOR KEY SHARE` conflicts with `FOR UPDATE`. > - When issue A mentions B and issue B mentions A, two concurrent syncs each hold one `FOR UPDATE` lock and wait for the other. Postgres reports `deadlock detected`, and one update fails. We saw this on staging. > - This pull request changes the lock to `FOR NO KEY UPDATE`. > - The benefit is that syncs of one issue stay serialized, and foreign-key checks from other issues do not wait on that lock. ## Linked Issues or Issue Description Refs #13898 **What happened?** A staging server reported `Failed query: insert into "issue_reference_mentions" ...` caused by `PostgresError: deadlock detected`. The insert came from `replaceSourceMentions` inside the `syncIssue` transaction. The source issue description mentioned the target issue. **Steps to reproduce** 1. Create issue A with a description that mentions issue B. 2. Create issue B with a description that mentions issue A. 3. Run `syncIssue(A)` and `syncIssue(B)` at the same time, several times. 4. One sync fails with `deadlock detected`. **Expected behavior** Both syncs complete, and each issue has its mention rows. ## What Changed - `server/src/services/issue-references.ts`: `syncIssue` now locks the source issue with `FOR NO KEY UPDATE`, not `FOR UPDATE`. `FOR NO KEY UPDATE` still conflicts with itself, so concurrent syncs of one issue stay serialized (the #13898 behavior). It does not conflict with the `FOR KEY SHARE` lock of a foreign-key check, so the lock cycle cannot form. - `server/src/__tests__/issue-references-service.test.ts`: two regression tests. - One test holds a sync lock in an open transaction. It then syncs a second issue that mentions the locked issue, and expects that sync to complete without a wait. - One test runs 20 concurrent syncs of two issues that mention each other, and expects all four mention rows. ## Verification - `cd server && npx vitest run src/__tests__/issue-references-service.test.ts` - Before the fix (old lock): the two new tests fail. One test shows `expected 'blocked' to be 'completed'`. The other test fails with `PostgresError: deadlock detected` on `select ... from "issues" ... for update`. This is the same error as the staging event. - After the fix: 5/5 tests pass. I ran the file three times and all runs passed. The existing test "serializes concurrent syncs for a description with repeated references" still passes. ## Risks - Low risk. There is no schema change or migration. The lock mode changes only for this read. `UPDATE` statements that do not change key columns already take `FOR NO KEY UPDATE`, so this lock blocks the same issue writers as before. Only foreign-key `KEY SHARE` checks can now continue at the same time. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in a Claude Code agent session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
30c4d2f4a1 |
perf(server): read the issue detail with fewer database statements (#21713)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI loads the task page through `GET /api/issues/:id`, and
the server runs all requests on one Node main thread
> - A CPU profile of the server shows that the cost is per database
statement: SQL building, parameter encoding, result parsing and one
socket write each
> - The issue detail read sent 73 statements for a task with two
ancestors, one blocker and a project mention
> - Several of those statements read the issue row again, only to get
values that the route already loaded
> - This pull request passes the loaded issue to those reads, walks the
ancestor chain in one query, and skips reads whose result the response
does not use
> - The benefit is 9 fewer statements on each task page load (73 to 64),
plus one fewer statement for each extra ancestor level, with a
byte-identical response
## Linked Issues or Issue Description
**What existing behavior does this improve?**
`GET /api/issues/:id` reads the same issue row several times in one
request, and walks the ancestor chain with one statement for each level.
**Subsystem affected**
Server: issue routes and the issue, issue reference and project
services.
**Current behavior**
The route loads the issue, then `getRelationSummaries`,
`getCurrentScheduledRetry`, `findMentionedProjectIds`,
`listIssueReferenceSummary` and `getAncestors` each read the issue row
again. `getAncestors` sends one statement for each ancestor. The inbox
archive fields read two activity aggregates even when the issue has no
archive row. The project embed reads the workspace provider catalog, but
the compact embed does not return it. The test fixture request sends 73
statements.
**Proposed behavior**
The route passes the values it already has. The ancestor walk is one
recursive query. The activity aggregates run only when an archive row
exists. The project embed skips the provider catalog. The same fixture
request sends 64 statements, and the JSON response is the same.
**Reason and benefit**
Fewer statements per request means less CPU on the main thread for every
open task page.
**Breaking changes**
None. Other callers of the changed service functions keep the old
behavior, because the new arguments are optional.
## What Changed
- `issueService.getRelationSummaries` and `getCurrentScheduledRetry`
accept an optional known `companyId` and then skip the issue read.
- `issueService.findMentionedProjectIds` accepts the loaded issue
(`companyId`, `title`, `description`).
- `issueService.getAncestors` accepts the known `parentId` and walks the
chain in one recursive query. It keeps the order (nearest first), the
limit of 50, and the stop at a cycle or at a task that is its own
parent.
- `issueService.getActiveInboxArchiveFields` reads the archive row first
and reads the activity aggregates only when an archive row exists.
- `issueReferenceService.listIssueReferenceSummary` accepts an optional
known `companyId`.
- `issueThreadInteractionService.listForIssue` accepts the loaded issue
(`status`, `companyId`, `executionWorkspaceId`). `GET
/api/issues/:id/interactions`, which the board polls, passes it (11 to
10 statements).
- `projectService.getById` accepts `{ workspaceProvider: false }`. The
issue detail route uses it, because `compactIssueProject` never returns
`workspaceProvider`.
- Test helpers: `createStatementCensus` (statements per request, grouped
by statement shape with the server callers), and a board route fixture
with the real session actor middleware. `meterDatabaseStatements` can
record a call stack for each statement.
- New tests: `issue-detail-statements.test.ts` (statement budget,
byte-identical normalized JSON against a fixture file, access rules,
membership removal on the next request, ancestor walk with a cycle) plus
an interactions poll test, and `board-route-statement-census.test.ts`
(census of the polled and page-load board routes).
## Verification
- Red: before the change, the fixture request sent 73 statements, and
the budget test failed (`expected 73 to be less than or equal to 64`).
- Green: after the change, the request sends 64 statements. The
normalized JSON is byte-identical to the fixture file that I wrote
before the change.
- Access: another company's user gets 404, a missing issue gets 404, an
anonymous request gets 404, a hidden issue stays readable for a member,
and a removed membership gets 404 on the very next request.
- `pnpm exec vitest run` on `issue-detail-statements`,
`board-route-statement-census`, `issues-goal-context-routes`,
`issues-service`, `issue-references-service`, `inbox-archive-routes`,
`heartbeat-task-context`, `project-workspace-provider.integration`,
`workspace-provider-failure`, `issue-telemetry-routes`,
`issue-thread-interaction-routes`, `issue-document-restore-routes`,
`heartbeat-referenced-projects`, `project-environments`: 14 files, 414
tests pass.
- `pnpm --filter @paperclipai/server typecheck` passes.
- Interactions: the polled route sends 10 statements instead of 11, with
a response equal to the service result without the known issue.
- Census: `PAPERCLIP_STATEMENT_CENSUS=/tmp/census.txt pnpm exec vitest
run src/__tests__/board-route-statement-census.test.ts` writes
statements per request and the top statement shapes for each board
route.
## Risks
- Low risk. The new arguments are optional, so other callers keep the
old reads.
- When the route passes a known `companyId`, a concurrent delete of the
issue no longer gives a 404 from these sub-reads. The route already
loaded and authorized the issue in the same request, so the response is
the same in practice.
- The JSON fixture file fails when someone changes the issue detail
response. Regenerate it with `PAPERCLIP_WRITE_ISSUE_DETAIL_GOLDEN=1` and
review the diff.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
1b2632facc |
fix: serialize concurrent issue reference syncs (#13898)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server keeps issue references synchronized with issue title and description changes. > - Concurrent updates can rebuild the same source issue's reference mentions at the same time. > - The replacement writes can then collide on the unique mention key and return a 500 error. > - This pull request serializes each source issue's reference rebuild and adds concurrent regression coverage. > - The benefit is that repeated unchanged descriptions complete without duplicate-key failures. ## Linked Issues or Issue Description Fixes #13731 ## What Changed - Lock the source issue row before replacing title and description reference mentions. - Add a 20-way concurrent regression test with repeated `COM-2` references. ## Verification - `pnpm exec vitest run server/src/__tests__/issue-references-service.test.ts packages/shared/src/issue-references.test.ts` passed with 10 tests. - `pnpm --filter @paperclipai/server typecheck` passed under Node 22.22.1; the repository declares Node >=24.11 and emitted engine warnings. - `git diff --check` passed. ## Risks - The row lock serializes rebuilds for one source issue and may reduce concurrency for simultaneous updates to that issue. - A theoretical lock-order risk remains for concurrent syncs of different mutually referencing source issues. That path is outside this fix and is not covered by the focused regression. - Remote CI and Greptile review have not run yet. ## Model Used OpenCode runtime, model `supercodes5/gpt-5.6-luna`; context window size is not exposed. Repository tool use and reasoning were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
7f70759e61 | Redact deleted issue comments | ||
|
|
b7545823be |
[codex] Add document annotations and comments (#6733)
## Thinking Path > - Paperclip orchestrates AI-agent companies through issues, documents, runs, and durable company-scoped state. > - Issue documents are where agents and operators capture plans, handoffs, and work products. > - Before this change, document collaboration could only happen through whole-document edits and detached issue comments. > - Inline document annotations need stable anchors, revision-aware persistence, and UI affordances that do not break existing document editing. > - This pull request adds company-scoped document annotation threads, comments, anchor snapshots, API routes, and board UI. > - The benefit is that operators and agents can discuss specific document passages without losing context as documents evolve. ## What Changed - Added document annotation tables, schema exports, shared types, validators, anchor hashing, and text-anchor helpers. - Added server-side document annotation services and issue routes for listing, creating, commenting, resolving, and reopening annotation threads. - Included annotation summaries in relevant issue document reads and backup/recovery document workspace behavior. - Added React UI for inline document highlights, comment panels, mobile sheet behavior, deep-link focus, and resolved/open filtering. - Added annotation design artifacts, Storybook coverage, screenshots, and a screenshot helper script. - Rebased the branch onto current `paperclipai/paperclip` `master` and renumbered the annotation migration from `0085_old_swarm` to `0091_old_swarm`; the SQL uses `IF NOT EXISTS` guards so environments that previously applied the old migration number can safely apply the new one. - Adjusted the new annotation UI tests to use a local async flush helper because this workspace's React 19.2.4 export does not expose `React.act`. ## Verification - `pnpm run preflight:workspace-links && pnpm exec vitest run packages/shared/src/document-anchors.test.ts server/src/__tests__/document-annotation-routes.test.ts server/src/__tests__/document-annotations-service.test.ts ui/src/components/DocumentAnnotationLayer.test.tsx ui/src/components/IssueDocumentAnnotations.test.tsx ui/src/lib/document-annotation-hash.test.ts ui/src/lib/document-annotation-selection.test.ts` - Confirmed `git diff --check` passes. - Confirmed no `pnpm-lock.yaml` or `.github/workflows/*` files are included in the PR diff. ## Risks - Medium risk: this adds new persisted annotation tables and routes across db/shared/server/ui. - Migration risk is reduced by moving the branch migration to `0091_old_swarm` after upstream `0090_resource_memberships` and keeping the SQL idempotent for old `0085_old_swarm` adopters. - UI risk is mostly around text range anchoring and panel positioning across long documents, folded content, and mobile layouts; the PR includes focused unit coverage and design screenshots. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-using software engineering mode. Context window size is not exposed in this Paperclip runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
ab9051b595 |
Add first-class issue references (#4214)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Operators and agents coordinate through company-scoped issues, comments, documents, and task relationships. > - Issue text can mention other tickets, but those references were previously plain markdown/text without durable relationship data. > - That made it harder to understand related work, surface backlinks, and keep cross-ticket context visible in the board. > - This pull request adds first-class issue reference extraction, storage, API responses, and UI surfaces. > - The benefit is that issue references become queryable, navigable, and visible without relying on ad hoc text scanning. ## What Changed - Added shared issue-reference parsing utilities and exported reference-related types/constants. - Added an `issue_reference_mentions` table, idempotent migration DDL, schema exports, and database documentation. - Added server-side issue reference services, route integration, activity summaries, and a backfill command for existing issue content. - Added UI reference pills, related-work panels, markdown/editor mention handling, and issue detail/property rendering updates. - Added focused shared, server, and UI tests for parsing, persistence, display, and related-work behavior. - Rebased `PAP-735-first-class-task-references` cleanly onto `public-gh/master`; no `pnpm-lock.yaml` changes are included. ## Verification - `pnpm -r typecheck` - `pnpm test:run packages/shared/src/issue-references.test.ts server/src/__tests__/issue-references-service.test.ts ui/src/components/IssueRelatedWorkPanel.test.tsx ui/src/components/IssueProperties.test.tsx ui/src/components/MarkdownBody.test.tsx` ## Risks - Medium risk because this adds a new issue-reference persistence path that touches shared parsing, database schema, server routes, and UI rendering. - Migration risk is mitigated by `CREATE TABLE IF NOT EXISTS`, guarded foreign-key creation, and `CREATE INDEX IF NOT EXISTS` statements so users who have applied an older local version of the numbered migration can re-run safely. - UI risk is limited by focused component coverage, but reviewers should still manually inspect issue detail pages containing ticket references before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent, tool-using shell workflow with repository inspection, git rebase/push, typecheck, and focused Vitest verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: dotta <dotta@example.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |