DottaandPaperclip
d5766e4074
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Standing chats show recent messages and current task questions. > - Recent-message pagination bounds history work, but it does not bound question work. > - The question read built the full company attention feed before it selected the chat's tasks. > - Each feed read also checked seeded queue membership with one insert per item. > - This pull request scopes question collection in the attention owner and batches queue membership checks. > - Chats keep their current question cards and initial layout while doing less work. ## Linked Issues or Issue Description Follow-up to #18358 and #18323. Related work: #18312. **What happened?** A hosted standing chat remained slow after recent-message pagination shipped. On commit `4eaa435bf4`, an embedded-browser reload made the conversation usable at about 8.6 seconds. Recent comments finished at about 1.9 seconds. The pending-question request held the initial reveal until about 8.4 seconds. That request called the complete company attention feed and then filtered its results. The separate badge summary also did broad work. **Expected behavior** Chat question cost should depend on the relevant tasks. Reading an existing queue must not require a separate insert for every member. Current access and decision policy must remain in force. **Steps to reproduce** 1. Create a standing chat with questions on tasks assigned to its agent. 2. Add many unrelated tasks and pending questions to the company. 3. Open the chat from a fresh application cache. 4. Compare recent-comment arrival with the pending-question response and content reveal. **Paperclip version or commit** Baseline: `909e29dd071b1aeeb24c2933af049da90a6066d3`. Its affected services match the observed hosted revision `4eaa435bf4`. ## What Changed - Add an internal task-question projection to the attention service. Filter pending interactions by company, task IDs and question kind before card enrichment. - Keep the same access, audience, dismissal, queue, snooze, retention, archive and training-example logic. Ranks are local to the returned question list. - Use the projection from the conversation receipt service. Keep current chat audience and session checks there. - Batch seeded membership inserts in groups of 500. Keep the queue lock, conflict protection, audit events and activity logging for each new membership. Existing members produce no new audit records. - Add real PostgreSQL regression cases and a reproducible paired benchmark. No schema migration or UI reveal change is needed. ## Verification - `pnpm -r typecheck` passed. The final server typecheck also passed after the queue change. - `pnpm build` passed. - Real PostgreSQL service coverage: 57 attention and chat-handoff tests passed. Seven queue-route tests passed, including concurrent reads, mixed existing/new items, duplicates, disabled seeds and audit preservation. - Red/green proof: the unrelated-attention regression fails with the baseline conversation implementation and passes with the scoped read. - `node scripts/check-module-boundaries.mjs` and `git diff --check` passed. - Full `pnpm test:run` was attempted and stopped with exit 130 after embedded PostgreSQL startup failures and 17 failures in the untouched native-session executor suite. It is not a local pass. The focused Docker/PostgreSQL results above completed before Docker Desktop became unavailable. A direct native PostgreSQL probe identified the host semaphore limit (`could not create semaphores: No space left on device`). No shared kernel settings or other test clusters were changed. [Full CI](https://github.com/paperclipai/paperclip/actions/runs/41162314995) passed on `2809f096b3fb2c01b44a96a1b82a27809c9359df`, including all eight browser shards. Final status: 55 successful checks/statuses, three skipped and one neutral. Greptile is 5/5 with no findings or unresolved threads. The fresh local browser retest is blocked because Docker Desktop cannot start and native PostgreSQL cannot allocate semaphores. The original hosted reproduction above is complete; no candidate browser pass is claimed. Local service benchmark: PostgreSQL 17.11, two relevant tasks, five alternating pairs per scenario, after warmup. These are local service timings, not a claim about deployed latency. | Fixture | Read | Baseline median (range) | Candidate median (range) | SQL calls | | --- | --- | --- | --- | --- | | 1,000 unrelated tasks/questions | Chat questions | 670 ms (632–1,905) | 12 ms (11–15) | 1,076 → 22 | | 1,000 unrelated tasks/questions | Company badge summary | 641 ms (622–717) | 167 ms (161–171) | 1,068 → 69 | | Small company | Chat questions | 38 ms (37–52) | 13 ms (12–16) | 52 → 22 | | Small company | Company badge summary | 34 ms (33–39) | 34 ms (32–40) | 44 → 43 | All question fields and ordering matched, except rank is relative to the scoped list. Summary fields matched except the generation timestamp. The interaction plan changed from 1,002 selected rows to two rows through the existing company/issue/status index. The benchmark loads baseline attention, conversation and queue modules together so the queue change cannot contaminate the baseline. ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-questions.ts \ --baseline-ref=909e29dd07 --unrelated=1000 ``` ## Risks - Scoped and complete feeds must share policy. This change reuses the same attention collection and finalization path and tests dismissals, named audiences, snoozing, archives and private access. - Batched inserts must audit only new members. The concurrent PostgreSQL test checks unique membership and one event/activity record per new item across batch boundaries. - Full inbox summaries still collect the company feed. The small-company summary timing is unchanged. Other chat requests and transcript hydration can still contribute to hosted latency. - The first reveal still waits for question cards to preserve layout. This follow-up has not been deployed, so its hosted effect remains unverified. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution and browser profiling. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
4.6 KiB
Long-running agent chats
agent-chat-startup.spec.ts drives the real standing-chat route with a retained
600-message, 300-run synthetic conversation. Completed runs have valid native
assistant events; there are no model calls. It verifies recent-only startup,
one backward page per upward gesture, delayed/error recovery, an unobscured latest reply, retained rows and
drafts, Back, older links, one saved live message, and sending before the first upward scroll. The send check uses separate retained fixture files per browser project so queued board input cannot alter the main history dataset. The live-message check posts one reply with a temporary key for the fixture agent, then revokes it. It starts no agent execution. Delayed inline images verify both following the latest reply and retaining an unpinned reading anchor. Additional checks keep message paging available during a receipts outage and verify that successive pages request each newly revealed metadata id once and avoid rereading previously hydrated terminal streams. Native stream recovery still refreshes on retry, resume, mode change, or reconnect. The disposable board user dismisses the current optional announcement before the journeys.
Use a disposable loopback instance with Agent Chat enabled. The first seed needs
its matching PostgreSQL URL, with a database name ending in _test:
PAPERCLIP_ISSUE_PERF_BASE_URL=http://127.0.0.1:3221 \
PAPERCLIP_CHAT_PERF_DATABASE_URL=postgres://paperclip:paperclip@127.0.0.1:55503/fast_chat_preview_test \
pnpm exec playwright test --config tests/perf/agent-chat/playwright.config.ts
The recipe enables Streamlined UI and the disposable board user's Coordinator
preference. It creates isolated fixture records through the API and database;
never point it at a shared instance. The fixture is retained at
test-results/agent-chat/fixture-600.json. Override
PAPERCLIP_CHAT_PERF_SEED_FILE to reuse an existing matching fixture; retained
fixtures need no database connection for browser journeys.
PAPERCLIP_CHAT_PERF_MESSAGES=6000 creates a larger fixture for the same assertions.
The rig blocks service workers so fixture responses are intercepted deterministically and build-update reloads cannot change the selected UI artifact. The ordinary local prototype keeps the normal service worker enabled.
The rig runs the same journeys on desktop and mobile. Use --project=desktop
for the smaller iteration loop, then run both projects before handoff.
PAPERCLIP_TASK_COLD_OPEN_DATABASE_URL lets the focused server suite reuse a
separate disposable loopback database ending in _test; it applies migrations
and retains randomly scoped fixture records until that database is discarded.
The normal CI path still uses the embedded PostgreSQL helper.
Measure startup latency separately from these assertions: use matched production builds, the same retained fixture and server, alternate baseline/candidate order, and stop this task's builds and suites during timing samples. Readiness is the visible recent conversation at its intended position with an editable composer, not the loading shell. Retain per-frame geometry and server timings alongside sample medians/ranges; do not treat fewer transcript requests as latency proof.
The chat's pending task questions use the attention service's scoped question projection. It keeps current access, audience, dismissal, queue, snooze and archive rules without constructing the company feed. Ranks are relative to the returned questions. Seeded queue membership checks use bounded batches on both scoped questions and the company feed. New memberships keep their existing audit path. The first reveal still waits for these cards so their arrival does not move an already visible conversation.
Compare the full service against a pinned commit before this optimization:
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-questions.ts \
--baseline-ref=909e29dd07 --unrelated=1000
The benchmark creates a disposable PostgreSQL database with two relevant tasks
and the requested number of unrelated blocked tasks and questions. It alternates
five baseline/candidate pairs for chat questions and the company badge summary,
compares all returned fields except contextual rank and the generated timestamp,
counts SQL calls and captures the actual interaction query plans.
PAPERCLIP_CHAT_QUESTIONS_TEST_DATABASE_URL can reuse a dedicated database ending
in _test. PAPERCLIP_COORDINATOR_HANDOFF_TEST_DATABASE_URL provides the same
option for the canonical handoff service tests. These paths retain synthetic
fixtures when an external test database is used.