mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat queue chooses which queued run starts when an agent at its concurrency limit frees a slot > - The queue puts runs a person asked for in a "person" tier ahead of agent "work" > - Subtasks of a person's task, and the parent's resume after its children complete, ranked as agent work > - Under load these runs started only through the 60-minute oldest-run share, so a half-finished task waited about 55 minutes per step behind newer tasks > - This pull request puts every non-recovery run on a task tree a person created in the person tier > - The benefit is that a person's started work finishes before newer work starts, and the fairness bounds stay in place ## Linked Issues or Issue Description **What existing behavior does this improve?** The order in which an agent at its concurrency limit starts queued runs. **Subsystem affected** Server heartbeat queue: `server/src/services/heartbeat/queued-run-dispatch-order.ts` and `queued-run-line.ts`. **Current behavior** A board user's new assignment ranks as person work. An agent's subtask of that person's task, and the parent's `issue_children_completed` resume, rank as agent work. In a restart chaos run with agents at capacity, the median queue wait of a parent resume was about 1,640 s. The median wait of a new assignment was about 190 to 290 s. One parent waited while 14 newer tasks started first. Subtasks and parent resumes started only near the 60-minute aging bound. **Proposed behavior** A non-recovery run on a task tree that a person created goes in the person tier. A task is in such a tree when the task, or any task above it, has `createdByUserId`. In the tier, runs keep the existing order: readiness, then priority, then age (oldest first). **Reason and benefit** A person waits on the whole tree, not only on the top task. Work that a person started now finishes before newer tasks take its slots. **Breaking changes** None. There is no schema or API change. ## What Changed - `queuedRunDispatchClass` puts a run with `personTask` in the person tier. System recovery stays in the recovery tiers, also on a person's tree. - `personTask` now means "the task, or a task above it, was created by a person". The existing `createdByUserId` marker is the source. - New `listPersonTreeIssueIds`: one recursive query per dispatch pass. It walks each queued task's parent chain through the `(company_id, id)` unique index. It stops at the first person-created task, at most 32 levels. It runs only when the agent has more than one queued run. The start-history read uses the same query in place of its old direct `createdByUserId` read, so the query count does not change there. - Recovery's 1-in-4 share is held only while a run a person asked for (`isPersonRequestedRun`) is ready, not for the rest of a person's tree. So a deep tree cannot hold recovery indefinitely. The dispatcher and the place-in-line replay use the same rule. - The claim backoff for a declined claim keeps the short person backoff only for runs a person asked for (unchanged behavior for tree runs). - `doc/heartbeat-internals.md`: new "Queued run dispatch order" section with the tiers, the order and the fairness bounds. Fairness bound: the oldest-run share is unchanged. Every 4th start goes to the longest-waiting ready run past 60 minutes, preferring runs people wait on. So another person's new task starts within 60 minutes plus 4 x (aged person-tier runs queued before it + 1) starts, even while a deep tree keeps adding in-progress or higher-priority work. A newer tree run never passes an older, equally ranked person task. Queues are per agent, and an agent belongs to one company, so no agent or company gets ahead of another. ## Verification - Red on master, green with this change: - `queued-run-dispatch-order.test.ts`: an older parent resume (`issue_children_completed`, person tree) starts before a newer person assignment. The place-in-line replay agrees. - `queued-run-dispatch-order.test.ts`: an older subtask of a person's task starts before a newer person assignment. - `queued-run-dispatch-order.test.ts`: fairness simulation. A deep tree adds two in-progress, higher-priority runs per start. Another person's new task starts within `QUEUED_RUN_AGING_MS` + `OLDEST_DISPATCH_SHARE` starts. - `execution-projection-agent-queue.test.ts` (embedded Postgres, real SQL): line is subtask (grandchild of a person's task), parent resume, newer person assignment, then work on an agent-created tree. Master gives the person assignment first and the tree runs last. - Negative and guard tests (green on both): work on an agent-created tree stays in the work tier behind a newer person assignment; system recovery on a person's tree stays in the recovery tier; newer tree work does not pass an older equally ranked person task; recovery still gets its share while only tree work waits. - Query plan on embedded Postgres with 300,000 issues and 200 queued tasks six levels deep: every step is an index scan on `issues_company_id_uq`, 1.5 ms execution. - `pnpm verify` exited 2 (a doc is a global CI input). Focused run: `queued-run-dispatch-order`, `queued-run-line`, `execution-projection-agent-queue`, `fresh-continuation-retry-delivery`, `assigned-work-stalled-queued-run`, `automatic-run-start-gate`, `automatic-run-brake-recovery-passes`, `heartbeat-stale-queue-invalidation`: 8 files, 173 tests passed. `tsc --noEmit` for the server passed. ## Risks - Behavior shift: in a company where people create most top-level tasks, most issue runs are now person tier. The "work" tier then holds agent-started trees, routines and runs without a task. Recovery keeps its share because only runs a person asked for hold it. - An in-progress run on a person's tree now ranks ahead of a new person task that is not in progress (readiness rank). The oldest-run share bounds that wait at about 60 minutes plus a few starts, the same bound agent work had before. - A tree deeper than 32 levels above a queued task is treated as not a person tree. It falls back to the work tier and the aging share. - The extra query runs only when an agent has more than one queued run, and only for queued tasks without `createdByUserId`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>