mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
feat: show the assignee's provider usage limit in task properties (#19270)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run on the native Runner through the Codex app-server and the Claude ACP harness > - Both harnesses report the provider account's rate-limit windows: Codex sends `account/rateLimits/updated`, and Claude sends its SDK `rate_limit_event` as `usage_update._meta["_claude/rateLimit"]` > - Paperclip dropped both signals, so an operator could not see that an agent was near or past its provider limit > - This pull request normalizes both signals into one account-scoped snapshot, keeps the latest snapshot per agent, and shows it as one "Usage limit" line in the task properties pane > - The benefit is that an operator sees "5h 78% · resets 15:40" (amber above 80%, "Limited until …" when the provider refuses work, "using credits" when a used-up window falls back to credits) next to the assignee, and can click it for every window ## Linked Issues or Issue Description Refs #19246 Refs #19259 **Is your feature request related to a problem?** Codex and Claude report provider rate-limit windows during a run. Paperclip dropped them: the Codex notification fell through the Rust normalizer with no branch, and ACPX dropped Claude's `_meta`. Operators could not see how close an agent was to its provider limit. **Describe the solution you'd like** Store the latest reported windows per agent and show one compact line in the task properties pane, with a popover that lists every window. **Describe alternatives you've considered** A full gauge in the agent list and the run header. A design review chose one property line with a details popover instead. ## What Changed - Runner (Rust, the runnerd path): new `provider_rate_limits.rs`. Codex `account/rateLimits/updated` becomes a `provider.notice.recorded` event (category `provider.rate_limits`, scope `account`). `CodexProviderState` merges sparse updates and emits a notice only when the windows change. ACPX `usage_update` with a sanitized `rateLimit` field also emits the notice. - Runner (sidecar): `acpx-runtime-sidecar.ts` forwards only `status`, `rateLimitType`, `utilization` and `resetsAt` from `_meta["_claude/rateLimit"]`. This needs the ACPX `_meta` passthrough from #19259. Until that merges, Claude emits nothing. - Runner (TypeScript in-process drivers): the same mapping in `codex-session-notifications.ts` and `codex-acpx-driver.ts` through `drivers/provider-rate-limits.ts`. - Shared: `provider-rate-limits.ts` holds the snapshot type, the parser, the merge, and `summarizeProviderRateLimits` (fullest current window, expiry by reset time, limited versus using credits). - Server: `agent-provider-rate-limits.ts` merges each notice into `agent_runtime_state.state_json.providerRateLimits`, scoped by company and agent. A session reset keeps the snapshot. No migration. - UI: `AgentRateLimitPropertyValue` adds a "Usage limit" row under the assignee in `IssueProperties`. Transcripts treat the notice as run-log only. A run status event refreshes the assignee's runtime state. - Storybook: `Tasks and conversations/Task creation and lifecycle/Usage limit property`. - Docs: `doc/run-log-events.md` describes the notice and its storage. ## Verification - `cd packages/paperclip-runner/runner && cargo test -p paperclip-runner-core --lib rate_limit` and `--test acpx_provider_events`: sparse Codex merge, dedupe, reached window, checkpoint round trip, Claude utilization and rejection, notice shape. - `cd packages/paperclip-runner && npx vitest run src/drivers/provider-rate-limits.test.ts src/drivers/codex` - `cd packages/shared && npx vitest run src/provider-rate-limits.test.ts`: summary, expiry, limited, using credits, parse and merge. - `cd server && npx vitest run src/__tests__/agent-provider-rate-limits.test.ts`: create and merge, out-of-order snapshot, company isolation, unrelated notices, session reset keeps the snapshot. - `cd ui && npx vitest run src/components/transcript src/components/issue-properties src/context/LiveUpdatesProvider` - `pnpm -r typecheck` and `pnpm check:token-gates` pass. - Live test on a local worktree instance: a native Codex agent on a ChatGPT subscription login ran a real task. The runner recorded one `provider.rate_limits` notice. The server stored it. The task properties pane showed "7d 100% · using credits" (this account's weekly window was used up and had credits, and the run succeeded). The transcript showed no extra row. - Not verified live: Claude. The local Claude CLI was not signed in, and the Claude path also needs #19259. The Rust and sidecar paths have tests. - Not run locally: the full `pnpm test:run` and `pnpm build`. CI runs them. ## Risks - Low risk. The snapshot is display data. It does not hold or start work. The existing provider-limit wait (#19246) still reads typed run failures. - Codex can send a rate-limit update often. The runner emits a notice only when the windows change, so run-log growth stays small. - With API-key auth, the harnesses report no windows, so no row appears. This is the intended result. - `CodexProviderState` gains two optional serde fields with defaults, so older checkpoints still load. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits, browser) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6a66c09f7f
commit
dd889e6a0b
27 files changed
+1911
-5
No files matched your search
@@ -0,0 +1,143 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { agentRuntimeState, agents, companies, createDb } from "@paperclipai/db";
|
||||
import { readProviderRateLimits, type ProviderRateLimitSnapshot } from "@paperclipai/shared";
|
||||
import { recordAgentProviderRateLimits, preservedAccountRuntimeState } from "../services/agent-provider-rate-limits.js";
|
||||
import { heartbeatService } from "../services/heartbeat.js";
|
||||
import { recoveryPostgresSupport, startRecoveryPostgres } from "./helpers/recovery-postgres.js";
|
||||
|
||||
const describePostgres = recoveryPostgresSupport.supported ? describe : describe.skip;
|
||||
if (!recoveryPostgresSupport.supported) console.warn(`Skipping provider rate limit tests: ${recoveryPostgresSupport.reason}`);
|
||||
|
||||
function snapshot(input: Partial<ProviderRateLimitSnapshot> & { observedAt: string }): ProviderRateLimitSnapshot {
|
||||
return {
|
||||
schema: "paperclip.provider.rate_limits.v1",
|
||||
provider: "openai",
|
||||
windows: [],
|
||||
credits: null,
|
||||
limitReached: null,
|
||||
...input,
|
||||
};
|
||||
}
|
||||
|
||||
function notice(value: ProviderRateLimitSnapshot) {
|
||||
return {
|
||||
eventType: "provider.notice.recorded",
|
||||
payload: {
|
||||
schema: "paperclip.provider.notice.v1",
|
||||
noticeId: "codex-rate-limits:1",
|
||||
severity: "info",
|
||||
category: "provider.rate_limits",
|
||||
scope: "account",
|
||||
recoverable: true,
|
||||
userActionable: false,
|
||||
summary: "Codex rate limits",
|
||||
details: [{ name: "rateLimits", value: JSON.stringify(value) }],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const fiveHour = (usedPercent: number) => ({
|
||||
id: "codex.primary", label: "5-hour window", shortLabel: "5h", windowMinutes: 300,
|
||||
usedPercent, resetsAt: "2026-11-05T15:40:00.000Z", status: null,
|
||||
});
|
||||
const weekly = (usedPercent: number) => ({
|
||||
id: "codex.secondary", label: "Weekly", shortLabel: "7d", windowMinutes: 10080,
|
||||
usedPercent, resetsAt: "2026-11-09T09:00:00.000Z", status: null,
|
||||
});
|
||||
|
||||
/** The latest provider rate-limit snapshot is kept per agent, in its own company. */
|
||||
describePostgres("agent provider rate limits", () => {
|
||||
let temporary: Awaited<ReturnType<typeof startRecoveryPostgres>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
|
||||
beforeAll(async () => {
|
||||
temporary = await startRecoveryPostgres("agent-provider-rate-limits-");
|
||||
db = createDb(temporary.connectionString);
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await db?.$client.end({ timeout: 0 });
|
||||
await temporary?.cleanup();
|
||||
});
|
||||
|
||||
async function seed() {
|
||||
const companyId = randomUUID(), agentId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Rate limit fixture", issuePrefix: `R${companyId.slice(0, 6)}` });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Codex planner", adapterType: "paperclip_runner", status: "idle", adapterConfig: {} });
|
||||
return { companyId, agentId };
|
||||
}
|
||||
|
||||
async function stored(agentId: string) {
|
||||
const row = await db.select().from(agentRuntimeState).where(eq(agentRuntimeState.agentId, agentId)).then((rows) => rows[0]);
|
||||
return row ? readProviderRateLimits(row.stateJson) : null;
|
||||
}
|
||||
|
||||
it("creates runtime state and merges later snapshots by window", async () => {
|
||||
const binding = await seed();
|
||||
await expect(recordAgentProviderRateLimits(db, binding, notice(snapshot({
|
||||
observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(70), weekly(40)],
|
||||
})))).resolves.toBe(true);
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({
|
||||
observedAt: "2026-11-05T15:10:00.000Z", windows: [fiveHour(78)],
|
||||
})));
|
||||
expect((await stored(binding.agentId))?.windows.map((window) => [window.id, window.usedPercent])).toEqual([
|
||||
["codex.primary", 78],
|
||||
["codex.secondary", 40],
|
||||
]);
|
||||
});
|
||||
|
||||
it("merges concurrent reports for one agent without losing a window", async () => {
|
||||
const binding = await seed();
|
||||
await Promise.all([
|
||||
recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(70)] }))),
|
||||
recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:01.000Z", windows: [weekly(40)] }))),
|
||||
]);
|
||||
expect((await stored(binding.agentId))?.windows.map((window) => window.id).sort()).toEqual(["codex.primary", "codex.secondary"]);
|
||||
});
|
||||
|
||||
it("reports no change when the same notice is replayed", async () => {
|
||||
const binding = await seed();
|
||||
const value = notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(70)] }));
|
||||
await expect(recordAgentProviderRateLimits(db, binding, value)).resolves.toBe(true);
|
||||
await expect(recordAgentProviderRateLimits(db, binding, value)).resolves.toBe(false);
|
||||
});
|
||||
|
||||
it("ignores an older snapshot delivered late", async () => {
|
||||
const binding = await seed();
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:10:00.000Z", windows: [fiveHour(78)] })));
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(60)] })));
|
||||
expect((await stored(binding.agentId))?.windows[0]?.usedPercent).toBe(78);
|
||||
});
|
||||
|
||||
it("does not write for another company's agent or for unrelated notices", async () => {
|
||||
const binding = await seed();
|
||||
const other = await seed();
|
||||
await expect(recordAgentProviderRateLimits(db, { companyId: other.companyId, agentId: binding.agentId }, notice(snapshot({
|
||||
observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(10)],
|
||||
})))).resolves.toBe(false);
|
||||
await expect(recordAgentProviderRateLimits(db, binding, {
|
||||
eventType: "provider.notice.recorded",
|
||||
payload: { schema: "paperclip.provider.notice.v1", category: "warning", details: [] },
|
||||
})).resolves.toBe(false);
|
||||
expect(await stored(binding.agentId)).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps the snapshot when the board resets the agent's session", async () => {
|
||||
const binding = await seed();
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(55)] })));
|
||||
await db.update(agentRuntimeState)
|
||||
.set({ stateJson: { ...(await db.select().from(agentRuntimeState).where(eq(agentRuntimeState.agentId, binding.agentId)).then((rows) => rows[0]!.stateJson)), sessionScratch: true } })
|
||||
.where(eq(agentRuntimeState.agentId, binding.agentId));
|
||||
await heartbeatService(db).resetRuntimeSession(binding.agentId);
|
||||
const row = await db.select().from(agentRuntimeState).where(eq(agentRuntimeState.agentId, binding.agentId)).then((rows) => rows[0]!);
|
||||
expect(row.stateJson.sessionScratch).toBeUndefined();
|
||||
expect(readProviderRateLimits(row.stateJson)?.windows[0]?.usedPercent).toBe(55);
|
||||
});
|
||||
|
||||
it("preserves only account keys", () => {
|
||||
expect(preservedAccountRuntimeState({ providerRateLimits: { a: 1 }, other: 2 })).toEqual({ providerRateLimits: { a: 1 } });
|
||||
expect(preservedAccountRuntimeState(null)).toEqual({});
|
||||
});
|
||||
});
|
||||
@@ -278,6 +278,11 @@ describe("live-event run read boundaries", () => {
|
||||
await once(socket, "open");
|
||||
socket.send(JSON.stringify({ type: "live.interests", version: 1, revision: 1, runIds: [] }));
|
||||
await expect.poll(() => messages.some(message => message.type === "live.subscribed" && message.revision === 1)).toBe(true);
|
||||
// Runtime-state refreshes reach the company without a transcript subscription.
|
||||
const refreshedAgentId = randomUUID();
|
||||
publishLiveEvent({ companyId, type: "agent.runtime_state.updated", payload: { agentId: refreshedAgentId } });
|
||||
await barrier();
|
||||
expect(messages.find(message => message.type === "agent.runtime_state.updated")?.payload).toMatchObject({ agentId: refreshedAgentId });
|
||||
const baseline = queries.length;
|
||||
publishLiveEvent({ companyId, type: "heartbeat.run.event", payload: sourcePayload }); await barrier();
|
||||
expect(messages.some(message => message.type === "heartbeat.run.event")).toBe(false);
|
||||
|
||||
Reference in new issue
Block a user