mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 01:57:31 +01:00
feat: show the assignee's provider usage limit in task properties (#19270)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run on the native Runner through the Codex app-server and the Claude ACP harness > - Both harnesses report the provider account's rate-limit windows: Codex sends `account/rateLimits/updated`, and Claude sends its SDK `rate_limit_event` as `usage_update._meta["_claude/rateLimit"]` > - Paperclip dropped both signals, so an operator could not see that an agent was near or past its provider limit > - This pull request normalizes both signals into one account-scoped snapshot, keeps the latest snapshot per agent, and shows it as one "Usage limit" line in the task properties pane > - The benefit is that an operator sees "5h 78% · resets 15:40" (amber above 80%, "Limited until …" when the provider refuses work, "using credits" when a used-up window falls back to credits) next to the assignee, and can click it for every window ## Linked Issues or Issue Description Refs #19246 Refs #19259 **Is your feature request related to a problem?** Codex and Claude report provider rate-limit windows during a run. Paperclip dropped them: the Codex notification fell through the Rust normalizer with no branch, and ACPX dropped Claude's `_meta`. Operators could not see how close an agent was to its provider limit. **Describe the solution you'd like** Store the latest reported windows per agent and show one compact line in the task properties pane, with a popover that lists every window. **Describe alternatives you've considered** A full gauge in the agent list and the run header. A design review chose one property line with a details popover instead. ## What Changed - Runner (Rust, the runnerd path): new `provider_rate_limits.rs`. Codex `account/rateLimits/updated` becomes a `provider.notice.recorded` event (category `provider.rate_limits`, scope `account`). `CodexProviderState` merges sparse updates and emits a notice only when the windows change. ACPX `usage_update` with a sanitized `rateLimit` field also emits the notice. - Runner (sidecar): `acpx-runtime-sidecar.ts` forwards only `status`, `rateLimitType`, `utilization` and `resetsAt` from `_meta["_claude/rateLimit"]`. This needs the ACPX `_meta` passthrough from #19259. Until that merges, Claude emits nothing. - Runner (TypeScript in-process drivers): the same mapping in `codex-session-notifications.ts` and `codex-acpx-driver.ts` through `drivers/provider-rate-limits.ts`. - Shared: `provider-rate-limits.ts` holds the snapshot type, the parser, the merge, and `summarizeProviderRateLimits` (fullest current window, expiry by reset time, limited versus using credits). - Server: `agent-provider-rate-limits.ts` merges each notice into `agent_runtime_state.state_json.providerRateLimits`, scoped by company and agent. A session reset keeps the snapshot. No migration. - UI: `AgentRateLimitPropertyValue` adds a "Usage limit" row under the assignee in `IssueProperties`. Transcripts treat the notice as run-log only. A run status event refreshes the assignee's runtime state. - Storybook: `Tasks and conversations/Task creation and lifecycle/Usage limit property`. - Docs: `doc/run-log-events.md` describes the notice and its storage. ## Verification - `cd packages/paperclip-runner/runner && cargo test -p paperclip-runner-core --lib rate_limit` and `--test acpx_provider_events`: sparse Codex merge, dedupe, reached window, checkpoint round trip, Claude utilization and rejection, notice shape. - `cd packages/paperclip-runner && npx vitest run src/drivers/provider-rate-limits.test.ts src/drivers/codex` - `cd packages/shared && npx vitest run src/provider-rate-limits.test.ts`: summary, expiry, limited, using credits, parse and merge. - `cd server && npx vitest run src/__tests__/agent-provider-rate-limits.test.ts`: create and merge, out-of-order snapshot, company isolation, unrelated notices, session reset keeps the snapshot. - `cd ui && npx vitest run src/components/transcript src/components/issue-properties src/context/LiveUpdatesProvider` - `pnpm -r typecheck` and `pnpm check:token-gates` pass. - Live test on a local worktree instance: a native Codex agent on a ChatGPT subscription login ran a real task. The runner recorded one `provider.rate_limits` notice. The server stored it. The task properties pane showed "7d 100% · using credits" (this account's weekly window was used up and had credits, and the run succeeded). The transcript showed no extra row. - Not verified live: Claude. The local Claude CLI was not signed in, and the Claude path also needs #19259. The Rust and sidecar paths have tests. - Not run locally: the full `pnpm test:run` and `pnpm build`. CI runs them. ## Risks - Low risk. The snapshot is display data. It does not hold or start work. The existing provider-limit wait (#19246) still reads typed run failures. - Codex can send a rate-limit update often. The runner emits a notice only when the windows change, so run-log growth stays small. - With API-key auth, the harnesses report no windows, so no row appears. This is the intended result. - `CodexProviderState` gains two optional serde fields with defaults, so older checkpoints still load. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits, browser) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6a66c09f7f
commit
dd889e6a0b
27 files changed
+1911
-5
No files matched your search
@@ -0,0 +1,143 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { agentRuntimeState, agents, companies, createDb } from "@paperclipai/db";
|
||||
import { readProviderRateLimits, type ProviderRateLimitSnapshot } from "@paperclipai/shared";
|
||||
import { recordAgentProviderRateLimits, preservedAccountRuntimeState } from "../services/agent-provider-rate-limits.js";
|
||||
import { heartbeatService } from "../services/heartbeat.js";
|
||||
import { recoveryPostgresSupport, startRecoveryPostgres } from "./helpers/recovery-postgres.js";
|
||||
|
||||
const describePostgres = recoveryPostgresSupport.supported ? describe : describe.skip;
|
||||
if (!recoveryPostgresSupport.supported) console.warn(`Skipping provider rate limit tests: ${recoveryPostgresSupport.reason}`);
|
||||
|
||||
function snapshot(input: Partial<ProviderRateLimitSnapshot> & { observedAt: string }): ProviderRateLimitSnapshot {
|
||||
return {
|
||||
schema: "paperclip.provider.rate_limits.v1",
|
||||
provider: "openai",
|
||||
windows: [],
|
||||
credits: null,
|
||||
limitReached: null,
|
||||
...input,
|
||||
};
|
||||
}
|
||||
|
||||
function notice(value: ProviderRateLimitSnapshot) {
|
||||
return {
|
||||
eventType: "provider.notice.recorded",
|
||||
payload: {
|
||||
schema: "paperclip.provider.notice.v1",
|
||||
noticeId: "codex-rate-limits:1",
|
||||
severity: "info",
|
||||
category: "provider.rate_limits",
|
||||
scope: "account",
|
||||
recoverable: true,
|
||||
userActionable: false,
|
||||
summary: "Codex rate limits",
|
||||
details: [{ name: "rateLimits", value: JSON.stringify(value) }],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const fiveHour = (usedPercent: number) => ({
|
||||
id: "codex.primary", label: "5-hour window", shortLabel: "5h", windowMinutes: 300,
|
||||
usedPercent, resetsAt: "2026-11-05T15:40:00.000Z", status: null,
|
||||
});
|
||||
const weekly = (usedPercent: number) => ({
|
||||
id: "codex.secondary", label: "Weekly", shortLabel: "7d", windowMinutes: 10080,
|
||||
usedPercent, resetsAt: "2026-11-09T09:00:00.000Z", status: null,
|
||||
});
|
||||
|
||||
/** The latest provider rate-limit snapshot is kept per agent, in its own company. */
|
||||
describePostgres("agent provider rate limits", () => {
|
||||
let temporary: Awaited<ReturnType<typeof startRecoveryPostgres>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
|
||||
beforeAll(async () => {
|
||||
temporary = await startRecoveryPostgres("agent-provider-rate-limits-");
|
||||
db = createDb(temporary.connectionString);
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await db?.$client.end({ timeout: 0 });
|
||||
await temporary?.cleanup();
|
||||
});
|
||||
|
||||
async function seed() {
|
||||
const companyId = randomUUID(), agentId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Rate limit fixture", issuePrefix: `R${companyId.slice(0, 6)}` });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Codex planner", adapterType: "paperclip_runner", status: "idle", adapterConfig: {} });
|
||||
return { companyId, agentId };
|
||||
}
|
||||
|
||||
async function stored(agentId: string) {
|
||||
const row = await db.select().from(agentRuntimeState).where(eq(agentRuntimeState.agentId, agentId)).then((rows) => rows[0]);
|
||||
return row ? readProviderRateLimits(row.stateJson) : null;
|
||||
}
|
||||
|
||||
it("creates runtime state and merges later snapshots by window", async () => {
|
||||
const binding = await seed();
|
||||
await expect(recordAgentProviderRateLimits(db, binding, notice(snapshot({
|
||||
observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(70), weekly(40)],
|
||||
})))).resolves.toBe(true);
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({
|
||||
observedAt: "2026-11-05T15:10:00.000Z", windows: [fiveHour(78)],
|
||||
})));
|
||||
expect((await stored(binding.agentId))?.windows.map((window) => [window.id, window.usedPercent])).toEqual([
|
||||
["codex.primary", 78],
|
||||
["codex.secondary", 40],
|
||||
]);
|
||||
});
|
||||
|
||||
it("merges concurrent reports for one agent without losing a window", async () => {
|
||||
const binding = await seed();
|
||||
await Promise.all([
|
||||
recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(70)] }))),
|
||||
recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:01.000Z", windows: [weekly(40)] }))),
|
||||
]);
|
||||
expect((await stored(binding.agentId))?.windows.map((window) => window.id).sort()).toEqual(["codex.primary", "codex.secondary"]);
|
||||
});
|
||||
|
||||
it("reports no change when the same notice is replayed", async () => {
|
||||
const binding = await seed();
|
||||
const value = notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(70)] }));
|
||||
await expect(recordAgentProviderRateLimits(db, binding, value)).resolves.toBe(true);
|
||||
await expect(recordAgentProviderRateLimits(db, binding, value)).resolves.toBe(false);
|
||||
});
|
||||
|
||||
it("ignores an older snapshot delivered late", async () => {
|
||||
const binding = await seed();
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:10:00.000Z", windows: [fiveHour(78)] })));
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(60)] })));
|
||||
expect((await stored(binding.agentId))?.windows[0]?.usedPercent).toBe(78);
|
||||
});
|
||||
|
||||
it("does not write for another company's agent or for unrelated notices", async () => {
|
||||
const binding = await seed();
|
||||
const other = await seed();
|
||||
await expect(recordAgentProviderRateLimits(db, { companyId: other.companyId, agentId: binding.agentId }, notice(snapshot({
|
||||
observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(10)],
|
||||
})))).resolves.toBe(false);
|
||||
await expect(recordAgentProviderRateLimits(db, binding, {
|
||||
eventType: "provider.notice.recorded",
|
||||
payload: { schema: "paperclip.provider.notice.v1", category: "warning", details: [] },
|
||||
})).resolves.toBe(false);
|
||||
expect(await stored(binding.agentId)).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps the snapshot when the board resets the agent's session", async () => {
|
||||
const binding = await seed();
|
||||
await recordAgentProviderRateLimits(db, binding, notice(snapshot({ observedAt: "2026-11-05T15:00:00.000Z", windows: [fiveHour(55)] })));
|
||||
await db.update(agentRuntimeState)
|
||||
.set({ stateJson: { ...(await db.select().from(agentRuntimeState).where(eq(agentRuntimeState.agentId, binding.agentId)).then((rows) => rows[0]!.stateJson)), sessionScratch: true } })
|
||||
.where(eq(agentRuntimeState.agentId, binding.agentId));
|
||||
await heartbeatService(db).resetRuntimeSession(binding.agentId);
|
||||
const row = await db.select().from(agentRuntimeState).where(eq(agentRuntimeState.agentId, binding.agentId)).then((rows) => rows[0]!);
|
||||
expect(row.stateJson.sessionScratch).toBeUndefined();
|
||||
expect(readProviderRateLimits(row.stateJson)?.windows[0]?.usedPercent).toBe(55);
|
||||
});
|
||||
|
||||
it("preserves only account keys", () => {
|
||||
expect(preservedAccountRuntimeState({ providerRateLimits: { a: 1 }, other: 2 })).toEqual({ providerRateLimits: { a: 1 } });
|
||||
expect(preservedAccountRuntimeState(null)).toEqual({});
|
||||
});
|
||||
});
|
||||
@@ -278,6 +278,11 @@ describe("live-event run read boundaries", () => {
|
||||
await once(socket, "open");
|
||||
socket.send(JSON.stringify({ type: "live.interests", version: 1, revision: 1, runIds: [] }));
|
||||
await expect.poll(() => messages.some(message => message.type === "live.subscribed" && message.revision === 1)).toBe(true);
|
||||
// Runtime-state refreshes reach the company without a transcript subscription.
|
||||
const refreshedAgentId = randomUUID();
|
||||
publishLiveEvent({ companyId, type: "agent.runtime_state.updated", payload: { agentId: refreshedAgentId } });
|
||||
await barrier();
|
||||
expect(messages.find(message => message.type === "agent.runtime_state.updated")?.payload).toMatchObject({ agentId: refreshedAgentId });
|
||||
const baseline = queries.length;
|
||||
publishLiveEvent({ companyId, type: "heartbeat.run.event", payload: sourcePayload }); await barrier();
|
||||
expect(messages.some(message => message.type === "heartbeat.run.event")).toBe(false);
|
||||
|
||||
@@ -355,6 +355,7 @@ export function setupLiveEventsWebSocketServer(
|
||||
...(issueId ? await activityToastIdentity(db, event.companyId, payload) : {}),
|
||||
} };
|
||||
if (event.type === "agent.status") return { ...event, payload: { agentId: payload.agentId, status: payload.status } };
|
||||
if (event.type === "agent.runtime_state.updated") return { ...event, payload: { agentId: payload.agentId } };
|
||||
if (event.type === "external_object.updated") return { ...event, payload: { externalObjectId: payload.externalObjectId } };
|
||||
if (event.type.startsWith("plugin.")) return event;
|
||||
return null;
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { agentRuntimeState, agents } from "@paperclipai/db";
|
||||
import {
|
||||
PROVIDER_RATE_LIMITS_STATE_KEY,
|
||||
mergeProviderRateLimitSnapshots,
|
||||
providerRateLimitSnapshotFromNotice,
|
||||
readProviderRateLimits,
|
||||
} from "@paperclipai/shared";
|
||||
import { publishLiveEvent } from "./live-events.js";
|
||||
|
||||
/**
|
||||
* Keep the latest provider rate-limit snapshot an agent's harness reported.
|
||||
* The snapshot is account data the harness observed while running this agent;
|
||||
* it lives in the agent's company-scoped runtime state and never drives
|
||||
* execution (the provider-limit wait reads typed run failures instead).
|
||||
*
|
||||
* Safe to repeat: replaying the same notice merges to the same snapshot.
|
||||
* Returns true when the stored snapshot changed.
|
||||
*/
|
||||
export async function recordAgentProviderRateLimits(
|
||||
db: Db,
|
||||
binding: { companyId: string; agentId: string },
|
||||
event: { eventType: string; payload: Record<string, unknown> },
|
||||
): Promise<boolean> {
|
||||
const next = providerRateLimitSnapshotFromNotice(event.eventType, event.payload);
|
||||
if (!next) return false;
|
||||
const changed = await db.transaction(async (tx) => {
|
||||
const agent = await tx
|
||||
.select({ adapterType: agents.adapterType })
|
||||
.from(agents)
|
||||
.where(and(eq(agents.id, binding.agentId), eq(agents.companyId, binding.companyId)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!agent) return false;
|
||||
await tx
|
||||
.insert(agentRuntimeState)
|
||||
.values({ agentId: binding.agentId, companyId: binding.companyId, adapterType: agent.adapterType, stateJson: {} })
|
||||
.onConflictDoNothing({ target: agentRuntimeState.agentId });
|
||||
// Lock the row so concurrent runs merge in order instead of overwriting.
|
||||
const existing = await tx
|
||||
.select({ stateJson: agentRuntimeState.stateJson })
|
||||
.from(agentRuntimeState)
|
||||
.where(and(eq(agentRuntimeState.agentId, binding.agentId), eq(agentRuntimeState.companyId, binding.companyId)))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!existing) return false;
|
||||
const stored = readProviderRateLimits(existing.stateJson);
|
||||
const merged = mergeProviderRateLimitSnapshots(stored, next);
|
||||
if (stored && JSON.stringify(stored) === JSON.stringify(merged)) return false;
|
||||
const patch = JSON.stringify({ [PROVIDER_RATE_LIMITS_STATE_KEY]: merged });
|
||||
await tx
|
||||
.update(agentRuntimeState)
|
||||
.set({ stateJson: sql`coalesce(${agentRuntimeState.stateJson}, '{}'::jsonb) || ${patch}::jsonb` })
|
||||
.where(and(eq(agentRuntimeState.agentId, binding.agentId), eq(agentRuntimeState.companyId, binding.companyId)));
|
||||
return true;
|
||||
});
|
||||
if (changed) {
|
||||
publishLiveEvent({
|
||||
companyId: binding.companyId,
|
||||
type: "agent.runtime_state.updated",
|
||||
payload: { agentId: binding.agentId },
|
||||
});
|
||||
}
|
||||
return changed;
|
||||
}
|
||||
|
||||
/** Runtime state keys that describe the provider account, not the session. */
|
||||
export function preservedAccountRuntimeState(stateJson: Record<string, unknown> | null | undefined): Record<string, unknown> {
|
||||
const snapshot = stateJson?.[PROVIDER_RATE_LIMITS_STATE_KEY];
|
||||
return snapshot === undefined ? {} : { [PROVIDER_RATE_LIMITS_STATE_KEY]: snapshot };
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import { notifyExternalConversationSettlement } from "../external-conversation-settlement-notifications.js";
|
||||
import { preservedAccountRuntimeState } from "../agent-provider-rate-limits.js";
|
||||
import { readRuntimeAccountingTotals } from "../runtime-accounting-totals.js";
|
||||
import { conversationEventCondition, conversationEventColumns } from "./conversation-events.js";
|
||||
import { deriveTaskKey } from "./run-preparation.js";
|
||||
@@ -1940,7 +1941,7 @@ export function createHeartbeatRunState(db: Db) {
|
||||
) => {
|
||||
const agent = await getAgent(agentId);
|
||||
if (!agent) throw notFound("Agent not found");
|
||||
await ensureRuntimeState(agent);
|
||||
const runtimeBefore = await ensureRuntimeState(agent);
|
||||
const taskKey = readNonEmptyString(opts?.taskKey);
|
||||
const clearedTaskSessions = await clearTaskSessions(
|
||||
agent.companyId,
|
||||
@@ -1959,7 +1960,8 @@ export function createHeartbeatRunState(db: Db) {
|
||||
updatedAt: new Date(),
|
||||
};
|
||||
if (!taskKey) {
|
||||
runtimePatch.stateJson = {};
|
||||
// Provider rate limits describe the account, not the session.
|
||||
runtimePatch.stateJson = preservedAccountRuntimeState(runtimeBefore.stateJson);
|
||||
}
|
||||
|
||||
const updated = await db
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { NativeSessionDiagnostics, nativeSessionDiagnostics, type NativeDiagnosticLease } from "./native-session-diagnostics.js";
|
||||
import { recordAgentProviderRateLimits } from "../agent-provider-rate-limits.js";
|
||||
import { nativePostResultFailureCategory, type NativePostResultFailureCategory } from "./native-post-result-failure.js";
|
||||
import { logger } from "../../middleware/logger.js";
|
||||
import { canRestoreMigratedNativeSession } from "./native-workspace-migration.js";
|
||||
@@ -9215,6 +9216,16 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
throw error;
|
||||
}
|
||||
const controlPlaneInstanceId = `${effectiveRunnerInstanceId}:control`;
|
||||
// Display data only; a failure never fails the run. The replay path repeats
|
||||
// it so a snapshot lost between the event commit and this write recovers.
|
||||
const recordRateLimits = async (event: unknown) => {
|
||||
await recordAgentProviderRateLimits(input.db, input.execution.binding, event as {
|
||||
eventType: string;
|
||||
payload: Record<string, unknown>;
|
||||
}).catch((error) => {
|
||||
logger.warn({ err: error, runId: input.execution.binding.runId }, "failed to record provider rate limits");
|
||||
});
|
||||
};
|
||||
const planSynchronizations: PlanSynchronization[] = [];
|
||||
const upsertPlanSynchronization = (
|
||||
synchronization: PlanSynchronization,
|
||||
@@ -9682,6 +9693,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
event,
|
||||
event.eventType === "item.completed" || questionFallback !== null,
|
||||
);
|
||||
await recordRateLimits(event);
|
||||
await recordPlanSynchronization(
|
||||
event as {
|
||||
sourceEventId: string;
|
||||
@@ -9708,6 +9720,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
event,
|
||||
event.eventType === "item.completed" || questionFallback !== null,
|
||||
);
|
||||
await recordRateLimits(event);
|
||||
const planSynchronization = await synchronizeCompletedProviderPlan({
|
||||
db: input.db,
|
||||
execution: input.execution,
|
||||
|
||||
Reference in new issue
Block a user