mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 09:50:48 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native agent runs call Paperclip tools, and
`RepeatedToolFailureGuard` refuses an exact repeat of a call that
already failed validation twice
> - The refusals are not counted and never end the turn, so a model
whose host drops an argument can resend the same refused call until the
run's wall clock
> - In a recorded restart-chaos wave, an opencode/DeepSeek agent sent
the same `PATCH /api/issues/{id}` through `call_api` 431 times in 40
minutes without `body`; 427 were refused. The run held an agent slot and
delayed other tasks
> - The system knows the next step (stop the turn and continue on a
fresh session), so the harness must own it, with no Retry banner
> - This pull request counts refusals, stops the provider turn after a
bounded number with the classified cause `agent_tool_call_loop`, and
queues an automatic fresh-session continuation that tells the model to
use a dedicated tool
> - It also records the declared argument names that each opencode tool
call delivered, so we can tell whether the bridge or the model drops
`body`
> - The benefit is that a looping agent frees its slot in seconds
instead of 40 minutes, and the task continues by itself
## Linked Issues or Issue Description
**What happened?**
An opencode agent on DeepSeek created two subtasks and then tried to set
their descriptions with `call_api` `PATCH /api/issues/{id}`. Every call
arrived with `operationId`, `pathParams`, `query` and `responseText`,
but without `body`. The guard refused each repeat. The model said "I
need to include body" more than 20 times and kept sending the same call:
431 calls, 427 refused, over 40 minutes. The run stayed active and its
agent slot delayed other queued work.
**Expected behavior**
After a small, bounded number of refusals, Paperclip stops the provider
turn and continues the task automatically on a fresh session, with a
hint to use a dedicated Paperclip tool. The board shows no Retry banner
because the system owns this recovery.
**Steps to reproduce**
1. Run a native agent whose model resends one tool call with a missing
required argument.
2. The guard refuses the third and later identical calls.
3. On master, the refusals never end the turn.
**Paperclip version or commit**
`master` at 6400698a8e.
## What Changed
- `repeated-tool-failure-guard.ts`: counts refusals per exact call and
per run. At 6 refusals of one call, or 20 refusals in the run, it throws
`ToolCallLoopError` (`code: "agent_tool_call_loop"`, tool, counts and
argument names, never values). The message tells the model the turn
stops and gives a short hint.
- `paperclip-runner-tool-authority.ts`:
`scheduleToolCallLoopContinuation` writes one idempotent native
status-wake outbox row (key `agent_tool_call_loop:<runId>`) with
`forceFreshSession: true` and the hint as the wake instruction. The
outbox dispatcher admits it with `allowRunCoalescing: false`, so it
never merges into a deferred comment wake or the active run. An assignee
run uses `issue_continuation_needed`. A completion-review run uses
`native_completion_review` with its review assignment, which the
dispatcher revalidates, so the fresh reviewer session keeps the review.
- `native-session-executor.ts`: the session authority epoch queues that
continuation when the guard raises the loop error. The governed-wait
lookup finds the wake on the next tool event and ends the provider turn
as `yielded` with a `same_agent` continuation, the same way the
connection-tools refresh ends a turn. If the wake cannot be queued, the
refusal still reaches the model and the error is logged.
- `status-decision-committer.ts`: a yielded continuation with that key
binds to the queued wake instead of creating a second one.
- `status-arbiter.ts` and `native-run-finalizer.ts`: a pending review
run that ended through this continuation skips the
`native_review_unresolved` resume recovery, because the queued review
wake owns the next reviewer run.
- `mcp-bridge.ts` and `opencode-proxy-finish-diagnostics.ts`: the
opencode tool-delivery diagnostic records `argumentNames` (only names
declared in the tool's input schema, sorted, at most 1,024 characters of
JSON), `argumentNamesTruncated` and `undeclaredArgumentCount`. Values
and undeclared names are never recorded. The forwarder's closed key
allowlist accepts the new fields and validates them, so a record stays
under its 2,048-character limit.
### Threshold choice
- **6 refusals of one exact call.** The guard already runs a call twice
and then refuses with an explicit correction. Six more refusals (eight
identical attempts in total) give a model that reads errors many chances
to change its arguments. In the recorded run the model never changed
them in 427 refusals, so six refusals end that loop within seconds.
- **20 refusals in the run.** A model can alternate between a few
refused variants and keep each one under 6. Refusals happen only for
exact repeats of a certain validation failure, so a healthy run almost
never reaches 20.
### Previous-release story
The guard counters live in memory for one run's tool authority, as
before. A deploy or restart during a run starts the counters again at
zero on the new process; that only delays the stop. The continuation is
an ordinary wake row. If a server on the previous release finalizes a
run that a new server parked, it does not know the key and queues its
own `same_agent` continuation too; the issue execution lock lets one run
and the other coalesces.
## Verification
- Red then green, `repeated-tool-failure-guard.test.ts`: "after N
refusals the guard raises a turn-stopping loop error" and "stops the
turn once refusals across different calls reach the run limit". Both
fail on master.
- Red then green, `native-session-executor.test.ts`: "a turn whose model
repeats a refused call N times ends with agent_tool_call_loop and
schedules a continuation". With the epoch handling disabled it fails (13
calls instead of 8, nothing parked). Also "keeps the refused call's
error when the loop continuation cannot be queued".
- `paperclip-runner-tool-authority.test.ts`: "queues one fresh-session
continuation outbox for the assignee / a completion reviewer stopped by
a tool-call loop" (real Postgres; idempotent).
- `native-tool-call-loop-continuation.test.ts`: with a comment wake
already deferred behind the looping run, the continuation keeps its own
keyed deferred wake, the comment wake is unchanged, and after release a
run starts with `forceFreshSession` and the loop cause.
- `status-arbiter.test.ts`: a pending reviewer stopped by a loop
releases its checkout without a resume recovery.
- `native-status-arbiter-corpus.test.ts`: the existing wake-binding test
now also covers the `agent_tool_call_loop` key.
- `mcp-bridge-delivery.test.ts` and
`opencode-proxy-finish-diagnostics.test.ts`: argument names are
recorded, values and undeclared names are not, invalid records are
dropped, and a boundary test with 32 names of 64 characters stays
forwardable with `argumentNamesTruncated: true`.
- Server: the six touched test files pass (1755 tests). Runner:
`src/drivers/opencode` and `src/cli` pass (569 tests). `tsc --noEmit`
passes for `server` and `packages/paperclip-runner`.
- `pnpm verify` exits 2 (runner package changes need broad coverage), so
required CI supplies the broad run.
## Risks
- A run that hits the limit ends its turn early. The limits are far
above what a model that reads its errors needs, and the task continues
automatically on a fresh session.
- The fresh session loses the in-turn conversation. The task, its
comments, documents and workspace stay, and the continuation names the
cause and the hint.
- The diagnostic now carries declared argument names. These are schema
names, not values, and the forwarder validates the shape before it
forwards a record.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Refs #21360
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
171 lines
8.4 KiB
TypeScript
171 lines
8.4 KiB
TypeScript
import { HttpError } from "../../errors.js";
|
|
import { nativeSha256 } from "./canonical.js";
|
|
|
|
/** Identical failures after which the exact call is no longer run. */
|
|
export const REPEATED_TOOL_FAILURE_LIMIT = 2;
|
|
export const TOOL_CALL_LOOP_REFUSAL_LIMIT = 6;
|
|
export const TOOL_CALL_LOOP_RUN_REFUSAL_LIMIT = 20;
|
|
export const AGENT_TOOL_CALL_LOOP = "agent_tool_call_loop";
|
|
export function toolCallLoopContinuationKey(runId: string): string {
|
|
return `${AGENT_TOOL_CALL_LOOP}:${runId}`;
|
|
}
|
|
|
|
export const TOOL_CALL_LOOP_HINT = "The previous turn kept sending one tool call that Paperclip had already refused. "
|
|
+ "Do not send it again unchanged. Prefer a dedicated Paperclip tool, such as set_task_title, report_progress or write_document, over call_api. "
|
|
+ "If call_api is the only way, send every required argument, including body, as a top-level argument.";
|
|
const MAX_TRACKED_FAILURES = 64;
|
|
const MAX_RESPONSE_EXCERPT = 2_000;
|
|
|
|
type ToolCall = { tool: string; callId: string; arguments: unknown };
|
|
type Failure = {
|
|
count: number;
|
|
status: number;
|
|
message: string;
|
|
details?: unknown;
|
|
/** Returned by the route as an `ok: false` response rather than refused before sending. */
|
|
returned: boolean;
|
|
refusals?: number;
|
|
};
|
|
|
|
export class ToolCallLoopError extends HttpError {
|
|
readonly code = AGENT_TOOL_CALL_LOOP;
|
|
constructor(readonly tool: string, readonly refusals: number, readonly runRefusals: number,
|
|
readonly argumentNames: string[], status: number, message: string, details?: unknown) {
|
|
super(status, message, details);
|
|
this.name = "ToolCallLoopError";
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Stops a model from resending the same rejected tool call. Some model hosts
|
|
* drop optional arguments, so the model can believe it fixed a call that
|
|
* arrives unchanged every time. Only failures that are certain to repeat for
|
|
* identical arguments count: request validation (400/422) thrown before the
|
|
* request is sent, schema errors, and a route's own 400/422 returned as an
|
|
* `ok: false` response. Timeouts, rate limits, conflicts, server errors and
|
|
* any change to the arguments pass through untouched.
|
|
*
|
|
* A route can reject a request because of the state it found, so a returned
|
|
* rejection is forgotten as soon as a different call succeeds; a refusal made
|
|
* before sending depends on the arguments alone and is remembered for the run.
|
|
*
|
|
* State lives for one run's tool authority, in memory, and is bounded.
|
|
*/
|
|
export class RepeatedToolFailureGuard {
|
|
readonly #failures = new Map<string, Failure>();
|
|
// Identical calls a model sends in parallel run one after another, so each
|
|
// sees the failures before it. Different calls never wait on each other.
|
|
readonly #inFlight = new Map<string, Promise<unknown>>();
|
|
#runRefusals = 0;
|
|
|
|
run<T>(call: ToolCall, execute: () => Promise<T>): Promise<T> {
|
|
const key = callKey(call);
|
|
const previous = this.#inFlight.get(key);
|
|
const current = previous ? previous.then(() => this.#run(key, call, execute), () => this.#run(key, call, execute)) : this.#run(key, call, execute);
|
|
const settled = current.then(() => undefined, () => undefined);
|
|
this.#inFlight.set(key, settled);
|
|
void settled.then(() => { if (this.#inFlight.get(key) === settled) this.#inFlight.delete(key); });
|
|
return current;
|
|
}
|
|
|
|
async #run<T>(key: string, call: ToolCall, execute: () => Promise<T>): Promise<T> {
|
|
const prior = this.#failures.get(key);
|
|
if (prior && prior.count >= REPEATED_TOOL_FAILURE_LIMIT) {
|
|
prior.refusals = (prior.refusals ?? 0) + 1;
|
|
this.#runRefusals += 1;
|
|
if (prior.refusals >= TOOL_CALL_LOOP_REFUSAL_LIMIT || this.#runRefusals >= TOOL_CALL_LOOP_RUN_REFUSAL_LIMIT) {
|
|
throw new ToolCallLoopError(call.tool, prior.refusals, this.#runRefusals, argumentNames(call.arguments), prior.status,
|
|
`This call was not run, and Paperclip stopped this turn: ${call.tool} was refused ${prior.refusals} times in this run with these exact arguments. `
|
|
+ `${receivedArguments(call.arguments)} The task continues in a fresh session. ${TOOL_CALL_LOOP_HINT}`,
|
|
prior.details);
|
|
}
|
|
throw new HttpError(prior.status, `This call was not run: ${call.tool} already failed ${prior.count} times in this run with these exact arguments. `
|
|
+ `${receivedArguments(call.arguments)} ${prior.returned ? `The response each time was HTTP ${prior.status}: ` : "The error each time was: "}${prior.message} `
|
|
+ "Repeating it unchanged cannot succeed. Send it again only with the change that error asks for, or stop calling it and tell the user what blocked you.", prior.details);
|
|
}
|
|
let result: T;
|
|
try {
|
|
result = await execute();
|
|
} catch (error) {
|
|
const failure = thrownFailure(error);
|
|
if (!failure) throw error;
|
|
const count = this.#record(key, failure);
|
|
if (count < REPEATED_TOOL_FAILURE_LIMIT) throw error;
|
|
throw new HttpError(failure.status, `${failure.message} ${repeatedNote(count, call)}`, failure.details);
|
|
}
|
|
const failure = returnedFailure(result);
|
|
if (!failure) {
|
|
// Only a success can have changed the state a route checked. A transient
|
|
// failure (5xx, 429, timeout) leaves every remembered rejection in place.
|
|
if (!isFailedResponse(result)) {
|
|
this.#failures.delete(key);
|
|
this.#forgetReturnedFailures();
|
|
}
|
|
return result;
|
|
}
|
|
const count = this.#record(key, failure);
|
|
if (count < REPEATED_TOOL_FAILURE_LIMIT) return result;
|
|
const response = result as Record<string, unknown>;
|
|
const guidance = typeof response.guidance === "string" ? `${response.guidance} ` : "";
|
|
return { ...response, guidance: `${guidance}${repeatedNote(count, call)}` } as T;
|
|
}
|
|
|
|
#record(key: string, failure: Omit<Failure, "count">): number {
|
|
const latest = this.#failures.get(key);
|
|
const count = latest && latest.message === failure.message && latest.status === failure.status ? latest.count + 1 : 1;
|
|
this.#failures.delete(key);
|
|
this.#failures.set(key, { ...failure, count });
|
|
if (this.#failures.size > MAX_TRACKED_FAILURES) this.#failures.delete(this.#failures.keys().next().value!);
|
|
return count;
|
|
}
|
|
|
|
#forgetReturnedFailures() {
|
|
for (const [key, failure] of this.#failures) if (failure.returned) this.#failures.delete(key);
|
|
}
|
|
}
|
|
|
|
function callKey(call: ToolCall): string {
|
|
return nativeSha256({ tool: call.tool, arguments: call.arguments ?? null });
|
|
}
|
|
|
|
function isFailedResponse(result: unknown): boolean {
|
|
return Boolean(result && typeof result === "object" && (result as Record<string, unknown>).ok === false);
|
|
}
|
|
|
|
function isValidationStatus(status: unknown): status is 400 | 422 {
|
|
return status === 400 || status === 422;
|
|
}
|
|
|
|
function thrownFailure(error: unknown): Omit<Failure, "count"> | null {
|
|
if (error instanceof HttpError && isValidationStatus(error.status)) {
|
|
return { status: error.status, message: error.message, details: error.details, returned: false };
|
|
}
|
|
if (error instanceof Error && error.name === "ZodError") return { status: 400, message: error.message, returned: false };
|
|
return null;
|
|
}
|
|
|
|
/** A route's validation rejection, returned as data (call_api's `ok: false` with HTTP 400/422). */
|
|
function returnedFailure(result: unknown): Omit<Failure, "count"> | null {
|
|
if (!result || typeof result !== "object" || Array.isArray(result)) return null;
|
|
const response = result as Record<string, unknown>;
|
|
if (response.ok !== false || !isValidationStatus(response.status)) return null;
|
|
const reason = response.data ?? response.message ?? response.error ?? null;
|
|
const text = typeof reason === "string" ? reason : JSON.stringify(reason);
|
|
return { status: response.status, message: text.length > MAX_RESPONSE_EXCERPT ? `${text.slice(0, MAX_RESPONSE_EXCERPT)}…` : text, returned: true };
|
|
}
|
|
|
|
function repeatedNote(count: number, call: ToolCall): string {
|
|
return `This exact call has now failed ${count} times with the same arguments. ${receivedArguments(call.arguments)} `
|
|
+ "If you meant to send another argument, it did not arrive; send it as a top-level argument exactly as described above. "
|
|
+ "Paperclip will not run this exact call again in this run. Change the arguments, or stop and tell the user what blocked you.";
|
|
}
|
|
|
|
function argumentNames(value: unknown): string[] {
|
|
return value && typeof value === "object" && !Array.isArray(value) ? Object.keys(value).sort() : [];
|
|
}
|
|
|
|
function receivedArguments(value: unknown): string {
|
|
const names = argumentNames(value);
|
|
return `Arguments received each time: ${names.length ? names.join(", ") : "none"}.`;
|
|
}
|