Files
PaperClipAI/doc
79e28dc0d0 fix(runner): record pending calls at sidecar shutdown (#20272)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The ACPX sidecar carries task tools between a provider and its
controller.
> - A closed controller pipe rejects all pending tools with `sidecar
stdin closed`.
> - Current logs do not record the first shutdown trigger or the pending
calls at that point.
> - The sampled runs record a server SIGTERM before the 300-second tool
idle timeouts and later pipe failures. The exact pipe-close cause
remains unresolved.
> - This pull request adds one bounded shutdown record before cleanup
removes the evidence.
> - Operators can use it to compare controller shutdown with pending
tool calls.

## Linked Issues or Issue Description

**What happened?**

Several task document reads failed together with `sidecar stdin closed`.
In two sampled staging traces, earlier task tools reached a 300-second
idle timeout. The sidecar source maps the closure error to pending-call
rejection during shutdown. It does not record the trigger or the pending
call population.

Pattern: [Task reads lose their tool
connection](https://www.braintrust.dev/app/Paperclip/p/Paperclip%20Staging/patterns/5544e6f3-9e33-4661-95c3-5849c9a8f2b6).

**Expected behavior**

Record the local shutdown reason and a bounded snapshot of pending calls
before cleanup removes them. Keep tool cancellation and authorization
unchanged.

**Steps to reproduce**

1. Start the sidecar with pending task calls.
2. Close controller stdin, or send SIGTERM or SIGINT.
3. Inspect the sidecar event stream and stderr.

**Paperclip version or commit**

Baseline: `33a23e024697552a4a639e49957a3443c75a5bd5`.

### Hypothesis and thesis for a fix

The provisional hypothesis is that controller teardown causes the batch
failures after an earlier stall. The source proves how EOF rejects
pending tools. The traces establish ordering. They do not prove why the
controller closed stdin or why the earlier calls stalled. Run logs now
confirm an earlier server shutdown in both examples. One later recovery
could not find the provider; the other later ended at a durable receipt
limit. The first failed settlement and exact pipe-close trigger remain
unresolved. The later example already used a provider revision
containing #19365 and #19290, so those earlier fixes do not alone
explain all occurrences.

This is a **diagnostic-only change**. The sidecar owns the local
shutdown signal and its pending waiters. Record that state once at the
first shutdown request. Do not add retries or change deadlines without
evidence. This supports each provider that uses this sidecar. It
preserves company/task/run authority, approval checks, tool admission
and cleanup ownership. It does not resolve or claim recovery of the
original read stalls.

## What Changed

- Emit `sidecar_shutdown_requested` through the existing diagnostic
stream and stderr before shutdown can remove waiters.
- Record pending tool, input, permission and completion-normalization
counts. Include at most 16 oldest tool calls with elapsed time and a
SHA-256 call ID. Exclude arguments, results, operation names and raw
call IDs.
- Add real sidecar process tests for EOF and both shutdown signals.
Check successful resolution, cancelled calls, pending-call rejection,
provider cleanup and sidecar exit with code 0 and no signal before
fallback cleanup.
- Document the record and its limits. Saved session formats and
previous-release behavior stay unchanged.

## Verification

- Candidate: `85a929eef87d5f775bbed9f2c98409432c86c702`. The baseline
replay uses an isolated detached checkout at the baseline SHA with the
exact candidate regression file.
- Baseline red: the three process regressions failed because the
expected shutdown record was absent. The sidecar initialized, handled
calls and shut down before each assertion.
- Unchanged process regressions pass with the implementation. The
focused sidecar and snapshot suites passed: 117 tests, one unrelated
macOS candidate-admission test skipped on Linux x64.
- `cd packages/paperclip-runner && pnpm exec vitest run
src/cli/acpx-runtime-sidecar.test.ts
src/cli/acpx-sidecar-shutdown.test.ts`
- `cd packages/paperclip-runner && node --test
test/acpx-sidecar-contract.test.mjs`: 16 passed.
- `cd packages/paperclip-runner && pnpm exec tsc -p tsconfig.json
--noEmit`: passed after building the local eval-kernel dependency.
- `git diff --check`: passed.
- These tests use the real sidecar pipe and process boundary with a
controlled provider host. Independent outcome and cleanup files prove
rejection and process cleanup. They do not exercise a live authenticated
Claude/staging run or prove deployed recovery.
- The local preflight could not build Rust because this host has no
Cargo. CI must supply the applicable Rust/build checks. At head
`85a929eef87d5f775bbed9f2c98409432c86c702`, all 61 check runs are
complete: 58 passed and three conditional jobs skipped. The legacy Snyk
status passed. Greptile is 5/5 with both threads resolved. [CI
result](https://github.com/paperclipai/paperclip/actions/runs/42630335496).

## Risks

- The controller can stop reading before it receives the diagnostic
frame. Stderr is an additional path, not a delivery guarantee.
- EOF records a local pipe closure. It cannot identify the upstream
controller cause by itself.
- Hashed IDs require offline hashing of the existing tool-call journal
for correlation. At most 16 samples are retained; counts include the
full pending population.
- No persistent state, retry, cancellation or authorization contract
changes.

## Model Used

OpenAI Codex, GPT-6, with reasoning, tool use and code execution. The
runtime does not expose a more specific model ID or context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-10 09:57:01 -06:00
..
2026-02-16 14:25:00 -06:00