Files
PaperClipAI/scripts/request-hot-restart.ts
T
DottaandPaperclip 992389480a fix(server): restore hot-restart run adoption (#9647)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The local heartbeat/runtime subsystem starts long-running local
agent processes and records their run state.
> - Operators sometimes need to rebuild and restart the Paperclip server
while local agent processes are still alive.
> - A normal restart should remain conservative, but a guarded
production hot restart needs an explicit marker, startup reconciliation,
and an inspectable report.
> - The broader hot-restart PR is currently merge-conflicted, so this
pull request lands the minimal server-side recovery path on current
`master`.
> - The benefit is that deploy operators can restart from a current
branch without reverting production changes and without marking adopted
live runs as `process_lost`.

## Linked Issues or Issue Description

No public GitHub issue exists for this deploy-safety fix.

Bug fix:

- What happened: the current deployable `master` branch did not include
the hot-restart marker CLI, startup adoption report path, or health
version proof needed by guarded service restarts.
- Expected behavior: a deploy operator can write a one-shot marker
before restarting, the old server snapshots eligible running child
processes, the new server reports adopted/finalized/lost runs, and
adopted live runs are not reaped as `process_lost`.
- Steps to reproduce: restart a server with running local child-process
heartbeat runs without the marker/adoption path; startup orphan reaping
has no adoption metadata and treats live detached children as lost.
- Paperclip version/commit: fixed on top of `master` at `b606869a6`.
- Deployment mode: production/local-service style deployments that
rebuild and restart the primary `paperclip.service`.
- Related PR: Refs #9628. This PR intentionally lands a smaller
deploy-safe subset because #9628 is currently merge-conflicted.
- Duplicate search: searched public PRs/issues for `hot restart` and
`process_lost adoption`; #9628 is the directly related prior
implementation.

## What Changed

- Added `scripts/request-hot-restart.ts` to write a one-shot hot-restart
intent marker under `PAPERCLIP_HOME`.
- Added `server/src/services/hot-restart.ts` for intent/report path
resolution, parsing, atomic writes, shutdown snapshots, and marker
cleanup.
- Wired server shutdown/startup so explicit hot restarts snapshot active
runs, skip the normal heartbeat drain, reconcile live child processes on
boot, and write `hot-restart-report.json`.
- Preserved adopted run metadata so normal orphan reaping does not
regress adopted live runs to `process_lost`.
- Added `serverVersion` health proof alongside existing `version`, plus
docs and regression coverage.

## Verification

- `pnpm vitest run server/src/__tests__/health.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts` — 2 files
passed, 100 tests passed.
- `pnpm --filter @paperclipai/server typecheck`
- `env PAPERCLIP_HOME="$PAPERCLIP_RUN_SCRATCH_DIR/hot-restart-cli-smoke"
pnpm --filter @paperclipai/server exec tsx
../scripts/request-hot-restart.ts --server-pid 12345`
- Branch ancestry checked after `git fetch origin master`:
`origin/master` was `b606869a6`, and `HEAD..origin/master` was empty.

## Risks

- Medium risk: process adoption depends on PID/PGID metadata and the
service manager leaving child processes alive for the guarded restart.
- Normal restarts remain conservative, but an incorrect marker PID
intentionally falls back to graceful drain instead of adoption.
- The PR is server-only and does not include the broader
UI/experimental-setting work from #9628.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5 via Codex coding agent in a Paperclip execution
workspace; tool use and shell/code execution enabled; context window not
surfaced by this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-16 02:46:09 -05:00

84 lines
2.4 KiB
JavaScript

#!/usr/bin/env -S node --import tsx
import {
resolveHotRestartIntentPath,
writeHotRestartIntent,
} from "../server/src/services/hot-restart.js";
function usage(): never {
console.error([
"Usage: tsx scripts/request-hot-restart.ts --server-pid <pid> [--drain-required]",
"",
"Writes a one-shot hot-restart intent marker under PAPERCLIP_HOME.",
].join("\n"));
process.exit(2);
}
function readArgs(argv: string[]) {
let serverPid: number | null = null;
let drainRequired = false;
for (let index = 0; index < argv.length; index += 1) {
const arg = argv[index];
if (arg === "--server-pid") {
const raw = argv[index + 1];
if (!raw) usage();
const parsed = Number(raw);
if (!Number.isInteger(parsed) || parsed <= 0) usage();
serverPid = parsed;
index += 1;
continue;
}
if (arg === "--drain-required") {
drainRequired = true;
continue;
}
if (arg === "--help" || arg === "-h") usage();
console.error(`Unknown argument: ${arg}`);
usage();
}
if (!serverPid) usage();
return { serverPid, drainRequired };
}
function normalizeApiBase(raw: string | undefined) {
const trimmed = raw?.trim();
if (!trimmed) return null;
return trimmed.replace(/\/+$/, "").replace(/\/api$/, "");
}
async function readPreviousServerVersion() {
const apiBase = normalizeApiBase(process.env.PAPERCLIP_API_URL);
if (!apiBase) return null;
try {
const response = await fetch(`${apiBase}/api/health`, {
signal: AbortSignal.timeout(2_000),
});
if (!response.ok) return null;
const body = await response.json() as Record<string, unknown>;
return typeof body.serverVersion === "string"
? body.serverVersion
: typeof body.version === "string"
? body.version
: null;
} catch {
return null;
}
}
const { serverPid, drainRequired } = readArgs(process.argv.slice(2));
const intent = await writeHotRestartIntent({
previousServerPid: serverPid,
previousServerVersion: await readPreviousServerVersion(),
drainRequired,
requestedByRunId: process.env.PAPERCLIP_RUN_ID?.trim() || null,
});
console.log(JSON.stringify({
status: "hot_restart_intent_written",
intentPath: resolveHotRestartIntentPath(),
previousServerPid: intent.previousServerPid,
previousServerVersion: intent.previousServerVersion,
drainRequired: intent.drainRequired,
}, null, 2));