Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-12 14:19:00 +01:00
Code Issues Packages Projects Releases Wiki Activity
7,042 Commits 1,880 Branches 2,440 Tags
0a64c9b4abfb1aed8ed199a03e3ed90a595cbff2
Commit Graph
5 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 7475577cdb fix(daytona): keep a runner sandbox alive through the controller reconnect grace (#19754) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A remote agent run executes in a Daytona sandbox and keeps working
while the Paperclip server restarts
> - The server reattaches that runner within a 7-minute reconnect grace
> - Daytona stops an idle sandbox after its `autoStopInterval`, and it
counts only SDK and lifecycle calls as activity
> - With the server down, nothing touches the sandbox, so a 5-minute
auto-stop kills the runner before the server returns
> - This pull request makes the host tell the provider its reconnect
grace, so Daytona never auto-stops a runner sandbox inside that grace
> - The benefit is that a controller outage of up to 7 minutes no longer
turns a healthy remote run into a blocked task

## Linked Issues or Issue Description

Refs #19199 (the same floor for the warm runner idle window). Refs
#19741 (follow-up delivery through restarts; no overlap).

**What happened?**
A restart-chaos wave on Daytona killed the server for 4m51s while a
per-turn remote run was active. The environment used `autoStopInterval:
5`. The last sandbox activity was at 06:09:16. The server came back at
06:14:19 and then crashed again. At 06:15:21 recovery called the
sandbox. The plugin found it not started and started it again. The
runner identity marker was still on disk, but its process was gone:
`native.remote_runner_recovery_failed {stage: process_verification,
reason: process_not_running}`, then
`runner_remote_process_identity_unavailable`. The task went from done to
blocked. An earlier wave showed the same failure after a 4m45s outage.

**Expected behavior**
A sandbox that holds a live remote runner survives any controller outage
up to the reconnect grace. The server reattaches the runner and the run
continues.

**Steps to reproduce**
1. Create a Daytona environment with `autoStopInterval: 5` and a
`paperclip_runner` agent.
2. Start a remote run and kill the server for more than 5 minutes (less
than 7).
3. Restart the server. Recovery reports `process_not_running` and the
task becomes blocked.

## What Changed

- `server`: add `resolveRunnerReconnectGraceMsForRun`. It returns
`REMOTE_RUNNER_RECONNECT_GRACE_MS` for `paperclip_runner` on a sandbox
environment.
- `server`: forward `runnerReconnectGraceMs` to the provider worker on
lease acquisition and on the recovery reattach resume. It uses the same
provider-only path as `warmRunnerIdleTimeoutMs`. It is not lease
identity, lease metadata, or saved environment config.
- Daytona plugin: the effective auto-stop is at least the largest host
floor (warm idle window or reconnect grace) plus 5 minutes. With the
current grace this is 12 minutes. The plugin applies it at creation and
raises an older sandbox's shorter interval on resume (bounded, best
effort).
- Daytona plugin: keep the forwarded grace out of the normalized config.
Update the `autoStopInterval` manifest description.
- `doc/execution-semantics.md`: document the reconnect-grace floor next
to the warm floor.

## Verification

- Red/green Daytona plugin tests: `cd
packages/plugins/sandbox-providers/daytona && npx vitest run --config
vitest.config.ts`. The new cases failed before the fix (for example
`expected 120000 to be >= 720000`, and no `setAutostopInterval` call on
resume). All 605 tests pass after the fix.
- Server tests: `cd server && npx vitest run
src/__tests__/environment-run-orchestrator.test.ts
src/__tests__/environment-runtime.test.ts`. 247 pass. They prove
acquisition forwards the grace only for `paperclip_runner` on a sandbox,
that the reattach resume forwards it, and that it does not reach lease
metadata or saved config.
- `pnpm typecheck` for the plugin and `tsc --noEmit` for the server are
clean.
- `pnpm verify` asks for broad CI coverage because a doc changed.
Required CI supplies it.
- Evidence from the kept wave database: lease metadata shows
`autoStopInterval: 5`, `reuseLease: false`, `runnerLifecycleMode:
per_turn`. The run had no events between 06:09:16 and 06:14:19.

## Risks

- Cost: a runner sandbox that leaks (no cleanup) now auto-stops after 12
minutes instead of a shorter configured value. Cleanup still stops or
destroys released sandboxes. Explicit `0` and longer values are
unchanged. Auto-archive and auto-delete are unchanged.
- I did not inspect the sandbox state in Daytona directly. Cleanup
destroyed it before this investigation. The proof is the timing, the
lease config, and the `process_not_running` result after a restart that
kept the disk.
- No live Daytona replay ran.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
file edit, code execution) in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 01:36:18 -06:00
DottaandClaude Opus 5.5 037c58bb22 fix(daytona): keep the provider auto-stop strictly after the warm runner idle window (#19199) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A warm remote agent keeps its sandbox running after a turn. The host
keeps a warm runner for an idle window (`runnerIdleTimeoutMs`, or the
agent's `idleTimeoutMs`)
> - The Daytona provider also stops an idle sandbox after
`autoStopInterval` minutes. The default is 15 minutes
> - On staging, the warm idle window was 900,000 ms (15 minutes). The
two windows were equal
> - A follow-up turn at the end of the warm window resumed a sandbox
that Daytona was stopping. Daytona refused the resume in about 4
seconds, and the run failed at setup
> - #19195 added a bounded retry for that refusal. It did not remove the
cause
> - This pull request makes the two windows non-overlapping by
construction. Daytona derives its auto-stop for a warm reusable sandbox
from the same warm idle value that the host uses
> - The benefit is that a follow-up inside the warm window always finds
a running sandbox, and idle sandboxes still stop

## Linked Issues or Issue Description

Refs #19195 (bounded retry for a refused resume; this change removes the
common cause).

**What happened?**

On a staging soak, three follow-up runs failed at setup with a refused
sandbox resume. Each resume began 15 minutes 8 to 25 seconds after the
previous turn released the sandbox. The warm idle timeout was 900,000
ms. Daytona's default `autoStopInterval` is 15 minutes. Nothing in the
code kept these two values apart.

**Expected behavior**

A reusable sandbox that keeps a warm runner must not auto-stop inside
the warm idle window or at its edge. The provider backstop must fire
after the host's own idle handling, with a margin. Idle sandboxes must
still stop.

**Steps to reproduce**

1. Configure a Daytona sandbox environment with `reuseLease` and a warm
runner with `runnerIdleTimeoutMs: 900000` (or an agent with
`lifecycleMode: warm`, `idleTimeoutMs: 900000`).
2. Run a turn. Do not set `autoStopInterval`.
3. Before this change: the sandbox is created with `autoStopInterval:
15`, equal to the warm window. A follow-up near minute 15 races the
provider stop.

**Paperclip version or commit**

`c656b95b6b` (current `master`).

## What Changed

- Daytona plugin (`plugin.ts`): add `effectiveAutoStopIntervalMinutes`.
For a reusable lease that keeps a warm runner, the effective auto-stop
is at least `ceil(warm idle / 1 min) + 5` minutes. The plugin applies
the floor only when the host forwards `warmRunnerIdleTimeoutMs`, or when
the environment itself selects `runnerLifecycleMode: warm`. An explicit
`0` (disabled), a longer configured value, `inherit` with a per-turn
agent, per-turn, and non-reusable sandboxes keep the configured value.
- Daytona plugin: use the effective value at sandbox creation. On
resume, raise a shorter auto-stop on an existing sandbox. This update is
best effort and bounded to 5 seconds; the #19195 retry still covers a
refusal. The stale-handle refresh threshold keeps the configured
interval, so a cached handle is not trusted for longer before the
provider confirms the new interval. The warm window is not written to
the saved normalized config.
- Host (`runner-environment-lifecycle.ts`): add
`resolveWarmRunnerIdleTimeoutMsForRun`. It returns the run's effective
warm idle window (environment `runnerIdleTimeoutMs`, else the agent
`idleTimeoutMs`) for a warm `paperclip_runner` sandbox run. It shares
the lifecycle-mode resolution with `resolveRunnerEnvironmentForRun`.
- Host (`environment-run-orchestrator.ts`, `environment-runtime.ts`):
pass that value as `warmRunnerIdleTimeoutMs` to the sandbox-provider
driver on acquisition and on remote recovery resume. The driver adds it
to the provider worker config only. Lease identity, reuse matching, and
lease metadata are unchanged.
- Daytona manifest: describe the warm floor on `autoStopInterval`.
- `doc/execution-semantics.md`: document the contract.

## Verification

- Red (before the fix, same tests, with the floor disabled): 5 failures
in `plugin.test.ts` "warm idle window versus Daytona auto-stop". The
staging case failed with `expected 900000 to be greater than or equal to
1200000`. The resume case failed because `setAutostopInterval` was not
called.
- Red (host): the new `environment-runtime.test.ts` case first failed
because the provider worker config had no `runnerIdleTimeoutMs`. The
service wrapper dropped the field. The fix forwards it.
- Green: `npx vitest run --config vitest.config.ts` in
`packages/plugins/sandbox-providers/daytona`: 563 passed. This includes
a setter that never settles (resume completes after the 5-second bound)
and `inherit` with no forwarded window (explicit `autoStopInterval: 2`
stays 2).
- Green: `npx vitest run
src/services/runner-environment-lifecycle.test.ts
src/__tests__/environment-runtime.test.ts
src/__tests__/environment-run-orchestrator.test.ts` in `server/`: 242
passed. This includes recovery resume forwarding the admitted warm
window.
- The regression tests assert the relationship `autoStop >= warm idle +
margin` for several warm windows (125 s, 300 s default, 900 s, 1,800 s)
and for an explicit `autoStopInterval: 15`. They are not constant
checks.
- `node scripts/typecheck-low-memory.mjs` passes. `pnpm typecheck` in
the Daytona package passes.

## Risks

- A warm reusable Daytona sandbox now stays running up to the warm
window plus 5 minutes after its last activity, instead of 15 minutes.
With a 15-minute window this is 20 minutes. Cost is bounded and idle
sandboxes still stop.
- The resume-time `setAutostopInterval` call is best effort. If it
fails, the old interval stays and the #19195 retry still applies.
- Other providers are unchanged. They receive one extra host-owned
config key (`warmRunnerIdleTimeoutMs`) only for warm `paperclip_runner`
runs, and they ignore unknown keys.
- No lease identity change, so existing retained sandboxes are still
reused.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (shell, file edits, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-05 01:10:19 -06:00
Nicky Leach ea727dc3c2 Preserve plugin session continuity through warm execution and checkpoint recovery (#17820) 2026-10-26 20:25:10 -07:00
Nicky Leach aa0037a137 Support Codex warm plugin task reuse and recovery (#17390) 2026-10-23 19:14:14 -07:00
DottaandPaperclip 640dee1802 fix(runner): acquire reusable leases when switching to warm sessions (#14187) 
## Thinking Path
> - Paperclip coordinates agent work across task turns.
> - Warm runners need reusable sandbox leases.
> - Lease acquisition read only the environment reuse setting.
> - Startup then read the agent's new warm setting and rejected the
ephemeral lease.
> - This fix requests reuse before acquisition, with provider checks
intact.

## Linked Issues or Issue Description
**What happened?**
Switching an existing native runner task from per-turn to warm fails
with `runner_warm_environment_requires_reusable_lease`. Environment
`reuseLease` defaults to false. Acquisition therefore creates an
ephemeral lease, and capability narrowing correctly denies reuse.

**Expected behavior**
Changing lifecycle between turns requests a compatible lease without
replacing the task workspace or changing the shared environment.

**Steps to reproduce**
1. Start a native runner task with inherited lifecycle and environment
reuse disabled.
2. Change the agent from per-turn to warm.
3. Continue the task on a reusable-capable sandbox provider.

**Paperclip version or commit**
Reproduced on `a6c4e7a`. Related: #12904 established warm workspace
continuity; this fixes the earlier lease-selection mismatch.

## What Changed
- Pass agent settings into acquisition and derive a run-scoped reuse
request.
- Respect explicit environment lifecycle overrides; leave other adapters
unchanged.
- Keep provider capability, ownership, cleanup and restore gates intact.
- Add orchestration and database-backed transition tests; document the
contract.

## Verification
- Red: three new assertions failed before the fix.
- Green: 156 tests pass in environment-run-orchestrator,
environment-runtime, native-sandbox-lifecycle, and
environment-execution-target-capabilities.
- Database-backed regression proves ephemeral → reusable → resumed
lease, same workspace, and unchanged stored environment. Provider RPCs
are mocked, not live Daytona.
- `git diff --check` passes.
- Latest-head CI passes typecheck, build, tests, runner checks, E2E, and
canary dry run. Greptile: 5/5; no open threads.
- Recovery uses the persisted lifecycle. Direct regression: red before,
six cases green after. CI runs the added Vitest cases.
- Full local checks were unavailable (missing dependencies and earlier
memory limits).

## Risks
Warm mode now requests retained resources despite an environment's
default `reuseLease:false`. Existing idle timeout and cleanup still
apply. Unsupported providers remain denied. No migration, credential
[REDACTED], or shared-environment mutation.

## Model Used
OpenAI Codex agent; reasoning, code editing and test execution. Exact
model ID and context size were not exposed to this run.

## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket [REDACTED] or instance-derived details
- [x] I have run targeted tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-09-26 21:23:33 -05:00