Commit Graph
2 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 1b7ec13acb perf(native): default remote sandbox runs to a warm, reusable sandbox for measured harnesses (#20048)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents can run in remote sandboxes (Daytona), where each turn needs
the task's workspace and a runner inside the sandbox
> - With a default environment, every remote turn got a brand-new
sandbox: the whole workspace was uploaded again and a new runner
started, even for a follow-up comment
> - In a paperclip-sized workspace that made a follow-up take 30 s to
first token, against 8–10 s with a reusable warm sandbox
> - This pull request makes an agent that never chose a lifecycle run
warm on a sandbox for the measured harnesses (Codex, OpenCode, Claude),
so a follow-up keeps its sandbox, runner and workspace
> - The default applies only where warm can work: a sandbox provider
that keeps reusable leases, and a managed AI account that supports a
warm session. Everywhere else the run is per-turn, exactly as before
> - The benefit is fast remote follow-ups by default, without
re-uploading a large repository every turn

## Linked Issues or Issue Description

**What happened?**
On Daytona, with a fresh paperclip clone as the project workspace, a
follow-up comment took 30 s to its first token. The run log showed
`environment: fresh/created` on every follow-up. It spent 13.3 s in
`workspace_sync.transfer`, a full upload into the new sandbox, and 8–9 s
in `runner.session.resume`.

**Steps to reproduce**
1. Create a Daytona environment with the defaults: lifecycle inherited,
lease reuse off.
2. Run a Codex task in a large repository, then post a follow-up.
3. The follow-up gets a new sandbox, uploads the whole workspace again
and starts a new runner.

**Expected behavior**
A follow-up on the same task reuses the live sandbox, its runner and its
workspace (plan rule 2: no full re-upload per turn).

## What Changed

- **One rule for the default.** `defaultsRemoteSandboxRunToWarm`
(`native-runtime/native-warm-capability.ts`) decides the remote-sandbox
warm default for both the lease (environment view) and the runner
(native session policy), in one vocabulary (runtime backend plus ACPX
agent). It is applied only when no lifecycle was chosen anywhere:
admitted session, environment `runnerLifecycleMode`, or agent
`lifecycleMode`. It holds only when:
  1. the harness is measured (Codex, OpenCode, ACPX Claude);
2. the sandbox provider keeps reusable leases (Daytona, CreateOS and
boxd declare them; E2B, Modal and the built-in `fake` do not); and
3. with a managed AI account, that harness supports a managed warm
session (`supportsManagedNativeWarmSession`: Codex).
- **The harness resolves as the runtime resolves it.**
`resolveWarmProviderSelection` uses
`resolvePaperclipRunnerProviderProfile`, so an omitted `provider` is
Codex and ACPX without an agent is Claude, in both decisions. A config
the runtime would refuse gets no default.
- **Lease side** (`runner-environment-lifecycle.ts`,
`environment-run-orchestrator.ts`): for an eligible run only, the
orchestrator asks the provider before acquisition
(`providerSupportsReusableLeases` in `environment-runtime.ts`: the
acquisition rule, one registry lookup for a plugin provider, no
worker-ready wait). With support, the run-scoped view sets `reuseLease:
true` and forwards the warm idle window to the provider's auto-stop. The
shared environment is never rewritten. Every other run resolves exactly
as on master, without the probe.
- **Runner side** (`native-lifecycle.ts`, `runtime-selection.ts`): the
native policy applies the same rule to the acquired lease
(`reusableLeaseConfigured && effectiveCapabilities.reusableLeases`). If
acquisition downgrades the lease anyway, the run falls back to per-turn;
a default never produces
`runner_warm_environment_requires_reusable_lease`.
- **Idle window.** The remote default uses the window the environment
view forwards to the provider: an inheriting environment's
`runnerIdleTimeoutMs`, then the agent's `idleTimeoutMs`, then 5 minutes.
The sandbox execution target now carries the environment's window
(`runnerIdleTimeoutMs`, an optional field in `adapter-utils`'s execution
target) so the runner and the provider auto-stop agree. An explicit warm
choice and the local default are unchanged.
- Unchanged: explicit agent choices, environments that pin a lifecycle,
plugin-task environments, admitted sessions (`persistedPolicy`), and
local behavior.
- `tests/restart-chaos`: the vCPU budget check counts `--daytona-inherit
--lifecycle default` sandboxes as retained, because they now default to
warm.
- `doc/plans/2026-11-08-large-workspace-speed.md`: the measurements, the
ranked waits, and the gate.

## Verification

- Unit and integration tests, one per rule:
- Rule 1 (provider gate): `native-lifecycle.test.ts` (ephemeral lease →
per-turn), `runner-environment-lifecycle.test.ts` (no reusable leases →
environment unchanged, no warm window),
`environment-runtime-reusable-lease-probe.test.ts` (`fake`, an
undeclared plugin, a declared plugin whose worker does not verify the
reuse methods, and a plugin without a running worker read false),
`runtime-selection.test.ts` ("defaults an agent without a lifecycle on
...": reusable → warm/keep_running; reuse configured on an unsupported
provider → per_turn/destroy_after_turn; ephemeral → per_turn),
`environment-run-orchestrator.test.ts` (probe result decides
`reuseLease` and the forwarded window).
- Rule 2 (explicit configuration wins): explicit per-turn agent, pinned
environment, admitted session, configured `idleTimeoutMs`, managed
Claude per-turn, managed Codex warm; the orchestrator does not probe
when a lifecycle is explicit.
- Rule 3 (omitted provider): `{}` resolves to Codex in the environment
view; `runtime-selection.test.ts` "agrees with the environment view's
lease for ..." feeds the environment view's lease into selection for
Codex, a provider-less Codex agent, OpenCode, ACPX Claude, ACPX without
an agent, and Copilot (per-turn), and checks both decisions agree. A
plugin-task target stays out of the default.
- The two suites that failed on the previous head
(`runtime-selection.test.ts`,
`workspace-recovery-preparation-dispatch.test.ts`) pass locally.
Locally, 6 connector-skill cases in `runtime-selection.test.ts` fail
with `ENOENT .claude/skills/slack/SKILL.md` because this checkout is a
nested worktree; they do not touch this change and pass in CI.
- End-to-end check of this default on Daytona (before the gate was
added; Daytona keeps reusable leases, so the gate does not change this
path): a user-default environment (lifecycle inherited, lease reuse
unset, `--daytona-inherit`), a fresh paperclip clone, built from the
image's source revision with this change (branch measure/remote-warm).

  | | Before (per-turn sandbox) | After (this change) |
  |---|---|---|
  | Follow-up, Codex | 30.4 s | 7.6 s |
  | Follow-up, Claude | about 30 s | 11.0 s |
  | First follow-up after an idle K1, Codex | 45.9 s | 20.3 s |
  | First follow-up after an idle K1, Claude | 38.3 s | 18.3 s |

The run-log reuse records show the follow-ups reusing the workspace,
environment, runner and provider session, with no
`workspace_sync.transfer`. First turns still upload the workspace once
(14.8 s and 26.1 s). Every sandbox was cleaned up.
- Daytona restart chaos wave (K1/K2/K4 with a mid-run comment,
inheriting environment, agents with no lifecycle, runner image built
from this branch): wave `w-20261110T171438-s20048`, seed 20048, on
b7f6243d60 with image `paperclip-daytona-runner@sha256:76a33275…6fe29`
(built from this branch by the `image_only` runner workflow; runner
content id `ed503454…a479` matches every later commit here, which change
only server, adapter-utils types and tests). `--daytona-inherit
--lifecycle default --adapters codex,claude --workloads A1,C1 --triggers
time,comment --faults K1,K2,K4 --tasks 6 --concurrency 3`.
- **6/6 tasks pass** (R1–R6 and T), all 6 crossed a restart; cells:
Codex A1 K2, Codex A1 K4, Claude A1 K4, Claude C1 K1, Claude C1 K4,
Codex C1 K2 at the mid-run comment. **No task needed Retry** (R3 n/a on
every task; one Claude task recovered automatically from
`process_lost`). No open signatures, 0 speed breaches.
- The default applied: the R6 sweep found 6 retained reusable leases
(one per task, Daytona keeps them) and no orphan or leaked sandbox; the
environment was deleted cleanly.
- Timings (Daytona, first turns, n=3 each): TTFT p50 Codex 21.9 s,
Claude 31.0 s. Restart recovery p50/p95: Codex 10.5 s / 29.6 s (n=3),
Claude 8.8 s / 23.9 s (n=2). Warm follow-up TTFT has no clean sample in
this wave (both follow-ups were crossed by a fault), so the follow-up
comparison above (30.4 s → 7.6 s Codex, ~30 s → 11.0 s Claude) remains
the speed evidence; no separate per-turn chaos wave was run.
- An earlier attempt, `w-20261110T170357-s20048`, never started tasks:
the first sandbox from the new image stayed `creating` past the
harness's 300 s Codex qualification bound (Daytona building the new
snapshot). It left one sandbox that the harness reported after deleting
the environment; the rerun on the cached snapshot is the wave above.
- The idle-window fix (9a74116082) landed after the wave; it changes
only runs whose inheriting environment sets `runnerIdleTimeoutMs`, which
this wave's environment does not.

## Previous-release story

No persisted state changes shape. A session admitted per-turn before
this change keeps its saved policy (`persistedPolicy` wins) until a new
session starts. A run in flight at the cutover keeps its admitted
lifecycle for recovery (`admittedLifecycleMode` wins in the environment
view).

## Cost and opt-out

- **Cost:** after a turn, the sandbox and its warm runner stay up for
the warm idle window (the environment's `runnerIdleTimeoutMs`, else the
agent's `idleTimeoutMs`, else 5 minutes). Then the controller suspends
the runner, and the provider auto-stop follows the same window. One idle
window per active task.
- **Opt-out:** set the agent's lifecycle to "Turn by turn"
(`lifecycleMode: per_turn`), or set the environment's
`runnerLifecycleMode` to `per_turn`. Either wins over this default.

## Reuse scope, restart and saved work

- **One lineage per sandbox.** The Daytona plugin keys a reusable lease
by the agent plus the execution workspace (or issue). Another task never
gets a sandbox holding a different task's files, and subtasks still get
fresh workspaces from the project base, so they get their own sandbox.
The workspace is adopted only after its stamp matches the host's.
- **After a restart,** a warm sandbox is reattached or replaced through
the existing warm recovery and owner fencing. A sandbox with a live
owner is never claimed twice. The Daytona chaos wave above checks this.
- **No lost work.** Each turn's workspace copyback
(`workspace_finalize`) runs at the end of the turn, before any idle
suspend, as for per-turn runs.

## Risks

- **Cost:** an idle sandbox stays up for the warm idle window (5
minutes) after a turn, then suspends. Only on providers with reusable
leases.
- **Probe cost:** one plugin-registry lookup before acquisition, only
for runs the default can apply to. No wait for a worker: a provider
whose worker is not running yet reads as unsupported, so that run is
per-turn.
- **Known gap (pre-existing, not widened):** with an explicit agent warm
choice on an inheriting environment that sets `runnerIdleTimeoutMs`, the
provider gets the environment's window while the host uses the agent's,
as on master.
- **Restart behavior on warm remote sandboxes** uses the existing warm
recovery paths. The Daytona chaos wave above checks K1, K2 and K4.
- Sessions admitted per-turn before this change stay per-turn until a
new session starts.
## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file editing, test execution, Daytona measurements).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 11:49:14 -06:00
DottaandPaperclip 92b238137b Connect and configure GitHub Copilot through Apps (#15705)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps and AI Connections store credentials and govern access to agent
runtimes.
> - Users need a supported saved-token connection and config values that
survive save and reopen.
> - The release must preserve saved-token access and task lifecycle
guarantees.
> - This pull request will connect Copilot through the normal Apps and
AI Connections path.
> - The benefit is a reviewable part of the Copilot production release.

## Linked Issues or Issue Description

Refs #15701 and #15703. Supersedes setup work in #15562. Base: #15703.
Next: #15706.

## What Changed

- Require configuration update authority before Copilot
credential-bearing model discovery; default to the saved environment and
preserve explicit selection for editors.

- Keep Copilot discovery passive during reconnect reconciliation;
refresh on explicit opening, account/environment changes and reconnect.

- Add GitHub AI token connections with encrypted storage, ownership,
grants and audit logging.
- Keep repository access separate from Copilot runtime authentication.
- Probe the selected environment without sending a model prompt and
preserve authenticated model selection through save/reopen.
- Keep fresh execution admission consistent with normal Copilot setup;
persist task model choices without changing provider or permission
policy.
- Refresh authenticated models after reconnecting the same saved Copilot
account, including recovery from a previous authentication error.
- Apply the additive provider-default constraint migration through the
normal generated migration workflow.

## Verification

Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`.
History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`.
Remaining source
[`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d)
contains the four-file authorization repair. The remaining PR trees
preserve six nonoverlapping files from the newer public mainline
managed-settings fix and the CI-owned lockfile. No unrelated provider is
enabled.

The security finding is repaired: existing-agent discovery requires
`agent_config:update` before connection selection or token resolution.
The saved environment is the default. Authorized editors can select a
prospective environment. Both denied board-viewer and agent-reader
regressions fail on the original source. All49 route tests and server
typecheck pass with the fix. Denied requests perform no connection
selection, credential resolution or probe. Current-head security scans
pass on both remaining PRs; the repaired thread is resolved.

Copilot CLI1.0.88 and profile47 retain digest
`sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`.
The [qualification
record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json)
reports20/20 Product cells,154 assertions and7/7 protocol cases,46
assertions against frozen normal installed source
`c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification
model is `gpt-5.6-luna`. Every passing Product case confirms cleanup.
Original failures remain recorded; automatic behavior retries are
disabled. The route-only authorization repair changes no bound
runtime/profile bytes and does not relabel the frozen campaign.

The ordinary installed remote journey passed saved-token selection,
model discovery, normal UI save/reopen, metadata-only Test, completed
task, accepted receipt and exact19-byte browser download. An open model
catalog caused no additional discovery requests during117seconds. Actual
installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta
and native Linux x64 on Daytona. Raw token browser entry was not
exercised; encrypted connection creation used the normal API.

The first Daytona file attempt timed out during delayed semantic
dispatch; its precise stalled layer remains unresolved. Independent
ordinary remote delivery and the other19 Product cells passed before one
instrumented recheck passed8/8 with unchanged source,prompt,assertions
and deadlines. No causal latency repair is claimed.

Before the authorization repair, all four final PR heads passed required
verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads.
Current-head CI/review is running for the two repaired heads. Local
focused checks retain129 connection,59 runner,72 shared-eval and88
recovery/workspace passes, plus migration safety and token gates. The
additive Copilot migration is361; released master history is preserved.

[Cloud normal-package
run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704)
builds the pre-security source4f3f. All four native producers and Linux
package assembly pass; the final clean installed consumers are
finishing. All18 retained archives match their hashes;37,863 regular
files contain zero selected credentials. That package does not contain
the later route repair. The actual final published package must include
it and pass the local/Daytona live canaries. The frozen Daytona image
remains published and independently verified. No separate full-image
scan is claimed. Docker builds run in the cloud.

**Remaining:** current-head CI/review, setup and qualification merges,
final publication, and ordinary local/Daytona canaries against the
actual shipped package. The user explicitly authorizes the maintainer
review override; CI remains a merge gate. No production deployment is
claimed. Provider USD remains unknown and GitHub-attributed.

## Risks

Bad credentials, missing entitlement, runtime installation and
unavailable models must remain distinct failures. A connection must not
adopt ambient credentials. The constraint migration changes accepted
provider values, with no credential-storage migration.

Current-master NewAgent fixture extraction is retained. Copilot cases
use that shared support and test both passive discovery and explicit
reconnect under the same setup lifecycle.

The onboarding retry regression shares the component suite’s
saved-connection, authenticated-model, draft, agent-list, hire and
render lifecycle fixtures. Keeping it beside the existing onboarding
failures makes adoption versus duplicate hire observable under the same
lifecycle; this is the concrete cohesion reason for the existing large
test file.

## Model Used

OpenAI Codex, based on GPT-6. The exact serving model ID and context
window are not exposed in this task. Used for code changes, review and
tool-assisted verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 07:02:22 -05:00