Files
PaperClipAI/doc/plugins/PLUGIN_RUNNER_EXECUTION.md
T
Nicky LeachandPaperclip 731b52453c fix(environments): verify workspaces on run leases and release settled tasks (#21289)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core owns run leases and delegates remote execution to environment
plugins.
> - A retry must verify its old checkout on the placement that will run
the task.
> - A completed task must still release its provider resources.
> - This change uses the existing lease for workspace verification and
always calls the release hook after settlement.
> - Saved settlement proof lets cleanup retry without repeating
execution work.

## Linked Issues or Issue Description

Refs #21237. That PR records stop proof before lease release. This PR
keeps provider cleanup after settlement and changes workspace
verification sequencing. Both touch task completion; preserve both
changes if the merge order requires conflict resolution.

**What happened?**

Workspace verification ran before a portable task had a lease, so a
provider could not verify the checkout on its selected placement. Core
also skipped the provider release hook after successful task completion.
This could leave resources allocated after the run ended.

**Expected behavior**

Verify the old checkout through the selected run lease. A confirmed
refusal can use the authorized fresh-workspace fallback. A transport
failure must remain retryable. Call the provider release hook after
settlement and retry failed cleanup without changing the run result.

**Steps to reproduce**

1. Retry a task whose prior checkout uses plugin-managed storage and
portable task placement.
2. Verify that the plugin receives the same task lease for verification
and preparation.
3. Complete a task, fail the provider release once, and reconstruct the
cleanup controller.
4. Confirm that cleanup calls only the release hook and retains the
completed run outcome.

## What Changed

- Acquire the logical run lease before provider workspace verification
and reuse it for preparation and execution.
- Preserve the existing session and workspace choices. Recompute
workspace-dependent inputs only after a definite refusal; acquire
credentials and managed runtime inputs once.
- Save the exact refusal and lease before cleanup. Use a
workspace-scoped creation key only for the resulting fresh attempt.
- Replay interrupted verification on the same unlaunched lease. Keep
transport errors retryable and reject replay after execution starts.
- Always call the provider release hook after task settlement. Pass
refreshed settlement proof and route failed releases through the
existing cleanup reaper.
- Add database regressions for call order, restart, retained writers,
refusal, transport errors, and settlement-only cleanup replay.

`plugin-workspaces.ts` grows from 997 to 1,001 lines. The four added
lines extend the existing workspace transaction and lease callbacks.
Keeping the checks there preserves the ownership assertion, provider
result validation, and binding update in one operation. Extracting only
the verification branch would split that transaction without removing a
separate responsibility.

## Verification

- Server TypeScript check passed: `tsc -p server/tsconfig.json
--noEmit`. The complete `server/scripts/typecheck-layers.mjs` check also
passed, including all test-tree layers.
- The first CI typecheck caught a readonly array in a test fixture that
the narrower source check did not include. Commit 8d9ec19e7a corrects
that fixture; all layered checks pass afterward.
- Added a public heartbeat refusal regression in 7c598a4da3. All ten
session-preparation tests pass; independent execution of the refusal
case also passes. It checks successful completion, the saved fresh
workspace, a new conversation, and one credential acquisition.
- Nine database/service suites passed 335 tests: plugin warm tasks,
plugin runner tasks, startup cleanup recovery, credential writeback,
plugin workspaces, session preparation, plugin environments, retry
workspace continuity, and heartbeat workspace preparation.
- After the final restart change, both targeted refusal/restart tests
passed. The primary agent independently reran those cases and the final
server typecheck. After integrating current master, the primary agent's
independent rerun passed all 237 tests across the warm-task,
plugin-workspace, session-preparation, and heartbeat-workspace suites.
- `pnpm verify` returned exit 2 and requested broad CI coverage because
the plugin execution contract documentation changed. No full local
Core-suite pass is claimed.
- Previous-revision CI was green on 8d9ec19e7a: 59 checks passed and 3
were skipped. The first run failed the unchanged legacy-continuation
browser assertion and credential-cleanup server shard. Each passed on
one targeted retry with no code changes. The credential-cleanup
integration suite also passed all 21 tests locally on Node 22. The
browser failure concerned an existing continuation metadata marker; the
server failure cascaded after a cleanup error left a test spy installed.
The successful retries do not establish the cause of the first failures.
- Local database fixtures use simulated provider workers. No live
provider or deployed environment was changed.

- Additional verification for 59c7a1dae8: the layered server typecheck
passed. Independent execution passed all 265 tests across durable task
retries, workspace preparation, and public heartbeat preparation. The
new shutdown regression preserves the fresh Retry choice, waits for
parent cleanup, and prevents the successor from inheriting the parent
lease reference. Same-run recovery retains its receipt. Required CI is
green on 59c7a1dae8: 58 checks passed and 3 were skipped. Standard
review scored 5/5 with no unresolved comments. The separate Superagent
Security Scan is neutral/inconclusive because its workflow failed; a
direct rerun request returned HTTP 404. That scan is not claimed as
passed.

## Risks

- Providers must use the supplied task lease for verification and return
a definite `binding_mismatch` only when the checkout is unconfirmed.
Transport errors retain the retry decision.
- A refused checkout waits for confirmed release before a fresh
allocation. Failed deletion remains owned by the existing cleanup
reaper.
- Task settlement and physical resource release stay separate. Retained
handoff leases do not enter this release path.
- No migration or SDK schema change is required.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) implemented the changes and tests.
The primary Codex agent independently reviewed the connected Core and
plugin paths and ran checks. A separate GPT-6.1 Sol agent reviewed
lifecycle and restart boundaries.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue using the bug template labels
- [x] I have not referenced internal or private project details
- [x] My branch name describes the change
- [x] I have added relevant tests and documentation
- [x] I have considered and documented risks
- [x] All required CI gates are green
- [x] Greptile is 5/5 with no unresolved comments
- [x] I will address reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-12 21:56:02 -08:00

33 KiB

Plugin Runner execution

Core can run paperclip_runner tasks in a plugin environment. The plugin owns the execution host and its files. Core owns run identity, selected policy, assigned tools, and the PRP journal.

Managed AI connection required

A task-capable environment runs only agents with a managed AI connection. Core does not use an account installed on the plugin's execution host. Before it acquires a lease or calls the plugin, Core fails a new run that has no managed AI connection. A homeMode: "preconfigured" Runner home does not use the agent's AI connection, so Core refuses it in the same way. The run fails as configuration_incomplete with reason ai_connection_required. The task is blocked with a notice that names the fix: choose an AI connection, or switch to an isolated Runner home. Agent verification applies the same rule before it resolves the probe target, so it acquires no lease. The agent Test route also checks before it resolves secrets or allocates a test target. A setup probe without a managed AI connection, or with a preconfigured home, fails the ai_connection_required check and calls no task operation. Verification then waits for the configuration to change. A resumed native run keeps the contract it was admitted under.

Required hooks

Set supportsTasks: true on the environment driver. Declare the environment.drivers.register capability. Implement these hooks:

  • onEnvironmentAcquireLease
  • onEnvironmentTask
  • onEnvironmentReleaseLease
  • onEnvironmentDestroyLease

Task execution does not require onEnvironmentExecute or onEnvironmentRealizeWorkspace. The task API has no shell command or file installation operation. Use a separate workflow to install agent files. Providers that advertise workspaceFileReadVersion: 1 can return task output files through the bounded read_file operation for attachments and deliverables.

Core supports per-turn workspace sessions with Codex, OpenCode, and ACPX. OpenCode and ACPX require the existing qualified provider profiles. Workspace-free providers are not supported here. Drivers opt into warm retention through warmTaskVersion: 1. Core prefers warm execution for new task runs on capable drivers, including ACPX; existing provider admission still determines which harnesses the driver supports. The plugin owns project workspace preparation. Core does not prepare or reuse a Core Git worktree for a task-only environment. Selecting a plain task directory does not add redundant default configuration. Retaining that directory does not itself change the run configuration. For plugin-backed project storage, Core uses the workspace provider contract. The storage plugin must also provide the selected execution environment. Its declared workspace modes and strategies remain in effect. Core passes the pinned workspace binding with each task operation. Core rejects configured workspace services for task-only environments. Fresh retries that import retained files are not supported for plugin storage. An agent-owned workspace cannot move to another agent or environment. Core also rejects reuse of a local workspace in a task-only environment. Keep its original owner, or use the workspace storage provider contract.

Prepare and launch

  1. Return an opaque task ID from lease acquisition as providerLeaseId.
  2. Return the installed Runner version and SHA-256 digest from describe.
  3. Return launchPreparationVersion: 1 from describe.
  4. Handle prepare with the launch preparation contract.
  5. Save the result under its configuration ID.
  6. Handle submit with that ID in preparedLaunchId.
  7. Return an authenticated WSS endpoint from connection.

Preparation must not start a process. It must preserve Core's provider, model, policy, instructions, and resource assignment identities. Resolve assigned resources to execution-host paths. Use bindRunnerLaunchResources and createRunnerProviderLaunch when the built-in descriptor builder is suitable.

Core checks the returned requirements digest and descriptor before it registers PRP or submits the task. A failed check does not enable a fallback launch. Recovery within the same run must return the exact saved configuration.

The plugin supplies executable paths and the provider workspace in the descriptor. It owns the process environment and account installation. For example, a persistent computer can use these locations:

HOME=/home/agent
AGENT_HOME=/home/agent/.agents
CODEX_HOME=/home/agent/.codex
instructions=/home/agent/.agents/AGENTS.md
skills=/home/agent/.agents/skills/<name>/SKILL.md

These are execution-host locations. Core does not copy the home for each run. The plugin must declare projectAuthCheckVersion: 1 for all native tasks. Core requires an allowed check_project_auth result, or a prepared result with projectAuthCheckAtSubmit: true. The latter delegates the check to the trusted plugin's submit handler before credential installation or task start. The plugin must reject conflicting project or parent authentication settings. It must keep the checked configuration fixed until submission. See launch authentication validation.

Core sends authorized project IDs, the PRP identity, launch limits, and the bootstrap ticket through submit. It does not send a process command. The plugin must submit each task at most once. A lost reply requires status reconciliation for the same task.

Credentials and tools

Core can transfer a selected Codex account or a direct Claude credential. Other managed routes fail closed until they have a typed credential contract. Do not put credentials in task metadata, journals, logs, or errors.

Assigned MCP tools run through the Core gateway over PRP. The gateway credentials stay on Core. The plugin must preserve the MCP binding identity in the prepared context.

Core records credential writeback authority before dispatch. After process termination, Core saves refreshed credentials before it completes the task. A failed save retains the task grants for cleanup.

Stop and recovery

A terminal phase does not prove process termination. Return executionStopped: true only after the task and its descendants stop. Return sessionStateAvailable: true only when the suspended state remains usable. Keep retained state when Core completes the task. Core checks managed plugin workspaces through their storage provider before completion, including after a restart. Core does not inspect these folders locally. If this check fails, Core defers the run and continues recovery for other runs. Core keeps known branch and owner error codes with the repair instructions.

The task contract exposes two checkpoint operations:

  • read_state returns the exact Runner durable state for the supplied identity.
  • archive_state moves a suspended Runner authority checkpoint under the supplied archive key. It returns the same state on retry. It keeps provider session files.

These operations do not accept paths. Core checks the active lease and retained source task before it calls the plugin. The existing Runner transport validates state identity and suspension before it rotates authority.

For controller replacement, Core restores its encrypted PRP journal from Postgres and reuses the original active task lease. The task owner checks running status and exact Runner identity, then reacquires the connection. It does not call prepare, submit, or stop. The running artifact remains pinned even if describe reports a newer installed version. Launch preparation files on the old controller are not required to adopt an already running task. The five-minute outage contract and its prerequisites are described in the continuity recipe.

Core checks company, agent, environment, workspace, driver, and termination receipts before it supplies resumeFrom. Historical controller recovery without the durable journal first stops and releases the old task. A failed stop prevents replacement. A new run prepares its current assignments. Provider compatibility checks remain in the existing backends. Missing or incompatible retained state fails closed.

Setup verification

Core creates a private probe lease with an expiry. It runs the same describe, prepare, submit, and WSS/PRP path. The probe checks provider authentication and the selected model with tools denied. Core bounds Runner lifetime to the remaining probe time. It then requires suspension, confirmed termination, and task cleanup. For a new agent-owned computer, the saved-agent lifecycle runs verification after it saves the agent. Core does not require a computer before this save.

Verification

The Runner fixture uses a synthetic Codex provider and local sockets. It does not qualify a deployed plugin, live account, or production WSS proxy.

Workspace completion and maintenance

When the selected task driver declares sessionHandoffVersion: 1, Core reserves the managed workspace before it acquires the run's task lease. Core reconciles placement ownership, then passes taskLease to the workspace preparation call. The plugin uses the same allocation for checkout and Runner admission. Finalization records that placement before calling the plugin and reuses it on retry. If an interrupted finalization has no recorded placement, Core retries its original request without placement. This preserves the operation ID and the provider's established routing for requests made before this upgrade.

Before dispatching preparation for an unbound workspace, Core records the run and agent, plus the task placement when present. Inspection replays the saved request. A retry with the same run, agent, and placement uses the same operation ID. When a retry acquires a different task, Core records its placement and prepares the same workspace. The provider settles the prior preparation before using the retained checkout. Reservation reconciliation uses project Git consent while keeping the original run and agent in the provider request.

A portable restore uses the durable ownership claim and the same account identity after a credential refresh or controller replacement. It starts a new Runner on the target. Run attachment applies the current tools and runtime configuration to the restored provider session. After a confirmed workspace transfer, Core can use the same target to start a new conversation when its session compatibility check requires one. The new conversation starts without the prior provider checkpoint, and the ownership claim continues to fence the source.

A storage provider can declare workspaceProvider.workspaceVerificationVersion: 1. Before creating a fresh workspace for an admitted Retry, Core may send the original task-owned workspace's prepare request with verifyExisting: true. Core sends verification through workspaceOperation with a 10-second deadline and a 12-second RPC timeout. Drivers with task placement receive the selected run's taskLease before Core describes the Runner task. Use that lease to select the same execution placement for verification and later preparation. The provider must verify a completed checkout without creating or fetching one and return prepared with created: false. Core checks prior writer settlement and continues normal preparation on the retained workspace after verification. Return the workspace failure reason binding_mismatch for a definitively missing or invalid checkout. Core can then release the verification lease and use the authorized fresh retry fallback. Transport failures and timeouts keep the current checkout decision retryable. Local workspaces use the same ownership and writer checks with Core's existing local Git verification.

A storage provider can declare workspaceProvider.finalizationVersion: 1. Core then sends workspaceOperation with operation.kind: "finalize", the retained binding, lifecycle generation, and originating runId and agentId. The operation must verify workspace identity and make the run's retained writes durable before returning { kind: "finalized", headSha, branchName }. Git workspaces must still report the assigned branch. Non-Git workspaces return null Git fields.

Completion permits dirty, untracked, ignored, and unpublished work. It does not prove remote recoverability or authorize deletion. The provider chooses how to meet this contract; Core does not prescribe a VM, filesystem, or Git layout. Core gives completion a 30-second deadline. An unavailable execution host or an unconfirmed durability barrier must fail retryably, never return success. Recovery receives the original run and agent context.

The existing agent-independent inspect and remove operations continue to protect storage maintenance. Providers that do not declare finalization version 1 retain the inspection-based completion path. No stored binding migration is required.

A provider whose inspection cannot prove merge ancestry may declare workspaceProvider.supportsMergeAncestry: false. Omission preserves the existing ancestry inspection behavior. For an opted-out provider, automatic cleanup uses its latest successful finalization HEAD as a hint for destination-aware GitHub checks. It discovers associated PRs without requiring a recorded work product. A direct ancestry proof or a merged PR with the exact final HEAD, intended base repository/branch, and reachable integration commit must precede storage inspection. A fresh matching HEAD and branch, remote recoverability, no local data, terminal tasks, and the existing lifecycle fences still govern deletion. Open or unknown delivery for a known commit retains a bounded remote-only retry. Missing candidate identity parks until a relevant write or startup recovery. Explicit close-readiness and pending lifecycle recovery retain their checks.

A provider may declare workspaceProvider.recoveryPullRequestVersion: 1. Core can then include recoveryPullRequest: { number, headSha } in inspect and remove. This is a narrow recovery hint, not permission to delete. The provider must use its trusted source repository, validate that the specific PR recovery reference still identifies headSha, and independently check the actual workspace and all owned data. Arbitrary remote URLs and reference names are not accepted. Absent capability means Core does not send the extra field. Before dispatch, Core persists the negotiated removal payload and its operation ID. Retries replay both unchanged, even if capabilities change, while still enforcing company, provider enablement, binding and generation checks. A downgraded provider can reject an unsupported payload; Core does not silently change its identity. Legacy intents without a recovery hint retain their deterministic ID. A hinted intent without a recorded ID is ambiguous and fails closed. None of these receipts replaces the provider's checks.

A reservation is a workspace whose prepare never returned a binding. When its source task is done or cancelled, no task points at it, and no run for the task is queued or running, Core archives it and sends remove with binding: null and expectedHeadSha: null instead of reconciling it with an agent-free prepare. The provider must resolve its own record for that execution workspace, discard anything the unfinished preparation created, and return { kind: "removed" }, including when nothing was created. The removal is persisted and replayed with the same operation ID like any other removal. A provider that refuses it fails the removal, which backs off like any other cleanup failure. A reservation that a task still points at, or whose task is open, keeps the prepare reconciliation.

When terminal cleanup inspection or removal fails, Core records metadata.terminalCleanupRetry on the workspace with the failure count, the next attempt time, and the failure kind, reason, and Core's message. Retries back off from 5 minutes, doubling up to 4 hours. A successful inspection or removal clears the record, and a reopen, which raises the lifecycle generation, discards it. Other wakes, including provider upgrades, wait for the next attempt time. Core keeps the record through workspace metadata updates.

The optional prepared.destinationRef records the resolved delivery branch as refs/heads/<branch>, up to 266 characters including the prefix. Providers should return null when preparation cannot establish an unambiguous branch destination or its canonical ref exceeds this receipt limit; an optional receipt must not fail an otherwise successful preparation. Core stores this alongside the source snapshot; callers cannot overwrite it through workspace metadata updates. Older providers omit it. Core only falls back to their captured explicit source refs, not a mutable project configuration or a newly chosen default branch. A source SHA or tag may support direct ancestry, but does not imply a PR base branch. Provider upgrades, activation, company re-enablement, and finalization settlement wake cleanup transactionally. No table, independent timer, or background scan of all remote PR refs is required.

Asynchronous workspace preparation

A provider can implement onWorkspacePrepare alongside onWorkspaceOperation. Core calls the optional method with phase: "submit" and a short submission deadline. A provider that cannot prepare asynchronously returns { kind: "unsupported" } before starting work, so Core can use the original operation. Otherwise the provider returns the completed result or a { kind: "pending", deadline } receipt. The provider sets and enforces that operation deadline. Core holds the workspace lifecycle lock during preparation and rejects offers beyond the previous synchronous RPC budget before polling. It honors accepted deadlines as supplied. Replayed submission must not duplicate an active prepare.

Core sends phase: "poll" with the same operation identity and the deadline from the receipt. The provider returns the same receipt while work is pending and the usual scoped prepared result after success. Core rejects late results and stops polling when the deadline expires. Each submission and poll has a short transport bound independent of the operation deadline. A provider can keep reporting pending while it restores recovery snapshots, under the same deadline.

Each invocation has its own scoped host access. Complete scoped host calls inside the submit or poll that owns them. Detached work can use a remote operation already authorized by that invocation. It cannot reuse the invocation to call host services after the reply.

Workspace operation failures

Core separates a failed workspaceOperation call by what it knows about the outcome:

Failure Run error Core behavior
The worker was not running, so no request was sent Workspace storage provider is not running; retry the same workspace A run that has not started provider work is retried as plugin_worker_unavailable.
The call timed out, or the worker exited with the call pending A fixed timeout or stopped-provider message that says to retry the same workspace The operation may have run, so the same workspace and operation identity must be retried.
The provider threw an error with a known reason The fixed message Core keeps for that reason Only transient reasons tell the user to retry.
The provider threw any other error Workspace storage provider reported a failure; retry the same workspace Unchanged setup failure.

To report a reason, throw PluginWorkspaceOperationError from onWorkspaceOperation or onWorkspacePrepare, or any error with an own workspaceFailureReason property. The reason must come from WORKSPACE_OPERATION_FAILURE_REASONS:

Reason Retrying the same workspace can help
unsupported_operation, configuration_unavailable, credential_unavailable, storage_grant_unavailable, binding_mismatch, runtime_connection_invalid No
storage_pending, runtime_unavailable, remote_incomplete, context_changed, deadline_exceeded Yes
import { PluginWorkspaceOperationError } from "@paperclipai/plugin-sdk";

throw new PluginWorkspaceOperationError("storage_grant_unavailable", "Project mount grants unavailable");

The SDK sends only the reason. Core never reads the provider's message, so a message cannot reach the run, the API, or Core logs. The run keeps its setup_failed error code and records resultJson.workspaceProviderFailure with the failure kind, reason, and retryable flag. The heartbeat execution setup failed log carries the same fields. A provider may also record a plugin.workspace.failure span whose plugin.failure_reason is a reason from this set or unknown.

Warm task retention

A driver declares warmTaskVersion: 1, implements onEnvironmentResumeLease, and returns warmTaskVersion: 1 from describe only when its execution host supports authorized run attachment. Newly hired agents targeting such a driver are configured for warm execution. Existing agents keep their explicit policy; plugin runs without an explicit policy prefer warm execution. There is no provider-specific warm gate or additional setting. Existing runs retain their admitted lifecycle policy during restart recovery. The existing runnerIdleTimeoutMs controls bounded retention. Fresh warm admission requires a Core-managed credential identity before lease acquisition. The saved warm preference does not bypass account ownership checks. Drivers without warm capability and setup probes remain ephemeral.

Plugin task execution uses the public, run-scoped GitHub credential broker when the provider advertises its versioned contract. Warm preparation updates the command capability before admitting the new run. Restart recovery retains the original run binding; new work must not reactivate an older capability. Project authorization is still checked for each new run.

A Launcher task owns one Runner process, independently of Core heartbeat runs. Successful warm run completion retains the task and its mounts. The reuse policy is saved with the selected native runtime, independently of workspace copy-back records. Recovery uses that policy when no workspace-sync record exists, so it does not stop a Runner that foreground completion retains. Failed and cancelled runs still stop instead of staying warm. The reuse fingerprint includes the authorized project set; changing it retires the previous task before checkpoint recovery. Adoption and each warm handoff require a current, configuration-bound project-auth check against the exact retained mount scope. Core atomically hands its retained provider identity to one new run lease, revalidates current project authentication, prepares the new run's resources, and uses PRP run.attach. Preparation includes the exact priorIdentity and stable accountIdentity. The provider journals that transition idempotently before allowing the new authority. The original ingress address remains unchanged. Runner accepts connection: { mode: "external" } on warm attachment to preserve the process owner's listener and TLS settings while rotating the run identity.

Core permits connection refresh for the saved successful run while its warm task lease is retained, unexpired, and not retiring or stopped. PRP authentication still applies. A superseded lease cannot acquire a connection. Retention does not authorize preparation or submission of new work through the completed lease.

Idle retirement suspends and stops Runner, saves its exact checkpoint, writes credentials through the durable lease handoff, then completes the provider task. Core permits read-only suspension proof for the exact saved identity of a successful warm run after its reuse lease is retained or superseded. This does not authorize new work or file access through that lease. Temporary credential homes are not needed after the originating run ends. An explicit reset to a different normalized session can retire a warm owner whose close failed only because it could not prove checkpoint suspension. Core requires the exact lease-bound task's executionStopped: true receipt and a successful complete handoff first. When the close left them unproven, the new run requests them itself: one confirmed stop (not resent once confirmed), the credential snapshot, then complete. A warm owner without a task needs its Runner's own durable suspension instead. Unsettled Core tool results do not block this reset: a governed-wait park leaves one by design, its run is already terminal, its provider is stopped, and the new session never replays it. Core then collects stopped instruction files before it releases the in-memory owner. With that task receipt, any close error qualifies, including a controller journal drain that outlived the close grace: the old journal lives in the discarded session's own state root. The old checkpoint remains evidence; it is not reused by the new session. A handoff that still fails, failed file collection, another close error from an owner without a task, or continuation of the same session still blocks replacement. A controller restart is not required for this stopped-owner reset path.

Deleting an environment (with consent to destroy its reusable sandboxes) closes its idle warm owners first. An idle owner whose close cannot settle is retired the same way: one confirmed stop, the credential snapshot, complete, then instruction collection. An owner whose sandbox lease already carries a destroyed termination receipt has no process left to stop, so Core releases it without contacting the sandbox. Only then are the environment's sandboxes destroyed, through the usual workspace-export and restore-source guards. When an owner still cannot be retired, the delete returns 409 and names each run and the close and stop errors. Idle-timeout, restart and sleep sweeps do not force a stop. They log the exact close error for each owner instead of a count.

When retiring the previous owner stays unproven anyway (any close failure, on any of the paths a new run uses to retire it: instruction preparation, configuration change, session reset, or a prior-owner fence on a resumed sandbox), the new run has done no provider work. It fails with the typed pre-provider error native_predecessor_retirement_unproven instead of a provider failure. The owner stays registered for its own sandbox, so no run on that sandbox launches over it. The heartbeat then:

  • marks this run's reusable lease on that sandbox replaceBeforeReuse, so the next acquisition creates a fresh sandbox and hands the old one to the cleanup sweep. That replacement uses the same rule as a failed resume: a clean, committed workspace save and a verified harness backup. Without them the next acquisition resumes the sandbox as before;
  • after the run ends, hands its prepared workspace on at the saved version only through the shared unchanged-files proof (no runner command, workspace operation such as a provisioning command, execution event or result). Without that proof the workspace stays dirty and is preserved as today, and the run rests with Retry;
  • re-queues the run in the uncharged interrupted_preparation lane instead of a native reconciliation hold. A run that answered a task interaction keeps its own wake in that lane, so the retry still carries the answer.

A run on a different sandbox forgets the unretired owner instead of replaying its rejected close. A previous run parked on a durable governed interaction leaves its question's tool input unanswered by design. Once its Runner's sandbox lease has ended elsewhere, that committed park settles the input for retirement.

Core settles a dispatched task with stop, status, and complete, then calls onEnvironmentReleaseLease to release the allocation. If the release hook fails after settlement, Core retries cleanup using the recorded settlement proof. The hook can use that proof to avoid repeating task operations. A retained lease stays available for handoff and Core calls the release hook when it releases the lease.

When onEnvironmentResumeLease returns no provider lease, Core sends status first. It sends stop only when that status lacks executionStopped: true or required preparationStopped: true proof. An old warm-start preparation marker does not force a repeated stop when both proofs are present.

Termination receipts cover all run leases that owned the retired physical task. A stopped task gets a new provider task ID for checkpoint resumption. Inspection precedes in-memory session selection so a replacement cannot inherit the stopped session. Core saves the fresh provider expiry and forwards and enforces any caller-imposed deadline. A missing or unreachable computer never authorizes a replacement. Controller recovery can adopt a verified surviving idle process through its authenticated PRP connection; ambiguous transitions remain fenced by the existing durable recovery checks.

Changes to account, credential generation, workspace, runtime or process policy retire the old owner before recovery. A new authority epoch archives the stopped checkpoint under its stable archive key and preserves provider session files. Warm status monitoring relies on PRP after startup and does not poll an idle Launcher. ACPX uses this task-provider contract; other execution transports retain their provider-specific lifecycle policies.

Operation-time GitHub credentials

A task provider can advertise githubCredentialBrokerVersion: 1 in its describe result. Core then supplies an optional githubBroker: { url, token } in each prepare operation. This is a runtime capability, not a GitHub token. No controller path, executable, script, or command implementation is part of this contract. Plugin authors own how their command environment uses it.

At the start of each authenticated operation, send POST with body {} to <url without trailing /api>/runtime-tools/github/credentials, with Authorization: Bearer <token>. The response is either { status: "available", env: { ... } } or { status: "unavailable", reason: "..." }. Core rechecks the live run, accepted instruction identity, grants, and trust policy on each request. Use the returned environment only in the command and its descendants. Never retain raw GitHub credentials in plugin state, launch descriptors, logs, or files.

A warm task must replace its broker capability before admitting the next run's commands. An absent broker clears the old capability and allows anonymous/local operations only. Isolate ambient host credentials and Git/GitHub configuration; an unavailable broker must never select a host account. Capture once per operation so steering does not split a command's Git descendants across people. The runtime capability may be staged privately for command-time access; it must not enter the returned launch descriptor or entity records. Core's managed launchers are an optional reference implementation. Compatible native helpers, credential helpers, or command proxies can implement the same semantics.

Deploy the updated Core and plugin together. A configured GitHub connection on an older task provider returns an explicit upgrade error. Unconfigured providers continue ordinary work without authenticated GitHub access.

Managed GitHub command providers must retain the qualified provider executable and arguments. Supply helper discovery through the task environment instead of wrapping that executable. Claude ACPX preserves explicit BASH_ENV, ZDOTDIR, GH_CONFIG_DIR, and PAPERCLIP_GITHUB_LAUNCHER_DIR selectors across the sidecar boundary. It does not forward GitHub tokens or broker capabilities to the provider. The helper reads the current task capability at invocation, including after warm reuse. These selectors must come from the trusted environment owner. Reference helpers mark only their child operation and reject re-entry before resolving another executable or capturing credentials. A PATH shim that calls a helper's absolute path fails with launcher_recursion_rejected (exit 126), preventing recursive process growth. Offline Git fixtures must resolve the native binary outside managed helper paths and exclude runtime shell startup hooks.

The pure launch factory exports RUNNER_GITHUB_LAUNCHER_VERSION = 1 for this contract. A provider must check it as well as the host broker schema and its Launcher capability before advertising managed GitHub commands. The installed Runner must match the factory's source revision.

The helper projection is marked by the controller-owned PAPERCLIP_GITHUB_LAUNCHER_DIR. Task overrides cannot supply that marker or the shell selectors. Run preparation strips stored bindings before secret resolution, and configured task-environment projections reject the shell selectors too.