Commit Graph
4 Commits
Author SHA1 Message Date
Nicky LeachandPaperclip a924d2e2c7 Persist portable Runner ownership through controller recovery (#21066)
## Thinking Path

> - Paperclip delegates remote Runner execution to environment plugins.
> - A plugin can move a session to another task allocation.
> - The Core controller can lose an ownership reply or restart before
the Runner starts.
> - The source and target need a durable record of the same request.
> - This change binds restart and attachment to that request and the
original unlaunched task lease.

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Recovery of plugin Runner sessions on replacement compute.

**Current behavior**
Core cannot replay a session ownership request from saved lease state
during startup.

**Proposed behavior**
Record the operation on both leases and check the matched ownership
receipt before attaching. Preserve the original pre-admission lease
across controller restart.

Managed environment configuration is in
[#21065](https://github.com/paperclipai/paperclip/pull/21065). Related
warm sandbox cleanup is in
[#20065](https://github.com/paperclipai/paperclip/pull/20065). This
layer preserves the same-sandbox fallback from
[#21001](https://github.com/paperclipai/paperclip/pull/21001). Workspace
placement follows in
[#20743](https://github.com/paperclipai/paperclip/pull/20743).

## What Changed

- Add the optional `sessionHandoffVersion` capability and the ownership
task operation.
- Save matching ownership claims and fence source checkpoint access.
Replay a pending request on the original target.
- Restore a portable session using the durable claim and the current
credentials for the same account. Start a fresh conversation when the
provider reports an unavailable checkpoint, or when Core's persisted
compatibility decision selects a different session after transfer. The
latter requires an unlaunched target; subsequent recovery must match the
fresh task's saved identity. Both paths retain the workspace transfer
and source fencing.
- Reuse the original unlaunched task lease after hard or graceful
controller restart. After a move has attached, validate the surviving
Runner through the saved claim rather than requiring its logical lease
ID to equal the new physical lease.
- Keep older retained leases without a saved account identity on the
existing resume path.
- Apply the existing supported MCP-only compatibility rule to warm
recovery and predecessor accounting. Persist the capability discovered
at admission and reuse it after restart without repeating discovery.
- Keep predecessor-retirement fallback scoped to the same physical
sandbox. A refusal after a portable move preserves the session instead
of requesting a replacement that the plugin driver cannot enforce.

## Verification

- On `1eb527be27`, server typecheck (including Runner build and SDK
dependencies) passed.
- Independent final verification passed 168 tests across portable source
recovery, session resume, controller recovery, and task authority; two
existing tests were skipped. The full portable source integration suite
passed 64 tests.
- The new integration follows confirmed ownership through an
incompatible instruction change, persisted fresh admission, target-only
prepare/auth/submit without the source checkpoint, and reattachment
after controller replacement. It also rejects unpersisted session
selection, ambiguous target leases, and incorrect saved identities.
- Before this final change, independent verification passed 1,667 tests
across executor, session resume, recovery usage, remote warm recovery,
runtime selection, and runtime execution; the environment lifecycle
suite passed 208 tests.
- Existing regressions cover previous-release account metadata,
supported and unsupported MCP changes, provider/instruction
incompatibility, and admission capability reuse without another
discovery call.
- On `51d7eec373`, all four S3 multipart throttle tests passed
independently. CI exposed an existing timing-sensitive test twice; its
20 ms overlap assumption now uses synchronized retries and an event-loop
yield. Production upload code is unchanged. Temporary concurrency caps
of one and eight both fail the retained exact-four assertion.
- Exact SDK source-pin checks passed for both stack heads. The upper
#20743 code patch remains unchanged; its documentation preserves both
layers' behavior.

## Risks

- The handoff protocol is opt-in. Existing plugin leases without
portable ownership metadata keep the previous release's resume path.
Live attachment still requires end-to-end qualification. An unverified
claim blocks startup. Git workspaces prepared on a portable allocation
also need the workspace placement layer.

## Model Used

- OpenAI Codex GPT-6 with GPT-6.1 Sol agents for implementation and
independent review.

## Checklist

- [x] Thinking path and model are included.
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] The issue is described using the enhancement template.
- [x] Local checks passed and new behavior has tests.
- [x] All Paperclip CI gates are green.
- [x] Greptile is 5/5 with no open findings.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-12 10:43:48 -08:00
Nicky Leach 17136a6c25 fix(runtime): persist managed Runner credential refreshes across execution targets (#20580) 2026-11-10 10:15:29 -08:00
Nicky LeachandPaperclip 31bf6e4482 Pin plugin Runner cleanup to its physical task (#19300)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Plugin Runner owners manage a physical task through an environment
lease.
> - The existing replacement path can assign a new physical task to that
same lease.
> - A delayed old owner can then stop the replacement or mark it
stopped.
> - Cleanup must keep the original physical task identity across RPC and
database writes.
> - This pull request pins that identity and keeps cleanup available to
the new owner.

## Linked Issues or Issue Description

Refs #19134 and #19157. This fix is based on master and does not enable
repeated stopped-source recovery. A search found no duplicate fix.

**What happened?**

The maintained `createOwnedSandbox` replacement path changes
`providerLeaseId` on one lease. An old task owner uses that mutable
lease for a later Stop. A stopped-state response already in flight can
also update the new task's flags.

**Expected behavior**

The old owner must address its original task. Its late cleanup writes
must not alter the replacement. The replacement's own owner must still
stop and complete it.

**Steps to reproduce**

1. Create a plugin task owner for a stopped task.
2. Replace that task through the existing describe/startup path on the
same lease.
3. Resume an old checkpoint or a held old status response.
4. Observe a Stop for the replacement or an incorrect stopped flag
before this fix.

## What Changed

- Require each task owner to keep the physical ID returned by its
description. Reject a rebind to a different task.
- Carry the ID through the existing internal task context after startup
ends. Reject a changed physical task before RPC dispatch.
- Match the physical task in stopped-state, completion and
credential-handoff writes.
- Hold the exact lease row through the retained-session database update
and local checkpoint save.
- Cover the maintained replacement path with real PostgreSQL tests.
Preserve positive cleanup for the replacement owner. Document this
boundary.

## Verification

- Both replacement regressions failed on unmodified master `10a7d039c8`:
delayed old-owner Stop targeted the replacement, and a held old status
response marked it stopped.
- Six focused files passed 160 tests: task authority, task runtime, warm
tasks, execution target, credential writeback and task owner.
- Four selected executor factory cases passed. They cover task-only
providers and the physical task binding after the startup context ends.
- Runner TypeScript, plugin SDK and adapter-utils builds passed. Server
TypeScript check passed. Diff and added-line secret scans passed.
- All 15 setup-probe cases passed with the actual Runner and fake
provider binaries. The first local attempt lacked cargo on PATH. A
sandboxed native attempt timed out in seven process cases; the unchanged
run with loopback permission passed all 15. A separate initial DB
integration attempt was denied a loopback listener; its
permission-correct run passed.
- Review found three missed constructors in the existing SDK/Core
lifecycle fixture. They now use the actual describe result. All four
cases in that real PostgreSQL and worker RPC file passed.
- The first GitHub attempt failed in the unchanged GitHub recovery
pagination case in `chat-channels.integration.test.ts`: its zero-request
assertion observed a configuration GET. That exact case passed locally
without code or assertion changes. The one failed-job-only retry passed;
the initial failure log is retained.
- [GitHub workflow
42082587576](https://github.com/paperclipai/paperclip/actions/runs/42082587576)
passed on its second attempt. All CI checks completed, and Snyk passed.
Standard Greptile scored exact head `fb914d98b99e` 5/5 with no
unresolved threads. The branch is mergeable and ready for review.

## Risks

The task context change is internal. It adds no plugin wire field,
schema or Runner mode. A stale owner now gets an error and cannot clean
up a replacement. A new owner can still clean up that task. The retained
checkpoint write holds one existing lease row during a bounded local
file write.

Repeated stopped-source incarnation admission, runtime pins and
deployments are outside this change.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning. Used repository
tools, TypeScript and real PostgreSQL tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR using the bug template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] I have run the focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no unresolved findings
- [x] I will address reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 12:14:09 -08:00
Nicky LeachandPaperclip 33cc567f75 Add lease-scoped Runner task hooks to environment plugins (#15406)
## Thinking Path

> - Paperclip coordinates AI agents and their runs.
> - Environment plugins own remote capacity.
> - A plugin needs a typed API to start and stop its installed Runner.
> - Core must derive authority from the saved lease.
> - A lost reply or controller restart must not lose cleanup authority.
> - This PR adds the worker contract and the host task service.
> - Native execution and Runner changes are separate PRs.

## Linked Issues or Issue Description

**Subsystem affected**

Plugin SDK and environment lease services.

**Current behavior**

Environment plugins have no typed Runner task hook. Core cannot dispatch
these tasks or recover their credential handoff through a saved lease.

**Proposed behavior**

Add validated task operations, worker negotiation, company and project
checks, retained-session authorization, and durable cleanup. Keep
credentials out of lease metadata.

**Additional context**

This PR is the first part of a smaller series. Runner installed-home
support is in #16977. Native execution and setup are in #16978. Both are
stacked drafts. This PR does not enable native plugin execution, change
default environments, add a route, or add a database migration.

## What Changed

- Add `supportsTasks` and `onEnvironmentTask` with seven typed
operations.
- Validate worker requests and responses. Reject shell commands and file
transfer operations.
- Pin dispatch and cleanup to the saved company, lease, plugin, and
driver.
- Authorize projects and retained sessions. Validate the current run
separately from the retained PRP identity.
- Save dispatch before the provider call. Fence admission during
cleanup. Require observed process termination.
- Save account writeback authority on the lease. Recover Codex refreshes
through the existing account and secret services.
- Document the task API. Keep the Runner, heartbeat, setup, workspace,
and provider changes in later PRs.

## Verification

- Full `pnpm -r typecheck` and `pnpm build` passed on the separated API
scope.
- 201 focused tests passed: SDK worker dispatch, task validation,
PostgreSQL task lineage, credential recovery, and existing environment
cleanup.
- The lineage test covers a third run, repeated same-run replacements, a
stale submitting run, and exact termination receipts.
- Initial commands hit sandbox socket/IPC restrictions. The permitted
runs executed PostgreSQL and passed. A typecheck attempt lacked Cargo in
PATH; the run with the installed toolchain passed.
- The earlier combined implementation had a full local run with 22,286
passing tests, 120 skips, and eight failed assertions. Task fixture
failures were fixed before this split. Remaining host Git and
temporary-path assertions are recorded in the previous PR revision. This
split does not claim a fresh full local suite pass.
- The standard Greptile review scored this API head 5/5 with no
unresolved threads. CI is green. The single Apex review also scored this
head 5/5 with no actionable findings.
- Fresh API CI lost a self-hosted runner during server shard 9. GitHub
reported lost communication, with no failed test step. The affected jobs
passed on retry without code changes.
- CI on the prior combined head failed an unchanged workspace dev-server
readiness case. No unrelated test fix is included here. Fresh CI is
required for this smaller head.
- This scope does not run a provider or a Cloud test. The user owns
deployment and the intermediate Cloud test.

## Risks

- The API is opt-in. Native plugin execution requires the later
integrations.
- Providers must make task submission idempotent and prove descendant
termination.
- A failed status read or credential export keeps the lease available
for cleanup.
- Codex refresh writeback requires the same active grant, secret
reference, and session epoch. Revocation or account replacement discards
the old refresh.

## Model Used

OpenAI GPT-6 through Codex. Used source inspection, code execution,
tests, and GitHub tools. The exact served model ID and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run local checks and recorded failures and their results
above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 09:14:23 -07:00