Pin plugin Runner cleanup to its physical task (#19300)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Plugin Runner owners manage a physical task through an environment
lease.
> - The existing replacement path can assign a new physical task to that
same lease.
> - A delayed old owner can then stop the replacement or mark it
stopped.
> - Cleanup must keep the original physical task identity across RPC and
database writes.
> - This pull request pins that identity and keeps cleanup available to
the new owner.

## Linked Issues or Issue Description

Refs #19134 and #19157. This fix is based on master and does not enable
repeated stopped-source recovery. A search found no duplicate fix.

**What happened?**

The maintained `createOwnedSandbox` replacement path changes
`providerLeaseId` on one lease. An old task owner uses that mutable
lease for a later Stop. A stopped-state response already in flight can
also update the new task's flags.

**Expected behavior**

The old owner must address its original task. Its late cleanup writes
must not alter the replacement. The replacement's own owner must still
stop and complete it.

**Steps to reproduce**

1. Create a plugin task owner for a stopped task.
2. Replace that task through the existing describe/startup path on the
same lease.
3. Resume an old checkpoint or a held old status response.
4. Observe a Stop for the replacement or an incorrect stopped flag
before this fix.

## What Changed

- Require each task owner to keep the physical ID returned by its
description. Reject a rebind to a different task.
- Carry the ID through the existing internal task context after startup
ends. Reject a changed physical task before RPC dispatch.
- Match the physical task in stopped-state, completion and
credential-handoff writes.
- Hold the exact lease row through the retained-session database update
and local checkpoint save.
- Cover the maintained replacement path with real PostgreSQL tests.
Preserve positive cleanup for the replacement owner. Document this
boundary.

## Verification

- Both replacement regressions failed on unmodified master `10a7d039c8`:
delayed old-owner Stop targeted the replacement, and a held old status
response marked it stopped.
- Six focused files passed 160 tests: task authority, task runtime, warm
tasks, execution target, credential writeback and task owner.
- Four selected executor factory cases passed. They cover task-only
providers and the physical task binding after the startup context ends.
- Runner TypeScript, plugin SDK and adapter-utils builds passed. Server
TypeScript check passed. Diff and added-line secret scans passed.
- All 15 setup-probe cases passed with the actual Runner and fake
provider binaries. The first local attempt lacked cargo on PATH. A
sandboxed native attempt timed out in seven process cases; the unchanged
run with loopback permission passed all 15. A separate initial DB
integration attempt was denied a loopback listener; its
permission-correct run passed.
- Review found three missed constructors in the existing SDK/Core
lifecycle fixture. They now use the actual describe result. All four
cases in that real PostgreSQL and worker RPC file passed.
- The first GitHub attempt failed in the unchanged GitHub recovery
pagination case in `chat-channels.integration.test.ts`: its zero-request
assertion observed a configuration GET. That exact case passed locally
without code or assertion changes. The one failed-job-only retry passed;
the initial failure log is retained.
- [GitHub workflow
42082587576](https://github.com/paperclipai/paperclip/actions/runs/42082587576)
passed on its second attempt. All CI checks completed, and Snyk passed.
Standard Greptile scored exact head `fb914d98b99e` 5/5 with no
unresolved threads. The branch is mergeable and ready for review.

## Risks

The task context change is internal. It adds no plugin wire field,
schema or Runner mode. A stale owner now gets an error and cannot clean
up a replacement. A new owner can still clean up that task. The retained
checkpoint write holds one existing lease row during a bounded local
file write.

Repeated stopped-source incarnation admission, runtime pins and
deployments are outside this change.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning. Used repository
tools, TypeScript and real PostgreSQL tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR using the bug template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] I have run the focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no unresolved findings
- [x] I will address reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-11-05 12:14:09 -08:00
1 parent 8aa28940bf
commit 31bf6e4482
16 files changed
+173 -72

No files matched your search

@@ -35,6 +35,8 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
leaseId: string;
operation: PluginEnvironmentTaskOperation;
executionContext?: unknown;
/** Physical task captured by an existing owner, independent of the mutable lease. */
expectedTaskId?: string;
}) {
const operation = environmentTaskOperationSchema.parse(input.operation);
const [row] = await db.select({ lease: environmentLeases, environment: environments, run: heartbeatRuns })
@@ -52,6 +54,7 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
context.assertScope({ companyId: input.companyId, agentId: run.agentId, runId: run.id });
}
const taskId = row.lease.providerLeaseId;
if (input.expectedTaskId !== undefined && input.expectedTaskId !== taskId) throw new Error("Plugin Runner physical task changed");
const metadata = lease.metadata ?? {};
const probe = metadata.runnerTaskProbe as { runId?: unknown; sessionId?: unknown; expiresAtMs?: unknown } | undefined;
const activeProbe = lease.heartbeatRunId === null && lease.issueId === null && probe
@@ -338,8 +341,8 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
}
}
if (operation.kind === "complete") {
await settlePluginTaskCredentialHandoff(db, { companyId: input.companyId, leaseId: lease.id }, metadata, async accountIdentity => {
const snapshot = await executeEnvironmentTask(db, workers, { ...input, operation: { kind: "credentials", accountIdentity } });
await settlePluginTaskCredentialHandoff(db, { companyId: input.companyId, leaseId: lease.id, expectedTaskId: taskId }, metadata, async accountIdentity => {
const snapshot = await executeEnvironmentTask(db, workers, { ...input, expectedTaskId: taskId, operation: { kind: "credentials", accountIdentity } });
if (snapshot.kind !== "credentials") throw new Error("Plugin Runner credential snapshot unavailable");
return snapshot.credential;
});
@@ -405,7 +408,8 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
metadata: sql`coalesce(${environmentLeases.metadata}, '{}'::jsonb) || ${JSON.stringify({ runnerTaskExecutionStopped: true,
runnerTaskSessionRetained: parsed.kind === "status" && parsed.sessionStateAvailable === true,
...(parsed.kind === "status" && parsed.preparationStopped === true ? { runnerTaskPreparationStopped: true } : {}) })}::jsonb`,
}).where(and(eq(environmentLeases.id, lease.id), eq(environmentLeases.companyId, input.companyId)));
}).where(and(eq(environmentLeases.id, lease.id), eq(environmentLeases.companyId, input.companyId),
eq(environmentLeases.providerLeaseId, taskId)));
}
if (operation.kind === "complete") {
if (freshWorkspace && metadata.runnerTaskExecutionStopped === true)
@@ -420,13 +424,13 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
await writeEnvironmentCleanup(db, tx => tx.update(environmentLeases).set({
...(owner.status === "retained" ? {status:"released" as const,releasedAt:new Date(),cleanupStatus:"success" as const}:{}),
metadata:sql`coalesce(${environmentLeases.metadata}, '{}'::jsonb) || ${JSON.stringify({remoteExecutionTermination:receipt,runnerTaskExecutionStopped:true})}::jsonb`,
}).where(eq(environmentLeases.id,owner.id)), { companyId: owner.companyId, runId: owner.heartbeatRunId ?? undefined });
}).where(and(eq(environmentLeases.id,owner.id),eq(environmentLeases.providerLeaseId,taskId))), { companyId: owner.companyId, runId: owner.heartbeatRunId ?? undefined });
}
}
await db.update(environmentLeases).set({
metadata: sql`coalesce(${environmentLeases.metadata}, '{}'::jsonb) || '{"runnerTaskDispatched":false}'::jsonb`,
}).where(and(eq(environmentLeases.id, lease.id), eq(environmentLeases.companyId, input.companyId),
sql`${environmentLeases.metadata}->>'runnerTaskExecutionStopped' = 'true'`));
eq(environmentLeases.providerLeaseId, taskId), sql`${environmentLeases.metadata}->>'runnerTaskExecutionStopped' = 'true'`));
}
return parsed;
} catch {