Guard Native recovery with process-space proof and explicit containment (#19813)

## Thinking Path

> - Paperclip manages agents and their work.
> - Recovery must preserve execution ownership and accepted results.
> - A saved PID proves nothing outside its recorded process namespace.
> - Uncertain unfinished work must stay held until its original owners
stop.
> - Operators need a source-bound containment action and a separate
ordinary Retry.
> - Accepted results must use the existing finalizer without another
provider turn.
> - This PR keeps those paths safe, recoverable, and quiet between real
obligations.

## Linked Issues or Issue Description

Refs: #19713

Refs: #19591

Recovery could use a local PID to release or replace a controller from
another process namespace. Connected-chat Retry also rejected successful
containment on a task in `todo`. The new containment action used a
broader permission than the existing owner-stop action. A later review
found that result-less guards also rejected the existing accepted-result
confirmation action.

## What Changed

- Record and verify the controller's kernel/PID namespace. Keep unknown
and foreign unfinished ownership held. Preserve independent durable stop
evidence and the existing remote lease fences.
- Use the shared accepted-result ownership reconciler. Bind settlement
to the observed lease token, generation, boot, PID, namespace and
process birth. Retain finalizer, cleanup, current-controller, live-owner
and remote exclusions.
- Share the remote-owner reader across recovery consumers. Scope runner
leases to the same company, task and run. Do not make local PID probes
for remote accepted-result ownership.
- Gate original-host containment on `runtime:confirm_owner_stop`. Bind
it to the current task, source, assignee, original expiry, lease and
Stop state. Audit separate controller/provider containment and reviewed
action outcomes.
- Keep containment separate from ordinary Retry. Use one canonical
committed receipt and the normal admission, budget, workspace and
lineage checks. Preserve the recorded external-action outcomes.
- Keep the controller-only confirmation actionable for both result-less
work and accepted results. Result-less work gets its next source-bound
attention in the same transaction. Accepted results keep their original
result and use normal shared settlement/finalization.
- Keep identity inspection quiet. Preserve the original foreign-owner
expiry and use the existing scoped finalizer in accepted-result recovery
controls.
- Update the recovery card, validators, execution projection, docs and
access maps. Reuse existing canonical columns; add no migration or
ownership backfill.

## Verification

Current publication head: `71ea76b568de09d6313d8992e803131d9d5c54f3`.

**The current fix has a causal red/green check.** On the test-only
revision, both foreign-local stopped/fenced controller cases failed
because recovery probed saved runner, process-group and provider PIDs in
this host's namespace. This measured the wrong probes; it did not run
duplicate providers. After the fix, the same two cases passed every
no-probe, no-claim, no-controller-stamp, no-stop-receipt and
unchanged-authority assertion. Three existing remote preparation, grace
and fenced-controller progress cases also passed. Total: **5 passed, 0
failed;170 filtered rows excluded**. All three normal affected App
TypeScript programs passed on this head (660,681 and428 roots).

The fix separates a controller wait from execution stop proof.
Independent live/grace evidence still schedules recovery. A
stopped/fenced foreign controller can pass this namespace guard only for
remote execution, which keeps its later checkpoint and never-launched
checks. Foreign local execution stays held before runner or provider PID
inspection.

Earlier source-scoped results remain evidence, not fresh executions on
the current head:

- On `c86dd741`, the complete failed-owner file passed **25/25**, with
no skips or filters. Empty scans wrote no controller stamps. Foreign
ownership kept its original expiry. Normal returned-claim settlement and
attention at the exact deadline passed.
- On `c86dd741`, **15 selected accepted-result/restart controls
passed**. The81 filtered rows are excluded. Both completion cases used
the existing scoped finalizer and kept their accepted result without
another provider turn.
- Trusted CI at synthetic merge `50d072a5` (actual master `1b7fe334`
plus `c86dd741`) passed the complete **17-case maintenance-server
file**. This includes the original90-real-second quiet observation and
two qualified hourly windows. The quiet case took118,918ms. The four
jobs that failed on the earlier publication all passed on that synthetic
merge.
- On `6243b32c`, the complete ownership-report file passed **29/29**.
Native NULL namespace stayed unprovable and generic compatibility
remained. Three selected recovery controls passed;170 filtered rows are
excluded.
- On `f81f2c3c`,356 business cases passed: whole ownership-route221,
recoveryUI70, owner-proof38 and controller-confirmation8; selected
accepted-result authority18; and one real connected-chat containment,
separate authorized Retry, useful Native result and downstream reply
case. The2,024 filtered rows are excluded. A separate earlier
accepted-result causal case kept the original result and completed after
restart without replay.

Normal Shared exports, App programs, UI typecheck, token gates and map
checks passed on their recorded revisions. Unchanged DB, adapter, Eval
and Runner inputs/outputs have explicit carry proofs. The clean normal
merge with fetched master `8fe524f7` preserves all31 exclusive owned
paths and14,827 unrelated master entries. The nine overlapping paths
were reviewed, including the upstream unidentified-run stop action.
Fresh CI must validate that actual merge composition.

PG cases used existing external PG17, fresh owned databases, pools at
most4, one worker and no SQL observer during cases. Registered clients
and recorded process groups closed. Independent final SQL audits found
zero campaign sessions before normal database cleanup. Original failing
fixtures remain retained.

Original failures are preserved. One new test first failed compilation
due to a missing getter import; an import-only correction passed its
full owning programs before the causal red/green check. A local quiet
attempt stopped in the normal prerequisite hook before any business case
and its collector rejected two normal build markers. The later17-case CI
pass establishes quiet recovery; it does not turn that failed local
collector into a pass. An earlier useful case passed before its private
cleanup collector failed on an owned read-only directory; normal cleanup
completed after all owners closed. Earlier runner-shutdown cancellation
and a separate private whole-suite failure remain unexplained.

Current-head checks are complete:57 success,1 neutral and3 skipped,
including the successful aggregate CI gate. The legacy status passed.
Greptile is5/5 and Ponytail gives an explicit5/5 Ship verdict for this
head. All seven review threads are resolved. Full recursive typecheck,
the full test suite and release build were not repeated locally. No
result here proves deployed suspension, a live provider or termination
on another host.

## Risks

- Unknown physical ownership can require an owner/admin action. A false
containment statement can be unsafe.
- New work, reassignment, Stop, renewed authority, accepted results or
pending cleanup can invalidate an action. The operator must refresh.
- Controller confirmation does not prove provider/tool containment.
Completed external actions must not be replayed.
- Generic legacy and child-process compatibility remains separate from
the strict Native controller policy.
- Existing saved state and canonical schema are preserved. Permanent
safety refusals remain held with their normal named recovery path.

## Model Used

OpenAI `gpt-6.1-sol` agents implemented and tested the changes with
reasoning, repository tools and code execution. A GPT-6-based Codex
managing agent reviewed and integrated them. The host does not expose
the manager's exact build or the context-window sizes.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Dotta authored and GitHub committed 2026-11-09 12:06:11 -06:00
1 parent 365c771030
commit a3e7e63330
40 files changed
+2674 -147

No files matched your search

@@ -369,6 +369,37 @@ describe("issue validators", () => {
} }).success).toBe(true);
});
it.each(["backlog", "in_progress", "blocked"] as const)("allows a complete controller containment request to retain %s status for locked route validation", sourceIssueStatus => {
const input = { actionId: "22222222-2222-4222-8222-222222222222", outcome: "restored", sourceIssueStatus,
executionReconciliation: { runId: "11111111-1111-4111-8111-111111111111", providerStopped: true,
controllerStopped: true, controllerAuthorityDigest: "a".repeat(64), actionOutcome: "mixed",
outcomeEvidence: "The original-host controller and provider stopped; action outcomes were reviewed." } };
// The real historical HTTP request failed here before its route could
// enforce same-status/current-authority checks. Syntax grants no authority.
expect(resolveIssueRecoveryActionSchema.parse(input)).toEqual(input);
});
it.each(["controller-only", "digest-only", "missing-action", "wrong-outcome", "partial-with-workspace-evidence"] as const)(
"rejects incomplete controller containment even when ordinary status validation would pass: %s", mode => {
const complete = { runId: "11111111-1111-4111-8111-111111111111", providerStopped: true,
controllerStopped: true, controllerAuthorityDigest: "a".repeat(64), actionOutcome: "mixed",
outcomeEvidence: "The original-host controller and provider stopped; action outcomes were reviewed." };
const ordinary = { runId: complete.runId, providerStopped: complete.providerStopped,
actionOutcome: complete.actionOutcome, outcomeEvidence: complete.outcomeEvidence };
const input = { actionId: mode === "missing-action" ? undefined : "22222222-2222-4222-8222-222222222222",
outcome: mode === "wrong-outcome" ? "blocked" : "restored",
sourceIssueStatus: mode === "wrong-outcome" ? "blocked" : "todo",
executionReconciliation: mode === "controller-only" ? { ...ordinary, controllerStopped: true }
: mode === "digest-only" ? { ...ordinary, controllerAuthorityDigest: complete.controllerAuthorityDigest }
: mode === "partial-with-workspace-evidence" ? { ...ordinary, controllerStopped: true,
workspaceRepairEvidence: "Retained workspace files were repaired and inspected separately." }
: complete };
const result = resolveIssueRecoveryActionSchema.safeParse(input);
expect(result.success).toBe(false);
if (!result.success) expect(result.error.issues).toContainEqual(expect.objectContaining({ path: ["executionReconciliation"] }));
},
);
it("allows cancelled recovery resolutions to atomically restore the source issue status", () => {
expect(
resolveIssueRecoveryActionSchema.parse({
+15 -3
View File
@@ -652,6 +652,8 @@ export const resolveIssueRecoveryActionSchema = z
.object({
runId: z.string().guid(),
providerStopped: z.literal(true),
controllerStopped: z.literal(true).optional(),
controllerAuthorityDigest: z.string().regex(/^[0-9a-f]{64}$/).optional(),
actionOutcome: z.enum(["completed", "not_performed", "mixed"]),
outcomeEvidence: z.string().trim().min(20).max(12000),
workspaceRepairEvidence: z.string().trim().min(20).max(12000).optional(),
@@ -668,10 +670,20 @@ export const resolveIssueRecoveryActionSchema = z
if (value.workspaceBaseRef && (!value.actionId || value.outcome !== "restored" || value.sourceIssueStatus !== "todo" || value.executionReconciliation)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["workspaceBaseRef"], message: "Branch repair must retry the exact recovery action" });
}
const reconciliation = value.executionReconciliation;
const controllerContainment = reconciliation?.controllerStopped !== undefined
|| reconciliation?.controllerAuthorityDigest !== undefined;
if (controllerContainment && (!value.actionId || value.outcome !== "restored"
|| reconciliation?.controllerStopped !== true || !reconciliation?.controllerAuthorityDigest)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["executionReconciliation"],
message: "Controller containment requires the exact recovery action, both stop confirmations, and its authority digest" });
return;
}
if (value.outcome === "restored") {
// A retained-source repair records evidence without changing task state.
// The route verifies the exact server-owned source and unchanged status.
if (value.executionReconciliation?.workspaceRepairEvidence) return;
// Source repair and complete controller containment record evidence
// without changing task state. The route checks the exact locked source,
// unchanged status, board authority, lease expiry, and cleanup evidence.
if (value.executionReconciliation?.workspaceRepairEvidence || controllerContainment) return;
if (
value.sourceIssueStatus !== "todo" &&
value.sourceIssueStatus !== "done" &&