Files
PaperClipAI/server/src/services/native-runtime/provider-pack-upload.ts
T
DottaandPaperclip 909f4d1a33 fix(native): recover exact historical provider pack code (#19779)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native sessions retain the provider pack that admitted their saved
state.
> - An image update can replace that pack's executable, proxy, or
contract.
> - Older sessions can predate the durable three-file launch artifact
receipt.
> - The current image's archive cannot satisfy a different pinned
manifest.
> - This pull request obtains the exact public code from its recorded
release image.
> - Full source, archive, and installed-file checks still control
provider admission.

## Linked Issues or Issue Description

Refs #19558, #19622, #19755.

A saved OpenCode session can fail with
`runner_remote_provider_archive_invalid` after an image update. Its
original provider state and checkpoint remain valid, but its pinned pack
differs from the controller's pack. The old manifest can name a release
that still has the exact code in GHCR. Matching the OpenCode version
does not prove matching proxy or contract bytes.

## What Changed

- Resolve a pinned historical pack from the recorded source-SHA image in
the public Paperclip GHCR repository. Verify the platform manifest, raw
config, source identity, dedicated code layer, selected pack manifest,
archive contents, paths, and hashes.
- Keep a private temporary cache of verified public code. Rebuild it on
an empty host. Bound each source to 4 GiB and three minutes, with two
active imports or cache verifications, 32 distinct queued sources, and
disk-headroom checks. Share warm-cache verification by manifest while
each caller keeps its own cancellation.
- Select this verified archive through the existing uploader. Preserve
installed-pack and preinstalled-pack checks. Refuse unavailable
historical transport before clearing the target. Keep the current-source
identity guard from #19755. Refuse an oversized layer argument before
any target effect.
- Attach the source wait to the actual per-turn startup. Stop, lease
loss, and startup completion cancel it. Keep existing Runner admission
and turn limits.
- Document the registry retention requirement and the distinction
between public code recovery and encrypted provider-state backup.

## Verification

- Genuine baseline RED: the real uploader rejects the selected old pack
before any remote effect. Treatment installs the exact old code through
that uploader.
- Current head: all 19 source/integrity/cache/deadline controls, all 45
archive controls, and 29 actual-caller and manifest/startup controls
pass (93 total). Four review regressions fail against the prior head and
pass with the fixes. The server and colocated test TypeScript graph
passes. Module-boundary and whitespace checks pass.
- Actual public GHCR cold reconstruction: 1,054,587,008 compressed code
bytes, 115.4 s, about 248 MiB maximum owner RSS. All 114 file layers and
the overlay passed source validation.
- Linux Node 24.22.2 fixture: the real historical pack passes the normal
uploader, full installed-file verifier, and pinned OpenCode version
check. It used one transfer, no application startup, no network, and no
provider calls. Revision12b0d2ba passed in 220.2 s with about 282 MiB
owner RSS. The earlier prototype passed in 108.5 s; both measurements
are retained. The three-minute source limit and the fixture's
four-minute limit were unchanged. The later review fixes add shared
cache verification, hash cancellation, and an early size guard. They do
not change extraction or the accepted Linux fixture input.
- Earlier fixture failures are retained: unsafe hardlink classification
was corrected with confined-link controls; a cached image lacked the
required zstd CLI; Docker Desktop bind storage caused GNU tar
directory-identity errors. The passing fixture uses the official Debian
zstd binary and an owned Linux tmpfs target. Production extraction code
is unchanged.
- Fresh Greptile review is 5/5 on
7b1a249478 with no open findings. All
required CI gates passed on the reviewed head. [CI
run](https://github.com/paperclipai/paperclip/actions/runs/42353011485),
[fresh
review](https://github.com/paperclipai/paperclip/pull/19779#issuecomment-6704065994).
No hosted session was retried by this change.

## Risks

- Cold historical recovery reads about 1 GiB of public code for the
measured release. Registry latency or missing images can block it. The
source deadline remains three minutes.
- GHCR administrators can delete release images. Source-SHA tags and
this temporary cache are not a permanent code-retention guarantee. A
provider-state S3 backup does not replace legacy executable bytes.
- Targets must support the existing qualified archive transport and
selected platform. Failed checks preserve the saved state and prevent
provider dispatch.
- The Linux fixture proves code installation and verification. Hosted
conversation continuation still needs acceptance after normal delivery.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools, and local
code execution. The host does not expose the exact deployment ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes:`
/ `Refs:` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub references)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-07 02:06:02 -06:00

76 lines
4.4 KiB
TypeScript

import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
import type { NativeRunTrace } from "./native-run-trace.js";
import { readProviderPackArchive, stageProviderPackArchive } from "./provider-pack-archive.js";
import type { ProviderPackIdentity } from "./remote-provider-pack-verification.js";
import { resolveProviderPackUploadSource } from "./retained-provider-pack-source.js";
import { providerPackExtractionProbe } from "./provider-pack-extraction.js";
import { providerPackLayersArgument } from "./provider-pack-layers.js";
const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`;
/** Upload a pack and verify the installed bytes before exposing it to a provider. */
export async function uploadVerifiedProviderPack(input: {
root: string;
/** Identity of the controller pack at `root`. */
sourceDigest: string;
/** Identity selected for this execution; a retained session can pin an older release. */
expectedDigest: string;
manifest?: ProviderPackIdentity;
signal?: AbortSignal;
runner: CommandManagedRuntimeRunner;
cwd: string;
targetPath: string;
reuseRoots: readonly string[];
trace?: NativeRunTrace;
verifyDirectory: (root: string) => void | Promise<void>;
stageDirectory: (root: string) => Promise<void>;
verifyUploaded: () => Promise<void>;
}): Promise<void> {
if (input.sourceDigest !== input.expectedDigest && !input.manifest) {
const short = (digest: string) => digest.replace(/^sha256:/, "").slice(0, 12);
throw new Error(`runner_remote_provider_artifact_incompatible: selected provider pack ${short(input.expectedDigest)} is not the controller's pack ${short(input.sourceDigest)}; install the matching Paperclip package and Daytona image`);
}
const measure = <T>(name: string, fn: () => Promise<T>) => input.trace ? input.trace.measure(name, fn) : fn();
input.signal?.throwIfAborted();
if (input.manifest && input.manifest.digest !== input.expectedDigest) throw Error("runner_remote_provider_source_invalid");
const source = input.manifest ? await measure("provider_pack.source", () => resolveProviderPackUploadSource({
root: input.root, sourceDigest: input.sourceDigest, manifest: input.manifest!, signal: input.signal,
})) : { root: input.root, archiveOnly: false };
const verifyDirectory = () => measure("provider_pack.directory.verify", async () => input.verifyDirectory(source.root));
// Archive installation reads the sealed archives, not the expanded tree.
// Hash the directory only when it is actually the upload source. The
// installed pack still passes full verification against the selected identity.
const archive = await measure("provider_pack.archive.verify", () =>
readProviderPackArchive(source.root, input.expectedDigest, input.signal));
if (source.archiveOnly) {
if (!archive) throw Error("runner_remote_provider_source_invalid");
if ("kind" in archive && providerPackLayersArgument(archive, input.targetPath, input.reuseRoots) === null) {
throw Error("runner_remote_provider_archive_transport_unavailable");
}
const capability = await measure("provider_pack.archive.probe", () => input.runner.execute({
command: "sh", args: ["-c", `${providerPackExtractionProbe}${"kind" in archive ? " && node -e 'process.exit(Number(process.versions.node.split(\".\")[0])>=18?0:1)'" : ""}`],
bypassSession: true, timeoutMs: 10_000,
}));
if (capability.exitCode !== 0 || capability.timedOut) throw Error("runner_remote_provider_archive_transport_unavailable");
}
if (!archive) await verifyDirectory();
input.signal?.throwIfAborted();
const cleared = await measure("provider_pack.clear_stale", () => input.runner.execute({
command: "sh", args: ["-c", `rm -rf ${quote(input.targetPath)}`],
cwd: input.cwd, bypassSession: true, timeoutMs: 10_000,
}));
if (cleared.exitCode !== 0 || cleared.timedOut) {
throw new Error("runner_remote_provider_artifact_incompatible: stale provider pack could not be replaced");
}
await measure("provider_pack.upload", async () => {
if (archive && await stageProviderPackArchive({
archive, runner: input.runner, targetPath: input.targetPath,
reuseRoots: input.reuseRoots, trace: input.trace,
})) return;
if (source.archiveOnly) throw Error("runner_remote_provider_archive_transport_unavailable");
if (archive) await verifyDirectory();
await input.stageDirectory(source.root);
});
await measure("provider_pack.verify_uploaded", input.verifyUploaded);
}