mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 05:11:49 +01:00
fix: preserve compatible historical Codex WEB evidence (#20665)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native session migration preserves local evidence in encrypted archives. > - Each WEB receipt binds the exact provider directory and exclusion policy. > - The Codex credential refresh change added two excluded files and changed that policy hash. > - A new reader then rejected old receipts even when their facts, files, skills, and authority still matched. > - This pull request accepts the exact prior Codex policy while checking archive paths against the current credential policy. > - Operators can keep valid saved evidence across the upgrade without copying it again. ## Linked Issues or Issue Description Refs: #19814, #20646. **What happened?** A reader upgrade rejected saved Codex WEB evidence because the profile hash changed when `auth-refresh.json` and `auth-refresh.json.tmp` became excluded credentials. The other receipt bindings still matched. This blocked the native harness migration gate. **Expected behavior** The exact known prior profile remains readable when the current archive path checks accept every saved entry. An archive that includes either newly excluded credential file must still fail. Unknown profiles and changed authority must fail. **Steps to reproduce** 1. Preserve WEB evidence with the frozen prior Codex profile and no credential refresh files. 2. Read its unchanged receipt with the current profile. 3. Observe that the old reader returns false solely because the profile hash changed. **Paperclip version or commit** Reproduced against `a3b34a9d88`; final branch is based on `e9343d8501`. ## What Changed - Accept the exact prior Codex profile hash only when the complete current profile has the reviewed successor hash and the provider is Codex. - Keep the current entry and excluded-path checks. Keep facts, controller, archive, skill, and ownership checks. - Add a frozen prior policy fixture with its literal hash. Test old receipt reads, both credential files, unknown policies, changed authority, and current writes. ## Verification - Real PostgreSQL baseline: the frozen old-policy receipt test failed because the reader returned false. The unchanged fixture passes with this fix. - All 18 selected PostgreSQL cases pass on final head after the master rebase: six new cases and twelve existing WEB archive and authority guards (14.93 seconds). - Full layered server typecheck passes on final head after the rebase. The module boundary check passes. - A single read-only native-only candidate check on the deployed dependency graph validated all 54 retained scopes. The 29 prior failures passed, and the other 25 stayed valid. The check made no storage calls or data writes. It did not replace final deployed enrollment acceptance. - Final head `14112d0e3abb6e318a86fe9799d95bddd2dbf76b` passes all [required CI checks](https://github.com/paperclipai/paperclip/actions/runs/42919867108): 55 successful checks, four expected skips, and one neutral review. The legacy security status also passes. Exact-head Greptile review is 5/5 with no open review threads. ## Risks The compatibility branch accepts only the two reviewed policy hashes. A future policy change can require another explicit compatibility decision. The reader still rejects saved credential refresh files. It does not rewrite receipts or change execution authority. No stored schema or operator command changes. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window capacity are not exposed in this session. Used reasoning, source editing, shell execution, and tool calls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
aa2a7be8b4
commit
de7ceac0de
2 files changed
+76
-1
No files matched your search
+69
@@ -16,6 +16,7 @@ import { materializeAsset } from "./runtime-context.js";
|
||||
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
|
||||
import { preserveRuntimeSkillAssets } from "./runtime-skill-preservation.js";
|
||||
import { nativeControllerMigrationDescriptor, scopedRunnerdStateRoot } from "./native-session-executor.js";
|
||||
import * as nativeSessionExecutor from "./native-session-executor.js";
|
||||
import { hasLocalNativeHarnessObligations } from "./native-harness-backup-migration.js";
|
||||
import { preserveNativeHarnessFilesystemIndependence, hasValidNativeHarnessFilesystemIndependence } from "./native-harness-filesystem-independence.js";
|
||||
import * as filesystemIndependence from "./native-harness-filesystem-independence.js";
|
||||
@@ -39,6 +40,20 @@ import { LEGACY_WORKSPACE_RECOVERY_MIGRATION } from "../legacy-workspace-recover
|
||||
|
||||
vi.mock("../../build-commit.js", () => ({ readBuildCommit: () => "a".repeat(40) }));
|
||||
|
||||
const CODEX_WEB_PROFILE_8E4 = {
|
||||
providerKind: "codex", driverKind: "codex_app_server",
|
||||
directories: [
|
||||
{ name: "runner", location: "runner", excludeEntries: [] },
|
||||
{ name: "codex-home", location: "filesystem", excludeEntries: [
|
||||
"tmp", ".tmp", "auth.json", "config.toml", ".remote-plugin-install-staging", "cache/remote_plugin_catalog",
|
||||
"plugins/.remote-plugin-install-staging", "plugins/cache/openai-curated-remote", "plugins/cache/created-by-me-remote",
|
||||
"plugins/cache/workspace-directory", "plugins/cache/workspace-shared-with-me",
|
||||
"plugins/cache/workspace-shared-with-me-private", "plugins/cache/workspace-shared-with-me-unlisted",
|
||||
] },
|
||||
],
|
||||
} as const;
|
||||
const CODEX_WEB_PROFILE_8E4_DIGEST = "6f10c4b4424e84defcdb4c7e932921d09ea73dc0f39c96e61777c8b20c5f637b";
|
||||
|
||||
describeEmbeddedPostgres("owner-verified native harness filesystem independence", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>, db: ReturnType<typeof createDb>;
|
||||
const roots: string[] = [];
|
||||
@@ -198,6 +213,60 @@ describeEmbeddedPostgres("owner-verified native harness filesystem independence"
|
||||
return { ...f, execution, controllerPath, raw, supervisor, supervisorPath: descriptor.supervisorPath, identity };
|
||||
}
|
||||
|
||||
it("reads the frozen pre-auth-refresh Codex WEB proof without changing authority", async () => {
|
||||
const f = await controllerFixture(true, "none");
|
||||
expect(nativeSha256(CODEX_WEB_PROFILE_8E4.directories)).toBe(CODEX_WEB_PROFILE_8E4_DIGEST);
|
||||
const oldPolicy = vi.spyOn(nativeSessionExecutor, "resolveNativeHarnessPersistenceProfile").mockReturnValue(structuredClone(CODEX_WEB_PROFILE_8E4));
|
||||
try { await preserveNativeHarnessFilesystemIndependence(db, { ...f.input, mode: "web_evidence" }); }
|
||||
finally { oldPolicy.mockRestore(); }
|
||||
const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
|
||||
expect((before!.runnerProfileJson!.nativeHarnessFilesystemIndependence as { web: { profileDigest: string } }).web.profileDigest)
|
||||
.toBe(CODEX_WEB_PROFILE_8E4_DIGEST);
|
||||
expect(await hasValidNativeHarnessFilesystemIndependence(db, f.input)).toBe(true);
|
||||
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId)))[0]).toEqual(before);
|
||||
await db.update(nativeRunFinalizations).set({ leaseOwner: randomUUID(), leaseExpiresAt: new Date(Date.now() + 60_000) })
|
||||
.where(eq(nativeRunFinalizations.runId, f.runId));
|
||||
expect(await hasValidNativeHarnessFilesystemIndependence(db, f.input)).toBe(false);
|
||||
});
|
||||
|
||||
it.each(["auth-refresh.json", "auth-refresh.json.tmp"])("refuses a legacy WEB archive that contains %s", async name => {
|
||||
const f = await controllerFixture(true, "none");
|
||||
await mkdir(join(f.sourceRoot, "codex-home"), { mode: 0o700 });
|
||||
await writeFile(join(f.sourceRoot, "codex-home", name), "private credential fixture", { mode: 0o600 });
|
||||
const oldPolicy = vi.spyOn(nativeSessionExecutor, "resolveNativeHarnessPersistenceProfile").mockReturnValue(structuredClone(CODEX_WEB_PROFILE_8E4));
|
||||
try { await preserveNativeHarnessFilesystemIndependence(db, { ...f.input, mode: "web_evidence" }); }
|
||||
finally { oldPolicy.mockRestore(); }
|
||||
const [row] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
|
||||
const web = (row!.runnerProfileJson!.nativeHarnessFilesystemIndependence as { web: { entries: Array<{ path: string }> } }).web;
|
||||
expect(web.entries.some(entry => entry.path === `web-home/codex-home/${name}`)).toBe(true);
|
||||
expect(await hasValidNativeHarnessFilesystemIndependence(db, f.input)).toBe(false);
|
||||
expect(await readFile(join(f.sourceRoot, "codex-home", name), "utf8")).toBe("private credential fixture");
|
||||
});
|
||||
|
||||
it.each(["extra-exclusion", "missing-existing-exclusion"])("refuses an unknown historical Codex WEB policy: %s", async change => {
|
||||
const f = await controllerFixture(true, "none");
|
||||
const exclusions = change === "extra-exclusion" ? [...CODEX_WEB_PROFILE_8E4.directories[1].excludeEntries, "unrecognized-cache"]
|
||||
: CODEX_WEB_PROFILE_8E4.directories[1].excludeEntries.filter(entry => entry !== "auth.json");
|
||||
const unknownPolicy = vi.spyOn(nativeSessionExecutor, "resolveNativeHarnessPersistenceProfile").mockReturnValue({
|
||||
...CODEX_WEB_PROFILE_8E4, directories: [CODEX_WEB_PROFILE_8E4.directories[0], { ...CODEX_WEB_PROFILE_8E4.directories[1], excludeEntries: exclusions }],
|
||||
});
|
||||
try { await preserveNativeHarnessFilesystemIndependence(db, { ...f.input, mode: "web_evidence" }); }
|
||||
finally { unknownPolicy.mockRestore(); }
|
||||
expect(await hasValidNativeHarnessFilesystemIndependence(db, f.input)).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps current Codex WEB writers on the current credential exclusion policy", async () => {
|
||||
const f = await controllerFixture(true, "none");
|
||||
await mkdir(join(f.sourceRoot, "codex-home"), { mode: 0o700 });
|
||||
await writeFile(join(f.sourceRoot, "codex-home/auth-refresh.json"), "private credential fixture", { mode: 0o600 });
|
||||
await preserveNativeHarnessFilesystemIndependence(db, { ...f.input, mode: "web_evidence" });
|
||||
const [row] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
|
||||
const web = (row!.runnerProfileJson!.nativeHarnessFilesystemIndependence as { web: { profileDigest: string; excluded: Array<{ path: string }> } }).web;
|
||||
expect(web.profileDigest).toBe("e5248e68685fb06ae32336d9517d7054e8aff785bdfb220b4481823a82ff8e78");
|
||||
expect(web.excluded.some(entry => entry.path === "codex-home/auth-refresh.json")).toBe(true);
|
||||
expect(await hasValidNativeHarnessFilesystemIndependence(db, f.input)).toBe(true);
|
||||
});
|
||||
|
||||
it("validates retained WEB receipts with bounded parallel database proofs", async () => {
|
||||
const fixtures = [];
|
||||
for (let index = 0; index < 5; index++) {
|
||||
|
||||
@@ -38,6 +38,8 @@ import { verifyDurableNativeHarnessBackup } from "./native-harness-backup-storag
|
||||
const KEY = "nativeHarnessFilesystemIndependence";
|
||||
const SCHEMA = "paperclip.native-harness-filesystem-independence.v1";
|
||||
const WEB_SCHEMA = "paperclip.native-harness-web-evidence.v1";
|
||||
const CODEX_WEB_PROFILE_BEFORE_AUTH_REFRESH = "6f10c4b4424e84defcdb4c7e932921d09ea73dc0f39c96e61777c8b20c5f637b";
|
||||
const CODEX_WEB_PROFILE_WITH_AUTH_REFRESH = "e5248e68685fb06ae32336d9517d7054e8aff785bdfb220b4481823a82ff8e78";
|
||||
const terminal = ["succeeded", "failed", "cancelled", "interrupted", "timed_out"];
|
||||
const sha = (bytes: string | Buffer) => createHash("sha256").update(bytes).digest("hex");
|
||||
const digest = (value: unknown) => nativeSha256(JSON.parse(JSON.stringify(value)));
|
||||
@@ -495,8 +497,12 @@ function validWebBinding(facts: Awaited<ReturnType<typeof databaseFacts>>, web:
|
||||
if (!web || typeof web.rootPresent !== "boolean" || typeof web.controllerAbsent !== "boolean" || !Array.isArray(web.entries)
|
||||
|| !Array.isArray(web.skills) || !Array.isArray(web.excluded)
|
||||
|| web.entries.length + web.excluded.length > NATIVE_CONTROLLER_WEB_LIMITS.entries
|
||||
|| web.profileDigest !== digest(webProfile(facts)) || digest(web.skills) !== digest(webSkills(facts))) return false;
|
||||
|| digest(web.skills) !== digest(webSkills(facts))) return false;
|
||||
const profiles = webProfile(facts), manifests = new Map(web.skills.map(skill => [skill.bundleDigest, skill.manifestDigest]));
|
||||
const profileDigest = digest(profiles);
|
||||
if (web.profileDigest !== profileDigest && !(resolveNativeHarnessPersistenceProfile(facts.execution).providerKind === "codex"
|
||||
&& profileDigest === CODEX_WEB_PROFILE_WITH_AUTH_REFRESH
|
||||
&& web.profileDigest === CODEX_WEB_PROFILE_BEFORE_AUTH_REFRESH)) return false;
|
||||
for (const [bundle, manifest] of manifests) if (!web.entries.some(entry => entry.path === `web-manifests/${bundle}.json`
|
||||
&& entry.type === "file" && entry.sha256 === manifest)) return false;
|
||||
if (web.entries.some(entry => {
|
||||
|
||||
Reference in new issue
Block a user