fix(skills): reuse validated runtime revisions during preparation (#13042)

## Thinking Path

> - Paperclip manages AI agents and prepares their runtime inputs before
each turn.
> - Shared company skills are part of those inputs for native and legacy
adapters.
> - Runtime materialization refreshed the full inventory again for every
declared file.
> - Remote skill directories were also downloaded and rebuilt on every
turn.
> - Measured preparation took 42–73 seconds while runner execution took
7–9 seconds.
> - This change reads the inventory once and reuses validated installed
revisions.
> - Agents retain their selected skills while repeated preparation
avoids upstream work.

## Linked Issues or Issue Description

**What happened?**

One 114-skill preparation performed 407 inventory refreshes, 48
directory rebuilds, and 388 GitHub file fetches. Reusing existing local
copies took 151 ms.

**Expected behavior**

Each listing refreshes inventory once. Unchanged installed remote
revisions reuse complete, validated local copies. Local edits remain
visible. Explicit updates select new revisions.

**Steps to reproduce**

1. Import GitHub skills with supporting files.
2. Run an agent turn, then run another with the same installed
revisions.
3. Observe repeated inventory scans, downloads, and runtime directory
replacement before execution.

Related prior attempts: #2330 and #9268 (still open; #9268 last updated
July 9). Those use a marker compared with `updatedAt`. This patch
follows the required content validation, immutable revision, company
isolation, atomic publication, and read-only semantics, and removes
refresh-per-file multiplication.

## What Changed

- Split public file reading from reading an already loaded skill.
Runtime listing refreshes inventory once.
- Add a company-scoped revision cache with file manifests outside the
delivered skill directory. Fingerprints omit cosmetic metadata.
- Validate exact file inventory, sizes, and hashes before warm reuse.
Reject traversal and symlinks. Stage complete builds and serialize
atomic publication across processes.
- Preserve local/catalog direct sources, stored Markdown fallback,
explicit version snapshots, and legacy mutable-ref compatibility. Report
missing supporting files and keep older valid revisions readable.
- Clean both runtime layouts on rename/removal and record
`skills.prepare` under preparation timing.
- Add service/cache regressions and an isolated 114-skill benchmark,
including a new-process warm run.

## Verification

- Final targeted skill-service/cache/trace validation: 86 tests pass (61
embedded-PostgreSQL service tests, 19 cache tests, 6 trace tests).
Database tests executed rather than skipped. Focused skill routes,
adapter selection, and native runtime context also pass.
- `pnpm -r typecheck` and `pnpm build` pass locally at `22caa1fe4`.
- The full `pnpm test:run` matrix passes on supported Linux CI at the
final head: [CI
run](https://github.com/paperclipai/paperclip/actions/runs/34236097762).
Local full-suite execution encountered PostgreSQL startup contention, a
random allocated-port boundary, and a socket hang-up; every affected
suite passed on an isolated rerun. The interrupted local serialized run
is not claimed as a complete local pass.
- Repeatable benchmark: `pnpm --filter @paperclipai/server exec tsx
../scripts/benchmark-skill-preparation.ts`. Mixed 114-skill inventory
with 429 remote files on Linux: cold 286 ms, warm median 96 ms / maximum
153 ms including a new process. Every warm sample performs one refresh,
zero upstream fetches/rebuilds, and reports no missing entries; content
assertions pass.
- Controlled deployment against the previously deployed revision
completed with zero lost runs. Real inventory: 114 skills, 670 declared
files; 402 cached files match the prior installed copies byte-for-byte.
Ten post-deployment warm preparations: median 129 ms / maximum 208 ms;
new-process warm 194 ms, zero downloads/rebuilds/missing entries.
- Five sequential real browser questions persisted in 10.6–20.7 s
(median 12.2 s), versus 50–83 s before. Skill preparation median 240 ms,
with one 2.37 s outlier. Total preparation median 3.337 s / maximum
8.728 s **does not fully meet** the <3 s / <5 s target. The excluded
historical-run redaction query takes about 1.36 s per scan at two
preparation call sites; wider application latency coincided with the
outlier, without a cache rebuild. These residuals are reported rather
than discarded.
- Disposable skill reimport verified through actual selected-skill runs:
the next run read the changed code. Fixture removed and agent
configuration verified unchanged.
- Greptile 5/5, zero unresolved review threads, all final-head CI checks
green.

## Risks

- Cold preparation still requires upstream availability for supporting
files. An unavailable revision is reported missing and never falls back
to an older revision.
- Valid older revisions and quarantined invalid entries consume additive
disk space until skill cleanup. An abruptly killed publisher can leave a
lock that requires operator cleanup after confirming its PID is dead.
- Warm validation reads all cached file bytes. Very large inventories
still have proportional local I/O cost.
- No HTTP API, schema, agent configuration, or first-party Telemetry
changes. OpenTelemetry retains its operator endpoint gate.

## Model Used

OpenAI GPT-6 in Codex, with reasoning, repository inspection, code
editing, and test execution. The exact serving snapshot and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted and isolated
reruns; full Linux CI matrix passes, local full-run caveats above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

server/src/__tests__/runtime-skill-cache.test.ts

@@ -0,0 +1,174 @@
+import { randomUUID } from "node:crypto";
+import { spawn } from "node:child_process";
+import { fileURLToPath } from "node:url";
+import { promises as fs } from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import type { CompanySkill } from "@paperclipai/shared";
+import { removeRuntimeSkillCache, resolveRuntimeSkillCache, runtimeSkillCacheSpec } from "../services/runtime-skill-cache.js";
+
+async function makeWritable(root: string): Promise<void> {
+  const stat = await fs.lstat(root);
+  if (!stat.isDirectory() || stat.isSymbolicLink()) return;
+  await fs.chmod(root, 0o700);
+  for (const item of await fs.readdir(root)) await makeWritable(path.join(root, item));
+}
+
+async function makeReadonly(root: string): Promise<void> {
+  const stat = await fs.lstat(root);
+  if (stat.isSymbolicLink()) return;
+  if (stat.isDirectory()) {
+    for (const item of await fs.readdir(root)) await makeReadonly(path.join(root, item));
+  }
+  await fs.chmod(root, stat.isDirectory() ? 0o555 : 0o444);
+}
+
+describe("runtime skill revision cache", () => {
+  let root: string;
+  let skill: CompanySkill;
+  const contents: Record<string, string> = { "SKILL.md": "# Test", "references/a.md": "Supporting content" };
+  beforeEach(async () => {
+    root = await fs.mkdtemp(path.join(os.tmpdir(), "runtime-cache-"));
+    skill = { id: randomUUID(), companyId: randomUUID(), sourceType: "github", sourceLocator: "https://github.com/org/repo",
+      sourceRef: "a".repeat(40), slug: "test", markdown: contents["SKILL.md"], metadata: { owner: "org", repo: "repo", repoSkillDir: "skills/test" },
+      fileInventory: [{ path: "SKILL.md", kind: "skill" }, { path: "references/a.md", kind: "reference" }],
+    } as CompanySkill;
+  });
+  afterEach(async () => { await makeWritable(root); await fs.rm(root, { recursive: true, force: true }); });
+  const reader = () => vi.fn(async (file: string) => contents[file]);
+
+  it("publishes complete contents once for twenty callers and leaves warm files untouched", async () => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    const read = reader();
+    const sources = await Promise.all(Array.from({ length: 20 }, () => resolveRuntimeSkillCache(spec, read)));
+    expect(new Set(sources).size).toBe(1);
+    expect(read).toHaveBeenCalledTimes(2);
+    expect((await fs.stat(sources[0]!)).mode & 0o222).toBe(0);
+    expect((await fs.stat(path.join(sources[0]!, "SKILL.md"))).mode & 0o222).toBe(0);
+    const before = await fs.stat(path.join(sources[0]!, "SKILL.md"));
+    read.mockRejectedValue(new Error("Upstream offline"));
+    expect(await resolveRuntimeSkillCache(runtimeSkillCacheSpec(root, { ...skill })!, read)).toBe(sources[0]);
+    expect(await resolveRuntimeSkillCache(spec, read, false)).toBe(sources[0]);
+    expect((await fs.stat(path.join(sources[0]!, "SKILL.md"))).mtimeMs).toBe(before.mtimeMs);
+    expect(await fs.readdir(sources[0]!)).toEqual(["SKILL.md", "references"]);
+    expect(read).toHaveBeenCalledTimes(2);
+  });
+
+  it("reuses the winning complete directory across concurrent processes and a process restart", async () => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    const launch = () => new Promise<{ source: string; reads: number }>((resolve, reject) => {
+      const code = `
+        import { resolveRuntimeSkillCache } from ${JSON.stringify(new URL("../services/runtime-skill-cache.ts", import.meta.url).href)};
+        let reads = 0;
+        const source = await resolveRuntimeSkillCache(${JSON.stringify(spec)}, async (file) => {
+          reads++;
+          await new Promise(resolve => setTimeout(resolve, 30));
+          return ${JSON.stringify(contents)}[file];
+        });
+        console.log(JSON.stringify({ source, reads }));
+      `;
+      const child = spawn(process.execPath, ["--import", fileURLToPath(new URL("../../node_modules/tsx/dist/loader.mjs", import.meta.url)), "--input-type=module", "-e", code], { stdio: ["ignore", "pipe", "pipe"] });
+      let output = "", errors = "";
+      child.stdout.on("data", (chunk) => { output += chunk; });
+      child.stderr.on("data", (chunk) => { errors += chunk; });
+      child.on("error", reject);
+      child.on("exit", (exitCode) => exitCode === 0 ? resolve(JSON.parse(output)) : reject(new Error(errors)));
+    });
+    const [first, second] = await Promise.all([launch(), launch()]);
+    expect(first.source).toBe(second.source);
+    expect(await fs.readdir(spec.root)).toEqual([spec.fingerprint]);
+    expect(await launch()).toEqual({ source: first.source, reads: 0 });
+  });
+
+  it("fingerprints installed content and ownership, excluding cosmetic metadata", async () => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    expect(runtimeSkillCacheSpec(root, { ...skill, name: "Renamed", updatedAt: new Date(), metadata: { ...skill.metadata, starred: true, ref: "new-branch" } })!.fingerprint).toBe(spec.fingerprint);
+    expect(runtimeSkillCacheSpec(root, { ...skill, fileInventory: [...skill.fileInventory].reverse() })!.fingerprint).toBe(spec.fingerprint);
+    for (const update of [{ companyId: randomUUID() }, { sourceRef: "b".repeat(40) }, { markdown: "changed" },
+      { metadata: { ...skill.metadata, repoSkillDir: "other" } }]) {
+      expect(runtimeSkillCacheSpec(root, { ...skill, ...update })!.fingerprint).not.toBe(spec.fingerprint);
+    }
+    expect(runtimeSkillCacheSpec(root, { ...skill, sourceRef: "main" })).toBeNull();
+  });
+
+  it("retains the old revision when publishing an explicit update", async () => {
+    const old = await resolveRuntimeSkillCache(runtimeSkillCacheSpec(root, skill)!, reader());
+    const next = await resolveRuntimeSkillCache(runtimeSkillCacheSpec(root, { ...skill, sourceRef: "b".repeat(40) })!, async () => "updated");
+    expect(next).not.toBe(old);
+    expect(await fs.readFile(path.join(old!, "references/a.md"), "utf8")).toBe(contents["references/a.md"]);
+    expect(await fs.readFile(path.join(next!, "references/a.md"), "utf8")).toBe("updated");
+  });
+
+  it.each(["manifest-missing", "manifest-malformed", "changed", "deleted", "extra", "symlink"])("rejects %s without read-only repair, then rebuilds", async (corruption) => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    const source = (await resolveRuntimeSkillCache(spec, reader()))!;
+    await makeWritable(spec.entry);
+    await fs.chmod(path.join(source, "SKILL.md"), 0o600);
+    await fs.chmod(path.join(spec.entry, "manifest.json"), 0o600);
+    const manifest = path.join(spec.entry, "manifest.json");
+    if (corruption === "manifest-missing") await fs.unlink(manifest);
+    if (corruption === "manifest-malformed") await fs.writeFile(manifest, "{bad");
+    if (corruption === "changed") await fs.writeFile(path.join(source, "SKILL.md"), "tampered");
+    if (corruption === "deleted") await fs.unlink(path.join(source, "SKILL.md"));
+    if (corruption === "extra") await fs.writeFile(path.join(source, "extra.txt"), "extra");
+    if (corruption === "symlink") {
+      await fs.unlink(path.join(source, "SKILL.md"));
+      await fs.symlink(path.join(root, "outside.txt"), path.join(source, "SKILL.md"));
+      await fs.writeFile(path.join(root, "outside.txt"), "outside");
+    }
+    await makeReadonly(spec.entry);
+    const read = reader();
+    expect(await resolveRuntimeSkillCache(spec, read, false)).toBeNull();
+    expect(read).not.toHaveBeenCalled();
+    expect(await resolveRuntimeSkillCache(spec, read)).toBe(source);
+    expect(read).toHaveBeenCalledTimes(2);
+    expect(await fs.readFile(path.join(source, "SKILL.md"), "utf8")).toBe(contents["SKILL.md"]);
+    if (corruption === "symlink") expect(await fs.readFile(path.join(root, "outside.txt"), "utf8")).toBe("outside");
+  });
+
+  it("does not publish partial builds and retries after a failed upstream read", async () => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    const read = reader().mockRejectedValueOnce(new Error("offline"));
+    await expect(resolveRuntimeSkillCache(spec, read)).rejects.toThrow("offline");
+    expect(await fs.readdir(spec.root)).toEqual([]);
+    expect(await resolveRuntimeSkillCache(spec, read, false)).toBeNull();
+    expect(await resolveRuntimeSkillCache(spec, read)).toBe(path.join(spec.entry, "files"));
+  });
+
+  it("coordinates cleanup with an active build and prevents a removed skill from being republished", async () => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    let installed = true;
+    let release!: () => void;
+    let entered!: () => void;
+    const gate = new Promise<void>((resolve) => { release = resolve; });
+    const started = new Promise<void>((resolve) => { entered = resolve; });
+    const build = resolveRuntimeSkillCache(spec, async (file) => {
+      entered(); await gate; return contents[file];
+    }, true, async () => installed);
+    const rejected = expect(build).rejects.toThrow("renamed or removed");
+    await started;
+    installed = false;
+    const removal = removeRuntimeSkillCache(root, skill.id);
+    release();
+    await rejected;
+    await removal;
+    await expect(fs.stat(spec.root)).rejects.toMatchObject({ code: "ENOENT" });
+    await expect(resolveRuntimeSkillCache(spec, reader(), true, async () => installed)).rejects.toThrow("renamed or removed");
+    await expect(fs.stat(spec.root)).rejects.toMatchObject({ code: "ENOENT" });
+  });
+
+  it.each(["../escape", "/absolute", "a/../../escape", "a/../b", "C:\\escape", "a\\..\\escape"])("rejects traversal %s", (file) => {
+    expect(() => runtimeSkillCacheSpec(root, { ...skill, fileInventory: [...skill.fileInventory, { path: file, kind: "reference" }] })).toThrow("Invalid runtime skill file path");
+  });
+
+  it("rejects a symlink cache ancestor before invoking the source reader", async () => {
+    const spec = runtimeSkillCacheSpec(root, skill)!;
+    const outside = await fs.mkdtemp(path.join(root, "outside-"));
+    await fs.symlink(outside, path.dirname(spec.root));
+    const read = reader();
+    await expect(resolveRuntimeSkillCache(spec, read)).rejects.toThrow("Unsafe");
+    expect(read).not.toHaveBeenCalled();
+    expect(await fs.readdir(outside)).toEqual([]);
+  });
+});