mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 15:35:12 +01:00
Carry qualified landed master through 18395
Preserve the complete owner patch and all current master behavior. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
commit
493f7e61e4
36 files changed
+4248
-3326
No files matched your search
@@ -0,0 +1,59 @@
|
||||
## Thinking Path
|
||||
|
||||
> - Paperclip is the open source app people use to manage AI agents for work.
|
||||
> - Contributors need predictable CI feedback.
|
||||
> - Test shards use historical per-file costs to distribute their work.
|
||||
> - New and changed tests make those costs stale.
|
||||
> - This automated PR refreshes the costs from successful protected-master runs.
|
||||
> - The refresh changes placement, while preserving test inventory and runner authorization.
|
||||
|
||||
## Linked Issues or Issue Description
|
||||
|
||||
**What happened?**
|
||||
|
||||
Test costs changed after the last timing refresh. Each manifest records the source runs and measurement coverage.
|
||||
|
||||
**Expected behavior**
|
||||
|
||||
Parallel test jobs should have similar workloads. Timing metadata must never authorize skipping tests.
|
||||
|
||||
**Steps to reproduce**
|
||||
|
||||
Run `node scripts/ci/refresh-test-durations.mjs` with GitHub Actions read access and a full master checkout.
|
||||
|
||||
## What Changed
|
||||
|
||||
- Refresh general-server and serialized timing weights from recent successful master verification runs.
|
||||
|
||||
## Verification
|
||||
|
||||
- The collector validates repository, workflow, branch, event, attempt, successful jobs, commit ancestry and measurement coverage.
|
||||
- Timing validation and full shard coverage tests pass before this branch is pushed.
|
||||
- Review the source run links and coverage recorded in both manifests.
|
||||
- Required PR checks and review must pass before merge. GitHub may require approval to start checks for a PR created with `GITHUB_TOKEN`.
|
||||
|
||||
## Risks
|
||||
|
||||
- Runner load can distort measurements. Medians reduce this noise. Missing measurements retain previous weights.
|
||||
- The changes affect balance, not which tests execute. There is no automatic merge or AWS authorization change.
|
||||
|
||||
## Model Used
|
||||
|
||||
- No model ran this refresh. The generator was authored with OpenAI Codex (GPT-6); exact deployment ID and context window were not exposed. It uses code execution and repository inspection.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] I have included a thinking path that traces from project context to this change
|
||||
- [x] I have specified the model used (with version and capability details)
|
||||
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work (routine CI timing maintenance)
|
||||
- [x] I have searched GitHub for duplicate or related PRs (the workflow pauses while a refresh PR is open)
|
||||
- [x] I have described the issue in-PR following the bug issue template
|
||||
- [x] I have not referenced internal/instance-local Paperclip issues or links
|
||||
- [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details
|
||||
- [x] I have run the relevant timing and shard tests and they pass
|
||||
- [x] I have added or updated tests where applicable (existing timing and shard tests cover these generated values)
|
||||
- [x] I have updated relevant documentation (the manifests record source runs and coverage)
|
||||
- [x] I have considered and documented any risks above
|
||||
- [ ] All Paperclip CI gates are green
|
||||
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
|
||||
- [x] I will address all Greptile and reviewer comments before requesting merge
|
||||
@@ -294,6 +294,10 @@ test("complete merge trees handle more than 300 files, base advancement, renames
|
||||
assert.ok(result.changed.includes("ui/src/deleted.ts")); assert.ok(result.changed.includes("ui/src/renamed.ts"));
|
||||
assert.ok(!result.changed.includes(".github/workflows/unrelated.yml"));
|
||||
assert.equal(plan(result.changed).forceReason, null);
|
||||
const shallow = path.join(directory, "shallow");
|
||||
git("clone", "--quiet", "--depth", "2", `file://${directory}`, shallow);
|
||||
assert.equal(execFileSync("git", ["rev-parse", "--is-shallow-repository"], { cwd: shallow, encoding: "utf8" }).trim(), "true");
|
||||
assert.deepEqual(mergeDiff(head, prHead, shallow), result);
|
||||
assert.throws(() => mergeDiff(head, base, directory), /expected PR head/);
|
||||
assert.throws(() => mergeDiff(prHead, prHead, directory), /merge commit/);
|
||||
assert.throws(() => mergeDiff("f".repeat(40), prHead, directory));
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
name: Refresh CI test durations
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '41 7 * * 1'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
actions: read
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
concurrency:
|
||||
group: refresh-ci-test-durations
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
refresh:
|
||||
if: github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020
|
||||
with:
|
||||
node-version: 24
|
||||
package-manager-cache: false
|
||||
- name: Check for an outstanding refresh
|
||||
id: pending
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
count="$(gh pr list --repo paperclipai/paperclip --state open --search 'in:title "refresh measured test durations"' --limit 100 --json headRefName --jq '[.[] | select(.headRefName | startswith("codex/refresh-ci-durations-"))] | length')"
|
||||
echo "count=$count" >> "$GITHUB_OUTPUT"
|
||||
- name: Refresh from successful protected-master runs
|
||||
if: steps.pending.outputs.count == '0'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: node scripts/ci/refresh-test-durations.mjs
|
||||
- name: Verify timing data and complete shard coverage
|
||||
if: steps.pending.outputs.count == '0'
|
||||
run: node --test scripts/ci/test-selection-timings.test.mjs scripts/__tests__/run-vitest-stable-shard.test.mjs
|
||||
- name: Open a timing-only pull request
|
||||
if: steps.pending.outputs.count == '0'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
if git diff --quiet; then exit 0; fi
|
||||
# The collector must not cause unrelated code to enter the bot PR.
|
||||
git diff --name-only -z | node --input-type=module -e '
|
||||
import {readFileSync} from "node:fs";
|
||||
import assert from "node:assert/strict";
|
||||
const allowed = new Set(["scripts/general-server-shard-durations.json", "scripts/serialized-shard-durations.json"]);
|
||||
for (const file of readFileSync(0,"utf8").split("\0").filter(Boolean)) assert.ok(allowed.has(file), file);
|
||||
'
|
||||
branch="codex/refresh-ci-durations-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT"
|
||||
git switch -c "$branch"
|
||||
git config user.name 'github-actions[bot]'
|
||||
git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
|
||||
git add scripts/general-server-shard-durations.json scripts/serialized-shard-durations.json
|
||||
git commit -m 'chore(ci): refresh measured test durations' -m 'Co-Authored-By: Paperclip <noreply@paperclip.ing>'
|
||||
gh auth setup-git
|
||||
git push origin "$branch"
|
||||
gh pr create --repo paperclipai/paperclip --base master --head "$branch" \
|
||||
--title 'chore(ci): refresh measured test durations' --body-file .github/ci-duration-pr.md
|
||||
@@ -44,7 +44,8 @@ jobs:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
# The merge commit and its two parent trees are the complete diff.
|
||||
fetch-depth: 2
|
||||
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
@@ -274,6 +274,30 @@ ID saved with the plan, even when GitHub increments the run attempt. Turbo hashe
|
||||
`PAPERCLIP_CI_SELECTION_HASH`, so a passing subset cannot satisfy a full run or a
|
||||
different selection.
|
||||
|
||||
Graph discovery runs at most two isolated Node processes concurrently. Each
|
||||
project retains its own Vite configuration, aliases and module graph. Any worker
|
||||
error or timeout rejects the whole refinement and retains the original task plan.
|
||||
The selector checkout needs only the merge commit and its two parents, not the
|
||||
repository's complete history.
|
||||
|
||||
Server test costs are recorded in CI with tests, hooks, collection, setup and
|
||||
environment preparation included. The weekly **Refresh CI test durations**
|
||||
workflow reads two successful protected-master Cloud readiness runs and opens a
|
||||
timing-only PR. It never executes code from logs, changes test selection rules,
|
||||
or grants AWS access. GitHub may require a maintainer to approve the bot PR's
|
||||
workflow runs; the refresh never merges itself. An outstanding refresh PR pauses
|
||||
new refreshes. Both timing manifests include source runs and measurement coverage.
|
||||
The refresh requires at least 90% coverage of each current lane and uses median
|
||||
measurements; missing files retain their old weights. Historical logs can bootstrap
|
||||
test/hook costs until structured loading/setup measurements are available.
|
||||
|
||||
To reproduce a refresh from a full checkout with GitHub Actions read access:
|
||||
|
||||
```sh
|
||||
node scripts/ci/refresh-test-durations.mjs [master-run-id ...]
|
||||
node --test scripts/ci/test-selection-timing*.test.mjs
|
||||
```
|
||||
|
||||
Set the repository variable `CI_VITEST_SELECTION=off` to disable file selection
|
||||
while keeping the existing task-level optimization. Empty or `enabled` enables
|
||||
it; other values disable it. `full-ci`, profiling, and task-level `full`/`shadow`
|
||||
|
||||
@@ -2240,6 +2240,27 @@ replacement launches use current release authority. A missing historical receipt
|
||||
retains the current release gate; guest files cannot establish old authority. A missing, changed, or incompatible package
|
||||
must be replaced and verified before launch.
|
||||
|
||||
The server records the closed `runnerProfileJson.nativeProviderPackAdmission`
|
||||
receipt after actual session admission and attachment. Its `session_admitted`
|
||||
stage and `core_verified_selection` authority describe Core's verified selection,
|
||||
not an independent provider statement. It includes only the bound run, session,
|
||||
runner, lease and pack identities. Retained attachments keep the original pack
|
||||
and identify `inheritedFromRunId`. The first receipt is immutable history.
|
||||
`nativeProviderPackAdmissionState` separately describes current admission proof:
|
||||
`pending`, `verified` or `unknown`, with an opaque admission epoch. The mandatory
|
||||
coordinator claim atomically sets a new pending epoch before session admission.
|
||||
Only the matching owner and epoch can publish post-attachment proof. Failed
|
||||
receipt writes leave pending evidence; changed or unsupported admission leaves
|
||||
unknown evidence and does not prevent authorized execution. A missing legacy
|
||||
receipt remains unknown; the current release cannot supply that missing history.
|
||||
Retained recovery needs positively verified prior provenance, captured before
|
||||
same-run invalidation or read from the exact source run. A fresh replacement
|
||||
uses its actual newly verified selection. Recording the receipt does not grant
|
||||
execution or change the existing recovery gates. Admission proof does not certify
|
||||
provider liveness, physical termination or workspace export.
|
||||
The `runnerEnvironmentLeaseId` names the logical Runner and provider-pack
|
||||
authority scope. It can differ from a physical `environment_leases.id`.
|
||||
|
||||
Warm attachment requires two consecutive authenticated readiness snapshots.
|
||||
Blocked readiness probes back off within the reconnect deadline so they do not
|
||||
fill the durable command journal while waiting. The fast ready path keeps its
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile, execFileSync } from "node:child_process";
|
||||
import { readFileSync, writeFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify, stripVTControlCharacters } from "node:util";
|
||||
import { timingPrefix } from "./test-timing-reporter.mjs";
|
||||
|
||||
const repository = "paperclipai/paperclip";
|
||||
const api = `/repos/${repository}`;
|
||||
const execute = promisify(execFile);
|
||||
const github = async (endpoint, json = true) => {
|
||||
const { stdout } = await execute("gh", ["api", endpoint], {
|
||||
encoding: "utf8", maxBuffer: json ? 4 * 1024 * 1024 : 128 * 1024 * 1024, timeout: 120_000,
|
||||
});
|
||||
return json ? JSON.parse(stdout) : stdout;
|
||||
};
|
||||
|
||||
export function validateRun(run) {
|
||||
assert.equal(run.repository?.full_name, repository);
|
||||
assert.equal(run.repository?.id, 1170821064);
|
||||
assert.equal(run.head_repository?.id, 1170821064);
|
||||
assert.equal(run.head_branch, "master");
|
||||
assert.equal(run.event, "push");
|
||||
assert.equal(run.path, ".github/workflows/cloud-readiness.yml");
|
||||
assert.equal(run.status, "completed");
|
||||
assert.equal(run.conclusion, "success");
|
||||
assert.match(run.head_sha, /^[a-f0-9]{40}$/);
|
||||
assert.ok(Number.isSafeInteger(run.id) && run.id > 0);
|
||||
assert.ok(Number.isSafeInteger(run.run_attempt) && run.run_attempt > 0);
|
||||
}
|
||||
|
||||
export function parseTimings(log, inventory) {
|
||||
const fallback = new Map(), measured = new Map();
|
||||
for (const raw of log.split("\n")) {
|
||||
const line = stripVTControlCharacters(raw);
|
||||
const offset = line.indexOf(timingPrefix);
|
||||
if (offset >= 0) {
|
||||
const record = JSON.parse(line.slice(offset + timingPrefix.length));
|
||||
assert.equal(record.version, 1);
|
||||
assert.equal(typeof record.file, "string");
|
||||
assert.match(record.file, /^server\/(src|scripts)\/.+\.test\.[cm]?[jt]s$/);
|
||||
assert.ok(!record.file.split("/").some(part => part === ".." || part === "." || part === ""));
|
||||
assert.ok(!record.file.includes("\\"));
|
||||
assert.ok(Number.isFinite(record.durationMs) && record.durationMs >= 0 && record.durationMs <= 3_600_000);
|
||||
// A trusted source run can predate a deletion or a lane reassignment.
|
||||
if (!inventory.has(record.file)) continue;
|
||||
measured.set(record.file, Math.max(measured.get(record.file) ?? 0, record.durationMs));
|
||||
continue;
|
||||
}
|
||||
// Bootstrap from pre-reporter runs. These values omit import/setup cost;
|
||||
// structured measurements take precedence as soon as master emits them.
|
||||
const match = line.match(/@paperclipai\/server\s+([^\s]+\.test\.[cm]?[jt]s)\s+\(\d+ tests?[^)]*\)\s+(\d+)ms\s*$/);
|
||||
if (!match) continue;
|
||||
const file = `server/${match[1]}`, ms = Number(match[2]);
|
||||
if (inventory.has(file) && ms <= 3_600_000) fallback.set(file, Math.max(fallback.get(file) ?? 0, ms));
|
||||
}
|
||||
return new Map([...fallback].map(([file, ms]) => [file, { ms, complete: false }])
|
||||
.concat([...measured].map(([file, ms]) => [file, { ms, complete: true }])));
|
||||
}
|
||||
|
||||
export function updatedDurations(previous, inventory, samples, sources) {
|
||||
const measured = [...inventory].filter(file => samples.some(sample => sample.has(file)));
|
||||
assert.ok(measured.length >= inventory.size * 0.9, `Incomplete timing coverage: ${measured.length}/${inventory.size}`);
|
||||
const durations = {};
|
||||
let complete = 0;
|
||||
for (const file of [...inventory].sort()) {
|
||||
const values = samples.flatMap(sample => sample.has(file) ? [sample.get(file)] : []);
|
||||
const full = values.filter(value => value.complete);
|
||||
if (full.length) complete++;
|
||||
const ms = (full.length ? full : values).map(value => value.ms).sort((a, b) => a - b);
|
||||
if (!ms.length) {
|
||||
if (Number.isFinite(previous.durations[file]) && previous.durations[file] >= 0) durations[file] = previous.durations[file];
|
||||
continue;
|
||||
}
|
||||
const middle = Math.floor(ms.length / 2);
|
||||
// A small floor prevents clock resolution from giving a suite zero cost.
|
||||
durations[file] = Math.max(100, Math.ceil(ms.length % 2 ? ms[middle] : (ms[middle - 1] + ms[middle]) / 2));
|
||||
}
|
||||
return {
|
||||
$comment: "Generated by node scripts/ci/refresh-test-durations.mjs from successful protected-master Cloud readiness runs. Median per-file cost includes tests, hooks, collection, setup and environment preparation when the timing reporter is present; older logs bootstrap test/hook durations only. Missing files retain their prior weight or the partitioner's median fallback. Timing changes only partition work; they never authorize skipping a test.",
|
||||
unit: "ms", sources, coverage: { measured: measured.length, withSetup: complete, inventory: inventory.size }, durations,
|
||||
};
|
||||
}
|
||||
|
||||
export async function refreshDurations(runIds = []) {
|
||||
const inventory = JSON.parse(execFileSync(process.execPath, ["scripts/run-vitest-stable.mjs", "--mode", "general",
|
||||
"--group", "general-server", "--shard-index", "0", "--shard-count", "1", "--dry-run"], {
|
||||
encoding: "utf8", env: { ...process.env, GITHUB_WORKFLOW: "PR", PAPERCLIP_CI_TEST_SELECTION: "" },
|
||||
}));
|
||||
const lanes = {
|
||||
general: { files: new Set(inventory.selectedGeneralServerSuites), path: "scripts/general-server-shard-durations.json", samples: [] },
|
||||
serialized: { files: new Set(inventory.selectedSerializedSuites), path: "scripts/serialized-shard-durations.json", samples: [] },
|
||||
};
|
||||
if (!runIds.length) {
|
||||
const listed = await github(`${api}/actions/workflows/cloud-readiness.yml/runs?branch=master&event=push&status=success&per_page=3`);
|
||||
runIds = listed.workflow_runs.map(run => run.id).slice(0, 2);
|
||||
}
|
||||
assert.ok(runIds.length > 0 && runIds.length <= 3, "Expected one to three successful master runs");
|
||||
const sources = [];
|
||||
for (const id of runIds) {
|
||||
assert.match(String(id), /^\d+$/);
|
||||
const run = await github(`${api}/actions/runs/${id}`);
|
||||
validateRun(run);
|
||||
execFileSync("git", ["merge-base", "--is-ancestor", run.head_sha, "HEAD"]);
|
||||
const jobs = [];
|
||||
for (let page = 1; ; page++) {
|
||||
const result = await github(`${api}/actions/runs/${id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`);
|
||||
jobs.push(...result.jobs);
|
||||
if (result.jobs.length < 100) break;
|
||||
}
|
||||
const samples = { general: new Map(), serialized: new Map() };
|
||||
const selected = jobs.map(job => ({ job, lane: /General tests \(server \(\d+\/\d+\)\)$/.test(job.name) ? "general"
|
||||
: /Serialized tests \(\d+\/\d+\)$/.test(job.name) ? "serialized" : null })).filter(item => item.lane);
|
||||
assert.ok(selected.length, "No server timing jobs found");
|
||||
// Keep API traffic and log buffers bounded. Logs are parsed data, never code.
|
||||
for (let start = 0; start < selected.length; start += 3) {
|
||||
await Promise.all(selected.slice(start, start + 3).map(async ({ job, lane }) => {
|
||||
assert.equal(job.run_id, run.id);
|
||||
assert.equal(job.conclusion, "success");
|
||||
const parsed = parseTimings(await github(`${api}/actions/jobs/${job.id}/logs`, false), lanes[lane].files);
|
||||
for (const [file, value] of parsed) {
|
||||
const previous = samples[lane].get(file);
|
||||
if (!previous || (value.complete && !previous.complete)
|
||||
|| (value.complete === previous.complete && value.ms > previous.ms)) samples[lane].set(file, value);
|
||||
}
|
||||
}));
|
||||
}
|
||||
for (const lane of Object.keys(lanes)) lanes[lane].samples.push(samples[lane]);
|
||||
sources.push({ run: run.id, attempt: run.run_attempt, sha: run.head_sha, url: run.html_url });
|
||||
}
|
||||
// Validate both lanes before replacing either checked-in manifest.
|
||||
const updates = Object.values(lanes).map(lane => [lane.path,
|
||||
updatedDurations(JSON.parse(readFileSync(lane.path, "utf8")), lane.files, lane.samples, sources)]);
|
||||
for (const [file, data] of updates) {
|
||||
writeFileSync(file, JSON.stringify(data, null, 2) + "\n");
|
||||
console.log(`${file}: ${data.coverage.measured}/${data.coverage.inventory} measured; ${data.coverage.withSetup} include setup`);
|
||||
}
|
||||
}
|
||||
|
||||
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
refreshDurations(process.argv.slice(2)).catch(error => { console.error(error); process.exitCode = 1; });
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
import { fork } from "node:child_process";
|
||||
import { availableParallelism } from "node:os";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// Each Vite context stays in its own process: aliases, plugins, globals and
|
||||
// heaps cannot leak between projects. Two contexts bound memory on hosted CI.
|
||||
export async function selectProjects({ projects, changed, root = process.cwd(), concurrency = Math.min(2, availableParallelism()) }) {
|
||||
if (!Number.isInteger(concurrency) || concurrency < 1 || concurrency > 2) throw new Error("Expected one or two graph workers");
|
||||
const results = new Array(projects.length), children = new Set();
|
||||
let next = 0, failure;
|
||||
const work = async () => {
|
||||
while (!failure && next < projects.length) {
|
||||
const index = next++;
|
||||
try {
|
||||
results[index] = await new Promise((resolve, reject) => {
|
||||
const env = { ...process.env };
|
||||
delete env.NODE_TEST_CONTEXT;
|
||||
const child = fork(fileURLToPath(import.meta.url), [], {
|
||||
cwd: root, env, execArgv: ["--max-old-space-size=2048"], stdio: ["ignore", "inherit", "inherit", "ipc"],
|
||||
});
|
||||
children.add(child);
|
||||
let result;
|
||||
const timer = setTimeout(() => child.kill("SIGKILL"), 180_000);
|
||||
child.on("message", message => { result = message; });
|
||||
child.once("error", error => {
|
||||
clearTimeout(timer);
|
||||
children.delete(child);
|
||||
child.kill("SIGKILL");
|
||||
reject(error);
|
||||
});
|
||||
child.once("exit", (code, signal) => {
|
||||
clearTimeout(timer);
|
||||
children.delete(child);
|
||||
if (code === 0 && result?.graph?.project === projects[index]) resolve(result.graph);
|
||||
else reject(new Error(result?.error || `Graph worker ${projects[index]} exited ${signal || code}`));
|
||||
});
|
||||
child.send({ root, project: projects[index], changed });
|
||||
});
|
||||
} catch (error) {
|
||||
failure ??= error;
|
||||
for (const child of children) child.kill("SIGKILL");
|
||||
}
|
||||
}
|
||||
};
|
||||
await Promise.all(Array.from({ length: Math.min(concurrency, projects.length) }, work));
|
||||
if (failure) throw failure;
|
||||
return results;
|
||||
}
|
||||
|
||||
if (process.send && process.argv[1] === fileURLToPath(import.meta.url)) {
|
||||
process.once("message", async input => {
|
||||
try {
|
||||
const { selectProjectTests } = await import("./vitest-selection-graph.mjs");
|
||||
const graph = await selectProjectTests(input);
|
||||
process.send({ graph }, () => process.exit(0));
|
||||
} catch (error) {
|
||||
process.send({ error: error.message }, () => process.exit(1));
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import assert from "node:assert/strict";
|
||||
import { refineTestPlan, selectionDurations } from "./test-selection-plan.mjs";
|
||||
import { productInputs } from "../../.github/scripts/ci-inputs.mjs";
|
||||
import { mergeDiff } from "../../.github/scripts/ci-diff.mjs";
|
||||
import { selectProjects } from "./select-projects.mjs";
|
||||
|
||||
export function selectionEligibility(plan) {
|
||||
if (plan?.version !== 1 || plan.mode !== "enabled" || plan.forceReason) return "Full/shadow/forced task plan";
|
||||
@@ -43,9 +44,13 @@ export async function createTestSelection(plan, head) {
|
||||
if (plan.jobs[job].run) for (const entry of plan.jobs[job].matrix.include) groups.add(entry.group ?? "serialized");
|
||||
}
|
||||
groups.delete("general-chat"); // Keep its line sharding and DB ownership intact.
|
||||
const owners = {}, projects = new Set();
|
||||
const owners = {}, projects = new Set(), inventories = new Map();
|
||||
const inventoryFor = group => {
|
||||
if (!inventories.has(group)) inventories.set(group, runnerInventory(group));
|
||||
return inventories.get(group);
|
||||
};
|
||||
for (const group of groups) {
|
||||
const inventory = runnerInventory(group);
|
||||
const inventory = inventoryFor(group);
|
||||
if (group === "serialized" || group === "general-server-without-chat") {
|
||||
owners[group] = { "@paperclipai/server": group === "serialized"
|
||||
? inventory.selectedSerializedSuites : inventory.selectedGeneralServerSuites };
|
||||
@@ -55,12 +60,10 @@ export async function createTestSelection(plan, head) {
|
||||
for (const project of inventory.workspaceProjects) projects.add(project);
|
||||
}
|
||||
}
|
||||
const { selectProjectTests } = await import("./vitest-selection-graph.mjs");
|
||||
const graphs = [];
|
||||
const changed = productInputs(plan.changed);
|
||||
for (const project of projects) {
|
||||
const graph = await selectProjectTests({ project, changed });
|
||||
graphs.push(graph);
|
||||
const graphs = await selectProjects({ projects: [...projects], changed });
|
||||
for (const graph of graphs) {
|
||||
const { project } = graph;
|
||||
console.log(`${project}: ${graph.selected.length}/${graph.inventory.length} files, ${graph.alwaysRun.length} retained conservatively, ${graph.seconds.toFixed(1)}s`);
|
||||
for (const owner of Object.values(owners)) if (owner[project] === null) owner[project] = graph.inventory;
|
||||
}
|
||||
@@ -68,8 +71,8 @@ export async function createTestSelection(plan, head) {
|
||||
for (const file of changed) assert.ok(covered.has(file), `Change outside the resolved test graphs: ${file}`);
|
||||
const server = graphs.find((graph) => graph.project === "@paperclipai/server");
|
||||
if (server) {
|
||||
const general = runnerInventory("general-server-without-chat");
|
||||
const native = runnerInventory("general-server-native-runner");
|
||||
const general = inventoryFor("general-server-without-chat");
|
||||
const native = inventoryFor("general-server-native-runner");
|
||||
assert.deepEqual([...general.selectedGeneralServerSuites, ...general.selectedSerializedSuites,
|
||||
...native.selectedGeneralServerSuites, "server/src/__tests__/chat-channels.integration.test.ts"].sort(),
|
||||
server.inventory, "Server inventory must be completely covered by existing lane ownership");
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import { selectProjects } from "./select-projects.mjs";
|
||||
|
||||
test("parallel project graphs preserve isolation, ordering and conservative selection", async t => {
|
||||
const root = realpathSync(mkdtempSync(path.join(tmpdir(), "parallel-graphs-")));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
symlinkSync(path.resolve("node_modules"), path.join(root, "node_modules"), "dir");
|
||||
const write = (file, source) => writeFileSync(path.join(root, file), source);
|
||||
write(".gitignore", "node_modules/\n");
|
||||
write("vitest.config.mjs", 'export default {test:{projects:["one", "two"]}}');
|
||||
for (const name of ["one", "two"]) {
|
||||
mkdirSync(path.join(root, name));
|
||||
write(`${name}/vitest.config.mjs`, `export default {resolve:{alias:{"@leaf":${JSON.stringify(path.join(root, name, "leaf.ts"))}}},test:{name:${JSON.stringify(name)},include:["*.test.ts"]}}`);
|
||||
write(`${name}/leaf.ts`, "export const value = 1;");
|
||||
write(`${name}/leaf.test.ts`, 'import {value} from "@leaf"; throw new Error("must not execute tests " + value);');
|
||||
write(`${name}/opaque.test.ts`, 'import {readFileSync} from "node:fs"; readFileSync(process.env.INPUT);');
|
||||
}
|
||||
execFileSync("git", ["init", "-q"], { cwd: root });
|
||||
execFileSync("git", ["add", "."], { cwd: root });
|
||||
const options = { root, projects: ["two", "one"], changed: ["one/leaf.ts"] };
|
||||
const serial = await selectProjects({ ...options, concurrency: 1 });
|
||||
const parallel = await selectProjects({ ...options, concurrency: 2 });
|
||||
const stable = graphs => graphs.map(({ seconds, ...graph }) => graph);
|
||||
assert.deepEqual(stable(parallel), stable(serial));
|
||||
assert.deepEqual(parallel[0].selected, ["two/opaque.test.ts"]);
|
||||
assert.deepEqual(parallel[1].selected, ["one/leaf.test.ts", "one/opaque.test.ts"]);
|
||||
await assert.rejects(selectProjects({ ...options, projects: ["missing", "one"] }), /missing/);
|
||||
await assert.rejects(selectProjects({ ...options, concurrency: 3 }), /one or two/);
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import { parseTimings } from "./refresh-test-durations.mjs";
|
||||
|
||||
test("the real Vitest reporter includes import/setup cost and preserves failures", t => {
|
||||
const repo = process.cwd(), root = realpathSync(mkdtempSync(path.join(tmpdir(), "file-cost-")));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
mkdirSync(path.join(root, "server/src"), { recursive: true });
|
||||
symlinkSync(path.join(repo, "node_modules"), path.join(root, "node_modules"), "dir");
|
||||
writeFileSync(path.join(root, "vitest.config.mjs"), `export default {test:{name:"@paperclipai/server",include:["server/src/*.test.mjs"],setupFiles:["./setup.mjs"],reporters:["default",${JSON.stringify(path.join(repo, "scripts/ci/test-timing-reporter.mjs"))}]}}`);
|
||||
writeFileSync(path.join(root, "setup.mjs"), 'await new Promise(resolve=>setTimeout(resolve,60));');
|
||||
writeFileSync(path.join(root, "server/src/probe.test.mjs"), 'import {test} from "vitest"; await new Promise(resolve=>setTimeout(resolve,60)); test("pass",()=>{});');
|
||||
const env = { ...process.env }; delete env.NODE_TEST_CONTEXT;
|
||||
const run = () => spawnSync(process.execPath, [path.join(repo, "node_modules/vitest/vitest.mjs"), "run"], {
|
||||
cwd: root, env, encoding: "utf8", timeout: 30_000,
|
||||
});
|
||||
const result = run();
|
||||
assert.equal(result.status, 0, result.stdout + result.stderr);
|
||||
const file = "server/src/probe.test.mjs";
|
||||
const measurement = parseTimings(result.stdout, new Set([file])).get(file);
|
||||
assert.equal(measurement.complete, true);
|
||||
assert.ok(measurement.ms >= 110, JSON.stringify(measurement));
|
||||
writeFileSync(path.join(root, "server/src/probe.test.mjs"), 'import {test} from "vitest"; test("fail",()=>{throw new Error("expected failure")});');
|
||||
const failed = run();
|
||||
assert.equal(failed.status, 1);
|
||||
assert.equal(parseTimings(failed.stdout, new Set([file])).size, 0);
|
||||
});
|
||||
@@ -0,0 +1,54 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { fileTiming, timingPrefix } from "./test-timing-reporter.mjs";
|
||||
import { parseTimings, updatedDurations, validateRun } from "./refresh-test-durations.mjs";
|
||||
|
||||
const file = "server/src/slow.test.ts", second = "server/src/fast.test.ts";
|
||||
test("file costs include loading and setup, and failed files cannot become samples", () => {
|
||||
const module = { project: { name: "@paperclipai/server" }, moduleId: `${process.cwd()}/${file}`,
|
||||
state: () => "passed", diagnostic: () => ({ duration: 100, collectDuration: 200, setupDuration: 30,
|
||||
prepareDuration: 20, environmentSetupDuration: 10 }) };
|
||||
assert.deepEqual(fileTiming(module), { version: 1, file, durationMs: 360 });
|
||||
assert.equal(fileTiming({ ...module, state: () => "failed" }), null);
|
||||
assert.equal(fileTiming({ ...module, diagnostic: () => ({ duration: NaN }) }), null);
|
||||
});
|
||||
|
||||
test("structured measurements replace bootstrap logs without allowing arbitrary paths", () => {
|
||||
const inventory = new Set([file]);
|
||||
const log = `2026-10-31T00:00:00Z ✓ @paperclipai/server src/slow.test.ts (2 tests) 100ms\n`;
|
||||
assert.deepEqual(parseTimings(log, inventory).get(file), { ms: 100, complete: false });
|
||||
const record = { version: 1, file, durationMs: 360 };
|
||||
assert.deepEqual(parseTimings(log + timingPrefix + JSON.stringify(record), inventory).get(file), { ms: 360, complete: true });
|
||||
assert.equal(parseTimings(timingPrefix + JSON.stringify({ ...record, file: "server/src/deleted.test.ts" }), inventory).size, 0);
|
||||
for (const invalid of [{ ...record, file: "../../package.json" }, { ...record, durationMs: -1 },
|
||||
{ ...record, file: "server/src/../outside.test.ts" },
|
||||
{ ...record, durationMs: "360" }, { ...record, version: 2 }]) {
|
||||
assert.throws(() => parseTimings(timingPrefix + JSON.stringify(invalid), inventory));
|
||||
}
|
||||
});
|
||||
|
||||
test("refresh uses medians, removes obsolete paths and refuses partial lane coverage", () => {
|
||||
const previous = { durations: { [file]: 2, "server/src/removed.test.ts": 42 } };
|
||||
const samples = [new Map([[file, { ms: 2500, complete: false }], [second, { ms: 1, complete: false }]]),
|
||||
new Map([[file, { ms: 3000, complete: true }], [second, { ms: 200, complete: false }]]),
|
||||
new Map([[file, { ms: 4000, complete: true }]])];
|
||||
const result = updatedDurations(previous, new Set([file, second]), samples, []);
|
||||
assert.deepEqual(result.durations, { [second]: 101, [file]: 3500 });
|
||||
assert.throws(() => updatedDurations(previous, new Set([file, second]), [new Map()], []), /Incomplete timing/);
|
||||
});
|
||||
|
||||
test("only successful same-repository master push verification can supply weights", () => {
|
||||
const run = { repository: { id: 1170821064, full_name: "paperclipai/paperclip" }, head_repository: { id: 1170821064 },
|
||||
head_branch: "master", event: "push", path: ".github/workflows/cloud-readiness.yml", status: "completed",
|
||||
conclusion: "success", head_sha: "a".repeat(40), id: 1, run_attempt: 1 };
|
||||
validateRun(run);
|
||||
for (const override of [{ head_repository: { id: 123 } }, { head_branch: "feature" }, { event: "pull_request" },
|
||||
{ conclusion: "failure" }, { status: "in_progress" }, { path: ".github/workflows/pr.yml" }, { run_attempt: 0 }]) {
|
||||
assert.throws(() => validateRun({ ...run, ...override }));
|
||||
}
|
||||
const workflow = readFileSync(".github/workflows/ci-test-durations.yml", "utf8");
|
||||
assert.match(workflow, /runs-on: ubuntu-latest/);
|
||||
assert.match(workflow, /github.ref == 'refs\/heads\/master'/);
|
||||
assert.doesNotMatch(workflow, /pull_request_target|runs-on\/fleet|--force/);
|
||||
});
|
||||
@@ -61,6 +61,7 @@ test("a published selection controls the matrix and the required aggregate", ()
|
||||
|
||||
test("publication follows successful upload and manifests are downloaded without ignoring failures", () => {
|
||||
const selection = block("test_selection");
|
||||
assert.match(selection, /fetch-depth: 2/);
|
||||
assert.ok(selection.indexOf("Upload validated selection") < selection.indexOf("Publish the executable plan"));
|
||||
assert.match(selection, /timeout-minutes: 5/);
|
||||
assert.match(selection, /vars.CI_VITEST_SELECTION == '' \|\| vars.CI_VITEST_SELECTION == 'enabled'/);
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import path from "node:path";
|
||||
|
||||
// One bounded metadata record per server file. No test names, output, or secrets.
|
||||
// The refresh job reads these as data from successful protected-master runs.
|
||||
export const timingPrefix = "PAPERCLIP_TEST_TIMING ";
|
||||
|
||||
export function fileTiming(module, root = process.cwd()) {
|
||||
if (module.project.name !== "@paperclipai/server" || module.state() !== "passed") return null;
|
||||
const file = path.relative(root, module.moduleId).split(path.sep).join("/");
|
||||
if (!/^server\/(?:src|scripts)\/.+\.test\.[cm]?[jt]s$/.test(file)) return null;
|
||||
const timing = module.diagnostic();
|
||||
const phases = [timing.duration, timing.collectDuration, timing.setupDuration,
|
||||
timing.prepareDuration, timing.environmentSetupDuration];
|
||||
if (phases.some(value => !Number.isFinite(value) || value < 0)) return null;
|
||||
return { version: 1, file, durationMs: Math.ceil(phases.reduce((sum, value) => sum + value, 0)) };
|
||||
}
|
||||
|
||||
export default class TestTimingReporter {
|
||||
onTestModuleEnd(module) {
|
||||
const record = fileTiming(module);
|
||||
if (record) console.log(timingPrefix + JSON.stringify(record));
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,150 +1,188 @@
|
||||
{
|
||||
"$comment": "Per-suite wall-clock durations (ms) for serialized route/authz verification, used by scripts/run-vitest-stable.mjs in PR and release shards. Refreshed from successful Cloud readiness run 34705914878 (2026-09-12), jobs 103585803684, 103585803688, 103585803716, 103585803734, and 103585803754. Each weight spans consecutive [test:run] suite-start timestamps; the last suite ends at the first post-job cleanup timestamp. This includes each isolated Vitest process startup, import, collection, tests, and shutdown. All 145 suites are sampled exactly once. New or renamed suites use the median weight. Refresh from a successful complete run when observed shard times drift.",
|
||||
"$comment": "Generated by node scripts/ci/refresh-test-durations.mjs from successful protected-master Cloud readiness runs. Median per-file cost includes tests, hooks, collection, setup and environment preparation when the timing reporter is present; older logs bootstrap test/hook durations only. Missing files retain their prior weight or the partitioner's median fallback. Timing changes only partition work; they never authorize skipping a test.",
|
||||
"unit": "ms",
|
||||
"sources": [
|
||||
{
|
||||
"run": 41202111422,
|
||||
"attempt": 1,
|
||||
"sha": "c2971badde04b602fa9fe698015e1f3653b39e6b",
|
||||
"url": "https://github.com/paperclipai/paperclip/actions/runs/41202111422"
|
||||
},
|
||||
{
|
||||
"run": 41201965500,
|
||||
"attempt": 1,
|
||||
"sha": "d5766e40748ac5881c4b9879fad1d5a61d9b493d",
|
||||
"url": "https://github.com/paperclipai/paperclip/actions/runs/41201965500"
|
||||
}
|
||||
],
|
||||
"coverage": {
|
||||
"measured": 163,
|
||||
"withSetup": 0,
|
||||
"inventory": 163
|
||||
},
|
||||
"durations": {
|
||||
"server/src/__tests__/access-routes-hidden-floor.test.ts": 11187,
|
||||
"server/src/__tests__/access-routes-permissions-upgrade.test.ts": 18919,
|
||||
"server/src/__tests__/activity-routes.test.ts": 5963,
|
||||
"server/src/__tests__/adapter-auth-signal-routes.test.ts": 14105,
|
||||
"server/src/__tests__/adapter-model-refresh-routes.test.ts": 10257,
|
||||
"server/src/__tests__/adapter-routes-authz.test.ts": 7991,
|
||||
"server/src/__tests__/adapter-routes.test.ts": 8580,
|
||||
"server/src/__tests__/agent-action-audit-routes.test.ts": 17869,
|
||||
"server/src/__tests__/agent-adapter-validation-routes.test.ts": 24906,
|
||||
"server/src/__tests__/agent-cross-tenant-authz-routes.test.ts": 9297,
|
||||
"server/src/__tests__/agent-device-login-routes.test.ts": 25216,
|
||||
"server/src/__tests__/agent-hire-idempotency-routes.test.ts": 16903,
|
||||
"server/src/__tests__/agent-instructions-routes.test.ts": 15221,
|
||||
"server/src/__tests__/agent-live-run-routes.test.ts": 30747,
|
||||
"server/src/__tests__/agent-permissions-routes.test.ts": 9849,
|
||||
"server/src/__tests__/agent-secrets-routes.test.ts": 16629,
|
||||
"server/src/__tests__/agent-skills-routes.test.ts": 24106,
|
||||
"server/src/__tests__/agent-test-environment-routes.test.ts": 20121,
|
||||
"server/src/__tests__/approval-routes-idempotency.test.ts": 6366,
|
||||
"server/src/__tests__/artifact-review-document-routes.test.ts": 11415,
|
||||
"server/src/__tests__/assets.test.ts": 7738,
|
||||
"server/src/__tests__/auth-routes.test.ts": 3471,
|
||||
"server/src/__tests__/auth-session-route.test.ts": 4282,
|
||||
"server/src/__tests__/authz-company-access.test.ts": 3348,
|
||||
"server/src/__tests__/authz-existence-oracle-guard.test.ts": 312,
|
||||
"server/src/__tests__/authz-secret-context.test.ts": 3899,
|
||||
"server/src/__tests__/board-chat-route-feature-flag.test.ts": 1251,
|
||||
"server/src/__tests__/bootstrap-claim-routes.test.ts": 5821,
|
||||
"server/src/__tests__/built-in-agent-routes.test.ts": 6607,
|
||||
"server/src/__tests__/cases-routes.test.ts": 17725,
|
||||
"server/src/__tests__/cli-auth-routes.test.ts": 8422,
|
||||
"server/src/__tests__/cloud-routes.test.ts": 3475,
|
||||
"server/src/__tests__/companies-route-cross-company-authz.test.ts": 10608,
|
||||
"server/src/__tests__/companies-route-path-guard.test.ts": 3520,
|
||||
"server/src/__tests__/company-branding-route.test.ts": 6319,
|
||||
"server/src/__tests__/company-import-transfer-routes.test.ts": 10852,
|
||||
"server/src/__tests__/company-portability-routes.test.ts": 5037,
|
||||
"server/src/__tests__/company-portability.test.ts": 6030,
|
||||
"server/src/__tests__/company-search-extract-routes.test.ts": 11382,
|
||||
"server/src/__tests__/company-search-rate-limit-routes.test.ts": 11592,
|
||||
"server/src/__tests__/company-skill-policy-routes.test.ts": 9969,
|
||||
"server/src/__tests__/company-skills-import-authz-routes.test.ts": 16725,
|
||||
"server/src/__tests__/company-skills-routes.test.ts": 6666,
|
||||
"server/src/__tests__/company-user-directory-route.test.ts": 5254,
|
||||
"server/src/__tests__/costs-service.test.ts": 10178,
|
||||
"server/src/__tests__/decision-queues-routes.test.ts": 9909,
|
||||
"server/src/__tests__/document-annotation-routes.test.ts": 12372,
|
||||
"server/src/__tests__/environment-custom-image-routes.test.ts": 8712,
|
||||
"server/src/__tests__/environment-instance-routes.test.ts": 8738,
|
||||
"server/src/__tests__/environment-routes.test.ts": 5462,
|
||||
"server/src/__tests__/environment-selection-route-guards.test.ts": 9749,
|
||||
"server/src/__tests__/execution-workspace-runtime-lease-route.test.ts": 12231,
|
||||
"server/src/__tests__/execution-workspaces-routes.test.ts": 6598,
|
||||
"server/src/__tests__/express5-auth-wildcard.test.ts": 1098,
|
||||
"server/src/__tests__/external-object-routes.test.ts": 14559,
|
||||
"server/src/__tests__/folders-routes.test.ts": 3122,
|
||||
"server/src/__tests__/health-dev-server-token.test.ts": 3808,
|
||||
"server/src/__tests__/health.test.ts": 3664,
|
||||
"server/src/__tests__/heartbeat-dependency-scheduling.test.ts": 22771,
|
||||
"server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts": 26411,
|
||||
"server/src/__tests__/heartbeat-process-recovery.test.ts": 165058,
|
||||
"server/src/__tests__/inbox-agent-policy-routes.test.ts": 17418,
|
||||
"server/src/__tests__/inbox-archive-routes.test.ts": 18276,
|
||||
"server/src/__tests__/instance-database-backups-routes.test.ts": 3602,
|
||||
"server/src/__tests__/instance-settings-routes.test.ts": 3263,
|
||||
"server/src/__tests__/invite-accept-existing-member.test.ts": 5373,
|
||||
"server/src/__tests__/invite-accept-gateway-defaults.test.ts": 16661,
|
||||
"server/src/__tests__/invite-accept-replay.test.ts": 11438,
|
||||
"server/src/__tests__/invite-create-route.test.ts": 5108,
|
||||
"server/src/__tests__/invite-expiry.test.ts": 11055,
|
||||
"server/src/__tests__/invite-join-manager.test.ts": 10519,
|
||||
"server/src/__tests__/invite-list-route.test.ts": 10911,
|
||||
"server/src/__tests__/invite-logo-route.test.ts": 10811,
|
||||
"server/src/__tests__/invite-onboarding-text.test.ts": 9920,
|
||||
"server/src/__tests__/invite-rate-limit-route.test.ts": 11305,
|
||||
"server/src/__tests__/invite-summary-route.test.ts": 12300,
|
||||
"server/src/__tests__/invite-test-resolution-route.test.ts": 10497,
|
||||
"server/src/__tests__/invite-url-public-base-url.test.ts": 5781,
|
||||
"server/src/__tests__/issue-activity-events-routes.test.ts": 13914,
|
||||
"server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts": 50945,
|
||||
"server/src/__tests__/issue-assigned-backlog-contract-routes.test.ts": 9454,
|
||||
"server/src/__tests__/issue-assignee-invokability-routes.test.ts": 9340,
|
||||
"server/src/__tests__/issue-attachment-routes.test.ts": 18688,
|
||||
"server/src/__tests__/issue-blocker-diagnostics-routes.test.ts": 17348,
|
||||
"server/src/__tests__/issue-closed-workspace-routes.test.ts": 9378,
|
||||
"server/src/__tests__/issue-comment-attribution-audit-routes.test.ts": 16687,
|
||||
"server/src/__tests__/issue-comment-cancel-routes.test.ts": 9120,
|
||||
"server/src/__tests__/issue-comment-reopen-routes.test.ts": 9744,
|
||||
"server/src/__tests__/issue-create-deduplication-routes.test.ts": 18131,
|
||||
"server/src/__tests__/issue-created-from-routes.test.ts": 18051,
|
||||
"server/src/__tests__/issue-dependency-wakeups-routes.test.ts": 12754,
|
||||
"server/src/__tests__/issue-document-restore-routes.test.ts": 11341,
|
||||
"server/src/__tests__/issue-execution-policy-routes.test.ts": 17384,
|
||||
"server/src/__tests__/issue-feedback-routes.test.ts": 10562,
|
||||
"server/src/__tests__/issue-identifier-routes.test.ts": 17386,
|
||||
"server/src/__tests__/issue-list-assignee-filter-routes.test.ts": 18762,
|
||||
"server/src/__tests__/issue-list-updatedsince-filter-routes.test.ts": 17534,
|
||||
"server/src/__tests__/issue-onboarding-first-task-routes.test.ts": 19600,
|
||||
"server/src/__tests__/issue-queued-comments-routes.test.ts": 28267,
|
||||
"server/src/__tests__/issue-scheduled-retry-routes.test.ts": 17617,
|
||||
"server/src/__tests__/issue-stale-execution-lock-routes.test.ts": 17166,
|
||||
"server/src/__tests__/issue-stalled-review-decision-routes.test.ts": 18751,
|
||||
"server/src/__tests__/issue-subtree-diagnostics-routes.test.ts": 18263,
|
||||
"server/src/__tests__/issue-telemetry-routes.test.ts": 9904,
|
||||
"server/src/__tests__/issue-thread-interaction-routes.test.ts": 39373,
|
||||
"server/src/__tests__/issue-tree-control-routes.test.ts": 3616,
|
||||
"server/src/__tests__/issue-update-comment-wakeup-routes.test.ts": 15540,
|
||||
"server/src/__tests__/issue-wake-diagnostics-routes.test.ts": 17422,
|
||||
"server/src/__tests__/issue-watchdogs-routes.test.ts": 22439,
|
||||
"server/src/__tests__/issue-workspace-command-authz.test.ts": 9358,
|
||||
"server/src/__tests__/issues-checkout-wakeup.test.ts": 1064,
|
||||
"server/src/__tests__/issues-goal-context-routes.test.ts": 9431,
|
||||
"server/src/__tests__/issues-service.test.ts": 76576,
|
||||
"server/src/__tests__/llms-routes.test.ts": 3399,
|
||||
"server/src/__tests__/low-trust-red-team-routes.test.ts": 27945,
|
||||
"server/src/__tests__/managed-agent-profile-routes-authz.test.ts": 3624,
|
||||
"server/src/__tests__/multilingual-issues-routes.test.ts": 17623,
|
||||
"server/src/__tests__/onboarding-seed-route.test.ts": 13936,
|
||||
"server/src/__tests__/openapi-routes.test.ts": 4089,
|
||||
"server/src/__tests__/openclaw-invite-prompt-route.test.ts": 5131,
|
||||
"server/src/__tests__/opencode-local-adapter-environment.test.ts": 8676,
|
||||
"server/src/__tests__/permissions-upgrade-boundary-routes.test.ts": 16917,
|
||||
"server/src/__tests__/plugin-install-route-security.test.ts": 13924,
|
||||
"server/src/__tests__/plugin-routes-authz.test.ts": 5693,
|
||||
"server/src/__tests__/plugin-scoped-api-routes.test.ts": 6245,
|
||||
"server/src/__tests__/project-goal-telemetry-routes.test.ts": 6735,
|
||||
"server/src/__tests__/project-routes-env.test.ts": 5412,
|
||||
"server/src/__tests__/project-workspace-managed-sandbox-routes.test.ts": 6840,
|
||||
"server/src/__tests__/projects-list-archived-routes.test.ts": 17390,
|
||||
"server/src/__tests__/redaction.test.ts": 1125,
|
||||
"server/src/__tests__/resource-memberships-routes.test.ts": 17692,
|
||||
"server/src/__tests__/routine-document-annotation-routes.test.ts": 3749,
|
||||
"server/src/__tests__/routines-e2e.test.ts": 20181,
|
||||
"server/src/__tests__/routines-routes.test.ts": 6778,
|
||||
"server/src/__tests__/secret-proposals-routes.test.ts": 21824,
|
||||
"server/src/__tests__/secrets-routes-claude-oauth-service.test.ts": 11110,
|
||||
"server/src/__tests__/secrets-routes.test.ts": 10410,
|
||||
"server/src/__tests__/sidebar-preferences-routes.test.ts": 4495,
|
||||
"server/src/__tests__/summary-slot-routes.test.ts": 6451,
|
||||
"server/src/__tests__/teams-catalog-routes.test.ts": 4948,
|
||||
"server/src/__tests__/user-profile-routes.test.ts": 9198,
|
||||
"server/src/__tests__/workspace-command-authz.test.ts": 1044,
|
||||
"server/src/__tests__/workspace-runtime-routes-authz.test.ts": 6915,
|
||||
"server/src/__tests__/workspace-runtime-service-authz.test.ts": 10428
|
||||
"server/src/__tests__/access-routes-hidden-floor.test.ts": 4202,
|
||||
"server/src/__tests__/access-routes-permissions-upgrade.test.ts": 3661,
|
||||
"server/src/__tests__/activity-routes.test.ts": 15018,
|
||||
"server/src/__tests__/adapter-auth-signal-routes.test.ts": 14386,
|
||||
"server/src/__tests__/adapter-model-refresh-routes.test.ts": 7225,
|
||||
"server/src/__tests__/adapter-routes-authz.test.ts": 13466,
|
||||
"server/src/__tests__/adapter-routes.test.ts": 3359,
|
||||
"server/src/__tests__/agent-action-audit-routes.test.ts": 4488,
|
||||
"server/src/__tests__/agent-adapter-validation-routes.test.ts": 77234,
|
||||
"server/src/__tests__/agent-cross-tenant-authz-routes.test.ts": 4565,
|
||||
"server/src/__tests__/agent-device-login-routes.test.ts": 4861,
|
||||
"server/src/__tests__/agent-hire-auth-inheritance-routes.test.ts": 3045,
|
||||
"server/src/__tests__/agent-hire-idempotency-routes.test.ts": 1271,
|
||||
"server/src/__tests__/agent-instructions-routes.test.ts": 3836,
|
||||
"server/src/__tests__/agent-live-run-routes.test.ts": 69136,
|
||||
"server/src/__tests__/agent-permissions-routes.test.ts": 3946,
|
||||
"server/src/__tests__/agent-runner-qualification-routes.test.ts": 1009,
|
||||
"server/src/__tests__/agent-secrets-routes.test.ts": 1104,
|
||||
"server/src/__tests__/agent-skills-routes.test.ts": 4859,
|
||||
"server/src/__tests__/agent-test-environment-routes.test.ts": 6760,
|
||||
"server/src/__tests__/ai-connection-router.test.ts": 19925,
|
||||
"server/src/__tests__/announcements-routes.test.ts": 626,
|
||||
"server/src/__tests__/approval-routes-idempotency.test.ts": 3352,
|
||||
"server/src/__tests__/artifact-review-document-routes.test.ts": 7798,
|
||||
"server/src/__tests__/assets.test.ts": 8543,
|
||||
"server/src/__tests__/auth-routes.test.ts": 100,
|
||||
"server/src/__tests__/auth-session-route.test.ts": 100,
|
||||
"server/src/__tests__/authz-company-access.test.ts": 100,
|
||||
"server/src/__tests__/authz-existence-oracle-guard.test.ts": 100,
|
||||
"server/src/__tests__/authz-secret-context.test.ts": 100,
|
||||
"server/src/__tests__/board-chat-route-feature-flag.test.ts": 895,
|
||||
"server/src/__tests__/bootstrap-claim-routes.test.ts": 100,
|
||||
"server/src/__tests__/browser-use-route-scope.test.ts": 136,
|
||||
"server/src/__tests__/built-in-agent-routes.test.ts": 6740,
|
||||
"server/src/__tests__/cases-routes.test.ts": 4496,
|
||||
"server/src/__tests__/cli-auth-routes.test.ts": 17331,
|
||||
"server/src/__tests__/cloud-routes.test.ts": 100,
|
||||
"server/src/__tests__/companies-route-cross-company-authz.test.ts": 11340,
|
||||
"server/src/__tests__/companies-route-path-guard.test.ts": 100,
|
||||
"server/src/__tests__/company-branding-route.test.ts": 4804,
|
||||
"server/src/__tests__/company-import-transfer-routes.test.ts": 1428,
|
||||
"server/src/__tests__/company-portability-routes.test.ts": 2672,
|
||||
"server/src/__tests__/company-portability.test.ts": 1315,
|
||||
"server/src/__tests__/company-search-extract-routes.test.ts": 115,
|
||||
"server/src/__tests__/company-search-rate-limit-routes.test.ts": 100,
|
||||
"server/src/__tests__/company-skill-policy-routes.test.ts": 626,
|
||||
"server/src/__tests__/company-skills-import-authz-routes.test.ts": 1524,
|
||||
"server/src/__tests__/company-skills-routes.test.ts": 3409,
|
||||
"server/src/__tests__/company-user-directory-route.test.ts": 100,
|
||||
"server/src/__tests__/cost-data-quality-routes.test.ts": 100,
|
||||
"server/src/__tests__/costs-service.test.ts": 1070,
|
||||
"server/src/__tests__/decision-queues-routes.test.ts": 2092,
|
||||
"server/src/__tests__/document-annotation-routes.test.ts": 8418,
|
||||
"server/src/__tests__/email-routes-provider-errors.test.ts": 100,
|
||||
"server/src/__tests__/environment-custom-image-routes.test.ts": 100,
|
||||
"server/src/__tests__/environment-instance-routes.test.ts": 100,
|
||||
"server/src/__tests__/environment-routes.test.ts": 198,
|
||||
"server/src/__tests__/environment-selection-route-guards.test.ts": 119,
|
||||
"server/src/__tests__/execution-workspace-runtime-lease-route.test.ts": 2694,
|
||||
"server/src/__tests__/execution-workspaces-routes.test.ts": 100,
|
||||
"server/src/__tests__/express5-auth-wildcard.test.ts": 100,
|
||||
"server/src/__tests__/external-object-routes.test.ts": 4096,
|
||||
"server/src/__tests__/folders-routes.test.ts": 1358,
|
||||
"server/src/__tests__/health-dev-server-token.test.ts": 104,
|
||||
"server/src/__tests__/health.test.ts": 100,
|
||||
"server/src/__tests__/heartbeat-dependency-scheduling.test.ts": 27885,
|
||||
"server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts": 47816,
|
||||
"server/src/__tests__/heartbeat-process-recovery.test.ts": 324638,
|
||||
"server/src/__tests__/heartbeat-run-privacy-routes.test.ts": 4967,
|
||||
"server/src/__tests__/inbox-agent-policy-routes.test.ts": 831,
|
||||
"server/src/__tests__/inbox-archive-routes.test.ts": 2794,
|
||||
"server/src/__tests__/instance-database-backups-routes.test.ts": 100,
|
||||
"server/src/__tests__/instance-settings-routes.test.ts": 3161,
|
||||
"server/src/__tests__/invite-accept-existing-member.test.ts": 100,
|
||||
"server/src/__tests__/invite-accept-gateway-defaults.test.ts": 584,
|
||||
"server/src/__tests__/invite-accept-replay.test.ts": 100,
|
||||
"server/src/__tests__/invite-create-route.test.ts": 7648,
|
||||
"server/src/__tests__/invite-expiry.test.ts": 100,
|
||||
"server/src/__tests__/invite-join-manager.test.ts": 100,
|
||||
"server/src/__tests__/invite-list-route.test.ts": 457,
|
||||
"server/src/__tests__/invite-logo-route.test.ts": 100,
|
||||
"server/src/__tests__/invite-onboarding-text.test.ts": 100,
|
||||
"server/src/__tests__/invite-rate-limit-route.test.ts": 5390,
|
||||
"server/src/__tests__/invite-summary-route.test.ts": 6262,
|
||||
"server/src/__tests__/invite-test-resolution-route.test.ts": 4135,
|
||||
"server/src/__tests__/invite-url-public-base-url.test.ts": 4075,
|
||||
"server/src/__tests__/issue-access-grants-routes.test.ts": 4910,
|
||||
"server/src/__tests__/issue-activity-events-routes.test.ts": 9271,
|
||||
"server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts": 9524,
|
||||
"server/src/__tests__/issue-assigned-backlog-contract-routes.test.ts": 4067,
|
||||
"server/src/__tests__/issue-assignee-invokability-routes.test.ts": 125,
|
||||
"server/src/__tests__/issue-attachment-routes.test.ts": 4226,
|
||||
"server/src/__tests__/issue-blocker-diagnostics-routes.test.ts": 1268,
|
||||
"server/src/__tests__/issue-closed-workspace-routes.test.ts": 4032,
|
||||
"server/src/__tests__/issue-comment-attribution-audit-routes.test.ts": 4466,
|
||||
"server/src/__tests__/issue-comment-cancel-routes.test.ts": 3762,
|
||||
"server/src/__tests__/issue-comment-reopen-routes.test.ts": 501,
|
||||
"server/src/__tests__/issue-create-deduplication-routes.test.ts": 1646,
|
||||
"server/src/__tests__/issue-created-from-routes.test.ts": 828,
|
||||
"server/src/__tests__/issue-creation-source-routes.test.ts": 1836,
|
||||
"server/src/__tests__/issue-dependency-wakeups-routes.test.ts": 8947,
|
||||
"server/src/__tests__/issue-document-restore-routes.test.ts": 7787,
|
||||
"server/src/__tests__/issue-execution-policy-routes.test.ts": 3678,
|
||||
"server/src/__tests__/issue-feedback-routes.test.ts": 3786,
|
||||
"server/src/__tests__/issue-identifier-routes.test.ts": 622,
|
||||
"server/src/__tests__/issue-list-assignee-filter-routes.test.ts": 6773,
|
||||
"server/src/__tests__/issue-list-updatedsince-filter-routes.test.ts": 772,
|
||||
"server/src/__tests__/issue-onboarding-first-task-routes.test.ts": 6124,
|
||||
"server/src/__tests__/issue-queued-comments-routes.test.ts": 47964,
|
||||
"server/src/__tests__/issue-scheduled-retry-routes.test.ts": 7804,
|
||||
"server/src/__tests__/issue-stale-execution-lock-routes.test.ts": 2222,
|
||||
"server/src/__tests__/issue-stalled-review-decision-routes.test.ts": 2218,
|
||||
"server/src/__tests__/issue-status-version-route-service.test.ts": 881,
|
||||
"server/src/__tests__/issue-subtree-diagnostics-routes.test.ts": 1299,
|
||||
"server/src/__tests__/issue-summaries-routes.test.ts": 832,
|
||||
"server/src/__tests__/issue-telemetry-routes.test.ts": 4919,
|
||||
"server/src/__tests__/issue-thread-interaction-routes.test.ts": 14647,
|
||||
"server/src/__tests__/issue-tree-control-routes.test.ts": 1004,
|
||||
"server/src/__tests__/issue-update-comment-wakeup-routes.test.ts": 4204,
|
||||
"server/src/__tests__/issue-wake-diagnostics-routes.test.ts": 1493,
|
||||
"server/src/__tests__/issue-watchdogs-routes.test.ts": 6085,
|
||||
"server/src/__tests__/issue-workspace-command-authz.test.ts": 4361,
|
||||
"server/src/__tests__/issues-checkout-wakeup.test.ts": 100,
|
||||
"server/src/__tests__/issues-goal-context-routes.test.ts": 100,
|
||||
"server/src/__tests__/issues-service.test.ts": 14248,
|
||||
"server/src/__tests__/llms-routes.test.ts": 1700,
|
||||
"server/src/__tests__/low-trust-red-team-routes.test.ts": 21637,
|
||||
"server/src/__tests__/managed-agent-profile-routes-authz.test.ts": 100,
|
||||
"server/src/__tests__/multilingual-issues-routes.test.ts": 990,
|
||||
"server/src/__tests__/muse-qualification-routes.test.ts": 100,
|
||||
"server/src/__tests__/onboarding-seed-route.test.ts": 2184,
|
||||
"server/src/__tests__/openapi-routes.test.ts": 808,
|
||||
"server/src/__tests__/openclaw-invite-prompt-route.test.ts": 100,
|
||||
"server/src/__tests__/opencode-local-adapter-environment.test.ts": 6051,
|
||||
"server/src/__tests__/openrouter-attribution.test.ts": 100,
|
||||
"server/src/__tests__/permissions-upgrade-boundary-routes.test.ts": 1292,
|
||||
"server/src/__tests__/plugin-install-route-security.test.ts": 3548,
|
||||
"server/src/__tests__/plugin-routes-authz.test.ts": 3421,
|
||||
"server/src/__tests__/plugin-scoped-api-routes.test.ts": 2549,
|
||||
"server/src/__tests__/project-goal-telemetry-routes.test.ts": 3418,
|
||||
"server/src/__tests__/project-routes-env.test.ts": 4210,
|
||||
"server/src/__tests__/project-workspace-managed-sandbox-routes.test.ts": 6883,
|
||||
"server/src/__tests__/projects-list-archived-routes.test.ts": 1152,
|
||||
"server/src/__tests__/redaction.test.ts": 100,
|
||||
"server/src/__tests__/resource-memberships-routes.test.ts": 1629,
|
||||
"server/src/__tests__/routine-document-annotation-routes.test.ts": 2027,
|
||||
"server/src/__tests__/routines-e2e.test.ts": 17403,
|
||||
"server/src/__tests__/routines-routes.test.ts": 6826,
|
||||
"server/src/__tests__/secret-proposals-routes.test.ts": 9151,
|
||||
"server/src/__tests__/secrets-routes-claude-oauth-service.test.ts": 1360,
|
||||
"server/src/__tests__/secrets-routes.test.ts": 198,
|
||||
"server/src/__tests__/sidebar-preferences-routes.test.ts": 3043,
|
||||
"server/src/__tests__/subscription-routes.test.ts": 100,
|
||||
"server/src/__tests__/summary-slot-routes.test.ts": 5832,
|
||||
"server/src/__tests__/task-connector-read-routes.test.ts": 122,
|
||||
"server/src/__tests__/task-title-routes.test.ts": 4602,
|
||||
"server/src/__tests__/teams-catalog-routes.test.ts": 3849,
|
||||
"server/src/__tests__/user-profile-routes.test.ts": 1209,
|
||||
"server/src/__tests__/workspace-command-authz.test.ts": 100,
|
||||
"server/src/__tests__/workspace-preferences-routes.test.ts": 100,
|
||||
"server/src/__tests__/workspace-restore-repair-route.test.ts": 1666,
|
||||
"server/src/__tests__/workspace-runtime-routes-authz.test.ts": 3430,
|
||||
"server/src/__tests__/workspace-runtime-service-authz.test.ts": 926
|
||||
}
|
||||
}
|
||||
@@ -165,6 +165,7 @@ const mockObserveCrossIssueInfluence = vi.hoisted(() => vi.fn(async () => null))
|
||||
const mockRetryWorkspaceExport = vi.hoisted(() => vi.fn());
|
||||
const mockRequestWorkspaceRecovery = vi.hoisted(() => vi.fn());
|
||||
const mockRequestTaskRetry = vi.hoisted(() => vi.fn());
|
||||
const mockOnboardingFirstTaskDependenciesAllowed = vi.hoisted(() => vi.fn(async () => undefined));
|
||||
const mockGetExperimental = vi.hoisted(() => vi.fn());
|
||||
const mockResumeWorkspaceFinalization = vi.hoisted(() => vi.fn());
|
||||
|
||||
@@ -176,6 +177,10 @@ function registerRouteMocks() {
|
||||
}) };
|
||||
});
|
||||
vi.doMock("../services/task-retry.js", () => ({ requestTaskRetry: mockRequestTaskRetry }));
|
||||
vi.doMock("../services/onboarding-first-task-admission.js", async () => ({
|
||||
...await vi.importActual<typeof import("../services/onboarding-first-task-admission.js")>("../services/onboarding-first-task-admission.js"),
|
||||
assertOnboardingFirstTaskDependenciesAllowed: mockOnboardingFirstTaskDependenciesAllowed,
|
||||
}));
|
||||
vi.doMock("../services/task-interruption-presentation.js", () => ({
|
||||
taskInterruptionPresentation: vi.fn(async () => null),
|
||||
taskInterruptionPresentations: vi.fn(async () => new Map()),
|
||||
@@ -2980,6 +2985,118 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("native run changing its own task status", () => {
|
||||
const nativeRun = {
|
||||
id: ownerRunId,
|
||||
companyId,
|
||||
agentId: ownerAgentId,
|
||||
runtimeMode: "native",
|
||||
status: "running",
|
||||
contextSnapshot: { issueId },
|
||||
};
|
||||
const runActor = () => ({ ...ownerActor(), source: "agent_jwt" });
|
||||
|
||||
beforeEach(() => {
|
||||
mockIssueService.getById.mockResolvedValue(makeIssue({ executionRunId: ownerRunId, checkoutRunId: ownerRunId }));
|
||||
mockHeartbeatService.getRun.mockResolvedValue(nativeRun);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["in_review", {}],
|
||||
["in_review", { assigneeAgentId: ownerAgentId }],
|
||||
["done", {}],
|
||||
["blocked", {}],
|
||||
["todo", { blockedByIssueIds: [] }],
|
||||
["blocked", { blockedByIssueIds: ["bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"] }],
|
||||
])("rejects %s before it releases the run's own execution lock (%j)", async (status, extra) => {
|
||||
const res = await request(createApp(runActor())).patch(`/api/issues/${issueId}`).send({ status, ...extra });
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(409);
|
||||
expect(res.body.details).toMatchObject({ code: "native_run_status_reported_by_finish", requestedStatus: status });
|
||||
expect(res.body.error).toContain("paperclip_finish");
|
||||
expect(mockIssueService.update).not.toHaveBeenCalled();
|
||||
expect(mockHeartbeatService.cancelRun).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still lets the run edit other fields of its task", async () => {
|
||||
const res = await request(createApp(runActor())).patch(`/api/issues/${issueId}`).send({ title: "Renamed by its run" });
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(200);
|
||||
expect(mockIssueService.update).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["a legacy run", { runtimeMode: "legacy" }, {}],
|
||||
["a native run that no longer owns execution", {}, { executionRunId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", checkoutRunId: null }],
|
||||
["a native run that is no longer running", { status: "failed" }, {}],
|
||||
])("leaves the status change to the existing rules for %s", async (_label, runOverrides, issueOverrides) => {
|
||||
mockHeartbeatService.getRun.mockResolvedValue({ ...nativeRun, ...runOverrides });
|
||||
mockIssueService.getById.mockResolvedValue(makeIssue({ executionRunId: ownerRunId, checkoutRunId: ownerRunId, ...issueOverrides }));
|
||||
|
||||
const res = await request(createApp(runActor())).patch(`/api/issues/${issueId}`).send({ status: "in_review" });
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(200);
|
||||
expect(res.body).toMatchObject({ status: "in_review" });
|
||||
expect(mockIssueService.update).toHaveBeenCalledTimes(1);
|
||||
expect(mockIssueService.update.mock.calls[0]?.[1]).toMatchObject({ status: "in_review" });
|
||||
});
|
||||
|
||||
it("does not intercept a native run handing its task to a human", async () => {
|
||||
// A native handoff still stops the outgoing run first; confirm that stop
|
||||
// so the request reaches the update instead of the stop-unconfirmed 409.
|
||||
mockHeartbeatService.cancelRun.mockResolvedValue({ ...nativeRun, status: "cancelled" });
|
||||
const handoff = { assigneeAgentId: null, assigneeUserId: "board-user", status: "in_review" };
|
||||
|
||||
const res = await request(createApp(runActor(), createRunContextDb({}, ownerAgentId, ownerRunId, [], true)))
|
||||
.patch(`/api/issues/${issueId}`).send(handoff);
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(200);
|
||||
expect(res.body).toMatchObject(handoff);
|
||||
expect(mockHeartbeatService.cancelRun).toHaveBeenCalledWith(ownerRunId, expect.anything(), expect.anything());
|
||||
expect(mockIssueService.update).toHaveBeenCalledTimes(1);
|
||||
expect(mockHeartbeatService.cancelRun.mock.invocationCallOrder[0])
|
||||
.toBeLessThan(mockIssueService.update.mock.invocationCallOrder[0]!);
|
||||
expect(mockIssueService.update.mock.calls[0]?.[1]).toMatchObject(handoff);
|
||||
});
|
||||
|
||||
it("does not intercept a native conversation reply moving to review", async () => {
|
||||
mockIssueService.getById.mockResolvedValue(makeIssue({ executionRunId: ownerRunId, checkoutRunId: ownerRunId,
|
||||
conversationAgentId: ownerAgentId, conversationUserId: "board-user" }));
|
||||
|
||||
const res = await request(createApp(runActor())).patch(`/api/issues/${issueId}`).send({ status: "in_review" });
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(200);
|
||||
expect(res.body).toMatchObject({ status: "in_review" });
|
||||
expect(mockIssueService.update).toHaveBeenCalledTimes(1);
|
||||
expect(mockIssueService.update.mock.calls[0]?.[1]).toMatchObject({ status: "in_review" });
|
||||
});
|
||||
|
||||
it("keeps the dependency rule's error for a blocker it rejects", async () => {
|
||||
const { HttpError: CurrentHttpError } = await vi.importActual<typeof import("../errors.js")>("../errors.js");
|
||||
mockOnboardingFirstTaskDependenciesAllowed.mockRejectedValueOnce(
|
||||
new CurrentHttpError(400, "Remove the onboarding task from blockedByIssueIds.", { code: "onboarding_first_task_parent_blocker" }));
|
||||
|
||||
const res = await request(createApp(runActor())).patch(`/api/issues/${issueId}`)
|
||||
.send({ status: "blocked", blockedByIssueIds: ["bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"] });
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(400);
|
||||
expect(res.body.details).toMatchObject({ code: "onboarding_first_task_parent_blocker" });
|
||||
expect(mockOnboardingFirstTaskDependenciesAllowed).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
|
||||
companyId, actorAgentId: ownerAgentId, actorRunId: ownerRunId, blockedByIssueIds: ["bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"],
|
||||
}));
|
||||
expect(mockIssueService.update).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not intercept a board user changing the status", async () => {
|
||||
const res = await request(createApp(boardActor())).patch(`/api/issues/${issueId}`).send({ status: "in_review" });
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(200);
|
||||
expect(res.body).toMatchObject({ status: "in_review" });
|
||||
expect(mockIssueService.update).toHaveBeenCalledTimes(1);
|
||||
expect(mockIssueService.update.mock.calls[0]?.[1]).toMatchObject({ status: "in_review" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("task watchdog scope grants", () => {
|
||||
const watchdogRunId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaab";
|
||||
const watchdogReportIssueId = "cccccccc-cccc-4ccc-8ccc-cccccccccccd";
|
||||
|
||||
@@ -36,6 +36,7 @@ import { reconstructNativeWorkspace, unreconciledNativeExportPredicate } from ".
|
||||
import { reconstructWorkspaceSchema } from "@paperclipai/shared";
|
||||
import { queuedInteractionId, readQueuedInteractionResponse, hasQueuedInteractionResponse } from "../services/queued-interaction-response.js";
|
||||
import { deliverConversationComments, isConversation } from "../services/agent-conversations.js";
|
||||
import { assertOnboardingFirstTaskDependenciesAllowed } from "../services/onboarding-first-task-admission.js";
|
||||
import { resolveChatTaskAssignment } from "../services/chat-task-assignment.js";
|
||||
import { issueRecoveryActionReadModel } from "../services/issue-recovery-actions.js";
|
||||
import { inspectWorkspaceSource } from "../services/workspace-source-inspection.js";
|
||||
@@ -13857,6 +13858,51 @@ export function issueRoutes(
|
||||
if (active) throw conflict("An agent currently owns execution. Wait for it to finish, or manage execution in Paperclip.", { code: "MCP_ACTIVE_EXECUTION" });
|
||||
}
|
||||
|
||||
/**
|
||||
* A native run reports its outcome through paperclip_finish/paperclip_block,
|
||||
* and the status arbiter applies the task status. A status change sent here
|
||||
* by that same run (for example through call_api) releases the run's own
|
||||
* execution lock. Every runner tool after that, paperclip_finish included,
|
||||
* is unauthorized, so the run can never settle and recovery blocks the task.
|
||||
* Reject the change before it commits, so the run can correct it in the same
|
||||
* turn. Handoffs and reassignments are a different path and are unaffected.
|
||||
*
|
||||
* Conversation tasks are excluded: a conversation turn settles from its final
|
||||
* reply (nativeConversationReplyResult), not from a runner tool, so a reply
|
||||
* that moves the conversation to review still settles without the lock.
|
||||
*/
|
||||
async function assertNativeRunKeepsOwnExecutionLock(
|
||||
req: Request,
|
||||
issue: { companyId: string; status: string; parentId: string | null; assigneeAgentId: string | null; assigneeUserId: string | null; checkoutRunId: string | null; executionRunId: string | null; conversationAgentId?: string | null; conversationUserId?: string | null },
|
||||
) {
|
||||
const runId = req.actor.type === "agent" ? req.actor.runId : null;
|
||||
if (!runId || !req.actor.agentId || req.actor.agentId !== issue.assigneeAgentId) return;
|
||||
if (runId !== issue.executionRunId && runId !== issue.checkoutRunId) return;
|
||||
if (req.body.status === undefined || req.body.status === issue.status) return;
|
||||
if (isConversation(issue)) return;
|
||||
const keepsAssignment =
|
||||
(req.body.assigneeAgentId === undefined || req.body.assigneeAgentId === issue.assigneeAgentId) &&
|
||||
(req.body.assigneeUserId === undefined || req.body.assigneeUserId === issue.assigneeUserId);
|
||||
if (!keepsAssignment) return;
|
||||
const run = await heartbeat.getRun(runId);
|
||||
if (!run || run.runtimeMode !== "native" || run.status !== "running"
|
||||
|| run.agentId !== req.actor.agentId || run.companyId !== issue.companyId) return;
|
||||
// A blocker that the dependency rules reject anyway keeps its specific
|
||||
// error; otherwise the run reports the block with paperclip_block.
|
||||
if (Array.isArray(req.body.blockedByIssueIds) && req.body.blockedByIssueIds.length > 0) {
|
||||
await assertOnboardingFirstTaskDependenciesAllowed(db, {
|
||||
companyId: issue.companyId, actorAgentId: req.actor.agentId, actorRunId: runId,
|
||||
blockedByIssueIds: req.body.blockedByIssueIds, parentIssueId: issue.parentId,
|
||||
});
|
||||
}
|
||||
throw conflict(
|
||||
"This run owns the task's execution, so it cannot change the task status directly; doing so would end its access to the task before it reports. " +
|
||||
"Report the outcome with paperclip_finish instead: reportedWorkDisposition \"needs_review\" for review, \"done\" when complete, or \"yielded\" with continuation.kind \"monitor\" to wait on a scheduled monitor. " +
|
||||
"Use paperclip_block for a blocker. Paperclip sets the task status from that report.",
|
||||
{ code: "native_run_status_reported_by_finish", requestedStatus: req.body.status },
|
||||
);
|
||||
}
|
||||
|
||||
router.patch(
|
||||
"/issues/:id",
|
||||
(req, _res, next) => {
|
||||
@@ -13887,6 +13933,7 @@ export function issueRoutes(
|
||||
);
|
||||
if (!existing) return;
|
||||
if (!(await assertIssueReadAllowed(req, res, existing))) return;
|
||||
await assertNativeRunKeepsOwnExecutionLock(req, existing);
|
||||
if (req.body.expectedStatusVersion !== undefined && (
|
||||
existing.statusVersion !== req.body.expectedStatusVersion || existing.status !== "backlog"
|
||||
)) {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { subscribeDatabaseWork, databaseWorkPending, reconcileDatabaseWork } from "@paperclipai/db";
|
||||
import { subscribeDatabaseWork, databaseWorkPending, reconcileDatabaseWork } from "@paperclipai/db/work-signals";
|
||||
import { createWorkScheduler } from "./work-scheduler.js";
|
||||
import { type DeliveryQueue } from "./delivery-work-notifications.js";
|
||||
import { beginIdleTrackedWork } from "./task-admission.js";
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { signalDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db";
|
||||
import { signalDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db/work-signals";
|
||||
|
||||
/** Topics for existing durable queues; register intent before their writes. */
|
||||
export const DELIVERY_QUEUES = {
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { agents, companies, createDb, heartbeatRuns, issues, nativeRunFinalizations, type Db } from "@paperclipai/db";
|
||||
import type { NativeSession } from "../../vendor/paperclip-runner/index.js";
|
||||
import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
import { canonicalProviderPackJson, type ProviderPackIdentity } from "./remote-provider-pack-verification.js";
|
||||
import { admittedNativeProviderPack, nativeProviderPackAdmissionPending, recordNativeProviderPackAdmission,
|
||||
rememberAdmittedNativeProviderPack, rememberRecoveredNativeProviderPack } from "./native-provider-pack-admission.js";
|
||||
|
||||
const hash = (value: string) => `sha256:${createHash("sha256").update(value).digest("hex")}`;
|
||||
function manifest(revision = "a".repeat(40)): ProviderPackIdentity {
|
||||
const artifact = { path: "/private/provider-path", sha256: hash(revision) };
|
||||
const payload = { runnerSourceRevision: revision, pins: { opencode: "1.18.34" },
|
||||
target: { platform: "linux", architecture: "x64" },
|
||||
artifacts: { nodeCommand: artifact, opencodeCommand: artifact, opencodeProxy: artifact } };
|
||||
return { schema: "paperclip-runner/remote-provider-pack/v1", payload, digest: hash(canonicalProviderPackJson(payload)) };
|
||||
}
|
||||
const initial = { companyId: "company", agentId: "agent", issueId: "issue", runId: "source-run", sessionId: "session" };
|
||||
function sessionFixture(identity = initial) {
|
||||
let current = { ...identity };
|
||||
const session = { identity: () => ({ ...current }), attachRun: async ({ identity: next }: { identity: typeof initial }) => { current = { ...next }; } } as NativeSession;
|
||||
return { session, attach: (next: typeof initial) => session.attachRun!({ identity: next }) };
|
||||
}
|
||||
const binding = { runnerInstanceId: "runner", environmentLeaseId: "lease" };
|
||||
function remember(session: NativeSession, identity = initial, selected = manifest()) {
|
||||
return rememberAdmittedNativeProviderPack(session, { identity, ...binding, manifest: selected });
|
||||
}
|
||||
const verified = { ...nativeProviderPackAdmissionPending(randomUUID()), state: "verified" };
|
||||
function receiptDb(receipt: unknown, state: unknown = verified): Db {
|
||||
return { select: () => ({ from: () => ({ where: () => ({ limit: async () => [{ receipt, state }] }) }) }) } as unknown as Db;
|
||||
}
|
||||
|
||||
describe("provider pack admission identity", () => {
|
||||
it("does not publish selected configuration before successful session admission", async () => {
|
||||
const { session } = sessionFixture();
|
||||
expect(admittedNativeProviderPack(session, initial)).toBeNull();
|
||||
});
|
||||
it("keeps a closed original pack identity through an actual target attachment", async () => {
|
||||
const { session, attach } = sessionFixture();
|
||||
const original = manifest();
|
||||
expect(remember(session, initial, original)).toBe(true);
|
||||
const before = admittedNativeProviderPack(session, initial)!;
|
||||
const target = { ...initial, runId: "target-run", issueId: "target-issue" };
|
||||
expect(admittedNativeProviderPack(session, target)).toBeNull();
|
||||
await attach(target);
|
||||
const next = admittedNativeProviderPack(session, target)!;
|
||||
expect(next).toEqual({ ...before, runId: target.runId, inheritedFromRunId: initial.runId });
|
||||
expect(JSON.stringify(next)).not.toContain("private/provider-path");
|
||||
expect(Object.keys(next).sort()).toEqual(["schema", "stage", "authority", "companyId", "agentId", "runId", "sessionId",
|
||||
"runnerInstanceId", "runnerEnvironmentLeaseId", "providerPackDigest", "opencodeProxySha256", "sourceRevision", "inheritedFromRunId"].sort());
|
||||
expect(remember(session, target, original)).toBe(true);
|
||||
expect(admittedNativeProviderPack(session, target)).toEqual(next);
|
||||
expect(remember(session, target, manifest("b".repeat(40)))).toBe(true);
|
||||
expect(admittedNativeProviderPack(session, target)).toMatchObject({ sourceRevision: "b".repeat(40), runId: target.runId });
|
||||
});
|
||||
it.each(["companyId", "agentId", "issueId", "runId", "sessionId"] as const)("rejects a returned session with a different %s", key => {
|
||||
const { session } = sessionFixture();
|
||||
expect(remember(session, { ...initial, [key]: "foreign" })).toBe(false);
|
||||
expect(admittedNativeProviderPack(session, initial)).toBeNull();
|
||||
});
|
||||
it("rejects stale manifest digests and unavailable source identity", () => {
|
||||
const { session } = sessionFixture();
|
||||
const stale = manifest(); stale.payload.artifacts.opencodeProxy.sha256 = hash("changed");
|
||||
expect(remember(session, initial, stale)).toBe(false);
|
||||
expect(remember(session, initial, manifest("unknown"))).toBe(false);
|
||||
expect(admittedNativeProviderPack(session, initial)).toBeNull();
|
||||
});
|
||||
it("does not reuse an earlier in-memory proof for an unsupported new admission", async () => {
|
||||
const { session } = sessionFixture(); remember(session);
|
||||
expect(remember(session, initial, manifest("unknown"))).toBe(false);
|
||||
expect(admittedNativeProviderPack(session, initial)).toBeNull();
|
||||
remember(session);
|
||||
await expect(rememberRecoveredNativeProviderPack(receiptDb(undefined), session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding })).resolves.toBe(false);
|
||||
expect(admittedNativeProviderPack(session, initial)).toBeNull();
|
||||
});
|
||||
it("restores only the exact prior closed Core receipt and leaves legacy absence unknown", async () => {
|
||||
const source = sessionFixture(); remember(source.session);
|
||||
const saved = admittedNativeProviderPack(source.session, initial)!;
|
||||
const recovered = sessionFixture();
|
||||
await expect(rememberRecoveredNativeProviderPack(receiptDb(undefined), recovered.session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding })).resolves.toBe(false);
|
||||
expect(admittedNativeProviderPack(recovered.session, initial)).toBeNull();
|
||||
await expect(rememberRecoveredNativeProviderPack(receiptDb(saved), recovered.session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding })).resolves.toBe(true);
|
||||
const target = { ...initial, runId: "target-run" };
|
||||
expect(admittedNativeProviderPack(recovered.session, target)).toBeNull();
|
||||
await recovered.attach(target);
|
||||
expect(admittedNativeProviderPack(recovered.session, target)).toEqual({ ...saved, runId: target.runId, inheritedFromRunId: initial.runId });
|
||||
});
|
||||
it.each(["companyId", "agentId", "runId", "sessionId", "runnerInstanceId", "runnerEnvironmentLeaseId"] as const)("rejects a prior receipt with a different %s", async key => {
|
||||
const source = sessionFixture(); remember(source.session);
|
||||
const saved = { ...admittedNativeProviderPack(source.session, initial)!, [key]: "foreign" };
|
||||
const recovered = sessionFixture();
|
||||
await expect(rememberRecoveredNativeProviderPack(receiptDb(saved), recovered.session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding })).resolves.toBe(false);
|
||||
expect(admittedNativeProviderPack(recovered.session, initial)).toBeNull();
|
||||
});
|
||||
it.each([undefined, { ...verified, state: "pending" }, { ...verified, state: "unknown" }])("does not revive a historical receipt without verified current provenance: %j", async state => {
|
||||
const source = sessionFixture(); remember(source.session);
|
||||
const recovered = sessionFixture();
|
||||
const absentState = state === undefined ? null : state;
|
||||
await expect(rememberRecoveredNativeProviderPack(receiptDb(admittedNativeProviderPack(source.session, initial), absentState), recovered.session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding })).resolves.toBe(false);
|
||||
expect(admittedNativeProviderPack(recovered.session, initial)).toBeNull();
|
||||
});
|
||||
it("uses only the exact same-run locked pre-invalidation source snapshot", async () => {
|
||||
const source = sessionFixture(); remember(source.session);
|
||||
const saved = admittedNativeProviderPack(source.session, initial)!;
|
||||
const recovered = sessionFixture();
|
||||
const db = receiptDb(saved, nativeProviderPackAdmissionPending(randomUUID()));
|
||||
await expect(rememberRecoveredNativeProviderPack(db, recovered.session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding, source: { runId: "foreign-run", receipt: saved, state: verified } })).resolves.toBe(false);
|
||||
await expect(rememberRecoveredNativeProviderPack(db, recovered.session,
|
||||
{ identity: initial, sourceRunId: initial.runId, ...binding, source: { runId: initial.runId, receipt: saved, state: verified } })).resolves.toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("provider pack admission PostgreSQL fencing", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
beforeAll(async () => { database = await startEmbeddedPostgresTestDatabase("pack-admission-"); db = createDb(database.connectionString); }, 90_000);
|
||||
afterAll(async () => { await database?.cleanup(); });
|
||||
async function fixture() {
|
||||
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Pack admission", issuePrefix: randomUUID().slice(0, 8) });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Pack agent" });
|
||||
await db.insert(issues).values({ id: issueId, companyId, title: "Pack task", assigneeAgentId: agentId });
|
||||
const identity = { companyId, agentId, issueId, runId, sessionId: `session-${runId}` };
|
||||
const admissionEpoch = randomUUID();
|
||||
const profile = { preserved: { value: "unchanged" }, nativeExecutionInput: { binding: identity },
|
||||
nativeProviderPackAdmissionState: nativeProviderPackAdmissionPending(admissionEpoch) };
|
||||
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, nativeIssueId: issueId, runtimeMode: "native", status: "running", runnerProfileJson: profile });
|
||||
const claim = { leaseOwner: randomUUID(), attempt: 1, controllerGeneration: 2, controllerBootId: randomUUID() };
|
||||
await db.insert(nativeRunFinalizations).values({ runId, companyId, issueId, phase: "observed", ...claim,
|
||||
leaseExpiresAt: new Date(Date.now() + 60_000) });
|
||||
const { session } = sessionFixture(identity); remember(session, identity);
|
||||
return { identity, session, claim, profile, admissionEpoch };
|
||||
}
|
||||
const read = async (runId: string) => (await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId)))[0]!;
|
||||
it("merges the exact closed receipt without changing other profile data and is idempotent", async () => {
|
||||
const f = await fixture();
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(true);
|
||||
const saved = await read(f.identity.runId);
|
||||
expect(saved.runnerProfileJson).toEqual({ ...f.profile, nativeProviderPackAdmission: admittedNativeProviderPack(f.session, f.identity),
|
||||
nativeProviderPackAdmissionState: { ...f.profile.nativeProviderPackAdmissionState, state: "verified" } });
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(true);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual(saved.runnerProfileJson);
|
||||
});
|
||||
it.each(["leaseOwner", "attempt", "controllerGeneration", "controllerBootId"] as const)("rejects stale %s without publishing a receipt", async key => {
|
||||
const f = await fixture();
|
||||
const changed = typeof f.claim[key] === "number" ? 99 : randomUUID();
|
||||
await expect(recordNativeProviderPackAdmission(db, { ...f, claim: { ...f.claim, [key]: changed } as typeof f.claim })).resolves.toBe(false);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual(f.profile);
|
||||
});
|
||||
it("rejects an expired claim and a terminal run", async () => {
|
||||
const f = await fixture();
|
||||
await db.update(nativeRunFinalizations).set({ leaseExpiresAt: new Date(0) }).where(eq(nativeRunFinalizations.runId, f.identity.runId));
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(false);
|
||||
await db.update(nativeRunFinalizations).set({ leaseExpiresAt: new Date(Date.now() + 60_000) }).where(eq(nativeRunFinalizations.runId, f.identity.runId));
|
||||
await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, f.identity.runId));
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(false);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual(f.profile);
|
||||
});
|
||||
it("preserves the original immutable receipt when another pack is proposed", async () => {
|
||||
const f = await fixture(); await recordNativeProviderPackAdmission(db, f);
|
||||
const before = (await read(f.identity.runId)).runnerProfileJson!;
|
||||
const other = sessionFixture(f.identity); remember(other.session, f.identity, manifest("b".repeat(40)));
|
||||
await expect(recordNativeProviderPackAdmission(db, { ...f, session: other.session })).resolves.toBe(false);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual({ ...before,
|
||||
nativeProviderPackAdmissionState: { ...f.profile.nativeProviderPackAdmissionState, state: "unknown" } });
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(false);
|
||||
});
|
||||
it("preserves the original inheritance when the same run recovers the same verified pack", async () => {
|
||||
const f = await fixture(), sourceRunId = randomUUID();
|
||||
const receipt = admittedNativeProviderPack(f.session, f.identity)!;
|
||||
await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId: f.identity.companyId, agentId: f.identity.agentId,
|
||||
nativeIssueId: f.identity.issueId, runtimeMode: "native", status: "succeeded",
|
||||
runnerProfileJson: { nativeProviderPackAdmission: { ...receipt, runId: sourceRunId } } });
|
||||
const original = { ...f.profile, nativeProviderPackAdmission: { ...receipt, inheritedFromRunId: sourceRunId } };
|
||||
await db.update(heartbeatRuns).set({ runnerProfileJson: original }).where(eq(heartbeatRuns.id, f.identity.runId));
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(true);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual({ ...original,
|
||||
nativeProviderPackAdmissionState: { ...f.profile.nativeProviderPackAdmissionState, state: "verified" } });
|
||||
});
|
||||
it("leaves a newer pending epoch untouched by an older same-claim callback", async () => {
|
||||
const f = await fixture(); await recordNativeProviderPackAdmission(db, f);
|
||||
const previous = (await read(f.identity.runId)).runnerProfileJson!;
|
||||
const next = { ...previous, nativeProviderPackAdmissionState: nativeProviderPackAdmissionPending(randomUUID()) };
|
||||
await db.update(heartbeatRuns).set({ runnerProfileJson: next }).where(eq(heartbeatRuns.id, f.identity.runId));
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(false);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual(next);
|
||||
});
|
||||
it.each(["runner", "lease", "unsupported"])("keeps the first receipt historical after an authorized %s replacement", async change => {
|
||||
const f = await fixture(); await recordNativeProviderPackAdmission(db, f);
|
||||
const previous = (await read(f.identity.runId)).runnerProfileJson!;
|
||||
const admissionEpoch = randomUUID();
|
||||
await db.update(heartbeatRuns).set({ runnerProfileJson: { ...previous,
|
||||
nativeProviderPackAdmissionState: nativeProviderPackAdmissionPending(admissionEpoch) } }).where(eq(heartbeatRuns.id, f.identity.runId));
|
||||
const replacement = sessionFixture(f.identity);
|
||||
if (change !== "unsupported") rememberAdmittedNativeProviderPack(replacement.session, { identity: f.identity, manifest: manifest(),
|
||||
...binding, ...(change === "runner" ? { runnerInstanceId: "replacement" } : { environmentLeaseId: "replacement" }) });
|
||||
await expect(recordNativeProviderPackAdmission(db, { ...f, admissionEpoch, session: replacement.session })).resolves.toBe(false);
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual({ ...previous,
|
||||
nativeProviderPackAdmissionState: { ...nativeProviderPackAdmissionPending(admissionEpoch), state: "unknown" } });
|
||||
});
|
||||
it("cannot publish while another transaction holds the coordinator claim", async () => {
|
||||
const f = await fixture();
|
||||
await recordNativeProviderPackAdmission(db, f);
|
||||
const previous = (await read(f.identity.runId)).runnerProfileJson!;
|
||||
f.admissionEpoch = randomUUID();
|
||||
const pending = { ...previous, nativeProviderPackAdmissionState: nativeProviderPackAdmissionPending(f.admissionEpoch) };
|
||||
await db.update(heartbeatRuns).set({ runnerProfileJson: pending }).where(eq(heartbeatRuns.id, f.identity.runId));
|
||||
let release!: () => void, locked!: () => void;
|
||||
const acquired = new Promise<void>(resolve => { locked = resolve; });
|
||||
const hold = new Promise<void>(resolve => { release = resolve; });
|
||||
const owner = db.transaction(async tx => {
|
||||
await tx.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, f.identity.runId)).for("update");
|
||||
locked(); await hold;
|
||||
});
|
||||
try {
|
||||
await acquired;
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).rejects.toThrow();
|
||||
expect((await read(f.identity.runId)).runnerProfileJson).toEqual(pending);
|
||||
} finally { release(); await owner; }
|
||||
await expect(recordNativeProviderPackAdmission(db, f)).resolves.toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,167 @@
|
||||
import { and, eq, gt, isNull, sql } from "drizzle-orm";
|
||||
import { createHash } from "node:crypto";
|
||||
import { heartbeatRuns, nativeRunFinalizations, type Db } from "@paperclipai/db";
|
||||
import type { NativeSession } from "../../vendor/paperclip-runner/index.js";
|
||||
import { canonicalProviderPackJson, type ProviderPackIdentity } from "./remote-provider-pack-verification.js";
|
||||
|
||||
type Selection = {
|
||||
admittedRunId: string;
|
||||
companyId: string;
|
||||
agentId: string;
|
||||
sessionId: string;
|
||||
runnerInstanceId: string;
|
||||
environmentLeaseId: string;
|
||||
providerPackDigest: string;
|
||||
opencodeProxySha256: string;
|
||||
sourceRevision: string;
|
||||
};
|
||||
|
||||
export type NativeProviderPackAdmission = Omit<Selection, "admittedRunId" | "environmentLeaseId"> & {
|
||||
schema: "paperclip.native-provider-pack-admission.v1";
|
||||
stage: "session_admitted";
|
||||
authority: "core_verified_selection";
|
||||
runId: string;
|
||||
runnerEnvironmentLeaseId: string;
|
||||
inheritedFromRunId?: string;
|
||||
};
|
||||
|
||||
const admitted = new WeakMap<NativeSession, Readonly<Selection>>();
|
||||
const digest = /^sha256:[a-f0-9]{64}$/;
|
||||
type NativeRunIdentity = ReturnType<NativeSession["identity"]>;
|
||||
|
||||
export type NativeProviderPackAdmissionSource = { runId: string; receipt: unknown; state: unknown };
|
||||
|
||||
export function nativeProviderPackAdmissionPending(admissionEpoch: string) {
|
||||
return { schema: "paperclip.native-provider-pack-admission-state.v1" as const, state: "pending" as const, admissionEpoch };
|
||||
}
|
||||
|
||||
function verifiedState(value: unknown): boolean {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
|
||||
const state = value as Record<string, unknown>;
|
||||
return Object.keys(state).sort().join(",") === "admissionEpoch,schema,state"
|
||||
&& state.schema === "paperclip.native-provider-pack-admission-state.v1" && state.state === "verified"
|
||||
&& typeof state.admissionEpoch === "string" && /^[a-f0-9-]{36}$/.test(state.admissionEpoch);
|
||||
}
|
||||
|
||||
function rememberSelection(session: NativeSession, selected: Selection): boolean {
|
||||
const prior = admitted.get(session);
|
||||
if (prior && canonicalProviderPackJson(prior) === canonicalProviderPackJson({ ...selected, admittedRunId: prior.admittedRunId })) return true;
|
||||
admitted.set(session, Object.freeze(selected));
|
||||
return true;
|
||||
}
|
||||
|
||||
export function rememberAdmittedNativeProviderPack(session: NativeSession, input: {
|
||||
identity: NativeRunIdentity;
|
||||
runnerInstanceId: string;
|
||||
environmentLeaseId: string;
|
||||
manifest: ProviderPackIdentity;
|
||||
}): boolean {
|
||||
const unavailable = () => { admitted.delete(session); return false; };
|
||||
if (typeof session.identity !== "function") return unavailable();
|
||||
const actual = session.identity();
|
||||
if (canonicalProviderPackJson(actual) !== canonicalProviderPackJson(input.identity)) return unavailable();
|
||||
const proxy = input.manifest.payload.artifacts.opencodeProxy?.sha256;
|
||||
const revision = input.manifest.payload.runnerSourceRevision;
|
||||
if (!digest.test(input.manifest.digest)
|
||||
|| input.manifest.digest !== `sha256:${createHash("sha256").update(canonicalProviderPackJson(input.manifest.payload)).digest("hex")}`
|
||||
|| !digest.test(proxy ?? "") || !/^[a-f0-9]{40}$/.test(revision)
|
||||
|| !input.runnerInstanceId || !input.environmentLeaseId) return unavailable();
|
||||
const selected: Selection = {
|
||||
admittedRunId: actual.runId,
|
||||
companyId: actual.companyId, agentId: actual.agentId, sessionId: actual.sessionId,
|
||||
runnerInstanceId: input.runnerInstanceId, environmentLeaseId: input.environmentLeaseId,
|
||||
providerPackDigest: input.manifest.digest, opencodeProxySha256: proxy!, sourceRevision: revision,
|
||||
};
|
||||
return rememberSelection(session, selected);
|
||||
}
|
||||
|
||||
export async function rememberRecoveredNativeProviderPack(db: Db, session: NativeSession, input: {
|
||||
identity: NativeRunIdentity;
|
||||
sourceRunId: string;
|
||||
runnerInstanceId: string;
|
||||
environmentLeaseId: string;
|
||||
source?: NativeProviderPackAdmissionSource;
|
||||
}): Promise<boolean> {
|
||||
admitted.delete(session);
|
||||
if (typeof session.identity !== "function") return false;
|
||||
if (canonicalProviderPackJson(session.identity()) !== canonicalProviderPackJson(input.identity)) return false;
|
||||
const source = input.source?.runId === input.sourceRunId ? input.source : (await db.select({
|
||||
receipt: sql<unknown>`${heartbeatRuns.runnerProfileJson}->'nativeProviderPackAdmission'`,
|
||||
state: sql<unknown>`${heartbeatRuns.runnerProfileJson}->'nativeProviderPackAdmissionState'`,
|
||||
}).from(heartbeatRuns).where(and(eq(heartbeatRuns.id, input.sourceRunId), eq(heartbeatRuns.companyId, input.identity.companyId),
|
||||
eq(heartbeatRuns.agentId, input.identity.agentId), eq(heartbeatRuns.runtimeMode, "native"))).limit(1))[0];
|
||||
if (!verifiedState(source?.state)) return false;
|
||||
const value = source?.receipt;
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
|
||||
const receipt = value as Record<string, unknown>;
|
||||
const keys = new Set(["schema", "stage", "authority", "companyId", "agentId", "runId", "sessionId", "runnerInstanceId",
|
||||
"runnerEnvironmentLeaseId", "providerPackDigest", "opencodeProxySha256", "sourceRevision", "inheritedFromRunId"]);
|
||||
if (Object.keys(receipt).some(key => !keys.has(key)) || receipt.schema !== "paperclip.native-provider-pack-admission.v1"
|
||||
|| receipt.stage !== "session_admitted" || receipt.authority !== "core_verified_selection"
|
||||
|| receipt.runId !== input.sourceRunId || receipt.companyId !== input.identity.companyId || receipt.agentId !== input.identity.agentId
|
||||
|| receipt.sessionId !== input.identity.sessionId || receipt.runnerInstanceId !== input.runnerInstanceId
|
||||
|| receipt.runnerEnvironmentLeaseId !== input.environmentLeaseId || !digest.test(String(receipt.providerPackDigest))
|
||||
|| !digest.test(String(receipt.opencodeProxySha256)) || !/^[a-f0-9]{40}$/.test(String(receipt.sourceRevision))
|
||||
|| (receipt.inheritedFromRunId !== undefined && (typeof receipt.inheritedFromRunId !== "string" || !receipt.inheritedFromRunId))) return false;
|
||||
return rememberSelection(session, { admittedRunId: String(receipt.inheritedFromRunId ?? receipt.runId),
|
||||
companyId: input.identity.companyId, agentId: input.identity.agentId, sessionId: input.identity.sessionId,
|
||||
runnerInstanceId: input.runnerInstanceId, environmentLeaseId: input.environmentLeaseId,
|
||||
providerPackDigest: String(receipt.providerPackDigest), opencodeProxySha256: String(receipt.opencodeProxySha256),
|
||||
sourceRevision: String(receipt.sourceRevision) });
|
||||
}
|
||||
|
||||
export function admittedNativeProviderPack(session: NativeSession, identity: NativeRunIdentity): NativeProviderPackAdmission | null {
|
||||
const selected = admitted.get(session);
|
||||
if (!selected) return null;
|
||||
const actual = session.identity();
|
||||
if (canonicalProviderPackJson(actual) !== canonicalProviderPackJson(identity)
|
||||
|| actual.companyId !== selected.companyId || actual.agentId !== selected.agentId || actual.sessionId !== selected.sessionId) return null;
|
||||
const { admittedRunId, environmentLeaseId, ...identityFields } = selected;
|
||||
return { schema: "paperclip.native-provider-pack-admission.v1", stage: "session_admitted",
|
||||
authority: "core_verified_selection", ...identityFields, runId: actual.runId, runnerEnvironmentLeaseId: environmentLeaseId,
|
||||
...(admittedRunId === actual.runId ? {} : { inheritedFromRunId: admittedRunId }) };
|
||||
}
|
||||
|
||||
export async function recordNativeProviderPackAdmission(db: Db, input: {
|
||||
session: NativeSession;
|
||||
identity: NativeRunIdentity;
|
||||
admissionEpoch: string;
|
||||
claim: { leaseOwner: string; attempt: number; controllerGeneration: number; controllerBootId: string };
|
||||
}): Promise<boolean> {
|
||||
const receipt = admittedNativeProviderPack(input.session, input.identity);
|
||||
return db.transaction(async tx => {
|
||||
const [owner] = await tx.select({ runId: nativeRunFinalizations.runId }).from(nativeRunFinalizations).where(and(
|
||||
eq(nativeRunFinalizations.runId, input.identity.runId), eq(nativeRunFinalizations.companyId, input.identity.companyId),
|
||||
eq(nativeRunFinalizations.issueId, input.identity.issueId), eq(nativeRunFinalizations.leaseOwner, input.claim.leaseOwner),
|
||||
eq(nativeRunFinalizations.attempt, input.claim.attempt), eq(nativeRunFinalizations.controllerGeneration, input.claim.controllerGeneration),
|
||||
eq(nativeRunFinalizations.controllerBootId, input.claim.controllerBootId), gt(nativeRunFinalizations.leaseExpiresAt, sql`now()`),
|
||||
eq(nativeRunFinalizations.phase, "observed"), isNull(nativeRunFinalizations.resultId),
|
||||
)).for("update", { noWait: true }).limit(1);
|
||||
if (!owner) return false;
|
||||
const [run] = await tx.select({ profile: heartbeatRuns.runnerProfileJson }).from(heartbeatRuns).where(and(
|
||||
eq(heartbeatRuns.id, input.identity.runId), eq(heartbeatRuns.companyId, input.identity.companyId),
|
||||
eq(heartbeatRuns.agentId, input.identity.agentId), eq(heartbeatRuns.nativeIssueId, input.identity.issueId),
|
||||
eq(heartbeatRuns.runtimeMode, "native"), eq(heartbeatRuns.status, "running"),
|
||||
)).for("update", { noWait: true }).limit(1);
|
||||
if (!run) return false;
|
||||
const current = run.profile?.nativeProviderPackAdmissionState as Record<string, unknown> | undefined;
|
||||
if (current?.schema !== "paperclip.native-provider-pack-admission-state.v1" || current.admissionEpoch !== input.admissionEpoch
|
||||
|| !["pending", "verified", "unknown"].includes(String(current.state))) return false;
|
||||
if (current.state === "unknown") return false;
|
||||
const previous = run.profile?.nativeProviderPackAdmission;
|
||||
const verified = receipt != null && (previous == null ? current.state === "pending"
|
||||
: canonicalProviderPackJson({ ...(previous as Record<string, unknown>), inheritedFromRunId: null })
|
||||
=== canonicalProviderPackJson({ ...receipt, inheritedFromRunId: null }));
|
||||
if (verified && current.state === "verified") return true;
|
||||
const patch = { ...(verified && previous == null ? { nativeProviderPackAdmission: receipt } : {}),
|
||||
nativeProviderPackAdmissionState: { ...nativeProviderPackAdmissionPending(input.admissionEpoch), state: verified ? "verified" : "unknown" } };
|
||||
const saved = await tx.update(heartbeatRuns).set({
|
||||
runnerProfileJson: sql`coalesce(${heartbeatRuns.runnerProfileJson}, '{}'::jsonb) || ${JSON.stringify(patch)}::jsonb`,
|
||||
updatedAt: new Date(),
|
||||
}).where(and(eq(heartbeatRuns.id, input.identity.runId), eq(heartbeatRuns.companyId, input.identity.companyId),
|
||||
eq(heartbeatRuns.agentId, input.identity.agentId), eq(heartbeatRuns.nativeIssueId, input.identity.issueId),
|
||||
eq(heartbeatRuns.runtimeMode, "native"), eq(heartbeatRuns.status, "running"),
|
||||
)).returning({ id: heartbeatRuns.id });
|
||||
return verified && saved.length === 1;
|
||||
});
|
||||
}
|
||||
@@ -100,6 +100,9 @@ import { NativeRunnerOwnershipUnverifiedError } from "./native-runner-ownership.
|
||||
import type { AdapterRuntimeEvent } from "../../adapters/index.js";
|
||||
import type { AdapterUsageCheckpoint } from "@paperclipai/adapter-utils";
|
||||
import { acknowledgedNativeStopExecutionHasStopped, isAcknowledgedNativeStop } from "../acknowledged-native-stop.js";
|
||||
import { admittedNativeProviderPack, nativeProviderPackAdmissionPending, rememberAdmittedNativeProviderPack } from "./native-provider-pack-admission.js";
|
||||
import { canonicalProviderPackJson } from "./remote-provider-pack-verification.js";
|
||||
import * as providerPackAdmission from "./native-provider-pack-admission.js";
|
||||
|
||||
vi.mock("node:fs", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("node:fs")>();
|
||||
@@ -1700,7 +1703,7 @@ describe("remote provider pack manifest", () => {
|
||||
return JSON.stringify(value);
|
||||
};
|
||||
|
||||
it.each(["opencode", "opencode-diagnostics", "opencode-diagnostics-stale", "cursor", "claude-x64", "claude-arm64"] as const)("uses the bundled Linux image authority for remote %s without controller overrides", async provider => {
|
||||
it.each(["opencode", "opencode-dirty", "opencode-diagnostics", "opencode-diagnostics-stale", "cursor", "claude-x64", "claude-arm64"] as const)("uses the bundled Linux image authority for remote %s without controller overrides", async provider => {
|
||||
const architecture = provider === "claude-arm64" ? "arm64" : "x64";
|
||||
const claude = provider.startsWith("claude-");
|
||||
const opencode = provider.startsWith("opencode");
|
||||
@@ -1746,7 +1749,7 @@ describe("remote provider pack manifest", () => {
|
||||
const syncIn = vi.fn(async (_operations: { files: { kind: string }[] }[]) => undefined);
|
||||
const installed = structuredClone(manifest);
|
||||
if (opencode) {
|
||||
installed.payload.runnerSourceRevision = "c".repeat(40);
|
||||
installed.payload.runnerSourceRevision = "c".repeat(40) + (provider === "opencode-dirty" ? "-dirty" : "");
|
||||
installed.payload.artifacts.nodeCommand.sha256 = "sha256:" + "d".repeat(64);
|
||||
if (provider !== "opencode-diagnostics") installed.payload.artifacts.opencodeProxy.sha256 = "sha256:" + "e".repeat(64);
|
||||
installed.digest = "sha256:" + createHash("sha256").update(canonical(installed.payload)).digest("hex");
|
||||
@@ -1761,6 +1764,7 @@ describe("remote provider pack manifest", () => {
|
||||
}) : "",
|
||||
}));
|
||||
try {
|
||||
let coldAdmission: ReturnType<typeof admittedNativeProviderPack> = null;
|
||||
const prepare = createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
|
||||
runnerInstanceId: "bundled-image-runner", runnerIngressAuthorized: true,
|
||||
...(opencode ? { runnerEnvironment: { ...configuredEnvironmentProjection(diagnostics ? { PAPERCLIP_OPENCODE_PERFORMANCE_DIAGNOSTICS: "1" } : {}), PAPERCLIP_NATIVE_MCP_NAME: "assigned-fixture", PAPERCLIP_NATIVE_MCP_URL: "http://127.0.0.1/mcp/gateways/assigned-fixture", PAPERCLIP_NATIVE_MCP_TOKEN: "private-fixture" } } : {}),
|
||||
@@ -1772,7 +1776,7 @@ describe("remote provider pack manifest", () => {
|
||||
expect(state.createBackend).not.toHaveBeenCalled();
|
||||
return;
|
||||
}
|
||||
await prepare;
|
||||
const managedBackend = await prepare;
|
||||
state.createTransport.mockClear();
|
||||
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
|
||||
const transport = state.createTransport.mock.calls.at(-1)![0] as RunnerTransportOptions & {
|
||||
@@ -1798,6 +1802,28 @@ describe("remote provider pack manifest", () => {
|
||||
expect(transport).toMatchObject({ providerPackAuthorityDigest: installed.digest,
|
||||
providerNodeCommand: "/opt/paperclip-runner/provider-pack/node_modules/node/bin/node",
|
||||
providerNodeCommandSha256: installed.payload.artifacts.nodeCommand.sha256 });
|
||||
if (provider === "opencode" || provider === "opencode-dirty") {
|
||||
const identity = { ...remoteExecution.binding, sessionId: remoteExecution.session.normalizedSessionId! };
|
||||
const session = { identity: () => identity } as never;
|
||||
const backend = state.createBackend.mock.results.at(-1)!.value;
|
||||
const openSession = vi.fn().mockRejectedValueOnce(new Error("provider open failed")).mockResolvedValue(session);
|
||||
Object.assign(backend, { openSession });
|
||||
expect(admittedNativeProviderPack(session, identity)).toBeNull();
|
||||
await expect(managedBackend.openSession({ identity })).rejects.toThrow("provider open failed");
|
||||
expect(admittedNativeProviderPack(session, identity)).toBeNull();
|
||||
const opened = await managedBackend.openSession({ identity });
|
||||
if (provider === "opencode-dirty") {
|
||||
expect(admittedNativeProviderPack(opened, identity)).toBeNull();
|
||||
expect(existsSync(join(transport.stateDirectory!, "provider-pack-authority.json"))).toBe(true);
|
||||
} else expect(admittedNativeProviderPack(opened, identity)).toMatchObject({
|
||||
schema: "paperclip.native-provider-pack-admission.v1", stage: "session_admitted", authority: "core_verified_selection",
|
||||
companyId: identity.companyId, agentId: identity.agentId, runId: identity.runId, sessionId: identity.sessionId,
|
||||
runnerInstanceId: transport.prpIdentity!.runnerInstanceId, runnerEnvironmentLeaseId: transport.prpIdentity!.environmentLeaseId,
|
||||
providerPackDigest: installed.digest, opencodeProxySha256: installed.payload.artifacts.opencodeProxy.sha256,
|
||||
sourceRevision: installed.payload.runnerSourceRevision,
|
||||
});
|
||||
coldAdmission = admittedNativeProviderPack(opened, identity);
|
||||
}
|
||||
expect(execute.mock.calls.filter(([call]) => call.args?.[0] === "--build-metadata")).toHaveLength(1);
|
||||
expect(execute.mock.calls.some(([call]) => call.args?.[0] === "-e" || call.args?.[0] === "--version")).toBe(false);
|
||||
expect(syncIn.mock.calls.every(([operations]) => operations.every(operation => operation.files.every(file => file.kind === "file")))).toBe(true);
|
||||
@@ -1831,7 +1857,7 @@ describe("remote provider pack manifest", () => {
|
||||
: script.includes("cat --") ? `nonce\n123\n2026-09-19T10:00:00.000Z\n${identity.runnerInstanceId}\nlinux:abcd-1234:100\n` : "";
|
||||
return { stdout, stderr: "", exitCode: 0, timedOut: false };
|
||||
});
|
||||
await createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
|
||||
const recoveryInput: Parameters<typeof createRunnerdBackend>[0] = { db: leaseDb(remoteExecution), execution: remoteExecution,
|
||||
runnerInstanceId: identity.runnerInstanceId, runnerIngressAuthorized: true,
|
||||
...(opencode ? { runnerEnvironment: configuredEnvironmentProjection({ PAPERCLIP_OPENCODE_PERFORMANCE_DIAGNOSTICS: "1" }) } : {}),
|
||||
runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace",
|
||||
@@ -1840,10 +1866,58 @@ describe("remote provider pack manifest", () => {
|
||||
restartRecovery: { kind: "reattach_remote_runner", runId: remoteExecution.binding.runId,
|
||||
leaseOwner: "controller", controllerGeneration: 2, providerAttempt: 1, restartKind: "graceful", recoveryRequestId: null,
|
||||
remote: { providerLeaseId: "sandbox", remoteCwd: "/workspace" } },
|
||||
});
|
||||
};
|
||||
const recoveredBackend = await createRunnerdBackend(recoveryInput);
|
||||
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
|
||||
const recovered = state.createTransport.mock.calls.at(-1)![0];
|
||||
expect(recovered).toMatchObject({ providerPackAuthorityDigest: admitted.digest, runnerArtifact: artifact, adoptExistingRunner: { pid: 123 } });
|
||||
if (provider === "opencode") {
|
||||
const nativeIdentity = { ...remoteExecution.binding, sessionId: remoteExecution.session.normalizedSessionId! };
|
||||
const restored = { identity: () => nativeIdentity } as never;
|
||||
const checkpoint = { backendKind: "runner", identity: nativeIdentity, sessionId: nativeIdentity.sessionId } as never;
|
||||
Object.assign(state.createBackend.mock.results.at(-1)!.value, {
|
||||
recoverSession: async () => ({ recovered: true, session: restored }),
|
||||
});
|
||||
const result = await recoveredBackend.recoverSession!(checkpoint, { signal: new AbortController().signal });
|
||||
expect(result.recovered).toBe(true);
|
||||
expect(admittedNativeProviderPack(result.session!, nativeIdentity)).toBeNull();
|
||||
const legacyBackend = await createRunnerdBackend({ ...recoveryInput,
|
||||
controllerPersistence: { bootId: "replacement-controller", runnerArtifact: artifact } });
|
||||
const legacy = { identity: () => nativeIdentity } as never;
|
||||
Object.assign(state.createBackend.mock.results.at(-1)!.value, {
|
||||
recoverSession: async () => ({ recovered: true, session: legacy }),
|
||||
});
|
||||
const legacyResult = await legacyBackend.recoverSession!(checkpoint, { signal: new AbortController().signal });
|
||||
expect(legacyResult.recovered).toBe(true);
|
||||
expect(admittedNativeProviderPack(legacyResult.session!, nativeIdentity)).toBeNull();
|
||||
const withPrior = new Proxy(recoveryInput.db, {
|
||||
get(target, key, receiver) {
|
||||
if (key !== "select") return Reflect.get(target, key, receiver);
|
||||
return (fields?: Record<string, unknown>) => fields && "receipt" in fields
|
||||
? { from: () => ({ where: () => ({ limit: async () => [{ receipt: coldAdmission,
|
||||
state: { ...nativeProviderPackAdmissionPending(randomUUID()), state: "verified" } }] }) }) }
|
||||
: target.select(fields as never);
|
||||
},
|
||||
});
|
||||
for (const pinned of [false, true]) {
|
||||
const knownBackend = await createRunnerdBackend({ ...recoveryInput, db: withPrior,
|
||||
...(pinned ? { controllerPersistence: { bootId: "replacement-controller", runnerArtifact: artifact }, runnerRemoteBinaryPath: state.bundledRunnerBinary! } : {}) });
|
||||
const known = { identity: () => nativeIdentity } as never;
|
||||
Object.assign(state.createBackend.mock.results.at(-1)!.value, {
|
||||
recoverSession: async () => ({ recovered: true, session: known }),
|
||||
});
|
||||
const knownResult = await knownBackend.recoverSession!(checkpoint, { signal: new AbortController().signal });
|
||||
expect(knownResult.recovered).toBe(true);
|
||||
expect(admittedNativeProviderPack(knownResult.session!, nativeIdentity)).toEqual(coldAdmission);
|
||||
if (pinned) {
|
||||
const replacement = { identity: () => nativeIdentity } as never;
|
||||
Object.assign(state.createBackend.mock.results.at(-1)!.value, { openSession: async () => replacement });
|
||||
const replaced = await knownBackend.openReplacementSession!({ identity: nativeIdentity }, checkpoint);
|
||||
expect(replaced).toBe(replacement);
|
||||
expect(admittedNativeProviderPack(replaced, nativeIdentity)).toBeNull();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
state.bundledPackManifestPath = null; state.bundledRunnerBinary = null;
|
||||
@@ -8077,6 +8151,105 @@ describe("native session cancellation", () => {
|
||||
});
|
||||
|
||||
describe("native session execution lease fencing", () => {
|
||||
it("persists current admission from the executor callback and never writes for null publication", async () => {
|
||||
const identity = { runId: execution.binding.runId, companyId: execution.binding.companyId, agentId: execution.binding.agentId,
|
||||
issueId: execution.binding.issueId, sessionId: execution.session.normalizedSessionId! };
|
||||
const session = { identity: () => identity } as never;
|
||||
const artifact = { path: "/fixture/provider", sha256: "sha256:" + "a".repeat(64) };
|
||||
const payload = { runnerSourceRevision: "b".repeat(40), target: { platform: "linux", architecture: "x64" },
|
||||
pins: { opencode: "1.18.34" }, artifacts: { nodeCommand: artifact, opencodeCommand: artifact, opencodeProxy: artifact } };
|
||||
expect(rememberAdmittedNativeProviderPack(session, { identity, runnerInstanceId: "actual-runner", environmentLeaseId: "runner-scope",
|
||||
manifest: { schema: "paperclip-runner/remote-provider-pack/v1", payload,
|
||||
digest: "sha256:" + createHash("sha256").update(canonicalProviderPackJson(payload)).digest("hex") } })).toBe(true);
|
||||
const profile: Record<string, unknown> = { preserved: { value: "unchanged" } };
|
||||
const updates: Array<{ table: unknown; values: Record<string, unknown> }> = [];
|
||||
const recordAdmission = vi.spyOn(providerPackAdmission, "recordNativeProviderPackAdmission");
|
||||
let profileWrites = 0;
|
||||
const wrap = (source: Db): Db => new Proxy(source, { get(target, key, receiver) {
|
||||
if (key === "transaction") return (operation: (tx: Db) => Promise<unknown>) => target.transaction(tx => operation(wrap(tx as unknown as Db)) as never);
|
||||
if (key === "select") return (fields?: Record<string, unknown>) => {
|
||||
if (!fields || !("profile" in fields)) return target.select(fields as never);
|
||||
const query = { where: () => query, for: () => query, limit: async () => [{ profile }] };
|
||||
return { from: () => query };
|
||||
};
|
||||
if (key === "update") return (table: unknown) => ({ set: (values: Record<string, unknown>) => {
|
||||
const update = target.update(table as never).set(values);
|
||||
return { where: (condition: unknown) => {
|
||||
if (table === heartbeatRuns && values.runnerProfileJson) {
|
||||
const query = new PgDialect().sqlToQuery(values.runnerProfileJson as SQL);
|
||||
const patch = query.params.map(value => typeof value === "string" ? (() => { try { return JSON.parse(value); } catch { return null; } })() : null)
|
||||
.find(value => value?.nativeProviderPackAdmissionState);
|
||||
if (patch) { Object.assign(profile, patch); profileWrites++; }
|
||||
}
|
||||
return update.where(condition as never);
|
||||
} };
|
||||
} });
|
||||
return Reflect.get(target, key, receiver);
|
||||
} });
|
||||
const db = wrap(leaseDb(execution, {}, {}, updates, profile));
|
||||
let nullWrites = -1;
|
||||
state.execute.mockReset().mockImplementationOnce(async options => {
|
||||
await options.onSession(session);
|
||||
const before = profileWrites;
|
||||
await options.onSession(null);
|
||||
nullWrites = profileWrites - before;
|
||||
return { result: { summary: "cancelled" }, terminal: { runTerminalState: "cancelled" }, turnId: "turn",
|
||||
normalizedSessionId: identity.sessionId, providerSessionId: null, driverKind: "test", driverVersion: "1",
|
||||
nativeEventCount: 0, highestContiguousSourceSeq: 0, usage: null };
|
||||
});
|
||||
await executePaperclipNativeSession({ db, execution, runnerInstanceId: "actual-runner" });
|
||||
expect(profile.preserved).toEqual({ value: "unchanged" });
|
||||
expect(profile.nativeProviderPackAdmission).toEqual(admittedNativeProviderPack(session, identity));
|
||||
expect(profile.nativeProviderPackAdmissionState).toMatchObject({ state: "verified", admissionEpoch: expect.any(String) });
|
||||
expect(profileWrites).toBe(2);
|
||||
expect(nullWrites).toBe(0);
|
||||
const claim = updates.find(update => update.table === nativeRunFinalizations && update.values.phase === "observed" && update.values.leaseOwner)!.values;
|
||||
expect(recordAdmission).toHaveBeenCalledExactlyOnceWith(db, { session, identity,
|
||||
admissionEpoch: (profile.nativeProviderPackAdmissionState as Record<string, unknown>).admissionEpoch,
|
||||
claim: { leaseOwner: claim.leaseOwner, attempt: claim.attempt, controllerGeneration: claim.controllerGeneration, controllerBootId: claim.controllerBootId } });
|
||||
});
|
||||
it("atomically invalidates previous admission proof before provider admission", async () => {
|
||||
const updates: Array<{ table: unknown; values: Record<string, unknown> }> = [];
|
||||
let observed: Record<string, unknown> | null = null;
|
||||
const profile = { nativeProviderPackAdmission: { historical: "preserved" },
|
||||
nativeProviderPackAdmissionState: { ...nativeProviderPackAdmissionPending(randomUUID()), state: "verified" } };
|
||||
state.execute.mockReset().mockImplementationOnce(async () => {
|
||||
const invalidation = updates.find(update => update.table === heartbeatRuns && update.values.nativePhase === "observed")!;
|
||||
if (invalidation.values.runnerProfileJson) {
|
||||
const query = new PgDialect().sqlToQuery(invalidation.values.runnerProfileJson as SQL);
|
||||
observed = query.params.map(value => typeof value === "string" ? (() => { try { return JSON.parse(value); } catch { return null; } })() : null)
|
||||
.find(value => value?.nativeProviderPackAdmissionState);
|
||||
}
|
||||
return { result: { summary: "cancelled" }, terminal: { runTerminalState: "cancelled" }, turnId: "turn",
|
||||
normalizedSessionId: "session", providerSessionId: null, driverKind: "test", driverVersion: "1",
|
||||
nativeEventCount: 0, highestContiguousSourceSeq: 0, usage: null };
|
||||
});
|
||||
await executePaperclipNativeSession({ db: leaseDb(execution, {}, {}, updates, profile), execution, runnerInstanceId: "runner" });
|
||||
expect(state.execute).toHaveBeenCalledOnce();
|
||||
expect(observed).toEqual({ nativeProviderPackAdmissionState: { schema: "paperclip.native-provider-pack-admission-state.v1",
|
||||
state: "pending", admissionEpoch: expect.any(String) } });
|
||||
expect(JSON.stringify(observed)).not.toContain(profile.nativeProviderPackAdmissionState.admissionEpoch);
|
||||
expect(profile.nativeProviderPackAdmissionState.state).toBe("verified");
|
||||
});
|
||||
it("does not admit a session if mandatory claim invalidation fails", async () => {
|
||||
const source = leaseDb(execution);
|
||||
const failed = new Error("claim profile write failed");
|
||||
const db = new Proxy(source, { get(target, key, receiver) {
|
||||
if (key !== "transaction") return Reflect.get(target, key, receiver);
|
||||
return (operation: (tx: Db) => Promise<unknown>) => target.transaction(tx => operation(new Proxy(tx, { get(transaction, name, innerReceiver) {
|
||||
if (name !== "update") return Reflect.get(transaction, name, innerReceiver);
|
||||
return (table: unknown) => ({ set: (values: Record<string, unknown>) => {
|
||||
if (table === heartbeatRuns && values.nativePhase === "observed" && values.runnerProfileJson) throw failed;
|
||||
return transaction.update(table as never).set(values);
|
||||
} });
|
||||
} }) as unknown as Db) as never);
|
||||
} });
|
||||
state.execute.mockClear(); state.createBackend.mockClear(); state.createTransport.mockClear();
|
||||
await expect(executePaperclipNativeSession({ db, execution, runnerInstanceId: "runner" })).rejects.toBe(failed);
|
||||
expect(state.execute).not.toHaveBeenCalled();
|
||||
expect(state.createBackend).not.toHaveBeenCalled();
|
||||
expect(state.createTransport).not.toHaveBeenCalled();
|
||||
});
|
||||
it("renews only when the exact fenced owner remains current", async () => {
|
||||
const returning = vi
|
||||
.fn()
|
||||
|
||||
@@ -17,6 +17,7 @@ import { notifyUnsafeWorkspaceExportRecovery } from "../unsafe-workspace-export-
|
||||
import { notifyCancelledWorkspacePreservation } from "../cancelled-workspace-preservation-notifications.js";
|
||||
import { notifyInstructionCollection } from "../instruction-collection-notifications.js";
|
||||
import { providerPackWorkspaceId, recordAdmittedRemoteProviderPack, resolveRemoteProviderPackAuthority } from "./remote-provider-pack-authority.js";
|
||||
import { nativeProviderPackAdmissionPending, rememberAdmittedNativeProviderPack, rememberRecoveredNativeProviderPack, recordNativeProviderPackAdmission, type NativeProviderPackAdmissionSource } from "./native-provider-pack-admission.js";
|
||||
import { type ProviderPackInstallation, verifyRemoteProviderPack as verifyRemoteProviderPackIdentity } from "./remote-provider-pack-verification.js";
|
||||
import { writeEnvironmentCleanup } from "../environment-cleanup-notifications.js";
|
||||
import { NATIVE_SESSION_EXECUTION_LEASE_TTL_MS, startNativeSessionExecutionLeaseRenewal, type NativeExecutionLeaseRenewal } from "./native-execution-lease.js";
|
||||
@@ -8439,6 +8440,8 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
? REMOTE_CONTROLLER_LEASE_MS : NATIVE_SESSION_EXECUTION_LEASE_TTL_MS;
|
||||
let attempt: number;
|
||||
let controllerGeneration = 0;
|
||||
const providerPackAdmissionEpoch = randomUUID();
|
||||
let providerPackAdmissionSource: NativeProviderPackAdmissionSource | undefined;
|
||||
try {
|
||||
attempt = await trace.measure(
|
||||
"native.coordinator.claim",
|
||||
@@ -8480,6 +8483,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
resultJson: heartbeatRuns.resultJson,
|
||||
runtimeMode: heartbeatRuns.runtimeMode,
|
||||
status: heartbeatRuns.status,
|
||||
runnerProfileJson: heartbeatRuns.runnerProfileJson,
|
||||
})
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, input.execution.binding.runId))
|
||||
@@ -8619,12 +8623,16 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
.returning({ runId: nativeRunFinalizations.runId })
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!claimed) throw new Error("native_session_lease_lost");
|
||||
providerPackAdmissionSource = { runId: coordinator.runId,
|
||||
receipt: boundRun.runnerProfileJson?.nativeProviderPackAdmission,
|
||||
state: boundRun.runnerProfileJson?.nativeProviderPackAdmissionState };
|
||||
controllerGeneration = nextControllerGeneration;
|
||||
await tx
|
||||
.update(heartbeatRuns)
|
||||
.set({
|
||||
nativePhase: "observed",
|
||||
nativePhaseUpdatedAt: leaseNow,
|
||||
runnerProfileJson: sql`coalesce(${heartbeatRuns.runnerProfileJson}, '{}'::jsonb) || ${JSON.stringify({ nativeProviderPackAdmissionState: nativeProviderPackAdmissionPending(providerPackAdmissionEpoch) })}::jsonb`,
|
||||
updatedAt: leaseNow,
|
||||
})
|
||||
.where(eq(heartbeatRuns.id, coordinator.runId));
|
||||
@@ -9414,6 +9422,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
input.useRunnerd && input.backend === undefined
|
||||
? await timeController("backend_setup", () => createRunnerdBackend({
|
||||
...input,
|
||||
providerPackAdmissionSource,
|
||||
onAccountingEvent: producerUsage?.capture,
|
||||
controllerPersistence: input.runnerExecutionTarget?.kind === "remote" ? {
|
||||
bootId: controller.bootId,
|
||||
@@ -9461,7 +9470,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
persistWarmNativeCheckpoint(remoteWarmRecovery.previousExecution,
|
||||
nativeSessionConfigDigest(remoteWarmRecovery.previousExecution, "remote"), await recovered.session.snapshot());
|
||||
persistedWarmSession = loadWarmNativeCheckpoint(input.execution, warmConfigDigest!, "remote");
|
||||
runnerdBackend = await createRunnerdBackend({ ...input, onAccountingEvent: producerUsage?.capture, runnerInstanceId: effectiveRunnerInstanceId,
|
||||
runnerdBackend = await createRunnerdBackend({ ...input, providerPackAdmissionSource, onAccountingEvent: producerUsage?.capture, runnerInstanceId: effectiveRunnerInstanceId,
|
||||
controllerPersistence: input.runnerExecutionTarget?.kind === "remote" ? { bootId: controller.bootId } : undefined,
|
||||
durableEnvironmentLeaseId: durableRunnerBinding?.environmentLeaseId, trace, toolTrace });
|
||||
}
|
||||
@@ -9787,6 +9796,18 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
if (session.detachControllerForRestart) await detachActiveNativeSessionForRestart(active);
|
||||
throw new NativeControllerDetachedForRestartError();
|
||||
}
|
||||
try {
|
||||
await recordNativeProviderPackAdmission(input.db, {
|
||||
session,
|
||||
admissionEpoch: providerPackAdmissionEpoch,
|
||||
identity: { runId: input.execution.binding.runId, companyId: input.execution.binding.companyId,
|
||||
agentId: input.execution.binding.agentId, issueId: input.execution.binding.issueId,
|
||||
sessionId: nativeSessionKey(input.execution) },
|
||||
claim: { leaseOwner, attempt, controllerGeneration, controllerBootId: controller.bootId },
|
||||
});
|
||||
} catch {
|
||||
try { await input.onLog?.("stderr", "[paperclip-runner] native provider-pack admission receipt could not be saved\n"); } catch {}
|
||||
}
|
||||
if (session.resolveRuntimeRequest) await liveQuestions.attach();
|
||||
if (active.cancelRequested) {
|
||||
await active.cancellationSettled;
|
||||
@@ -12139,6 +12160,7 @@ export function remoteCheckpointIncompleteFailure(
|
||||
export async function createRunnerdBackend(input: {
|
||||
casesEnabled?: boolean;
|
||||
db: Db;
|
||||
providerPackAdmissionSource?: NativeProviderPackAdmissionSource;
|
||||
controllerPersistence?: { bootId: string; runnerArtifact?: { version: string; digest: string } };
|
||||
onAccountingEvent?: (event: PrpEvent) => Promise<void>;
|
||||
execution: NativeExecutionInput;
|
||||
@@ -14361,7 +14383,22 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
priorAuthorityEpoch.revoke();
|
||||
}
|
||||
sessionToolAuthorityEpochs.set(sessionScopeId, authorityEpoch);
|
||||
const recordAdmittedProviderPack = async () => {
|
||||
const recordAdmittedProviderPack = async (session: NativeSession, identity: Parameters<NativeSessionBackend["openSession"]>[0]["identity"], retained: boolean) => {
|
||||
if (retained && remoteTarget && requiresRemoteProviderPack) {
|
||||
const sourceRunId = input.remoteWarmRecovery?.identity.runId ?? durableIdentity?.runId;
|
||||
if (typeof sourceRunId === "string" && sourceRunId) {
|
||||
try {
|
||||
await rememberRecoveredNativeProviderPack(input.db, session, { identity, sourceRunId,
|
||||
runnerInstanceId: effectiveRunnerInstanceId, environmentLeaseId: effectiveEnvironmentLeaseId,
|
||||
source: input.providerPackAdmissionSource });
|
||||
} catch {
|
||||
try { await input.onLog?.("stderr", "[paperclip-runner] prior provider-pack admission receipt is unavailable\n"); } catch {}
|
||||
}
|
||||
}
|
||||
} else if (providerPackVerified && providerPackAuthority && expectedProviderPackManifest) {
|
||||
rememberAdmittedNativeProviderPack(session, { identity, manifest: expectedProviderPackManifest,
|
||||
runnerInstanceId: effectiveRunnerInstanceId, environmentLeaseId: effectiveEnvironmentLeaseId });
|
||||
}
|
||||
if (providerPackVerified && providerPackAuthority) {
|
||||
await recordAdmittedRemoteProviderPack(providerPackAuthority, message => input.onLog?.("stderr", message));
|
||||
}
|
||||
@@ -14380,13 +14417,15 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
descriptor: () => backend.descriptor(),
|
||||
openSession: async (sessionInput) => {
|
||||
const session = await backend.openSession(sessionInput);
|
||||
await recordAdmittedProviderPack();
|
||||
return wrapManagedSession(session);
|
||||
const managed = wrapManagedSession(session);
|
||||
await recordAdmittedProviderPack(managed, sessionInput.identity, Boolean(adoptedProcess));
|
||||
return managed;
|
||||
},
|
||||
recoverSession: async (snapshot, options) => {
|
||||
const result = backend.recoverSession ? await backend.recoverSession(snapshot, options) : { recovered: false, reason: "driver does not support recovery" };
|
||||
if (result.session) await recordAdmittedProviderPack();
|
||||
return result.session ? { ...result, session: wrapManagedSession(result.session) } : result;
|
||||
const session = result.session ? wrapManagedSession(result.session) : undefined;
|
||||
if (result.recovered && session) await recordAdmittedProviderPack(session, snapshot.identity, Boolean(adoptedProcess));
|
||||
return session ? { ...result, session } : result;
|
||||
},
|
||||
openReplacementSession: async (sessionInput) => {
|
||||
await measureNativeRunnerSpan(
|
||||
@@ -14396,8 +14435,9 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
{ parentName: "native.session.execute" },
|
||||
);
|
||||
const session = await backend.openSession(sessionInput);
|
||||
await recordAdmittedProviderPack();
|
||||
return wrapManagedSession(session);
|
||||
const managed = wrapManagedSession(session);
|
||||
await recordAdmittedProviderPack(managed, sessionInput.identity, false);
|
||||
return managed;
|
||||
},
|
||||
} satisfies ManagedRunnerdBackend;
|
||||
}
|
||||
|
||||
@@ -10,6 +10,37 @@ const fail = code => {
|
||||
throw new SetupFailure(match ? `${match[1]}_http` : code, match ? Number(match[2]) : null);
|
||||
};
|
||||
const bootstrapRequestClasses = new Set(['JOB_RUN_GET', 'LAUNCH_GET', 'JOB_GET']);
|
||||
const uploadFailureChecks = {
|
||||
context: ['selected_run', 'selected_attempt', 'selected_workflow_sha', 'selected_target_sha', 'selected_jobrun', 'env_jobrun', 'selected_execution',
|
||||
'selected_scenario', 'selected_project', 'selected_full_name', 'plan_id', 'job_id', 'case_id', 'launch_id', 'export_case',
|
||||
'export_campaign', 'export_count', 'export_status', 'export_cleanup', 'passed_cleanup', 'attachment_array', 'attachment_count'],
|
||||
attachment_contract: ['attachment_rows', 'provenance_count', 'record_count', 'passed_screenshot'],
|
||||
jobrun: ['jobrun_id', 'job_id', 'job_type', 'launch_id', 'run_url', 'stage'],
|
||||
launch: ['launch_id', 'project_id'],
|
||||
result_page: ['content_array', 'content_limit', 'total_integer', 'total_matches_content', 'pages_integer', 'pages_range'],
|
||||
result_row: ['result_id', 'case_id'],
|
||||
result: ['result_id', 'project_id', 'launch_id', 'case_id', 'full_name', 'jobrun_id', 'jobrun_launch', 'jobrun_url',
|
||||
'embedded_job', 'links_array', 'known_links'],
|
||||
result_status: ['status_supported', 'status_matches_export'],
|
||||
attachment_page: ['content_array', 'content_limit', 'total_integer', 'total_matches_content', 'pages_integer', 'pages_range',
|
||||
'unique_ids', 'attachment_rows'],
|
||||
};
|
||||
const uploadFailures = new WeakMap();
|
||||
function stopUpload(code, phase, readChecks) {
|
||||
const error = new SetupFailure(code);
|
||||
let values;
|
||||
try { values = readChecks(); } catch {}
|
||||
const names = uploadFailureChecks[phase].filter(name => Array.isArray(values?.[name]) && typeof values[name][0] === 'boolean');
|
||||
const checks = Object.fromEntries(names.map(name => [name, values[name][0]]));
|
||||
const types = Object.fromEntries(names.filter(name => values[name].length === 2).map(name => {
|
||||
const value = values[name][1], type = typeof value;
|
||||
return [name, value === undefined ? 'missing' : value === null ? 'null' : Array.isArray(value) ? 'array'
|
||||
: ['string', 'number', 'boolean', 'object'].includes(type) ? type : 'other'];
|
||||
}));
|
||||
const failedPredicates = Object.keys(checks).filter(name => !checks[name]);
|
||||
uploadFailures.set(error, { phase, firstFailedPredicate: failedPredicates[0] ?? 'checks_unavailable', failedPredicates, checks, types });
|
||||
throw error;
|
||||
}
|
||||
const diagnosticCodes = new Set(['untrusted_chat_testops_context', 'testops_token_missing', 'testops_auth_http',
|
||||
'testops_api_http', 'testops_auth_invalid', 'testops_route_invalid', 'testops_response_too_large',
|
||||
'testops_response_invalid', 'testops_metadata_page_incomplete', 'testops_case_identity_invalid',
|
||||
@@ -22,10 +53,13 @@ const diagnosticCodes = new Set(['untrusted_chat_testops_context', 'testops_toke
|
||||
'chat_testops_job_parameters_limit', 'chat_testops_job_url_mismatch', 'chat_testops_jobrun_mismatch', 'chat_testops_launch_mismatch', 'chat_testops_selection_mismatch', 'chat_testops_parameter_mismatch', 'chat_testops_nonce_invalid', 'chat_testops_nonce_conflict', 'chat_testops_catalog_incomplete', 'chat_testops_launch_admission', 'chat_testops_github_http', 'chat_testops_integration_mismatch', 'chat_testops_upload_mismatch', 'chat_testops_upload_unverified']);
|
||||
export function failureProjection(error, stage) {
|
||||
const request = error instanceof SetupFailure ? error.request : undefined;
|
||||
const verification = error instanceof SetupFailure ? uploadFailures.get(error) : undefined;
|
||||
const stages = ['context', 'output_admission', 'authentication', 'metadata_inspection', 'plan_bootstrap', 'selection_validation', 'launch_admission', 'launch_preflight', 'plan_launch', 'bootstrap_job_metadata', 'result_export', 'upload_verification', 'bootstrap_context', 'receipt'];
|
||||
return { stage: stages.includes(stage) ? stage : 'unknown',
|
||||
code: error instanceof SetupFailure && diagnosticCodes.has(error.code) ? error.code : 'unknown_failure',
|
||||
httpStatus: error instanceof SetupFailure && Number.isInteger(error.status) && error.status >= 100 && error.status < 600 ? error.status : null,
|
||||
...(verification ? { verification: { ...verification, failedPredicates: [...verification.failedPredicates],
|
||||
checks: { ...verification.checks }, types: { ...verification.types } } } : {}),
|
||||
...(request && bootstrapRequestClasses.has(request.class) && id(request.resourceId) && ['STARTED', 'SUCCEEDED', 'FAILED'].includes(request.state)
|
||||
? { request: { class: request.class, resourceId: request.resourceId, state: request.state,
|
||||
httpStatus: Number.isInteger(request.httpStatus) && request.httpStatus >= 100 && request.httpStatus < 600 ? request.httpStatus : null } } : {}) };
|
||||
@@ -345,14 +379,36 @@ export async function verifyChatUpload(api, env, selected, exported, { wait = ms
|
||||
|| exported.caseId !== selected.caseId || exported.campaignId !== `gha-${context.runId}-1` || exported.count !== 1
|
||||
|| !['passed', 'failed'].includes(exported.status) || !['passed', 'failed', 'not_started'].includes(exported.cleanup)
|
||||
|| exported.status === 'passed' && exported.cleanup !== 'passed' || !Array.isArray(exported.attachmentMetadata)
|
||||
|| exported.attachmentMetadata.length < 2 || exported.attachmentMetadata.length > 10) fail('chat_testops_upload_mismatch');
|
||||
|| exported.attachmentMetadata.length < 2 || exported.attachmentMetadata.length > 10) stopUpload('chat_testops_upload_mismatch', 'context', () => ({
|
||||
selected_run: [selected.runId === context.runId, selected.runId], selected_attempt: [selected.attempt === '1', selected.attempt],
|
||||
selected_workflow_sha: [selected.workflowSha === context.workflowSha, selected.workflowSha],
|
||||
selected_target_sha: [selected.targetSha === context.targetSha, selected.targetSha],
|
||||
selected_jobrun: [selected.allureJobRunId === env.ALLURE_JOB_RUN_ID, selected.allureJobRunId],
|
||||
env_jobrun: [selected.allureJobRunId === env.ALLURE_JOB_RUN_ID, env.ALLURE_JOB_RUN_ID],
|
||||
selected_execution: [selected.executionId === executionId, selected.executionId], selected_scenario: [selected.canonicalScenario === scenario, selected.canonicalScenario],
|
||||
selected_project: [selected.projectId === 1, selected.projectId], selected_full_name: [aliases.includes(selected.fullName), selected.fullName],
|
||||
plan_id: [id(selected.planId), selected.planId], job_id: [id(selected.jobId), selected.jobId],
|
||||
case_id: [id(selected.caseId), selected.caseId], launch_id: [id(selected.launchId), selected.launchId],
|
||||
export_case: [exported.caseId === selected.caseId, exported.caseId], export_campaign: [exported.campaignId === `gha-${context.runId}-1`, exported.campaignId],
|
||||
export_count: [exported.count === 1, exported.count], export_status: [['passed', 'failed'].includes(exported.status), exported.status],
|
||||
export_cleanup: [['passed', 'failed', 'not_started'].includes(exported.cleanup), exported.cleanup],
|
||||
passed_cleanup: [!(exported.status === 'passed' && exported.cleanup !== 'passed')],
|
||||
attachment_array: [Array.isArray(exported.attachmentMetadata), exported.attachmentMetadata],
|
||||
attachment_count: [Array.isArray(exported.attachmentMetadata) && exported.attachmentMetadata.length >= 2 && exported.attachmentMetadata.length <= 10],
|
||||
}));
|
||||
const allowedAttachments = new Map([['Synthetic task and source conversation', 'image/png'],
|
||||
['Execution provenance and checks', 'application/json'], ['Sanitized Product E2E record and accounting', 'application/json']]);
|
||||
if (exported.attachmentMetadata.some(row => allowedAttachments.get(row.name) !== row.contentType
|
||||
|| !Number.isSafeInteger(row.contentLength) || row.contentLength <= 0 || row.contentLength > 12 * 1024 * 1024)
|
||||
|| exported.attachmentMetadata.filter(row => row.name === 'Execution provenance and checks').length !== 1
|
||||
|| exported.attachmentMetadata.filter(row => row.name === 'Sanitized Product E2E record and accounting').length !== 1
|
||||
|| exported.status === 'passed' && !exported.attachmentMetadata.some(row => row.contentType === 'image/png')) fail('chat_testops_upload_mismatch');
|
||||
|| exported.status === 'passed' && !exported.attachmentMetadata.some(row => row.contentType === 'image/png')) stopUpload('chat_testops_upload_mismatch', 'attachment_contract', () => ({
|
||||
attachment_rows: [!exported.attachmentMetadata.some(row => allowedAttachments.get(row?.name) !== row?.contentType
|
||||
|| !Number.isSafeInteger(row?.contentLength) || row?.contentLength <= 0 || row?.contentLength > 12 * 1024 * 1024)],
|
||||
provenance_count: [exported.attachmentMetadata.filter(row => row?.name === 'Execution provenance and checks').length === 1],
|
||||
record_count: [exported.attachmentMetadata.filter(row => row?.name === 'Sanitized Product E2E record and accounting').length === 1],
|
||||
passed_screenshot: [!(exported.status === 'passed' && !exported.attachmentMetadata.some(row => row?.contentType === 'image/png'))],
|
||||
}));
|
||||
const signature = rows => rows.map(row => JSON.stringify([row.name, row.contentType, row.contentLength])).sort();
|
||||
const runUrl = `https://github.com/paperclipai/paperclip/actions/runs/${context.runId}`;
|
||||
const sourceUrl = `https://github.com/paperclipai/paperclip/commit/${context.targetSha}`;
|
||||
@@ -367,15 +423,29 @@ export async function verifyChatUpload(api, env, selected, exported, { wait = ms
|
||||
for (let poll = 0; poll < 12 && now() < deadline; poll++) {
|
||||
const run = await get(`/api/jobrun/${selected.allureJobRunId}`);
|
||||
if (run.id !== Number(selected.allureJobRunId) || run.job?.id !== selected.jobId || run.job?.type !== 'github'
|
||||
|| run.launchId !== selected.launchId || run.url !== runUrl || !['in_progress', 'finished'].includes(run.stage)) fail('chat_testops_jobrun_mismatch');
|
||||
|| run.launchId !== selected.launchId || run.url !== runUrl || !['in_progress', 'finished'].includes(run.stage)) stopUpload('chat_testops_jobrun_mismatch', 'jobrun', () => ({
|
||||
jobrun_id: [run.id === Number(selected.allureJobRunId), run.id], job_id: [run.job?.id === selected.jobId, run.job?.id],
|
||||
job_type: [run.job?.type === 'github', run.job?.type], launch_id: [run.launchId === selected.launchId, run.launchId],
|
||||
run_url: [run.url === runUrl, run.url], stage: [['in_progress', 'finished'].includes(run.stage), run.stage],
|
||||
}));
|
||||
const launch = await get(`/api/launch/${selected.launchId}`);
|
||||
if (launch.id !== selected.launchId || launch.projectId !== 1) fail('chat_testops_launch_mismatch');
|
||||
if (launch.id !== selected.launchId || launch.projectId !== 1) stopUpload('chat_testops_launch_mismatch', 'launch', () => ({
|
||||
launch_id: [launch.id === selected.launchId, launch.id], project_id: [launch.projectId === 1, launch.projectId],
|
||||
}));
|
||||
const listed = await get(`/api/testresult?${new URLSearchParams({ launchId: String(selected.launchId), page: '0', size: '100' })}`);
|
||||
if (!Array.isArray(listed?.content) || listed.content.length > 1 || !Number.isSafeInteger(listed.totalElements)
|
||||
|| listed.totalElements !== listed.content.length || !Number.isSafeInteger(listed.totalPages) || listed.totalPages < 0 || listed.totalPages > 1) fail('chat_testops_upload_mismatch');
|
||||
|| listed.totalElements !== listed.content.length || !Number.isSafeInteger(listed.totalPages) || listed.totalPages < 0 || listed.totalPages > 1) stopUpload('chat_testops_upload_mismatch', 'result_page', () => ({
|
||||
content_array: [Array.isArray(listed?.content), listed?.content], content_limit: [Array.isArray(listed?.content) && listed.content.length <= 1],
|
||||
total_integer: [Number.isSafeInteger(listed?.totalElements), listed?.totalElements],
|
||||
total_matches_content: [Array.isArray(listed?.content) && listed.totalElements === listed.content.length],
|
||||
pages_integer: [Number.isSafeInteger(listed?.totalPages), listed?.totalPages],
|
||||
pages_range: [Number.isSafeInteger(listed?.totalPages) && listed.totalPages >= 0 && listed.totalPages <= 1],
|
||||
}));
|
||||
if (listed.content.length) {
|
||||
const row = listed.content[0];
|
||||
if (!id(row.id) || row.testCaseId !== selected.caseId) fail('chat_testops_upload_mismatch');
|
||||
if (!id(row.id) || row.testCaseId !== selected.caseId) stopUpload('chat_testops_upload_mismatch', 'result_row', () => ({
|
||||
result_id: [id(row.id), row.id], case_id: [row.testCaseId === selected.caseId, row.testCaseId],
|
||||
}));
|
||||
const result = await get(`/api/testresult/${row.id}`);
|
||||
if (result.id !== row.id || result.projectId !== 1 || result.launchId !== selected.launchId || result.testCaseId !== selected.caseId
|
||||
|| result.fullName !== selected.fullName || result.jobRun?.id !== Number(selected.allureJobRunId)
|
||||
@@ -385,14 +455,37 @@ export async function verifyChatUpload(api, env, selected, exported, { wait = ms
|
||||
|| result.jobRun.job.url != null && !publicJobUrl(result.jobRun.job.url, env))
|
||||
|| !Array.isArray(result.links) || result.links.some(link => typeof link.url === 'string'
|
||||
&& (link.url.startsWith('https://github.com/paperclipai/paperclip/actions/runs/') && link.url !== runUrl
|
||||
|| link.url.startsWith('https://github.com/paperclipai/paperclip/commit/') && link.url !== sourceUrl))) fail('chat_testops_upload_mismatch');
|
||||
if (!['passed', 'failed', 'broken', 'skipped', 'unknown'].includes(result.status)) fail('chat_testops_upload_mismatch');
|
||||
if (result.status !== 'unknown' && result.status !== exported.status) fail('chat_testops_upload_mismatch');
|
||||
|| link.url.startsWith('https://github.com/paperclipai/paperclip/commit/') && link.url !== sourceUrl))) stopUpload('chat_testops_upload_mismatch', 'result', () => ({
|
||||
result_id: [result.id === row.id, result.id], project_id: [result.projectId === 1, result.projectId],
|
||||
launch_id: [result.launchId === selected.launchId, result.launchId], case_id: [result.testCaseId === selected.caseId, result.testCaseId],
|
||||
full_name: [result.fullName === selected.fullName, result.fullName], jobrun_id: [result.jobRun?.id === Number(selected.allureJobRunId), result.jobRun?.id],
|
||||
jobrun_launch: [result.jobRun?.launchId === selected.launchId, result.jobRun?.launchId], jobrun_url: [result.jobRun?.url === runUrl, result.jobRun?.url],
|
||||
embedded_job: [!(result.jobRun?.job != null && (result.jobRun.job.id !== run.job.id
|
||||
|| result.jobRun.job.type != null && result.jobRun.job.type !== run.job.type
|
||||
|| result.jobRun.job.url != null && !publicJobUrl(result.jobRun.job.url, env))), result.jobRun?.job],
|
||||
links_array: [Array.isArray(result.links), result.links], known_links: [Array.isArray(result.links) && !result.links.some(link => typeof link?.url === 'string'
|
||||
&& (link?.url.startsWith('https://github.com/paperclipai/paperclip/actions/runs/') && link?.url !== runUrl
|
||||
|| link?.url.startsWith('https://github.com/paperclipai/paperclip/commit/') && link?.url !== sourceUrl))],
|
||||
}));
|
||||
if (!['passed', 'failed', 'broken', 'skipped', 'unknown'].includes(result.status)) stopUpload('chat_testops_upload_mismatch', 'result_status', () => ({
|
||||
status_supported: [['passed', 'failed', 'broken', 'skipped', 'unknown'].includes(result.status), result.status],
|
||||
}));
|
||||
if (result.status !== 'unknown' && result.status !== exported.status) stopUpload('chat_testops_upload_mismatch', 'result_status', () => ({
|
||||
status_matches_export: [result.status === 'unknown' || result.status === exported.status, result.status],
|
||||
}));
|
||||
const attached = await get(`/api/testresult/attachment?${new URLSearchParams({ testResultId: String(row.id), page: '0', size: '100' })}`);
|
||||
if (!Array.isArray(attached?.content) || attached.content.length > 10 || !Number.isSafeInteger(attached.totalElements)
|
||||
|| attached.totalElements !== attached.content.length || !Number.isSafeInteger(attached.totalPages) || attached.totalPages < 0 || attached.totalPages > 1
|
||||
|| new Set(attached.content.map(value => value.id)).size !== attached.content.length
|
||||
|| attached.content.some(value => !id(value.id) || value.missed === true || value.fromTestCase === true)) fail('chat_testops_upload_mismatch');
|
||||
|| attached.content.some(value => !id(value.id) || value.missed === true || value.fromTestCase === true)) stopUpload('chat_testops_upload_mismatch', 'attachment_page', () => ({
|
||||
content_array: [Array.isArray(attached?.content), attached?.content], content_limit: [Array.isArray(attached?.content) && attached.content.length <= 10],
|
||||
total_integer: [Number.isSafeInteger(attached?.totalElements), attached?.totalElements],
|
||||
total_matches_content: [Array.isArray(attached?.content) && attached.totalElements === attached.content.length],
|
||||
pages_integer: [Number.isSafeInteger(attached?.totalPages), attached?.totalPages],
|
||||
pages_range: [Number.isSafeInteger(attached?.totalPages) && attached.totalPages >= 0 && attached.totalPages <= 1],
|
||||
unique_ids: [Array.isArray(attached?.content) && new Set(attached.content.map(value => value?.id)).size === attached.content.length],
|
||||
attachment_rows: [Array.isArray(attached?.content) && !attached.content.some(value => !id(value?.id) || value?.missed === true || value?.fromTestCase === true)],
|
||||
}));
|
||||
if (now() < deadline && result.status === exported.status && same(signature(attached.content), signature(exported.attachmentMetadata))) {
|
||||
return { schema: 'paperclip.chat-feature-testops-upload-verification/v1', ...context, projectId: 1, planId: selected.planId,
|
||||
jobId: selected.jobId, allureJobRunId: selected.allureJobRunId, launchId: selected.launchId, caseId: selected.caseId,
|
||||
|
||||
@@ -387,6 +387,110 @@ test('readback rejects authoritative or embedded identity contradictions and a f
|
||||
}
|
||||
});
|
||||
|
||||
async function uploadFailureDiagnostic(api, context = uploadContext, selection = uploadSelection, receipt = uploadReceipt) {
|
||||
const { verifyChatUpload } = await import('./chat-feature-testops-api.mjs');
|
||||
let error;
|
||||
await assert.rejects(verifyChatUpload(api, context, selection, receipt), value => { error = value; return true; });
|
||||
return { error, projected: failureProjection(error, 'upload_verification') };
|
||||
}
|
||||
|
||||
test('upload diagnostics identify selected, environment and exported context without an HTTP call', async () => {
|
||||
const noHttp = () => assert.fail('context rejection must precede HTTP');
|
||||
const cases = [
|
||||
[{}, { ...uploadSelection, targetSha: 'b'.repeat(40) }, uploadReceipt, 'selected_target_sha', 'string'],
|
||||
[{ ALLURE_JOB_RUN_ID: undefined }, uploadSelection, uploadReceipt, 'selected_jobrun', 'string'],
|
||||
[{}, uploadSelection, { ...uploadReceipt, count: 'private-count' }, 'export_count', 'string'],
|
||||
];
|
||||
for (const [envChange, selected, receipt, first, type] of cases) {
|
||||
const { projected } = await uploadFailureDiagnostic(noHttp, { ...uploadContext, ...envChange }, selected, receipt);
|
||||
assert.equal(projected.code, 'chat_testops_upload_mismatch');
|
||||
assert.equal(projected.verification.phase, 'context');
|
||||
assert.equal(projected.verification.firstFailedPredicate, first);
|
||||
assert.equal(projected.verification.checks[first], false);
|
||||
assert.equal(projected.verification.types[first], type);
|
||||
assert.ok(!JSON.stringify(projected).includes('private-count'));
|
||||
if (first === 'selected_jobrun') assert.equal(projected.verification.types.env_jobrun, 'missing');
|
||||
}
|
||||
});
|
||||
|
||||
test('upload diagnostics distinguish absent page and row fields from contradictory numeric identities', async () => {
|
||||
const base = uploadApi();
|
||||
for (const [response, phase, first, type] of [
|
||||
[{ content: [], totalPages: 0 }, 'result_page', 'total_integer', 'missing'],
|
||||
[{ content: [], totalElements: 0, totalPages: 'private-page' }, 'result_page', 'pages_integer', 'string'],
|
||||
[{ content: [{ id: 45 }], totalElements: 1, totalPages: 1 }, 'result_row', 'case_id', 'missing'],
|
||||
[{ content: [{ id: 45, testCaseId: 92 }], totalElements: 1, totalPages: 1 }, 'result_row', 'case_id', 'number'],
|
||||
]) {
|
||||
const { projected } = await uploadFailureDiagnostic((route, ...args) => route.startsWith('/api/testresult?') ? response : base(route, ...args));
|
||||
assert.equal(projected.verification.phase, phase); assert.equal(projected.verification.firstFailedPredicate, first);
|
||||
assert.equal(projected.verification.types[first], type); assert.equal(projected.verification.checks[first], false);
|
||||
assert.ok(!JSON.stringify(projected).includes('private-page'));
|
||||
}
|
||||
});
|
||||
|
||||
test('upload diagnostics retain exact authoritative and result rejection predicates', async () => {
|
||||
const base = uploadApi();
|
||||
const cases = [
|
||||
['/api/jobrun/9', { ...uploadRun, job: { type: 'github' } }, 'jobrun', 'job_id', 'missing'],
|
||||
['/api/jobrun/9', { ...uploadRun, job: { id: 8, type: 'github' } }, 'jobrun', 'job_id', 'number'],
|
||||
['/api/launch/10', { id: 10, projectId: 2 }, 'launch', 'project_id', 'number'],
|
||||
];
|
||||
for (const [target, response, phase, first, type] of cases) {
|
||||
const { projected } = await uploadFailureDiagnostic((route, ...args) => route === target ? response : base(route, ...args));
|
||||
assert.equal(projected.verification.phase, phase); assert.equal(projected.verification.firstFailedPredicate, first);
|
||||
assert.equal(projected.verification.types[first], type);
|
||||
}
|
||||
const result = await uploadFailureDiagnostic(uploadApi({ status: 'private-status' }));
|
||||
assert.equal(result.projected.verification.phase, 'result_status');
|
||||
assert.equal(result.projected.verification.firstFailedPredicate, 'status_supported');
|
||||
assert.ok(!JSON.stringify(result.projected).includes('private-status'));
|
||||
});
|
||||
|
||||
test('upload diagnostics identify local attachment and backend attachment-page failures', async () => {
|
||||
const local = await uploadFailureDiagnostic(() => assert.fail('attachment contract precedes HTTP'), uploadContext, uploadSelection,
|
||||
{ ...uploadReceipt, attachmentMetadata: uploadReceipt.attachmentMetadata.map((row, index) => index === 0 ? { ...row, contentType: 'private-type' } : row) });
|
||||
assert.equal(local.projected.verification.phase, 'attachment_contract');
|
||||
assert.equal(local.projected.verification.firstFailedPredicate, 'attachment_rows');
|
||||
assert.ok(!JSON.stringify(local.projected).includes('private-type'));
|
||||
const base = uploadApi();
|
||||
const page = await uploadFailureDiagnostic((route, ...args) => route.startsWith('/api/testresult/attachment?')
|
||||
? { content: null, totalElements: 0, totalPages: 0 } : base(route, ...args));
|
||||
assert.equal(page.projected.verification.phase, 'attachment_page');
|
||||
assert.equal(page.projected.verification.firstFailedPredicate, 'content_array');
|
||||
assert.equal(page.projected.verification.types.content_array, 'null');
|
||||
});
|
||||
|
||||
test('upload diagnostic projection never retains secret values, unknown fields or mutable aliases', async () => {
|
||||
const secret = 'DO_NOT_RETAIN_SYNTHETIC_CREDENTIAL_OR_PRIVATE_URL';
|
||||
const { error, projected } = await uploadFailureDiagnostic(uploadApi({ fullName: secret, unknown: secret }),
|
||||
{ ...uploadContext, ALLURE_TOKEN: secret });
|
||||
assert.equal(projected.verification.phase, 'result');
|
||||
assert.equal(projected.verification.firstFailedPredicate, 'full_name');
|
||||
assert.equal(projected.verification.types.full_name, 'string');
|
||||
assert.ok(!JSON.stringify(projected).includes(secret));
|
||||
projected.verification.phase = secret; projected.verification.failedPredicates.push(secret);
|
||||
projected.verification.checks.full_name = secret; projected.verification.types.full_name = secret;
|
||||
const again = failureProjection(error, 'upload_verification');
|
||||
assert.equal(again.verification.phase, 'result'); assert.equal(again.verification.checks.full_name, false);
|
||||
assert.ok(!JSON.stringify(again).includes(secret));
|
||||
const unknown = failureProjection(Object.assign(new Error(secret), { verification: { phase: secret } }), 'upload_verification');
|
||||
assert.equal(unknown.code, 'unknown_failure'); assert.ok(!Object.hasOwn(unknown, 'verification'));
|
||||
});
|
||||
|
||||
test('upload diagnostics preserve an early mismatch when later array entries are null', async () => {
|
||||
const result = await uploadFailureDiagnostic(uploadApi({ id: 46, links: [null] }));
|
||||
assert.equal(result.projected.verification.phase, 'result');
|
||||
assert.equal(result.projected.verification.firstFailedPredicate, 'result_id');
|
||||
assert.equal(result.projected.verification.checks.result_id, false);
|
||||
const base = uploadApi();
|
||||
const page = await uploadFailureDiagnostic((route, ...args) => route.startsWith('/api/testresult/attachment?')
|
||||
? { content: [null], totalElements: 2, totalPages: 1 } : base(route, ...args));
|
||||
assert.equal(page.projected.verification.firstFailedPredicate, 'total_matches_content');
|
||||
const local = await uploadFailureDiagnostic(() => assert.fail('no HTTP'), uploadContext, uploadSelection,
|
||||
{ ...uploadReceipt, attachmentMetadata: [{ ...uploadReceipt.attachmentMetadata[0], contentType: 'unexpected' }, null, uploadReceipt.attachmentMetadata[2]] });
|
||||
assert.equal(local.projected.verification.firstFailedPredicate, 'attachment_rows');
|
||||
});
|
||||
|
||||
test('readback missing result or screenshot metadata is bounded unverified, never a fabricated pass', async () => {
|
||||
const { verifyChatUpload } = await import('./chat-feature-testops-api.mjs');
|
||||
let waits = 0;
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import { apiErrorMessage, parseRetryAfter } from "./errors";
|
||||
|
||||
export interface ApiErrorOptions {
|
||||
/** Raw server error code; defaults to `body.error` when that is a string. */
|
||||
code?: string | null;
|
||||
/** Server-requested delay from the `Retry-After` header, in milliseconds. */
|
||||
retryAfterMs?: number | null;
|
||||
}
|
||||
|
||||
export class ApiError extends Error {
|
||||
status: number;
|
||||
body: unknown;
|
||||
/**
|
||||
* The raw `body.error` string (for example `tenant_app_unavailable`). Match
|
||||
* on this, never on `message`: `message` is readable copy for people and may
|
||||
* differ from the code (see `apiErrorMessage` in `api/errors.ts`).
|
||||
*/
|
||||
code: string | null;
|
||||
retryAfterMs: number | null;
|
||||
|
||||
constructor(message: string, status: number, body: unknown, options: ApiErrorOptions = {}) {
|
||||
super(message);
|
||||
this.name = "ApiError";
|
||||
this.status = status;
|
||||
this.body = body;
|
||||
const rawError = body && typeof body === "object" ? (body as { error?: unknown }).error : undefined;
|
||||
this.code = options.code !== undefined ? options.code : typeof rawError === "string" ? rawError : null;
|
||||
this.retryAfterMs = options.retryAfterMs ?? null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Build the `ApiError` for a non-OK response whose body has been read. */
|
||||
export function apiErrorFromResponse(res: Response, body: unknown): ApiError {
|
||||
return new ApiError(apiErrorMessage(res.status, body), res.status, body, {
|
||||
retryAfterMs: parseRetryAfter(res.headers?.get("Retry-After") ?? null),
|
||||
});
|
||||
}
|
||||
+2
-37
@@ -1,46 +1,11 @@
|
||||
import { getPageVisibility, getVisibilityHeaderValue } from "@/lib/page-visibility";
|
||||
import { tenantSessionRecovery } from "@/lib/tenant-session-recovery";
|
||||
import { apiErrorMessage, parseRetryAfter } from "./errors";
|
||||
import { ApiError, apiErrorFromResponse } from "./api-error";
|
||||
export { ApiError, apiErrorFromResponse, type ApiErrorOptions } from "./api-error";
|
||||
import { readApiJson } from "./response";
|
||||
|
||||
const BASE = "/api";
|
||||
|
||||
export interface ApiErrorOptions {
|
||||
/** Raw server error code; defaults to `body.error` when that is a string. */
|
||||
code?: string | null;
|
||||
/** Server-requested delay from the `Retry-After` header, in milliseconds. */
|
||||
retryAfterMs?: number | null;
|
||||
}
|
||||
|
||||
export class ApiError extends Error {
|
||||
status: number;
|
||||
body: unknown;
|
||||
/**
|
||||
* The raw `body.error` string (for example `tenant_app_unavailable`). Match
|
||||
* on this, never on `message`: `message` is readable copy for people and may
|
||||
* differ from the code (see `apiErrorMessage` in `api/errors.ts`).
|
||||
*/
|
||||
code: string | null;
|
||||
retryAfterMs: number | null;
|
||||
|
||||
constructor(message: string, status: number, body: unknown, options: ApiErrorOptions = {}) {
|
||||
super(message);
|
||||
this.name = "ApiError";
|
||||
this.status = status;
|
||||
this.body = body;
|
||||
const rawError = body && typeof body === "object" ? (body as { error?: unknown }).error : undefined;
|
||||
this.code = options.code !== undefined ? options.code : typeof rawError === "string" ? rawError : null;
|
||||
this.retryAfterMs = options.retryAfterMs ?? null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Build the `ApiError` for a non-OK response whose body has been read. */
|
||||
export function apiErrorFromResponse(res: Response, body: unknown): ApiError {
|
||||
return new ApiError(apiErrorMessage(res.status, body), res.status, body, {
|
||||
retryAfterMs: parseRetryAfter(res.headers?.get("Retry-After") ?? null),
|
||||
});
|
||||
}
|
||||
|
||||
// --- Dev/QA outage simulator ----------------------------------------------------
|
||||
//
|
||||
// Reproduces a deploy blip without a deploy. Dev and QA builds only:
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { ApiError } from "./client";
|
||||
import { ApiError } from "./api-error";
|
||||
import {
|
||||
apiErrorMessage,
|
||||
classifyError,
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { renderToStaticMarkup } from "react-dom/server";
|
||||
import { ThemeProvider } from "../context/ThemeContext";
|
||||
import { RunInvocationCard } from "../pages/AgentDetail";
|
||||
import { RunInvocationCard } from "./run-log/RunDetails";
|
||||
|
||||
describe("RunInvocationCard", () => {
|
||||
it("keeps verbose invocation details collapsed by default", () => {
|
||||
|
||||
@@ -0,0 +1,646 @@
|
||||
import { TranscriptSequence } from "../../lib/transcript-sequence";
|
||||
import { readTranscriptRequest } from "../transcript/read-transcript-request";
|
||||
import { useTranscriptLiveStream } from "../../context/LiveUpdatesProvider";
|
||||
import { liveRunEvent, mergeRunEvents } from "../../lib/live-run-event";
|
||||
import { mergeRunLogChunks, readChunkSeq } from "../../lib/run-log-chunks";
|
||||
import { usePageVisibility } from "../../lib/page-visibility";
|
||||
import { useCallback, useEffect, useMemo, useState, useRef } from "react";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { heartbeatsApi } from "../../api/heartbeats";
|
||||
import { instanceSettingsApi } from "../../api/instanceSettings";
|
||||
import { ApiError } from "../../api/api-error";
|
||||
import { queryKeys } from "../../lib/queryKeys";
|
||||
import { toolsApi } from "../../api/tools";
|
||||
import { getUIAdapter, buildTranscript, onAdapterChange } from "../../adapters";
|
||||
import { cn } from "../../lib/utils";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { RunTranscriptView, type TranscriptMode } from "../transcript/RunTranscriptView";
|
||||
import { LIVE_TRANSCRIPT_RENDER_LIMIT, MAX_LIVE_EVENTS, MAX_LIVE_LOG_LINES } from "../../lib/live-log-buffer";
|
||||
import { type HeartbeatRun, type HeartbeatRunEvent } from "@paperclipai/shared";
|
||||
import {
|
||||
type RunLogChunk,
|
||||
type ScrollContainer,
|
||||
shouldPollRunShellLog,
|
||||
asRecord,
|
||||
asNonEmptyString,
|
||||
buildHeartbeatProgressLogLine,
|
||||
heartbeatProgressLogLineKey,
|
||||
findScrollContainer,
|
||||
readScrollMetrics,
|
||||
LIVE_SCROLL_BOTTOM_TOLERANCE_PX,
|
||||
scrollToContainerBottom,
|
||||
RUN_LOG_PAGE_BYTES,
|
||||
redactPathValue,
|
||||
WorkspaceOperationsSection,
|
||||
RunInvocationCard,
|
||||
redactPathText,
|
||||
} from "./RunDetails";
|
||||
|
||||
export function LogViewer({ run, adapterType }: { run: HeartbeatRun; adapterType: string }) {
|
||||
const { visible } = usePageVisibility();
|
||||
const [events, setEvents] = useState<HeartbeatRunEvent[]>([]);
|
||||
const [hasMoreEvents, setHasMoreEvents] = useState(false);
|
||||
const [eventsError, setEventsError] = useState<string | null>(null);
|
||||
const hasMoreEventsRef = useRef(false);
|
||||
const [loadingMoreEvents, setLoadingMoreEvents] = useState(false);
|
||||
const loadMoreEventsRef = useRef<(() => void) | null>(null);
|
||||
const [logLines, setLogLines] = useState<RunLogChunk[]>([]);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [logLoading, setLogLoading] = useState(!!run.logRef);
|
||||
const [logError, setLogError] = useState<string | null>(null);
|
||||
const [logOffset, setLogOffsetState] = useState(0);
|
||||
const logOffsetRef = useRef(0);
|
||||
const setLogOffset = useCallback((next: number | ((previous: number) => number)) => {
|
||||
logOffsetRef.current = typeof next === "function" ? next(logOffsetRef.current) : next;
|
||||
setLogOffsetState(logOffsetRef.current);
|
||||
}, []);
|
||||
const logMergeRefs = useRef({ seenChunkKeys: new Set<string>(), trimmedSeqFloorByRun: new Map<string, number>() });
|
||||
const [hasMoreLog, setHasMoreLog] = useState(false);
|
||||
const [loadingMoreLog, setLoadingMoreLog] = useState(false);
|
||||
const [isFollowing, setIsFollowing] = useState(false);
|
||||
const [transcriptMode, setTranscriptMode] = useState<TranscriptMode>("nice");
|
||||
const logEndRef = useRef<HTMLDivElement>(null);
|
||||
const pendingLogLineRef = useRef("");
|
||||
const seenProgressLogLineKeysRef = useRef<Set<string>>(new Set());
|
||||
const scrollContainerRef = useRef<ScrollContainer | null>(null);
|
||||
const isFollowingRef = useRef(false);
|
||||
const lastMetricsRef = useRef<{ scrollHeight: number; distanceFromBottom: number }>({
|
||||
scrollHeight: 0,
|
||||
distanceFromBottom: Number.POSITIVE_INFINITY,
|
||||
});
|
||||
const isLive = run.status === "running" || run.status === "queued";
|
||||
const shouldPollShellLog = shouldPollRunShellLog(run.status);
|
||||
const { data: workspaceOperations = [] } = useQuery({
|
||||
queryKey: queryKeys.runWorkspaceOperations(run.id),
|
||||
queryFn: () => heartbeatsApi.workspaceOperations(run.id),
|
||||
refetchInterval: isLive ? 2000 : false,
|
||||
});
|
||||
|
||||
function isRunLogUnavailable(err: unknown): boolean {
|
||||
return err instanceof ApiError && err.status === 404;
|
||||
}
|
||||
|
||||
function appendLogLines(incoming: RunLogChunk[]) {
|
||||
setLogLines((previous) => mergeRunLogChunks(run.id, previous, incoming.map((line) => ({
|
||||
...line, dedupeKey: `log:${run.id}:${line.ts}:${line.stream}:${line.chunk}`,
|
||||
})), logMergeRefs.current, isLive ? MAX_LIVE_LOG_LINES : Number.POSITIVE_INFINITY).chunks);
|
||||
}
|
||||
|
||||
function appendLogContent(content: string, finalize = false) {
|
||||
if (!content && !finalize) return;
|
||||
const combined = `${pendingLogLineRef.current}${content}`;
|
||||
const split = combined.split("\n");
|
||||
pendingLogLineRef.current = split.pop() ?? "";
|
||||
if (finalize && pendingLogLineRef.current) {
|
||||
split.push(pendingLogLineRef.current);
|
||||
pendingLogLineRef.current = "";
|
||||
}
|
||||
|
||||
const parsed: RunLogChunk[] = [];
|
||||
for (const line of split) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
try {
|
||||
const raw = JSON.parse(trimmed) as { ts?: unknown; stream?: unknown; chunk?: unknown; seq?: unknown };
|
||||
const stream =
|
||||
raw.stream === "stderr" || raw.stream === "system" ? raw.stream : "stdout";
|
||||
const chunk = typeof raw.chunk === "string" ? raw.chunk : "";
|
||||
const ts = typeof raw.ts === "string" ? raw.ts : new Date().toISOString();
|
||||
if (!chunk) continue;
|
||||
const seq = readChunkSeq(raw.seq);
|
||||
logSequenceRef.current.confirm(seq);
|
||||
parsed.push({ ts, stream, chunk, seq });
|
||||
} catch {
|
||||
// ignore malformed lines
|
||||
}
|
||||
}
|
||||
|
||||
if (parsed.length > 0) {
|
||||
// Live runs stream forever, so cap the retained tail. Terminated runs are
|
||||
// paginated by the user via "Load more log" and keep their full history.
|
||||
appendLogLines(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
const refreshEventsRef = useRef<((force?: boolean) => void) | null>(null);
|
||||
const refreshLogRef = useRef<((force?: boolean) => void) | null>(null);
|
||||
const eventSequenceRef = useRef(new TranscriptSequence());
|
||||
const logSequenceRef = useRef(new TranscriptSequence());
|
||||
const eventsReadCursorRef = useRef(0);
|
||||
const eventsCatchupCursorRef = useRef(0);
|
||||
const retainedEventHistoryThroughRef = useRef(0);
|
||||
const logIdentityRef = useRef<string | null>(run.logRef ?? null);
|
||||
const retiredLogRefs = useRef(new Set<string>());
|
||||
const wasLiveRef = useRef(isLive);
|
||||
const bindLog = (ref: unknown): "same" | "changed" | "stale" => {
|
||||
if (typeof ref !== "string" || !ref || ref === logIdentityRef.current) return "same";
|
||||
if (retiredLogRefs.current.has(ref)) return "stale";
|
||||
const prior = logIdentityRef.current;
|
||||
logIdentityRef.current = ref;
|
||||
if (!prior) return "same";
|
||||
retiredLogRefs.current.add(prior);
|
||||
if (retiredLogRefs.current.size > 8) retiredLogRefs.current.delete(retiredLogRefs.current.values().next().value!);
|
||||
pendingLogLineRef.current = "";
|
||||
logMergeRefs.current = { seenChunkKeys: new Set(), trimmedSeqFloorByRun: new Map() };
|
||||
logSequenceRef.current = new TranscriptSequence();
|
||||
setLogLines([]); setLogOffset(0);
|
||||
return "changed";
|
||||
};
|
||||
useEffect(() => {
|
||||
eventsReadCursorRef.current = 0; eventsCatchupCursorRef.current = 0; retainedEventHistoryThroughRef.current = 0; eventSequenceRef.current = new TranscriptSequence(); logSequenceRef.current = new TranscriptSequence();
|
||||
wasLiveRef.current = isLive;
|
||||
logIdentityRef.current = run.logRef ?? null; retiredLogRefs.current.clear();
|
||||
hasMoreEventsRef.current = false; setHasMoreEvents(false); setLoadingMoreEvents(false); setEventsError(null);
|
||||
setEvents([]); setLoading(true);
|
||||
}, [run.id, run.companyId]);
|
||||
useEffect(() => { if (isLive) wasLiveRef.current = true; }, [isLive]);
|
||||
|
||||
function appendEvents(rows: HeartbeatRunEvent[], retainHistory = false) {
|
||||
if (retainHistory) retainedEventHistoryThroughRef.current = Math.max(
|
||||
retainedEventHistoryThroughRef.current, rows.at(-1)?.seq ?? 0,
|
||||
);
|
||||
setEvents(previous => {
|
||||
const merged = mergeRunEvents(previous, rows);
|
||||
if (!isLive && !wasLiveRef.current) return merged;
|
||||
const retainedThrough = Math.max(200, retainedEventHistoryThroughRef.current);
|
||||
const prefix = merged.filter(row => row.seq <= retainedThrough);
|
||||
const tail = merged.filter(row => row.seq > retainedThrough);
|
||||
return [...prefix, ...tail.slice(-MAX_LIVE_EVENTS)];
|
||||
});
|
||||
}
|
||||
|
||||
const stream = useTranscriptLiveStream(isLive ? [run.id] : [], { companyId: run.companyId, enabled: visible }, event => {
|
||||
if (event.companyId !== run.companyId) return;
|
||||
const payload = asRecord(event.payload);
|
||||
if (!payload || payload.runId !== run.id) return;
|
||||
if (event.type === "heartbeat.run.log") {
|
||||
const binding = bindLog(payload.logRef);
|
||||
if (binding === "stale") return;
|
||||
const seq = readChunkSeq(payload.seq);
|
||||
if (payload.truncated === true) logSequenceRef.current.requireHistory(seq);
|
||||
const gap = logSequenceRef.current.observe(seq);
|
||||
if (binding === "changed" || payload.truncated === true || gap) refreshLogRef.current?.();
|
||||
if (payload.truncated === true) return;
|
||||
const chunk = typeof payload.chunk === "string" ? payload.chunk : "";
|
||||
if (!chunk) return;
|
||||
const channel = payload.stream === "stderr" || payload.stream === "system" ? payload.stream : "stdout";
|
||||
appendLogLines([{ ts: asNonEmptyString(payload.ts) ?? event.createdAt, stream: channel, chunk, seq }]);
|
||||
return;
|
||||
}
|
||||
if (event.type === "heartbeat.run.progress") {
|
||||
const line = buildHeartbeatProgressLogLine(payload, event.createdAt);
|
||||
if (!line) return;
|
||||
const key = heartbeatProgressLogLineKey(line);
|
||||
if (seenProgressLogLineKeysRef.current.has(key)) return;
|
||||
seenProgressLogLineKeysRef.current.add(key); appendLogLines([line]);
|
||||
return;
|
||||
}
|
||||
if (event.type === "heartbeat.run.status") {
|
||||
refreshEventsRef.current?.(true); refreshLogRef.current?.(true); return;
|
||||
}
|
||||
const row = liveRunEvent(event);
|
||||
if (!row) return;
|
||||
const commitSignal = row.message === null && row.payload === null;
|
||||
if (commitSignal) eventSequenceRef.current.requireHistory(row.seq);
|
||||
if (eventSequenceRef.current.observe(row.seq)) refreshEventsRef.current?.();
|
||||
if (commitSignal) return;
|
||||
appendEvents([row]);
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
setLoadingMoreEvents(false);
|
||||
if (!visible) return;
|
||||
let cancelled = false;
|
||||
let timer: number | undefined;
|
||||
const abort = new AbortController();
|
||||
let reading = false;
|
||||
let pendingRefresh = false;
|
||||
let failedRead: { requested: boolean; catchup: boolean } | null = null;
|
||||
const read = async (requested = false, catchup = false) => {
|
||||
if (cancelled) return;
|
||||
if (reading) { if (catchup) pendingRefresh = true; return; }
|
||||
if (hasMoreEventsRef.current && !requested && !catchup) return;
|
||||
reading = true; pendingRefresh = false; setLoadingMoreEvents(true); setEventsError(null);
|
||||
window.clearTimeout(timer); timer = undefined;
|
||||
let failed = false;
|
||||
try {
|
||||
const cursor = catchup ? Math.max(eventsCatchupCursorRef.current, eventsReadCursorRef.current) : eventsReadCursorRef.current;
|
||||
const initializesHistory = eventsReadCursorRef.current === 0;
|
||||
const rows = await readTranscriptRequest(signal => heartbeatsApi.events(run.id, cursor, 1000, { signal }), abort.signal);
|
||||
if (cancelled) return;
|
||||
appendEvents(rows, !catchup || initializesHistory);
|
||||
const next = Math.max(cursor, rows.at(-1)?.seq ?? cursor);
|
||||
eventsCatchupCursorRef.current = Math.max(eventsCatchupCursorRef.current, next);
|
||||
eventSequenceRef.current.confirm(next);
|
||||
if (catchup) {
|
||||
if (rows.length === 1000 && next > cursor) pendingRefresh = true;
|
||||
}
|
||||
if (!catchup || initializesHistory) {
|
||||
eventsReadCursorRef.current = next;
|
||||
hasMoreEventsRef.current = rows.length === 1000 && next > cursor;
|
||||
setHasMoreEvents(hasMoreEventsRef.current);
|
||||
}
|
||||
failed = eventSequenceRef.current.needsCatchup;
|
||||
failedRead = null;
|
||||
} catch {
|
||||
failed = true; failedRead = { requested, catchup };
|
||||
if (!cancelled) setEventsError("Could not load events. Try again.");
|
||||
}
|
||||
if (cancelled) return;
|
||||
reading = false; setLoadingMoreEvents(false); setLoading(false);
|
||||
if (pendingRefresh) void read(false, true);
|
||||
else if (failed || (isLive && !stream.healthy && !hasMoreEventsRef.current)) {
|
||||
timer = window.setTimeout(() => void read(requested, catchup || eventSequenceRef.current.needsCatchup), 2000);
|
||||
}
|
||||
};
|
||||
refreshEventsRef.current = () => { void read(false, wasLiveRef.current); };
|
||||
loadMoreEventsRef.current = () => { void read(failedRead?.requested ?? true, failedRead?.catchup ?? false); };
|
||||
void read(false, wasLiveRef.current && (stream.healthy || eventsReadCursorRef.current > 0));
|
||||
return () => {
|
||||
cancelled = true; refreshEventsRef.current = null; loadMoreEventsRef.current = null;
|
||||
abort.abort(); if (timer !== undefined) window.clearTimeout(timer);
|
||||
};
|
||||
}, [visible, run.id, run.companyId, run.status, stream.healthy, stream.epoch]);
|
||||
|
||||
const getScrollContainer = useCallback((): ScrollContainer => {
|
||||
if (scrollContainerRef.current) return scrollContainerRef.current;
|
||||
const container = findScrollContainer(logEndRef.current);
|
||||
scrollContainerRef.current = container;
|
||||
return container;
|
||||
}, []);
|
||||
|
||||
const updateFollowingState = useCallback(() => {
|
||||
const container = getScrollContainer();
|
||||
const metrics = readScrollMetrics(container);
|
||||
lastMetricsRef.current = metrics;
|
||||
const nearBottom = metrics.distanceFromBottom <= LIVE_SCROLL_BOTTOM_TOLERANCE_PX;
|
||||
isFollowingRef.current = nearBottom;
|
||||
setIsFollowing((prev) => (prev === nearBottom ? prev : nearBottom));
|
||||
}, [getScrollContainer]);
|
||||
|
||||
useEffect(() => {
|
||||
scrollContainerRef.current = null;
|
||||
lastMetricsRef.current = {
|
||||
scrollHeight: 0,
|
||||
distanceFromBottom: Number.POSITIVE_INFINITY,
|
||||
};
|
||||
|
||||
if (!isLive) {
|
||||
isFollowingRef.current = false;
|
||||
setIsFollowing(false);
|
||||
return;
|
||||
}
|
||||
|
||||
updateFollowingState();
|
||||
}, [isLive, run.id, updateFollowingState]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) return;
|
||||
const container = getScrollContainer();
|
||||
updateFollowingState();
|
||||
|
||||
if (container === window) {
|
||||
window.addEventListener("scroll", updateFollowingState, { passive: true });
|
||||
} else {
|
||||
container.addEventListener("scroll", updateFollowingState, { passive: true });
|
||||
}
|
||||
window.addEventListener("resize", updateFollowingState);
|
||||
return () => {
|
||||
if (container === window) {
|
||||
window.removeEventListener("scroll", updateFollowingState);
|
||||
} else {
|
||||
container.removeEventListener("scroll", updateFollowingState);
|
||||
}
|
||||
window.removeEventListener("resize", updateFollowingState);
|
||||
};
|
||||
}, [isLive, run.id, getScrollContainer, updateFollowingState]);
|
||||
|
||||
// Auto-scroll only for live runs when following
|
||||
useEffect(() => {
|
||||
if (!isLive || !isFollowingRef.current) return;
|
||||
|
||||
const container = getScrollContainer();
|
||||
const previous = lastMetricsRef.current;
|
||||
const current = readScrollMetrics(container);
|
||||
const growth = Math.max(0, current.scrollHeight - previous.scrollHeight);
|
||||
const expectedDistance = previous.distanceFromBottom + growth;
|
||||
const movedAwayBy = current.distanceFromBottom - expectedDistance;
|
||||
|
||||
// If user moved away from bottom between updates, release auto-follow immediately.
|
||||
if (movedAwayBy > LIVE_SCROLL_BOTTOM_TOLERANCE_PX) {
|
||||
isFollowingRef.current = false;
|
||||
setIsFollowing(false);
|
||||
lastMetricsRef.current = current;
|
||||
return;
|
||||
}
|
||||
|
||||
scrollToContainerBottom(container, "auto");
|
||||
const after = readScrollMetrics(container);
|
||||
lastMetricsRef.current = after;
|
||||
if (!isFollowingRef.current) {
|
||||
isFollowingRef.current = true;
|
||||
}
|
||||
setIsFollowing((prev) => (prev ? prev : true));
|
||||
}, [events.length, logLines.length, isLive, getScrollContainer]);
|
||||
|
||||
// Reset only when the log source changes, never when visibility changes.
|
||||
useEffect(() => {
|
||||
pendingLogLineRef.current = "";
|
||||
logMergeRefs.current = { seenChunkKeys: new Set(), trimmedSeqFloorByRun: new Map() };
|
||||
seenProgressLogLineKeysRef.current = new Set();
|
||||
setLogLines([]);
|
||||
setLogOffset(0);
|
||||
setHasMoreLog(false);
|
||||
setLoadingMoreLog(false);
|
||||
setLogError(null);
|
||||
}, [run.id, run.logRef, setLogOffset]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!visible) return;
|
||||
if (!run.logRef && !shouldPollShellLog) { setLogLoading(false); return; }
|
||||
let cancelled = false;
|
||||
let timer: number | undefined;
|
||||
const abort = new AbortController();
|
||||
setLogLoading(true);
|
||||
let reading = false;
|
||||
let pendingRefresh = false;
|
||||
const load = async () => {
|
||||
if (cancelled || reading) return;
|
||||
reading = true; pendingRefresh = false;
|
||||
window.clearTimeout(timer); timer = undefined;
|
||||
let failed = false;
|
||||
try {
|
||||
for (;;) {
|
||||
const offset = logOffsetRef.current;
|
||||
const result = await readTranscriptRequest(signal => heartbeatsApi.log(run.id, offset, RUN_LOG_PAGE_BYTES, { signal }), abort.signal);
|
||||
if (cancelled) return;
|
||||
const binding = bindLog(result.logRef);
|
||||
if (binding === "stale") { failed = true; break; }
|
||||
if (binding === "changed" && offset !== 0) continue;
|
||||
appendLogContent(result.content, result.nextOffset === undefined);
|
||||
const next = result.nextOffset ?? offset + new TextEncoder().encode(result.content).byteLength;
|
||||
setLogOffset(next); setLogError(null);
|
||||
setHasMoreLog(!wasLiveRef.current && result.nextOffset !== undefined && !!result.content);
|
||||
if (!wasLiveRef.current || result.nextOffset === undefined || !result.content || next <= offset) break;
|
||||
}
|
||||
failed ||= logSequenceRef.current.needsCatchup;
|
||||
} catch (err) {
|
||||
if (cancelled) return;
|
||||
failed = shouldPollShellLog || !isRunLogUnavailable(err);
|
||||
if (!isRunLogUnavailable(err)) setLogError(err instanceof Error ? err.message : "Failed to load run log");
|
||||
}
|
||||
if (cancelled) return;
|
||||
reading = false;
|
||||
setLogLoading(false);
|
||||
if (pendingRefresh) void load();
|
||||
else if (failed || (shouldPollShellLog && !stream.healthy)) timer = window.setTimeout(() => void load(), 2000);
|
||||
};
|
||||
refreshLogRef.current = (force = false) => { if (force) pendingRefresh = true; void load(); };
|
||||
void load();
|
||||
return () => { cancelled = true; refreshLogRef.current = null; abort.abort(); if (timer !== undefined) window.clearTimeout(timer); };
|
||||
}, [visible, run.id, run.companyId, run.logRef, isLive ? null : run.logBytes, shouldPollShellLog, stream.healthy, stream.epoch]);
|
||||
|
||||
async function loadMorePersistedLog() {
|
||||
if (loadingMoreLog || !hasMoreLog) return;
|
||||
setLoadingMoreLog(true);
|
||||
setLogError(null);
|
||||
try {
|
||||
const result = await heartbeatsApi.log(run.id, logOffset, RUN_LOG_PAGE_BYTES);
|
||||
appendLogContent(result.content, result.nextOffset === undefined);
|
||||
const next = result.nextOffset ?? logOffset + new TextEncoder().encode(result.content).byteLength;
|
||||
setLogOffset(next);
|
||||
setHasMoreLog(result.nextOffset !== undefined);
|
||||
} catch (err) {
|
||||
setLogError(err instanceof Error ? err.message : "Failed to load more run log");
|
||||
} finally {
|
||||
setLoadingMoreLog(false);
|
||||
}
|
||||
}
|
||||
|
||||
const censorUsernameInLogs = useQuery({
|
||||
queryKey: queryKeys.instance.generalSettings,
|
||||
queryFn: () => instanceSettingsApi.getGeneral(),
|
||||
}).data?.censorUsernameInLogs === true;
|
||||
|
||||
const adapterInvokePayload = useMemo(() => {
|
||||
const evt = events.find((e) => e.eventType === "adapter.invoke");
|
||||
return redactPathValue(asRecord(evt?.payload ?? null), censorUsernameInLogs);
|
||||
}, [censorUsernameInLogs, events]);
|
||||
|
||||
// NOTE: adapter is NOT memoized because external adapters replace their
|
||||
// parseStdoutLine asynchronously after dynamic parser loading. Memoizing
|
||||
// on adapterType alone would stale the transcript with the fallback parser.
|
||||
// We subscribe to adapter registry changes to force transcript recomputation.
|
||||
const [parserTick, setParserTick] = useState(0);
|
||||
const adapter = getUIAdapter(adapterType);
|
||||
|
||||
useEffect(() => {
|
||||
return onAdapterChange(() => setParserTick((t) => t + 1));
|
||||
}, []);
|
||||
|
||||
const transcript = useMemo(
|
||||
() => buildTranscript(logLines, adapter, { censorUsernameInLogs }),
|
||||
[adapter, censorUsernameInLogs, logLines, parserTick],
|
||||
);
|
||||
const toolDecisionLookup = useQuery({
|
||||
queryKey: queryKeys.tools.runDecisions(run.companyId, run.id),
|
||||
queryFn: () => toolsApi.getRunDecisionLookup(run.companyId, run.id),
|
||||
enabled: Boolean(run.companyId && run.id),
|
||||
refetchInterval: isLive ? 3000 : false,
|
||||
staleTime: isLive ? 1000 : 30_000,
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
setTranscriptMode("nice");
|
||||
}, [run.id]);
|
||||
|
||||
if (loading && logLoading) {
|
||||
return <p className="text-xs text-muted-foreground">Loading run logs...</p>;
|
||||
}
|
||||
|
||||
if (events.length === 0 && logLines.length === 0 && !logError && !eventsError) {
|
||||
return <p className="text-xs text-muted-foreground">No log events.</p>;
|
||||
}
|
||||
|
||||
const levelColors: Record<string, string> = {
|
||||
info: "text-foreground",
|
||||
warn: "text-yellow-600 dark:text-yellow-400",
|
||||
error: "text-red-600 dark:text-red-400",
|
||||
};
|
||||
|
||||
const streamColors: Record<string, string> = {
|
||||
stdout: "text-foreground",
|
||||
stderr: "text-red-600 dark:text-red-300",
|
||||
system: "text-blue-600 dark:text-blue-300",
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
<WorkspaceOperationsSection
|
||||
operations={workspaceOperations}
|
||||
censorUsernameInLogs={censorUsernameInLogs}
|
||||
/>
|
||||
{adapterInvokePayload && (
|
||||
<RunInvocationCard payload={adapterInvokePayload} censorUsernameInLogs={censorUsernameInLogs} />
|
||||
)}
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-xs font-medium text-muted-foreground">
|
||||
Transcript ({transcript.length})
|
||||
</span>
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="inline-flex rounded-lg border border-border/70 bg-background/70 p-0.5">
|
||||
{(["nice", "raw"] as const).map((mode) => (
|
||||
<button
|
||||
key={mode}
|
||||
type="button"
|
||||
className={cn(
|
||||
"rounded-md px-2.5 py-1 text-(length:--text-micro) font-medium capitalize transition-colors",
|
||||
transcriptMode === mode
|
||||
? "bg-accent text-foreground shadow-sm"
|
||||
: "text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
onClick={() => setTranscriptMode(mode)}
|
||||
>
|
||||
{mode}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
{isLive && !isFollowing && (
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="xs"
|
||||
onClick={() => {
|
||||
const container = getScrollContainer();
|
||||
isFollowingRef.current = true;
|
||||
setIsFollowing(true);
|
||||
scrollToContainerBottom(container, "auto");
|
||||
lastMetricsRef.current = readScrollMetrics(container);
|
||||
}}
|
||||
>
|
||||
Jump to live
|
||||
</Button>
|
||||
)}
|
||||
{isLive && (
|
||||
<span className="flex items-center gap-1 text-xs text-blue-600 dark:text-blue-400">
|
||||
<span className="relative flex h-2 w-2">
|
||||
<span className="animate-pulse absolute inline-flex h-full w-full rounded-full bg-blue-400 opacity-75" />
|
||||
<span className="relative inline-flex rounded-full h-2 w-2 bg-blue-500" />
|
||||
</span>
|
||||
Live
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="max-h-(--sz-38rem) overflow-y-auto rounded-2xl border border-border/70 bg-background/40 p-3 sm:p-4">
|
||||
<RunTranscriptView
|
||||
entries={transcript}
|
||||
toolDecisions={toolDecisionLookup.data?.decisions ?? []}
|
||||
mode={transcriptMode}
|
||||
streaming={isLive}
|
||||
limit={isLive ? LIVE_TRANSCRIPT_RENDER_LIMIT : undefined}
|
||||
emptyMessage={run.logRef ? "Waiting for transcript..." : "No persisted transcript for this run."}
|
||||
/>
|
||||
{hasMoreLog && (
|
||||
<div className="mt-3 flex flex-wrap items-center gap-2 border-t border-border/60 pt-3">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="xs"
|
||||
onClick={loadMorePersistedLog}
|
||||
disabled={loadingMoreLog}
|
||||
>
|
||||
{loadingMoreLog ? "Loading..." : "Load more log"}
|
||||
</Button>
|
||||
<span className="text-xs text-muted-foreground">
|
||||
Showing the first {Math.round(logOffset / 1024).toLocaleString("en-US")} KB
|
||||
{typeof run.logBytes === "number" && run.logBytes > 0
|
||||
? ` of ${Math.round(run.logBytes / 1024).toLocaleString("en-US")} KB`
|
||||
: ""}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{logError && (
|
||||
<div className="mt-3 rounded-xl border border-red-500/20 bg-red-500/[0.06] px-3 py-2 text-xs text-red-700 dark:text-red-300">
|
||||
{logError}
|
||||
</div>
|
||||
)}
|
||||
<div ref={logEndRef} />
|
||||
</div>
|
||||
|
||||
{(run.status === "failed" || run.status === "timed_out") && (
|
||||
<div className="rounded-lg border border-red-300 dark:border-red-500/30 bg-red-50 dark:bg-red-950/20 p-3 space-y-2">
|
||||
<div className="text-xs font-medium text-red-700 dark:text-red-300">Failure details</div>
|
||||
{run.error && (
|
||||
<div className="text-xs text-red-600 dark:text-red-200">
|
||||
<span className="text-red-700 dark:text-red-300">Error: </span>
|
||||
{redactPathText(run.error, censorUsernameInLogs)}
|
||||
</div>
|
||||
)}
|
||||
{run.stderrExcerpt && run.stderrExcerpt.trim() && (
|
||||
<div>
|
||||
<div className="text-xs text-red-700 dark:text-red-300 mb-1">stderr excerpt</div>
|
||||
<pre className="bg-red-50 dark:bg-neutral-950 rounded-md p-2 text-xs overflow-x-auto whitespace-pre-wrap text-red-800 dark:text-red-100">
|
||||
{redactPathText(run.stderrExcerpt, censorUsernameInLogs)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
{run.resultJson && (
|
||||
<div>
|
||||
<div className="text-xs text-red-700 dark:text-red-300 mb-1">adapter result JSON</div>
|
||||
<pre className="bg-red-50 dark:bg-neutral-950 rounded-md p-2 text-xs overflow-x-auto whitespace-pre-wrap text-red-800 dark:text-red-100">
|
||||
{JSON.stringify(redactPathValue(run.resultJson, censorUsernameInLogs), null, 2)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
{run.stdoutExcerpt && run.stdoutExcerpt.trim() && !run.resultJson && (
|
||||
<div>
|
||||
<div className="text-xs text-red-700 dark:text-red-300 mb-1">stdout excerpt</div>
|
||||
<pre className="bg-red-50 dark:bg-neutral-950 rounded-md p-2 text-xs overflow-x-auto whitespace-pre-wrap text-red-800 dark:text-red-100">
|
||||
{redactPathText(run.stdoutExcerpt, censorUsernameInLogs)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{(events.length > 0 || eventsError) && (
|
||||
<div>
|
||||
<div className="mb-2 text-xs font-medium text-muted-foreground">Events ({events.length})</div>
|
||||
{eventsError && <p className="text-sm text-destructive">{eventsError}</p>}
|
||||
{(hasMoreEvents || eventsError) && (
|
||||
<Button variant="outline" size="sm" disabled={loadingMoreEvents} onClick={() => loadMoreEventsRef.current?.()}>
|
||||
{loadingMoreEvents ? "Loading..." : eventsError ? "Retry events" : "Load more events"}
|
||||
</Button>
|
||||
)}
|
||||
<div className="bg-neutral-100 dark:bg-neutral-950 rounded-lg p-3 font-mono text-xs space-y-0.5">
|
||||
{events.map((evt) => {
|
||||
const color = evt.color
|
||||
?? (evt.level ? levelColors[evt.level] : null)
|
||||
?? (evt.stream ? streamColors[evt.stream] : null)
|
||||
?? "text-foreground";
|
||||
|
||||
return (
|
||||
<div key={`${evt.runId}:${evt.seq}`} className="flex gap-2">
|
||||
<span className="text-neutral-400 dark:text-neutral-600 shrink-0 select-none w-16">
|
||||
{new Date(evt.createdAt).toLocaleTimeString("en-US", { hour12: false })}
|
||||
</span>
|
||||
<span className={cn("shrink-0 w-14", evt.stream ? (streamColors[evt.stream] ?? "text-neutral-500") : "text-neutral-500")}>
|
||||
{evt.stream ? `[${evt.stream}]` : ""}
|
||||
</span>
|
||||
<span className={cn("break-all", color)}>
|
||||
{evt.message
|
||||
? redactPathText(evt.message, censorUsernameInLogs)
|
||||
: evt.payload
|
||||
? JSON.stringify(redactPathValue(evt.payload, censorUsernameInLogs))
|
||||
: ""}
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,490 @@
|
||||
import { useMemo, useState } from "react";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { heartbeatsApi } from "../../api/heartbeats";
|
||||
import { redactCommandText as redactCommandSecretText } from "@paperclipai/adapter-utils";
|
||||
import { cn, relativeTime } from "../../lib/utils";
|
||||
import { ChevronRight } from "lucide-react";
|
||||
import { Collapsible, CollapsibleTrigger, CollapsibleContent } from "@/components/ui/collapsible";
|
||||
import { type HeartbeatRun, type WorkspaceOperation } from "@paperclipai/shared";
|
||||
import { redactHomePathUserSegments, redactHomePathUserSegmentsInValue } from "@paperclipai/adapter-utils";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
|
||||
export const RUN_LOG_PAGE_BYTES = 256_000;
|
||||
|
||||
const REDACTED_ENV_VALUE = "***REDACTED***";
|
||||
|
||||
const SECRET_ENV_KEY_RE =
|
||||
/(api[-_]?key|access[-_]?token|auth(?:_?token)?|authorization|bearer|secret|passwd|password|credential|jwt|private[-_]?key|cookie|connectionstring)/i;
|
||||
|
||||
const COMMAND_ENV_KEY_RE = /(^command$|^cmd$|command[-_]?line|resolved[-_]?command|PAPERCLIP_RESOLVED_COMMAND)/i;
|
||||
|
||||
const JWT_VALUE_RE = /^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+(?:\.[A-Za-z0-9_-]+)?$/;
|
||||
|
||||
export function redactPathText(value: string, censorUsernameInLogs: boolean) {
|
||||
return redactHomePathUserSegments(value, { enabled: censorUsernameInLogs });
|
||||
}
|
||||
|
||||
export function redactPathValue<T>(value: T, censorUsernameInLogs: boolean): T {
|
||||
return redactHomePathUserSegmentsInValue(value, { enabled: censorUsernameInLogs });
|
||||
}
|
||||
|
||||
function redactCommandText(value: string, censorUsernameInLogs: boolean): string {
|
||||
return redactPathText(redactCommandSecretText(value, REDACTED_ENV_VALUE), censorUsernameInLogs);
|
||||
}
|
||||
|
||||
function shouldRedactSecretValue(key: string, value: unknown): boolean {
|
||||
if (SECRET_ENV_KEY_RE.test(key)) return true;
|
||||
if (typeof value !== "string") return false;
|
||||
return JWT_VALUE_RE.test(value);
|
||||
}
|
||||
|
||||
function redactEnvValue(key: string, value: unknown, censorUsernameInLogs: boolean): string {
|
||||
if (
|
||||
typeof value === "object" &&
|
||||
value !== null &&
|
||||
!Array.isArray(value) &&
|
||||
(value as { type?: unknown }).type === "secret_ref"
|
||||
) {
|
||||
return "***SECRET_REF***";
|
||||
}
|
||||
if (shouldRedactSecretValue(key, value)) return REDACTED_ENV_VALUE;
|
||||
if (value === null || value === undefined) return "";
|
||||
if (typeof value === "string" && COMMAND_ENV_KEY_RE.test(key)) return redactCommandText(value, censorUsernameInLogs);
|
||||
if (typeof value === "string") return redactPathText(value, censorUsernameInLogs);
|
||||
try {
|
||||
return JSON.stringify(redactPathValue(value, censorUsernameInLogs));
|
||||
} catch {
|
||||
return redactPathText(String(value), censorUsernameInLogs);
|
||||
}
|
||||
}
|
||||
|
||||
function formatEnvForDisplay(envValue: unknown, censorUsernameInLogs: boolean): string {
|
||||
const env = asRecord(envValue);
|
||||
if (!env) return "<unable-to-parse>";
|
||||
|
||||
const keys = Object.keys(env);
|
||||
if (keys.length === 0) return "<empty>";
|
||||
|
||||
return keys
|
||||
.sort()
|
||||
.map((key) => `${key}=${redactEnvValue(key, env[key], censorUsernameInLogs)}`)
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
export const LIVE_SCROLL_BOTTOM_TOLERANCE_PX = 32;
|
||||
|
||||
export type ScrollContainer = Window | HTMLElement;
|
||||
|
||||
function isWindowContainer(container: ScrollContainer): container is Window {
|
||||
return container === window;
|
||||
}
|
||||
|
||||
function isElementScrollContainer(element: HTMLElement): boolean {
|
||||
const overflowY = window.getComputedStyle(element).overflowY;
|
||||
return overflowY === "auto" || overflowY === "scroll" || overflowY === "overlay";
|
||||
}
|
||||
|
||||
export function findScrollContainer(anchor: HTMLElement | null): ScrollContainer {
|
||||
let parent = anchor?.parentElement ?? null;
|
||||
while (parent) {
|
||||
if (isElementScrollContainer(parent)) return parent;
|
||||
parent = parent.parentElement;
|
||||
}
|
||||
return window;
|
||||
}
|
||||
|
||||
export function readScrollMetrics(container: ScrollContainer): { scrollHeight: number; distanceFromBottom: number } {
|
||||
if (isWindowContainer(container)) {
|
||||
const pageHeight = Math.max(
|
||||
document.documentElement.scrollHeight,
|
||||
document.body.scrollHeight,
|
||||
);
|
||||
const viewportBottom = window.scrollY + window.innerHeight;
|
||||
return {
|
||||
scrollHeight: pageHeight,
|
||||
distanceFromBottom: Math.max(0, pageHeight - viewportBottom),
|
||||
};
|
||||
}
|
||||
|
||||
const viewportBottom = container.scrollTop + container.clientHeight;
|
||||
return {
|
||||
scrollHeight: container.scrollHeight,
|
||||
distanceFromBottom: Math.max(0, container.scrollHeight - viewportBottom),
|
||||
};
|
||||
}
|
||||
|
||||
export function scrollToContainerBottom(container: ScrollContainer, behavior: ScrollBehavior = "auto") {
|
||||
if (isWindowContainer(container)) {
|
||||
const pageHeight = Math.max(
|
||||
document.documentElement.scrollHeight,
|
||||
document.body.scrollHeight,
|
||||
);
|
||||
window.scrollTo({ top: pageHeight, behavior });
|
||||
return;
|
||||
}
|
||||
|
||||
container.scrollTo({ top: container.scrollHeight, behavior });
|
||||
}
|
||||
|
||||
export type RunLogChunk = {
|
||||
seq?: number;
|
||||
ts: string;
|
||||
stream: "stdout" | "stderr" | "system";
|
||||
chunk: string;
|
||||
};
|
||||
|
||||
export function asRecord(value: unknown): Record<string, unknown> | null {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value)) return null;
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
export function asNonEmptyString(value: unknown): string | null {
|
||||
if (typeof value !== "string") return null;
|
||||
const trimmed = value.trim();
|
||||
return trimmed.length > 0 ? trimmed : null;
|
||||
}
|
||||
|
||||
export function buildHeartbeatProgressLogLine(
|
||||
payload: Record<string, unknown>,
|
||||
fallbackTimestamp: string,
|
||||
): RunLogChunk | null {
|
||||
const message = asNonEmptyString(payload.message);
|
||||
if (!message) return null;
|
||||
const phase = asNonEmptyString(payload.phase);
|
||||
const ts = asNonEmptyString(payload.updatedAt) ?? fallbackTimestamp;
|
||||
const chunk = phase ? `[${phase}] ${message}` : message;
|
||||
return { ts, stream: "system", chunk };
|
||||
}
|
||||
|
||||
export function heartbeatProgressLogLineKey(line: RunLogChunk): string {
|
||||
return `${line.ts}\u0000${line.stream}\u0000${line.chunk}`;
|
||||
}
|
||||
|
||||
export function shouldPollRunShellLog(status: HeartbeatRun["status"]): boolean {
|
||||
return status === "running";
|
||||
}
|
||||
|
||||
export function RunInvocationCard({
|
||||
payload,
|
||||
censorUsernameInLogs,
|
||||
}: {
|
||||
payload: Record<string, unknown>;
|
||||
censorUsernameInLogs: boolean;
|
||||
}) {
|
||||
const rawCommandLine = [
|
||||
typeof payload.command === "string" ? payload.command : null,
|
||||
...(Array.isArray(payload.commandArgs)
|
||||
? payload.commandArgs.filter((value): value is string => typeof value === "string")
|
||||
: []),
|
||||
]
|
||||
.filter((value): value is string => Boolean(value))
|
||||
.join(" ");
|
||||
const commandLine = rawCommandLine ? redactCommandText(rawCommandLine, censorUsernameInLogs) : "";
|
||||
|
||||
const hasAdvancedDetails =
|
||||
commandLine.length > 0
|
||||
|| (Array.isArray(payload.commandNotes) && payload.commandNotes.length > 0)
|
||||
|| payload.prompt !== undefined
|
||||
|| payload.context !== undefined
|
||||
|| payload.env !== undefined;
|
||||
|
||||
return (
|
||||
<div className="rounded-lg border border-border bg-background/60 p-3 space-y-2">
|
||||
<div className="text-xs font-medium text-muted-foreground">Invocation</div>
|
||||
{typeof payload.adapterType === "string" && (
|
||||
<div className="text-xs"><span className="text-muted-foreground">Adapter: </span>{payload.adapterType}</div>
|
||||
)}
|
||||
{typeof payload.cwd === "string" && (
|
||||
<div className="text-xs break-all"><span className="text-muted-foreground">Working dir: </span><span className="font-mono">{payload.cwd}</span></div>
|
||||
)}
|
||||
{hasAdvancedDetails && (
|
||||
<Collapsible>
|
||||
<CollapsibleTrigger className="flex items-center gap-1 text-xs text-muted-foreground hover:text-foreground transition-colors group">
|
||||
<ChevronRight className="h-3 w-3 transition-transform group-data-[state=open]:rotate-90" />
|
||||
Details
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent className="pt-2 space-y-2">
|
||||
{commandLine && (
|
||||
<div className="text-xs break-all">
|
||||
<span className="text-muted-foreground">Command: </span>
|
||||
<span className="font-mono">{commandLine}</span>
|
||||
</div>
|
||||
)}
|
||||
{Array.isArray(payload.commandNotes) && payload.commandNotes.length > 0 && (
|
||||
<div>
|
||||
<div className="text-xs text-muted-foreground mb-1">Command notes</div>
|
||||
<ul className="list-disc pl-5 space-y-1">
|
||||
{payload.commandNotes
|
||||
.filter((value): value is string => typeof value === "string" && value.trim().length > 0)
|
||||
.map((note, idx) => (
|
||||
<li key={`${idx}-${note}`} className="text-xs break-all font-mono">
|
||||
{note}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
)}
|
||||
{payload.prompt !== undefined && (
|
||||
<div>
|
||||
<div className="text-xs text-muted-foreground mb-1">Prompt</div>
|
||||
<pre className="bg-neutral-100 dark:bg-neutral-950 rounded-md p-2 text-xs overflow-x-auto whitespace-pre-wrap">
|
||||
{typeof payload.prompt === "string"
|
||||
? redactPathText(payload.prompt, censorUsernameInLogs)
|
||||
: JSON.stringify(redactPathValue(payload.prompt, censorUsernameInLogs), null, 2)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
{payload.context !== undefined && (
|
||||
<div>
|
||||
<div className="text-xs text-muted-foreground mb-1">Context</div>
|
||||
<pre className="bg-neutral-100 dark:bg-neutral-950 rounded-md p-2 text-xs overflow-x-auto whitespace-pre-wrap">
|
||||
{JSON.stringify(redactPathValue(payload.context, censorUsernameInLogs), null, 2)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
{payload.env !== undefined && (
|
||||
<div>
|
||||
<div className="text-xs text-muted-foreground mb-1">Environment</div>
|
||||
<pre className="bg-neutral-100 dark:bg-neutral-950 rounded-md p-2 text-xs overflow-x-auto whitespace-pre-wrap font-mono">
|
||||
{formatEnvForDisplay(payload.env, censorUsernameInLogs)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
</CollapsibleContent>
|
||||
</Collapsible>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function parseStoredLogContent(content: string): RunLogChunk[] {
|
||||
const parsed: RunLogChunk[] = [];
|
||||
for (const line of content.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
try {
|
||||
const raw = JSON.parse(trimmed) as { ts?: unknown; stream?: unknown; chunk?: unknown };
|
||||
const stream =
|
||||
raw.stream === "stderr" || raw.stream === "system" ? raw.stream : "stdout";
|
||||
const chunk = typeof raw.chunk === "string" ? raw.chunk : "";
|
||||
const ts = typeof raw.ts === "string" ? raw.ts : new Date().toISOString();
|
||||
if (!chunk) continue;
|
||||
parsed.push({ ts, stream, chunk });
|
||||
} catch {
|
||||
// Ignore malformed log lines.
|
||||
}
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function workspaceOperationPhaseLabel(phase: WorkspaceOperation["phase"]) {
|
||||
switch (phase) {
|
||||
case "worktree_prepare":
|
||||
return "Worktree setup";
|
||||
case "workspace_config_freshness":
|
||||
return "Config freshness";
|
||||
case "workspace_provision":
|
||||
return "Provision";
|
||||
case "workspace_teardown":
|
||||
return "Teardown";
|
||||
case "worktree_cleanup":
|
||||
return "Worktree cleanup";
|
||||
default:
|
||||
return phase;
|
||||
}
|
||||
}
|
||||
|
||||
function workspaceOperationStatusTone(status: WorkspaceOperation["status"]) {
|
||||
switch (status) {
|
||||
case "succeeded":
|
||||
return "border-green-500/20 bg-green-500/10 text-green-700 dark:text-green-300";
|
||||
case "failed":
|
||||
return "border-red-500/20 bg-red-500/10 text-red-700 dark:text-red-300";
|
||||
case "running":
|
||||
return "border-blue-500/20 bg-blue-500/10 text-blue-700 dark:text-blue-300";
|
||||
case "skipped":
|
||||
return "border-yellow-500/20 bg-yellow-500/10 text-yellow-700 dark:text-yellow-300";
|
||||
default:
|
||||
return "border-border bg-muted/40 text-muted-foreground";
|
||||
}
|
||||
}
|
||||
|
||||
function WorkspaceOperationStatusBadge({ status }: { status: WorkspaceOperation["status"] }) {
|
||||
return (
|
||||
<Badge variant="outline"
|
||||
className={cn(
|
||||
"text-(length:--text-micro) capitalize",
|
||||
workspaceOperationStatusTone(status),
|
||||
)}
|
||||
>
|
||||
{status.replace("_", " ")}
|
||||
</Badge>
|
||||
);
|
||||
}
|
||||
|
||||
function WorkspaceOperationLogViewer({
|
||||
operation,
|
||||
censorUsernameInLogs,
|
||||
}: {
|
||||
operation: WorkspaceOperation;
|
||||
censorUsernameInLogs: boolean;
|
||||
}) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const { data: logData, isLoading, error } = useQuery({
|
||||
queryKey: ["workspace-operation-log", operation.id],
|
||||
queryFn: () => heartbeatsApi.workspaceOperationLog(operation.id),
|
||||
enabled: open && Boolean(operation.logRef),
|
||||
refetchInterval: open && operation.status === "running" ? 2000 : false,
|
||||
});
|
||||
|
||||
const chunks = useMemo(
|
||||
() => (logData?.content ? parseStoredLogContent(logData.content) : []),
|
||||
[logData?.content],
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<button
|
||||
type="button"
|
||||
className="text-(length:--text-micro) text-muted-foreground underline underline-offset-2 hover:text-foreground"
|
||||
onClick={() => setOpen((value) => !value)}
|
||||
>
|
||||
{open ? "Hide full log" : "Show full log"}
|
||||
</button>
|
||||
{open && (
|
||||
<div className="rounded-md border border-border bg-background/70 p-2">
|
||||
{isLoading && <div className="text-xs text-muted-foreground">Loading log...</div>}
|
||||
{error && (
|
||||
<div className="text-xs text-destructive">
|
||||
{error instanceof Error ? error.message : "Failed to load workspace operation log"}
|
||||
</div>
|
||||
)}
|
||||
{!isLoading && !error && chunks.length === 0 && (
|
||||
<div className="text-xs text-muted-foreground">No persisted log lines.</div>
|
||||
)}
|
||||
{chunks.length > 0 && (
|
||||
<div className="max-h-64 overflow-y-auto rounded bg-neutral-100 p-2 font-mono text-xs dark:bg-neutral-950">
|
||||
{chunks.map((chunk, index) => (
|
||||
<div key={`${chunk.ts}-${index}`} className="flex gap-2">
|
||||
<span className="shrink-0 text-neutral-500">
|
||||
{new Date(chunk.ts).toLocaleTimeString("en-US", { hour12: false })}
|
||||
</span>
|
||||
<span
|
||||
className={cn(
|
||||
"shrink-0 w-14",
|
||||
chunk.stream === "stderr"
|
||||
? "text-red-600 dark:text-red-300"
|
||||
: chunk.stream === "system"
|
||||
? "text-blue-600 dark:text-blue-300"
|
||||
: "text-muted-foreground",
|
||||
)}
|
||||
>
|
||||
[{chunk.stream}]
|
||||
</span>
|
||||
<span className="whitespace-pre-wrap break-all">{redactPathText(chunk.chunk, censorUsernameInLogs)}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function WorkspaceOperationsSection({
|
||||
operations,
|
||||
censorUsernameInLogs,
|
||||
}: {
|
||||
operations: WorkspaceOperation[];
|
||||
censorUsernameInLogs: boolean;
|
||||
}) {
|
||||
if (operations.length === 0) return null;
|
||||
|
||||
return (
|
||||
<div className="rounded-lg border border-border bg-background/60 p-3 space-y-3">
|
||||
<div className="text-xs font-medium text-muted-foreground">
|
||||
Workspace ({operations.length})
|
||||
</div>
|
||||
<div className="space-y-3">
|
||||
{operations.map((operation) => {
|
||||
const metadata = asRecord(operation.metadata);
|
||||
return (
|
||||
<div key={operation.id} className="rounded-md border border-border/70 bg-background/70 p-3 space-y-2">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<div className="text-sm font-medium">{workspaceOperationPhaseLabel(operation.phase)}</div>
|
||||
<WorkspaceOperationStatusBadge status={operation.status} />
|
||||
<div className="text-(length:--text-micro) text-muted-foreground">
|
||||
{relativeTime(operation.startedAt)}
|
||||
{operation.finishedAt && ` to ${relativeTime(operation.finishedAt)}`}
|
||||
</div>
|
||||
</div>
|
||||
{operation.command && (
|
||||
<div className="text-xs break-all">
|
||||
<span className="text-muted-foreground">Command: </span>
|
||||
<span className="font-mono">{operation.command}</span>
|
||||
</div>
|
||||
)}
|
||||
{operation.cwd && (
|
||||
<div className="text-xs break-all">
|
||||
<span className="text-muted-foreground">Working dir: </span>
|
||||
<span className="font-mono">{operation.cwd}</span>
|
||||
</div>
|
||||
)}
|
||||
{(asNonEmptyString(metadata?.branchName)
|
||||
|| asNonEmptyString(metadata?.baseRef)
|
||||
|| asNonEmptyString(metadata?.worktreePath)
|
||||
|| asNonEmptyString(metadata?.repoRoot)
|
||||
|| asNonEmptyString(metadata?.cleanupAction)) && (
|
||||
<div className="grid gap-1 text-xs sm:grid-cols-2">
|
||||
{asNonEmptyString(metadata?.branchName) && (
|
||||
<div><span className="text-muted-foreground">Branch: </span><span className="font-mono">{metadata?.branchName as string}</span></div>
|
||||
)}
|
||||
{asNonEmptyString(metadata?.baseRef) && (
|
||||
<div><span className="text-muted-foreground">Base ref: </span><span className="font-mono">{metadata?.baseRef as string}</span></div>
|
||||
)}
|
||||
{asNonEmptyString(metadata?.worktreePath) && (
|
||||
<div className="break-all"><span className="text-muted-foreground">Worktree: </span><span className="font-mono">{metadata?.worktreePath as string}</span></div>
|
||||
)}
|
||||
{asNonEmptyString(metadata?.repoRoot) && (
|
||||
<div className="break-all"><span className="text-muted-foreground">Repo root: </span><span className="font-mono">{metadata?.repoRoot as string}</span></div>
|
||||
)}
|
||||
{asNonEmptyString(metadata?.cleanupAction) && (
|
||||
<div><span className="text-muted-foreground">Cleanup: </span><span className="font-mono">{metadata?.cleanupAction as string}</span></div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
{typeof metadata?.created === "boolean" && (
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{metadata.created ? "Created by this run" : "Reused existing workspace"}
|
||||
</div>
|
||||
)}
|
||||
{operation.stderrExcerpt && operation.stderrExcerpt.trim() && (
|
||||
<div>
|
||||
<div className="mb-1 text-xs text-red-700 dark:text-red-300">stderr excerpt</div>
|
||||
<pre className="rounded-md bg-red-50 p-2 text-xs whitespace-pre-wrap break-all text-red-800 dark:bg-neutral-950 dark:text-red-100">
|
||||
{redactPathText(operation.stderrExcerpt, censorUsernameInLogs)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
{operation.stdoutExcerpt && operation.stdoutExcerpt.trim() && (
|
||||
<div>
|
||||
<div className="mb-1 text-xs text-muted-foreground">stdout excerpt</div>
|
||||
<pre className="rounded-md bg-neutral-100 p-2 text-xs whitespace-pre-wrap break-all dark:bg-neutral-950">
|
||||
{redactPathText(operation.stdoutExcerpt, censorUsernameInLogs)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
{operation.logRef && (
|
||||
<WorkspaceOperationLogViewer
|
||||
operation={operation}
|
||||
censorUsernameInLogs={censorUsernameInLogs}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -5,8 +5,7 @@ import type { HeartbeatRun, HeartbeatRunEvent, LiveEvent } from "@paperclipai/sh
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import { __liveUpdatesTestUtils, type CompanyLiveEventHandler } from "../context/LiveUpdatesProvider";
|
||||
import { LiveTranscriptInterests } from "../lib/live-transcript-interests";
|
||||
import { LogViewer } from "./AgentDetail";
|
||||
import { LogViewer as ProductionLogViewer } from "./AgentDetail.production";
|
||||
import { LogViewer } from "../components/run-log/LogViewer";
|
||||
|
||||
const { log, events, empty } = vi.hoisted(() => ({ log: vi.fn(), events: vi.fn(async (_runId?: string, _cursor?: number, _limit?: number, _options?: { signal?: AbortSignal }): Promise<import("@paperclipai/shared").HeartbeatRunEvent[]> => []), empty: [] }));
|
||||
vi.mock("../api/heartbeats", () => ({ heartbeatsApi: { log, events } }));
|
||||
@@ -25,7 +24,7 @@ vi.mock("../components/transcript/RunTranscriptView", () => ({
|
||||
(globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
afterEach(() => { vi.restoreAllMocks(); vi.unstubAllGlobals(); vi.useRealTimers(); log.mockReset(); events.mockReset(); events.mockResolvedValue([]); });
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("retains legacy history and reads only the next offset on visibility recovery (%#)", async (Viewer) => {
|
||||
it("retains legacy history and reads only the next offset on visibility recovery", async () => {
|
||||
const visibility = vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
const row = (seq: number, chunk: string) => JSON.stringify({ seq, ts: `2026-09-10T12:00:0${seq}Z`, stream: "stdout", chunk }) + "\n";
|
||||
const first = row(1, "retained history");
|
||||
@@ -36,7 +35,7 @@ it.each([LogViewer, ProductionLogViewer])("retains legacy history and reads only
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
try {
|
||||
await act(async () => root.render(<Viewer run={run} adapterType="codex_local" />));
|
||||
await act(async () => root.render(<LogViewer run={run} adapterType="codex_local" />));
|
||||
expect(container.textContent).toContain("retained history");
|
||||
await act(async () => {
|
||||
visibility.mockReturnValue("hidden");
|
||||
@@ -55,7 +54,7 @@ it.each([LogViewer, ProductionLogViewer])("retains legacy history and reads only
|
||||
await act(async () => complete({ content: second, nextOffset: first.length + second.length }));
|
||||
expect(container.textContent).toContain("retained history new output");
|
||||
log.mockResolvedValueOnce({ content: first, nextOffset: first.length });
|
||||
await act(async () => root.render(<Viewer run={{ ...run, id: "run-2" }} adapterType="codex_local" />));
|
||||
await act(async () => root.render(<LogViewer run={{ ...run, id: "run-2" }} adapterType="codex_local" />));
|
||||
expect(log).toHaveBeenLastCalledWith("run-2", 0, expect.any(Number), expect.objectContaining({ signal: expect.any(AbortSignal) }));
|
||||
expect(container.textContent).not.toContain("new output");
|
||||
} finally {
|
||||
@@ -64,7 +63,7 @@ it.each([LogViewer, ProductionLogViewer])("retains legacy history and reads only
|
||||
}
|
||||
});
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("uses shared streaming and retains fallback reads until negotiation (%#)", async (Viewer) => {
|
||||
it("uses shared streaming and retains fallback reads until negotiation", async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
const socket = vi.fn();
|
||||
@@ -77,7 +76,7 @@ it.each([LogViewer, ProductionLogViewer])("uses shared streaming and retains fal
|
||||
const subscription = { companyId: "company-1", transcripts, subscribe: () => () => {} };
|
||||
const { LiveEventSubscriptionContext } = __liveUpdatesTestUtils;
|
||||
try {
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={run} adapterType="codex_local" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={run} adapterType="codex_local" /></LiveEventSubscriptionContext.Provider>));
|
||||
const initialReads = log.mock.calls.length;
|
||||
await act(async () => vi.advanceTimersByTimeAsync(2000));
|
||||
expect(log.mock.calls.length).toBeGreaterThan(initialReads);
|
||||
@@ -98,7 +97,7 @@ it.each([LogViewer, ProductionLogViewer])("uses shared streaming and retains fal
|
||||
expect(messages.at(-1)?.runIds).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("re-enables cancelled history paging after hiding and showing a completed run (%#)", async Viewer => {
|
||||
it("re-enables cancelled history paging after hiding and showing a completed run", async () => {
|
||||
const visibility = vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
const rows = Array.from({ length: 1001 }, (_, index) => ({ id: index + 1, seq: index + 1,
|
||||
@@ -113,7 +112,7 @@ it.each([LogViewer, ProductionLogViewer])("re-enables cancelled history paging a
|
||||
const root = createRoot(container);
|
||||
const more = () => [...container.querySelectorAll("button")].find(button => /Load more events|Loading\.\.\./.test(button.textContent ?? ""));
|
||||
try {
|
||||
await act(async () => root.render(<Viewer run={run} adapterType="codex_local" />));
|
||||
await act(async () => root.render(<LogViewer run={run} adapterType="codex_local" />));
|
||||
await act(async () => more()!.click());
|
||||
expect(more()!.disabled).toBe(true);
|
||||
const cancelledSignal = events.mock.calls[1]![3]!.signal;
|
||||
@@ -132,7 +131,7 @@ it.each([LogViewer, ProductionLogViewer])("re-enables cancelled history paging a
|
||||
} finally { await act(async () => root.unmount()); }
|
||||
}, 30_000);
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("bounds automatic event history and pages further only on request (%#)", async Viewer => {
|
||||
it("bounds automatic event history and pages further only on request", async () => {
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
const rows = Array.from({ length: 2_001 }, (_, index) => ({ id: index + 1, seq: index + 1,
|
||||
@@ -144,7 +143,7 @@ it.each([LogViewer, ProductionLogViewer])("bounds automatic event history and pa
|
||||
const container = document.createElement("div");
|
||||
const root = createRoot(container);
|
||||
try {
|
||||
await act(async () => root.render(<Viewer run={run} adapterType="codex_local" />));
|
||||
await act(async () => root.render(<LogViewer run={run} adapterType="codex_local" />));
|
||||
expect(events).toHaveBeenCalledTimes(1);
|
||||
expect(container.textContent).toContain("invocation-retained-proof");
|
||||
const more = [...container.querySelectorAll("button")].find(button => button.textContent === "Load more events");
|
||||
@@ -161,7 +160,7 @@ it.each([LogViewer, ProductionLogViewer])("bounds automatic event history and pa
|
||||
} finally { await act(async () => root.unmount()); }
|
||||
}, 30_000);
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("preserves saved event details and catches up payload-free commit signals (%#)", async Viewer => {
|
||||
it("preserves saved event details and catches up payload-free commit signals", async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
@@ -186,7 +185,7 @@ it.each([LogViewer, ProductionLogViewer])("preserves saved event details and cat
|
||||
const root = createRoot(container);
|
||||
const { LiveEventSubscriptionContext } = __liveUpdatesTestUtils;
|
||||
try {
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => {
|
||||
transcripts.open({ send: text => messages.push(JSON.parse(text)) });
|
||||
transcripts.receive({ type: "live.capabilities", version: 1, capabilities: ["run-interests"] });
|
||||
@@ -214,13 +213,13 @@ it.each([LogViewer, ProductionLogViewer])("preserves saved event details and cat
|
||||
expect(container.textContent).toContain("saved event details");
|
||||
expect(container.textContent).toContain("live row 2205");
|
||||
expect(container.textContent).toContain("Events (2200)");
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={{ ...run, status: "succeeded" }} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={{ ...run, status: "succeeded" }} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
expect(container.textContent).toContain("saved event details");
|
||||
expect(container.textContent).toContain("live row 2205");
|
||||
} finally { await act(async () => root.unmount()); }
|
||||
}, 30_000);
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("merges saved and live rows with colliding database and run-sequence IDs (%#)", async Viewer => {
|
||||
it("merges saved and live rows with colliding database and run-sequence IDs", async () => {
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
const consoleError = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
@@ -240,7 +239,7 @@ it.each([LogViewer, ProductionLogViewer])("merges saved and live rows with colli
|
||||
for (const listener of listeners) listener(event);
|
||||
};
|
||||
try {
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => { emit(2, "live second row"); emit(3, "live third row"); });
|
||||
expect(container.textContent).toContain("saved first row");
|
||||
expect(container.textContent).toContain("live second row");
|
||||
@@ -250,9 +249,9 @@ it.each([LogViewer, ProductionLogViewer])("merges saved and live rows with colli
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ Viewer: LogViewer, stalled: "log" }, { Viewer: ProductionLogViewer, stalled: "log" },
|
||||
{ Viewer: LogViewer, stalled: "events" }, { Viewer: ProductionLogViewer, stalled: "events" },
|
||||
])("finishes a slow $stalled page while the other reader catches up (%#)", async ({ Viewer, stalled }) => {
|
||||
{ stalled: "log" },
|
||||
{ stalled: "events" },
|
||||
])("finishes a slow $stalled page while the other reader catches up", async ({ stalled }) => {
|
||||
vi.useFakeTimers();
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "" });
|
||||
@@ -277,7 +276,7 @@ it.each([
|
||||
let finishEvents!: (value: HeartbeatRunEvent[]) => void;
|
||||
const held = stalled === "log" ? log : events;
|
||||
try {
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => {
|
||||
transcripts.open({ send: text => messages.push(JSON.parse(text)) });
|
||||
transcripts.receive({ type: "live.capabilities", version: 1, capabilities: ["run-interests"] });
|
||||
@@ -312,7 +311,7 @@ it.each([
|
||||
} finally { await act(async () => root.unmount()); }
|
||||
});
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("shows and retries an initial event-history failure instead of claiming empty history (%#)", async Viewer => {
|
||||
it("shows and retries an initial event-history failure instead of claiming empty history", async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
@@ -321,7 +320,7 @@ it.each([LogViewer, ProductionLogViewer])("shows and retries an initial event-hi
|
||||
const container = document.createElement("div");
|
||||
const root = createRoot(container);
|
||||
try {
|
||||
await act(async () => root.render(<Viewer run={run} adapterType="paperclip_runner" />));
|
||||
await act(async () => root.render(<LogViewer run={run} adapterType="paperclip_runner" />));
|
||||
expect(container.textContent).toContain("Could not load events. Try again.");
|
||||
expect(container.textContent).not.toContain("No log events.");
|
||||
await act(async () => vi.advanceTimersByTimeAsync(2000));
|
||||
@@ -341,7 +340,7 @@ it.each([LogViewer, ProductionLogViewer])("shows and retries an initial event-hi
|
||||
});
|
||||
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("recovers a paginated live tail and preserves pages explicitly loaded afterward (%#)", async Viewer => {
|
||||
it("recovers a paginated live tail and preserves pages explicitly loaded afterward", async () => {
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
const run = { id: "run-1", companyId: "company-1", agentId: "agent-1", status: "running" } as HeartbeatRun;
|
||||
@@ -356,7 +355,7 @@ it.each([LogViewer, ProductionLogViewer])("recovers a paginated live tail and pr
|
||||
subscribe(fn: CompanyLiveEventHandler) { listeners.add(fn); return () => { listeners.delete(fn); }; } };
|
||||
const container = document.createElement("div"), root = createRoot(container);
|
||||
const { LiveEventSubscriptionContext } = __liveUpdatesTestUtils;
|
||||
const render = async (status = "running") => act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={{ ...run, status } as HeartbeatRun} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
const render = async (status = "running") => act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={{ ...run, status } as HeartbeatRun} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
const emit = (row: HeartbeatRunEvent) => { for (const listener of listeners) listener({ id: row.seq,
|
||||
companyId: run.companyId, type: "heartbeat.run.event", createdAt: row.createdAt.toISOString(), payload: { ...row } }); };
|
||||
try {
|
||||
@@ -389,7 +388,7 @@ it.each([LogViewer, ProductionLogViewer])("recovers a paginated live tail and pr
|
||||
}, 60_000);
|
||||
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer].flatMap(Viewer => ["completed-return", "catchup-retry"].map(mode => ({ Viewer, mode }))))("preserves event read mode for $mode (%#)", async ({ Viewer, mode }) => {
|
||||
it.each(["completed-return", "catchup-retry"].map(mode => ({ mode })))("preserves event read mode for $mode", async ({ mode }) => {
|
||||
vi.useFakeTimers();
|
||||
const visibility = vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
@@ -409,7 +408,7 @@ it.each([LogViewer, ProductionLogViewer].flatMap(Viewer => ["completed-return",
|
||||
const container = document.createElement("div"), root = createRoot(container);
|
||||
const { LiveEventSubscriptionContext } = __liveUpdatesTestUtils;
|
||||
try {
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
expect(events).toHaveBeenCalledTimes(1);
|
||||
if (mode === "completed-return") {
|
||||
await act(async () => { visibility.mockReturnValue("hidden"); document.dispatchEvent(new Event("visibilitychange")); });
|
||||
@@ -435,7 +434,7 @@ it.each([LogViewer, ProductionLogViewer].flatMap(Viewer => ["completed-return",
|
||||
} finally { await act(async () => root.unmount()); }
|
||||
}, 60_000);
|
||||
|
||||
it.each([LogViewer, ProductionLogViewer])("recovers the latest saved tail when acknowledgement aborts the initial history read (%#)", async Viewer => {
|
||||
it("recovers the latest saved tail when acknowledgement aborts the initial history read", async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.spyOn(document, "visibilityState", "get").mockReturnValue("visible");
|
||||
log.mockResolvedValue({ content: "", nextOffset: 0 });
|
||||
@@ -453,7 +452,7 @@ it.each([LogViewer, ProductionLogViewer])("recovers the latest saved tail when a
|
||||
const container = document.createElement("div"), root = createRoot(container);
|
||||
const { LiveEventSubscriptionContext } = __liveUpdatesTestUtils;
|
||||
try {
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><Viewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
await act(async () => root.render(<LiveEventSubscriptionContext.Provider value={subscription}><LogViewer run={run} adapterType="paperclip_runner" /></LiveEventSubscriptionContext.Provider>));
|
||||
const initialSignal = events.mock.calls[0]![3]!.signal!;
|
||||
await act(async () => {
|
||||
transcripts.open({ send: text => sent.push(JSON.parse(text)) });
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+11
-1107
File diff suppressed because it is too large.
Load diff
+3
-3
@@ -2,9 +2,9 @@ import { configDefaults, defineConfig } from "vitest/config";
|
||||
|
||||
export default defineConfig({
|
||||
test: {
|
||||
...(process.env.PAPERCLIP_CI_PROFILE === "true"
|
||||
? { reporters: [...configDefaults.reporters, "./scripts/ci-profile/vitest-reporter.mjs"] }
|
||||
: {}),
|
||||
reporters: [...configDefaults.reporters,
|
||||
...(process.env.GITHUB_ACTIONS === "true" ? ["./scripts/ci/test-timing-reporter.mjs"] : []),
|
||||
...(process.env.PAPERCLIP_CI_PROFILE === "true" ? ["./scripts/ci-profile/vitest-reporter.mjs"] : [])],
|
||||
projects: [
|
||||
"packages/shared",
|
||||
"packages/skills-catalog",
|
||||
|
||||
Reference in new issue
Block a user