mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 11:01:16 +01:00
fix(native): publish fresh retry sessions for later turns (#18713)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native tasks save a provider session for later comments and monitor checks. > - The goal projection also records the run that owns the session event sequence. > - A fresh retry uses a new native session, but the projection accepted only the old session. > - The accepted checkpoint then rejected session publication because the goal source still named the old run. > - This pull request lets the current task executor advance that source after a terminal predecessor with no unfinished goal. > - Later turns can then use the fresh retry's saved session. ## Linked Issues or Issue Description Related: #12669, #18343. I searched open and closed PRs for source epochs and native task-session publication. I found no duplicate repair. **What happened?** A successful fresh native retry could retain its predecessor's task-session pointer. A later comment or monitor selected the old provider session. The goal source transition required both runs to have the same native session. The terminal checkpoint then saved a `suppressed_newer` publication result. **Expected behavior** The current task executor can publish its accepted fresh session. A later turn resumes that session. Delayed events cannot restore the old source or revive a cleared goal. **Steps to reproduce** 1. Complete a native turn and retain its task-session row. 2. Admit a fresh session on the same task without deleting the row. 3. Complete that turn, then send an ordinary follow-up. 4. Before this change, the saved pointer can still name the first session. **Paperclip version or commit** Reproduced on source commit `36df8933f0` with a PostgreSQL regression before the production change. **Deployment mode** Source checkout. The affected producer runs for local and remote native sessions. ## What Changed - Permit a different native session to advance the goal source only when its run holds the task execution lock and assignee, and the task is in progress. - Keep the terminal predecessor, native runtime, runner identity, absent or completed goal, source sequence, and conversation reset checks. - Test accepted session publication, a fresh heartbeat followed by an ordinary resumed turn, stale events, current-owner loss, and unfinished or unreadable goals. - Add an opt-in live Codex file-persistence check. Update execution documentation and both recovery maps. ## Verification - Red: the new PostgreSQL publication regression failed before the producer change. The saved session, display ID and goal source still named the predecessor. - Green: 72 distinct tests passed across the final focused runs of `runner-goals.test.ts`, `native-task-session-publication.test.ts`, `native-session-resumption.test.ts`, and `remote-warm-session-recovery.test.ts`. - The assembled heartbeat test admits three turns, opens two provider sessions, resumes the second session once, and stores three results. It uses a deterministic provider backend. - `node scripts/check-maps.mjs` and `git diff --check` passed. - Real local Codex acceptance passed on macOS arm64 with `gpt-6.1-sol` and explicit warm mode. Three heartbeat turns saved three exact file lines. The fresh second turn published its new normalized and provider session IDs. The ordinary third turn retained both IDs. The fixture closed its warm owners and removed its private credential copy and workspace. The production code was at `450e15e`; the final test-only commit selects explicit warm mode and uses the existing owned-home cleanup helper. The production change is identical in the final head. - The earlier per-turn fixture preserved all three file steps and selected the correct normalized session, but provider-thread reuse failed. `run.attach` reported that the Codex provider had pending events; the safe fallback used another provider thread. This remains a separate failure. The first fixture cleanup also failed on immutable instruction assets. The owned-home helper fixed cleanup, and the second per-turn attempt exposed the provider identity assertion directly. - Full current-head CI passes, including typecheck, build, general and serialized tests, Rust/Runner checks, all eight E2E shards, and the `ci / e2e` and `ci / verify` aggregate gates. Greptile is 5/5 on the current commit, with no unresolved review threads. Broad local workspace checks were not duplicated; the focused regressions and actual provider acceptance above were run locally. No merge, deployment, or recovery of an original task is included. ## Risks - Source epoch changes affect saved goal state. The fresh-session path requires current task authority and refuses unfinished or unreadable saved goals. - The change starts no provider work and changes no prior-owner recovery guard. Unknown external outcomes retain their reconciliation hold. - A prior session remains held when it lacks the required authority or state proof. - Per-turn provider-thread reuse remains an open coverage gap described above. This change does not alter that attachment guard. ## Model Used OpenAI Codex, GPT-6 family. The exact model ID and context window are unavailable in this session. The agent used code inspection, tools, and executable tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
89e3c23062
commit
db8a273562
9 files changed
+231
-7
No files matched your search
@@ -3018,8 +3018,22 @@ identity mismatches.
|
||||
Native task-session publication is part of the terminal checkpoint transaction,
|
||||
not resource cleanup. `native-task-session-publication.ts` owns its accepted-result,
|
||||
owner, task-epoch and reset checks; heartbeat supplies prepared session metadata.
|
||||
The goal projection also advances its source epoch when a fresh native session
|
||||
holds the task's current execution lock and assignee. The predecessor must be
|
||||
terminal, and its saved goal must be absent or complete. This lets the accepted
|
||||
checkpoint publish the fresh session as the next continuation baseline. A
|
||||
different session without current task authority, an unfinished or unreadable
|
||||
goal, a reset conversation, and delayed predecessor events retain their fences.
|
||||
The goal-less regression in `native-session-resumption.test.ts` holds provider
|
||||
cleanup open and then exercises a normal follow-up and an explicit session reset.
|
||||
Its fresh-session case keeps the prior task-session row, opens one new provider
|
||||
session, and verifies that the following ordinary comment resumes that session.
|
||||
The opt-in `native-session-epoch.live.test.ts` runs the same path with an existing
|
||||
Codex login and a built native Runner. It uses warm mode, makes three model calls,
|
||||
checks exact file contents and provider-thread reuse, and removes its temporary
|
||||
credential copy and workspace. Run it with
|
||||
`PAPERCLIP_LIVE_SESSION_EPOCH=true pnpm exec vitest run server/src/services/native-runtime/native-session-epoch.live.test.ts`.
|
||||
`PAPERCLIP_LIVE_EVIDENCE_PATH` can select a JSON evidence file outside the fixture.
|
||||
Control-port tests also cover terminal snapshots before accepted results and
|
||||
result-replay contention with the status writer's coordinator-first lock order.
|
||||
Remote adoption tests must invoke actual external-state preparation. Inspecting
|
||||
|
||||
@@ -2438,6 +2438,14 @@ configuration, and controller-owned credential or instruction resources use
|
||||
verified suspension and saved-history resume. This recovery can restart a
|
||||
provider; it does not suspend every healthy warm turn.
|
||||
|
||||
A fresh native session admitted for the current task may replace a terminal
|
||||
predecessor's goal source epoch when the saved goal is absent or complete.
|
||||
It must hold the task's execution lock and assignee. Its accepted terminal
|
||||
checkpoint can then publish the new session for later comments or monitors.
|
||||
An unfinished or unreadable goal retains its prior epoch. Delayed predecessor
|
||||
events, stale source sequences, conversation resets and unverified outcomes
|
||||
retain their existing guards. Source projection does not start provider work.
|
||||
|
||||
Changing configuration can retire a completed warm runner during instruction
|
||||
preparation, before the next native coordinator is claimed. Its maintenance
|
||||
checkpoints may use the next run's live preparation lease from the same
|
||||
|
||||
@@ -4,6 +4,11 @@ Product map: [Recovery](../feature-map/recovery.md).
|
||||
|
||||
Feature ID: `recovery`.
|
||||
|
||||
Fresh native retry session publication remains server-owned. Goal source
|
||||
replacement requires the current task execution lock and assignee, a terminal
|
||||
predecessor, and an absent or completed saved goal. It grants no agent API or
|
||||
permission to start another executor or replay an unknown external outcome.
|
||||
|
||||
Native finalization deadline reconciliation records unfinished workspace
|
||||
preservation through the existing server-owned recovery path. It grants no
|
||||
additional agent API, provider-stop authority or permission to replay work.
|
||||
|
||||
@@ -36,6 +36,10 @@ Agent access: [companion](../agent-access-map/recovery.md).
|
||||
Close still requires confirmed provider exit, durable event drain, and exact
|
||||
runner suspension; the cleanup purpose alone cannot certify termination.
|
||||
Hosted provider settlement remains separate acceptance.
|
||||
An authorized fresh native retry publishes its own session for the next
|
||||
ordinary comment or monitor. The [heartbeat regression](../../server/src/__tests__/native-session-resumption.test.ts)
|
||||
covers fresh admission, saved session selection, and one subsequent resume.
|
||||
Unfinished goals and unverified prior outcomes retain their recovery gates.
|
||||
- `source-action`: `recovery/source-action` — source-scoped recovery actions remain attached to the task;
|
||||
independent repair work may have its own issue and return owner.
|
||||
- `exhaustion`: `recovery/exhaustion` — an exhausted infrastructure attempt offers ordinary Retry.
|
||||
|
||||
@@ -1534,7 +1534,7 @@ describe("goal-less native task-session publication", () => {
|
||||
await database?.cleanup();
|
||||
});
|
||||
|
||||
it.each(["retained", "reset"] as const)("publishes the completed task session before provider cleanup and resumes an ordinary comment (%s)", async mode => {
|
||||
it.each(["retained", "reset", "fresh"] as const)("publishes the completed task session before provider cleanup and resumes an ordinary comment (%s)", async mode => {
|
||||
const home = createIsolatedTestHome();
|
||||
const previousHome = process.env.PAPERCLIP_HOME;
|
||||
process.env.PAPERCLIP_HOME = home.home;
|
||||
@@ -1643,7 +1643,8 @@ describe("goal-less native task-session publication", () => {
|
||||
const comment = await issuesSvc.addComment(issueId, "Continue using the saved artifact.", { userId: "pointer-user" });
|
||||
const second = await heartbeat.wakeup(agentId, { source: "automation", triggerDetail: "system", reason: "issue_reopened_via_comment",
|
||||
requestedByActorType: "user", requestedByActorId: "pointer-user", payload: { issueId, commentId: comment.id, mutation: "comment", reopenedFrom: "done" },
|
||||
contextSnapshot: { issueId, taskId: issueId, wakeCommentId: comment.id, reopenedFrom: "done", skipIssueComment: true } });
|
||||
contextSnapshot: { issueId, taskId: issueId, wakeCommentId: comment.id, reopenedFrom: "done", skipIssueComment: true,
|
||||
...(mode === "fresh" ? { forceFreshSession: true } : {}) } });
|
||||
expect(second).not.toBeNull();
|
||||
await heartbeat.drainActiveRunExecutions();
|
||||
const after = await heartbeat.getRun(second!.id);
|
||||
@@ -1664,6 +1665,27 @@ describe("goal-less native task-session publication", () => {
|
||||
}
|
||||
expect(startTurn).toHaveBeenCalledTimes(2);
|
||||
expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, second!.id))).toHaveLength(1);
|
||||
if (mode === "fresh") {
|
||||
expect(await heartbeat.listTaskSessions(agentId)).toEqual([expect.objectContaining({
|
||||
lastRunId: second!.id, sessionParamsJson: expect.objectContaining({ sessionId: after!.nativeSessionId }),
|
||||
sessionDisplayId: after!.sessionIdAfter,
|
||||
})]);
|
||||
await issuesSvc.update(issueId, { status: "todo", actorUserId: "pointer-user" });
|
||||
const followup = await issuesSvc.addComment(issueId, "Continue the fresh session's saved work.", { userId: "pointer-user" });
|
||||
const third = await heartbeat.wakeup(agentId, { source: "automation", triggerDetail: "system", reason: "issue_reopened_via_comment",
|
||||
requestedByActorType: "user", requestedByActorId: "pointer-user",
|
||||
payload: { issueId, commentId: followup.id, mutation: "comment", reopenedFrom: "done" },
|
||||
contextSnapshot: { issueId, taskId: issueId, wakeCommentId: followup.id, reopenedFrom: "done", skipIssueComment: true } });
|
||||
expect(third).not.toBeNull();
|
||||
await heartbeat.drainActiveRunExecutions();
|
||||
expect(await heartbeat.getRun(third!.id)).toMatchObject({ status: "succeeded",
|
||||
nativeSessionId: after!.nativeSessionId, sessionIdAfter: after!.sessionIdAfter });
|
||||
expect(inputs).toHaveLength(3);
|
||||
expect(openSession).toHaveBeenCalledTimes(2);
|
||||
expect(recoverSession).toHaveBeenCalledOnce();
|
||||
expect(startTurn).toHaveBeenCalledTimes(3);
|
||||
expect(await db.select().from(nativeRunResults).where(inArray(nativeRunResults.runId, [first!.id, second!.id, third!.id]))).toHaveLength(3);
|
||||
}
|
||||
} finally {
|
||||
releaseClose();
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { copyFile, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { homedir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { agents, agentTaskSessions, companies, heartbeatRuns, issues } from "@paperclipai/db";
|
||||
import { expect, it } from "vitest";
|
||||
import { startRunnerApiTestServer } from "../../__tests__/helpers/runner-api-server.js";
|
||||
import { createIsolatedTestHome } from "../../__tests__/helpers/isolated-test-home.js";
|
||||
import { heartbeatService } from "../heartbeat.js";
|
||||
import { issueService } from "../issues.js";
|
||||
import { instanceSettingsService } from "../instance-settings.js";
|
||||
import { closeIdleWarmNativeSessionsForRestart } from "./native-session-executor.js";
|
||||
|
||||
it.skipIf(process.env.PAPERCLIP_LIVE_SESSION_EPOCH !== "true")("publishes a fresh live Codex session and resumes its saved file", async () => {
|
||||
const home = createIsolatedTestHome(), root = home.home;
|
||||
const lifecycleMode = process.env.PAPERCLIP_LIVE_SESSION_LIFECYCLE === "per_turn" ? "per_turn" : "warm";
|
||||
const keys = ["PAPERCLIP_HOME", "CODEX_HOME", "PAPERCLIP_API_URL", "PAPERCLIP_AGENT_JWT_SECRET", "PAPERCLIP_TELEMETRY_DISABLED"];
|
||||
const saved = Object.fromEntries(keys.map(key => [key, process.env[key]]));
|
||||
let server: Awaited<ReturnType<typeof startRunnerApiTestServer>> | undefined;
|
||||
let heartbeat: ReturnType<typeof heartbeatService> | undefined;
|
||||
try {
|
||||
const codexHome = join(root, "codex");
|
||||
await mkdir(codexHome, { mode: 0o700 });
|
||||
await copyFile(join(process.env.PAPERCLIP_LIVE_CODEX_HOME ?? join(homedir(), ".codex"), "auth.json"), join(codexHome, "auth.json"));
|
||||
await writeFile(join(codexHome, "config.toml"), 'model_reasoning_effort = "low"\n');
|
||||
process.env.CODEX_HOME = codexHome;
|
||||
process.env.PAPERCLIP_HOME = join(root, "instance");
|
||||
process.env.PAPERCLIP_AGENT_JWT_SECRET = randomUUID();
|
||||
process.env.PAPERCLIP_TELEMETRY_DISABLED = "1";
|
||||
server = await startRunnerApiTestServer({ deploymentMode: "local_trusted" });
|
||||
process.env.PAPERCLIP_API_URL = server.apiUrl;
|
||||
const fixture = await server.fixture({ connectionScenario: "fresh" });
|
||||
const db = server.db;
|
||||
await db.update(issues).set({ executionRunId: null, executionWorkspaceSettings: { mode: "shared_workspace" },
|
||||
title: "Preserve one verification file across native sessions",
|
||||
description: "Follow the latest comment's single file step. The deliverable is epoch.txt in this workspace. Read the file to verify the step. Complete this step with paperclip_finish, reportedWorkDisposition done, and file evidence. Do not do later steps or repeat earlier steps.",
|
||||
}).where(eq(issues.id, fixture.issueId));
|
||||
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, fixture.runId));
|
||||
await db.update(companies).set({ defaultResponsibleUserId: fixture.userId }).where(eq(companies.id, fixture.companyId));
|
||||
await db.update(agents).set({ adapterConfig: { provider: "codex", model: "gpt-6.1-sol", modelReasoningEffort: "low",
|
||||
cwd: fixture.workspace, workspaceStrategy: { type: "project_primary" }, env: { CODEX_HOME: codexHome }, timeoutSec: 180,
|
||||
lifecycleMode, idleTimeoutMs: 900_000 },
|
||||
runtimeConfig: { heartbeat: { enabled: false, wakeOnDemand: true, maxConcurrentRuns: 1 } },
|
||||
}).where(eq(agents.id, fixture.agentId));
|
||||
await instanceSettingsService(db).updateExperimental({ enableNativeRunner: true, enableIsolatedWorkspaces: true });
|
||||
heartbeat = heartbeatService(db);
|
||||
const service = issueService(db), markers = [randomUUID(), randomUUID(), randomUUID()];
|
||||
const completed: Array<{ runId: string; sessionId: string; providerSessionId: string; publication: unknown }> = [];
|
||||
for (const [index, marker] of markers.entries()) {
|
||||
await service.update(fixture.issueId, { status: "todo", actorUserId: fixture.userId! });
|
||||
const comment = await service.addComment(fixture.issueId,
|
||||
index === 0 ? `Write exactly ${marker} followed by a newline to epoch.txt. Read it and finish this step.`
|
||||
: `Append exactly ${marker} followed by a newline once to epoch.txt. Preserve all existing lines. Read the result and finish this step.`,
|
||||
{ userId: fixture.userId! });
|
||||
const run = await heartbeat.wakeup(fixture.agentId, { source: "automation", triggerDetail: "system", reason: "issue_reopened_via_comment",
|
||||
requestedByActorType: "user", requestedByActorId: fixture.userId,
|
||||
payload: { issueId: fixture.issueId, commentId: comment.id, mutation: "comment", reopenedFrom: "done" },
|
||||
contextSnapshot: { issueId: fixture.issueId, taskId: fixture.issueId, wakeCommentId: comment.id, skipIssueComment: true,
|
||||
...(index === 1 ? { forceFreshSession: true } : {}) },
|
||||
});
|
||||
expect(run?.id).toBeTruthy();
|
||||
await heartbeat.drainActiveRunExecutions();
|
||||
const after = await heartbeat.getRun(run!.id);
|
||||
console.log(JSON.stringify({ stage: index, status: after?.status, errorCode: after?.errorCode, error: after?.error }));
|
||||
expect(after?.status).toBe("succeeded");
|
||||
const [session] = await db.select().from(agentTaskSessions).where(eq(agentTaskSessions.taskKey, fixture.issueId));
|
||||
expect(session.lastRunId).toBe(run!.id);
|
||||
expect(session.sessionParamsJson?.sessionId).toBe(after!.nativeSessionId);
|
||||
expect(await readFile(join(fixture.workspace, "epoch.txt"), "utf8")).toBe(markers.slice(0, index + 1).join("\n") + "\n");
|
||||
completed.push({ runId: run!.id, sessionId: after!.nativeSessionId!, providerSessionId: after!.sessionIdAfter!,
|
||||
publication: after!.runnerProfileJson?.nativeTaskSessionPublication });
|
||||
}
|
||||
console.log(JSON.stringify({ observedSessions: completed }));
|
||||
expect(completed[1].sessionId).not.toBe(completed[0].sessionId);
|
||||
expect(completed[1].providerSessionId).not.toBe(completed[0].providerSessionId);
|
||||
expect(completed[2].sessionId).toBe(completed[1].sessionId);
|
||||
expect(completed[2].providerSessionId).toBe(completed[1].providerSessionId);
|
||||
const evidence = { passed: true, model: "gpt-6.1-sol", lifecycleMode, provider: "real local Codex login", platform: `${process.platform}-${process.arch}`,
|
||||
turns: 3, freshSessionPublished: true, ordinaryFollowupResumedFreshSession: true, exactFileLines: 3, completed };
|
||||
if (process.env.PAPERCLIP_LIVE_EVIDENCE_PATH) await writeFile(process.env.PAPERCLIP_LIVE_EVIDENCE_PATH, JSON.stringify(evidence, null, 2) + "\n");
|
||||
console.log(JSON.stringify(evidence));
|
||||
} finally {
|
||||
await heartbeat?.drainActiveRunExecutions();
|
||||
await closeIdleWarmNativeSessionsForRestart();
|
||||
await server?.close();
|
||||
for (const key of keys) { if (saved[key] === undefined) delete process.env[key]; else process.env[key] = saved[key]; }
|
||||
home.cleanup();
|
||||
}
|
||||
}, 600_000);
|
||||
@@ -12,6 +12,7 @@ import { resolveExternalTestDatabaseUrl } from "../../__tests__/helpers/external
|
||||
import { startRecoveryPostgres } from "../../__tests__/helpers/recovery-postgres.js";
|
||||
import { redactRegisteredSecretValues } from "../run-secret-redaction.js";
|
||||
import { nativeSha256 } from "./canonical.js";
|
||||
import { applyRunnerGoalPrpEvent } from "../runner-goals.js";
|
||||
import { PaperclipControlPlanePort, type PaperclipControlPlaneBinding } from "./paperclip-control-plane-port.js";
|
||||
import { nativeTaskSessionPublicationHandled, publishNativeTaskSessionInTransaction, NATIVE_TASK_SESSION_PUBLICATION_KEY,
|
||||
type NativeTaskSessionPublicationContext } from "./native-task-session-publication.js";
|
||||
@@ -115,6 +116,30 @@ describe("accepted native task session publication", () => {
|
||||
expect(repeated.filter(query => /^update /i.test(query.trim()))).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("publishes a fresh authorized retry after a different provider session owned the goal epoch", async () => {
|
||||
const f = await fixture();
|
||||
const priorRunId = randomUUID(), priorSessionId = randomUUID(), priorRunnerId = randomUUID();
|
||||
await db.insert(heartbeatRuns).values({ id: priorRunId, companyId: f.binding.companyId,
|
||||
agentId: f.binding.agentId, nativeIssueId: f.binding.issueId, nativeSessionId: priorSessionId,
|
||||
runnerInstanceId: priorRunnerId, runtimeMode: "native", status: "failed",
|
||||
createdAt: new Date(Date.now() - 10_000), finishedAt: new Date(Date.now() - 5_000) });
|
||||
await db.update(agentTaskSessions).set({ lastRunId: priorRunId,
|
||||
sessionParamsJson: { sessionId: priorSessionId }, sessionDisplayId: "prior-provider-thread",
|
||||
goalSourceId: `${priorRunnerId}:${priorRunId}`, goalSourceCursor: 1_000,
|
||||
}).where(eq(agentTaskSessions.taskKey, f.binding.issueId));
|
||||
|
||||
await applyRunnerGoalPrpEvent(db, { ...f.binding, adapterType: f.context.adapterType }, {
|
||||
eventType: "session.goal.snapshot", sourceInstanceId: f.binding.sourceInstanceId,
|
||||
sourceRunId: f.binding.runId, sourceSeq: 1, payload: { goal: null, workingNow: false },
|
||||
});
|
||||
await f.port.checkpointSession(f.snapshot);
|
||||
const { run, session } = await f.read();
|
||||
expect(session).toMatchObject({ lastRunId: f.binding.runId,
|
||||
sessionParamsJson: { ...f.context.sessionParamsJson, sessionId: f.binding.sessionId },
|
||||
sessionDisplayId: "provider-thread", goalSourceId: `${f.binding.sourceInstanceId}:${f.binding.runId}` });
|
||||
expect(run.runnerProfileJson?.nativeTaskSessionPublication).toMatchObject({ outcome: "published" });
|
||||
});
|
||||
|
||||
it("persists a pre-acceptance terminal observation and publishes only at the existing accepted checkpoint", async () => {
|
||||
const f = await fixture("runner", false);
|
||||
// Driver event/startup checkpoints can expose a semantic terminal observation
|
||||
|
||||
@@ -409,6 +409,60 @@ describeEmbeddedPostgres("runner goal service", () => {
|
||||
expect(session).toMatchObject({ goalSourceId: `${runnerId}:${nextRunId}`, goalSourceCursor: 2 });
|
||||
});
|
||||
|
||||
it.each(["current", "current_complete", "not_current", "reassigned", "blocked", "prior_running", "next_finished", "foreign_runner", "missing_session", "unfinished_goal", "unreadable_goal", "conversation_reset"] as const)("fences a fresh goal source by current execution authority (%s)", async kind => {
|
||||
const binding = await seed(), priorRunId = randomUUID(), nextRunId = randomUUID();
|
||||
const priorRunnerId = randomUUID(), nextRunnerId = randomUUID();
|
||||
const priorSessionId = randomUUID(), nextSessionId = randomUUID();
|
||||
const eventBinding = { ...binding, adapterType: "paperclip_runner" };
|
||||
await db.insert(heartbeatRuns).values([
|
||||
{ id: priorRunId, ...binding, nativeIssueId: binding.issueId, runtimeMode: "native", status: "failed",
|
||||
runnerInstanceId: priorRunnerId, nativeSessionId: priorSessionId },
|
||||
{ id: nextRunId, ...binding, nativeIssueId: binding.issueId, runtimeMode: "native", status: "running",
|
||||
runnerInstanceId: nextRunnerId, nativeSessionId: nextSessionId },
|
||||
]);
|
||||
await db.update(issues).set({ executionRunId: nextRunId }).where(eq(issues.id, binding.issueId));
|
||||
const priorEvent = { eventType: "session.goal.updated", sourceInstanceId: priorRunnerId,
|
||||
sourceRunId: priorRunId, sourceSeq: 20,
|
||||
payload: { goal: { objective: "Retained goal", status: kind === "current_complete" ? "complete" : "active" } } };
|
||||
await applyRunnerGoalPrpEvent(db, eventBinding, priorEvent);
|
||||
if (kind !== "unfinished_goal" && kind !== "current_complete") await applyRunnerGoalPrpEvent(db, eventBinding, {
|
||||
...priorEvent, eventType: "session.goal.cleared", sourceSeq: 21, payload: { goal: null },
|
||||
});
|
||||
if (kind === "not_current") await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, binding.issueId));
|
||||
if (kind === "reassigned") await db.update(issues).set({ assigneeAgentId: null }).where(eq(issues.id, binding.issueId));
|
||||
if (kind === "blocked") await db.update(issues).set({ status: "blocked" }).where(eq(issues.id, binding.issueId));
|
||||
if (kind === "prior_running") await db.update(heartbeatRuns).set({ status: "running" }).where(eq(heartbeatRuns.id, priorRunId));
|
||||
if (kind === "next_finished") await db.update(heartbeatRuns).set({ status: "succeeded" }).where(eq(heartbeatRuns.id, nextRunId));
|
||||
if (kind === "foreign_runner") await db.update(heartbeatRuns).set({ runnerInstanceId: randomUUID() }).where(eq(heartbeatRuns.id, nextRunId));
|
||||
if (kind === "missing_session") await db.update(heartbeatRuns).set({ nativeSessionId: null }).where(eq(heartbeatRuns.id, nextRunId));
|
||||
if (kind === "unreadable_goal") await db.update(agentTaskSessions).set({ goalJson: { legacy: "unknown" } }).where(eq(agentTaskSessions.taskKey, binding.issueId));
|
||||
if (kind === "conversation_reset") await db.update(issues).set({ conversationAgentId: binding.agentId,
|
||||
conversationUserId: "goal-reset-user", conversationState: "active", conversationSessionGeneration: 2,
|
||||
}).where(eq(issues.id, binding.issueId));
|
||||
const nextEvent = { eventType: "session.goal.updated", sourceInstanceId: nextRunnerId,
|
||||
sourceRunId: nextRunId, sourceSeq: 1,
|
||||
payload: { goal: { objective: "Current fresh session", status: "active" } } };
|
||||
const changed = await applyRunnerGoalPrpEvent(db, eventBinding, nextEvent);
|
||||
const [session] = await db.select().from(agentTaskSessions).where(eq(agentTaskSessions.taskKey, binding.issueId));
|
||||
if (kind !== "current" && kind !== "current_complete") {
|
||||
expect(changed).toBeNull();
|
||||
expect(session.goalSourceId).toBe(`${priorRunnerId}:${priorRunId}`);
|
||||
expect(session.goalSourceCursor).toBe(kind === "unfinished_goal" ? 20 : 21);
|
||||
return;
|
||||
}
|
||||
expect(changed?.goal).toMatchObject({ objective: "Current fresh session", status: "active" });
|
||||
expect(session).toMatchObject({ goalSourceId: `${nextRunnerId}:${nextRunId}`, goalSourceCursor: 1 });
|
||||
await applyRunnerGoalPrpEvent(db, eventBinding, { ...nextEvent, eventType: "session.goal.cleared",
|
||||
sourceSeq: 2, payload: { goal: null } });
|
||||
await db.update(heartbeatRuns).set({ status: "succeeded" }).where(eq(heartbeatRuns.id, nextRunId));
|
||||
for (const stale of [{ ...priorEvent, sourceSeq: 1_000 }, nextEvent,
|
||||
{ eventType: "session.goal.updated", sourceSeq: 2_000, payload: priorEvent.payload }]) {
|
||||
expect(await applyRunnerGoalPrpEvent(db, eventBinding, stale)).toBeNull();
|
||||
}
|
||||
expect(await runnerGoalService(db).projection(binding.companyId, binding.issueId)).toMatchObject({ goal: null });
|
||||
expect(await db.select().from(agentSessionGoalActions)).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("keeps a missing resumed goal blocked across restored empty snapshots", async () => {
|
||||
const binding = await seed();
|
||||
const eventBinding = { ...binding, adapterType: "paperclip_runner" };
|
||||
|
||||
@@ -687,10 +687,6 @@ export async function applyRunnerGoalPrpEvent(
|
||||
: undefined;
|
||||
const sameSource = !sourceId || session?.goalSourceId === sourceId;
|
||||
if (session?.goalSourceId && sourceId !== session.goalSourceId) {
|
||||
// goalSourceId is the persisted run epoch, not just a sequence namespace.
|
||||
// Only a running, authenticated successor of the same native session may
|
||||
// replace it. A delayed terminal predecessor must never reclaim the epoch
|
||||
// or resurrect a clear tombstone, even with a higher source sequence.
|
||||
const priorRunId = session.goalSourceId.split(":").at(-1);
|
||||
const nextRunId = event.sourceRunId;
|
||||
const isUuid = (value: string | undefined): value is string =>
|
||||
@@ -701,6 +697,7 @@ export async function applyRunnerGoalPrpEvent(
|
||||
status: heartbeatRuns.status,
|
||||
nativeSessionId: heartbeatRuns.nativeSessionId,
|
||||
runnerInstanceId: heartbeatRuns.runnerInstanceId,
|
||||
runtimeMode: heartbeatRuns.runtimeMode,
|
||||
}).from(heartbeatRuns).where(and(
|
||||
inArray(heartbeatRuns.id, [priorRunId, nextRunId]),
|
||||
eq(heartbeatRuns.companyId, binding.companyId),
|
||||
@@ -709,10 +706,15 @@ export async function applyRunnerGoalPrpEvent(
|
||||
));
|
||||
const prior = owners.find((run) => run.id === priorRunId);
|
||||
const next = owners.find((run) => run.id === nextRunId);
|
||||
const priorGoal = storedGoal(session);
|
||||
const currentFreshSession = issue?.executionRunId === nextRunId &&
|
||||
issue.assigneeAgentId === binding.agentId && issue.status === "in_progress" &&
|
||||
next?.runtimeMode === "native" && prior?.runtimeMode === "native" &&
|
||||
!!prior.nativeSessionId && (session.goalJson === null || priorGoal?.status === "complete");
|
||||
if (
|
||||
!prior || !next || next.status !== "running" ||
|
||||
next.runnerInstanceId !== event.sourceInstanceId ||
|
||||
!next.nativeSessionId || next.nativeSessionId !== prior.nativeSessionId ||
|
||||
!next.nativeSessionId || (next.nativeSessionId !== prior.nativeSessionId && !currentFreshSession) ||
|
||||
(prior.id !== next.id && !["succeeded", "failed", "cancelled", "timed_out"].includes(prior.status))
|
||||
) return false;
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user