mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 09:41:04 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs keep a finalization row, and tasks can keep recovery
actions while work is stuck
> - Git cache expiration and native harness backup expiration read these
rows to decide if closed work is still live
> - Commit paths never cleared
`native_run_finalizations.recovery_state`, so committed rows kept
`resuming_session` forever
> - Recovery actions on done or cancelled tasks also stayed active when
the close did not go through the issue routes
> - So workspaces and backups of tasks that closed weeks ago stayed live
and used disk space
> - This pull request clears the stale state at commit, backfills old
rows, resolves stale recovery actions on closed tasks, and lets the two
expiration jobs ignore settled state on closed trees
> - The benefit is that closed work expires on schedule, while live
leases, unsettled phases, unverified owners, and in-flight system
recovery still keep it
## Linked Issues or Issue Description
Refs #17248
**What happened?**
On one instance, Git cache expiration and native harness backup
expiration kept the workspaces and backups of tasks that were done for
more than 7 days. Three kinds of stale state caused this:
1. 51 `native_run_finalizations` rows had `phase = 'committed'` and
`recovery_state = 'resuming_session'`. The commit writes (status
decision commit, attention replay, result persistence, and the finalizer
claim) never cleared `recovery_state`. No reader uses `recovery_state`
on a row that has a result, except the expiration predicates.
2. Rows with `phase = 'terminal_failure'` and `recovery_state =
'blocked'` (or NULL) are permanent states. Nothing clears them, and
continuations are refused for closed tasks.
3. `issue_recovery_actions` with status `active` or `escalated` stayed
open after the source task became done or cancelled. Only the issue
routes cancel them at close. The periodic sweep
`reconcileActiveRecoveryActions` skipped every action unless
`wakePolicy.type` was a bounded owner policy or the owner was the board.
Agent-owned `resume_native_run` actions (which use `wakePolicy.kind`)
and system-owned actions were never resolved.
**Expected behavior**
When a task tree is closed for the configured time, its Git cache and
native backup expire. Only live state keeps them: a held finalization
lease, an unsettled finalization phase, an unverified native workspace
owner, or system recovery that is scheduled, running, or awaiting
evidence.
**Steps to reproduce**
1. Run a native task. Let the runner restart so that restart recovery
sets `recovery_state = 'resuming_session'` on the finalization row.
2. Let the resumed run commit its result and close the task as done.
3. Wait past the expiration window (or run the sweep with a later
clock).
4. See that the finalization row is still `committed` +
`resuming_session`, and that Git cache expiration and native backup
expiration skip the closed workspace.
**Paperclip version or commit**
`master` at 837add0caa
**Deployment mode**
Self-hosted server with native runtime enabled
**Agent adapter(s) involved**
Not adapter-specific (core bug)
## What Changed
- Clear `recovery_state` and `recovery_request_id` in every write that
persists or commits a native result: `recordCoordinatorDecision` in the
status decision committer, the attention replay in
`routePersistedNativeResultAttention`, `completeRun` in both
`NativeRunCoordinatorStore` and `PaperclipControlPlanePort`, and
`claimCoordinator` in the finalizer.
- Add data migration `0376_clear_committed_native_recovery_state.sql`
with its unchanged schema snapshot (the oldest snapshot is pruned, as
the generator does). It clears `recovery_state` and
`recovery_request_id` on rows that are `committed` with no lease owner.
`recovery_history` keeps the audit trail.
- In `reconcileActiveRecoveryActions`, resolve active and escalated
actions on done or cancelled tasks (`source_terminal`) before the wake
policy and owner filter. The existing owner-unverified guard still runs
first. Actions with system recovery in `scheduled`, `running`, or
`awaiting_evidence` are skipped.
- In `isGitWorkspaceExpirationEligible` (closed-tree mode only), a
recovery action blocks only if its cause is
`native_workspace_finalization_owner_unverified` or its system recovery
is in flight. A finalization blocks only if it has a lease owner or its
phase is not `committed` or `terminal_failure`. Idle mode keeps the old
strict checks.
- In native harness backup expiration, a settled finalization
(`committed` or `terminal_failure`, no lease owner) no longer blocks
once the session's tasks are closed for the configured days.
## Verification
- `pnpm exec vitest run` in `server/` on the touched and related files:
`git-workspace-expiration`, `git-workspace-expiration-eligibility`,
`native-harness-backup-expiration`, `native-finalization-migration`,
`native-interaction-bridge`, `native-run-finalizer-telemetry`,
`paperclip-control-plane-port`, `issue-recovery-actions`,
`native-status-arbiter-corpus`: 9 files, 321 tests passed. After the
tests moved to `native-finalization-recovery-state.test.ts`, that file,
`git-workspace-expiration` and `native-run-finalizer-telemetry` passed
again (95 tests).
- `pnpm exec vitest run src/migration-snapshot-drift.test.ts` in
`packages/db`: passed.
- `pnpm exec vitest run
src/__tests__/heartbeat-process-recovery.test.ts`: 408 tests passed.
- `pnpm run typecheck` in `server/`: passed.
- `pnpm run check:migrations` in `packages/db`: passed.
- New tests:
- Result persistence (port and coordinator store), finalizer claim and
status decision commit (new focused file
`native-finalization-recovery-state.test.ts`), and attention replay each
clear `recovery_state` and `recovery_request_id`.
- The migration clears committed, unleased rows. It keeps leased
committed rows, `observed` rows, and `terminal_failure` rows unchanged,
and keeps `recovery_history`.
- The sweep resolves an agent-owned `resume_native_run` action and an
escalated system action on a done task, and keeps an action with
scheduled system recovery. The existing test "retains the exact native
owner hold without changing a terminal task's disposition" still covers
the owner-unverified hold.
- Git expiration eligibility: a closed tree expires with `committed` +
`resuming_session`, `terminal_failure` + `blocked`, an active agent
recovery action, or an escalated system action. It stays with a leased
finalization, an `observed` finalization, an owner-unverified hold,
in-flight system recovery, an open tree, and in idle mode.
- A full Git expiration of a closed tree with stale native state, then a
reopen to the recorded commit. The existing "preserves scheduled runs,
unresolved recovery, and live previews" test now uses an `observed` and
a leased finalization as the unresolved recovery.
- Native backup expiration expires `committed` + `resuming_session` and
`terminal_failure` + `blocked`, and keeps an `observed` or leased
finalization.
- Mutation checks. I reverted each fix on its own, saw the matching test
fail, and restored it:
- Status decision commit, finalizer claim, port `completeRun`, store
`completeRun`, attention replay: each new clearing test failed.
- Migration (`WHERE false`): the backfill test failed.
- Sweep order (wake policy filter moved back in front) and the in-flight
system recovery guard: the sweep test failed.
- Git eligibility: the closed-tree action relaxation, the finalization
relaxation, the owner-unverified guard, and idle strictness each made
the matching eligibility tests fail.
- Backup expiration (`recovery_state IS NOT NULL` restored): both new
expiry cases failed.
- Not run: the full server suite and the e2e suites. CI runs them.
## Risks
- The migration is a data-only `UPDATE` on `native_run_finalizations`.
It touches only committed rows with no lease owner and a non-null
`recovery_state`. The table is not on the known-large list, and the
migration safety check passes. `recovery_history` is not changed.
- Clearing `recovery_state` at commit and claim: the readers of this
column need `phase = 'observed'` (continuation contract), `result_id IS
NULL` (recovery service, failed-run retry), or running runs (heartbeat
recovery). A row that has a result does not match any of them. The
restart recovery summary counts will drop because they no longer count
committed rows.
- The sweep now resolves more actions on closed tasks. A done or
cancelled task cannot continue, so these actions cannot act. Unverified
owner holds and in-flight system recovery are kept.
- Expiration now removes more closed workspaces and backups. Restore
points and Git cache restoration still allow a reopen to rebuild the
workspace.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code
with tool use (shell, file edits, tests).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
866 lines
34 KiB
TypeScript
866 lines
34 KiB
TypeScript
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
|
import { randomUUID } from "node:crypto";
|
|
import {
|
|
activityLog,
|
|
agentWakeupRequests,
|
|
agents,
|
|
companies,
|
|
completionContracts,
|
|
createDb,
|
|
heartbeatRuns,
|
|
issueRecoveryActions,
|
|
issueThreadInteractions,
|
|
issues,
|
|
nativeRunFinalizations,
|
|
nativeRunResults,
|
|
statusDecisionEffects,
|
|
statusDecisions,
|
|
workAssessments,
|
|
} from "@paperclipai/db";
|
|
import { and, eq } from "drizzle-orm";
|
|
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
|
import {
|
|
materializeLegacyQuestionResponseWakeProjection,
|
|
materializeNativeInteractionResponses,
|
|
NativeInteractionBridgeError,
|
|
routePersistedNativeResultAttention,
|
|
} from "../services/native-runtime/native-interaction-bridge.js";
|
|
import { PaperclipControlPlanePort } from "../services/native-runtime/paperclip-control-plane-port.js";
|
|
import { finalizeNativeRun } from "../services/native-runtime/native-run-finalizer.js";
|
|
|
|
describe("P6-19 native interaction bridge", () => {
|
|
let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
|
let db: ReturnType<typeof createDb>;
|
|
const companyId = "78000000-0000-4000-8000-000000000001";
|
|
const agentId = "78000000-0000-4000-8000-000000000002";
|
|
const issueId = "78000000-0000-4000-8000-000000000003";
|
|
const runId = "78000000-0000-4000-8000-000000000004";
|
|
const confirmationId = "78000000-0000-4000-8000-000000000005";
|
|
const questionsId = "78000000-0000-4000-8000-000000000006";
|
|
const governedId = "78000000-0000-4000-8000-000000000007";
|
|
const selfApprovedId = "78000000-0000-4000-8000-000000000008";
|
|
const suggestedTasksId = "78000000-0000-4000-8000-000000000012";
|
|
const checkboxId = "78000000-0000-4000-8000-000000000013";
|
|
const itemVerdictsId = "78000000-0000-4000-8000-000000000014";
|
|
|
|
beforeAll(async () => {
|
|
temporary = await startEmbeddedPostgresTestDatabase("paperclip-native-interaction-");
|
|
db = createDb(temporary.connectionString);
|
|
await db.insert(companies).values({ id: companyId, name: "Native interaction", issuePrefix: "NIB" });
|
|
await db.insert(agents).values({
|
|
id: agentId,
|
|
companyId,
|
|
name: "Native interaction agent",
|
|
adapterType: "codex_local",
|
|
status: "running",
|
|
});
|
|
await db.insert(issues).values({
|
|
id: issueId,
|
|
companyId,
|
|
title: "Consume an authorized interaction response",
|
|
status: "in_progress",
|
|
assigneeAgentId: agentId,
|
|
workMode: "standard",
|
|
});
|
|
await db.insert(heartbeatRuns).values({
|
|
id: runId,
|
|
companyId,
|
|
agentId,
|
|
status: "running",
|
|
contextSnapshot: { issueId, interactionId: confirmationId },
|
|
});
|
|
await db.insert(issueThreadInteractions).values([
|
|
{
|
|
id: confirmationId,
|
|
companyId,
|
|
issueId,
|
|
kind: "request_confirmation",
|
|
status: "accepted",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: new Date(),
|
|
payload: { version: 1, prompt: "Continue?" },
|
|
result: { version: 1, outcome: "accepted" },
|
|
},
|
|
{
|
|
id: questionsId,
|
|
companyId,
|
|
issueId,
|
|
kind: "ask_user_questions",
|
|
status: "answered",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: new Date(),
|
|
payload: {
|
|
version: 1,
|
|
questions: [{
|
|
id: "choice",
|
|
prompt: "Which path?",
|
|
selectionMode: "single",
|
|
options: [{ id: "safe", label: "Safe" }],
|
|
}],
|
|
},
|
|
result: { version: 1, answers: [{ questionId: "choice", optionIds: ["safe"] }] },
|
|
},
|
|
{
|
|
id: governedId,
|
|
companyId,
|
|
issueId,
|
|
kind: "request_confirmation",
|
|
status: "accepted",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: new Date(),
|
|
payload: {
|
|
version: 1,
|
|
prompt: "Execute write?",
|
|
toolAction: {
|
|
version: 1,
|
|
actionRequestId: "78000000-0000-4000-8000-000000000010",
|
|
invocationId: "78000000-0000-4000-8000-000000000011",
|
|
toolName: "write",
|
|
toolDisplayName: "Write",
|
|
connectionId: null,
|
|
applicationId: null,
|
|
appDisplayName: null,
|
|
risk: "write",
|
|
previewMarkdown: "write",
|
|
argumentsSummaryJson: "{}",
|
|
argumentsHash: "hash",
|
|
expiresAt: new Date(Date.now() + 60_000).toISOString(),
|
|
},
|
|
},
|
|
result: { version: 1, outcome: "accepted" },
|
|
},
|
|
{
|
|
id: selfApprovedId,
|
|
companyId,
|
|
issueId,
|
|
kind: "request_confirmation",
|
|
status: "accepted",
|
|
createdByAgentId: agentId,
|
|
resolvedByAgentId: agentId,
|
|
resolvedByRunId: runId,
|
|
resolvedAt: new Date(),
|
|
payload: { version: 1, prompt: "Self approve?" },
|
|
result: { version: 1, outcome: "accepted" },
|
|
},
|
|
{
|
|
id: suggestedTasksId,
|
|
companyId,
|
|
issueId,
|
|
kind: "suggest_tasks",
|
|
status: "accepted",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: new Date(),
|
|
payload: {
|
|
version: 1,
|
|
tasks: [{ clientKey: "calculator", title: "Build calculator" }],
|
|
},
|
|
result: {
|
|
version: 1,
|
|
createdTasks: [{ clientKey: "calculator", issueId: "78000000-0000-4000-8000-000000000015" }],
|
|
skippedClientKeys: [],
|
|
},
|
|
},
|
|
{
|
|
id: checkboxId,
|
|
companyId,
|
|
issueId,
|
|
kind: "request_checkbox_confirmation",
|
|
status: "accepted",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: new Date(),
|
|
payload: {
|
|
version: 1,
|
|
prompt: "Choose outputs",
|
|
options: [{ id: "guide", label: "Guide" }],
|
|
},
|
|
result: { version: 1, outcome: "accepted", selectedOptionIds: ["guide"] },
|
|
},
|
|
{
|
|
id: itemVerdictsId,
|
|
companyId,
|
|
issueId,
|
|
kind: "request_item_verdicts",
|
|
status: "pending",
|
|
payload: {
|
|
version: 1,
|
|
prompt: "Review items",
|
|
items: [
|
|
{ id: "one", label: "One" },
|
|
{ id: "two", label: "Two" },
|
|
],
|
|
},
|
|
result: {
|
|
version: 1,
|
|
outcome: "resolved",
|
|
complete: false,
|
|
items: [{
|
|
id: "one",
|
|
verdict: "approve",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: new Date().toISOString(),
|
|
}],
|
|
},
|
|
},
|
|
]);
|
|
}, 30_000);
|
|
|
|
afterAll(async () => temporary?.cleanup());
|
|
|
|
it("projects supported typed responses through the authorized interaction service", async () => {
|
|
await expect(materializeNativeInteractionResponses({
|
|
db,
|
|
companyId,
|
|
issueId,
|
|
runId,
|
|
agentId,
|
|
interactionIds: [questionsId, confirmationId, suggestedTasksId, checkboxId, itemVerdictsId],
|
|
})).resolves.toEqual([
|
|
{
|
|
interactionId: confirmationId,
|
|
kind: "request_confirmation",
|
|
response: { status: "accepted", result: { version: 1, outcome: "accepted" } },
|
|
},
|
|
{
|
|
interactionId: questionsId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [{ questionId: "choice", optionIds: ["safe"] }],
|
|
summaryMarkdown: [
|
|
"Resolved questions and answers:",
|
|
"- Which path?: Safe",
|
|
].join("\n"),
|
|
},
|
|
},
|
|
},
|
|
{
|
|
interactionId: suggestedTasksId,
|
|
kind: "suggest_tasks",
|
|
response: {
|
|
status: "accepted",
|
|
result: {
|
|
version: 1,
|
|
createdTasks: [{ clientKey: "calculator", issueId: "78000000-0000-4000-8000-000000000015" }],
|
|
skippedClientKeys: [],
|
|
},
|
|
},
|
|
},
|
|
{
|
|
interactionId: checkboxId,
|
|
kind: "request_checkbox_confirmation",
|
|
response: {
|
|
status: "accepted",
|
|
result: { version: 1, outcome: "accepted", selectedOptionIds: ["guide"] },
|
|
},
|
|
},
|
|
{
|
|
interactionId: itemVerdictsId,
|
|
kind: "request_item_verdicts",
|
|
response: {
|
|
status: "pending",
|
|
result: {
|
|
version: 1,
|
|
outcome: "resolved",
|
|
complete: false,
|
|
items: [{
|
|
id: "one",
|
|
verdict: "approve",
|
|
resolvedByUserId: "board-user",
|
|
resolvedAt: expect.any(String),
|
|
}],
|
|
},
|
|
},
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("projects answered questions for legacy adapter wake prompts", async () => {
|
|
await expect(materializeLegacyQuestionResponseWakeProjection({
|
|
db,
|
|
companyId,
|
|
issueId,
|
|
runId,
|
|
agentId,
|
|
interactionId: questionsId,
|
|
})).resolves.toEqual({
|
|
interactionId: questionsId,
|
|
summaryMarkdown: [
|
|
"Resolved questions and answers:",
|
|
"- Which path?: Safe",
|
|
].join("\n"),
|
|
});
|
|
});
|
|
|
|
it.each([
|
|
[governedId, "native_interaction_governed_request_unresolved"],
|
|
[selfApprovedId, "native_interaction_self_approval"],
|
|
])("fails closed for governed or self-approved interaction %s", async (interactionId, code) => {
|
|
const error = await materializeNativeInteractionResponses({
|
|
db,
|
|
companyId,
|
|
issueId,
|
|
runId,
|
|
agentId,
|
|
interactionIds: [interactionId],
|
|
}).catch((caught) => caught);
|
|
expect(error).toBeInstanceOf(NativeInteractionBridgeError);
|
|
expect(error).toMatchObject({ code });
|
|
});
|
|
|
|
it("routes accepted package-result attention through live issue and interaction owners", async () => {
|
|
const delegateId = "78000000-0000-4000-8000-000000000020";
|
|
const outsideCompanyId = "78000000-0000-4000-8000-000000000021";
|
|
const outsideAgentId = "78000000-0000-4000-8000-000000000022";
|
|
await db.insert(companies).values({ id: outsideCompanyId, name: "Outside attention", issuePrefix: "NIO" });
|
|
await db.insert(agents).values([
|
|
{
|
|
id: delegateId,
|
|
companyId,
|
|
name: "Native attention delegate",
|
|
adapterType: "codex_local",
|
|
status: "idle",
|
|
},
|
|
{
|
|
id: outsideAgentId,
|
|
companyId: outsideCompanyId,
|
|
name: "Outside attention delegate",
|
|
adapterType: "codex_local",
|
|
status: "idle",
|
|
},
|
|
]);
|
|
|
|
const scenarios = [
|
|
{
|
|
key: "agent",
|
|
issueId: "78000000-0000-4000-8000-000000000023",
|
|
runId: "78000000-0000-4000-8000-000000000024",
|
|
contractId: "78000000-0000-4000-8000-000000000025",
|
|
disposition: "yielded" as const,
|
|
request: {
|
|
id: "attention-agent",
|
|
requestedCapability: "domain_expertise",
|
|
summary: "Delegate a same-company native investigation",
|
|
target: { ownerClass: "agent", agentId: delegateId, companyId },
|
|
},
|
|
},
|
|
{
|
|
key: "human",
|
|
issueId: "78000000-0000-4000-8000-000000000026",
|
|
runId: "78000000-0000-4000-8000-000000000027",
|
|
contractId: "78000000-0000-4000-8000-000000000028",
|
|
disposition: "needs_review" as const,
|
|
request: {
|
|
id: "attention-human",
|
|
requestedCapability: "subjective_decision",
|
|
requiredAuthority: "board",
|
|
summary: "Choose the acceptable native result",
|
|
target: { ownerClass: "board_user", companyId },
|
|
},
|
|
},
|
|
{
|
|
key: "cross-company",
|
|
issueId: "78000000-0000-4000-8000-000000000029",
|
|
runId: "78000000-0000-4000-8000-000000000030",
|
|
contractId: "78000000-0000-4000-8000-000000000031",
|
|
disposition: "yielded" as const,
|
|
request: {
|
|
id: "attention-cross-company",
|
|
requestedCapability: "domain_expertise",
|
|
summary: "Reject an outside-company native delegate",
|
|
target: { ownerClass: "agent", agentId: outsideAgentId, companyId: outsideCompanyId },
|
|
},
|
|
},
|
|
];
|
|
|
|
for (const scenario of scenarios) {
|
|
const contractSha = `attention-contract:${scenario.key}`;
|
|
await db.insert(issues).values({
|
|
id: scenario.issueId,
|
|
companyId,
|
|
title: `Native attention ${scenario.key}`,
|
|
status: "in_progress",
|
|
assigneeAgentId: agentId,
|
|
workMode: "standard",
|
|
});
|
|
await db.insert(completionContracts).values({
|
|
id: scenario.contractId,
|
|
companyId,
|
|
issueId: scenario.issueId,
|
|
revision: 1,
|
|
schemaVersion: "paperclip.completion-contract.v1",
|
|
policyVersion: "phase6-v1",
|
|
risk: "standard",
|
|
completionAuthority: "server_arbiter",
|
|
incompleteCriteriaPolicy: "preserve_non_terminal",
|
|
contractJson: { revision: "attention-v1", objective: scenario.key, criteria: [{ id: "objective" }] },
|
|
canonicalSha256: contractSha,
|
|
createdByActorType: "system",
|
|
createdByActorId: "test",
|
|
});
|
|
await db.insert(heartbeatRuns).values({
|
|
id: scenario.runId,
|
|
companyId,
|
|
agentId,
|
|
status: "running",
|
|
runtimeMode: "native",
|
|
runtimeModeResolverVersion: "phase6-v1",
|
|
runtimeModeReason: "eligible_opt_in",
|
|
runtimeModeResolvedAt: new Date(),
|
|
nativeIssueId: scenario.issueId,
|
|
nativeSessionId: scenario.runId,
|
|
runnerInstanceId: scenario.contractId,
|
|
completionContractId: scenario.contractId,
|
|
completionContractSha256: contractSha,
|
|
contextSnapshot: { issueId: scenario.issueId },
|
|
});
|
|
const port = new PaperclipControlPlanePort(db, {
|
|
companyId,
|
|
issueId: scenario.issueId,
|
|
runId: scenario.runId,
|
|
agentId,
|
|
sessionId: scenario.runId,
|
|
completionContractId: scenario.contractId,
|
|
completionContractSha256: contractSha,
|
|
sourceInstanceId: scenario.contractId,
|
|
controlPlaneSourceInstanceId: `control:${scenario.key}`,
|
|
});
|
|
await port.completeRun({
|
|
result: {
|
|
schema: "paperclip.run_result.v1",
|
|
reportedWorkDisposition: scenario.disposition,
|
|
summary: `Native attention ${scenario.key}`,
|
|
completionClaim: {
|
|
contractRevision: "attention-v1",
|
|
objectiveSatisfied: false,
|
|
criteria: [{ criterionId: "objective", status: "unknown", evidenceRefs: [] }],
|
|
remainingWork: [{ description: "Resolve attention", blocksCompletion: true }],
|
|
},
|
|
evidence: [],
|
|
verification: [{ commandOrCheck: "attention", status: "not_run" }],
|
|
attentionRequests: [scenario.request],
|
|
artifacts: [],
|
|
...(scenario.disposition === "yielded" ? {
|
|
continuation: {
|
|
kind: "response_wake" as const,
|
|
summary: "Resume after attention",
|
|
idempotencyKey: `attention:${scenario.key}`,
|
|
},
|
|
} : {}),
|
|
},
|
|
terminal: {
|
|
schema: "paperclip.prp.terminal.v1",
|
|
turnTerminalState: "completed",
|
|
runTerminalState: "succeeded",
|
|
reportedWorkDisposition: scenario.disposition,
|
|
},
|
|
turnId: `turn:${scenario.key}`,
|
|
});
|
|
await finalizeNativeRun({
|
|
db,
|
|
runId: scenario.runId,
|
|
workspaceFinalizeStatus: "succeeded",
|
|
projectRunStatus: true,
|
|
});
|
|
}
|
|
|
|
const delegated = await db.select().from(issues).where(and(
|
|
eq(issues.parentId, scenarios[0]!.issueId),
|
|
eq(issues.companyId, companyId),
|
|
));
|
|
expect(delegated).toEqual([]);
|
|
const agentInteractions = await db.select().from(issueThreadInteractions)
|
|
.where(eq(issueThreadInteractions.issueId, scenarios[0]!.issueId));
|
|
expect(agentInteractions).toEqual([
|
|
expect.objectContaining({
|
|
kind: "request_confirmation",
|
|
status: "pending",
|
|
sourceRunId: scenarios[0]!.runId,
|
|
addresseeAgentId: delegateId,
|
|
}),
|
|
]);
|
|
const humanInteractions = await db.select().from(issueThreadInteractions)
|
|
.where(eq(issueThreadInteractions.issueId, scenarios[1]!.issueId));
|
|
expect(humanInteractions).toEqual([
|
|
expect.objectContaining({ kind: "request_confirmation", status: "pending", sourceRunId: scenarios[1]!.runId }),
|
|
]);
|
|
await expect(db.select().from(issues).where(eq(issues.id, scenarios[0]!.issueId))).resolves.toEqual([
|
|
expect.objectContaining({ status: "in_review" }),
|
|
]);
|
|
await expect(db.select().from(issues).where(eq(issues.id, scenarios[1]!.issueId))).resolves.toEqual([
|
|
expect.objectContaining({ status: "in_review" }),
|
|
]);
|
|
await expect(db.select().from(issues).where(eq(issues.parentId, scenarios[2]!.issueId))).resolves.toHaveLength(0);
|
|
await expect(db.select().from(issueRecoveryActions)
|
|
.where(eq(issueRecoveryActions.sourceIssueId, scenarios[2]!.issueId))).resolves.toEqual([]);
|
|
await expect(db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, scenarios[2]!.runId))).resolves.toEqual([
|
|
expect.objectContaining({ status: "succeeded", nativePhase: "committed" }),
|
|
]);
|
|
|
|
for (const scenario of scenarios) {
|
|
const assessments = await db.select().from(workAssessments)
|
|
.where(eq(workAssessments.issueId, scenario.issueId));
|
|
expect(assessments.map((row) => row.triggerKind)).toEqual(["native_result"]);
|
|
const decisions = await db.select().from(statusDecisions)
|
|
.where(eq(statusDecisions.issueId, scenario.issueId));
|
|
expect(decisions).toHaveLength(1);
|
|
const effects = await db.select().from(statusDecisionEffects)
|
|
.where(eq(statusDecisionEffects.decisionId, decisions[0]!.id));
|
|
expect(effects.length).toBeGreaterThan(0);
|
|
await expect(db.select().from(activityLog).where(and(
|
|
eq(activityLog.entityId, scenario.issueId),
|
|
eq(activityLog.runId, scenario.runId),
|
|
))).resolves.toEqual(expect.arrayContaining([
|
|
expect.objectContaining({ actorId: "native-status-committer" }),
|
|
]));
|
|
await expect(db.select().from(nativeRunFinalizations)
|
|
.where(eq(nativeRunFinalizations.runId, scenario.runId))).resolves.toEqual([
|
|
expect.objectContaining({ assessmentId: decisions[0]!.assessmentId, decisionId: decisions[0]!.id }),
|
|
]);
|
|
}
|
|
}, 30_000);
|
|
|
|
it("commits duplicate and stale attention as replay-stable zero-decision audit outcomes", async () => {
|
|
const seedAuditRun = async (input: {
|
|
label: string;
|
|
requests: Record<string, unknown>[];
|
|
interactions: Array<typeof issueThreadInteractions.$inferInsert>;
|
|
}) => {
|
|
const localIssueId = randomUUID();
|
|
const localRunId = randomUUID();
|
|
const localContractId = randomUUID();
|
|
const contractSha = `audit-contract:${input.label}`;
|
|
await db.insert(issues).values({
|
|
id: localIssueId,
|
|
companyId,
|
|
title: `Audit-only native attention ${input.label}`,
|
|
status: "in_progress",
|
|
assigneeAgentId: agentId,
|
|
workMode: "standard",
|
|
});
|
|
await db.insert(completionContracts).values({
|
|
id: localContractId,
|
|
companyId,
|
|
issueId: localIssueId,
|
|
revision: 1,
|
|
schemaVersion: "paperclip.completion-contract.v1",
|
|
policyVersion: "phase6-v1",
|
|
risk: "standard",
|
|
completionAuthority: "server_arbiter",
|
|
incompleteCriteriaPolicy: "preserve_non_terminal",
|
|
contractJson: {
|
|
revision: "audit-v1",
|
|
objective: "Retain superseded attention for audit",
|
|
criteria: [{ id: "objective", requirement: "Preserve the authoritative issue state" }],
|
|
},
|
|
canonicalSha256: contractSha,
|
|
createdByActorType: "system",
|
|
createdByActorId: "test",
|
|
});
|
|
await db.insert(heartbeatRuns).values({
|
|
id: localRunId,
|
|
companyId,
|
|
agentId,
|
|
status: "running",
|
|
runtimeMode: "native",
|
|
runtimeModeResolverVersion: "phase6-v1",
|
|
runtimeModeReason: "eligible_opt_in",
|
|
runtimeModeResolvedAt: new Date(),
|
|
nativeIssueId: localIssueId,
|
|
nativeSessionId: localRunId,
|
|
runnerInstanceId: localContractId,
|
|
completionContractId: localContractId,
|
|
completionContractSha256: contractSha,
|
|
contextSnapshot: { issueId: localIssueId },
|
|
});
|
|
if (input.interactions.length > 0) {
|
|
await db.insert(issueThreadInteractions).values(input.interactions.map((interaction) => ({
|
|
...interaction,
|
|
companyId: interaction.companyId ?? companyId,
|
|
issueId: interaction.issueId ?? localIssueId,
|
|
})));
|
|
}
|
|
const port = new PaperclipControlPlanePort(db, {
|
|
companyId,
|
|
issueId: localIssueId,
|
|
runId: localRunId,
|
|
agentId,
|
|
sessionId: localRunId,
|
|
completionContractId: localContractId,
|
|
completionContractSha256: contractSha,
|
|
sourceInstanceId: localContractId,
|
|
controlPlaneSourceInstanceId: `audit-control:${input.label}`,
|
|
});
|
|
await port.completeRun({
|
|
result: {
|
|
schema: "paperclip.run_result.v1",
|
|
reportedWorkDisposition: "yielded",
|
|
summary: `Audit-only native attention ${input.label}`,
|
|
completionClaim: {
|
|
contractRevision: "audit-v1",
|
|
objectiveSatisfied: false,
|
|
criteria: [{ criterionId: "objective", status: "unknown", evidenceRefs: [] }],
|
|
remainingWork: [{ description: "The canonical request remains authoritative", blocksCompletion: true }],
|
|
},
|
|
evidence: [],
|
|
verification: [{ commandOrCheck: "audit-only attention", status: "not_run" }],
|
|
attentionRequests: input.requests,
|
|
artifacts: [],
|
|
continuation: {
|
|
kind: "response_wake",
|
|
summary: "Wait for the canonical request",
|
|
idempotencyKey: `audit-only:${input.label}`,
|
|
},
|
|
},
|
|
terminal: {
|
|
schema: "paperclip.prp.terminal.v1",
|
|
turnTerminalState: "completed",
|
|
runTerminalState: "succeeded",
|
|
reportedWorkDisposition: "yielded",
|
|
},
|
|
turnId: `audit-turn:${input.label}`,
|
|
});
|
|
return { issueId: localIssueId, runId: localRunId };
|
|
};
|
|
|
|
const cases = ["duplicate", "stale", "mixed"] as const;
|
|
for (const label of cases) {
|
|
const canonicalId = randomUUID();
|
|
const duplicateId = randomUUID();
|
|
const staleId = randomUUID();
|
|
const duplicateRequest = {
|
|
id: `audit:${label}:duplicate`,
|
|
requestedCapability: "duplicate",
|
|
requiredAuthority: "agent",
|
|
target: { ownerClass: "current_agent", companyId },
|
|
summary: `Duplicate attention ${label}`,
|
|
responseState: "none",
|
|
targetInteractionId: duplicateId,
|
|
canonicalRequestId: canonicalId,
|
|
};
|
|
const staleRequest = {
|
|
id: `audit:${label}:stale`,
|
|
requestedCapability: "context_lookup",
|
|
requiredAuthority: "agent",
|
|
target: { ownerClass: "current_agent", companyId },
|
|
summary: `Stale attention ${label}`,
|
|
responseState: "stale",
|
|
targetInteractionId: staleId,
|
|
};
|
|
const requests = label === "duplicate"
|
|
? [duplicateRequest]
|
|
: label === "stale" ? [staleRequest] : [duplicateRequest, staleRequest];
|
|
const interactions: Array<typeof issueThreadInteractions.$inferInsert> = [];
|
|
if (label !== "stale") {
|
|
interactions.push(
|
|
{
|
|
id: canonicalId,
|
|
kind: "request_confirmation",
|
|
status: "pending",
|
|
payload: { version: 1, prompt: `Canonical ${label}` },
|
|
},
|
|
{
|
|
id: duplicateId,
|
|
kind: "request_confirmation",
|
|
status: "expired",
|
|
resolvedAt: new Date(),
|
|
payload: { version: 1, prompt: `Duplicate ${label}` },
|
|
result: { version: 1, outcome: "superseded_by_newer_request", supersededByInteractionId: canonicalId },
|
|
},
|
|
);
|
|
}
|
|
if (label !== "duplicate") {
|
|
interactions.push({
|
|
id: staleId,
|
|
kind: "request_confirmation",
|
|
status: "expired",
|
|
resolvedAt: new Date(),
|
|
payload: { version: 1, prompt: `Stale ${label}` },
|
|
result: { version: 1, outcome: "superseded_by_comment", commentId: randomUUID() },
|
|
});
|
|
}
|
|
const seeded = await seedAuditRun({ label, requests, interactions });
|
|
const finalized = await finalizeNativeRun({
|
|
db,
|
|
runId: seeded.runId,
|
|
workspaceFinalizeStatus: "succeeded",
|
|
projectRunStatus: true,
|
|
});
|
|
expect(finalized).toMatchObject({ phase: "committed", decisionId: expect.any(String) });
|
|
const finalDecisionId = finalized.decisionId;
|
|
|
|
await expect(db.select().from(issues).where(eq(issues.id, seeded.issueId))).resolves.toEqual([
|
|
expect.objectContaining({
|
|
status: label === "stale" ? "in_progress" : "in_review",
|
|
statusVersion: 1,
|
|
lastStatusDecisionId: expect.any(String),
|
|
}),
|
|
]);
|
|
await expect(db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId))).resolves.toEqual([
|
|
expect.objectContaining({
|
|
status: "succeeded",
|
|
nativePhase: "committed",
|
|
resultJson: expect.objectContaining({
|
|
finalizationPhase: "committed",
|
|
decisionId: expect.any(String),
|
|
authoritativeDecision: label === "stale" ? "in_progress" : "in_review",
|
|
ignoredAttentionRequests: expect.any(Array),
|
|
}),
|
|
}),
|
|
]);
|
|
await expect(db.select().from(nativeRunFinalizations)
|
|
.where(eq(nativeRunFinalizations.runId, seeded.runId))).resolves.toEqual([
|
|
expect.objectContaining({ phase: "committed", decisionId: expect.any(String), failureCode: null }),
|
|
]);
|
|
await expect(db.select().from(statusDecisions).where(eq(statusDecisions.issueId, seeded.issueId))).resolves.toHaveLength(1);
|
|
await expect(db.select().from(statusDecisionEffects).where(eq(statusDecisionEffects.issueId, seeded.issueId)))
|
|
.resolves.not.toHaveLength(0);
|
|
const issueWakeups = (await db.select().from(agentWakeupRequests)
|
|
.where(eq(agentWakeupRequests.companyId, companyId)))
|
|
.filter((request) => (request.payload as Record<string, unknown> | null)?.issueId === seeded.issueId);
|
|
if (label === "stale") {
|
|
expect(issueWakeups).toEqual([
|
|
expect.objectContaining({ payload: expect.objectContaining({ issueId: seeded.issueId }) }),
|
|
]);
|
|
} else {
|
|
// Duplicate actionable attention is owned by the review interaction;
|
|
// it must not wake the agent before that human decision is resolved.
|
|
expect(issueWakeups).toEqual([]);
|
|
}
|
|
|
|
const beforeReplay = await db.select({
|
|
id: issueThreadInteractions.id,
|
|
summary: issueThreadInteractions.summary,
|
|
updatedAt: issueThreadInteractions.updatedAt,
|
|
}).from(issueThreadInteractions).where(and(
|
|
eq(issueThreadInteractions.issueId, seeded.issueId),
|
|
));
|
|
const assessmentCount = await db.select().from(workAssessments)
|
|
.where(eq(workAssessments.runId, seeded.runId)).then((rows) => rows.length);
|
|
await expect(finalizeNativeRun({
|
|
db,
|
|
runId: seeded.runId,
|
|
workspaceFinalizeStatus: "succeeded",
|
|
projectRunStatus: true,
|
|
})).resolves.toMatchObject({ phase: "committed", decisionId: finalDecisionId });
|
|
const afterReplay = await db.select({
|
|
id: issueThreadInteractions.id,
|
|
summary: issueThreadInteractions.summary,
|
|
updatedAt: issueThreadInteractions.updatedAt,
|
|
}).from(issueThreadInteractions).where(eq(issueThreadInteractions.issueId, seeded.issueId));
|
|
expect(afterReplay).toEqual(beforeReplay);
|
|
await expect(db.select().from(workAssessments)
|
|
.where(eq(workAssessments.runId, seeded.runId)).then((rows) => rows.length)).resolves.toBe(assessmentCount);
|
|
const coordinatorBeforeCommittedReplay = await db.select({ attempt: nativeRunFinalizations.attempt })
|
|
.from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seeded.runId))
|
|
.then((rows) => rows[0] ?? null);
|
|
await expect(finalizeNativeRun({
|
|
db,
|
|
runId: seeded.runId,
|
|
workspaceFinalizeStatus: "succeeded",
|
|
projectRunStatus: true,
|
|
})).resolves.toMatchObject({ phase: "committed", decisionId: finalDecisionId });
|
|
await expect(db.select({ attempt: nativeRunFinalizations.attempt })
|
|
.from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seeded.runId))
|
|
.then((rows) => rows[0] ?? null)).resolves.toEqual(coordinatorBeforeCommittedReplay);
|
|
}
|
|
|
|
const outsideCompanyId = randomUUID();
|
|
const outsideIssueId = randomUUID();
|
|
const outsideInteractionId = randomUUID();
|
|
await db.insert(companies).values({ id: outsideCompanyId, name: "Outside audit target", issuePrefix: "OAT" });
|
|
await db.insert(issues).values({
|
|
id: outsideIssueId,
|
|
companyId: outsideCompanyId,
|
|
title: "Outside audit target",
|
|
status: "in_progress",
|
|
workMode: "standard",
|
|
});
|
|
await db.insert(issueThreadInteractions).values({
|
|
id: outsideInteractionId,
|
|
companyId: outsideCompanyId,
|
|
issueId: outsideIssueId,
|
|
kind: "request_confirmation",
|
|
status: "expired",
|
|
resolvedAt: new Date(),
|
|
payload: { version: 1, prompt: "Outside stale response" },
|
|
result: { version: 1, outcome: "superseded_by_comment", commentId: randomUUID() },
|
|
});
|
|
for (const [label, targetInteractionId] of [
|
|
["missing", randomUUID()],
|
|
["cross-company", outsideInteractionId],
|
|
] as const) {
|
|
const seeded = await seedAuditRun({
|
|
label: `invalid-${label}`,
|
|
requests: [{
|
|
id: `audit:invalid:${label}`,
|
|
requestedCapability: "context_lookup",
|
|
requiredAuthority: "agent",
|
|
target: { ownerClass: "current_agent", companyId },
|
|
summary: `Invalid audit target ${label}`,
|
|
responseState: "stale",
|
|
targetInteractionId,
|
|
}],
|
|
interactions: [],
|
|
});
|
|
await expect(finalizeNativeRun({
|
|
db,
|
|
runId: seeded.runId,
|
|
workspaceFinalizeStatus: "succeeded",
|
|
projectRunStatus: true,
|
|
})).resolves.toMatchObject({ phase: "committed", failureCode: null });
|
|
await expect(db.select().from(issues).where(eq(issues.id, seeded.issueId))).resolves.toEqual([
|
|
expect.objectContaining({ status: "in_progress", statusVersion: 1, lastStatusDecisionId: expect.any(String) }),
|
|
]);
|
|
await expect(db.select().from(statusDecisions).where(eq(statusDecisions.issueId, seeded.issueId))).resolves.toHaveLength(1);
|
|
await expect(db.select({ resultJson: heartbeatRuns.resultJson }).from(heartbeatRuns)
|
|
.where(eq(heartbeatRuns.id, seeded.runId)).then((rows) => rows[0]?.resultJson)).resolves.toEqual(
|
|
expect.objectContaining({ ignoredAttentionRequests: expect.any(Array) }),
|
|
);
|
|
}
|
|
await expect(db.select({ summary: issueThreadInteractions.summary })
|
|
.from(issueThreadInteractions)
|
|
.where(eq(issueThreadInteractions.id, outsideInteractionId))).resolves.toEqual([{ summary: null }]);
|
|
}, 30_000);
|
|
|
|
it("clears stale recovery state when replaying a committed attention decision", async () => {
|
|
const localIssueId = randomUUID();
|
|
const localRunId = randomUUID();
|
|
const localContractId = randomUUID();
|
|
const resultId = randomUUID();
|
|
const assessmentId = randomUUID();
|
|
const requestId = "attention:replayed";
|
|
await db.insert(issues).values({ id: localIssueId, companyId, title: "Replayed native attention",
|
|
status: "in_progress", assigneeAgentId: agentId, workMode: "standard" });
|
|
await db.insert(completionContracts).values({ id: localContractId, companyId, issueId: localIssueId, revision: 1,
|
|
schemaVersion: "paperclip.completion-contract.v1", policyVersion: "phase6-v1", risk: "standard",
|
|
completionAuthority: "server_arbiter", incompleteCriteriaPolicy: "preserve_non_terminal",
|
|
contractJson: { revision: "replay-v1", objective: "Replay attention", criteria: [{ id: "objective", requirement: "Replay" }] },
|
|
canonicalSha256: `replay-contract:${localContractId}`, createdByActorType: "system", createdByActorId: "test" });
|
|
await db.insert(heartbeatRuns).values({ id: localRunId, companyId, agentId, status: "succeeded", runtimeMode: "native",
|
|
nativeIssueId: localIssueId, nativeSessionId: localRunId, completionContractId: localContractId,
|
|
completionContractSha256: `replay-contract:${localContractId}`, contextSnapshot: { issueId: localIssueId } });
|
|
await db.insert(nativeRunResults).values({ id: resultId, companyId, issueId: localIssueId, runId: localRunId,
|
|
completionContractId: localContractId, serverFingerprint: `fp-${resultId}`, schemaStatus: "accepted",
|
|
resultJson: { result: { attentionRequests: [{ id: requestId, requestedCapability: "context_lookup", summary: "Look up context" }] } },
|
|
canonicalSha256: `sha-${resultId}` });
|
|
await db.insert(workAssessments).values({ id: assessmentId, companyId, issueId: localIssueId, runId: localRunId,
|
|
contractId: localContractId, resultId, triggerKind: "native_attention", triggerRef: requestId,
|
|
triggerActorCompanyId: companyId, priorIssueStatus: "in_progress", priorStatusVersion: 0,
|
|
policyVersion: "phase6-v1", assessmentJson: {}, inputDigest: `digest-${assessmentId}` });
|
|
const [decision] = await db.insert(statusDecisions).values({ companyId, issueId: localIssueId, runId: localRunId,
|
|
assessmentId, decisionVersion: 1, policyVersion: "phase6-v1", fromStatus: "in_progress", toStatus: "in_progress",
|
|
reasonCode: "attention_context", decisionJson: { effects: [] }, decisionDigest: `digest-${assessmentId}` }).returning();
|
|
await db.insert(nativeRunFinalizations).values({ runId: localRunId, companyId, issueId: localIssueId,
|
|
phase: "assessing", resultId, recoveryState: "resuming_session", recoveryRequestId: "restart-request" });
|
|
|
|
const receipts = await routePersistedNativeResultAttention({ db, runId: localRunId });
|
|
|
|
expect(receipts).toEqual([expect.objectContaining({ requestId, decisionId: decision!.id, replayed: true })]);
|
|
await expect(db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, localRunId))).resolves.toEqual([
|
|
expect.objectContaining({ phase: "committed", decisionId: decision!.id, recoveryState: null, recoveryRequestId: null }),
|
|
]);
|
|
});
|
|
});
|