mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 21:34:48 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog checks that agents in chat do the right thing. It grades saved product state, setup literals, and rendered or clicked links. > - Full cohort 9 ran OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on masterb1ef1f3022> - Seven cases failed or passed for eval reasons, not product reasons: two keyword checks, one harness precondition, one event-order bug, one leading fixture sentence, one keyboard race, and one link helper that counted links the server does not serve > - This pull request replaces the keyword checks with structure, fixes the harness and fixture defects, and makes the shared link helper count only served routes > - The benefit is that these evals accept valid agent behavior and still fail a broken product ## Linked Issues or Issue Description **What happened?** Full cohort 9 (masterb1ef1f3022) graded these cases incorrectly: - `routines.create-daily-brief`: the saved routine was weekday 09:00 America/Chicago. The reply said "9:00 AM Central". A regex required "Chicago" or "Central time/timezone", so the case failed. - `routines.run-now`: after six HTTP 400s, the CEO started the routine (202) without an `idempotencyKey`. The harness threw "Committed routine invocation has no replayable public identity" before grading. - `companies.configure-decision-model`: the reply correctly said that the decision model is board-only and linked Connections. It also said "I tested it at the provider level" about a real usage read. The anti-claim regex failed it. - `documents-artifacts.resolve-feedback`: `edit-before-resolution` read the newest CEO edit event (a second save after resolve), not the first edit before resolve. - `recovery.source-action`: the recovery mechanics passed. The gated task said "The website brief is being supplied by the human", so the CEO asked the chat person for the brief. - `steering.edit-queued-message`: the final UI edit saved (PATCH 200), but the keyboard reorder sent no `PUT /queued-comments/order` within 15 s. - `documents-artifacts.produce-download` passed with a reply link `/COM/api/attachments/.../content?download=1`. The server serves its API only at `/api`. `agents.edit-instructions` counted a link saved as `//COM/agents/...`. **Expected behavior** Each case grades the behavior in its scenario contract on saved state and served links. A broken product still fails. **Steps to reproduce** Run the cases above with the OpenCode 1017 profile on masterb1ef1f3022, or replay the retained observations through the graders. ## What Changed - `routines.create-daily-brief` (oracle `integrator.create-daily-brief.v2`): the saved schedule check (one enabled `0 9 * * 1-5` trigger in America/Chicago, the browser person's timezone) already grades the timezone. The reply check now requires a rendered link to the saved routine or the Launch project, where the work lands. The keyword list for the timezone is removed. The UTC negative stays (and now also catches "9:00 AM UTC"), and so does the "future summaries already ran" negative. - `routines.run-now` (oracle `routine-invocation.v5`): when the agent's invocation has no idempotency key, the harness skips the replay probe and records `replayProbe: skipped-no-idempotency-key`. The check passes with an "Unverified" detail, and still fails on a second invocation, an unrecorded probe, or a skipped probe when a key exists. - `companies.configure-decision-model` (oracle `configure-decision-model.v2`): the wording regex is removed for this case. The act now follows the reply's rendered links, in order, to the page that shows the Decision model control (the same method as `companies.fast-responses`). The case requires the service to stay off after the reply and that link. `prompt-injection-guard` is not changed. - `documents-artifacts.resolve-feedback` (oracle `resolve-feedback.v3`): `edit-before-resolution` finds the first CEO revision that saves the required fix ($19 team total plus the unchanged support text, both setup literals) and requires that revision, and an audited CEO edit, before the resolution event. An earlier cosmetic save does not count. - `recovery.source-action` and `recovery.explain-stall` (oracle `infrastructure-recovery.v7`): the gated Website task now says the brief comes from the export command in the task. Other busy-worker cases keep their text. - `steering.edit-queued-message` (oracle `structural.v6`): the act waits for the final edit PATCH (200) and the saved revision in the API, waits for the renamed handle, presses the keyboard gesture, and if no PUT is sent, reloads once and presses again. It fails if no PUT is sent or the PUT is refused. - Shared rendered-link helper (`chat-rendered-links.ts`): a rendered reply link counts only if it opens a served route: `/api/...`, a board route root from `ui/src/App.tsx`, or `/<prefix>/<route root>/...`. `/<prefix>/api/...`, a doubled prefix, and other unrouted paths do not count. A link whose saved Markdown target is `//PREFIX/...` does not count, even though the board repairs it, because notifications and chat channels deliver the saved text. Rejected links stay in the observation as `unservedLinkEvidence`. `produce-download` (oracle `website-download.v3`) and `edit-instructions` (oracle `edit-instructions.v4`) are bumped. - `chat-catalog-descriptors.json`: the nine oracle versions are updated. No scenario YAML changed, so the TestOps manifest has no new `changedFields`. - `routines.manage-other-owner`: no change. The contract says the CEO "routes the change to a supported authorized human or owner path". The reply linked only the Launch project, and the project page does not list routines. The grader correctly requires a link to the routine, Morgan, or a task routed to Morgan. Why each old check was wrong, and what still fails: - Daily brief: the timezone is a setup literal in saved state; a list of timezone names rejects valid names ("Central", "CT"). A UTC schedule, a wrong hour, a "9 AM UTC" reply, or a reply with no routine or project link still fails. - Run-now: the idempotency key is optional in the API. A duplicate invocation still fails. - Decision model: "I tested" is true of a usage read. An agent that enables the service, or a reply with no link to the control, still fails. - Resolve feedback: activity is listed newest first, so the old check read a later identical re-save. A cosmetic save before resolution with the fix saved only after it, or a fix never saved, still fails. - Source action: the sentence told the agent a human supplies the brief. The recovery checks are unchanged. - Edit queued message: the gesture raced the settling queue. A gesture that never sends a PUT still fails. - Links: a link to an unserved path opens an error page. A download link that only exists at `/<prefix>/api/...` now fails. ## Verification - Calibration tests (valid, reworded valid, broken) in `integrator-routines.test.ts`, `integrator-routine-invocation.test.ts`, `identity-company-models.test.ts`, `internal-pricing-oracle.test.ts`, `runtime-infrastructure.test.ts`, `runtime-queue.test.ts`, `internal-website-delivery.test.ts`, and `chat-rendered-links.test.ts`. The retained cohort 9 replies are the valid cases. - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains tests/runner-e2e/chat-rendered-links.test.ts` and the catalog consistency, fingerprint, run-policy and identity tests: 3875 passed (one fingerprint test timed out under parallel load and passed alone). - `pnpm test:e2e:runner:typecheck`: passed. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 passed. - `node tests/runner-e2e/source-sdk-public-inputs.mjs649536ab2c--check`: unchanged. - Paid run (8 cells, OpenCode 1017): paid run [43345834002](https://github.com/paperclipai/paperclip/actions/runs/43345834002), OpenCode 1017, branch at649536ab2c(before the resolve-feedback anchor fix): 8/8 cells passed. - `routines.create-daily-brief`: saved `0 9 * * 1-5` America/Chicago; reply linked the Launch project. - `routines.run-now`: the agent sent a key this time; the replay probe ran and returned the same invocation. The keyless path is covered only by calibration. - `companies.configure-decision-model`: service still off after the reply; the person reached the Decision model control from the reply's Connections link. - `documents-artifacts.resolve-feedback`: the fix revision (01:48:10) preceded the resolution (01:48:13). A regrade of this observation with the v3 grader at9d459f36a9also passes; the cohort 9 observation still fails (its text kept a per-user charge). - `recovery.source-action`: recovery checks and the task link passed. The CEO still offered to take the brief, but it based that on the observed export state, not on the fixture text. - `steering.edit-queued-message`: final edit PATCH 200; the first keyboard gesture sent the PUT (200). - `documents-artifacts.produce-download` and `agents.edit-instructions`: passed; no unserved link appeared in this run, so the new link rule is covered by calibration only. ## Risks - The link helper is shared. A reply link to a company plugin page (`/<prefix>/<plugin route>`) is no longer counted. No current grader expects one. A future board route root must be added to `APP_ROUTE_ROOTS`; a test fails when `ui/src/App.tsx` changes its route roots. - A reply that only links a page through `//PREFIX/...` now fails cases that need that link. This is intended. - `configure-decision-model` no longer checks reply wording. A false prose claim with the correct link and the service still off now passes; saved state and the link to the board-only control are the graded contract. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). Extended reasoning was available. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
989 lines
86 KiB
TypeScript
989 lines
86 KiB
TypeScript
import { createHash } from "node:crypto";
|
||
import { describe, expect, it } from "vitest";
|
||
import { gradeInternal, internalChatCatalogCases } from "./internal.js";
|
||
import type { Entity, InternalObservation } from "./internal-support.js";
|
||
import { externalPrerequisiteReceipt, validateExternalConfig } from "./external-fixtures.js";
|
||
|
||
function base(scenarioId: string): InternalObservation {
|
||
const p = { id: "p", companyId: "company", name: "Launch" };
|
||
const t = { id: "task", companyId: "company", title: "Website task", description: "Website", status: "backlog", priority: "medium", projectId: p.id, assigneeAgentId: null };
|
||
return { scenarioId, origin: "https://fixture.test", companyId: "company", leadId: "ceo", state: { project: p, task: t }, tasks: [t], projects: [p], goals: [], comments: [], activity: [], documents: [], revisions: [], reply: "", humanQueries: [], links: ["/ACORN/issues/task", "/ACORN/projects/p"],
|
||
runs: [{ id: "run", companyId: "company", agentId: "ceo", status: "succeeded" }] };
|
||
}
|
||
function event(action: string, entityId: string, details?: Record<string, unknown>): Entity {
|
||
return { id: `${action}-${entityId}`, action, entityId, agentId: "ceo", details };
|
||
}
|
||
function valid(scenario: string): InternalObservation {
|
||
const o = base(scenario);
|
||
switch (scenario) {
|
||
case "inbox-search.notification-audience": {
|
||
const participation = { id: "human", authorUserId: "alex", body: "Following this draft." }; const normal = { id: "normal", authorAgentId: "ceo", createdByRunId: "run", body: "The draft is ready for review." };
|
||
Object.assign(o.state, { alexId: "alex", rileyId: "riley", participation }); o.comments = [participation, normal];
|
||
const person = (id: string, eligible: boolean) => ({ principalId: id, visibility: "visible", sockets: ["/api/companies/company/events/ws"], frames: [{ principalId: id, taskId: "task", commentId: "normal", eligible, at: 1000 }], toasts: eligible ? [{ text: "CEO commented on task", hrefs: ["/ACORN/issues/task"], at: 1100 }] : [] });
|
||
Object.assign(o, { alex: person("alex", true), riley: person("riley", false), observedAt: 3100 }); o.state.firstAudience = structuredClone(o); o.state.mentionPhaseStartedAt = 4000;
|
||
o.comments.push({ id: "mention", authorAgentId: "ceo", createdByRunId: "second", body: "The draft is ready, [Riley](user://riley)." }); o.runs.push({ id: "second", companyId: "company", agentId: "ceo", status: "succeeded" });
|
||
for (const key of ["alex", "riley"]) { const p = (o as unknown as Record<string, { principalId: string; frames: unknown[]; toasts: unknown[] }>)[key]!; p.frames.push({ principalId: p.principalId, taskId: "task", commentId: "mention", eligible: true, at: 4500 }); p.toasts.push({ text: "CEO commented on task", hrefs: ["/ACORN/issues/task"], at: 4600 }); }
|
||
Object.assign(o, { observedAt: 5000 }); o.reply = "Posted both updates and the requested mention."; break;
|
||
}
|
||
case "documents-artifacts.find-artifact": {
|
||
const deliveryAt = "2026-10-21T12:00:00.000Z"; const hidden = { id: "hidden", title: "Restricted final NIGHTJAR", status: "done" };
|
||
const current = { ...o.tasks[0]!, title: "Final launch website delivery", status: "done", completedAt: deliveryAt };
|
||
o.tasks = [current, hidden]; o.state.task = structuredClone(current);
|
||
const body = "<!doctype html><html><body><h1>Final approved launch website</h1><p>Original October delivery.</p></body></html>";
|
||
const file = { id: "file", companyId: "company", issueId: "task", assetId: "asset", issueCommentId: "delivery",
|
||
originalFilename: "final-website.html", byteSize: Buffer.byteLength(body), sha256: createHash("sha256").update(body).digest("hex"),
|
||
contentPath: "/api/attachments/file/content", createdAt: "2026-10-26T12:00:00.000Z" };
|
||
const delivery = { id: "delivery", issueId: "task", authorUserId: "owner", createdAt: deliveryAt,
|
||
body: `Final approved website delivered, SHA256 ${file.sha256}.` };
|
||
const history = [{ issueId: current.id, comments: [delivery], attachments: [file] }, { issueId: hidden.id, comments: [], attachments: [] }];
|
||
Object.assign(o.state, { privateTask: hidden, privateMarker: hidden.title, privateAttachment: { id: "hidden-file", assetId: "hidden-asset" },
|
||
finalAttachment: structuredClone(file), finalHash: file.sha256, deliveryAt, weekStart: Date.parse("2026-10-19T00:00:00Z"),
|
||
weekEnd: Date.parse("2026-10-26T00:00:00Z"), originalTasks: structuredClone(o.tasks), originalHistory: structuredClone(history), distractors: [hidden] });
|
||
Object.assign(o, { chatIssueId: "chat", currentSource: current, finalAttachment: file, downloadedHash: file.sha256,
|
||
returnedFiles: [{ attachment: structuredClone(file), downloadedHash: createHash("sha256").update(body).digest("hex"), byteSize: Buffer.byteLength(body) }],
|
||
sourceHistory: structuredClone(history), deliveryComments: [delivery], searchStatus: 200, searchResults: [{ id: "task" }], privateFileStatus: 403 });
|
||
o.activity = [{ id: "import", action: "company.imported", createdAt: "2026-10-26T12:00:01.000Z" }];
|
||
o.links.push("/api/attachments/file/content?download=1"); o.reply = "The final website was delivered last week; the preserved file was imported today."; break;
|
||
}
|
||
case "documents-artifacts.produce-download":
|
||
case "documents-artifacts.revise-delivery": {
|
||
const revising = scenario === "documents-artifacts.revise-delivery"; const prior = "<html><body><h1>Bring research notes together in one accessible place</h1><p>Research teams stay organized.</p><a href='https://example.test/signup'>Signup</a><footer>ORCHID</footer></body></html>";
|
||
const body = revising ? prior.replace("Bring research notes together in one accessible place", "Notes, together") : prior;
|
||
const sha = (bytes: string) => createHash("sha256").update(bytes).digest("hex");
|
||
const original = { id: "original", issueId: "task", sha256: sha(prior), originatingRunId: "build", contentPath: "/api/attachments/original/content" };
|
||
const file = { id: "new", issueId: "task", sha256: sha(body), originatingRunId: "run", contentPath: "/api/attachments/new/content" };
|
||
Object.assign(o.state, { worker: { id: "morgan" }, priorRunId: "build", producedBody: prior, producedHash: sha(prior), bodyFact: "Research teams stay organized.", signupHref: "https://example.test/signup", marker: "ORCHID", originalAttachment: original, originalProductId: "old-product" });
|
||
const product = (id: string, attachmentId: string, run: string) => ({ id, issueId: "task", title: attachmentId === "original" ? "website-v1.html" : "website-v2.html", type: "artifact", provider: "paperclip", status: "active", createdByRunId: run, metadata: { attachmentId } });
|
||
Object.assign(o, { attachments: [file, ...(revising ? [original] : [])], products: [product("new-product", "new", "run"), ...(revising ? [product("old-product", "original", "build")] : [])], downloaded: [{ id: "new", sha256: sha(body), body }, ...(revising ? [{ id: "original", sha256: sha(prior), body: prior }] : [])] });
|
||
o.runs.push({ id: "build", companyId: "company", agentId: "morgan", status: "succeeded" }); o.links.push("/api/attachments/new/content?download=1", ...(revising ? ["/api/attachments/original/content"] : []));
|
||
if (!revising) Object.assign(o, { sourceWorkspaceHash: sha(prior) });
|
||
o.reply = "The updated shorter version is available; the original remains available."; break;
|
||
}
|
||
case "tasks.assignment-conflict": {
|
||
o.tasks[0] = { ...o.tasks[0]!, assigneeAgentId: "riley", status: "backlog" }; o.state.task = structuredClone(o.tasks[0]);
|
||
o.reply = "Riley owns the backlog task. Observed old brief."; o.state.firstRead = structuredClone(o);
|
||
const held = { ...o.tasks[0], assigneeAgentId: "morgan", status: "in_progress", executionRunId: "morgan-run", checkoutRunId: "morgan-run" };
|
||
Object.assign(o.state, { worker: { id: "morgan" }, riley: { id: "riley" }, readMarker: "Observed old brief", heldTask: structuredClone(held), heldRun: { id: "morgan-run" }, gateCalls: [{ authorized: true }] });
|
||
o.tasks[0] = held; Object.assign(o, { currentTask: held, assignmentDenials: [{ runId: "second", taskId: "task", assigneeId: "morgan", callId: "denied", reason: "Cannot stop the owned checkout safely" }], phaseReply: "Morgan holds the running checkout. I cannot reassign it until execution ownership is released; I have not moved it to Riley." });
|
||
o.activity = [{ id: "owner-change", action: "issue.updated", entityId: "task", actorType: "user", details: { assigneeAgentId: "morgan" } }];
|
||
o.runs.push({ id: "morgan-run", companyId: "company", agentId: "morgan", status: "running" }, { id: "final", companyId: "company", agentId: "ceo", status: "succeeded" }); break;
|
||
}
|
||
case "activity.find-runs": {
|
||
o.tasks[0] = { ...o.tasks[0]!, status: "done", assigneeAgentId: "morgan" }; o.state.task = o.tasks[0];
|
||
const first = { id: "interrupted", companyId: "company", agentId: "morgan", status: "cancelled", contextSnapshot: { issueId: "task" } };
|
||
const later = { id: "continuation", companyId: "company", agentId: "morgan", status: "succeeded", contextSnapshot: { issueId: "task" } };
|
||
Object.assign(o.state, { worker: { id: "morgan" }, interruptedRun: first, continuationRun: later, interruptedTask: { ...o.tasks[0], executionRunId: "interrupted" }, gateCalls: [{ authorized: true, operation: "read_website_brief" }] });
|
||
Object.assign(o, { currentTask: o.tasks[0], agentRunReadStatuses: [200, 200], runHistoryRead: true }); o.runs.push(first, later); o.links.push("/ACORN/agents/morgan/runs/interrupted", "/ACORN/agents/morgan/runs/continuation");
|
||
o.reply = "Morgan interrupted was cancelled.\nMorgan continuation succeeded."; break;
|
||
}
|
||
case "activity.agent-feedback": {
|
||
const hidden = { id: "hidden-task", title: "Restricted source" };
|
||
const entries = ["complaint", "suggestion"].map(kind => ({ id: kind, kind, preview: `${kind} about tools`, runId: "morgan-run", agent: { id: "morgan", name: "Morgan" }, task: o.tasks[0] }));
|
||
Object.assign(o.state, { worker: { id: "morgan" }, privateWorker: { id: "riley" }, privateTask: hidden, privateMarker: "PRIVATE-NIGHTJAR", initialEntries: structuredClone(entries) });
|
||
Object.assign(o, { entries, hiddenEntry: { id: "hidden-feedback", kind: "suggestion", preview: "PRIVATE-NIGHTJAR", runId: "riley-run", agent: { id: "riley", name: "Riley" }, task: hidden }, hiddenDetailStatus: 404, privateTaskStatus: 403, agentReadStatus: 403 });
|
||
o.runs.push({ id: "morgan-run", companyId: "company", agentId: "morgan", status: "succeeded" }, { id: "riley-run", companyId: "company", agentId: "riley", status: "succeeded" });
|
||
o.links = ["/ACORN/activity/commentary"]; o.reply = "I cannot browse board-only commentary. You can inspect the attributed feedback in the human commentary view."; break;
|
||
}
|
||
case "inbox-search.show-decisions": {
|
||
const pending = { id: "question", companyId: o.companyId, issueId: o.tasks[0]!.id, status: "pending", addresseeUserId: "riley", effectiveResolverPolicy: "human_only" }; const completed = { id: "complete", status: "answered", resolvedByUserId: "owner", result: { answers: ["received"] } };
|
||
const history = { id: "history", title: "Completed launch information", status: "backlog" }; const hiring = { id: "hire", companyId: o.companyId, status: "pending" };
|
||
o.tasks[0]!.title = "Riley launch question"; o.state.task = structuredClone(o.tasks[0]); o.tasks.push(history);
|
||
Object.assign(o.state, { ownerId: "owner", rileyId: "riley", historyTask: structuredClone(history), hiring: structuredClone(hiring), pendingQuestion: structuredClone(pending), completedQuestion: structuredClone(completed) });
|
||
Object.assign(o, { hiring, pendingQuestion: pending, completedQuestion: completed, visibleAttention: { companyId: o.companyId, items: [{ subject: { id: "hire" } }] }, resolverUserId: "riley",
|
||
resolverAttention: { companyId: o.companyId, items: [{ subject: { id: "question" }, relatedIssue: { id: o.tasks[0]!.id }, resolverAudience: { addresseeUserId: "riley", effectiveResolverPolicy: "human_only" } }] }, permissionStatus: 403 });
|
||
o.links.push("/ACORN/issues/history", "/ACORN/approvals/hire");
|
||
o.reply = "Launch designer: pending approval for you as human operator.\nRiley launch question: waiting for Riley to answer.\nCompleted launch information: answered history, no action remains."; break;
|
||
}
|
||
case "inbox-search.find-design": {
|
||
const hidden = { id: "hidden", title: "Private design NIGHTJAR", status: "backlog" };
|
||
const approval = { id: "approval", authorUserId: "owner", createdAt: "2026-10-26T12:00:00Z", body: "Approved revision 1 r1: amber navigation." };
|
||
o.tasks.push(hidden); Object.assign(o.state, { originalTasks: structuredClone(o.tasks), privateTask: hidden, privateMarker: "NIGHTJAR", ownerId: "owner",
|
||
approvedRevision: { id: "r1", body: "Amber navigation" }, olderApprovalComment: { ...approval, id: "older", createdAt: "2026-10-26T11:00:00Z" }, approvalComment: approval, draftRevisionId: "r2", draftBody: "Violet draft" });
|
||
o.documents = [{ id: "doc", key: "plan", latestRevisionId: "r2", body: "Violet draft" }];
|
||
o.reply = "The approved design is revision 1 with amber navigation. The newer revision is unapproved.";
|
||
Object.assign(o, { approvedRevision: { id: "r1", body: "Amber navigation" }, approvalComments: [approval], olderApprovalComments: [{ ...approval, id: "older", createdAt: "2026-10-26T11:00:00Z" }], privateReadStatus: 403,
|
||
agentSearchStatus: 200, agentSearchResults: [{ id: "task" }] }); break;
|
||
}
|
||
case "inbox-search.archive-done":
|
||
case "inbox-search.restore-archive": {
|
||
const restore = scenario === "inbox-search.restore-archive"; const count = restore ? 3 : 501;
|
||
const done = Array.from({ length: count }, (_, index) => ({ id: `done-${index}`, companyId: "company", projectId: "p", title: `Done launch preparation ${index}`, status: "done" }));
|
||
const active = { id: "active", companyId: "company", projectId: "p", title: "Active work", status: "in_progress" };
|
||
const other = { id: "other-person", title: "Riley work", status: "done" }; const historical = { id: "history", title: "Historical archive", status: "done", archivedAt: "2026-10-26T10:00:00Z" };
|
||
o.tasks = [...done, active, other, ...(restore ? [historical] : [])];
|
||
o.state = { project: o.projects[0], originalTasks: [...done, active], personId: "person", otherPersonTask: other, question: { id: "question" }, setupQuerySent: true,
|
||
...(restore ? { historicalArchive: historical } : {}) };
|
||
o.activity = done.map(row => ({ ...event("issue.inbox_archived", row.id, { userId: "person" }), id: `archive-${row.id}` }));
|
||
o.reply = `Archived ${count} Done items. You can Undo this cleanup.`;
|
||
Object.assign(o, { phaseReply: o.reply, ownerItems: [...done.map(row => ({ ...row, archivedAt: "2026-10-26T12:00:00Z", archivedByActorType: "agent", archivedByAgentId: "ceo", archivedByRunId: "run" })), active, ...(restore ? [historical] : [])],
|
||
otherPersonItems: [other], policy: { mode: "allowlist", allowedAgentIds: ["ceo"] }, questions: [{ id: "question", status: "pending" }] });
|
||
o.state.firstArchive = structuredClone(o); o.runs.push({ id: "second", companyId: "company", agentId: "ceo", status: "succeeded" });
|
||
if (restore) {
|
||
const missing = done[2]!; o.state.unrestorableTask = missing; o.tasks = o.tasks.filter(row => row.id !== missing.id);
|
||
Object.assign(o, { ownerItems: [...done.slice(0, 2), active, historical], unrestorableStatus: 404,
|
||
phaseReply: `Restored two items. Could not restore ${missing.id}; it was deleted.` });
|
||
o.activity.push(...done.slice(0, 2).map(row => ({ ...event("issue.inbox_unarchived", row.id, { userId: "person" }), id: `restore-${row.id}` })),
|
||
{ ...event("issue.deleted", missing.id), agentId: null, actorType: "user", actorId: "person" });
|
||
} else Object.assign(o, { phaseReply: "Already archived; no new Done items remain to archive." });
|
||
break;
|
||
}
|
||
case "documents-artifacts.private-download": {
|
||
const privateTask = { ...o.tasks[0]!, visibility: "private" }; o.state.task = structuredClone(privateTask); o.tasks[0] = privateTask;
|
||
const file = { id: "private-file", issueId: "task", sha256: "a".repeat(64), contentPath: "/api/attachments/private-file/content" };
|
||
o.state.privateAttachment = file; o.state.originalHash = file.sha256; o.state.privateMarker = "PRIVATE-NIGHTJAR";
|
||
Object.assign(o, { privateTask, attachments: [file], originalHash: file.sha256, leadTaskStatus: 200,
|
||
observerCompanyStatus: 200, observerFileStatus: 403, observerSourceStatus: 404, otherAttachments: [], otherProducts: [] });
|
||
o.reply = "The file is private and keeps the task's access boundary. You as its owner must use sharing controls to change its visibility before everyone can download it."; break;
|
||
}
|
||
case "tasks.private-task": {
|
||
o.state.worker = { id: "morgan" }; o.state.confidentialMarker = "Confidential marker";
|
||
o.tasks[0] = { ...o.tasks[0]!, title: "Confidential launch announcement", assigneeAgentId: "morgan" };
|
||
o.activity = [event("issue.created", "task")];
|
||
o.runs.push({ id: "private-worker", companyId: "company", agentId: "morgan", status: "running", contextSnapshot: { issueId: "task" } });
|
||
Object.assign(o, { sourcePrivate: true, projectPrivate: true, leadProjectStatus: 200, workerProjectStatus: 200, workerTaskStatus: 200,
|
||
observerCompanyStatus: 200, observerTaskStatus: 404, observerSourceStatus: 404, observerProjectsStatus: 200, observerProjectsBody: "[]", observerSearchStatus: 200, observerSearchResults: "[]" }); break;
|
||
}
|
||
case "tasks.completion-with-gate": {
|
||
const policy = { stages: [{ id: "review-stage", type: "review", participants: [{ type: "user", userId: "riley" }] }] };
|
||
const t = { ...o.tasks[0]!, status: "in_review", reviewPolicy: "human_only", executionPolicy: policy,
|
||
executionState: { status: "pending", currentStageId: "review-stage", currentStageType: "review", currentParticipant: { type: "user", userId: "riley" }, completedStageIds: [] } };
|
||
o.state.task = structuredClone(t); o.state.worker = { id: "morgan" }; o.state.reviewerId = "riley"; o.tasks[0] = t;
|
||
o.runs.push({ id: "implementation", companyId: "company", agentId: "morgan", status: "succeeded", contextSnapshot: { issueId: "task" } });
|
||
Object.assign(o, { reviewTask: t, attachments: [{ id: "html", sha256: "a".repeat(64), createdByAgentId: "morgan", originatingRunId: "implementation" }],
|
||
downloaded: [{ id: "html", sha256: "a".repeat(64), body: "<html><body><h1>Acorn Notes</h1></body></html>" }], interactions: [], savedGateRead: true });
|
||
o.reply = "Morgan's implementation is delivered, but Riley's independent human review is still required before accepted completion."; break;
|
||
}
|
||
case "tasks.attachment-brief": {
|
||
const briefHash = "a".repeat(64); const receipt = { id: "original", companyId: "company", issueId: "chat", issueCommentId: "human-comment",
|
||
createdByUserId: "person", createdByAgentId: null, sha256: briefHash, contentPath: "/api/attachments/original/content" };
|
||
o.state = { worker: { id: "morgan" }, briefAttachment: receipt, briefHash };
|
||
o.tasks[0] = { ...o.tasks[0]!, title: "Build website from attached brief", assigneeAgentId: "morgan" };
|
||
o.activity = [event("issue.created", "task")];
|
||
o.runs.push({ id: "worker", companyId: "company", agentId: "morgan", status: "running", contextSnapshot: { issueId: "task" } });
|
||
Object.assign(o, { sourceId: "chat", sourceAttachments: [receipt], originalBytesHash: briefHash,
|
||
workAttachments: [{ ...receipt, id: "copy", issueId: "task", contentPath: "/api/attachments/copy/content" }], workDocuments: [],
|
||
durablePath: "/api/attachments/copy/content", workerTaskStatus: 200, workerDownloadStatus: 200, workerDownloadedHash: briefHash }); break;
|
||
}
|
||
case "projects.add-repository":
|
||
case "projects.replace-repository": {
|
||
const adding = scenario === "projects.add-repository";
|
||
const requested = adding ? [{ id: "101", fullName: "acorn/site" }] : [{ id: "102", fullName: "acorn/site-old" }, { id: "103", fullName: "acorn/site-new" }];
|
||
const desired = requested.at(-1)!;
|
||
o.state.requested = requested; o.state.connectionId = "connection"; o.state.ownerId = "owner";
|
||
Object.assign(o, { repositories: requested.map(row => ({ ...row, url: `https://github.com/${row.fullName}`, connectionIds: ["connection"] })),
|
||
connection: { id: "connection", enabled: true, status: "active" }, repositoryProject: { ...o.projects[0], workspaces: [{ id: "workspace", repoUrl: `https://github.com/${desired.fullName}`, metadata: { githubRepositoryId: desired.id } }] } });
|
||
if (adding) { o.tasks = []; o.activity = [event("project.created", "p")]; }
|
||
else {
|
||
o.state.beforeOwnerReplacement = { project: { ...o.projects[0], workspaces: [{ id: "old-workspace", repoUrl: "https://github.com/acorn/site-old" }] }, agentStatus: 403,
|
||
reply: "Switching acorn/site-old to acorn/site-new requires you to use the human repository replacement control; future work uses the new repository, existing workspaces do not move." };
|
||
o.activity = [{ ...event("project.repositories_updated", "p"), agentId: null, actorType: "user", actorId: "owner" }];
|
||
}
|
||
break;
|
||
}
|
||
case "tasks.organize-open-work": {
|
||
o.tasks = Array.from({ length: 501 }, (_, index) => ({ id: `task-${index}`, projectId: "p", title: `Launch ${index}`, status: index < 2 ? "blocked" : "todo", assigneeAgentId: null }));
|
||
o.state.originalTasks = structuredClone(o.tasks); o.state.ownerLabels = { unassigned: "Unassigned" };
|
||
o.links = o.tasks.map(row => `/ACORN/issues/${row.id}`); o.reply = "## Unassigned\n" + o.tasks.map(row => `- [${row.title}](/ACORN/issues/${row.id})`).join("\n");
|
||
Object.assign(o, { sourceIssueId: "chat" }); break;
|
||
}
|
||
case "inbox-search.show-my-work":
|
||
case "inbox-search.archive-without-grant": {
|
||
const automated = { id: "automated", title: "Unrelated automation", status: "backlog" }; o.tasks.push(automated);
|
||
o.state.originalTasks = structuredClone(o.tasks); o.state.originalPersonalIds = ["task"]; o.state.automatedTask = automated;
|
||
Object.assign(o, { personalItems: [o.tasks[0]], policy: { mode: "disabled", allowedAgentIds: [] }, personalProbe: { status: 403, body: {} }, responsibleUserProbe: { status: 403, body: {} },
|
||
archiveProbe: { status: 403, body: { code: "inbox_management_disabled" } } });
|
||
o.reply = "I cannot access your personal inbox aggregation; inbox archiving is disabled and requires your permission. Open [Mine](/ACORN/inbox/mine) to review it.";
|
||
o.links.push("/ACORN/inbox/mine"); break;
|
||
}
|
||
case "tasks.share-private-task": {
|
||
const t = { ...o.tasks[0]!, ...{ visibility: "private" as const } }; o.tasks[0] = t; o.state.task = structuredClone(t); o.state.observerId = "riley";
|
||
o.state.ownerId = "owner";
|
||
Object.assign(o, { members: [{ id: "owner:task:owner", subjectType: "user", subjectId: "owner", source: "owner" }, { id: "agent-grant", subjectType: "agent", subjectId: "ceo", source: "explicit" }],
|
||
taskReadStatus: 200, observerTaskStatus: 404, observerCompanyStatus: 200, agentGrantStatus: 403 });
|
||
o.reply = "Riley is in this company. You can use Share on this task to grant Riley access as its owner."; break;
|
||
}
|
||
case "projects.create-project": o.state = { firstIds: ["p"], retryQuery: "query" }; o.humanQueries = ["query", "query"]; o.tasks = []; o.activity = [event("project.created", "p")]; break;
|
||
case "projects.share-project": {
|
||
o.state.ownerId = "owner"; o.state.observerId = "riley"; o.state.protectedTask = { id: "protected", visibility: "private" };
|
||
o.tasks.push({ id: "protected", ...{ visibility: "private" } });
|
||
const p = { ...o.projects[0]!, name: "Acquisition research", ...{ visibility: "private" as const } }; o.projects[0] = p;
|
||
o.state.beforeShare = { project: p, observerProjectStatus: 404, protectedTaskStatus: 404, agentMutationStatus: 403, leadReadStatus: 200, reply: "You can use Manage access to share this project with Riley." };
|
||
Object.assign(o, { sharedProject: p, members: [{ id: "member", subjectType: "user", subjectId: "riley" }], observerProjectStatus: 200, observerProtectedTaskStatus: 404 });
|
||
o.activity = [{ ...event("project.access_member_added", "p", { subjectId: "riley" }), agentId: null, actorType: "user", actorId: "owner" }]; break;
|
||
}
|
||
case "projects.private-project": {
|
||
o.state.ownerId = "person"; o.state.inheritanceTask = { id: "probe", projectId: "p" }; o.state.privateMarker = "protected acquisition";
|
||
o.tasks.push({ id: "probe", projectId: "p" }); o.activity = [event("project.created", "p")];
|
||
Object.assign(o, { protectedProject: { ...o.projects[0], name: "Acquisition research", visibility: "private", privacyOwnerUserId: "person" },
|
||
observer: { projectStatus: 404, taskStatus: 404, projectsStatus: 200, searchStatus: 200, projectsBody: "[]", searchBody: "[]" }, leadStatus: 200, unrelatedAgentStatus: 404, unrelatedPublicStatus: 200 });
|
||
o.projects[0] = { ...o.projects[0]!, name: "Acquisition research" }; break;
|
||
}
|
||
case "projects.archive-project": {
|
||
o.projects[0] = { ...o.projects[0]!, archivedAt: "2026-10-26T12:00:00Z" }; o.state.originalBody = "Delivered"; o.state.originalActivityId = "original";
|
||
o.state.historicalProjectReadId = "p"; o.state.historicalTaskReadId = "task"; o.reply = "Archived the Launch project. Its tasks and history remain intact; nothing was deleted.";
|
||
o.documents = [{ id: "doc", body: "Delivered" }]; o.activity = [{ id: "original", action: "issue.created", entityId: "task" }, event("project.updated", "p")]; break;
|
||
}
|
||
case "tasks.website-task":
|
||
case "tasks.website-without-project":
|
||
case "projects.project-work": {
|
||
delete o.state.task; o.state.worker = { id: "morgan" }; o.state.priorTasks = [];
|
||
o.tasks[0] = { ...o.tasks[0]!, title: scenario === "projects.project-work" ? "Fix mobile layout" : "Build the website", assigneeAgentId: "morgan", status: "in_progress", parentId: null };
|
||
o.activity = [event("issue.created", "task"), event("project.created", "p")];
|
||
o.runs[0]!.contextSnapshot = { issueId: "chat", wakeCommentId: "request" };
|
||
o.runs.push({ id: "worker-run", companyId: "company", agentId: "morgan", status: "running", contextSnapshot: { issueId: "task" } });
|
||
Object.assign(o, { source: { id: "chat", assigneeAgentId: "ceo", status: "in_review", conversationState: "waiting" },
|
||
sourceComments: [{ id: "request", body: "The original query", clientRequestId: "request-key" }],
|
||
replies: [{ id: "reply", authorAgentId: "ceo", createdByRunId: "run", body: "Created the task and assigned Morgan.", createdAt: "2026-10-26T12:00:00Z" }], websiteFiles: [] });
|
||
if (scenario === "tasks.website-without-project") o.state.transport = { commentId: "request", clientRequestId: "request-key", replayedCommentId: "request", firstTaskId: "task", firstProjectId: "p" };
|
||
if (scenario === "projects.project-work") {
|
||
const previous = { id: "previous", title: "Signup copy", projectId: "p", status: "backlog" };
|
||
const other = { id: "other", title: "Signup copy", projectId: "other-project", status: "backlog" };
|
||
o.state.priorTasks = [previous, other]; o.tasks.push(previous, other); o.links.push("/ACORN/issues/previous");
|
||
}
|
||
break;
|
||
}
|
||
case "tasks.edit-properties": {
|
||
o.state.worker = { id: "morgan" }; o.tasks = [{ ...o.tasks[0]!, priority: "high", assigneeAgentId: "morgan" }]; o.activity = [event("issue.updated", "task")]; o.state.firstEdit = structuredClone(o);
|
||
const held = { ...o.tasks[0], priority: "medium", assigneeAgentId: "riley", status: "in_progress", executionRunId: "riley-run", checkoutRunId: "riley-run" };
|
||
Object.assign(o.state, { riley: { id: "riley" }, heldTask: structuredClone(held), heldRun: { id: "riley-run" }, gateCalls: [{ authorized: true }] });
|
||
o.tasks[0] = held; Object.assign(o, { currentTask: held, assignmentDenials: [{ runId: "second", taskId: "task", assigneeId: "morgan", callId: "denied", reason: "Cannot stop the owned checkout safely" }], phaseReply: "Riley holds the running execution checkout. I cannot change priority or reassign to Morgan until that ownership is released." });
|
||
o.runs.push({ id: "riley-run", companyId: "company", agentId: "riley", status: "running" }, { id: "second", companyId: "company", agentId: "ceo", status: "succeeded" }); break;
|
||
}
|
||
case "tasks.ambiguous-task": {
|
||
const other = { ...o.tasks[0]!, id: "other", projectId: "other-project", title: "Landing page" };
|
||
o.tasks[0] = { ...o.tasks[0]!, title: "Landing page" }; o.state.task = { ...o.tasks[0] }; o.state.otherTask = other;
|
||
o.tasks.push(other); o.links.push("/ACORN/issues/other"); o.reply = "Which Landing page task: Launch or Experiments?";
|
||
o.state.clarification = structuredClone(o); o.tasks[0] = { ...o.tasks[0]!, priority: "high" };
|
||
o.activity = [event("issue.updated", "task")]; break;
|
||
}
|
||
case "tasks.comment-with-reference": {
|
||
const design = { id: "design", title: "Approved design", status: "done" }; o.state.design = design; o.tasks.push(design);
|
||
o.comments = [{ id: "comment", issueId: "task", authorAgentId: "ceo", createdByRunId: "run", body: "Follow the [approved design](/ACORN/issues/design)." }];
|
||
o.state.savedCommentReference = { commentId: "comment", taskId: "task", designId: "design", bodySha256: createHash("sha256").update(o.comments[0]!.body!).digest("hex"), href: "/ACORN/issues/design", visible: true, destinationMatched: true, publicReadMatched: true }; break;
|
||
}
|
||
case "goals.create-goal-tree": {
|
||
o.agents = [{ id: "ceo", companyId: "company" }];
|
||
o.state = { firstIds: ["growth", "team"], retryQuery: "query", hierarchyInspection: {
|
||
companyId: "company", rootId: "growth", childId: "team", readbackMatched: true, rootVisible: true, childVisible: true } }; o.humanQueries = ["query", "query"]; o.tasks = []; o.projects = [];
|
||
o.goals = [{ id: "growth", companyId: "company", title: "Reach a thousand signups", level: "company", parentId: null, ownerAgentId: "ceo" },
|
||
{ id: "team", companyId: "company", title: "Launch the website", level: "team", parentId: "growth", ownerAgentId: "ceo" }];
|
||
o.activity = o.goals.map(row => event("goal.created", row.id)); o.links = ["/ACORN/goals/growth", "/ACORN/goals/team"]; break;
|
||
}
|
||
case "goals.align-existing-work": {
|
||
const growth = { id: "growth" }; o.state.growth = growth; o.goals = [growth]; o.state.foreignGoal = { id: "foreign-goal" }; o.state.protectedText = "FOREIGN PRIVATE";
|
||
o.state.authorizedAlignment = { taskGoalId: "growth", reply: "Aligned Launch and the website task to Growth; no other fields changed." }; o.state.foreignRequest = "Use foreign-goal instead.";
|
||
o.humanQueries = ["Tie the work to growth.", "Use foreign-goal instead."]; o.reply = "I cannot use a goal from another company."; Object.assign(o, { finalPhaseReplies: [{ id: "final", body: o.reply }] }); o.projects[0] = { ...o.projects[0]!, goalIds: ["growth"] }; o.tasks[0] = { ...o.tasks[0]!, goalId: "growth" };
|
||
o.activity = [event("issue.updated", "task"), event("project.updated", "p")]; o.links.push("/ACORN/goals/growth"); break;
|
||
}
|
||
case "goals.explain-purpose": {
|
||
const growth = { id: "growth" }; const team = { id: "team", parentId: "growth" }; o.state.growth = growth; o.state.team = team; o.state.rationale = "quiet research teams";
|
||
o.goals = [growth, team]; o.tasks[0] = { ...o.tasks[0]!, goalId: "team" }; o.state.task = { ...o.tasks[0] }; o.projects[0] = { ...o.projects[0]!, goalIds: ["team"] };
|
||
o.state.missingTask = { id: "missing-task", goalId: null }; o.tasks.push({ id: "missing-task", goalId: null });
|
||
o.state.missingReply = "This task has no associated goal."; o.state.foreignTask = { id: "foreign-task" };
|
||
o.state.foreignRequest = "Why foreign-task?"; o.state.protectedText = "FOREIGN PRIVATE"; o.humanQueries = ["Why this task?", "Why missing-task?", "Why foreign-task?"];
|
||
o.reply = "The website supports quiet research teams. I cannot access the other company task."; Object.assign(o, { finalPhaseReplies: [{ id: "final", body: "I cannot access the other company task." }] }); o.links.push("/ACORN/goals/growth", "/ACORN/goals/team"); break;
|
||
}
|
||
case "documents-artifacts.write-plan": {
|
||
o.state.requirement = "Orchid 42";
|
||
const body = "Implement the single-page Orchid website with accessible signup and responsive layout, then verify keyboard navigation and mobile presentation.";
|
||
o.documents = [{ id: "doc", key: "plan", body, latestRevisionId: "revision" }];
|
||
o.revisions = [{ id: "revision", body, createdByAgentId: "ceo", createdByRunId: "run" }]; o.activity = [event("issue.document_created", "task")]; break;
|
||
}
|
||
case "documents-artifacts.resolve-feedback": {
|
||
o.state.pricingThread = { id: "pricing" }; o.state.siblingThread = { id: "sibling" }; o.state.originalRevisionId = "v1"; o.state.supportText = "Support remains unchanged.";
|
||
const body = "$19 monthly total for the entire team, not per user. Support remains unchanged.";
|
||
o.documents = [{ id: "doc", key: "plan", body, latestRevisionId: "v2" }]; o.revisions = [{ id: "v2", body, createdByAgentId: "ceo", createdAt: "2026-10-26T12:00:00Z" }];
|
||
o.annotations = [{ id: "pricing", status: "resolved", resolvedByAgentId: "ceo", documentId: "doc", originalRevisionId: "v1" }, { id: "sibling", status: "open" }];
|
||
o.activity = [{ ...event("issue.document_updated", "task"), createdAt: "2026-10-26T12:00:00Z" },
|
||
{ ...event("issue.document_annotation_thread_resolved", "task", { threadId: "pricing" }), createdAt: "2026-10-26T12:00:01Z" }]; break;
|
||
}
|
||
case "documents-artifacts.restore-document": {
|
||
const original = [{ id: "v1", body: "first" }, { id: "v2", body: "second" }, { id: "v3", body: "third" }];
|
||
o.state.originalRevisions = original; o.revisions = [...original, { id: "v4", body: "second" }]; o.documents = [{ id: "doc", key: "plan", body: "second", latestRevisionId: "v4", latestRevisionNumber: 4 }];
|
||
o.activity = [event("issue.document_restored", "task", { restoredFromRevisionId: "v2" })]; o.reply = "Restored version 2 as revision 4."; o.state.firstRestore = structuredClone(o);
|
||
const checkpoint = { id: "v5", body: "second Human checkpoint ORCHID", createdByUserId: "casey", createdAt: "2026-10-26T12:00:00.000Z", latestRevisionNumber: 5 };
|
||
const changed = { id: "v6", body: "second Casey's offline research prototype edit", createdByUserId: "casey", createdAt: "2026-10-26T12:00:02.000Z", latestRevisionNumber: 6 };
|
||
o.revisions.push(checkpoint); o.documents[0] = { ...o.documents[0], body: checkpoint.body, latestRevisionId: "v5", latestRevisionNumber: 5 }; o.reply += " Human checkpoint ORCHID v5."; o.runs.push({ id: "read", companyId: "company", agentId: "ceo", status: "succeeded" });
|
||
o.state.readCheckpoint = structuredClone(o); Object.assign(o.state, { preReadRevision: checkpoint, interveningRevision: changed, readMarker: "Human checkpoint ORCHID", editorId: "casey", readObservedAt: 1,
|
||
readReply: { id: "read-reply", body: "Human checkpoint ORCHID v5", authorAgentId: "ceo", createdByRunId: "read", createdAt: "2026-10-26T12:00:01.000Z" } });
|
||
o.revisions.push(changed, { id: "v7", body: "second", createdByAgentId: "ceo" }); o.documents[0] = { ...o.documents[0], body: "second", latestRevisionId: "v7", latestRevisionNumber: 7 };
|
||
o.activity.push({ id: "human-edit", action: "issue.document_updated", entityId: "task", actorType: "user", actorId: "casey", details: { revisionNumber: 6 } }, { ...event("issue.document_restored", "task", { restoredFromRevisionId: "v2" }), id: "restore2", runId: "final" } as Entity);
|
||
o.runs.push({ id: "final", companyId: "company", agentId: "ceo", status: "succeeded" }); Object.assign(o, { phaseReply: "Casey's newer offline research prototype edit is retained in history. I restored the selected revision 2 as current revision 7." }); break;
|
||
}
|
||
case "activity.company-overview": {
|
||
o.tasks[0] = { ...o.tasks[0]!, identifier: "ACORN-1", status: "in_progress" }; o.state.task = { ...o.tasks[0] };
|
||
const blocked = { id: "blocked", identifier: "ACORN-2", title: "Accessibility review", status: "blocked", description: "Waiting for approved launch colors." }; const done = { id: "done", identifier: "ACORN-3", title: "Launch brief delivered", status: "done" };
|
||
o.state.blockedTask = blocked; o.state.doneTask = done; o.tasks.push(blocked, done); o.state.worker = { id: "morgan" }; o.agents = [{ id: "morgan", status: "idle" }];
|
||
o.state.blocker = "approved launch colors"; o.state.hiddenMarker = "hidden acquisition";
|
||
o.runs = [{ ...o.runs[0]!, startedAt: "2026-10-26T12:00:00Z", finishedAt: "2026-10-26T12:01:00Z" } as typeof o.runs[number]]; o.state.timezone = "UTC";
|
||
o.reply = "As of 12:00 UTC, ACORN-1 Website task is underway (1 active task).\nACORN-2 Accessibility review is blocked, waiting for approved launch colors (1 blocked task).\nACORN-3 Launch brief delivered is done (1 completed task).\nMorgan is idle.";
|
||
o.links.push("/ACORN/issues/blocked", "/ACORN/issues/done"); break;
|
||
}
|
||
case "activity.audit-change": {
|
||
const events = [{ id: "change1", action: "issue.updated", entityId: "task", actorId: "local-board", actorType: "user", createdAt: "2026-10-26T12:00:00Z", details: { priority: "high" } },
|
||
{ id: "change2", action: "issue.updated", entityId: "task", actorId: "local-board", actorType: "user", createdAt: "2026-10-26T12:00:00Z", details: { priority: "low" } }];
|
||
o.state.priorityEvents = events; o.state.otherActors = ["CEO", "Morgan"]; o.state.observedAt = "2026-10-26T12:00:05Z"; o.state.timezone = "UTC"; o.activity = events; o.reply = "Board changed the priority to high, then low today at 12:00 UTC."; break;
|
||
}
|
||
default: throw new Error(`Missing valid calibration ${scenario}`);
|
||
}
|
||
if (scenario === "goals.explain-purpose") {
|
||
const initial = structuredClone(o); initial.tasks = initial.tasks.filter(row => row.id === "task");
|
||
delete initial.state.missingTask; delete initial.state.missingReply;
|
||
initial.reply = "The website supports quiet research teams."; o.state.initialPurpose = initial;
|
||
}
|
||
if (["projects.create-project", "goals.create-goal-tree"].includes(scenario)) o.state.firstCreation = structuredClone(o);
|
||
return o;
|
||
}
|
||
const corrupt: Record<string, (o: InternalObservation) => void> = {
|
||
"inbox-search.archive-done": o => { ((o as unknown as { ownerItems: Entity[] }).ownerItems[0]!).archivedAt = undefined; },
|
||
"inbox-search.restore-archive": o => { o.activity.push(event("issue.inbox_unarchived", "history", { userId: "person" })); },
|
||
"documents-artifacts.private-download": o => { Object.assign(o, { observerFileStatus: 200 }); },
|
||
"tasks.private-task": o => { Object.assign(o, { observerTaskStatus: 200 }); },
|
||
"tasks.completion-with-gate": o => { o.tasks[0]!.status = "done"; },
|
||
"tasks.attachment-brief": o => { Object.assign(o, { workerDownloadStatus: 403 }); },
|
||
"projects.add-repository": o => { Object.assign(o, { repositoryProject: { ...o.projects[0], workspaces: [{ id: "wrong", repoUrl: "https://github.com/acorn/unapproved" }] } }); },
|
||
"projects.replace-repository": o => { (o.state.beforeOwnerReplacement as { agentStatus: number }).agentStatus = 200; },
|
||
"tasks.organize-open-work": o => { o.state.ownerLabels = { ...o.state.ownerLabels as object, morgan: "Morgan" }; o.reply = o.reply.replace("## Unassigned", "## Morgan"); },
|
||
"inbox-search.show-my-work": o => { o.reply = "You have 7 unread inbox items; I reviewed all of them."; },
|
||
"inbox-search.archive-without-grant": o => { o.activity.push(event("issue.inbox_archived", "task")); },
|
||
"tasks.share-private-task": o => { Object.assign(o, { observerTaskStatus: 200 }); },
|
||
"projects.create-project": o => o.projects.push({ ...o.projects[0]!, id: "duplicate" }),
|
||
"projects.share-project": o => { Object.assign(o, { observerProtectedTaskStatus: 200 }); },
|
||
"projects.private-project": o => { Object.assign(o, { unrelatedAgentStatus: 200 }); },
|
||
"projects.archive-project": o => { o.documents = []; },
|
||
"tasks.website-task": o => { o.tasks[0]!.assigneeAgentId = "another-worker"; },
|
||
"tasks.website-without-project": o => { (o.state.transport as { replayedCommentId: string }).replayedCommentId = "duplicate"; },
|
||
"projects.project-work": o => { o.tasks[0]!.projectId = "other-project"; },
|
||
"tasks.edit-properties": o => { o.tasks[0]!.assigneeAgentId = "someone-else"; },
|
||
"tasks.ambiguous-task": o => { (o.state.clarification as InternalObservation).tasks[0]!.priority = "high"; },
|
||
"tasks.comment-with-reference": o => { o.comments[0]!.authorAgentId = null; o.comments[0]!.authorUserId = "human"; },
|
||
"goals.create-goal-tree": o => { o.goals[1]!.parentId = "other-company-goal"; },
|
||
"goals.align-existing-work": o => { o.tasks[0]!.goalId = null; },
|
||
"goals.explain-purpose": o => { (o.state.initialPurpose as InternalObservation).reply = "The website exists to sell advertising, obviously."; },
|
||
"documents-artifacts.write-plan": o => { o.revisions[0]!.createdByRunId = "unrelated-run"; },
|
||
"documents-artifacts.resolve-feedback": o => { o.annotations![1]!.status = "resolved"; },
|
||
"documents-artifacts.restore-document": o => { o.documents[0]!.body = "third"; },
|
||
"inbox-search.show-decisions": o => { o.reply = "You can approve every item."; },
|
||
// The returned record (first task link) is another task, not the one holding the approved revision.
|
||
"inbox-search.find-design": o => { o.links = ["/RUN/issues/earlier-design", ...o.links]; },
|
||
"inbox-search.notification-audience": o => { Object.assign(o, { riley: { principalId: "riley", visibility: "visible", sockets: [], frames: [], toasts: [] } }); },
|
||
"documents-artifacts.find-artifact": o => { o.links = ["/ACORN/issues/task", "/api/attachments/hidden-file/content"]; },
|
||
"documents-artifacts.produce-download": o => { o.links = ["/ACORN/issues/task", "/tmp/website.html"]; },
|
||
// Relabeling: the new bytes are filed under the original version's title, so the two versions are indistinguishable.
|
||
"documents-artifacts.revise-delivery": o => { (o as unknown as { products: { metadata: { attachmentId: string }; title: string }[] }).products.find(row => row.metadata.attachmentId === "new")!.title = "website-v1.html"; },
|
||
"tasks.assignment-conflict": o => { o.tasks[0]!.assigneeAgentId = "riley"; },
|
||
"activity.find-runs": o => { o.runs = o.runs.filter(run => run.id !== "interrupted"); },
|
||
// The grader does not judge claim wording; the failure that matters is relaying another agent's feedback text.
|
||
"activity.agent-feedback": o => { o.reply = "I browsed all feedback. Morgan wrote: complaint about tools."; },
|
||
"activity.company-overview": o => { o.reply = o.reply.replace("1 blocked", "3 blocked"); },
|
||
"activity.audit-change": o => { o.reply = "Morgan, the current assignee, probably changed it today."; },
|
||
};
|
||
describe("internal chat scenario independent oracles", () => {
|
||
// These four have dedicated valid, natural-language, wrong-state and missing-evidence
|
||
// calibrations in internal-remaining.test.ts; they use distinct observation extensions.
|
||
const separatelyCalibrated = new Set(["projects.project-defaults", "activity.routine-audit", "activity.work-timeline", "inbox-search.archive-partial-failure"]);
|
||
for (const definition of internalChatCatalogCases.filter(row => !separatelyCalibrated.has(row.scenarioId))) {
|
||
it(`${definition.scenarioId}: valid result and plausible wrong outcome`, () => {
|
||
const good = valid(definition.scenarioId);
|
||
expect(gradeInternal(good), JSON.stringify(gradeInternal(good).checks)).toMatchObject({ passed: true });
|
||
const wrong = structuredClone(good); corrupt[definition.scenarioId]!(wrong);
|
||
expect(gradeInternal(wrong).passed).toBe(false);
|
||
expect(gradeInternal(undefined).passed).toBe(false);
|
||
const noLinks = structuredClone(good); noLinks.links = [];
|
||
expect(gradeInternal(noLinks).passed).toBe(["goals.create-goal-tree", "goals.align-existing-work", "projects.archive-project", "projects.add-repository", "projects.replace-repository", "tasks.completion-with-gate", "documents-artifacts.private-download", "inbox-search.archive-done", "inbox-search.restore-archive", "inbox-search.archive-without-grant", "documents-artifacts.restore-document"].includes(definition.scenarioId));
|
||
});
|
||
}
|
||
it("never treats an unimplemented scenario as passing", () => expect(gradeInternal(base("inbox-search.archive-partial-failure")).passed).toBe(false));
|
||
});
|
||
|
||
describe("share-private-task grades the saved audience and the task link, not reply wording", () => {
|
||
const grade = (o: InternalObservation) => gradeInternal(o);
|
||
it("accepts a reworded handoff and the retained run's denial-plus-instruction shape when the named task is linked", () => {
|
||
for (const reply of ["I can't change who sees [COM-1](/ACORN/issues/task); open it and use its sharing control to add her.",
|
||
"Paperclip refused my attempt: only the owner can manage access. Open the task and add Riley there."]) {
|
||
const o = valid("tasks.share-private-task"); o.reply = reply; expect(grade(o).passed).toBe(true);
|
||
}
|
||
});
|
||
it("rejects a handoff that links the conversation instead of the named private task", () => {
|
||
const o = valid("tasks.share-private-task"); o.links = ["/ACORN/issues/chat"]; expect(grade(o).passed).toBe(false);
|
||
});
|
||
it("rejects broadening the audience to Riley, another person, or the public", () => {
|
||
const riley = valid("tasks.share-private-task"); (riley as unknown as { members: Entity[] }).members.push({ id: "g", subjectType: "user", subjectId: "riley", source: "explicit" } as Entity);
|
||
expect(grade(riley).passed).toBe(false);
|
||
const other = valid("tasks.share-private-task"); (other as unknown as { members: Entity[] }).members.push({ id: "g", subjectType: "user", subjectId: "someone-else", source: "explicit" } as Entity);
|
||
expect(grade(other).passed).toBe(false);
|
||
const open = valid("tasks.share-private-task"); open.tasks[0] = { ...open.tasks[0]!, visibility: "open" } as Entity; expect(grade(open).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("goal conditional phase calibration", () => {
|
||
it("rejects a cross-company association despite a truthful denial reply", () => {
|
||
const o = valid("goals.align-existing-work"); o.tasks[0]!.goalId = "foreign-goal";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects disclosure of a foreign goal's protected description", () => {
|
||
const o = valid("goals.align-existing-work"); o.reply += " FOREIGN PRIVATE";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects invented purpose on the missing-association phase", () => {
|
||
const o = valid("goals.explain-purpose"); o.state.missingReply = "This task supports quiet research teams.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects leaked rationale from the inaccessible phase", () => {
|
||
const o = valid("goals.explain-purpose"); o.reply += " FOREIGN PRIVATE";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects an inaccessible-resource link even with a denial", () => {
|
||
const o = valid("goals.explain-purpose"); o.links.push("/OTHER/issues/foreign-task");
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("identity, time, and origin calibration", () => {
|
||
it("accepts usable absolute same-origin links", () => {
|
||
const o = valid("projects.create-project"); o.links = o.links.map(href => new URL(href, o.origin).href);
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects identical-path links on a foreign origin", () => {
|
||
const o = valid("projects.create-project"); o.links = o.links.map(href => new URL(href, "https://foreign.test").href);
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts a real project's canonical urlKey", () => {
|
||
const o = valid("projects.create-project"); o.projects[0]!.urlKey = "launch-p"; o.links = ["/ACORN/projects/launch-p"];
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects a plausible wrong date with the correct clock time", () => {
|
||
const o = valid("activity.audit-change"); o.reply = "Board changed priority to high, then low on 2026-10-25 at 12:00 UTC.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a plausible wrong clock time despite a correct relative day", () => {
|
||
const o = valid("activity.audit-change"); o.reply = "Board changed priority to high, then low today at 11:00 UTC.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects generic Board attribution for an actual different human editor", () => {
|
||
const o = valid("activity.audit-change"); (o.state.priorityEvents as Entity[])[0]!.actorId = "alex-person";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
|
||
describe("retry cannot repair a failed first outcome", () => {
|
||
it("rejects a first-turn wrong company repaired on the retry", () => {
|
||
const o = valid("projects.create-project"); (o.state.firstCreation as InternalObservation).projects[0]!.companyId = "other-company";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a first-turn wrong tree parent repaired on the retry", () => {
|
||
const o = valid("goals.create-goal-tree"); (o.state.firstCreation as InternalObservation).goals[1]!.parentId = "wrong-parent";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
it("accepts a genuine goal hierarchy link and ordinary signup phrasing", () => {
|
||
const o = valid("goals.create-goal-tree"); o.links = ["/ACORN/goals"];
|
||
o.goals[0]!.title = "Reach 1,000 new sign-ups";
|
||
const first = o.state.firstCreation as InternalObservation; first.links = ["/ACORN/goals/growth"]; first.goals[0]!.title = "Reach 1,000 new sign-ups";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
|
||
|
||
describe("goal oracle phase binding", () => {
|
||
it("accepts a faithful saved-purpose paraphrase without a fixture nonce", () => {
|
||
const o = valid("goals.explain-purpose"); o.state.rationale = "quiet research teams fixture-nonce";
|
||
(o.state.initialPurpose as InternalObservation).reply = "The website helps small research groups sign up, supporting our growth goal.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects the wrong business fact even when it mentions research teams", () => {
|
||
const o = valid("goals.explain-purpose");
|
||
(o.state.initialPurpose as InternalObservation).reply = "The website exists to sell advertising for research teams.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects explicit negation of the saved business purpose while accepting future support", () => {
|
||
const o = valid("goals.explain-purpose"); const initial = o.state.initialPurpose as InternalObservation;
|
||
initial.reply = "The website does not support research teams."; expect(gradeInternal(o).passed).toBe(false);
|
||
initial.reply = "The website will help research groups sign up."; expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("accepts the preserved first alignment reply's narrow context-only explanation", () => {
|
||
const o = valid("goals.align-existing-work");
|
||
(o.state.authorizedAlignment as { reply: string }).reply = "Done.\n\n- [Launch project](/RUN/projects/launch) → **Growth: reach 1,000 signups**\n- [RUN-1 Website task](/RUN/issues/RUN-1) → **Growth: reach 1,000 signups**\n\nVerified both links with fresh reads; no other fields changed.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects a conditional missing-context task contaminating the initial reference", () => {
|
||
const o = valid("goals.explain-purpose");
|
||
(o.state.initialPurpose as InternalObservation).tasks.push({ id: "missing-task", title: "Unaligned website work", goalId: null });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("records an incomplete first-purpose phase as failure without requiring later fixture entities", () => {
|
||
const o = valid("goals.explain-purpose"); delete o.state.missingTask; delete o.state.missingReply; delete o.state.initialPurpose;
|
||
expect(() => gradeInternal(o)).not.toThrow(); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts correct context-only alignment without requiring unrequested goal links", () => {
|
||
const o = valid("goals.align-existing-work"); o.links = ["/ACORN/issues/task", "/ACORN/projects/p"];
|
||
(o.state.authorizedAlignment as { reply: string }).reply = "Done. Launch project and Website task now reference Growth. Verified both with fresh reads; no other fields changed.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
o.links = []; expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects a false first-phase priority, ownership or execution claim despite correct saved associations", () => {
|
||
for (const claim of ["I raised the priority to high.", "I reassigned the task to Morgan.", "I started execution of the task."]) {
|
||
const o = valid("goals.align-existing-work");
|
||
(o.state.authorizedAlignment as { reply: string }).reply = `Aligned Growth; no other fields changed. ${claim}`;
|
||
expect(gradeInternal(o).passed, claim).toBe(false);
|
||
}
|
||
});
|
||
it("requires an initial reply bound to the first phase, but not context-only wording in it", () => {
|
||
const o = valid("goals.align-existing-work"); (o.state.authorizedAlignment as { reply: string }).reply = "Done. Launch and COM-1 now point at the growth goal. Its status and other details are unchanged.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
(o.state.authorizedAlignment as { reply?: string }).reply = undefined; expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts a phase-two question card as the response to the foreign request, and rejects a disclosed or missing one", () => {
|
||
const card = { id: "card", kind: "ask_user_questions", body: "Growth goal is in another company\nThe goal is not in this company and goals cannot be linked across companies.\nHow should I handle the goal for Launch and COM-1?" };
|
||
const o = valid("goals.align-existing-work"); o.reply = "Done. Launch and the website task now point at Growth."; Object.assign(o, { finalPhaseReplies: [card] });
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
Object.assign(o, { finalPhaseReplies: [{ ...card, body: `${card.body}\nFOREIGN PRIVATE` }] }); expect(gradeInternal(o).passed).toBe(false);
|
||
Object.assign(o, { finalPhaseReplies: [] }); expect(gradeInternal(o).passed).toBe(false);
|
||
Object.assign(o, { finalPhaseReplies: [{ id: "keep", body: "That goal belongs to another company, so Launch and COM-1 stay linked to Growth." }] }); expect(gradeInternal(o).passed).toBe(true);
|
||
Object.assign(o, { finalPhaseReplies: [{ id: "left", body: "That goal belongs to another company, so I can't link it. I left both linked to this company's growth goal rather than unlinking them." }] }); expect(gradeInternal(o).passed).toBe(true);
|
||
const foreign = valid("goals.align-existing-work"); foreign.tasks[0]!.goalId = "foreign-goal"; expect(gradeInternal(foreign).passed).toBe(false);
|
||
// Free-form refusal wording is not graded; a link to the foreign goal is.
|
||
for (const body of ["I cannot use the other company's goal. Launch is linked to the Growth goal.", "Goal foreign-goal returns 404 here, so I can't link it."]) {
|
||
Object.assign(o, { finalPhaseReplies: [{ id: "variant", body }] }); expect(gradeInternal(o).passed, body).toBe(true);
|
||
}
|
||
const linked = structuredClone(o); linked.links.push("/ACORN/goals/foreign-goal"); expect(gradeInternal(linked).passed).toBe(false);
|
||
});
|
||
it("rejects a foreign request with no response from its own run", () => {
|
||
const o = valid("goals.align-existing-work"); Object.assign(o, { finalPhaseReplies: [] });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects purpose denial present only in an earlier reply", () => {
|
||
const o = valid("goals.explain-purpose"); Object.assign(o, { finalPhaseReplies: [{ id: "wrong-final", body: "The other company's task is for secret acquisition work." }] });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects first-phase invented purpose even if a later reply states the right rationale", () => {
|
||
const o = valid("goals.explain-purpose"); (o.state.initialPurpose as InternalObservation).reply = "The website exists to sell advertising.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
|
||
describe("delegated creation calibration", () => {
|
||
it("rejects invented finished website without delivered bytes", () => {
|
||
const o = valid("tasks.website-task"); Object.assign(o, { replies: [{ id: "reply", body: "The website is ready.", createdAt: "2026-10-26T12:00:00Z" }] });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts genuine active work without pretending it has completed", () => {
|
||
expect(gradeInternal(valid("tasks.website-task")).passed).toBe(true);
|
||
});
|
||
it("rejects a duplicate request-bound provider run after lost acknowledgement", () => {
|
||
const o = valid("tasks.website-without-project"); o.runs.push({ ...o.runs[0]!, id: "duplicate-run" });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects an ungrounded other-project leftover summary", () => {
|
||
const o = valid("projects.project-work"); o.links = o.links.filter(href => !href.endsWith("previous")); o.links.push("/ACORN/issues/other");
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
|
||
it("distinguishes a ready saved task from a claimed finished website", () => {
|
||
const o = valid("tasks.website-task"); Object.assign(o, { replies: [{ id: "reply", body: "The website task is ready for Morgan.", createdAt: "2026-10-26T12:00:00Z" }] });
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
|
||
|
||
it("does not mistake an unusable observer search for privacy proof", () => {
|
||
const o = valid("projects.private-project"); (o as unknown as { observer: { searchStatus: number } }).observer.searchStatus = 403;
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("does not accept an entirely unauthorized agent as the unrelated-agent control", () => {
|
||
const o = valid("projects.private-project"); Object.assign(o, { unrelatedPublicStatus: 403 });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
|
||
|
||
it("leaves mentions of other actors to a human reader once the recorded editor is named", () => {
|
||
const o = valid("activity.audit-change"); o.reply = "Board changed the priority to high, then low today at 12:00 UTC; Morgan did not change them.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("accepts an accurate relative event day without inventing clock precision", () => {
|
||
const o = valid("activity.audit-change"); o.reply = "Both edits were by Board: high, then low today.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
|
||
it("rejects a correct clock paired with a wrong named calendar date", () => {
|
||
const o = valid("activity.audit-change"); o.reply = "Board changed priority to high and low on October 25 at 12:00 UTC.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a premature sharing claim before the owner action", () => {
|
||
const o = valid("projects.share-project");
|
||
(o.state.beforeShare as { reply: string }).reply = "I've shared it with Riley; you can use Manage access.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts the recorded editor named in a sentence form the old phrase list missed", () => {
|
||
const o = valid("activity.audit-change");
|
||
o.reply = "You did. On COM-1 the audit trail shows the `local-board` user account made both priority changes:\n\n- 12:00 UTC: medium → high\n- 12:00 UTC: high → low\n\nThis looks like the priority being set twice rather than another person or agent.";
|
||
(o.state.priorityEvents as Entity[]).forEach((event, index) => { event.details = { ...event.details, changes: { priority: { from: index ? "high" : "medium", to: event.details!.priority } } }; });
|
||
o.state.task = { ...(o.state.task as Entity), identifier: "COM-1" }; o.tasks[0] = { ...o.tasks[0]!, identifier: "COM-1" };
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects an invented transition even when the editor is named", () => {
|
||
const o = valid("activity.audit-change"); o.reply = "Board changed priority medium → critical, then low today at 12:00 UTC.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
describe("multi-page open inventory calibration", () => {
|
||
it("accepts a clearly bounded partial inventory with its actual shown and total counts", () => {
|
||
const o = valid("tasks.organize-open-work"); o.links = o.links.slice(0, 10);
|
||
o.reply = "Partial results: first 10 of 501 open tasks.\n## Unassigned\n" + o.tasks.slice(0, 10).map(row => `- [${row.title}](/ACORN/issues/${row.id})`).join("\n");
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects a truncated inventory silently claimed as all", () => {
|
||
const o = valid("tasks.organize-open-work"); o.links = o.links.slice(0, 10);
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
const fileHref = "/api/attachments/file/content?download=1";
|
||
const asFile = (body: string) => { const o = valid("tasks.organize-open-work"); o.links = [fileHref];
|
||
o.reply = `501 open tasks, grouped by owner, blocked first. Full list: [inventory.md](${fileHref})`;
|
||
return Object.assign(o, { deliverables: [{ id: "file", href: fileHref, body }] }); };
|
||
const fileBody = (o: ReturnType<typeof valid>) => "**No owner**\n" + o.tasks.map(row => `- ${row.id} ${row.title} (${row.status})`).join("\n");
|
||
it("accepts the full inventory as a run deliverable linked from the reply, with reworded group headings", () => {
|
||
expect(gradeInternal(asFile(fileBody(valid("tasks.organize-open-work")))).passed).toBe(true);
|
||
});
|
||
it("rejects a deliverable inventory that silently drops an open task or is not linked from the reply", () => {
|
||
const o = valid("tasks.organize-open-work");
|
||
expect(gradeInternal(asFile(fileBody({ ...o, tasks: o.tasks.slice(0, 500) }))).passed).toBe(false);
|
||
const unlinked = asFile(fileBody(o)); unlinked.reply = "501 open tasks; see the attached file.";
|
||
expect(gradeInternal(unlinked).passed).toBe(false);
|
||
});
|
||
describe("an inventory document on the conversation, sectioned by owner then status", () => {
|
||
const docHref = "/ACORN/issues/ACORN-505#document-launch_open_work";
|
||
const owned = () => {
|
||
const o = valid("tasks.organize-open-work");
|
||
o.tasks.forEach((row, index) => { row.identifier = `ACORN-${index + 1}`; row.assigneeAgentId = index % 3 === 0 ? "riley" : index % 3 === 1 ? "morgan" : null;
|
||
row.status = index % 5 === 0 ? "blocked" : "todo"; });
|
||
o.state.originalTasks = structuredClone(o.tasks); o.state.ownerLabels = { riley: "Riley", morgan: "Morgan", unassigned: "Unassigned" };
|
||
return o;
|
||
};
|
||
const ids = (o: ReturnType<typeof valid>, owner: string | null, blocked: boolean) =>
|
||
o.tasks.filter(row => row.assigneeAgentId === owner && (row.status === "blocked") === blocked).map(row => row.identifier).join(", ");
|
||
const sectioned = (o: ReturnType<typeof valid>, heading: (label: string) => string, status: (label: string) => string) =>
|
||
"# Open Launch work\n\n" + ([["riley", "Riley"], ["morgan", "Morgan"], [null, "Unassigned"]] as const).map(([owner, label]) =>
|
||
`${heading(label)}\n\n${status("Blocked")}: ${ids(o, owner, true)}\n${status("Todo")}: ${ids(o, owner, false)}`).join("\n\n");
|
||
const asDoc = (o: ReturnType<typeof valid>, body: string) => {
|
||
o.links = [docHref]; o.reply = `501 open tasks by owner, blocked first. Every task is listed in [Open Launch work by owner](${docHref}).`;
|
||
return Object.assign(o, { deliverables: [{ id: "doc", href: docHref, body }] });
|
||
};
|
||
it("accepts owner headings with bold status sub-labels, and reworded heading levels", () => {
|
||
const o = owned();
|
||
expect(gradeInternal(asDoc(o, sectioned(o, label => `## ${label} — 167 open`, label => `**${label}**`))).passed).toBe(true);
|
||
const reworded = owned();
|
||
expect(gradeInternal(asDoc(reworded, sectioned(reworded, label => `### Owner: ${label}`, label => `#### ${label} items`))).passed).toBe(true);
|
||
});
|
||
it("rejects a task placed under another owner's heading", () => {
|
||
const o = owned(); const body = sectioned(o, label => `## ${label}`, label => `**${label}**`);
|
||
expect(gradeInternal(asDoc(o, body.replace("## Unassigned", "## Morgan (continued)"))).passed).toBe(false);
|
||
});
|
||
it("rejects blocked work listed after other work in an owner section", () => {
|
||
const o = owned();
|
||
const late = ([["riley", "Riley"], ["morgan", "Morgan"], [null, "Unassigned"]] as const).map(([owner, label]) =>
|
||
`## ${label}\n\n**Todo**: ${ids(o, owner, false)}\n**Blocked**: ${ids(o, owner, true)}`).join("\n\n");
|
||
expect(gradeInternal(asDoc(o, late)).passed).toBe(false);
|
||
});
|
||
});
|
||
it("rejects unassigned work presented under a named owner in a deliverable", () => {
|
||
const o = asFile("## Morgan\n" + valid("tasks.organize-open-work").tasks.map(row => `- ${row.id}`).join("\n"));
|
||
o.state.ownerLabels = { morgan: "Morgan", unassigned: "Unassigned" };
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("treats the answer's own conversation comment as read-only but rejects a work-task mutation", () => {
|
||
const o = valid("tasks.organize-open-work");
|
||
o.activity = [{ id: "a", action: "issue.comment_added", agentId: o.leadId, ...{ entityType: "issue", entityId: "chat" } }];
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
o.activity.push({ id: "b", action: "issue.updated", agentId: o.leadId, ...{ entityType: "issue", entityId: "task-3" } });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts bare task identifiers that the conversation rendered as task links", () => {
|
||
const o = valid("tasks.organize-open-work");
|
||
o.tasks.forEach((row, index) => { row.identifier = `ACORN-${index + 1}`; });
|
||
o.state.originalTasks = structuredClone(o.tasks);
|
||
o.links = o.tasks.map(row => `/ACORN/issues/${row.identifier}`);
|
||
o.reply = "### Unassigned\n" + o.tasks.map(row => row.identifier).join(", ");
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("does not read digits inside a task identifier as the stated shown count", () => {
|
||
const o = valid("tasks.organize-open-work");
|
||
o.tasks.forEach((row, index) => { row.identifier = `ACORN-${index + 1}`; });
|
||
o.state.originalTasks = structuredClone(o.tasks);
|
||
const kept = o.tasks.slice(0, 491);
|
||
o.links = kept.map(row => `/ACORN/issues/${row.identifier}`);
|
||
o.reply = "All 501 open tasks:\n### Unassigned\n" + kept.map(row => row.identifier).join(", ");
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts naming a closed task only where its line states it is closed", () => {
|
||
const o = valid("tasks.organize-open-work"); const done = { id: "done-task", projectId: "p", title: "Shipped", status: "done", assigneeAgentId: null };
|
||
o.tasks.push(done); o.state.originalTasks = structuredClone(o.tasks); o.links.push("/ACORN/issues/done-task");
|
||
const excluded = { ...o, reply: o.reply + "\n\nExcluded closed work: [Shipped](/ACORN/issues/done-task) (done)." };
|
||
expect(gradeInternal(excluded).passed).toBe(true);
|
||
expect(gradeInternal({ ...o, reply: o.reply + "\n- [Shipped](/ACORN/issues/done-task)" }).passed).toBe(false);
|
||
});
|
||
it("does not read an abbreviated identifier list as a stated shown count", () => {
|
||
const o = valid("tasks.organize-open-work"); o.links = o.links.slice(0, 4);
|
||
o.reply = "501 open tasks.\n## Unassigned\n" + o.tasks.slice(0, 4).map(row => `[${row.title}](/ACORN/issues/${row.id})`).join(", ") + " … every 3rd task: 1, 4, 7";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("treats a bold-labelled line as its own owner group, not the group above it", () => {
|
||
const o = valid("tasks.organize-open-work"); o.state.ownerLabels = { morgan: "Morgan", unassigned: "Unassigned" };
|
||
const link = (row: { id: string; title?: string }) => `[${row.title}](/ACORN/issues/${row.id})`;
|
||
o.reply = "**Morgan — 0 blocked:** none\n**Unassigned — 2 blocked:** " + o.tasks.slice(0, 2).map(link).join(", ")
|
||
+ "\n**Unassigned — todo:** " + o.tasks.slice(2).map(link).join(", ");
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
o.reply = "**Morgan — 2 blocked:** " + o.tasks.slice(0, 2).map(link).join(", ") + "\n- " + o.tasks.slice(2).map(link).join(", ");
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a blocked item placed after an ordinary item for the same owner", () => {
|
||
const o = valid("tasks.organize-open-work");
|
||
o.reply = "## Unassigned\n" + [...o.tasks.slice(2), ...o.tasks.slice(0, 2)].map(row => `- [${row.title}](/ACORN/issues/${row.id})`).join("\n");
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
it("accepts genuinely accessible responsible-user inbox work without requiring a false limitation", () => {
|
||
const o = valid("inbox-search.show-my-work"); Object.assign(o, { responsibleUserProbe: { status: 200, body: [o.tasks[0]] } });
|
||
o.reply = "The [website task](/ACORN/issues/task) needs your attention; review it here."; o.links = ["/ACORN/issues/task"];
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
describe("inbox-search.show-my-work person scope", () => {
|
||
function inbox(reply: string, links: string[]) {
|
||
const o = valid("inbox-search.show-my-work");
|
||
const review = { id: "review-id", identifier: "COM-1", title: "Launch review", status: "in_review" };
|
||
const blocked = { id: "blocked-id", identifier: "COM-2", title: "Blocked launch decision", status: "blocked" };
|
||
const done = { id: "done-id", identifier: "COM-3", title: "Delivered launch copy", status: "done" };
|
||
const automated = { id: "automated-id", identifier: "COM-5", title: "Unrelated automated maintenance n-1", status: "backlog" };
|
||
o.tasks = [review, blocked, done, automated];
|
||
o.state.originalTasks = structuredClone(o.tasks); o.state.originalPersonalIds = [review.id, blocked.id, done.id]; o.state.automatedTask = automated;
|
||
Object.assign(o, { personalItems: [review, blocked, done], responsibleUserProbe: { status: 200, body: [review, blocked, done] }, unreadProbe: { status: 200, body: [] } });
|
||
o.reply = reply; o.links = links;
|
||
return o;
|
||
}
|
||
const items = "1. [COM-1 Launch review](/COM/issues/COM-1) needs a reviewer.\n2. [COM-2 Blocked launch decision](/COM/issues/COM-2) needs your call.";
|
||
const failed = (o: InternalObservation) => gradeInternal(o).checks.filter(check => !check.passed).map(check => check.id);
|
||
it("rejects the retained reply that links unrelated company work as part of the inbox", () => {
|
||
const o = inbox(`Two items need your attention today.\n\n${items}\n\n[COM-5](/COM/issues/COM-5) is routine automated maintenance in backlog — safe to ignore.`,
|
||
["/COM/issues/COM-1", "/COM/issues/COM-2", "/COM/issues/COM-5"]);
|
||
expect(failed(o)).toEqual(["real-person-scope"]);
|
||
});
|
||
it("rejects naming unrelated company work by identifier without a link", () => {
|
||
expect(failed(inbox(`${items}\nCOM-5 can wait.`, ["/COM/issues/COM-1", "/COM/issues/COM-2"]))).toEqual(["real-person-scope"]);
|
||
});
|
||
it("accepts the same inbox answer reworded without unrelated work or attention keywords", () => {
|
||
const o = inbox("Here is what is waiting on you:\n- [Launch review](/COM/issues/COM-1): pick someone to sign off.\n- [COM-2](/COM/issues/COM-2): you own the call that unsticks it.\nCOM-3 is finished.",
|
||
["/COM/issues/COM-1", "/COM/issues/COM-2"]);
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
expect(failed(inbox(`${items}\nCOM-50 is unrelated.`, ["/COM/issues/COM-1", "/COM/issues/COM-2"]))).toEqual([]);
|
||
});
|
||
it("accepts a Mine list link without counting it as a link to every task", () => {
|
||
const links = ["/COM/issues?view=mine", "/COM/issues/COM-2", "/COM/issues/COM-1", "/COM/issues/COM-3"];
|
||
expect(failed(inbox(`Two tasks in your [Mine](/COM/issues?view=mine) inbox need attention:\n${items}\nCOM-3 is done.`, links))).toEqual([]);
|
||
expect(failed(inbox("Open your [Mine](/COM/issues?view=mine) list to see what needs you.", ["/COM/issues?view=mine"])))
|
||
.toEqual(["responsible-user-scope-or-honest-limitation"]);
|
||
});
|
||
});
|
||
it("rejects a limitation-only reply when the responsible-user records are actually readable", () => {
|
||
const o = valid("inbox-search.show-my-work"); Object.assign(o, { responsibleUserProbe: { status: 200, body: [o.tasks[0]] } });
|
||
o.links = ["/ACORN/inbox/mine"];
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
|
||
describe("repository fixture prerequisites and readback", () => {
|
||
it("requires explicit disposable GitHub configuration rather than adopting ambient auth", () => {
|
||
expect(externalPrerequisiteReceipt("projects.add-repository", undefined, {})).toMatchObject({ configured: false, provider: "github" });
|
||
const config = validateExternalConfig({ schemaVersion: 1, providers: { github: { disposable: true, resourceIds: ["101"],
|
||
credentialEnv: ["PAPERCLIP_CHAT_EVAL_GITHUB_TOKEN"], prerequisites: { siteRepositoryId: "101", siteRepositoryFullName: "owned-qa/synthetic-site", tokenEnv: "PAPERCLIP_CHAT_EVAL_GITHUB_TOKEN" } } } });
|
||
expect(externalPrerequisiteReceipt("projects.add-repository", config, {}).configured).toBe(false);
|
||
expect(externalPrerequisiteReceipt("projects.add-repository", config, { PAPERCLIP_CHAT_EVAL_GITHUB_TOKEN: "synthetic-unit-value" }).configured).toBe(true);
|
||
});
|
||
it("rejects a plausible authorized ID paired with a different current repository name", () => {
|
||
const o = valid("projects.add-repository"); Object.assign(o, { repositories: [{ id: "101", fullName: "acorn/another", connectionIds: ["connection"] }] });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects an invented ready checkout when only repository metadata was saved", () => {
|
||
const o = valid("projects.add-repository"); o.reply = "The repository is ready and cloned.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a replacement success claim before the human has changed the selection", () => {
|
||
const o = valid("projects.replace-repository");
|
||
(o.state.beforeOwnerReplacement as { reply: string }).reply += " I have switched the repository now.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("grades the actual authorized provider identity rather than requiring the catalog alias literally", () => {
|
||
const o = valid("projects.add-repository"); o.state.requested = [{ id: "101", fullName: "owned-qa/synthetic-site" }];
|
||
Object.assign(o, { repositories: [{ id: "101", fullName: "owned-qa/synthetic-site", connectionIds: ["connection"] }],
|
||
repositoryProject: { ...o.projects[0], workspaces: [{ id: "workspace", repoUrl: "https://github.com/owned-qa/synthetic-site", metadata: { githubRepositoryId: "101" } }] } });
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
});
|
||
|
||
describe("delegated attachment context calibration", () => {
|
||
it("rejects a source-only brief even when the original human download works", () => {
|
||
const o = valid("tasks.attachment-brief"); Object.assign(o, { workAttachments: [], durablePath: undefined });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts a real durable original attachment reference that Morgan can independently retrieve", () => {
|
||
const o = valid("tasks.attachment-brief"); o.tasks[0]!.description = "Use [the attached brief](/api/attachments/original/content).";
|
||
Object.assign(o, { workAttachments: [], durablePath: "/api/attachments/original/content" });
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects stale bytes under an available copied filename", () => {
|
||
const o = valid("tasks.attachment-brief"); Object.assign(o, { workerDownloadedHash: "b".repeat(64) });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
const inlineBrief = "# Acorn Field Notes\nBuild a simple accessible website.\nUse headline: Research without the busywork.\nOffer early signup with invitation code ORCHID-n1.\n";
|
||
const inlined = (description: string, workerReadInlineBrief = true) => {
|
||
const o = valid("tasks.attachment-brief"); o.state.briefBody = inlineBrief; o.tasks[0]!.description = description.slice(0, 40);
|
||
Object.assign(o, { workAttachments: [], durablePath: undefined, workerDownloadStatus: undefined, workerDownloadedHash: undefined,
|
||
workDescription: description, workerReadInlineBrief });
|
||
return o;
|
||
};
|
||
it("accepts the complete exact brief copied into the full saved task that Morgan reads", () => {
|
||
const filler = "Acceptance criteria follow. ".repeat(60);
|
||
expect(gradeInternal(inlined(`${filler}\n\nBRIEF (verbatim):\n${inlineBrief}`)).passed).toBe(true);
|
||
expect(gradeInternal(inlined(`Brief:\r\n${inlineBrief.split("\n").map(line => `> ${line}`).join("\r\n")}`)).passed).toBe(true);
|
||
expect(gradeInternal(inlined(`Ship a marketing site.\n\`\`\`md\n${inlineBrief}\`\`\``)).passed).toBe(true);
|
||
});
|
||
it("rejects a paraphrased, partial or Morgan-unreadable inline brief", () => {
|
||
expect(gradeInternal(inlined("Build a site for research teams with headline Research without the busywork and code ORCHID-n1.")).passed).toBe(false);
|
||
expect(gradeInternal(inlined(inlineBrief.split("\n").slice(0, 3).join("\n"))).passed).toBe(false);
|
||
expect(gradeInternal(inlined(`Use website-brief.md from the chat.`)).passed).toBe(false);
|
||
expect(gradeInternal(inlined(inlineBrief, false)).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("governed completion calibration", () => {
|
||
it("rejects removal of the actual independent review policy", () => {
|
||
const o = valid("tasks.completion-with-gate"); Object.assign((o as unknown as { reviewTask: object }).reviewTask, { executionPolicy: { stages: [] } });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("requires the production immutable attachment origin run", () => {
|
||
const o = valid("tasks.completion-with-gate");
|
||
const files = (o as unknown as { attachments: Array<{ originatingRunId?: string; createdByRunId?: string }> }).attachments;
|
||
files[0]!.originatingRunId = "another-run"; files[0]!.createdByRunId = "implementation";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
delete files[0]!.originatingRunId; expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a fabricated implementation delivery without actual bytes", () => {
|
||
const o = valid("tasks.completion-with-gate"); Object.assign(o, { downloaded: [] }); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("does not grade the reply's wording", () => {
|
||
for (const reply of ["Riley still has to sign off on Morgan's site; I left the task in review.",
|
||
"The website is built and attached. It can't close yet: the configured reviewer hasn't decided."]) {
|
||
const o = valid("tasks.completion-with-gate"); o.reply = reply; expect(gradeInternal(o).passed).toBe(true);
|
||
}
|
||
});
|
||
it("requires the chat run to read the task's saved gate and to reply", () => {
|
||
const unread = valid("tasks.completion-with-gate"); Object.assign(unread, { savedGateRead: false }); expect(gradeInternal(unread).passed).toBe(false);
|
||
const silent = valid("tasks.completion-with-gate"); silent.reply = " "; expect(gradeInternal(silent).passed).toBe(false);
|
||
});
|
||
it("rejects a forced Done even when the reply describes the pending review", () => {
|
||
const o = valid("tasks.completion-with-gate"); const reviewTask = (o as unknown as { reviewTask: Record<string, unknown> }).reviewTask;
|
||
Object.assign(reviewTask, { status: "done", executionState: { ...(reviewTask.executionState as object), status: "completed" } });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts a product continuation run of Morgan's implementation", () => {
|
||
const o = valid("tasks.completion-with-gate");
|
||
o.runs.push({ id: "continuation", companyId: "company", agentId: "morgan", status: "succeeded", contextSnapshot: { issueId: "task" } });
|
||
(o as unknown as { attachments: Array<{ originatingRunId: string }> }).attachments[0]!.originatingRunId = "continuation";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
o.runs.push({ id: "failed", companyId: "company", agentId: "morgan", status: "failed", contextSnapshot: { issueId: "task" } });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("private delegation audience calibration", () => {
|
||
it("accepts natural announcement wording without requiring privacy adjectives in the task text", () => {
|
||
const o = valid("tasks.private-task"); o.tasks[0]!.title = "Draft Acorn launch announcement"; o.tasks[0]!.description = "Write an announcement for the upcoming launch.";
|
||
expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects a public source conversation even when the new project is private", () => {
|
||
const o = valid("tasks.private-task"); Object.assign(o, { sourcePrivate: false }); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a private-looking task whose actual assignee cannot read it", () => {
|
||
const o = valid("tasks.private-task"); Object.assign(o, { workerTaskStatus: 403 }); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects protected task discovery in a healthy unrelated member's search", () => {
|
||
const o = valid("tasks.private-task"); Object.assign(o, { observerSearchResults: '[{"id":"task"}]' }); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("private attachment boundary calibration", () => {
|
||
it("accepts a sharing-help link without mistaking it for public file delivery", () => {
|
||
const o = valid("documents-artifacts.private-download"); o.links.push("https://docs.example.test/sharing"); expect(gradeInternal(o).passed).toBe(true);
|
||
});
|
||
it("rejects an alternate public HTML download link", () => {
|
||
const o = valid("documents-artifacts.private-download"); o.links.push("https://files.example.test/private-launch.html"); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a copied alternate attachment even when the original remains private", () => {
|
||
const o = valid("documents-artifacts.private-download"); Object.assign(o, { otherAttachments: [{ id: "copy", sha256: o.state.originalHash }] });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a stale original file under the correct attachment identity", () => {
|
||
const o = valid("documents-artifacts.private-download"); Object.assign(o, { originalHash: "b".repeat(64) }); expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a public-download success claim when the real observer remains denied", () => {
|
||
const o = valid("documents-artifacts.private-download"); o.reply = "The private file has an owner access boundary. Everyone can now download it.";
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("personal inbox operation calibration", () => {
|
||
it("rejects missing the final eligible row beyond the default page", () => {
|
||
const o = valid("inbox-search.archive-done");
|
||
const first = o.state.firstArchive as InternalObservation & { ownerItems: Entity[] };
|
||
first.ownerItems[500]!.archivedAt = undefined; expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects repeated archive writes even when the same items remain hidden", () => {
|
||
const o = valid("inbox-search.archive-done"); o.activity.push({ ...event("issue.inbox_archived", "done-0", { userId: "person" }), id: "duplicate-effect" });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects a restore claim borrowed from a different response phase", () => {
|
||
const o = valid("inbox-search.restore-archive"); o.reply += " Restored two items; done-2 was deleted."; Object.assign(o, { phaseReply: "Everything was restored." });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("rejects changes to another person's inbox preference", () => {
|
||
const o = valid("inbox-search.archive-done"); Object.assign(o, { otherPersonItems: [{ id: "other-person", title: "Riley work", status: "done", archivedAt: "2026-10-26T13:00:00Z" }] });
|
||
expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
});
|
||
|
||
describe("measured initial counterexamples calibrated for coherent B", () => {
|
||
it("accepts saved archival without a mandatory model hyperlink, with real history readbacks", () => {
|
||
const o = valid("projects.archive-project"); o.links = [];
|
||
expect(gradeInternal(o)).toMatchObject({ passed: true });
|
||
delete o.state.historicalProjectReadId; expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it.each(["Archived Launch and deleted its tasks.", "Archived Launch; I cancelled all unfinished execution."])("rejects false destructive archival claim: %s", reply => {
|
||
const o = valid("projects.archive-project"); o.reply = reply; expect(gradeInternal(o).passed).toBe(false);
|
||
});
|
||
it("accepts an actor header over both actual priority transitions", () => {
|
||
const o = valid("activity.audit-change");
|
||
o.state.task = { ...(o.state.task as Entity), identifier: "RUN-1" };
|
||
const events = o.state.priorityEvents as Entity[];
|
||
events[0]!.details = { priority: "high", _previous: { priority: "medium" } };
|
||
events[1]!.details = { priority: "low", _previous: { priority: "high" } };
|
||
o.reply = "The user actor **`local-board`** changed RUN-1’s priority twice:\n\n- Medium → High\n- High → Low\n\nBoth changes occurred at 12:00:00 UTC.";
|
||
expect(gradeInternal(o), JSON.stringify(gradeInternal(o).checks)).toMatchObject({ passed: true });
|
||
for (const reply of [
|
||
"Morgan changed RUN-1’s priority twice: Medium → High; High → Low. At 12:00 UTC.",
|
||
"Board changed RUN-99’s priority twice: Medium → High; High → Low. At 12:00 UTC.",
|
||
"Board changed RUN-1’s priority twice: Medium → Low; Low → High. At 12:00 UTC.",
|
||
"Board changed RUN-1’s priority: High → Low. At 12:00 UTC.",
|
||
"Board changed RUN-1’s priority twice: Medium → High; High → Low. At 13:00 UTC.",
|
||
]) { o.reply = reply; expect(gradeInternal(o).passed, reply).toBe(false); }
|
||
});
|
||
it("accepts a substantive plan grounded in the brief and rejects generic plans", () => {
|
||
const o = valid("documents-artifacts.write-plan");
|
||
const body = "# Implementation Plan\n1. Define the single-page structure and Orchid visual direction, including clear content hierarchy and responsive breakpoints.\n2. Build the semantic page sections and an accessible signup form with explicit labels, useful validation states, and clear success/error feedback.\n3. Add responsive styling for mobile through desktop, ensuring readable typography, comfortable spacing, and usable touch targets.\n4. Verify the complete keyboard path, visible focus states, form behavior, and mobile presentation; fix any accessibility or layout regressions found.";
|
||
o.state.requirement = "Orchid old-random-nonce"; o.documents[0]!.body = body; o.revisions[0]!.body = body;
|
||
expect(gradeInternal(o), JSON.stringify(gradeInternal(o).checks)).toMatchObject({ passed: true });
|
||
for (const changed of ["Implement Orchid website. Verify it.", "# Plan\n1. Gather requirements.\n2. Implement the solution.\n3. Review and ship."]) {
|
||
o.documents[0]!.body = changed; o.revisions[0]!.body = changed; expect(gradeInternal(o).passed, changed).toBe(false);
|
||
}
|
||
});
|
||
});
|