mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 11:01:16 +01:00
test(e2e): cover browser task editing and reopening (#18782)
## Thinking Path > - Paperclip lets people manage tasks and agent work. > - Daily QA uses reusable Product E2E journeys against a named target. > - Existing task journeys cover creation and assignment. > - They lack saved title and label edits across completion and reopening. > - This pull request adds one browser-only lifecycle journey with independent saved-state checks. > - It also puts shared registry entries on separate lines so independent case PRs can use distinct insertion points. > - The registry formatting preserves every token, value, entry order and case body. ## Linked Issues or Issue Description Refs: #18655. **What existing behavior does this improve?** Product E2E coverage of ordinary task editing and completion. **Current behavior** The reusable catalog covers task creation and assignment. It lacks one journey that edits a task, reloads it, completes it and reopens it while retaining its saved context. Several shared registries put all entries on one line, so independent case additions conflict. **Proposed behavior** Create a task assigned to the signed-in person. Edit its title and create one label through the visible UI. Change its status to In Progress, reload it, mark it Done, reload it, reopen it to Todo and reload it again. Require the exact saved task and label throughout. Preserve its description, assignee, project and priority. Allow no agent run or model usage. Put the existing shared registry entries on separate lines without changing their contents or order. ## What Changed - Add `task-edit-lifecycle` and six required checks for browser creation, saved edits, reload, completion and reopening, preserved context and no agent work. - Add valid, wrong and missing evidence controls. Retain allowlisted observations and bounded GET failure receipts in private evidence. - Register the case, feature associations, catalog expectation, source-hash inputs and browser-only result validation. - Extend target ownership for run-owned labels and one exact predeclared title change. Require compatible scoped cleanup before selecting the case. - Open the real side panel or Properties control when needed. Exclude inert and aria-hidden Labels rows and require visible readback. Document inline title editing and the case prerequisites. - Keep priority editing disabled. Verify that the saved priority remains unchanged. - Keep Cursor test observations outside its existing Stop deadline, preserve the full rejection/Stop overlap matrix and add pre-release no-provider-call controls. Production behavior and deadline values stay unchanged. - Format five shared registry files with one entry per line: the JourneyId and AgentJourney unions, definition filenames, expected catalog IDs and agent run counts. No registry entry moves in this PR. ## Verification - Current head is `d1af3aae3712455c90db937fa7a5141555710506`. Its only change from integrated head d40f28 is the Cursor driver test-fixture ordering repair described below. The prior normal merge consumes actual master `d59791828482f213713ca55000c40e6c9c6c03e5`, including reviewed UI repair #18848 and the existing idle-history fixture repair from #18853. It has both real parents, no conflicts and the required trailer. All seven lifecycle case/oracle/catalog/documentation blobs and six selected registry constructs remain exact 10625b. The idle-history fixture is byte-identical to master. The PR diff has 13 paths, including the one Cursor test file, and no feature-map or production-driver changes. - 167 focused tests pass across catalog, target contracts, agent evidence, lifecycle evidence and observation reads. E2E TypeScript and credential-free discovery of21 Product E2E cases pass. The real merged idle-history and exposure-reservation suites pass all 109 tests in132 seconds, including the owned HTTP-process adoption assertion and explicit unknown-listener denial/drift controls. Canonical shared/SDK, database and Runner TypeScript dependencies were rebuilt for the changed master inputs. All4,220 tracked package/script/root inputs and 4,954 dependency outputs are recorded; the outputs remain unchanged after the tests and browser journey. The first database build stopped because child PATH selected a different global pnpm and its version switch could not fetch signed metadata. The preserved attempt made no package or source change. Using the already-installed verified pnpm9 on child PATH passed. No install, Rust or provider-build global setup ran. - One fresh canonical managed-local browser journey passes on exact d40f28: all six checks and authentication, setup, journey, collection and cleanup pass. Cleanup releases the disposable instance. The journey takes 12.059 seconds and has explicit zero-run readback at all five saved states. Its runtime commit isd40f28. Definition hash is `f262707de2e544cecdb9c5c4c5222d61f04a122575b438658dec069e1a93f9cb`. The prior10625b local6-check/5-phase pass remains separately recorded with its own source and definition hash. - A real closed-panel control previously reproduced the original label-interaction failure using only the presentation preference `paperclip:panel-visible=false`. The corrected journey passed all six unchanged checks through an ordinary Show properties click. A separate canonical743 local run passed all six checks and five phases with released cleanup. No domain outcome was supplied by that control. - A separate QA staging qualification passed all six checks and five phases on exact743, with closed cleanup and verified zero agent work. That original pass remains bound to743. It is not regraded or relabeled as a staging run of10625b. Daily admission remains a separate coordinator action. - Earlier material failures remain part of the verification record. The first managed-local setup failed before authentication with an embedded PostgreSQL exit code of null; its original cause remains unresolved. A later loader diagnostic found missing library aliases in this isolated checkout. Existing bundled libraries were hash-verified and the aliases were restored only there. Subsequent real runs exposed an overlong label name, a status picker name mismatch and the closed-panel issue. Each failed result and cleanup record is retained. The fixes preserve the full campaign name, validate the production48-character label limit before mutations, and use the actual visible controls. - Original full CI [41652807327](https://github.com/paperclipai/paperclip/actions/runs/41652807327) failed an unchanged Discord fixture assertion, attention-retention fixture limits and the CLI import/export startup limit. One bounded recheck recovered Discord but repeated the other failures. The reviewed merged retention-fixture repair was consumed normally through #18770. Full CI [41667811850](https://github.com/paperclipai/paperclip/actions/runs/41667811850) passed all E2E shards but exposed inbox/account-fixture failures. This branch consumed the actual reviewed repairs from #18798 and #18799. Original logs and failed assertions remain retained; no timeout or oracle was weakened. - Original743 CI [41682757346](https://github.com/paperclipai/paperclip/actions/runs/41682757346) failed the unchanged CLI suite's30-second child-server startup hook. A bounded local phase probe passed the same startup oracle but did not prove the original CI cause. One coordinator-approved official recheck followed. Timestamp comparison shows46 non-skipped jobs retained their original start and end times; only the failed CLI job and final aggregate have new start and end times. The skipped selection job remains unclassified. New GitHub job IDs alone did not prove new executions. The original startup failure remains preserved and its cause remains unproven. - Formatting-head CI [41726104217](https://github.com/paperclipai/paperclip/actions/runs/41726104217) failed the upstream idle-history process-adoption test and dependent verify job:46 jobs passed, one was skipped and two failed. Its actual synthetic checkout had parents masterf187 and10625b; the failed test is absent from 10625b itself. The assertion expected one adopted process and received zero. A pure production-function control proves a conditional reserved-port drift path, but the actual failed CI port and full reconciliation counters were not logged. The actual CI cause remains unproven. This branch consumes the already-reviewed bounded fixture-port repair in #18853 through its normal actual-master merge. The original failure is retained. No unchanged-code retry, timeout increase, skipped assertion or ownership-policy change was used. - Integrated-head CI [41730231620](https://github.com/paperclipai/paperclip/actions/runs/41730231620) passed the formerly failing server1/12 lane, Typecheck, Build and E2E. It failed the Cursor package fixture below and the dependent verify job. A separate server4/12 job lost runner communication; the exact runner instance and Spot-request readback confirm a capacity interruption. Its unavailable log is not treated as a test pass. The original failures remain retained. No old-head retry was made. - The Cursor fixture sets a 300 ms cancellation budget, holds a rejected-create checkpoint and originally performed directory/file observations while that clock ran. Controlled400ms observation latency on the original fixture reproduces the same fenced Stop error. The same diagnostic passes after moving file observations before the cancellation clock, or while the initial Stop-checkpoint gate still prevents it from starting. The driver,300ms configuration and timeout negatives are unchanged. Every original assertion AST remains, and three new assertions forbid provider observation/cancel calls before releasing the checkpoint. All81 driver tests,12 client tests and43 checkpoint-archive tests pass; Runner TypeScript build and E2E TypeScript pass. The actual CI inner error and observation duration were not logged, so its precise original cause remains unproven. - Final d1af3a full CI [41733938939](https://github.com/paperclipai/paperclip/actions/runs/41733938939) attempt3 is successful. Attempt1 lost four exact-bound Spot runners; attempt2 recovered the Runner and both server lanes but lost the replacement E2E runner to another confirmed Spot capacity interruption. Two separate coordinator-authorized official failed-job rechecks were recorded. Attempt3 passed the E2E path and final aggregate. Successful executions retain their original timestamps; new job IDs alone were not used as proof of re-execution. All original failures remain retained. - Fresh Greptile check126258142754 completed successfully on exact d1af3a with 5/5 and zero unresolved threads. Full recursive workspace typecheck, all workspace tests and the full build were not repeated locally; the corresponding hosted full CI gates pass. ## Risks The case requires target support for `label` ownership and scoped `taskEdit.revisedTitle` cleanup. Older target adapters must not select it. Cleanup must reconcile lost creation responses and exact predeclared title saves without touching other resources. Registry formatting changes the harness definition hash because that hash includes source bytes. Historical runs must keep their original source and hash. The formatting does not change runtime values, order, dispatch, case checks or product behavior. Later case PRs must verify their actual merge resolutions and final-head checks before admission. ## Model Used OpenAI GPT-6 through Codex with reasoning, tool use and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
ab2d2098a2
commit
8771606fd8
13 files changed
+610
-13
No files matched your search
@@ -994,21 +994,32 @@ describe("Cursor Cloud native driver", () => {
|
||||
try {
|
||||
const turn = await f.session.startTurn({ message: { role: "user", text: rejection === "unsent_payload" ? "\u0001".repeat(400_000) : "Rejected task" } });
|
||||
await entered.promise;
|
||||
const observeHeldCheckpoint = async () => {
|
||||
const checkpointPath = join(f.root, (await readdir(f.root)).find(name => /^checkpoint-.*\.json$/.test(name))!);
|
||||
expect(JSON.parse(await readFile(checkpointPath, "utf8")).phase).not.toBe("idle");
|
||||
return checkpointPath;
|
||||
};
|
||||
let checkpointPath: string;
|
||||
if (overlap === "stop_write") {
|
||||
stop(); await stopHeld.promise;
|
||||
rejectCreate.resolve(); await new Promise<void>(accept => setImmediate(accept));
|
||||
// The Stop checkpoint gate still holds its cancellation clock here.
|
||||
checkpointPath = await observeHeldCheckpoint();
|
||||
releaseStop.resolve();
|
||||
await waitFor(() => pausedRejection || stopDone);
|
||||
await Promise.race([rejectionHeld.promise, stopped!]);
|
||||
expect(pausedRejection, "the definite rejection must reach durable settlement even while stopping").toBe(true);
|
||||
} else {
|
||||
rejectCreate.resolve(); await rejectionHeld.promise;
|
||||
// File observations must not consume Stop's 300ms settlement budget.
|
||||
checkpointPath = await observeHeldCheckpoint();
|
||||
stop(); await new Promise<void>(accept => setImmediate(accept));
|
||||
}
|
||||
expect(stopDone, "Stop must wait for the rejected-create checkpoint").toBe(false);
|
||||
expect(await f.session.read()).toMatchObject({ phase: "stopping" });
|
||||
await expect(f.session.startTurn({ message: { role: "user", text: "Too early" } })).rejects.toThrow("unresolved remote work");
|
||||
const checkpointPath = join(f.root, (await readdir(f.root)).find(name => /^checkpoint-.*\.json$/.test(name))!);
|
||||
expect(JSON.parse(await readFile(checkpointPath, "utf8")).phase).not.toBe("idle");
|
||||
expect(peer.getAgent).toHaveBeenCalledTimes(before.getAgent);
|
||||
expect(peer.getRun).toHaveBeenCalledTimes(before.getRun);
|
||||
expect(peer.cancelRun).toHaveBeenCalledTimes(before.cancel);
|
||||
releaseRejection.resolve();
|
||||
expect(await stopped).toEqual({ error: null });
|
||||
await waitFor(() => f.events.some(event => event.turnId === turn.turnId && event.eventType === "turn.failed" && event.payload.remoteWorkAccepted === false));
|
||||
|
||||
@@ -33,6 +33,7 @@ never start a server or depend on a tenant filesystem.
|
||||
| `ask-question` | Company, configured agent capability, positive model allowance and exactly one permitted work run | Reuse the existing cross-model Ask question prompt; require one successful run with matching company/task/agent/model/environment, one saved exact answer, and visible readback after reload |
|
||||
| `question-answer-resume` | Native agent; up to three work runs | Reuse the classic two-question story: answer a choice and a free-text question through the browser, verify each native continuation, then read the exact saved answers in an agent-authored task document after reload |
|
||||
| `task-properties` | Company, signed-in board user, task-write capability, prepared run-owned backlog task | Open its properties, assign it to the signed-in user, reload, check both UI and saved assignment; verify no agent run |
|
||||
| `task-edit-lifecycle` | Company, signed-in board user, task-write capability; target support for run-owned labels and exact predeclared `taskEdit.revisedTitle` cleanup; zero model runs | Create a self-assigned task, edit its inline header title and label, reload, complete and reopen it; independently verify exact saved state, unchanged context and zero runs. Open a closed Properties panel through its shipped toggle |
|
||||
| `task-document-persistence` | Company, signed-in board user, task-write capability, prepared run-owned backlog task; no model allowance or runs | Create a general document through Artifacts, edit its title and body, reopen and reload it; independently verify exact canonical content, preserved original and latest revisions, board-user authorship, and no agent work |
|
||||
| `task-search-filter` | Company, signed-in board user, task-write capability; zero model runs | Search three owned target/control tasks through the task list, require exact matching identities, filter Backlog to exclude the completed control, reload and return from the exact saved task with search/filter context intact |
|
||||
| `plan-revise-accept` | Native agent; three work runs | Reuse the classic Plan story: canonical draft, browser revision request, new revision-bound confirmation, approval and exact saved final response |
|
||||
@@ -488,3 +489,38 @@ within the original journey deadline. Binary attachment and browser downloads
|
||||
remain single attempts; transport failures and digest mismatches fail visibly. Git inspection consumes the exact digest-verified download
|
||||
buffer; browser downloads remain an independent result-access check. The
|
||||
repository journey shares the observation reader and browser helpers above.
|
||||
|
||||
|
||||
## Task editing and lifecycle
|
||||
|
||||
`task-edit-lifecycle` creates a self-assigned task through the browser with an
|
||||
explicit title. It edits the title inline in the task header and adds one
|
||||
run-owned label through the Properties panel. If the panel starts closed, the
|
||||
journey opens it through the ordinary Toggle side panel or Show properties
|
||||
control. It changes the status to In Progress, reloads, marks
|
||||
the task Done, reloads, reopens it to Todo and reloads again. Independent API
|
||||
reads bind every observed state to the same company, task and human owner.
|
||||
The exact description, assignee, project and priority remain unchanged.
|
||||
Priority editing is not part of this journey because the production UI hides
|
||||
that control. No agent or model work is allowed.
|
||||
|
||||
Labels use `Journey QA label ${campaignId}`, retaining the complete campaign
|
||||
identity within the production 48-character limit for ordinary generated
|
||||
campaigns. The production schema rejects an overlong name before any browser
|
||||
mutation or ownership intent; names are never truncated or hashed.
|
||||
The target must journal `label` resources by campaign name before browser label
|
||||
creation and by exact ID after its 201 response. Before title editing, the owned
|
||||
issue receipt retains its original name and ID and declares
|
||||
`taskEdit: { revisedTitle }` for this case. Cleanup may match only that one
|
||||
predeclared title or the original title on the same company and issue ID.
|
||||
The private target must reconcile a lost label-creation response, remove only
|
||||
its proven run-owned label, and verify absence. Targets without these ownership
|
||||
and cleanup semantics must not select this journey.
|
||||
|
||||
The case uses no private fixture writes. Its limits require zero model calls and
|
||||
zero task runs. `private-stories/task-edit-lifecycle/task-lifecycle-observations.json`
|
||||
retains the five allowlisted task observations. The adjacent
|
||||
`observation-read-failures.json` retains bounded read-transport failures.
|
||||
Valid, wrong and missing evidence
|
||||
controls calibrate the independent oracle. Registration and local checks do not
|
||||
claim a Cloud staging pass or admission to a daily plan.
|
||||
@@ -126,7 +126,29 @@ describe("runner E2E catalog", () => {
|
||||
});
|
||||
|
||||
it("exports deployment journeys separately from the model matrix and default selection", () => {
|
||||
expect(productJourneys.map(journey => journey.id)).toEqual(["company-access-isolation", "repository-continuation", "routine-scheduled-run", "inbox-triage", "file-input-answer", "task-create", "project-task-context", "ask-question", "question-answer-resume", "task-properties", "task-document-persistence", "task-search-filter", "plan-revise-accept", "build-revise", "protected-actions", "create-skill-studio", "stop-redirect", "followup-while-running", "hire-delegate-reuse", "cloud-onboarding"]);
|
||||
expect(productJourneys.map(journey => journey.id)).toEqual([
|
||||
"task-edit-lifecycle",
|
||||
"company-access-isolation",
|
||||
"repository-continuation",
|
||||
"routine-scheduled-run",
|
||||
"inbox-triage",
|
||||
"file-input-answer",
|
||||
"task-create",
|
||||
"project-task-context",
|
||||
"ask-question",
|
||||
"question-answer-resume",
|
||||
"task-properties",
|
||||
"task-document-persistence",
|
||||
"task-search-filter",
|
||||
"plan-revise-accept",
|
||||
"build-revise",
|
||||
"protected-actions",
|
||||
"create-skill-studio",
|
||||
"stop-redirect",
|
||||
"followup-while-running",
|
||||
"hire-delegate-reuse",
|
||||
"cloud-onboarding"
|
||||
]);
|
||||
expect(runnerSuites.some(suite => suite.id === "product-journeys")).toBe(false);
|
||||
expect(runnerMatrix.some(execution => execution.suite.id === "product-journeys")).toBe(false);
|
||||
expect(selectRunnerExecutions(parseRunnerSelectors(["--all"])).some(execution => execution.suite.id === "product-journeys")).toBe(false);
|
||||
|
||||
@@ -28,6 +28,7 @@ associate(["product-journeys"], {
|
||||
"followup-while-running": ["steering/queue"],
|
||||
"hire-delegate-reuse": ["agents/hire", "documents-artifacts/documents"],
|
||||
"project-task-context": ["projects/lifecycle", "projects/task-context", "tasks/create"],
|
||||
"task-edit-lifecycle": ["tasks/properties", "tasks/completion"],
|
||||
"task-properties": ["tasks/properties"], "task-document-persistence": ["documents-artifacts/documents"], "task-search-filter": ["tasks/organize"], "cloud-onboarding": ["onboarding/first-task"] });
|
||||
associate(["antigravity-agent-journey"], {
|
||||
"saved-subscription-two-turn": ["agents/hire", "agents/instructions", "runs-adapters/validation", "runs-adapters/session",
|
||||
|
||||
@@ -20,7 +20,13 @@ import { agentJourneyRuns } from "./catalog.js";
|
||||
import { JourneyError, type JourneyContext, type JourneyEvidence, type TargetSession } from "./types.js";
|
||||
|
||||
type Row = Record<string, any>;
|
||||
type AgentJourney = Exclude<keyof typeof agentJourneyRuns, "ask-question" | "question-answer-resume" | "followup-while-running" | "file-input-answer" | "routine-scheduled-run" | "repository-continuation">;
|
||||
type AgentJourney = Exclude<keyof typeof agentJourneyRuns,
|
||||
"ask-question"
|
||||
| "question-answer-resume"
|
||||
| "followup-while-running"
|
||||
| "file-input-answer"
|
||||
| "routine-scheduled-run"
|
||||
| "repository-continuation">;
|
||||
interface TaskState { currentIssue: Row & { id: string; status: string }; taskRuns: Array<Row & { status: string }>; comments: Row[]; interactions: Array<Row & { id: string; status: string }>; attachments: StoryDelivery[] }
|
||||
interface Options { deadlineAt: number; directory: string; capture(id: string, label: string, file: string): Promise<void> }
|
||||
const active = (run: Row) => ["queued", "running"].includes(run.status);
|
||||
|
||||
@@ -2,6 +2,7 @@ import { agentJourneyRuns } from "./success-contract.js";
|
||||
export { agentJourneyRuns, agentJourneyRunCountValid } from "./success-contract.js";
|
||||
import type { JourneyId, TargetConfig } from "./types.js";
|
||||
export const productJourneys = [
|
||||
{ id: "task-edit-lifecycle", label: "Edit task properties, complete and reopen with saved context", requires: ["company", "board-user", "task-write"], capabilities: ["tasks/properties", "tasks/completion"], paid: false },
|
||||
{ id: "company-access-isolation", label: "Switch QA workspaces and verify restricted agent isolation", requires: ["company", "board-user", "task-write", "company-access"], capabilities: ["companies/selection", "access/resources"], paid: false },
|
||||
{ id: "repository-continuation", label: "Continue repository work with the same saved workspace", requires: ["company", "board-user", "task-write", "agent"], capabilities: ["projects/task-context", "documents-artifacts/attachments"], paid: true },
|
||||
{ id: "routine-scheduled-run", label: "Run a scheduled routine once and verify pause and saved history", requires: ["company", "board-user", "task-write", "agent"], capabilities: ["routines/definition", "routines/triggers", "routines/history", "routines/management"], paid: true },
|
||||
|
||||
@@ -9,6 +9,7 @@ import { runnerSuites } from "../catalog.js";
|
||||
import { productJourneys, parseTargetConfig, preparationPreview, isAgentJourney } from "./catalog.js";
|
||||
import { agentJourneyRunCountValid } from "./success-contract.js";
|
||||
import { journeyFixtures } from "./fixtures.js";
|
||||
import { taskEditLifecycle } from "./task-lifecycle.js";
|
||||
import { taskProperties } from "./task-properties.js";
|
||||
import { taskSearchFilter } from "./task-search-filter.js";
|
||||
import { askQuestion, taskCreate } from "./basic-flows.js";
|
||||
@@ -113,7 +114,73 @@ export async function launchProductJourneys(args: string[], repositoryRoot: stri
|
||||
try { sha = execFileSync("git", ["rev-parse", "HEAD"], { cwd: repositoryRoot, encoding: "utf8", stdio: ["ignore", "pipe", "ignore"] }).trim(); } catch { /* Packaged harnesses may not include Git metadata. */ }
|
||||
const source = resolveRunnerE2ESource({ sha, ref: null, workflowRunUrl: null });
|
||||
const definitionHash = createHash("sha256").update(JSON.stringify(productJourneys));
|
||||
for (const name of ["observation-read.ts", "../connection-config.ts", "catalog.ts", "company-access-flow.ts", "company-access-contract.ts", "repository-continuation-flow.ts", "repository-continuation-contract.ts", "repository-continuation-evidence.ts", "../registered-artifact.ts", "../user-actions.ts", "../connection-target.ts", "../../../packages/shared/src/project-mentions.ts", "routine-schedule-flow.ts", "routine-schedule-evidence.ts", "input-file-flow.ts", "input-file-evidence.ts", "inbox-triage-flow.ts", "inbox-triage-evidence.ts", "inbox-triage-fixture.ts", "inbox-author-contract.ts", "task-properties.ts", "task-search-filter.ts", "task-search-evidence.ts", "basic-flows.ts", "basic-evidence.ts", "task-document-flow.ts", "task-document-evidence.ts", "project-context-flow.ts", "project-context-readback.ts", "project-context-evidence.ts", "agent-flows.ts", "hire-delegate-flow.ts", "hire-delegate-composer.ts", "hire-delegate-evidence.ts", "active-followup-flow.ts", "active-followup-evidence.ts", "stop-redirect-evidence.ts", "clarification-flow.ts", "clarification-evidence.ts", "../question-resume-ui.ts", "../question-resume-scoring.ts", "../runtime-question-readiness.ts", "../chat-flow.ts", "../continuation-readiness.ts", "../continuation-cases.ts", "success-contract.ts", "product-screenshot.ts", "cloud-onboarding.ts", "onboarding-connect.ts", "onboarding-first-task.ts", "workspace-deferral.ts", "onboarding-pre-acceptance.ts", "artifact-evidence.ts", "fixtures.ts", "launch.ts", "types.ts", "collection.ts", "local-target.ts", "../../../ui/src/lib/markdown-reply-content.ts", "../../../ui/src/lib/issue-reference.ts", "../../../ui/src/lib/company-routes.ts", "../../../ui/package.json"]) definitionHash.update(await readFile(path.join(import.meta.dirname, name)));
|
||||
for (const name of [
|
||||
"task-lifecycle.ts",
|
||||
"task-lifecycle-evidence.ts",
|
||||
"../../../packages/shared/src/validators/issue.ts",
|
||||
"observation-read.ts",
|
||||
"../connection-config.ts",
|
||||
"catalog.ts",
|
||||
"company-access-flow.ts",
|
||||
"company-access-contract.ts",
|
||||
"repository-continuation-flow.ts",
|
||||
"repository-continuation-contract.ts",
|
||||
"repository-continuation-evidence.ts",
|
||||
"../registered-artifact.ts",
|
||||
"../user-actions.ts",
|
||||
"../connection-target.ts",
|
||||
"../../../packages/shared/src/project-mentions.ts",
|
||||
"routine-schedule-flow.ts",
|
||||
"routine-schedule-evidence.ts",
|
||||
"input-file-flow.ts",
|
||||
"input-file-evidence.ts",
|
||||
"inbox-triage-flow.ts",
|
||||
"inbox-triage-evidence.ts",
|
||||
"inbox-triage-fixture.ts",
|
||||
"inbox-author-contract.ts",
|
||||
"task-properties.ts",
|
||||
"task-search-filter.ts",
|
||||
"task-search-evidence.ts",
|
||||
"basic-flows.ts",
|
||||
"basic-evidence.ts",
|
||||
"task-document-flow.ts",
|
||||
"task-document-evidence.ts",
|
||||
"project-context-flow.ts",
|
||||
"project-context-readback.ts",
|
||||
"project-context-evidence.ts",
|
||||
"agent-flows.ts",
|
||||
"hire-delegate-flow.ts",
|
||||
"hire-delegate-composer.ts",
|
||||
"hire-delegate-evidence.ts",
|
||||
"active-followup-flow.ts",
|
||||
"active-followup-evidence.ts",
|
||||
"stop-redirect-evidence.ts",
|
||||
"clarification-flow.ts",
|
||||
"clarification-evidence.ts",
|
||||
"../question-resume-ui.ts",
|
||||
"../question-resume-scoring.ts",
|
||||
"../runtime-question-readiness.ts",
|
||||
"../chat-flow.ts",
|
||||
"../continuation-readiness.ts",
|
||||
"../continuation-cases.ts",
|
||||
"success-contract.ts",
|
||||
"product-screenshot.ts",
|
||||
"cloud-onboarding.ts",
|
||||
"onboarding-connect.ts",
|
||||
"onboarding-first-task.ts",
|
||||
"workspace-deferral.ts",
|
||||
"onboarding-pre-acceptance.ts",
|
||||
"artifact-evidence.ts",
|
||||
"fixtures.ts",
|
||||
"launch.ts",
|
||||
"types.ts",
|
||||
"collection.ts",
|
||||
"local-target.ts",
|
||||
"../../../ui/src/lib/markdown-reply-content.ts",
|
||||
"../../../ui/src/lib/issue-reference.ts",
|
||||
"../../../ui/src/lib/company-routes.ts",
|
||||
"../../../ui/package.json"
|
||||
]) definitionHash.update(await readFile(path.join(import.meta.dirname, name)));
|
||||
for (const file of ["catalog.ts", "api.ts", "run-observations.ts", "native-task-reply.ts", "native-event-envelope.ts", "user-actions.ts", "plan-revision-flow.ts", "skill-studio-flow.ts", "protected-action-flow.ts", "story-artifact.ts", "stop-redirect-flow.ts", "everyday-cases.ts", "everyday-delivery.ts", "chat-hardening.ts", "everyday-decisions.ts", "connection-guidance-evidence.ts", "everyday-artifact.py"]) definitionHash.update(await readFile(path.join(import.meta.dirname, "..", file)));
|
||||
definitionHash.update(runnerSuites.flatMap(suite => suite.tasks).find(task => task.id === "ask-question")!.buildPrompt("definition"));
|
||||
const suiteDefinitionHash = definitionHash.digest("hex");
|
||||
@@ -234,6 +301,8 @@ export async function launchProductJourneys(args: string[], repositoryRoot: stri
|
||||
} });
|
||||
} else if (id === "company-access-isolation") await companyAccessIsolation(ready, session!, campaignId, evidence);
|
||||
else if (id === "inbox-triage") await inboxTriage(ready, session!, campaignId, evidence);
|
||||
else if (id === "task-edit-lifecycle") await taskEditLifecycle(ready, session!, campaignId, evidence,
|
||||
{ deadlineAt, directory: path.join(outputDir, "private-stories", id) });
|
||||
else if (id === "task-create") await taskCreate(ready, session!, campaignId, evidence);
|
||||
else if (id === "task-document-persistence") await taskDocumentPersistence(ready, session!, campaignId, evidence);
|
||||
else if (id === "project-task-context") await projectTaskContext(ready, session!, campaignId, evidence);
|
||||
|
||||
@@ -1,6 +1,18 @@
|
||||
/** Closed, target-independent success requirements used by dispatch and retained evidence. */
|
||||
export const agentJourneyRuns = { "repository-continuation": 2, "routine-scheduled-run": 1, "file-input-answer": 1, "ask-question": 1, "plan-revise-accept": 3, "build-revise": 2,
|
||||
"followup-while-running": 2, "protected-actions": 4, "create-skill-studio": 1, "stop-redirect": 3, "question-answer-resume": 3, "hire-delegate-reuse": 7 } as const;
|
||||
export const agentJourneyRuns = {
|
||||
"repository-continuation": 2,
|
||||
"routine-scheduled-run": 1,
|
||||
"file-input-answer": 1,
|
||||
"ask-question": 1,
|
||||
"plan-revise-accept": 3,
|
||||
"build-revise": 2,
|
||||
"followup-while-running": 2,
|
||||
"protected-actions": 4,
|
||||
"create-skill-studio": 1,
|
||||
"stop-redirect": 3,
|
||||
"question-answer-resume": 3,
|
||||
"hire-delegate-reuse": 7
|
||||
} as const;
|
||||
|
||||
export function agentJourneyRunCountValid(id: string, count: unknown): boolean {
|
||||
if (!Object.hasOwn(agentJourneyRuns, id) || !Number.isSafeInteger(count)) return false;
|
||||
@@ -9,6 +21,7 @@ export function agentJourneyRunCountValid(id: string, count: unknown): boolean {
|
||||
return (count as number) >= minimum && (count as number) <= maximum;
|
||||
}
|
||||
export const journeyRequiredChecks: Record<string, readonly string[]> = {
|
||||
"task-edit-lifecycle": ["task-lifecycle-browser-created", "task-lifecycle-visible-edits-persisted", "task-lifecycle-edits-reloaded", "task-lifecycle-completed-and-reopened", "task-lifecycle-unedited-context-preserved", "no-agent-work"],
|
||||
"company-access-isolation": ["isolation-owner-resource-controls", "cloud-workspace-switch-and-return", "workspace-data-scoped", "restricted-agent-positive-controls", "cross-workspace-agent-read-write-denied", "isolation-credential-revoked", "no-agent-work"],
|
||||
"repository-continuation": ["repository-configured-source-bound", "repository-first-commit-verified", "repository-original-files-saved", "repository-followup-stored-once", "repository-same-workspace-retained", "repository-second-commit-verified", "repository-original-and-result-reopened", "configured-model-and-environment"],
|
||||
"routine-scheduled-run": ["routine-browser-created", "routine-scheduler-created-execution", "routine-result-attributed", "routine-history-preserved", "routine-pause-prevented-next-run", "routine-result-reopened", "configured-model-and-environment"],
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { createIssueLabelSchema } from "../../../packages/shared/src/validators/issue.js";
|
||||
import { taskLifecycleEvidence, taskLifecycleLabelName, type LifecycleObservation, type LifecycleTask } from "./task-lifecycle-evidence.js";
|
||||
import { parseTargetConfig, preparationPreview } from "./catalog.js";
|
||||
import { journeyRequiredChecks } from "./success-contract.js";
|
||||
import { validateRetainedRunnerResult } from "../result-validation.js";
|
||||
import type { RunnerE2EResult } from "../types.js";
|
||||
|
||||
function fixture(): Parameters<typeof taskLifecycleEvidence>[0] {
|
||||
const expected = { taskId: "task-1", companyId: "company-1", userId: "user-1", originalTitle: "Original task",
|
||||
revisedTitle: "Revised task", description: "Exact persisted context", labelName: "Owned label" };
|
||||
const initial: LifecycleTask = { id: expected.taskId, companyId: expected.companyId, identifier: "QA-1", title: expected.originalTitle,
|
||||
description: expected.description, status: "todo", priority: "medium", projectId: "project-1", assigneeUserId: expected.userId,
|
||||
assigneeAgentId: null, createdByUserId: expected.userId, createdByAgentId: null, labelIds: [], completedAt: null };
|
||||
const observe = (task: LifecycleTask, labelVisible: boolean): LifecycleObservation => ({ task: structuredClone(task), runCount: 0,
|
||||
titleVisible: true, descriptionVisible: true, statusVisible: true, labelVisible });
|
||||
const edited = { ...initial, title: expected.revisedTitle, status: "in_progress", labelIds: ["label-1"] };
|
||||
return { expected, label: { id: "label-1", name: expected.labelName, companyId: expected.companyId, color: "#0088ff" },
|
||||
created: observe(initial, false), edited: observe(edited, true), reloaded: observe(edited, true),
|
||||
completed: observe({ ...edited, status: "done", completedAt: "2026-01-01T00:00:00.000Z" }, true),
|
||||
reopened: observe({ ...edited, status: "todo" }, true) };
|
||||
}
|
||||
const stages = ["created", "edited", "reloaded", "completed", "reopened"] as const;
|
||||
|
||||
describe("task lifecycle saved-state oracle", () => {
|
||||
it("keeps a full generated campaign identity within the production label-name limit", () => {
|
||||
const campaigns = ["journey-1793720364049-f3055878", "journey-9999999999999-ffffffff"];
|
||||
const labels = campaigns.map(taskLifecycleLabelName);
|
||||
for (const [index, name] of labels.entries()) {
|
||||
expect(name).toContain(campaigns[index]);
|
||||
expect(createIssueLabelSchema.shape.name.safeParse(name).success).toBe(true);
|
||||
}
|
||||
expect(new Set(labels).size).toBe(campaigns.length);
|
||||
});
|
||||
it("rejects an overlong campaign before it can produce a browser label", () => {
|
||||
expect(() => taskLifecycleLabelName("c".repeat(32))).toThrow("task_lifecycle_campaign_label_name_invalid");
|
||||
});
|
||||
it("accepts the complete browser-edit/reload/complete/reopen sequence", () => {
|
||||
expect(taskLifecycleEvidence(fixture())).toEqual({ created: true, edited: true, reloaded: true,
|
||||
completedAndReopened: true, uneditedFieldsPreserved: true, noAgentWork: true });
|
||||
});
|
||||
it("accepts a fresh backlog task and an absent project without changing either contract", () => {
|
||||
const input = fixture(); input.created.task!.status = "backlog";
|
||||
for (const stage of stages) input[stage].task!.projectId = null;
|
||||
expect(Object.values(taskLifecycleEvidence(input)).every(Boolean)).toBe(true);
|
||||
});
|
||||
it.each(stages)("rejects a missing saved task at %s", stage => {
|
||||
const input = fixture(); input[stage].task = null;
|
||||
const result = taskLifecycleEvidence(input);
|
||||
expect(result.uneditedFieldsPreserved).toBe(false);
|
||||
expect(Object.values(result).every(Boolean)).toBe(false);
|
||||
});
|
||||
it.each(["id", "companyId", "assigneeUserId", "assigneeAgentId", "createdByUserId", "createdByAgentId", "description"] as const)(
|
||||
"rejects a wrong owner or saved context field: %s", key => {
|
||||
const input = fixture(); Object.assign(input.reopened.task!, { [key]: "foreign" });
|
||||
expect(taskLifecycleEvidence(input).completedAndReopened).toBe(false);
|
||||
expect(taskLifecycleEvidence(input).uneditedFieldsPreserved).toBe(false);
|
||||
});
|
||||
it.each(["identifier", "priority", "projectId"] as const)("requires %s to survive every edit", key => {
|
||||
for (const stage of stages.slice(1)) {
|
||||
const input = fixture(); Object.assign(input[stage].task!, { [key]: "changed" });
|
||||
expect(taskLifecycleEvidence(input).uneditedFieldsPreserved).toBe(false);
|
||||
}
|
||||
});
|
||||
it.each(["edited", "reloaded", "completed", "reopened"] as const)("rejects a lost title or label at %s", stage => {
|
||||
for (const patch of [{ title: "Original task" }, { labelIds: [] }, { labelIds: ["foreign"] }, { labelIds: ["label-1", "label-1"] }]) {
|
||||
const input = fixture(); Object.assign(input[stage].task!, patch);
|
||||
expect(Object.values(taskLifecycleEvidence(input)).every(Boolean)).toBe(false);
|
||||
}
|
||||
});
|
||||
it.each(["titleVisible", "descriptionVisible", "statusVisible", "labelVisible"] as const)("requires visible reloaded %s", key => {
|
||||
for (const stage of ["reloaded", "completed", "reopened"] as const) {
|
||||
const input = fixture(); input[stage][key] = false;
|
||||
expect(Object.values(taskLifecycleEvidence(input)).every(Boolean)).toBe(false);
|
||||
}
|
||||
});
|
||||
it("does not accept a completed-looking UI without saved Done or completion time", () => {
|
||||
const wrongStatus = fixture(); wrongStatus.completed.task!.status = "in_review";
|
||||
expect(taskLifecycleEvidence(wrongStatus).completedAndReopened).toBe(false);
|
||||
for (const completedAt of [null, "", "not a timestamp"]) {
|
||||
const input = fixture(); input.completed.task!.completedAt = completedAt;
|
||||
expect(taskLifecycleEvidence(input).completedAndReopened).toBe(false);
|
||||
}
|
||||
});
|
||||
it("requires reopened Todo with its completion time cleared", () => {
|
||||
for (const patch of [{ status: "done" }, { completedAt: "2026-01-01T00:00:00.000Z" }]) {
|
||||
const input = fixture(); Object.assign(input.reopened.task!, patch);
|
||||
expect(taskLifecycleEvidence(input).completedAndReopened).toBe(false);
|
||||
}
|
||||
});
|
||||
it.each(stages)("requires an explicit zero-run observation at %s", stage => {
|
||||
for (const runCount of [null, 1, -1]) {
|
||||
const input = fixture(); input[stage].runCount = runCount;
|
||||
expect(taskLifecycleEvidence(input).noAgentWork).toBe(false);
|
||||
}
|
||||
});
|
||||
it("rejects stale starting data, a wrong label and an unchanged expected title", () => {
|
||||
for (const change of [
|
||||
(input: ReturnType<typeof fixture>) => { input.created.task!.labelIds = ["existing"]; },
|
||||
(input: ReturnType<typeof fixture>) => { input.created.task!.status = "done"; },
|
||||
(input: ReturnType<typeof fixture>) => { input.label = null; },
|
||||
(input: ReturnType<typeof fixture>) => { input.label!.companyId = "foreign"; },
|
||||
(input: ReturnType<typeof fixture>) => { input.label!.name = "Other label"; },
|
||||
(input: ReturnType<typeof fixture>) => { input.label!.color = "invalid"; },
|
||||
(input: ReturnType<typeof fixture>) => { input.expected.revisedTitle = input.expected.originalTitle; },
|
||||
]) {
|
||||
const input = fixture(); change(input);
|
||||
expect(Object.values(taskLifecycleEvidence(input)).every(Boolean)).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
function result(): RunnerE2EResult {
|
||||
return { schema: "paperclip.runner-e2e.result/v2", suiteId: "product-journeys", suiteDefinitionHash: "test",
|
||||
executionId: "product-journeys.qa.browser.daytona.task-edit-lifecycle", profileId: "browser", environmentId: "daytona",
|
||||
caseId: "task-edit-lifecycle", provider: null, model: null, runtimeMode: "none", status: "passed", cleanup: "passed",
|
||||
attempt: 1, startedAt: "2026-01-01T00:00:00Z", finishedAt: "2026-01-01T00:00:01Z", durationMs: 1000,
|
||||
productJourney: { outcome: "passed", startingState: "recipe", phases: [{ phase: "journey", outcome: "passed" }],
|
||||
checks: journeyRequiredChecks["task-edit-lifecycle"]!.map(id => ({ id, passed: true })), privacy: "private", resourceDisposition: "released",
|
||||
deployment: { kind: "cloud", targetId: "qa", origin: "https://qa.example.test", commit: "a".repeat(40),
|
||||
cloudRelease: "test", runtimeImage: null, configurationHash: "test" } } };
|
||||
}
|
||||
|
||||
describe("task lifecycle discovery and report contract", () => {
|
||||
it("previews a browser-only existing target with no provider allowance", () => {
|
||||
const config = parseTargetConfig({ version: 1, id: "qa", mode: "existing-cloud", adapterModule: "/private/adapter.mjs",
|
||||
journeys: ["task-edit-lifecycle"], executionEnvironment: "daytona", baseline: { features: {}, connections: [] },
|
||||
limits: { durationMs: 180000, maxRuns: 0, modelBudgetCents: 0 } });
|
||||
expect(preparationPreview(config).journeys).toMatchObject([{ id: "task-edit-lifecycle", paid: false }]);
|
||||
expect(config.limits.maxRuns).toBe(0);
|
||||
});
|
||||
it("rejects every missing required check and unexpected native work", () => {
|
||||
expect(() => validateRetainedRunnerResult(result())).not.toThrow();
|
||||
for (const id of journeyRequiredChecks["task-edit-lifecycle"]!) {
|
||||
const missing = result(); missing.productJourney!.checks = missing.productJourney!.checks.filter(check => check.id !== id);
|
||||
expect(() => validateRetainedRunnerResult(missing)).toThrow();
|
||||
}
|
||||
for (const patch of [{ provider: "openai" }, { model: "wrong" }, { runtimeMode: "native" }, { runIds: ["unexpected"] }]) {
|
||||
expect(() => validateRetainedRunnerResult({ ...result(), ...patch })).toThrow();
|
||||
}
|
||||
const usage = result(); usage.usage = { runs: [{ runId: "unexpected", usage: { inputTokens: 1 } }] };
|
||||
expect(() => validateRetainedRunnerResult(usage)).toThrow("result.browserOnly");
|
||||
const emptyUsage = result(); emptyUsage.usage = { runs: [] };
|
||||
expect(() => validateRetainedRunnerResult(emptyUsage)).not.toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,98 @@
|
||||
import { createIssueLabelSchema } from "../../../packages/shared/src/validators/issue.js";
|
||||
import { JourneyError } from "./types.js";
|
||||
|
||||
export function taskLifecycleLabelName(campaignId: string): string {
|
||||
const name = `Journey QA label ${campaignId}`;
|
||||
if (!createIssueLabelSchema.shape.name.safeParse(name).success) {
|
||||
throw new JourneyError("unsupported", "task_lifecycle_campaign_label_name_invalid");
|
||||
}
|
||||
return name;
|
||||
}
|
||||
|
||||
export interface LifecycleTask {
|
||||
id: string;
|
||||
companyId: string;
|
||||
identifier: string;
|
||||
title: string;
|
||||
description: string;
|
||||
status: string;
|
||||
priority: string;
|
||||
projectId: string | null;
|
||||
assigneeUserId: string;
|
||||
assigneeAgentId: null;
|
||||
createdByUserId: string;
|
||||
createdByAgentId: null;
|
||||
labelIds: string[];
|
||||
completedAt: string | null;
|
||||
}
|
||||
export interface LifecycleLabel { id: string; companyId: string; name: string; color: string }
|
||||
export interface LifecycleObservation {
|
||||
task: LifecycleTask | null;
|
||||
runCount: number | null;
|
||||
titleVisible: boolean;
|
||||
descriptionVisible: boolean;
|
||||
statusVisible: boolean;
|
||||
labelVisible: boolean;
|
||||
}
|
||||
export interface LifecycleExpectation {
|
||||
taskId: string;
|
||||
companyId: string;
|
||||
userId: string;
|
||||
originalTitle: string;
|
||||
revisedTitle: string;
|
||||
description: string;
|
||||
labelName: string;
|
||||
}
|
||||
|
||||
function ownedTask(task: LifecycleTask | null, expected: LifecycleExpectation) {
|
||||
return Boolean(task && task.id === expected.taskId && task.companyId === expected.companyId && task.identifier
|
||||
&& task.description === expected.description && task.assigneeUserId === expected.userId && task.assigneeAgentId === null
|
||||
&& task.createdByUserId === expected.userId && task.createdByAgentId === null);
|
||||
}
|
||||
function singleton(ids: unknown, id: string) { return Array.isArray(ids) && ids.length === 1 && ids[0] === id; }
|
||||
function sameUneditedFields(task: LifecycleTask | null, initial: LifecycleTask | null) {
|
||||
return Boolean(task && initial && task.identifier === initial.identifier && task.priority === initial.priority && task.projectId === initial.projectId);
|
||||
}
|
||||
|
||||
/** Independent saved states accompany visible UI readback. Empty or missing observations cannot pass. */
|
||||
export function taskLifecycleEvidence(input: {
|
||||
expected: LifecycleExpectation;
|
||||
label: LifecycleLabel | null;
|
||||
created: LifecycleObservation;
|
||||
edited: LifecycleObservation;
|
||||
reloaded: LifecycleObservation;
|
||||
completed: LifecycleObservation;
|
||||
reopened: LifecycleObservation;
|
||||
}) {
|
||||
const { expected, label, created, edited, reloaded, completed, reopened } = input;
|
||||
const validExpected = Boolean(expected.taskId && expected.companyId && expected.userId && expected.originalTitle
|
||||
&& expected.revisedTitle && expected.originalTitle !== expected.revisedTitle && expected.description && expected.labelName);
|
||||
const fresh = validExpected && ownedTask(created.task, expected) && created.task!.title === expected.originalTitle
|
||||
&& ["backlog", "todo"].includes(created.task!.status) && created.task!.completedAt === null
|
||||
&& ["critical", "high", "medium", "low"].includes(created.task!.priority)
|
||||
&& (created.task!.projectId === null || typeof created.task!.projectId === "string" && created.task!.projectId.length > 0)
|
||||
&& Array.isArray(created.task!.labelIds) && created.task!.labelIds.length === 0
|
||||
&& created.titleVisible && created.descriptionVisible && created.statusVisible;
|
||||
const labelValid = Boolean(label && label.id && label.companyId === expected.companyId && label.name === expected.labelName
|
||||
&& /^#[a-fA-F0-9]{6}$/.test(label.color));
|
||||
const saved = (observation: LifecycleObservation, status: string) => fresh && labelValid && ownedTask(observation.task, expected)
|
||||
&& sameUneditedFields(observation.task, created.task) && observation.task!.title === expected.revisedTitle
|
||||
&& observation.task!.status === status && singleton(observation.task!.labelIds, label!.id);
|
||||
const shown = (observation: LifecycleObservation) => observation.titleVisible && observation.descriptionVisible
|
||||
&& observation.statusVisible && observation.labelVisible;
|
||||
const editedMatches = saved(edited, "in_progress") && shown(edited) && edited.task!.completedAt === null;
|
||||
const reloadedMatches = saved(reloaded, "in_progress") && shown(reloaded) && reloaded.task!.completedAt === null;
|
||||
const completedMatches = saved(completed, "done") && shown(completed)
|
||||
&& typeof completed.task!.completedAt === "string" && Number.isFinite(Date.parse(completed.task!.completedAt));
|
||||
const reopenedMatches = saved(reopened, "todo") && shown(reopened) && reopened.task!.completedAt === null;
|
||||
const observations = [created, edited, reloaded, completed, reopened];
|
||||
return {
|
||||
created: fresh,
|
||||
edited: editedMatches,
|
||||
reloaded: reloadedMatches,
|
||||
completedAndReopened: completedMatches && reopenedMatches,
|
||||
uneditedFieldsPreserved: fresh && observations.every(observation => ownedTask(observation.task, expected)
|
||||
&& sameUneditedFields(observation.task, created.task)),
|
||||
noAgentWork: fresh && observations.every(observation => observation.runCount === 0),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,173 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { expect } from "@playwright/test";
|
||||
import { createTaskThroughUi, withBrowserEvent } from "../user-actions.js";
|
||||
import { createObservedReader } from "./observation-read.js";
|
||||
import { taskLifecycleEvidence, taskLifecycleLabelName, type LifecycleExpectation, type LifecycleLabel, type LifecycleObservation, type LifecycleTask } from "./task-lifecycle-evidence.js";
|
||||
import { JourneyError, type JourneyContext, type JourneyEvidence, type TargetSession } from "./types.js";
|
||||
|
||||
type Stage = "create" | "initial_read" | "rename" | "label" | "in_progress" | "reload" | "complete" | "reopen";
|
||||
function taskSnapshot(task: LifecycleTask): LifecycleTask {
|
||||
return { id: task.id, companyId: task.companyId, identifier: task.identifier, title: task.title,
|
||||
description: task.description, status: task.status, priority: task.priority, projectId: task.projectId,
|
||||
assigneeUserId: task.assigneeUserId, assigneeAgentId: task.assigneeAgentId, createdByUserId: task.createdByUserId,
|
||||
createdByAgentId: task.createdByAgentId, labelIds: task.labelIds, completedAt: task.completedAt };
|
||||
}
|
||||
const emptyObservation = (): LifecycleObservation => ({ task: null, runCount: null,
|
||||
titleVisible: false, descriptionVisible: false, statusVisible: false, labelVisible: false });
|
||||
|
||||
async function properties(context: JourneyContext) {
|
||||
const { page } = context;
|
||||
// Closed panes keep their rows mounted. Their geometry alone does not make
|
||||
// an inert or accessibility-hidden panel available for interaction.
|
||||
const row = page.locator('[data-property-row]:not([inert] *):not([aria-hidden="true"] *)')
|
||||
.filter({ has: page.locator('[data-property-label="Labels"]') }).filter({ visible: true });
|
||||
const toggle = page.getByRole("button", { name: "Toggle side panel", exact: true, pressed: false }).filter({ visible: true });
|
||||
const propertiesButton = page.getByRole("button", { name: /^(Show properties|Properties)$/ }).filter({ visible: true });
|
||||
await expect.poll(async () => await row.isVisible() || await toggle.isVisible() || await propertiesButton.isVisible()).toBe(true);
|
||||
if (!await row.isVisible()) {
|
||||
if (await toggle.isVisible()) await toggle.click();
|
||||
else await propertiesButton.click();
|
||||
}
|
||||
await expect(row).toBeVisible();
|
||||
await row.scrollIntoViewIfNeeded();
|
||||
await expect(row).toBeInViewport();
|
||||
return row;
|
||||
}
|
||||
|
||||
async function setStatus(context: JourneyContext, label: string) {
|
||||
const { page } = context;
|
||||
await page.getByTestId("issue-detail-header").filter({ visible: true })
|
||||
.getByRole("button", { name: /^Change status \(current:/ }).click();
|
||||
// The shipped picker includes both a labeled status glyph and visible text.
|
||||
// Bind the choice to its exact accessible glyph inside the open popover.
|
||||
await page.locator('[data-slot="popover-content"]').filter({ visible: true })
|
||||
.getByRole("button").filter({ has: page.getByRole("img", { name: label, exact: true }) }).click();
|
||||
}
|
||||
|
||||
/** All measured edits use the shipped browser. Independent GETs verify saved state. */
|
||||
export async function taskEditLifecycle(context: JourneyContext, session: TargetSession, campaignId: string, evidence: JourneyEvidence,
|
||||
options: { deadlineAt: number; directory: string }) {
|
||||
const { page, api, company, user } = context;
|
||||
const read = createObservedReader(api, options.deadlineAt);
|
||||
const journeyId = "task-edit-lifecycle" as const;
|
||||
const originalTitle = `Journey QA task lifecycle ${campaignId}`;
|
||||
const revisedTitle = `Journey QA task lifecycle revised ${campaignId}`;
|
||||
const description = `Keep this task assigned to me. Saved task context ${campaignId} must survive every edit.`;
|
||||
const labelName = taskLifecycleLabelName(campaignId);
|
||||
const observed = { created: emptyObservation(), edited: emptyObservation(), reloaded: emptyObservation(),
|
||||
completed: emptyObservation(), reopened: emptyObservation() };
|
||||
let expected: LifecycleExpectation | undefined;
|
||||
let label: LifecycleLabel | null = null;
|
||||
let stage: Stage = "create";
|
||||
const check = (id: string, passed: boolean) => {
|
||||
evidence.checks.push({ id, passed });
|
||||
if (!passed) throw new JourneyError("failed", id.replaceAll("-", "_"));
|
||||
};
|
||||
const preserve = async () => {
|
||||
await mkdir(options.directory, { recursive: true, mode: 0o700 });
|
||||
await writeFile(path.join(options.directory, "task-lifecycle-observations.json"),
|
||||
JSON.stringify({ schema: "paperclip.task-lifecycle-observations/v1", stage, expected, label, ...observed }, null, 2), { mode: 0o600 });
|
||||
};
|
||||
const grade = () => taskLifecycleEvidence({ expected: expected!, label, ...observed });
|
||||
try {
|
||||
for (const capability of ["company", "board-user", "task-write"]) {
|
||||
if (!context.capabilities.includes(capability)) throw new JourneyError("unsupported", `missing_${capability}`);
|
||||
}
|
||||
if (!campaignId) throw new JourneyError("blocked", "task_lifecycle_campaign_required");
|
||||
const created = await createTaskThroughUi({ page, origin: api.origin, issuePrefix: company.issuePrefix,
|
||||
agentName: "Me", title: originalTitle, prompt: description, workMode: "standard", requireExplicitTitle: true,
|
||||
beforeDispatch: () => session.own({ kind: "issue", name: originalTitle, journeyId }) });
|
||||
await session.own({ kind: "issue", name: originalTitle, id: created.issueId, journeyId });
|
||||
const route = `/api/issues/${created.issueId}`;
|
||||
expected = { taskId: created.issueId, companyId: company.id, userId: user.id, originalTitle, revisedTitle, description, labelName };
|
||||
const task = await read.get<LifecycleTask>(route);
|
||||
await page.goto(`${api.origin}/${company.issuePrefix}/issues/${task.identifier}`);
|
||||
const capture = async (title: string, statusLabel: string, withLabel: boolean): Promise<LifecycleObservation> => {
|
||||
await expect(page.getByRole("heading", { name: title, exact: true })).toBeVisible();
|
||||
await expect(page.getByText(description, { exact: true }).first()).toBeVisible();
|
||||
const status = page.getByRole("button", { name: `Change status (current: ${statusLabel})`, exact: true }).filter({ visible: true }).first();
|
||||
await expect(status).toBeVisible();
|
||||
const labelRow = await properties(context);
|
||||
if (withLabel) await expect(labelRow.getByText(labelName, { exact: true })).toBeVisible();
|
||||
const saved = await read.get<LifecycleTask>(route);
|
||||
const runs = await read.get<unknown[]>(`${route}/runs`);
|
||||
return { task: structuredClone(taskSnapshot(saved)), runCount: Array.isArray(runs) ? runs.length : null,
|
||||
titleVisible: await page.getByRole("heading", { name: title, exact: true }).isVisible(),
|
||||
descriptionVisible: await page.getByText(description, { exact: true }).first().isVisible(),
|
||||
statusVisible: await status.isVisible(), labelVisible: withLabel && await labelRow.getByText(labelName, { exact: true }).isVisible() };
|
||||
};
|
||||
stage = "initial_read";
|
||||
const initialStatus = task.status === "backlog" ? "Backlog" : "Todo";
|
||||
observed.created = await capture(originalTitle, initialStatus, false);
|
||||
await preserve();
|
||||
check("task-lifecycle-browser-created", grade().created);
|
||||
stage = "rename";
|
||||
// Keep the creation receipt stable. Cleanup may accept this one exact,
|
||||
// predeclared title if the browser save commits but its response is lost.
|
||||
await session.own({ kind: "issue", name: originalTitle, id: created.issueId, journeyId, taskEdit: { revisedTitle } });
|
||||
await page.getByRole("heading", { name: originalTitle, exact: true }).click();
|
||||
const titleEditor = page.getByTestId("issue-detail-header").locator("textarea").first();
|
||||
await titleEditor.fill(revisedTitle);
|
||||
await titleEditor.press("Enter");
|
||||
await expect.poll(async () => (await read.get<LifecycleTask>(route)).title).toBe(revisedTitle);
|
||||
stage = "label";
|
||||
const labelsRoute = `/api/companies/${company.id}/labels`;
|
||||
const beforeLabels = await read.get<LifecycleLabel[]>(labelsRoute);
|
||||
if (beforeLabels.some(row => row.name === labelName)) throw new JourneyError("failed", "task_lifecycle_label_not_fresh");
|
||||
const labels = await properties(context);
|
||||
await labels.locator("[data-property-value]").getByRole("button").first().click();
|
||||
await page.getByPlaceholder("New label", { exact: true }).fill(labelName);
|
||||
await session.own({ kind: "label", name: labelName, journeyId });
|
||||
const response = await withBrowserEvent(() => page.waitForResponse(response => response.request().method() === "POST"
|
||||
&& new URL(response.url()).pathname === labelsRoute),
|
||||
() => page.getByRole("button", { name: "Create label", exact: true }).click());
|
||||
if (response.status() !== 201) throw new JourneyError("failed", "task_lifecycle_label_create_failed");
|
||||
const createdLabel = await response.json() as LifecycleLabel;
|
||||
if (typeof createdLabel.id !== "string" || !createdLabel.id || createdLabel.name !== labelName || createdLabel.companyId !== company.id) {
|
||||
throw new JourneyError("failed", "task_lifecycle_label_receipt_mismatch");
|
||||
}
|
||||
await session.own({ kind: "label", name: labelName, id: createdLabel.id, journeyId });
|
||||
label = { id: createdLabel.id, companyId: createdLabel.companyId, name: createdLabel.name, color: createdLabel.color };
|
||||
await page.keyboard.press("Escape");
|
||||
const savedLabels = await read.get<LifecycleLabel[]>(labelsRoute);
|
||||
if (savedLabels.filter(row => row.name === labelName).length !== 1
|
||||
|| !savedLabels.some(row => row.id === label!.id && row.name === labelName && row.companyId === company.id && row.color === label!.color)) {
|
||||
throw new JourneyError("failed", "task_lifecycle_label_saved_mismatch");
|
||||
}
|
||||
stage = "in_progress";
|
||||
await setStatus(context, "In Progress");
|
||||
await expect.poll(async () => (await read.get<LifecycleTask>(route)).status).toBe("in_progress");
|
||||
await expect.poll(async () => {
|
||||
const task = await read.get<LifecycleTask>(route);
|
||||
return Array.isArray(task.labelIds) && task.labelIds.length === 1 && task.labelIds[0] === label!.id;
|
||||
}).toBe(true);
|
||||
observed.edited = await capture(revisedTitle, "In Progress", true);
|
||||
await preserve();
|
||||
check("task-lifecycle-visible-edits-persisted", grade().edited);
|
||||
stage = "reload";
|
||||
await page.reload();
|
||||
observed.reloaded = await capture(revisedTitle, "In Progress", true);
|
||||
await preserve();
|
||||
check("task-lifecycle-edits-reloaded", grade().reloaded);
|
||||
stage = "complete";
|
||||
await setStatus(context, "Done");
|
||||
await expect.poll(async () => (await read.get<LifecycleTask>(route)).status).toBe("done");
|
||||
await page.reload();
|
||||
observed.completed = await capture(revisedTitle, "Done", true);
|
||||
await preserve();
|
||||
stage = "reopen";
|
||||
await setStatus(context, "Todo");
|
||||
await expect.poll(async () => (await read.get<LifecycleTask>(route)).status).toBe("todo");
|
||||
await page.reload();
|
||||
observed.reopened = await capture(revisedTitle, "Todo", true);
|
||||
await preserve();
|
||||
const final = grade();
|
||||
check("task-lifecycle-completed-and-reopened", final.completedAndReopened);
|
||||
check("task-lifecycle-unedited-context-preserved", final.uneditedFieldsPreserved);
|
||||
check("no-agent-work", final.noAgentWork);
|
||||
} catch (error) {
|
||||
if (error instanceof JourneyError) throw error;
|
||||
throw new JourneyError("failed", `task_lifecycle_${stage}_failed`);
|
||||
} finally { await preserve(); await read.preserve(options.directory); }
|
||||
}
|
||||
@@ -2,8 +2,28 @@ import type { BrowserContext, Page } from "@playwright/test";
|
||||
import type { ConnectionApi } from "../connection-target.js";
|
||||
|
||||
export type JourneyOutcome = "passed" | "failed" | "blocked" | "unsupported";
|
||||
export type JourneyId = "task-properties" | "task-create" | "project-task-context" | "task-document-persistence" | "ask-question" | "cloud-onboarding"
|
||||
| "question-answer-resume" | "task-search-filter" | "followup-while-running" | "plan-revise-accept" | "build-revise" | "protected-actions" | "create-skill-studio" | "stop-redirect" | "hire-delegate-reuse" | "file-input-answer" | "inbox-triage" | "routine-scheduled-run" | "repository-continuation" | "company-access-isolation";
|
||||
export type JourneyId =
|
||||
"task-edit-lifecycle"
|
||||
| "task-properties"
|
||||
| "task-create"
|
||||
| "project-task-context"
|
||||
| "task-document-persistence"
|
||||
| "ask-question"
|
||||
| "cloud-onboarding"
|
||||
| "question-answer-resume"
|
||||
| "task-search-filter"
|
||||
| "followup-while-running"
|
||||
| "plan-revise-accept"
|
||||
| "build-revise"
|
||||
| "protected-actions"
|
||||
| "create-skill-studio"
|
||||
| "stop-redirect"
|
||||
| "hire-delegate-reuse"
|
||||
| "file-input-answer"
|
||||
| "inbox-triage"
|
||||
| "routine-scheduled-run"
|
||||
| "repository-continuation"
|
||||
| "company-access-isolation";
|
||||
export type TargetMode = "managed-self-hosted" | "existing-cloud" | "new-cloud";
|
||||
export interface DeploymentIdentity {
|
||||
kind: "self-hosted" | "cloud";
|
||||
@@ -113,7 +133,8 @@ export interface TargetSession {
|
||||
/** Publish the observed finalization failure without releasing resources or leases. */
|
||||
checkpointEvidence?(input?: TargetLifecycleInput): Promise<void>;
|
||||
/** Durable intent before creation; receipt immediately after creation. */
|
||||
own(resource: { kind: "issue" | "project" | "agent" | "connection" | "tool-connection" | "skill" | "routine"; name: string; id?: string; journeyId?: JourneyId;
|
||||
own(resource: { kind: "issue" | "project" | "agent" | "connection" | "tool-connection" | "skill" | "routine" | "label"; name: string; id?: string; journeyId?: JourneyId;
|
||||
taskEdit?: { revisedTitle: string };
|
||||
freshChat?: { companyId: string; userId: string; leadAgentId: string; absent: true } }): Promise<void>;
|
||||
}
|
||||
export interface TargetProvider {
|
||||
|
||||
@@ -434,8 +434,8 @@ export function validateRetainedRunnerResult(
|
||||
if (!evidence) return invalid("result.productJourney");
|
||||
const required = journeyRequiredChecks[checked.caseId];
|
||||
if (evidence.outcome !== "passed" || !evidence.deployment || !required || !required.every(id => evidence.checks.some(c => c.id === id && c.passed)) || evidence.checks.some(check => !check.passed) || evidence.phases.some(p => p.outcome !== "passed") || checked.cleanup !== "passed") invalid("result.productJourney");
|
||||
if (["inbox-triage", "task-properties", "task-create", "project-task-context", "task-document-persistence", "task-search-filter", "company-access-isolation"].includes(checked.caseId) && (checked.runtimeMode !== "none" || checked.provider !== null || checked.model !== null || checked.runIds?.length)) invalid("result.browserOnly");
|
||||
if (checked.caseId === "company-access-isolation" && checked.usage != null && Object.entries(checked.usage).some(([key, value]) => key !== "runs" || !Array.isArray(value) || value.length > 0)) invalid("result.browserOnly");
|
||||
if (["task-edit-lifecycle", "inbox-triage", "task-properties", "task-create", "project-task-context", "task-document-persistence", "task-search-filter", "company-access-isolation"].includes(checked.caseId) && (checked.runtimeMode !== "none" || checked.provider !== null || checked.model !== null || checked.runIds?.length)) invalid("result.browserOnly");
|
||||
if (["company-access-isolation", "task-edit-lifecycle"].includes(checked.caseId) && checked.usage != null && Object.entries(checked.usage).some(([key, value]) => key !== "runs" || !Array.isArray(value) || value.length > 0)) invalid("result.browserOnly");
|
||||
if (Object.hasOwn(agentJourneyRuns, checked.caseId) && (!agentJourneyRunCountValid(checked.caseId, checked.runIds?.length)
|
||||
|| !checked.model || !checked.provider || checked.runtimeMode !== "native")) invalid("result.agentJourneyRun");
|
||||
if (checked.caseId === "cloud-onboarding" && (!checked.runIds?.length || !evidence.operations || !evidence.operations.requestCount || !evidence.operations.traceIds.length || evidence.operations.campaignId !== evidence.campaignId || evidence.operations.attemptId !== evidence.attemptId)) invalid("result.paidJourneyReceipt");
|
||||
|
||||
Reference in new issue
Block a user