DottaandPaperclip
0a5f1546ee
## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
6.2 KiB
Minimal reusable workflows
Implement the approved name-and-prompt workflow model on branch
codex/reusable-case-workflows, under experimental.enableCases.
Workflow definitions own reusable text; cases retain one primary task and use
its revisioned workflow document. Handoffs own explicit adoption/refresh/detach,
captured prompts, and transition history. No interpreter or new execution service.
Acceptance: reuse across cases; same-task A → B → A; shared edits do not change case copies; retries/restarts do not duplicate revisions; cancelled pending adoptions and document conflicts preserve instructions; old inline cases, authorization, and flag behavior remain compatible. Verify through real database, native tools, existing case UI, and a bounded configured-provider journey.
Endpoint: one reviewable PR with required checks. No merge or deployment requested. Deferred: workflow authoring UI, deletion/archive, import/export, role binding, graph validation, and Spec Kit integration.
Status: implementation and live acceptance complete. Full-suite verification and review are tracked in https://github.com/paperclipai/paperclip/pull/17114.
Implementation: schema/migration, definition REST service and optional native tools, transactional adoption with private prompt snapshots, bounded reads, case context, Work/history/Audit presentation, contracts, docs and maps are in place. The arithmetic Product E2E journey now measures A → B → A and observed workflow-document reads; inline receipt coverage remains.
Verification so far: real-database handoff tests, authenticated native-tool/API contract tests, 37 case/Audit UI tests, and arithmetic oracle calibration pass. Repository-wide typecheck and build pass, including after a clean rebase onto master. Full-suite results are tracked in the PR. Token gates, migration checks, module boundaries and map structure checks pass. Earlier failed attempts were an incorrect assertion about conflict wording, missing fresh-worktree build dependencies, outdated API-schema assertions and generated inventories; these are retained in temporary logs and corrected before reruns.
Live evidence: the subscription attempt reached kickoff and browser readback,
then the first provider run failed with HTTP 429 (responseTooManyFailedAttempts).
That failure is retained separately. An explicitly authorized API-key attempt
uses the same arithmetic procedure, oracle and single-attempt run limits in a
fresh isolated campaign. Both arithmetic credential modes are now selectable.
The first API attempt completed four successful provider runs, A → B → A, and
normal terminal case completion. Its old four-handoff oracle kept waiting for
exactly four rows despite the valid fifth terminal row. The attempt was stopped
with normal launcher cleanup and remains failed. The fixture now explicitly
requires the terminal intent and its retained procedure revision; positive and
negative oracle tests pass. A fresh single-attempt campaign verifies that fix.
The terminal-aware API attempt passed all nine durable workflow checks and the
case Work/history browser assertions. Its final chat-marker assertion failed:
the existing presentation policy chooses the explicit terminal handoff comment
over the later completion summary. The fixture now puts the marker in that
authoritative visible comment and retains the earlier failure separately.
Runner contract checks exposed strict JSON-schema branches and an omitted public semantic catalog registration for the four tools. Both are corrected; all 144 affected catalog/discovery tests and 27 prompt/coverage tests pass. Generated projections were refreshed from a fresh TypeScript build. The repository-wide suite uses the repository-supported isolated shard groups.
Live acceptance passed in reusable-workflows-api-final-2026-10-21, using local
code matching commit a1d73b4652: four actual Codex API runs, all nine durable
checks, current Work summary, historical workflow/phase names, original task
completion and visible terminal comment. Cleanup passed. The result reports
token usage but incomplete aggregate dollar accounting; it does not establish
zero provider cost. Earlier campaigns remain failed and unchanged. The report
is retained at tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/dashboard.html
(ignored local evidence, not committed screenshots).
Final verification retains the interrupted default serial run as incomplete. The replacement covers all 1,082 general server suites and 161 serialized suites without gaps or overlap, plus all three workspace groups. CI caught a missing workflow route-inventory registration; explicit route declarations and that registration now pass the OpenAPI/native-tool integration checks (49 tests). Case context selects only incoming-handoff metadata, avoiding reads of the private captured workflow prompt on each resume.
The final context projection passes all 90 database handoff checks and server typecheck. CI also detected the changed shared instruction source in Copilot's pinned identity. Version 56 binds the new guidance, retains v55 decoding and historical fixtures, and passes 142 installation/sidecar checks. This is static compatibility evidence; live acceptance here covers native Codex. One local environment-route socket disconnect remains recorded; its separate 11-test rerun passed without changes to that unrelated route or test.
Review found that separate prompt pages could observe different definition edits. Workflow reads now return a content fingerprint and require it on continuation pages, rejecting edits with 409 before returning another page. The HTTP test covers a same-length edit and restarting with a four-byte window; compression preserves the fingerprint through its weak ETag prefix. No workflow version entity or stored field is added. Final check and review results live on the PR.
The shared native execution prompt omits experimental Cases/workflow guidance,
including the Mermaid diagram recommendation. Runtime-context prompt text lives
in the CODEOWNERS-protected runtime-context-prompts.ts as exported strings;
composition and snapshot validation remain in runtime-context.ts. Current
prompt metadata advances to v7, while saved prompt snapshots remain readable.
Copilot v57 binds both source files and retains historical v56 decoding.