From 45862dd2102e4aa506712338a70b334b54060b97 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Mon, 5 Oct 2026 21:08:47 -0500 Subject: [PATCH 01/12] docs: add a product feature map (#15288) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its user journeys span tasks, agents, projects, connected apps, governance, and CLI operations. > - Existing tests do not provide a shared index of entry points and verification steps. > - Contributors need to see which surfaces a change affects and what evidence exists. > - This pull request adds an optional product feature map with recipes and explicit coverage gaps. > - The map adds no CI checks or required maintenance for future pull requests. > - Contributors can use it to find verification steps and state what they checked. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** User-journey verification guidance in AGENTS.md and doc/DEVELOPING.md. **What's wrong?** There is no shared index of product features, user entry points, available test evidence, and remaining coverage gaps. Shared components can hide differences between their hosts. **Suggested fix** Add a documentation-only capability index and verification recipes. The format takes inspiration from [Omnigent's feature map](https://github.com/omnigent-ai/omnigent/tree/91acfbbb59f6fc210ff95a9e9428aadd62e06582/feature-map). The recipes describe Paperclip's own behavior and tests. Searched GitHub PRs and issues for `feature map` and `feature-map`. No duplicate change was found. Checked ROADMAP.md. This PR documents existing capabilities. ## What Changed - Added 35 feature recipes, 171 named sub-features, and 91 entry points across product, CLI, operator, and developer surfaces. - Each recipe describes setup, expected results, existing automated evidence, manual verification, and coverage gaps. - Added a dated source snapshot of 185 non-test page TSX modules in 15 areas. The snapshot describes documentation coverage, not runtime health. - Included entry points within existing pages, CLI/API operations, and experimental surfaces. Identified helper-only test evidence where a rendered journey has no automated proof. - Linked the map from AGENTS.md and doc/DEVELOPING.md as an optional reference. - The final diff contains only Markdown and the inventory JSON. It adds no checker, tests, package commands, workflows, dependencies, scheduled work, or mandatory inventory updates. ## Verification - PASS: local documentation links resolve and the inventory JSON parses. Confirmed the final PR diff contains only 39 documentation files. - PASS: `node --test '.github/scripts/tests/*.test.mjs'` — 381 existing tests after removal of the feature-map tests. - PASS: `git diff --check`. - Earlier local build and typecheck passed. The full local test run was stopped after 26 minutes with failures in unchanged chat-channel and native-runner integration tests. It did not complete. These application checks were not repeated for the documentation-only removal. - [CI on the preceding head](https://github.com/paperclipai/paperclip/actions/runs/37398407523) passed all applicable checks. Checks on the final documentation-only head are pending. - PASS: Greptile review on final head `45c4b0aeb26540324825da43e81bee2336ad1c1f` is 5/5. There are no unresolved findings. - Live product/provider journeys were not run to author the map. The recipes identify available evidence and manual steps, not new qualification results. ## Risks Low product risk: the PR changes documentation only. Recipes and the source snapshot can become stale. Maintenance is optional and based on review. Linked tests do not prove that every documented journey works. The map states remaining coverage gaps. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, reasoning, tool use, and code execution. The exact serving model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- AGENTS.md | 4 + doc/DEVELOPING.md | 5 + feature-map/README.md | 177 +++++++++++ feature-map/access.md | 55 ++++ feature-map/activity.md | 45 +++ feature-map/agent-chat.md | 54 ++++ feature-map/agents.md | 55 ++++ feature-map/app-permissions.md | 45 +++ feature-map/budgets-costs.md | 44 +++ feature-map/cases.md | 43 +++ feature-map/chat-channels.md | 45 +++ feature-map/cli-operations.md | 55 ++++ feature-map/companies.md | 44 +++ feature-map/connection-setup.md | 150 +++++++++ feature-map/coverage.json | 401 +++++++++++++++++++++++++ feature-map/developer-labs.md | 43 +++ feature-map/documents-artifacts.md | 55 ++++ feature-map/execution-environments.md | 45 +++ feature-map/gateways-profiles.md | 45 +++ feature-map/goals.md | 42 +++ feature-map/inbox-search.md | 54 ++++ feature-map/instance-operations.md | 45 +++ feature-map/navigation-preferences.md | 45 +++ feature-map/onboarding.md | 45 +++ feature-map/pipelines.md | 44 +++ feature-map/plugins.md | 44 +++ feature-map/projects.md | 44 +++ feature-map/questions-and-approvals.md | 182 +++++++++++ feature-map/recovery.md | 159 ++++++++++ feature-map/routines.md | 45 +++ feature-map/runs-adapters.md | 44 +++ feature-map/secrets.md | 45 +++ feature-map/skills.md | 55 ++++ feature-map/status-cards.md | 44 +++ feature-map/steering.md | 135 +++++++++ feature-map/task-coordination.md | 44 +++ feature-map/tasks.md | 55 ++++ feature-map/teams.md | 44 +++ feature-map/workspaces.md | 45 +++ 39 files changed, 2670 insertions(+) create mode 100644 feature-map/README.md create mode 100644 feature-map/access.md create mode 100644 feature-map/activity.md create mode 100644 feature-map/agent-chat.md create mode 100644 feature-map/agents.md create mode 100644 feature-map/app-permissions.md create mode 100644 feature-map/budgets-costs.md create mode 100644 feature-map/cases.md create mode 100644 feature-map/chat-channels.md create mode 100644 feature-map/cli-operations.md create mode 100644 feature-map/companies.md create mode 100644 feature-map/connection-setup.md create mode 100644 feature-map/coverage.json create mode 100644 feature-map/developer-labs.md create mode 100644 feature-map/documents-artifacts.md create mode 100644 feature-map/execution-environments.md create mode 100644 feature-map/gateways-profiles.md create mode 100644 feature-map/goals.md create mode 100644 feature-map/inbox-search.md create mode 100644 feature-map/instance-operations.md create mode 100644 feature-map/navigation-preferences.md create mode 100644 feature-map/onboarding.md create mode 100644 feature-map/pipelines.md create mode 100644 feature-map/plugins.md create mode 100644 feature-map/projects.md create mode 100644 feature-map/questions-and-approvals.md create mode 100644 feature-map/recovery.md create mode 100644 feature-map/routines.md create mode 100644 feature-map/runs-adapters.md create mode 100644 feature-map/secrets.md create mode 100644 feature-map/skills.md create mode 100644 feature-map/status-cards.md create mode 100644 feature-map/steering.md create mode 100644 feature-map/task-coordination.md create mode 100644 feature-map/tasks.md create mode 100644 feature-map/teams.md create mode 100644 feature-map/workspaces.md diff --git a/AGENTS.md b/AGENTS.md index c957ca41ea..53f25d6d51 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -152,6 +152,10 @@ Notes: ## 7. Verification Before Hand-off +[feature-map/README.md](feature-map/README.md) is an optional reference for user +entry points, verification recipes, and coverage gaps. The map records coverage +scope, not proof that a live journey passed. + Default local/agent test path: ```sh diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index c30e261158..fb3f6d9447 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -460,6 +460,11 @@ npx paperclipai allowed-hostname dotta-macbook-pro ## Test Commands +The [feature map](../feature-map/README.md) is an optional reference for user +entry points, targeted tests, manual verification recipes, and coverage gaps. +Its page inventory is a source snapshot. The documented journeys have separate +verification steps and do not run automatically from the map. + Use the cheap local default unless you are specifically working on browser flows: ```sh diff --git a/feature-map/README.md b/feature-map/README.md new file mode 100644 index 0000000000..b38d7fb650 --- /dev/null +++ b/feature-map/README.md @@ -0,0 +1,177 @@ +# Paperclip feature map + +Start here when reproducing a user-facing bug, verifying a change, or deciding +which surfaces a fix must cover. Each recipe describes what a person can do, +where they can do it, the expected result, and the evidence needed to verify it. +Product behavior is still governed by [the implementation spec](../doc/SPEC-implementation.md). + +## Features + +The map is organized by user capability, not by source file. It covers **35 feature +families** across the product, CLI, and operator workflows, checked against source +on 2026-10-05. Experimental and developer-only surfaces are labeled explicitly. +Each recipe includes sub-features, current entry points, automated evidence, +manual verification steps, and gotchas. + +### Getting started and access + +| Feature | What it covers | +| --- | --- | +| [Onboarding and first work](./onboarding.md) | Instance setup, company wizard, first agent and task. | +| [Login, invitations, and access](./access.md) | Sessions, bootstrap, membership, roles, and CLI authorization. | +| [Companies and portability](./companies.md) | Company switching/settings, archival, package import/export. | + +### Tasks and conversations + +| Feature | What it covers | +| --- | --- | +| [Task creation and lifecycle](./tasks.md) | Creation, assignment, lists, properties, comments, and completion. | +| [Delegation, dependencies, and signoff](./task-coordination.md) | Child work, prerequisites, reviewers, and task-tree controls. | +| [Inbox, decisions, and search](./inbox-search.md) | Personal triage, decision queues, unread/blocked views, and search. | +| [Agent conversations and project handoff](./agent-chat.md) | Persistent conversations, discovery, handoff, and gated board chat. | +| [Questions and approvals](./questions-and-approvals.md) | Question/plan responses, formal approvals, and app-tool review. | +| [Steering and queued messages](./steering.md) | Follow-ups, queue edits, interruption, and pause/resume. | +| [Documents, attachments, and work products](./documents-artifacts.md) | Versioned documents, annotations, files, outputs, and artifact library. | + +### Agents and reusable capabilities + +| Feature | What it covers | +| --- | --- | +| [Hiring, configuration, and organization](./agents.md) | Agent identity, instructions, reporting lines, lifecycle, and built-ins. | +| [Runs, harnesses, and model accounts](./runs-adapters.md) | Adapter/model setup, account validation, transcripts, and run history. | +| [Skills and Skill Studio](./skills.md) | Discovery, sources, authoring, revisions, tests, and agent policies. | +| [Team packages and installation](./teams.md) | Catalog preview/install via CLI/API; catalog UI availability called out. | + +### Projects and execution + +| Feature | What it covers | +| --- | --- | +| [Projects and repositories](./projects.md) | Project lifecycle, task context, repository configuration, and defaults. | +| [Goals and work alignment](./goals.md) | Goal hierarchy, ownership, status, and project/task context. | +| [Workspaces, services, and files](./workspaces.md) | Provisioning, task bindings, services/logs, Git/files, and closure. | +| [Execution environments](./execution-environments.md) | Local, SSH, sandbox providers, target probes, and custom images. | +| [Recovery](./recovery.md) | Stopped work, workspace repair, bounded continuation, and reconnect. | + +### Apps, channels, and extensions + +| Feature | What it covers | +| --- | --- | +| [Connection setup](./connection-setup.md) | Catalog, MCP links, task requests, authentication, and setup resumption. | +| [App access and action permissions](./app-permissions.md) | Agent grants, off/ask/allowed actions, discovery, tests, and revocation. | +| [External chat and email](./chat-channels.md) | Provider-specific identity, threads, files, delivery, and endpoint upkeep. | +| [Tool gateways and access profiles](./gateways-profiles.md) | Tool exposure, client configuration, tokens, profiles, and activity. | +| [Secrets and proposals](./secrets.md) | Company/user credentials, grants, proposals, and vault import. | +| [Plugins](./plugins.md) | Installation/configuration, contributed pages/tools, and lifecycle. | + +### Automation and structured work + +| Feature | What it covers | +| --- | --- | +| [Routines, schedules, and triggers](./routines.md) | Definitions, variables, scheduled/webhook/manual runs, and history. | +| [Pipelines, review queues, and learnings](./pipelines.md) | Experimental stages, automation, items, review, and learning records. | +| [Cases](./cases.md) | Experimental structured fields, relationships, revisions, and task links. | +| [Status cards](./status-cards.md) | Experimental summaries, watched work, refresh history, and settings. | + +### Oversight and operation + +| Feature | What it covers | +| --- | --- | +| [Costs and budgets](./budgets-costs.md) | Spend reports, scoped limits, incidents, hard stops, and resumption. | +| [Dashboards and audit trails](./activity.md) | Company health, live work, activity, runs, routines, and timeline. | +| [Navigation, profile, and announcements](./navigation-preferences.md) | Sidebar state, favorites/recents, personal identity, and dismissals. | +| [Instance operations](./instance-operations.md) | Installation, updates, service health, configuration, and backups. | +| [CLI/API and local worktrees](./cli-operations.md) | Explicit context, resource commands, outputs, runs, and isolated instances. | + +### Contributor surfaces + +| Feature | What it covers | +| --- | --- | +| [Developer previews and diagnostic labs](./developer-labs.md) | Design examples, interaction fixtures, performance checks; not production acceptance. | + +These are verification instructions, not a claim that every journey passed a live +test. A component test proves its component; a scripted provider proves that +fixture integration. Record executed results separately from the map. + +## Before driving a journey + +1. Use this checkout's [isolated test drive](../doc/DEVELOPING.md#one-command-isolated-manual-test-drive) + for manual product checks. It creates a temporary instance and prints its URL + and data directory. Do not guess that a server on port 3100 belongs to you. +2. Record the commit, URL, company, login role, relevant experimental settings, + adapter, runtime mode, and live versus simulated dependencies. Use disposable + tasks and provider resources. A fresh test drive creates a company and CEO, + but no task or first run. +3. Use the current navigation and company prefix. Paths in recipes omit that + prefix. Record whether the streamlined or production shell is selected; + Agent Chat, chat connectors, and the combined Inbox/Tasks view have separate + gates. A hidden surface is an unmet prerequisite, not a successful test. +4. Run the smallest relevant test first. Vitest commands below run from the + repository root after installing dependencies. Playwright recipes use their + named configuration and its isolated environment, not an unrelated running + instance. Reserve expensive runner/provider tests for the behavior at issue. +5. Exercise the actual user action, inspect its result, reload, and verify the + persisted state or continuation. Use read-only API checks to corroborate UI + evidence; creating state through the API does not prove the creation UI. + +For agent-driven acceptance work, the existing +[dev-workspace run/verify skill](../.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md) +and [evaluation skill](../.agents/skills/paperclip-evals/SKILL.md) describe runtime +ownership and evidence handling. Reuse them; this map introduces no environment +launcher, credentials, scheduled job, or second test framework. + +## Evidence contract + +Report each entry point as **passed**, **failed**, **blocked** (with its missing +prerequisite), or **not run**. Include the feature filename and entry-point ID, +commit/environment, user action, expected and observed result, command/exit code, +and evidence links. Screenshots or traces should show both the action and the +discriminating result. Include task/run/request IDs when relevant, without secrets. + +A fix is verified across its affected surfaces only when each has evidence or +an explicit reason it does not apply. Shared code alone does not prove parity. +Keep product regressions visible; do not change the recipe to bless a failure. +For Paperclip-assigned work, attach evidence through the +[artifact workflow](../doc/AGENT-ARTIFACTS.md). + +## Inventory and remaining depth + +[The UI coverage inventory](./coverage.json) accounts for every non-test TSX +module under `ui/src/pages`, including supporting panels, legacy variants, and +labs. Every product area now links to concrete recipes. **Partial** means the +area still has the named variant or workflow gaps; **unmapped** is reserved for +an area with no recipe. These statuses describe documentation, not runtime health. + +The page inventory is a source snapshot dated 2026-10-05. The map also +includes entry points hosted inside other modules (pipeline Review Queue and +Learnings, task documents, onboarding), CLI-only operations, and operator work. +The team recipe explicitly distinguishes the current CLI/API path from catalog +UI components without a current top-level route. Refer to [route registration](../ui/src/App.tsx) +and the [CLI registry](../cli/src/index.ts) when changing reachability. + +Remaining depth includes complete provider/auth/attachment matrices, every +harness/model/environment capability combination, third-party plugin features, +and every role/error/mobile/legacy-shell permutation. The recipes name relevant +gaps instead of equating source presence with a working user journey. The index +can include capabilities that do not add a page file. + +## Maintaining the reference + +The map is documentation only. It adds no CI checks, automatic journey execution, +or required inventory updates for future pull requests. + +When maintaining a recipe, compare its entry points and expected results with +the current source. Check that linked tests still exercise the stated behavior +and that local references still exist. Refresh the inventory snapshot when it +helps explain the current product. Maintenance is optional and review-based. + +Each recipe starts with an H1 and a user-visible description, then exactly: + +1. **Sub-features** — stable backticked IDs and observable behavior/states. +2. **How to get to it (user POV)** — one H3 backticked entry-point ID per surface. +3. **Driving it** — starts with `Preconditions:`; repeat each entry-point H3, + with `Automated:` and `Manual:` paragraphs. Name test scope and gaps honestly. +4. **Gotchas** — misleading look-alikes, feature gates, and invalid evidence. + +The format is inspired by +[Omnigent's feature map](https://github.com/omnigent-ai/omnigent/tree/91acfbbb59f6fc210ff95a9e9428aadd62e06582/feature-map). +Paperclip's recipes and checks follow its own task model and test infrastructure. diff --git a/feature-map/access.md b/feature-map/access.md new file mode 100644 index 0000000000..d7248c8b63 --- /dev/null +++ b/feature-map/access.md @@ -0,0 +1,55 @@ +# Login, invitations, and access + +People and agents enter the correct instance and company with the permissions granted to their identity. Login, membership, resource access, and CLI authorization have separate controls. + +Implementation: [auth page](../ui/src/pages/Auth.tsx), [company members](../ui/src/pages/CompanyAccess.tsx), [invitation](../ui/src/pages/InviteLanding.tsx). + +## Sub-features + +- `login`: sign in or claim the board in the configured deployment mode. +- `invitations`: join through a valid invite and handle expired or already-used links. +- `membership`: review join requests and manage company/instance membership and permissions. +- `resources`: respect company and resource boundaries on deep links and mutations. +- `cli-auth`: authorize a CLI session and revoke or replace credentials. + +## How to get to it (user POV) + +### `login` + +Open `/auth`; first-owner bootstrap uses `/board-claim/:token` when configured. + +### `invite-members` + +Open `/invite/:token`; operators manage members in company settings and join requests at `/inbox/requests`. + +### `cli-authorization` + +Start the CLI auth/connect flow and approve its `/cli-auth/:id` browser page; instance access is under Settings when visible. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). For login tests use authenticated mode and two test users; local-trusted mode does not prove session enforcement. Use two companies for isolation checks. + +### `login` + +Automated: [auth UI](../ui/src/pages/Auth.test.tsx) and [session routes](../server/src/__tests__/auth-session-route.test.ts) cover their respective layers. + +Manual: Sign in, reload a company deep link, sign out, and revisit it. Confirm the browser requests authentication rather than showing cached company data. Test an invalid claim separately on disposable state. + +### `invite-members` + +Automated: [member UI](../ui/src/pages/CompanyAccess.test.tsx) and [invite replay](../server/src/__tests__/invite-accept-replay.test.ts) provide component and route evidence. + +Manual: Accept an invite as the intended user, inspect the resulting company and permissions, then reuse and expire disposable invites. Change a member permission, reload as that member, and verify a denied operation on another company remains denied. + +### `cli-authorization` + +Automated: [board auth client](../cli/src/__tests__/board-auth.test.ts) and [token commands](../cli/src/__tests__/token.test.ts) cover the client contracts. + +Manual: Authorize the intended CLI context, make a read-only company request, then revoke its token and retry. Verify failure is explicit and does not silently fall back to another identity. + +## Gotchas + +- Settings visibility is not authorization; direct requests still need permission checks. +- An agent bearer key must not gain another company’s access. +- Never put invite tokens, session cookies, or API keys in verification evidence. diff --git a/feature-map/activity.md b/feature-map/activity.md new file mode 100644 index 0000000000..a50206a9fc --- /dev/null +++ b/feature-map/activity.md @@ -0,0 +1,45 @@ +# Dashboards, activity, and audit trails + +People inspect company health, recent activity, run history, and routine activity, then follow evidence back to the task or agent responsible. + +Implementation: [dashboard](../ui/src/pages/Dashboard.tsx), [live dashboard](../ui/src/pages/DashboardLive.tsx), [audit hub](../ui/src/pages/audit/AuditHub.tsx). + +## Sub-features + +- `overview`: inspect the company dashboard and live-work summary. +- `activity`: filter and page through recorded actor/resource events. +- `runs`: find executions and follow their task/agent context. +- `routine-audit`: inspect routine execution activity separately from its definition. +- `timeline`: use the available timeline view to understand work over time. + +## How to get to it (user POV) + +### `dashboard` + +Open `/dashboard` or `/dashboard/live`. + +### `activity-audit` + +Open `/activity`, `/activity/runs`, or `/activity/timeline` in the streamlined shell; legacy routes may redirect. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare known task mutations and completed/failed runs in a disposable company, recording their IDs and actors. + +### `dashboard` + +Automated: [dashboard service](../server/src/__tests__/dashboard-service.test.ts) covers server aggregation; [dashboard helpers](../ui/src/pages/Dashboard.test.ts) is narrower UI helper coverage. + +Manual: Compare the dashboard with the known fixture tasks and runs. Start and finish a small task, refresh/reload, and verify the summary changes and links reach the right records. Exercise empty data and a failed fetch. + +### `activity-audit` + +Automated: [activity service](../server/src/__tests__/activity-service.test.ts), [audit runs](../ui/src/pages/audit/AuditRuns.test.tsx), and [routine audit](../ui/src/pages/audit/RoutineAuditActivity.test.tsx) cover these separate data/host layers. + +Manual: Find a known mutation by actor/resource and a known run by status. Change filters, load more, and follow the source links. Reload the chosen view and verify no company leakage or duplicate events. Inspect a routine run from its audit entry. + +## Gotchas + +- Audit/activity events, run-log rows, first-party Telemetry, and operator OpenTelemetry traces are separate data paths. +- A healthy stream or dashboard counter alone does not prove task success. +- Cost and budget reports have their own [recipe](./budgets-costs.md). diff --git a/feature-map/agent-chat.md b/feature-map/agent-chat.md new file mode 100644 index 0000000000..e47de311b5 --- /dev/null +++ b/feature-map/agent-chat.md @@ -0,0 +1,54 @@ +# Agent conversations and project handoff + +A person can hold a persistent conversation with an agent, return to its history, and hand useful work into a project or task. The gated conference-room chat is a separate conversation surface. + +Implementation: [agent chat](../ui/src/pages/AgentChat.tsx), [chat landing](../ui/src/pages/AgentChats.tsx), [conference room](../ui/src/pages/BoardChat.tsx). + +## Sub-features + +- `discovery`: find an agent and reopen an accessible recent conversation. +- `history`: send messages, inspect replies/files, and return to the same conversation. +- `handoff`: create or identify a project/task from chat and follow the resulting work. +- `conference-room`: use the separately gated board chat without assuming parity with agent chat. + +## How to get to it (user POV) + +### `agent-conversation` + +Open `/chats`, choose an agent, or use `/chats/:agentRef` and its conversation sidebar. + +### `project-handoff` + +Ask for project work in Agent Chat and follow its created project/task links. + +### `conference-room` + +Open `/board-chat` only when its conference-room gate is enabled. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable the applicable chat feature, use a configured test agent, and distinguish model replies from a mocked conversation fixture. + +### `agent-conversation` + +Automated: [chat landing](../ui/src/pages/AgentChats.test.tsx) checks recent-chat routing; [conversation service](../server/src/__tests__/agent-conversations.test.ts) covers persisted conversation behavior. + +Manual: Start a conversation, send a recognizable request, and inspect its reply. Navigate away and reopen the same history. Test a new conversation, unavailable historical agent, and an inaccessible saved conversation; verify no cross-user or cross-company history appears. + +### `project-handoff` + +Automated: [chat project tools](../server/src/__tests__/chat-project-tools.test.ts) checks tool contracts; live model choice and UI handoff require a manual run. + +Manual: Ask the configured agent to create bounded project work, inspect the proposed/created target, and open the resulting task. Verify the request context and intended project survived and the task produces a useful output. + +### `conference-room` + +Automated: [board chat gate](../server/src/__tests__/board-chat-route-feature-flag.test.ts) proves route gating, not a live conversation. + +Manual: Check disabled access first, enable in the disposable instance, then send and reload a conversation. Inspect the actual responding agent and persisted history; report this result separately from Agent Chat. + +## Gotchas + +- Idle chat is not failed execution. +- See [steering](./steering.md) for in-flight messages and [questions](./questions-and-approvals.md) for interactions. +- An API tool test does not prove that every harness will choose the same handoff tool. diff --git a/feature-map/agents.md b/feature-map/agents.md new file mode 100644 index 0000000000..a1633a52da --- /dev/null +++ b/feature-map/agents.md @@ -0,0 +1,55 @@ +# Hiring, agent configuration, and organization + +Operators hire agents, configure their identity and instructions, assign reporting relationships, and manage their lifecycle. Built-in agents have their own setup and lifecycle constraints. + +Implementation: [hiring](../ui/src/pages/NewAgent.tsx), [configuration](../ui/src/components/AgentConfigForm.tsx), [organization](../ui/src/pages/OrgChart.tsx). + +## Sub-features + +- `hire`: choose role, manager, adapter, model, and required account settings. +- `instructions`: edit agent configuration and instruction files with durable revisions. +- `organization`: inspect and change reporting relationships through supported controls. +- `lifecycle`: pause/resume or retire an agent while preserving history. +- `built-in`: configure required agents without treating them as ordinary removable hires. + +## How to get to it (user POV) + +### `hire-agent` + +Use New agent from the roster or `/agents/new`. + +### `agent-configuration` + +Open `/agents/:agentId` and the configuration/instruction surfaces it exposes. + +### `organization` + +Use the roster’s organization view; `/org` redirects to the roster in the streamlined shell. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable company and supported adapter; hiring may require a formal approval. + +### `hire-agent` + +Automated: [new agent page](../ui/src/pages/NewAgent.test.tsx) and [configuration form](../ui/src/components/AgentConfigForm.render.test.tsx) cover mocked UI behavior. + +Manual: Hire an agent with a distinct name and role. Complete any required approval, reload the roster, and open its configuration. Confirm manager, adapter, and model persisted; run a small task to prove execution readiness. + +### `agent-configuration` + +Automated: [instruction service](../server/src/__tests__/agent-instructions-service.test.ts) and [instruction revisions](../server/src/__tests__/agent-instruction-revisions.test.ts) cover file/revision contracts. + +Manual: Save a harmless instruction edit, reload, and verify its revision and effect in the next eligible run. Pause and resume the disposable agent; confirm pause prevents new work and does not erase previous runs. + +### `organization` + +Automated: [org chart](../ui/src/pages/OrgChart.test.tsx) is component evidence. Built-in setup also has [configuration modal tests](../ui/src/components/ConfigureBuiltInAgentModal.test.tsx). + +Manual: Inspect a manager and report, change the relationship through the offered controls, and reload both views. Exercise a rejected invalid relationship. Configure a built-in agent and verify it retains its required identity and restrictions. + +## Gotchas + +- A roster entry is not proof of an authenticated or invokable adapter. +- Pending hires may live in approval/task surfaces rather than the active roster. +- Skill and tool grants are separate: see [skills](./skills.md) and [app permissions](./app-permissions.md). diff --git a/feature-map/app-permissions.md b/feature-map/app-permissions.md new file mode 100644 index 0000000000..fd8a825c51 --- /dev/null +++ b/feature-map/app-permissions.md @@ -0,0 +1,45 @@ +# App access, action permissions, and testing + +After connecting an app, an operator decides which agents may use it and whether each action is off, requires approval, or is allowed. Activity and action tests help verify the resulting access. + +Implementation: [app detail](../ui/src/pages/apps/AppDetail.tsx), [permissions](../ui/src/pages/apps/app-detail/PermissionsPanel.tsx), [activity](../ui/src/pages/apps/app-detail/ActivityPanel.tsx). + +## Sub-features + +- `agent-access`: grant the connection to all or selected eligible agents. +- `action-policy`: set off/ask/allowed independently for available actions. +- `catalog-change`: review newly discovered/quarantined actions before enabling them. +- `test-action`: choose an actor and inputs and inspect a real action result. +- `account-maintenance`: inspect accounts, reconnect/revoke, and review connection activity. + +## How to get to it (user POV) + +### `permissions` + +Open an app at `/apps/:connectionId/permissions`; this is the default connected-app view. + +### `test-and-maintain` + +Use the action test control in Permissions, then the connection’s Activity and available account/advanced controls. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare a connection to a fixture or test provider, two agents, and a harmless action. Record provider and credential ownership. + +### `permissions` + +Automated: [app detail](../ui/src/pages/apps/AppDetail.test.tsx) covers agent/action controls, quarantined actions, and navigation using mocked requests. + +Manual: Grant one agent access, leave another excluded, and exercise off, ask, and allowed states for a harmless action. Reload between changes. Verify actual denied calls stay denied and ask-first creates a decision rather than executing early. + +### `test-and-maintain` + +Automated: [app detail tests](../ui/src/pages/apps/AppDetail.test.tsx) cover test-dialog/UI contracts. Real provider execution and revocation require the selected provider’s live check. + +Manual: Run a read-only action with explicit inputs and inspect returned data and activity. Refresh discovery and review new actions. Revoke or disconnect only the disposable credential, then verify subsequent use fails with an actionable reconnect state. + +## Gotchas + +- Connected does not imply an agent has access or an action is allowed. +- Credential-only connections may replace tool-action controls with account-specific controls. +- See [connection setup](./connection-setup.md) for auth and [questions and approvals](./questions-and-approvals.md) for pending action review. diff --git a/feature-map/budgets-costs.md b/feature-map/budgets-costs.md new file mode 100644 index 0000000000..e1a1efc573 --- /dev/null +++ b/feature-map/budgets-costs.md @@ -0,0 +1,44 @@ +# Costs, budgets, and spending controls + +Operators inspect recorded spend and configure budget controls for the company, projects, and agents. Hard stops pause or prevent eligible work until the governing incident is resolved. + +Implementation: [costs and budget host](../ui/src/pages/Costs.tsx), [audit hub](../ui/src/pages/audit/AuditHub.tsx). + +## Sub-features + +- `cost-reporting`: inspect spend by the offered time range and resource dimensions. +- `budgets`: configure supported company/project/agent limits and inspect usage. +- `incidents`: distinguish warnings, hard stops, and manual pauses. +- `resolution`: raise or resolve a budget incident through authorized controls and verify resumption. + +## How to get to it (user POV) + +### `cost-overview` + +Use Activity → Costs/Budgets in the streamlined shell, `/costs` in the production shell, or scoped project/agent budget panels. + +### `budget-enforcement` + +Open the governing budget configuration and any resulting budget incident card. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use synthetic low-cost events and small disposable budgets. Do not buy model usage merely to hit a threshold. + +### `cost-overview` + +Automated: [costs service](../server/src/__tests__/costs-service.test.ts) covers cost contracts; [embedded costs UI](../ui/src/pages/Costs.test.tsx) has narrow Audit-host coverage. + +Manual: Inspect a known fixture run and reconcile its recorded spend with the selected report period and resource. Switch company/project filters and reload. Check empty and unavailable cost data without treating missing data as zero spend. + +### `budget-enforcement` + +Automated: [budget service](../server/src/__tests__/budgets-service.test.ts) exercises soft incidents, hard stops, scope, and valid budget raises. + +Manual: Create a small fixture budget, cross its threshold with synthetic recorded spend, and verify the warning or hard-stop state. Attempt new work and confirm it is gated. Raise the correct budget through an authorized action and verify eligible continuation without clearing unrelated manual pauses. + +## Gotchas + +- Paperclip cost accounting is not a provider invoice or a promise about externally billed spend. +- Company, project, and agent gates can overlap; fixing one may leave another active. +- A successful retry must not bypass an unresolved budget hold. diff --git a/feature-map/cases.md b/feature-map/cases.md new file mode 100644 index 0000000000..10cde78e5e --- /dev/null +++ b/feature-map/cases.md @@ -0,0 +1,43 @@ +# Cases and structured work records + +The cases experiment exposes structured records with types, fields, parent/child relationships, activity, revisions, and related task work. + +Implementation: [case list](../ui/src/pages/Cases.tsx), [case detail](../ui/src/pages/CaseDetail.tsx). + +## Sub-features + +- `browse`: search, filter, sort, group, and navigate case records. +- `structure`: inspect fields and parent/child relationships without losing matching descendants. +- `history`: inspect revisions and activity for a record. +- `task-links`: follow work associated with a case in both directions. + +## How to get to it (user POV) + +### `case-list` + +Open `/cases` and switch between available grouped/tree views. + +### `case-record` + +Open `/cases/:caseIdentifier` or a task’s linked case. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable cases and prepare related records with different types/statuses plus an associated task. + +### `case-list` + +Automated: [case list](../ui/src/pages/Cases.test.tsx) covers filters, persisted view state, and keyboard/tree behavior with mocked data. + +Manual: Search for a child, verify its parent context remains understandable, change filters/grouping/columns, and reload. Open the record by keyboard and direct link. Include empty results and terminal records. + +### `case-record` + +Automated: [case routes](../server/src/__tests__/cases-routes.test.ts) covers API behavior; [case fields](../ui/src/components/CaseFieldsPanel.test.tsx) covers field UI separately. + +Manual: Edit an offered field, reload, inspect its revision/activity, and follow child and task links. Verify cross-company and invalid field changes fail clearly. Distinguish a pipeline-owned item from a stand-alone case. + +## Gotchas + +- A case is not a task execution; inspect linked tasks to prove work occurred. +- Feature gating and case-type schema determine which actions are available. diff --git a/feature-map/chat-channels.md b/feature-map/chat-channels.md new file mode 100644 index 0000000000..3399965dca --- /dev/null +++ b/feature-map/chat-channels.md @@ -0,0 +1,45 @@ +# External chat and email conversations + +People can talk to an assigned agent through configured external channels, bind their identity, exchange files, and inspect delivery state. Each provider has a different identity, threading, and attachment contract. + +Implementation: [endpoint setup](../ui/src/pages/apps/chat/ChatEndpointSetup.tsx), [endpoint detail](../ui/src/pages/apps/chat/ChatEndpointDetail.tsx), [email setup](../ui/src/pages/apps/chat/EmailEndpointSetup.tsx). + +## Sub-features + +- `providers`: configure the available Slack, Discord, Microsoft Teams, Telegram, GitHub, AgentMail, and experimental iMessage Photon paths; availability still depends on gates and provider support. +- `identity`: link the external sender to the intended authorized Paperclip identity. +- `conversation`: route inbound messages to the selected agent and send replies to the correct thread. +- `attachments`: transfer supported files with inspectable delivery and failure states. +- `lifecycle`: inspect endpoint activity, repair auth, and disable delivery when required. + +## How to get to it (user POV) + +### `external-thread` + +Message the configured bot/address from its external provider and follow the corresponding Paperclip conversation/task. + +### `endpoint-management` + +Open `/apps/chat/:endpointId/settings` and the endpoint’s other available tabs; identity confirmation uses `/chat-identity/confirm`. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable chat connectors and use a test provider account/channel with permission to send messages. Setup is covered in [connection setup](./connection-setup.md). + +### `external-thread` + +Automated: [channel integration](../server/src/__tests__/chat-channels.integration.test.ts) uses provider fixtures; it does not prove a live Slack, Discord, Teams, Telegram, GitHub, email, or Photon delivery. + +Manual: For each supported provider being changed, send a unique message, verify the intended agent/company receives it, and inspect the actual external reply in the original thread. Repeat with an unlinked/unauthorized sender, duplicate inbound delivery, and one supported attachment. Record provider-specific results separately. + +### `endpoint-management` + +Automated: [channel integration](../server/src/__tests__/chat-channels.integration.test.ts) covers shared endpoint/delivery behavior; live credential repair and provider ownership remain manual checks. + +Manual: Inspect the bound agent and identity, recent activity, and delivery failure details. Repair a disposable credential, retry a safe message when offered, and verify exactly one external reply. Disable the endpoint and confirm new traffic is not silently processed. + +## Gotchas + +- One provider’s passing fixture does not qualify another provider or live delivery. +- An accepted send request does not prove the recipient received the message or file. +- Use [steering](./steering.md) and [questions](./questions-and-approvals.md) for their channel-specific interaction checks. diff --git a/feature-map/cli-operations.md b/feature-map/cli-operations.md new file mode 100644 index 0000000000..34c95f01c6 --- /dev/null +++ b/feature-map/cli-operations.md @@ -0,0 +1,55 @@ +# CLI, API clients, and local worktrees + +Operators and agents can work through explicit CLI/API context, manage domain resources, inspect runs, and create isolated local worktree instances for development. + +Implementation: [command registry](../cli/src/index.ts), [client context](../cli/src/commands/client/context.ts), [worktree commands](../cli/src/commands/worktree.ts). + +## Sub-features + +- `context`: choose instance/API base, profile, identity, and company explicitly. +- `domain-commands`: read/write tasks, agents, projects, goals, approvals, routines, and other registered resources. +- `outputs`: work with documents/assets/skills and inspect machine-readable results. +- `run-controls`: invoke/inspect supported runs and observe actual outcomes. +- `worktree-instances`: initialize, seed, diagnose, and clean up only an owned development instance. + +## How to get to it (user POV) + +### `client-context` + +Use `paperclipai context show/list/use/set`, auth/connect, and the registered resource command help. + +### `subresources-and-runs` + +Use issue subresource, asset/skill, and run/heartbeat commands exposed by the CLI help. + +### `worktree-instance` + +Use the documented worktree/test-drive commands in an owned checkout; inspect `paperclipai worktree --help` first. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use `--help` from the checked-out CLI to discover exact arguments. Select a disposable company and isolated context; do not inherit an unknown API target. + +### `client-context` + +Automated: [context](../cli/src/__tests__/context.test.ts), [HTTP client](../cli/src/__tests__/http.test.ts), and [operations parity](../cli/src/__tests__/operations-parity.test.ts) cover client contracts, not every live endpoint. + +Manual: Inspect the selected context and identity, list the disposable company’s resources, create a harmless task through the supported command, and verify it in the browser. Exercise an invalid token and wrong-company resource. Inspect JSON output and nonzero errors rather than only printed success text. + +### `subresources-and-runs` + +Automated: [issue subresources](../cli/src/__tests__/issue-subresources.test.ts) and [asset/skill parity](../cli/src/__tests__/admin-asset-skill-parity.test.ts) cover command/API contracts. + +Manual: Write a small task document or attach a fixture using the supported command, open it in the UI, and compare contents. Invoke eligible test work, inspect the resulting run ID/status/output, and verify denial when the actor lacks permission. + +### `worktree-instance` + +Automated: [worktree commands](../cli/src/__tests__/worktree.test.ts) and [test drive](../cli/src/__tests__/test-drive.test.ts) cover isolation/setup contracts. + +Manual: Initialize an owned disposable worktree instance, inspect its printed paths and ports, verify health and login, and confirm any seed data belongs to the intended source. Stop and clean up only that instance; inspect preserved work before deleting any checkout. + +## Gotchas + +- API success proves its contract, not the equivalent browser creation journey. +- The CLI command registry is broader than the UI page inventory; new command families need a recipe review. +- See [instance operations](./instance-operations.md) for install/update/backup and feature-specific recipes for domain behavior. diff --git a/feature-map/companies.md b/feature-map/companies.md new file mode 100644 index 0000000000..d0ce685318 --- /dev/null +++ b/feature-map/companies.md @@ -0,0 +1,44 @@ +# Companies and portability + +An operator can switch between companies, change their identity and settings, archive them, and preview an import or export. Each company remains a separate scope for work and agents. + +Implementation: [companies](../ui/src/pages/Companies.tsx), [settings](../ui/src/pages/CompanySettings.tsx), [import](../ui/src/pages/CompanyImport.tsx), [export](../ui/src/pages/CompanyExport.tsx). + +## Sub-features + +- `selection`: create/select a company and keep navigation in its scope. +- `identity`: update company identity and configured settings. +- `archive`: archive and restore through the available controls without confusing archival with deletion. +- `portability`: preview a company package and inspect imported agents, projects, skills, and conflicts. + +## How to get to it (user POV) + +### `company-switcher` + +Use the company switcher or `/companies`, then enter company Settings. + +### `package-transfer` + +Use `/company/export` and `/company/import` where enabled, or the corresponding CLI company commands. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use two disposable companies and a small package fixture. Record cloud-managed restrictions and visible settings gates. + +### `company-switcher` + +Automated: [company list](../ui/src/pages/Companies.test.tsx) and [archive UI](../ui/src/pages/CompanySettingsArchive.test.tsx) cover UI behavior with mocks. + +Manual: Create or select a company, rename it, and reload. Switch to the second company and back; verify tasks, agents, and settings follow the selection. Archive the disposable company and verify its visibility and offered restoration path. + +### `package-transfer` + +Automated: [portability routes](../server/src/__tests__/company-portability-routes.test.ts) and [CLI import/export](../cli/src/__tests__/company-import-export-e2e.test.ts) cover package and command behavior, not every external Git host. + +Manual: Export a small company, inspect the package preview, and import into a disposable target. Verify counts, references, conflict choices, and agent configuration. Run one imported task after supplying its required local credentials. Confirm source company data is unchanged. + +## Gotchas + +- Package presence does not prove a runnable imported company; credentials and environment bindings need separate verification. +- Company deletion is a distinct gated operation; do not use it as the archive test. +- Settings hidden in a managed deployment should be recorded as unavailable, not bypassed. diff --git a/feature-map/connection-setup.md b/feature-map/connection-setup.md new file mode 100644 index 0000000000..00c9328a9a --- /dev/null +++ b/feature-map/connection-setup.md @@ -0,0 +1,150 @@ +# Connection setup + +People connect an app or account, choose who may use it, and recover incomplete +setup from either Apps or a task that needs access. A saved draft, a completed +provider login, a usable connection, and a resumed task are separate checkpoints. +Chat channels additionally require an endpoint and linked sender identity. + +## Sub-features + +- `catalog`: find a provider, inspect its available methods, and open setup; + unavailable methods explain their prerequisite rather than silently switching. +- `identity-and-access`: select personal/shared identity and eligible agents; + an empty explicit selection and company-wide access must not be confused. +- `credentials`: provider sign-in, key, or generic MCP URL follows its supported + setup method; credentials are not retained in browser draft storage. +- `handoff`: provider popup/full-page callback returns to the correct setup + host with the original task intent and identity choice intact. +- `draft-recovery`: cancel, Back, Save & exit, reload, Finish setup, and reconnect + preserve the exact draft rather than creating duplicate connections. +- `task-request`: only the addressed person resolves the requested connection; + usable access releases the pending task request through normal continuation. +- `channel-setup`: provider endpoint, represented agent, linked sender, and + communication instructions are configured before an external message is used. +- `failure`: denied login, unreachable/private URL, expired credentials, and + permission errors leave an actionable state rather than a false success. + +## How to get to it (user POV) + +### `apps-catalog` + +Open Apps (`/apps`), choose an app, and connect from its landing page. Select an +available method and access, then complete setup. Existing connections open +their details; unfinished ones offer a way to finish setup. + +### `mcp-link` + +Use **Connect with a link** or `/apps/byo` for a generic MCP server. Paste its +URL, select access, and follow the probe's sign-in/key guidance. Provider-specific +catalog setup and a generic endpoint may use different flows. + +### `task-request` + +Open the task's pending connection request and its setup action as the named +user. Setup can open in a task dialog and may use an OAuth popup. Follow the +request's original link if setup falls back to a standalone page. + +### `resume-connection` + +From Apps or connection details, use Finish setup, reconnect, or the offered +authentication recovery action. Reload or return from the provider before +checking whether the same saved connection became usable. + +### `chat-channel` + +With chat connectors enabled, select a supported chat/email provider in Apps +or open `/apps/chat/connect`. Complete that provider's endpoint setup, then +open its settings (`/apps/chat/:endpointId/settings`). Follow identity linking +as the intended sender before messaging the agent externally. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Choose a disposable provider account/resource and record its auth method, +deployment mode, account ownership, and access selection. For live setup, use +the provider's documented prerequisites in the +[connector playbook](../doc/connections/CONNECTOR-PLAYBOOK.md). Automated fixtures +do not establish OAuth registration, public callbacks, or production provider access. + +### `apps-catalog` + +Automated: [Browse](../ui/src/pages/apps/Browse.test.tsx), +[unconnected app](../ui/src/pages/apps/AppNotConnected.test.tsx), and +[AppsConnect](../ui/src/pages/apps/AppsConnect.test.tsx) cover catalog entry, +access selection, provider-specific method choices, and setup failures with +mocked APIs. [Connection intents](../tests/e2e/connection-intents.spec.ts) +drives store and task setup against one fake provider through a useful continuation: + +```sh +pnpm exec vitest run ui/src/pages/apps/Browse.test.tsx ui/src/pages/apps/AppsConnect.test.tsx +pnpm exec playwright test -c tests/e2e/playwright.config.ts tests/e2e/connection-intents.spec.ts +``` + +Manual: connect from an unconnected provider landing. Select the intended +identity and agent access, complete authentication, reload, and inspect the +saved connection. Run a permitted read-only action with an eligible agent and +verify its useful result. Try an ineligible agent and confirm denial. Repeat +with a user who cannot install company-wide; the UI must not imply broad access. + +### `mcp-link` + +Automated: AppsConnect's generic MCP cases cover probe-driven auth, private or +unreachable endpoints, preregistered clients, secret-backed headers, and unsafe +authorization URLs. These are component/API mocks, not remote provider proof. + +Manual: connect a known test MCP URL through the link flow, verify its saved +endpoint and a read-only action, then repeat with a bad URL and wrong credential. +The error should explain the next action and preserve non-secret setup context. +Test no-auth and authenticated endpoints separately; never infer one from the other. + +### `task-request` + +Automated: [interaction card](../ui/src/components/IssueThreadInteractionCard.test.tsx) +checks addressed-user controls. AppsConnect tests cover page/dialog parity, +task-bound OAuth, popup handoff, and verifying a saved connection before closing. +The connection-intents browser suite above exercises fixture-backed continuation. + +Manual: ask an agent to use an unavailable app, open its request as the named +user, and complete setup in the dialog. Verify the card settles, the exact task +resumes, and the agent returns a useful provider result. Repeat the page-host +fallback and denied OAuth path. As another user, confirm that merely seeing the +task does not grant the right to resolve the request. + +### `resume-connection` + +Automated: AppsConnect tests cover exact-draft resume, declined OAuth, reconnect +lookup failure/retry, response-lost creation, archived connection revival, and +saved account reuse. [App detail](../ui/src/pages/apps/AppDetail.test.tsx) covers +the connection host. These tests do not complete a real provider login. + +Manual: leave setup through its footer, reload Apps, choose Finish setup, and +confirm the same connection is resumed. Decline provider login and recover from +that checkpoint. Reconnect an expired disposable account and verify a permitted +action afterwards. Count connections before/after to catch accidental duplicates. + +### `chat-channel` + +Automated: [setup routing](../ui/src/pages/apps/chat/ChatEndpointSetup.routing.test.tsx), +[saved setup state](../ui/src/pages/apps/chat/ChatEndpointSetup.state.test.ts), +[email setup](../ui/src/pages/apps/chat/EmailEndpointSetup.test.tsx), and +[identity confirmation](../ui/src/pages/apps/chat/ChatIdentityConfirm.test.tsx) +cover shared UI logic with fixtures. Provider-specific live setup is not implied. + +Manual: complete setup for one named provider, reload endpoint settings, link +the intended sender, and send a disposable message. Verify one task and a useful +reply in the originating conversation. Inspect an unlinked sender's denial and +the endpoint's pause behavior. Repeat independently for each provider at issue; +Slack setup does not qualify Discord, Telegram, GitHub, email, or iMessage. + +## Gotchas + +- A successful OAuth callback is not proof of saved connection access or task + continuation. Verify all three checkpoints. +- Personal identity, shared account access, and per-agent access are distinct. + Connecting an account does not waive an Off action or Ask first review. +- A tool app connection is not a chat endpoint. The same provider can offer both. +- Old Apps/Connections URLs can redirect to Apps. Use the current navigation and + record the actual destination rather than documenting an obsolete screen. +- Full provider matrix, AI-account selection, gateway/profile administration, + and account revocation lifecycles remain outside this seed recipe. +- For governed actions after setup, use [questions and approvals](./questions-and-approvals.md). diff --git a/feature-map/coverage.json b/feature-map/coverage.json new file mode 100644 index 0000000000..7fc99df425 --- /dev/null +++ b/feature-map/coverage.json @@ -0,0 +1,401 @@ +{ + "version": 1, + "areas": [ + { + "id": "tasks-and-attention", + "title": "Tasks, conversations, and attention", + "status": "partial", + "features": [ + "tasks.md", + "task-coordination.md", + "inbox-search.md", + "agent-chat.md", + "questions-and-approvals.md", + "steering.md", + "recovery.md", + "documents-artifacts.md" + ], + "gap": "Recipes cover the main task/conversation/attention workflows. Full shell, permission, attachment, long-thread, and concurrency permutations remain to be enumerated.", + "paths": [ + "ui/src/pages/AgentChat.tsx", + "ui/src/pages/AgentChats.tsx", + "ui/src/pages/BoardChat.tsx", + "ui/src/pages/DecisionQueuePage.tsx", + "ui/src/pages/Inbox.tsx", + "ui/src/pages/IssueDetail.tsx", + "ui/src/pages/Issues.tsx", + "ui/src/pages/LegacyInbox.tsx", + "ui/src/pages/MyIssues.tsx", + "ui/src/pages/Search.tsx", + "ui/src/pages/WhatNeedsMe.tsx" + ] + }, + { + "id": "agents", + "title": "Agents, organization, and runtime setup", + "status": "partial", + "features": [ + "agents.md", + "runs-adapters.md", + "skills.md", + "app-permissions.md", + "recovery.md", + "execution-environments.md" + ], + "gap": "Per-harness/model/account capabilities, every instruction-file error, and complete built-in-agent lifecycle variants remain separate qualification matrices.", + "paths": [ + "ui/src/pages/AdapterManager.tsx", + "ui/src/pages/AgentDetail.production.tsx", + "ui/src/pages/AgentDetail.tsx", + "ui/src/pages/AgentToolsTab.tsx", + "ui/src/pages/Agents.production.tsx", + "ui/src/pages/Agents.tsx", + "ui/src/pages/NewAgent.tsx", + "ui/src/pages/Org.tsx", + "ui/src/pages/OrgChart.production.tsx", + "ui/src/pages/OrgChart.tsx" + ] + }, + { + "id": "apps", + "title": "App catalog and connection setup", + "status": "partial", + "features": [ + "connection-setup.md", + "app-permissions.md", + "questions-and-approvals.md" + ], + "gap": "Provider-specific authentication/revocation, credential-only accounts, and aggregator administrative variants still need their own detailed matrices.", + "paths": [ + "ui/src/pages/apps/AggregatorAppManager.tsx", + "ui/src/pages/apps/AggregatorConnectDialog.tsx", + "ui/src/pages/apps/AppDetail.tsx", + "ui/src/pages/apps/AppLogo.tsx", + "ui/src/pages/apps/AppNotConnected.tsx", + "ui/src/pages/apps/AppsConnect.tsx", + "ui/src/pages/apps/AppsReview.tsx", + "ui/src/pages/apps/ArcadeDiscoverySetup.tsx", + "ui/src/pages/apps/Browse.tsx", + "ui/src/pages/apps/CatalogSourceFilters.tsx", + "ui/src/pages/apps/ComposioAppManager.tsx", + "ui/src/pages/apps/ComposioAppSetup.tsx", + "ui/src/pages/apps/ConnectionProvenanceChip.tsx", + "ui/src/pages/apps/Connections.tsx", + "ui/src/pages/apps/ExecutorManagementSetup.tsx", + "ui/src/pages/apps/PaperclipCloudOAuthHandoff.tsx", + "ui/src/pages/apps/ReviewQueueCard.tsx", + "ui/src/pages/apps/UnverifiedServerBadge.tsx", + "ui/src/pages/apps/app-detail/ActionTestDialog.tsx", + "ui/src/pages/apps/app-detail/ActivityPanel.tsx", + "ui/src/pages/apps/app-detail/AdvancedPanel.tsx", + "ui/src/pages/apps/app-detail/AgentConnectionAccess.tsx", + "ui/src/pages/apps/app-detail/BrowserUseSettingsPanel.tsx", + "ui/src/pages/apps/app-detail/ConnectedAggregatorApps.tsx", + "ui/src/pages/apps/app-detail/IdentitiesSection.tsx", + "ui/src/pages/apps/app-detail/PermissionsPanel.tsx", + "ui/src/pages/apps/app-detail/RailwayAccessPanel.tsx", + "ui/src/pages/apps/app-detail/ReviewPanel.tsx", + "ui/src/pages/apps/app-detail/SetupPanel.tsx", + "ui/src/pages/apps/connection-owner.tsx" + ] + }, + { + "id": "chat-channels", + "title": "Chat and email channel setup", + "status": "partial", + "features": [ + "chat-channels.md", + "connection-setup.md", + "questions-and-approvals.md", + "steering.md" + ], + "gap": "Shared recipes name provider checks; each provider transport/thread/identity/file-size/type combination needs separate live evidence and deeper instructions.", + "paths": [ + "ui/src/pages/apps/chat/ChatCommunicationInstructions.tsx", + "ui/src/pages/apps/chat/ChatEndpointDetail.tsx", + "ui/src/pages/apps/chat/ChatEndpointSetup.tsx", + "ui/src/pages/apps/chat/ChatIdentityConfirm.tsx", + "ui/src/pages/apps/chat/EmailEndpointSetup.tsx", + "ui/src/pages/apps/chat/GitHubBotConfiguration.tsx", + "ui/src/pages/apps/chat/GitHubBotManagement.tsx", + "ui/src/pages/apps/chat/GitHubChatSetup.tsx", + "ui/src/pages/apps/chat/GitHubSetupPrompt.tsx", + "ui/src/pages/apps/chat/PhotonConnectStep.tsx", + "ui/src/pages/apps/chat/SetupPrompt.tsx", + "ui/src/pages/apps/chat/SlackAvatarStep.tsx", + "ui/src/pages/apps/chat/SlackIdentityStep.tsx", + "ui/src/pages/apps/chat/SlackSetupPrompt.tsx", + "ui/src/pages/apps/chat/SlackToolSettings.tsx" + ] + }, + { + "id": "skills-and-teams", + "title": "Skills, Skill Studio, and teams", + "status": "partial", + "features": [ + "skills.md", + "teams.md", + "questions-and-approvals.md" + ], + "gap": "Main library/Studio/source/policy and CLI team-install paths are mapped. Full source-provider conflicts, harness sync, and remote package variants remain; catalog UI is not a current top-level route.", + "paths": [ + "ui/src/pages/CompanySkills.production.tsx", + "ui/src/pages/CompanySkills.tsx", + "ui/src/pages/SkillSources.tsx", + "ui/src/pages/SkillStudio.tsx", + "ui/src/pages/TeamCatalog.tsx", + "ui/src/pages/agent-skills/AgentSkillReleasePicker.tsx", + "ui/src/pages/agent-skills/AgentSkillRow.tsx", + "ui/src/pages/agent-skills/AgentSkillsTab.tsx", + "ui/src/pages/skills/ImportSkillsFromProjectDialog.tsx", + "ui/src/pages/skills/SkillImportProgress.tsx", + "ui/src/pages/skills/SkillPackagePreview.tsx", + "ui/src/pages/skills/SkillSourceTree.tsx" + ] + }, + { + "id": "projects-and-workspaces", + "title": "Projects and execution workspaces", + "status": "partial", + "features": [ + "projects.md", + "workspaces.md", + "goals.md", + "execution-environments.md", + "budgets-costs.md", + "recovery.md" + ], + "gap": "Provider-specific provisioning, multi-repository/submodule edge cases, Git reconciliation variants, and all service exposure combinations require deeper recipes.", + "paths": [ + "ui/src/pages/ExecutionWorkspaceDetail.tsx", + "ui/src/pages/ProjectDetail.tsx", + "ui/src/pages/ProjectWorkspaceDetail.tsx", + "ui/src/pages/Projects.tsx", + "ui/src/pages/Workspaces.tsx" + ] + }, + { + "id": "formal-approvals", + "title": "Formal governance approvals", + "status": "partial", + "features": [ + "questions-and-approvals.md", + "task-coordination.md", + "agents.md" + ], + "gap": "Formal board decisions and signoff have recipes; every approval payload/type and actor/host error permutation is not enumerated.", + "paths": [ + "ui/src/pages/ApprovalDetail.tsx", + "ui/src/pages/Approvals.tsx" + ] + }, + { + "id": "auth-and-access", + "title": "Login, invitations, and access", + "status": "partial", + "features": [ + "onboarding.md", + "access.md", + "navigation-preferences.md", + "cli-operations.md" + ], + "gap": "Deployment modes and core login/invite/member/token paths are mapped; exhaustive identity-provider, expiry, role, and resource-policy combinations remain.", + "paths": [ + "ui/src/pages/Auth.tsx", + "ui/src/pages/BoardClaim.tsx", + "ui/src/pages/CliAuth.tsx", + "ui/src/pages/CompanyAccess.tsx", + "ui/src/pages/InstanceAccess.tsx", + "ui/src/pages/InviteLanding.tsx", + "ui/src/pages/JoinRequestQueue.tsx", + "ui/src/pages/UserProfile.tsx" + ] + }, + { + "id": "company-and-instance", + "title": "Company and instance administration", + "status": "partial", + "features": [ + "companies.md", + "onboarding.md", + "instance-operations.md", + "execution-environments.md", + "navigation-preferences.md" + ], + "gap": "Core administration/import/export/operator workflows are mapped; cloud/self-hosted differences, migration/restore compatibility, and platform-specific service variants need deeper recipes.", + "paths": [ + "ui/src/pages/Companies.tsx", + "ui/src/pages/CompanyEnvironments.tsx", + "ui/src/pages/CompanyExport.tsx", + "ui/src/pages/CompanyImport.tsx", + "ui/src/pages/CompanySettings.tsx", + "ui/src/pages/InstanceExperimentalSettings.tsx", + "ui/src/pages/InstanceGeneralSettings.tsx", + "ui/src/pages/ProfileSettings.tsx" + ] + }, + { + "id": "goals-costs-and-activity", + "title": "Goals, dashboards, costs, and activity", + "status": "partial", + "features": [ + "goals.md", + "budgets-costs.md", + "activity.md", + "runs-adapters.md", + "routines.md" + ], + "gap": "Core goal/report/budget/audit journeys are mapped. Every aggregation/filter combination, cost source, and overlapping budget window still needs dedicated cases.", + "paths": [ + "ui/src/pages/Costs.production.tsx", + "ui/src/pages/Costs.tsx", + "ui/src/pages/Dashboard.tsx", + "ui/src/pages/DashboardLive.tsx", + "ui/src/pages/GoalDetail.tsx", + "ui/src/pages/Goals.tsx", + "ui/src/pages/Timeline.tsx", + "ui/src/pages/audit/AuditFeed.production.tsx", + "ui/src/pages/audit/AuditFeed.tsx", + "ui/src/pages/audit/AuditHub.tsx", + "ui/src/pages/audit/AuditRuns.tsx", + "ui/src/pages/audit/CompanyActivity.production.tsx", + "ui/src/pages/audit/CompanyActivity.tsx", + "ui/src/pages/audit/RoutineAuditActivity.tsx" + ] + }, + { + "id": "artifacts", + "title": "Artifacts and work products", + "status": "partial", + "features": [ + "documents-artifacts.md" + ], + "gap": "Document/attachment/library/work-product journeys are mapped; every media preview type, storage backend, annotation conflict, and runtime-resource lifecycle variant remains separate.", + "paths": [ + "ui/src/pages/Artifacts.tsx" + ] + }, + { + "id": "routines-and-pipelines", + "title": "Routines, pipelines, cases, and status cards", + "status": "partial", + "features": [ + "routines.md", + "pipelines.md", + "cases.md", + "status-cards.md" + ], + "gap": "All four capabilities have recipes. Trigger/provider/timezone combinations, custom case schemas, stage automation types, and summarizer quality need deeper cases.", + "paths": [ + "ui/src/pages/CaseDetail.tsx", + "ui/src/pages/Cases.tsx", + "ui/src/pages/PipelineSettings.tsx", + "ui/src/pages/Pipelines.tsx", + "ui/src/pages/RoutineDetail.production.tsx", + "ui/src/pages/RoutineDetail.tsx", + "ui/src/pages/Routines.production.tsx", + "ui/src/pages/Routines.tsx", + "ui/src/pages/StatusCards/ArchivedStatusCardRow.tsx", + "ui/src/pages/StatusCards/CreateStatusCardDialog.tsx", + "ui/src/pages/StatusCards/StatusCardDetailDrawer.tsx", + "ui/src/pages/StatusCards/StatusCardSettingsForm.tsx", + "ui/src/pages/StatusCards/StatusCardTile.tsx", + "ui/src/pages/StatusCards/SummarizerAgentSelect.tsx", + "ui/src/pages/StatusCards/index.tsx" + ] + }, + { + "id": "secrets", + "title": "Secrets and secret proposals", + "status": "partial", + "features": [ + "secrets.md", + "access.md", + "app-permissions.md" + ], + "gap": "Company/user/proposal/vault and runtime-binding paths are mapped; every secret provider, rotation scenario, and delivery context needs separate validation.", + "paths": [ + "ui/src/pages/Secrets.tsx", + "ui/src/pages/secrets/ImportFromVaultDialog.tsx", + "ui/src/pages/secrets/MissingUserSecretsBanner.tsx", + "ui/src/pages/secrets/MyUserSecretsTab.tsx", + "ui/src/pages/secrets/ProposalsTab.tsx", + "ui/src/pages/secrets/SecretPathName.tsx", + "ui/src/pages/secrets/SetMyUserSecretDialog.tsx", + "ui/src/pages/secrets/UserSecretDefinitionsTab.tsx", + "ui/src/pages/secrets/proposal-review.tsx", + "ui/src/pages/secrets/user-secret-presentation.tsx" + ] + }, + { + "id": "plugins-and-tools", + "title": "Plugins, gateways, and tool profiles", + "status": "partial", + "features": [ + "plugins.md", + "gateways-profiles.md", + "app-permissions.md", + "execution-environments.md" + ], + "gap": "Host plugin/gateway/profile lifecycles are mapped. Third-party plugin behaviors and every client transport/provider combination are outside host-wide coverage.", + "paths": [ + "ui/src/pages/CompanySettingsPluginPage.tsx", + "ui/src/pages/PluginManager.tsx", + "ui/src/pages/PluginPage.tsx", + "ui/src/pages/PluginSettings.tsx", + "ui/src/pages/apps/gateways/ConnectClientDialog.tsx", + "ui/src/pages/apps/gateways/CopyableGatewayUrl.tsx", + "ui/src/pages/apps/gateways/EditGatewayDialog.tsx", + "ui/src/pages/apps/gateways/GatewayDetail.tsx", + "ui/src/pages/apps/gateways/GatewaysList.tsx", + "ui/src/pages/apps/gateways/NewGatewayDialog.tsx", + "ui/src/pages/apps/gateways/panels/AppsToolsPanel.tsx", + "ui/src/pages/apps/gateways/panels/GatewayActivityPanel.tsx", + "ui/src/pages/apps/gateways/panels/GatewayAdvancedPanel.tsx", + "ui/src/pages/apps/gateways/panels/OverviewPanel.tsx", + "ui/src/pages/apps/gateways/panels/TokensPanel.tsx", + "ui/src/pages/tools/AdvancedToolsRoute.tsx", + "ui/src/pages/tools/AuditTab.tsx", + "ui/src/pages/tools/GatewaysTab.tsx", + "ui/src/pages/tools/McpConfigHelpDialog.tsx", + "ui/src/pages/tools/PasteConfigTab.tsx", + "ui/src/pages/tools/ProfilesTab.tsx", + "ui/src/pages/tools/SmokeLabTab.tsx", + "ui/src/pages/tools/ToolsAccess.tsx", + "ui/src/pages/tools/connection-dialogs.tsx", + "ui/src/pages/tools/profiles/ProfileActionDialog.tsx", + "ui/src/pages/tools/profiles/ProfileDetail.tsx", + "ui/src/pages/tools/profiles/ProfileDetailRoute.tsx", + "ui/src/pages/tools/profiles/ProfileWizard.tsx", + "ui/src/pages/tools/profiles/ProfileWizardRoute.tsx", + "ui/src/pages/tools/profiles/ProfilesIndex.tsx", + "ui/src/pages/tools/profiles/ToolsAdminGate.tsx", + "ui/src/pages/tools/profiles/WizardToolsStep.tsx", + "ui/src/pages/tools/shared.tsx" + ] + }, + { + "id": "labs-and-fallbacks", + "title": "Developer labs and fallback pages", + "status": "partial", + "features": [ + "developer-labs.md", + "navigation-preferences.md" + ], + "gap": "Developer and fallback surfaces are identified. Each synthetic fixture and performance budget is not a production journey and needs its own measurement.", + "paths": [ + "ui/src/pages/BootstrapSetupUxLab.tsx", + "ui/src/pages/CrossIssueCollaborationUxLab.tsx", + "ui/src/pages/DesignGuide.tsx", + "ui/src/pages/InviteUxLab.tsx", + "ui/src/pages/IssueChatLongThreadPerf.tsx", + "ui/src/pages/IssueChatUxLab.tsx", + "ui/src/pages/NotFound.tsx", + "ui/src/pages/ResponsibleUserDenialUxLab.tsx", + "ui/src/pages/RunTranscriptUxLab.tsx", + "ui/src/pages/SystemNoticeUxLab.tsx", + "ui/src/pages/TaskChatLab.tsx" + ] + } + ] +} diff --git a/feature-map/developer-labs.md b/feature-map/developer-labs.md new file mode 100644 index 0000000000..dc9a7fbec0 --- /dev/null +++ b/feature-map/developer-labs.md @@ -0,0 +1,43 @@ +# Developer previews and diagnostic labs + +Contributors can inspect design examples, interaction labs, and performance fixtures. These are development surfaces and do not count as production feature verification. + +Implementation: [route registration](../ui/src/App.tsx), [design guide](../ui/src/pages/DesignGuide.tsx), [long-thread fixture](../ui/src/pages/IssueChatLongThreadPerf.tsx). + +## Sub-features + +- `design`: inspect shared components and layout examples in the design guide or Storybook. +- `interaction-labs`: exercise synthetic bootstrap, task-chat, denial, and collaboration states. +- `performance`: measure the provided long-thread fixture with known data and environment. +- `route-gates`: distinguish dev-only, experimental, historical, and fallback surfaces. + +## How to get to it (user POV) + +### `preview-lab` + +Use `/design-guide`, registered `/ux-lab/...` routes, or Storybook; consult `ui/src/App.tsx` for the current gated set. + +### `performance-fixture` + +Open `/tests/perf/long-thread` only in an appropriate development environment. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a development/test environment. Confirm the route is registered for the current build before claiming the page is available. + +### `preview-lab` + +Automated: [onboarding variant](../ui/src/components/OnboardingWizardVariant.test.tsx) is an example component contract. There is no claim of full automated acceptance coverage for every lab page. + +Manual: Open the specific fixture state, inspect controls and layout, then repeat the affected interaction in the actual product host. Report fixture results separately from persisted task/company results. + +### `performance-fixture` + +Automated: [task thread component](../ui/src/components/IssueChatThread.test.tsx) does not establish a performance budget. Timing/scroll measurements need a dedicated browser run. + +Manual: Record browser/build/data size, exercise scrolling and composing in the fixture, and retain timing or trace evidence. Confirm a representative real task does not regress before extrapolating fixture measurements. + +## Gotchas + +- An unmounted or obsolete lab file remains in the conservative page inventory but is not a shipped entry point. +- Screenshots of a lab do not prove live provider behavior, persistence, or authorization. diff --git a/feature-map/documents-artifacts.md b/feature-map/documents-artifacts.md new file mode 100644 index 0000000000..39d96ade8b --- /dev/null +++ b/feature-map/documents-artifacts.md @@ -0,0 +1,55 @@ +# Documents, attachments, and work products + +People read and revise task documents, annotate specific content, open attached files, and inspect the outputs an agent hands back. The artifact library provides another route to the same work. + +Implementation: [task documents](../ui/src/components/IssueDocumentsSection.tsx), [artifact library](../ui/src/pages/Artifacts.tsx), [document panel](../ui/src/components/task-side-panel/TaskDocumentPanel.tsx). + +## Sub-features + +- `documents`: edit versioned task documents and inspect or restore revisions. +- `annotations`: attach feedback to document content and inspect resolution state. +- `attachments`: upload, preview, and download allowed files with company access checks. +- `work-products`: link deliverables to their source task, resource, and review context. +- `library`: search, filter, group, and open artifact stacks. + +## How to get to it (user POV) + +### `task-document` + +Open a document from the task thread or document side panel. + +### `task-output` + +Open an attachment, output card, or file link from a task. + +### `artifact-library` + +Open `/artifacts` and its grouping, search, and media controls. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a task with a text document, a small attachment, and a generated work product. Use nonsensitive fixture files. + +### `task-document` + +Automated: [documents service](../server/src/__tests__/documents-service.test.ts), [annotations](../server/src/__tests__/document-annotations-service.test.ts), and [revision restore](../server/src/__tests__/issue-document-restore-routes.test.ts) cover server contracts. + +Manual: Edit a paragraph, save, reload, and compare revisions. Add an annotation to a specific selection and inspect it after another edit. Resolve it through the supported action. Restore a disposable revision and confirm the new current document and history agree. + +### `task-output` + +Automated: [attachment routes](../server/src/__tests__/issue-attachment-routes.test.ts) and [work products](../server/src/__tests__/work-products.test.ts) cover data/access behavior; [artifact arrival browser suite](../tests/e2e/artifact-tab-arrival.spec.ts) targets the arrival experience. + +Manual: Upload a fixture and have the test task produce an output. Preview and download each, compare the contents, and follow the source task link. Reload and test an inaccessible user/company; verify missing workspace-only resources have an explicit state. + +### `artifact-library` + +Automated: [artifact page](../ui/src/pages/Artifacts.test.tsx) covers grouping, search requests, and pagination using mocks. + +Manual: Find the task’s output through search and grouping, open its stack, page through results, and reload a direct stack link. Confirm the same file/content appears as from the task. + +## Gotchas + +- A local path is not automatically an uploaded artifact accessible to another user. +- A preview screenshot is not proof that download bytes or revision references are correct. +- Version conflicts and stale annotations must stay visible rather than silently targeting different content. diff --git a/feature-map/execution-environments.md b/feature-map/execution-environments.md new file mode 100644 index 0000000000..feb89f353d --- /dev/null +++ b/feature-map/execution-environments.md @@ -0,0 +1,45 @@ +# Local, SSH, and sandbox environments + +Operators configure where agents execute, probe readiness, select an environment for work, and inspect remote provisioning or custom-image state when the installed provider supports it. + +Implementation: [environment settings](../ui/src/pages/CompanyEnvironments.tsx), [environment runtime](../server/src/services/environment-runtime.ts). + +## Sub-features + +- `configuration`: inspect local environments and configure available SSH/sandbox targets. +- `capabilities`: select only drivers/providers that advertise the required execution capability. +- `probe`: test reachability and runtime prerequisites in the actual target. +- `assignment`: bind the intended agent/project/workspace to its execution target. +- `images`: inspect supported sandbox custom-image preparation and terminal workflows. + +## How to get to it (user POV) + +### `environment-settings` + +Open Settings → Environments, then create/edit a supported target. + +### `remote-execution` + +Choose the environment in agent/project/workspace configuration, then inspect the resulting task/run and workspace. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an owned disposable target and a supported provider plugin. Record provider, target identity, account, and spending constraints before a live remote run. + +### `environment-settings` + +Automated: [environment UI](../ui/src/pages/CompanyEnvironments.test.tsx), [driver configuration UI](../ui/src/pages/CompanySettings.test.tsx), and [environment service](../server/src/__tests__/environment-service.test.ts) cover configuration behavior. + +Manual: Save a disposable target, reload it, and run its available probe. Verify both success and missing-credential/reachability errors identify the target. Confirm unavailable providers are not offered as working run targets. + +### `remote-execution` + +Automated: [execution target](../server/src/__tests__/environment-execution-target.test.ts) and [custom images](../server/src/__tests__/environment-custom-images-service.test.ts) cover contracts; [SSH live test](../server/src/__tests__/environment-live-ssh.test.ts) requires its own external setup. + +Manual: Run a bounded command in the intended target, inspect its output and workspace, and verify execution did not silently fall back to the server host. For custom-image changes, build/probe the disposable image before assigning it. Inspect cleanup and shutdown after the run. + +## Gotchas + +- A probe is narrower evidence than a complete task with tools, files, and resumption. +- The local option may appear only when editing an existing local environment; availability follows current UI/provider capabilities. +- Workspace service reachability and environment reachability are separate checks. diff --git a/feature-map/gateways-profiles.md b/feature-map/gateways-profiles.md new file mode 100644 index 0000000000..616c73d8c3 --- /dev/null +++ b/feature-map/gateways-profiles.md @@ -0,0 +1,45 @@ +# Tool gateways and access profiles + +Administrators expose selected tools through gateways and reusable access profiles, connect clients, manage tokens, and inspect gateway activity. + +Implementation: [gateways](../ui/src/pages/apps/gateways/GatewaysList.tsx), [gateway detail](../ui/src/pages/apps/gateways/GatewayDetail.tsx), [profiles](../ui/src/pages/tools/profiles/ProfileWizard.tsx). + +## Sub-features + +- `gateway`: create/edit an endpoint and select its exposed apps/tools. +- `profiles`: define reusable tool access configuration and inspect draft/detail state. +- `client-connect`: copy the intended endpoint/configuration to an authorized test client. +- `tokens`: create/revoke client credentials and verify their scope. +- `activity`: inspect tool discovery/calls and explicit errors. + +## How to get to it (user POV) + +### `gateway-client` + +Open `/apps/gateways`, then a gateway’s Overview, Apps/Tools, Tokens, and Activity tabs. + +### `access-profile` + +Open `/apps/advanced/profiles`, create a profile, or follow its detail/edit route. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a fixture tool connection and test client. Record gateway/profile, token owner, transport, and selected company. + +### `gateway-client` + +Automated: [gateway service](../server/src/__tests__/tool-gateway-service.test.ts), [connect client dialog](../ui/src/pages/apps/gateways/ConnectClientDialog.test.tsx), and [token panel](../ui/src/pages/apps/gateways/panels/TokensPanel.test.tsx) cover service/UI contracts. + +Manual: Create a disposable gateway with one harmless tool, connect a client using its provided configuration, discover/call that tool, and inspect activity. Revoke the client token and confirm the same call fails. Test a tool outside the allowed set. + +### `access-profile` + +Automated: [profile wizard](../ui/src/pages/tools/profiles/ProfileWizard.test.tsx) and [profile detail](../ui/src/pages/tools/profiles/ProfileDetail.test.tsx) cover component behavior with mocks. + +Manual: Create a bounded profile, choose its offered tools and options, save or resume its draft, and reload the detail. Apply/connect it through the offered flow and inspect the actual client-visible tool set. Verify cancellation and invalid selections retain an understandable draft. + +## Gotchas + +- A copied endpoint is not proof of client authentication or tool discovery. +- Gateway access cannot replace underlying connection/action permission checks. +- Advanced tool routes require the appropriate admin gate; legacy tool URLs may redirect. diff --git a/feature-map/goals.md b/feature-map/goals.md new file mode 100644 index 0000000000..a282d35345 --- /dev/null +++ b/feature-map/goals.md @@ -0,0 +1,42 @@ +# Goals and work alignment + +People define company and lower-level goals and connect projects and tasks to the outcome they serve. + +Implementation: [goals](../ui/src/pages/Goals.tsx), [goal detail](../ui/src/pages/GoalDetail.tsx). + +## Sub-features + +- `hierarchy`: create and inspect goal level, parent, owner, and status. +- `alignment`: associate work with a valid goal in the same company. +- `navigation`: follow goal/project/task context to understand why work exists. + +## How to get to it (user POV) + +### `goal-detail` + +Open `/goals`, create a goal, and enter `/goals/:goalId`. + +### `task-goal-context` + +Inspect the goal context from a project or task and compare explicit versus inherited context. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare two goals and a disposable project/task in the selected company. + +### `goal-detail` + +Automated: [project-goal validation](../server/src/__tests__/project-goal-validation.test.ts) covers relationship validation; [goal detail control](../ui/src/pages/GoalDetail.test.tsx) only tests the properties toggle, not the whole creation journey. + +Manual: Create a parent and child goal, save owner/status changes, and reload. Link a project and task through the offered controls, follow their goal context, and verify a cross-company or invalid relationship is rejected. + +### `task-goal-context` + +Automated: [task goal routes](../server/src/__tests__/issues-goal-context-routes.test.ts) and [goal fallback](../server/src/__tests__/issue-goal-fallback.test.ts) cover the server rules. + +Manual: Compare a task with an explicit goal to one relying on project context. Change the allowed association, reload, and verify the displayed/injected context follows the documented server rule rather than retaining an unrelated goal. + +## Gotchas + +- A goal status is not an independently measured proof of business success. +- Do not infer a complete goal browser test from the small properties-toggle suite. diff --git a/feature-map/inbox-search.md b/feature-map/inbox-search.md new file mode 100644 index 0000000000..2ef31ac7c7 --- /dev/null +++ b/feature-map/inbox-search.md @@ -0,0 +1,54 @@ +# Inbox, decisions, and search + +People find work that needs their attention, distinguish unread activity from required decisions, archive handled items, and search across the company. + +Implementation: [inbox](../ui/src/pages/Inbox.tsx), [decisions](../ui/src/pages/WhatNeedsMe.tsx), [search](../ui/src/pages/Search.tsx). + +## Sub-features + +- `views`: use Mine, Recent, Unread, Blocked, and broader task views. +- `triage`: read or archive attention items without pretending the underlying task is complete. +- `decisions`: open the source of a decision and act in the right context. +- `search`: query company work and follow a result to its original record. + +## How to get to it (user POV) + +### `inbox` + +Use Inbox links or the corresponding task views when the combined list is enabled. + +### `decisions` + +Open `/decisions`, then a queue at `/decisions/queues/:key` and its source item. + +### `search` + +Open `/search` or the navigation search entry. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Seed assigned, unread, blocked, and resolved tasks plus a pending decision. Record the combined Inbox/Tasks experimental setting. + +### `inbox` + +Automated: [archive routes](../server/src/__tests__/inbox-archive-routes.test.ts) cover server mutations. [Task-list helpers](../ui/src/pages/Issues.test.tsx) cover search URLs, pagination, and presentation constants; they do not render the combined Inbox/Tasks page. Reading, archiving, and navigating that page remain manual checks. + +Manual: Read an unread task, return to Unread, archive a handled item, and reload. Verify personal triage state changes while task status stays correct. Switch users and companies to check isolation. + +### `decisions` + +Automated: [decision queues](../server/src/__tests__/decision-queues-routes.test.ts) covers route behavior. End-to-end queue-to-source navigation is manual. + +Manual: Open a pending decision, follow its source, resolve it through the supported control, and reload the queue. Confirm it retires only after the decision persisted and unrelated pending items remain. + +### `search` + +Automated: [search page](../ui/src/pages/Search.test.tsx) and [company search service](../server/src/__tests__/company-search-service.test.ts) cover UI and server search separately. + +Manual: Search a unique task title and content phrase, open the result, and compare the actual record. Test filters, no results, and a second company. Verify stale results do not expose inaccessible data. + +## Gotchas + +- Personal archival is not task cancellation or completion. +- Legacy Inbox routes can redirect to the combined task view; record which host actually rendered. +- Search service tests do not establish the relevance quality of every query. diff --git a/feature-map/instance-operations.md b/feature-map/instance-operations.md new file mode 100644 index 0000000000..dbed223088 --- /dev/null +++ b/feature-map/instance-operations.md @@ -0,0 +1,45 @@ +# Installation, configuration, updates, and backups + +An operator can install and run Paperclip, diagnose the instance, configure supported deployment settings, manage its service, and produce a recoverable database backup. + +Implementation: [CLI entry point](../cli/src/index.ts), [instance settings](../ui/src/pages/InstanceExperimentalSettings.tsx), [backup command](../cli/src/commands/db-backup.ts). + +## Sub-features + +- `install-update`: inspect install channel/version and apply supported update or rollback operations. +- `server-service`: start/stop the intended instance and inspect its service health. +- `configuration`: configure database, server binding, storage, secrets, and available experimental settings. +- `diagnostics`: use doctor/health output to distinguish readiness from a listening port. +- `backup`: create and inspect backups and verify restore only into a separate disposable target. + +## How to get to it (user POV) + +### `operator-cli` + +Use `paperclipai install`, `run`, `service`, `doctor`, `configure`, `channels`, or `update --help` for the intended operation. + +### `settings-backup` + +Use currently visible instance settings for experiments/access and `paperclipai db:backup --help` for one-off backup. Some legacy general-settings URLs redirect. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable data directory/service identity. Do not update, restart, or restore over an unrelated live instance as a documentation test. + +### `operator-cli` + +Automated: [doctor](../cli/src/__tests__/doctor.test.ts), [update command](../cli/src/__tests__/update-command.test.ts), and [service manager](../cli/src/__tests__/service-manager.test.ts) cover command contracts. + +Manual: Inspect the selected installation/channel and configuration, start the isolated instance, confirm health and browser access, then stop that instance. For update changes, use the dry-run/check path first and verify installed version and rollback state on an owned disposable installation. + +### `settings-backup` + +Automated: [experimental settings](../ui/src/pages/InstanceExperimentalSettings.test.tsx) and [database backup routes](../server/src/__tests__/instance-database-backups-routes.test.ts) cover their UI/API contracts. + +Manual: Change a harmless experimental setting, reload, and confirm the affected route honors it. Back up a disposable instance, inspect the reported file and metadata, and restore using the documented procedure into a second isolated target. Verify a known record after restoration. + +## Gotchas + +- A successful backup write does not prove restoration works. +- Settings visibility and mutability differ for cloud-managed and self-hosted instances. +- Keep the three data paths distinct: first-party Telemetry, operator-configured OpenTelemetry, and local run logs. diff --git a/feature-map/navigation-preferences.md b/feature-map/navigation-preferences.md new file mode 100644 index 0000000000..66794028f8 --- /dev/null +++ b/feature-map/navigation-preferences.md @@ -0,0 +1,45 @@ +# Navigation, profile, and announcements + +People organize their sidebar, move between company resources, maintain their profile, and dismiss product announcements with the correct persistence and identity scope. + +Implementation: [sidebar](../ui/src/components/Sidebar.tsx), [profile](../ui/src/pages/ProfileSettings.tsx), [announcements](../ui/src/components/AnnouncementWell.tsx). + +## Sub-features + +- `navigation`: use current sidebar links, favorites, recents, and available layout controls. +- `preferences`: persist supported ordering/collapse choices for the intended user/company. +- `profile`: edit personal identity and inspect public profile links. +- `announcements`: inspect and dismiss an announcement with its intended instance-wide user scope. +- `fallbacks`: handle missing resources and old deep links without presenting another resource as the target. + +## How to get to it (user POV) + +### `sidebar-deep-links` + +Use starred/recent/sidebar links and direct company-prefixed URLs. + +### `profile-announcements` + +Use Settings → Profile, `/u/:userSlug`, and the announcement well when a publication is available. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use two test users and companies; record the selected UI shell and any client-local preference storage. + +### `sidebar-deep-links` + +Automated: [sidebar](../ui/src/components/Sidebar.test.tsx) and [sidebar preference routes](../server/src/__tests__/sidebar-preferences-routes.test.ts) cover UI/server behavior. + +Manual: Star and reorder supported resources, collapse a section, navigate and reload. Switch company/user and inspect the intended preference scope. Open an unavailable resource and a legacy route; verify clear fallback or correct redirect without leaking another company’s state. + +### `profile-announcements` + +Automated: [profile page](../ui/src/pages/ProfileSettings.test.tsx), [announcement hook](../ui/src/hooks/useAnnouncement.test.tsx), and [announcement routes](../server/src/__tests__/announcements-routes.test.ts) cover separate UI/persistence layers. + +Manual: Change a disposable user’s display details, reload, and inspect an attributed profile link. Dismiss a fixture announcement, switch companies, and verify it remains dismissed for that user; check another user still has their own dismissal state. + +## Gotchas + +- Announcement dismissals are intentionally instance-wide user preferences, unlike company-scoped domain data. +- Legacy/production shells need their own navigation check when affected. +- A route redirect is part of the journey; do not document a removed screen as if it still renders. diff --git a/feature-map/onboarding.md b/feature-map/onboarding.md new file mode 100644 index 0000000000..4a47e0a9fd --- /dev/null +++ b/feature-map/onboarding.md @@ -0,0 +1,45 @@ +# Onboarding and first work + +An operator can start an instance, create a company, configure its first agent, and reach a real first task. Instance setup and company onboarding are separate steps. + +Implementation: [web wizard](../ui/src/components/OnboardingWizard.tsx), [CLI setup](../cli/src/commands/onboard.ts). + +## Sub-features + +- `instance`: choose local instance configuration and start the server. +- `company`: create a company with its identity and initial agent. +- `adapter`: choose an available harness, model, account, and execution environment. +- `first-task`: leave setup with a task whose execution or missing prerequisite is visible. +- `resume`: return to an incomplete wizard without losing saved state. + +## How to get to it (user POV) + +### `web-wizard` + +Open `/onboarding` or the onboarding launcher in an empty company context. + +### `cli-setup` + +Use `paperclipai onboard --help` or `paperclipai run --help` to select an isolated data directory before setup. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a new isolated data directory and record deployment mode. A configured adapter may still need account authentication. + +### `web-wizard` + +Automated: [wizard tests](../ui/src/components/OnboardingWizard.test.tsx) cover mocked wizard behavior; [onboarding browser suite](../tests/e2e/onboarding.spec.ts) supplies its own test environment. Live model execution is a separate check. + +Manual: Create a disposable company and first agent. Save and leave midway, reopen, and finish. Follow the created first task, verify the selected agent and company, and inspect its output or explicit authentication gate. Reload both company and task. + +### `cli-setup` + +Automated: [CLI onboarding](../cli/src/__tests__/onboard.test.ts) and [isolated test drive](../cli/src/__tests__/test-drive.test.ts) cover their setup contracts; they do not establish that every model account is usable. + +Manual: Run setup in the disposable directory, open the URL it prints, and confirm the instance identity and company. Restart using that same configuration and verify saved state. Exercise an unavailable adapter and confirm an actionable error. + +## Gotchas + +- An empty company, an empty instance, and an unauthenticated browser are different states. +- Creating an agent does not prove its provider login or first execution succeeded. +- See [access](./access.md) for authenticated deployments and [agents](./agents.md) for later hiring. diff --git a/feature-map/pipelines.md b/feature-map/pipelines.md new file mode 100644 index 0000000000..ac0bd5d48b --- /dev/null +++ b/feature-map/pipelines.md @@ -0,0 +1,44 @@ +# Pipelines, review queues, and learnings + +With the pipeline experiment enabled, people move structured work through configured stages, inspect stage automation, review outputs, and retain learning records. + +Implementation: [pipeline board and item hosts](../ui/src/pages/Pipelines.tsx), [pipeline configuration](../ui/src/pages/PipelineSettings.tsx). + +## Sub-features + +- `stages`: configure stage order, behavior, and automation assignees. +- `items`: create or inspect items, grouping, stage state, and work references. +- `review`: act on the pipeline review queue and inspect the resulting transition. +- `learnings`: inspect the separate learning view and its source context. + +## How to get to it (user POV) + +### `pipeline-board` + +Open `/pipelines`, `/pipelines/:pipelineId`, and its Settings route. + +### `item-review-learning` + +Open `/pipelines/:pipelineId/items/:caseId`, `/review-queue`, or `/learnings`. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable pipelines and prepare a disposable pipeline with at least two stages and a fixture agent. Record any associated case type. + +### `pipeline-board` + +Automated: [pipeline UI helpers](../ui/src/pages/Pipelines.test.tsx) cover grouping and presentation helpers; [pipeline service](../server/src/__tests__/pipelines-service.test.ts) covers service behavior. + +Manual: Configure stages and a harmless automation, add an item, and inspect its stage. Follow its task/work references and observe the resulting transition. Reload grouping and settings; verify rejected or failed transitions leave the item in an intelligible state. + +### `item-review-learning` + +Automated: [pipeline tutorial browser suite](../tests/e2e/pipelines-tutorial-flow.spec.ts) targets a fixture journey; it does not qualify arbitrary stage automation or learning quality. + +Manual: Open a pending review, inspect its source output and revision, accept or reject, and verify the persisted stage outcome. Follow a learning record to its source context and confirm it belongs to the intended pipeline/company. + +## Gotchas + +- These routes are experimental and may redirect or deny access when disabled. +- ReviewQueue and Learnings are hosted from the pipeline module; a page-file inventory alone misses them. +- Pipeline items and free-standing [cases](./cases.md) share records but have different navigation contexts. diff --git a/feature-map/plugins.md b/feature-map/plugins.md new file mode 100644 index 0000000000..b97f79758e --- /dev/null +++ b/feature-map/plugins.md @@ -0,0 +1,44 @@ +# Plugins and contributed product surfaces + +Operators install and configure plugins, inspect their status and errors, and use their contributed pages, tools, and managed resources within the plugin’s granted capabilities. + +Implementation: [plugin manager](../ui/src/pages/PluginManager.tsx), [plugin settings](../ui/src/pages/PluginSettings.tsx), [plugin page](../ui/src/pages/PluginPage.tsx). + +## Sub-features + +- `installation`: install a trusted test package or local development plugin through supported controls. +- `configuration`: save validated configuration and inspect activation/failure state. +- `contributions`: open plugin routes, settings, tools, and contributed resource tabs. +- `lifecycle`: disable/remove or reload a disposable plugin and observe cleanup. + +## How to get to it (user POV) + +### `plugin-administration` + +Open Settings → Plugins where permitted, or the CLI plugin commands. + +### `plugin-contribution` + +Open `/plugins/:pluginId`, a registered plugin route, or its contributed company/project settings surface. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a known local fixture plugin with bounded capabilities. Record installed version and any declared external dependencies. + +### `plugin-administration` + +Automated: [install authorization](../server/src/__tests__/plugin-install-route-security.test.ts) and [plugin settings](../ui/src/pages/PluginSettings.test.tsx) cover route/UI contracts. + +Manual: Install the fixture, supply its configuration, activate it, and inspect status. Save an invalid configuration and verify a useful error. Disable/remove the disposable plugin and check its tools and managed resources no longer behave as active. + +### `plugin-contribution` + +Automated: [static plugin UI](../server/src/__tests__/plugin-ui-static.test.ts) and [company settings contribution](../ui/src/pages/CompanySettingsPluginPage.test.tsx) cover serving/host behavior. + +Manual: Open the contributed page by navigation and cold deep link, perform a harmless plugin action, and inspect its real outcome. Switch company and test an actor without the capability. Reload with the plugin unavailable and verify the host reports that state. + +## Gotchas + +- Installing a plugin does not authorize every host capability or external action. +- Plugin-provided routes are dynamic; this map covers the host lifecycle, not every third-party feature. +- Environment-driver plugins also need the [environment](./execution-environments.md) checks. diff --git a/feature-map/projects.md b/feature-map/projects.md new file mode 100644 index 0000000000..f2c7f45591 --- /dev/null +++ b/feature-map/projects.md @@ -0,0 +1,44 @@ +# Projects and repositories + +People group tasks into projects, attach goals and repositories, and configure the context and defaults used by project work. + +Implementation: [project list](../ui/src/pages/Projects.tsx), [project detail](../ui/src/pages/ProjectDetail.tsx). + +## Sub-features + +- `lifecycle`: create, name, organize, and archive projects. +- `task-context`: inspect project tasks and create work in the selected project. +- `repositories`: configure repository/workspace sources and verify usable checkouts. +- `defaults`: maintain project instructions, environment, goal, and budget context where offered. + +## How to get to it (user POV) + +### `project-list` + +Open `/projects`, create a project, and enter its overview or Issues view. + +### `repository-settings` + +Open `/projects/:projectId/configuration` and its repository/workspace settings. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable project and a small repository you can access; record checkout and credential ownership. + +### `project-list` + +Automated: [projects UI](../ui/src/pages/Projects.test.tsx) and [project detail](../ui/src/pages/ProjectDetail.test.tsx) are component tests. + +Manual: Create a project, add a task from its view, and reload. Verify the task belongs to that project, list counts are plausible, and links preserve company scope. Archive only the disposable project and inspect list/task behavior. + +### `repository-settings` + +Automated: [repository persistence](../server/src/__tests__/project-repositories-persistence.test.ts) and [repository browser suite](../tests/e2e/project-repositories.spec.ts) cover configured repository workflows. + +Manual: Attach a small repository, save, and create a project task. Inspect the actual checkout/revision used by its run. Test an invalid repository and missing credentials without treating a saved URL as a working workspace. + +## Gotchas + +- Project repository configuration and a provisioned execution workspace are different records. +- See [workspaces](./workspaces.md), [goals](./goals.md), and [budgets](./budgets-costs.md) for those features. +- Do not assume a deleted or archived project should erase its historical task evidence. diff --git a/feature-map/questions-and-approvals.md b/feature-map/questions-and-approvals.md new file mode 100644 index 0000000000..0446cd11c3 --- /dev/null +++ b/feature-map/questions-and-approvals.md @@ -0,0 +1,182 @@ +# Questions and approvals + +People answer an agent's questions and decide requests without losing the task's +history. A question, plan confirmation, governance approval, and app-tool review +can look similar but have different authority and continuation rules. Each +decision must stay attached to its original request and survive reload. + +## Sub-features + +- `questions`: single-choice, multiple-choice, custom, and free-text answers; + paged forms retain answers until submission and show validation failures. +- `dismiss-and-reopen`: unanswered questions remain in the feed after dismissal + or a newer message; reopening restores the form without inventing an answer. +- `confirmations`: accept, decline, or request changes; stale, expired, resolved, + and withdrawn requests cannot be acted on again. +- `plan-review`: inspect the requested plan revision before accepting or asking + for changes. An ordinary planning task can proceed to execution; Agent Chat + hands work to linked tasks and remains a conversation. +- `audience`: the current actor, company, named audience, and review policy govern + resolution. A visible request is not necessarily actionable by that viewer. +- `governance`: pending/history views, approve/reject, revision request, and + resubmission for formal board approvals. +- `tool-review`: approve once, always allow the displayed scope, or decline; + decision status and provider execution outcome remain distinct. +- `continuation`: one recorded response leads to the eligible waiting run or + continuation; repeated clicks, reloads, and competing tabs do not repeat work. + +## How to get to it (user POV) + +### `task-thread` + +Open a task from Tasks, search, a project, or its direct `/issues/:issueId` link. +Answer the question near the composer, or reopen its compact unanswered entry +in the history. Open the plan preview before using the confirmation controls. +Exercise both available task presentations when a change affects shared cards. + +### `agent-chat` + +With Agent Chat enabled, choose Chat and an agent (`/chats/:agentRef`). Answer, +dismiss, or reopen requests in this persistent conversation. A plan handoff here +creates linked work rather than turning the conversation into implementation. + +### `attention` + +Open Decisions (`/decisions`, including a saved queue) or an Inbox attention row. +Expand an actionable interaction inline; follow its task link to inspect context. +With combined Inbox/Tasks enabled, open the corresponding Tasks view instead. + +### `board-approval` + +Open Approvals → Pending or All (`/approvals/pending`, `/approvals/all`). Use the +card action or open `/approvals/:approvalId` for details, comments, revision +request, and resubmission. These are formal approvals, not ordinary questions. + +### `app-review` + +In a task, open **Review request** for an Ask first app call. The same request +appears in Apps → Review (`/apps/review`) and the connection's Review tab. +Use **Approve & run**, **Always allow**, or **Decline** on the surface under test. + +### `skill-test` + +Open Skill Studio, run a skill test that produces an interaction, and inspect +its output card. Skill-test authority is narrower than an ordinary company run. + +### `external-channel` + +In a configured chat channel, answer a published question or confirmation as a +linked sender. Open the associated Paperclip task to inspect the saved outcome. +Channel-specific forms and identity linking require separate provider evidence. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Use separate disposable requests for accept, decline, and revision paths. Record +the actor and request revision. Prepare a waiting question through a test agent; +a component fixture alone cannot prove live response delivery. + +### `task-thread` + +Automated: [interaction cards](../ui/src/components/IssueThreadInteractionCard.test.tsx), +[paged task cards](../ui/src/components/task-chat/TaskChatInteractionCard.test.tsx), +and [task thread](../ui/src/components/TaskChatThread.test.tsx) cover form, +receipt, and dismissal behavior. [Resolution routes](../server/src/__tests__/issue-thread-interaction-routes.test.ts) +and [response delivery](../server/src/__tests__/question-response-delivery.test.ts) +cover authority and durable continuation. Run a targeted set with: + +```sh +pnpm exec vitest run ui/src/components/TaskChatThread.test.tsx server/src/__tests__/question-response-delivery.test.ts +``` + +Manual: answer a multi-page question, go back, edit, then submit. Reload and +confirm the saved answers and one agent continuation. On a fresh request, +dismiss, reload, reopen, and verify the draft. Send a new comment while a +question remains unanswered; it must stay reopenable in history. Try a denied +actor and a request already resolved in another tab; expect an explicit error +or settled receipt, never a second decision. For plans, change the plan revision +before trying the old approval, then review the current revision. + +### `agent-chat` + +Automated: the [task thread suite](../ui/src/components/TaskChatThread.test.tsx) +parameterizes unanswered-question history for conversation mode on and off. +This proves component parity, not a model's plan handoff. + +Manual: repeat dismissal, reload, reopening, and answer submission in Chat. +Approve a plan and inspect the linked execution task and copied plan. The chat +history remains and the conversation does not close as a completed task. + +### `attention` + +Automated: [attention resolver](../ui/src/components/AttentionInteractionResolver.test.tsx) +covers preparing-request refresh without accepting it. The resolution-route +tests above cover the API, not every Decisions or Inbox navigation path. + +Manual: open the same pending request in an attention row and task tab. Resolve +from the row; both surfaces must settle after refresh. Check an empty queue, +loading, permission denial, and a stale request. Repeat with combined Inbox/Tasks +on when that surface is part of the change. Full navigation parity remains a +manual coverage gap. + +### `board-approval` + +Automated: [approval service](../server/src/__tests__/approvals-service.test.ts) +and [idempotency routes](../server/src/__tests__/approval-routes-idempotency.test.ts) +cover persistence and duplicate decisions. They do not drive the approval pages. + +Manual: create disposable formal approval requests; approve one from the list, +reject another from details, and request revision on a third. Resubmit that +third request and verify Pending/All and detail history after reload. Confirm +the recorded decision and its governed effect, not just a success toast. + +### `app-review` + +Automated: [connection reviews](../tests/e2e/connection-reviews.spec.ts) drives +task/queue synchronization, approve, decline, remembered permission, provider +failure, and restart with a scripted process agent and local MCP provider: + +```sh +pnpm exec playwright test -c tests/e2e/connection-reviews.config.ts +``` + +Manual: dismiss and reopen a review, decide it from each entry point using fresh +requests, and inspect the continuation's useful result. Always allow must show +its scope; a future action with changed arguments should obey the saved rule. +Provider failure must remain an approved decision with a failed execution. +Real provider and model checks follow [Task reviews](../doc/connections/TASK-REVIEWS.md#verification-workflows). + +### `skill-test` + +Automated: [read-only interaction contract](../server/src/__tests__/issue-interactions-read-only-contract.test.ts) +checks that fetching interactions does not mutate them. Shared card suites cover +rendering. Neither proves the complete Skill Studio host or its permission matrix. + +Manual: run a skill test with a question, answer through its visible output, +and inspect the test result. Attempt a governed action outside the test's scope; +expect denial. The complete Skill Studio browser journey is not covered by the +referenced unit tests. + +### `external-channel` + +Automated: [chat question forms](../server/src/services/chat-question-forms.test.ts), +[Discord forms](../server/src/services/chat-discord-question-forms.test.ts), +and [interaction publications](../server/src/__tests__/chat-interaction-publications.test.ts) +cover serialization and delivery logic with fixtures. + +Manual: use a disposable provider conversation, answer as the linked sender, +and check the task's persisted result and one continuation. Repeat from an +unlinked or wrong sender and with an expired card. A browser answer does not +qualify Slack, Discord, Telegram, email, or iMessage; record each as not run +unless its actual provider journey was driven. + +## Gotchas + +- Dismissing a question does not answer it. Dismissing an app review neither + grants permission nor resumes the agent. Ordinary comments cannot approve it. +- A resolved request and a successful tool execution are separate facts. +- Do not reuse an accepted request to test rejection; use another request. +- Agent Chat and skill tests have distinct lifecycle/authority rules despite + sharing UI. A watchdog cannot acquire board authority by resolving a card. +- An answer to an old question can queue behind a successor run; it does not + automatically steer that run. See [steering](./steering.md). diff --git a/feature-map/recovery.md b/feature-map/recovery.md new file mode 100644 index 0000000000..3e8a856467 --- /dev/null +++ b/feature-map/recovery.md @@ -0,0 +1,159 @@ +# Recovery + +People can see why a task stopped, who owns the next action, and whether a safe +continuation is available. Recovery retains task context and ownership, honors +pause and budget gates, and distinguishes confirmed failure from an uncertain +external outcome. A retry acknowledgement alone is not proof that work resumed. + +## Sub-features + +- `diagnosis`: a task/run exposes the recorded cause, evidence, owner, attempt + progress, and next action without requiring a raw-log interpretation. +- `bounded-continuation`: supported interrupted work continues with preserved + conversation context and bounded attempts, without blindly replaying tools. +- `source-action`: source-scoped recovery actions remain attached to the task; + independent repair work may have its own issue and return owner. +- `exhaustion`: exhausted or overdue recovery exposes the current gate and + permitted operator action; it does not claim an automatic path is still live. +- `workspace`: workspace divergence/export failures show the applicable repair, + reconcile, or isolated reissue action with evidence and authority checks. +- `unknown-outcome`: uncertain external side effects require inspection and + an explicit decision, not automatic replay. +- `user-continuation`: saved user messages after a stop are reconsidered once + cleanup proves the old execution is stopped; duplicate wakes are avoided. +- `reconnect`: browser stream recovery refreshes persisted state without claiming + the agent itself restarted or that a failed run succeeded. + +## How to get to it (user POV) + +### `source-task` + +Open the affected task from Tasks, Inbox/Blocked, search, or a direct task link. +Inspect the recovery notice/card in its history, expand details if needed, and +use the currently offered action. Follow any linked repair task and return owner. + +### `run-detail` + +Open the agent's run (`/agents/:agentId/runs/:runId`) from activity or the task's +run link. The run page hosts workspace recovery controls only for a failed run +with error code `workspace_validation_failed` and a linked source task that still +has a live `workspace_validation` recovery action. Use the source task for other +stopped-run recovery checks. + +### `operator-retry` + +On an exhausted disposition or workspace recovery card, inspect the current +reason and use the permitted retry/repair/reconcile action. Some repairs require +confirmation; unavailable actions must explain the gate or remain absent. + +### `post-stop-message` + +After a stop, send a new instruction on the same task or inspect one already +saved while cleanup was pending. Observe the waiting message, continuation, and +eventual result rather than manually changing the task status to look complete. + +### `browser-reconnect` + +While viewing an active or recovering task, interrupt the browser connection +and restore it, or suspend and return to the tab. Reload is a separate check. +The current task and run state should reconcile with the server. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Use a disposable instance and reproducible failure. Record source task/run, +adapter/runtime mode, cause, attempt count, current owner, and verified stop +evidence. Do not kill an unrelated server or induce provider side effects just +to get a failure screen. Use the existing deterministic recovery suite first. + +### `source-task` + +Automated: [recovery actions](../server/src/__tests__/issue-recovery-actions.test.ts) +covers source scoping, bounded attempts, operator stops, quota monitoring, +ownership, stale action retirement, and continuation deduplication. +[Recovery cards](../ui/src/components/IssueRecoveryActionCard.test.tsx) cover +diagnosis, retry progress, exhaustion, and action availability. The fixture-backed +[execution-recovery journeys](../tests/e2e/execution-recovery/recovery.spec.ts) +start their own test-drive instances and exercise safe, uncertain, restart, +manager-lineage, and legacy-unknown paths: + +```sh +pnpm exec vitest run server/src/__tests__/issue-recovery-actions.test.ts ui/src/components/IssueRecoveryActionCard.test.tsx +pnpm exec playwright test -c tests/e2e/execution-recovery/playwright.config.ts recovery.spec.ts +``` + +Manual: open a stopped task through each affected navigation path. Confirm its +explanation, original owner, recovery owner if different, and next action. Follow +the supported recovery until the source task produces a useful result or a +specific escalation. Reload and verify that resolved notices retire and attempt +history remains. A fixture pass does not qualify every live harness/provider. + +### `run-detail` + +Automated: [run workspace recovery](../ui/src/components/RunWorkspaceRecoverySurface.test.tsx) +tests this host; it is component coverage, not an end-to-end repair. + +Manual: prepare a failed run with error code `workspace_validation_failed`, a +linked source task, and a live `workspace_validation` recovery action on that +task. Open that exact run and compare its recovery diagnosis to the source task. +Use an eligible action from the run surface, return to the task, +and verify the same action state, persisted outcome, and restored execution. +Repeat with an actor who lacks authority. Verify that the run-page controls are +absent for other error codes or after the source action retires; continue other +recovery checks on the source task. Full run-detail navigation remains a manual +gap. + +### `operator-retry` + +Automated: [disposition notices](../ui/src/components/DispositionRecoveryNotice.test.tsx) +tests pending/error/gated states and waiting for the real retry acknowledgement. +The recovery-card suite covers workspace divergence, reissue, reconcile, and +confirmation. [Workspace export](../ui/src/components/WorkspaceExportRecovery.test.tsx) +covers its separate repair surface. + +Manual: reproduce an exhausted disposable action. Click once, verify pending +state prevents repeated clicks, and inspect a successful continuation or an +explicit rejection. Reload to confirm the durable outcome. For a workspace +repair, record the compared branches/revisions and preserve uncommitted work; +confirm the repair's actual result before resuming. Check the denied and stale +action paths. A changed card label is insufficient proof of workspace integrity. + +### `post-stop-message` + +Automated: [legacy continuation](../server/src/services/recovery/legacy-continuation.test.ts), +[queue routes](../server/src/__tests__/issue-queued-comments-routes.test.ts), and +[native restart recovery](../server/src/services/native-runtime/native-restart-recovery.test.ts) +cover their respective recovery paths. [Legacy failure continuation](../tests/e2e/legacy-failure-continuation.spec.ts) +adds a browser journey; it is not native runtime proof. + +Manual: stop a disposable run, submit a distinguishable follow-up, and observe +any cleanup wait. After confirmed stop, verify the follow-up reaches one eligible +continuation and produces the requested answer. Repeat with a task/ancestor +pause and a budget hold: recovery must not bypass the gate. For unknown external +outcomes, inspect the provider before authorizing another operation. + +### `browser-reconnect` + +Automated: [live-update recovery](../ui/src/context/LiveUpdatesProvider.recovery.test.tsx) +covers client transport recovery with mocks. It does not prove runner restart. + +Manual: go offline during a disposable run, let the server reach a new state, +then reconnect. Confirm task/run/attention data refresh without duplicate +comments or decisions; reload and compare. A healthy websocket proves transport +only. Separately inspect the underlying execution if it is still stuck. + +## Gotchas + +- Legacy and native runtimes have different ownership and restart paths. Record + which one was exercised instead of inferring coverage from the adapter name. +- A process disappearance, elapsed timeout, or successful API retry does not + prove an external tool operation never happened. Unknown outcomes stay unknown + until corroborated; automatic replay can duplicate a side effect. +- Active-run watchdogs, task watchdogs, reviewers, and recovery owners have + different authority. No recovery card grants board permission by itself. +- Idle Agent Chat is not stranded work. An intentional operator stop must not + silently trigger recovery that fights the user's instruction. +- Source inspection and mocked component tests do not qualify real server + restart continuity. Use the appropriate runtime suite for that claim. +- See [execution semantics](../doc/execution-semantics.md) for the governing + recovery contract and [steering](./steering.md) for saved follow-ups. diff --git a/feature-map/routines.md b/feature-map/routines.md new file mode 100644 index 0000000000..c951e17091 --- /dev/null +++ b/feature-map/routines.md @@ -0,0 +1,45 @@ +# Routines, schedules, and triggers + +Operators define repeatable work, configure its inputs and triggers, run it on demand or automatically, and inspect the execution history and resulting tasks. + +Implementation: [routine list](../ui/src/pages/Routines.tsx), [routine detail](../ui/src/pages/RoutineDetail.tsx), [trigger editor](../ui/src/components/routine-triggers/TriggerWizard.tsx). + +## Sub-features + +- `definition`: save the request, agent, project, variables, and execution context. +- `triggers`: configure supported schedules/webhooks or other offered triggers. +- `run-now`: supply inputs and create an observable routine execution. +- `history`: inspect resulting tasks and past outcomes. +- `management`: organize, pause/disable, archive, and distinguish plugin-managed routines. + +## How to get to it (user POV) + +### `routine-editor` + +Open `/routines`, create one, and inspect `/routines/:routineId` and its edit sections. + +### `trigger-history` + +Use the routine’s trigger controls and History/Audit links; lists also offer Run now. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable routine, deterministic test agent, and harmless target. Record schedule timezone and enabled state. + +### `routine-editor` + +Automated: [routine detail helper](../ui/src/pages/RoutineDetail.test.tsx) tests project-selector options, not the rendered editor. [Routine service](../server/src/__tests__/routines-service.test.ts) covers server contracts. Saving and running through the editor remain manual checks. + +Manual: Save an agent, request, project, and a required variable. Reload and inspect the saved definition. Run now with a distinguishable input, follow the resulting task, and verify its output and history entry. Check a missing required input. + +### `trigger-history` + +Automated: [trigger wizard](../ui/src/components/routine-triggers/TriggerWizard.test.tsx) is component coverage; [routine integration](../server/src/__tests__/routines-e2e.test.ts) is server integration rather than a provider-wide browser proof. + +Manual: Set a near-term disposable schedule or authorized test webhook, verify one eligible execution and its source trigger, then disable it and confirm further invocations stop. Inspect a failed execution without losing its history. Remove the temporary trigger after testing. + +## Gotchas + +- Saving a schedule does not prove the scheduler invoked it. +- Timezone, overlap/concurrency rules, and plugin ownership affect behavior. +- A routine run’s task must still respect pause, access, and budget gates. diff --git a/feature-map/runs-adapters.md b/feature-map/runs-adapters.md new file mode 100644 index 0000000000..6b1f1a4f1e --- /dev/null +++ b/feature-map/runs-adapters.md @@ -0,0 +1,44 @@ +# Runs, harnesses, and model accounts + +Operators choose an agent harness and model account, inspect execution history and transcripts, and distinguish a working configuration from a successful task result. + +Implementation: [adapter manager](../ui/src/pages/AdapterManager.tsx), [run host](../ui/src/pages/AgentDetail.tsx), [configuration](../ui/src/components/AgentConfigForm.tsx). + +## Sub-features + +- `availability`: discover installed adapters and available models/accounts. +- `validation`: test an adapter in the selected execution environment and surface authentication failures. +- `runs`: inspect run status, events, output, costs, and the source task. +- `session`: preserve or reset execution context through supported lifecycle actions. + +## How to get to it (user POV) + +### `adapter-setup` + +Open an agent’s model/adapter configuration or the instance adapter settings page when permitted. + +### `run-history` + +Open an agent run at `/agents/:agentId/runs/:runId` from the agent or task activity. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Record adapter, model, account, native/legacy runtime, and environment. Capabilities vary; one harness result is not a matrix-wide qualification. + +### `adapter-setup` + +Automated: [adapter validation routes](../server/src/__tests__/agent-adapter-validation-routes.test.ts) and [model refresh](../server/src/__tests__/adapter-model-refresh-routes.test.ts) cover server behavior. + +Manual: Select the intended adapter and account, refresh model choices, and run its environment test. Save, reload, then execute a small task. Repeat with a deliberately unavailable account and verify the failure remains visible. + +### `run-history` + +Automated: [live run routes](../server/src/__tests__/agent-live-run-routes.test.ts) and [run event sequencing](../server/src/__tests__/heartbeat-run-event-sequencing.test.ts) cover run data contracts. + +Manual: Start a disposable task, open its running transcript, and follow it to a terminal outcome. Reload and compare task/run IDs, status, output, and cost. Verify a failed run remains distinguishable from a completed run with useful output. + +## Gotchas + +- The run log is instance database data; it is not the opt-out first-party Telemetry system or operator-configured OpenTelemetry tracing. +- Authentication/model discovery does not establish tool, image, steering, or resumption capability parity. +- Use [recovery](./recovery.md) for interrupted runs and [execution environments](./execution-environments.md) for local/SSH/sandbox differences. diff --git a/feature-map/secrets.md b/feature-map/secrets.md new file mode 100644 index 0000000000..71dcfb7c26 --- /dev/null +++ b/feature-map/secrets.md @@ -0,0 +1,45 @@ +# Secrets, personal credentials, and proposals + +Operators manage company secret references, users supply their own credentials where required, and reviewers decide proposed secret changes without exposing secret values in ordinary UI evidence. + +Implementation: [secrets page](../ui/src/pages/Secrets.tsx), [personal secrets](../ui/src/pages/secrets/MyUserSecretsTab.tsx), [proposals](../ui/src/pages/secrets/ProposalsTab.tsx). + +## Sub-features + +- `company-secrets`: create/update secret references and control their bindings. +- `user-secrets`: define required user credentials and set a value for the current user. +- `proposals`: review proposed changes with the appropriate authority. +- `vault-import`: import eligible external-vault references through the offered flow. +- `runtime-delivery`: deliver only the secret references authorized for the run/actor. + +## How to get to it (user POV) + +### `secret-management` + +Open company Settings → Secrets and the company/personal secret views. + +### `proposal-import` + +Use the Proposals tab or Import from vault where available. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use dummy credentials in an isolated company, two user identities, and a test agent. Never print actual secret values in evidence. + +### `secret-management` + +Automated: [secrets service](../server/src/__tests__/secrets-service.test.ts) and [user ownership](../server/src/__tests__/secrets-service-user-secret-owner-scoped.test.ts) cover storage/access contracts. + +Manual: Create a dummy secret reference, bind it through a supported field, and verify its use in a harmless run without logging the value. Update it and reload. Switch users and confirm personal values remain owner-scoped and missing credentials produce an actionable state. + +### `proposal-import` + +Automated: [proposal routes](../server/src/__tests__/secret-proposals-routes.test.ts) and [vault import dialog](../ui/src/pages/secrets/ImportFromVaultDialog.test.tsx) cover route/UI layers; a real vault integration needs live validation. + +Manual: Review a dummy proposal, reject one and accept another, then inspect the resulting reference state and history. Preview a vault import, select only disposable entries, and verify imported references and denied-access errors after reload. + +## Gotchas + +- A displayed reference name is not proof that the actor can resolve its value at runtime. +- Company secrets and user-specific values have different ownership. +- Redaction needs verification in errors, activity, and run logs as well as the form. diff --git a/feature-map/skills.md b/feature-map/skills.md new file mode 100644 index 0000000000..c8565f2b2a --- /dev/null +++ b/feature-map/skills.md @@ -0,0 +1,55 @@ +# Skill discovery, authoring, and policy + +Operators discover or import skills, author and test them in Skill Studio, manage source versions, and decide which agents may use them. + +Implementation: [skill library](../ui/src/pages/CompanySkills.tsx), [studio](../ui/src/pages/SkillStudio.tsx), [sources](../ui/src/pages/SkillSources.tsx), [agent skills](../ui/src/pages/agent-skills/AgentSkillsTab.tsx). + +## Sub-features + +- `discovery`: browse installed/discover views and import a skill or source. +- `authoring`: create/edit a writable skill or fork a read-only source. +- `versions`: inspect revisions, source updates, diffs, and selected releases. +- `policy`: apply agent enablement and skill usage policy independently from installation. +- `testing`: run a bounded Studio test and inspect its result and interactions. + +## How to get to it (user POV) + +### `library-sources` + +Open `/skills`, Discover, or `/skills/sources`; project import is available through the library’s import flow. + +### `studio` + +Use New skill or `/skills/studio/:skillId`; writable copies and source-backed originals differ. + +### `agent-enablement` + +Use skill policy/agent selection in the library or an agent’s Skills tab. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a harmless skill fixture and test agent with skill support. Record whether the source is local, bundled, project, or remote. + +### `library-sources` + +Automated: [library UI](../ui/src/pages/CompanySkills.test.tsx) and [source service](../server/src/__tests__/skill-sources.test.ts) cover their respective layers. + +Manual: Install a fixture, locate it in Installed, inspect its source and files, and reload. Import a source update and compare versions before accepting it. Check a source requiring unavailable repository access and verify the failure is explicit. + +### `studio` + +Automated: [Studio](../ui/src/pages/SkillStudio.test.tsx) covers UI behavior; model-driven skill testing remains a separate manual result. + +Manual: Create or fork a disposable skill, edit instructions, save, and reload. Run a bounded test, respond to any question, and inspect the output. Verify the saved version matches what was tested and failed saves retain the draft. + +### `agent-enablement` + +Automated: [policy service](../server/src/__tests__/company-skill-policy-service.test.ts) and [agent skills state](../ui/src/pages/agent-skills/AgentSkillsTab.test.ts) cover policy/state contracts. + +Manual: Enable the skill for one test agent and leave another without access. Run a small task on each, inspect the available skill context, then change the release/policy and verify the next eligible run uses the intended version. + +## Gotchas + +- Installed does not mean enabled for every agent. +- Read-only GitHub/bundled skills may require a writable copy rather than an in-place edit. +- Studio interaction coverage is in [questions and approvals](./questions-and-approvals.md); a passing mock is not a live harness test. diff --git a/feature-map/status-cards.md b/feature-map/status-cards.md new file mode 100644 index 0000000000..ec0b2b240c --- /dev/null +++ b/feature-map/status-cards.md @@ -0,0 +1,44 @@ +# Status cards and refreshed summaries + +The status-card experiment turns a saved prompt/query into an inspectable summary of watched work, with refresh history, configuration, and archive/restore controls. + +Implementation: [status page](../ui/src/pages/StatusCards/index.tsx), [card detail](../ui/src/pages/StatusCards/StatusCardDetailDrawer.tsx). + +## Sub-features + +- `authoring`: create a card and configure its prompt, refresh behavior, and summarizer. +- `generation`: observe compilation/refresh tasks and their successful or failed output. +- `inspection`: read the summary, watched work, history, and available query debugging. +- `archive`: archive and restore without confusing a stale summary with a current result. + +## How to get to it (user POV) + +### `status-board` + +Open `/status`, choose New card, and follow a card at `/status/:cardId`. + +### `card-settings` + +Open the card detail’s settings/query-debug controls and archive/restore actions. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable status cards, use a configured summarizer and disposable tasks, and record whether summary generation is simulated or live. + +### `status-board` + +Automated: [status-card service/routes](../server/src/__tests__/status-cards.test.ts) covers gated lifecycle and generation contracts; [tile UI](../ui/src/pages/StatusCards/StatusCardTile.test.tsx) covers rendering. + +Manual: Create a bounded card, inspect its generation task, and wait for a saved summary. Change a watched task, refresh, and verify both the summary and update history reflect the source change. Exercise a failed generation and confirm it is not presented as fresh. + +### `card-settings` + +Automated: [settings form](../ui/src/pages/StatusCards/StatusCardSettingsForm.test.tsx) and [update engine](../server/src/__tests__/status-card-update-engine.test.ts) cover separate UI/engine layers. + +Manual: Change a disposable card’s prompt or summarizer, run the offered rebuild/refresh, and inspect the resulting query and summary. Archive and restore it, reload its direct link, and check the actual new generation outcome. + +## Gotchas + +- Summary quality needs comparison against source work, not just a successful task status. +- A saved old summary must not be counted as a successful new refresh. +- Experimental visibility, author permissions, and agent ownership constrain mutation. diff --git a/feature-map/steering.md b/feature-map/steering.md new file mode 100644 index 0000000000..02d1cfa91b --- /dev/null +++ b/feature-map/steering.md @@ -0,0 +1,135 @@ +# Steering and queued messages + +People can send follow-ups while an agent works, manage saved messages that have +not been consumed, and choose when to steer or interrupt. Stopping a run and +pausing a task are separate controls. The UI must preserve the message, explain +why it is waiting, and only claim delivery after an authoritative acknowledgement. + +## Sub-features + +- `queue`: a busy task saves follow-ups and shows their order and wait reason. +- `edit-order-discard`: edit full Markdown, reorder, and discard eligible saved + messages; ownership, revision conflicts, and already-dispatched states matter. +- `steer`: explicit user action sends a queued message into a compatible active + run; unavailable capabilities and provider rejection remain visible. +- `interrupt`: stop the current execution and carry accepted follow-ups into + the eligible continuation without delivering them twice. +- `stop-and-pause`: distinguish stop-in-flight, confirmed stop, task/ancestor + pause, agent pause, and budget hold. Resume must respect the actual gate. +- `drafts`: switching tasks, reloading, and uncertain submission retain the + correct task's draft and attachment receipts. +- `approval-queue`: answered questions and accepted approvals have durable + delivery distinct from editable ordinary comments. + +## How to get to it (user POV) + +### `task-composer` + +Open an assigned task (`/issues/:issueId`). Start work, then send follow-ups while +the agent runs. Use the queued-message controls to edit, reorder, discard, +steer, or interrupt when offered. Use the composer stop/pause control separately. +Both task presentations need attention when changing their shared behavior. + +### `agent-chat` + +Enable Agent Chat and open an agent conversation. Send another message during +a reply and inspect the shared queue. `/new` starts fresh provider context in +the same conversation; it is an ordered boundary, not a normal model prompt. + +### `request-response` + +Answer an earlier question or accept a confirmation while successor work is +active on the task. Inspect the resulting queued response and its available +steer/interrupt control instead of assuming acceptance interrupted execution. + +### `external-channel` + +Send a follow-up from a linked external chat conversation while its task is +running. Use that provider's supported stop/control affordance where available; +inspect the resulting task and run in Paperclip. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Use an agent that stays busy long enough to inspect the queue. Record the adapter, +native/legacy mode, and advertised steering capability. Test supported and +unsupported steering without treating an absent capability as a product failure. + +### `task-composer` + +Automated: [queued-message UI](../ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx) +covers order, failure restoration, unavailable steering, stale revisions, and +discard acknowledgements. [Queue routes](../server/src/__tests__/issue-queued-comments-routes.test.ts) +cover durable ordering, authorization, steering acknowledgements, interrupt +recovery, and concurrent dispatch. [Composer](../ui/src/components/task-chat/TaskChatComposer.test.tsx) +covers drafts and uncertain saves. + +```sh +pnpm exec vitest run ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx server/src/__tests__/issue-queued-comments-routes.test.ts +``` + +Automated stop coverage is separate: +[composer stop](../tests/e2e/composer-stop.spec.ts) with +[its configuration](../tests/e2e/playwright-composer-stop.config.ts) and +[ACP stop/continuation](../tests/e2e/acp-stop-continuation.spec.ts). Inspect their +fixture and opt-in prerequisites; skipped native cases are not native proof. + +Manual: queue three distinguishable messages. Edit one, reorder them, reload, +and confirm the saved order. Discard one and confirm it never reaches execution. +Steer one while busy and verify the agent receives that exact message once. +Repeat with a rejected steer and a stale second-tab edit; the UI must preserve +retryable content and explain the conflict. On a fresh run, interrupt with a +queued follow-up and verify confirmed stop followed by one continuation. Pause +the task (and separately its ancestor), preserve a draft, then resume the actual +hold; no hidden send should occur while paused. Check both task presentations +when available; the named component tests do not prove every host integration. + +### `agent-chat` + +Automated: [agent chat sessions](../tests/e2e/agent-chat-sessions.spec.ts) covers +persistent conversation/session behavior with deterministic harness fixtures; +shared composer tests cover the controls. These do not qualify live steering +for every harness. + +Manual: queue two follow-ups while the conversation replies. Reload, edit or +discard one, then let the other run. Verify one reply and unchanged conversation +identity. Send `/new`, then a new prompt; old history remains visible, and the +new turn uses fresh provider context. Verify a useful answer rather than only +the session-divider UI. Repeat the queue operations with a live capable harness +when the change touches delivery. + +### `request-response` + +Automated: [response delivery](../server/src/__tests__/question-response-delivery.test.ts) +proves queueing/coalescing and retry-safe receipts. The queue-route suite tests +separate comment and approval queues, explicit steering, and fresh-session +restrictions on plan approvals. + +Manual: answer an old question during a new run, confirm that it is saved and +queued, and let the active run end. Verify exactly one continuation with the +answer. Repeat an eligible approval with explicit steer; another approval that +requires a fresh session must not offer same-turn delivery. Do not edit an +approval outcome through an ordinary queued-message editor. + +### `external-channel` + +Automated: [chat interaction arbitration](../server/src/services/chat-interaction-arbitration.test.ts) +covers backend arbitration with simulated requests. It is not a provider UI test. + +Manual: send two distinguishable messages through one disposable linked +conversation while busy. Inspect their saved task order, run delivery, and +external reply. Check a duplicate provider event and unauthorized sender. +Provider-specific control semantics and real transport timing remain a manual +gap; do not assume browser queue controls are available in every channel. + +## Gotchas + +- Paperclip's queue is server-owned once saved; it is not Omnigent's unsent + browser draft buffer. A stale edit must not overwrite a consumed message. +- Steer availability depends on the active runtime's capability, not its brand + name. A UI row disappearing optimistically does not prove provider receipt. +- Interrupt, stop, task pause, agent pause, and budget pause have different + effects. A running indicator disappearing does not prove remote work stopped. +- An interrupted run may need verified cleanup before a saved follow-up can + start. Preserve and inspect the wait reason; see [recovery](./recovery.md). +- A healthy idle Agent Chat is waiting for input, not stranded unfinished work. diff --git a/feature-map/task-coordination.md b/feature-map/task-coordination.md new file mode 100644 index 0000000000..9549f7606d --- /dev/null +++ b/feature-map/task-coordination.md @@ -0,0 +1,44 @@ +# Delegation, dependencies, and signoff + +A larger task can coordinate child work, dependencies, reviewers, and execution controls while preserving a clear owner and a traceable completion decision. + +Implementation: [task thread](../ui/src/pages/IssueDetail.tsx), [execution policy](../server/src/services/issues.ts). + +## Sub-features + +- `delegation`: inspect parent/child tasks and the original request behind delegated work. +- `dependencies`: show blocking tasks and wake eligible work when prerequisites resolve. +- `signoff`: apply the configured implementation/review completion policy. +- `tree-control`: pause or resume a task tree without bypassing its gates. + +## How to get to it (user POV) + +### `task-relationships` + +Open the task’s parent/child and dependency links in task detail. + +### `review-and-controls` + +Inspect the task’s execution policy and available review, pause, resume, or stop controls. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare a parent, child, and dependent task plus implementer/reviewer agents. Record execution policy and use a disposable tree. + +### `task-relationships` + +Automated: [dependency wakeups](../server/src/__tests__/issue-dependency-wakeups-routes.test.ts) checks server behavior; visual relationship navigation remains a manual check. + +Manual: Block one task on another, finish the prerequisite, and observe the dependent task’s eligible continuation. Follow parent/child links back to the source request. Verify no duplicate run or false parent completion occurs. + +### `review-and-controls` + +Automated: [execution policy](../server/src/__tests__/issue-execution-policy.test.ts), [tree control](../server/src/__tests__/issue-tree-control-service.test.ts), and [signoff browser suite](../tests/e2e/signoff-policy.spec.ts) target these layers. A linked suite is not a claim that its current run is green. + +Manual: Complete implementation under a review-required policy and confirm it waits for the required review outcome. Exercise rejection and later approval. Pause the parent while child work is pending, then verify resumption respects budget and ownership gates. + +## Gotchas + +- Review completion and task completion are different events under some policies. +- Pausing a tree must not be “tested” by changing statuses until the screen looks quiet. +- Use [questions and approvals](./questions-and-approvals.md) for decision forms and [recovery](./recovery.md) for interrupted work. diff --git a/feature-map/tasks.md b/feature-map/tasks.md new file mode 100644 index 0000000000..a2a7e42bc3 --- /dev/null +++ b/feature-map/tasks.md @@ -0,0 +1,55 @@ +# Task creation and lifecycle + +People create, assign, organize, discuss, and finish tasks. A task retains its owner, project, priority, history, and outputs as it moves through the work lifecycle. + +Implementation: [new task](../ui/src/components/NewIssueDialog.tsx), [task list](../ui/src/pages/Issues.tsx), [task detail](../ui/src/pages/IssueDetail.tsx). + +## Sub-features + +- `create`: capture a title/request with the intended agent, project, and attachments. +- `organize`: filter, sort, group, and navigate tasks without losing the selected context. +- `properties`: change status, priority, assignee, project, and relationships where authorized. +- `discussion`: persist comments, references, and files on the task. +- `completion`: distinguish backlog, active work, review, blocked, done, and cancelled outcomes. + +## How to get to it (user POV) + +### `new-task` + +Use New task from the global launcher, task list, or project task view. + +### `task-list` + +Open `/issues` or a project’s Issues view; legacy `/tasks` links redirect. + +### `task-detail` + +Open `/issues/:issueId` from a row, mention, or direct link. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an invokable test agent and a project; record whether a real or simulated harness handles the request. + +### `new-task` + +Automated: [creation dialog](../ui/src/components/NewIssueDialog.test.tsx) is component coverage with mocked requests. + +Manual: Create a uniquely named task with an agent and project. Open its resulting link and reload. Confirm text, attachment, assignment, and project persisted, and verify execution started only when the task and agent are eligible. + +### `task-list` + +Automated: [task-list helpers](../ui/src/pages/Issues.test.tsx) cover search URLs, pagination, deduplication, and presentation constants. The suite does not render the page or exercise filtering, grouping, opening a task, and returning; those interactions still need a running-browser check. + +Manual: Filter by status and assignee, change grouping/sort, open a task, and return. Verify the correct records and selected view survive navigation and reload. Include an empty result and a failed request. + +### `task-detail` + +Automated: [task detail](../ui/src/pages/IssueDetail.test.tsx) covers component behavior, while [issue service](../server/src/__tests__/issues-service.test.ts) covers server contracts. + +Manual: Add a comment and edit authorized properties. Reload and verify history, actor, and values. Complete a disposable task and inspect its deliverable; reopen through an available action and confirm the next execution follows current policy. + +## Gotchas + +- A status label alone does not prove that the requested work exists. +- Single-assignee and atomic checkout rules still apply when tasks are created from different hosts. +- Use [coordination](./task-coordination.md), [steering](./steering.md), and [documents](./documents-artifacts.md) for their distinct workflows. diff --git a/feature-map/teams.md b/feature-map/teams.md new file mode 100644 index 0000000000..a1370c75eb --- /dev/null +++ b/feature-map/teams.md @@ -0,0 +1,44 @@ +# Team packages and catalog installation + +Operators can preview and install a team package with its agents and related resources. Catalog/package installation is available through the CLI/API; the repository also contains catalog UI components that are not a current top-level route. + +Implementation: [CLI teams](../cli/src/commands/client/teams.ts), [catalog components](../ui/src/pages/TeamCatalog.tsx). + +## Sub-features + +- `discovery`: list available team catalog entries and inspect package contents. +- `preview`: inspect proposed resources and installation options before mutation. +- `install`: install into the intended company and inspect resulting agents and resources. +- `configuration`: supply required adapters/accounts and verify installed agents can work. + +## How to get to it (user POV) + +### `catalog-cli` + +Use the CLI teams catalog/preview/install operations in the selected context. + +### `catalog-components` + +The catalog components and install wizard are available in developer tests/design surfaces. No top-level `/teams` route is registered; use the CLI/API for the current installation workflow. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable company, a small catalog entry, and explicit company context. Check `paperclipai teams --help` for current subcommands. + +### `catalog-cli` + +Automated: [team commands](../cli/src/__tests__/teams.test.ts) and [catalog service](../server/src/__tests__/teams-catalog-service.test.ts) cover command and service contracts. + +Manual: List an entry, preview its proposed agents and resources, install it into the disposable company, and inspect each created resource. Run a small task after configuring accounts. Retry preview/install as supported and inspect duplicate/conflict behavior. + +### `catalog-components` + +Automated: [catalog component](../ui/src/pages/TeamCatalog.test.tsx) and [install hook](../ui/src/pages/useInstallTeamCatalogEntry.test.tsx) cover preview and install UI logic with mocks. + +Manual: When changing or mounting these components, exercise preview, manager/source options, cancellation, failed install, and success. Follow created agents in the actual company. Record whether this was a component harness or a real product entry point. + +## Gotchas + +- Source files do not prove that a screen is reachable in the current app. +- Installed team configuration may still need credentials or environment setup. +- Company package import is a separate [portability](./companies.md) workflow. diff --git a/feature-map/workspaces.md b/feature-map/workspaces.md new file mode 100644 index 0000000000..d7b24fca7c --- /dev/null +++ b/feature-map/workspaces.md @@ -0,0 +1,45 @@ +# Execution workspaces, services, and files + +A project or task can use a managed workspace with inspectable files, Git state, services, and runtime controls. Operators can follow its lifecycle through provisioning, use, and closure. + +Implementation: [workspace list](../ui/src/pages/Workspaces.tsx), [execution workspace](../ui/src/pages/ExecutionWorkspaceDetail.tsx), [project workspace](../ui/src/pages/ProjectWorkspaceDetail.tsx). + +## Sub-features + +- `binding`: identify the workspace actually bound to a task/run. +- `provisioning`: observe readiness and failures instead of inferring readiness from a created record. +- `services`: start/stop supported services and inspect logs and exposed endpoints. +- `files-git`: inspect workspace files and repository state used for the deliverable. +- `closure`: close/reopen or reconcile through permitted controls without losing work. + +## How to get to it (user POV) + +### `task-workspace` + +Use the task’s workspace card or Files panel and follow its workspace link. + +### `workspace-management` + +Open `/workspaces`, a project workspace, or `/execution-workspaces/:workspaceId` and its Services/Configuration/Runtime logs views. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable isolated-workspace UI where needed. Use an owned disposable repository and environment; record workspace ID and branch. + +### `task-workspace` + +Automated: [workspace card](../ui/src/components/IssueWorkspaceCard.test.tsx) and [task binding](../server/src/__tests__/issue-runtime-workspace-binding.test.ts) cover host/binding contracts. + +Manual: Run a task that writes a small file, follow the bound workspace, and inspect the file and revision. Reload and verify links still refer to the same workspace, not another checkout or a stale run. + +### `workspace-management` + +Automated: [workspace detail](../ui/src/pages/ExecutionWorkspaceDetail.test.tsx) and [runtime service](../server/src/__tests__/workspace-runtime.test.ts) cover UI and runtime contracts separately. + +Manual: Provision a disposable workspace, wait for readiness, start an available service, open its endpoint, and inspect a real response and logs. Stop it and verify it actually stops. Exercise close/reopen only after inspecting pending changes and active runs. + +## Gotchas + +- A link or running badge does not prove the service is reachable from the user’s browser. +- Workspace IDs, runtime leases, and repository paths are not interchangeable. +- Use [recovery](./recovery.md) for divergence/export repair and [CLI operations](./cli-operations.md) for local worktree tooling. From 81cd03b3b542ea4af291ef56d5084fbf6410eb9c Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Mon, 5 Oct 2026 21:09:09 -0500 Subject: [PATCH 02/12] test(ui): keep initial reasoning with the saved reply (#15107) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The issue thread shows saved replies and agent run history. > - History that appears after the reply can move the page. > - Master now waits for initial run history before it shows the thread in #15228. > - This pull request checks that reasoning and the saved reply appear together in the first visible frame. > - The test protects that behavior from a later change. ## Linked Issues or Issue Description No public issue tracks this report. This is a bug regression test. **What happened?** The issue thread showed a saved answer before its reasoning and tool history loaded. The later history moved the page. **Expected behavior** The first visible thread contains the initial reasoning and the saved reply in order. **Steps to reproduce** 1. Open an issue with an agent run and a saved reply. 2. Delay initial run-history hydration. 3. Observe the saved reply before its reasoning appears. Related work: #15228 now contains the reveal gate. This PR adds a direct regression assertion for that gate. #14667 takes a different loading approach and remains open. ## What Changed - Add a saved agent reply to the initial-history test. - Assert that the native run's reasoning appears before that reply when the thread becomes visible. - Assert that the thread stays inert until history is ready. ## Verification - `vitest run ui/src/components/TaskChatThread.test.tsx ui/src/pages/IssueDetail.test.tsx` passed: 353 tests. - `git diff --check origin/master...HEAD` passed. - GitHub CI checks are in progress for the rebased head. ## Risks - Low risk. This PR changes only a test. - The production fix is already in #15228. This PR must not restore the older code from the previous branch head. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI assisted with tool use and code execution. The exact model ID and context window were not exposed to this agent session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My remote branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no user-facing documentation changed) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- ui/src/components/TaskChatThread.test.tsx | 29 +++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/ui/src/components/TaskChatThread.test.tsx b/ui/src/components/TaskChatThread.test.tsx index 11ca30d76a..10854e06cc 100644 --- a/ui/src/components/TaskChatThread.test.tsx +++ b/ui/src/components/TaskChatThread.test.tsx @@ -437,7 +437,15 @@ describe.each(["legacy", "native"] as const)("%s task history readiness", (runti async (status) => { const props = { issueId: "issue-1", - comments: createLongThreadComments(), + comments: [...createLongThreadComments(), { + ...createLongThreadComments()[3], + id: "agent-answer", + authorType: "agent" as const, + authorAgentId: "agent-1", + body: "Final saved reply", + runId: "started-run", + createdAt: new Date("2026-08-25T18:00:02.000Z"), + }], onAdd: async () => {}, linkedRuns: [ retryRun, @@ -455,9 +463,17 @@ describe.each(["legacy", "native"] as const)("%s task history readiness", (runti container.querySelector('[data-testid="task-chat-history-loading"]'), ).not.toBeNull(); expect(container.textContent).toContain("Thread message 1"); - + expect(container.querySelector('[data-thread-anchor="comment-1"]')?.closest('[inert]')).not.toBeNull(); transcriptState.hydratedRunIds = new Set(["started-run"]); nativeTranscriptState.hydratedRunIds = new Set(["started-run"]); + transcriptState.transcriptByRun.set("started-run", [{ + kind: "thinking", text: "Reasoning before the reply", + ts: "2026-08-25T18:00:01.000Z", + }]); + nativeTranscriptState.transcriptByRun.set("started-run", [{ + kind: "assistant", text: "Reasoning before the reply", channel: "progress", + ts: "2026-08-25T18:00:01.000Z", + }]); render(); await act(async () => { await new Promise((resolve) => requestAnimationFrame(resolve)); @@ -467,6 +483,15 @@ describe.each(["legacy", "native"] as const)("%s task history readiness", (runti expect( container.querySelector('[data-testid="task-chat-history-loading"]'), ).toBeNull(); + expect(container.querySelector('[data-thread-anchor="comment-1"]')?.closest('[inert]')).toBeNull(); + expect(container.textContent).toContain("Thread message 1"); + if (status === "succeeded" && runtimeMode === "native") { + expect(container.textContent).toContain("Reasoning before the reply"); + expect(container.textContent).toContain("Final saved reply"); + expect(container.textContent!.indexOf("Reasoning before the reply")).toBeLessThan( + container.textContent!.indexOf("Final saved reply"), + ); + } }, ); }); From 858094ba8123c7edb56623597cd96f0391f7e2d4 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Mon, 5 Oct 2026 19:41:52 -0700 Subject: [PATCH 03/12] Fix browser polling for unsaved agent chats (#15298) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat uses the shared task surface and browser panel. > - An unsaved chat has a temporary ID instead of a stored task UUID. > - The browser poll sent that ID to a PostgreSQL UUID query and returned a server error. > - This pull request waits for a saved task and validates task IDs before the query. > - Saved chats keep their browser tools and access checks. ## Linked Issues or Issue Description Refs #14331. That report describes the same draft-ID problem in email requests. This change covers the browser routes; it does not change email behavior. Opening an unsaved Agent Chat sends `chat:` to `/issues/:issueId/browsers` every three seconds. PostgreSQL rejects this as a task UUID. The draft is only a UI view model. The first send or upload creates its stored task. ## What Changed - Disable browser polling for unsaved chat IDs. Resume polling when the chat has a task UUID. - Return the existing task-not-found response for malformed task IDs before database access. Keep board authentication first and company and credential checks unchanged. - Test all six task browser routes, draft-to-saved polling, normal tasks, saved chats, and denied access. Keep canonical UUID v4, v7, and nil values queryable. - Document the draft and saved-chat browser contract. ## Verification - `pnpm install --frozen-lockfile` passed with pnpm 9.15.4 and Node 24.21.0. - `pnpm exec vitest run server/src/__tests__/browser-use-route-scope.test.ts server/src/__tests__/browser-use-connection.test.ts` passed all 35 tests. - `pnpm exec vitest run ui/src/hooks/useTaskBrowsers.test.ts` passed all 5 tests. - Independent review passed 14 hook and route tests plus both real task and saved-chat authorization cases. - `pnpm check:token-gates` and `git diff --check` passed. - Full workspace `pnpm -r typecheck` and `pnpm build` passed. - Full Linux CI passed on `dafff44a32`: 53 successful checks and two intentional Storybook skips. This includes all general and serialized test groups, UI and browser tests, typecheck, build, and the canary dry run. No CI retries were needed. - The full local `TMPDIR=/private/tmp pnpm test:run` started but did not complete. It was stopped after full Linux CI passed. Before the stop, three company-skills cache cases failed on macOS. A focused rerun reproduced all three as `EACCES` while renaming immutable cache staging directories. The test, company-skills service, and runtime cache files are byte-identical to the baseline previously reproduced on `e99854249c` for #15291. Other local groups were not reached; the full Linux CI run provides aggregate coverage. - Greptile scored the exact head `dafff44a32` 5/5 with no findings or unresolved review threads. ## Risks - Malformed task IDs now return 404 instead of reaching PostgreSQL and returning 500. The route does not map draft IDs to agents or tasks. - No schema, browser provider, task creation, or authorization policy changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository editing, code execution, and independent agent review. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run limitations are reported above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip --- doc/DEVELOPING.md | 5 ++ .../__tests__/browser-use-connection.test.ts | 26 ++++++- .../__tests__/browser-use-route-scope.test.ts | 73 +++++++++++++++++++ server/src/routes/browser-use.ts | 10 +++ ui/src/hooks/useTaskBrowsers.test.ts | 40 +++++++++- ui/src/hooks/useTaskBrowsers.ts | 2 +- 6 files changed, 150 insertions(+), 6 deletions(-) create mode 100644 server/src/__tests__/browser-use-route-scope.test.ts diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index fb3f6d9447..47e399e98c 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -1208,6 +1208,11 @@ operator-controlled [runner API tools](runner-api-tools.md) rollout; enabling Agent Chat does not enable that API surface. Failed-turn retries restore the selected run's user comments so the agent can answer the original request. +An unsaved Agent Chat uses an ephemeral `chat:` view-model ID. Task +browser polling starts after the first send or upload creates a persisted task +UUID. Saved chats retain the task browser's company and credential access checks. +Task browser routes reject malformed task IDs with `404` before querying PostgreSQL. + A native continuation that requires reconciliation shows **Recovery needed** with **Inspect run**; inspect the original outcome before resolving its recovery hold. A generic retry cannot resolve this incident. The runner's diff --git a/server/src/__tests__/browser-use-connection.test.ts b/server/src/__tests__/browser-use-connection.test.ts index 49a11010fe..b206fabe93 100644 --- a/server/src/__tests__/browser-use-connection.test.ts +++ b/server/src/__tests__/browser-use-connection.test.ts @@ -983,8 +983,15 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; )[0], ).toMatchObject({ eventCursor: 9, eventsDrained: 1, accountedCents: 15 }); }); - it("protects interactive viewer routes from agent, other-company and read-only access", async () => { + it.each(["task", "saved chat"])("protects %s browser routes from agent, other-company and read-only access", async (kind) => { const f = await fixture(); + if (kind === "saved chat") { + await db.update(issues).set({ + conversationAgentId: f.agent.id, + conversationUserId: actor.actorId, + conversationState: "active", + }).where(eq(issues.id, f.issue.id)); + } await f.service.execute( f.binding, f.grant, @@ -1021,11 +1028,19 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; }); app.use(browserUseRoutes(db, f.service)); app.use(errorHandler); + const listPath = `/issues/${f.issue.id}/browsers`; + const listed = await http(app).get(listPath); + expect(listed.status).toBe(200); + expect(listed.body).toMatchObject([{ id: browser.id, issueId: f.issue.id }]); + const missing = await http(app).get(`/issues/${randomUUID()}/browsers`); + expect(missing.status).toBe(404); + expect(missing.body).toEqual({ error: "Task not found" }); const path = `/issues/${f.issue.id}/browsers/${browser.id}/viewer`; const ok = await http(app).get(path); expect(ok.status).toBe(200); expect(ok.headers["cache-control"]).toBe("no-store"); expect(ok.headers["referrer-policy"]).toBe("no-referrer"); + const controlPath = path.replace("/viewer", "/control"); const presencePath = path.replace("/viewer", "/presence"); expect((await http(app).post(presencePath)).status).toBe(200); const resizePath = path.replace("/viewer", "/viewport"); @@ -1065,7 +1080,10 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; (await http(app).post(resizePath).send({ preset: "custom", width: 1 })) .status, ).toBe(400); + expect((await http(app).post(controlPath).send({ action: "cancel" })).status).toBe(200); mode = "agent"; + expect((await http(app).get(listPath)).status).toBe(403); + expect((await http(app).post(controlPath).send({ action: "keep_open" })).status).toBe(403); expect((await http(app).post(presencePath)).status).toBe(403); expect( (await http(app).post(`${resizePath}/release`).send({ viewerId })) @@ -1076,6 +1094,10 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; ).toBe(403); expect((await http(app).get(path)).status).toBe(403); mode = "other-company"; + const inaccessible = await http(app).get(listPath); + expect(inaccessible.status).toBe(missing.status); + expect(inaccessible.body).toEqual(missing.body); + expect((await http(app).post(controlPath).send({ action: "keep_open" })).status).toBe(404); expect((await http(app).post(presencePath)).status).toBe(404); expect( (await http(app).post(`${resizePath}/release`).send({ viewerId })) @@ -1091,6 +1113,8 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; .set({ membershipRole: "viewer" }) .where(eq(companyMemberships.companyId, f.company.id)); expect((await http(app).get(path)).status).toBe(403); + expect((await http(app).get(listPath)).body).toEqual([]); + expect((await http(app).post(controlPath).send({ action: "keep_open" })).status).toBe(403); expect((await http(app).post(presencePath)).status).toBe(403); expect( (await http(app).post(resizePath).send({ preset: "phone" })).status, diff --git a/server/src/__tests__/browser-use-route-scope.test.ts b/server/src/__tests__/browser-use-route-scope.test.ts new file mode 100644 index 0000000000..113b408184 --- /dev/null +++ b/server/src/__tests__/browser-use-route-scope.test.ts @@ -0,0 +1,73 @@ +import { randomUUID } from "node:crypto"; +import express from "express"; +import http from "supertest"; +import { describe, expect, it, vi } from "vitest"; +import type { Db } from "@paperclipai/db"; +import { errorHandler } from "../middleware/error-handler.js"; +import { browserUseRoutes } from "../routes/browser-use.js"; + +const routes = [ + { method: "get", suffix: "" }, + { method: "get", suffix: "/viewer" }, + { method: "post", suffix: "/presence" }, + { method: "post", suffix: "/viewport" }, + { method: "post", suffix: "/viewport/release" }, + { method: "post", suffix: "/control" }, +] as const; + +describe("browser task route scope", () => { + it.each([ + "12345678-1234-4234-8234-123456789abc", + "12345678-1234-7234-8234-123456789ABC", + "00000000-0000-0000-0000-000000000000", + ])("keeps canonical PostgreSQL task UUIDs queryable: %s", async (id) => { + const companyId = randomUUID(); + const limit = vi.fn().mockResolvedValue([{ companyId }]); + const db = { select: () => ({ from: () => ({ where: () => ({ limit }) }) }) } as unknown as Db; + const list = vi.fn().mockResolvedValue([]); + const service = { list } as unknown as NonNullable[1]>; + const app = express(); + app.use((req, _res, next) => { + req.actor = { type: "board", userId: "browser-reviewer", source: "session", companyIds: [companyId] }; + next(); + }); + app.use(browserUseRoutes(db, service)); + app.use(errorHandler); + expect((await http(app).get(`/issues/${id}/browsers`)).status).toBe(200); + expect(limit).toHaveBeenCalledExactlyOnceWith(1); + expect(list).toHaveBeenCalledExactlyOnceWith(companyId, id, "browser-reviewer"); + }); + + it.each(routes)("rejects invalid task IDs before any query: $method $suffix", async ({ method, suffix }) => { + const select = vi.fn(() => { throw new Error("Invalid scope reached the database"); }); + const db = { select } as unknown as Db; + const service = Object.fromEntries( + ["list", "viewer", "presence", "resize", "releaseViewport", "humanSession", "control"] + .map((name) => [name, vi.fn()]), + ) as unknown as NonNullable[1]>; + let actorType: "board" | "agent" | "none" = "board"; + const app = express(); + app.use((req, _res, next) => { + req.actor = actorType === "board" + ? { type: "board", userId: "browser-reviewer", source: "session", companyIds: [] } + : actorType === "agent" + ? { type: "agent", companyId: randomUUID(), agentId: randomUUID(), source: "agent_key" } + : { type: "none" }; + next(); + }); + app.use(browserUseRoutes(db, service)); + app.use(errorHandler); + const id = randomUUID(); + for (const invalidId of [`chat:${id}`, "not-a-uuid", `${id}'`, ` ${id}`, `${id}\n`]) { + const path = `/issues/${encodeURIComponent(invalidId)}/browsers${suffix ? `/${id}${suffix}` : ""}`; + for (const type of ["board", "agent", "none"] as const) { + actorType = type; + const response = await http(app)[method](path); + expect(response.status).toBe(type === "board" ? 404 : 403); + expect(response.body).toEqual({ error: type === "board" ? "Task not found" : "Board access required" }); + } + } + expect(select).not.toHaveBeenCalled(); + for (const handler of Object.values(service)) expect(handler).not.toHaveBeenCalled(); + }); +}); diff --git a/server/src/routes/browser-use.ts b/server/src/routes/browser-use.ts index 343237998f..0b45801e51 100644 --- a/server/src/routes/browser-use.ts +++ b/server/src/routes/browser-use.ts @@ -1,5 +1,6 @@ import { Router } from "express"; import { eq } from "drizzle-orm"; +import { z } from "zod"; import { issues, type Db } from "@paperclipai/db"; import { browserUseControlSchema, @@ -15,10 +16,19 @@ import { } from "./authz.js"; import { browserUseService } from "../services/browser-use.js"; +const issueIdSchema = z.string().uuid(); + export function browserUseRoutes(db: Db, service = browserUseService(db)) { const router = Router(); router.use("/issues/:issueId/browsers", async (req, res, next) => { assertBoard(req); + if ( + req.params.issueId !== req.params.issueId.trim() || + !issueIdSchema.safeParse(req.params.issueId).success + ) { + res.status(404).json({ error: "Task not found" }); + return; + } const [issue] = await db .select({ companyId: issues.companyId }) .from(issues) diff --git a/ui/src/hooks/useTaskBrowsers.test.ts b/ui/src/hooks/useTaskBrowsers.test.ts index 603c84db1d..23b8e0638d 100644 --- a/ui/src/hooks/useTaskBrowsers.test.ts +++ b/ui/src/hooks/useTaskBrowsers.test.ts @@ -3,18 +3,25 @@ import { act, createElement } from "react"; import { createRoot } from "react-dom/client"; import { afterEach, beforeEach, expect, it, vi } from "vitest"; import type { TaskBrowser } from "@paperclipai/shared"; -import { useBrowserArrivals } from "./useTaskBrowsers"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { browserUseApi } from "@/api/browser-use"; +import { useBrowserArrivals, useTaskBrowsers } from "./useTaskBrowsers"; (globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true; const cleanups = new Set<() => void>(); function cleanup() { for (const unmount of cleanups) unmount(); } -function renderHook(callback: (props: P) => R, options?: { initialProps: P }) { +function renderHook(callback: (props: P) => R, options?: { initialProps: P; client?: QueryClient }) { const host = document.createElement("div"); document.body.append(host); const root = createRoot(host); let props = options?.initialProps as P; const result = {} as { current: R }; function Harness() { result.current = callback(props); return null; } - function rerender(next = props) { props = next; act(() => root.render(createElement(Harness))); } + function rerender(next = props) { + props = next; + act(() => root.render(options?.client + ? createElement(QueryClientProvider, { client: options.client }, createElement(Harness)) + : createElement(Harness))); + } function unmount() { act(() => root.unmount()); host.remove(); cleanups.delete(unmount); } cleanups.add(unmount); rerender(); @@ -26,7 +33,32 @@ const browser = (id: string): TaskBrowser => ({ error: null, createdAt: "2026-09-29T10:00:00Z", }); beforeEach(() => { localStorage.clear(); sessionStorage.clear(); }); -afterEach(() => { cleanup(); vi.restoreAllMocks(); }); +afterEach(() => { cleanup(); vi.restoreAllMocks(); vi.useRealTimers(); }); +it("starts browser polling when a chat draft becomes a saved task and stops on the next draft", async () => { + vi.useFakeTimers(); + const list = vi.spyOn(browserUseApi, "list").mockResolvedValue([]); + const client = new QueryClient(); + const taskId = "12345678-1234-4234-8234-123456789abc"; + const { rerender, unmount } = renderHook(useTaskBrowsers, { + initialProps: undefined as string | undefined, + client, + }); + await act(async () => { await vi.advanceTimersByTimeAsync(6000); }); + expect(list).not.toHaveBeenCalled(); + rerender(`chat:${taskId}`); + await act(async () => { await vi.advanceTimersByTimeAsync(6000); }); + expect(list).not.toHaveBeenCalled(); + rerender(taskId); + await act(async () => { await vi.advanceTimersByTimeAsync(1); }); + expect(list).toHaveBeenCalledExactlyOnceWith(taskId); + await act(async () => { await vi.advanceTimersByTimeAsync(3000); }); + expect(list).toHaveBeenCalledTimes(2); + rerender(`chat:${taskId}`); + await act(async () => { await vi.advanceTimersByTimeAsync(6000); }); + expect(list).toHaveBeenCalledTimes(2); + unmount(); + client.clear(); +}); it("queues simultaneous arrivals and claims only after the panel acknowledges", () => { const browsers = [browser("first"), browser("next")]; const { result, rerender } = renderHook(() => useBrowserArrivals("user", "task", browsers)); diff --git a/ui/src/hooks/useTaskBrowsers.ts b/ui/src/hooks/useTaskBrowsers.ts index 0d59497756..be8a7d27ac 100644 --- a/ui/src/hooks/useTaskBrowsers.ts +++ b/ui/src/hooks/useTaskBrowsers.ts @@ -6,7 +6,7 @@ export function useTaskBrowsers(issueId?: string) { return useQuery({ queryKey: ["task-browsers", issueId], queryFn: () => browserUseApi.list(issueId!), - enabled: Boolean(issueId), + enabled: Boolean(issueId) && !issueId?.startsWith("chat:"), refetchInterval: 3000, retry: false, }); From d21fe312432d5f47ed5d1a953a8d2d75f08b2a2c Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Mon, 5 Oct 2026 20:09:18 -0700 Subject: [PATCH 04/12] Retry pooled execution projection reads after disconnects (#15300) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task run lists show current execution and recovery state. > - Their execution projection reads runs, finalizations, pending interactions, and recovery records. > - A temporary database disconnect in one lookup makes the run list fail. > - This pull request uses the existing bounded retry policy for these read-only lookups. > - It keeps transaction reads, authorization, and liveness writes outside the retry scope. ## Linked Issues or Issue Description Refs #15248. This extends the same bounded SELECT retry policy to the execution projection used by task run lists. A wrapped `CONNECTION_CLOSED` error from the pending-interaction SELECT makes `GET /issues/:id/runs` fail. These lookups are safe to repeat on the pooled database connection. The shared projection helper also serves transaction callers, so retries must remain an explicit opt-in. ## What Changed - Opt the activity run list into per-query projection retries. Keep the default path single-attempt for transaction callers. - Rebuild only the failed SELECT. Use the existing three-attempt limit and 50/100 ms delays. - Keep completed reads, route authorization, and liveness backfill writes outside the retry callbacks. - Test each query, exhaustion, exact error preservation, non-transient errors, empty results, transaction defaults, run-list backfill behavior, and denied route access. - Document the pooled-read boundary. ## Verification - Frozen install passed with Node 24.21.0 and pnpm 9.15.4. - 92 focused tests passed across execution projection, activity service/routes, prior service-read retries, and the shared retry policy. - Independent review passed 56 projection, retry-policy, and route authorization tests with no blocking findings. - `git diff --check` and the staged diff and PR-text secret/PII scan passed. - Full workspace `pnpm -r typecheck` and `pnpm build` passed. - Greptile scored exact head `78783e84a5` 5/5 with no findings or open review threads. - Initial CI had one Runner test timeout: `stops and closes a timed-out consumer even when the caller requested a warm session` exceeded 5 seconds. Its fixture uses a 1 ms session timeout. The Runner package is unchanged from the base. The focused case passed locally (1 passed, 138 skipped). One unchanged-head rerun of that job passed all 1,219 Runner tests and the additional native-runtime integration test. All other CI lanes passed on the initial run. - Full Linux CI passed on exact head `78783e84a5`: 53 successful checks, two intentional Storybook skips, and no pending checks. It provides aggregate test coverage. A full local aggregate was not launched. The same host's prior aggregate exposed three known macOS immutable-cache rename failures in unchanged company-skills code, documented in #15298. ## Risks - An affected lookup can take up to two additional connection attempts plus 150 ms of delay. Persistent failures still return the final error. - The opt-in is only used by the pooled activity read path. No transaction, mutation, full request, or provider action is replayed. - No schema or response-shape changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository editing, code execution, and independent agent review. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; aggregate coverage is described above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip --- doc/DATABASE.md | 6 + server/src/__tests__/activity-routes.test.ts | 20 +++ server/src/services/activity.ts | 2 +- .../execution-projection-read-retry.test.ts | 150 ++++++++++++++++++ server/src/services/execution-projection.ts | 23 ++- 5 files changed, 192 insertions(+), 9 deletions(-) create mode 100644 server/src/services/execution-projection-read-retry.test.ts diff --git a/doc/DATABASE.md b/doc/DATABASE.md index a0f30babbb..2b4480f9d6 100644 --- a/doc/DATABASE.md +++ b/doc/DATABASE.md @@ -186,6 +186,12 @@ workflow, or issue label and watchdog enrichment. Missing resources, authentication errors, and other database errors retain their usual behavior. This does not enable general SQL replay or retry a full request. +The task run list also opts its execution-status projection into these bounded +retries. Each of its four read-only lookups rebuilds only the failed query; +completed lookups and the separate liveness backfill are not replayed. Projection +callers that use a transaction retain single-attempt reads. The opt-in is only +for a pooled database handle outside a transaction. + ## Execution identity row locks Identity initialization, credential acquisition, and steering reconciliation lock diff --git a/server/src/__tests__/activity-routes.test.ts b/server/src/__tests__/activity-routes.test.ts index e6d9d61113..45871e3236 100644 --- a/server/src/__tests__/activity-routes.test.ts +++ b/server/src/__tests__/activity-routes.test.ts @@ -277,6 +277,26 @@ describe.sequential("activity routes", () => { expect(mockActivityService.create).not.toHaveBeenCalled(); }); + it.each([null, { id: "issue-uuid-1", companyId: "company-2" }])( + "does not start run reads or retries for a missing or cross-company task", async (issue) => { + mockIssueService.getByIdentifier.mockResolvedValue(issue); + const app = await createApp(); + const res = await request(app).get("/api/issues/TEST-1/runs"); + expect(res.status).toBe(404); + expect(res.body).toEqual({ error: "Issue not found" }); + expect(mockActivityService.runsForIssue).not.toHaveBeenCalled(); + }, + ); + + it("does not start run reads or retries when task read authorization is denied", async () => { + mockIssueService.getByIdentifier.mockResolvedValue({ id: "issue-uuid-1", companyId: "company-1" }); + mockAccessService.decide.mockResolvedValue({ allowed: false, reason: "denied", explanation: "Denied by test policy." }); + const app = await createApp(); + const res = await request(app).get("/api/issues/TEST-1/runs"); + expect(res.status).toBe(403); + expect(mockActivityService.runsForIssue).not.toHaveBeenCalled(); + }); + it("returns 200 [] (not 404) when listing issues for another company's run, preserving API contract and the cross-tenant oracle", async () => { mockHeartbeatService.getRun.mockResolvedValue({ id: "run-2", diff --git a/server/src/services/activity.ts b/server/src/services/activity.ts index 5d2b3ebdba..8a2310cbad 100644 --- a/server/src/services/activity.ts +++ b/server/src/services/activity.ts @@ -497,7 +497,7 @@ export function activityService(db: Db) { const [exhaustionRows, leaseRows, executionByRunId, [savedPlan]] = await Promise.all([ exhaustionRowsQuery, leaseRowsQuery, - executionProjectionsForRuns(db, companyId, runIds), + executionProjectionsForRuns(db, companyId, runIds, new Date(), { retryDatabaseReads: true }), savedPlanQuery, ]); const retryExhaustedReasonByRunId = new Map(); diff --git a/server/src/services/execution-projection-read-retry.test.ts b/server/src/services/execution-projection-read-retry.test.ts new file mode 100644 index 0000000000..383835d802 --- /dev/null +++ b/server/src/services/execution-projection-read-retry.test.ts @@ -0,0 +1,150 @@ +import { getTableName, type SQL } from "drizzle-orm"; +import { PgDialect } from "drizzle-orm/pg-core"; +import type { Db } from "@paperclipai/db"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { activityService } from "./activity.js"; +import { executionProjectionsForRuns } from "./execution-projection.js"; +import { logger } from "../middleware/logger.js"; + +const companyId = "00000000-0000-4000-8000-000000000001"; +const issueId = "00000000-0000-4000-8000-000000000002"; +const runId = "00000000-0000-4000-8000-000000000003"; +const now = new Date("2026-01-01T00:00:00Z"); +const tables = ["heartbeat_runs", "native_run_finalizations", "issue_thread_interactions", "issue_recovery_actions"]; +const dialect = new PgDialect(); +function disconnected(code = "CONNECTION_CLOSED") { + return new Error("Failed query", { cause: Object.assign(new Error("Connection failed"), { code }) }); +} + +function fixture(options: { + failures?: Record; + missing?: boolean; + noIssue?: boolean; + backfill?: boolean; + writeFailure?: Error; +} = {}) { + const attempts: Array<{ key: string; params: unknown[]; order: string[] }> = []; + const row = { + id: runId, companyId, runId, status: "succeeded", runtimeMode: "native", errorCode: null, + nativeIssueId: options.noIssue ? null : issueId, contextSnapshot: {}, + finishedAt: now, startedAt: now, createdAt: now, resultJson: null, usageJson: null, + scheduledRetryAt: null, scheduledRetryAttempt: 0, retryOfRunId: null, nextAction: null, + }; + const select = vi.fn((fields: Record = {}) => ({ + from(table: Parameters[0]) { + const name = getTableName(table); + const key = name === "heartbeat_runs" && "runId" in fields ? "run_list" + : name === "heartbeat_runs" && "stdoutExcerpt" in fields ? "liveness_backfill" + : name; + let condition: SQL; + let order: SQL[] = []; + let result: Promise | undefined; + const query = { + where(value: SQL) { condition = value; return query; }, + innerJoin() { return query; }, + orderBy(...values: SQL[]) { order = values; return query; }, + limit() { return query; }, + then(resolve: (value: unknown[]) => unknown, reject: (error: unknown) => unknown) { + // Re-awaiting the same failed builder cannot recover the connection. + result ??= Promise.resolve().then(() => { + const attempt = attempts.filter((read) => read.key === key).length; + attempts.push({ key, params: dialect.sqlToQuery(condition).params, + order: order.map((value) => dialect.sqlToQuery(value).sql) }); + const error = options.failures?.[key]?.[attempt]; + if (error) throw error; + if (options.missing) return []; + if (key === "heartbeat_runs" || key === "run_list") return [row]; + if (key === "liveness_backfill") return options.backfill ? [row] : []; + if (key === "issues") return [{ status: "in_progress", title: "Fixture", workMode: "standard" }]; + if (key === "issue_thread_interactions") return [{ issueId, kind: "ask_user_questions" }]; + return []; + }); + return result.then(resolve, reject); + }, + }; + return query; + }, + })); + const updateWhere = vi.fn(async () => { if (options.writeFailure) throw options.writeFailure; }); + const update = vi.fn(() => ({ set: () => ({ where: updateWhere }) })); + const transaction = vi.fn(() => { throw new Error("Must not start or replay a transaction"); }); + const db = { select, update, transaction } as unknown as Db; + return { db, attempts, select, update, updateWhere, transaction, + count: (key: string) => attempts.filter((read) => read.key === key).length }; +} + +afterEach(() => vi.restoreAllMocks()); + +describe("execution projection read retries", () => { + it.each(tables)("rebuilds only the failed %s SELECT within the existing retry budget", async (table) => { + const test = fixture({ failures: { [table]: [disconnected(), disconnected("CONNECTION_ENDED")] } }); + const result = await executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true }); + expect(result.get(runId)).toMatchObject({ phase: "waiting_for_answer", label: "Waiting for answer" }); + for (const name of tables) expect(test.count(name)).toBe(name === table ? 3 : 1); + expect(test.select).toHaveBeenCalledTimes(6); + const reads = test.attempts.filter((read) => read.key === table); + expect(reads[1]).toEqual(reads[0]); + expect(reads[2]).toEqual(reads[0]); + expect(reads[0].params).toContain(companyId); + expect(reads[0].params).toContain(table === "heartbeat_runs" || table === "native_run_finalizations" ? runId : issueId); + if (table === "issue_thread_interactions") expect(reads[0].params).toContain("pending"); + if (table === "issue_recovery_actions") expect(reads[0].order).toEqual(['"issue_recovery_actions"."updated_at" desc']); + expect(test.update).not.toHaveBeenCalled(); + expect(test.transaction).not.toHaveBeenCalled(); + }); + + it.each(tables)("preserves the last %s error after three attempts", async (table) => { + const errors = [disconnected(), disconnected(), disconnected()]; + const test = fixture({ failures: { [table]: errors } }); + await expect(executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true })).rejects.toBe(errors[2]); + for (const [index, name] of tables.entries()) { + expect(test.count(name)).toBe(index < tables.indexOf(table) ? 1 : name === table ? 3 : 0); + } + }); + + it.each(tables)("keeps the default transaction-compatible %s path single-attempt", async (table) => { + const error = disconnected(); + const test = fixture({ failures: { [table]: [error] } }); + await expect(executionProjectionsForRuns(test.db, companyId, [runId], now)).rejects.toBe(error); + expect(test.count(table)).toBe(1); + expect(test.transaction).not.toHaveBeenCalled(); + }); + + it.each([new Error("CONNECTION_CLOSED"), disconnected("28P01"), disconnected("42501"), disconnected("22P02"), disconnected("25P02")])( + "does not retry an unclassified or non-transient error", async (error) => { + const test = fixture({ failures: { issue_thread_interactions: [error] } }); + await expect(executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true })).rejects.toBe(error); + expect(test.count("issue_thread_interactions")).toBe(1); + expect(test.count("issue_recovery_actions")).toBe(0); + }, + ); + + it("preserves empty and taskless projection behavior without unnecessary reads", async () => { + const empty = fixture(); + expect(await executionProjectionsForRuns(empty.db, companyId, [], now, { retryDatabaseReads: true })).toEqual(new Map()); + expect(empty.select).not.toHaveBeenCalled(); + for (const options of [{ missing: true }, { noIssue: true }]) { + const test = fixture(options); + const result = await executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true }); + expect(test.select).toHaveBeenCalledTimes(2); + expect(options.missing ? result.size : result.get(runId)?.phase).toBe(options.missing ? 0 : "completed"); + } + }); + + it.each([false, true])("the run list retries projection reads without replaying other reads or backfill writes (write fails: %s)", async (writeFails) => { + vi.spyOn(logger, "warn").mockImplementation(() => undefined); + const test = fixture({ backfill: true, writeFailure: writeFails ? disconnected() : undefined, + failures: { issue_thread_interactions: [disconnected(), disconnected()] } }); + const result = await activityService(test.db).runsForIssue(companyId, issueId); + expect(result).toMatchObject([{ runId, execution: { phase: "waiting_for_answer" } }]); + expect(test.count("issue_thread_interactions")).toBe(3); + for (const key of ["run_list", "liveness_backfill", "heartbeat_runs", "native_run_finalizations", "environment_leases", "issue_recovery_actions"]) { + expect(test.count(key)).toBe(1); + } + // One retry-exhaustion lookup plus the independent backfill's event count. + expect(test.count("heartbeat_run_events")).toBe(2); + expect(test.update).toHaveBeenCalledTimes(1); + expect(test.updateWhere).toHaveBeenCalledTimes(1); + expect(test.transaction).not.toHaveBeenCalled(); + }); +}); diff --git a/server/src/services/execution-projection.ts b/server/src/services/execution-projection.ts index cab39d73e3..ae88b4b739 100644 --- a/server/src/services/execution-projection.ts +++ b/server/src/services/execution-projection.ts @@ -9,6 +9,7 @@ import { import type { ExecutionProjection } from "@paperclipai/shared"; import { EXECUTION_CONTROL_DEADLINE_MS } from "./execution-control-deadline.js"; import { executionFailureRetryCount } from "./execution-recovery-attempt.js"; +import { retryIdempotentDatabaseOperation } from "../database-retry.js"; const text = (v: unknown) => (typeof v === "string" ? v : null); const executionRunColumns = { id: heartbeatRuns.id, @@ -48,10 +49,16 @@ export async function executionProjectionsForRuns( companyId: string, runIds: string[], now = new Date(), + options: { retryDatabaseReads?: boolean } = {}, ) { const projections = new Map(); if (!runIds.length) return projections; - const runs = await db + // Opt in only with a pooled Db outside a transaction. Issue enrichment also + // calls this helper with a transaction, whose failed reads must not replay. + const read = (query: () => Promise) => options.retryDatabaseReads + ? retryIdempotentDatabaseOperation(query) + : query(); + const runs = await read(async () => db .select(executionRunColumns) .from(heartbeatRuns) .where( @@ -59,8 +66,8 @@ export async function executionProjectionsForRuns( eq(heartbeatRuns.companyId, companyId), inArray(heartbeatRuns.id, runIds), ), - ); - const coordinators = await db + )); + const coordinators = await read(async () => db .select() .from(nativeRunFinalizations) .where( @@ -68,7 +75,7 @@ export async function executionProjectionsForRuns( eq(nativeRunFinalizations.companyId, companyId), inArray(nativeRunFinalizations.runId, runIds), ), - ); + )); const issueIds = [ ...new Set( runs @@ -77,7 +84,7 @@ export async function executionProjectionsForRuns( ), ]; const pending = issueIds.length - ? await db + ? await read(async () => db .select({ issueId: issueThreadInteractions.issueId, kind: issueThreadInteractions.kind, @@ -89,10 +96,10 @@ export async function executionProjectionsForRuns( inArray(issueThreadInteractions.issueId, issueIds), eq(issueThreadInteractions.status, "pending"), ), - ) + )) : []; const recovery = issueIds.length - ? await db + ? await read(async () => db .select({ issueId: issueRecoveryActions.sourceIssueId, cause: issueRecoveryActions.cause, @@ -112,7 +119,7 @@ export async function executionProjectionsForRuns( ]), ), ) - .orderBy(desc(issueRecoveryActions.updatedAt)) + .orderBy(desc(issueRecoveryActions.updatedAt))) : []; const coordinatorByRun = new Map(coordinators.map((row) => [row.runId, row])); for (const run of runs) { From 7eadc714d239368eda4fdbfdbdcf9d6a958f761f Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Mon, 5 Oct 2026 20:21:01 -0700 Subject: [PATCH 05/12] Verify native semantic input against its raw wire digest (#15301) ## Thinking Path > - Paperclip manages AI agents and their work. > - Native runners send authenticated semantic tool inputs to the control plane. > - The runner hashes the complete input, but the receiver used a redacted receipt hash. > - Protected fields and credential-like document text can therefore fail integrity validation even when the input is unchanged. > - This pull request verifies the complete input with the existing canonical hash function. > - Receipt redaction and permanent rejection of altered input remain in place. ## Linked Issues or Issue Description **What happened?** The Rust runner preserves tool arguments and hashes their canonical JSON. The TypeScript receiver instead redacts the input before hashing. A valid input such as a synthetic document containing `Bearer fixture_token_123456` fails with `native_event_replay_conflict`. A digest of redacted input can also pass without proving the original protected values. **Expected behavior** Verify the complete transmitted input after authentication and exact scope checks. Reject any incorrect digest before durable commit, dispatch, or ACK. **Steps to reproduce** Run the new authenticated controller regressions against the prior receiver. The protected-field and credential-like document cases fail, and the redacted-digest rejection case receives an ACK. The same tests pass with this change. **Paperclip version or commit** Reproduced from source at `858094ba8123c7edb56623597cd96f0391f7e2d4` with synthetic fixtures. Applies to native runner deployments. Related: #14937 preserves semantic input bytes for execution. GitHub issue and PR searches found no duplicate fix; #14591 touches a separate question-draft contract. ## What Changed - Use the existing bounded raw canonical digest for incoming semantic and MCP tool inputs. - Keep receipt and diagnostic redaction unchanged. - Share four digest fixtures between Rust and the authenticated TypeScript controller, including protected fields, document text, Unicode keys, and number boundaries. - Test raw acceptance, altered protected values, forged and redacted digests, canonicalization limits, permanent reconnect fences, and authentication/scope rejection. - Document the separate wire and receipt contracts and the unchanged recovery fence. ## Verification - Before the production fix, the new selected regressions produced three expected failures and eight passes. - Focused controller, receipt, and semantic-tool suites: 138 tests passed. - Rust shared digest fixture test: 1 test passed. - Full workspace typecheck and build passed; the final receiver delta also passed its TypeScript typecheck. Canonical Linux PR CI passed the full aggregate test gate, runner checks, build, and canary dry run at `efcd38e2d8a9fa6340db4f4e863b6febbe8ca32a`. - Independent review passed, including 18 independently run authenticated regressions and the Rust golden test, plus both matching-digest size-limit cases after the test-only follow-up. Greptile is 5/5 on the current head with no unresolved threads. Diff whitespace and local secret/PII scan passed; fixtures are synthetic. ## Risks The verifier remains strict: there is no redacted-digest fallback. Existing authentication, scope, sequence, replay, settlement, and authorization checks remain in force. Receipt storage and redaction behavior are unchanged. This patch does not clear failed-run fences or replay prior work. Synthetic tests prove the protocol mismatch; they do not identify the contents of any historical rejected input. ## Model Used OpenAI Codex (GPT-6), with reasoning, code execution, and independent agent review. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- .../paperclip-runner/docs/durable-recovery.md | 9 + .../tests/semantic_input_digest.rs | 18 ++ .../durable-prp-control-plane.test.ts | 161 +++++++++++++++++- .../durable-prp-control-plane.ts | 19 ++- .../test/fixtures/semantic-input-digests.json | 42 +++++ 5 files changed, 240 insertions(+), 9 deletions(-) create mode 100644 packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs create mode 100644 packages/paperclip-runner/test/fixtures/semantic-input-digests.json diff --git a/packages/paperclip-runner/docs/durable-recovery.md b/packages/paperclip-runner/docs/durable-recovery.md index 8ee0b22c97..0a9d5d9a7a 100644 --- a/packages/paperclip-runner/docs/durable-recovery.md +++ b/packages/paperclip-runner/docs/durable-recovery.md @@ -95,6 +95,15 @@ welcome and every control envelope against the authenticated connection, runner, environment lease, run, normalized session, turn, item, protocol, lease ID, expiry, and revocation metadata before applying an ACK or command. +Semantic input integrity uses SHA-256 of the complete transmitted input's +canonical JSON, with UTF-16 object-key order and JavaScript number formatting. +The controller checks this proof after authentication and exact run/session/ +turn/item correlation, before committing, dispatching, or acknowledging the +input. Receipt redaction has a separate digest; a redacted digest cannot stand +in for wire integrity, including when only a protected field changes. A failed +integrity proof keeps the existing operator-required recovery fence. Updating +the verifier does not clear a previously failed run or replay its work. + The daemon captures and removes the bootstrap environment variable before it parses arguments or starts child work. Secret buffers are overwritten when they are dropped. It resolves the destination once before sending a bearer value and diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs new file mode 100644 index 0000000000..1ef2c8f34b --- /dev/null +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs @@ -0,0 +1,18 @@ +use paperclip_runner_core::provider_bridge::semantic_value_digest; +use serde_json::Value; + +#[test] +fn semantic_input_digests_match_the_authenticated_controller_fixtures() { + let fixtures: Value = serde_json::from_str(include_str!( + "../../../../test/fixtures/semantic-input-digests.json" + )) + .expect("parse shared semantic input digest fixtures"); + for fixture in fixtures.as_array().expect("digest fixture array") { + assert_eq!( + semantic_value_digest(&fixture["input"]), + fixture["digest"].as_str().expect("fixture digest"), + "fixture {}", + fixture["name"] + ); + } +} diff --git a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts index 9a1e145f6e..8980c1c4e3 100644 --- a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts +++ b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts @@ -26,6 +26,7 @@ import { validatePrpEvent } from "../protocol/replay-contract.js"; import { digestPaperclipSemanticContent } from "../semantic-tools/receipts.js"; import { DurablePrpControlPlane, + durableRecoveryInternals, SemanticToolNotDispatchedError, inspectWarmRunTransition, spawnRunner, @@ -1164,7 +1165,17 @@ async function receiveSecure( return value; } -function semanticInputEvent(sourceSeq = 1): Record { +const semanticInputDigestFixtures = JSON.parse(readFileSync( + new URL("../../test/fixtures/semantic-input-digests.json", import.meta.url), + "utf8", +)) as Array<{ name: string; input: Record; digest: string }>; + +function rawSemanticInputDigest(input: unknown): string { + return `sha256:${createHash("sha256") + .update(durableRecoveryInternals.canonicalJson(input)).digest("hex")}`; +} + +function semanticInputEvent(sourceSeq = 1, input: Record = {}): Record { return { protocol: "paperclip.runner", version: 1, @@ -1204,11 +1215,11 @@ function semanticInputEvent(sourceSeq = 1): Record { }, idempotencyKey: null, content: { - digest: digestPaperclipSemanticContent({}), + digest: rawSemanticInputDigest(input), redactionDisposition: "digest_only", references: [], }, - input: {}, + input, }, }, }, @@ -1687,6 +1698,144 @@ describe.sequential("DurablePrpControlPlane", () => { }, ); + it.each(semanticInputDigestFixtures.flatMap((fixture) => + ["semantic_tool.input", "mcp_app.tool_input"].map((eventType) => ({ ...fixture, eventType })), + ))( + "accepts the Rust wire digest and preserves exact input: $eventType / $name", + async ({ input, digest, eventType }) => { + // The JSON transport normalizes negative zero before the receiver sees it. + const wireInput = JSON.parse(JSON.stringify(input)) as Record; + const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-raw-input-")); + const onProtocolIntegrityError = vi.fn(); + const onSemanticToolInput = vi.fn(async () => ({ result: { ok: true } })); + const core = new DurablePrpControlPlane({ + stateDirectory: root, identity, expectedRunnerVersion, expectedRunnerDigest, + onProtocolIntegrityError, onSemanticToolInput, + }); + try { + await core.start(); + const client = (await authenticate(core, core.issueBootstrapTicket()))!; + const event = semanticInputEvent(1, wireInput); + (event.payload as any).eventType = eventType; + const semantic = (event.payload as any).payload.semantic_tool; + // This shared golden also runs against the Rust producer's hash. + semantic.content.digest = digest; + expect(rawSemanticInputDigest(input)).toBe(digest); + sendSecure(client, event); + const frames = [await receiveSecure(client), await receiveSecure(client)]; + expect(frames).toContainEqual(expect.objectContaining({ + kind: "ack", payload: expect.objectContaining({ ackedSourceSeq: 1 }), + })); + const command = frames.find((frame) => frame?.kind === "command")!.payload as any; + expect(command.payload.inputDigest).toBe(digest.slice("sha256:".length)); + expect(onSemanticToolInput).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({ input: wireInput })); + expect(core.store.state.committedEvents).toHaveLength(1); + expect(core.store.state.committedEvents[0]?.envelope).toEqual(event); + expect(onProtocolIntegrityError).not.toHaveBeenCalled(); + client.socket.destroy(); + } finally { await core.stop(); rmSync(root, { recursive: true, force: true }); } + }, + ); + + it.each([ + "redacted_digest", "protected_field_tampering", "forged_digest", + "over_node_limit", "over_depth_limit", + ] as const)( + "rejects invalid raw semantic integrity without commit, dispatch or ACK: %s", + async (fault) => { + const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-raw-integrity-")); + const onProtocolIntegrityError = vi.fn(); + const onCommittedEvent = vi.fn(async () => undefined); + const onSemanticToolInput = vi.fn(async () => ({ result: { ok: true } })); + const core = new DurablePrpControlPlane({ + stateDirectory: root, identity, expectedRunnerVersion, expectedRunnerDigest, + onProtocolIntegrityError, onCommittedEvent, onSemanticToolInput, + }); + try { + await core.start(); + const client = (await authenticate(core, core.issueBootstrapTicket()))!; + const input = { password: "fixture-before", safe: true }; + const event = semanticInputEvent(1, input); + const semantic = (event.payload as any).payload.semantic_tool; + const redactedDigest = digestPaperclipSemanticContent(input); + expect(redactedDigest).not.toBe(semantic.content.digest); + if (fault === "redacted_digest") semantic.content.digest = redactedDigest; + if (fault === "forged_digest") semantic.content.digest = `sha256:${"0".repeat(64)}`; + if (fault === "protected_field_tampering") { + semantic.input = { ...input, password: "fixture-after" }; + // Redacted receipt hashes intentionally cannot prove byte integrity. + expect(digestPaperclipSemanticContent(semantic.input)).toBe(redactedDigest); + } + if (fault === "over_node_limit") semantic.input = { values: Array(10_001).fill(0) }; + if (fault === "over_depth_limit") { + let nested: unknown = {}; + for (let depth = 0; depth < 65; depth += 1) nested = { child: nested }; + semantic.input = nested; + } + if (fault === "over_node_limit" || fault === "over_depth_limit") { + // Each fixture object has one key, so JSON.stringify is canonical. + // Hash independently of the bounded helper: the limits themselves + // must reject even a matching raw digest and keep the permanent fence. + semantic.content.digest = `sha256:${createHash("sha256") + .update(JSON.stringify(semantic.input)).digest("hex")}`; + } + sendSecure(client, event); + await expect(receiveSecure(client)).resolves.toBeNull(); + expect(onProtocolIntegrityError).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({ + code: "native_event_replay_conflict", reason: "semantic_input_digest_mismatch", + recovery: "operator_required", + })); + expect(core.store.state.ackedSourceSeq).toBe(0); + expect(core.store.state.committedEvents).toEqual([]); + expect(onCommittedEvent).not.toHaveBeenCalled(); + expect(onSemanticToolInput).not.toHaveBeenCalled(); + expect(JSON.stringify(onProtocolIntegrityError.mock.calls)).not.toContain("fixture-before"); + expect(JSON.stringify(onProtocolIntegrityError.mock.calls)).not.toContain("fixture-after"); + const replay = (await authenticate(core, client.leaseToken!))!; + sendSecure(replay, semanticInputEvent()); + await expect(receiveSecure(replay)).resolves.toBeNull(); + expect(onProtocolIntegrityError).toHaveBeenCalledTimes(1); + expect(onSemanticToolInput).not.toHaveBeenCalled(); + } finally { await core.stop(); rmSync(root, { recursive: true, force: true }); } + }, + ); + + it.each(["authentication", "envelope", "event", "correlation"] as const)( + "rejects raw semantic input outside its exact authenticated scope: %s", + async (boundary) => { + const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-raw-scope-")); + const onProtocolIntegrityError = vi.fn(); + const onCommittedEvent = vi.fn(async () => undefined); + const onSemanticToolInput = vi.fn(async () => ({ result: { ok: true } })); + const core = new DurablePrpControlPlane({ + stateDirectory: root, identity, expectedRunnerVersion, expectedRunnerDigest, + onProtocolIntegrityError, onCommittedEvent, onSemanticToolInput, + }); + try { + await core.start(); + const wrongRunId = "00000000-0000-4000-8000-000000000999"; + const client = await authenticate(core, core.issueBootstrapTicket(), + boundary === "authentication" ? { ...identity, runId: wrongRunId } : identity); + if (boundary === "authentication") { + expect(client).toBeNull(); + } else { + const event = semanticInputEvent(1, { password: "fixture-only" }); + const payload = event.payload as any; + if (boundary === "envelope") event.runId = wrongRunId; + if (boundary === "event") payload.runId = wrongRunId; + if (boundary === "correlation") payload.payload.semantic_tool.correlation.runId = wrongRunId; + sendSecure(client!, event); + await expect(receiveSecure(client!)).resolves.toBeNull(); + } + expect(core.store.state.ackedSourceSeq).toBe(0); + expect(core.store.state.committedEvents).toEqual([]); + expect(onCommittedEvent).not.toHaveBeenCalled(); + expect(onSemanticToolInput).not.toHaveBeenCalled(); + expect(onProtocolIntegrityError).not.toHaveBeenCalled(); + } finally { await core.stop(); rmSync(root, { recursive: true, force: true }); } + }, + ); + it("latches an authenticated semantic integrity fault without ACK or dispatch and still permits suspension", async () => { const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-integrity-")); const onProtocolIntegrityError = vi.fn(); @@ -1848,7 +1997,7 @@ describe.sequential("DurablePrpControlPlane", () => { ).semantic_tool as Record; semantic.input = { changed: true }; (semantic.content as Record).digest = - digestPaperclipSemanticContent(semantic.input); + rawSemanticInputDigest(semantic.input); sendSecure(client, changed); await expect(receiveSecure(client)).resolves.toBeNull(); expect(onProtocolIntegrityError).toHaveBeenCalledTimes(1); @@ -2232,13 +2381,13 @@ describe.sequential("DurablePrpControlPlane", () => { const event = semanticInputEvent(); const semantic = ((event.payload as any).payload as any).semantic_tool; semantic.input = args; - semantic.content.digest = digestPaperclipSemanticContent(args); + semantic.content.digest = rawSemanticInputDigest(args); sendSecure(client, event); const frames = [await receiveSecure(client), await receiveSecure(client)]; const wire = frames.find((frame) => frame?.kind === "command")!.payload as any; expect(wire.payload.result.content).toBe(content); expect(wire.payload.input).toBeUndefined(); - expect(wire.payload.inputDigest).toBe(digestPaperclipSemanticContent(args).slice("sha256:".length)); + expect(wire.payload.inputDigest).toBe(rawSemanticInputDigest(args).slice("sha256:".length)); expect(Buffer.byteLength(JSON.stringify(wire))).toBeLessThan(1024 * 1024); sendSecure(client, { protocol: "paperclip.runner", version: 1, kind: "command_result", payload: { commandId: wire.commandId, controllerSeq: wire.controllerSeq, commandType: wire.type, diff --git a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts index dfcaf5bb8d..c7f8cc9e87 100644 --- a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts +++ b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts @@ -35,7 +35,6 @@ import { validatePrpEvent, type PrpEvent, } from "../protocol/replay-contract.js"; -import { digestPaperclipSemanticContent } from "../semantic-tools/receipts.js"; import { type DurableRecoveryCommittedEvent, type DurableRecoveryCoreCommand, @@ -416,6 +415,18 @@ function canonicalDigest(value: unknown): string { return createHash("sha256").update(canonicalJson(value)).digest("hex"); } +function matchesSemanticInputDigest(input: unknown, digest: unknown): boolean { + try { + // Wire integrity covers the complete input, including protected fields. + // Receipt redaction can erase those differences and has a separate hash. + return digest === `sha256:${canonicalDigest(input)}`; + } catch { + // Canonicalization bounds must keep the same permanent integrity fence. + // Never attach an input-derived error or payload to the diagnostic. + return false; + } +} + function exactIdentity(value: unknown): value is DurableRecoveryIdentity { return ( isRecord(value) && @@ -3141,8 +3152,10 @@ export class DurablePrpControlPlane { if ( isSemanticInput && semantic !== undefined && - (semantic.content as Record).digest !== - digestPaperclipSemanticContent(semantic.input) + !matchesSemanticInputDigest( + semantic.input, + (semantic.content as Record).digest, + ) ) { // Only the authenticated, schema-valid, exactly correlated input may // permanently fail its owner. Never commit, dispatch, or ACK these bytes. diff --git a/packages/paperclip-runner/test/fixtures/semantic-input-digests.json b/packages/paperclip-runner/test/fixtures/semantic-input-digests.json new file mode 100644 index 0000000000..db60546267 --- /dev/null +++ b/packages/paperclip-runner/test/fixtures/semantic-input-digests.json @@ -0,0 +1,42 @@ +[ + { + "name": "plain document", + "input": { + "body": "Plain fixture document" + }, + "digest": "sha256:2ba38770004c4e7672a5097c4d8e3241c62750b21e034903796fb035fcf9a2a4" + }, + { + "name": "protected fields", + "input": { + "password": "fixture-only-not-a-real-credential", + "safe": true + }, + "digest": "sha256:98ecc80f0e6d2d4b99ce3620d18a21392356f99def55ed599c1b1c002ccd2d47" + }, + { + "name": "credential-like document text", + "input": { + "body": "Example Bearer fixture_token_123456\nhttps://example.test/callback?token=fixture-only" + }, + "digest": "sha256:aa7f429cc5dd63e85fd11aa93e0659fa1c7d5041f5f35843fef62092ff86fdc0" + }, + { + "name": "canonical Unicode and numbers", + "input": { + "": "private-use key", + "😀": "astral key", + "values": [ + -0, + 0.000001, + 9.99999e-7, + 100000000000000000000, + 1e+21, + true, + null, + "日本語\n\"text\"" + ] + }, + "digest": "sha256:3104a5e5db1fe2c6dcdaa0bfcf362d7bd36441e7e08905a48ae03be06eea9dc4" + } +] From 1d5a439e2b0e551f7a5129e414d7bdc8582ddd00 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Mon, 5 Oct 2026 20:57:33 -0700 Subject: [PATCH 06/12] Diagnose native model rejections and preserve repairable reviews (#15304) ## Thinking Path > - Paperclip manages agent execution and recovery. > - A native provider can return a failed turn with an accepted result. > - This path replaced a structured model rejection with a generic failure. > - Finalization could also queue the same failed request for recovery. > - This change retains a bounded model and authentication diagnosis. > - Workers wait for a configuration fix; reviews retain their assignment. ## Linked Issues or Issue Description **Describe the bug** A native Codex turn can report HTTP 400 because the selected model is not supported by its ChatGPT connection. When the turn also returns a failed semantic result, the run shows `Native session failed` and `adapter_failed`. The native failed-result finalizer can create a retry intent before the adapter error is saved. **To Reproduce** Use a native Codex session that returns a structured `invalid_request_error` model/ChatGPT rejection and an accepted failed result. The regression tests reproduce this with a stub provider and a real isolated PostgreSQL database. No provider account or live request is needed. **Expected behavior** Show an actionable error and stop automatic retries of the rejected request. Block the current worker for a configuration fix. Preserve a pending review so its reviewer can explicitly retry after repair and resolve the original assignment. A stale run must not block a new task owner or override a completed review. **Version and deployment mode** The affected source path is present at `858094ba81`. This change applies to native Codex execution in local and remote environments. Related work: #13853 covers CLI protocol compatibility, #14942 refreshes runtime/model pins, and #12767 proposes legacy adapter model validation. This change concerns accepted native failed results and does not replace those changes. ## What Changed - Recognize only a bounded structured HTTP 400 provider error for the exact selected model and failed turn. - Save a fixed actionable error and four fixed diagnostic fields. Exclude raw provider text and model names from the new diagnostic. - Re-derive finalization evidence from the pinned execution and committed runner event, including its canonical hash and identity bindings. - Block only the current worker. For a current pending reviewer, preserve the original review decision and status version, release execution, and create no automatic recovery action. Recheck execution ownership under the issue lock. Preserve superseded and completed reviews. - Keep unknown failures, accepted results, cleanup, and ownership fences intact. Advance the status policy version to `phase6-v8`. - Route this exact failure code through the existing configuration recovery path. Permit an authenticated Board retry of this native failure only for its saved, still-pending review; discard caller review markers and revalidate at dispatch. ## Verification - Node 24: `pnpm -r typecheck` and `pnpm build` passed. - Independent focused validation: 222 tests passed across the new observer and PostgreSQL integration suites, status arbiter, diagnostic redaction, recovery classification, wake policy, and wake use cases. - The integration tests use actual result persistence, finalization, and the wake dispatcher. They cover zero retry dispatch for an authorized rejection, repeated reconciliation, pending/superseded/resolved reviews, reassignment, and a same-agent successor at an unchanged status version. - Final-commit verification: full typecheck and build passed; independent route/integration/arbiter/review validation passed 136 tests with no remaining source findings. This includes authenticated retry refusals, actual heartbeat retry admission, atomic queued-review claim, and acceptance of the original review. - The local aggregate test run was stopped to avoid duplicating the canonical Linux CI aggregate on this development host. Its partial log showed no failures; no full local-suite pass is claimed. The full hosted Linux CI aggregate passed on the final commit. - No live provider calls, runtime changes, or schema changes were made. ## Risks The classifier is intentionally narrow. A changed provider response format remains an unknown failure. No global model restriction is introduced. The worker blocker requires current task authority and does not replace newer assignments or completed review decisions. A pending review remains in review and requires an explicit retry after configuration repair; it is not rebound to a new status decision. The retry decision is durable. There remains a small existing gap between status/effect commit and diagnostic metadata persistence; a crash there can leave a generic diagnostic while preserving the blocked decision. No schema migration is required. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The exact serving model suffix and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #123` / `Refs #123` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/observability.md | 24 ++ .../__tests__/agent-live-run-routes.test.ts | 68 +++++ .../heartbeat-process-recovery.test.ts | 4 +- .../src/modules/wake-queue/domain/values.ts | 3 +- server/src/routes/agents.ts | 22 +- .../__tests__/run-failure-diagnostics.test.ts | 13 + .../native-provider-failure-evidence.ts | 45 +++ ...ative-provider-failure.integration.test.ts | 272 ++++++++++++++++++ .../native-provider-failure.test.ts | 73 +++++ .../native-runtime/native-provider-failure.ts | 60 ++++ .../native-runtime/native-run-finalizer.ts | 21 +- .../native-runtime/native-session-executor.ts | 9 +- .../native-runtime/status-arbiter.test.ts | 19 +- .../services/native-runtime/status-arbiter.ts | 30 +- .../status-decision-committer.ts | 19 ++ .../provider-failure-classification.test.ts | 12 + server/src/services/recovery/service.ts | 3 +- .../src/services/run-failure-diagnostics.ts | 3 + 18 files changed, 691 insertions(+), 9 deletions(-) create mode 100644 server/src/services/native-runtime/native-provider-failure-evidence.ts create mode 100644 server/src/services/native-runtime/native-provider-failure.integration.test.ts create mode 100644 server/src/services/native-runtime/native-provider-failure.test.ts create mode 100644 server/src/services/native-runtime/native-provider-failure.ts diff --git a/doc/observability.md b/doc/observability.md index d8c4b0e4a2..e877610b3d 100644 --- a/doc/observability.md +++ b/doc/observability.md @@ -443,6 +443,30 @@ values become `unknown`. A signal such as `SIGKILL` does not establish who sent it or prove an out-of-memory kill. These fields do not change error grouping or run outcomes, and do not include process output or adapter result payloads. +For a native Codex turn that ends with a structured HTTP 400 model/ChatGPT +compatibility rejection, the run records `native_provider_model_rejected` +and a fixed message that asks the user to select a supported model or connection. +This requires a committed runner terminal for the selected model and the exact +failed turn. Agent text, tool output, unrelated turns, and unknown provider +errors do not create this classification. The provider diagnostic contains only +`provider=codex`, `category=model_auth_incompatible`, `status=400`, and +`authMode=chatgpt`. It excludes the model name and provider response. This does +not change credentials, select a fallback model, replay a turn, or alter the +accepted semantic result or cleanup. The existing configuration-blocker recovery +path requires a configuration change instead of automatically retrying the same +model and connection. Native finalization also derives this evidence from the +committed event and the run's pinned provider configuration before it can queue +a failed-result retry. This survives a controller restart before the adapter +diagnostic is saved. Invalid bindings, missing evidence, or a mismatched event +hash do not grant this classification. A current worker's task is blocked for a +configuration change. A pending review instead keeps its original decision and +status version, releases execution, and creates no automatic recovery action. +After repairing the configuration, explicitly retry the reviewer to resolve the +same pending review. The Board's exact failed-run Retry accepts this classified +native failure only while its saved review is still current and pending; caller +payloads cannot select a different review. Dispatch checks that assignment again. +Superseded or completed reviews retain their authority. + An unconfirmed adapter Stop timeout has an event-local `adapter_stop` context: the run UUID, built-in adapter type, native/legacy runtime mode, configured wait duration, and whether abort was requested. Invalid identities become diff --git a/server/src/__tests__/agent-live-run-routes.test.ts b/server/src/__tests__/agent-live-run-routes.test.ts index 55d978e079..d5d1805a69 100644 --- a/server/src/__tests__/agent-live-run-routes.test.ts +++ b/server/src/__tests__/agent-live-run-routes.test.ts @@ -23,6 +23,7 @@ const mockIssueService = vi.hoisted(() => ({ getById: vi.fn(), getByIdentifier: vi.fn(), })); +const mockNativeReviewAssignment = vi.hoisted(() => vi.fn()); const mockExecutionProjection = vi.hoisted(() => ({ executionProjectionForRun: vi.fn(async () => null), @@ -79,6 +80,10 @@ const mockChatRunRetries = vi.hoisted(() => ({ })); function registerModuleMocks() { + vi.doMock("../services/native-runtime/native-review-participant.js", async () => ({ + ...(await vi.importActual("../services/native-runtime/native-review-participant.js")), + getNativeReviewAssignment: mockNativeReviewAssignment, + })); vi.doMock("../services/execution-projection.js", () => mockExecutionProjection); vi.doMock("../routes/authz.js", async () => vi.importActual("../routes/authz.js"), @@ -287,6 +292,7 @@ describe("agent live run routes", () => { vi.clearAllMocks(); mockChatRunRetries.prepareFailedChatRunRetry.mockReset(); mockChatRunRetries.processFailedChatRunRetry.mockReset(); + mockNativeReviewAssignment.mockReset().mockResolvedValue(null); mockAccessService.canUser.mockResolvedValue(true); mockAccessService.decide.mockImplementation(async (input: { action?: string }) => ({ allowed: true, @@ -966,6 +972,68 @@ describe("agent live run routes", () => { expect(mockHeartbeatService.wakeup).toHaveBeenCalledWith(routeAgentId, expect.objectContaining({ manualUserWake: true })); }); + describe("exact native model-rejected review retry", () => { + const reviewContext = { + nativeReviewInteractionId: "55555555-5555-4555-8555-555555555555", + nativeReviewDecisionId: "66666666-6666-4666-8666-666666666666", + }; + const selectedRun = { + id: failedChatRunId, companyId: "company-1", agentId: routeAgentId, + status: "failed", runtimeMode: "native", errorCode: "native_provider_model_rejected", + nativeIssueId: failedChatIssueId, + contextSnapshot: { issueId: failedChatIssueId, ...reviewContext }, + }; + const selectedIssue = { id: failedChatIssueId, companyId: "company-1", status: "in_review", assigneeAgentId: "worker", assigneeUserId: null }; + const retryBody = { failedRunId: failedChatRunId, reason: "retry_failed_run" }; + beforeEach(() => { + mockAgentService.getById.mockResolvedValue({ id: routeAgentId, companyId: "company-1" }); + mockHeartbeatService.getRun.mockResolvedValue(selectedRun); + mockIssueService.getById.mockResolvedValue(selectedIssue); + mockNativeReviewAssignment.mockResolvedValue({ interaction: { status: "pending" } }); + }); + + it("permits the Board to retry the exact pending review using only saved context", async () => { + const fixture = createFailedChatRetryDb(false); + const res = await requestApp(await createApp(fixture.db), url => request(url).post(`/api/agents/${routeAgentId}/wakeup`).send({ + ...retryBody, payload: { issueId: "forged", nativeReviewInteractionId: "forged", nativeReviewDecisionId: "forged" }, + })); + expect(res.status, JSON.stringify(res.body)).toBe(202); + expect(mockNativeReviewAssignment).toHaveBeenCalledExactlyOnceWith(fixture.db, { + companyId: "company-1", issueId: failedChatIssueId, agentId: routeAgentId, contextSnapshot: reviewContext, + }); + expect(mockHeartbeatService.wakeup).toHaveBeenCalledExactlyOnceWith(routeAgentId, expect.objectContaining({ + failedRunId: failedChatRunId, reason: "retry_failed_run", payload: { issueId: failedChatIssueId }, + requestedByActorType: "user", contextSnapshot: expect.objectContaining(reviewContext), + })); + expect(JSON.stringify(mockHeartbeatService.wakeup.mock.calls)).not.toContain("forged"); + }); + + it.each(["agent-caller", "denied-task", "foreign-company", "foreign-agent", "foreign-issue", "legacy", "unknown-error", "missing-context", "stale", "resolved", "changed-reviewer"])( + "rejects an ineligible exact review retry (%s)", async scenario => { + let actor: Record | undefined; + if (scenario === "agent-caller") actor = { type: "agent", agentId: routeAgentId, companyId: "company-1" }; + if (scenario === "denied-task") mockAccessService.decide.mockImplementation(async ({ action }) => ({ allowed: action !== "issue:comment", explanation: "Denied" })); + if (scenario === "foreign-company") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, companyId: "company-2" }); + if (scenario === "foreign-agent") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, agentId: "other-reviewer" }); + if (scenario === "foreign-issue") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, nativeIssueId: "other-task" }); + if (scenario === "legacy") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, runtimeMode: "legacy" }); + if (scenario === "unknown-error") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, errorCode: "adapter_failed" }); + if (scenario === "missing-context") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, contextSnapshot: { issueId: failedChatIssueId } }); + if (scenario === "stale" || scenario === "changed-reviewer") mockNativeReviewAssignment.mockResolvedValue(null); + if (scenario === "resolved") mockNativeReviewAssignment.mockResolvedValue({ interaction: { status: "accepted" } }); + const fixture = createFailedChatRetryDb(false); + const res = await requestApp(await createApp(fixture.db, actor), url => request(url).post(`/api/agents/${routeAgentId}/wakeup`).send({ + ...retryBody, payload: { ...reviewContext }, + })); + expect(res.status).toBe(["agent-caller", "denied-task"].includes(scenario) ? 403 : ["foreign-company", "foreign-agent"].includes(scenario) ? 404 : 409); + expect(mockHeartbeatService.wakeup).not.toHaveBeenCalled(); + if (["agent-caller", "denied-task", "foreign-company", "foreign-agent", "foreign-issue", "legacy", "unknown-error", "missing-context"].includes(scenario)) { + expect(mockNativeReviewAssignment).not.toHaveBeenCalled(); + } + }, + ); + }); + describe("exact failed chat run retry", () => { const retryBody = { failedRunId: failedChatRunId, diff --git a/server/src/__tests__/heartbeat-process-recovery.test.ts b/server/src/__tests__/heartbeat-process-recovery.test.ts index 80d2011761..88c1a64cfe 100644 --- a/server/src/__tests__/heartbeat-process-recovery.test.ts +++ b/server/src/__tests__/heartbeat-process-recovery.test.ts @@ -14310,7 +14310,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { await commitNativeStatusDecision({ db, companyId: fixture.companyId, issueId: fixture.issueId, runId: fixture.runId, assessmentId: coordinator!.assessmentId!, priorStatus: issue!.status, priorStatusVersion: issue!.statusVersion, priorDecisionId: issue!.lastStatusDecisionId, - decision: { policyVersion: "phase6-v7", statusAction: "in_progress", toStatus: "in_progress", + decision: { policyVersion: "phase6-v8", statusAction: "in_progress", toStatus: "in_progress", reasonCode: "board_response_waiting", unblockDescriptor: null, effects: [] }, requireBoardResponseWaitSource: source!.source, requireBoardResponseWaitOrigin: origin!, }); @@ -14934,7 +14934,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { priorStatusVersion: issue!.statusVersion, priorDecisionId: issue!.lastStatusDecisionId, decision: { - policyVersion: "phase6-v7", + policyVersion: "phase6-v8", statusAction: reasonCode === "prior_status_preserved_no_live_path" ? "preserve" : "in_progress", toStatus: "in_progress", reasonCode, diff --git a/server/src/modules/wake-queue/domain/values.ts b/server/src/modules/wake-queue/domain/values.ts index 48191b69d2..fc4fa8fa5e 100644 --- a/server/src/modules/wake-queue/domain/values.ts +++ b/server/src/modules/wake-queue/domain/values.ts @@ -22,5 +22,6 @@ export function isWorkspaceValidationFailedRun(run: { errorCode: string | null } } export function isConfigurationIncompleteFailedRun(run: { errorCode: string | null }): boolean { - return run.errorCode === CONFIGURATION_INCOMPLETE_FAILURE_CODE || run.errorCode === "model_not_found"; + return run.errorCode === CONFIGURATION_INCOMPLETE_FAILURE_CODE || run.errorCode === "model_not_found" || + run.errorCode === "native_provider_model_rejected"; } diff --git a/server/src/routes/agents.ts b/server/src/routes/agents.ts index 93ab497d32..6e4ce96fd2 100644 --- a/server/src/routes/agents.ts +++ b/server/src/routes/agents.ts @@ -15,6 +15,7 @@ import { isAiConnectionCompatible } from "@paperclipai/shared"; import { applyConnectorSkills, resolveConnectorAssignments, annotateConnectorSkills, isConnectorSkill } from "../services/connector-runtime.js"; import { getExecutionBlocker } from "../services/execution-blocker.js"; import { canRetryStoppedRun } from "../services/cancelled-native-startup.js"; +import { getNativeReviewAssignment, readNativeReviewAssignmentContext, type NativeReviewAssignmentContext } from "../services/native-runtime/native-review-participant.js"; import { paperclipRunnerTransitionConfig, normalizeLegacyRunnerProvider, isPaperclipRunnerProvider } from "@paperclipai/adapter-utils"; import { executionProjectionForRun, executionProjectionsForRuns } from "../services/execution-projection.js"; import { selectDashboardRunIds } from "../services/dashboard-run-selection.js"; @@ -6036,6 +6037,7 @@ export function agentRoutes( let wakePayload = req.body.payload ?? null; let retryConversationContext: Record = {}; + let retryNativeReviewContext: NativeReviewAssignmentContext | null = null; if (req.body.failedRunId) { assertBoard(req); if ( @@ -6089,7 +6091,22 @@ export function agentRoutes( }, }); if (!decision.allowed) throw forbidden(decision.explanation, authorizationDeniedDetails(decision)); - if (issue.assigneeAgentId !== agent.id) throw conflict("The task is no longer assigned to this agent."); + const savedReviewContext = readNativeReviewAssignmentContext(failedContext); + if (failedRun.runtimeMode === "native" && failedRun.status === "failed" && + failedRun.errorCode === "native_provider_model_rejected" && + failedRun.nativeIssueId === issue.id && savedReviewContext) { + const review = await getNativeReviewAssignment(db, { + companyId: agent.companyId, issueId: issue.id, agentId: agent.id, + contextSnapshot: savedReviewContext, + }); + if (review?.interaction.status !== "pending") { + throw conflict("The failed run's review is no longer current."); + } + retryNativeReviewContext = savedReviewContext; + } + if (issue.assigneeAgentId !== agent.id && !retryNativeReviewContext) { + throw conflict("The task is no longer assigned to this agent."); + } if (issue.conversationAgentId) { // Agent Chat has no task description to replay. Recover the exact // request from the selected server-owned run, never caller markers. @@ -6196,6 +6213,9 @@ export function agentRoutes( requestedByActorId: req.actor.type === "agent" ? req.actor.agentId ?? null : req.actor.userId ?? null, contextSnapshot: { ...retryConversationContext, + // Only the selected server-owned run can carry review authority into + // an exact retry. Dispatch revalidates the current review assignment. + ...retryNativeReviewContext, triggeredBy: req.actor.type, originIdentityContextId: req.actor.identityContextId ?? null, responsibleUserId: req.actor.type === "agent" ? req.actor.onBehalfOfUserId ?? null : req.actor.userId ?? null, diff --git a/server/src/services/__tests__/run-failure-diagnostics.test.ts b/server/src/services/__tests__/run-failure-diagnostics.test.ts index 98a585aa7c..6ee721b7a3 100644 --- a/server/src/services/__tests__/run-failure-diagnostics.test.ts +++ b/server/src/services/__tests__/run-failure-diagnostics.test.ts @@ -7,6 +7,19 @@ const run = (overrides: Partial = {}) => ({ resultJson: null, ...overrides const collect = (error: unknown) => collectRunFailureDiagnostics(run(), { error }); describe("run failure diagnostics", () => { + it("exports only the closed native model/auth rejection vocabulary", () => { + const diagnostic = { provider: "codex", category: "model_auth_incompatible", status: 400, authMode: "chatgpt" }; + const result = sanitizeRunFailureDiagnostics(collectRunFailureDiagnostics(run({ resultJson: { + nativeProviderFailure: { ...diagnostic, model: "private-model", response: "private-provider-response", prompt: "private-prompt" }, + } }), {})); + expect(result.provider).toEqual(diagnostic); + expect(JSON.stringify(result)).not.toContain("private"); + for (const value of [null, [], { ...diagnostic, category: "private-category" }, + Object.defineProperty({}, "provider", { get() { throw new Error("private"); } })]) { + expect(collectRunFailureDiagnostics(run({ resultJson: { nativeProviderFailure: value } }), {}).provider).toEqual({}); + } + }); + it("selects bounded lock-owner evidence from a caught timeout cause", () => { const error = new Error("outer", { cause: Object.assign(new Error("lock timeout"), { code: "ERR_WORKSPACE_RESTORE_LOCK_TIMEOUT", diff --git a/server/src/services/native-runtime/native-provider-failure-evidence.ts b/server/src/services/native-runtime/native-provider-failure-evidence.ts new file mode 100644 index 0000000000..e97143aca9 --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure-evidence.ts @@ -0,0 +1,45 @@ +import { and, desc, eq, sql } from "drizzle-orm"; +import { heartbeatRunEvents, heartbeatRuns, type Db } from "@paperclipai/db"; +import { parseNativeExecutionInput, validatePrpEvent, type PrpTerminalState } from "../../vendor/paperclip-runner/index.js"; +import { nativeSha256 } from "./canonical.js"; +import { createNativeProviderFailureObservation } from "./native-provider-failure.js"; + +function record(value: unknown): Record { + return value !== null && typeof value === "object" && !Array.isArray(value) + ? value as Record : {}; +} + +/** Re-derive authority from immutable run-bound evidence, including after restart. */ +export async function readPersistedNativeModelRejection( + db: Db, + run: typeof heartbeatRuns.$inferSelect, + turnId: string | null, + terminal: PrpTerminalState, +) { + if (run.runtimeMode !== "native" || !turnId || !run.nativeSessionId || !run.runnerInstanceId || + terminal.turnTerminalState !== "failed" || terminal.runTerminalState !== "failed") return null; + let execution; + try { execution = parseNativeExecutionInput(record(run.runnerProfileJson).nativeExecutionInput); } catch { return null; } + if (execution.provider.kind !== "codex" || !execution.provider.model || + execution.binding.runId !== run.id || execution.binding.companyId !== run.companyId || + execution.binding.agentId !== run.agentId || execution.binding.issueId !== run.nativeIssueId || + execution.session.normalizedSessionId !== run.nativeSessionId || + execution.completionContract.id !== run.completionContractId || + execution.completionContract.sha256 !== run.completionContractSha256) return null; + + const [row] = await db.select().from(heartbeatRunEvents).where(and( + eq(heartbeatRunEvents.companyId, run.companyId), eq(heartbeatRunEvents.runId, run.id), + eq(heartbeatRunEvents.sourceInstanceId, run.runnerInstanceId), eq(heartbeatRunEvents.eventType, "turn.failed"), + sql`${heartbeatRunEvents.payload}->'prpEvent'->>'turnId' = ${turnId}`, + )).orderBy(desc(heartbeatRunEvents.seq)).limit(1); + if (!row) return null; + const parsed = validatePrpEvent(record(row.payload).prpEvent); + if (!parsed.ok || parsed.event.runId !== run.id || parsed.event.normalizedSessionId !== run.nativeSessionId || + parsed.event.sourceKind !== "runner" || parsed.event.sourceInstanceId !== run.runnerInstanceId || + parsed.event.sourceEventId !== row.sourceEventId || parsed.event.sourceSeq !== row.sourceSeq || + parsed.event.eventType !== row.eventType || parsed.event.turnId !== turnId || + row.sourcePayloadSha256 !== nativeSha256(parsed.event)) return null; + const observed = createNativeProviderFailureObservation(execution.provider); + observed.observe(parsed.event); + return observed.forTerminal(turnId, terminal); +} diff --git a/server/src/services/native-runtime/native-provider-failure.integration.test.ts b/server/src/services/native-runtime/native-provider-failure.integration.test.ts new file mode 100644 index 0000000000..2da7891abb --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure.integration.test.ts @@ -0,0 +1,272 @@ +import { randomUUID } from "node:crypto"; +import { eq } from "drizzle-orm"; +import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { agents, agentWakeupRequests, companies, completionContracts, createDb, heartbeatRunEvents, heartbeatRuns, issueRecoveryActions, issueThreadInteractions, issues, nativeRunFinalizations, nativeRunResults, statusDecisions, workAssessments } from "@paperclipai/db"; +import type { ControlPlanePort, NativeExecutionInputV1, PrpEvent, PrpTerminalState } from "@paperclipai/paperclip-runner"; +import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js"; +import { CONTROL_PLANE_CONFORMANCE_RESULT } from "../../vendor/paperclip-runner/testing.js"; + +const provider = vi.hoisted(() => ({ execute: vi.fn() })); +vi.mock("../../vendor/paperclip-runner/index.js", async (importOriginal) => ({ + ...(await importOriginal()), + executeNativeSession: provider.execute, +})); + +import { executePaperclipNativeSession } from "./native-session-executor.js"; +import { buildNativeCompletionContract } from "./completion-contracts.js"; +import { PaperclipControlPlanePort } from "./paperclip-control-plane-port.js"; +import { prepareNativeHeartbeatRun } from "./prepare-native-run.js"; +import { NATIVE_MODEL_REJECTION_DIAGNOSTIC, NATIVE_MODEL_REJECTION_MESSAGE } from "./native-provider-failure.js"; +import { collectRunFailureDiagnostics } from "../run-failure-diagnostics.js"; +import { finalizeNativeRun } from "./native-run-finalizer.js"; +import { readPersistedNativeModelRejection } from "./native-provider-failure-evidence.js"; +import { heartbeatService } from "../heartbeat.js"; +import { getNativeReviewAssignment, readNativeReviewAssignmentContext } from "./native-review-participant.js"; +import { claimQueuedNativeReviewRun } from "./native-review-dispatch.js"; +import { commitNativeStatusDecision, NativeStatusRaceError } from "./status-decision-committer.js"; +import { NATIVE_STATUS_ARBITER_POLICY_VERSION } from "./status-arbiter.js"; +import { issueThreadInteractionService } from "../issue-thread-interactions.js"; +import { createReleaseIssueExecution } from "../../modules/wake-queue/application/use-cases.js"; +import { createPostgresWakeQueueAdapter } from "../../modules/wake-queue/adapters/postgres.js"; + +describe("native completed failure diagnostics", () => { + let temporary: Awaited>; + let db: ReturnType; + const companyId = randomUUID(); + const agentId = randomUUID(); + const workerId = randomUUID(); + const model = "test-model"; + + beforeAll(async () => { + temporary = await startEmbeddedPostgresTestDatabase("native-model-failure-"); + db = createDb(temporary.connectionString); + await db.insert(companies).values({ id: companyId, name: "Native failure regression", issuePrefix: "NMF" }); + await db.insert(agents).values({ id: agentId, companyId, name: "Native failure agent", status: "active", adapterType: "paperclip_runner", adapterConfig: { provider: "codex", model } }); + await db.insert(agents).values({ id: workerId, companyId, name: "Review source worker", status: "active", adapterType: "paperclip_runner" }); + }); + afterAll(async () => { await temporary?.cleanup(); }); + + it.each(["committed", "duplicate", "pending-review", "superseded-review", "resolved-review", "reassigned", "successor-race", "different-turn", "unknown-error", "untrusted-source", "untrusted-run", "untrusted-session", "succeeded", "cancelled"] as const)( + "preserves accepted results and only labels the matching provider rejection (%s)", async (scenario) => { + const issueId = randomUUID(); + const runId = randomUUID(); + const [issue] = await db.insert(issues).values({ id: issueId, companyId, title: "Explain a provider rejection", status: "in_progress", workMode: "standard", assigneeAgentId: agentId }).returning(); + const [run] = await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native", nativeIssueId: issueId, invocationSource: "assignment", triggerDetail: "system", contextSnapshot: { issueId } }).returning(); + await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId)); + const native = await prepareNativeHeartbeatRun({ db, run, issue, environmentLeaseId: randomUUID() }); + const [contract] = await db.select().from(completionContracts).where(eq(completionContracts.issueId, issueId)); + const completionInput = buildNativeCompletionContract(issue, { revision: contract.revision }); + await db.insert(nativeRunFinalizations).values({ runId, companyId, issueId, phase: "observed" }); + const execution: NativeExecutionInputV1 = { + schema: "paperclip.native-execution-input.v1", + binding: { companyId, runId, issueId, agentId, executionWorkspaceId: runId }, + provider: { kind: "codex", model }, + task: { identifier: issueId, title: issue.title, description: null, prompt: "Reply to the current message.", workMode: "standard" }, + workspace: { cwd: "/tmp", repoUrl: null, repoRef: null, branchName: null }, + session: { normalizedSessionId: native.normalizedSessionId, driverKind: "codex_app_server", protocolVersion: 1, lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null } }, + completionContract: { id: contract.id, sha256: contract.canonicalSha256, schemaVersion: "paperclip.completion-contract.v1", contract: completionInput }, + interactionResponses: [], credentialBindings: [], + }; + await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeExecutionInput: execution } }).where(eq(heartbeatRuns.id, runId)); + if (scenario.endsWith("-review")) { + const sourceRunId = randomUUID(), sourceResultId = randomUUID(), assessmentId = randomUUID(); + const decisionId = randomUUID(), interactionId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId, agentId: workerId, nativeIssueId: issueId, runtimeMode: "native", status: "succeeded", completionContractId: contract.id, completionContractSha256: contract.canonicalSha256 }); + await db.insert(nativeRunResults).values({ id: sourceResultId, companyId, issueId, runId: sourceRunId, completionContractId: contract.id, serverFingerprint: sourceResultId, schemaStatus: "accepted", resultJson: {}, canonicalSha256: sourceResultId }); + await db.insert(workAssessments).values({ id: assessmentId, companyId, issueId, runId: sourceRunId, contractId: contract.id, resultId: sourceResultId, triggerKind: "native_result", triggerActorCompanyId: companyId, priorIssueStatus: "in_progress", priorStatusVersion: 0, policyVersion: "fixture", assessmentJson: {}, inputDigest: assessmentId }); + const [reviewIssue] = await db.update(issues).set({ status: "in_review", assigneeAgentId: workerId }).where(eq(issues.id, issueId)).returning(); + await db.insert(statusDecisions).values({ id: decisionId, companyId, issueId, runId: sourceRunId, assessmentId, decisionVersion: 1, policyVersion: "fixture", fromStatus: "in_progress", toStatus: "in_review", reasonCode: "explicit_review", decisionJson: { projectedStatusVersion: reviewIssue.statusVersion }, decisionDigest: decisionId, applicationState: "applied" }); + await db.update(issues).set({ lastStatusDecisionId: decisionId }).where(eq(issues.id, issueId)); + await db.insert(issueThreadInteractions).values({ id: interactionId, companyId, issueId, kind: "request_confirmation", sourceRunId, addresseeAgentId: agentId, createdByAgentId: workerId, + continuationPolicy: "wake_assignee", + status: scenario === "resolved-review" ? "accepted" : "pending", + ...(scenario === "resolved-review" ? { resolvedByRunId: runId, resolvedByAgentId: agentId } : {}), + payload: { version: 1, prompt: "Review the completed work.", target: { type: "custom", key: "native_completion_review", revisionId: decisionId } }, + }); + const contextSnapshot = { issueId, wakeReason: "native_completion_review", nativeReviewInteractionId: interactionId, nativeReviewDecisionId: decisionId }; + await db.update(heartbeatRuns).set({ contextSnapshot }).where(eq(heartbeatRuns.id, runId)); + if (scenario === "superseded-review") await db.update(issues).set({ lastStatusDecisionId: null }).where(eq(issues.id, issueId)); + const review = await getNativeReviewAssignment(db, { companyId, issueId, agentId, contextSnapshot, allowResolvedByRunId: runId }); + if (scenario === "superseded-review") expect(review).toBeNull(); + else expect(review?.interaction.status).toBe(scenario === "resolved-review" ? "accepted" : "pending"); + } + const event: PrpEvent = { + schema: "paperclip.prp.event.v1", runId, normalizedSessionId: native.normalizedSessionId, + turnId: native.turnId, sourceInstanceId: native.runnerInstanceId, sourceEventId: "model-terminal", sourceSeq: 1, + sourceKind: "runner", eventType: "turn.failed", schemaVersion: 1, priority: 0, emittedAt: new Date().toISOString(), + payload: { status: "failed", error: { codexErrorInfo: "other", message: JSON.stringify({ + type: "error", status: 400, error: { type: "invalid_request_error", message: scenario === "unknown-error" ? "Private unknown failure" : `The '${model}' model is not supported when using Codex with a ChatGPT account.` }, + }) } }, + }; + const openInput = { identity: { companyId, issueId, runId, agentId, sessionId: native.normalizedSessionId }, backendKind: "mock" as const, sourceInstanceId: native.runnerInstanceId }; + if (scenario === "duplicate") { + const prior = new PaperclipControlPlanePort(db, { companyId, issueId, runId, agentId, sessionId: native.normalizedSessionId, sourceInstanceId: native.runnerInstanceId, controlPlaneSourceInstanceId: "prior-controller", completionContractId: contract.id, completionContractSha256: contract.canonicalSha256 }); + await prior.openRun(openInput); + expect((await prior.appendEvent(event)).disposition).toBe("committed"); + } + const turnId = scenario === "different-turn" ? "later-turn" : native.turnId; + const terminal: PrpTerminalState = { + schema: "paperclip.prp.terminal.v1", turnTerminalState: scenario === "succeeded" ? "completed" : scenario === "cancelled" ? "cancelled" : "failed", + runTerminalState: scenario === "succeeded" ? "succeeded" : scenario === "cancelled" ? "cancelled" : "failed", reportedWorkDisposition: "blocked", + }; + const result = { ...CONTROL_PLANE_CONFORMANCE_RESULT, reportedWorkDisposition: "blocked" as const, + blocker: { reasonCode: "provider_rejected", owner: { kind: "user" as const, name: "Task owner" }, unblockAction: "Select a compatible connection.", scope: "task_wide" as const }, + completionClaim: { ...CONTROL_PLANE_CONFORMANCE_RESULT.completionClaim, contractRevision: completionInput.revision } }; + provider.execute.mockReset().mockImplementation(async (options: { controlPlane: ControlPlanePort }) => { + await options.controlPlane.openRun(openInput); + if (scenario === "untrusted-source") { + await expect(options.controlPlane.appendEvent({ ...event, sourceInstanceId: "wrong-runner" })).rejects.toThrow("native_event_source_binding_mismatch"); + } else if (scenario === "untrusted-run" || scenario === "untrusted-session") { + await expect(options.controlPlane.appendEvent({ ...event, + ...(scenario === "untrusted-run" ? { runId: randomUUID() } : { normalizedSessionId: "wrong-session" }), + })).rejects.toThrow("native_event_binding_mismatch"); + } else { + expect((await options.controlPlane.appendEvent(event)).disposition).toBe(scenario === "duplicate" ? "duplicate" : "committed"); + } + await options.controlPlane.completeRun({ result, terminal, turnId }); + return { result, terminal, turnId, normalizedSessionId: native.normalizedSessionId, providerSessionId: "fixture-provider-session", driverKind: "codex_app_server", driverVersion: "fixture", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null }; + }); + const adapter = await executePaperclipNativeSession({ db, execution, runnerInstanceId: native.runnerInstanceId, backend: { + descriptor: async () => { throw new Error("Unexpected provider call"); }, + openSession: async () => { throw new Error("Unexpected provider call"); }, + } }); + const classified = scenario === "committed" || scenario === "duplicate" || scenario.endsWith("-review") || scenario === "reassigned" || scenario === "successor-race"; + const reviewAuthorityEnded = scenario === "superseded-review" || scenario === "resolved-review"; + const ownerChanged = scenario === "reassigned" || scenario === "successor-race"; + expect(adapter.errorCode).toBe(classified ? "native_provider_model_rejected" : undefined); + expect(adapter.errorMessage).toBe(classified ? NATIVE_MODEL_REJECTION_MESSAGE : scenario === "succeeded" ? null : `Native session ${terminal.runTerminalState}`); + expect(adapter.resultJson?.nativeProviderFailure).toEqual(classified ? NATIVE_MODEL_REJECTION_DIAGNOSTIC : undefined); + expect(adapter.nativeFinalization).toMatchObject({ result, terminal, turnId, workspaceFinalizeStatus: "pending" }); + expect(provider.execute).toHaveBeenCalledTimes(1); + expect(execution.provider).toEqual({ kind: "codex", model }); + const [coordinator] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, runId)); + expect(coordinator).toMatchObject({ phase: "workspace_finalizing", leaseOwner: null, attempt: 1, nextAttemptAt: null }); + expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId))).toHaveLength(1); + expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(0); + expect(await db.select().from(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, runId))).toHaveLength(scenario.startsWith("untrusted-") ? 0 : 1); + const diagnostics = collectRunFailureDiagnostics({ ...run, resultJson: adapter.resultJson ?? null }, {}); + expect(diagnostics.provider).toEqual(classified ? NATIVE_MODEL_REJECTION_DIAGNOSTIC : {}); + expect(JSON.stringify(diagnostics)).not.toContain(model); + expect(JSON.stringify(diagnostics)).not.toContain("Private"); + const [persistedRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId)); + expect(persistedRun.errorCode).toBeNull(); // No adapter metadata has been saved yet. + expect(await readPersistedNativeModelRejection(db, persistedRun, turnId, terminal)).toEqual(classified ? { + errorCode: "native_provider_model_rejected", errorMessage: NATIVE_MODEL_REJECTION_MESSAGE, diagnostic: NATIVE_MODEL_REJECTION_DIAGNOSTIC, + } : null); + if (classified) { + if (scenario === "committed") { + // Neither saved display diagnostics nor a different execution binding + // can confer the right to change the finalizer's retry decision. + for (const changed of [ + { ...persistedRun, runnerProfileJson: null }, + { ...persistedRun, nativeSessionId: "foreign-session" }, + { ...persistedRun, runnerInstanceId: "foreign-source" }, + { ...persistedRun, companyId: randomUUID() }, + { ...persistedRun, completionContractSha256: "foreign-contract" }, + { ...persistedRun, runnerProfileJson: { nativeExecutionInput: { ...execution, provider: { kind: "codex", model: "different-model" } } } }, + { ...persistedRun, runnerProfileJson: { nativeExecutionInput: { ...execution, binding: { ...execution.binding, runId: randomUUID() } } } }, + ]) expect(await readPersistedNativeModelRejection(db, changed, turnId, terminal)).toBeNull(); + const [stored] = await db.select().from(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, runId)); + await db.update(heartbeatRunEvents).set({ sourcePayloadSha256: "invalid-hash" }).where(eq(heartbeatRunEvents.id, stored.id)); + expect(await readPersistedNativeModelRejection(db, persistedRun, turnId, terminal)).toBeNull(); + await db.update(heartbeatRunEvents).set({ sourcePayloadSha256: stored.sourcePayloadSha256 }).where(eq(heartbeatRunEvents.id, stored.id)); + } + // Simulate a restart after native result acceptance, before heartbeat's + // adapter update. Finalization must use durable provider evidence. + const acceptedBefore = await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId)); + if (scenario === "reassigned") { + await db.update(issues).set({ assigneeAgentId: workerId, executionRunId: null }).where(eq(issues.id, issueId)); + } else if (scenario === "successor-race") { + const [snapshot] = await db.select().from(issues).where(eq(issues.id, issueId)); + const newerRunId = randomUUID(), raceAssessmentId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: newerRunId, companyId, agentId, status: "running", nativeIssueId: issueId }); + await db.update(issues).set({ executionRunId: newerRunId }).where(eq(issues.id, issueId)); + const [changed] = await db.select().from(issues).where(eq(issues.id, issueId)); + expect(changed.statusVersion).toBe(snapshot.statusVersion); + await db.insert(workAssessments).values({ id: raceAssessmentId, companyId, issueId, runId, contractId: contract.id, resultId: acceptedBefore[0].id, triggerKind: "native_result", triggerActorCompanyId: companyId, priorIssueStatus: snapshot.status, priorStatusVersion: snapshot.statusVersion, policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, assessmentJson: {}, inputDigest: raceAssessmentId }); + await expect(commitNativeStatusDecision({ db, companyId, issueId, runId, assessmentId: raceAssessmentId, + priorStatus: snapshot.status, priorStatusVersion: snapshot.statusVersion, priorDecisionId: snapshot.lastStatusDecisionId, + requireModelRejectionOwner: { agentId, reviewContext: null }, + decision: { policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, statusAction: "blocked", toStatus: "blocked", reasonCode: "native_provider_model_rejected", unblockDescriptor: { owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }, effects: [{ kind: "bind_blocker", owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }] }, + })).rejects.toBeInstanceOf(NativeStatusRaceError); + expect(await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId))).toHaveLength(0); + } + for (let retry = 0; retry < 2; retry++) { + await finalizeNativeRun({ db, runId, workspaceFinalizeStatus: "succeeded", projectRunStatus: true }); + } + expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId))).toEqual(acceptedBefore); + const [finalRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId)); + expect(finalRun).toMatchObject({ status: "failed", errorCode: "native_provider_model_rejected", error: NATIVE_MODEL_REJECTION_MESSAGE }); + const [finalIssue] = await db.select().from(issues).where(eq(issues.id, issueId)); + expect(finalIssue.status).toBe(scenario.endsWith("-review") ? "in_review" : ownerChanged ? "in_progress" : "blocked"); + const decisions = await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId)); + expect(decisions).toHaveLength(1); + expect(decisions[0].reasonCode).toBe(reviewAuthorityEnded ? "native_review_action_finished" : ownerChanged ? "run_failed_partial_evidence_preserved" : "native_provider_model_rejected"); + const wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId)); + expect(wakes.filter(wake => wake.payload?.issueId === issueId)).toHaveLength(ownerChanged ? 1 : 0); + if (!ownerChanged) expect(await heartbeatService(db).dispatchPendingNativeStatusWakeups({ companyId })).toEqual({ scanned: 0, dispatched: 0, recovered: 0, deferred: 0 }); + expect(provider.execute).toHaveBeenCalledTimes(1); + if (scenario === "pending-review") { + const reviewContext = readNativeReviewAssignmentContext(finalRun.contextSnapshot)!; + expect(finalIssue).toMatchObject({ lastStatusDecisionId: reviewContext.nativeReviewDecisionId, executionRunId: null, checkoutRunId: null }); + const originalReview = await getNativeReviewAssignment(db, { companyId, issueId, agentId, contextSnapshot: reviewContext }); + expect(originalReview?.interaction.status).toBe("pending"); + expect(finalIssue.statusVersion).toBe(originalReview!.sourceDecision.decisionJson.projectedStatusVersion); + expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(0); + + // Exercise the real release and periodic recovery lanes, rather than + // assuming that absence of a finalizer wake prevents later retries. + const unexpected = vi.fn(async () => { throw new Error("Unexpected automatic recovery"); }); + const release = createReleaseIssueExecution({ + issueLock: createPostgresWakeQueueAdapter(db, { resolveResponsibleUserId: unexpected, getRoutineEnv: unexpected, resolveSessionBeforeForWakeup: unexpected }), + recovery: { escalateStrandedAssignedIssue: unexpected, escalateStrandedRecoveryIssueInPlace: unexpected }, + }); + expect((await release({ companyId, runId, now: new Date() })).outcome.kind).toBe("released"); + await heartbeatService(db).reconcileStrandedAssignedIssues(); + expect(unexpected).not.toHaveBeenCalled(); + expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(0); + expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId))).filter(wake => wake.payload?.issueId === issueId)).toHaveLength(0); + const [afterRecovery] = await db.select().from(issues).where(eq(issues.id, issueId)); + expect(afterRecovery).toMatchObject({ status: "in_review", statusVersion: finalIssue.statusVersion, lastStatusDecisionId: finalIssue.lastStatusDecisionId }); + + // The operator repairs configuration and explicitly retries the + // assigned reviewer. No review rebinding or status reset is needed. + await db.update(agents).set({ adapterConfig: { provider: "codex", model: "repaired-model" }, runtimeConfig: { heartbeat: { wakeOnDemand: true, maxConcurrentRuns: 1 } } }).where(eq(agents.id, agentId)); + // Occupy the execution slot to exercise real wake admission without + // starting a provider. The normal atomic review claim follows below. + const occupiedRunId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: occupiedRunId, companyId, agentId, status: "running", startedAt: new Date() }); + const retryRun = await heartbeatService(db, { runtimeEnv: {} }).wakeup(agentId, { + failedRunId: runId, source: "on_demand", triggerDetail: "manual", reason: "retry_failed_run", + requestedByActorType: "user", requestedByActorId: "review-repair-operator", + payload: { issueId }, contextSnapshot: { ...reviewContext }, + }); + expect(retryRun).toMatchObject({ status: "queued", retryOfRunId: runId, contextSnapshot: { issueId, ...reviewContext } }); + const retryRunId = retryRun!.id; + await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date() }).where(eq(heartbeatRuns.id, occupiedRunId)); + expect(await claimQueuedNativeReviewRun(db, { run: retryRun!, agentNameKey: "reviewer", claimedAt: new Date(), claimValues: { nativeIssueId: issueId, runtimeMode: "native" } })) + .toMatchObject({ id: retryRunId, status: "running" }); + expect(await getNativeReviewAssignment(db, { companyId, issueId, agentId, contextSnapshot: reviewContext, actingRunId: retryRunId, issueExecutionRunId: retryRunId })).not.toBeNull(); + await issueThreadInteractionService(db).acceptInteraction( + { id: issueId, companyId, projectId: null, goalId: null, status: "in_review" }, + reviewContext.nativeReviewInteractionId, {}, { agentId, runId: retryRunId }, + ); + const [resolved] = await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, reviewContext.nativeReviewInteractionId)); + expect(resolved).toMatchObject({ status: "accepted", resolvedByAgentId: agentId, resolvedByRunId: retryRunId }); + expect(await db.select().from(issues).where(eq(issues.id, issueId))).toMatchObject([{ status: "done", assigneeAgentId: workerId }]); + expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId))).toEqual(acceptedBefore); + expect(provider.execute).toHaveBeenCalledTimes(1); + await db.update(agents).set({ adapterConfig: { provider: "codex", model } }).where(eq(agents.id, agentId)); + } + } else if (scenario === "unknown-error") { + // A lookalike diagnostic flag does not alter unknown failed-result policy. + await db.update(heartbeatRuns).set({ resultJson: { nativeProviderFailure: NATIVE_MODEL_REJECTION_DIAGNOSTIC } }).where(eq(heartbeatRuns.id, runId)); + await finalizeNativeRun({ db, runId, workspaceFinalizeStatus: "succeeded" }); + const [decision] = await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId)); + expect(decision.reasonCode).toBe("run_failed_partial_evidence_preserved"); + const wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId)); + expect(wakes.filter(wake => wake.payload?.issueId === issueId)).toHaveLength(1); + } + }, + ); +}); diff --git a/server/src/services/native-runtime/native-provider-failure.test.ts b/server/src/services/native-runtime/native-provider-failure.test.ts new file mode 100644 index 0000000000..5db2942e4c --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure.test.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from "vitest"; +import type { PrpEvent, PrpTerminalState } from "../../vendor/paperclip-runner/index.js"; +import { createNativeProviderFailureObservation, NATIVE_MODEL_REJECTION_DIAGNOSTIC, NATIVE_MODEL_REJECTION_MESSAGE } from "./native-provider-failure.js"; + +const model = "test-model"; +const response = { + type: "error", status: 400, + error: { type: "invalid_request_error", message: `The '${model}' model is not supported when using Codex with a ChatGPT account.` }, +}; +const event: PrpEvent = { + schema: "paperclip.prp.event.v1", runId: "run", normalizedSessionId: "session", turnId: "turn", + sourceInstanceId: "runner", sourceEventId: "failure", sourceSeq: 1, sourceKind: "runner", + eventType: "turn.failed", schemaVersion: 1, priority: 0, emittedAt: "2026-01-01T00:00:00.000Z", + payload: { status: "failed", error: { message: JSON.stringify(response), codexErrorInfo: "other" } }, +}; +const terminal: PrpTerminalState = { + schema: "paperclip.prp.terminal.v1", turnTerminalState: "failed", runTerminalState: "failed", reportedWorkDisposition: "blocked", +}; +const observation = () => createNativeProviderFailureObservation({ kind: "codex", model }); + +describe("native provider failure observation", () => { + it("binds the structured rejection to the failed turn and emits no response or model text", () => { + const observed = observation(); + observed.observe(event); + observed.observe(event); // A byte-identical committed replay remains idempotent. + const result = observed.forTerminal("turn", terminal); + expect(result).toEqual({ errorCode: "native_provider_model_rejected", errorMessage: NATIVE_MODEL_REJECTION_MESSAGE, diagnostic: NATIVE_MODEL_REJECTION_DIAGNOSTIC }); + expect(JSON.stringify(result)).not.toContain(model); + expect(JSON.stringify(result)).not.toContain("invalid_request_error"); + expect(observed.forTerminal("later-turn", terminal)).toBeNull(); + expect(observed.forTerminal(null, terminal)).toBeNull(); + expect(observed.forTerminal("turn", { ...terminal, runTerminalState: "succeeded" })).toBeNull(); + expect(observed.forTerminal("turn", { ...terminal, turnTerminalState: "cancelled" })).toBeNull(); + }); + + it.each([ + { ...event, sourceKind: "control_plane" as const }, + { ...event, eventType: "item.completed" as const }, + { ...event, eventType: "harness.diagnostic" as const }, + { ...event, turnId: undefined }, + { ...event, payload: { ...event.payload, status: "completed" } }, + { ...event, payload: { ...event.payload, recoverable: true } }, + { ...event, payload: { status: "failed", error: { message: JSON.stringify(response), recoverable: true } } }, + ])("ignores non-authoritative or retryable evidence %#", (unrelated) => { + const observed = observation(); + observed.observe(unrelated); + expect(observed.forTerminal("turn", terminal)).toBeNull(); + }); + + it.each([ + response.error.message, "malformed JSON", " ".repeat(4_097), + JSON.stringify({ ...response, status: 401 }), + JSON.stringify({ ...response, status: "400" }), + JSON.stringify({ ...response, type: "notice" }), + JSON.stringify({ ...response, error: { ...response.error, type: "server_error" } }), + JSON.stringify({ ...response, error: { ...response.error, message: "Private arbitrary provider text" } }), + JSON.stringify({ ...response, error: { ...response.error, message: response.error.message + " Private suffix" } }), + JSON.stringify({ ...response, error: { ...response.error, message: response.error.message.replace(model, "another-model") } }), + JSON.stringify([response]), "null", + ])("leaves unknown failures unchanged %#", (message) => { + const observed = observation(); + observed.observe({ ...event, payload: { status: "failed", error: { message } } }); + expect(observed.forTerminal("turn", terminal)).toBeNull(); + }); + + it.each([{ kind: "claude", model }, { kind: "codex" }, { kind: "codex", model: "other" }])( + "does not apply a rejection to another selected provider/model (%j)", (provider) => { + const observed = createNativeProviderFailureObservation(provider); + observed.observe(event); + expect(observed.forTerminal("turn", terminal)).toBeNull(); + }, + ); +}); diff --git a/server/src/services/native-runtime/native-provider-failure.ts b/server/src/services/native-runtime/native-provider-failure.ts new file mode 100644 index 0000000000..ec7380ffc8 --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure.ts @@ -0,0 +1,60 @@ +import type { PrpEvent, PrpTerminalState } from "../../vendor/paperclip-runner/index.js"; + +export const NATIVE_MODEL_REJECTION_MESSAGE = + "The selected model is not supported by the current ChatGPT connection. Choose a supported model or a compatible AI connection."; + +export const NATIVE_MODEL_REJECTION_DIAGNOSTIC = { + provider: "codex", + category: "model_auth_incompatible", + status: 400, + authMode: "chatgpt", +} as const; + +function record(value: unknown): Record { + return value !== null && typeof value === "object" && !Array.isArray(value) + ? value as Record + : {}; +} + +/** Reconstruct only the closed diagnostic vocabulary, never provider text. */ +export function readNativeModelRejectionDiagnostic(value: unknown) { + try { + const entry = record(value); + return Object.entries(NATIVE_MODEL_REJECTION_DIAGNOSTIC).every(([key, expected]) => entry[key] === expected) + ? { ...NATIVE_MODEL_REJECTION_DIAGNOSTIC } + : null; + } catch { return null; } +} + +/** Observe only events accepted by the bound control-plane persistence port. */ +export function createNativeProviderFailureObservation(provider: { kind: string; model?: string | null }) { + let rejectedTurnId: string | null = null; + return { + observe(event: PrpEvent) { + if (provider.kind !== "codex" || !provider.model || + event.sourceKind !== "runner" || event.eventType !== "turn.failed" || !event.turnId) return; + const payload = record(event.payload); + const failure = record(payload.error); + if (payload.status !== "failed" || payload.recoverable === true || failure.recoverable === true || + typeof failure.message !== "string" || failure.message.length > 4_096) return; + let response: Record; + try { response = record(JSON.parse(failure.message)); } catch { return; } + const error = record(response.error); + // This is a provider terminal envelope, not text emitted by a tool or an + // agent. Bind the exact rejection to the configured model without storing + // its name or the response. Unknown provider errors retain their behavior. + if (response.type !== "error" || response.status !== 400 || error.type !== "invalid_request_error" || + error.message !== `The '${provider.model}' model is not supported when using Codex with a ChatGPT account.`) return; + rejectedTurnId = event.turnId; + }, + forTerminal(turnId: string | null | undefined, terminal: PrpTerminalState) { + if (!rejectedTurnId || turnId !== rejectedTurnId || + terminal.turnTerminalState !== "failed" || terminal.runTerminalState !== "failed") return null; + return { + errorCode: "native_provider_model_rejected" as const, + errorMessage: NATIVE_MODEL_REJECTION_MESSAGE, + diagnostic: { ...NATIVE_MODEL_REJECTION_DIAGNOSTIC }, + }; + }, + }; +} diff --git a/server/src/services/native-runtime/native-run-finalizer.ts b/server/src/services/native-runtime/native-run-finalizer.ts index b6aea881b4..fb248551b9 100644 --- a/server/src/services/native-runtime/native-run-finalizer.ts +++ b/server/src/services/native-runtime/native-run-finalizer.ts @@ -1,4 +1,5 @@ import { settleSlackConversation } from "../slack-conversation-lifecycle.js"; +import { readPersistedNativeModelRejection } from "./native-provider-failure-evidence.js"; import { dismissAutomaticCompletionReviews } from "./automatic-completion-reviews.js"; import { getNativeReviewAssignment, readNativeReviewAssignmentContext } from "./native-review-participant.js"; import { conversationNativeDecision, isConversation } from "../agent-conversations.js"; @@ -24,7 +25,7 @@ import { workspaceOperations, } from "@paperclipai/db"; import { classifyNativeEvidence } from "./evidence-classifier.js"; -import type { PrpIgnoredAttentionRequest } from "@paperclipai/paperclip-runner"; +import type { PrpIgnoredAttentionRequest, PrpTerminalState } from "@paperclipai/paperclip-runner"; import { arbitrateNativeStatus, NATIVE_STATUS_ARBITER_POLICY_VERSION, @@ -1275,7 +1276,15 @@ export async function finalizeNativeRun(input: { companyId: run.companyId, issueId: authoritativeIssue.id, agentId: run.agentId, contextSnapshot: reviewContext, allowResolvedByRunId: run.id, }) : null; + const providerModelRejection = await readPersistedNativeModelRejection( + input.db, run, resultRow.turnId, envelope.terminal as PrpTerminalState, + ); + const ownsModelRejectionDecision = + (!authoritativeIssue.executionRunId || authoritativeIssue.executionRunId === run.id) && + (reviewContext ? nativeReview?.interaction.status === "pending" + : authoritativeIssue.assigneeAgentId === run.agentId && !authoritativeIssue.assigneeUserId); const proposedDecision = resolveNativeFinalizerStatus({ + providerModelRejected: providerModelRejection !== null && ownsModelRejectionDecision, ...(reviewContext ? { nativeReviewOutcome: nativeReview ? nativeReview.interaction.status === "pending" ? "pending" as const : "resolved" as const : "stale" as const } : {}), @@ -1358,6 +1367,9 @@ export async function finalizeNativeRun(input: { priorStatusVersion: Number(authoritativeIssue.statusVersion), priorDecisionId: authoritativeIssue.lastStatusDecisionId, decision, + requireModelRejectionOwner: decision.reasonCode === "native_provider_model_rejected" + ? { agentId: run.agentId, reviewContext } + : undefined, requireBoardResponseWaitSource: decision.reasonCode === "board_response_waiting" || repairBoardResponseWait ? boardResponseWait?.source @@ -1400,6 +1412,7 @@ export async function finalizeNativeRun(input: { ); const clearExecutionFailure = input.projectRunStatus && finalizationPhase === "committed" && terminalState === "succeeded"; const finalizationMetadata = { + ...(providerModelRejection ? { nativeProviderFailure: providerModelRejection.diagnostic } : {}), finalizationPhase, ...(finalizationPhase === "committed" ? { failureCode: null, originalFailureCode: null, nextAttemptAt: null } : {}), assessmentId: assessmentRow.id, @@ -1450,6 +1463,12 @@ export async function finalizeNativeRun(input: { nativePhase: finalizationPhase, nativePhaseUpdatedAt: now, ...(clearExecutionFailure ? { error: null, errorCode: null } : {}), + ...(providerModelRejection ? { + // Recovery can finalize before heartbeat saves the adapter result. + // Retain any independently recorded execution/cleanup failure. + error: sql`coalesce(${heartbeatRuns.error}, ${providerModelRejection.errorMessage})`, + errorCode: sql`coalesce(${heartbeatRuns.errorCode}, ${providerModelRejection.errorCode})`, + } : {}), resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || ${JSON.stringify(finalizationMetadata)}::jsonb || ${clearExecutionFailure ? recoveredExecutionFailureMetadata() : sql`'{}'::jsonb`}`, diff --git a/server/src/services/native-runtime/native-session-executor.ts b/server/src/services/native-runtime/native-session-executor.ts index cc217b9452..ec28b68800 100644 --- a/server/src/services/native-runtime/native-session-executor.ts +++ b/server/src/services/native-runtime/native-session-executor.ts @@ -9,6 +9,7 @@ import { REMOTE_CODEX_SUPPORTED_RANGE, } from "./codex-runtime-compatibility.js"; import { createNativeToolTrace, type NativeToolTrace } from "./native-tool-trace.js"; +import { createNativeProviderFailureObservation } from "./native-provider-failure.js"; import { createNativeGitHubAccess, type NativeGitHubAccess } from "./native-github-access.js"; import { resolveGitHubOperationCredentials } from "../github-operation-credentials.js"; import { bindManagedNativeCredentialTurn, completeManagedNativeCredentialTurn } from "./managed-native-credentials.js"; @@ -7805,6 +7806,7 @@ async function executePaperclipNativeSessionWithinScope( let turnStartedAtMs: number | null = null; let firstAgentEventRecorded = false; let providerUsageLimitObserved = false; + const providerFailureObservation = createNativeProviderFailureObservation(input.execution.provider); let turnCompletedAtMs: number | null = null; let runnerSessionStartupScope: NativeRunSpanScope | null = null; let agentTurnScope: NativeRunSpanScope | null = null; @@ -7861,6 +7863,7 @@ async function executePaperclipNativeSessionWithinScope( await liveQuestions.observe(event); await projectSessionGoalEvent(event); providerUsageLimitObserved ||= nativeProviderUsageLimitFromEvent(event); + providerFailureObservation.observe(event); const eventAtMs = Date.parse(event.emittedAt); const milestoneAtMs = Number.isFinite(eventAtMs) ? eventAtMs @@ -8064,6 +8067,7 @@ async function executePaperclipNativeSessionWithinScope( await liveQuestions.observe(event); await projectSessionGoalEvent(event); providerUsageLimitObserved ||= nativeProviderUsageLimitFromEvent(event); + providerFailureObservation.observe(event); const questionFallback = await materializeRuntimeQuestionFallback({ db: input.db, binding: input.execution.binding, @@ -9203,6 +9207,7 @@ async function executePaperclipNativeSessionWithinScope( ); if (collectInstructions && !collectedByOwner) await instructionCopy!.collectStopped(); } + const providerFailure = providerFailureObservation.forTerminal(native.turnId, native.terminal); const adapterResult: AdapterExecutionResult = { exitCode: native.terminal.runTerminalState === "succeeded" ? 0 : 1, signal: null, @@ -9210,10 +9215,12 @@ async function executePaperclipNativeSessionWithinScope( errorMessage: native.terminal.runTerminalState === "succeeded" ? null - : `Native session ${native.terminal.runTerminalState}`, + : providerFailure?.errorMessage ?? `Native session ${native.terminal.runTerminalState}`, + ...(providerFailure ? { errorCode: providerFailure.errorCode } : {}), resultJson: { nativeResult: native.result as unknown as Record, nativeTerminal: native.terminal as unknown as Record, + ...(providerFailure ? { nativeProviderFailure: providerFailure.diagnostic } : {}), ...(native.goalRolloverRequired ? { goalRolloverRequired: true } : {}), planSynchronizations, }, diff --git a/server/src/services/native-runtime/status-arbiter.test.ts b/server/src/services/native-runtime/status-arbiter.test.ts index 0f3b1abad9..4162548217 100644 --- a/server/src/services/native-runtime/status-arbiter.test.ts +++ b/server/src/services/native-runtime/status-arbiter.test.ts @@ -44,6 +44,23 @@ function arbitrate( } describe("native status authority", () => { + it.each(["in_progress", "in_review"] as const)("blocks a current worker's proven model rejection without a retry (%s)", (priorIssueStatus) => { + const decision = arbitrate({ priorIssueStatus, terminalState: "failed", providerModelRejected: true }); + expect(decision).toMatchObject({ statusAction: "blocked", toStatus: "blocked", reasonCode: "native_provider_model_rejected", unblockDescriptor: { owner: "board" } }); + expect(decision.effects).toEqual([{ kind: "bind_blocker", owner: "board", action: expect.any(String) }]); + expect(arbitrate({ terminalState: "failed", providerModelRejected: false }).effects).toContainEqual(expect.objectContaining({ kind: "schedule_retry" })); + expect(arbitrate({ terminalState: "succeeded", providerModelRejected: true }).reasonCode).not.toBe("native_provider_model_rejected"); + expect(arbitrate({ terminalState: "failed", providerModelRejected: true, workspaceFinalizeStatus: "failed" }).reasonCode).toBe("finalization_failed_claim_preserved"); + expect(arbitrate({ terminalState: "failed", providerModelRejected: true, priorIssueStatus: "done" }).toStatus).toBe("done"); + for (const nativeReviewOutcome of ["stale", "resolved"] as const) { + expect(arbitrate({ terminalState: "failed", providerModelRejected: true, priorIssueStatus, nativeReviewOutcome })) + .toMatchObject({ statusAction: "preserve", toStatus: priorIssueStatus, reasonCode: "native_review_action_finished" }); + } + }); + it("preserves pending review authority without automatic recovery after model rejection", () => { + expect(arbitrate({ priorIssueStatus: "in_review", terminalState: "failed", providerModelRejected: true, nativeReviewOutcome: "pending" })) + .toMatchObject({ statusAction: "preserve", toStatus: "in_review", reasonCode: "native_provider_model_rejected", unblockDescriptor: null, effects: [{ kind: "release_checkout" }] }); + }); it("a reviewer finishes its decision without completing rejected or still-reviewed work", () => { for (const priorIssueStatus of ["in_progress", "in_review"] as const) { const decision = arbitrate({ priorIssueStatus, nativeReviewOutcome: "resolved" }); @@ -511,7 +528,7 @@ describe("native status authority", () => { expect.objectContaining({ statusAction: "blocked", toStatus: "blocked", - policyVersion: "phase6-v7", + policyVersion: "phase6-v8", reasonCode: "current_track_blocker_waiting", unblockDescriptor: { owner: "board", diff --git a/server/src/services/native-runtime/status-arbiter.ts b/server/src/services/native-runtime/status-arbiter.ts index 55e298bf5a..c4235536a6 100644 --- a/server/src/services/native-runtime/status-arbiter.ts +++ b/server/src/services/native-runtime/status-arbiter.ts @@ -1,6 +1,7 @@ import type { NativeEvidenceAssessment } from "./evidence-classifier.js"; +import { NATIVE_MODEL_REJECTION_MESSAGE } from "./native-provider-failure.js"; -export const NATIVE_STATUS_ARBITER_POLICY_VERSION = "phase6-v7"; +export const NATIVE_STATUS_ARBITER_POLICY_VERSION = "phase6-v8"; export type NativeAuthoritativeIssueStatus = | "backlog" @@ -115,6 +116,8 @@ export function arbitrateNativeStatus(input: { reviewOwnerUserId?: string | null; /** Review decisions own task state; a reviewer's finish report cannot override them. */ nativeReviewOutcome?: "resolved" | "pending" | "stale"; + /** Re-derived from the committed runner terminal and pinned execution identity. */ + providerModelRejected?: boolean; agentId: string; priorIssueStatus: NativeAuthoritativeIssueStatus; }): NativeStatusDecision { @@ -146,6 +149,31 @@ export function arbitrateNativeStatus(input: { ], }; } + if (input.providerModelRejected && input.terminalState === "failed" && + input.nativeReviewOutcome !== "stale" && input.nativeReviewOutcome !== "resolved") { + if (input.nativeReviewOutcome === "pending") { + // The pending review is bound to the worker's status decision and + // version. Preserve that authority so an explicit retry after a + // configuration repair can resolve the same review. In particular, + // do not create the normal unresolved-review recovery action. + return { + policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, + statusAction: "preserve", + toStatus: input.priorIssueStatus, + reasonCode: "native_provider_model_rejected", + unblockDescriptor: null, + effects: [{ kind: "release_checkout" }], + }; + } + return { + policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, + statusAction: "blocked", + toStatus: "blocked", + reasonCode: "native_provider_model_rejected", + unblockDescriptor: { owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }, + effects: [{ kind: "bind_blocker", owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }], + }; + } if (input.nativeReviewOutcome) { return { policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, diff --git a/server/src/services/native-runtime/status-decision-committer.ts b/server/src/services/native-runtime/status-decision-committer.ts index 307efb532d..21c14f5e93 100644 --- a/server/src/services/native-runtime/status-decision-committer.ts +++ b/server/src/services/native-runtime/status-decision-committer.ts @@ -24,6 +24,7 @@ import { type NativeStatusEffect, } from "./status-arbiter.js"; import { nativeSha256 } from "./canonical.js"; +import { getNativeReviewAssignment, type NativeReviewAssignmentContext } from "./native-review-participant.js"; import { readNativeBoardResponseWaitSource, readNativeBoardResponseWaitOrigin, @@ -1535,6 +1536,7 @@ export async function commitNativeStatusDecision(input: { preMaterializedEffects?: NativeMaterializedStatusEffect[]; supersedesCommittedDecisionId?: string; requireExternalChatResponseWaitAuthorization?: { agentId: string }; + requireModelRejectionOwner?: { agentId: string; reviewContext: NativeReviewAssignmentContext | null }; requireBoardResponseWaitSource?: NativeBoardResponseWaitSource; requireBoardResponseWaitOrigin?: NativeBoardResponseWaitOrigin; reviewResponsePresentation?: { @@ -1547,6 +1549,9 @@ export async function commitNativeStatusDecision(input: { throw new Error("native_status_reason_code_required"); } const reasonCode = input.decision.reasonCode; + if (reasonCode === "native_provider_model_rejected" && !input.requireModelRejectionOwner) { + throw new Error("native_model_rejection_owner_required"); + } const publications: ActivityPublication[] = []; const terminalRunsToEmit: (typeof heartbeatRuns.$inferSelect)[] = []; const committed = await input.db.transaction(async (tx) => { @@ -1616,6 +1621,20 @@ export async function commitNativeStatusDecision(input: { ) { throw new NativeStatusRaceError(); } + if (input.requireModelRejectionOwner) { + const owner = input.requireModelRejectionOwner; + // A successor can claim execution without changing statusVersion. Keep + // this new blocking authority behind the same locked issue snapshot. + if (issue.executionRunId && issue.executionRunId !== input.runId) throw new NativeStatusRaceError(); + if (owner.reviewContext) { + const review = await getNativeReviewAssignment(tx as unknown as Db, { + companyId: input.companyId, issueId: input.issueId, agentId: owner.agentId, contextSnapshot: owner.reviewContext, + }); + if (review?.interaction.status !== "pending") throw new NativeStatusRaceError(); + } else if (issue.assigneeAgentId !== owner.agentId || issue.assigneeUserId) { + throw new NativeStatusRaceError(); + } + } if (input.requireExternalChatResponseWaitAuthorization) { let authorization; try { diff --git a/server/src/services/recovery/provider-failure-classification.test.ts b/server/src/services/recovery/provider-failure-classification.test.ts index d58bcf42a9..9343e87813 100644 --- a/server/src/services/recovery/provider-failure-classification.test.ts +++ b/server/src/services/recovery/provider-failure-classification.test.ts @@ -7,8 +7,20 @@ import { classifyContinuationFailure, } from "./service.js"; import { legacyExecutionNeedsReconciliation } from "../legacy-execution-recovery.js"; +import { isConfigurationIncompleteFailedRun } from "../../modules/wake-queue/domain/values.js"; describe("classifyAdapterFailureForRecovery", () => { + it("waits for model configuration changes instead of retrying a typed native model rejection", () => { + const run = { errorCode: "native_provider_model_rejected", error: "Fixed model configuration message", resultJson: null }; + expect(classifyAdapterFailureForRecovery(run)).toEqual({ kind: "configuration_incomplete" }); + expect(classifyContinuationFailure(run as never)).toMatchObject({ kind: "non_retryable", maxAttempts: 0 }); + expect(isConfigurationIncompleteFailedRun(run)).toBe(true); + expect(isAiAuthenticationBlocked(run as never)).toBe(false); + const unknown = { ...run, errorCode: "adapter_failed", error: "native_provider_model_rejected" }; + expect(classifyAdapterFailureForRecovery(unknown)).toBeNull(); + expect(classifyContinuationFailure(unknown as never).kind).toBe("transient_infra"); + expect(isConfigurationIncompleteFailedRun(unknown)).toBe(false); + }); it.each(["acpx_auth_required", "claude_auth_required", "codex_auth_required", "adapter_auth_missing", "refresh_token_reused", "authentication_required"])("does not automatically retry provider authentication failure %s", (errorCode) => { const classification = classifyRunLiveness({ runStatus: "failed", issue: null, errorCode, authenticationRepairRequested: true }); const run = { errorCode, ...classification }; diff --git a/server/src/services/recovery/service.ts b/server/src/services/recovery/service.ts index cfda20e5bc..2cb7ac951f 100644 --- a/server/src/services/recovery/service.ts +++ b/server/src/services/recovery/service.ts @@ -491,6 +491,7 @@ const TRANSIENT_INFRA_CONTINUATION_ERROR_CODES = new Set([ ]); const NON_RETRYABLE_CONTINUATION_ERROR_CODES = new Set([ + "native_provider_model_rejected", "provider_tool_definition_invalid", "adapter_engine_unavailable", "agent_not_invokable", @@ -626,7 +627,7 @@ export function classifyAdapterFailureForRecovery( ): AdapterFailureRecoveryClassification { // An engine prerequisite cannot be repaired by asking the same unavailable // engine to retry. Use the existing configuration-blocker path. - if (latestRun.errorCode === "adapter_engine_unavailable" || latestRun.errorCode === "provider_tool_definition_invalid") { + if (latestRun.errorCode === "adapter_engine_unavailable" || latestRun.errorCode === "provider_tool_definition_invalid" || latestRun.errorCode === "native_provider_model_rejected") { return { kind: "configuration_incomplete" }; } if ( diff --git a/server/src/services/run-failure-diagnostics.ts b/server/src/services/run-failure-diagnostics.ts index 5a83450794..c5c9d7e820 100644 --- a/server/src/services/run-failure-diagnostics.ts +++ b/server/src/services/run-failure-diagnostics.ts @@ -5,6 +5,7 @@ import { redactDiagnosticText } from "@paperclipai/adapter-utils/command-redacti import { sanitizeWorkspaceRestoreDiagnostic } from "@paperclipai/adapter-utils/workspace-restore-diagnostics"; import { redactCurrentUserText } from "../log-redaction.js"; import { redactSensitiveText, REDACTED_EVENT_VALUE } from "../redaction.js"; +import { readNativeModelRejectionDiagnostic } from "./native-runtime/native-provider-failure.js"; type Run = typeof heartbeatRuns.$inferSelect; type Context = Record; @@ -184,6 +185,8 @@ export function collectRunFailureDiagnostics(run: Run, options: RunFailureReport "stackPreview", "status", "statusCode", "requestId", ]); const provider = scalars(read(result, "terminalSessionFailure"), ["category", "title", "details"]); + const nativeProviderFailure = readNativeModelRejectionDiagnostic(read(result, "nativeProviderFailure")); + if (nativeProviderFailure) Object.assign(provider, nativeProviderFailure); const truncatedFields: string[] = []; const providerTruncation = read(read(result, "terminalSessionFailure"), "truncatedFields"); if (Array.isArray(providerTruncation)) { From bb73f2fe392696d1470bdd521e734ce975a830dd Mon Sep 17 00:00:00 2001 From: Valentin Palkovic Date: Tue, 6 Oct 2026 07:11:59 +0200 Subject: [PATCH 07/12] feat(exe-dev): copy a source VM with exe.dev cp (#14975) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The exe.dev sandbox provider plugin gives each run its own exe.dev VM > - Today the plugin always creates that VM with `exe.dev new`, so every run starts from a bare image > - Large repositories need a long setup on each VM: toolchain, agent CLIs, package caches, and browsers for tests > - exe.dev has a `cp` command that copies an existing VM, disk and config included > - This pull request adds an optional `sourceVm` setting. When it is set, the plugin copies that VM with `exe.dev cp` instead of creating a new one > - The benefit is that operators prepare one VM once, and each run starts from it ## Linked Issues or Issue Description Refs #13575. That open PR rewrites this plugin for durable exe.dev environments. It does not add `cp`. The two changes touch the same files and can conflict. I found no issue for this. Feature description: **Subsystem affected** packages/plugins: the exe.dev sandbox provider plugin (`packages/plugins/sandbox-providers/exe-dev`). **Problem or motivation** Each run gets a fresh exe.dev VM from `exe.dev new`. We use a large monorepo (Storybook). Before the agent can work, each run must install Node, the agent CLIs, and Playwright browsers. Each run must also fill the package manager cache. This setup takes a long time on each run. **Proposed solution** Add a `sourceVm` setting ("Source VM" in the environment form). When it is set, lease acquisition and probes run `exe.dev cp --json`. The command also sends the configured `cpu`, `memory`, and `disk`. The VM name, the SSH setup, the workspace, and the release and destroy steps do not change. When the setting is blank, the plugin uses `exe.dev new` as before. **Alternatives considered** - A custom image with `--image`: the operator must build and push a large image for each change. A private registry needs `--registry-auth`, and the plugin does not support it. - `--setup-script`: it runs on every new VM, so each run still pays the setup cost. It also has a 10 KiB limit. - `reuseLease`: it keeps one VM for one lease. It does not give each run a fresh copy of a prepared VM. **Roadmap alignment** The change stays inside an existing sandbox provider plugin. `ROADMAP.md` lists "Cloud / Sandbox agent support" as done and does not plan VM copies. CONTRIBUTING.md asks to discuss features in Discord `#dev` first. I open this pull request as a draft and start that discussion in `#dev`. **Additional context** exe.dev documents `cp` here: https://exe.dev/docs/cli-cp. exe.dev token permissions are documented here: https://exe.dev/docs/https-api. ## What Changed - `plugin.ts`: add `sourceVm` to the driver config. - `plugin.ts`: `buildCreateCommand` sends `cp` when `sourceVm` is set. - `plugin.ts`: config validation rejects `sourceVm` together with settings that `cp` cannot apply (`image`, `command`, `comment`, `env`, `integrations`, `tags`, `setupScript`, `prompt`). The error names the settings to clear. The server shows validation errors in the form, but it does not show warnings after a successful save. - `manifest.ts`: add the "Source VM" field to the "VM creation" group. Its description says that the API token must allow `cp`, because exe.dev returns 403 for a command that the token does not list. The API key description now also mentions `cp`. - `README.md`: document `sourceVm`, its limits, and the token permission. - `plugin.test.ts`: add tests for the `cp` command, the validation error, and the form field. Add `sourceVm: null` to the expected normalized config. ## Verification - `vitest run --config vitest.config.ts` in `packages/plugins/sandbox-providers/exe-dev`: 38 tests pass. The three new tests fail without the change. - `tsc --noEmit -p packages/plugins/sandbox-providers/exe-dev`: no errors. - Manual test on a self-hosted Paperclip instance (2026.1001.0). I applied the same change to the installed plugin. I prepared a source VM and set "Source VM" on an exe.dev environment. Then I ran agent tasks. Each run copied the source VM and ran in the copy. Paperclip deleted the copy at release. - With an API token that does not list `cp`, the run fails with `exe.dev API command failed (403) for: cp '' '' --json ...`. The new field description tells operators about this. ## Risks - Low risk. The new code runs only when `sourceVm` is set. The `new` path is unchanged. - `cp` uses the same `/exec` endpoint and its 30-second request limit. A copy of a very large disk can take longer than the limit. - Every copy inherits the source VM disk. The README tells operators not to keep secrets on the source VM. - Can conflict with #13575. ## Model Used - Provider and model: Anthropic Claude Opus 5.5. - Model ID: `claude-opus-5-5`. - Tool: Claude Code, with tool use (shell commands and file edits) and extended thinking. - Context window: the tool does not report it. - The model wrote the change and the tests. A human tested the feature on a real exe.dev setup. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Bildschirmfoto 2026-10-02 um 23 04 32 --------- Co-authored-by: Claude Opus 5.5 --- .../sandbox-providers/exe-dev/README.md | 3 +- .../sandbox-providers/exe-dev/src/manifest.ts | 10 +++- .../exe-dev/src/plugin.test.ts | 58 +++++++++++++++++++ .../sandbox-providers/exe-dev/src/plugin.ts | 32 ++++++++++ 4 files changed, 101 insertions(+), 2 deletions(-) diff --git a/packages/plugins/sandbox-providers/exe-dev/README.md b/packages/plugins/sandbox-providers/exe-dev/README.md index 52d420c25d..f1091e4896 100644 --- a/packages/plugins/sandbox-providers/exe-dev/README.md +++ b/packages/plugins/sandbox-providers/exe-dev/README.md @@ -23,7 +23,7 @@ Configure exe.dev from `Instance Settings -> Environments`, not from the plugin' To use the provider successfully, the environment/host needs all of the following: -- An exe.dev API token that allows the lifecycle commands the provider uses: `new`, `ls`, and `rm`. `whoami` and `help` are recommended for manual debugging. `restart` is only needed if you extend the provider to restart retained VMs. +- An exe.dev API token that allows the lifecycle commands the provider uses: `new`, `ls`, and `rm`, plus `cp` when `sourceVm` is set. exe.dev answers 403 for a command the token does not list. `whoami` and `help` are recommended for manual debugging. `restart` is only needed if you extend the provider to restart retained VMs. - SSH access from the Paperclip host to the resulting `*.exe.xyz` VMs. - An SSH private key that exe.dev already recognizes. You can either: - paste the private key into the environment config via `sshPrivateKey` @@ -37,6 +37,7 @@ Operational notes: - Reusable leases keep the VM alive between runs. exe.dev does not expose a documented "stop and later resume" command in the public CLI docs, so `reuseLease: true` means "retain the VM" rather than "suspend it." - The provisioning path uses `https://exe.dev/exec`, which exe.dev documents as a command-style HTTPS API with a 30-second request timeout. Typical `new` calls are expected to fit inside that limit; command execution itself does not use `/exec`. - Probes still create and delete a real exe.dev VM through `/exec`, and so do the `new`/`rm` calls inside the normal acquire/release lifecycle. Treat all of those as real provisioning cost, not just probes. +- Set `sourceVm` ("Source VM" in the form) to the name of an existing exe.dev VM to copy it for each run with `exe.dev cp`, disk and config included, instead of creating a fresh VM with `exe.dev new`. This lets you prepare one VM with your toolchain and caches and start every run from it. `cp` accepts only the VM name, `cpu`, `memory`, and `disk`, so config validation rejects `sourceVm` together with `image`, `command`, `comment`, `env`, `integrations`, `tags`, `setupScript`, or `prompt`. The default setup script does not run either, so the source VM must already have Node 24.11+ and accept the configured SSH key. Do not keep secrets on the source VM: every copy inherits its disk. `cp` also goes through `/exec`, so a large disk copy must finish inside the same 30-second limit. - exe.dev runs `--setup-script` as the unprivileged `exedev` user, not as root. That user has passwordless `sudo`, so any system-level steps in a custom `setupScript` must invoke `sudo` explicitly (for example `sudo apt-get install -y …`). When you omit `setupScript`, the plugin supplies a default that installs Node 24 via the official nodesource script — Paperclip's sandbox callback bridge is a Node program, so the VM needs `node` on `PATH` before the bridge can launch. ## Local development diff --git a/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts b/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts index bef86f3def..da176d0f96 100644 --- a/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts +++ b/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts @@ -31,7 +31,7 @@ const manifest: PaperclipPluginManifestV1 = { type: "string", format: "secret-ref", description: - "Paste your exe.dev API token, or pick a saved Paperclip secret. Create one at exe.dev → Settings → API tokens with `/exec` scope (`new`, `ls`, `rm`).", + "Paste your exe.dev API token, or pick a saved Paperclip secret. Create one at exe.dev → Settings → API tokens with `/exec` scope (`new`, `ls`, `rm`, plus `cp` if you use Source VM).", }, sshPrivateKey: { type: "string", @@ -99,6 +99,14 @@ const manifest: PaperclipPluginManifestV1 = { "x-paperclip-group": "VM resources", }, // ---- Advanced: VM creation ---- + sourceVm: { + type: "string", + title: "Source VM", + description: + "Name of an existing exe.dev VM to copy for each run with `exe.dev cp`, disk and config included. Leave blank to create a fresh VM with `exe.dev new`. Your API token must allow `cp`: tokens list their permitted commands, and one without `cp` fails with a 403. When set, leave image, command, env, integrations, tags, setup script, prompt, and comment empty: `cp` cannot apply them, and saving fails until they are cleared.", + "x-paperclip-advanced": true, + "x-paperclip-group": "VM creation", + }, command: { type: "string", description: "Optional container command passed to `exe.dev new --command`.", diff --git a/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts b/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts index 16618bf507..912105db85 100644 --- a/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts +++ b/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts @@ -106,6 +106,7 @@ describe("exe.dev sandbox provider plugin", () => { integrations: ["github"], tags: ["prod", "sandbox"], setupScript: null, + sourceVm: null, prompt: null, timeoutMs: 450000, reuseLease: true, @@ -319,6 +320,57 @@ describe("exe.dev sandbox provider plugin", () => { }); }); + it("copies the source VM with exe.dev cp instead of creating a new VM", async () => { + fetchMock.mockResolvedValueOnce( + new Response(JSON.stringify({ + vm_name: "paperclip-env1-run1", + ssh_dest: "paperclip-env1-run1.exe.xyz", + status: "running", + }), { status: 200 }), + ); + queueSpawnResult({ stdout: "/home/exedev\nbash\n" }); + queueSpawnResult({}); + + const lease = await plugin.definition.onEnvironmentAcquireLease?.({ + driverKey: "exe-dev", + companyId: "company-1", + environmentId: "env-1", + runId: "run-1", + config: { + apiKey: "api-key", + sourceVm: " golden-vm ", + image: "ubuntu:22.04", + env: { FOO: "bar" }, + cpu: 4, + memory: "8GB", + disk: "40GB", + }, + }); + + expect(String(fetchMock.mock.calls[0]?.[1]?.body ?? "")).toBe( + "cp 'golden-vm' 'paperclip-env1-run1' --json --cpu='4' --memory='8GB' --disk='40GB'", + ); + expect(lease).toMatchObject({ providerLeaseId: "paperclip-env1-run1" }); + }); + + it("rejects VM creation settings that exe.dev cp cannot apply when sourceVm is set", async () => { + const result = await plugin.definition.onEnvironmentValidateConfig?.({ + driverKey: "exe-dev", + config: { + apiKey: "api-key", + sourceVm: "golden-vm", + image: "ubuntu:22.04", + env: { FOO: "bar" }, + setupScript: "echo hi", + }, + }); + + expect(result?.ok).toBe(false); + expect(result?.errors).toContain( + "sourceVm copies an existing VM with `exe.dev cp`, which cannot apply image, env, setupScript. Clear these settings or clear sourceVm.", + ); + }); + it("uses a pasted sshPrivateKey when connecting to the VM", async () => { fetchMock.mockResolvedValueOnce( new Response(JSON.stringify({ @@ -860,6 +912,12 @@ describe("exe-dev manifest form defaults", () => { expect(properties.disk?.default).toBe("20GB"); }); + it("explains the cp token permission on the Source VM field", () => { + const sourceVm = properties.sourceVm as { title?: string; description?: string } | undefined; + expect(sourceVm?.title).toBe("Source VM"); + expect(sourceVm?.description).toMatch(/token must allow `cp`.*403/); + }); + it("declares no default on secret-ref fields, which would be persisted as a company secret", () => { for (const prop of Object.values(properties)) { if (prop.format === "secret-ref") { diff --git a/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts b/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts index a586b665f4..d45f562fc0 100644 --- a/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts +++ b/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts @@ -24,6 +24,7 @@ interface ExeDevDriverConfig { apiKey: string | null; apiUrl: string; namePrefix: string; + sourceVm: string | null; image: string | null; command: string | null; cpu: number | null; @@ -246,6 +247,7 @@ function parseDriverConfig(raw: Record): ExeDevDriverConfig { apiKey: parseOptionalString(raw.apiKey), apiUrl: normalizeApiUrl(parseOptionalString(raw.apiUrl)), namePrefix: normalizeNamePrefix(parseOptionalString(raw.namePrefix)), + sourceVm: parseOptionalString(raw.sourceVm), image: parseOptionalString(raw.image), command: parseOptionalString(raw.command), cpu: parseOptionalInteger(raw.cpu), @@ -313,10 +315,23 @@ function resolveSetupScript(config: ExeDevDriverConfig): string | null { return trimmed.length > 0 ? config.setupScript : null; } +function buildCopyCommand(sourceVm: string, config: ExeDevDriverConfig, vmName: string): string { + return [ + "cp", + shellQuote(sourceVm), + shellQuote(vmName), + "--json", + ...buildFlag("cpu", config.cpu), + ...buildFlag("memory", config.memory), + ...buildFlag("disk", config.disk), + ].join(" "); +} + function buildCreateCommand( config: ExeDevDriverConfig, vmName: string, ): string { + if (config.sourceVm) return buildCopyCommand(config.sourceVm, config, vmName); return [ "new", "--json", @@ -774,6 +789,23 @@ const plugin = definePlugin({ warnings.push( "The Paperclip host must have SSH access to the created exe.dev VM, and its SSH key must be registered with exe.dev. The API token only covers provisioning.", ); + if (config.sourceVm) { + const ignored = Object.entries({ + image: config.image, + command: config.command, + comment: config.comment, + env: Object.keys(config.env).length > 0, + integrations: config.integrations.length > 0, + tags: config.tags.length > 0, + setupScript: config.setupScript, + prompt: config.prompt, + }).filter(([, value]) => value).map(([key]) => key); + if (ignored.length > 0) { + errors.push( + `sourceVm copies an existing VM with \`exe.dev cp\`, which cannot apply ${ignored.join(", ")}. Clear these settings or clear sourceVm.`, + ); + } + } if (config.reuseLease) { warnings.push("reuseLease keeps the VM alive between runs; this provider does not suspend retained VMs."); } From bf14f803d580f0bfb964e07b479856c0bfaac38c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bar=C4=B1=C5=9F=20=C3=96ZDEM=C4=B0R?= Date: Tue, 6 Oct 2026 08:21:00 +0300 Subject: [PATCH 08/12] fix(ssh): transport project repositories as their own git checkouts (#14782) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A project can attach more than one repository. The task workspace keeps the selected repository at its root and puts the other project repositories under `.paperclip-repositories/-`, each with its own `.git` > - Agents can run on an SSH execution environment. Paperclip copies the task workspace to the remote host before the run and restores it after the run > - The SSH copy excludes `.git` at every depth, but the restore baseline excludes it only at the workspace root > - So the other project repositories reach the remote host without Git, and the restore then deletes their `.git` directories on the Paperclip host > - The next run of the same task fails during workspace setup, and the agent cannot commit to those repositories on the remote host > - This pull request transports each project repository as a Git workspace of its own, the same way the sandbox path already handles them > - The benefit is that multi-repository projects work on SSH environments across consecutive runs ## Linked Issues or Issue Description Refs #11632 (SSH workspace transfer exclude list). Related SSH workspace PRs: #14233, #14428, #14472. I found no issue or PR for this bug. **What happened?** A project has two repositories and its agent runs on an SSH environment. After the first run, the second repository under `.paperclip-repositories/` has no `.git` directory on the Paperclip host. The next run of the same task fails during setup with `Managed workspace path "…/.paperclip-repositories/" already exists but is not a git checkout.` On the remote host, `git` inside that repository resolves to the parent repository. **Expected behavior** Each project repository reaches the remote host as a Git checkout with its local changes. Remote commits and edits come back after the run. The next run of the same task starts normally. **Steps to reproduce** 1. Create a project with two repositories. 2. Configure an SSH execution environment and make it the agent's default environment. 3. Assign a task to the agent and let it run once. 4. Look at `.paperclip-repositories/` in the task workspace: `.git` is gone. 5. Wake the agent on the same task again: the run fails with `setup_failed`. **Paperclip version or commit** Reproduced on `v2026.916.1` and on `master` (`5edf55d73`). **Deployment mode** Self-hosted (Docker), authenticated, with an SSH execution environment. ## What Changed - `ssh.ts`: `prepareWorkspaceForSshExecution` lists the project repositories under `.paperclip-repositories/`. It applies the discovery rules of `readGitWorkspaceSnapshot`: each entry must be a directory with a valid name and must be a Git repository root, else the prepare step fails before any transfer. - `ssh.ts`: the anchor copy leaves `.paperclip-repositories/` out. Each project repository then gets the same import, sync, and deleted-path steps as the anchor. The remote anchor repository ignores `/.paperclip-repositories/`, as the local checkout does. - `ssh.ts`: `prepareWorkspaceForSshExecution` returns the transported repositories (the field is present only when there are repositories). `restoreWorkspaceFromSshExecution` accepts them with their baselines. It validates each path and baseline first, then restores the repositories before the anchor and stops at the first failure, as the sandbox restore does. - `remote-managed-runtime.ts`: the anchor baseline excludes `.paperclip-repositories/`, and each project repository gets its own baseline for the restore merge. - `ssh-fixture.test.ts`: regression tests for two consecutive managed runs and for the direct restore path, on a workspace with a project repository (commits, dirty edits, and a deleted file). Two tests for the new validation. - `docs/guides/board-operator/execution-workspaces-and-runtime-services.md`: one line about project repositories in the SSH round trip. ## Verification - The new regression test fails on `master` (`expected 'backend initial\n?? ../\n' to contain 'frontend initial'`) and passes with this change. - `PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1 npx vitest run packages/adapter-utils/src/ssh-fixture.test.ts packages/adapter-utils/src/remote-managed-runtime.test.ts`: 32 passed, with the sshd fixture running. - `tsc --noEmit` passes for `packages/adapter-utils` and `server`, and `pnpm -r typecheck` passes for the other workspaces. The Rust step of `@paperclipai/paperclip-runner` did not run locally because `cargo` is not installed. - `node ./scripts/check-no-git-push.mjs` and `pnpm check:module-boundaries` pass. - `pnpm test:run` did not complete locally. Before it stopped, 5 tests failed: 2 in `server/src/__tests__/workspace-runtime.test.ts` and 3 in `server/src/__tests__/company-skills-service.test.ts`. The same 5 tests also fail on the base commit `5edf55d73` without this change. CI runs the full suite. - `pnpm build` passes for all workspaces except `@paperclipai/paperclip-runner` and `server`, because their build compiles the Rust runner binary and `cargo` is not installed. `tsc --noEmit` passes for `server`. - Manual test on a self-hosted `v2026.916.1` instance with the same change applied: a project with two repositories and an SSH environment. Two runs on the same task passed. After each run, the second repository keeps its `.git` on the host. On the remote host it is a Git checkout, and the remote anchor ignores it. ## Risks - Low risk. Workspaces without `.paperclip-repositories/` take the same path as before, and the return value is unchanged for them. - A workspace with an invalid entry under `.paperclip-repositories/` now fails the SSH prepare step. The sandbox path already rejects such entries. - If one repository fails to restore, the restore stops, as in the sandbox path. The remote run directory keeps the agent's work. - Each project repository adds one bundle import and one restore per run. The time grows with the number and size of the repositories. - Out of scope: other nested `.git` directories (for example a vendored checkout inside a repository) keep the existing SSH behavior. ## Model Used - Provider and model: Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code. - Capabilities: extended thinking, tool use, and code execution. The context window size was not recorded. - Use: the model investigated the bug, wrote the change and the tests, and ran the checks. A separate Claude Code agent reviewed the diff. The author reviewed the change. The manual test ran on the author's self-hosted instance. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- ...ecution-workspaces-and-runtime-services.md | 1 + .../src/remote-managed-runtime.ts | 22 +- .../adapter-utils/src/ssh-fixture.test.ts | 209 ++++++++++++++++++ packages/adapter-utils/src/ssh.ts | 187 ++++++++++++++-- 4 files changed, 396 insertions(+), 23 deletions(-) diff --git a/docs/guides/board-operator/execution-workspaces-and-runtime-services.md b/docs/guides/board-operator/execution-workspaces-and-runtime-services.md index e38b1b2692..16eeefe881 100644 --- a/docs/guides/board-operator/execution-workspaces-and-runtime-services.md +++ b/docs/guides/board-operator/execution-workspaces-and-runtime-services.md @@ -138,6 +138,7 @@ fetches with it. Details and caveats: Code state moves between runs through the local execution-workspace cwd alone — not through a git remote. - Each run's prepare step bundles the local worktree to the run's remote dir over ssh, with no `git remote` configured. +- Other project repositories under `.paperclip-repositories/` make the same round trip. Each one is bundled and restored as its own Git checkout. - The adapter's restore step at the end of the run writes any new remote commits back into the local worktree directly. - Adapters must never `git push` from runtime code, and must never assume a remote exists. - A failed restore is a run-level error and records `workspace_finalize=failed` on the execution workspace, which gates dependent issue wakes until the next successful finalize. diff --git a/packages/adapter-utils/src/remote-managed-runtime.ts b/packages/adapter-utils/src/remote-managed-runtime.ts index cb47f3caed..8c5d2581e1 100644 --- a/packages/adapter-utils/src/remote-managed-runtime.ts +++ b/packages/adapter-utils/src/remote-managed-runtime.ts @@ -1,5 +1,5 @@ import path from "node:path"; -import { GIT_ARCHIVE_EXCLUDES } from "./git-workspace-sync.js"; +import { GIT_ARCHIVE_EXCLUDES, PROJECT_REPOSITORIES_DIR } from "./git-workspace-sync.js"; import { type SshRemoteExecutionSpec, prepareWorkspaceForSshExecution, @@ -13,7 +13,7 @@ import { type SandboxAdditionalSource, type SandboxManagedRuntimeAssetRestoreContext, } from "./sandbox-managed-runtime.js"; -import { captureDirectorySnapshot } from "./workspace-restore-merge.js"; +import { captureDirectorySnapshot, type DirectorySnapshot } from "./workspace-restore-merge.js"; import type { RuntimeProgressSink } from "./runtime-progress.js"; // The fixed heavy-directory excludes every referenced project drops, @@ -145,13 +145,27 @@ export async function prepareRemoteManagedRuntime(input: { workspaceExclude: input.workspaceExclude, }) : null; + const projectRepositories = preparedWorkspace?.repositories ?? []; const baselineSnapshot = preparedWorkspace ? await captureDirectorySnapshot(input.workspaceLocalDir, { exclude: preparedWorkspace.gitBacked - ? [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"] + ? [ + ...GIT_ARCHIVE_EXCLUDES, + ".paperclip-runtime", + ...(projectRepositories.length > 0 ? [PROJECT_REPOSITORIES_DIR] : []), + ] : [".paperclip-runtime", ...(input.workspaceFileMode === "all" ? input.workspaceExclude ?? [] : [])], }) : null; + const repositoryBaselines: Array<{ path: string; baselineSnapshot: DirectorySnapshot }> = []; + for (const repository of projectRepositories) { + repositoryBaselines.push({ + path: repository, + baselineSnapshot: await captureDirectorySnapshot(path.join(input.workspaceLocalDir, repository), { + exclude: [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"], + }), + }); + } const assetDirs: Record = {}; try { @@ -177,6 +191,7 @@ export async function prepareRemoteManagedRuntime(input: { baselineSnapshot, restoreGitHistory: preparedWorkspace.gitBacked, onProgress: input.onProgress, + repositories: repositoryBaselines, }); } throw error; @@ -244,6 +259,7 @@ export async function prepareRemoteManagedRuntime(input: { baselineSnapshot, restoreGitHistory: preparedWorkspace.gitBacked, onProgress, + repositories: repositoryBaselines, }); } for (const asset of input.assets ?? []) { diff --git a/packages/adapter-utils/src/ssh-fixture.test.ts b/packages/adapter-utils/src/ssh-fixture.test.ts index 83c4ad7e6d..39d3c52515 100644 --- a/packages/adapter-utils/src/ssh-fixture.test.ts +++ b/packages/adapter-utils/src/ssh-fixture.test.ts @@ -21,6 +21,16 @@ import { import { prepareRemoteManagedRuntime } from "./remote-managed-runtime.js"; const SSH_FIXTURE_TEST_TIMEOUT_MS = 30_000; +const UNREACHABLE_SSH_SPEC = { + host: "127.0.0.1", + port: 1, + username: "nobody", + remoteWorkspacePath: "/nonexistent", + remoteCwd: "/nonexistent", + privateKey: null, + knownHosts: null, + strictHostKeyChecking: false, +} as const; let sshEnvLabUnsupportedReason: string | null = null; // One entry per fixture root directory, registered at creation time so @@ -850,6 +860,205 @@ describe("ssh env-lab fixture", () => { await expect(readFile(path.join(localRepo, "run-b.txt"), "utf8")).resolves.toBe("from run b\n"); }, SSH_FIXTURE_TEST_TIMEOUT_MS); + it("round-trips project repositories nested in .paperclip-repositories across consecutive SSH runs", async () => { + const rootDir = await createFixtureRootDir(); + const statePath = path.join(rootDir, "state.json"); + const localRepo = path.join(rootDir, "local-workspace"); + const nestedRelative = ".paperclip-repositories/frontend-0123456789ab"; + const nestedRepo = path.join(localRepo, nestedRelative); + + await mkdir(localRepo, { recursive: true }); + await git(localRepo, ["init"]); + await git(localRepo, ["checkout", "-b", "main"]); + await git(localRepo, ["config", "user.name", "Paperclip Test"]); + await git(localRepo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(localRepo, "backend.txt"), "backend base\n", "utf8"); + await git(localRepo, ["add", "backend.txt"]); + await git(localRepo, ["commit", "-m", "backend initial"]); + await writeFile(path.join(localRepo, ".git", "info", "exclude"), "\n/.paperclip-repositories/\n", { flag: "a" }); + + await mkdir(nestedRepo, { recursive: true }); + await git(nestedRepo, ["init"]); + await git(nestedRepo, ["checkout", "-b", "main"]); + await git(nestedRepo, ["config", "user.name", "Paperclip Test"]); + await git(nestedRepo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(nestedRepo, "frontend.txt"), "frontend base\n", "utf8"); + await writeFile(path.join(nestedRepo, "obsolete.txt"), "tracked\n", "utf8"); + await git(nestedRepo, ["add", "frontend.txt", "obsolete.txt"]); + await git(nestedRepo, ["commit", "-m", "frontend initial"]); + await writeFile(path.join(nestedRepo, "frontend.txt"), "frontend dirty local\n", "utf8"); + await rm(path.join(nestedRepo, "obsolete.txt")); + + const started = await startSshEnvLabFixtureOrSkip(statePath, "SSH nested project repositories test"); + if (!started) return; + const config = await buildSshEnvLabFixtureConfig(started); + const spec = { + ...config, + remoteCwd: started.workspaceDir, + } as const; + + const first = await prepareRemoteManagedRuntime({ + spec, + runId: "run-1", + adapterKey: "test-adapter", + workspaceLocalDir: localRepo, + }); + const remoteNested = path.posix.join(first.workspaceRemoteDir, nestedRelative); + + const remoteNestedStatus = await runSshCommand( + config, + `cd ${JSON.stringify(remoteNested)} && git log -1 --pretty=%s && git status --short`, + ); + expect(remoteNestedStatus.stdout).toContain("frontend initial"); + expect(remoteNestedStatus.stdout).toContain("M frontend.txt"); + expect(remoteNestedStatus.stdout).toContain("D obsolete.txt"); + const remoteAnchorStatus = await runSshCommand( + config, + `cd ${JSON.stringify(first.workspaceRemoteDir)} && git status --short --untracked-files=all`, + ); + expect(remoteAnchorStatus.stdout).not.toContain(".paperclip-repositories"); + + await runSshCommand( + config, + [ + `cd ${JSON.stringify(remoteNested)}`, + `git config user.name "Paperclip SSH"`, + `git config user.email "ssh@paperclip.dev"`, + `git add frontend.txt`, + `git commit -m "remote frontend update" >/dev/null`, + `printf "frontend remote dirty\\n" > frontend.txt`, + `cd ${JSON.stringify(first.workspaceRemoteDir)}`, + `git config user.name "Paperclip SSH"`, + `git config user.email "ssh@paperclip.dev"`, + `printf "backend remote\\n" > backend.txt`, + `git add backend.txt`, + `git commit -m "remote backend update" >/dev/null`, + ].join(" && "), + { timeoutMs: 30_000, maxBuffer: 256 * 1024 }, + ); + + await first.restoreWorkspace(); + + await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined(); + expect(await git(nestedRepo, ["log", "-1", "--pretty=%s"])).toBe("remote frontend update"); + await expect(readFile(path.join(nestedRepo, "frontend.txt"), "utf8")).resolves.toBe("frontend remote dirty\n"); + expect(await git(localRepo, ["log", "-1", "--pretty=%s"])).toBe("remote backend update"); + expect(await git(localRepo, ["status", "--short", "--untracked-files=all"])).not.toContain(".paperclip-repositories"); + + const second = await prepareRemoteManagedRuntime({ + spec, + runId: "run-2", + adapterKey: "test-adapter", + workspaceLocalDir: localRepo, + }); + const secondNestedLog = await runSshCommand( + config, + `cd ${JSON.stringify(path.posix.join(second.workspaceRemoteDir, nestedRelative))} && git log -1 --pretty=%s`, + ); + expect(secondNestedLog.stdout.trim()).toBe("remote frontend update"); + await second.restoreWorkspace(); + await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined(); + await expect(readFile(path.join(nestedRepo, "frontend.txt"), "utf8")).resolves.toBe("frontend remote dirty\n"); + }, SSH_FIXTURE_TEST_TIMEOUT_MS * 2); + + it("restores project repositories through the direct SSH restore path", async () => { + const rootDir = await createFixtureRootDir(); + const statePath = path.join(rootDir, "state.json"); + const localRepo = path.join(rootDir, "local-workspace"); + const nestedRelative = ".paperclip-repositories/frontend-0123456789ab"; + const nestedRepo = path.join(localRepo, nestedRelative); + + for (const repo of [localRepo, nestedRepo]) { + await mkdir(repo, { recursive: true }); + await git(repo, ["init"]); + await git(repo, ["checkout", "-b", "main"]); + await git(repo, ["config", "user.name", "Paperclip Test"]); + await git(repo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(repo, "tracked.txt"), "base\n", "utf8"); + await git(repo, ["add", "tracked.txt"]); + await git(repo, ["commit", "-m", "initial"]); + } + await writeFile(path.join(localRepo, ".git", "info", "exclude"), "\n/.paperclip-repositories/\n", { flag: "a" }); + + const started = await startSshEnvLabFixtureOrSkip(statePath, "SSH direct restore with project repositories test"); + if (!started) return; + const config = await buildSshEnvLabFixtureConfig(started); + const spec = { + ...config, + remoteCwd: started.workspaceDir, + } as const; + + const prepared = await prepareWorkspaceForSshExecution({ + spec, + localDir: localRepo, + remoteDir: started.workspaceDir, + }); + expect(prepared).toEqual({ gitBacked: true, repositories: [nestedRelative] }); + + await runSshCommand( + config, + [ + `cd ${JSON.stringify(path.posix.join(started.workspaceDir, nestedRelative))}`, + `git config user.name "Paperclip SSH"`, + `git config user.email "ssh@paperclip.dev"`, + `printf "remote\\n" > tracked.txt`, + `git commit -am "remote nested update" >/dev/null`, + ].join(" && "), + { timeoutMs: 30_000, maxBuffer: 256 * 1024 }, + ); + + await restoreWorkspaceFromSshExecution({ + spec, + localDir: localRepo, + remoteDir: started.workspaceDir, + repositories: (prepared.repositories ?? []).map((repository) => ({ path: repository })), + }); + + await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined(); + expect(await git(nestedRepo, ["log", "-1", "--pretty=%s"])).toBe("remote nested update"); + await expect(readFile(path.join(nestedRepo, "tracked.txt"), "utf8")).resolves.toBe("remote\n"); + expect(await git(localRepo, ["status", "--short", "--untracked-files=all"])).toBe(""); + }, SSH_FIXTURE_TEST_TIMEOUT_MS * 2); + + it("fails closed when .paperclip-repositories holds something other than a git checkout", async () => { + const rootDir = await createFixtureRootDir(); + const localRepo = path.join(rootDir, "local-workspace"); + await mkdir(path.join(localRepo, ".paperclip-repositories", "not-a-repo"), { recursive: true }); + await git(localRepo, ["init"]); + await git(localRepo, ["checkout", "-b", "main"]); + await git(localRepo, ["config", "user.name", "Paperclip Test"]); + await git(localRepo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8"); + await git(localRepo, ["add", "tracked.txt"]); + await git(localRepo, ["commit", "-m", "initial"]); + + await expect(prepareWorkspaceForSshExecution({ spec: UNREACHABLE_SSH_SPEC, localDir: localRepo, remoteDir: "/nonexistent" })) + .rejects.toThrow("Project repository is not a Git checkout: .paperclip-repositories/not-a-repo"); + }, SSH_FIXTURE_TEST_TIMEOUT_MS); + + it("rejects inconsistent project repository restores before any SSH connection", async () => { + const localDir = await createFixtureRootDir(); + const repositoryBaseline = { exclude: [".git", ".git/*", ".paperclip-runtime"], entries: new Map() }; + const restore = (input: Partial[0]>) => + restoreWorkspaceFromSshExecution({ + spec: UNREACHABLE_SSH_SPEC, + localDir, + remoteDir: "/nonexistent", + baselineSnapshot: { exclude: [".git", ".git/*", ".paperclip-runtime", ".paperclip-repositories"], entries: new Map() }, + restoreGitHistory: true, + ...input, + }); + + await expect(restore({ repositories: [{ path: "../outside", baselineSnapshot: repositoryBaseline }] })) + .rejects.toThrow("Invalid project repository path: ../outside"); + await expect(restore({ repositories: [{ path: ".paperclip-repositories/frontend-0123456789ab" }] })) + .rejects.toThrow("Project repository has no workspace baseline: .paperclip-repositories/frontend-0123456789ab"); + await expect(restore({ + baselineSnapshot: { exclude: [".git", ".git/*", ".paperclip-runtime"], entries: new Map() }, + repositories: [{ path: ".paperclip-repositories/frontend-0123456789ab", baselineSnapshot: repositoryBaseline }], + })).rejects.toThrow("Workspace baseline must exclude .paperclip-repositories"); + }, SSH_FIXTURE_TEST_TIMEOUT_MS); + it("preserves nested per-run files across sequential SSH restores with stale baselines", async () => { const rootDir = await createFixtureRootDir(); const statePath = path.join(rootDir, "state.json"); diff --git a/packages/adapter-utils/src/ssh.ts b/packages/adapter-utils/src/ssh.ts index 89fd456c5d..68e61fc639 100644 --- a/packages/adapter-utils/src/ssh.ts +++ b/packages/adapter-utils/src/ssh.ts @@ -9,6 +9,7 @@ import type { CommandManagedRuntimeRunner } from "./command-managed-runtime.js"; import { createUnrelatedHistoryGraftCommit, GIT_SYNC_COMMIT_IDENTITY_ARGS, + PROJECT_REPOSITORIES_DIR, readSanitizedOriginRemoteUrl, } from "./git-workspace-sync.js"; import type { RunProcessResult } from "./server-utils.js"; @@ -1071,6 +1072,91 @@ async function removeDeletedPathsOnSsh(input: { }); } +const PROJECT_REPOSITORY_NAME_PATTERN = /^[a-zA-Z0-9_-]+$/; + +async function isLocalGitRepositoryRoot(localDir: string): Promise { + try { + const toplevel = await runLocalGit(localDir, ["rev-parse", "--show-toplevel"], { + timeout: 10_000, + maxBuffer: 16 * 1024, + }); + const [directory, repository] = await Promise.all([ + fs.realpath(localDir), + fs.realpath(toplevel.stdout.trim()), + ]); + return directory === repository; + } catch { + return false; + } +} + +async function listLocalProjectRepositories(localDir: string): Promise { + const root = path.join(localDir, PROJECT_REPOSITORIES_DIR); + const rootStat = await fs.lstat(root).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (!rootStat) return []; + if (!rootStat.isDirectory() || rootStat.isSymbolicLink()) { + throw new Error("Invalid project repositories directory"); + } + const repositories: string[] = []; + for (const entry of await fs.readdir(root, { withFileTypes: true })) { + if (!entry.isDirectory() || !PROJECT_REPOSITORY_NAME_PATTERN.test(entry.name)) { + throw new Error("Invalid project repository directory"); + } + const relative = `${PROJECT_REPOSITORIES_DIR}/${entry.name}`; + if (!(await isLocalGitRepositoryRoot(path.join(localDir, relative)))) { + throw new Error(`Project repository is not a Git checkout: ${relative}`); + } + repositories.push(relative); + } + return repositories.sort(); +} + +async function transportGitWorkspaceToSsh(input: { + spec: SshRemoteExecutionSpec; + localDir: string; + remoteDir: string; + snapshot: LocalGitWorkspaceSnapshot; + exclude?: string[]; + onProgress?: RuntimeProgressSink; + progressLabel: string; +}): Promise { + await importGitWorkspaceToSsh({ + spec: input.spec, + localDir: input.localDir, + remoteDir: input.remoteDir, + snapshot: input.snapshot, + onProgress: input.onProgress, + }); + await syncDirectoryToSsh({ + spec: input.spec, + localDir: input.localDir, + remoteDir: input.remoteDir, + exclude: [".git", ".paperclip-runtime", ...(input.exclude ?? [])], + onProgress: input.onProgress, + progressLabel: input.progressLabel, + }); + await removeDeletedPathsOnSsh({ + spec: input.spec, + remoteDir: input.remoteDir, + deletedPaths: input.snapshot.deletedPaths, + }); +} + +async function excludeProjectRepositoriesOnSsh(spec: SshConnectionConfig, remoteDir: string): Promise { + const pattern = `/${PROJECT_REPOSITORIES_DIR}/`; + const excludeFile = path.posix.join(remoteDir, ".git", "info", "exclude"); + await runSshScript( + spec, + `mkdir -p ${shellQuote(path.posix.dirname(excludeFile))} && ` + + `{ grep -qxF ${shellQuote(pattern)} ${shellQuote(excludeFile)} 2>/dev/null || ` + + `printf '\\n%s\\n' ${shellQuote(pattern)} >> ${shellQuote(excludeFile)}; }`, + { timeoutMs: 30_000 }, + ); +} + async function allocateLoopbackPort(host: string): Promise { return await new Promise((resolve, reject) => { const server = net.createServer(); @@ -1560,32 +1646,38 @@ export async function prepareWorkspaceForSshExecution(input: { onProgress?: RuntimeProgressSink; workspaceFileMode?: "all"; workspaceExclude?: string[]; -}): Promise<{ gitBacked: boolean }> { +}): Promise<{ gitBacked: boolean; repositories?: string[] }> { const remoteDir = input.remoteDir ?? input.spec.remoteCwd; const gitSnapshot = input.workspaceFileMode === "all" ? null : await readLocalGitWorkspaceSnapshot(input.localDir); if (gitSnapshot) { - await importGitWorkspaceToSsh({ + const repositories = await listLocalProjectRepositories(input.localDir); + await transportGitWorkspaceToSsh({ spec: input.spec, localDir: input.localDir, remoteDir, snapshot: gitSnapshot, - onProgress: input.onProgress, - }); - await syncDirectoryToSsh({ - spec: input.spec, - localDir: input.localDir, - remoteDir, - exclude: [".git", ".paperclip-runtime"], + exclude: repositories.length > 0 ? [PROJECT_REPOSITORIES_DIR] : [], onProgress: input.onProgress, progressLabel: "workspace", }); - await removeDeletedPathsOnSsh({ - spec: input.spec, - remoteDir, - deletedPaths: gitSnapshot.deletedPaths, - }); - return { gitBacked: true }; + if (repositories.length > 0) { + await excludeProjectRepositoriesOnSsh(input.spec, remoteDir); + } + for (const relative of repositories) { + const localDir = path.join(input.localDir, relative); + const snapshot = await readLocalGitWorkspaceSnapshot(localDir); + if (!snapshot) throw new Error(`Cannot read the Git state of project repository: ${relative}`); + await transportGitWorkspaceToSsh({ + spec: input.spec, + localDir, + remoteDir: path.posix.join(remoteDir, relative), + snapshot, + onProgress: input.onProgress, + progressLabel: relative, + }); + } + return { gitBacked: true, ...(repositories.length > 0 ? { repositories } : {}) }; } await clearRemoteDirectory({ @@ -1611,8 +1703,62 @@ export async function restoreWorkspaceFromSshExecution(input: { baselineSnapshot?: DirectorySnapshot; restoreGitHistory?: boolean; onProgress?: RuntimeProgressSink; + repositories?: Array<{ path: string; baselineSnapshot?: DirectorySnapshot }>; }): Promise { const remoteDir = input.remoteDir ?? input.spec.remoteCwd; + const repositories = input.repositories ?? []; + for (const repository of repositories) { + if ( + path.posix.dirname(repository.path) !== PROJECT_REPOSITORIES_DIR || + !PROJECT_REPOSITORY_NAME_PATTERN.test(path.posix.basename(repository.path)) + ) { + throw new Error(`Invalid project repository path: ${repository.path}`); + } + if (input.baselineSnapshot && !repository.baselineSnapshot) { + throw new Error(`Project repository has no workspace baseline: ${repository.path}`); + } + } + if ( + input.baselineSnapshot && + repositories.length > 0 && + !input.baselineSnapshot.exclude.includes(PROJECT_REPOSITORIES_DIR) + ) { + throw new Error(`Workspace baseline must exclude ${PROJECT_REPOSITORIES_DIR} when project repositories are restored separately`); + } + for (const repository of repositories) { + await restoreWorkspaceRootFromSsh({ + spec: input.spec, + localDir: path.join(input.localDir, repository.path), + remoteDir: path.posix.join(remoteDir, repository.path), + baselineSnapshot: repository.baselineSnapshot, + restoreGitHistory: input.restoreGitHistory, + onProgress: input.onProgress, + progressLabel: repository.path, + }); + } + await restoreWorkspaceRootFromSsh({ + spec: input.spec, + localDir: input.localDir, + remoteDir, + baselineSnapshot: input.baselineSnapshot, + restoreGitHistory: input.restoreGitHistory, + onProgress: input.onProgress, + progressLabel: "workspace", + hasProjectRepositories: repositories.length > 0, + }); +} + +async function restoreWorkspaceRootFromSsh(input: { + spec: SshRemoteExecutionSpec; + localDir: string; + remoteDir: string; + baselineSnapshot?: DirectorySnapshot; + restoreGitHistory?: boolean; + onProgress?: RuntimeProgressSink; + progressLabel: string; + hasProjectRepositories?: boolean; +}): Promise { + const remoteDir = input.remoteDir; if (input.baselineSnapshot) { const stagingDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-sync-back-")); const importedRef = input.restoreGitHistory @@ -1635,7 +1781,7 @@ export async function restoreWorkspaceFromSshExecution(input: { localDir: stagingDir, exclude: input.baselineSnapshot.exclude, onProgress: input.onProgress, - progressLabel: "workspace", + progressLabel: input.progressLabel, }); await mergeDirectoryWithBaseline({ baseline: input.baselineSnapshot, @@ -1666,6 +1812,7 @@ export async function restoreWorkspaceFromSshExecution(input: { const gitSnapshot = await readLocalGitWorkspaceSnapshot(input.localDir); if (gitSnapshot) { + const projectRepositoryEntries = input.hasProjectRepositories ? [PROJECT_REPOSITORIES_DIR] : []; await exportGitWorkspaceFromSsh({ spec: input.spec, remoteDir, @@ -1676,10 +1823,10 @@ export async function restoreWorkspaceFromSshExecution(input: { spec: input.spec, remoteDir, localDir: input.localDir, - exclude: [".git", ".paperclip-runtime"], - preserveLocalEntries: [".git"], + exclude: [".git", ".paperclip-runtime", ...projectRepositoryEntries], + preserveLocalEntries: [".git", ...projectRepositoryEntries], onProgress: input.onProgress, - progressLabel: "workspace", + progressLabel: input.progressLabel, }); return; } @@ -1690,7 +1837,7 @@ export async function restoreWorkspaceFromSshExecution(input: { localDir: input.localDir, exclude: [".paperclip-runtime"], onProgress: input.onProgress, - progressLabel: "workspace", + progressLabel: input.progressLabel, }); } From f858207161ba29c01c82f4674aef83d91b74480f Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Mon, 5 Oct 2026 22:54:25 -0700 Subject: [PATCH 09/12] Guard routine UUID lookups without narrowing valid inputs (#15313) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip manages work for AI agents and their companies. > - Routines expose details, triggers, and management actions through resource IDs. > - These resource IDs are UUIDs. The UI and CLI do not resolve short prefixes. > - A malformed ID reaches a PostgreSQL UUID comparison and returns a server error. > - A lookup guard can return the existing not-found response before that query. > - This PR preserves valid PostgreSQL UUID input forms and the existing access checks. ## Linked Issues or Issue Description Refs #11471. This is a credited continuation of the lookup-guard approach from @mv2woods. That older PR remains open and unchanged. Its review requested the required PR description sections. This continuation uses current master and adds UUID compatibility and authorization coverage. The earlier `isUuidLike` guard restricts UUID versions to 1–5 and trims input. The database currently accepts other UUID values and input forms. This guard preserves the existing [PostgreSQL UUID input contract](https://www.postgresql.org/docs/current/datatype-uuid.html), including uppercase, paired braces, omitted hyphens, and hyphens after groups of four digits. It rejects whitespace and short prefixes without rewriting the value sent to the database. ## What Changed - Guard the shared routine and private-trigger UUID lookups. Malformed resource IDs return null, so existing routes return their normal 404 response. - Keep company access, assignee permissions, body validation order, and database error propagation unchanged. - Add real PostgreSQL tests for stored UUIDv4, UUIDv7, nil, and max values in six input forms. Add no-query checks for malformed input and HTTP coverage across all 15 root-resource route handlers. - Document complete resource IDs and distinguish them from opaque public webhook IDs. ## Verification - `pnpm exec vitest run server/src/__tests__/routines-service.test.ts server/src/__tests__/routines-e2e.test.ts`: 91 passed, including the new PostgreSQL and HTTP regressions. - Independent review found no blockers and passed all 9 focused PostgreSQL and HTTP regression cases on this commit. - Full local `pnpm -r typecheck` and `pnpm build` passed on Node 24.21.0 with pnpm 9.15.4. - Full Linux CI passed on `0a2990eca2f904335821027a322e4db868151eb3`: 53 successful checks and 2 inapplicable Storybook skips. No retries were needed. This provides aggregate suite coverage; a duplicate full local aggregate was not run. - [Greptile reviewed this exact commit](https://github.com/paperclipai/paperclip/pull/15313#issuecomment-6010222312) at 5/5 with no findings or unresolved threads. The PR is ready for review and has no merge conflicts. - `git diff --check` and the added-diff secret and PII scan passed. ## Risks Malformed routine and private-trigger IDs now return 404 instead of a database error. Full UUIDs retain the existing company and assignee checks. This change does not add prefix lookup, alter public webhook IDs, or validate unrelated nested revision/thread IDs or query parameters. No migration is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository editing, code execution, and independent agent review. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip --- docs/api/routines.md | 7 ++ server/src/__tests__/routines-e2e.test.ts | 113 ++++++++++++++++++ server/src/__tests__/routines-service.test.ts | 57 +++++++++ server/src/services/routines.ts | 10 ++ 4 files changed, 187 insertions(+) diff --git a/docs/api/routines.md b/docs/api/routines.md index 6ac4a2d5b4..729efcdf49 100644 --- a/docs/api/routines.md +++ b/docs/api/routines.md @@ -21,6 +21,13 @@ GET /api/routines/{routineId} Returns routine details including triggers. +Routine and trigger resource IDs must be complete UUIDs from the API response. +Short prefixes and malformed IDs return the same `404` as a missing resource. +UUID lookup accepts PostgreSQL's existing input forms, including uppercase, +compact and braced UUIDs; it does not trim whitespace or resolve prefixes. +Company access and routine-assignee permissions still apply. Public trigger +IDs in webhook URLs are separate opaque identifiers, not resource UUIDs. + ## Create Routine ``` diff --git a/server/src/__tests__/routines-e2e.test.ts b/server/src/__tests__/routines-e2e.test.ts index 1f6742be74..42e7ef0b46 100644 --- a/server/src/__tests__/routines-e2e.test.ts +++ b/server/src/__tests__/routines-e2e.test.ts @@ -238,6 +238,119 @@ describeEmbeddedPostgres("routine routes end-to-end", () => { return { companyId, agentId, projectId, userId }; } + function routineResourceRequests(app: express.Express, routineId: string, triggerId = routineId) { + const path = `/api/routines/${encodeURIComponent(routineId)}`; + const triggerPath = `/api/routine-triggers/${encodeURIComponent(triggerId)}`; + const nestedId = "12345678-1234-4234-8234-123456789abc"; + return [ + request(app).get(path), + request(app).get(`${path}/runs`), + request(app).get(`${path}/revisions`), + request(app).get(`${path}/description/annotations`), + request(app).get(`${path}/description/annotations/${nestedId}`), + request(app).post(`${path}/description/annotations`).send({ + baseRevisionId: nestedId, baseRevisionNumber: 1, body: "Review this text", + selector: { + quote: { exact: "text" }, + position: { normalizedStart: 0, normalizedEnd: 4, markdownStart: 0, markdownEnd: 4 }, + }, + }), + request(app).post(`${path}/description/annotations/${nestedId}/comments`).send({ body: "Review" }), + request(app).patch(`${path}/description/annotations/${nestedId}`).send({ status: "resolved" }), + request(app).patch(path).send({ title: "Updated routine" }), + request(app).post(`${path}/revisions/${nestedId}/restore`).send({}), + request(app).post(`${path}/triggers`).send({ kind: "api" }), + request(app).post(`${path}/run`).send({}), + request(app).patch(triggerPath).send({ enabled: false }), + request(app).delete(triggerPath), + request(app).post(`${triggerPath}/rotate-secret`).send({}), + ]; + } + + it("returns not found without queries for malformed routine and trigger IDs on every resource route", async () => { + const { companyId, agentId, userId } = await seedFixture(); + const actors = [ + { type: "board", userId, source: "session", companyIds: [companyId] }, + { type: "agent", agentId, companyId }, + { type: "none" }, + ]; + const fullId = "12345678-1234-4234-8234-123456789abc"; + for (const actor of actors) { + const app = await createApp(actor); + const spies = [ + vi.spyOn(db, "select"), vi.spyOn(db, "insert"), + vi.spyOn(db, "update"), vi.spyOn(db, "delete"), + ]; + try { + for (const id of [fullId.slice(0, 8), "not-a-uuid", ` ${fullId}`, `${fullId}\n`, `{${fullId}\n}`]) { + for (const pending of routineResourceRequests(app, id)) { + const response = await pending; + expect(response.status, JSON.stringify(response.body)).toBe(404); + expect(response.body).toEqual({ error: response.req.path.includes("/routine-triggers/") + ? "Routine trigger not found" : "Routine not found" }); + } + } + const invalidBody = await request(app).patch("/api/routines/not-a-uuid").send({ title: 42 }); + expect(invalidBody.status).toBe(400); + for (const spy of spies) expect(spy).not.toHaveBeenCalled(); + } finally { + for (const spy of spies) spy.mockRestore(); + } + } + }, 20_000); + + it("keeps missing and cross-company routine resources indistinguishable", async () => { + const { companyId, agentId, userId } = await seedFixture(); + const routineId = randomUUID(); + const triggerId = randomUUID(); + await db.insert(routines).values({ id: routineId, companyId, assigneeAgentId: agentId, title: "Private routine" }); + await db.insert(routineTriggers).values({ id: triggerId, companyId, routineId, kind: "api" }); + for (const actor of [ + { type: "board", userId, source: "session", isInstanceAdmin: true, companyIds: [randomUUID()] }, + { type: "agent", agentId: randomUUID(), companyId: randomUUID() }, + { type: "none" }, + ]) { + const app = await createApp(actor); + for (const [id, tid] of [[routineId, triggerId], [routineId.toUpperCase(), `{${triggerId}}`], [randomUUID(), randomUUID()]]) { + for (const pending of routineResourceRequests(app, id!, tid!)) { + const response = await pending; + expect(response.status, JSON.stringify(response.body)).toBe(404); + expect(response.body).toEqual({ error: response.req.path.includes("/routine-triggers/") + ? "Routine trigger not found" : "Routine not found" }); + } + } + } + expect(await db.select().from(routines)).toMatchObject([{ id: routineId, title: "Private routine" }]); + expect(await db.select().from(routineTriggers)).toMatchObject([{ id: triggerId, enabled: true }]); + expect(await db.select().from(activityLog)).toEqual([]); + }, 20_000); + + it("preserves UUID routine reads and assignee-only mutations", async () => { + const { companyId, agentId, userId } = await seedFixture(); + const routineId = randomUUID(); + const triggerId = randomUUID(); + await db.insert(routines).values({ id: routineId, companyId, assigneeAgentId: agentId, title: "Private routine" }); + await db.insert(routineTriggers).values({ id: triggerId, companyId, routineId, kind: "api" }); + const owner = await createApp({ type: "agent", agentId, companyId }); + const otherAgent = await createApp({ type: "agent", agentId: randomUUID(), companyId }); + const board = await createApp({ type: "board", userId, source: "session", companyIds: [companyId] }); + for (const app of [owner, otherAgent, board]) { + const response = await request(app).get(`/api/routines/${routineId.toUpperCase()}`); + expect(response.status).toBe(200); + expect(response.body.id).toBe(routineId); + } + for (const [index, pending] of routineResourceRequests(otherAgent, routineId, triggerId).entries()) { + const response = await pending; + expect(response.status, JSON.stringify(response.body)).toBe(index < 2 ? 200 : 403); + } + const updated = await request(owner).patch(`/api/routines/${routineId}`).send({ title: "Owner update" }); + expect(updated.status, JSON.stringify(updated.body)).toBe(200); + expect(updated.body.title).toBe("Owner update"); + const changedTrigger = await request(board).patch(`/api/routine-triggers/${triggerId}`).send({ enabled: false }); + expect(changedTrigger.status, JSON.stringify(changedTrigger.body)).toBe(200); + expect(changedTrigger.body.enabled).toBe(false); + }, 20_000); + it.each(["bearer", "hmac_sha256", "github_hmac", "none"] as const)( "authenticates %s HTTP deliveries, persists payloads, and enforces trigger lifecycle", async (signingMode) => { diff --git a/server/src/__tests__/routines-service.test.ts b/server/src/__tests__/routines-service.test.ts index bc4badd189..9d4949e30a 100644 --- a/server/src/__tests__/routines-service.test.ts +++ b/server/src/__tests__/routines-service.test.ts @@ -246,6 +246,63 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => { .then((rows) => rows[0]!); } + it("rejects malformed routine and trigger IDs before any query", async () => { + const { routine, svc } = await seedFixture(); + const malformed = [ + "", routine.id.slice(0, 8), "not-a-uuid", `${routine.id} `, ` ${routine.id}`, + `${routine.id}\n`, `{${routine.id}`, `${routine.id}}`, `{${routine.id}\n}`, + `${routine.id}-`, routine.id.replace("-", "--"), `g${routine.id.slice(1)}`, + `${routine.id}\0`, `é${routine.id.slice(1)}`, + `${routine.id.slice(0, 3)}-${routine.id.slice(3)}`, + ]; + const select = vi.spyOn(db, "select"); + try { + for (const id of malformed) { + await expect(svc.get(id)).resolves.toBeNull(); + await expect(svc.getDetail(id)).resolves.toBeNull(); + await expect(svc.getTrigger(id)).resolves.toBeNull(); + } + expect(select).not.toHaveBeenCalled(); + } finally { + select.mockRestore(); + } + }); + + it("preserves lookup failures for valid routine and trigger IDs", async () => { + const { routine, svc } = await seedFixture(); + const failure = new Error("Database unavailable"); + const select = vi.spyOn(db, "select").mockImplementation(() => { throw failure; }); + try { + await expect(svc.get(routine.id)).rejects.toBe(failure); + await expect(svc.getDetail(routine.id)).rejects.toBe(failure); + await expect(svc.getTrigger(routine.id)).rejects.toBe(failure); + } finally { + select.mockRestore(); + } + }); + + it.each([ + "12345678-1234-4234-8234-123456789abc", + "12345678-1234-7234-8234-123456789abc", + "00000000-0000-0000-0000-000000000000", + "ffffffff-ffff-ffff-ffff-ffffffffffff", + ])("preserves PostgreSQL UUID lookup forms for %s", async (id) => { + const { companyId, svc } = await seedFixture(); + await db.insert(routines).values({ id, companyId, title: "UUID lookup fixture" }); + await db.insert(routineTriggers).values({ id, companyId, routineId: id, kind: "api" }); + const compact = id.replaceAll("-", ""); + const forms = [ + id, id.toUpperCase(), `{${id}}`, compact, + compact.match(/.{4}/g)!.join("-"), + `{${compact.slice(0, 8)}-${compact.slice(8, 16)}-${compact.slice(16)}}`, + ]; + for (const input of forms) { + await expect(svc.get(input)).resolves.toMatchObject({ id, companyId }); + await expect(svc.getDetail(input)).resolves.toMatchObject({ id, companyId }); + await expect(svc.getTrigger(input)).resolves.toMatchObject({ id, companyId, routineId: id }); + } + }); + it("clears transient routine run failures when execution issues resume", async () => { const { companyId, issueSvc, routine, svc } = await seedFixture(); const runId = randomUUID(); diff --git a/server/src/services/routines.ts b/server/src/services/routines.ts index c681e73b0b..98bffd4f18 100644 --- a/server/src/services/routines.ts +++ b/server/src/services/routines.ts @@ -665,6 +665,14 @@ function mapRoutineDescriptionDocument(row: { }; } +// Match PostgreSQL UUID input, including compact/braced forms and any version. +// Do not trim or resolve prefixes: the exact input must be safe for a UUID query. +function isRoutineUuidInput(id: string): boolean { + if (/\s/.test(id)) return false; + const value = id.startsWith("{") && id.endsWith("}") ? id.slice(1, -1) : id; + return /^(?:[0-9a-f]{4}-?){7}[0-9a-f]{4}$/i.test(value); +} + export function routineService( db: Db, deps: { @@ -682,6 +690,7 @@ export function routineService( }); async function getRoutineById(id: string) { + if (!isRoutineUuidInput(id)) return null; return db .select() .from(routines) @@ -766,6 +775,7 @@ export function routineService( } async function getTriggerById(id: string) { + if (!isRoutineUuidInput(id)) return null; return db .select() .from(routineTriggers) From a1ab55a56d3d535931efb18fc6b5e6f13b5fcdbd Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 6 Oct 2026 05:28:48 -0500 Subject: [PATCH 10/12] fix(agents): grant configuration access by default (#15283) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip manages agent work and permissions for a company. > - Agent setup can require one agent to configure another agent. > - New standard agents could create agents, but they had no direct configuration grant. > - Existing agents must keep their current permissions after an upgrade. > - The requested new-agent defaults include 13 more direct permissions for suggestions, skills, tools, audit, inbox, and task assignment. > - This pull request adds the 14-grant set at creation and approval activation, retains it through invitations, and keeps existing agents unchanged. > - The change keeps low trust and built-in agents on their narrower permissions. ## Linked Issues or Issue Description **What existing behavior does this improve?** The agent creation and approval flows, and the agent configuration authorization path. **Current behavior** A new standard agent can create an agent. It cannot make protected changes to a peer agent unless an operator adds an `agents:configure` grant. **Proposed behavior** Add direct grants for `agents:configure`, `agents:suggest-changes`, `skills:create`, `skills:suggest-changes`, `tools:manage_connections`, `tools:manage_profiles`, `tools:view_audit`, `audit:view_agent_actions`, `tools:use`, `tools:manage_runtime`, `inbox:manage`, `tasks:assign`, `tasks:assign_scope`, and `tasks:manage_active_checkouts` to new standard agents. Scope `tasks:assign_scope` to the agent’s reporting subtree. Keep existing agents and their grants unchanged. **Reason and benefit** New standard agents can complete agent setup and the requested tool, skill, inbox, and task workflows under existing route, scope, and approval checks. **Breaking changes** Existing agents keep their current permissions. There is no permission migration. Related PR: #12212 adds scoped grant routes. This PR changes the default grant. ## What Changed - Add the 14 requested direct grants in agent creation and approval activation. Keep them when invitation approval replaces grants, preserving explicit scopes. Remove them when an agent is deleted. - Apply defaults only to new agents. Remove the existing-agent permission migration, its snapshot, and its journal entry. Preserve existing scoped grants. - Add permission, scope, invitation, and existing-agent regression tests. Document all default and excluded permissions. - Prevent agent keys from creating, changing, or restoring host-executed process or local adapter command settings, and from restoring workspace commands through rollback. ## Verification - Run `node_modules/.bin/vitest run server/src/__tests__/agent-default-configure-grants.test.ts server/src/__tests__/invite-join-grants.test.ts packages/db/src/migration-snapshot-drift.test.ts`. All 15 tests pass. These cover new-agent defaults, unchanged existing grants, pending approval, invitation grants, and migration history. - Run `node_modules/.bin/tsc -p server/tsconfig.json --noEmit`. It passes. - Run `packages/db/node_modules/.bin/tsx packages/db/src/check-migration-numbering.ts` and `packages/db/node_modules/.bin/tsx packages/db/src/check-migration-safety.ts`. Both pass. - Confirm that this PR has no files under `packages/db/src/migrations/` in its final diff. - GitHub CI at `e8173b2c41` passes build, typecheck, server suites, browser shards, and canary verification. One unchanged OpenCode transport test reached its five-second timeout in the first Runner shard run. That test passes locally in 2.55 seconds. The shard passed on one retry. All 54 checks pass, with two expected skips. - Greptile gives this exact head 5/5. Security review passes. The PR has no unresolved review threads or merge conflicts. ## Risks - The 14 default permissions apply only to new standard agents. Existing permissions stay unchanged. Low trust and managed built-in agents are excluded. - New grants include connection, runtime, and active-checkout management. Existing company, responsible-user, scope, and approval checks remain in force. The default inbox grant carries a responsible-user-only scope so it cannot override another user's inbox settings. - Protected changes still need the responsible user's authority when that check applies. Company boundaries and approval gates still apply. - Agent-authenticated requests cannot configure process adapters or host command settings on local adapters. Known provider credential references remain allowed, as do narrow plain authentication overrides on new peers when the caller uses an AI connection pool. Arbitrary environment settings remain blocked. Board operators retain the host configuration paths. > I checked `ROADMAP.md`. This is a narrow fix to existing agent configuration behavior. ## Model Used - OpenAI Codex, GPT-6 series. This runtime did not expose its exact hosted model ID or context window. This revision uses OpenAI GPT-6 through Codex with tool use, code execution, and tests. The runtime does not expose the exact hosted model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/SPEC-implementation.md | 10 +- doc/agent-permission-defaults.md | 31 +++ .../agent-default-configure-grants.test.ts | 240 ++++++++++++++++++ .../agent-hire-ai-connections.test.ts | 29 ++- .../agent-permissions-routes.test.ts | 211 +++++++++++++++ .../agents-claude-oauth-binding.test.ts | 2 + .../agents-pending-approval-config.test.ts | 2 + .../agents-service-secret-bindings.test.ts | 2 + .../__tests__/authorization-service.test.ts | 38 +++ server/src/__tests__/built-in-agents.test.ts | 6 +- .../src/__tests__/invite-join-grants.test.ts | 48 ++-- .../__tests__/plugin-managed-agents.test.ts | 2 + .../__tests__/plugin-managed-routines.test.ts | 2 + server/src/routes/access.ts | 20 +- server/src/routes/agents.ts | 82 ++++++ server/src/services/agent-permissions.ts | 29 ++- server/src/services/agents.ts | 33 ++- server/src/services/authorization.ts | 3 +- server/src/services/invite-grants.ts | 20 +- 19 files changed, 744 insertions(+), 66 deletions(-) create mode 100644 doc/agent-permission-defaults.md create mode 100644 server/src/__tests__/agent-default-configure-grants.test.ts diff --git a/doc/SPEC-implementation.md b/doc/SPEC-implementation.md index 41d7435d3a..7e0a3b8ea2 100644 --- a/doc/SPEC-implementation.md +++ b/doc/SPEC-implementation.md @@ -592,7 +592,7 @@ conversation lifecycles, and protection against replaying superseded requests. | Action | Board | Agent | |---|---|---| | Create company | yes | no | -| Hire/create agent | yes (direct) | request via approval | +| Hire/create agent | yes (direct) | new standard agents: direct via `canCreateAgents`; low-trust policy or approval gates can restrict | | Pause/resume agent | yes | pause: no; resume: direct `agents:configure` grant only | | Create/update task | yes | yes | | Force reassign task | yes | limited | @@ -611,6 +611,14 @@ access does not bypass that decision, and `agents:suggest-changes` alone cannot apply the lifecycle change. Pause, clear-error, terminate, approval, and key-management routes remain board-only. +An ordinary standard agent receives a direct `agents:configure` grant on +creation. Existing agents keep their current permissions; no backfill runs. +Low-trust and managed built-in agents do not receive this default. See +[agent permission defaults](agent-permission-defaults.md) for the full inventory. +Agent-authenticated changes cannot set or restore host-executed process adapter +configuration, including commands and environment values. Agent-authenticated +rollbacks cannot restore host-executed workspace commands either. + ### 9.3.1 Shared default-open issue writes For standard-trust agents, issue comments, issue field/status updates, child diff --git a/doc/agent-permission-defaults.md b/doc/agent-permission-defaults.md new file mode 100644 index 0000000000..49ee70192f --- /dev/null +++ b/doc/agent-permission-defaults.md @@ -0,0 +1,31 @@ +# Agent permission defaults + +This document describes the server defaults for an ordinary, standard trust agent. An agent must belong to the same company and be active where the action requires active membership. A responsible user's authority, project or run trust policy, scope rules, and approval gates can narrow these defaults. + +## Granted by default + +| Capability | Source and limit | +| --- | --- | +| Agent configuration and suggestions | New standard agents receive direct, company-scoped `agents:configure` and `agents:suggest-changes` grants. Protected-change and responsible-user checks still apply. | +| Skill changes | New standard agents receive direct `skills:create` and `skills:suggest-changes` grants. | +| Tool access and management | New standard agents receive `tools:manage_connections`, `tools:manage_profiles`, `tools:view_audit`, `tools:use`, and `tools:manage_runtime` grants. Tool, connection, profile, and runtime policy checks still apply. | +| Audit and inbox | New standard agents receive `audit:view_agent_actions` and `inbox:manage` grants. The default inbox grant is limited to the responsible-user path and still follows that user's inbox policy. Cross-user management requires a separately configured grant or the target user's saved consent policy. | +| Task assignment and checkout control | New standard agents receive `tasks:assign`, `tasks:assign_scope`, and `tasks:manage_active_checkouts` grants, subject to company and route checks. The `tasks:assign_scope` grant covers the new agent's own reporting subtree; an explicit invitation scope is preserved. | +| Agent creation | New standard agents have `canCreateAgents: true`. This is a legacy authorization path for `agents:create`, not an `agents:create` grant row. Stored legacy records without the flag stay closed. | +| Skill creation setting | `canCreateSkills: true` is the normalized setting. Protected skill configuration changes still require a direct `skills:create` grant or a consented `skills:suggest-changes` grant. | +| Same-company visibility and work | Standard agents can read same-company agents, projects, issues, and company scope; read and manage decision queues; read runtime and secret metadata where the route allows it; comment or mutate their own or unassigned issues; and assign tasks under the assignment policy. The route and resource checks still apply. | +| Own configuration and wake | An agent can read its own configuration, update unprotected parts of it, and wake itself. Protected changes still require the relevant grant. | + +## Not granted by default + +These permission keys have no blanket grant for new standard agents: `agents:create` (the flag above is separate), `environments:manage`, `tools:admin`, `users:invite`, `users:manage_permissions`, `pipelines:write`, and `joins:approve`. Existing agents keep their current permissions. The defaults above apply when a new standard agent is created or a pending new hire is activated. Some actions have separate bounded paths; the absence of a grant row does not describe every route decision. + +The new direct grants do not grant company administration, user permission management, blanket tool administration, or cross-company access. The responsible user's permissions, route checks, and the normal approval gates still apply. + +Agent-authenticated callers cannot set process adapter configuration or switch an existing agent onto the process adapter. For local adapters, they cannot change host command, argument, working-directory, arbitrary environment, or filesystem sandbox command settings. Provider credential secret references on known environment keys remain allowed so agents can hire peers with their own credentials. When a caller uses an AI connection pool, a new peer can use a plain provider authentication override on the small allowlist of supported keys. Arbitrary environment keys remain blocked. The same limits apply to creation, hiring, adapter switches, and configuration rollback. They also cannot restore host-executed workspace commands through a configuration revision. This keeps host-executed commands under board control while standard agents configure other supported agent settings. + +## Exceptions and rollout + +The new direct grants are withheld from low trust agents and managed built-in agents. Low trust run or project policies can also deny privileged actions even if an agent has a grant. There is no permission backfill or upgrade migration for existing agents. A standard pending new hire receives the new default set when approved and activated. Invitation approval retains the set when it replaces a new agent's grants and preserves any explicitly scoped default grant. Existing scoped grants stay unchanged. A grant removed by an operator stays removed; ordinary updates and startup do not reapply it. + +The standard agent configuration default supports the agent setup in the linked runner task. Enabling a warm runtime can still depend on the target agent's adapter and runtime configuration, provider availability, and any responsible-user or approval checks. diff --git a/server/src/__tests__/agent-default-configure-grants.test.ts b/server/src/__tests__/agent-default-configure-grants.test.ts new file mode 100644 index 0000000000..e67f68a545 --- /dev/null +++ b/server/src/__tests__/agent-default-configure-grants.test.ts @@ -0,0 +1,240 @@ +import { randomUUID } from "node:crypto"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { eq } from "drizzle-orm"; +import { agents, companies, companyMemberships, createDb, principalPermissionGrants } from "@paperclipai/db"; +import { LOW_TRUST_REVIEW_PRESET, type PermissionKey } from "@paperclipai/shared"; +import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { agentService } from "../services/agents.js"; +import { authorizationService } from "../services/authorization.js"; +import { accessService } from "../services/access.js"; +import { agentJoinGrantsFromDefaults } from "../services/invite-grants.js"; + +const support = await getEmbeddedPostgresTestSupport(); +const describeDatabase = support.supported ? describe : describe.skip; +const expectedNewAgentGrantKeys: PermissionKey[] = [ + "agents:configure", + "agents:suggest-changes", + "skills:create", + "skills:suggest-changes", + "tools:manage_connections", + "tools:manage_profiles", + "tools:view_audit", + "audit:view_agent_actions", + "tools:use", + "tools:manage_runtime", + "inbox:manage", + "tasks:assign", + "tasks:assign_scope", + "tasks:manage_active_checkouts", +].sort(); + +describeDatabase("new agent configuration defaults", () => { + let db!: ReturnType; + let cleanup: (() => Promise) | undefined; + + beforeAll(async () => { + const started = await startEmbeddedPostgresTestDatabase("agent-configure-defaults"); + db = createDb(started.connectionString); + cleanup = started.cleanup; + }, 20_000); + + afterAll(async () => { + await cleanup?.(); + }); + + it("grants standard new agents direct peer configuration, but keeps restricted agents narrow", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Agent permission defaults", + issuePrefix: `PD${companyId.slice(0, 6).toUpperCase()}`, + }); + const create = (name: string, permissions: Record = {}, metadata?: Record) => + agentService(db).create(companyId, { + name, + role: "engineer", + adapterType: "process", + adapterConfig: {}, + runtimeConfig: {}, + permissions, + metadata, + }, metadata ? { allowBuiltInAgentMetadata: true } : undefined); + + const standard = await create("Standard"); + const peer = await create("Peer"); + const lowTrust = await create("Low trust", { trustPreset: LOW_TRUST_REVIEW_PRESET }); + const bundled = await create("Bundled", {}, { + paperclipBuiltInAgent: { key: "reflection-coach", featureKeys: [] }, + }); + + const grants = await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId)); + for (const agent of [standard, peer]) { + expect(grants.filter((grant) => grant.principalId === agent.id) + .map((grant) => grant.permissionKey).sort()).toEqual(expectedNewAgentGrantKeys); + expect(grants.find((grant) => grant.principalId === agent.id && grant.permissionKey === "tasks:assign_scope")?.scope) + .toEqual({ subtreeRootAgentId: agent.id }); + expect(grants.find((grant) => grant.principalId === agent.id && grant.permissionKey === "inbox:manage")?.scope) + .toEqual({ responsibleUserOnly: true }); + } + for (const agent of [lowTrust, bundled]) { + expect(grants.filter((grant) => grant.principalId === agent.id)).toEqual([]); + } + + await db.insert(companyMemberships).values({ + companyId, + principalType: "agent", + principalId: standard.id, + status: "active", + membershipRole: "member", + }); + + for (const permissionKey of expectedNewAgentGrantKeys) { + if (permissionKey === "tasks:assign_scope" || permissionKey === "inbox:manage") continue; // These grants require a target scope or the responsible-user inbox path. + expect(await accessService(db).hasPermission(companyId, "agent", standard.id, permissionKey)).toBe(true); + } + expect(await authorizationService(db).decidePrincipalGrant({ + companyId, + principalType: "agent", + principalId: standard.id, + action: "tasks:assign", + permissionKey: "tasks:assign_scope", + scope: { assigneeAgentId: standard.id }, + })).toMatchObject({ allowed: true }); + expect(await authorizationService(db).decidePrincipalGrant({ + companyId, + principalType: "agent", + principalId: standard.id, + action: "tasks:assign", + permissionKey: "tasks:assign_scope", + scope: { assigneeAgentId: peer.id }, + })).toMatchObject({ allowed: false, reason: "deny_scope" }); + for (const permissionKey of [ + "agents:create", "environments:manage", "tools:admin", "users:invite", + "users:manage_permissions", "pipelines:write", "joins:approve", + ] as const) { + expect(await accessService(db).hasPermission(companyId, "agent", standard.id, permissionKey)).toBe(false); + } + + const decision = await authorizationService(db).decide({ + actor: { type: "agent", agentId: standard.id, companyId, source: "agent_key" }, + action: "agent_config:update", + resource: { type: "agent", agentId: peer.id, companyId }, + scope: { requiresChangeGrant: true }, + }); + expect(decision).toMatchObject({ allowed: true, reason: "allow_direct_change" }); + + // A later ordinary self-update must not recreate a grant removed by an operator. + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, standard.id)); + await agentService(db).update(standard.id, { title: "Updated own title" }); + expect(await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.principalId, standard.id))).toEqual([]); + + await agentService(db).remove(peer.id); + expect(await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.principalId, peer.id))).toEqual([]); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(companyMemberships).where(eq(companyMemberships.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); + + it("keeps existing agents and their scoped grants unchanged during ordinary updates", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Existing permission defaults", + issuePrefix: `PE${companyId.slice(0, 6).toUpperCase()}`, + }); + const rows = await db.insert(agents).values([ + { companyId, name: "Existing", role: "engineer", permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Scoped", role: "engineer", permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Restricted", role: "engineer", permissions: { trustPreset: LOW_TRUST_REVIEW_PRESET }, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Built in", role: "engineer", metadata: { paperclipBuiltInAgent: { key: "reflection-coach", featureKeys: [] } }, permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Pending", role: "engineer", status: "pending_approval", permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + ]).returning(); + await db.insert(principalPermissionGrants).values({ + companyId, + principalType: "agent", + principalId: rows[1]!.id, + permissionKey: "agents:configure", + scope: { agentIds: [rows[0]!.id] }, + }); + + for (const row of rows) { + if (row.status === "pending_approval") { + await expect(agentService(db).update(row.id, { title: "Updated existing agent" })) + .rejects.toThrow("Pending approval agent configuration cannot be changed before board approval"); + } else { + await agentService(db).update(row.id, { title: "Updated existing agent" }); + } + } + const grants = await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId)); + expect(grants.map((grant) => grant.principalId).sort()).toEqual([rows[1]!.id]); + expect(grants.map((grant) => grant.permissionKey)).toEqual(["agents:configure"]); + expect(grants.find((grant) => grant.principalId === rows[1]!.id)?.scope).toEqual({ agentIds: [rows[0]!.id] }); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); + + it("adds the default grant when a standard pending hire is approved", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Pending permission default", + issuePrefix: `PP${companyId.slice(0, 6).toUpperCase()}`, + }); + const pending = await agentService(db).create(companyId, { + name: "Pending hire", + status: "pending_approval", + role: "engineer", + adapterType: "process", + adapterConfig: {}, + runtimeConfig: {}, + }); + expect(await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId))).toEqual([]); + + const result = await agentService(db).activatePendingApproval(pending.id); + expect(result?.activated).toBe(true); + expect((await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId))) + .map((grant) => grant.permissionKey).sort()).toEqual(expectedNewAgentGrantKeys); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); + + it("keeps configuration access when invitation approval replaces a new agent's grants", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Invited permission default", + issuePrefix: `PI${companyId.slice(0, 6).toUpperCase()}`, + }); + const invited = await agentService(db).create(companyId, { + name: "Invited agent", + role: "engineer", + adapterType: "process", + adapterConfig: {}, + runtimeConfig: {}, + }); + await accessService(db).ensureMembership(companyId, "agent", invited.id, "member", "active"); + await accessService(db).setPrincipalGrants( + companyId, "agent", invited.id, agentJoinGrantsFromDefaults(null, invited.id), null, + ); + const grants = await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.principalId, invited.id)); + expect(grants.map((grant) => grant.permissionKey).sort()).toEqual(expectedNewAgentGrantKeys); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(companyMemberships).where(eq(companyMemberships.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); +}); diff --git a/server/src/__tests__/agent-hire-ai-connections.test.ts b/server/src/__tests__/agent-hire-ai-connections.test.ts index 8fb798e261..cd12debd64 100644 --- a/server/src/__tests__/agent-hire-ai-connections.test.ts +++ b/server/src/__tests__/agent-hire-ai-connections.test.ts @@ -28,6 +28,7 @@ beforeAll(async () => { home = await mkdtemp(path.join(os.tmpdir(), "paperclip-hire-ai-")); vi.stubEnv("PAPERCLIP_HOME", home); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "hire-ai"); + vi.stubEnv("PAPERCLIP_IN_WORKTREE", "false"); database = await startEmbeddedPostgresTestDatabase("paperclip-hire-ai-db-"); db = createDb(database.connectionString); }, 90_000); @@ -310,15 +311,15 @@ describe("agent-created hires use managed AI connections", () => { ["ANTHROPIC_API_KEY", ""], ["CLAUDE_CONFIG_DIR", "/tmp/child-claude-home"], ["ANTHROPIC_BASE_URL", "https://example.invalid"], - ])(`${endpoint}: preserves an explicit child auth setting %s=%s`, async (key, value) => { + ])(`${endpoint}: rejects an agent-supplied local environment setting %s=%s`, async (key, value) => { const f = await fixture("anthropic"); - const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ + const response = await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ name: "Explicit auth", role: "engineer", adapterType: f.adapterType, adapterConfig: { env: { [key]: value } }, - })); - expect(agent.runtimeConfig.aiConnection).toBeUndefined(); - const [saved] = await db.select().from(agents).where(eq(agents.id, agent.id)); - expect((saved.adapterConfig.env as Record)[key]).toEqual({ type: "plain", value }); + }); + expect(response.status).toBe(403); + expect(response.body.error).toContain("host-executed local adapter settings"); + expect(await db.select().from(agents).where(eq(agents.companyId, f.companyId))).toHaveLength(1); }); } @@ -366,13 +367,17 @@ describe("agent-created hires use managed AI connections", () => { ["openai", "paperclip_runner", { provider: "acpx", acpxAgent: "claude" }, "OPENAI_API_KEY"], ] as const)(`${endpoint}: ignores the %s auth key for a different provider in %s`, async (provider, adapterType, config, key) => { const f = await fixture(provider); - const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ + const response = await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ name: "Cross-provider config", role: "engineer", adapterType, adapterConfig: { ...config, env: { [key]: "leftover-parent-setting" } }, - })); - expect(agent.runtimeConfig.aiConnection).toMatchObject({ - provider: provider === "anthropic" ? "openai" : "anthropic", mode: "responsible_user", }); + if (adapterType.endsWith("_local")) { + expect(response.status).toBe(403); + expect(response.body.error).toContain("host-executed local adapter settings"); + return; + } + const agent = hired(response); + expect(agent.runtimeConfig.aiConnection).toMatchObject({ provider: provider === "anthropic" ? "openai" : "anthropic", mode: "responsible_user" }); }); } @@ -465,7 +470,9 @@ describe("agent-created hires use managed AI connections", () => { describe("hired agents sharing a subscription", () => { it.each(["openai", "anthropic"] as const)("runs the %s child alongside a live parent and inherits its connection", async (provider) => { const f = await fixture(provider, "subscription"); - const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Concurrent teammate", role: "engineer", adapterType: f.adapterType, reportsTo: f.agentId, adapterConfig: { cwd: home, engine: "cli" }, runtimeConfig: { heartbeat: { enabled: false } } })); + const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Concurrent teammate", role: "engineer", adapterType: f.adapterType, reportsTo: f.agentId, adapterConfig: { engine: "cli" }, runtimeConfig: { heartbeat: { enabled: false } } })); + // Host working directories are configured by an operator, not an agent key. + await db.update(agents).set({ adapterConfig: { ...agent.adapterConfig, cwd: home } }).where(eq(agents.id, agent.id)); const [issue] = await db.insert(issues).values({ companyId: f.companyId, title: "Subscription child task", status: "todo", assigneeAgentId: agent.id, responsibleUserId: f.userId, createdByUserId: f.userId }).returning(); const parentRuntime = await prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: f.agentId, responsibleUserId: f.userId, adapterType: f.adapterType, binding: f.binding, config: { cwd: home } }); const execute = vi.fn(async () => { diff --git a/server/src/__tests__/agent-permissions-routes.test.ts b/server/src/__tests__/agent-permissions-routes.test.ts index 2997c9b928..4322844170 100644 --- a/server/src/__tests__/agent-permissions-routes.test.ts +++ b/server/src/__tests__/agent-permissions-routes.test.ts @@ -918,6 +918,217 @@ describe.sequential("agent permission routes", () => { expect(mockLogActivity).not.toHaveBeenCalled(); }); + it.each([ + ["command", { command: "sh" }], + ["arguments", { args: ["-c", "id"] }], + ["environment", { env: { PATH: "/tmp" } }], + ])("blocks agent-authenticated process adapter %s updates", async (_label, adapterConfig) => { + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it("blocks an agent from switching a peer onto the process adapter", async () => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType: "codex_local" }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterType: "process" })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it.each([ + ["claude_local", { engine: "cli", command: "/tmp/untrusted" }], + ["codex_local", { args: ["-c", "id"] }], + ["cursor_local", { env: { PATH: "/tmp" } }], + ["hermes_local", { hermesCommand: "/tmp/untrusted" }], + ["gemini_local", { cwd: "/tmp" }], + ["opencode_local", { filesystemSandboxCommand: "/tmp/untrusted" }], + ])("blocks agent-authenticated host settings for %s", async (adapterType, adapterConfig) => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it("allows an agent to update a peer's local model without host settings", async () => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType: "claude_local" }); + const app = await createApp({ type: "agent", agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig: { model: "claude-sonnet-4-5" } })); + + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(mockAgentService.update).toHaveBeenCalledOnce(); + }); + + it("allows a provider credential reference without allowing arbitrary environment variables", async () => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType: "claude_local" }); + const app = await createApp({ type: "agent", agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig: { env: { ANTHROPIC_API_KEY: { type: "secret_ref", secretId: "33333333-3333-4333-8333-333333333333" } } } })); + + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(mockAgentService.update).toHaveBeenCalledOnce(); + }); + + it("blocks an agent from activating inherited host settings by switching onto a local adapter", async () => { + mockAgentService.getById.mockResolvedValue({ + ...baseAgent, + adapterType: "paperclip_runner", + adapterConfig: { env: { PATH: "/tmp" } }, + }); + const app = await createApp({ type: "agent", agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterType: "claude_local", adapterConfig: { engine: "cli" } })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it.each([ + ["create", `/api/companies/${companyId}/agents`], + ["hire", `/api/companies/${companyId}/agent-hires`], + ])("blocks agent-authenticated process adapter commands on %s", async (_label, path) => { + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(path) + .send({ name: "Host process", role: "engineer", adapterType: "process", adapterConfig: { command: "sh" } })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.create).not.toHaveBeenCalled(); + }); + + it.each([ + ["create", `/api/companies/${companyId}/agents`], + ["hire", `/api/companies/${companyId}/agent-hires`], + ])("blocks agent-authenticated local adapter commands on %s", async (_label, path) => { + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(path) + .send({ name: "Local peer", role: "engineer", adapterType: "claude_local", adapterConfig: { engine: "cli", command: "/tmp/untrusted" } })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.create).not.toHaveBeenCalled(); + }); + + it("blocks agent-authenticated rollback into process adapter configuration", async () => { + mockAgentService.getConfigRevision.mockResolvedValue({ + id: "33333333-3333-4333-8333-333333333333", + afterConfig: { adapterType: "process", adapterConfig: { command: "sh" }, runtimeConfig: {} }, + }); + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/33333333-3333-4333-8333-333333333333/rollback`)); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.rollbackConfigRevision).not.toHaveBeenCalled(); + }); + + it("blocks agent-authenticated rollback into local adapter command configuration", async () => { + const revisionId = "33333333-3333-4333-8333-333333333333"; + mockAgentService.getConfigRevision.mockResolvedValue({ + id: revisionId, + afterConfig: { adapterType: "claude_local", adapterConfig: { engine: "cli", command: "/tmp/untrusted" }, runtimeConfig: {} }, + }); + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/${revisionId}/rollback`)); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.rollbackConfigRevision).not.toHaveBeenCalled(); + }); + + it.each(["provisionCommand", "runtimeProvisionCommand", "teardownCommand"])( + "blocks agent-authenticated rollback of workspace %s", + async (commandKey) => { + const revisionId = "33333333-3333-4333-8333-333333333333"; + mockAgentService.getConfigRevision.mockResolvedValue({ + id: revisionId, + afterConfig: { + adapterType: "codex_local", + adapterConfig: { workspaceStrategy: { type: "git_worktree", [commandKey]: "sh -c id" } }, + runtimeConfig: {}, + }, + }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/${revisionId}/rollback`)); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed workspace commands"); + expect(mockAgentService.rollbackConfigRevision).not.toHaveBeenCalled(); + }, + ); + + it("allows an agent to restore a non-process revision for a process peer", async () => { + const revisionId = "33333333-3333-4333-8333-333333333333"; + mockAgentService.getConfigRevision.mockResolvedValue({ + id: revisionId, + afterConfig: { adapterType: "codex_local", adapterConfig: {}, runtimeConfig: {} }, + }); + mockAgentService.rollbackConfigRevision.mockResolvedValue({ + ...baseAgent, + adapterType: "codex_local", + }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/${revisionId}/rollback`)); + + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(mockAgentService.rollbackConfigRevision).toHaveBeenCalledOnce(); + }); + it("blocks agent-authenticated self-updates that set instructions bundle roots", async () => { const app = await createApp({ type: "agent", diff --git a/server/src/__tests__/agents-claude-oauth-binding.test.ts b/server/src/__tests__/agents-claude-oauth-binding.test.ts index 19d496b945..1a133380da 100644 --- a/server/src/__tests__/agents-claude-oauth-binding.test.ts +++ b/server/src/__tests__/agents-claude-oauth-binding.test.ts @@ -15,6 +15,7 @@ import { companySecrets, createDb, environments, + principalPermissionGrants, userSecretDeclarations, userSecretDefinitions, } from "@paperclipai/db"; @@ -245,6 +246,7 @@ describeEmbeddedPostgres("agent service Claude OAuth binding claim", () => { await db.delete(userSecretDeclarations); await db.delete(userSecretDefinitions); await db.delete(adapterAuthSessions); + await db.delete(principalPermissionGrants); await db.delete(agents); await db.delete(environments); await db.delete(companies); diff --git a/server/src/__tests__/agents-pending-approval-config.test.ts b/server/src/__tests__/agents-pending-approval-config.test.ts index 2977ae4d57..7c02a10da1 100644 --- a/server/src/__tests__/agents-pending-approval-config.test.ts +++ b/server/src/__tests__/agents-pending-approval-config.test.ts @@ -8,6 +8,7 @@ import { budgetPolicies, companies, createDb, + principalPermissionGrants, } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, @@ -40,6 +41,7 @@ describeEmbeddedPostgres("pending approval agent config integrity", () => { afterEach(async () => { await db.delete(activityLog); + await db.delete(principalPermissionGrants); await db.delete(budgetPolicies); await db.delete(approvals); await db.delete(agents); diff --git a/server/src/__tests__/agents-service-secret-bindings.test.ts b/server/src/__tests__/agents-service-secret-bindings.test.ts index d804917e72..a155bb72ab 100644 --- a/server/src/__tests__/agents-service-secret-bindings.test.ts +++ b/server/src/__tests__/agents-service-secret-bindings.test.ts @@ -13,6 +13,7 @@ import { companySecretVersions, companySecrets, createDb, + principalPermissionGrants, } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, @@ -46,6 +47,7 @@ describeEmbeddedPostgres("agent service secret binding sync", () => { afterEach(async () => { await db.delete(activityLog); + await db.delete(principalPermissionGrants); await db.delete(companySecretBindings); await db.delete(companySecretVersions); await db.delete(companySecrets); diff --git a/server/src/__tests__/authorization-service.test.ts b/server/src/__tests__/authorization-service.test.ts index b76683562e..02c1b562cd 100644 --- a/server/src/__tests__/authorization-service.test.ts +++ b/server/src/__tests__/authorization-service.test.ts @@ -2606,6 +2606,44 @@ describeEmbeddedPostgres("authorization service", () => { })).resolves.toMatchObject({ allowed: true, reason: "allow_explicit_grant" }); }); + it("keeps a default inbox grant within the responsible user's policy", async () => { + const company = await createCompany(db, "InboxDefaultScoped"); + const actorAgent = await createAgent(db, company.id); + const responsibleUserId = await createUser(db); + const disabledUserId = await createUser(db); + const disallowedUserId = await createUser(db); + await db.insert(companyMemberships).values( + [responsibleUserId, disabledUserId, disallowedUserId].map((principalId) => ({ + companyId: company.id, + principalType: "user" as const, + principalId, + status: "active" as const, + membershipRole: "operator" as const, + })), + ); + await db.insert(userInboxAgentPolicies).values([ + { companyId: company.id, userId: disabledUserId, mode: "disabled" }, + { companyId: company.id, userId: disallowedUserId, mode: "allowlist", allowedAgentIds: [] }, + ]); + await grantAgentPermission(db, company.id, actorAgent.id, "inbox:manage", { responsibleUserOnly: true }); + const decideFor = (userId: string) => authorizationService(db).decide({ + actor: { + type: "agent" as const, + agentId: actorAgent.id, + companyId: company.id, + onBehalfOfUserId: responsibleUserId, + source: "agent_jwt" as const, + }, + action: "inbox:manage" as const, + resource: { type: "company" as const, companyId: company.id }, + scope: { userId }, + }); + + await expect(decideFor(responsibleUserId)).resolves.toMatchObject({ allowed: true, reason: "allow_self" }); + await expect(decideFor(disabledUserId)).resolves.toMatchObject({ allowed: false, reason: "inbox_management_disabled" }); + await expect(decideFor(disallowedUserId)).resolves.toMatchObject({ allowed: false, reason: "inbox_agent_not_allowed" }); + }); + it("enforces user-scoped cross-user inbox grants", async () => { const company = await createCompany(db, "InboxCrossUserScoped"); const actorAgent = await createAgent(db, company.id); diff --git a/server/src/__tests__/built-in-agents.test.ts b/server/src/__tests__/built-in-agents.test.ts index 69de71c730..7a90b5b5ec 100644 --- a/server/src/__tests__/built-in-agents.test.ts +++ b/server/src/__tests__/built-in-agents.test.ts @@ -611,9 +611,9 @@ describeEmbeddedPostgres("built-in agents", () => { expect(result.defaultGrantsEnsured).toBeGreaterThanOrEqual(4); const rootGrantKeys = await permissionKeysForAgent(root.id); - expect(rootGrantKeys).toEqual(expect.arrayContaining(["agents:configure", "skills:create"])); - expect(rootGrantKeys).not.toContain("agents:suggest-changes"); - expect(rootGrantKeys).not.toContain("skills:suggest-changes"); + expect(rootGrantKeys).toEqual(expect.arrayContaining([ + "agents:configure", "agents:suggest-changes", "skills:create", "skills:suggest-changes", + ])); const state = await builtInAgentService(db).get(companyId, "reflection-coach"); expect(state).toMatchObject({ diff --git a/server/src/__tests__/invite-join-grants.test.ts b/server/src/__tests__/invite-join-grants.test.ts index 427fef0fa6..9c10d2d636 100644 --- a/server/src/__tests__/invite-join-grants.test.ts +++ b/server/src/__tests__/invite-join-grants.test.ts @@ -8,18 +8,20 @@ import { normalizeHumanRole, resolveHumanInviteRole, } from "../services/company-member-roles.js"; +import { NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS, newStandardAgentGrantScope } from "../services/agent-permissions.js"; + +const invitedAgentId = "agent-1"; +const defaultAgentGrants = NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({ + permissionKey, + scope: newStandardAgentGrantScope(permissionKey, invitedAgentId), +})); describe("agentJoinGrantsFromDefaults", () => { - it("adds tasks:assign when invite defaults do not specify agent grants", () => { - expect(agentJoinGrantsFromDefaults(null)).toEqual([ - { - permissionKey: "tasks:assign", - scope: null, - }, - ]); + it("adds all new-agent grants when invite defaults do not specify grants", () => { + expect(agentJoinGrantsFromDefaults(null, invitedAgentId)).toEqual(defaultAgentGrants); }); - it("preserves invite agent grants and appends tasks:assign", () => { + it("preserves invite agent grants and appends the missing defaults", () => { expect( agentJoinGrantsFromDefaults({ agent: { @@ -30,17 +32,8 @@ describe("agentJoinGrantsFromDefaults", () => { }, ], }, - }), - ).toEqual([ - { - permissionKey: "agents:create", - scope: null, - }, - { - permissionKey: "tasks:assign", - scope: null, - }, - ]); + }, invitedAgentId), + ).toEqual([{ permissionKey: "agents:create", scope: null }, ...defaultAgentGrants]); }); it("does not duplicate tasks:assign when invite defaults already include it", () => { @@ -54,12 +47,19 @@ describe("agentJoinGrantsFromDefaults", () => { }, ], }, - }), + }, invitedAgentId), ).toEqual([ - { - permissionKey: "tasks:assign", - scope: { projectId: "project-1" }, - }, + { permissionKey: "tasks:assign", scope: { projectId: "project-1" } }, + ...defaultAgentGrants.filter((grant) => grant.permissionKey !== "tasks:assign"), + ]); + }); + + it("preserves an explicit scoped agent configuration grant", () => { + expect(agentJoinGrantsFromDefaults({ + agent: { grants: [{ permissionKey: "agents:configure", scope: { agentIds: ["agent-1"] } }] }, + }, invitedAgentId)).toEqual([ + { permissionKey: "agents:configure", scope: { agentIds: ["agent-1"] } }, + ...defaultAgentGrants.filter((grant) => grant.permissionKey !== "agents:configure"), ]); }); }); diff --git a/server/src/__tests__/plugin-managed-agents.test.ts b/server/src/__tests__/plugin-managed-agents.test.ts index 0b06a55126..30a4725f47 100644 --- a/server/src/__tests__/plugin-managed-agents.test.ts +++ b/server/src/__tests__/plugin-managed-agents.test.ts @@ -17,6 +17,7 @@ import { pluginCompanySettings, pluginManagedResources, plugins, + principalPermissionGrants, } from "@paperclipai/db"; import type { PaperclipPluginManifestV1 } from "@paperclipai/shared"; import { @@ -106,6 +107,7 @@ describeEmbeddedPostgres("plugin-managed agents", () => { await db.delete(pluginManagedResources); await db.delete(pluginCompanySettings); await db.delete(approvals); + await db.delete(principalPermissionGrants); await db.delete(agents); await db.delete(plugins); await db.delete(companies); diff --git a/server/src/__tests__/plugin-managed-routines.test.ts b/server/src/__tests__/plugin-managed-routines.test.ts index 25a02531c2..e5c0a4d0f3 100644 --- a/server/src/__tests__/plugin-managed-routines.test.ts +++ b/server/src/__tests__/plugin-managed-routines.test.ts @@ -12,6 +12,7 @@ import { issues, pluginManagedResources, plugins, + principalPermissionGrants, projects, routineDocuments, routineRuns, @@ -121,6 +122,7 @@ describeEmbeddedPostgres("plugin-managed routines", () => { await db.delete(agentConfigRevisions); await db.delete(activityLog); await db.delete(pluginManagedResources); + await db.delete(principalPermissionGrants); await db.delete(agents); await db.delete(projects); await db.delete(plugins); diff --git a/server/src/routes/access.ts b/server/src/routes/access.ts index d35c4586d7..3aaa1df32e 100644 --- a/server/src/routes/access.ts +++ b/server/src/routes/access.ts @@ -93,7 +93,7 @@ import { normalizeHumanRole, resolveHumanInviteRole, } from "../services/company-member-roles.js"; -import { humanJoinGrantsFromDefaults } from "../services/invite-grants.js"; +import { agentJoinGrantsFromDefaults as standardAgentJoinGrantsFromDefaults, humanJoinGrantsFromDefaults } from "../services/invite-grants.js"; import { collapseDuplicatePendingHumanJoinRequests, findReusableHumanJoinRequest, @@ -2246,22 +2246,13 @@ function grantsFromDefaults( } export function agentJoinGrantsFromDefaults( - defaultsPayload: Record | null | undefined + defaultsPayload: Record | null | undefined, + agentId: string, ): Array<{ permissionKey: (typeof PERMISSION_KEYS)[number]; scope: Record | null; }> { - const grants = grantsFromDefaults(defaultsPayload, "agent"); - if (grants.some((grant) => grant.permissionKey === "tasks:assign")) { - return grants; - } - return [ - ...grants, - { - permissionKey: "tasks:assign", - scope: null - } - ]; + return standardAgentJoinGrantsFromDefaults(defaultsPayload, agentId); } type JoinRequestManagerCandidate = { @@ -4297,7 +4288,8 @@ export function accessRoutes( "active" ); const grants = agentJoinGrantsFromDefaults( - invite.defaultsPayload as Record | null + invite.defaultsPayload as Record | null, + created.id, ); await access.setPrincipalGrants( companyId, diff --git a/server/src/routes/agents.ts b/server/src/routes/agents.ts index 6e4ce96fd2..0f7e74bbee 100644 --- a/server/src/routes/agents.ts +++ b/server/src/routes/agents.ts @@ -2914,6 +2914,69 @@ export function agentRoutes( ); } + function assertNoAgentProcessAdapterMutation(req: Request, adapterType: string, mutatesConfiguration: boolean) { + if (req.actor.type !== "agent" || adapterType !== "process" || !mutatesConfiguration) return; + throw forbidden("Agent keys cannot configure host-executed process adapters."); + } + + const LOCAL_ADAPTER_HOST_COMMAND_KEYS = [ + "command", + "hermesCommand", + "args", + "extraArgs", + "cwd", + "env", + "filesystemSandboxCommand", + ] as const; + + const LOCAL_ADAPTER_CREDENTIAL_ENV_KEYS: Record = { + ...INHERITABLE_AGENT_CREDENTIAL_ENV_KEYS, + gemini_local: ["GEMINI_API_KEY", "GOOGLE_API_KEY"], + kimi_local: ["KIMI_MODEL_API_KEY"], + opencode_local: ["PAPERCLIP_OPENCODE_PROVIDERS", "OPENCODE_AUTH_JSON"], + }; + const POOL_AUTH_OVERRIDE_ENV_KEYS = [ + "OPENAI_API_KEY", "ANTHROPIC_API_KEY", "PAPERCLIP_OPENCODE_PROVIDERS", "OPENCODE_AUTH_JSON", + ] as const; + + async function callerUsesAiConnectionPool(req: Request, companyId: string): Promise { + if (req.actor.type !== "agent" || !req.actor.agentId) return false; + const caller = await svc.getById(req.actor.agentId); + if (!caller || caller.companyId !== companyId) return false; + return asRecord(asRecord(caller.runtimeConfig)?.aiConnection)?.mode === "router"; + } + + function containsOnlyLocalAdapterCredentialRefs( + adapterType: string, + env: unknown, + allowPlainCredentials: boolean, + ): boolean { + const envRecord = asRecord(env); + if (!envRecord) return false; + const allowedKeys = LOCAL_ADAPTER_CREDENTIAL_ENV_KEYS[adapterType] ?? []; + return Object.entries(envRecord).every(([key, value]) => + (allowedKeys.includes(key) && isInheritableCredentialReference(value)) || + (allowPlainCredentials && POOL_AUTH_OVERRIDE_ENV_KEYS.includes(key as typeof POOL_AUTH_OVERRIDE_ENV_KEYS[number]) && asEnvBindingString(value) !== null), + ); + } + + function assertNoAgentLocalAdapterHostCommandMutation( + req: Request, + adapterType: string, + adapterConfig: Record, + allowPlainCredentials = false, + ) { + if (req.actor.type !== "agent" || !adapterType.endsWith("_local")) return; + const changedKeys = LOCAL_ADAPTER_HOST_COMMAND_KEYS.filter((key) => + adapterConfig[key] !== undefined && + (key !== "env" || !containsOnlyLocalAdapterCredentialRefs(adapterType, adapterConfig.env, allowPlainCredentials)), + ); + if (changedKeys.length === 0) return; + throw forbidden( + `Agent keys cannot configure host-executed local adapter settings (${changedKeys.join(", ")}).`, + ); + } + function summarizeAgentUpdateDetails(patch: Record) { const changedTopLevelKeys = Object.keys(patch).sort(); const details: Record = { changedTopLevelKeys }; @@ -4419,6 +4482,13 @@ export function agentRoutes( await assertSelectableAdapterType(rollbackAdapterType); } const rollbackAdapterConfig = asRecord(rollbackConfig.adapterConfig) ?? {}; + assertNoAgentAdapterConfigMutation(req, rollbackAdapterConfig); + assertNoAgentLocalAdapterHostCommandMutation(req, rollbackAdapterType, rollbackAdapterConfig); + assertNoAgentProcessAdapterMutation( + req, + rollbackAdapterType, + true, + ); assertExternalInstructionsAdmin(req, existing); assertExternalInstructionsAdmin(req, { ...existing, @@ -4581,6 +4651,8 @@ export function agentRoutes( rawHireAdapterConfig, ); assertNoAgentAdapterConfigMutation(req, rawHireAdapterConfig); + assertNoAgentLocalAdapterHostCommandMutation(req, hireInput.adapterType, rawHireAdapterConfig, await callerUsesAiConnectionPool(req, companyId)); + assertNoAgentProcessAdapterMutation(req, hireInput.adapterType, Object.keys(rawHireAdapterConfig).length > 0); const hiredAgentId = randomUUID(); const authInheritance = await applyHiringAgentAuthInheritance( req, @@ -4890,6 +4962,8 @@ export function agentRoutes( rawCreateAdapterConfig, ); assertNoAgentAdapterConfigMutation(req, rawCreateAdapterConfig); + assertNoAgentLocalAdapterHostCommandMutation(req, createInput.adapterType, rawCreateAdapterConfig, await callerUsesAiConnectionPool(req, companyId)); + assertNoAgentProcessAdapterMutation(req, createInput.adapterType, Object.keys(rawCreateAdapterConfig).length > 0); const agentId = randomUUID(); const requestedAdapterConfig = applyCodexLocalKeyIsolation( companyId, @@ -5528,6 +5602,7 @@ export function agentRoutes( const touchesAdapterConfiguration = hasOwn(patchData, "adapterType") || hasOwn(patchData, "adapterConfig"); + assertNoAgentProcessAdapterMutation(req, requestedAdapterType, touchesAdapterConfiguration); if (touchesAdapterConfiguration) { assertExternalInstructionsAdmin(req, existing); const existingAdapterConfig = asRecord(existing.adapterConfig) ?? {}; @@ -5575,6 +5650,13 @@ export function agentRoutes( if (requestedAdapterType === "paperclip_runner") { rawEffectiveAdapterConfig = normalizePaperclipRunnerAdapterConfig(requestedAdapterType, rawEffectiveAdapterConfig); } + if (changingAdapterType || requestedAdapterConfig) { + assertNoAgentLocalAdapterHostCommandMutation( + req, + requestedAdapterType, + changingAdapterType ? rawEffectiveAdapterConfig : (requestedAdapterConfig ?? {}), + ); + } const existingRunnerProvider = existing.adapterType === "paperclip_runner" ? existingAdapterConfig.provider diff --git a/server/src/services/agent-permissions.ts b/server/src/services/agent-permissions.ts index e628a62ec5..24f9bc29d0 100644 --- a/server/src/services/agent-permissions.ts +++ b/server/src/services/agent-permissions.ts @@ -1,4 +1,31 @@ -import { LOW_TRUST_REVIEW_PRESET } from "@paperclipai/shared"; +import { LOW_TRUST_REVIEW_PRESET, type PermissionKey } from "@paperclipai/shared"; + +/** Direct grants for newly activated standard agents. Existing agents are not backfilled. */ +export const NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS = [ + "agents:configure", + "agents:suggest-changes", + "skills:create", + "skills:suggest-changes", + "tools:manage_connections", + "tools:manage_profiles", + "tools:view_audit", + "audit:view_agent_actions", + "tools:use", + "tools:manage_runtime", + "inbox:manage", + "tasks:assign", + "tasks:assign_scope", + "tasks:manage_active_checkouts", +] as const satisfies readonly PermissionKey[]; + +export function newStandardAgentGrantScope( + permissionKey: PermissionKey, + agentId: string, +): Record | null { + if (permissionKey === "tasks:assign_scope") return { subtreeRootAgentId: agentId }; + if (permissionKey === "inbox:manage") return { responsibleUserOnly: true }; + return null; +} export type NormalizedAgentPermissions = Record & { canCreateAgents: boolean; diff --git a/server/src/services/agents.ts b/server/src/services/agents.ts index 3640a90d4e..13450be91f 100644 --- a/server/src/services/agents.ts +++ b/server/src/services/agents.ts @@ -17,6 +17,7 @@ import { issueExecutionDecisions, issues, issueComments, + principalPermissionGrants, } from "@paperclipai/db"; import { AGENT_DEFAULT_MAX_CONCURRENT_RUNS, @@ -38,8 +39,8 @@ import { syncAgentAdapterEnvBindings, } from "./agent-secret-bindings.js"; import { logActivity } from "./activity-log.js"; +import { NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS, newStandardAgentGrantScope, normalizeAgentPermissions, permissionsImplyLowTrust } from "./agent-permissions.js"; import { recordAgentStatusEvent, recordResourceCreationEvent } from "./resource-lifecycle-events.js"; -import { normalizeAgentPermissions } from "./agent-permissions.js"; import { REDACTED_EVENT_VALUE, sanitizeRecord } from "../redaction.js"; import { assertClaudeOAuthBindingInvariant, @@ -944,6 +945,20 @@ export function agentService(db: Db) { }) .returning() .then((rows) => rows[0]); + // New standard agents receive the standard direct grants at activation. + // Low-trust and bundled agents keep their explicit, narrower grants. + if (created.status !== "pending_approval" && !permissionsImplyLowTrust(normalizedPermissions) && + !readBuiltInAgentMarker(created.metadata)) { + await tx.insert(principalPermissionGrants).values( + NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({ + companyId, + principalType: "agent" as const, + principalId: created.id, + permissionKey, + scope: newStandardAgentGrantScope(permissionKey, created.id), + })), + ).onConflictDoNothing(); + } if (options?.aiConnectionInstall) { const install = options.aiConnectionInstall; const connectionIds = [...new Set([install.connectionId, ...(install.memberConnectionIds ?? [])])]; @@ -1084,6 +1099,11 @@ export function agentService(db: Db) { await tx.delete(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, id)); await tx.delete(agentApiKeys).where(eq(agentApiKeys.agentId, id)); await tx.delete(agentRuntimeState).where(eq(agentRuntimeState.agentId, id)); + await tx.delete(principalPermissionGrants).where(and( + eq(principalPermissionGrants.companyId, existing.companyId), + eq(principalPermissionGrants.principalType, "agent"), + eq(principalPermissionGrants.principalId, id), + )); const deleted = await tx .delete(agents) .where(eq(agents.id, id)) @@ -1151,6 +1171,17 @@ export function agentService(db: Db) { }); } await syncAgentSecretBindings(updated, txDb, existing.adapterConfig); + if (!permissionsImplyLowTrust(updated.permissions) && !readBuiltInAgentMarker(updated.metadata)) { + await tx.insert(principalPermissionGrants).values( + NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({ + companyId: updated.companyId, + principalType: "agent" as const, + principalId: updated.id, + permissionKey, + scope: newStandardAgentGrantScope(permissionKey, updated.id), + })), + ).onConflictDoNothing(); + } await recordResourceCreationEvent(txDb, existing.companyId, "agent", updated.id); const agent = await agentService(txDb).getById(updated.id); if (!agent) { diff --git a/server/src/services/authorization.ts b/server/src/services/authorization.ts index d16e9f30dc..e3edf7a99f 100644 --- a/server/src/services/authorization.ts +++ b/server/src/services/authorization.ts @@ -2060,7 +2060,8 @@ export function authorizationService(db: Db | DbTransaction) { // implicit default-open policy remains responsible-user-only so an // absent row never becomes a company-wide cross-user grant. const grant = await findGrant(companyId, "agent", actorAgentId, "inbox:manage"); - if (grant && (await scopeAllows(db, companyId, grant.scope, { userId: targetUserId }))) { + if (grant && grant.scope?.responsibleUserOnly !== true && + (await scopeAllows(db, companyId, grant.scope, { userId: targetUserId }))) { return allow({ action: input.action, reason: "allow_explicit_grant", diff --git a/server/src/services/invite-grants.ts b/server/src/services/invite-grants.ts index 32c9aa42fe..b2141e0930 100644 --- a/server/src/services/invite-grants.ts +++ b/server/src/services/invite-grants.ts @@ -1,5 +1,6 @@ import { PERMISSION_KEYS } from "@paperclipai/shared"; import type { HumanCompanyMembershipRole } from "@paperclipai/shared"; +import { NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS, newStandardAgentGrantScope } from "./agent-permissions.js"; import { grantsForHumanRole } from "./company-member-roles.js"; export function grantsFromDefaults( @@ -38,22 +39,21 @@ export function grantsFromDefaults( } export function agentJoinGrantsFromDefaults( - defaultsPayload: Record | null | undefined + defaultsPayload: Record | null | undefined, + agentId: string, ): Array<{ permissionKey: (typeof PERMISSION_KEYS)[number]; scope: Record | null; }> { const grants = grantsFromDefaults(defaultsPayload, "agent"); - if (grants.some((grant) => grant.permissionKey === "tasks:assign")) { - return grants; + // Invitation approval replaces the whole grant set after agent creation. + // Retain new-agent defaults, but never widen an explicitly scoped grant. + for (const permissionKey of NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS) { + if (!grants.some((grant) => grant.permissionKey === permissionKey)) { + grants.push({ permissionKey, scope: newStandardAgentGrantScope(permissionKey, agentId) }); + } } - return [ - ...grants, - { - permissionKey: "tasks:assign", - scope: null, - }, - ]; + return grants; } export function humanJoinGrantsFromDefaults( From 63f3aa2dbfc5a9474301cd8e11c8311b4826f9a8 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 6 Oct 2026 07:49:47 -0500 Subject: [PATCH 11/12] fix(runner): continue restart-interrupted Codex turns (#15297) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs retain their provider conversation across server restarts. > - A dead runner can restore a Codex conversation after its active turn is lost. > - The old recovery path synthesized a failed task result from that interruption. > - The task then required operator action even though its conversation and workspace were available. > - This pull request preserves the interruption cause and uses the admitted restart attempt for one continuation in the same conversation. > - The agent can reconcile unfinished actions and complete the current request without resending the original task. ## Linked Issues or Issue Description Related: #12845 added native restart recovery. #15042 covers admission during shutdown. #14796 covers legacy shutdown recovery. This change covers a lost native Codex turn after successful conversation restoration. **What happened?** After a server restart killed the local runner, Paperclip restored the saved Codex thread. Runnerd found that the old turn was no longer active. It synthesized a failed terminal and a needs-review result from the last progress message. The transport also discarded the terminal error when it reconstructed thread history. The task failed instead of continuing. **Expected behavior** After proving that the old process stopped and admitting a bounded recovery attempt, resume the current request in the same conversation. Preserve the workspace. Inspect unfinished actions before proceeding. Keep real provider failures, accepted results, intentional stops, unknown unreconciled effects, and exhausted attempts subject to their existing rules. **Steps to reproduce** 1. Start a local native Codex run and leave its turn active. 2. Kill the isolated runner and provider processes, as can happen during a server restart. 3. Restore the same provider thread with no active turn. 4. Observe the synthetic task failure. The new real-process regression reproduces this boundary with a scripted provider. ## What Changed - Record an explicit recoverable process-loss cause without inventing a task result. - Preserve terminal errors and prior turns in reconstructed provider history. Recover the authoritative saved result when adopting an accepted continuation. - Send one continuation in the same conversation for an admitted dead-runner recovery. Require reconciliation of unfinished commands and external actions. - Persist the interrupted terminal before submission and retain the existing recovery marker across another controller loss. - Keep provider attempt limits, terminal failures, and intentional cancellation behavior. - Add red/green regressions, real process-kill coverage, restart checkpoint coverage, and retry-budget coverage. Document the behavior and run-log evidence. ## Verification - Red: the new native runtime regression rejected with `NativeProviderTerminalFailure` on the original code; the Rust restore regression found a missing recovery cause. - Red/green: if restoring the conversation fails and replacement is allowed, the replacement receives the full task and fresh-session handoff. Both prepared and legacy execution inputs are covered. - Red: a second controller crash after the provider accepted the continuation caused an extra `turn/start`. The regression now proves there are exactly two submissions total: the original and its continuation. - Green: focused runtime, backend, driver recovery, and real-process restart suites (207 tests). After the final history/result changes, driver recovery and real-process restart suites passed again (36 tests). - Green: complete Codex transport suite (186 tests), server restart classification/database integration suites (34 tests), and Rust Codex provider suite (92 passed, 2 ignored). - Full `pnpm -r typecheck` and `pnpm build` passed on `60141e649`. The subsequent replacement-prompt guard passed the Runner TypeScript check and the complete runtime plus process-restart suites (148 tests). - Local full-suite attempt: `pnpm test:run` reported two failures in the untouched chat integration suite. Both passed individually, and the complete chat suite passed on rerun (1,063 tests). After all remote test shards passed, the duplicate serial local run was stopped with SIGINT; it is not claimed as a full local-suite pass. - Latest-head CI (`b1297dcd4`): 55 successful checks and 4 intentionally skipped checks, including all test shards, typecheck, build, native Runner verification, end-to-end tests, and canary dry run. [CI run](https://github.com/paperclipai/paperclip/actions/runs/37405346303). - Greptile: 5/5 on the latest head, with no unresolved review threads. The PR is mergeable. - The process tests use the real runner binary and a scripted Codex provider. They do not call a live model service. ## Risks - This changes local Codex recovery after process loss. A continuation can execute more work in the retained conversation. Its prompt requires state inspection before repeating an uncertain action; the system does not replay tool calls. - Recovery shares the existing three-attempt budget and one-shot continuation marker. Real failures and older unmarked failed checkpoints are not reopened. - No database migration or API change is required. ## Model Used - OpenAI Codex, based on GPT-6. The exact model ID and context-window size are not exposed in this session. - Capabilities: reasoning, source inspection, tool use, code editing, code execution, and test analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/execution-semantics.md | 13 ++ doc/run-log-events.md | 8 ++ .../src/bin/fake-codex-app-server.rs | 4 + .../runner-core/src/provider_backend.rs | 30 +++-- .../runner-core/tests/codex_provider.rs | 59 +++++++++ .../src/backends/harness-driver-backend.ts | 1 + .../src/contracts/native-session-backend.ts | 27 +++- .../codex/codex-app-server-driver-impl.ts | 8 +- .../codex-app-server-driver.recovery.test.ts | 23 ++++ .../drivers/codex/codex-session-terminal.ts | 8 ++ .../src/live/runnerd-codex-transport.ts | 35 ++++- .../live/runnerd-restart-continuation.test.ts | 125 ++++++++++++++++++ .../src/native-session-runtime.test.ts | 86 +++++++++++- .../src/native-session-runtime.ts | 54 ++++++++ .../native-runtime/native-restart-recovery.ts | 4 +- ...unner-restart-recovery.integration.test.ts | 24 ++++ .../native-runtime/native-session-executor.ts | 1 + server/src/vendor/paperclip-runner/index.ts | 1 + 18 files changed, 492 insertions(+), 19 deletions(-) create mode 100644 packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts diff --git a/doc/execution-semantics.md b/doc/execution-semantics.md index f94909d6c8..701852da70 100644 --- a/doc/execution-semantics.md +++ b/doc/execution-semantics.md @@ -1000,6 +1000,19 @@ recording the proof and successor lineage. Retained provider files are not edite Bootstrap retries, exact-checkpoint resumes, and fresh replacement sessions share three total provider attempts, including the original attempt. Linked run IDs, controller restarts, and duplicate wakes do not reset this budget. Automatic attempts retain the 30-second delay. Replacement scheduling and predecessor lineage commit together, with one successor per predecessor and admission through the normal task locks, authorization, pause, approval, and budget gates. +An exact local Codex restart can restore the conversation after losing its active +turn. Runnerd records this as `provider_turn_lost_on_restore`, with no invented +task result. The admitted restart attempt may send one continuation in that same +conversation. It preserves the workspace and asks the agent to reconcile unfinished +commands and external actions before proceeding. It does not resend the task or +tool calls. An outcome that cannot be reconciled remains a blocker. The continuation +uses the existing durable one-shot recovery marker; recovery adopts an accepted +turn if the controller dies before checkpointing its ID. The interrupted terminal +is checkpointed before submission. Real provider failures, accepted semantic +results, intentional stops, and historical unmarked failures keep their existing +terminal behavior. This continuation uses the already charged restart attempt and +does not reset the provider-attempt budget. + Provider execution and control-plane finalization have different clocks. A healthy provider can think or execute a long tool without output. Once execution settles, recovery and finalization control steps have a 60-second deadline, checked on startup and every 15 seconds. With a healthy database and scheduler, an abandoned transition must be repaired or surfaced within 90 seconds. Terminal persistence must not wait on provider cleanup or publication; a late finalizer cannot change a reassigned or closed task or release another run's locks. Historical ambiguous runs are never automatically replayed after an upgrade. Every continuation carries the triggering request, ordered user direction, interaction outcomes, completed work, and explicit history coverage. A delivered message remains part of the task's request after its connection or approval resolves. The original title is background; a completed Notion read does not satisfy a later Gmail request. Author and source-trust boundaries survive rendering into both native and legacy prompts. Missing required history must be fetched before dispatch rather than described as complete. diff --git a/doc/run-log-events.md b/doc/run-log-events.md index 1757c04fe1..ed8f723652 100644 --- a/doc/run-log-events.md +++ b/doc/run-log-events.md @@ -110,6 +110,14 @@ proofs, encryption keys, environment variables, provider credentials, command arguments, or an unsanitized stderr stream. Detailed failed-attempt diagnostics remain in the bounded `native_run_finalizations.recovery_history` ledger. +For a local Codex turn lost while runnerd was stopped, the canonical `turn.failed` +event retains `error.code: "provider_turn_lost_on_restore"` and +`error.recoverable: true`. It records the interrupted turn, not a final task +outcome. There is no synthesized result for that interruption. An admitted +same-session continuation emits its own `turn.submitted`, accepted turn ID, and +terminal events; only the final outcome closes the run. Reconstructed thread +history retains the original error so reconciliation and live delivery agree. + ## Native Local Process Stop Evidence The server writes `native.local_process_stopped` in the same transaction that diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs b/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs index ce3b9e3e4b..b1ed069b09 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs @@ -652,6 +652,9 @@ fn run() -> Result<(), Box> { .iter() .any(|value| value == "--require-codex-home-auth"); let durable_turn_ids = args.iter().any(|value| value == "--durable-turn-ids"); + let hold_first_durable_turn = args + .iter() + .any(|value| value == "--hold-first-durable-turn"); let durable_tool_ids = args.iter().any(|value| value == "--durable-tool-ids"); let expected_canonical_task_context_file = argument(&args, "--expected-canonical-task-context-file"); @@ -1348,6 +1351,7 @@ fn run() -> Result<(), Box> { .checked_add(1) .ok_or("fake provider turn sequence exhausted")?; } + let hold_turn = hold_turn || (hold_first_durable_turn && state.next_turn == 1); if reject_second_turn_start && turn_start_count == 2 { send(json!({ "method": "warning", diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs b/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs index 545544c73e..8d038f5fe1 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs @@ -2103,17 +2103,29 @@ impl CodexCommandExecutor { "providerTurnId": previous_active_turn_id, "status": "failed", "providerTerminalObserved": false, + "error": if provider_label == "codex" { json!({ + "code": "provider_turn_lost_on_restore", + "recoverable": true, + "message": "The runner restored the conversation after process loss, but the previous turn is no longer active.", + }) } else { Value::Null }, }), }; - let outcome = terminal_events( - state, - "turn.failed", - state.goal.as_ref().map(|goal| goal.status.as_str()), - ); - state.extend_terminal_events(with_terminal_outcome( - vec![provider_terminal], - outcome, - ))?; + if provider_label == "codex" { + // No provider result was observed. Preserve the lost + // turn as a fact, but leave the run open for the + // controller's bounded same-conversation recovery. + state.push_terminal_event(provider_terminal)?; + } else { + let outcome = terminal_events( + state, + "turn.failed", + state.goal.as_ref().map(|goal| goal.status.as_str()), + ); + state.extend_terminal_events(with_terminal_outcome( + vec![provider_terminal], + outcome, + ))?; + } } } else { state.push_event(reconciled)?; diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs index 9550fb2e95..34e3b46d0c 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs @@ -3484,6 +3484,65 @@ fn durable_backend_replays_pending_tool_calls_without_mutating_the_event_queue() fs::remove_dir_all(directory).expect("remove Codex integration-test directory"); } +#[test] +fn restart_lost_turn_retains_a_recoverable_cause_without_inventing_a_result() { + let directory = temporary_directory("restart-lost-turn"); + let config = provider_config( + &directory, + &["--durable-turn-ids", "--linger-after-turn-start"], + ); + let mut first = CodexCommandExecutor::new(&directory); + first + .execute(&command( + "prepare", + 1, + "run.prepare", + json!({"provider": config}), + )) + .unwrap(); + first + .execute(&command("open", 2, "session.open", json!({}))) + .unwrap(); + first + .execute(&command( + "start", + 3, + "turn.start", + json!({"text": "Watch the existing check run."}), + )) + .unwrap(); + // The process disappeared while its command was in flight. Codex restores + // the conversation, but cannot restore the running turn. + drop(first); + fs::write( + directory.join("fake-state.json"), + serde_json::to_vec(&json!({ + "threadId": "codex-thread-1", "activeTurnId": null, "nextTurn": 1, + })) + .unwrap(), + ) + .unwrap(); + let mut restored = CodexCommandExecutor::new(&directory); + restored + .execute(&command("snapshot", 4, "session.snapshot", json!({}))) + .unwrap(); + let events = poll_and_ack(&mut restored).unwrap(); + let interrupted = events + .iter() + .find(|event| event.event_type == "turn.failed") + .unwrap(); + assert_eq!( + interrupted.payload["error"]["code"], + "provider_turn_lost_on_restore" + ); + assert_eq!(interrupted.payload["error"]["recoverable"], true); + assert!(!events + .iter() + .any(|event| event.event_type == "run.result.proposed")); + restored.shutdown().unwrap(); + fs::remove_dir_all(directory).unwrap(); +} + #[test] fn durable_backend_settles_pending_tools_when_recovery_finds_the_turn_ended() { let directory = temporary_directory("durable-tool-ended-offline"); diff --git a/packages/paperclip-runner/src/backends/harness-driver-backend.ts b/packages/paperclip-runner/src/backends/harness-driver-backend.ts index fd280b5df8..143dd877be 100644 --- a/packages/paperclip-runner/src/backends/harness-driver-backend.ts +++ b/packages/paperclip-runner/src/backends/harness-driver-backend.ts @@ -673,6 +673,7 @@ class HarnessNativeSession implements NativeSession { // The provider has no active turn to interrupt yet. Do not launch work // whose events cancellation would suppress and leave the owner waiting. if (this.#explicitlyCancelled) throw new Error("native_session_cancelled"); + this.#terminal = null; try { const started = await this.#session.startTurn(input); this.#assertProtocolIntegrity(); diff --git a/packages/paperclip-runner/src/contracts/native-session-backend.ts b/packages/paperclip-runner/src/contracts/native-session-backend.ts index 7a84e9577d..87af9e2ad0 100644 --- a/packages/paperclip-runner/src/contracts/native-session-backend.ts +++ b/packages/paperclip-runner/src/contracts/native-session-backend.ts @@ -63,7 +63,7 @@ export interface PersistedNativeSession { terminal?: PrpTerminalState | null; activeTurnId?: string | null; terminalTurns?: PersistedHarnessTurnTerminal[]; - /** Durable at-most-once marker for a resultless terminal recovery turn. */ + /** At-most-once marker for resultless disposition repair or restart continuation. */ dispositionOnlyRecoveryConsumed?: boolean; dispositionOnlyRecoveryTurnId?: string | null; pendingRuntimeRequests?: HarnessRuntimeRequest[]; @@ -77,6 +77,31 @@ export interface NativeSessionRecoveryResult { reason?: string; } +/** Runner-owned cause: the process was restored, but its active turn was lost. */ +export const NATIVE_RESTART_INTERRUPTION_CODE = "provider_turn_lost_on_restore"; + +export function isNativeRestartInterruption(error: unknown): boolean { + if (!error || typeof error !== "object" || Array.isArray(error)) return false; + const failure = error as Record; + return failure.code === NATIVE_RESTART_INTERRUPTION_CODE && failure.recoverable === true; +} + +/** The terminal fingerprint survives a crash between interruption and continuation. */ +export function nativeRestartInterruptedTurnId(snapshot: Pick): string | null { + if (snapshot.driverKind !== "codex_app_server" || snapshot.activeTurnId != null || + snapshot.semanticResult != null || snapshot.dispositionOnlyRecoveryConsumed || + snapshot.goal != null || snapshot.pendingRuntimeRequests?.length) return null; + const terminal = Array.isArray(snapshot.terminalTurns) ? snapshot.terminalTurns.at(-1) : null; + if (!terminal || typeof terminal.fingerprint !== "string" || !terminal.turnId) return null; + try { + const fingerprint = JSON.parse(terminal.fingerprint); + return fingerprint?.terminalState === "failed" && fingerprint.result === null && + isNativeRestartInterruption(fingerprint.error) ? terminal.turnId : null; + } catch { return null; } +} + /** * Cancellation is a synchronous authority transition followed by passive * provider cleanup. Once `cancel` returns, the provider session must no longer diff --git a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts index 8be0449173..6f3b5ad8a7 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts @@ -17,7 +17,7 @@ import type { PersistedHarnessSession, PersistedHarnessTurnTerminal, } from "../../contracts/harness-driver.js"; -import { NativeSessionProtocolIntegrityError } from "../../contracts/native-session-backend.js"; +import { NativeSessionProtocolIntegrityError, nativeRestartInterruptedTurnId } from "../../contracts/native-session-backend.js"; import { HarnessReconciliationError } from "../../contracts/harness-driver.js"; import { CODEX_CODEX_PROTOCOL_VERSION, @@ -528,6 +528,12 @@ export class CodexAppServerDriver implements HarnessDriver { turns.forEach((turn, index) => { if (terminalIds.has(text(turn.id))) lastKnownTerminalIndex = index; }); + if (nativeRestartInterruptedTurnId({ + ...snapshot, semanticResult: null, + }) && (!providerHistoryIsArray || lastKnownTerminalIndex < 0)) { + await cancellation.wait(cancellation.close()); + return { recovered: false, reason: "restart interruption history is incomplete" }; + } // Releasing a consumed marker requires both an actual history array // and a checkpointed terminal that anchors its ordering. An array that // omits every durable terminal may be truncated or inconsistent, so diff --git a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts index dc4e03e33c..7c76d0edff 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts @@ -45,6 +45,29 @@ import { import { NativeSessionProtocolIntegrityError } from "../../contracts/native-session-backend.js"; describe("Codex app-server Codex driver", () => { + it("does not resend a restart continuation when its history anchor is missing", async () => { + const first = new FakeCodexTransport(); + const second = new FakeCodexTransport(); + const driver = makeDriver([first, second]); + const original = await driver.openSession({ + runId: "run-restart-anchor", normalizedSessionId: "session-restart-anchor", workingDirectory: WORKSPACE, + }); + await original.startTurn({ message: { role: "user", text: "Finish the request." } }); + first.push("turn/completed", { threadId: "thread-1", turn: { + id: "turn-1", status: "failed", error: { code: "provider_turn_lost_on_restore", recoverable: true }, items: [], + } }); + await collectUntilTerminal(original.events()); + const snapshot = await original.snapshot(); + await original.close({ reason: "controller lost before continuation checkpoint" }); + second.readResponse = { thread: { id: "thread-1", sessionId: "provider-session-1", cwd: WORKSPACE, + turns: [{ id: "uncheckpointed-turn", status: "completed", items: [] }], + } }; + await expect(driver.recoverSession?.(snapshot)).resolves.toMatchObject({ + recovered: false, reason: "restart interruption history is incomplete", + }); + expect(second.calls.some(call => call.method === "turn/start")).toBe(false); + }); + it.each([null, "checkpointed-prior-turn"])("recovers an autonomous goal turn beyond checkpoint %s", async (checkpointTurnId) => { const first = new FakeCodexTransport(); const second = new FakeCodexTransport(); diff --git a/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts b/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts index f2c5d70f29..47612b9267 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts @@ -1,4 +1,5 @@ import { HarnessReconciliationError } from "../../contracts/harness-driver.js"; +import { isNativeRestartInterruption } from "../../contracts/native-session-backend.js"; import type { PrpStructuredRunResult } from "../../protocol/replay-contract.js"; import { boundedCodexPayload as boundedPayload, isRetainableCodexPayload } from "./codex-boundaries.js"; import type { CodexSessionState } from "./codex-session-state.js"; @@ -168,6 +169,13 @@ export function mapTerminalTurn( { turnId }, ); finalize(state, status); + if (status === "failed" && isNativeRestartInterruption(turn.error) && state.result === null && + !state.dispositionOnlyRecoveryConsumed) { + // Orchestration may use the paid restart attempt for one same-thread + // continuation. Its accepted turn id uses the existing durable marker. + state.terminal = false; + state.dispositionOnlyRecoveryAvailable = true; + } } function terminalFingerprint( diff --git a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts index 5fbca7d8fd..68a7c84cc8 100644 --- a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts +++ b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts @@ -3581,20 +3581,31 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { if (method === "thread/turns/list") { data = turns.map(turn => ({ ...turn, items: [], itemsView: "notLoaded" })); } else { - if (params.turnId !== this.#turnId) throw new Error("codex_history_unavailable: requested turn is outside the retained runner event window"); + if (!turns.some(turn => turn.id === params.turnId)) throw new Error("codex_history_unavailable: requested turn is outside the retained runner event window"); const items = new Map>(); + let semanticResultItem: Record | null = null; let observedTurn = ""; let observedStart = false; for (const event of this.#core?.store.state.committedEvents ?? []) { + if (event.envelope.runId !== this.#core?.store.state.identity.runId) continue; const payload = record(record(event.envelope.payload).payload); if (event.eventType === "turn.started") observedTurn = String(payload.providerTurnId ?? payload.turnId ?? record(payload.turn).id ?? ""); if (event.eventType === "turn.started" && observedTurn === params.turnId) observedStart = true; + if (event.eventType === "run.result.proposed" && observedTurn === params.turnId) { + // Reconciliation can precede notification delivery. Recover the + // runner's authoritative result with the exact retained turn, + // rather than launching work again just to obtain a disposition. + const id = `runner-result-${event.sourceSeq}`; + semanticResultItem = { turnId: observedTurn, item: { id, type: "agentMessage", text: JSON.stringify(payload) } }; + } if (event.eventType !== "item.completed" || observedTurn !== params.turnId) continue; const item = record(rehydrateRunnerdItemNotification(payload, this.#threadId, observedTurn).item); if (typeof item.id === "string") items.set(item.id, { turnId: observedTurn, item }); } if (!observedStart) throw new Error("codex_history_incomplete: requested turn start is outside the retained runner event window"); data = [...items.values()]; + // Runner authority wins over schema-shaped prose in an assistant item. + if (semanticResultItem) data.push(semanticResultItem); } if (params.sortDirection === "desc") data.reverse(); const offset = params.cursor == null ? 0 : Number(params.cursor); @@ -3655,8 +3666,12 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { await new Promise((resolveWait) => setTimeout(resolveWait, 10)); } if (terminal !== undefined) { + const payload = record(record(terminal.envelope.payload).payload); recoveredTurns.push({ id: this.#turnId, + // Reconciliation must retain the cause, including the runner's + // explicit process-loss marker, rather than inventing error:null. + error: payload.error ?? record(payload.turn).error ?? null, status: terminal.eventType === "turn.completed" ? "completed" @@ -3668,6 +3683,24 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { }); } } + // A controller can lose the checkpoint after a continuation is accepted. + // Keep its prior terminal as the history anchor, so driver recovery can + // adopt the later accepted turn instead of submitting it again. Items + // remain lazy and fail closed if their start left the retained window. + const priorTerminals = new Map>(); + for (const event of this.#core?.store.state.committedEvents ?? []) { + if (event.envelope.runId !== this.#core?.store.state.identity.runId || + !["turn.completed", "turn.failed", "turn.interrupted", "turn.cancelled"].includes(event.eventType)) continue; + const payload = record(record(event.envelope.payload).payload); + const turnId = payload.providerTurnId ?? payload.turnId ?? record(payload.turn).id; + if (typeof turnId !== "string" || !turnId || turnId === this.#turnId) continue; + priorTerminals.set(turnId, { + id: turnId, + status: event.eventType.slice("turn.".length), + error: payload.error ?? record(payload.turn).error ?? null, + }); + } + recoveredTurns.unshift(...priorTerminals.values()); this.#recoveryTurnBindingPending = false; this.#pumpEvents(); return { diff --git a/packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts b/packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts new file mode 100644 index 0000000000..e1e1095165 --- /dev/null +++ b/packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts @@ -0,0 +1,125 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { expect, it, vi } from "vitest"; +import { createRunnerdNativeSessionBackend } from "../backends/codex-native-backend.js"; +import type { NativeExecutionInputV1 } from "../contracts/native-execution.js"; +import type { PersistedNativeSession } from "../contracts/native-session-backend.js"; +import type { PrpEvent } from "../protocol/replay-contract.js"; +import { executeNativeSession } from "../native-session-runtime.js"; +import { createRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "./runnerd-codex-transport.js"; + +it.each(["none", "before-start", "after-start"] as const)("completes after the real runner and provider are killed (second crash: %s)", async (crash) => { + const root = await mkdtemp(join(tmpdir(), "runner-restart-continuation-")); + const identity = { runId: "run-restart", sessionId: "session-restart", companyId: "company-restart", issueId: "issue-restart", agentId: "agent-restart" }; + const execution: NativeExecutionInputV1 = { + schema: "paperclip.native-execution-input.v1", + binding: { runId: identity.runId, companyId: identity.companyId, issueId: identity.issueId, agentId: identity.agentId, executionWorkspaceId: "workspace-restart" }, + session: { normalizedSessionId: identity.sessionId, driverKind: "codex_app_server", protocolVersion: 1 }, + task: { identifier: "TEST-1", title: "Finish checks", description: null, prompt: "Watch the already running checks and finish the request.", workMode: "standard" }, + workspace: { cwd: root, repoUrl: null, repoRef: null, branchName: null }, + provider: { kind: "codex", model: null }, + completionContract: { id: "contract-restart", sha256: "digest", schemaVersion: "paperclip.completion-contract.v1", contract: { + revision: "1", objective: "Finish checks", criteria: [{ id: "checks", requirement: "Check the outcome" }], + } }, + interactionResponses: [], credentialBindings: [], + }; + const options = { + runnerBinary: defaultCapabilityRunnerdBinary(), + codexCommand: resolve(import.meta.dirname, "../../runner/target/debug/fake-codex-app-server"), + codexArgs: ["--state-file", join(root, "fake-state.json"), "--call-log", join(root, "calls.log"), "--durable-turn-ids", "--hold-first-durable-turn"], + stateDirectory: root, + prpIdentity: { runnerInstanceId: "runner-restart", environmentLeaseId: "lease-restart", runId: identity.runId, normalizedSessionId: identity.sessionId, turnId: "turn-restart", itemId: "item-restart" }, + }; + const first = createRunnerdCodexTransport(options); + let restored: ReturnType | undefined; + const alive = (pid: number | null) => { + if (!pid) return false; + try { process.kill(pid, 0); return true; } catch { return false; } + }; + const kill = (pid: number | null) => { + if (!pid) return; + try { process.kill(pid, "SIGKILL"); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error; } + }; + try { + const backend = createRunnerdNativeSessionBackend(execution, { runnerInstanceId: "runner-restart", transportFactory: () => first.transport }); + const session = await backend.openSession({ identity, workingDirectory: root }); + await session.startTurn({ message: { role: "user", text: execution.task.prompt } }); + const persisted = await session.snapshot(); + expect(persisted.activeTurnId).toBe("provider-turn-1"); + const durableEvents: PrpEvent[] = []; + const initialEvents = session.events()[Symbol.asyncIterator](); + while ((durableEvents.at(-1)?.sourceSeq ?? 0) < Number(persisted.cursor)) { + const event = await initialEvents.next(); + if (event.done) throw new Error("initial event history ended early"); + durableEvents.push(event.value); + } + const owner = first.evidence(); + await first.detachControllerForRestart(); + kill(owner.runnerPid); + kill(owner.providerPid); + await vi.waitFor(() => { expect(alive(owner.runnerPid)).toBe(false); expect(alive(owner.providerPid)).toBe(false); }); + // Simulate Codex's process-loss history: the thread remains, its in-memory + // turn does not. Keep the turn counter to prevent provider identity reuse. + const providerState = JSON.parse(await readFile(join(root, "fake-state.json"), "utf8")); + providerState.activeTurnId = null; + await writeFile(join(root, "fake-state.json"), JSON.stringify(providerState)); + restored = createRunnerdCodexTransport({ ...options, resumeDynamicTools: [], resumeActiveTurnId: persisted.activeTurnId }); + let recovering = createRunnerdNativeSessionBackend(execution, { runnerInstanceId: "runner-restart", transportFactory: () => restored!.transport }); + const events: PrpEvent[] = []; + const checkpoints: PersistedNativeSession[] = []; + let recoveryCheckpoint = persisted; + let failCheckpoint = crash !== "none"; + const interruptedController = new Error("controller lost before continuation submission"); + const execute = () => executeNativeSession({ + input: execution, backend: recovering, persistedSession: recoveryCheckpoint, resumeInterruptedTurn: true, + runnerInstanceId: "runner-restart", controlPlaneInstanceId: "controller-restart", timeoutMs: 15_000, + controlPlane: { + async openRun() {}, + async checkpointSession(snapshot) { + if (failCheckpoint && crash === "after-start" && snapshot.dispositionOnlyRecoveryTurnId === "provider-turn-2") { + failCheckpoint = false; + throw interruptedController; + } + checkpoints.push(structuredClone(snapshot)); + recoveryCheckpoint = structuredClone(snapshot); + if (failCheckpoint && crash === "before-start" && snapshot.terminal?.runTerminalState === "failed") { + failCheckpoint = false; + throw interruptedController; + } + }, + async appendEvent(event) { events.push(event); durableEvents.push(event); return { cursor: durableEvents.length, highestContiguousSourceSeq: event.sourceSeq, disposition: "committed" }; }, + async replayEvents(request) { + const source = durableEvents.filter(event => event.sourceInstanceId === request.sourceInstanceId); + return { events: source.filter(event => event.sourceSeq > request.afterSourceSeq), highestContiguousSourceSeq: source.at(-1)?.sourceSeq ?? 0 }; + }, + async completeRun() {}, + }, + }); + if (crash !== "none") { + await expect(execute()).rejects.toBe(interruptedController); + await restored.transport.close().catch(() => undefined); + restored = createRunnerdCodexTransport({ ...options, resumeDynamicTools: [], resumeActiveTurnId: recoveryCheckpoint.activeTurnId }); + recovering = createRunnerdNativeSessionBackend(execution, { runnerInstanceId: "runner-restart", transportFactory: () => restored!.transport }); + } + const completion = await execute(); + expect(completion).toMatchObject({ providerSessionId: persisted.providerSessionId, turnId: "provider-turn-2", terminal: { runTerminalState: "succeeded" } }); + expect(events.filter(event => event.eventType === "turn.failed")).toHaveLength(1); + expect(events.filter(event => event.eventType === "run.terminal")).toHaveLength(1); + if (crash !== "after-start") expect(checkpoints.some(snapshot => snapshot.activeTurnId === "provider-turn-2" && snapshot.terminal === null)).toBe(true); + expect((await readFile(join(root, "calls.log"), "utf8")).match(/^turn\/start$/gm)).toHaveLength(2); + const journal = JSON.parse(await readFile(join(root, "control-plane/control-plane-state.json"), "utf8")); + const submissions = journal.commands.filter((command: { type: string }) => command.type === "turn.start"); + expect(submissions).toHaveLength(2); + expect(submissions[1].payload.text).toContain("Reconcile any unfinished tool or command"); + } finally { + await restored?.transport.close().catch(() => undefined); + for (const bundle of [first, restored]) { + if (!bundle) continue; + const evidence = bundle.evidence(); + kill(evidence.runnerPid); + kill(evidence.providerPid); + } + await rm(root, { recursive: true, force: true }); + } +}, 30_000); diff --git a/packages/paperclip-runner/src/native-session-runtime.test.ts b/packages/paperclip-runner/src/native-session-runtime.test.ts index 5d3418e1cb..db0344becc 100644 --- a/packages/paperclip-runner/src/native-session-runtime.test.ts +++ b/packages/paperclip-runner/src/native-session-runtime.test.ts @@ -225,6 +225,71 @@ function highestContiguous(events: PrpEvent[]): number { } describe("executeNativeSession recovery", () => { + it.each([ + { label: "continues a restart-lost Codex turn", allowed: true, error: { code: "provider_turn_lost_on_restore", recoverable: true } }, + { label: "requires the control plane restart claim", allowed: false, error: { code: "provider_turn_lost_on_restore", recoverable: true } }, + { label: "preserves real provider errors", allowed: true, error: { code: "model_not_found", recoverable: false } }, + { label: "does not reopen historical error-null failures", allowed: true, error: null }, + ])("$label without replaying the task", async ({ label, allowed, error }) => { + const checkpoint: PersistedNativeSession = { + backendKind: "runner", driverKind: "codex_app_server", sessionId: "thread-recovery", identity, + providerSessionId: "provider-recovery", cursor: "0", activeTurnId: "turn-recovery", + terminalTurns: [], semanticResult: null, pendingRuntimeRequests: [], lineage: [], + }; + const failure = error; + const interrupted: PersistedNativeSession = { + ...checkpoint, activeTurnId: null, + terminalTurns: [{ turnId: "turn-recovery", fingerprint: JSON.stringify({ terminalState: "failed", error: failure, result: null }) }], + }; + const startTurn = vi.fn(async () => ({ turnId: "turn-continuation" })); + const session: NativeSession = { + identity: () => identity, + async capabilities() { return { resume: true, typedEvents: true, steering: false, interruption: true, structuredResult: true }; }, + async *events() { + yield runnerEvent(1, "turn.failed", { status: "failed", error: failure }); + if (!startTurn.mock.calls.length) return; + yield { ...runnerEvent(2, "turn.completed", { status: "completed" }), turnId: "turn-continuation" }; + }, + startTurn, + async result() { return startTurn.mock.calls.length ? { result, terminal, turnId: "turn-continuation" } : null; }, + async snapshot() { return structuredClone(interrupted); }, + async close() {}, + }; + const events: PrpEvent[] = []; + const checkpoints: PersistedNativeSession[] = []; + const backend: NativeSessionBackend = { + async descriptor() { return { kind: "runner", name: "restart", version: "1", capabilities: await session.capabilities() }; }, + async openSession() { throw new Error("must preserve the provider session"); }, + async recoverSession() { return { recovered: true, session }; }, + }; + const port: ControlPlanePort = { + async openRun() {}, + async checkpointSession(snapshot) { checkpoints.push(structuredClone(snapshot)); }, + async appendEvent(event) { events.push(event); return { cursor: events.length, highestContiguousSourceSeq: event.sourceSeq, disposition: "committed" }; }, + async replayEvents() { return { events: [], highestContiguousSourceSeq: 0 }; }, + async completeRun() {}, + }; + const execution = executeNativeSession({ + input, backend, controlPlane: port, persistedSession: checkpoint, + runnerInstanceId: "runner-recovery", controlPlaneInstanceId: "control-recovery", + resumeInterruptedTurn: allowed, + }); + if (label !== "continues a restart-lost Codex turn") { + await expect(execution).rejects.toMatchObject({ code: "native_provider_terminal_failed" }); + expect(startTurn).not.toHaveBeenCalled(); + return; + } + await expect(execution).resolves.toMatchObject({ result, turnId: "turn-continuation", terminal: { runTerminalState: "succeeded" } }); + expect(startTurn).toHaveBeenCalledOnce(); + const submitted = startTurn.mock.calls[0]![0] as Parameters[0]; + expect(submitted.continuation).toBe(true); + expect(submitted.message.text).toContain("Reconcile any unfinished tool or command"); + expect(submitted.message.text).not.toContain(input.task.prompt); + expect(events.filter(event => event.eventType === "turn.failed")).toHaveLength(1); + expect(events.filter(event => event.eventType === "run.terminal")).toHaveLength(1); + expect(checkpoints.some(snapshot => snapshot.terminalTurns?.some(turn => turn.turnId === "turn-recovery"))).toBe(true); + }); + it.each([undefined, 0, 7 * 24 * 60 * 60 * 1000, 30 * 24 * 60 * 60 * 1000])( "honors long-lived turn duration independently of operation bounds (%s)", async (turnTimeoutMs) => { @@ -6234,7 +6299,12 @@ describe("executeNativeSession recovery", () => { expect(openRun).not.toHaveBeenCalled(); }); - it.each([false, true])("replaces a provider session that already ended with a failed terminal (prepared: %s)", async (preparedMode) => { + it.each([ + { preparedMode: false, interrupted: false }, + { preparedMode: true, interrupted: false }, + { preparedMode: false, interrupted: true }, + { preparedMode: true, interrupted: true }, + ])("replaces a failed provider with full task context (prepared: $preparedMode, interrupted: $interrupted)", async ({ preparedMode, interrupted }) => { const digest = "0".repeat(64); const skill = { key: "company/recovery-skill", @@ -6286,6 +6356,7 @@ describe("executeNativeSession recovery", () => { } as const; const checkpoint: PersistedNativeSession = { backendKind: "mock", + driverKind: "codex_app_server", sessionId: "driver-failed", identity, providerSessionId: "provider-failed", @@ -6299,7 +6370,10 @@ describe("executeNativeSession recovery", () => { runTerminalState: "failed", reportedWorkDisposition: "yielded", }, - terminalTurns: [{ turnId: "turn-failed", fingerprint: "failed" }], + terminalTurns: [{ turnId: "turn-failed", fingerprint: interrupted ? JSON.stringify({ + terminalState: "failed", result: null, + error: { code: "provider_turn_lost_on_restore", recoverable: true }, + }) : "failed" }], pendingRuntimeRequests: [], lineage: [], }; @@ -6335,8 +6409,8 @@ describe("executeNativeSession recovery", () => { async close() {}, }; const recoverSession = vi.fn(async () => ({ - recovered: true as const, - session: replacementSession, + recovered: false as const, + reason: "provider thread is unavailable", })); const openReplacementSession = vi.fn(async () => replacementSession); const onContinuityBreak = vi.fn(async () => undefined); @@ -6397,7 +6471,7 @@ describe("executeNativeSession recovery", () => { ).resolves.toMatchObject({ providerSessionId: "provider-replacement" }); expect(getFreshSessionHandoff).toHaveBeenCalledOnce(); - expect(recoverSession).not.toHaveBeenCalled(); + expect(recoverSession).toHaveBeenCalledTimes(interrupted ? 1 : 0); expect(openReplacementSession).toHaveBeenCalledOnce(); const replacementEnvelope = JSON.parse( startTurn.mock.calls[0]![0].message.text, @@ -6419,7 +6493,7 @@ describe("executeNativeSession recovery", () => { expect(JSON.stringify(replacementEnvelope)).not.toContain("ONLY_NEW_COMMENT"); expect(startTurn.mock.calls[0]![0]).not.toHaveProperty("continuation"); expect(onContinuityBreak).toHaveBeenCalledWith({ - reason: "provider session ended with a failed terminal", + reason: interrupted ? "provider thread is unavailable" : "provider session ended with a failed terminal", previousDriverSessionId: "driver-failed", previousProviderSessionId: "provider-failed", replacementDriverSessionId: "driver-replacement", diff --git a/packages/paperclip-runner/src/native-session-runtime.ts b/packages/paperclip-runner/src/native-session-runtime.ts index 2ccfd7a4c1..e76409e7ab 100644 --- a/packages/paperclip-runner/src/native-session-runtime.ts +++ b/packages/paperclip-runner/src/native-session-runtime.ts @@ -23,6 +23,8 @@ import { NativeSessionCloseUnrecoverableError, NativeSessionCleanupQuarantinedError, NativeSessionProtocolIntegrityError, + isNativeRestartInterruption, + nativeRestartInterruptedTurnId, } from "./contracts/native-session-backend.js"; import { validatePrpStructuredRunResult, @@ -184,6 +186,8 @@ export interface NativeSessionGoalControl { } export interface ExecuteNativeSessionOptions { + /** The control plane proved the old local runner stopped and charged a recovery attempt. */ + resumeInterruptedTurn?: boolean; /** Read bounded task history only after recovery requires a fresh conversation. */ getFreshSessionHandoff?: () => Promise; /** Durable launch intent, after cleanup admission and before provider calls. */ @@ -865,6 +869,7 @@ async function consumeTurn( requestId: string; }, initialGoal?: HarnessThreadGoal | null, + resumeInterruptedTurn?: (event: PrpEvent, signal: AbortSignal) => Promise, ) { let timer: ReturnType | undefined; const appendAbort = new AbortController(); @@ -1174,6 +1179,8 @@ async function consumeTurn( } if (isTurnTerminal(event)) { if (providerFailure) throw providerFailure; + if (semanticResultProposal === null && governedResult === null && + inputTimers.size === 0 && await resumeInterruptedTurn?.(event, appendAbort.signal)) continue; return { event, eventCount, @@ -1871,6 +1878,12 @@ export async function executeNativeSession( previousProviderSessionId: string | null; } | null = null; let reconciledRecoveryCheckpoint: PersistedNativeSession | null = null; + const checkpointedInterruption = persistedSession && nativeRestartInterruptedTurnId(persistedSession); + if (checkpointedInterruption && persistedSession) { + // The turn failed because its process vanished; the conversation is still + // usable. Keep its terminal fingerprint so replay cannot execute it twice. + persistedSession = { ...persistedSession, terminal: null }; + } await options.onSessionAdmission?.(); if (options.existingSession) { if (options.existingSession.attachRun === undefined) { @@ -2157,6 +2170,42 @@ export async function executeNativeSession( await persistCheckpoint(snapshot, signal); }; const recoveredSnapshot = await session.snapshot(); + const restartContinuation = () => ({ + message: { role: "user" as const, text: JSON.stringify({ + schema: "paperclip.native-continuation.v1", + events: [ + "Paperclip restarted and restored this same conversation. The previous turn was interrupted.", + "Continue the current user request from the recorded progress and preserved workspace.", + "Reconcile any unfinished tool or command before proceeding: inspect its current state and recorded outcomes.", + "Do not repeat completed work or blindly rerun an action whose outcome is unknown. If an external action cannot be reconciled, report the specific blocker.", + ].join("\n"), + completion: { + revision: input.completionContract.contract.revision, + criterionIds: input.completionContract.contract.criteria.map(criterion => criterion.id), + }, + }) }, + continuation: true as const, + requestedCollaborationMode: "executionMode" in input ? input.executionMode : "default" as const, + }); + let restartContinuationStarted = false; + const resumeInterruptedTurn = async (event: PrpEvent, signal: AbortSignal) => { + if (restartContinuationStarted || !recovered || !options.resumeInterruptedTurn || + input.provider.kind !== "codex" || options.sessionGoalControl || options.resumeSessionGoalHeartbeat || + recoveredSnapshot.goal || persistedSession?.semanticResult || persistedSession?.pendingRuntimeRequests?.length || + event.eventType !== "turn.failed" || !isNativeRestartInterruption(event.payload.error) || + event.turnId !== persistedSession?.activeTurnId) return false; + const snapshot = await session.snapshot(); + signal.throwIfAborted(); + if (nativeRestartInterruptedTurnId(snapshot) !== event.turnId || snapshot.pendingRuntimeRequests?.length) return false; + restartContinuationStarted = true; + // Persist the old terminal before launching. Recovery inspects provider + // history to adopt an accepted continuation if its checkpoint was lost. + await persistCheckpoint(snapshot, signal); + signal.throwIfAborted(); + await session.startTurn(restartContinuation()); + await checkpoint(signal); + return true; + }; const recoveredActiveTurnId = recovered ? (recoveredSnapshot.activeTurnId ?? null) : (persistedSession?.activeTurnId ?? null); @@ -2294,6 +2343,7 @@ export async function executeNativeSession( } : undefined, recoveredSnapshot.goal, + resumeInterruptedTurn, ) : Promise.resolve({ event: recoveryTerminal, @@ -2333,6 +2383,10 @@ export async function executeNativeSession( await session.goal({ action: "get", requestId }); } await checkpoint(); + } else if (shouldStartFreshTurn && recovered && checkpointedInterruption) { + restartContinuationStarted = true; + await session.startTurn(restartContinuation()); + await checkpoint(); } else if (shouldStartFreshTurn) { let modelEnvelope = recovered ? buildNativeModelEnvelope(input, { resumedSession: true }) diff --git a/server/src/services/native-runtime/native-restart-recovery.ts b/server/src/services/native-runtime/native-restart-recovery.ts index 2fc54b97a2..96378b4bc2 100644 --- a/server/src/services/native-runtime/native-restart-recovery.ts +++ b/server/src/services/native-runtime/native-restart-recovery.ts @@ -16,6 +16,7 @@ import { redactSensitiveText } from "../../redaction.js"; import { issueRecoveryActionService } from "../issue-recovery-actions.js"; import { reportRunFailure } from "../run-failure-report.js"; import { isNativeRunnerOwnershipHeld } from "./native-runner-ownership.js"; +import { nativeRestartInterruptedTurnId, type PersistedNativeSession } from "../../vendor/paperclip-runner/index.js"; export type NativeControllerIdentity = { bootId: string; @@ -764,7 +765,8 @@ export async function claimNativeRestartRecoveries(input: { hasProviderEvidence, checkpointFailed: (checkpointRecord.terminal as Record | undefined) - ?.runTerminalState === "failed", + ?.runTerminalState === "failed" && + nativeRestartInterruptedTurnId(checkpointRecord as unknown as PersistedNativeSession) === null, providerAttempt: row.coordinator.attempt, }); const ownershipChanged = diff --git a/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts b/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts index 820661f889..c73db0b848 100644 --- a/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts +++ b/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts @@ -1018,6 +1018,30 @@ describeEmbeddedPostgres("native runner restart recovery with real processes", ( ]); }); + it.each([2, 3])("recovers a checkpointed restart interruption within provider budget %s", async (attempt) => { + const fixture = await seedRun(`INTERRUPTED-${attempt}`); + await fixture.db.update(heartbeatRuns).set({ runnerProfileJson: { sessionCheckpoint: { + driverKind: "codex_app_server", + sessionId: fixture.native.normalizedSessionId, + identity: { companyId, issueId: fixture.issueId, runId: fixture.runId, agentId, sessionId: fixture.native.normalizedSessionId }, + providerSessionId: "interrupted-provider", + activeTurnId: null, + semanticResult: null, + terminal: { runTerminalState: "failed", turnTerminalState: "failed" }, + terminalTurns: [{ turnId: "interrupted-turn", fingerprint: JSON.stringify({ + terminalState: "failed", result: null, + error: { code: "provider_turn_lost_on_restore", recoverable: true }, + }) }], + } } }).where(eq(heartbeatRuns.id, fixture.runId)); + await fixture.db.update(nativeRunFinalizations).set({ attempt }).where(eq(nativeRunFinalizations.runId, fixture.runId)); + const dispositions = await claimNativeRestartRecoveries({ + db: fixture.db, controller: successor, restartKind: "hot", runIds: [fixture.runId], + }); + expect(dispositions).toEqual([expect.objectContaining(attempt < 3 + ? { kind: "resume_dead_runner", runId: fixture.runId, providerAttempt: attempt } + : { kind: "blocked", runId: fixture.runId, reason: "execution_recovery_budget_exhausted" })]); + }); + it("terminalizes the recorded failed-checkpoint incident atomically instead of resuming it on upgrade", async () => { const fixture = await seedRun("FAILED-CHECKPOINT"); await fixture.db.update(heartbeatRuns).set({ runnerProfileJson: { sessionCheckpoint: { diff --git a/server/src/services/native-runtime/native-session-executor.ts b/server/src/services/native-runtime/native-session-executor.ts index ec28b68800..981169652e 100644 --- a/server/src/services/native-runtime/native-session-executor.ts +++ b/server/src/services/native-runtime/native-session-executor.ts @@ -8338,6 +8338,7 @@ async function executePaperclipNativeSessionWithinScope( trace.activate(runnerSessionStartupScope); const result = await trace.run(runnerSessionStartupScope, () => executeNativeSession({ + resumeInterruptedTurn: input.restartRecovery?.kind === "resume_dead_runner", getFreshSessionHandoff: input.getFreshSessionHandoff, onSessionAdmission: async () => { // Invalidate prior stop evidence before a backend can spawn. diff --git a/server/src/vendor/paperclip-runner/index.ts b/server/src/vendor/paperclip-runner/index.ts index 80f98ee6f0..e113899dfa 100644 --- a/server/src/vendor/paperclip-runner/index.ts +++ b/server/src/vendor/paperclip-runner/index.ts @@ -128,6 +128,7 @@ export const validatePrpStructuredRunResult = runner.validatePrpStructuredRunResult; export const NativeProviderTerminalFailure = runner.NativeProviderTerminalFailure; +export const nativeRestartInterruptedTurnId = runner.nativeRestartInterruptedTurnId; export const completeTerminatedRemoteNativeSessionCleanup = runner.completeTerminatedRemoteNativeSessionCleanup; export const completeTerminatedLocalNativeSessionCleanup = runner.completeTerminatedLocalNativeSessionCleanup; From 9b3fe260bac576d622ffdfefb923db73cc5273f7 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 6 Oct 2026 08:05:25 -0500 Subject: [PATCH 12/12] fix(tasks): surface Codex ChatGPT model rejection (#15299) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner runs report provider errors and can also save a structured failure result. > - Codex rejects a model that the user's ChatGPT account cannot use. > - Master now diagnoses this rejection, but the task's compact run data omits its message. The thread can still call it a generic run failure. > - Users need the account restriction and a clear step to repair the model selection. > - This pull request shows the existing diagnosis as “Model unavailable” on the task. ## Linked Issues or Issue Description **What happened?** Codex returns HTTP 400 with `invalid_request_error` and the message `The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.` Master now stores an actionable diagnosis for this rejection. The task thread still labels it “Run failed” and does not receive its error text in compact run data. **Expected behavior** Show the account restriction on the task and in the run error. Tell the user to choose a supported model or clear the task's model override before retrying. **Steps to reproduce** 1. Use a Native Runner Codex agent signed in with a ChatGPT account. 2. Select a model that produces the rejection above and start a task. 3. Let the runner save its generic failed result. Inspect the task's failure marker and recovery notice. **Additional context** Refs: #15304. That merged PR diagnoses the provider failure and preserves worker and review recovery rules. This PR adds its task-facing message and guidance without changing that diagnosis or those rules. Refs: #13134. That PR improves model discovery for ChatGPT accounts. This PR exposes the rejection when a configured model still fails at execution time. ## What Changed - Return a bounded model rejection message in compact issue-run data. - Show “Model unavailable” and model-change guidance in the task thread and recovery notice. - Add database and UI regression tests, including both the original rejection text and master's fixed diagnosis. Document the new failure message. ## Verification - Focused merged-branch validation: 279 tests passed across activity service, native provider failure observation and PostgreSQL integration, TaskChatThread, and ExecutionBlockerNotice. - `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates` passed on the merged source tree. - Greptile reviewed conflict-resolution commit `1db39c4e636a92e2e76ba224b71c6f1e2f556c81` at 5/5 with no findings or inline comments. - All 55 check runs completed without failure on that commit. The two optional Storybook jobs were skipped. The legacy Snyk status passed. The branch has no merge conflicts. - UI regression assertion: the task's failure marker says “Model unavailable” and retains the account restriction. It no longer says that this failure happened after a final response. ## Risks - Provider recognition and recovery are owned by the existing master implementation. This PR exposes only bounded error text for failed runs with `native_provider_model_rejected`. - Historical runs with the generic `adapter_failed` code are not reclassified. No stored run is rewritten. - Existing retry and reconciliation gates remain in place. No schema migration or model configuration change is required. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code execution, and browser inspection. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip --- doc/DEVELOPING.md | 5 ++++ server/src/__tests__/activity-service.test.ts | 7 +++-- server/src/services/activity.ts | 3 +++ ui/src/api/activity.ts | 2 ++ .../ExecutionBlockerNotice.test.tsx | 22 ++++++++++++++++ ui/src/components/ExecutionBlockerNotice.tsx | 4 ++- ui/src/components/TaskChatThread.test.tsx | 26 +++++++++++++++++++ ui/src/components/TaskChatThread.tsx | 6 +++-- ui/src/lib/issue-chat-messages.ts | 1 + 9 files changed, 71 insertions(+), 5 deletions(-) diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index 47e399e98c..e1927b4296 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -192,6 +192,11 @@ choices. Codex uses the curated adapter catalog unless the instance declares `PAPERCLIP_ADAPTER_MODELS`; general OpenAI API models are not Codex choices. The Paperclip Runner Codex profile shows the same known model effort levels. Its selected effort is saved with the run and sent to Codex for each turn. +When Codex reports that a selected model is not supported with a ChatGPT account, +the task shows **Model unavailable**, the provider's account restriction, and +guidance to choose a supported model or clear the task's model override before +retrying. The run retains this reason even when the runner saves a generic +failure result. Claude Code uses model-specific effort levels; Haiku has no effort slider. Grok uses its adapter's reasoning levels, including for its default model. Kimi shows effort only when its agent uses the CLI engine, including with its diff --git a/server/src/__tests__/activity-service.test.ts b/server/src/__tests__/activity-service.test.ts index 328b61acff..6f1a960163 100644 --- a/server/src/__tests__/activity-service.test.ts +++ b/server/src/__tests__/activity-service.test.ts @@ -150,7 +150,7 @@ describeEmbeddedPostgres("activity service", () => { expect(result.map((event) => event.action)).toEqual(["test.newest", "test.middle"]); }); - it("returns compact usage and result summaries for issue runs", async () => { + it.each([null, "native_provider_model_rejected", "adapter_failed"])("returns compact issue runs with bounded model rejection details: %s", async (errorCode) => { const companyId = randomUUID(); const agentId = randomUUID(); const issueId = randomUUID(); @@ -180,7 +180,9 @@ describeEmbeddedPostgres("activity service", () => { companyId, agentId, invocationSource: "assignment", - status: "succeeded", + status: errorCode ? "failed" : "succeeded", + errorCode, + error: "provider-error".repeat(200), contextSnapshot: { issueId }, usageJson: { inputTokens: 11, @@ -215,6 +217,7 @@ describeEmbeddedPostgres("activity service", () => { agentId, invocationSource: "assignment", contextIssueId: issueId, + error: errorCode === "native_provider_model_rejected" ? "provider-error".repeat(200).slice(0, 2000) : null, }); expect(runs[0]?.usageJson).toEqual({ inputTokens: 11, diff --git a/server/src/services/activity.ts b/server/src/services/activity.ts index 8a2310cbad..96685ad287 100644 --- a/server/src/services/activity.ts +++ b/server/src/services/activity.ts @@ -403,6 +403,9 @@ export function activityService(db: Db) { invocationSource: heartbeatRuns.invocationSource, responsibleUserId: heartbeatRuns.responsibleUserId, errorCode: heartbeatRuns.errorCode, + error: sql`case when ${heartbeatRuns.status} = 'failed' + and ${heartbeatRuns.errorCode} = 'native_provider_model_rejected' + then left(${heartbeatRuns.error}, 2000) else null end`, usageJson: summarizedUsageJson, resultJson: summarizedResultJson, logBytes: heartbeatRuns.logBytes, diff --git a/ui/src/api/activity.ts b/ui/src/api/activity.ts index 50ecab9a5a..5daa3701cc 100644 --- a/ui/src/api/activity.ts +++ b/ui/src/api/activity.ts @@ -16,6 +16,8 @@ export interface RunForIssue { invocationSource: string; responsibleUserId?: string | null; errorCode?: string | null; + /** Bounded provider message for a failed model rejection. */ + error?: string | null; usageJson: Record | null; resultJson: Record | null; logBytes?: number | null; diff --git a/ui/src/components/ExecutionBlockerNotice.test.tsx b/ui/src/components/ExecutionBlockerNotice.test.tsx index 3683c76883..d5d2b0a95d 100644 --- a/ui/src/components/ExecutionBlockerNotice.test.tsx +++ b/ui/src/components/ExecutionBlockerNotice.test.tsx @@ -107,6 +107,28 @@ describe("stopped task recovery notice", () => { expect(onRetried).toHaveBeenCalledOnce(); }); + it.each([ + "The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.", + "The selected model is not supported by the current ChatGPT connection. Choose a supported model or a compatible AI connection.", + ])("shows the model rejection and repair action without expanding run logs: %s", async (runError) => { + vi.mocked(activityApi.runsForIssue).mockResolvedValue([{ runId: "model-run", agentId: "agent", + status: "failed", errorCode: "native_provider_model_rejected" }] as never); + await act(async () => root.render( + + )); + await act(async () => { await new Promise(resolve => setTimeout(resolve, 10)); }); + expect(container.textContent).toContain("Model unavailable."); + expect(container.textContent).toContain(runError); + expect(container.textContent).toContain("clear the task's model override, then retry"); + expect(container.textContent).toContain("Inspect the original failure before continuing."); + expect(container.querySelector("button")?.textContent).toBe("Retry"); + }); + it.each(["native_continuation_requires_reconciliation", "uncertain_external_action"])("offers Retry for a server-admitted native failure: %s", async cause => { await act(async () => root.render( run.runId === blocker.runId && ["failed", "timed_out"].includes(run.status)); + const modelRejected = failedRun?.errorCode === "native_provider_model_rejected"; const requiresInspection = blocker.cause === "native_continuation_requires_reconciliation" || blocker.cause === "native_session_cleanup_quarantined"; const retry = useMutation({ @@ -34,7 +35,8 @@ export function ExecutionBlockerNotice({ companyId, issueId, blocker, onRetried return (
-

Recovery needed.{blocker.runError ? ` ${blocker.runError}` : ""}

+

{modelRejected ? "Model unavailable." : "Recovery needed."}{blocker.runError ? ` ${blocker.runError}` : ""}

+ {modelRejected &&

Choose a supported model or clear the task's model override, then retry.

}

{blocker.nextAction}

{Boolean(blocker.savedMessageCount) && (

{blocker.savedMessageCount} saved {blocker.savedMessageCount === 1 ? "message is" : "messages are"} waiting for recovery.

diff --git a/ui/src/components/TaskChatThread.test.tsx b/ui/src/components/TaskChatThread.test.tsx index 10854e06cc..1a111c1167 100644 --- a/ui/src/components/TaskChatThread.test.tsx +++ b/ui/src/components/TaskChatThread.test.tsx @@ -1803,6 +1803,32 @@ describe("TaskChatThread runtime transcript selection", () => { expect(container.textContent).not.toContain("The runner stopped"); }); + it.each([ + "The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.", + "The selected model is not supported by the current ChatGPT connection. Choose a supported model or a compatible AI connection.", + ])("promotes a rejected model above the runner's generated failure response: %s", (error) => { + nativeTranscriptState.transcriptByRun.set("model-rejected", [{ + kind: "run_result", ts: "2026-08-25T18:00:01.000Z", + summary: "The Codex run failed before it completed.", + disposition: "needs_review", objectiveSatisfied: false, verification: [], + remainingWork: [], blocker: null, artifacts: [], + }]); + render( {}} linkedRuns={[{ + runId: "model-rejected", runtimeMode: "native", status: "failed", + errorCode: "native_provider_model_rejected", + error, + agentId: "agent-1", agentName: "Runner", adapterType: "paperclip_runner", + createdAt: "2026-08-25T18:00:00.000Z", startedAt: "2026-08-25T18:00:00.000Z", + finishedAt: "2026-08-25T18:00:02.000Z", + }]} />); + const marker = container.querySelector('[data-testid="task-chat-collapsible-marker"]'); + expect(marker?.textContent).toContain("Model unavailable"); + flushSync(() => marker!.querySelector('button[aria-expanded="false"]')!.click()); + expect(container.textContent).toContain(error); + expect(container.textContent).toContain("clear the task's model override, then retry"); + expect(container.textContent).not.toContain("after returning a final response"); + }); + it("keeps workspace contention out of the conversation's cancellation markers", () => { render( {}} linkedRuns={[{ runId: "workspace-wait", runtimeMode: "native", status: "cancelled", errorCode: "workspace_busy", diff --git a/ui/src/components/TaskChatThread.tsx b/ui/src/components/TaskChatThread.tsx index 3c70f2aafc..45690233df 100644 --- a/ui/src/components/TaskChatThread.tsx +++ b/ui/src/components/TaskChatThread.tsx @@ -1637,7 +1637,9 @@ export function TaskChatThread(props: TaskChatThreadProps) { ? "native_runner_timed_out" : "native_runner_process_exited"); const label = - code === "native_provider_approval_required" && source.status === "failed" + code === "native_provider_model_rejected" && source.status === "failed" + ? "Model unavailable" + : code === "native_provider_approval_required" && source.status === "failed" ? "Approval required" : code === "native_provider_usage_limit" && source.status === "failed" ? "Usage limit reached" @@ -1659,7 +1661,7 @@ export function TaskChatThread(props: TaskChatThreadProps) { : code === "native_provider_approval_required" ? "This operation requires approval, but this runner has no interactive approval handler. Review the operation and update the agent's permission setting before retrying." : code === "native_provider_model_rejected" - ? "The provider rejected the selected model. Check the model ID and your account's access, save the agent configuration, then retry. View the run for the provider's full error." + ? `${meta?.error || "The provider rejected the selected model."} Choose a supported model or clear the task's model override, then retry.` : code === "native_provider_usage_limit" && source.status === "failed" ? "The model provider has reached its current usage limit. Try again after the limit resets." diff --git a/ui/src/lib/issue-chat-messages.ts b/ui/src/lib/issue-chat-messages.ts index bc6e7dfef1..cd184c0042 100644 --- a/ui/src/lib/issue-chat-messages.ts +++ b/ui/src/lib/issue-chat-messages.ts @@ -59,6 +59,7 @@ export interface IssueChatLinkedRun { hasStoredOutput?: boolean; logBytes?: number | null; errorCode?: string | null; + error?: string | null; scheduledRetryAt?: string | null; nextAction?: string | null; resultJson?: Record | null;