diff --git a/AGENTS.md b/AGENTS.md index c957ca41ea..53f25d6d51 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -152,6 +152,10 @@ Notes: ## 7. Verification Before Hand-off +[feature-map/README.md](feature-map/README.md) is an optional reference for user +entry points, verification recipes, and coverage gaps. The map records coverage +scope, not proof that a live journey passed. + Default local/agent test path: ```sh diff --git a/doc/DATABASE.md b/doc/DATABASE.md index a0f30babbb..2b4480f9d6 100644 --- a/doc/DATABASE.md +++ b/doc/DATABASE.md @@ -186,6 +186,12 @@ workflow, or issue label and watchdog enrichment. Missing resources, authentication errors, and other database errors retain their usual behavior. This does not enable general SQL replay or retry a full request. +The task run list also opts its execution-status projection into these bounded +retries. Each of its four read-only lookups rebuilds only the failed query; +completed lookups and the separate liveness backfill are not replayed. Projection +callers that use a transaction retain single-attempt reads. The opt-in is only +for a pooled database handle outside a transaction. + ## Execution identity row locks Identity initialization, credential acquisition, and steering reconciliation lock diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index c30e261158..e1927b4296 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -192,6 +192,11 @@ choices. Codex uses the curated adapter catalog unless the instance declares `PAPERCLIP_ADAPTER_MODELS`; general OpenAI API models are not Codex choices. The Paperclip Runner Codex profile shows the same known model effort levels. Its selected effort is saved with the run and sent to Codex for each turn. +When Codex reports that a selected model is not supported with a ChatGPT account, +the task shows **Model unavailable**, the provider's account restriction, and +guidance to choose a supported model or clear the task's model override before +retrying. The run retains this reason even when the runner saves a generic +failure result. Claude Code uses model-specific effort levels; Haiku has no effort slider. Grok uses its adapter's reasoning levels, including for its default model. Kimi shows effort only when its agent uses the CLI engine, including with its @@ -460,6 +465,11 @@ npx paperclipai allowed-hostname dotta-macbook-pro ## Test Commands +The [feature map](../feature-map/README.md) is an optional reference for user +entry points, targeted tests, manual verification recipes, and coverage gaps. +Its page inventory is a source snapshot. The documented journeys have separate +verification steps and do not run automatically from the map. + Use the cheap local default unless you are specifically working on browser flows: ```sh @@ -1203,6 +1213,11 @@ operator-controlled [runner API tools](runner-api-tools.md) rollout; enabling Agent Chat does not enable that API surface. Failed-turn retries restore the selected run's user comments so the agent can answer the original request. +An unsaved Agent Chat uses an ephemeral `chat:` view-model ID. Task +browser polling starts after the first send or upload creates a persisted task +UUID. Saved chats retain the task browser's company and credential access checks. +Task browser routes reject malformed task IDs with `404` before querying PostgreSQL. + A native continuation that requires reconciliation shows **Recovery needed** with **Inspect run**; inspect the original outcome before resolving its recovery hold. A generic retry cannot resolve this incident. The runner's diff --git a/doc/SPEC-implementation.md b/doc/SPEC-implementation.md index 41d7435d3a..7e0a3b8ea2 100644 --- a/doc/SPEC-implementation.md +++ b/doc/SPEC-implementation.md @@ -592,7 +592,7 @@ conversation lifecycles, and protection against replaying superseded requests. | Action | Board | Agent | |---|---|---| | Create company | yes | no | -| Hire/create agent | yes (direct) | request via approval | +| Hire/create agent | yes (direct) | new standard agents: direct via `canCreateAgents`; low-trust policy or approval gates can restrict | | Pause/resume agent | yes | pause: no; resume: direct `agents:configure` grant only | | Create/update task | yes | yes | | Force reassign task | yes | limited | @@ -611,6 +611,14 @@ access does not bypass that decision, and `agents:suggest-changes` alone cannot apply the lifecycle change. Pause, clear-error, terminate, approval, and key-management routes remain board-only. +An ordinary standard agent receives a direct `agents:configure` grant on +creation. Existing agents keep their current permissions; no backfill runs. +Low-trust and managed built-in agents do not receive this default. See +[agent permission defaults](agent-permission-defaults.md) for the full inventory. +Agent-authenticated changes cannot set or restore host-executed process adapter +configuration, including commands and environment values. Agent-authenticated +rollbacks cannot restore host-executed workspace commands either. + ### 9.3.1 Shared default-open issue writes For standard-trust agents, issue comments, issue field/status updates, child diff --git a/doc/agent-permission-defaults.md b/doc/agent-permission-defaults.md new file mode 100644 index 0000000000..49ee70192f --- /dev/null +++ b/doc/agent-permission-defaults.md @@ -0,0 +1,31 @@ +# Agent permission defaults + +This document describes the server defaults for an ordinary, standard trust agent. An agent must belong to the same company and be active where the action requires active membership. A responsible user's authority, project or run trust policy, scope rules, and approval gates can narrow these defaults. + +## Granted by default + +| Capability | Source and limit | +| --- | --- | +| Agent configuration and suggestions | New standard agents receive direct, company-scoped `agents:configure` and `agents:suggest-changes` grants. Protected-change and responsible-user checks still apply. | +| Skill changes | New standard agents receive direct `skills:create` and `skills:suggest-changes` grants. | +| Tool access and management | New standard agents receive `tools:manage_connections`, `tools:manage_profiles`, `tools:view_audit`, `tools:use`, and `tools:manage_runtime` grants. Tool, connection, profile, and runtime policy checks still apply. | +| Audit and inbox | New standard agents receive `audit:view_agent_actions` and `inbox:manage` grants. The default inbox grant is limited to the responsible-user path and still follows that user's inbox policy. Cross-user management requires a separately configured grant or the target user's saved consent policy. | +| Task assignment and checkout control | New standard agents receive `tasks:assign`, `tasks:assign_scope`, and `tasks:manage_active_checkouts` grants, subject to company and route checks. The `tasks:assign_scope` grant covers the new agent's own reporting subtree; an explicit invitation scope is preserved. | +| Agent creation | New standard agents have `canCreateAgents: true`. This is a legacy authorization path for `agents:create`, not an `agents:create` grant row. Stored legacy records without the flag stay closed. | +| Skill creation setting | `canCreateSkills: true` is the normalized setting. Protected skill configuration changes still require a direct `skills:create` grant or a consented `skills:suggest-changes` grant. | +| Same-company visibility and work | Standard agents can read same-company agents, projects, issues, and company scope; read and manage decision queues; read runtime and secret metadata where the route allows it; comment or mutate their own or unassigned issues; and assign tasks under the assignment policy. The route and resource checks still apply. | +| Own configuration and wake | An agent can read its own configuration, update unprotected parts of it, and wake itself. Protected changes still require the relevant grant. | + +## Not granted by default + +These permission keys have no blanket grant for new standard agents: `agents:create` (the flag above is separate), `environments:manage`, `tools:admin`, `users:invite`, `users:manage_permissions`, `pipelines:write`, and `joins:approve`. Existing agents keep their current permissions. The defaults above apply when a new standard agent is created or a pending new hire is activated. Some actions have separate bounded paths; the absence of a grant row does not describe every route decision. + +The new direct grants do not grant company administration, user permission management, blanket tool administration, or cross-company access. The responsible user's permissions, route checks, and the normal approval gates still apply. + +Agent-authenticated callers cannot set process adapter configuration or switch an existing agent onto the process adapter. For local adapters, they cannot change host command, argument, working-directory, arbitrary environment, or filesystem sandbox command settings. Provider credential secret references on known environment keys remain allowed so agents can hire peers with their own credentials. When a caller uses an AI connection pool, a new peer can use a plain provider authentication override on the small allowlist of supported keys. Arbitrary environment keys remain blocked. The same limits apply to creation, hiring, adapter switches, and configuration rollback. They also cannot restore host-executed workspace commands through a configuration revision. This keeps host-executed commands under board control while standard agents configure other supported agent settings. + +## Exceptions and rollout + +The new direct grants are withheld from low trust agents and managed built-in agents. Low trust run or project policies can also deny privileged actions even if an agent has a grant. There is no permission backfill or upgrade migration for existing agents. A standard pending new hire receives the new default set when approved and activated. Invitation approval retains the set when it replaces a new agent's grants and preserves any explicitly scoped default grant. Existing scoped grants stay unchanged. A grant removed by an operator stays removed; ordinary updates and startup do not reapply it. + +The standard agent configuration default supports the agent setup in the linked runner task. Enabling a warm runtime can still depend on the target agent's adapter and runtime configuration, provider availability, and any responsible-user or approval checks. diff --git a/doc/execution-semantics.md b/doc/execution-semantics.md index f94909d6c8..701852da70 100644 --- a/doc/execution-semantics.md +++ b/doc/execution-semantics.md @@ -1000,6 +1000,19 @@ recording the proof and successor lineage. Retained provider files are not edite Bootstrap retries, exact-checkpoint resumes, and fresh replacement sessions share three total provider attempts, including the original attempt. Linked run IDs, controller restarts, and duplicate wakes do not reset this budget. Automatic attempts retain the 30-second delay. Replacement scheduling and predecessor lineage commit together, with one successor per predecessor and admission through the normal task locks, authorization, pause, approval, and budget gates. +An exact local Codex restart can restore the conversation after losing its active +turn. Runnerd records this as `provider_turn_lost_on_restore`, with no invented +task result. The admitted restart attempt may send one continuation in that same +conversation. It preserves the workspace and asks the agent to reconcile unfinished +commands and external actions before proceeding. It does not resend the task or +tool calls. An outcome that cannot be reconciled remains a blocker. The continuation +uses the existing durable one-shot recovery marker; recovery adopts an accepted +turn if the controller dies before checkpointing its ID. The interrupted terminal +is checkpointed before submission. Real provider failures, accepted semantic +results, intentional stops, and historical unmarked failures keep their existing +terminal behavior. This continuation uses the already charged restart attempt and +does not reset the provider-attempt budget. + Provider execution and control-plane finalization have different clocks. A healthy provider can think or execute a long tool without output. Once execution settles, recovery and finalization control steps have a 60-second deadline, checked on startup and every 15 seconds. With a healthy database and scheduler, an abandoned transition must be repaired or surfaced within 90 seconds. Terminal persistence must not wait on provider cleanup or publication; a late finalizer cannot change a reassigned or closed task or release another run's locks. Historical ambiguous runs are never automatically replayed after an upgrade. Every continuation carries the triggering request, ordered user direction, interaction outcomes, completed work, and explicit history coverage. A delivered message remains part of the task's request after its connection or approval resolves. The original title is background; a completed Notion read does not satisfy a later Gmail request. Author and source-trust boundaries survive rendering into both native and legacy prompts. Missing required history must be fetched before dispatch rather than described as complete. diff --git a/doc/observability.md b/doc/observability.md index d8c4b0e4a2..e877610b3d 100644 --- a/doc/observability.md +++ b/doc/observability.md @@ -443,6 +443,30 @@ values become `unknown`. A signal such as `SIGKILL` does not establish who sent it or prove an out-of-memory kill. These fields do not change error grouping or run outcomes, and do not include process output or adapter result payloads. +For a native Codex turn that ends with a structured HTTP 400 model/ChatGPT +compatibility rejection, the run records `native_provider_model_rejected` +and a fixed message that asks the user to select a supported model or connection. +This requires a committed runner terminal for the selected model and the exact +failed turn. Agent text, tool output, unrelated turns, and unknown provider +errors do not create this classification. The provider diagnostic contains only +`provider=codex`, `category=model_auth_incompatible`, `status=400`, and +`authMode=chatgpt`. It excludes the model name and provider response. This does +not change credentials, select a fallback model, replay a turn, or alter the +accepted semantic result or cleanup. The existing configuration-blocker recovery +path requires a configuration change instead of automatically retrying the same +model and connection. Native finalization also derives this evidence from the +committed event and the run's pinned provider configuration before it can queue +a failed-result retry. This survives a controller restart before the adapter +diagnostic is saved. Invalid bindings, missing evidence, or a mismatched event +hash do not grant this classification. A current worker's task is blocked for a +configuration change. A pending review instead keeps its original decision and +status version, releases execution, and creates no automatic recovery action. +After repairing the configuration, explicitly retry the reviewer to resolve the +same pending review. The Board's exact failed-run Retry accepts this classified +native failure only while its saved review is still current and pending; caller +payloads cannot select a different review. Dispatch checks that assignment again. +Superseded or completed reviews retain their authority. + An unconfirmed adapter Stop timeout has an event-local `adapter_stop` context: the run UUID, built-in adapter type, native/legacy runtime mode, configured wait duration, and whether abort was requested. Invalid identities become diff --git a/doc/run-log-events.md b/doc/run-log-events.md index 1757c04fe1..ed8f723652 100644 --- a/doc/run-log-events.md +++ b/doc/run-log-events.md @@ -110,6 +110,14 @@ proofs, encryption keys, environment variables, provider credentials, command arguments, or an unsanitized stderr stream. Detailed failed-attempt diagnostics remain in the bounded `native_run_finalizations.recovery_history` ledger. +For a local Codex turn lost while runnerd was stopped, the canonical `turn.failed` +event retains `error.code: "provider_turn_lost_on_restore"` and +`error.recoverable: true`. It records the interrupted turn, not a final task +outcome. There is no synthesized result for that interruption. An admitted +same-session continuation emits its own `turn.submitted`, accepted turn ID, and +terminal events; only the final outcome closes the run. Reconstructed thread +history retains the original error so reconciliation and live delivery agree. + ## Native Local Process Stop Evidence The server writes `native.local_process_stopped` in the same transaction that diff --git a/docs/api/routines.md b/docs/api/routines.md index 6ac4a2d5b4..729efcdf49 100644 --- a/docs/api/routines.md +++ b/docs/api/routines.md @@ -21,6 +21,13 @@ GET /api/routines/{routineId} Returns routine details including triggers. +Routine and trigger resource IDs must be complete UUIDs from the API response. +Short prefixes and malformed IDs return the same `404` as a missing resource. +UUID lookup accepts PostgreSQL's existing input forms, including uppercase, +compact and braced UUIDs; it does not trim whitespace or resolve prefixes. +Company access and routine-assignee permissions still apply. Public trigger +IDs in webhook URLs are separate opaque identifiers, not resource UUIDs. + ## Create Routine ``` diff --git a/docs/guides/board-operator/execution-workspaces-and-runtime-services.md b/docs/guides/board-operator/execution-workspaces-and-runtime-services.md index e38b1b2692..16eeefe881 100644 --- a/docs/guides/board-operator/execution-workspaces-and-runtime-services.md +++ b/docs/guides/board-operator/execution-workspaces-and-runtime-services.md @@ -138,6 +138,7 @@ fetches with it. Details and caveats: Code state moves between runs through the local execution-workspace cwd alone — not through a git remote. - Each run's prepare step bundles the local worktree to the run's remote dir over ssh, with no `git remote` configured. +- Other project repositories under `.paperclip-repositories/` make the same round trip. Each one is bundled and restored as its own Git checkout. - The adapter's restore step at the end of the run writes any new remote commits back into the local worktree directly. - Adapters must never `git push` from runtime code, and must never assume a remote exists. - A failed restore is a run-level error and records `workspace_finalize=failed` on the execution workspace, which gates dependent issue wakes until the next successful finalize. diff --git a/feature-map/README.md b/feature-map/README.md new file mode 100644 index 0000000000..b38d7fb650 --- /dev/null +++ b/feature-map/README.md @@ -0,0 +1,177 @@ +# Paperclip feature map + +Start here when reproducing a user-facing bug, verifying a change, or deciding +which surfaces a fix must cover. Each recipe describes what a person can do, +where they can do it, the expected result, and the evidence needed to verify it. +Product behavior is still governed by [the implementation spec](../doc/SPEC-implementation.md). + +## Features + +The map is organized by user capability, not by source file. It covers **35 feature +families** across the product, CLI, and operator workflows, checked against source +on 2026-10-05. Experimental and developer-only surfaces are labeled explicitly. +Each recipe includes sub-features, current entry points, automated evidence, +manual verification steps, and gotchas. + +### Getting started and access + +| Feature | What it covers | +| --- | --- | +| [Onboarding and first work](./onboarding.md) | Instance setup, company wizard, first agent and task. | +| [Login, invitations, and access](./access.md) | Sessions, bootstrap, membership, roles, and CLI authorization. | +| [Companies and portability](./companies.md) | Company switching/settings, archival, package import/export. | + +### Tasks and conversations + +| Feature | What it covers | +| --- | --- | +| [Task creation and lifecycle](./tasks.md) | Creation, assignment, lists, properties, comments, and completion. | +| [Delegation, dependencies, and signoff](./task-coordination.md) | Child work, prerequisites, reviewers, and task-tree controls. | +| [Inbox, decisions, and search](./inbox-search.md) | Personal triage, decision queues, unread/blocked views, and search. | +| [Agent conversations and project handoff](./agent-chat.md) | Persistent conversations, discovery, handoff, and gated board chat. | +| [Questions and approvals](./questions-and-approvals.md) | Question/plan responses, formal approvals, and app-tool review. | +| [Steering and queued messages](./steering.md) | Follow-ups, queue edits, interruption, and pause/resume. | +| [Documents, attachments, and work products](./documents-artifacts.md) | Versioned documents, annotations, files, outputs, and artifact library. | + +### Agents and reusable capabilities + +| Feature | What it covers | +| --- | --- | +| [Hiring, configuration, and organization](./agents.md) | Agent identity, instructions, reporting lines, lifecycle, and built-ins. | +| [Runs, harnesses, and model accounts](./runs-adapters.md) | Adapter/model setup, account validation, transcripts, and run history. | +| [Skills and Skill Studio](./skills.md) | Discovery, sources, authoring, revisions, tests, and agent policies. | +| [Team packages and installation](./teams.md) | Catalog preview/install via CLI/API; catalog UI availability called out. | + +### Projects and execution + +| Feature | What it covers | +| --- | --- | +| [Projects and repositories](./projects.md) | Project lifecycle, task context, repository configuration, and defaults. | +| [Goals and work alignment](./goals.md) | Goal hierarchy, ownership, status, and project/task context. | +| [Workspaces, services, and files](./workspaces.md) | Provisioning, task bindings, services/logs, Git/files, and closure. | +| [Execution environments](./execution-environments.md) | Local, SSH, sandbox providers, target probes, and custom images. | +| [Recovery](./recovery.md) | Stopped work, workspace repair, bounded continuation, and reconnect. | + +### Apps, channels, and extensions + +| Feature | What it covers | +| --- | --- | +| [Connection setup](./connection-setup.md) | Catalog, MCP links, task requests, authentication, and setup resumption. | +| [App access and action permissions](./app-permissions.md) | Agent grants, off/ask/allowed actions, discovery, tests, and revocation. | +| [External chat and email](./chat-channels.md) | Provider-specific identity, threads, files, delivery, and endpoint upkeep. | +| [Tool gateways and access profiles](./gateways-profiles.md) | Tool exposure, client configuration, tokens, profiles, and activity. | +| [Secrets and proposals](./secrets.md) | Company/user credentials, grants, proposals, and vault import. | +| [Plugins](./plugins.md) | Installation/configuration, contributed pages/tools, and lifecycle. | + +### Automation and structured work + +| Feature | What it covers | +| --- | --- | +| [Routines, schedules, and triggers](./routines.md) | Definitions, variables, scheduled/webhook/manual runs, and history. | +| [Pipelines, review queues, and learnings](./pipelines.md) | Experimental stages, automation, items, review, and learning records. | +| [Cases](./cases.md) | Experimental structured fields, relationships, revisions, and task links. | +| [Status cards](./status-cards.md) | Experimental summaries, watched work, refresh history, and settings. | + +### Oversight and operation + +| Feature | What it covers | +| --- | --- | +| [Costs and budgets](./budgets-costs.md) | Spend reports, scoped limits, incidents, hard stops, and resumption. | +| [Dashboards and audit trails](./activity.md) | Company health, live work, activity, runs, routines, and timeline. | +| [Navigation, profile, and announcements](./navigation-preferences.md) | Sidebar state, favorites/recents, personal identity, and dismissals. | +| [Instance operations](./instance-operations.md) | Installation, updates, service health, configuration, and backups. | +| [CLI/API and local worktrees](./cli-operations.md) | Explicit context, resource commands, outputs, runs, and isolated instances. | + +### Contributor surfaces + +| Feature | What it covers | +| --- | --- | +| [Developer previews and diagnostic labs](./developer-labs.md) | Design examples, interaction fixtures, performance checks; not production acceptance. | + +These are verification instructions, not a claim that every journey passed a live +test. A component test proves its component; a scripted provider proves that +fixture integration. Record executed results separately from the map. + +## Before driving a journey + +1. Use this checkout's [isolated test drive](../doc/DEVELOPING.md#one-command-isolated-manual-test-drive) + for manual product checks. It creates a temporary instance and prints its URL + and data directory. Do not guess that a server on port 3100 belongs to you. +2. Record the commit, URL, company, login role, relevant experimental settings, + adapter, runtime mode, and live versus simulated dependencies. Use disposable + tasks and provider resources. A fresh test drive creates a company and CEO, + but no task or first run. +3. Use the current navigation and company prefix. Paths in recipes omit that + prefix. Record whether the streamlined or production shell is selected; + Agent Chat, chat connectors, and the combined Inbox/Tasks view have separate + gates. A hidden surface is an unmet prerequisite, not a successful test. +4. Run the smallest relevant test first. Vitest commands below run from the + repository root after installing dependencies. Playwright recipes use their + named configuration and its isolated environment, not an unrelated running + instance. Reserve expensive runner/provider tests for the behavior at issue. +5. Exercise the actual user action, inspect its result, reload, and verify the + persisted state or continuation. Use read-only API checks to corroborate UI + evidence; creating state through the API does not prove the creation UI. + +For agent-driven acceptance work, the existing +[dev-workspace run/verify skill](../.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md) +and [evaluation skill](../.agents/skills/paperclip-evals/SKILL.md) describe runtime +ownership and evidence handling. Reuse them; this map introduces no environment +launcher, credentials, scheduled job, or second test framework. + +## Evidence contract + +Report each entry point as **passed**, **failed**, **blocked** (with its missing +prerequisite), or **not run**. Include the feature filename and entry-point ID, +commit/environment, user action, expected and observed result, command/exit code, +and evidence links. Screenshots or traces should show both the action and the +discriminating result. Include task/run/request IDs when relevant, without secrets. + +A fix is verified across its affected surfaces only when each has evidence or +an explicit reason it does not apply. Shared code alone does not prove parity. +Keep product regressions visible; do not change the recipe to bless a failure. +For Paperclip-assigned work, attach evidence through the +[artifact workflow](../doc/AGENT-ARTIFACTS.md). + +## Inventory and remaining depth + +[The UI coverage inventory](./coverage.json) accounts for every non-test TSX +module under `ui/src/pages`, including supporting panels, legacy variants, and +labs. Every product area now links to concrete recipes. **Partial** means the +area still has the named variant or workflow gaps; **unmapped** is reserved for +an area with no recipe. These statuses describe documentation, not runtime health. + +The page inventory is a source snapshot dated 2026-10-05. The map also +includes entry points hosted inside other modules (pipeline Review Queue and +Learnings, task documents, onboarding), CLI-only operations, and operator work. +The team recipe explicitly distinguishes the current CLI/API path from catalog +UI components without a current top-level route. Refer to [route registration](../ui/src/App.tsx) +and the [CLI registry](../cli/src/index.ts) when changing reachability. + +Remaining depth includes complete provider/auth/attachment matrices, every +harness/model/environment capability combination, third-party plugin features, +and every role/error/mobile/legacy-shell permutation. The recipes name relevant +gaps instead of equating source presence with a working user journey. The index +can include capabilities that do not add a page file. + +## Maintaining the reference + +The map is documentation only. It adds no CI checks, automatic journey execution, +or required inventory updates for future pull requests. + +When maintaining a recipe, compare its entry points and expected results with +the current source. Check that linked tests still exercise the stated behavior +and that local references still exist. Refresh the inventory snapshot when it +helps explain the current product. Maintenance is optional and review-based. + +Each recipe starts with an H1 and a user-visible description, then exactly: + +1. **Sub-features** — stable backticked IDs and observable behavior/states. +2. **How to get to it (user POV)** — one H3 backticked entry-point ID per surface. +3. **Driving it** — starts with `Preconditions:`; repeat each entry-point H3, + with `Automated:` and `Manual:` paragraphs. Name test scope and gaps honestly. +4. **Gotchas** — misleading look-alikes, feature gates, and invalid evidence. + +The format is inspired by +[Omnigent's feature map](https://github.com/omnigent-ai/omnigent/tree/91acfbbb59f6fc210ff95a9e9428aadd62e06582/feature-map). +Paperclip's recipes and checks follow its own task model and test infrastructure. diff --git a/feature-map/access.md b/feature-map/access.md new file mode 100644 index 0000000000..d7248c8b63 --- /dev/null +++ b/feature-map/access.md @@ -0,0 +1,55 @@ +# Login, invitations, and access + +People and agents enter the correct instance and company with the permissions granted to their identity. Login, membership, resource access, and CLI authorization have separate controls. + +Implementation: [auth page](../ui/src/pages/Auth.tsx), [company members](../ui/src/pages/CompanyAccess.tsx), [invitation](../ui/src/pages/InviteLanding.tsx). + +## Sub-features + +- `login`: sign in or claim the board in the configured deployment mode. +- `invitations`: join through a valid invite and handle expired or already-used links. +- `membership`: review join requests and manage company/instance membership and permissions. +- `resources`: respect company and resource boundaries on deep links and mutations. +- `cli-auth`: authorize a CLI session and revoke or replace credentials. + +## How to get to it (user POV) + +### `login` + +Open `/auth`; first-owner bootstrap uses `/board-claim/:token` when configured. + +### `invite-members` + +Open `/invite/:token`; operators manage members in company settings and join requests at `/inbox/requests`. + +### `cli-authorization` + +Start the CLI auth/connect flow and approve its `/cli-auth/:id` browser page; instance access is under Settings when visible. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). For login tests use authenticated mode and two test users; local-trusted mode does not prove session enforcement. Use two companies for isolation checks. + +### `login` + +Automated: [auth UI](../ui/src/pages/Auth.test.tsx) and [session routes](../server/src/__tests__/auth-session-route.test.ts) cover their respective layers. + +Manual: Sign in, reload a company deep link, sign out, and revisit it. Confirm the browser requests authentication rather than showing cached company data. Test an invalid claim separately on disposable state. + +### `invite-members` + +Automated: [member UI](../ui/src/pages/CompanyAccess.test.tsx) and [invite replay](../server/src/__tests__/invite-accept-replay.test.ts) provide component and route evidence. + +Manual: Accept an invite as the intended user, inspect the resulting company and permissions, then reuse and expire disposable invites. Change a member permission, reload as that member, and verify a denied operation on another company remains denied. + +### `cli-authorization` + +Automated: [board auth client](../cli/src/__tests__/board-auth.test.ts) and [token commands](../cli/src/__tests__/token.test.ts) cover the client contracts. + +Manual: Authorize the intended CLI context, make a read-only company request, then revoke its token and retry. Verify failure is explicit and does not silently fall back to another identity. + +## Gotchas + +- Settings visibility is not authorization; direct requests still need permission checks. +- An agent bearer key must not gain another company’s access. +- Never put invite tokens, session cookies, or API keys in verification evidence. diff --git a/feature-map/activity.md b/feature-map/activity.md new file mode 100644 index 0000000000..a50206a9fc --- /dev/null +++ b/feature-map/activity.md @@ -0,0 +1,45 @@ +# Dashboards, activity, and audit trails + +People inspect company health, recent activity, run history, and routine activity, then follow evidence back to the task or agent responsible. + +Implementation: [dashboard](../ui/src/pages/Dashboard.tsx), [live dashboard](../ui/src/pages/DashboardLive.tsx), [audit hub](../ui/src/pages/audit/AuditHub.tsx). + +## Sub-features + +- `overview`: inspect the company dashboard and live-work summary. +- `activity`: filter and page through recorded actor/resource events. +- `runs`: find executions and follow their task/agent context. +- `routine-audit`: inspect routine execution activity separately from its definition. +- `timeline`: use the available timeline view to understand work over time. + +## How to get to it (user POV) + +### `dashboard` + +Open `/dashboard` or `/dashboard/live`. + +### `activity-audit` + +Open `/activity`, `/activity/runs`, or `/activity/timeline` in the streamlined shell; legacy routes may redirect. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare known task mutations and completed/failed runs in a disposable company, recording their IDs and actors. + +### `dashboard` + +Automated: [dashboard service](../server/src/__tests__/dashboard-service.test.ts) covers server aggregation; [dashboard helpers](../ui/src/pages/Dashboard.test.ts) is narrower UI helper coverage. + +Manual: Compare the dashboard with the known fixture tasks and runs. Start and finish a small task, refresh/reload, and verify the summary changes and links reach the right records. Exercise empty data and a failed fetch. + +### `activity-audit` + +Automated: [activity service](../server/src/__tests__/activity-service.test.ts), [audit runs](../ui/src/pages/audit/AuditRuns.test.tsx), and [routine audit](../ui/src/pages/audit/RoutineAuditActivity.test.tsx) cover these separate data/host layers. + +Manual: Find a known mutation by actor/resource and a known run by status. Change filters, load more, and follow the source links. Reload the chosen view and verify no company leakage or duplicate events. Inspect a routine run from its audit entry. + +## Gotchas + +- Audit/activity events, run-log rows, first-party Telemetry, and operator OpenTelemetry traces are separate data paths. +- A healthy stream or dashboard counter alone does not prove task success. +- Cost and budget reports have their own [recipe](./budgets-costs.md). diff --git a/feature-map/agent-chat.md b/feature-map/agent-chat.md new file mode 100644 index 0000000000..e47de311b5 --- /dev/null +++ b/feature-map/agent-chat.md @@ -0,0 +1,54 @@ +# Agent conversations and project handoff + +A person can hold a persistent conversation with an agent, return to its history, and hand useful work into a project or task. The gated conference-room chat is a separate conversation surface. + +Implementation: [agent chat](../ui/src/pages/AgentChat.tsx), [chat landing](../ui/src/pages/AgentChats.tsx), [conference room](../ui/src/pages/BoardChat.tsx). + +## Sub-features + +- `discovery`: find an agent and reopen an accessible recent conversation. +- `history`: send messages, inspect replies/files, and return to the same conversation. +- `handoff`: create or identify a project/task from chat and follow the resulting work. +- `conference-room`: use the separately gated board chat without assuming parity with agent chat. + +## How to get to it (user POV) + +### `agent-conversation` + +Open `/chats`, choose an agent, or use `/chats/:agentRef` and its conversation sidebar. + +### `project-handoff` + +Ask for project work in Agent Chat and follow its created project/task links. + +### `conference-room` + +Open `/board-chat` only when its conference-room gate is enabled. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable the applicable chat feature, use a configured test agent, and distinguish model replies from a mocked conversation fixture. + +### `agent-conversation` + +Automated: [chat landing](../ui/src/pages/AgentChats.test.tsx) checks recent-chat routing; [conversation service](../server/src/__tests__/agent-conversations.test.ts) covers persisted conversation behavior. + +Manual: Start a conversation, send a recognizable request, and inspect its reply. Navigate away and reopen the same history. Test a new conversation, unavailable historical agent, and an inaccessible saved conversation; verify no cross-user or cross-company history appears. + +### `project-handoff` + +Automated: [chat project tools](../server/src/__tests__/chat-project-tools.test.ts) checks tool contracts; live model choice and UI handoff require a manual run. + +Manual: Ask the configured agent to create bounded project work, inspect the proposed/created target, and open the resulting task. Verify the request context and intended project survived and the task produces a useful output. + +### `conference-room` + +Automated: [board chat gate](../server/src/__tests__/board-chat-route-feature-flag.test.ts) proves route gating, not a live conversation. + +Manual: Check disabled access first, enable in the disposable instance, then send and reload a conversation. Inspect the actual responding agent and persisted history; report this result separately from Agent Chat. + +## Gotchas + +- Idle chat is not failed execution. +- See [steering](./steering.md) for in-flight messages and [questions](./questions-and-approvals.md) for interactions. +- An API tool test does not prove that every harness will choose the same handoff tool. diff --git a/feature-map/agents.md b/feature-map/agents.md new file mode 100644 index 0000000000..a1633a52da --- /dev/null +++ b/feature-map/agents.md @@ -0,0 +1,55 @@ +# Hiring, agent configuration, and organization + +Operators hire agents, configure their identity and instructions, assign reporting relationships, and manage their lifecycle. Built-in agents have their own setup and lifecycle constraints. + +Implementation: [hiring](../ui/src/pages/NewAgent.tsx), [configuration](../ui/src/components/AgentConfigForm.tsx), [organization](../ui/src/pages/OrgChart.tsx). + +## Sub-features + +- `hire`: choose role, manager, adapter, model, and required account settings. +- `instructions`: edit agent configuration and instruction files with durable revisions. +- `organization`: inspect and change reporting relationships through supported controls. +- `lifecycle`: pause/resume or retire an agent while preserving history. +- `built-in`: configure required agents without treating them as ordinary removable hires. + +## How to get to it (user POV) + +### `hire-agent` + +Use New agent from the roster or `/agents/new`. + +### `agent-configuration` + +Open `/agents/:agentId` and the configuration/instruction surfaces it exposes. + +### `organization` + +Use the roster’s organization view; `/org` redirects to the roster in the streamlined shell. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable company and supported adapter; hiring may require a formal approval. + +### `hire-agent` + +Automated: [new agent page](../ui/src/pages/NewAgent.test.tsx) and [configuration form](../ui/src/components/AgentConfigForm.render.test.tsx) cover mocked UI behavior. + +Manual: Hire an agent with a distinct name and role. Complete any required approval, reload the roster, and open its configuration. Confirm manager, adapter, and model persisted; run a small task to prove execution readiness. + +### `agent-configuration` + +Automated: [instruction service](../server/src/__tests__/agent-instructions-service.test.ts) and [instruction revisions](../server/src/__tests__/agent-instruction-revisions.test.ts) cover file/revision contracts. + +Manual: Save a harmless instruction edit, reload, and verify its revision and effect in the next eligible run. Pause and resume the disposable agent; confirm pause prevents new work and does not erase previous runs. + +### `organization` + +Automated: [org chart](../ui/src/pages/OrgChart.test.tsx) is component evidence. Built-in setup also has [configuration modal tests](../ui/src/components/ConfigureBuiltInAgentModal.test.tsx). + +Manual: Inspect a manager and report, change the relationship through the offered controls, and reload both views. Exercise a rejected invalid relationship. Configure a built-in agent and verify it retains its required identity and restrictions. + +## Gotchas + +- A roster entry is not proof of an authenticated or invokable adapter. +- Pending hires may live in approval/task surfaces rather than the active roster. +- Skill and tool grants are separate: see [skills](./skills.md) and [app permissions](./app-permissions.md). diff --git a/feature-map/app-permissions.md b/feature-map/app-permissions.md new file mode 100644 index 0000000000..fd8a825c51 --- /dev/null +++ b/feature-map/app-permissions.md @@ -0,0 +1,45 @@ +# App access, action permissions, and testing + +After connecting an app, an operator decides which agents may use it and whether each action is off, requires approval, or is allowed. Activity and action tests help verify the resulting access. + +Implementation: [app detail](../ui/src/pages/apps/AppDetail.tsx), [permissions](../ui/src/pages/apps/app-detail/PermissionsPanel.tsx), [activity](../ui/src/pages/apps/app-detail/ActivityPanel.tsx). + +## Sub-features + +- `agent-access`: grant the connection to all or selected eligible agents. +- `action-policy`: set off/ask/allowed independently for available actions. +- `catalog-change`: review newly discovered/quarantined actions before enabling them. +- `test-action`: choose an actor and inputs and inspect a real action result. +- `account-maintenance`: inspect accounts, reconnect/revoke, and review connection activity. + +## How to get to it (user POV) + +### `permissions` + +Open an app at `/apps/:connectionId/permissions`; this is the default connected-app view. + +### `test-and-maintain` + +Use the action test control in Permissions, then the connection’s Activity and available account/advanced controls. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare a connection to a fixture or test provider, two agents, and a harmless action. Record provider and credential ownership. + +### `permissions` + +Automated: [app detail](../ui/src/pages/apps/AppDetail.test.tsx) covers agent/action controls, quarantined actions, and navigation using mocked requests. + +Manual: Grant one agent access, leave another excluded, and exercise off, ask, and allowed states for a harmless action. Reload between changes. Verify actual denied calls stay denied and ask-first creates a decision rather than executing early. + +### `test-and-maintain` + +Automated: [app detail tests](../ui/src/pages/apps/AppDetail.test.tsx) cover test-dialog/UI contracts. Real provider execution and revocation require the selected provider’s live check. + +Manual: Run a read-only action with explicit inputs and inspect returned data and activity. Refresh discovery and review new actions. Revoke or disconnect only the disposable credential, then verify subsequent use fails with an actionable reconnect state. + +## Gotchas + +- Connected does not imply an agent has access or an action is allowed. +- Credential-only connections may replace tool-action controls with account-specific controls. +- See [connection setup](./connection-setup.md) for auth and [questions and approvals](./questions-and-approvals.md) for pending action review. diff --git a/feature-map/budgets-costs.md b/feature-map/budgets-costs.md new file mode 100644 index 0000000000..e1a1efc573 --- /dev/null +++ b/feature-map/budgets-costs.md @@ -0,0 +1,44 @@ +# Costs, budgets, and spending controls + +Operators inspect recorded spend and configure budget controls for the company, projects, and agents. Hard stops pause or prevent eligible work until the governing incident is resolved. + +Implementation: [costs and budget host](../ui/src/pages/Costs.tsx), [audit hub](../ui/src/pages/audit/AuditHub.tsx). + +## Sub-features + +- `cost-reporting`: inspect spend by the offered time range and resource dimensions. +- `budgets`: configure supported company/project/agent limits and inspect usage. +- `incidents`: distinguish warnings, hard stops, and manual pauses. +- `resolution`: raise or resolve a budget incident through authorized controls and verify resumption. + +## How to get to it (user POV) + +### `cost-overview` + +Use Activity → Costs/Budgets in the streamlined shell, `/costs` in the production shell, or scoped project/agent budget panels. + +### `budget-enforcement` + +Open the governing budget configuration and any resulting budget incident card. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use synthetic low-cost events and small disposable budgets. Do not buy model usage merely to hit a threshold. + +### `cost-overview` + +Automated: [costs service](../server/src/__tests__/costs-service.test.ts) covers cost contracts; [embedded costs UI](../ui/src/pages/Costs.test.tsx) has narrow Audit-host coverage. + +Manual: Inspect a known fixture run and reconcile its recorded spend with the selected report period and resource. Switch company/project filters and reload. Check empty and unavailable cost data without treating missing data as zero spend. + +### `budget-enforcement` + +Automated: [budget service](../server/src/__tests__/budgets-service.test.ts) exercises soft incidents, hard stops, scope, and valid budget raises. + +Manual: Create a small fixture budget, cross its threshold with synthetic recorded spend, and verify the warning or hard-stop state. Attempt new work and confirm it is gated. Raise the correct budget through an authorized action and verify eligible continuation without clearing unrelated manual pauses. + +## Gotchas + +- Paperclip cost accounting is not a provider invoice or a promise about externally billed spend. +- Company, project, and agent gates can overlap; fixing one may leave another active. +- A successful retry must not bypass an unresolved budget hold. diff --git a/feature-map/cases.md b/feature-map/cases.md new file mode 100644 index 0000000000..10cde78e5e --- /dev/null +++ b/feature-map/cases.md @@ -0,0 +1,43 @@ +# Cases and structured work records + +The cases experiment exposes structured records with types, fields, parent/child relationships, activity, revisions, and related task work. + +Implementation: [case list](../ui/src/pages/Cases.tsx), [case detail](../ui/src/pages/CaseDetail.tsx). + +## Sub-features + +- `browse`: search, filter, sort, group, and navigate case records. +- `structure`: inspect fields and parent/child relationships without losing matching descendants. +- `history`: inspect revisions and activity for a record. +- `task-links`: follow work associated with a case in both directions. + +## How to get to it (user POV) + +### `case-list` + +Open `/cases` and switch between available grouped/tree views. + +### `case-record` + +Open `/cases/:caseIdentifier` or a task’s linked case. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable cases and prepare related records with different types/statuses plus an associated task. + +### `case-list` + +Automated: [case list](../ui/src/pages/Cases.test.tsx) covers filters, persisted view state, and keyboard/tree behavior with mocked data. + +Manual: Search for a child, verify its parent context remains understandable, change filters/grouping/columns, and reload. Open the record by keyboard and direct link. Include empty results and terminal records. + +### `case-record` + +Automated: [case routes](../server/src/__tests__/cases-routes.test.ts) covers API behavior; [case fields](../ui/src/components/CaseFieldsPanel.test.tsx) covers field UI separately. + +Manual: Edit an offered field, reload, inspect its revision/activity, and follow child and task links. Verify cross-company and invalid field changes fail clearly. Distinguish a pipeline-owned item from a stand-alone case. + +## Gotchas + +- A case is not a task execution; inspect linked tasks to prove work occurred. +- Feature gating and case-type schema determine which actions are available. diff --git a/feature-map/chat-channels.md b/feature-map/chat-channels.md new file mode 100644 index 0000000000..3399965dca --- /dev/null +++ b/feature-map/chat-channels.md @@ -0,0 +1,45 @@ +# External chat and email conversations + +People can talk to an assigned agent through configured external channels, bind their identity, exchange files, and inspect delivery state. Each provider has a different identity, threading, and attachment contract. + +Implementation: [endpoint setup](../ui/src/pages/apps/chat/ChatEndpointSetup.tsx), [endpoint detail](../ui/src/pages/apps/chat/ChatEndpointDetail.tsx), [email setup](../ui/src/pages/apps/chat/EmailEndpointSetup.tsx). + +## Sub-features + +- `providers`: configure the available Slack, Discord, Microsoft Teams, Telegram, GitHub, AgentMail, and experimental iMessage Photon paths; availability still depends on gates and provider support. +- `identity`: link the external sender to the intended authorized Paperclip identity. +- `conversation`: route inbound messages to the selected agent and send replies to the correct thread. +- `attachments`: transfer supported files with inspectable delivery and failure states. +- `lifecycle`: inspect endpoint activity, repair auth, and disable delivery when required. + +## How to get to it (user POV) + +### `external-thread` + +Message the configured bot/address from its external provider and follow the corresponding Paperclip conversation/task. + +### `endpoint-management` + +Open `/apps/chat/:endpointId/settings` and the endpoint’s other available tabs; identity confirmation uses `/chat-identity/confirm`. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable chat connectors and use a test provider account/channel with permission to send messages. Setup is covered in [connection setup](./connection-setup.md). + +### `external-thread` + +Automated: [channel integration](../server/src/__tests__/chat-channels.integration.test.ts) uses provider fixtures; it does not prove a live Slack, Discord, Teams, Telegram, GitHub, email, or Photon delivery. + +Manual: For each supported provider being changed, send a unique message, verify the intended agent/company receives it, and inspect the actual external reply in the original thread. Repeat with an unlinked/unauthorized sender, duplicate inbound delivery, and one supported attachment. Record provider-specific results separately. + +### `endpoint-management` + +Automated: [channel integration](../server/src/__tests__/chat-channels.integration.test.ts) covers shared endpoint/delivery behavior; live credential repair and provider ownership remain manual checks. + +Manual: Inspect the bound agent and identity, recent activity, and delivery failure details. Repair a disposable credential, retry a safe message when offered, and verify exactly one external reply. Disable the endpoint and confirm new traffic is not silently processed. + +## Gotchas + +- One provider’s passing fixture does not qualify another provider or live delivery. +- An accepted send request does not prove the recipient received the message or file. +- Use [steering](./steering.md) and [questions](./questions-and-approvals.md) for their channel-specific interaction checks. diff --git a/feature-map/cli-operations.md b/feature-map/cli-operations.md new file mode 100644 index 0000000000..34c95f01c6 --- /dev/null +++ b/feature-map/cli-operations.md @@ -0,0 +1,55 @@ +# CLI, API clients, and local worktrees + +Operators and agents can work through explicit CLI/API context, manage domain resources, inspect runs, and create isolated local worktree instances for development. + +Implementation: [command registry](../cli/src/index.ts), [client context](../cli/src/commands/client/context.ts), [worktree commands](../cli/src/commands/worktree.ts). + +## Sub-features + +- `context`: choose instance/API base, profile, identity, and company explicitly. +- `domain-commands`: read/write tasks, agents, projects, goals, approvals, routines, and other registered resources. +- `outputs`: work with documents/assets/skills and inspect machine-readable results. +- `run-controls`: invoke/inspect supported runs and observe actual outcomes. +- `worktree-instances`: initialize, seed, diagnose, and clean up only an owned development instance. + +## How to get to it (user POV) + +### `client-context` + +Use `paperclipai context show/list/use/set`, auth/connect, and the registered resource command help. + +### `subresources-and-runs` + +Use issue subresource, asset/skill, and run/heartbeat commands exposed by the CLI help. + +### `worktree-instance` + +Use the documented worktree/test-drive commands in an owned checkout; inspect `paperclipai worktree --help` first. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use `--help` from the checked-out CLI to discover exact arguments. Select a disposable company and isolated context; do not inherit an unknown API target. + +### `client-context` + +Automated: [context](../cli/src/__tests__/context.test.ts), [HTTP client](../cli/src/__tests__/http.test.ts), and [operations parity](../cli/src/__tests__/operations-parity.test.ts) cover client contracts, not every live endpoint. + +Manual: Inspect the selected context and identity, list the disposable company’s resources, create a harmless task through the supported command, and verify it in the browser. Exercise an invalid token and wrong-company resource. Inspect JSON output and nonzero errors rather than only printed success text. + +### `subresources-and-runs` + +Automated: [issue subresources](../cli/src/__tests__/issue-subresources.test.ts) and [asset/skill parity](../cli/src/__tests__/admin-asset-skill-parity.test.ts) cover command/API contracts. + +Manual: Write a small task document or attach a fixture using the supported command, open it in the UI, and compare contents. Invoke eligible test work, inspect the resulting run ID/status/output, and verify denial when the actor lacks permission. + +### `worktree-instance` + +Automated: [worktree commands](../cli/src/__tests__/worktree.test.ts) and [test drive](../cli/src/__tests__/test-drive.test.ts) cover isolation/setup contracts. + +Manual: Initialize an owned disposable worktree instance, inspect its printed paths and ports, verify health and login, and confirm any seed data belongs to the intended source. Stop and clean up only that instance; inspect preserved work before deleting any checkout. + +## Gotchas + +- API success proves its contract, not the equivalent browser creation journey. +- The CLI command registry is broader than the UI page inventory; new command families need a recipe review. +- See [instance operations](./instance-operations.md) for install/update/backup and feature-specific recipes for domain behavior. diff --git a/feature-map/companies.md b/feature-map/companies.md new file mode 100644 index 0000000000..d0ce685318 --- /dev/null +++ b/feature-map/companies.md @@ -0,0 +1,44 @@ +# Companies and portability + +An operator can switch between companies, change their identity and settings, archive them, and preview an import or export. Each company remains a separate scope for work and agents. + +Implementation: [companies](../ui/src/pages/Companies.tsx), [settings](../ui/src/pages/CompanySettings.tsx), [import](../ui/src/pages/CompanyImport.tsx), [export](../ui/src/pages/CompanyExport.tsx). + +## Sub-features + +- `selection`: create/select a company and keep navigation in its scope. +- `identity`: update company identity and configured settings. +- `archive`: archive and restore through the available controls without confusing archival with deletion. +- `portability`: preview a company package and inspect imported agents, projects, skills, and conflicts. + +## How to get to it (user POV) + +### `company-switcher` + +Use the company switcher or `/companies`, then enter company Settings. + +### `package-transfer` + +Use `/company/export` and `/company/import` where enabled, or the corresponding CLI company commands. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use two disposable companies and a small package fixture. Record cloud-managed restrictions and visible settings gates. + +### `company-switcher` + +Automated: [company list](../ui/src/pages/Companies.test.tsx) and [archive UI](../ui/src/pages/CompanySettingsArchive.test.tsx) cover UI behavior with mocks. + +Manual: Create or select a company, rename it, and reload. Switch to the second company and back; verify tasks, agents, and settings follow the selection. Archive the disposable company and verify its visibility and offered restoration path. + +### `package-transfer` + +Automated: [portability routes](../server/src/__tests__/company-portability-routes.test.ts) and [CLI import/export](../cli/src/__tests__/company-import-export-e2e.test.ts) cover package and command behavior, not every external Git host. + +Manual: Export a small company, inspect the package preview, and import into a disposable target. Verify counts, references, conflict choices, and agent configuration. Run one imported task after supplying its required local credentials. Confirm source company data is unchanged. + +## Gotchas + +- Package presence does not prove a runnable imported company; credentials and environment bindings need separate verification. +- Company deletion is a distinct gated operation; do not use it as the archive test. +- Settings hidden in a managed deployment should be recorded as unavailable, not bypassed. diff --git a/feature-map/connection-setup.md b/feature-map/connection-setup.md new file mode 100644 index 0000000000..00c9328a9a --- /dev/null +++ b/feature-map/connection-setup.md @@ -0,0 +1,150 @@ +# Connection setup + +People connect an app or account, choose who may use it, and recover incomplete +setup from either Apps or a task that needs access. A saved draft, a completed +provider login, a usable connection, and a resumed task are separate checkpoints. +Chat channels additionally require an endpoint and linked sender identity. + +## Sub-features + +- `catalog`: find a provider, inspect its available methods, and open setup; + unavailable methods explain their prerequisite rather than silently switching. +- `identity-and-access`: select personal/shared identity and eligible agents; + an empty explicit selection and company-wide access must not be confused. +- `credentials`: provider sign-in, key, or generic MCP URL follows its supported + setup method; credentials are not retained in browser draft storage. +- `handoff`: provider popup/full-page callback returns to the correct setup + host with the original task intent and identity choice intact. +- `draft-recovery`: cancel, Back, Save & exit, reload, Finish setup, and reconnect + preserve the exact draft rather than creating duplicate connections. +- `task-request`: only the addressed person resolves the requested connection; + usable access releases the pending task request through normal continuation. +- `channel-setup`: provider endpoint, represented agent, linked sender, and + communication instructions are configured before an external message is used. +- `failure`: denied login, unreachable/private URL, expired credentials, and + permission errors leave an actionable state rather than a false success. + +## How to get to it (user POV) + +### `apps-catalog` + +Open Apps (`/apps`), choose an app, and connect from its landing page. Select an +available method and access, then complete setup. Existing connections open +their details; unfinished ones offer a way to finish setup. + +### `mcp-link` + +Use **Connect with a link** or `/apps/byo` for a generic MCP server. Paste its +URL, select access, and follow the probe's sign-in/key guidance. Provider-specific +catalog setup and a generic endpoint may use different flows. + +### `task-request` + +Open the task's pending connection request and its setup action as the named +user. Setup can open in a task dialog and may use an OAuth popup. Follow the +request's original link if setup falls back to a standalone page. + +### `resume-connection` + +From Apps or connection details, use Finish setup, reconnect, or the offered +authentication recovery action. Reload or return from the provider before +checking whether the same saved connection became usable. + +### `chat-channel` + +With chat connectors enabled, select a supported chat/email provider in Apps +or open `/apps/chat/connect`. Complete that provider's endpoint setup, then +open its settings (`/apps/chat/:endpointId/settings`). Follow identity linking +as the intended sender before messaging the agent externally. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Choose a disposable provider account/resource and record its auth method, +deployment mode, account ownership, and access selection. For live setup, use +the provider's documented prerequisites in the +[connector playbook](../doc/connections/CONNECTOR-PLAYBOOK.md). Automated fixtures +do not establish OAuth registration, public callbacks, or production provider access. + +### `apps-catalog` + +Automated: [Browse](../ui/src/pages/apps/Browse.test.tsx), +[unconnected app](../ui/src/pages/apps/AppNotConnected.test.tsx), and +[AppsConnect](../ui/src/pages/apps/AppsConnect.test.tsx) cover catalog entry, +access selection, provider-specific method choices, and setup failures with +mocked APIs. [Connection intents](../tests/e2e/connection-intents.spec.ts) +drives store and task setup against one fake provider through a useful continuation: + +```sh +pnpm exec vitest run ui/src/pages/apps/Browse.test.tsx ui/src/pages/apps/AppsConnect.test.tsx +pnpm exec playwright test -c tests/e2e/playwright.config.ts tests/e2e/connection-intents.spec.ts +``` + +Manual: connect from an unconnected provider landing. Select the intended +identity and agent access, complete authentication, reload, and inspect the +saved connection. Run a permitted read-only action with an eligible agent and +verify its useful result. Try an ineligible agent and confirm denial. Repeat +with a user who cannot install company-wide; the UI must not imply broad access. + +### `mcp-link` + +Automated: AppsConnect's generic MCP cases cover probe-driven auth, private or +unreachable endpoints, preregistered clients, secret-backed headers, and unsafe +authorization URLs. These are component/API mocks, not remote provider proof. + +Manual: connect a known test MCP URL through the link flow, verify its saved +endpoint and a read-only action, then repeat with a bad URL and wrong credential. +The error should explain the next action and preserve non-secret setup context. +Test no-auth and authenticated endpoints separately; never infer one from the other. + +### `task-request` + +Automated: [interaction card](../ui/src/components/IssueThreadInteractionCard.test.tsx) +checks addressed-user controls. AppsConnect tests cover page/dialog parity, +task-bound OAuth, popup handoff, and verifying a saved connection before closing. +The connection-intents browser suite above exercises fixture-backed continuation. + +Manual: ask an agent to use an unavailable app, open its request as the named +user, and complete setup in the dialog. Verify the card settles, the exact task +resumes, and the agent returns a useful provider result. Repeat the page-host +fallback and denied OAuth path. As another user, confirm that merely seeing the +task does not grant the right to resolve the request. + +### `resume-connection` + +Automated: AppsConnect tests cover exact-draft resume, declined OAuth, reconnect +lookup failure/retry, response-lost creation, archived connection revival, and +saved account reuse. [App detail](../ui/src/pages/apps/AppDetail.test.tsx) covers +the connection host. These tests do not complete a real provider login. + +Manual: leave setup through its footer, reload Apps, choose Finish setup, and +confirm the same connection is resumed. Decline provider login and recover from +that checkpoint. Reconnect an expired disposable account and verify a permitted +action afterwards. Count connections before/after to catch accidental duplicates. + +### `chat-channel` + +Automated: [setup routing](../ui/src/pages/apps/chat/ChatEndpointSetup.routing.test.tsx), +[saved setup state](../ui/src/pages/apps/chat/ChatEndpointSetup.state.test.ts), +[email setup](../ui/src/pages/apps/chat/EmailEndpointSetup.test.tsx), and +[identity confirmation](../ui/src/pages/apps/chat/ChatIdentityConfirm.test.tsx) +cover shared UI logic with fixtures. Provider-specific live setup is not implied. + +Manual: complete setup for one named provider, reload endpoint settings, link +the intended sender, and send a disposable message. Verify one task and a useful +reply in the originating conversation. Inspect an unlinked sender's denial and +the endpoint's pause behavior. Repeat independently for each provider at issue; +Slack setup does not qualify Discord, Telegram, GitHub, email, or iMessage. + +## Gotchas + +- A successful OAuth callback is not proof of saved connection access or task + continuation. Verify all three checkpoints. +- Personal identity, shared account access, and per-agent access are distinct. + Connecting an account does not waive an Off action or Ask first review. +- A tool app connection is not a chat endpoint. The same provider can offer both. +- Old Apps/Connections URLs can redirect to Apps. Use the current navigation and + record the actual destination rather than documenting an obsolete screen. +- Full provider matrix, AI-account selection, gateway/profile administration, + and account revocation lifecycles remain outside this seed recipe. +- For governed actions after setup, use [questions and approvals](./questions-and-approvals.md). diff --git a/feature-map/coverage.json b/feature-map/coverage.json new file mode 100644 index 0000000000..7fc99df425 --- /dev/null +++ b/feature-map/coverage.json @@ -0,0 +1,401 @@ +{ + "version": 1, + "areas": [ + { + "id": "tasks-and-attention", + "title": "Tasks, conversations, and attention", + "status": "partial", + "features": [ + "tasks.md", + "task-coordination.md", + "inbox-search.md", + "agent-chat.md", + "questions-and-approvals.md", + "steering.md", + "recovery.md", + "documents-artifacts.md" + ], + "gap": "Recipes cover the main task/conversation/attention workflows. Full shell, permission, attachment, long-thread, and concurrency permutations remain to be enumerated.", + "paths": [ + "ui/src/pages/AgentChat.tsx", + "ui/src/pages/AgentChats.tsx", + "ui/src/pages/BoardChat.tsx", + "ui/src/pages/DecisionQueuePage.tsx", + "ui/src/pages/Inbox.tsx", + "ui/src/pages/IssueDetail.tsx", + "ui/src/pages/Issues.tsx", + "ui/src/pages/LegacyInbox.tsx", + "ui/src/pages/MyIssues.tsx", + "ui/src/pages/Search.tsx", + "ui/src/pages/WhatNeedsMe.tsx" + ] + }, + { + "id": "agents", + "title": "Agents, organization, and runtime setup", + "status": "partial", + "features": [ + "agents.md", + "runs-adapters.md", + "skills.md", + "app-permissions.md", + "recovery.md", + "execution-environments.md" + ], + "gap": "Per-harness/model/account capabilities, every instruction-file error, and complete built-in-agent lifecycle variants remain separate qualification matrices.", + "paths": [ + "ui/src/pages/AdapterManager.tsx", + "ui/src/pages/AgentDetail.production.tsx", + "ui/src/pages/AgentDetail.tsx", + "ui/src/pages/AgentToolsTab.tsx", + "ui/src/pages/Agents.production.tsx", + "ui/src/pages/Agents.tsx", + "ui/src/pages/NewAgent.tsx", + "ui/src/pages/Org.tsx", + "ui/src/pages/OrgChart.production.tsx", + "ui/src/pages/OrgChart.tsx" + ] + }, + { + "id": "apps", + "title": "App catalog and connection setup", + "status": "partial", + "features": [ + "connection-setup.md", + "app-permissions.md", + "questions-and-approvals.md" + ], + "gap": "Provider-specific authentication/revocation, credential-only accounts, and aggregator administrative variants still need their own detailed matrices.", + "paths": [ + "ui/src/pages/apps/AggregatorAppManager.tsx", + "ui/src/pages/apps/AggregatorConnectDialog.tsx", + "ui/src/pages/apps/AppDetail.tsx", + "ui/src/pages/apps/AppLogo.tsx", + "ui/src/pages/apps/AppNotConnected.tsx", + "ui/src/pages/apps/AppsConnect.tsx", + "ui/src/pages/apps/AppsReview.tsx", + "ui/src/pages/apps/ArcadeDiscoverySetup.tsx", + "ui/src/pages/apps/Browse.tsx", + "ui/src/pages/apps/CatalogSourceFilters.tsx", + "ui/src/pages/apps/ComposioAppManager.tsx", + "ui/src/pages/apps/ComposioAppSetup.tsx", + "ui/src/pages/apps/ConnectionProvenanceChip.tsx", + "ui/src/pages/apps/Connections.tsx", + "ui/src/pages/apps/ExecutorManagementSetup.tsx", + "ui/src/pages/apps/PaperclipCloudOAuthHandoff.tsx", + "ui/src/pages/apps/ReviewQueueCard.tsx", + "ui/src/pages/apps/UnverifiedServerBadge.tsx", + "ui/src/pages/apps/app-detail/ActionTestDialog.tsx", + "ui/src/pages/apps/app-detail/ActivityPanel.tsx", + "ui/src/pages/apps/app-detail/AdvancedPanel.tsx", + "ui/src/pages/apps/app-detail/AgentConnectionAccess.tsx", + "ui/src/pages/apps/app-detail/BrowserUseSettingsPanel.tsx", + "ui/src/pages/apps/app-detail/ConnectedAggregatorApps.tsx", + "ui/src/pages/apps/app-detail/IdentitiesSection.tsx", + "ui/src/pages/apps/app-detail/PermissionsPanel.tsx", + "ui/src/pages/apps/app-detail/RailwayAccessPanel.tsx", + "ui/src/pages/apps/app-detail/ReviewPanel.tsx", + "ui/src/pages/apps/app-detail/SetupPanel.tsx", + "ui/src/pages/apps/connection-owner.tsx" + ] + }, + { + "id": "chat-channels", + "title": "Chat and email channel setup", + "status": "partial", + "features": [ + "chat-channels.md", + "connection-setup.md", + "questions-and-approvals.md", + "steering.md" + ], + "gap": "Shared recipes name provider checks; each provider transport/thread/identity/file-size/type combination needs separate live evidence and deeper instructions.", + "paths": [ + "ui/src/pages/apps/chat/ChatCommunicationInstructions.tsx", + "ui/src/pages/apps/chat/ChatEndpointDetail.tsx", + "ui/src/pages/apps/chat/ChatEndpointSetup.tsx", + "ui/src/pages/apps/chat/ChatIdentityConfirm.tsx", + "ui/src/pages/apps/chat/EmailEndpointSetup.tsx", + "ui/src/pages/apps/chat/GitHubBotConfiguration.tsx", + "ui/src/pages/apps/chat/GitHubBotManagement.tsx", + "ui/src/pages/apps/chat/GitHubChatSetup.tsx", + "ui/src/pages/apps/chat/GitHubSetupPrompt.tsx", + "ui/src/pages/apps/chat/PhotonConnectStep.tsx", + "ui/src/pages/apps/chat/SetupPrompt.tsx", + "ui/src/pages/apps/chat/SlackAvatarStep.tsx", + "ui/src/pages/apps/chat/SlackIdentityStep.tsx", + "ui/src/pages/apps/chat/SlackSetupPrompt.tsx", + "ui/src/pages/apps/chat/SlackToolSettings.tsx" + ] + }, + { + "id": "skills-and-teams", + "title": "Skills, Skill Studio, and teams", + "status": "partial", + "features": [ + "skills.md", + "teams.md", + "questions-and-approvals.md" + ], + "gap": "Main library/Studio/source/policy and CLI team-install paths are mapped. Full source-provider conflicts, harness sync, and remote package variants remain; catalog UI is not a current top-level route.", + "paths": [ + "ui/src/pages/CompanySkills.production.tsx", + "ui/src/pages/CompanySkills.tsx", + "ui/src/pages/SkillSources.tsx", + "ui/src/pages/SkillStudio.tsx", + "ui/src/pages/TeamCatalog.tsx", + "ui/src/pages/agent-skills/AgentSkillReleasePicker.tsx", + "ui/src/pages/agent-skills/AgentSkillRow.tsx", + "ui/src/pages/agent-skills/AgentSkillsTab.tsx", + "ui/src/pages/skills/ImportSkillsFromProjectDialog.tsx", + "ui/src/pages/skills/SkillImportProgress.tsx", + "ui/src/pages/skills/SkillPackagePreview.tsx", + "ui/src/pages/skills/SkillSourceTree.tsx" + ] + }, + { + "id": "projects-and-workspaces", + "title": "Projects and execution workspaces", + "status": "partial", + "features": [ + "projects.md", + "workspaces.md", + "goals.md", + "execution-environments.md", + "budgets-costs.md", + "recovery.md" + ], + "gap": "Provider-specific provisioning, multi-repository/submodule edge cases, Git reconciliation variants, and all service exposure combinations require deeper recipes.", + "paths": [ + "ui/src/pages/ExecutionWorkspaceDetail.tsx", + "ui/src/pages/ProjectDetail.tsx", + "ui/src/pages/ProjectWorkspaceDetail.tsx", + "ui/src/pages/Projects.tsx", + "ui/src/pages/Workspaces.tsx" + ] + }, + { + "id": "formal-approvals", + "title": "Formal governance approvals", + "status": "partial", + "features": [ + "questions-and-approvals.md", + "task-coordination.md", + "agents.md" + ], + "gap": "Formal board decisions and signoff have recipes; every approval payload/type and actor/host error permutation is not enumerated.", + "paths": [ + "ui/src/pages/ApprovalDetail.tsx", + "ui/src/pages/Approvals.tsx" + ] + }, + { + "id": "auth-and-access", + "title": "Login, invitations, and access", + "status": "partial", + "features": [ + "onboarding.md", + "access.md", + "navigation-preferences.md", + "cli-operations.md" + ], + "gap": "Deployment modes and core login/invite/member/token paths are mapped; exhaustive identity-provider, expiry, role, and resource-policy combinations remain.", + "paths": [ + "ui/src/pages/Auth.tsx", + "ui/src/pages/BoardClaim.tsx", + "ui/src/pages/CliAuth.tsx", + "ui/src/pages/CompanyAccess.tsx", + "ui/src/pages/InstanceAccess.tsx", + "ui/src/pages/InviteLanding.tsx", + "ui/src/pages/JoinRequestQueue.tsx", + "ui/src/pages/UserProfile.tsx" + ] + }, + { + "id": "company-and-instance", + "title": "Company and instance administration", + "status": "partial", + "features": [ + "companies.md", + "onboarding.md", + "instance-operations.md", + "execution-environments.md", + "navigation-preferences.md" + ], + "gap": "Core administration/import/export/operator workflows are mapped; cloud/self-hosted differences, migration/restore compatibility, and platform-specific service variants need deeper recipes.", + "paths": [ + "ui/src/pages/Companies.tsx", + "ui/src/pages/CompanyEnvironments.tsx", + "ui/src/pages/CompanyExport.tsx", + "ui/src/pages/CompanyImport.tsx", + "ui/src/pages/CompanySettings.tsx", + "ui/src/pages/InstanceExperimentalSettings.tsx", + "ui/src/pages/InstanceGeneralSettings.tsx", + "ui/src/pages/ProfileSettings.tsx" + ] + }, + { + "id": "goals-costs-and-activity", + "title": "Goals, dashboards, costs, and activity", + "status": "partial", + "features": [ + "goals.md", + "budgets-costs.md", + "activity.md", + "runs-adapters.md", + "routines.md" + ], + "gap": "Core goal/report/budget/audit journeys are mapped. Every aggregation/filter combination, cost source, and overlapping budget window still needs dedicated cases.", + "paths": [ + "ui/src/pages/Costs.production.tsx", + "ui/src/pages/Costs.tsx", + "ui/src/pages/Dashboard.tsx", + "ui/src/pages/DashboardLive.tsx", + "ui/src/pages/GoalDetail.tsx", + "ui/src/pages/Goals.tsx", + "ui/src/pages/Timeline.tsx", + "ui/src/pages/audit/AuditFeed.production.tsx", + "ui/src/pages/audit/AuditFeed.tsx", + "ui/src/pages/audit/AuditHub.tsx", + "ui/src/pages/audit/AuditRuns.tsx", + "ui/src/pages/audit/CompanyActivity.production.tsx", + "ui/src/pages/audit/CompanyActivity.tsx", + "ui/src/pages/audit/RoutineAuditActivity.tsx" + ] + }, + { + "id": "artifacts", + "title": "Artifacts and work products", + "status": "partial", + "features": [ + "documents-artifacts.md" + ], + "gap": "Document/attachment/library/work-product journeys are mapped; every media preview type, storage backend, annotation conflict, and runtime-resource lifecycle variant remains separate.", + "paths": [ + "ui/src/pages/Artifacts.tsx" + ] + }, + { + "id": "routines-and-pipelines", + "title": "Routines, pipelines, cases, and status cards", + "status": "partial", + "features": [ + "routines.md", + "pipelines.md", + "cases.md", + "status-cards.md" + ], + "gap": "All four capabilities have recipes. Trigger/provider/timezone combinations, custom case schemas, stage automation types, and summarizer quality need deeper cases.", + "paths": [ + "ui/src/pages/CaseDetail.tsx", + "ui/src/pages/Cases.tsx", + "ui/src/pages/PipelineSettings.tsx", + "ui/src/pages/Pipelines.tsx", + "ui/src/pages/RoutineDetail.production.tsx", + "ui/src/pages/RoutineDetail.tsx", + "ui/src/pages/Routines.production.tsx", + "ui/src/pages/Routines.tsx", + "ui/src/pages/StatusCards/ArchivedStatusCardRow.tsx", + "ui/src/pages/StatusCards/CreateStatusCardDialog.tsx", + "ui/src/pages/StatusCards/StatusCardDetailDrawer.tsx", + "ui/src/pages/StatusCards/StatusCardSettingsForm.tsx", + "ui/src/pages/StatusCards/StatusCardTile.tsx", + "ui/src/pages/StatusCards/SummarizerAgentSelect.tsx", + "ui/src/pages/StatusCards/index.tsx" + ] + }, + { + "id": "secrets", + "title": "Secrets and secret proposals", + "status": "partial", + "features": [ + "secrets.md", + "access.md", + "app-permissions.md" + ], + "gap": "Company/user/proposal/vault and runtime-binding paths are mapped; every secret provider, rotation scenario, and delivery context needs separate validation.", + "paths": [ + "ui/src/pages/Secrets.tsx", + "ui/src/pages/secrets/ImportFromVaultDialog.tsx", + "ui/src/pages/secrets/MissingUserSecretsBanner.tsx", + "ui/src/pages/secrets/MyUserSecretsTab.tsx", + "ui/src/pages/secrets/ProposalsTab.tsx", + "ui/src/pages/secrets/SecretPathName.tsx", + "ui/src/pages/secrets/SetMyUserSecretDialog.tsx", + "ui/src/pages/secrets/UserSecretDefinitionsTab.tsx", + "ui/src/pages/secrets/proposal-review.tsx", + "ui/src/pages/secrets/user-secret-presentation.tsx" + ] + }, + { + "id": "plugins-and-tools", + "title": "Plugins, gateways, and tool profiles", + "status": "partial", + "features": [ + "plugins.md", + "gateways-profiles.md", + "app-permissions.md", + "execution-environments.md" + ], + "gap": "Host plugin/gateway/profile lifecycles are mapped. Third-party plugin behaviors and every client transport/provider combination are outside host-wide coverage.", + "paths": [ + "ui/src/pages/CompanySettingsPluginPage.tsx", + "ui/src/pages/PluginManager.tsx", + "ui/src/pages/PluginPage.tsx", + "ui/src/pages/PluginSettings.tsx", + "ui/src/pages/apps/gateways/ConnectClientDialog.tsx", + "ui/src/pages/apps/gateways/CopyableGatewayUrl.tsx", + "ui/src/pages/apps/gateways/EditGatewayDialog.tsx", + "ui/src/pages/apps/gateways/GatewayDetail.tsx", + "ui/src/pages/apps/gateways/GatewaysList.tsx", + "ui/src/pages/apps/gateways/NewGatewayDialog.tsx", + "ui/src/pages/apps/gateways/panels/AppsToolsPanel.tsx", + "ui/src/pages/apps/gateways/panels/GatewayActivityPanel.tsx", + "ui/src/pages/apps/gateways/panels/GatewayAdvancedPanel.tsx", + "ui/src/pages/apps/gateways/panels/OverviewPanel.tsx", + "ui/src/pages/apps/gateways/panels/TokensPanel.tsx", + "ui/src/pages/tools/AdvancedToolsRoute.tsx", + "ui/src/pages/tools/AuditTab.tsx", + "ui/src/pages/tools/GatewaysTab.tsx", + "ui/src/pages/tools/McpConfigHelpDialog.tsx", + "ui/src/pages/tools/PasteConfigTab.tsx", + "ui/src/pages/tools/ProfilesTab.tsx", + "ui/src/pages/tools/SmokeLabTab.tsx", + "ui/src/pages/tools/ToolsAccess.tsx", + "ui/src/pages/tools/connection-dialogs.tsx", + "ui/src/pages/tools/profiles/ProfileActionDialog.tsx", + "ui/src/pages/tools/profiles/ProfileDetail.tsx", + "ui/src/pages/tools/profiles/ProfileDetailRoute.tsx", + "ui/src/pages/tools/profiles/ProfileWizard.tsx", + "ui/src/pages/tools/profiles/ProfileWizardRoute.tsx", + "ui/src/pages/tools/profiles/ProfilesIndex.tsx", + "ui/src/pages/tools/profiles/ToolsAdminGate.tsx", + "ui/src/pages/tools/profiles/WizardToolsStep.tsx", + "ui/src/pages/tools/shared.tsx" + ] + }, + { + "id": "labs-and-fallbacks", + "title": "Developer labs and fallback pages", + "status": "partial", + "features": [ + "developer-labs.md", + "navigation-preferences.md" + ], + "gap": "Developer and fallback surfaces are identified. Each synthetic fixture and performance budget is not a production journey and needs its own measurement.", + "paths": [ + "ui/src/pages/BootstrapSetupUxLab.tsx", + "ui/src/pages/CrossIssueCollaborationUxLab.tsx", + "ui/src/pages/DesignGuide.tsx", + "ui/src/pages/InviteUxLab.tsx", + "ui/src/pages/IssueChatLongThreadPerf.tsx", + "ui/src/pages/IssueChatUxLab.tsx", + "ui/src/pages/NotFound.tsx", + "ui/src/pages/ResponsibleUserDenialUxLab.tsx", + "ui/src/pages/RunTranscriptUxLab.tsx", + "ui/src/pages/SystemNoticeUxLab.tsx", + "ui/src/pages/TaskChatLab.tsx" + ] + } + ] +} diff --git a/feature-map/developer-labs.md b/feature-map/developer-labs.md new file mode 100644 index 0000000000..dc9a7fbec0 --- /dev/null +++ b/feature-map/developer-labs.md @@ -0,0 +1,43 @@ +# Developer previews and diagnostic labs + +Contributors can inspect design examples, interaction labs, and performance fixtures. These are development surfaces and do not count as production feature verification. + +Implementation: [route registration](../ui/src/App.tsx), [design guide](../ui/src/pages/DesignGuide.tsx), [long-thread fixture](../ui/src/pages/IssueChatLongThreadPerf.tsx). + +## Sub-features + +- `design`: inspect shared components and layout examples in the design guide or Storybook. +- `interaction-labs`: exercise synthetic bootstrap, task-chat, denial, and collaboration states. +- `performance`: measure the provided long-thread fixture with known data and environment. +- `route-gates`: distinguish dev-only, experimental, historical, and fallback surfaces. + +## How to get to it (user POV) + +### `preview-lab` + +Use `/design-guide`, registered `/ux-lab/...` routes, or Storybook; consult `ui/src/App.tsx` for the current gated set. + +### `performance-fixture` + +Open `/tests/perf/long-thread` only in an appropriate development environment. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a development/test environment. Confirm the route is registered for the current build before claiming the page is available. + +### `preview-lab` + +Automated: [onboarding variant](../ui/src/components/OnboardingWizardVariant.test.tsx) is an example component contract. There is no claim of full automated acceptance coverage for every lab page. + +Manual: Open the specific fixture state, inspect controls and layout, then repeat the affected interaction in the actual product host. Report fixture results separately from persisted task/company results. + +### `performance-fixture` + +Automated: [task thread component](../ui/src/components/IssueChatThread.test.tsx) does not establish a performance budget. Timing/scroll measurements need a dedicated browser run. + +Manual: Record browser/build/data size, exercise scrolling and composing in the fixture, and retain timing or trace evidence. Confirm a representative real task does not regress before extrapolating fixture measurements. + +## Gotchas + +- An unmounted or obsolete lab file remains in the conservative page inventory but is not a shipped entry point. +- Screenshots of a lab do not prove live provider behavior, persistence, or authorization. diff --git a/feature-map/documents-artifacts.md b/feature-map/documents-artifacts.md new file mode 100644 index 0000000000..39d96ade8b --- /dev/null +++ b/feature-map/documents-artifacts.md @@ -0,0 +1,55 @@ +# Documents, attachments, and work products + +People read and revise task documents, annotate specific content, open attached files, and inspect the outputs an agent hands back. The artifact library provides another route to the same work. + +Implementation: [task documents](../ui/src/components/IssueDocumentsSection.tsx), [artifact library](../ui/src/pages/Artifacts.tsx), [document panel](../ui/src/components/task-side-panel/TaskDocumentPanel.tsx). + +## Sub-features + +- `documents`: edit versioned task documents and inspect or restore revisions. +- `annotations`: attach feedback to document content and inspect resolution state. +- `attachments`: upload, preview, and download allowed files with company access checks. +- `work-products`: link deliverables to their source task, resource, and review context. +- `library`: search, filter, group, and open artifact stacks. + +## How to get to it (user POV) + +### `task-document` + +Open a document from the task thread or document side panel. + +### `task-output` + +Open an attachment, output card, or file link from a task. + +### `artifact-library` + +Open `/artifacts` and its grouping, search, and media controls. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a task with a text document, a small attachment, and a generated work product. Use nonsensitive fixture files. + +### `task-document` + +Automated: [documents service](../server/src/__tests__/documents-service.test.ts), [annotations](../server/src/__tests__/document-annotations-service.test.ts), and [revision restore](../server/src/__tests__/issue-document-restore-routes.test.ts) cover server contracts. + +Manual: Edit a paragraph, save, reload, and compare revisions. Add an annotation to a specific selection and inspect it after another edit. Resolve it through the supported action. Restore a disposable revision and confirm the new current document and history agree. + +### `task-output` + +Automated: [attachment routes](../server/src/__tests__/issue-attachment-routes.test.ts) and [work products](../server/src/__tests__/work-products.test.ts) cover data/access behavior; [artifact arrival browser suite](../tests/e2e/artifact-tab-arrival.spec.ts) targets the arrival experience. + +Manual: Upload a fixture and have the test task produce an output. Preview and download each, compare the contents, and follow the source task link. Reload and test an inaccessible user/company; verify missing workspace-only resources have an explicit state. + +### `artifact-library` + +Automated: [artifact page](../ui/src/pages/Artifacts.test.tsx) covers grouping, search requests, and pagination using mocks. + +Manual: Find the task’s output through search and grouping, open its stack, page through results, and reload a direct stack link. Confirm the same file/content appears as from the task. + +## Gotchas + +- A local path is not automatically an uploaded artifact accessible to another user. +- A preview screenshot is not proof that download bytes or revision references are correct. +- Version conflicts and stale annotations must stay visible rather than silently targeting different content. diff --git a/feature-map/execution-environments.md b/feature-map/execution-environments.md new file mode 100644 index 0000000000..feb89f353d --- /dev/null +++ b/feature-map/execution-environments.md @@ -0,0 +1,45 @@ +# Local, SSH, and sandbox environments + +Operators configure where agents execute, probe readiness, select an environment for work, and inspect remote provisioning or custom-image state when the installed provider supports it. + +Implementation: [environment settings](../ui/src/pages/CompanyEnvironments.tsx), [environment runtime](../server/src/services/environment-runtime.ts). + +## Sub-features + +- `configuration`: inspect local environments and configure available SSH/sandbox targets. +- `capabilities`: select only drivers/providers that advertise the required execution capability. +- `probe`: test reachability and runtime prerequisites in the actual target. +- `assignment`: bind the intended agent/project/workspace to its execution target. +- `images`: inspect supported sandbox custom-image preparation and terminal workflows. + +## How to get to it (user POV) + +### `environment-settings` + +Open Settings → Environments, then create/edit a supported target. + +### `remote-execution` + +Choose the environment in agent/project/workspace configuration, then inspect the resulting task/run and workspace. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an owned disposable target and a supported provider plugin. Record provider, target identity, account, and spending constraints before a live remote run. + +### `environment-settings` + +Automated: [environment UI](../ui/src/pages/CompanyEnvironments.test.tsx), [driver configuration UI](../ui/src/pages/CompanySettings.test.tsx), and [environment service](../server/src/__tests__/environment-service.test.ts) cover configuration behavior. + +Manual: Save a disposable target, reload it, and run its available probe. Verify both success and missing-credential/reachability errors identify the target. Confirm unavailable providers are not offered as working run targets. + +### `remote-execution` + +Automated: [execution target](../server/src/__tests__/environment-execution-target.test.ts) and [custom images](../server/src/__tests__/environment-custom-images-service.test.ts) cover contracts; [SSH live test](../server/src/__tests__/environment-live-ssh.test.ts) requires its own external setup. + +Manual: Run a bounded command in the intended target, inspect its output and workspace, and verify execution did not silently fall back to the server host. For custom-image changes, build/probe the disposable image before assigning it. Inspect cleanup and shutdown after the run. + +## Gotchas + +- A probe is narrower evidence than a complete task with tools, files, and resumption. +- The local option may appear only when editing an existing local environment; availability follows current UI/provider capabilities. +- Workspace service reachability and environment reachability are separate checks. diff --git a/feature-map/gateways-profiles.md b/feature-map/gateways-profiles.md new file mode 100644 index 0000000000..616c73d8c3 --- /dev/null +++ b/feature-map/gateways-profiles.md @@ -0,0 +1,45 @@ +# Tool gateways and access profiles + +Administrators expose selected tools through gateways and reusable access profiles, connect clients, manage tokens, and inspect gateway activity. + +Implementation: [gateways](../ui/src/pages/apps/gateways/GatewaysList.tsx), [gateway detail](../ui/src/pages/apps/gateways/GatewayDetail.tsx), [profiles](../ui/src/pages/tools/profiles/ProfileWizard.tsx). + +## Sub-features + +- `gateway`: create/edit an endpoint and select its exposed apps/tools. +- `profiles`: define reusable tool access configuration and inspect draft/detail state. +- `client-connect`: copy the intended endpoint/configuration to an authorized test client. +- `tokens`: create/revoke client credentials and verify their scope. +- `activity`: inspect tool discovery/calls and explicit errors. + +## How to get to it (user POV) + +### `gateway-client` + +Open `/apps/gateways`, then a gateway’s Overview, Apps/Tools, Tokens, and Activity tabs. + +### `access-profile` + +Open `/apps/advanced/profiles`, create a profile, or follow its detail/edit route. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a fixture tool connection and test client. Record gateway/profile, token owner, transport, and selected company. + +### `gateway-client` + +Automated: [gateway service](../server/src/__tests__/tool-gateway-service.test.ts), [connect client dialog](../ui/src/pages/apps/gateways/ConnectClientDialog.test.tsx), and [token panel](../ui/src/pages/apps/gateways/panels/TokensPanel.test.tsx) cover service/UI contracts. + +Manual: Create a disposable gateway with one harmless tool, connect a client using its provided configuration, discover/call that tool, and inspect activity. Revoke the client token and confirm the same call fails. Test a tool outside the allowed set. + +### `access-profile` + +Automated: [profile wizard](../ui/src/pages/tools/profiles/ProfileWizard.test.tsx) and [profile detail](../ui/src/pages/tools/profiles/ProfileDetail.test.tsx) cover component behavior with mocks. + +Manual: Create a bounded profile, choose its offered tools and options, save or resume its draft, and reload the detail. Apply/connect it through the offered flow and inspect the actual client-visible tool set. Verify cancellation and invalid selections retain an understandable draft. + +## Gotchas + +- A copied endpoint is not proof of client authentication or tool discovery. +- Gateway access cannot replace underlying connection/action permission checks. +- Advanced tool routes require the appropriate admin gate; legacy tool URLs may redirect. diff --git a/feature-map/goals.md b/feature-map/goals.md new file mode 100644 index 0000000000..a282d35345 --- /dev/null +++ b/feature-map/goals.md @@ -0,0 +1,42 @@ +# Goals and work alignment + +People define company and lower-level goals and connect projects and tasks to the outcome they serve. + +Implementation: [goals](../ui/src/pages/Goals.tsx), [goal detail](../ui/src/pages/GoalDetail.tsx). + +## Sub-features + +- `hierarchy`: create and inspect goal level, parent, owner, and status. +- `alignment`: associate work with a valid goal in the same company. +- `navigation`: follow goal/project/task context to understand why work exists. + +## How to get to it (user POV) + +### `goal-detail` + +Open `/goals`, create a goal, and enter `/goals/:goalId`. + +### `task-goal-context` + +Inspect the goal context from a project or task and compare explicit versus inherited context. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare two goals and a disposable project/task in the selected company. + +### `goal-detail` + +Automated: [project-goal validation](../server/src/__tests__/project-goal-validation.test.ts) covers relationship validation; [goal detail control](../ui/src/pages/GoalDetail.test.tsx) only tests the properties toggle, not the whole creation journey. + +Manual: Create a parent and child goal, save owner/status changes, and reload. Link a project and task through the offered controls, follow their goal context, and verify a cross-company or invalid relationship is rejected. + +### `task-goal-context` + +Automated: [task goal routes](../server/src/__tests__/issues-goal-context-routes.test.ts) and [goal fallback](../server/src/__tests__/issue-goal-fallback.test.ts) cover the server rules. + +Manual: Compare a task with an explicit goal to one relying on project context. Change the allowed association, reload, and verify the displayed/injected context follows the documented server rule rather than retaining an unrelated goal. + +## Gotchas + +- A goal status is not an independently measured proof of business success. +- Do not infer a complete goal browser test from the small properties-toggle suite. diff --git a/feature-map/inbox-search.md b/feature-map/inbox-search.md new file mode 100644 index 0000000000..2ef31ac7c7 --- /dev/null +++ b/feature-map/inbox-search.md @@ -0,0 +1,54 @@ +# Inbox, decisions, and search + +People find work that needs their attention, distinguish unread activity from required decisions, archive handled items, and search across the company. + +Implementation: [inbox](../ui/src/pages/Inbox.tsx), [decisions](../ui/src/pages/WhatNeedsMe.tsx), [search](../ui/src/pages/Search.tsx). + +## Sub-features + +- `views`: use Mine, Recent, Unread, Blocked, and broader task views. +- `triage`: read or archive attention items without pretending the underlying task is complete. +- `decisions`: open the source of a decision and act in the right context. +- `search`: query company work and follow a result to its original record. + +## How to get to it (user POV) + +### `inbox` + +Use Inbox links or the corresponding task views when the combined list is enabled. + +### `decisions` + +Open `/decisions`, then a queue at `/decisions/queues/:key` and its source item. + +### `search` + +Open `/search` or the navigation search entry. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Seed assigned, unread, blocked, and resolved tasks plus a pending decision. Record the combined Inbox/Tasks experimental setting. + +### `inbox` + +Automated: [archive routes](../server/src/__tests__/inbox-archive-routes.test.ts) cover server mutations. [Task-list helpers](../ui/src/pages/Issues.test.tsx) cover search URLs, pagination, and presentation constants; they do not render the combined Inbox/Tasks page. Reading, archiving, and navigating that page remain manual checks. + +Manual: Read an unread task, return to Unread, archive a handled item, and reload. Verify personal triage state changes while task status stays correct. Switch users and companies to check isolation. + +### `decisions` + +Automated: [decision queues](../server/src/__tests__/decision-queues-routes.test.ts) covers route behavior. End-to-end queue-to-source navigation is manual. + +Manual: Open a pending decision, follow its source, resolve it through the supported control, and reload the queue. Confirm it retires only after the decision persisted and unrelated pending items remain. + +### `search` + +Automated: [search page](../ui/src/pages/Search.test.tsx) and [company search service](../server/src/__tests__/company-search-service.test.ts) cover UI and server search separately. + +Manual: Search a unique task title and content phrase, open the result, and compare the actual record. Test filters, no results, and a second company. Verify stale results do not expose inaccessible data. + +## Gotchas + +- Personal archival is not task cancellation or completion. +- Legacy Inbox routes can redirect to the combined task view; record which host actually rendered. +- Search service tests do not establish the relevance quality of every query. diff --git a/feature-map/instance-operations.md b/feature-map/instance-operations.md new file mode 100644 index 0000000000..dbed223088 --- /dev/null +++ b/feature-map/instance-operations.md @@ -0,0 +1,45 @@ +# Installation, configuration, updates, and backups + +An operator can install and run Paperclip, diagnose the instance, configure supported deployment settings, manage its service, and produce a recoverable database backup. + +Implementation: [CLI entry point](../cli/src/index.ts), [instance settings](../ui/src/pages/InstanceExperimentalSettings.tsx), [backup command](../cli/src/commands/db-backup.ts). + +## Sub-features + +- `install-update`: inspect install channel/version and apply supported update or rollback operations. +- `server-service`: start/stop the intended instance and inspect its service health. +- `configuration`: configure database, server binding, storage, secrets, and available experimental settings. +- `diagnostics`: use doctor/health output to distinguish readiness from a listening port. +- `backup`: create and inspect backups and verify restore only into a separate disposable target. + +## How to get to it (user POV) + +### `operator-cli` + +Use `paperclipai install`, `run`, `service`, `doctor`, `configure`, `channels`, or `update --help` for the intended operation. + +### `settings-backup` + +Use currently visible instance settings for experiments/access and `paperclipai db:backup --help` for one-off backup. Some legacy general-settings URLs redirect. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable data directory/service identity. Do not update, restart, or restore over an unrelated live instance as a documentation test. + +### `operator-cli` + +Automated: [doctor](../cli/src/__tests__/doctor.test.ts), [update command](../cli/src/__tests__/update-command.test.ts), and [service manager](../cli/src/__tests__/service-manager.test.ts) cover command contracts. + +Manual: Inspect the selected installation/channel and configuration, start the isolated instance, confirm health and browser access, then stop that instance. For update changes, use the dry-run/check path first and verify installed version and rollback state on an owned disposable installation. + +### `settings-backup` + +Automated: [experimental settings](../ui/src/pages/InstanceExperimentalSettings.test.tsx) and [database backup routes](../server/src/__tests__/instance-database-backups-routes.test.ts) cover their UI/API contracts. + +Manual: Change a harmless experimental setting, reload, and confirm the affected route honors it. Back up a disposable instance, inspect the reported file and metadata, and restore using the documented procedure into a second isolated target. Verify a known record after restoration. + +## Gotchas + +- A successful backup write does not prove restoration works. +- Settings visibility and mutability differ for cloud-managed and self-hosted instances. +- Keep the three data paths distinct: first-party Telemetry, operator-configured OpenTelemetry, and local run logs. diff --git a/feature-map/navigation-preferences.md b/feature-map/navigation-preferences.md new file mode 100644 index 0000000000..66794028f8 --- /dev/null +++ b/feature-map/navigation-preferences.md @@ -0,0 +1,45 @@ +# Navigation, profile, and announcements + +People organize their sidebar, move between company resources, maintain their profile, and dismiss product announcements with the correct persistence and identity scope. + +Implementation: [sidebar](../ui/src/components/Sidebar.tsx), [profile](../ui/src/pages/ProfileSettings.tsx), [announcements](../ui/src/components/AnnouncementWell.tsx). + +## Sub-features + +- `navigation`: use current sidebar links, favorites, recents, and available layout controls. +- `preferences`: persist supported ordering/collapse choices for the intended user/company. +- `profile`: edit personal identity and inspect public profile links. +- `announcements`: inspect and dismiss an announcement with its intended instance-wide user scope. +- `fallbacks`: handle missing resources and old deep links without presenting another resource as the target. + +## How to get to it (user POV) + +### `sidebar-deep-links` + +Use starred/recent/sidebar links and direct company-prefixed URLs. + +### `profile-announcements` + +Use Settings → Profile, `/u/:userSlug`, and the announcement well when a publication is available. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use two test users and companies; record the selected UI shell and any client-local preference storage. + +### `sidebar-deep-links` + +Automated: [sidebar](../ui/src/components/Sidebar.test.tsx) and [sidebar preference routes](../server/src/__tests__/sidebar-preferences-routes.test.ts) cover UI/server behavior. + +Manual: Star and reorder supported resources, collapse a section, navigate and reload. Switch company/user and inspect the intended preference scope. Open an unavailable resource and a legacy route; verify clear fallback or correct redirect without leaking another company’s state. + +### `profile-announcements` + +Automated: [profile page](../ui/src/pages/ProfileSettings.test.tsx), [announcement hook](../ui/src/hooks/useAnnouncement.test.tsx), and [announcement routes](../server/src/__tests__/announcements-routes.test.ts) cover separate UI/persistence layers. + +Manual: Change a disposable user’s display details, reload, and inspect an attributed profile link. Dismiss a fixture announcement, switch companies, and verify it remains dismissed for that user; check another user still has their own dismissal state. + +## Gotchas + +- Announcement dismissals are intentionally instance-wide user preferences, unlike company-scoped domain data. +- Legacy/production shells need their own navigation check when affected. +- A route redirect is part of the journey; do not document a removed screen as if it still renders. diff --git a/feature-map/onboarding.md b/feature-map/onboarding.md new file mode 100644 index 0000000000..4a47e0a9fd --- /dev/null +++ b/feature-map/onboarding.md @@ -0,0 +1,45 @@ +# Onboarding and first work + +An operator can start an instance, create a company, configure its first agent, and reach a real first task. Instance setup and company onboarding are separate steps. + +Implementation: [web wizard](../ui/src/components/OnboardingWizard.tsx), [CLI setup](../cli/src/commands/onboard.ts). + +## Sub-features + +- `instance`: choose local instance configuration and start the server. +- `company`: create a company with its identity and initial agent. +- `adapter`: choose an available harness, model, account, and execution environment. +- `first-task`: leave setup with a task whose execution or missing prerequisite is visible. +- `resume`: return to an incomplete wizard without losing saved state. + +## How to get to it (user POV) + +### `web-wizard` + +Open `/onboarding` or the onboarding launcher in an empty company context. + +### `cli-setup` + +Use `paperclipai onboard --help` or `paperclipai run --help` to select an isolated data directory before setup. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a new isolated data directory and record deployment mode. A configured adapter may still need account authentication. + +### `web-wizard` + +Automated: [wizard tests](../ui/src/components/OnboardingWizard.test.tsx) cover mocked wizard behavior; [onboarding browser suite](../tests/e2e/onboarding.spec.ts) supplies its own test environment. Live model execution is a separate check. + +Manual: Create a disposable company and first agent. Save and leave midway, reopen, and finish. Follow the created first task, verify the selected agent and company, and inspect its output or explicit authentication gate. Reload both company and task. + +### `cli-setup` + +Automated: [CLI onboarding](../cli/src/__tests__/onboard.test.ts) and [isolated test drive](../cli/src/__tests__/test-drive.test.ts) cover their setup contracts; they do not establish that every model account is usable. + +Manual: Run setup in the disposable directory, open the URL it prints, and confirm the instance identity and company. Restart using that same configuration and verify saved state. Exercise an unavailable adapter and confirm an actionable error. + +## Gotchas + +- An empty company, an empty instance, and an unauthenticated browser are different states. +- Creating an agent does not prove its provider login or first execution succeeded. +- See [access](./access.md) for authenticated deployments and [agents](./agents.md) for later hiring. diff --git a/feature-map/pipelines.md b/feature-map/pipelines.md new file mode 100644 index 0000000000..ac0bd5d48b --- /dev/null +++ b/feature-map/pipelines.md @@ -0,0 +1,44 @@ +# Pipelines, review queues, and learnings + +With the pipeline experiment enabled, people move structured work through configured stages, inspect stage automation, review outputs, and retain learning records. + +Implementation: [pipeline board and item hosts](../ui/src/pages/Pipelines.tsx), [pipeline configuration](../ui/src/pages/PipelineSettings.tsx). + +## Sub-features + +- `stages`: configure stage order, behavior, and automation assignees. +- `items`: create or inspect items, grouping, stage state, and work references. +- `review`: act on the pipeline review queue and inspect the resulting transition. +- `learnings`: inspect the separate learning view and its source context. + +## How to get to it (user POV) + +### `pipeline-board` + +Open `/pipelines`, `/pipelines/:pipelineId`, and its Settings route. + +### `item-review-learning` + +Open `/pipelines/:pipelineId/items/:caseId`, `/review-queue`, or `/learnings`. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable pipelines and prepare a disposable pipeline with at least two stages and a fixture agent. Record any associated case type. + +### `pipeline-board` + +Automated: [pipeline UI helpers](../ui/src/pages/Pipelines.test.tsx) cover grouping and presentation helpers; [pipeline service](../server/src/__tests__/pipelines-service.test.ts) covers service behavior. + +Manual: Configure stages and a harmless automation, add an item, and inspect its stage. Follow its task/work references and observe the resulting transition. Reload grouping and settings; verify rejected or failed transitions leave the item in an intelligible state. + +### `item-review-learning` + +Automated: [pipeline tutorial browser suite](../tests/e2e/pipelines-tutorial-flow.spec.ts) targets a fixture journey; it does not qualify arbitrary stage automation or learning quality. + +Manual: Open a pending review, inspect its source output and revision, accept or reject, and verify the persisted stage outcome. Follow a learning record to its source context and confirm it belongs to the intended pipeline/company. + +## Gotchas + +- These routes are experimental and may redirect or deny access when disabled. +- ReviewQueue and Learnings are hosted from the pipeline module; a page-file inventory alone misses them. +- Pipeline items and free-standing [cases](./cases.md) share records but have different navigation contexts. diff --git a/feature-map/plugins.md b/feature-map/plugins.md new file mode 100644 index 0000000000..b97f79758e --- /dev/null +++ b/feature-map/plugins.md @@ -0,0 +1,44 @@ +# Plugins and contributed product surfaces + +Operators install and configure plugins, inspect their status and errors, and use their contributed pages, tools, and managed resources within the plugin’s granted capabilities. + +Implementation: [plugin manager](../ui/src/pages/PluginManager.tsx), [plugin settings](../ui/src/pages/PluginSettings.tsx), [plugin page](../ui/src/pages/PluginPage.tsx). + +## Sub-features + +- `installation`: install a trusted test package or local development plugin through supported controls. +- `configuration`: save validated configuration and inspect activation/failure state. +- `contributions`: open plugin routes, settings, tools, and contributed resource tabs. +- `lifecycle`: disable/remove or reload a disposable plugin and observe cleanup. + +## How to get to it (user POV) + +### `plugin-administration` + +Open Settings → Plugins where permitted, or the CLI plugin commands. + +### `plugin-contribution` + +Open `/plugins/:pluginId`, a registered plugin route, or its contributed company/project settings surface. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a known local fixture plugin with bounded capabilities. Record installed version and any declared external dependencies. + +### `plugin-administration` + +Automated: [install authorization](../server/src/__tests__/plugin-install-route-security.test.ts) and [plugin settings](../ui/src/pages/PluginSettings.test.tsx) cover route/UI contracts. + +Manual: Install the fixture, supply its configuration, activate it, and inspect status. Save an invalid configuration and verify a useful error. Disable/remove the disposable plugin and check its tools and managed resources no longer behave as active. + +### `plugin-contribution` + +Automated: [static plugin UI](../server/src/__tests__/plugin-ui-static.test.ts) and [company settings contribution](../ui/src/pages/CompanySettingsPluginPage.test.tsx) cover serving/host behavior. + +Manual: Open the contributed page by navigation and cold deep link, perform a harmless plugin action, and inspect its real outcome. Switch company and test an actor without the capability. Reload with the plugin unavailable and verify the host reports that state. + +## Gotchas + +- Installing a plugin does not authorize every host capability or external action. +- Plugin-provided routes are dynamic; this map covers the host lifecycle, not every third-party feature. +- Environment-driver plugins also need the [environment](./execution-environments.md) checks. diff --git a/feature-map/projects.md b/feature-map/projects.md new file mode 100644 index 0000000000..f2c7f45591 --- /dev/null +++ b/feature-map/projects.md @@ -0,0 +1,44 @@ +# Projects and repositories + +People group tasks into projects, attach goals and repositories, and configure the context and defaults used by project work. + +Implementation: [project list](../ui/src/pages/Projects.tsx), [project detail](../ui/src/pages/ProjectDetail.tsx). + +## Sub-features + +- `lifecycle`: create, name, organize, and archive projects. +- `task-context`: inspect project tasks and create work in the selected project. +- `repositories`: configure repository/workspace sources and verify usable checkouts. +- `defaults`: maintain project instructions, environment, goal, and budget context where offered. + +## How to get to it (user POV) + +### `project-list` + +Open `/projects`, create a project, and enter its overview or Issues view. + +### `repository-settings` + +Open `/projects/:projectId/configuration` and its repository/workspace settings. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable project and a small repository you can access; record checkout and credential ownership. + +### `project-list` + +Automated: [projects UI](../ui/src/pages/Projects.test.tsx) and [project detail](../ui/src/pages/ProjectDetail.test.tsx) are component tests. + +Manual: Create a project, add a task from its view, and reload. Verify the task belongs to that project, list counts are plausible, and links preserve company scope. Archive only the disposable project and inspect list/task behavior. + +### `repository-settings` + +Automated: [repository persistence](../server/src/__tests__/project-repositories-persistence.test.ts) and [repository browser suite](../tests/e2e/project-repositories.spec.ts) cover configured repository workflows. + +Manual: Attach a small repository, save, and create a project task. Inspect the actual checkout/revision used by its run. Test an invalid repository and missing credentials without treating a saved URL as a working workspace. + +## Gotchas + +- Project repository configuration and a provisioned execution workspace are different records. +- See [workspaces](./workspaces.md), [goals](./goals.md), and [budgets](./budgets-costs.md) for those features. +- Do not assume a deleted or archived project should erase its historical task evidence. diff --git a/feature-map/questions-and-approvals.md b/feature-map/questions-and-approvals.md new file mode 100644 index 0000000000..0446cd11c3 --- /dev/null +++ b/feature-map/questions-and-approvals.md @@ -0,0 +1,182 @@ +# Questions and approvals + +People answer an agent's questions and decide requests without losing the task's +history. A question, plan confirmation, governance approval, and app-tool review +can look similar but have different authority and continuation rules. Each +decision must stay attached to its original request and survive reload. + +## Sub-features + +- `questions`: single-choice, multiple-choice, custom, and free-text answers; + paged forms retain answers until submission and show validation failures. +- `dismiss-and-reopen`: unanswered questions remain in the feed after dismissal + or a newer message; reopening restores the form without inventing an answer. +- `confirmations`: accept, decline, or request changes; stale, expired, resolved, + and withdrawn requests cannot be acted on again. +- `plan-review`: inspect the requested plan revision before accepting or asking + for changes. An ordinary planning task can proceed to execution; Agent Chat + hands work to linked tasks and remains a conversation. +- `audience`: the current actor, company, named audience, and review policy govern + resolution. A visible request is not necessarily actionable by that viewer. +- `governance`: pending/history views, approve/reject, revision request, and + resubmission for formal board approvals. +- `tool-review`: approve once, always allow the displayed scope, or decline; + decision status and provider execution outcome remain distinct. +- `continuation`: one recorded response leads to the eligible waiting run or + continuation; repeated clicks, reloads, and competing tabs do not repeat work. + +## How to get to it (user POV) + +### `task-thread` + +Open a task from Tasks, search, a project, or its direct `/issues/:issueId` link. +Answer the question near the composer, or reopen its compact unanswered entry +in the history. Open the plan preview before using the confirmation controls. +Exercise both available task presentations when a change affects shared cards. + +### `agent-chat` + +With Agent Chat enabled, choose Chat and an agent (`/chats/:agentRef`). Answer, +dismiss, or reopen requests in this persistent conversation. A plan handoff here +creates linked work rather than turning the conversation into implementation. + +### `attention` + +Open Decisions (`/decisions`, including a saved queue) or an Inbox attention row. +Expand an actionable interaction inline; follow its task link to inspect context. +With combined Inbox/Tasks enabled, open the corresponding Tasks view instead. + +### `board-approval` + +Open Approvals → Pending or All (`/approvals/pending`, `/approvals/all`). Use the +card action or open `/approvals/:approvalId` for details, comments, revision +request, and resubmission. These are formal approvals, not ordinary questions. + +### `app-review` + +In a task, open **Review request** for an Ask first app call. The same request +appears in Apps → Review (`/apps/review`) and the connection's Review tab. +Use **Approve & run**, **Always allow**, or **Decline** on the surface under test. + +### `skill-test` + +Open Skill Studio, run a skill test that produces an interaction, and inspect +its output card. Skill-test authority is narrower than an ordinary company run. + +### `external-channel` + +In a configured chat channel, answer a published question or confirmation as a +linked sender. Open the associated Paperclip task to inspect the saved outcome. +Channel-specific forms and identity linking require separate provider evidence. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Use separate disposable requests for accept, decline, and revision paths. Record +the actor and request revision. Prepare a waiting question through a test agent; +a component fixture alone cannot prove live response delivery. + +### `task-thread` + +Automated: [interaction cards](../ui/src/components/IssueThreadInteractionCard.test.tsx), +[paged task cards](../ui/src/components/task-chat/TaskChatInteractionCard.test.tsx), +and [task thread](../ui/src/components/TaskChatThread.test.tsx) cover form, +receipt, and dismissal behavior. [Resolution routes](../server/src/__tests__/issue-thread-interaction-routes.test.ts) +and [response delivery](../server/src/__tests__/question-response-delivery.test.ts) +cover authority and durable continuation. Run a targeted set with: + +```sh +pnpm exec vitest run ui/src/components/TaskChatThread.test.tsx server/src/__tests__/question-response-delivery.test.ts +``` + +Manual: answer a multi-page question, go back, edit, then submit. Reload and +confirm the saved answers and one agent continuation. On a fresh request, +dismiss, reload, reopen, and verify the draft. Send a new comment while a +question remains unanswered; it must stay reopenable in history. Try a denied +actor and a request already resolved in another tab; expect an explicit error +or settled receipt, never a second decision. For plans, change the plan revision +before trying the old approval, then review the current revision. + +### `agent-chat` + +Automated: the [task thread suite](../ui/src/components/TaskChatThread.test.tsx) +parameterizes unanswered-question history for conversation mode on and off. +This proves component parity, not a model's plan handoff. + +Manual: repeat dismissal, reload, reopening, and answer submission in Chat. +Approve a plan and inspect the linked execution task and copied plan. The chat +history remains and the conversation does not close as a completed task. + +### `attention` + +Automated: [attention resolver](../ui/src/components/AttentionInteractionResolver.test.tsx) +covers preparing-request refresh without accepting it. The resolution-route +tests above cover the API, not every Decisions or Inbox navigation path. + +Manual: open the same pending request in an attention row and task tab. Resolve +from the row; both surfaces must settle after refresh. Check an empty queue, +loading, permission denial, and a stale request. Repeat with combined Inbox/Tasks +on when that surface is part of the change. Full navigation parity remains a +manual coverage gap. + +### `board-approval` + +Automated: [approval service](../server/src/__tests__/approvals-service.test.ts) +and [idempotency routes](../server/src/__tests__/approval-routes-idempotency.test.ts) +cover persistence and duplicate decisions. They do not drive the approval pages. + +Manual: create disposable formal approval requests; approve one from the list, +reject another from details, and request revision on a third. Resubmit that +third request and verify Pending/All and detail history after reload. Confirm +the recorded decision and its governed effect, not just a success toast. + +### `app-review` + +Automated: [connection reviews](../tests/e2e/connection-reviews.spec.ts) drives +task/queue synchronization, approve, decline, remembered permission, provider +failure, and restart with a scripted process agent and local MCP provider: + +```sh +pnpm exec playwright test -c tests/e2e/connection-reviews.config.ts +``` + +Manual: dismiss and reopen a review, decide it from each entry point using fresh +requests, and inspect the continuation's useful result. Always allow must show +its scope; a future action with changed arguments should obey the saved rule. +Provider failure must remain an approved decision with a failed execution. +Real provider and model checks follow [Task reviews](../doc/connections/TASK-REVIEWS.md#verification-workflows). + +### `skill-test` + +Automated: [read-only interaction contract](../server/src/__tests__/issue-interactions-read-only-contract.test.ts) +checks that fetching interactions does not mutate them. Shared card suites cover +rendering. Neither proves the complete Skill Studio host or its permission matrix. + +Manual: run a skill test with a question, answer through its visible output, +and inspect the test result. Attempt a governed action outside the test's scope; +expect denial. The complete Skill Studio browser journey is not covered by the +referenced unit tests. + +### `external-channel` + +Automated: [chat question forms](../server/src/services/chat-question-forms.test.ts), +[Discord forms](../server/src/services/chat-discord-question-forms.test.ts), +and [interaction publications](../server/src/__tests__/chat-interaction-publications.test.ts) +cover serialization and delivery logic with fixtures. + +Manual: use a disposable provider conversation, answer as the linked sender, +and check the task's persisted result and one continuation. Repeat from an +unlinked or wrong sender and with an expired card. A browser answer does not +qualify Slack, Discord, Telegram, email, or iMessage; record each as not run +unless its actual provider journey was driven. + +## Gotchas + +- Dismissing a question does not answer it. Dismissing an app review neither + grants permission nor resumes the agent. Ordinary comments cannot approve it. +- A resolved request and a successful tool execution are separate facts. +- Do not reuse an accepted request to test rejection; use another request. +- Agent Chat and skill tests have distinct lifecycle/authority rules despite + sharing UI. A watchdog cannot acquire board authority by resolving a card. +- An answer to an old question can queue behind a successor run; it does not + automatically steer that run. See [steering](./steering.md). diff --git a/feature-map/recovery.md b/feature-map/recovery.md new file mode 100644 index 0000000000..3e8a856467 --- /dev/null +++ b/feature-map/recovery.md @@ -0,0 +1,159 @@ +# Recovery + +People can see why a task stopped, who owns the next action, and whether a safe +continuation is available. Recovery retains task context and ownership, honors +pause and budget gates, and distinguishes confirmed failure from an uncertain +external outcome. A retry acknowledgement alone is not proof that work resumed. + +## Sub-features + +- `diagnosis`: a task/run exposes the recorded cause, evidence, owner, attempt + progress, and next action without requiring a raw-log interpretation. +- `bounded-continuation`: supported interrupted work continues with preserved + conversation context and bounded attempts, without blindly replaying tools. +- `source-action`: source-scoped recovery actions remain attached to the task; + independent repair work may have its own issue and return owner. +- `exhaustion`: exhausted or overdue recovery exposes the current gate and + permitted operator action; it does not claim an automatic path is still live. +- `workspace`: workspace divergence/export failures show the applicable repair, + reconcile, or isolated reissue action with evidence and authority checks. +- `unknown-outcome`: uncertain external side effects require inspection and + an explicit decision, not automatic replay. +- `user-continuation`: saved user messages after a stop are reconsidered once + cleanup proves the old execution is stopped; duplicate wakes are avoided. +- `reconnect`: browser stream recovery refreshes persisted state without claiming + the agent itself restarted or that a failed run succeeded. + +## How to get to it (user POV) + +### `source-task` + +Open the affected task from Tasks, Inbox/Blocked, search, or a direct task link. +Inspect the recovery notice/card in its history, expand details if needed, and +use the currently offered action. Follow any linked repair task and return owner. + +### `run-detail` + +Open the agent's run (`/agents/:agentId/runs/:runId`) from activity or the task's +run link. The run page hosts workspace recovery controls only for a failed run +with error code `workspace_validation_failed` and a linked source task that still +has a live `workspace_validation` recovery action. Use the source task for other +stopped-run recovery checks. + +### `operator-retry` + +On an exhausted disposition or workspace recovery card, inspect the current +reason and use the permitted retry/repair/reconcile action. Some repairs require +confirmation; unavailable actions must explain the gate or remain absent. + +### `post-stop-message` + +After a stop, send a new instruction on the same task or inspect one already +saved while cleanup was pending. Observe the waiting message, continuation, and +eventual result rather than manually changing the task status to look complete. + +### `browser-reconnect` + +While viewing an active or recovering task, interrupt the browser connection +and restore it, or suspend and return to the tab. Reload is a separate check. +The current task and run state should reconcile with the server. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Use a disposable instance and reproducible failure. Record source task/run, +adapter/runtime mode, cause, attempt count, current owner, and verified stop +evidence. Do not kill an unrelated server or induce provider side effects just +to get a failure screen. Use the existing deterministic recovery suite first. + +### `source-task` + +Automated: [recovery actions](../server/src/__tests__/issue-recovery-actions.test.ts) +covers source scoping, bounded attempts, operator stops, quota monitoring, +ownership, stale action retirement, and continuation deduplication. +[Recovery cards](../ui/src/components/IssueRecoveryActionCard.test.tsx) cover +diagnosis, retry progress, exhaustion, and action availability. The fixture-backed +[execution-recovery journeys](../tests/e2e/execution-recovery/recovery.spec.ts) +start their own test-drive instances and exercise safe, uncertain, restart, +manager-lineage, and legacy-unknown paths: + +```sh +pnpm exec vitest run server/src/__tests__/issue-recovery-actions.test.ts ui/src/components/IssueRecoveryActionCard.test.tsx +pnpm exec playwright test -c tests/e2e/execution-recovery/playwright.config.ts recovery.spec.ts +``` + +Manual: open a stopped task through each affected navigation path. Confirm its +explanation, original owner, recovery owner if different, and next action. Follow +the supported recovery until the source task produces a useful result or a +specific escalation. Reload and verify that resolved notices retire and attempt +history remains. A fixture pass does not qualify every live harness/provider. + +### `run-detail` + +Automated: [run workspace recovery](../ui/src/components/RunWorkspaceRecoverySurface.test.tsx) +tests this host; it is component coverage, not an end-to-end repair. + +Manual: prepare a failed run with error code `workspace_validation_failed`, a +linked source task, and a live `workspace_validation` recovery action on that +task. Open that exact run and compare its recovery diagnosis to the source task. +Use an eligible action from the run surface, return to the task, +and verify the same action state, persisted outcome, and restored execution. +Repeat with an actor who lacks authority. Verify that the run-page controls are +absent for other error codes or after the source action retires; continue other +recovery checks on the source task. Full run-detail navigation remains a manual +gap. + +### `operator-retry` + +Automated: [disposition notices](../ui/src/components/DispositionRecoveryNotice.test.tsx) +tests pending/error/gated states and waiting for the real retry acknowledgement. +The recovery-card suite covers workspace divergence, reissue, reconcile, and +confirmation. [Workspace export](../ui/src/components/WorkspaceExportRecovery.test.tsx) +covers its separate repair surface. + +Manual: reproduce an exhausted disposable action. Click once, verify pending +state prevents repeated clicks, and inspect a successful continuation or an +explicit rejection. Reload to confirm the durable outcome. For a workspace +repair, record the compared branches/revisions and preserve uncommitted work; +confirm the repair's actual result before resuming. Check the denied and stale +action paths. A changed card label is insufficient proof of workspace integrity. + +### `post-stop-message` + +Automated: [legacy continuation](../server/src/services/recovery/legacy-continuation.test.ts), +[queue routes](../server/src/__tests__/issue-queued-comments-routes.test.ts), and +[native restart recovery](../server/src/services/native-runtime/native-restart-recovery.test.ts) +cover their respective recovery paths. [Legacy failure continuation](../tests/e2e/legacy-failure-continuation.spec.ts) +adds a browser journey; it is not native runtime proof. + +Manual: stop a disposable run, submit a distinguishable follow-up, and observe +any cleanup wait. After confirmed stop, verify the follow-up reaches one eligible +continuation and produces the requested answer. Repeat with a task/ancestor +pause and a budget hold: recovery must not bypass the gate. For unknown external +outcomes, inspect the provider before authorizing another operation. + +### `browser-reconnect` + +Automated: [live-update recovery](../ui/src/context/LiveUpdatesProvider.recovery.test.tsx) +covers client transport recovery with mocks. It does not prove runner restart. + +Manual: go offline during a disposable run, let the server reach a new state, +then reconnect. Confirm task/run/attention data refresh without duplicate +comments or decisions; reload and compare. A healthy websocket proves transport +only. Separately inspect the underlying execution if it is still stuck. + +## Gotchas + +- Legacy and native runtimes have different ownership and restart paths. Record + which one was exercised instead of inferring coverage from the adapter name. +- A process disappearance, elapsed timeout, or successful API retry does not + prove an external tool operation never happened. Unknown outcomes stay unknown + until corroborated; automatic replay can duplicate a side effect. +- Active-run watchdogs, task watchdogs, reviewers, and recovery owners have + different authority. No recovery card grants board permission by itself. +- Idle Agent Chat is not stranded work. An intentional operator stop must not + silently trigger recovery that fights the user's instruction. +- Source inspection and mocked component tests do not qualify real server + restart continuity. Use the appropriate runtime suite for that claim. +- See [execution semantics](../doc/execution-semantics.md) for the governing + recovery contract and [steering](./steering.md) for saved follow-ups. diff --git a/feature-map/routines.md b/feature-map/routines.md new file mode 100644 index 0000000000..c951e17091 --- /dev/null +++ b/feature-map/routines.md @@ -0,0 +1,45 @@ +# Routines, schedules, and triggers + +Operators define repeatable work, configure its inputs and triggers, run it on demand or automatically, and inspect the execution history and resulting tasks. + +Implementation: [routine list](../ui/src/pages/Routines.tsx), [routine detail](../ui/src/pages/RoutineDetail.tsx), [trigger editor](../ui/src/components/routine-triggers/TriggerWizard.tsx). + +## Sub-features + +- `definition`: save the request, agent, project, variables, and execution context. +- `triggers`: configure supported schedules/webhooks or other offered triggers. +- `run-now`: supply inputs and create an observable routine execution. +- `history`: inspect resulting tasks and past outcomes. +- `management`: organize, pause/disable, archive, and distinguish plugin-managed routines. + +## How to get to it (user POV) + +### `routine-editor` + +Open `/routines`, create one, and inspect `/routines/:routineId` and its edit sections. + +### `trigger-history` + +Use the routine’s trigger controls and History/Audit links; lists also offer Run now. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable routine, deterministic test agent, and harmless target. Record schedule timezone and enabled state. + +### `routine-editor` + +Automated: [routine detail helper](../ui/src/pages/RoutineDetail.test.tsx) tests project-selector options, not the rendered editor. [Routine service](../server/src/__tests__/routines-service.test.ts) covers server contracts. Saving and running through the editor remain manual checks. + +Manual: Save an agent, request, project, and a required variable. Reload and inspect the saved definition. Run now with a distinguishable input, follow the resulting task, and verify its output and history entry. Check a missing required input. + +### `trigger-history` + +Automated: [trigger wizard](../ui/src/components/routine-triggers/TriggerWizard.test.tsx) is component coverage; [routine integration](../server/src/__tests__/routines-e2e.test.ts) is server integration rather than a provider-wide browser proof. + +Manual: Set a near-term disposable schedule or authorized test webhook, verify one eligible execution and its source trigger, then disable it and confirm further invocations stop. Inspect a failed execution without losing its history. Remove the temporary trigger after testing. + +## Gotchas + +- Saving a schedule does not prove the scheduler invoked it. +- Timezone, overlap/concurrency rules, and plugin ownership affect behavior. +- A routine run’s task must still respect pause, access, and budget gates. diff --git a/feature-map/runs-adapters.md b/feature-map/runs-adapters.md new file mode 100644 index 0000000000..6b1f1a4f1e --- /dev/null +++ b/feature-map/runs-adapters.md @@ -0,0 +1,44 @@ +# Runs, harnesses, and model accounts + +Operators choose an agent harness and model account, inspect execution history and transcripts, and distinguish a working configuration from a successful task result. + +Implementation: [adapter manager](../ui/src/pages/AdapterManager.tsx), [run host](../ui/src/pages/AgentDetail.tsx), [configuration](../ui/src/components/AgentConfigForm.tsx). + +## Sub-features + +- `availability`: discover installed adapters and available models/accounts. +- `validation`: test an adapter in the selected execution environment and surface authentication failures. +- `runs`: inspect run status, events, output, costs, and the source task. +- `session`: preserve or reset execution context through supported lifecycle actions. + +## How to get to it (user POV) + +### `adapter-setup` + +Open an agent’s model/adapter configuration or the instance adapter settings page when permitted. + +### `run-history` + +Open an agent run at `/agents/:agentId/runs/:runId` from the agent or task activity. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Record adapter, model, account, native/legacy runtime, and environment. Capabilities vary; one harness result is not a matrix-wide qualification. + +### `adapter-setup` + +Automated: [adapter validation routes](../server/src/__tests__/agent-adapter-validation-routes.test.ts) and [model refresh](../server/src/__tests__/adapter-model-refresh-routes.test.ts) cover server behavior. + +Manual: Select the intended adapter and account, refresh model choices, and run its environment test. Save, reload, then execute a small task. Repeat with a deliberately unavailable account and verify the failure remains visible. + +### `run-history` + +Automated: [live run routes](../server/src/__tests__/agent-live-run-routes.test.ts) and [run event sequencing](../server/src/__tests__/heartbeat-run-event-sequencing.test.ts) cover run data contracts. + +Manual: Start a disposable task, open its running transcript, and follow it to a terminal outcome. Reload and compare task/run IDs, status, output, and cost. Verify a failed run remains distinguishable from a completed run with useful output. + +## Gotchas + +- The run log is instance database data; it is not the opt-out first-party Telemetry system or operator-configured OpenTelemetry tracing. +- Authentication/model discovery does not establish tool, image, steering, or resumption capability parity. +- Use [recovery](./recovery.md) for interrupted runs and [execution environments](./execution-environments.md) for local/SSH/sandbox differences. diff --git a/feature-map/secrets.md b/feature-map/secrets.md new file mode 100644 index 0000000000..71dcfb7c26 --- /dev/null +++ b/feature-map/secrets.md @@ -0,0 +1,45 @@ +# Secrets, personal credentials, and proposals + +Operators manage company secret references, users supply their own credentials where required, and reviewers decide proposed secret changes without exposing secret values in ordinary UI evidence. + +Implementation: [secrets page](../ui/src/pages/Secrets.tsx), [personal secrets](../ui/src/pages/secrets/MyUserSecretsTab.tsx), [proposals](../ui/src/pages/secrets/ProposalsTab.tsx). + +## Sub-features + +- `company-secrets`: create/update secret references and control their bindings. +- `user-secrets`: define required user credentials and set a value for the current user. +- `proposals`: review proposed changes with the appropriate authority. +- `vault-import`: import eligible external-vault references through the offered flow. +- `runtime-delivery`: deliver only the secret references authorized for the run/actor. + +## How to get to it (user POV) + +### `secret-management` + +Open company Settings → Secrets and the company/personal secret views. + +### `proposal-import` + +Use the Proposals tab or Import from vault where available. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use dummy credentials in an isolated company, two user identities, and a test agent. Never print actual secret values in evidence. + +### `secret-management` + +Automated: [secrets service](../server/src/__tests__/secrets-service.test.ts) and [user ownership](../server/src/__tests__/secrets-service-user-secret-owner-scoped.test.ts) cover storage/access contracts. + +Manual: Create a dummy secret reference, bind it through a supported field, and verify its use in a harmless run without logging the value. Update it and reload. Switch users and confirm personal values remain owner-scoped and missing credentials produce an actionable state. + +### `proposal-import` + +Automated: [proposal routes](../server/src/__tests__/secret-proposals-routes.test.ts) and [vault import dialog](../ui/src/pages/secrets/ImportFromVaultDialog.test.tsx) cover route/UI layers; a real vault integration needs live validation. + +Manual: Review a dummy proposal, reject one and accept another, then inspect the resulting reference state and history. Preview a vault import, select only disposable entries, and verify imported references and denied-access errors after reload. + +## Gotchas + +- A displayed reference name is not proof that the actor can resolve its value at runtime. +- Company secrets and user-specific values have different ownership. +- Redaction needs verification in errors, activity, and run logs as well as the form. diff --git a/feature-map/skills.md b/feature-map/skills.md new file mode 100644 index 0000000000..c8565f2b2a --- /dev/null +++ b/feature-map/skills.md @@ -0,0 +1,55 @@ +# Skill discovery, authoring, and policy + +Operators discover or import skills, author and test them in Skill Studio, manage source versions, and decide which agents may use them. + +Implementation: [skill library](../ui/src/pages/CompanySkills.tsx), [studio](../ui/src/pages/SkillStudio.tsx), [sources](../ui/src/pages/SkillSources.tsx), [agent skills](../ui/src/pages/agent-skills/AgentSkillsTab.tsx). + +## Sub-features + +- `discovery`: browse installed/discover views and import a skill or source. +- `authoring`: create/edit a writable skill or fork a read-only source. +- `versions`: inspect revisions, source updates, diffs, and selected releases. +- `policy`: apply agent enablement and skill usage policy independently from installation. +- `testing`: run a bounded Studio test and inspect its result and interactions. + +## How to get to it (user POV) + +### `library-sources` + +Open `/skills`, Discover, or `/skills/sources`; project import is available through the library’s import flow. + +### `studio` + +Use New skill or `/skills/studio/:skillId`; writable copies and source-backed originals differ. + +### `agent-enablement` + +Use skill policy/agent selection in the library or an agent’s Skills tab. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a harmless skill fixture and test agent with skill support. Record whether the source is local, bundled, project, or remote. + +### `library-sources` + +Automated: [library UI](../ui/src/pages/CompanySkills.test.tsx) and [source service](../server/src/__tests__/skill-sources.test.ts) cover their respective layers. + +Manual: Install a fixture, locate it in Installed, inspect its source and files, and reload. Import a source update and compare versions before accepting it. Check a source requiring unavailable repository access and verify the failure is explicit. + +### `studio` + +Automated: [Studio](../ui/src/pages/SkillStudio.test.tsx) covers UI behavior; model-driven skill testing remains a separate manual result. + +Manual: Create or fork a disposable skill, edit instructions, save, and reload. Run a bounded test, respond to any question, and inspect the output. Verify the saved version matches what was tested and failed saves retain the draft. + +### `agent-enablement` + +Automated: [policy service](../server/src/__tests__/company-skill-policy-service.test.ts) and [agent skills state](../ui/src/pages/agent-skills/AgentSkillsTab.test.ts) cover policy/state contracts. + +Manual: Enable the skill for one test agent and leave another without access. Run a small task on each, inspect the available skill context, then change the release/policy and verify the next eligible run uses the intended version. + +## Gotchas + +- Installed does not mean enabled for every agent. +- Read-only GitHub/bundled skills may require a writable copy rather than an in-place edit. +- Studio interaction coverage is in [questions and approvals](./questions-and-approvals.md); a passing mock is not a live harness test. diff --git a/feature-map/status-cards.md b/feature-map/status-cards.md new file mode 100644 index 0000000000..ec0b2b240c --- /dev/null +++ b/feature-map/status-cards.md @@ -0,0 +1,44 @@ +# Status cards and refreshed summaries + +The status-card experiment turns a saved prompt/query into an inspectable summary of watched work, with refresh history, configuration, and archive/restore controls. + +Implementation: [status page](../ui/src/pages/StatusCards/index.tsx), [card detail](../ui/src/pages/StatusCards/StatusCardDetailDrawer.tsx). + +## Sub-features + +- `authoring`: create a card and configure its prompt, refresh behavior, and summarizer. +- `generation`: observe compilation/refresh tasks and their successful or failed output. +- `inspection`: read the summary, watched work, history, and available query debugging. +- `archive`: archive and restore without confusing a stale summary with a current result. + +## How to get to it (user POV) + +### `status-board` + +Open `/status`, choose New card, and follow a card at `/status/:cardId`. + +### `card-settings` + +Open the card detail’s settings/query-debug controls and archive/restore actions. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable status cards, use a configured summarizer and disposable tasks, and record whether summary generation is simulated or live. + +### `status-board` + +Automated: [status-card service/routes](../server/src/__tests__/status-cards.test.ts) covers gated lifecycle and generation contracts; [tile UI](../ui/src/pages/StatusCards/StatusCardTile.test.tsx) covers rendering. + +Manual: Create a bounded card, inspect its generation task, and wait for a saved summary. Change a watched task, refresh, and verify both the summary and update history reflect the source change. Exercise a failed generation and confirm it is not presented as fresh. + +### `card-settings` + +Automated: [settings form](../ui/src/pages/StatusCards/StatusCardSettingsForm.test.tsx) and [update engine](../server/src/__tests__/status-card-update-engine.test.ts) cover separate UI/engine layers. + +Manual: Change a disposable card’s prompt or summarizer, run the offered rebuild/refresh, and inspect the resulting query and summary. Archive and restore it, reload its direct link, and check the actual new generation outcome. + +## Gotchas + +- Summary quality needs comparison against source work, not just a successful task status. +- A saved old summary must not be counted as a successful new refresh. +- Experimental visibility, author permissions, and agent ownership constrain mutation. diff --git a/feature-map/steering.md b/feature-map/steering.md new file mode 100644 index 0000000000..02d1cfa91b --- /dev/null +++ b/feature-map/steering.md @@ -0,0 +1,135 @@ +# Steering and queued messages + +People can send follow-ups while an agent works, manage saved messages that have +not been consumed, and choose when to steer or interrupt. Stopping a run and +pausing a task are separate controls. The UI must preserve the message, explain +why it is waiting, and only claim delivery after an authoritative acknowledgement. + +## Sub-features + +- `queue`: a busy task saves follow-ups and shows their order and wait reason. +- `edit-order-discard`: edit full Markdown, reorder, and discard eligible saved + messages; ownership, revision conflicts, and already-dispatched states matter. +- `steer`: explicit user action sends a queued message into a compatible active + run; unavailable capabilities and provider rejection remain visible. +- `interrupt`: stop the current execution and carry accepted follow-ups into + the eligible continuation without delivering them twice. +- `stop-and-pause`: distinguish stop-in-flight, confirmed stop, task/ancestor + pause, agent pause, and budget hold. Resume must respect the actual gate. +- `drafts`: switching tasks, reloading, and uncertain submission retain the + correct task's draft and attachment receipts. +- `approval-queue`: answered questions and accepted approvals have durable + delivery distinct from editable ordinary comments. + +## How to get to it (user POV) + +### `task-composer` + +Open an assigned task (`/issues/:issueId`). Start work, then send follow-ups while +the agent runs. Use the queued-message controls to edit, reorder, discard, +steer, or interrupt when offered. Use the composer stop/pause control separately. +Both task presentations need attention when changing their shared behavior. + +### `agent-chat` + +Enable Agent Chat and open an agent conversation. Send another message during +a reply and inspect the shared queue. `/new` starts fresh provider context in +the same conversation; it is an ordered boundary, not a normal model prompt. + +### `request-response` + +Answer an earlier question or accept a confirmation while successor work is +active on the task. Inspect the resulting queued response and its available +steer/interrupt control instead of assuming acceptance interrupted execution. + +### `external-channel` + +Send a follow-up from a linked external chat conversation while its task is +running. Use that provider's supported stop/control affordance where available; +inspect the resulting task and run in Paperclip. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). +Use an agent that stays busy long enough to inspect the queue. Record the adapter, +native/legacy mode, and advertised steering capability. Test supported and +unsupported steering without treating an absent capability as a product failure. + +### `task-composer` + +Automated: [queued-message UI](../ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx) +covers order, failure restoration, unavailable steering, stale revisions, and +discard acknowledgements. [Queue routes](../server/src/__tests__/issue-queued-comments-routes.test.ts) +cover durable ordering, authorization, steering acknowledgements, interrupt +recovery, and concurrent dispatch. [Composer](../ui/src/components/task-chat/TaskChatComposer.test.tsx) +covers drafts and uncertain saves. + +```sh +pnpm exec vitest run ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx server/src/__tests__/issue-queued-comments-routes.test.ts +``` + +Automated stop coverage is separate: +[composer stop](../tests/e2e/composer-stop.spec.ts) with +[its configuration](../tests/e2e/playwright-composer-stop.config.ts) and +[ACP stop/continuation](../tests/e2e/acp-stop-continuation.spec.ts). Inspect their +fixture and opt-in prerequisites; skipped native cases are not native proof. + +Manual: queue three distinguishable messages. Edit one, reorder them, reload, +and confirm the saved order. Discard one and confirm it never reaches execution. +Steer one while busy and verify the agent receives that exact message once. +Repeat with a rejected steer and a stale second-tab edit; the UI must preserve +retryable content and explain the conflict. On a fresh run, interrupt with a +queued follow-up and verify confirmed stop followed by one continuation. Pause +the task (and separately its ancestor), preserve a draft, then resume the actual +hold; no hidden send should occur while paused. Check both task presentations +when available; the named component tests do not prove every host integration. + +### `agent-chat` + +Automated: [agent chat sessions](../tests/e2e/agent-chat-sessions.spec.ts) covers +persistent conversation/session behavior with deterministic harness fixtures; +shared composer tests cover the controls. These do not qualify live steering +for every harness. + +Manual: queue two follow-ups while the conversation replies. Reload, edit or +discard one, then let the other run. Verify one reply and unchanged conversation +identity. Send `/new`, then a new prompt; old history remains visible, and the +new turn uses fresh provider context. Verify a useful answer rather than only +the session-divider UI. Repeat the queue operations with a live capable harness +when the change touches delivery. + +### `request-response` + +Automated: [response delivery](../server/src/__tests__/question-response-delivery.test.ts) +proves queueing/coalescing and retry-safe receipts. The queue-route suite tests +separate comment and approval queues, explicit steering, and fresh-session +restrictions on plan approvals. + +Manual: answer an old question during a new run, confirm that it is saved and +queued, and let the active run end. Verify exactly one continuation with the +answer. Repeat an eligible approval with explicit steer; another approval that +requires a fresh session must not offer same-turn delivery. Do not edit an +approval outcome through an ordinary queued-message editor. + +### `external-channel` + +Automated: [chat interaction arbitration](../server/src/services/chat-interaction-arbitration.test.ts) +covers backend arbitration with simulated requests. It is not a provider UI test. + +Manual: send two distinguishable messages through one disposable linked +conversation while busy. Inspect their saved task order, run delivery, and +external reply. Check a duplicate provider event and unauthorized sender. +Provider-specific control semantics and real transport timing remain a manual +gap; do not assume browser queue controls are available in every channel. + +## Gotchas + +- Paperclip's queue is server-owned once saved; it is not Omnigent's unsent + browser draft buffer. A stale edit must not overwrite a consumed message. +- Steer availability depends on the active runtime's capability, not its brand + name. A UI row disappearing optimistically does not prove provider receipt. +- Interrupt, stop, task pause, agent pause, and budget pause have different + effects. A running indicator disappearing does not prove remote work stopped. +- An interrupted run may need verified cleanup before a saved follow-up can + start. Preserve and inspect the wait reason; see [recovery](./recovery.md). +- A healthy idle Agent Chat is waiting for input, not stranded unfinished work. diff --git a/feature-map/task-coordination.md b/feature-map/task-coordination.md new file mode 100644 index 0000000000..9549f7606d --- /dev/null +++ b/feature-map/task-coordination.md @@ -0,0 +1,44 @@ +# Delegation, dependencies, and signoff + +A larger task can coordinate child work, dependencies, reviewers, and execution controls while preserving a clear owner and a traceable completion decision. + +Implementation: [task thread](../ui/src/pages/IssueDetail.tsx), [execution policy](../server/src/services/issues.ts). + +## Sub-features + +- `delegation`: inspect parent/child tasks and the original request behind delegated work. +- `dependencies`: show blocking tasks and wake eligible work when prerequisites resolve. +- `signoff`: apply the configured implementation/review completion policy. +- `tree-control`: pause or resume a task tree without bypassing its gates. + +## How to get to it (user POV) + +### `task-relationships` + +Open the task’s parent/child and dependency links in task detail. + +### `review-and-controls` + +Inspect the task’s execution policy and available review, pause, resume, or stop controls. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare a parent, child, and dependent task plus implementer/reviewer agents. Record execution policy and use a disposable tree. + +### `task-relationships` + +Automated: [dependency wakeups](../server/src/__tests__/issue-dependency-wakeups-routes.test.ts) checks server behavior; visual relationship navigation remains a manual check. + +Manual: Block one task on another, finish the prerequisite, and observe the dependent task’s eligible continuation. Follow parent/child links back to the source request. Verify no duplicate run or false parent completion occurs. + +### `review-and-controls` + +Automated: [execution policy](../server/src/__tests__/issue-execution-policy.test.ts), [tree control](../server/src/__tests__/issue-tree-control-service.test.ts), and [signoff browser suite](../tests/e2e/signoff-policy.spec.ts) target these layers. A linked suite is not a claim that its current run is green. + +Manual: Complete implementation under a review-required policy and confirm it waits for the required review outcome. Exercise rejection and later approval. Pause the parent while child work is pending, then verify resumption respects budget and ownership gates. + +## Gotchas + +- Review completion and task completion are different events under some policies. +- Pausing a tree must not be “tested” by changing statuses until the screen looks quiet. +- Use [questions and approvals](./questions-and-approvals.md) for decision forms and [recovery](./recovery.md) for interrupted work. diff --git a/feature-map/tasks.md b/feature-map/tasks.md new file mode 100644 index 0000000000..a2a7e42bc3 --- /dev/null +++ b/feature-map/tasks.md @@ -0,0 +1,55 @@ +# Task creation and lifecycle + +People create, assign, organize, discuss, and finish tasks. A task retains its owner, project, priority, history, and outputs as it moves through the work lifecycle. + +Implementation: [new task](../ui/src/components/NewIssueDialog.tsx), [task list](../ui/src/pages/Issues.tsx), [task detail](../ui/src/pages/IssueDetail.tsx). + +## Sub-features + +- `create`: capture a title/request with the intended agent, project, and attachments. +- `organize`: filter, sort, group, and navigate tasks without losing the selected context. +- `properties`: change status, priority, assignee, project, and relationships where authorized. +- `discussion`: persist comments, references, and files on the task. +- `completion`: distinguish backlog, active work, review, blocked, done, and cancelled outcomes. + +## How to get to it (user POV) + +### `new-task` + +Use New task from the global launcher, task list, or project task view. + +### `task-list` + +Open `/issues` or a project’s Issues view; legacy `/tasks` links redirect. + +### `task-detail` + +Open `/issues/:issueId` from a row, mention, or direct link. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an invokable test agent and a project; record whether a real or simulated harness handles the request. + +### `new-task` + +Automated: [creation dialog](../ui/src/components/NewIssueDialog.test.tsx) is component coverage with mocked requests. + +Manual: Create a uniquely named task with an agent and project. Open its resulting link and reload. Confirm text, attachment, assignment, and project persisted, and verify execution started only when the task and agent are eligible. + +### `task-list` + +Automated: [task-list helpers](../ui/src/pages/Issues.test.tsx) cover search URLs, pagination, deduplication, and presentation constants. The suite does not render the page or exercise filtering, grouping, opening a task, and returning; those interactions still need a running-browser check. + +Manual: Filter by status and assignee, change grouping/sort, open a task, and return. Verify the correct records and selected view survive navigation and reload. Include an empty result and a failed request. + +### `task-detail` + +Automated: [task detail](../ui/src/pages/IssueDetail.test.tsx) covers component behavior, while [issue service](../server/src/__tests__/issues-service.test.ts) covers server contracts. + +Manual: Add a comment and edit authorized properties. Reload and verify history, actor, and values. Complete a disposable task and inspect its deliverable; reopen through an available action and confirm the next execution follows current policy. + +## Gotchas + +- A status label alone does not prove that the requested work exists. +- Single-assignee and atomic checkout rules still apply when tasks are created from different hosts. +- Use [coordination](./task-coordination.md), [steering](./steering.md), and [documents](./documents-artifacts.md) for their distinct workflows. diff --git a/feature-map/teams.md b/feature-map/teams.md new file mode 100644 index 0000000000..a1370c75eb --- /dev/null +++ b/feature-map/teams.md @@ -0,0 +1,44 @@ +# Team packages and catalog installation + +Operators can preview and install a team package with its agents and related resources. Catalog/package installation is available through the CLI/API; the repository also contains catalog UI components that are not a current top-level route. + +Implementation: [CLI teams](../cli/src/commands/client/teams.ts), [catalog components](../ui/src/pages/TeamCatalog.tsx). + +## Sub-features + +- `discovery`: list available team catalog entries and inspect package contents. +- `preview`: inspect proposed resources and installation options before mutation. +- `install`: install into the intended company and inspect resulting agents and resources. +- `configuration`: supply required adapters/accounts and verify installed agents can work. + +## How to get to it (user POV) + +### `catalog-cli` + +Use the CLI teams catalog/preview/install operations in the selected context. + +### `catalog-components` + +The catalog components and install wizard are available in developer tests/design surfaces. No top-level `/teams` route is registered; use the CLI/API for the current installation workflow. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable company, a small catalog entry, and explicit company context. Check `paperclipai teams --help` for current subcommands. + +### `catalog-cli` + +Automated: [team commands](../cli/src/__tests__/teams.test.ts) and [catalog service](../server/src/__tests__/teams-catalog-service.test.ts) cover command and service contracts. + +Manual: List an entry, preview its proposed agents and resources, install it into the disposable company, and inspect each created resource. Run a small task after configuring accounts. Retry preview/install as supported and inspect duplicate/conflict behavior. + +### `catalog-components` + +Automated: [catalog component](../ui/src/pages/TeamCatalog.test.tsx) and [install hook](../ui/src/pages/useInstallTeamCatalogEntry.test.tsx) cover preview and install UI logic with mocks. + +Manual: When changing or mounting these components, exercise preview, manager/source options, cancellation, failed install, and success. Follow created agents in the actual company. Record whether this was a component harness or a real product entry point. + +## Gotchas + +- Source files do not prove that a screen is reachable in the current app. +- Installed team configuration may still need credentials or environment setup. +- Company package import is a separate [portability](./companies.md) workflow. diff --git a/feature-map/workspaces.md b/feature-map/workspaces.md new file mode 100644 index 0000000000..d7b24fca7c --- /dev/null +++ b/feature-map/workspaces.md @@ -0,0 +1,45 @@ +# Execution workspaces, services, and files + +A project or task can use a managed workspace with inspectable files, Git state, services, and runtime controls. Operators can follow its lifecycle through provisioning, use, and closure. + +Implementation: [workspace list](../ui/src/pages/Workspaces.tsx), [execution workspace](../ui/src/pages/ExecutionWorkspaceDetail.tsx), [project workspace](../ui/src/pages/ProjectWorkspaceDetail.tsx). + +## Sub-features + +- `binding`: identify the workspace actually bound to a task/run. +- `provisioning`: observe readiness and failures instead of inferring readiness from a created record. +- `services`: start/stop supported services and inspect logs and exposed endpoints. +- `files-git`: inspect workspace files and repository state used for the deliverable. +- `closure`: close/reopen or reconcile through permitted controls without losing work. + +## How to get to it (user POV) + +### `task-workspace` + +Use the task’s workspace card or Files panel and follow its workspace link. + +### `workspace-management` + +Open `/workspaces`, a project workspace, or `/execution-workspaces/:workspaceId` and its Services/Configuration/Runtime logs views. + +## Driving it + +Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable isolated-workspace UI where needed. Use an owned disposable repository and environment; record workspace ID and branch. + +### `task-workspace` + +Automated: [workspace card](../ui/src/components/IssueWorkspaceCard.test.tsx) and [task binding](../server/src/__tests__/issue-runtime-workspace-binding.test.ts) cover host/binding contracts. + +Manual: Run a task that writes a small file, follow the bound workspace, and inspect the file and revision. Reload and verify links still refer to the same workspace, not another checkout or a stale run. + +### `workspace-management` + +Automated: [workspace detail](../ui/src/pages/ExecutionWorkspaceDetail.test.tsx) and [runtime service](../server/src/__tests__/workspace-runtime.test.ts) cover UI and runtime contracts separately. + +Manual: Provision a disposable workspace, wait for readiness, start an available service, open its endpoint, and inspect a real response and logs. Stop it and verify it actually stops. Exercise close/reopen only after inspecting pending changes and active runs. + +## Gotchas + +- A link or running badge does not prove the service is reachable from the user’s browser. +- Workspace IDs, runtime leases, and repository paths are not interchangeable. +- Use [recovery](./recovery.md) for divergence/export repair and [CLI operations](./cli-operations.md) for local worktree tooling. diff --git a/packages/adapter-utils/src/remote-managed-runtime.ts b/packages/adapter-utils/src/remote-managed-runtime.ts index cb47f3caed..8c5d2581e1 100644 --- a/packages/adapter-utils/src/remote-managed-runtime.ts +++ b/packages/adapter-utils/src/remote-managed-runtime.ts @@ -1,5 +1,5 @@ import path from "node:path"; -import { GIT_ARCHIVE_EXCLUDES } from "./git-workspace-sync.js"; +import { GIT_ARCHIVE_EXCLUDES, PROJECT_REPOSITORIES_DIR } from "./git-workspace-sync.js"; import { type SshRemoteExecutionSpec, prepareWorkspaceForSshExecution, @@ -13,7 +13,7 @@ import { type SandboxAdditionalSource, type SandboxManagedRuntimeAssetRestoreContext, } from "./sandbox-managed-runtime.js"; -import { captureDirectorySnapshot } from "./workspace-restore-merge.js"; +import { captureDirectorySnapshot, type DirectorySnapshot } from "./workspace-restore-merge.js"; import type { RuntimeProgressSink } from "./runtime-progress.js"; // The fixed heavy-directory excludes every referenced project drops, @@ -145,13 +145,27 @@ export async function prepareRemoteManagedRuntime(input: { workspaceExclude: input.workspaceExclude, }) : null; + const projectRepositories = preparedWorkspace?.repositories ?? []; const baselineSnapshot = preparedWorkspace ? await captureDirectorySnapshot(input.workspaceLocalDir, { exclude: preparedWorkspace.gitBacked - ? [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"] + ? [ + ...GIT_ARCHIVE_EXCLUDES, + ".paperclip-runtime", + ...(projectRepositories.length > 0 ? [PROJECT_REPOSITORIES_DIR] : []), + ] : [".paperclip-runtime", ...(input.workspaceFileMode === "all" ? input.workspaceExclude ?? [] : [])], }) : null; + const repositoryBaselines: Array<{ path: string; baselineSnapshot: DirectorySnapshot }> = []; + for (const repository of projectRepositories) { + repositoryBaselines.push({ + path: repository, + baselineSnapshot: await captureDirectorySnapshot(path.join(input.workspaceLocalDir, repository), { + exclude: [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"], + }), + }); + } const assetDirs: Record = {}; try { @@ -177,6 +191,7 @@ export async function prepareRemoteManagedRuntime(input: { baselineSnapshot, restoreGitHistory: preparedWorkspace.gitBacked, onProgress: input.onProgress, + repositories: repositoryBaselines, }); } throw error; @@ -244,6 +259,7 @@ export async function prepareRemoteManagedRuntime(input: { baselineSnapshot, restoreGitHistory: preparedWorkspace.gitBacked, onProgress, + repositories: repositoryBaselines, }); } for (const asset of input.assets ?? []) { diff --git a/packages/adapter-utils/src/ssh-fixture.test.ts b/packages/adapter-utils/src/ssh-fixture.test.ts index 83c4ad7e6d..39d3c52515 100644 --- a/packages/adapter-utils/src/ssh-fixture.test.ts +++ b/packages/adapter-utils/src/ssh-fixture.test.ts @@ -21,6 +21,16 @@ import { import { prepareRemoteManagedRuntime } from "./remote-managed-runtime.js"; const SSH_FIXTURE_TEST_TIMEOUT_MS = 30_000; +const UNREACHABLE_SSH_SPEC = { + host: "127.0.0.1", + port: 1, + username: "nobody", + remoteWorkspacePath: "/nonexistent", + remoteCwd: "/nonexistent", + privateKey: null, + knownHosts: null, + strictHostKeyChecking: false, +} as const; let sshEnvLabUnsupportedReason: string | null = null; // One entry per fixture root directory, registered at creation time so @@ -850,6 +860,205 @@ describe("ssh env-lab fixture", () => { await expect(readFile(path.join(localRepo, "run-b.txt"), "utf8")).resolves.toBe("from run b\n"); }, SSH_FIXTURE_TEST_TIMEOUT_MS); + it("round-trips project repositories nested in .paperclip-repositories across consecutive SSH runs", async () => { + const rootDir = await createFixtureRootDir(); + const statePath = path.join(rootDir, "state.json"); + const localRepo = path.join(rootDir, "local-workspace"); + const nestedRelative = ".paperclip-repositories/frontend-0123456789ab"; + const nestedRepo = path.join(localRepo, nestedRelative); + + await mkdir(localRepo, { recursive: true }); + await git(localRepo, ["init"]); + await git(localRepo, ["checkout", "-b", "main"]); + await git(localRepo, ["config", "user.name", "Paperclip Test"]); + await git(localRepo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(localRepo, "backend.txt"), "backend base\n", "utf8"); + await git(localRepo, ["add", "backend.txt"]); + await git(localRepo, ["commit", "-m", "backend initial"]); + await writeFile(path.join(localRepo, ".git", "info", "exclude"), "\n/.paperclip-repositories/\n", { flag: "a" }); + + await mkdir(nestedRepo, { recursive: true }); + await git(nestedRepo, ["init"]); + await git(nestedRepo, ["checkout", "-b", "main"]); + await git(nestedRepo, ["config", "user.name", "Paperclip Test"]); + await git(nestedRepo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(nestedRepo, "frontend.txt"), "frontend base\n", "utf8"); + await writeFile(path.join(nestedRepo, "obsolete.txt"), "tracked\n", "utf8"); + await git(nestedRepo, ["add", "frontend.txt", "obsolete.txt"]); + await git(nestedRepo, ["commit", "-m", "frontend initial"]); + await writeFile(path.join(nestedRepo, "frontend.txt"), "frontend dirty local\n", "utf8"); + await rm(path.join(nestedRepo, "obsolete.txt")); + + const started = await startSshEnvLabFixtureOrSkip(statePath, "SSH nested project repositories test"); + if (!started) return; + const config = await buildSshEnvLabFixtureConfig(started); + const spec = { + ...config, + remoteCwd: started.workspaceDir, + } as const; + + const first = await prepareRemoteManagedRuntime({ + spec, + runId: "run-1", + adapterKey: "test-adapter", + workspaceLocalDir: localRepo, + }); + const remoteNested = path.posix.join(first.workspaceRemoteDir, nestedRelative); + + const remoteNestedStatus = await runSshCommand( + config, + `cd ${JSON.stringify(remoteNested)} && git log -1 --pretty=%s && git status --short`, + ); + expect(remoteNestedStatus.stdout).toContain("frontend initial"); + expect(remoteNestedStatus.stdout).toContain("M frontend.txt"); + expect(remoteNestedStatus.stdout).toContain("D obsolete.txt"); + const remoteAnchorStatus = await runSshCommand( + config, + `cd ${JSON.stringify(first.workspaceRemoteDir)} && git status --short --untracked-files=all`, + ); + expect(remoteAnchorStatus.stdout).not.toContain(".paperclip-repositories"); + + await runSshCommand( + config, + [ + `cd ${JSON.stringify(remoteNested)}`, + `git config user.name "Paperclip SSH"`, + `git config user.email "ssh@paperclip.dev"`, + `git add frontend.txt`, + `git commit -m "remote frontend update" >/dev/null`, + `printf "frontend remote dirty\\n" > frontend.txt`, + `cd ${JSON.stringify(first.workspaceRemoteDir)}`, + `git config user.name "Paperclip SSH"`, + `git config user.email "ssh@paperclip.dev"`, + `printf "backend remote\\n" > backend.txt`, + `git add backend.txt`, + `git commit -m "remote backend update" >/dev/null`, + ].join(" && "), + { timeoutMs: 30_000, maxBuffer: 256 * 1024 }, + ); + + await first.restoreWorkspace(); + + await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined(); + expect(await git(nestedRepo, ["log", "-1", "--pretty=%s"])).toBe("remote frontend update"); + await expect(readFile(path.join(nestedRepo, "frontend.txt"), "utf8")).resolves.toBe("frontend remote dirty\n"); + expect(await git(localRepo, ["log", "-1", "--pretty=%s"])).toBe("remote backend update"); + expect(await git(localRepo, ["status", "--short", "--untracked-files=all"])).not.toContain(".paperclip-repositories"); + + const second = await prepareRemoteManagedRuntime({ + spec, + runId: "run-2", + adapterKey: "test-adapter", + workspaceLocalDir: localRepo, + }); + const secondNestedLog = await runSshCommand( + config, + `cd ${JSON.stringify(path.posix.join(second.workspaceRemoteDir, nestedRelative))} && git log -1 --pretty=%s`, + ); + expect(secondNestedLog.stdout.trim()).toBe("remote frontend update"); + await second.restoreWorkspace(); + await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined(); + await expect(readFile(path.join(nestedRepo, "frontend.txt"), "utf8")).resolves.toBe("frontend remote dirty\n"); + }, SSH_FIXTURE_TEST_TIMEOUT_MS * 2); + + it("restores project repositories through the direct SSH restore path", async () => { + const rootDir = await createFixtureRootDir(); + const statePath = path.join(rootDir, "state.json"); + const localRepo = path.join(rootDir, "local-workspace"); + const nestedRelative = ".paperclip-repositories/frontend-0123456789ab"; + const nestedRepo = path.join(localRepo, nestedRelative); + + for (const repo of [localRepo, nestedRepo]) { + await mkdir(repo, { recursive: true }); + await git(repo, ["init"]); + await git(repo, ["checkout", "-b", "main"]); + await git(repo, ["config", "user.name", "Paperclip Test"]); + await git(repo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(repo, "tracked.txt"), "base\n", "utf8"); + await git(repo, ["add", "tracked.txt"]); + await git(repo, ["commit", "-m", "initial"]); + } + await writeFile(path.join(localRepo, ".git", "info", "exclude"), "\n/.paperclip-repositories/\n", { flag: "a" }); + + const started = await startSshEnvLabFixtureOrSkip(statePath, "SSH direct restore with project repositories test"); + if (!started) return; + const config = await buildSshEnvLabFixtureConfig(started); + const spec = { + ...config, + remoteCwd: started.workspaceDir, + } as const; + + const prepared = await prepareWorkspaceForSshExecution({ + spec, + localDir: localRepo, + remoteDir: started.workspaceDir, + }); + expect(prepared).toEqual({ gitBacked: true, repositories: [nestedRelative] }); + + await runSshCommand( + config, + [ + `cd ${JSON.stringify(path.posix.join(started.workspaceDir, nestedRelative))}`, + `git config user.name "Paperclip SSH"`, + `git config user.email "ssh@paperclip.dev"`, + `printf "remote\\n" > tracked.txt`, + `git commit -am "remote nested update" >/dev/null`, + ].join(" && "), + { timeoutMs: 30_000, maxBuffer: 256 * 1024 }, + ); + + await restoreWorkspaceFromSshExecution({ + spec, + localDir: localRepo, + remoteDir: started.workspaceDir, + repositories: (prepared.repositories ?? []).map((repository) => ({ path: repository })), + }); + + await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined(); + expect(await git(nestedRepo, ["log", "-1", "--pretty=%s"])).toBe("remote nested update"); + await expect(readFile(path.join(nestedRepo, "tracked.txt"), "utf8")).resolves.toBe("remote\n"); + expect(await git(localRepo, ["status", "--short", "--untracked-files=all"])).toBe(""); + }, SSH_FIXTURE_TEST_TIMEOUT_MS * 2); + + it("fails closed when .paperclip-repositories holds something other than a git checkout", async () => { + const rootDir = await createFixtureRootDir(); + const localRepo = path.join(rootDir, "local-workspace"); + await mkdir(path.join(localRepo, ".paperclip-repositories", "not-a-repo"), { recursive: true }); + await git(localRepo, ["init"]); + await git(localRepo, ["checkout", "-b", "main"]); + await git(localRepo, ["config", "user.name", "Paperclip Test"]); + await git(localRepo, ["config", "user.email", "test@paperclip.dev"]); + await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8"); + await git(localRepo, ["add", "tracked.txt"]); + await git(localRepo, ["commit", "-m", "initial"]); + + await expect(prepareWorkspaceForSshExecution({ spec: UNREACHABLE_SSH_SPEC, localDir: localRepo, remoteDir: "/nonexistent" })) + .rejects.toThrow("Project repository is not a Git checkout: .paperclip-repositories/not-a-repo"); + }, SSH_FIXTURE_TEST_TIMEOUT_MS); + + it("rejects inconsistent project repository restores before any SSH connection", async () => { + const localDir = await createFixtureRootDir(); + const repositoryBaseline = { exclude: [".git", ".git/*", ".paperclip-runtime"], entries: new Map() }; + const restore = (input: Partial[0]>) => + restoreWorkspaceFromSshExecution({ + spec: UNREACHABLE_SSH_SPEC, + localDir, + remoteDir: "/nonexistent", + baselineSnapshot: { exclude: [".git", ".git/*", ".paperclip-runtime", ".paperclip-repositories"], entries: new Map() }, + restoreGitHistory: true, + ...input, + }); + + await expect(restore({ repositories: [{ path: "../outside", baselineSnapshot: repositoryBaseline }] })) + .rejects.toThrow("Invalid project repository path: ../outside"); + await expect(restore({ repositories: [{ path: ".paperclip-repositories/frontend-0123456789ab" }] })) + .rejects.toThrow("Project repository has no workspace baseline: .paperclip-repositories/frontend-0123456789ab"); + await expect(restore({ + baselineSnapshot: { exclude: [".git", ".git/*", ".paperclip-runtime"], entries: new Map() }, + repositories: [{ path: ".paperclip-repositories/frontend-0123456789ab", baselineSnapshot: repositoryBaseline }], + })).rejects.toThrow("Workspace baseline must exclude .paperclip-repositories"); + }, SSH_FIXTURE_TEST_TIMEOUT_MS); + it("preserves nested per-run files across sequential SSH restores with stale baselines", async () => { const rootDir = await createFixtureRootDir(); const statePath = path.join(rootDir, "state.json"); diff --git a/packages/adapter-utils/src/ssh.ts b/packages/adapter-utils/src/ssh.ts index 89fd456c5d..68e61fc639 100644 --- a/packages/adapter-utils/src/ssh.ts +++ b/packages/adapter-utils/src/ssh.ts @@ -9,6 +9,7 @@ import type { CommandManagedRuntimeRunner } from "./command-managed-runtime.js"; import { createUnrelatedHistoryGraftCommit, GIT_SYNC_COMMIT_IDENTITY_ARGS, + PROJECT_REPOSITORIES_DIR, readSanitizedOriginRemoteUrl, } from "./git-workspace-sync.js"; import type { RunProcessResult } from "./server-utils.js"; @@ -1071,6 +1072,91 @@ async function removeDeletedPathsOnSsh(input: { }); } +const PROJECT_REPOSITORY_NAME_PATTERN = /^[a-zA-Z0-9_-]+$/; + +async function isLocalGitRepositoryRoot(localDir: string): Promise { + try { + const toplevel = await runLocalGit(localDir, ["rev-parse", "--show-toplevel"], { + timeout: 10_000, + maxBuffer: 16 * 1024, + }); + const [directory, repository] = await Promise.all([ + fs.realpath(localDir), + fs.realpath(toplevel.stdout.trim()), + ]); + return directory === repository; + } catch { + return false; + } +} + +async function listLocalProjectRepositories(localDir: string): Promise { + const root = path.join(localDir, PROJECT_REPOSITORIES_DIR); + const rootStat = await fs.lstat(root).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (!rootStat) return []; + if (!rootStat.isDirectory() || rootStat.isSymbolicLink()) { + throw new Error("Invalid project repositories directory"); + } + const repositories: string[] = []; + for (const entry of await fs.readdir(root, { withFileTypes: true })) { + if (!entry.isDirectory() || !PROJECT_REPOSITORY_NAME_PATTERN.test(entry.name)) { + throw new Error("Invalid project repository directory"); + } + const relative = `${PROJECT_REPOSITORIES_DIR}/${entry.name}`; + if (!(await isLocalGitRepositoryRoot(path.join(localDir, relative)))) { + throw new Error(`Project repository is not a Git checkout: ${relative}`); + } + repositories.push(relative); + } + return repositories.sort(); +} + +async function transportGitWorkspaceToSsh(input: { + spec: SshRemoteExecutionSpec; + localDir: string; + remoteDir: string; + snapshot: LocalGitWorkspaceSnapshot; + exclude?: string[]; + onProgress?: RuntimeProgressSink; + progressLabel: string; +}): Promise { + await importGitWorkspaceToSsh({ + spec: input.spec, + localDir: input.localDir, + remoteDir: input.remoteDir, + snapshot: input.snapshot, + onProgress: input.onProgress, + }); + await syncDirectoryToSsh({ + spec: input.spec, + localDir: input.localDir, + remoteDir: input.remoteDir, + exclude: [".git", ".paperclip-runtime", ...(input.exclude ?? [])], + onProgress: input.onProgress, + progressLabel: input.progressLabel, + }); + await removeDeletedPathsOnSsh({ + spec: input.spec, + remoteDir: input.remoteDir, + deletedPaths: input.snapshot.deletedPaths, + }); +} + +async function excludeProjectRepositoriesOnSsh(spec: SshConnectionConfig, remoteDir: string): Promise { + const pattern = `/${PROJECT_REPOSITORIES_DIR}/`; + const excludeFile = path.posix.join(remoteDir, ".git", "info", "exclude"); + await runSshScript( + spec, + `mkdir -p ${shellQuote(path.posix.dirname(excludeFile))} && ` + + `{ grep -qxF ${shellQuote(pattern)} ${shellQuote(excludeFile)} 2>/dev/null || ` + + `printf '\\n%s\\n' ${shellQuote(pattern)} >> ${shellQuote(excludeFile)}; }`, + { timeoutMs: 30_000 }, + ); +} + async function allocateLoopbackPort(host: string): Promise { return await new Promise((resolve, reject) => { const server = net.createServer(); @@ -1560,32 +1646,38 @@ export async function prepareWorkspaceForSshExecution(input: { onProgress?: RuntimeProgressSink; workspaceFileMode?: "all"; workspaceExclude?: string[]; -}): Promise<{ gitBacked: boolean }> { +}): Promise<{ gitBacked: boolean; repositories?: string[] }> { const remoteDir = input.remoteDir ?? input.spec.remoteCwd; const gitSnapshot = input.workspaceFileMode === "all" ? null : await readLocalGitWorkspaceSnapshot(input.localDir); if (gitSnapshot) { - await importGitWorkspaceToSsh({ + const repositories = await listLocalProjectRepositories(input.localDir); + await transportGitWorkspaceToSsh({ spec: input.spec, localDir: input.localDir, remoteDir, snapshot: gitSnapshot, - onProgress: input.onProgress, - }); - await syncDirectoryToSsh({ - spec: input.spec, - localDir: input.localDir, - remoteDir, - exclude: [".git", ".paperclip-runtime"], + exclude: repositories.length > 0 ? [PROJECT_REPOSITORIES_DIR] : [], onProgress: input.onProgress, progressLabel: "workspace", }); - await removeDeletedPathsOnSsh({ - spec: input.spec, - remoteDir, - deletedPaths: gitSnapshot.deletedPaths, - }); - return { gitBacked: true }; + if (repositories.length > 0) { + await excludeProjectRepositoriesOnSsh(input.spec, remoteDir); + } + for (const relative of repositories) { + const localDir = path.join(input.localDir, relative); + const snapshot = await readLocalGitWorkspaceSnapshot(localDir); + if (!snapshot) throw new Error(`Cannot read the Git state of project repository: ${relative}`); + await transportGitWorkspaceToSsh({ + spec: input.spec, + localDir, + remoteDir: path.posix.join(remoteDir, relative), + snapshot, + onProgress: input.onProgress, + progressLabel: relative, + }); + } + return { gitBacked: true, ...(repositories.length > 0 ? { repositories } : {}) }; } await clearRemoteDirectory({ @@ -1611,8 +1703,62 @@ export async function restoreWorkspaceFromSshExecution(input: { baselineSnapshot?: DirectorySnapshot; restoreGitHistory?: boolean; onProgress?: RuntimeProgressSink; + repositories?: Array<{ path: string; baselineSnapshot?: DirectorySnapshot }>; }): Promise { const remoteDir = input.remoteDir ?? input.spec.remoteCwd; + const repositories = input.repositories ?? []; + for (const repository of repositories) { + if ( + path.posix.dirname(repository.path) !== PROJECT_REPOSITORIES_DIR || + !PROJECT_REPOSITORY_NAME_PATTERN.test(path.posix.basename(repository.path)) + ) { + throw new Error(`Invalid project repository path: ${repository.path}`); + } + if (input.baselineSnapshot && !repository.baselineSnapshot) { + throw new Error(`Project repository has no workspace baseline: ${repository.path}`); + } + } + if ( + input.baselineSnapshot && + repositories.length > 0 && + !input.baselineSnapshot.exclude.includes(PROJECT_REPOSITORIES_DIR) + ) { + throw new Error(`Workspace baseline must exclude ${PROJECT_REPOSITORIES_DIR} when project repositories are restored separately`); + } + for (const repository of repositories) { + await restoreWorkspaceRootFromSsh({ + spec: input.spec, + localDir: path.join(input.localDir, repository.path), + remoteDir: path.posix.join(remoteDir, repository.path), + baselineSnapshot: repository.baselineSnapshot, + restoreGitHistory: input.restoreGitHistory, + onProgress: input.onProgress, + progressLabel: repository.path, + }); + } + await restoreWorkspaceRootFromSsh({ + spec: input.spec, + localDir: input.localDir, + remoteDir, + baselineSnapshot: input.baselineSnapshot, + restoreGitHistory: input.restoreGitHistory, + onProgress: input.onProgress, + progressLabel: "workspace", + hasProjectRepositories: repositories.length > 0, + }); +} + +async function restoreWorkspaceRootFromSsh(input: { + spec: SshRemoteExecutionSpec; + localDir: string; + remoteDir: string; + baselineSnapshot?: DirectorySnapshot; + restoreGitHistory?: boolean; + onProgress?: RuntimeProgressSink; + progressLabel: string; + hasProjectRepositories?: boolean; +}): Promise { + const remoteDir = input.remoteDir; if (input.baselineSnapshot) { const stagingDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-sync-back-")); const importedRef = input.restoreGitHistory @@ -1635,7 +1781,7 @@ export async function restoreWorkspaceFromSshExecution(input: { localDir: stagingDir, exclude: input.baselineSnapshot.exclude, onProgress: input.onProgress, - progressLabel: "workspace", + progressLabel: input.progressLabel, }); await mergeDirectoryWithBaseline({ baseline: input.baselineSnapshot, @@ -1666,6 +1812,7 @@ export async function restoreWorkspaceFromSshExecution(input: { const gitSnapshot = await readLocalGitWorkspaceSnapshot(input.localDir); if (gitSnapshot) { + const projectRepositoryEntries = input.hasProjectRepositories ? [PROJECT_REPOSITORIES_DIR] : []; await exportGitWorkspaceFromSsh({ spec: input.spec, remoteDir, @@ -1676,10 +1823,10 @@ export async function restoreWorkspaceFromSshExecution(input: { spec: input.spec, remoteDir, localDir: input.localDir, - exclude: [".git", ".paperclip-runtime"], - preserveLocalEntries: [".git"], + exclude: [".git", ".paperclip-runtime", ...projectRepositoryEntries], + preserveLocalEntries: [".git", ...projectRepositoryEntries], onProgress: input.onProgress, - progressLabel: "workspace", + progressLabel: input.progressLabel, }); return; } @@ -1690,7 +1837,7 @@ export async function restoreWorkspaceFromSshExecution(input: { localDir: input.localDir, exclude: [".paperclip-runtime"], onProgress: input.onProgress, - progressLabel: "workspace", + progressLabel: input.progressLabel, }); } diff --git a/packages/paperclip-runner/docs/durable-recovery.md b/packages/paperclip-runner/docs/durable-recovery.md index 8ee0b22c97..0a9d5d9a7a 100644 --- a/packages/paperclip-runner/docs/durable-recovery.md +++ b/packages/paperclip-runner/docs/durable-recovery.md @@ -95,6 +95,15 @@ welcome and every control envelope against the authenticated connection, runner, environment lease, run, normalized session, turn, item, protocol, lease ID, expiry, and revocation metadata before applying an ACK or command. +Semantic input integrity uses SHA-256 of the complete transmitted input's +canonical JSON, with UTF-16 object-key order and JavaScript number formatting. +The controller checks this proof after authentication and exact run/session/ +turn/item correlation, before committing, dispatching, or acknowledging the +input. Receipt redaction has a separate digest; a redacted digest cannot stand +in for wire integrity, including when only a protected field changes. A failed +integrity proof keeps the existing operator-required recovery fence. Updating +the verifier does not clear a previously failed run or replay its work. + The daemon captures and removes the bootstrap environment variable before it parses arguments or starts child work. Secret buffers are overwritten when they are dropped. It resolves the destination once before sending a bearer value and diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs b/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs index ce3b9e3e4b..b1ed069b09 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/bin/fake-codex-app-server.rs @@ -652,6 +652,9 @@ fn run() -> Result<(), Box> { .iter() .any(|value| value == "--require-codex-home-auth"); let durable_turn_ids = args.iter().any(|value| value == "--durable-turn-ids"); + let hold_first_durable_turn = args + .iter() + .any(|value| value == "--hold-first-durable-turn"); let durable_tool_ids = args.iter().any(|value| value == "--durable-tool-ids"); let expected_canonical_task_context_file = argument(&args, "--expected-canonical-task-context-file"); @@ -1348,6 +1351,7 @@ fn run() -> Result<(), Box> { .checked_add(1) .ok_or("fake provider turn sequence exhausted")?; } + let hold_turn = hold_turn || (hold_first_durable_turn && state.next_turn == 1); if reject_second_turn_start && turn_start_count == 2 { send(json!({ "method": "warning", diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs b/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs index 545544c73e..8d038f5fe1 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs @@ -2103,17 +2103,29 @@ impl CodexCommandExecutor { "providerTurnId": previous_active_turn_id, "status": "failed", "providerTerminalObserved": false, + "error": if provider_label == "codex" { json!({ + "code": "provider_turn_lost_on_restore", + "recoverable": true, + "message": "The runner restored the conversation after process loss, but the previous turn is no longer active.", + }) } else { Value::Null }, }), }; - let outcome = terminal_events( - state, - "turn.failed", - state.goal.as_ref().map(|goal| goal.status.as_str()), - ); - state.extend_terminal_events(with_terminal_outcome( - vec![provider_terminal], - outcome, - ))?; + if provider_label == "codex" { + // No provider result was observed. Preserve the lost + // turn as a fact, but leave the run open for the + // controller's bounded same-conversation recovery. + state.push_terminal_event(provider_terminal)?; + } else { + let outcome = terminal_events( + state, + "turn.failed", + state.goal.as_ref().map(|goal| goal.status.as_str()), + ); + state.extend_terminal_events(with_terminal_outcome( + vec![provider_terminal], + outcome, + ))?; + } } } else { state.push_event(reconciled)?; diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs index 9550fb2e95..34e3b46d0c 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/codex_provider.rs @@ -3484,6 +3484,65 @@ fn durable_backend_replays_pending_tool_calls_without_mutating_the_event_queue() fs::remove_dir_all(directory).expect("remove Codex integration-test directory"); } +#[test] +fn restart_lost_turn_retains_a_recoverable_cause_without_inventing_a_result() { + let directory = temporary_directory("restart-lost-turn"); + let config = provider_config( + &directory, + &["--durable-turn-ids", "--linger-after-turn-start"], + ); + let mut first = CodexCommandExecutor::new(&directory); + first + .execute(&command( + "prepare", + 1, + "run.prepare", + json!({"provider": config}), + )) + .unwrap(); + first + .execute(&command("open", 2, "session.open", json!({}))) + .unwrap(); + first + .execute(&command( + "start", + 3, + "turn.start", + json!({"text": "Watch the existing check run."}), + )) + .unwrap(); + // The process disappeared while its command was in flight. Codex restores + // the conversation, but cannot restore the running turn. + drop(first); + fs::write( + directory.join("fake-state.json"), + serde_json::to_vec(&json!({ + "threadId": "codex-thread-1", "activeTurnId": null, "nextTurn": 1, + })) + .unwrap(), + ) + .unwrap(); + let mut restored = CodexCommandExecutor::new(&directory); + restored + .execute(&command("snapshot", 4, "session.snapshot", json!({}))) + .unwrap(); + let events = poll_and_ack(&mut restored).unwrap(); + let interrupted = events + .iter() + .find(|event| event.event_type == "turn.failed") + .unwrap(); + assert_eq!( + interrupted.payload["error"]["code"], + "provider_turn_lost_on_restore" + ); + assert_eq!(interrupted.payload["error"]["recoverable"], true); + assert!(!events + .iter() + .any(|event| event.event_type == "run.result.proposed")); + restored.shutdown().unwrap(); + fs::remove_dir_all(directory).unwrap(); +} + #[test] fn durable_backend_settles_pending_tools_when_recovery_finds_the_turn_ended() { let directory = temporary_directory("durable-tool-ended-offline"); diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs new file mode 100644 index 0000000000..1ef2c8f34b --- /dev/null +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/semantic_input_digest.rs @@ -0,0 +1,18 @@ +use paperclip_runner_core::provider_bridge::semantic_value_digest; +use serde_json::Value; + +#[test] +fn semantic_input_digests_match_the_authenticated_controller_fixtures() { + let fixtures: Value = serde_json::from_str(include_str!( + "../../../../test/fixtures/semantic-input-digests.json" + )) + .expect("parse shared semantic input digest fixtures"); + for fixture in fixtures.as_array().expect("digest fixture array") { + assert_eq!( + semantic_value_digest(&fixture["input"]), + fixture["digest"].as_str().expect("fixture digest"), + "fixture {}", + fixture["name"] + ); + } +} diff --git a/packages/paperclip-runner/src/backends/harness-driver-backend.ts b/packages/paperclip-runner/src/backends/harness-driver-backend.ts index fd280b5df8..143dd877be 100644 --- a/packages/paperclip-runner/src/backends/harness-driver-backend.ts +++ b/packages/paperclip-runner/src/backends/harness-driver-backend.ts @@ -673,6 +673,7 @@ class HarnessNativeSession implements NativeSession { // The provider has no active turn to interrupt yet. Do not launch work // whose events cancellation would suppress and leave the owner waiting. if (this.#explicitlyCancelled) throw new Error("native_session_cancelled"); + this.#terminal = null; try { const started = await this.#session.startTurn(input); this.#assertProtocolIntegrity(); diff --git a/packages/paperclip-runner/src/contracts/native-session-backend.ts b/packages/paperclip-runner/src/contracts/native-session-backend.ts index 7a84e9577d..87af9e2ad0 100644 --- a/packages/paperclip-runner/src/contracts/native-session-backend.ts +++ b/packages/paperclip-runner/src/contracts/native-session-backend.ts @@ -63,7 +63,7 @@ export interface PersistedNativeSession { terminal?: PrpTerminalState | null; activeTurnId?: string | null; terminalTurns?: PersistedHarnessTurnTerminal[]; - /** Durable at-most-once marker for a resultless terminal recovery turn. */ + /** At-most-once marker for resultless disposition repair or restart continuation. */ dispositionOnlyRecoveryConsumed?: boolean; dispositionOnlyRecoveryTurnId?: string | null; pendingRuntimeRequests?: HarnessRuntimeRequest[]; @@ -77,6 +77,31 @@ export interface NativeSessionRecoveryResult { reason?: string; } +/** Runner-owned cause: the process was restored, but its active turn was lost. */ +export const NATIVE_RESTART_INTERRUPTION_CODE = "provider_turn_lost_on_restore"; + +export function isNativeRestartInterruption(error: unknown): boolean { + if (!error || typeof error !== "object" || Array.isArray(error)) return false; + const failure = error as Record; + return failure.code === NATIVE_RESTART_INTERRUPTION_CODE && failure.recoverable === true; +} + +/** The terminal fingerprint survives a crash between interruption and continuation. */ +export function nativeRestartInterruptedTurnId(snapshot: Pick): string | null { + if (snapshot.driverKind !== "codex_app_server" || snapshot.activeTurnId != null || + snapshot.semanticResult != null || snapshot.dispositionOnlyRecoveryConsumed || + snapshot.goal != null || snapshot.pendingRuntimeRequests?.length) return null; + const terminal = Array.isArray(snapshot.terminalTurns) ? snapshot.terminalTurns.at(-1) : null; + if (!terminal || typeof terminal.fingerprint !== "string" || !terminal.turnId) return null; + try { + const fingerprint = JSON.parse(terminal.fingerprint); + return fingerprint?.terminalState === "failed" && fingerprint.result === null && + isNativeRestartInterruption(fingerprint.error) ? terminal.turnId : null; + } catch { return null; } +} + /** * Cancellation is a synchronous authority transition followed by passive * provider cleanup. Once `cancel` returns, the provider session must no longer diff --git a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts index ad31242683..222c41abd2 100644 --- a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts +++ b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts @@ -26,6 +26,7 @@ import { validatePrpEvent } from "../protocol/replay-contract.js"; import { digestPaperclipSemanticContent } from "../semantic-tools/receipts.js"; import { DurablePrpControlPlane, + durableRecoveryInternals, SemanticToolNotDispatchedError, inspectWarmRunTransition, spawnRunner, @@ -1170,7 +1171,17 @@ async function receiveSecure( return value; } -function semanticInputEvent(sourceSeq = 1): Record { +const semanticInputDigestFixtures = JSON.parse(readFileSync( + new URL("../../test/fixtures/semantic-input-digests.json", import.meta.url), + "utf8", +)) as Array<{ name: string; input: Record; digest: string }>; + +function rawSemanticInputDigest(input: unknown): string { + return `sha256:${createHash("sha256") + .update(durableRecoveryInternals.canonicalJson(input)).digest("hex")}`; +} + +function semanticInputEvent(sourceSeq = 1, input: Record = {}): Record { return { protocol: "paperclip.runner", version: 1, @@ -1210,11 +1221,11 @@ function semanticInputEvent(sourceSeq = 1): Record { }, idempotencyKey: null, content: { - digest: digestPaperclipSemanticContent({}), + digest: rawSemanticInputDigest(input), redactionDisposition: "digest_only", references: [], }, - input: {}, + input, }, }, }, @@ -1693,6 +1704,144 @@ describe.sequential("DurablePrpControlPlane", () => { }, ); + it.each(semanticInputDigestFixtures.flatMap((fixture) => + ["semantic_tool.input", "mcp_app.tool_input"].map((eventType) => ({ ...fixture, eventType })), + ))( + "accepts the Rust wire digest and preserves exact input: $eventType / $name", + async ({ input, digest, eventType }) => { + // The JSON transport normalizes negative zero before the receiver sees it. + const wireInput = JSON.parse(JSON.stringify(input)) as Record; + const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-raw-input-")); + const onProtocolIntegrityError = vi.fn(); + const onSemanticToolInput = vi.fn(async () => ({ result: { ok: true } })); + const core = new DurablePrpControlPlane({ + stateDirectory: root, identity, expectedRunnerVersion, expectedRunnerDigest, + onProtocolIntegrityError, onSemanticToolInput, + }); + try { + await core.start(); + const client = (await authenticate(core, core.issueBootstrapTicket()))!; + const event = semanticInputEvent(1, wireInput); + (event.payload as any).eventType = eventType; + const semantic = (event.payload as any).payload.semantic_tool; + // This shared golden also runs against the Rust producer's hash. + semantic.content.digest = digest; + expect(rawSemanticInputDigest(input)).toBe(digest); + sendSecure(client, event); + const frames = [await receiveSecure(client), await receiveSecure(client)]; + expect(frames).toContainEqual(expect.objectContaining({ + kind: "ack", payload: expect.objectContaining({ ackedSourceSeq: 1 }), + })); + const command = frames.find((frame) => frame?.kind === "command")!.payload as any; + expect(command.payload.inputDigest).toBe(digest.slice("sha256:".length)); + expect(onSemanticToolInput).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({ input: wireInput })); + expect(core.store.state.committedEvents).toHaveLength(1); + expect(core.store.state.committedEvents[0]?.envelope).toEqual(event); + expect(onProtocolIntegrityError).not.toHaveBeenCalled(); + client.socket.destroy(); + } finally { await core.stop(); rmSync(root, { recursive: true, force: true }); } + }, + ); + + it.each([ + "redacted_digest", "protected_field_tampering", "forged_digest", + "over_node_limit", "over_depth_limit", + ] as const)( + "rejects invalid raw semantic integrity without commit, dispatch or ACK: %s", + async (fault) => { + const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-raw-integrity-")); + const onProtocolIntegrityError = vi.fn(); + const onCommittedEvent = vi.fn(async () => undefined); + const onSemanticToolInput = vi.fn(async () => ({ result: { ok: true } })); + const core = new DurablePrpControlPlane({ + stateDirectory: root, identity, expectedRunnerVersion, expectedRunnerDigest, + onProtocolIntegrityError, onCommittedEvent, onSemanticToolInput, + }); + try { + await core.start(); + const client = (await authenticate(core, core.issueBootstrapTicket()))!; + const input = { password: "fixture-before", safe: true }; + const event = semanticInputEvent(1, input); + const semantic = (event.payload as any).payload.semantic_tool; + const redactedDigest = digestPaperclipSemanticContent(input); + expect(redactedDigest).not.toBe(semantic.content.digest); + if (fault === "redacted_digest") semantic.content.digest = redactedDigest; + if (fault === "forged_digest") semantic.content.digest = `sha256:${"0".repeat(64)}`; + if (fault === "protected_field_tampering") { + semantic.input = { ...input, password: "fixture-after" }; + // Redacted receipt hashes intentionally cannot prove byte integrity. + expect(digestPaperclipSemanticContent(semantic.input)).toBe(redactedDigest); + } + if (fault === "over_node_limit") semantic.input = { values: Array(10_001).fill(0) }; + if (fault === "over_depth_limit") { + let nested: unknown = {}; + for (let depth = 0; depth < 65; depth += 1) nested = { child: nested }; + semantic.input = nested; + } + if (fault === "over_node_limit" || fault === "over_depth_limit") { + // Each fixture object has one key, so JSON.stringify is canonical. + // Hash independently of the bounded helper: the limits themselves + // must reject even a matching raw digest and keep the permanent fence. + semantic.content.digest = `sha256:${createHash("sha256") + .update(JSON.stringify(semantic.input)).digest("hex")}`; + } + sendSecure(client, event); + await expect(receiveSecure(client)).resolves.toBeNull(); + expect(onProtocolIntegrityError).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({ + code: "native_event_replay_conflict", reason: "semantic_input_digest_mismatch", + recovery: "operator_required", + })); + expect(core.store.state.ackedSourceSeq).toBe(0); + expect(core.store.state.committedEvents).toEqual([]); + expect(onCommittedEvent).not.toHaveBeenCalled(); + expect(onSemanticToolInput).not.toHaveBeenCalled(); + expect(JSON.stringify(onProtocolIntegrityError.mock.calls)).not.toContain("fixture-before"); + expect(JSON.stringify(onProtocolIntegrityError.mock.calls)).not.toContain("fixture-after"); + const replay = (await authenticate(core, client.leaseToken!))!; + sendSecure(replay, semanticInputEvent()); + await expect(receiveSecure(replay)).resolves.toBeNull(); + expect(onProtocolIntegrityError).toHaveBeenCalledTimes(1); + expect(onSemanticToolInput).not.toHaveBeenCalled(); + } finally { await core.stop(); rmSync(root, { recursive: true, force: true }); } + }, + ); + + it.each(["authentication", "envelope", "event", "correlation"] as const)( + "rejects raw semantic input outside its exact authenticated scope: %s", + async (boundary) => { + const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-raw-scope-")); + const onProtocolIntegrityError = vi.fn(); + const onCommittedEvent = vi.fn(async () => undefined); + const onSemanticToolInput = vi.fn(async () => ({ result: { ok: true } })); + const core = new DurablePrpControlPlane({ + stateDirectory: root, identity, expectedRunnerVersion, expectedRunnerDigest, + onProtocolIntegrityError, onCommittedEvent, onSemanticToolInput, + }); + try { + await core.start(); + const wrongRunId = "00000000-0000-4000-8000-000000000999"; + const client = await authenticate(core, core.issueBootstrapTicket(), + boundary === "authentication" ? { ...identity, runId: wrongRunId } : identity); + if (boundary === "authentication") { + expect(client).toBeNull(); + } else { + const event = semanticInputEvent(1, { password: "fixture-only" }); + const payload = event.payload as any; + if (boundary === "envelope") event.runId = wrongRunId; + if (boundary === "event") payload.runId = wrongRunId; + if (boundary === "correlation") payload.payload.semantic_tool.correlation.runId = wrongRunId; + sendSecure(client!, event); + await expect(receiveSecure(client!)).resolves.toBeNull(); + } + expect(core.store.state.ackedSourceSeq).toBe(0); + expect(core.store.state.committedEvents).toEqual([]); + expect(onCommittedEvent).not.toHaveBeenCalled(); + expect(onSemanticToolInput).not.toHaveBeenCalled(); + expect(onProtocolIntegrityError).not.toHaveBeenCalled(); + } finally { await core.stop(); rmSync(root, { recursive: true, force: true }); } + }, + ); + it("latches an authenticated semantic integrity fault without ACK or dispatch and still permits suspension", async () => { const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-integrity-")); const onProtocolIntegrityError = vi.fn(); @@ -1854,7 +2003,7 @@ describe.sequential("DurablePrpControlPlane", () => { ).semantic_tool as Record; semantic.input = { changed: true }; (semantic.content as Record).digest = - digestPaperclipSemanticContent(semantic.input); + rawSemanticInputDigest(semantic.input); sendSecure(client, changed); await expect(receiveSecure(client)).resolves.toBeNull(); expect(onProtocolIntegrityError).toHaveBeenCalledTimes(1); @@ -2238,13 +2387,13 @@ describe.sequential("DurablePrpControlPlane", () => { const event = semanticInputEvent(); const semantic = ((event.payload as any).payload as any).semantic_tool; semantic.input = args; - semantic.content.digest = digestPaperclipSemanticContent(args); + semantic.content.digest = rawSemanticInputDigest(args); sendSecure(client, event); const frames = [await receiveSecure(client), await receiveSecure(client)]; const wire = frames.find((frame) => frame?.kind === "command")!.payload as any; expect(wire.payload.result.content).toBe(content); expect(wire.payload.input).toBeUndefined(); - expect(wire.payload.inputDigest).toBe(digestPaperclipSemanticContent(args).slice("sha256:".length)); + expect(wire.payload.inputDigest).toBe(rawSemanticInputDigest(args).slice("sha256:".length)); expect(Buffer.byteLength(JSON.stringify(wire))).toBeLessThan(1024 * 1024); sendSecure(client, { protocol: "paperclip.runner", version: 1, kind: "command_result", payload: { commandId: wire.commandId, controllerSeq: wire.controllerSeq, commandType: wire.type, diff --git a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts index e5320e5faa..bbf604d4dc 100644 --- a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts +++ b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts @@ -35,7 +35,6 @@ import { validatePrpEvent, type PrpEvent, } from "../protocol/replay-contract.js"; -import { digestPaperclipSemanticContent } from "../semantic-tools/receipts.js"; import { type DurableRecoveryCommittedEvent, type DurableRecoveryCoreCommand, @@ -416,6 +415,18 @@ function canonicalDigest(value: unknown): string { return createHash("sha256").update(canonicalJson(value)).digest("hex"); } +function matchesSemanticInputDigest(input: unknown, digest: unknown): boolean { + try { + // Wire integrity covers the complete input, including protected fields. + // Receipt redaction can erase those differences and has a separate hash. + return digest === `sha256:${canonicalDigest(input)}`; + } catch { + // Canonicalization bounds must keep the same permanent integrity fence. + // Never attach an input-derived error or payload to the diagnostic. + return false; + } +} + function exactIdentity(value: unknown): value is DurableRecoveryIdentity { return ( isRecord(value) && @@ -3141,8 +3152,10 @@ export class DurablePrpControlPlane { if ( isSemanticInput && semantic !== undefined && - (semantic.content as Record).digest !== - digestPaperclipSemanticContent(semantic.input) + !matchesSemanticInputDigest( + semantic.input, + (semantic.content as Record).digest, + ) ) { // Only the authenticated, schema-valid, exactly correlated input may // permanently fail its owner. Never commit, dispatch, or ACK these bytes. diff --git a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts index 8be0449173..6f3b5ad8a7 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts @@ -17,7 +17,7 @@ import type { PersistedHarnessSession, PersistedHarnessTurnTerminal, } from "../../contracts/harness-driver.js"; -import { NativeSessionProtocolIntegrityError } from "../../contracts/native-session-backend.js"; +import { NativeSessionProtocolIntegrityError, nativeRestartInterruptedTurnId } from "../../contracts/native-session-backend.js"; import { HarnessReconciliationError } from "../../contracts/harness-driver.js"; import { CODEX_CODEX_PROTOCOL_VERSION, @@ -528,6 +528,12 @@ export class CodexAppServerDriver implements HarnessDriver { turns.forEach((turn, index) => { if (terminalIds.has(text(turn.id))) lastKnownTerminalIndex = index; }); + if (nativeRestartInterruptedTurnId({ + ...snapshot, semanticResult: null, + }) && (!providerHistoryIsArray || lastKnownTerminalIndex < 0)) { + await cancellation.wait(cancellation.close()); + return { recovered: false, reason: "restart interruption history is incomplete" }; + } // Releasing a consumed marker requires both an actual history array // and a checkpointed terminal that anchors its ordering. An array that // omits every durable terminal may be truncated or inconsistent, so diff --git a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts index dc4e03e33c..7c76d0edff 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.recovery.test.ts @@ -45,6 +45,29 @@ import { import { NativeSessionProtocolIntegrityError } from "../../contracts/native-session-backend.js"; describe("Codex app-server Codex driver", () => { + it("does not resend a restart continuation when its history anchor is missing", async () => { + const first = new FakeCodexTransport(); + const second = new FakeCodexTransport(); + const driver = makeDriver([first, second]); + const original = await driver.openSession({ + runId: "run-restart-anchor", normalizedSessionId: "session-restart-anchor", workingDirectory: WORKSPACE, + }); + await original.startTurn({ message: { role: "user", text: "Finish the request." } }); + first.push("turn/completed", { threadId: "thread-1", turn: { + id: "turn-1", status: "failed", error: { code: "provider_turn_lost_on_restore", recoverable: true }, items: [], + } }); + await collectUntilTerminal(original.events()); + const snapshot = await original.snapshot(); + await original.close({ reason: "controller lost before continuation checkpoint" }); + second.readResponse = { thread: { id: "thread-1", sessionId: "provider-session-1", cwd: WORKSPACE, + turns: [{ id: "uncheckpointed-turn", status: "completed", items: [] }], + } }; + await expect(driver.recoverSession?.(snapshot)).resolves.toMatchObject({ + recovered: false, reason: "restart interruption history is incomplete", + }); + expect(second.calls.some(call => call.method === "turn/start")).toBe(false); + }); + it.each([null, "checkpointed-prior-turn"])("recovers an autonomous goal turn beyond checkpoint %s", async (checkpointTurnId) => { const first = new FakeCodexTransport(); const second = new FakeCodexTransport(); diff --git a/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts b/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts index f2c5d70f29..47612b9267 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-session-terminal.ts @@ -1,4 +1,5 @@ import { HarnessReconciliationError } from "../../contracts/harness-driver.js"; +import { isNativeRestartInterruption } from "../../contracts/native-session-backend.js"; import type { PrpStructuredRunResult } from "../../protocol/replay-contract.js"; import { boundedCodexPayload as boundedPayload, isRetainableCodexPayload } from "./codex-boundaries.js"; import type { CodexSessionState } from "./codex-session-state.js"; @@ -168,6 +169,13 @@ export function mapTerminalTurn( { turnId }, ); finalize(state, status); + if (status === "failed" && isNativeRestartInterruption(turn.error) && state.result === null && + !state.dispositionOnlyRecoveryConsumed) { + // Orchestration may use the paid restart attempt for one same-thread + // continuation. Its accepted turn id uses the existing durable marker. + state.terminal = false; + state.dispositionOnlyRecoveryAvailable = true; + } } function terminalFingerprint( diff --git a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts index a5fd76bb17..b73efc381b 100644 --- a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts +++ b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts @@ -3583,20 +3583,31 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { if (method === "thread/turns/list") { data = turns.map(turn => ({ ...turn, items: [], itemsView: "notLoaded" })); } else { - if (params.turnId !== this.#turnId) throw new Error("codex_history_unavailable: requested turn is outside the retained runner event window"); + if (!turns.some(turn => turn.id === params.turnId)) throw new Error("codex_history_unavailable: requested turn is outside the retained runner event window"); const items = new Map>(); + let semanticResultItem: Record | null = null; let observedTurn = ""; let observedStart = false; for (const event of this.#core?.store.state.committedEvents ?? []) { + if (event.envelope.runId !== this.#core?.store.state.identity.runId) continue; const payload = record(record(event.envelope.payload).payload); if (event.eventType === "turn.started") observedTurn = String(payload.providerTurnId ?? payload.turnId ?? record(payload.turn).id ?? ""); if (event.eventType === "turn.started" && observedTurn === params.turnId) observedStart = true; + if (event.eventType === "run.result.proposed" && observedTurn === params.turnId) { + // Reconciliation can precede notification delivery. Recover the + // runner's authoritative result with the exact retained turn, + // rather than launching work again just to obtain a disposition. + const id = `runner-result-${event.sourceSeq}`; + semanticResultItem = { turnId: observedTurn, item: { id, type: "agentMessage", text: JSON.stringify(payload) } }; + } if (event.eventType !== "item.completed" || observedTurn !== params.turnId) continue; const item = record(rehydrateRunnerdItemNotification(payload, this.#threadId, observedTurn).item); if (typeof item.id === "string") items.set(item.id, { turnId: observedTurn, item }); } if (!observedStart) throw new Error("codex_history_incomplete: requested turn start is outside the retained runner event window"); data = [...items.values()]; + // Runner authority wins over schema-shaped prose in an assistant item. + if (semanticResultItem) data.push(semanticResultItem); } if (params.sortDirection === "desc") data.reverse(); const offset = params.cursor == null ? 0 : Number(params.cursor); @@ -3657,8 +3668,12 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { await new Promise((resolveWait) => setTimeout(resolveWait, 10)); } if (terminal !== undefined) { + const payload = record(record(terminal.envelope.payload).payload); recoveredTurns.push({ id: this.#turnId, + // Reconciliation must retain the cause, including the runner's + // explicit process-loss marker, rather than inventing error:null. + error: payload.error ?? record(payload.turn).error ?? null, status: terminal.eventType === "turn.completed" ? "completed" @@ -3670,6 +3685,24 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { }); } } + // A controller can lose the checkpoint after a continuation is accepted. + // Keep its prior terminal as the history anchor, so driver recovery can + // adopt the later accepted turn instead of submitting it again. Items + // remain lazy and fail closed if their start left the retained window. + const priorTerminals = new Map>(); + for (const event of this.#core?.store.state.committedEvents ?? []) { + if (event.envelope.runId !== this.#core?.store.state.identity.runId || + !["turn.completed", "turn.failed", "turn.interrupted", "turn.cancelled"].includes(event.eventType)) continue; + const payload = record(record(event.envelope.payload).payload); + const turnId = payload.providerTurnId ?? payload.turnId ?? record(payload.turn).id; + if (typeof turnId !== "string" || !turnId || turnId === this.#turnId) continue; + priorTerminals.set(turnId, { + id: turnId, + status: event.eventType.slice("turn.".length), + error: payload.error ?? record(payload.turn).error ?? null, + }); + } + recoveredTurns.unshift(...priorTerminals.values()); this.#recoveryTurnBindingPending = false; this.#pumpEvents(); return { diff --git a/packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts b/packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts new file mode 100644 index 0000000000..e1e1095165 --- /dev/null +++ b/packages/paperclip-runner/src/live/runnerd-restart-continuation.test.ts @@ -0,0 +1,125 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { expect, it, vi } from "vitest"; +import { createRunnerdNativeSessionBackend } from "../backends/codex-native-backend.js"; +import type { NativeExecutionInputV1 } from "../contracts/native-execution.js"; +import type { PersistedNativeSession } from "../contracts/native-session-backend.js"; +import type { PrpEvent } from "../protocol/replay-contract.js"; +import { executeNativeSession } from "../native-session-runtime.js"; +import { createRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "./runnerd-codex-transport.js"; + +it.each(["none", "before-start", "after-start"] as const)("completes after the real runner and provider are killed (second crash: %s)", async (crash) => { + const root = await mkdtemp(join(tmpdir(), "runner-restart-continuation-")); + const identity = { runId: "run-restart", sessionId: "session-restart", companyId: "company-restart", issueId: "issue-restart", agentId: "agent-restart" }; + const execution: NativeExecutionInputV1 = { + schema: "paperclip.native-execution-input.v1", + binding: { runId: identity.runId, companyId: identity.companyId, issueId: identity.issueId, agentId: identity.agentId, executionWorkspaceId: "workspace-restart" }, + session: { normalizedSessionId: identity.sessionId, driverKind: "codex_app_server", protocolVersion: 1 }, + task: { identifier: "TEST-1", title: "Finish checks", description: null, prompt: "Watch the already running checks and finish the request.", workMode: "standard" }, + workspace: { cwd: root, repoUrl: null, repoRef: null, branchName: null }, + provider: { kind: "codex", model: null }, + completionContract: { id: "contract-restart", sha256: "digest", schemaVersion: "paperclip.completion-contract.v1", contract: { + revision: "1", objective: "Finish checks", criteria: [{ id: "checks", requirement: "Check the outcome" }], + } }, + interactionResponses: [], credentialBindings: [], + }; + const options = { + runnerBinary: defaultCapabilityRunnerdBinary(), + codexCommand: resolve(import.meta.dirname, "../../runner/target/debug/fake-codex-app-server"), + codexArgs: ["--state-file", join(root, "fake-state.json"), "--call-log", join(root, "calls.log"), "--durable-turn-ids", "--hold-first-durable-turn"], + stateDirectory: root, + prpIdentity: { runnerInstanceId: "runner-restart", environmentLeaseId: "lease-restart", runId: identity.runId, normalizedSessionId: identity.sessionId, turnId: "turn-restart", itemId: "item-restart" }, + }; + const first = createRunnerdCodexTransport(options); + let restored: ReturnType | undefined; + const alive = (pid: number | null) => { + if (!pid) return false; + try { process.kill(pid, 0); return true; } catch { return false; } + }; + const kill = (pid: number | null) => { + if (!pid) return; + try { process.kill(pid, "SIGKILL"); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error; } + }; + try { + const backend = createRunnerdNativeSessionBackend(execution, { runnerInstanceId: "runner-restart", transportFactory: () => first.transport }); + const session = await backend.openSession({ identity, workingDirectory: root }); + await session.startTurn({ message: { role: "user", text: execution.task.prompt } }); + const persisted = await session.snapshot(); + expect(persisted.activeTurnId).toBe("provider-turn-1"); + const durableEvents: PrpEvent[] = []; + const initialEvents = session.events()[Symbol.asyncIterator](); + while ((durableEvents.at(-1)?.sourceSeq ?? 0) < Number(persisted.cursor)) { + const event = await initialEvents.next(); + if (event.done) throw new Error("initial event history ended early"); + durableEvents.push(event.value); + } + const owner = first.evidence(); + await first.detachControllerForRestart(); + kill(owner.runnerPid); + kill(owner.providerPid); + await vi.waitFor(() => { expect(alive(owner.runnerPid)).toBe(false); expect(alive(owner.providerPid)).toBe(false); }); + // Simulate Codex's process-loss history: the thread remains, its in-memory + // turn does not. Keep the turn counter to prevent provider identity reuse. + const providerState = JSON.parse(await readFile(join(root, "fake-state.json"), "utf8")); + providerState.activeTurnId = null; + await writeFile(join(root, "fake-state.json"), JSON.stringify(providerState)); + restored = createRunnerdCodexTransport({ ...options, resumeDynamicTools: [], resumeActiveTurnId: persisted.activeTurnId }); + let recovering = createRunnerdNativeSessionBackend(execution, { runnerInstanceId: "runner-restart", transportFactory: () => restored!.transport }); + const events: PrpEvent[] = []; + const checkpoints: PersistedNativeSession[] = []; + let recoveryCheckpoint = persisted; + let failCheckpoint = crash !== "none"; + const interruptedController = new Error("controller lost before continuation submission"); + const execute = () => executeNativeSession({ + input: execution, backend: recovering, persistedSession: recoveryCheckpoint, resumeInterruptedTurn: true, + runnerInstanceId: "runner-restart", controlPlaneInstanceId: "controller-restart", timeoutMs: 15_000, + controlPlane: { + async openRun() {}, + async checkpointSession(snapshot) { + if (failCheckpoint && crash === "after-start" && snapshot.dispositionOnlyRecoveryTurnId === "provider-turn-2") { + failCheckpoint = false; + throw interruptedController; + } + checkpoints.push(structuredClone(snapshot)); + recoveryCheckpoint = structuredClone(snapshot); + if (failCheckpoint && crash === "before-start" && snapshot.terminal?.runTerminalState === "failed") { + failCheckpoint = false; + throw interruptedController; + } + }, + async appendEvent(event) { events.push(event); durableEvents.push(event); return { cursor: durableEvents.length, highestContiguousSourceSeq: event.sourceSeq, disposition: "committed" }; }, + async replayEvents(request) { + const source = durableEvents.filter(event => event.sourceInstanceId === request.sourceInstanceId); + return { events: source.filter(event => event.sourceSeq > request.afterSourceSeq), highestContiguousSourceSeq: source.at(-1)?.sourceSeq ?? 0 }; + }, + async completeRun() {}, + }, + }); + if (crash !== "none") { + await expect(execute()).rejects.toBe(interruptedController); + await restored.transport.close().catch(() => undefined); + restored = createRunnerdCodexTransport({ ...options, resumeDynamicTools: [], resumeActiveTurnId: recoveryCheckpoint.activeTurnId }); + recovering = createRunnerdNativeSessionBackend(execution, { runnerInstanceId: "runner-restart", transportFactory: () => restored!.transport }); + } + const completion = await execute(); + expect(completion).toMatchObject({ providerSessionId: persisted.providerSessionId, turnId: "provider-turn-2", terminal: { runTerminalState: "succeeded" } }); + expect(events.filter(event => event.eventType === "turn.failed")).toHaveLength(1); + expect(events.filter(event => event.eventType === "run.terminal")).toHaveLength(1); + if (crash !== "after-start") expect(checkpoints.some(snapshot => snapshot.activeTurnId === "provider-turn-2" && snapshot.terminal === null)).toBe(true); + expect((await readFile(join(root, "calls.log"), "utf8")).match(/^turn\/start$/gm)).toHaveLength(2); + const journal = JSON.parse(await readFile(join(root, "control-plane/control-plane-state.json"), "utf8")); + const submissions = journal.commands.filter((command: { type: string }) => command.type === "turn.start"); + expect(submissions).toHaveLength(2); + expect(submissions[1].payload.text).toContain("Reconcile any unfinished tool or command"); + } finally { + await restored?.transport.close().catch(() => undefined); + for (const bundle of [first, restored]) { + if (!bundle) continue; + const evidence = bundle.evidence(); + kill(evidence.runnerPid); + kill(evidence.providerPid); + } + await rm(root, { recursive: true, force: true }); + } +}, 30_000); diff --git a/packages/paperclip-runner/src/native-session-runtime.test.ts b/packages/paperclip-runner/src/native-session-runtime.test.ts index 5d3418e1cb..db0344becc 100644 --- a/packages/paperclip-runner/src/native-session-runtime.test.ts +++ b/packages/paperclip-runner/src/native-session-runtime.test.ts @@ -225,6 +225,71 @@ function highestContiguous(events: PrpEvent[]): number { } describe("executeNativeSession recovery", () => { + it.each([ + { label: "continues a restart-lost Codex turn", allowed: true, error: { code: "provider_turn_lost_on_restore", recoverable: true } }, + { label: "requires the control plane restart claim", allowed: false, error: { code: "provider_turn_lost_on_restore", recoverable: true } }, + { label: "preserves real provider errors", allowed: true, error: { code: "model_not_found", recoverable: false } }, + { label: "does not reopen historical error-null failures", allowed: true, error: null }, + ])("$label without replaying the task", async ({ label, allowed, error }) => { + const checkpoint: PersistedNativeSession = { + backendKind: "runner", driverKind: "codex_app_server", sessionId: "thread-recovery", identity, + providerSessionId: "provider-recovery", cursor: "0", activeTurnId: "turn-recovery", + terminalTurns: [], semanticResult: null, pendingRuntimeRequests: [], lineage: [], + }; + const failure = error; + const interrupted: PersistedNativeSession = { + ...checkpoint, activeTurnId: null, + terminalTurns: [{ turnId: "turn-recovery", fingerprint: JSON.stringify({ terminalState: "failed", error: failure, result: null }) }], + }; + const startTurn = vi.fn(async () => ({ turnId: "turn-continuation" })); + const session: NativeSession = { + identity: () => identity, + async capabilities() { return { resume: true, typedEvents: true, steering: false, interruption: true, structuredResult: true }; }, + async *events() { + yield runnerEvent(1, "turn.failed", { status: "failed", error: failure }); + if (!startTurn.mock.calls.length) return; + yield { ...runnerEvent(2, "turn.completed", { status: "completed" }), turnId: "turn-continuation" }; + }, + startTurn, + async result() { return startTurn.mock.calls.length ? { result, terminal, turnId: "turn-continuation" } : null; }, + async snapshot() { return structuredClone(interrupted); }, + async close() {}, + }; + const events: PrpEvent[] = []; + const checkpoints: PersistedNativeSession[] = []; + const backend: NativeSessionBackend = { + async descriptor() { return { kind: "runner", name: "restart", version: "1", capabilities: await session.capabilities() }; }, + async openSession() { throw new Error("must preserve the provider session"); }, + async recoverSession() { return { recovered: true, session }; }, + }; + const port: ControlPlanePort = { + async openRun() {}, + async checkpointSession(snapshot) { checkpoints.push(structuredClone(snapshot)); }, + async appendEvent(event) { events.push(event); return { cursor: events.length, highestContiguousSourceSeq: event.sourceSeq, disposition: "committed" }; }, + async replayEvents() { return { events: [], highestContiguousSourceSeq: 0 }; }, + async completeRun() {}, + }; + const execution = executeNativeSession({ + input, backend, controlPlane: port, persistedSession: checkpoint, + runnerInstanceId: "runner-recovery", controlPlaneInstanceId: "control-recovery", + resumeInterruptedTurn: allowed, + }); + if (label !== "continues a restart-lost Codex turn") { + await expect(execution).rejects.toMatchObject({ code: "native_provider_terminal_failed" }); + expect(startTurn).not.toHaveBeenCalled(); + return; + } + await expect(execution).resolves.toMatchObject({ result, turnId: "turn-continuation", terminal: { runTerminalState: "succeeded" } }); + expect(startTurn).toHaveBeenCalledOnce(); + const submitted = startTurn.mock.calls[0]![0] as Parameters[0]; + expect(submitted.continuation).toBe(true); + expect(submitted.message.text).toContain("Reconcile any unfinished tool or command"); + expect(submitted.message.text).not.toContain(input.task.prompt); + expect(events.filter(event => event.eventType === "turn.failed")).toHaveLength(1); + expect(events.filter(event => event.eventType === "run.terminal")).toHaveLength(1); + expect(checkpoints.some(snapshot => snapshot.terminalTurns?.some(turn => turn.turnId === "turn-recovery"))).toBe(true); + }); + it.each([undefined, 0, 7 * 24 * 60 * 60 * 1000, 30 * 24 * 60 * 60 * 1000])( "honors long-lived turn duration independently of operation bounds (%s)", async (turnTimeoutMs) => { @@ -6234,7 +6299,12 @@ describe("executeNativeSession recovery", () => { expect(openRun).not.toHaveBeenCalled(); }); - it.each([false, true])("replaces a provider session that already ended with a failed terminal (prepared: %s)", async (preparedMode) => { + it.each([ + { preparedMode: false, interrupted: false }, + { preparedMode: true, interrupted: false }, + { preparedMode: false, interrupted: true }, + { preparedMode: true, interrupted: true }, + ])("replaces a failed provider with full task context (prepared: $preparedMode, interrupted: $interrupted)", async ({ preparedMode, interrupted }) => { const digest = "0".repeat(64); const skill = { key: "company/recovery-skill", @@ -6286,6 +6356,7 @@ describe("executeNativeSession recovery", () => { } as const; const checkpoint: PersistedNativeSession = { backendKind: "mock", + driverKind: "codex_app_server", sessionId: "driver-failed", identity, providerSessionId: "provider-failed", @@ -6299,7 +6370,10 @@ describe("executeNativeSession recovery", () => { runTerminalState: "failed", reportedWorkDisposition: "yielded", }, - terminalTurns: [{ turnId: "turn-failed", fingerprint: "failed" }], + terminalTurns: [{ turnId: "turn-failed", fingerprint: interrupted ? JSON.stringify({ + terminalState: "failed", result: null, + error: { code: "provider_turn_lost_on_restore", recoverable: true }, + }) : "failed" }], pendingRuntimeRequests: [], lineage: [], }; @@ -6335,8 +6409,8 @@ describe("executeNativeSession recovery", () => { async close() {}, }; const recoverSession = vi.fn(async () => ({ - recovered: true as const, - session: replacementSession, + recovered: false as const, + reason: "provider thread is unavailable", })); const openReplacementSession = vi.fn(async () => replacementSession); const onContinuityBreak = vi.fn(async () => undefined); @@ -6397,7 +6471,7 @@ describe("executeNativeSession recovery", () => { ).resolves.toMatchObject({ providerSessionId: "provider-replacement" }); expect(getFreshSessionHandoff).toHaveBeenCalledOnce(); - expect(recoverSession).not.toHaveBeenCalled(); + expect(recoverSession).toHaveBeenCalledTimes(interrupted ? 1 : 0); expect(openReplacementSession).toHaveBeenCalledOnce(); const replacementEnvelope = JSON.parse( startTurn.mock.calls[0]![0].message.text, @@ -6419,7 +6493,7 @@ describe("executeNativeSession recovery", () => { expect(JSON.stringify(replacementEnvelope)).not.toContain("ONLY_NEW_COMMENT"); expect(startTurn.mock.calls[0]![0]).not.toHaveProperty("continuation"); expect(onContinuityBreak).toHaveBeenCalledWith({ - reason: "provider session ended with a failed terminal", + reason: interrupted ? "provider thread is unavailable" : "provider session ended with a failed terminal", previousDriverSessionId: "driver-failed", previousProviderSessionId: "provider-failed", replacementDriverSessionId: "driver-replacement", diff --git a/packages/paperclip-runner/src/native-session-runtime.ts b/packages/paperclip-runner/src/native-session-runtime.ts index 2ccfd7a4c1..e76409e7ab 100644 --- a/packages/paperclip-runner/src/native-session-runtime.ts +++ b/packages/paperclip-runner/src/native-session-runtime.ts @@ -23,6 +23,8 @@ import { NativeSessionCloseUnrecoverableError, NativeSessionCleanupQuarantinedError, NativeSessionProtocolIntegrityError, + isNativeRestartInterruption, + nativeRestartInterruptedTurnId, } from "./contracts/native-session-backend.js"; import { validatePrpStructuredRunResult, @@ -184,6 +186,8 @@ export interface NativeSessionGoalControl { } export interface ExecuteNativeSessionOptions { + /** The control plane proved the old local runner stopped and charged a recovery attempt. */ + resumeInterruptedTurn?: boolean; /** Read bounded task history only after recovery requires a fresh conversation. */ getFreshSessionHandoff?: () => Promise; /** Durable launch intent, after cleanup admission and before provider calls. */ @@ -865,6 +869,7 @@ async function consumeTurn( requestId: string; }, initialGoal?: HarnessThreadGoal | null, + resumeInterruptedTurn?: (event: PrpEvent, signal: AbortSignal) => Promise, ) { let timer: ReturnType | undefined; const appendAbort = new AbortController(); @@ -1174,6 +1179,8 @@ async function consumeTurn( } if (isTurnTerminal(event)) { if (providerFailure) throw providerFailure; + if (semanticResultProposal === null && governedResult === null && + inputTimers.size === 0 && await resumeInterruptedTurn?.(event, appendAbort.signal)) continue; return { event, eventCount, @@ -1871,6 +1878,12 @@ export async function executeNativeSession( previousProviderSessionId: string | null; } | null = null; let reconciledRecoveryCheckpoint: PersistedNativeSession | null = null; + const checkpointedInterruption = persistedSession && nativeRestartInterruptedTurnId(persistedSession); + if (checkpointedInterruption && persistedSession) { + // The turn failed because its process vanished; the conversation is still + // usable. Keep its terminal fingerprint so replay cannot execute it twice. + persistedSession = { ...persistedSession, terminal: null }; + } await options.onSessionAdmission?.(); if (options.existingSession) { if (options.existingSession.attachRun === undefined) { @@ -2157,6 +2170,42 @@ export async function executeNativeSession( await persistCheckpoint(snapshot, signal); }; const recoveredSnapshot = await session.snapshot(); + const restartContinuation = () => ({ + message: { role: "user" as const, text: JSON.stringify({ + schema: "paperclip.native-continuation.v1", + events: [ + "Paperclip restarted and restored this same conversation. The previous turn was interrupted.", + "Continue the current user request from the recorded progress and preserved workspace.", + "Reconcile any unfinished tool or command before proceeding: inspect its current state and recorded outcomes.", + "Do not repeat completed work or blindly rerun an action whose outcome is unknown. If an external action cannot be reconciled, report the specific blocker.", + ].join("\n"), + completion: { + revision: input.completionContract.contract.revision, + criterionIds: input.completionContract.contract.criteria.map(criterion => criterion.id), + }, + }) }, + continuation: true as const, + requestedCollaborationMode: "executionMode" in input ? input.executionMode : "default" as const, + }); + let restartContinuationStarted = false; + const resumeInterruptedTurn = async (event: PrpEvent, signal: AbortSignal) => { + if (restartContinuationStarted || !recovered || !options.resumeInterruptedTurn || + input.provider.kind !== "codex" || options.sessionGoalControl || options.resumeSessionGoalHeartbeat || + recoveredSnapshot.goal || persistedSession?.semanticResult || persistedSession?.pendingRuntimeRequests?.length || + event.eventType !== "turn.failed" || !isNativeRestartInterruption(event.payload.error) || + event.turnId !== persistedSession?.activeTurnId) return false; + const snapshot = await session.snapshot(); + signal.throwIfAborted(); + if (nativeRestartInterruptedTurnId(snapshot) !== event.turnId || snapshot.pendingRuntimeRequests?.length) return false; + restartContinuationStarted = true; + // Persist the old terminal before launching. Recovery inspects provider + // history to adopt an accepted continuation if its checkpoint was lost. + await persistCheckpoint(snapshot, signal); + signal.throwIfAborted(); + await session.startTurn(restartContinuation()); + await checkpoint(signal); + return true; + }; const recoveredActiveTurnId = recovered ? (recoveredSnapshot.activeTurnId ?? null) : (persistedSession?.activeTurnId ?? null); @@ -2294,6 +2343,7 @@ export async function executeNativeSession( } : undefined, recoveredSnapshot.goal, + resumeInterruptedTurn, ) : Promise.resolve({ event: recoveryTerminal, @@ -2333,6 +2383,10 @@ export async function executeNativeSession( await session.goal({ action: "get", requestId }); } await checkpoint(); + } else if (shouldStartFreshTurn && recovered && checkpointedInterruption) { + restartContinuationStarted = true; + await session.startTurn(restartContinuation()); + await checkpoint(); } else if (shouldStartFreshTurn) { let modelEnvelope = recovered ? buildNativeModelEnvelope(input, { resumedSession: true }) diff --git a/packages/paperclip-runner/test/fixtures/semantic-input-digests.json b/packages/paperclip-runner/test/fixtures/semantic-input-digests.json new file mode 100644 index 0000000000..db60546267 --- /dev/null +++ b/packages/paperclip-runner/test/fixtures/semantic-input-digests.json @@ -0,0 +1,42 @@ +[ + { + "name": "plain document", + "input": { + "body": "Plain fixture document" + }, + "digest": "sha256:2ba38770004c4e7672a5097c4d8e3241c62750b21e034903796fb035fcf9a2a4" + }, + { + "name": "protected fields", + "input": { + "password": "fixture-only-not-a-real-credential", + "safe": true + }, + "digest": "sha256:98ecc80f0e6d2d4b99ce3620d18a21392356f99def55ed599c1b1c002ccd2d47" + }, + { + "name": "credential-like document text", + "input": { + "body": "Example Bearer fixture_token_123456\nhttps://example.test/callback?token=fixture-only" + }, + "digest": "sha256:aa7f429cc5dd63e85fd11aa93e0659fa1c7d5041f5f35843fef62092ff86fdc0" + }, + { + "name": "canonical Unicode and numbers", + "input": { + "": "private-use key", + "😀": "astral key", + "values": [ + -0, + 0.000001, + 9.99999e-7, + 100000000000000000000, + 1e+21, + true, + null, + "日本語\n\"text\"" + ] + }, + "digest": "sha256:3104a5e5db1fe2c6dcdaa0bfcf362d7bd36441e7e08905a48ae03be06eea9dc4" + } +] diff --git a/packages/plugins/sandbox-providers/exe-dev/README.md b/packages/plugins/sandbox-providers/exe-dev/README.md index 52d420c25d..f1091e4896 100644 --- a/packages/plugins/sandbox-providers/exe-dev/README.md +++ b/packages/plugins/sandbox-providers/exe-dev/README.md @@ -23,7 +23,7 @@ Configure exe.dev from `Instance Settings -> Environments`, not from the plugin' To use the provider successfully, the environment/host needs all of the following: -- An exe.dev API token that allows the lifecycle commands the provider uses: `new`, `ls`, and `rm`. `whoami` and `help` are recommended for manual debugging. `restart` is only needed if you extend the provider to restart retained VMs. +- An exe.dev API token that allows the lifecycle commands the provider uses: `new`, `ls`, and `rm`, plus `cp` when `sourceVm` is set. exe.dev answers 403 for a command the token does not list. `whoami` and `help` are recommended for manual debugging. `restart` is only needed if you extend the provider to restart retained VMs. - SSH access from the Paperclip host to the resulting `*.exe.xyz` VMs. - An SSH private key that exe.dev already recognizes. You can either: - paste the private key into the environment config via `sshPrivateKey` @@ -37,6 +37,7 @@ Operational notes: - Reusable leases keep the VM alive between runs. exe.dev does not expose a documented "stop and later resume" command in the public CLI docs, so `reuseLease: true` means "retain the VM" rather than "suspend it." - The provisioning path uses `https://exe.dev/exec`, which exe.dev documents as a command-style HTTPS API with a 30-second request timeout. Typical `new` calls are expected to fit inside that limit; command execution itself does not use `/exec`. - Probes still create and delete a real exe.dev VM through `/exec`, and so do the `new`/`rm` calls inside the normal acquire/release lifecycle. Treat all of those as real provisioning cost, not just probes. +- Set `sourceVm` ("Source VM" in the form) to the name of an existing exe.dev VM to copy it for each run with `exe.dev cp`, disk and config included, instead of creating a fresh VM with `exe.dev new`. This lets you prepare one VM with your toolchain and caches and start every run from it. `cp` accepts only the VM name, `cpu`, `memory`, and `disk`, so config validation rejects `sourceVm` together with `image`, `command`, `comment`, `env`, `integrations`, `tags`, `setupScript`, or `prompt`. The default setup script does not run either, so the source VM must already have Node 24.11+ and accept the configured SSH key. Do not keep secrets on the source VM: every copy inherits its disk. `cp` also goes through `/exec`, so a large disk copy must finish inside the same 30-second limit. - exe.dev runs `--setup-script` as the unprivileged `exedev` user, not as root. That user has passwordless `sudo`, so any system-level steps in a custom `setupScript` must invoke `sudo` explicitly (for example `sudo apt-get install -y …`). When you omit `setupScript`, the plugin supplies a default that installs Node 24 via the official nodesource script — Paperclip's sandbox callback bridge is a Node program, so the VM needs `node` on `PATH` before the bridge can launch. ## Local development diff --git a/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts b/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts index bef86f3def..da176d0f96 100644 --- a/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts +++ b/packages/plugins/sandbox-providers/exe-dev/src/manifest.ts @@ -31,7 +31,7 @@ const manifest: PaperclipPluginManifestV1 = { type: "string", format: "secret-ref", description: - "Paste your exe.dev API token, or pick a saved Paperclip secret. Create one at exe.dev → Settings → API tokens with `/exec` scope (`new`, `ls`, `rm`).", + "Paste your exe.dev API token, or pick a saved Paperclip secret. Create one at exe.dev → Settings → API tokens with `/exec` scope (`new`, `ls`, `rm`, plus `cp` if you use Source VM).", }, sshPrivateKey: { type: "string", @@ -99,6 +99,14 @@ const manifest: PaperclipPluginManifestV1 = { "x-paperclip-group": "VM resources", }, // ---- Advanced: VM creation ---- + sourceVm: { + type: "string", + title: "Source VM", + description: + "Name of an existing exe.dev VM to copy for each run with `exe.dev cp`, disk and config included. Leave blank to create a fresh VM with `exe.dev new`. Your API token must allow `cp`: tokens list their permitted commands, and one without `cp` fails with a 403. When set, leave image, command, env, integrations, tags, setup script, prompt, and comment empty: `cp` cannot apply them, and saving fails until they are cleared.", + "x-paperclip-advanced": true, + "x-paperclip-group": "VM creation", + }, command: { type: "string", description: "Optional container command passed to `exe.dev new --command`.", diff --git a/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts b/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts index 16618bf507..912105db85 100644 --- a/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts +++ b/packages/plugins/sandbox-providers/exe-dev/src/plugin.test.ts @@ -106,6 +106,7 @@ describe("exe.dev sandbox provider plugin", () => { integrations: ["github"], tags: ["prod", "sandbox"], setupScript: null, + sourceVm: null, prompt: null, timeoutMs: 450000, reuseLease: true, @@ -319,6 +320,57 @@ describe("exe.dev sandbox provider plugin", () => { }); }); + it("copies the source VM with exe.dev cp instead of creating a new VM", async () => { + fetchMock.mockResolvedValueOnce( + new Response(JSON.stringify({ + vm_name: "paperclip-env1-run1", + ssh_dest: "paperclip-env1-run1.exe.xyz", + status: "running", + }), { status: 200 }), + ); + queueSpawnResult({ stdout: "/home/exedev\nbash\n" }); + queueSpawnResult({}); + + const lease = await plugin.definition.onEnvironmentAcquireLease?.({ + driverKey: "exe-dev", + companyId: "company-1", + environmentId: "env-1", + runId: "run-1", + config: { + apiKey: "api-key", + sourceVm: " golden-vm ", + image: "ubuntu:22.04", + env: { FOO: "bar" }, + cpu: 4, + memory: "8GB", + disk: "40GB", + }, + }); + + expect(String(fetchMock.mock.calls[0]?.[1]?.body ?? "")).toBe( + "cp 'golden-vm' 'paperclip-env1-run1' --json --cpu='4' --memory='8GB' --disk='40GB'", + ); + expect(lease).toMatchObject({ providerLeaseId: "paperclip-env1-run1" }); + }); + + it("rejects VM creation settings that exe.dev cp cannot apply when sourceVm is set", async () => { + const result = await plugin.definition.onEnvironmentValidateConfig?.({ + driverKey: "exe-dev", + config: { + apiKey: "api-key", + sourceVm: "golden-vm", + image: "ubuntu:22.04", + env: { FOO: "bar" }, + setupScript: "echo hi", + }, + }); + + expect(result?.ok).toBe(false); + expect(result?.errors).toContain( + "sourceVm copies an existing VM with `exe.dev cp`, which cannot apply image, env, setupScript. Clear these settings or clear sourceVm.", + ); + }); + it("uses a pasted sshPrivateKey when connecting to the VM", async () => { fetchMock.mockResolvedValueOnce( new Response(JSON.stringify({ @@ -860,6 +912,12 @@ describe("exe-dev manifest form defaults", () => { expect(properties.disk?.default).toBe("20GB"); }); + it("explains the cp token permission on the Source VM field", () => { + const sourceVm = properties.sourceVm as { title?: string; description?: string } | undefined; + expect(sourceVm?.title).toBe("Source VM"); + expect(sourceVm?.description).toMatch(/token must allow `cp`.*403/); + }); + it("declares no default on secret-ref fields, which would be persisted as a company secret", () => { for (const prop of Object.values(properties)) { if (prop.format === "secret-ref") { diff --git a/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts b/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts index a586b665f4..d45f562fc0 100644 --- a/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts +++ b/packages/plugins/sandbox-providers/exe-dev/src/plugin.ts @@ -24,6 +24,7 @@ interface ExeDevDriverConfig { apiKey: string | null; apiUrl: string; namePrefix: string; + sourceVm: string | null; image: string | null; command: string | null; cpu: number | null; @@ -246,6 +247,7 @@ function parseDriverConfig(raw: Record): ExeDevDriverConfig { apiKey: parseOptionalString(raw.apiKey), apiUrl: normalizeApiUrl(parseOptionalString(raw.apiUrl)), namePrefix: normalizeNamePrefix(parseOptionalString(raw.namePrefix)), + sourceVm: parseOptionalString(raw.sourceVm), image: parseOptionalString(raw.image), command: parseOptionalString(raw.command), cpu: parseOptionalInteger(raw.cpu), @@ -313,10 +315,23 @@ function resolveSetupScript(config: ExeDevDriverConfig): string | null { return trimmed.length > 0 ? config.setupScript : null; } +function buildCopyCommand(sourceVm: string, config: ExeDevDriverConfig, vmName: string): string { + return [ + "cp", + shellQuote(sourceVm), + shellQuote(vmName), + "--json", + ...buildFlag("cpu", config.cpu), + ...buildFlag("memory", config.memory), + ...buildFlag("disk", config.disk), + ].join(" "); +} + function buildCreateCommand( config: ExeDevDriverConfig, vmName: string, ): string { + if (config.sourceVm) return buildCopyCommand(config.sourceVm, config, vmName); return [ "new", "--json", @@ -774,6 +789,23 @@ const plugin = definePlugin({ warnings.push( "The Paperclip host must have SSH access to the created exe.dev VM, and its SSH key must be registered with exe.dev. The API token only covers provisioning.", ); + if (config.sourceVm) { + const ignored = Object.entries({ + image: config.image, + command: config.command, + comment: config.comment, + env: Object.keys(config.env).length > 0, + integrations: config.integrations.length > 0, + tags: config.tags.length > 0, + setupScript: config.setupScript, + prompt: config.prompt, + }).filter(([, value]) => value).map(([key]) => key); + if (ignored.length > 0) { + errors.push( + `sourceVm copies an existing VM with \`exe.dev cp\`, which cannot apply ${ignored.join(", ")}. Clear these settings or clear sourceVm.`, + ); + } + } if (config.reuseLease) { warnings.push("reuseLease keeps the VM alive between runs; this provider does not suspend retained VMs."); } diff --git a/server/src/__tests__/activity-routes.test.ts b/server/src/__tests__/activity-routes.test.ts index e6d9d61113..45871e3236 100644 --- a/server/src/__tests__/activity-routes.test.ts +++ b/server/src/__tests__/activity-routes.test.ts @@ -277,6 +277,26 @@ describe.sequential("activity routes", () => { expect(mockActivityService.create).not.toHaveBeenCalled(); }); + it.each([null, { id: "issue-uuid-1", companyId: "company-2" }])( + "does not start run reads or retries for a missing or cross-company task", async (issue) => { + mockIssueService.getByIdentifier.mockResolvedValue(issue); + const app = await createApp(); + const res = await request(app).get("/api/issues/TEST-1/runs"); + expect(res.status).toBe(404); + expect(res.body).toEqual({ error: "Issue not found" }); + expect(mockActivityService.runsForIssue).not.toHaveBeenCalled(); + }, + ); + + it("does not start run reads or retries when task read authorization is denied", async () => { + mockIssueService.getByIdentifier.mockResolvedValue({ id: "issue-uuid-1", companyId: "company-1" }); + mockAccessService.decide.mockResolvedValue({ allowed: false, reason: "denied", explanation: "Denied by test policy." }); + const app = await createApp(); + const res = await request(app).get("/api/issues/TEST-1/runs"); + expect(res.status).toBe(403); + expect(mockActivityService.runsForIssue).not.toHaveBeenCalled(); + }); + it("returns 200 [] (not 404) when listing issues for another company's run, preserving API contract and the cross-tenant oracle", async () => { mockHeartbeatService.getRun.mockResolvedValue({ id: "run-2", diff --git a/server/src/__tests__/activity-service.test.ts b/server/src/__tests__/activity-service.test.ts index 328b61acff..6f1a960163 100644 --- a/server/src/__tests__/activity-service.test.ts +++ b/server/src/__tests__/activity-service.test.ts @@ -150,7 +150,7 @@ describeEmbeddedPostgres("activity service", () => { expect(result.map((event) => event.action)).toEqual(["test.newest", "test.middle"]); }); - it("returns compact usage and result summaries for issue runs", async () => { + it.each([null, "native_provider_model_rejected", "adapter_failed"])("returns compact issue runs with bounded model rejection details: %s", async (errorCode) => { const companyId = randomUUID(); const agentId = randomUUID(); const issueId = randomUUID(); @@ -180,7 +180,9 @@ describeEmbeddedPostgres("activity service", () => { companyId, agentId, invocationSource: "assignment", - status: "succeeded", + status: errorCode ? "failed" : "succeeded", + errorCode, + error: "provider-error".repeat(200), contextSnapshot: { issueId }, usageJson: { inputTokens: 11, @@ -215,6 +217,7 @@ describeEmbeddedPostgres("activity service", () => { agentId, invocationSource: "assignment", contextIssueId: issueId, + error: errorCode === "native_provider_model_rejected" ? "provider-error".repeat(200).slice(0, 2000) : null, }); expect(runs[0]?.usageJson).toEqual({ inputTokens: 11, diff --git a/server/src/__tests__/agent-default-configure-grants.test.ts b/server/src/__tests__/agent-default-configure-grants.test.ts new file mode 100644 index 0000000000..e67f68a545 --- /dev/null +++ b/server/src/__tests__/agent-default-configure-grants.test.ts @@ -0,0 +1,240 @@ +import { randomUUID } from "node:crypto"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { eq } from "drizzle-orm"; +import { agents, companies, companyMemberships, createDb, principalPermissionGrants } from "@paperclipai/db"; +import { LOW_TRUST_REVIEW_PRESET, type PermissionKey } from "@paperclipai/shared"; +import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { agentService } from "../services/agents.js"; +import { authorizationService } from "../services/authorization.js"; +import { accessService } from "../services/access.js"; +import { agentJoinGrantsFromDefaults } from "../services/invite-grants.js"; + +const support = await getEmbeddedPostgresTestSupport(); +const describeDatabase = support.supported ? describe : describe.skip; +const expectedNewAgentGrantKeys: PermissionKey[] = [ + "agents:configure", + "agents:suggest-changes", + "skills:create", + "skills:suggest-changes", + "tools:manage_connections", + "tools:manage_profiles", + "tools:view_audit", + "audit:view_agent_actions", + "tools:use", + "tools:manage_runtime", + "inbox:manage", + "tasks:assign", + "tasks:assign_scope", + "tasks:manage_active_checkouts", +].sort(); + +describeDatabase("new agent configuration defaults", () => { + let db!: ReturnType; + let cleanup: (() => Promise) | undefined; + + beforeAll(async () => { + const started = await startEmbeddedPostgresTestDatabase("agent-configure-defaults"); + db = createDb(started.connectionString); + cleanup = started.cleanup; + }, 20_000); + + afterAll(async () => { + await cleanup?.(); + }); + + it("grants standard new agents direct peer configuration, but keeps restricted agents narrow", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Agent permission defaults", + issuePrefix: `PD${companyId.slice(0, 6).toUpperCase()}`, + }); + const create = (name: string, permissions: Record = {}, metadata?: Record) => + agentService(db).create(companyId, { + name, + role: "engineer", + adapterType: "process", + adapterConfig: {}, + runtimeConfig: {}, + permissions, + metadata, + }, metadata ? { allowBuiltInAgentMetadata: true } : undefined); + + const standard = await create("Standard"); + const peer = await create("Peer"); + const lowTrust = await create("Low trust", { trustPreset: LOW_TRUST_REVIEW_PRESET }); + const bundled = await create("Bundled", {}, { + paperclipBuiltInAgent: { key: "reflection-coach", featureKeys: [] }, + }); + + const grants = await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId)); + for (const agent of [standard, peer]) { + expect(grants.filter((grant) => grant.principalId === agent.id) + .map((grant) => grant.permissionKey).sort()).toEqual(expectedNewAgentGrantKeys); + expect(grants.find((grant) => grant.principalId === agent.id && grant.permissionKey === "tasks:assign_scope")?.scope) + .toEqual({ subtreeRootAgentId: agent.id }); + expect(grants.find((grant) => grant.principalId === agent.id && grant.permissionKey === "inbox:manage")?.scope) + .toEqual({ responsibleUserOnly: true }); + } + for (const agent of [lowTrust, bundled]) { + expect(grants.filter((grant) => grant.principalId === agent.id)).toEqual([]); + } + + await db.insert(companyMemberships).values({ + companyId, + principalType: "agent", + principalId: standard.id, + status: "active", + membershipRole: "member", + }); + + for (const permissionKey of expectedNewAgentGrantKeys) { + if (permissionKey === "tasks:assign_scope" || permissionKey === "inbox:manage") continue; // These grants require a target scope or the responsible-user inbox path. + expect(await accessService(db).hasPermission(companyId, "agent", standard.id, permissionKey)).toBe(true); + } + expect(await authorizationService(db).decidePrincipalGrant({ + companyId, + principalType: "agent", + principalId: standard.id, + action: "tasks:assign", + permissionKey: "tasks:assign_scope", + scope: { assigneeAgentId: standard.id }, + })).toMatchObject({ allowed: true }); + expect(await authorizationService(db).decidePrincipalGrant({ + companyId, + principalType: "agent", + principalId: standard.id, + action: "tasks:assign", + permissionKey: "tasks:assign_scope", + scope: { assigneeAgentId: peer.id }, + })).toMatchObject({ allowed: false, reason: "deny_scope" }); + for (const permissionKey of [ + "agents:create", "environments:manage", "tools:admin", "users:invite", + "users:manage_permissions", "pipelines:write", "joins:approve", + ] as const) { + expect(await accessService(db).hasPermission(companyId, "agent", standard.id, permissionKey)).toBe(false); + } + + const decision = await authorizationService(db).decide({ + actor: { type: "agent", agentId: standard.id, companyId, source: "agent_key" }, + action: "agent_config:update", + resource: { type: "agent", agentId: peer.id, companyId }, + scope: { requiresChangeGrant: true }, + }); + expect(decision).toMatchObject({ allowed: true, reason: "allow_direct_change" }); + + // A later ordinary self-update must not recreate a grant removed by an operator. + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, standard.id)); + await agentService(db).update(standard.id, { title: "Updated own title" }); + expect(await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.principalId, standard.id))).toEqual([]); + + await agentService(db).remove(peer.id); + expect(await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.principalId, peer.id))).toEqual([]); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(companyMemberships).where(eq(companyMemberships.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); + + it("keeps existing agents and their scoped grants unchanged during ordinary updates", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Existing permission defaults", + issuePrefix: `PE${companyId.slice(0, 6).toUpperCase()}`, + }); + const rows = await db.insert(agents).values([ + { companyId, name: "Existing", role: "engineer", permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Scoped", role: "engineer", permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Restricted", role: "engineer", permissions: { trustPreset: LOW_TRUST_REVIEW_PRESET }, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Built in", role: "engineer", metadata: { paperclipBuiltInAgent: { key: "reflection-coach", featureKeys: [] } }, permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + { companyId, name: "Pending", role: "engineer", status: "pending_approval", permissions: {}, adapterType: "process", adapterConfig: {}, runtimeConfig: {} }, + ]).returning(); + await db.insert(principalPermissionGrants).values({ + companyId, + principalType: "agent", + principalId: rows[1]!.id, + permissionKey: "agents:configure", + scope: { agentIds: [rows[0]!.id] }, + }); + + for (const row of rows) { + if (row.status === "pending_approval") { + await expect(agentService(db).update(row.id, { title: "Updated existing agent" })) + .rejects.toThrow("Pending approval agent configuration cannot be changed before board approval"); + } else { + await agentService(db).update(row.id, { title: "Updated existing agent" }); + } + } + const grants = await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId)); + expect(grants.map((grant) => grant.principalId).sort()).toEqual([rows[1]!.id]); + expect(grants.map((grant) => grant.permissionKey)).toEqual(["agents:configure"]); + expect(grants.find((grant) => grant.principalId === rows[1]!.id)?.scope).toEqual({ agentIds: [rows[0]!.id] }); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); + + it("adds the default grant when a standard pending hire is approved", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Pending permission default", + issuePrefix: `PP${companyId.slice(0, 6).toUpperCase()}`, + }); + const pending = await agentService(db).create(companyId, { + name: "Pending hire", + status: "pending_approval", + role: "engineer", + adapterType: "process", + adapterConfig: {}, + runtimeConfig: {}, + }); + expect(await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId))).toEqual([]); + + const result = await agentService(db).activatePendingApproval(pending.id); + expect(result?.activated).toBe(true); + expect((await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.companyId, companyId))) + .map((grant) => grant.permissionKey).sort()).toEqual(expectedNewAgentGrantKeys); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); + + it("keeps configuration access when invitation approval replaces a new agent's grants", async () => { + const companyId = randomUUID(); + await db.insert(companies).values({ + id: companyId, + name: "Invited permission default", + issuePrefix: `PI${companyId.slice(0, 6).toUpperCase()}`, + }); + const invited = await agentService(db).create(companyId, { + name: "Invited agent", + role: "engineer", + adapterType: "process", + adapterConfig: {}, + runtimeConfig: {}, + }); + await accessService(db).ensureMembership(companyId, "agent", invited.id, "member", "active"); + await accessService(db).setPrincipalGrants( + companyId, "agent", invited.id, agentJoinGrantsFromDefaults(null, invited.id), null, + ); + const grants = await db.select().from(principalPermissionGrants) + .where(eq(principalPermissionGrants.principalId, invited.id)); + expect(grants.map((grant) => grant.permissionKey).sort()).toEqual(expectedNewAgentGrantKeys); + + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.companyId, companyId)); + await db.delete(companyMemberships).where(eq(companyMemberships.companyId, companyId)); + await db.delete(agents).where(eq(agents.companyId, companyId)); + await db.delete(companies).where(eq(companies.id, companyId)); + }); +}); diff --git a/server/src/__tests__/agent-hire-ai-connections.test.ts b/server/src/__tests__/agent-hire-ai-connections.test.ts index 363a2e92eb..fb30faa12f 100644 --- a/server/src/__tests__/agent-hire-ai-connections.test.ts +++ b/server/src/__tests__/agent-hire-ai-connections.test.ts @@ -28,6 +28,7 @@ beforeAll(async () => { home = await mkdtemp(path.join(os.tmpdir(), "paperclip-hire-ai-")); vi.stubEnv("PAPERCLIP_HOME", home); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "hire-ai"); + vi.stubEnv("PAPERCLIP_IN_WORKTREE", "false"); database = await startEmbeddedPostgresTestDatabase("paperclip-hire-ai-db-"); db = createDb(database.connectionString); }, 90_000); @@ -312,15 +313,15 @@ describe("agent-created hires use managed AI connections", () => { ["ANTHROPIC_API_KEY", ""], ["CLAUDE_CONFIG_DIR", "/tmp/child-claude-home"], ["ANTHROPIC_BASE_URL", "https://example.invalid"], - ])(`${endpoint}: preserves an explicit child auth setting %s=%s`, async (key, value) => { + ])(`${endpoint}: rejects an agent-supplied local environment setting %s=%s`, async (key, value) => { const f = await fixture("anthropic"); - const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ + const response = await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ name: "Explicit auth", role: "engineer", adapterType: f.adapterType, adapterConfig: { env: { [key]: value } }, - })); - expect(agent.runtimeConfig.aiConnection).toBeUndefined(); - const [saved] = await db.select().from(agents).where(eq(agents.id, agent.id)); - expect((saved.adapterConfig.env as Record)[key]).toEqual({ type: "plain", value }); + }); + expect(response.status).toBe(403); + expect(response.body.error).toContain("host-executed local adapter settings"); + expect(await db.select().from(agents).where(eq(agents.companyId, f.companyId))).toHaveLength(1); }); } @@ -368,13 +369,17 @@ describe("agent-created hires use managed AI connections", () => { ["openai", "paperclip_runner", { provider: "acpx", acpxAgent: "claude" }, "OPENAI_API_KEY"], ] as const)(`${endpoint}: ignores the %s auth key for a different provider in %s`, async (provider, adapterType, config, key) => { const f = await fixture(provider); - const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ + const response = await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ name: "Cross-provider config", role: "engineer", adapterType, adapterConfig: { ...config, env: { [key]: "leftover-parent-setting" } }, - })); - expect(agent.runtimeConfig.aiConnection).toMatchObject({ - provider: provider === "anthropic" ? "openai" : "anthropic", mode: "responsible_user", }); + if (adapterType.endsWith("_local")) { + expect(response.status).toBe(403); + expect(response.body.error).toContain("host-executed local adapter settings"); + return; + } + const agent = hired(response); + expect(agent.runtimeConfig.aiConnection).toMatchObject({ provider: provider === "anthropic" ? "openai" : "anthropic", mode: "responsible_user" }); }); } @@ -467,7 +472,9 @@ describe("agent-created hires use managed AI connections", () => { describe("hired agents sharing a subscription", () => { it.each(["openai", "anthropic"] as const)("runs the %s child alongside a live parent and inherits its connection", async (provider) => { const f = await fixture(provider, "subscription"); - const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Concurrent teammate", role: "engineer", adapterType: f.adapterType, reportsTo: f.agentId, adapterConfig: { cwd: home, engine: "cli" }, runtimeConfig: { heartbeat: { enabled: false } } })); + const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Concurrent teammate", role: "engineer", adapterType: f.adapterType, reportsTo: f.agentId, adapterConfig: { engine: "cli" }, runtimeConfig: { heartbeat: { enabled: false } } })); + // Host working directories are configured by an operator, not an agent key. + await db.update(agents).set({ adapterConfig: { ...agent.adapterConfig, cwd: home } }).where(eq(agents.id, agent.id)); const [issue] = await db.insert(issues).values({ companyId: f.companyId, title: "Subscription child task", status: "todo", assigneeAgentId: agent.id, responsibleUserId: f.userId, createdByUserId: f.userId }).returning(); const parentRuntime = await prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: f.agentId, responsibleUserId: f.userId, adapterType: f.adapterType, binding: f.binding, config: { cwd: home } }); const execute = vi.fn(async () => { diff --git a/server/src/__tests__/agent-live-run-routes.test.ts b/server/src/__tests__/agent-live-run-routes.test.ts index 55d978e079..d5d1805a69 100644 --- a/server/src/__tests__/agent-live-run-routes.test.ts +++ b/server/src/__tests__/agent-live-run-routes.test.ts @@ -23,6 +23,7 @@ const mockIssueService = vi.hoisted(() => ({ getById: vi.fn(), getByIdentifier: vi.fn(), })); +const mockNativeReviewAssignment = vi.hoisted(() => vi.fn()); const mockExecutionProjection = vi.hoisted(() => ({ executionProjectionForRun: vi.fn(async () => null), @@ -79,6 +80,10 @@ const mockChatRunRetries = vi.hoisted(() => ({ })); function registerModuleMocks() { + vi.doMock("../services/native-runtime/native-review-participant.js", async () => ({ + ...(await vi.importActual("../services/native-runtime/native-review-participant.js")), + getNativeReviewAssignment: mockNativeReviewAssignment, + })); vi.doMock("../services/execution-projection.js", () => mockExecutionProjection); vi.doMock("../routes/authz.js", async () => vi.importActual("../routes/authz.js"), @@ -287,6 +292,7 @@ describe("agent live run routes", () => { vi.clearAllMocks(); mockChatRunRetries.prepareFailedChatRunRetry.mockReset(); mockChatRunRetries.processFailedChatRunRetry.mockReset(); + mockNativeReviewAssignment.mockReset().mockResolvedValue(null); mockAccessService.canUser.mockResolvedValue(true); mockAccessService.decide.mockImplementation(async (input: { action?: string }) => ({ allowed: true, @@ -966,6 +972,68 @@ describe("agent live run routes", () => { expect(mockHeartbeatService.wakeup).toHaveBeenCalledWith(routeAgentId, expect.objectContaining({ manualUserWake: true })); }); + describe("exact native model-rejected review retry", () => { + const reviewContext = { + nativeReviewInteractionId: "55555555-5555-4555-8555-555555555555", + nativeReviewDecisionId: "66666666-6666-4666-8666-666666666666", + }; + const selectedRun = { + id: failedChatRunId, companyId: "company-1", agentId: routeAgentId, + status: "failed", runtimeMode: "native", errorCode: "native_provider_model_rejected", + nativeIssueId: failedChatIssueId, + contextSnapshot: { issueId: failedChatIssueId, ...reviewContext }, + }; + const selectedIssue = { id: failedChatIssueId, companyId: "company-1", status: "in_review", assigneeAgentId: "worker", assigneeUserId: null }; + const retryBody = { failedRunId: failedChatRunId, reason: "retry_failed_run" }; + beforeEach(() => { + mockAgentService.getById.mockResolvedValue({ id: routeAgentId, companyId: "company-1" }); + mockHeartbeatService.getRun.mockResolvedValue(selectedRun); + mockIssueService.getById.mockResolvedValue(selectedIssue); + mockNativeReviewAssignment.mockResolvedValue({ interaction: { status: "pending" } }); + }); + + it("permits the Board to retry the exact pending review using only saved context", async () => { + const fixture = createFailedChatRetryDb(false); + const res = await requestApp(await createApp(fixture.db), url => request(url).post(`/api/agents/${routeAgentId}/wakeup`).send({ + ...retryBody, payload: { issueId: "forged", nativeReviewInteractionId: "forged", nativeReviewDecisionId: "forged" }, + })); + expect(res.status, JSON.stringify(res.body)).toBe(202); + expect(mockNativeReviewAssignment).toHaveBeenCalledExactlyOnceWith(fixture.db, { + companyId: "company-1", issueId: failedChatIssueId, agentId: routeAgentId, contextSnapshot: reviewContext, + }); + expect(mockHeartbeatService.wakeup).toHaveBeenCalledExactlyOnceWith(routeAgentId, expect.objectContaining({ + failedRunId: failedChatRunId, reason: "retry_failed_run", payload: { issueId: failedChatIssueId }, + requestedByActorType: "user", contextSnapshot: expect.objectContaining(reviewContext), + })); + expect(JSON.stringify(mockHeartbeatService.wakeup.mock.calls)).not.toContain("forged"); + }); + + it.each(["agent-caller", "denied-task", "foreign-company", "foreign-agent", "foreign-issue", "legacy", "unknown-error", "missing-context", "stale", "resolved", "changed-reviewer"])( + "rejects an ineligible exact review retry (%s)", async scenario => { + let actor: Record | undefined; + if (scenario === "agent-caller") actor = { type: "agent", agentId: routeAgentId, companyId: "company-1" }; + if (scenario === "denied-task") mockAccessService.decide.mockImplementation(async ({ action }) => ({ allowed: action !== "issue:comment", explanation: "Denied" })); + if (scenario === "foreign-company") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, companyId: "company-2" }); + if (scenario === "foreign-agent") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, agentId: "other-reviewer" }); + if (scenario === "foreign-issue") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, nativeIssueId: "other-task" }); + if (scenario === "legacy") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, runtimeMode: "legacy" }); + if (scenario === "unknown-error") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, errorCode: "adapter_failed" }); + if (scenario === "missing-context") mockHeartbeatService.getRun.mockResolvedValue({ ...selectedRun, contextSnapshot: { issueId: failedChatIssueId } }); + if (scenario === "stale" || scenario === "changed-reviewer") mockNativeReviewAssignment.mockResolvedValue(null); + if (scenario === "resolved") mockNativeReviewAssignment.mockResolvedValue({ interaction: { status: "accepted" } }); + const fixture = createFailedChatRetryDb(false); + const res = await requestApp(await createApp(fixture.db, actor), url => request(url).post(`/api/agents/${routeAgentId}/wakeup`).send({ + ...retryBody, payload: { ...reviewContext }, + })); + expect(res.status).toBe(["agent-caller", "denied-task"].includes(scenario) ? 403 : ["foreign-company", "foreign-agent"].includes(scenario) ? 404 : 409); + expect(mockHeartbeatService.wakeup).not.toHaveBeenCalled(); + if (["agent-caller", "denied-task", "foreign-company", "foreign-agent", "foreign-issue", "legacy", "unknown-error", "missing-context"].includes(scenario)) { + expect(mockNativeReviewAssignment).not.toHaveBeenCalled(); + } + }, + ); + }); + describe("exact failed chat run retry", () => { const retryBody = { failedRunId: failedChatRunId, diff --git a/server/src/__tests__/agent-permissions-routes.test.ts b/server/src/__tests__/agent-permissions-routes.test.ts index 2997c9b928..4322844170 100644 --- a/server/src/__tests__/agent-permissions-routes.test.ts +++ b/server/src/__tests__/agent-permissions-routes.test.ts @@ -918,6 +918,217 @@ describe.sequential("agent permission routes", () => { expect(mockLogActivity).not.toHaveBeenCalled(); }); + it.each([ + ["command", { command: "sh" }], + ["arguments", { args: ["-c", "id"] }], + ["environment", { env: { PATH: "/tmp" } }], + ])("blocks agent-authenticated process adapter %s updates", async (_label, adapterConfig) => { + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it("blocks an agent from switching a peer onto the process adapter", async () => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType: "codex_local" }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterType: "process" })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it.each([ + ["claude_local", { engine: "cli", command: "/tmp/untrusted" }], + ["codex_local", { args: ["-c", "id"] }], + ["cursor_local", { env: { PATH: "/tmp" } }], + ["hermes_local", { hermesCommand: "/tmp/untrusted" }], + ["gemini_local", { cwd: "/tmp" }], + ["opencode_local", { filesystemSandboxCommand: "/tmp/untrusted" }], + ])("blocks agent-authenticated host settings for %s", async (adapterType, adapterConfig) => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it("allows an agent to update a peer's local model without host settings", async () => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType: "claude_local" }); + const app = await createApp({ type: "agent", agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig: { model: "claude-sonnet-4-5" } })); + + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(mockAgentService.update).toHaveBeenCalledOnce(); + }); + + it("allows a provider credential reference without allowing arbitrary environment variables", async () => { + mockAgentService.getById.mockResolvedValue({ ...baseAgent, adapterType: "claude_local" }); + const app = await createApp({ type: "agent", agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterConfig: { env: { ANTHROPIC_API_KEY: { type: "secret_ref", secretId: "33333333-3333-4333-8333-333333333333" } } } })); + + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(mockAgentService.update).toHaveBeenCalledOnce(); + }); + + it("blocks an agent from activating inherited host settings by switching onto a local adapter", async () => { + mockAgentService.getById.mockResolvedValue({ + ...baseAgent, + adapterType: "paperclip_runner", + adapterConfig: { env: { PATH: "/tmp" } }, + }); + const app = await createApp({ type: "agent", agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .patch(`/api/agents/${agentId}`) + .send({ adapterType: "claude_local", adapterConfig: { engine: "cli" } })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.update).not.toHaveBeenCalled(); + }); + + it.each([ + ["create", `/api/companies/${companyId}/agents`], + ["hire", `/api/companies/${companyId}/agent-hires`], + ])("blocks agent-authenticated process adapter commands on %s", async (_label, path) => { + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(path) + .send({ name: "Host process", role: "engineer", adapterType: "process", adapterConfig: { command: "sh" } })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.create).not.toHaveBeenCalled(); + }); + + it.each([ + ["create", `/api/companies/${companyId}/agents`], + ["hire", `/api/companies/${companyId}/agent-hires`], + ])("blocks agent-authenticated local adapter commands on %s", async (_label, path) => { + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(path) + .send({ name: "Local peer", role: "engineer", adapterType: "claude_local", adapterConfig: { engine: "cli", command: "/tmp/untrusted" } })); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.create).not.toHaveBeenCalled(); + }); + + it("blocks agent-authenticated rollback into process adapter configuration", async () => { + mockAgentService.getConfigRevision.mockResolvedValue({ + id: "33333333-3333-4333-8333-333333333333", + afterConfig: { adapterType: "process", adapterConfig: { command: "sh" }, runtimeConfig: {} }, + }); + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/33333333-3333-4333-8333-333333333333/rollback`)); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed process adapters"); + expect(mockAgentService.rollbackConfigRevision).not.toHaveBeenCalled(); + }); + + it("blocks agent-authenticated rollback into local adapter command configuration", async () => { + const revisionId = "33333333-3333-4333-8333-333333333333"; + mockAgentService.getConfigRevision.mockResolvedValue({ + id: revisionId, + afterConfig: { adapterType: "claude_local", adapterConfig: { engine: "cli", command: "/tmp/untrusted" }, runtimeConfig: {} }, + }); + const app = await createApp({ type: "agent", agentId, companyId, source: "agent_key", runId: "run-1" }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/${revisionId}/rollback`)); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed local adapter settings"); + expect(mockAgentService.rollbackConfigRevision).not.toHaveBeenCalled(); + }); + + it.each(["provisionCommand", "runtimeProvisionCommand", "teardownCommand"])( + "blocks agent-authenticated rollback of workspace %s", + async (commandKey) => { + const revisionId = "33333333-3333-4333-8333-333333333333"; + mockAgentService.getConfigRevision.mockResolvedValue({ + id: revisionId, + afterConfig: { + adapterType: "codex_local", + adapterConfig: { workspaceStrategy: { type: "git_worktree", [commandKey]: "sh -c id" } }, + runtimeConfig: {}, + }, + }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/${revisionId}/rollback`)); + + expect(res.status).toBe(403); + expect(res.body.error).toContain("host-executed workspace commands"); + expect(mockAgentService.rollbackConfigRevision).not.toHaveBeenCalled(); + }, + ); + + it("allows an agent to restore a non-process revision for a process peer", async () => { + const revisionId = "33333333-3333-4333-8333-333333333333"; + mockAgentService.getConfigRevision.mockResolvedValue({ + id: revisionId, + afterConfig: { adapterType: "codex_local", adapterConfig: {}, runtimeConfig: {} }, + }); + mockAgentService.rollbackConfigRevision.mockResolvedValue({ + ...baseAgent, + adapterType: "codex_local", + }); + const app = await createApp({ + type: "agent", + agentId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + companyId, + source: "agent_key", + runId: "run-1", + }); + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .post(`/api/agents/${agentId}/config-revisions/${revisionId}/rollback`)); + + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(mockAgentService.rollbackConfigRevision).toHaveBeenCalledOnce(); + }); + it("blocks agent-authenticated self-updates that set instructions bundle roots", async () => { const app = await createApp({ type: "agent", diff --git a/server/src/__tests__/agents-claude-oauth-binding.test.ts b/server/src/__tests__/agents-claude-oauth-binding.test.ts index 19d496b945..1a133380da 100644 --- a/server/src/__tests__/agents-claude-oauth-binding.test.ts +++ b/server/src/__tests__/agents-claude-oauth-binding.test.ts @@ -15,6 +15,7 @@ import { companySecrets, createDb, environments, + principalPermissionGrants, userSecretDeclarations, userSecretDefinitions, } from "@paperclipai/db"; @@ -245,6 +246,7 @@ describeEmbeddedPostgres("agent service Claude OAuth binding claim", () => { await db.delete(userSecretDeclarations); await db.delete(userSecretDefinitions); await db.delete(adapterAuthSessions); + await db.delete(principalPermissionGrants); await db.delete(agents); await db.delete(environments); await db.delete(companies); diff --git a/server/src/__tests__/agents-pending-approval-config.test.ts b/server/src/__tests__/agents-pending-approval-config.test.ts index 2977ae4d57..7c02a10da1 100644 --- a/server/src/__tests__/agents-pending-approval-config.test.ts +++ b/server/src/__tests__/agents-pending-approval-config.test.ts @@ -8,6 +8,7 @@ import { budgetPolicies, companies, createDb, + principalPermissionGrants, } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, @@ -40,6 +41,7 @@ describeEmbeddedPostgres("pending approval agent config integrity", () => { afterEach(async () => { await db.delete(activityLog); + await db.delete(principalPermissionGrants); await db.delete(budgetPolicies); await db.delete(approvals); await db.delete(agents); diff --git a/server/src/__tests__/agents-service-secret-bindings.test.ts b/server/src/__tests__/agents-service-secret-bindings.test.ts index d804917e72..a155bb72ab 100644 --- a/server/src/__tests__/agents-service-secret-bindings.test.ts +++ b/server/src/__tests__/agents-service-secret-bindings.test.ts @@ -13,6 +13,7 @@ import { companySecretVersions, companySecrets, createDb, + principalPermissionGrants, } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, @@ -46,6 +47,7 @@ describeEmbeddedPostgres("agent service secret binding sync", () => { afterEach(async () => { await db.delete(activityLog); + await db.delete(principalPermissionGrants); await db.delete(companySecretBindings); await db.delete(companySecretVersions); await db.delete(companySecrets); diff --git a/server/src/__tests__/authorization-service.test.ts b/server/src/__tests__/authorization-service.test.ts index b76683562e..02c1b562cd 100644 --- a/server/src/__tests__/authorization-service.test.ts +++ b/server/src/__tests__/authorization-service.test.ts @@ -2606,6 +2606,44 @@ describeEmbeddedPostgres("authorization service", () => { })).resolves.toMatchObject({ allowed: true, reason: "allow_explicit_grant" }); }); + it("keeps a default inbox grant within the responsible user's policy", async () => { + const company = await createCompany(db, "InboxDefaultScoped"); + const actorAgent = await createAgent(db, company.id); + const responsibleUserId = await createUser(db); + const disabledUserId = await createUser(db); + const disallowedUserId = await createUser(db); + await db.insert(companyMemberships).values( + [responsibleUserId, disabledUserId, disallowedUserId].map((principalId) => ({ + companyId: company.id, + principalType: "user" as const, + principalId, + status: "active" as const, + membershipRole: "operator" as const, + })), + ); + await db.insert(userInboxAgentPolicies).values([ + { companyId: company.id, userId: disabledUserId, mode: "disabled" }, + { companyId: company.id, userId: disallowedUserId, mode: "allowlist", allowedAgentIds: [] }, + ]); + await grantAgentPermission(db, company.id, actorAgent.id, "inbox:manage", { responsibleUserOnly: true }); + const decideFor = (userId: string) => authorizationService(db).decide({ + actor: { + type: "agent" as const, + agentId: actorAgent.id, + companyId: company.id, + onBehalfOfUserId: responsibleUserId, + source: "agent_jwt" as const, + }, + action: "inbox:manage" as const, + resource: { type: "company" as const, companyId: company.id }, + scope: { userId }, + }); + + await expect(decideFor(responsibleUserId)).resolves.toMatchObject({ allowed: true, reason: "allow_self" }); + await expect(decideFor(disabledUserId)).resolves.toMatchObject({ allowed: false, reason: "inbox_management_disabled" }); + await expect(decideFor(disallowedUserId)).resolves.toMatchObject({ allowed: false, reason: "inbox_agent_not_allowed" }); + }); + it("enforces user-scoped cross-user inbox grants", async () => { const company = await createCompany(db, "InboxCrossUserScoped"); const actorAgent = await createAgent(db, company.id); diff --git a/server/src/__tests__/browser-use-connection.test.ts b/server/src/__tests__/browser-use-connection.test.ts index 49a11010fe..b206fabe93 100644 --- a/server/src/__tests__/browser-use-connection.test.ts +++ b/server/src/__tests__/browser-use-connection.test.ts @@ -983,8 +983,15 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; )[0], ).toMatchObject({ eventCursor: 9, eventsDrained: 1, accountedCents: 15 }); }); - it("protects interactive viewer routes from agent, other-company and read-only access", async () => { + it.each(["task", "saved chat"])("protects %s browser routes from agent, other-company and read-only access", async (kind) => { const f = await fixture(); + if (kind === "saved chat") { + await db.update(issues).set({ + conversationAgentId: f.agent.id, + conversationUserId: actor.actorId, + conversationState: "active", + }).where(eq(issues.id, f.issue.id)); + } await f.service.execute( f.binding, f.grant, @@ -1021,11 +1028,19 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; }); app.use(browserUseRoutes(db, f.service)); app.use(errorHandler); + const listPath = `/issues/${f.issue.id}/browsers`; + const listed = await http(app).get(listPath); + expect(listed.status).toBe(200); + expect(listed.body).toMatchObject([{ id: browser.id, issueId: f.issue.id }]); + const missing = await http(app).get(`/issues/${randomUUID()}/browsers`); + expect(missing.status).toBe(404); + expect(missing.body).toEqual({ error: "Task not found" }); const path = `/issues/${f.issue.id}/browsers/${browser.id}/viewer`; const ok = await http(app).get(path); expect(ok.status).toBe(200); expect(ok.headers["cache-control"]).toBe("no-store"); expect(ok.headers["referrer-policy"]).toBe("no-referrer"); + const controlPath = path.replace("/viewer", "/control"); const presencePath = path.replace("/viewer", "/presence"); expect((await http(app).post(presencePath)).status).toBe(200); const resizePath = path.replace("/viewer", "/viewport"); @@ -1065,7 +1080,10 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; (await http(app).post(resizePath).send({ preset: "custom", width: 1 })) .status, ).toBe(400); + expect((await http(app).post(controlPath).send({ action: "cancel" })).status).toBe(200); mode = "agent"; + expect((await http(app).get(listPath)).status).toBe(403); + expect((await http(app).post(controlPath).send({ action: "keep_open" })).status).toBe(403); expect((await http(app).post(presencePath)).status).toBe(403); expect( (await http(app).post(`${resizePath}/release`).send({ viewerId })) @@ -1076,6 +1094,10 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; ).toBe(403); expect((await http(app).get(path)).status).toBe(403); mode = "other-company"; + const inaccessible = await http(app).get(listPath); + expect(inaccessible.status).toBe(missing.status); + expect(inaccessible.body).toEqual(missing.body); + expect((await http(app).post(controlPath).send({ action: "keep_open" })).status).toBe(404); expect((await http(app).post(presencePath)).status).toBe(404); expect( (await http(app).post(`${resizePath}/release`).send({ viewerId })) @@ -1091,6 +1113,8 @@ const actor = { actorType: "user" as const, actorId: "browser-reviewer" }; .set({ membershipRole: "viewer" }) .where(eq(companyMemberships.companyId, f.company.id)); expect((await http(app).get(path)).status).toBe(403); + expect((await http(app).get(listPath)).body).toEqual([]); + expect((await http(app).post(controlPath).send({ action: "keep_open" })).status).toBe(403); expect((await http(app).post(presencePath)).status).toBe(403); expect( (await http(app).post(resizePath).send({ preset: "phone" })).status, diff --git a/server/src/__tests__/browser-use-route-scope.test.ts b/server/src/__tests__/browser-use-route-scope.test.ts new file mode 100644 index 0000000000..113b408184 --- /dev/null +++ b/server/src/__tests__/browser-use-route-scope.test.ts @@ -0,0 +1,73 @@ +import { randomUUID } from "node:crypto"; +import express from "express"; +import http from "supertest"; +import { describe, expect, it, vi } from "vitest"; +import type { Db } from "@paperclipai/db"; +import { errorHandler } from "../middleware/error-handler.js"; +import { browserUseRoutes } from "../routes/browser-use.js"; + +const routes = [ + { method: "get", suffix: "" }, + { method: "get", suffix: "/viewer" }, + { method: "post", suffix: "/presence" }, + { method: "post", suffix: "/viewport" }, + { method: "post", suffix: "/viewport/release" }, + { method: "post", suffix: "/control" }, +] as const; + +describe("browser task route scope", () => { + it.each([ + "12345678-1234-4234-8234-123456789abc", + "12345678-1234-7234-8234-123456789ABC", + "00000000-0000-0000-0000-000000000000", + ])("keeps canonical PostgreSQL task UUIDs queryable: %s", async (id) => { + const companyId = randomUUID(); + const limit = vi.fn().mockResolvedValue([{ companyId }]); + const db = { select: () => ({ from: () => ({ where: () => ({ limit }) }) }) } as unknown as Db; + const list = vi.fn().mockResolvedValue([]); + const service = { list } as unknown as NonNullable[1]>; + const app = express(); + app.use((req, _res, next) => { + req.actor = { type: "board", userId: "browser-reviewer", source: "session", companyIds: [companyId] }; + next(); + }); + app.use(browserUseRoutes(db, service)); + app.use(errorHandler); + expect((await http(app).get(`/issues/${id}/browsers`)).status).toBe(200); + expect(limit).toHaveBeenCalledExactlyOnceWith(1); + expect(list).toHaveBeenCalledExactlyOnceWith(companyId, id, "browser-reviewer"); + }); + + it.each(routes)("rejects invalid task IDs before any query: $method $suffix", async ({ method, suffix }) => { + const select = vi.fn(() => { throw new Error("Invalid scope reached the database"); }); + const db = { select } as unknown as Db; + const service = Object.fromEntries( + ["list", "viewer", "presence", "resize", "releaseViewport", "humanSession", "control"] + .map((name) => [name, vi.fn()]), + ) as unknown as NonNullable[1]>; + let actorType: "board" | "agent" | "none" = "board"; + const app = express(); + app.use((req, _res, next) => { + req.actor = actorType === "board" + ? { type: "board", userId: "browser-reviewer", source: "session", companyIds: [] } + : actorType === "agent" + ? { type: "agent", companyId: randomUUID(), agentId: randomUUID(), source: "agent_key" } + : { type: "none" }; + next(); + }); + app.use(browserUseRoutes(db, service)); + app.use(errorHandler); + const id = randomUUID(); + for (const invalidId of [`chat:${id}`, "not-a-uuid", `${id}'`, ` ${id}`, `${id}\n`]) { + const path = `/issues/${encodeURIComponent(invalidId)}/browsers${suffix ? `/${id}${suffix}` : ""}`; + for (const type of ["board", "agent", "none"] as const) { + actorType = type; + const response = await http(app)[method](path); + expect(response.status).toBe(type === "board" ? 404 : 403); + expect(response.body).toEqual({ error: type === "board" ? "Task not found" : "Board access required" }); + } + } + expect(select).not.toHaveBeenCalled(); + for (const handler of Object.values(service)) expect(handler).not.toHaveBeenCalled(); + }); +}); diff --git a/server/src/__tests__/built-in-agents.test.ts b/server/src/__tests__/built-in-agents.test.ts index 69de71c730..7a90b5b5ec 100644 --- a/server/src/__tests__/built-in-agents.test.ts +++ b/server/src/__tests__/built-in-agents.test.ts @@ -611,9 +611,9 @@ describeEmbeddedPostgres("built-in agents", () => { expect(result.defaultGrantsEnsured).toBeGreaterThanOrEqual(4); const rootGrantKeys = await permissionKeysForAgent(root.id); - expect(rootGrantKeys).toEqual(expect.arrayContaining(["agents:configure", "skills:create"])); - expect(rootGrantKeys).not.toContain("agents:suggest-changes"); - expect(rootGrantKeys).not.toContain("skills:suggest-changes"); + expect(rootGrantKeys).toEqual(expect.arrayContaining([ + "agents:configure", "agents:suggest-changes", "skills:create", "skills:suggest-changes", + ])); const state = await builtInAgentService(db).get(companyId, "reflection-coach"); expect(state).toMatchObject({ diff --git a/server/src/__tests__/heartbeat-process-recovery.test.ts b/server/src/__tests__/heartbeat-process-recovery.test.ts index 80d2011761..88c1a64cfe 100644 --- a/server/src/__tests__/heartbeat-process-recovery.test.ts +++ b/server/src/__tests__/heartbeat-process-recovery.test.ts @@ -14310,7 +14310,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { await commitNativeStatusDecision({ db, companyId: fixture.companyId, issueId: fixture.issueId, runId: fixture.runId, assessmentId: coordinator!.assessmentId!, priorStatus: issue!.status, priorStatusVersion: issue!.statusVersion, priorDecisionId: issue!.lastStatusDecisionId, - decision: { policyVersion: "phase6-v7", statusAction: "in_progress", toStatus: "in_progress", + decision: { policyVersion: "phase6-v8", statusAction: "in_progress", toStatus: "in_progress", reasonCode: "board_response_waiting", unblockDescriptor: null, effects: [] }, requireBoardResponseWaitSource: source!.source, requireBoardResponseWaitOrigin: origin!, }); @@ -14934,7 +14934,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { priorStatusVersion: issue!.statusVersion, priorDecisionId: issue!.lastStatusDecisionId, decision: { - policyVersion: "phase6-v7", + policyVersion: "phase6-v8", statusAction: reasonCode === "prior_status_preserved_no_live_path" ? "preserve" : "in_progress", toStatus: "in_progress", reasonCode, diff --git a/server/src/__tests__/invite-join-grants.test.ts b/server/src/__tests__/invite-join-grants.test.ts index 427fef0fa6..9c10d2d636 100644 --- a/server/src/__tests__/invite-join-grants.test.ts +++ b/server/src/__tests__/invite-join-grants.test.ts @@ -8,18 +8,20 @@ import { normalizeHumanRole, resolveHumanInviteRole, } from "../services/company-member-roles.js"; +import { NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS, newStandardAgentGrantScope } from "../services/agent-permissions.js"; + +const invitedAgentId = "agent-1"; +const defaultAgentGrants = NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({ + permissionKey, + scope: newStandardAgentGrantScope(permissionKey, invitedAgentId), +})); describe("agentJoinGrantsFromDefaults", () => { - it("adds tasks:assign when invite defaults do not specify agent grants", () => { - expect(agentJoinGrantsFromDefaults(null)).toEqual([ - { - permissionKey: "tasks:assign", - scope: null, - }, - ]); + it("adds all new-agent grants when invite defaults do not specify grants", () => { + expect(agentJoinGrantsFromDefaults(null, invitedAgentId)).toEqual(defaultAgentGrants); }); - it("preserves invite agent grants and appends tasks:assign", () => { + it("preserves invite agent grants and appends the missing defaults", () => { expect( agentJoinGrantsFromDefaults({ agent: { @@ -30,17 +32,8 @@ describe("agentJoinGrantsFromDefaults", () => { }, ], }, - }), - ).toEqual([ - { - permissionKey: "agents:create", - scope: null, - }, - { - permissionKey: "tasks:assign", - scope: null, - }, - ]); + }, invitedAgentId), + ).toEqual([{ permissionKey: "agents:create", scope: null }, ...defaultAgentGrants]); }); it("does not duplicate tasks:assign when invite defaults already include it", () => { @@ -54,12 +47,19 @@ describe("agentJoinGrantsFromDefaults", () => { }, ], }, - }), + }, invitedAgentId), ).toEqual([ - { - permissionKey: "tasks:assign", - scope: { projectId: "project-1" }, - }, + { permissionKey: "tasks:assign", scope: { projectId: "project-1" } }, + ...defaultAgentGrants.filter((grant) => grant.permissionKey !== "tasks:assign"), + ]); + }); + + it("preserves an explicit scoped agent configuration grant", () => { + expect(agentJoinGrantsFromDefaults({ + agent: { grants: [{ permissionKey: "agents:configure", scope: { agentIds: ["agent-1"] } }] }, + }, invitedAgentId)).toEqual([ + { permissionKey: "agents:configure", scope: { agentIds: ["agent-1"] } }, + ...defaultAgentGrants.filter((grant) => grant.permissionKey !== "agents:configure"), ]); }); }); diff --git a/server/src/__tests__/plugin-managed-agents.test.ts b/server/src/__tests__/plugin-managed-agents.test.ts index 0b06a55126..30a4725f47 100644 --- a/server/src/__tests__/plugin-managed-agents.test.ts +++ b/server/src/__tests__/plugin-managed-agents.test.ts @@ -17,6 +17,7 @@ import { pluginCompanySettings, pluginManagedResources, plugins, + principalPermissionGrants, } from "@paperclipai/db"; import type { PaperclipPluginManifestV1 } from "@paperclipai/shared"; import { @@ -106,6 +107,7 @@ describeEmbeddedPostgres("plugin-managed agents", () => { await db.delete(pluginManagedResources); await db.delete(pluginCompanySettings); await db.delete(approvals); + await db.delete(principalPermissionGrants); await db.delete(agents); await db.delete(plugins); await db.delete(companies); diff --git a/server/src/__tests__/plugin-managed-routines.test.ts b/server/src/__tests__/plugin-managed-routines.test.ts index 25a02531c2..e5c0a4d0f3 100644 --- a/server/src/__tests__/plugin-managed-routines.test.ts +++ b/server/src/__tests__/plugin-managed-routines.test.ts @@ -12,6 +12,7 @@ import { issues, pluginManagedResources, plugins, + principalPermissionGrants, projects, routineDocuments, routineRuns, @@ -121,6 +122,7 @@ describeEmbeddedPostgres("plugin-managed routines", () => { await db.delete(agentConfigRevisions); await db.delete(activityLog); await db.delete(pluginManagedResources); + await db.delete(principalPermissionGrants); await db.delete(agents); await db.delete(projects); await db.delete(plugins); diff --git a/server/src/__tests__/routines-e2e.test.ts b/server/src/__tests__/routines-e2e.test.ts index 1f6742be74..42e7ef0b46 100644 --- a/server/src/__tests__/routines-e2e.test.ts +++ b/server/src/__tests__/routines-e2e.test.ts @@ -238,6 +238,119 @@ describeEmbeddedPostgres("routine routes end-to-end", () => { return { companyId, agentId, projectId, userId }; } + function routineResourceRequests(app: express.Express, routineId: string, triggerId = routineId) { + const path = `/api/routines/${encodeURIComponent(routineId)}`; + const triggerPath = `/api/routine-triggers/${encodeURIComponent(triggerId)}`; + const nestedId = "12345678-1234-4234-8234-123456789abc"; + return [ + request(app).get(path), + request(app).get(`${path}/runs`), + request(app).get(`${path}/revisions`), + request(app).get(`${path}/description/annotations`), + request(app).get(`${path}/description/annotations/${nestedId}`), + request(app).post(`${path}/description/annotations`).send({ + baseRevisionId: nestedId, baseRevisionNumber: 1, body: "Review this text", + selector: { + quote: { exact: "text" }, + position: { normalizedStart: 0, normalizedEnd: 4, markdownStart: 0, markdownEnd: 4 }, + }, + }), + request(app).post(`${path}/description/annotations/${nestedId}/comments`).send({ body: "Review" }), + request(app).patch(`${path}/description/annotations/${nestedId}`).send({ status: "resolved" }), + request(app).patch(path).send({ title: "Updated routine" }), + request(app).post(`${path}/revisions/${nestedId}/restore`).send({}), + request(app).post(`${path}/triggers`).send({ kind: "api" }), + request(app).post(`${path}/run`).send({}), + request(app).patch(triggerPath).send({ enabled: false }), + request(app).delete(triggerPath), + request(app).post(`${triggerPath}/rotate-secret`).send({}), + ]; + } + + it("returns not found without queries for malformed routine and trigger IDs on every resource route", async () => { + const { companyId, agentId, userId } = await seedFixture(); + const actors = [ + { type: "board", userId, source: "session", companyIds: [companyId] }, + { type: "agent", agentId, companyId }, + { type: "none" }, + ]; + const fullId = "12345678-1234-4234-8234-123456789abc"; + for (const actor of actors) { + const app = await createApp(actor); + const spies = [ + vi.spyOn(db, "select"), vi.spyOn(db, "insert"), + vi.spyOn(db, "update"), vi.spyOn(db, "delete"), + ]; + try { + for (const id of [fullId.slice(0, 8), "not-a-uuid", ` ${fullId}`, `${fullId}\n`, `{${fullId}\n}`]) { + for (const pending of routineResourceRequests(app, id)) { + const response = await pending; + expect(response.status, JSON.stringify(response.body)).toBe(404); + expect(response.body).toEqual({ error: response.req.path.includes("/routine-triggers/") + ? "Routine trigger not found" : "Routine not found" }); + } + } + const invalidBody = await request(app).patch("/api/routines/not-a-uuid").send({ title: 42 }); + expect(invalidBody.status).toBe(400); + for (const spy of spies) expect(spy).not.toHaveBeenCalled(); + } finally { + for (const spy of spies) spy.mockRestore(); + } + } + }, 20_000); + + it("keeps missing and cross-company routine resources indistinguishable", async () => { + const { companyId, agentId, userId } = await seedFixture(); + const routineId = randomUUID(); + const triggerId = randomUUID(); + await db.insert(routines).values({ id: routineId, companyId, assigneeAgentId: agentId, title: "Private routine" }); + await db.insert(routineTriggers).values({ id: triggerId, companyId, routineId, kind: "api" }); + for (const actor of [ + { type: "board", userId, source: "session", isInstanceAdmin: true, companyIds: [randomUUID()] }, + { type: "agent", agentId: randomUUID(), companyId: randomUUID() }, + { type: "none" }, + ]) { + const app = await createApp(actor); + for (const [id, tid] of [[routineId, triggerId], [routineId.toUpperCase(), `{${triggerId}}`], [randomUUID(), randomUUID()]]) { + for (const pending of routineResourceRequests(app, id!, tid!)) { + const response = await pending; + expect(response.status, JSON.stringify(response.body)).toBe(404); + expect(response.body).toEqual({ error: response.req.path.includes("/routine-triggers/") + ? "Routine trigger not found" : "Routine not found" }); + } + } + } + expect(await db.select().from(routines)).toMatchObject([{ id: routineId, title: "Private routine" }]); + expect(await db.select().from(routineTriggers)).toMatchObject([{ id: triggerId, enabled: true }]); + expect(await db.select().from(activityLog)).toEqual([]); + }, 20_000); + + it("preserves UUID routine reads and assignee-only mutations", async () => { + const { companyId, agentId, userId } = await seedFixture(); + const routineId = randomUUID(); + const triggerId = randomUUID(); + await db.insert(routines).values({ id: routineId, companyId, assigneeAgentId: agentId, title: "Private routine" }); + await db.insert(routineTriggers).values({ id: triggerId, companyId, routineId, kind: "api" }); + const owner = await createApp({ type: "agent", agentId, companyId }); + const otherAgent = await createApp({ type: "agent", agentId: randomUUID(), companyId }); + const board = await createApp({ type: "board", userId, source: "session", companyIds: [companyId] }); + for (const app of [owner, otherAgent, board]) { + const response = await request(app).get(`/api/routines/${routineId.toUpperCase()}`); + expect(response.status).toBe(200); + expect(response.body.id).toBe(routineId); + } + for (const [index, pending] of routineResourceRequests(otherAgent, routineId, triggerId).entries()) { + const response = await pending; + expect(response.status, JSON.stringify(response.body)).toBe(index < 2 ? 200 : 403); + } + const updated = await request(owner).patch(`/api/routines/${routineId}`).send({ title: "Owner update" }); + expect(updated.status, JSON.stringify(updated.body)).toBe(200); + expect(updated.body.title).toBe("Owner update"); + const changedTrigger = await request(board).patch(`/api/routine-triggers/${triggerId}`).send({ enabled: false }); + expect(changedTrigger.status, JSON.stringify(changedTrigger.body)).toBe(200); + expect(changedTrigger.body.enabled).toBe(false); + }, 20_000); + it.each(["bearer", "hmac_sha256", "github_hmac", "none"] as const)( "authenticates %s HTTP deliveries, persists payloads, and enforces trigger lifecycle", async (signingMode) => { diff --git a/server/src/__tests__/routines-service.test.ts b/server/src/__tests__/routines-service.test.ts index bc4badd189..9d4949e30a 100644 --- a/server/src/__tests__/routines-service.test.ts +++ b/server/src/__tests__/routines-service.test.ts @@ -246,6 +246,63 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => { .then((rows) => rows[0]!); } + it("rejects malformed routine and trigger IDs before any query", async () => { + const { routine, svc } = await seedFixture(); + const malformed = [ + "", routine.id.slice(0, 8), "not-a-uuid", `${routine.id} `, ` ${routine.id}`, + `${routine.id}\n`, `{${routine.id}`, `${routine.id}}`, `{${routine.id}\n}`, + `${routine.id}-`, routine.id.replace("-", "--"), `g${routine.id.slice(1)}`, + `${routine.id}\0`, `é${routine.id.slice(1)}`, + `${routine.id.slice(0, 3)}-${routine.id.slice(3)}`, + ]; + const select = vi.spyOn(db, "select"); + try { + for (const id of malformed) { + await expect(svc.get(id)).resolves.toBeNull(); + await expect(svc.getDetail(id)).resolves.toBeNull(); + await expect(svc.getTrigger(id)).resolves.toBeNull(); + } + expect(select).not.toHaveBeenCalled(); + } finally { + select.mockRestore(); + } + }); + + it("preserves lookup failures for valid routine and trigger IDs", async () => { + const { routine, svc } = await seedFixture(); + const failure = new Error("Database unavailable"); + const select = vi.spyOn(db, "select").mockImplementation(() => { throw failure; }); + try { + await expect(svc.get(routine.id)).rejects.toBe(failure); + await expect(svc.getDetail(routine.id)).rejects.toBe(failure); + await expect(svc.getTrigger(routine.id)).rejects.toBe(failure); + } finally { + select.mockRestore(); + } + }); + + it.each([ + "12345678-1234-4234-8234-123456789abc", + "12345678-1234-7234-8234-123456789abc", + "00000000-0000-0000-0000-000000000000", + "ffffffff-ffff-ffff-ffff-ffffffffffff", + ])("preserves PostgreSQL UUID lookup forms for %s", async (id) => { + const { companyId, svc } = await seedFixture(); + await db.insert(routines).values({ id, companyId, title: "UUID lookup fixture" }); + await db.insert(routineTriggers).values({ id, companyId, routineId: id, kind: "api" }); + const compact = id.replaceAll("-", ""); + const forms = [ + id, id.toUpperCase(), `{${id}}`, compact, + compact.match(/.{4}/g)!.join("-"), + `{${compact.slice(0, 8)}-${compact.slice(8, 16)}-${compact.slice(16)}}`, + ]; + for (const input of forms) { + await expect(svc.get(input)).resolves.toMatchObject({ id, companyId }); + await expect(svc.getDetail(input)).resolves.toMatchObject({ id, companyId }); + await expect(svc.getTrigger(input)).resolves.toMatchObject({ id, companyId, routineId: id }); + } + }); + it("clears transient routine run failures when execution issues resume", async () => { const { companyId, issueSvc, routine, svc } = await seedFixture(); const runId = randomUUID(); diff --git a/server/src/modules/wake-queue/domain/values.ts b/server/src/modules/wake-queue/domain/values.ts index 48191b69d2..fc4fa8fa5e 100644 --- a/server/src/modules/wake-queue/domain/values.ts +++ b/server/src/modules/wake-queue/domain/values.ts @@ -22,5 +22,6 @@ export function isWorkspaceValidationFailedRun(run: { errorCode: string | null } } export function isConfigurationIncompleteFailedRun(run: { errorCode: string | null }): boolean { - return run.errorCode === CONFIGURATION_INCOMPLETE_FAILURE_CODE || run.errorCode === "model_not_found"; + return run.errorCode === CONFIGURATION_INCOMPLETE_FAILURE_CODE || run.errorCode === "model_not_found" || + run.errorCode === "native_provider_model_rejected"; } diff --git a/server/src/routes/access.ts b/server/src/routes/access.ts index d35c4586d7..3aaa1df32e 100644 --- a/server/src/routes/access.ts +++ b/server/src/routes/access.ts @@ -93,7 +93,7 @@ import { normalizeHumanRole, resolveHumanInviteRole, } from "../services/company-member-roles.js"; -import { humanJoinGrantsFromDefaults } from "../services/invite-grants.js"; +import { agentJoinGrantsFromDefaults as standardAgentJoinGrantsFromDefaults, humanJoinGrantsFromDefaults } from "../services/invite-grants.js"; import { collapseDuplicatePendingHumanJoinRequests, findReusableHumanJoinRequest, @@ -2246,22 +2246,13 @@ function grantsFromDefaults( } export function agentJoinGrantsFromDefaults( - defaultsPayload: Record | null | undefined + defaultsPayload: Record | null | undefined, + agentId: string, ): Array<{ permissionKey: (typeof PERMISSION_KEYS)[number]; scope: Record | null; }> { - const grants = grantsFromDefaults(defaultsPayload, "agent"); - if (grants.some((grant) => grant.permissionKey === "tasks:assign")) { - return grants; - } - return [ - ...grants, - { - permissionKey: "tasks:assign", - scope: null - } - ]; + return standardAgentJoinGrantsFromDefaults(defaultsPayload, agentId); } type JoinRequestManagerCandidate = { @@ -4297,7 +4288,8 @@ export function accessRoutes( "active" ); const grants = agentJoinGrantsFromDefaults( - invite.defaultsPayload as Record | null + invite.defaultsPayload as Record | null, + created.id, ); await access.setPrincipalGrants( companyId, diff --git a/server/src/routes/agents.ts b/server/src/routes/agents.ts index 5caf6ec9a1..418bf32237 100644 --- a/server/src/routes/agents.ts +++ b/server/src/routes/agents.ts @@ -16,6 +16,7 @@ import { isAiConnectionCompatible } from "@paperclipai/shared"; import { applyConnectorSkills, resolveConnectorAssignments, annotateConnectorSkills, isConnectorSkill } from "../services/connector-runtime.js"; import { getExecutionBlocker } from "../services/execution-blocker.js"; import { canRetryStoppedRun } from "../services/cancelled-native-startup.js"; +import { getNativeReviewAssignment, readNativeReviewAssignmentContext, type NativeReviewAssignmentContext } from "../services/native-runtime/native-review-participant.js"; import { paperclipRunnerTransitionConfig, normalizeLegacyRunnerProvider, isPaperclipRunnerProvider } from "@paperclipai/adapter-utils"; import { executionProjectionForRun, executionProjectionsForRuns } from "../services/execution-projection.js"; import { selectDashboardRunIds } from "../services/dashboard-run-selection.js"; @@ -2914,6 +2915,69 @@ export function agentRoutes( ); } + function assertNoAgentProcessAdapterMutation(req: Request, adapterType: string, mutatesConfiguration: boolean) { + if (req.actor.type !== "agent" || adapterType !== "process" || !mutatesConfiguration) return; + throw forbidden("Agent keys cannot configure host-executed process adapters."); + } + + const LOCAL_ADAPTER_HOST_COMMAND_KEYS = [ + "command", + "hermesCommand", + "args", + "extraArgs", + "cwd", + "env", + "filesystemSandboxCommand", + ] as const; + + const LOCAL_ADAPTER_CREDENTIAL_ENV_KEYS: Record = { + ...INHERITABLE_AGENT_CREDENTIAL_ENV_KEYS, + gemini_local: ["GEMINI_API_KEY", "GOOGLE_API_KEY"], + kimi_local: ["KIMI_MODEL_API_KEY"], + opencode_local: ["PAPERCLIP_OPENCODE_PROVIDERS", "OPENCODE_AUTH_JSON"], + }; + const POOL_AUTH_OVERRIDE_ENV_KEYS = [ + "OPENAI_API_KEY", "ANTHROPIC_API_KEY", "PAPERCLIP_OPENCODE_PROVIDERS", "OPENCODE_AUTH_JSON", + ] as const; + + async function callerUsesAiConnectionPool(req: Request, companyId: string): Promise { + if (req.actor.type !== "agent" || !req.actor.agentId) return false; + const caller = await svc.getById(req.actor.agentId); + if (!caller || caller.companyId !== companyId) return false; + return asRecord(asRecord(caller.runtimeConfig)?.aiConnection)?.mode === "router"; + } + + function containsOnlyLocalAdapterCredentialRefs( + adapterType: string, + env: unknown, + allowPlainCredentials: boolean, + ): boolean { + const envRecord = asRecord(env); + if (!envRecord) return false; + const allowedKeys = LOCAL_ADAPTER_CREDENTIAL_ENV_KEYS[adapterType] ?? []; + return Object.entries(envRecord).every(([key, value]) => + (allowedKeys.includes(key) && isInheritableCredentialReference(value)) || + (allowPlainCredentials && POOL_AUTH_OVERRIDE_ENV_KEYS.includes(key as typeof POOL_AUTH_OVERRIDE_ENV_KEYS[number]) && asEnvBindingString(value) !== null), + ); + } + + function assertNoAgentLocalAdapterHostCommandMutation( + req: Request, + adapterType: string, + adapterConfig: Record, + allowPlainCredentials = false, + ) { + if (req.actor.type !== "agent" || !adapterType.endsWith("_local")) return; + const changedKeys = LOCAL_ADAPTER_HOST_COMMAND_KEYS.filter((key) => + adapterConfig[key] !== undefined && + (key !== "env" || !containsOnlyLocalAdapterCredentialRefs(adapterType, adapterConfig.env, allowPlainCredentials)), + ); + if (changedKeys.length === 0) return; + throw forbidden( + `Agent keys cannot configure host-executed local adapter settings (${changedKeys.join(", ")}).`, + ); + } + function summarizeAgentUpdateDetails(patch: Record) { const changedTopLevelKeys = Object.keys(patch).sort(); const details: Record = { changedTopLevelKeys }; @@ -4419,6 +4483,13 @@ export function agentRoutes( await assertSelectableAdapterType(rollbackAdapterType); } const rollbackAdapterConfig = asRecord(rollbackConfig.adapterConfig) ?? {}; + assertNoAgentAdapterConfigMutation(req, rollbackAdapterConfig); + assertNoAgentLocalAdapterHostCommandMutation(req, rollbackAdapterType, rollbackAdapterConfig); + assertNoAgentProcessAdapterMutation( + req, + rollbackAdapterType, + true, + ); assertExternalInstructionsAdmin(req, existing); assertExternalInstructionsAdmin(req, { ...existing, @@ -4581,6 +4652,8 @@ export function agentRoutes( rawHireAdapterConfig, ); assertNoAgentAdapterConfigMutation(req, rawHireAdapterConfig); + assertNoAgentLocalAdapterHostCommandMutation(req, hireInput.adapterType, rawHireAdapterConfig, await callerUsesAiConnectionPool(req, companyId)); + assertNoAgentProcessAdapterMutation(req, hireInput.adapterType, Object.keys(rawHireAdapterConfig).length > 0); const hiredAgentId = randomUUID(); const authInheritance = await applyHiringAgentAuthInheritance( req, @@ -4890,6 +4963,8 @@ export function agentRoutes( rawCreateAdapterConfig, ); assertNoAgentAdapterConfigMutation(req, rawCreateAdapterConfig); + assertNoAgentLocalAdapterHostCommandMutation(req, createInput.adapterType, rawCreateAdapterConfig, await callerUsesAiConnectionPool(req, companyId)); + assertNoAgentProcessAdapterMutation(req, createInput.adapterType, Object.keys(rawCreateAdapterConfig).length > 0); const agentId = randomUUID(); const requestedAdapterConfig = applyCodexLocalKeyIsolation( companyId, @@ -5528,6 +5603,7 @@ export function agentRoutes( const touchesAdapterConfiguration = hasOwn(patchData, "adapterType") || hasOwn(patchData, "adapterConfig"); + assertNoAgentProcessAdapterMutation(req, requestedAdapterType, touchesAdapterConfiguration); if (touchesAdapterConfiguration) { assertExternalInstructionsAdmin(req, existing); const existingAdapterConfig = asRecord(existing.adapterConfig) ?? {}; @@ -5575,6 +5651,13 @@ export function agentRoutes( if (requestedAdapterType === "paperclip_runner") { rawEffectiveAdapterConfig = normalizePaperclipRunnerAdapterConfig(requestedAdapterType, rawEffectiveAdapterConfig); } + if (changingAdapterType || requestedAdapterConfig) { + assertNoAgentLocalAdapterHostCommandMutation( + req, + requestedAdapterType, + changingAdapterType ? rawEffectiveAdapterConfig : (requestedAdapterConfig ?? {}), + ); + } const existingRunnerProvider = existing.adapterType === "paperclip_runner" ? existingAdapterConfig.provider @@ -6037,6 +6120,7 @@ export function agentRoutes( let wakePayload = req.body.payload ?? null; let retryConversationContext: Record = {}; + let retryNativeReviewContext: NativeReviewAssignmentContext | null = null; if (req.body.failedRunId) { assertBoard(req); if ( @@ -6090,7 +6174,22 @@ export function agentRoutes( }, }); if (!decision.allowed) throw forbidden(decision.explanation, authorizationDeniedDetails(decision)); - if (issue.assigneeAgentId !== agent.id) throw conflict("The task is no longer assigned to this agent."); + const savedReviewContext = readNativeReviewAssignmentContext(failedContext); + if (failedRun.runtimeMode === "native" && failedRun.status === "failed" && + failedRun.errorCode === "native_provider_model_rejected" && + failedRun.nativeIssueId === issue.id && savedReviewContext) { + const review = await getNativeReviewAssignment(db, { + companyId: agent.companyId, issueId: issue.id, agentId: agent.id, + contextSnapshot: savedReviewContext, + }); + if (review?.interaction.status !== "pending") { + throw conflict("The failed run's review is no longer current."); + } + retryNativeReviewContext = savedReviewContext; + } + if (issue.assigneeAgentId !== agent.id && !retryNativeReviewContext) { + throw conflict("The task is no longer assigned to this agent."); + } if (issue.conversationAgentId) { // Agent Chat has no task description to replay. Recover the exact // request from the selected server-owned run, never caller markers. @@ -6197,6 +6296,9 @@ export function agentRoutes( requestedByActorId: req.actor.type === "agent" ? req.actor.agentId ?? null : req.actor.userId ?? null, contextSnapshot: { ...retryConversationContext, + // Only the selected server-owned run can carry review authority into + // an exact retry. Dispatch revalidates the current review assignment. + ...retryNativeReviewContext, triggeredBy: req.actor.type, originIdentityContextId: req.actor.identityContextId ?? null, responsibleUserId: req.actor.type === "agent" ? req.actor.onBehalfOfUserId ?? null : req.actor.userId ?? null, diff --git a/server/src/routes/browser-use.ts b/server/src/routes/browser-use.ts index 343237998f..0b45801e51 100644 --- a/server/src/routes/browser-use.ts +++ b/server/src/routes/browser-use.ts @@ -1,5 +1,6 @@ import { Router } from "express"; import { eq } from "drizzle-orm"; +import { z } from "zod"; import { issues, type Db } from "@paperclipai/db"; import { browserUseControlSchema, @@ -15,10 +16,19 @@ import { } from "./authz.js"; import { browserUseService } from "../services/browser-use.js"; +const issueIdSchema = z.string().uuid(); + export function browserUseRoutes(db: Db, service = browserUseService(db)) { const router = Router(); router.use("/issues/:issueId/browsers", async (req, res, next) => { assertBoard(req); + if ( + req.params.issueId !== req.params.issueId.trim() || + !issueIdSchema.safeParse(req.params.issueId).success + ) { + res.status(404).json({ error: "Task not found" }); + return; + } const [issue] = await db .select({ companyId: issues.companyId }) .from(issues) diff --git a/server/src/services/__tests__/run-failure-diagnostics.test.ts b/server/src/services/__tests__/run-failure-diagnostics.test.ts index 98a585aa7c..6ee721b7a3 100644 --- a/server/src/services/__tests__/run-failure-diagnostics.test.ts +++ b/server/src/services/__tests__/run-failure-diagnostics.test.ts @@ -7,6 +7,19 @@ const run = (overrides: Partial = {}) => ({ resultJson: null, ...overrides const collect = (error: unknown) => collectRunFailureDiagnostics(run(), { error }); describe("run failure diagnostics", () => { + it("exports only the closed native model/auth rejection vocabulary", () => { + const diagnostic = { provider: "codex", category: "model_auth_incompatible", status: 400, authMode: "chatgpt" }; + const result = sanitizeRunFailureDiagnostics(collectRunFailureDiagnostics(run({ resultJson: { + nativeProviderFailure: { ...diagnostic, model: "private-model", response: "private-provider-response", prompt: "private-prompt" }, + } }), {})); + expect(result.provider).toEqual(diagnostic); + expect(JSON.stringify(result)).not.toContain("private"); + for (const value of [null, [], { ...diagnostic, category: "private-category" }, + Object.defineProperty({}, "provider", { get() { throw new Error("private"); } })]) { + expect(collectRunFailureDiagnostics(run({ resultJson: { nativeProviderFailure: value } }), {}).provider).toEqual({}); + } + }); + it("selects bounded lock-owner evidence from a caught timeout cause", () => { const error = new Error("outer", { cause: Object.assign(new Error("lock timeout"), { code: "ERR_WORKSPACE_RESTORE_LOCK_TIMEOUT", diff --git a/server/src/services/activity.ts b/server/src/services/activity.ts index 5d2b3ebdba..96685ad287 100644 --- a/server/src/services/activity.ts +++ b/server/src/services/activity.ts @@ -403,6 +403,9 @@ export function activityService(db: Db) { invocationSource: heartbeatRuns.invocationSource, responsibleUserId: heartbeatRuns.responsibleUserId, errorCode: heartbeatRuns.errorCode, + error: sql`case when ${heartbeatRuns.status} = 'failed' + and ${heartbeatRuns.errorCode} = 'native_provider_model_rejected' + then left(${heartbeatRuns.error}, 2000) else null end`, usageJson: summarizedUsageJson, resultJson: summarizedResultJson, logBytes: heartbeatRuns.logBytes, @@ -497,7 +500,7 @@ export function activityService(db: Db) { const [exhaustionRows, leaseRows, executionByRunId, [savedPlan]] = await Promise.all([ exhaustionRowsQuery, leaseRowsQuery, - executionProjectionsForRuns(db, companyId, runIds), + executionProjectionsForRuns(db, companyId, runIds, new Date(), { retryDatabaseReads: true }), savedPlanQuery, ]); const retryExhaustedReasonByRunId = new Map(); diff --git a/server/src/services/agent-permissions.ts b/server/src/services/agent-permissions.ts index e628a62ec5..24f9bc29d0 100644 --- a/server/src/services/agent-permissions.ts +++ b/server/src/services/agent-permissions.ts @@ -1,4 +1,31 @@ -import { LOW_TRUST_REVIEW_PRESET } from "@paperclipai/shared"; +import { LOW_TRUST_REVIEW_PRESET, type PermissionKey } from "@paperclipai/shared"; + +/** Direct grants for newly activated standard agents. Existing agents are not backfilled. */ +export const NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS = [ + "agents:configure", + "agents:suggest-changes", + "skills:create", + "skills:suggest-changes", + "tools:manage_connections", + "tools:manage_profiles", + "tools:view_audit", + "audit:view_agent_actions", + "tools:use", + "tools:manage_runtime", + "inbox:manage", + "tasks:assign", + "tasks:assign_scope", + "tasks:manage_active_checkouts", +] as const satisfies readonly PermissionKey[]; + +export function newStandardAgentGrantScope( + permissionKey: PermissionKey, + agentId: string, +): Record | null { + if (permissionKey === "tasks:assign_scope") return { subtreeRootAgentId: agentId }; + if (permissionKey === "inbox:manage") return { responsibleUserOnly: true }; + return null; +} export type NormalizedAgentPermissions = Record & { canCreateAgents: boolean; diff --git a/server/src/services/agents.ts b/server/src/services/agents.ts index 3640a90d4e..13450be91f 100644 --- a/server/src/services/agents.ts +++ b/server/src/services/agents.ts @@ -17,6 +17,7 @@ import { issueExecutionDecisions, issues, issueComments, + principalPermissionGrants, } from "@paperclipai/db"; import { AGENT_DEFAULT_MAX_CONCURRENT_RUNS, @@ -38,8 +39,8 @@ import { syncAgentAdapterEnvBindings, } from "./agent-secret-bindings.js"; import { logActivity } from "./activity-log.js"; +import { NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS, newStandardAgentGrantScope, normalizeAgentPermissions, permissionsImplyLowTrust } from "./agent-permissions.js"; import { recordAgentStatusEvent, recordResourceCreationEvent } from "./resource-lifecycle-events.js"; -import { normalizeAgentPermissions } from "./agent-permissions.js"; import { REDACTED_EVENT_VALUE, sanitizeRecord } from "../redaction.js"; import { assertClaudeOAuthBindingInvariant, @@ -944,6 +945,20 @@ export function agentService(db: Db) { }) .returning() .then((rows) => rows[0]); + // New standard agents receive the standard direct grants at activation. + // Low-trust and bundled agents keep their explicit, narrower grants. + if (created.status !== "pending_approval" && !permissionsImplyLowTrust(normalizedPermissions) && + !readBuiltInAgentMarker(created.metadata)) { + await tx.insert(principalPermissionGrants).values( + NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({ + companyId, + principalType: "agent" as const, + principalId: created.id, + permissionKey, + scope: newStandardAgentGrantScope(permissionKey, created.id), + })), + ).onConflictDoNothing(); + } if (options?.aiConnectionInstall) { const install = options.aiConnectionInstall; const connectionIds = [...new Set([install.connectionId, ...(install.memberConnectionIds ?? [])])]; @@ -1084,6 +1099,11 @@ export function agentService(db: Db) { await tx.delete(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, id)); await tx.delete(agentApiKeys).where(eq(agentApiKeys.agentId, id)); await tx.delete(agentRuntimeState).where(eq(agentRuntimeState.agentId, id)); + await tx.delete(principalPermissionGrants).where(and( + eq(principalPermissionGrants.companyId, existing.companyId), + eq(principalPermissionGrants.principalType, "agent"), + eq(principalPermissionGrants.principalId, id), + )); const deleted = await tx .delete(agents) .where(eq(agents.id, id)) @@ -1151,6 +1171,17 @@ export function agentService(db: Db) { }); } await syncAgentSecretBindings(updated, txDb, existing.adapterConfig); + if (!permissionsImplyLowTrust(updated.permissions) && !readBuiltInAgentMarker(updated.metadata)) { + await tx.insert(principalPermissionGrants).values( + NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({ + companyId: updated.companyId, + principalType: "agent" as const, + principalId: updated.id, + permissionKey, + scope: newStandardAgentGrantScope(permissionKey, updated.id), + })), + ).onConflictDoNothing(); + } await recordResourceCreationEvent(txDb, existing.companyId, "agent", updated.id); const agent = await agentService(txDb).getById(updated.id); if (!agent) { diff --git a/server/src/services/authorization.ts b/server/src/services/authorization.ts index d16e9f30dc..e3edf7a99f 100644 --- a/server/src/services/authorization.ts +++ b/server/src/services/authorization.ts @@ -2060,7 +2060,8 @@ export function authorizationService(db: Db | DbTransaction) { // implicit default-open policy remains responsible-user-only so an // absent row never becomes a company-wide cross-user grant. const grant = await findGrant(companyId, "agent", actorAgentId, "inbox:manage"); - if (grant && (await scopeAllows(db, companyId, grant.scope, { userId: targetUserId }))) { + if (grant && grant.scope?.responsibleUserOnly !== true && + (await scopeAllows(db, companyId, grant.scope, { userId: targetUserId }))) { return allow({ action: input.action, reason: "allow_explicit_grant", diff --git a/server/src/services/execution-projection-read-retry.test.ts b/server/src/services/execution-projection-read-retry.test.ts new file mode 100644 index 0000000000..383835d802 --- /dev/null +++ b/server/src/services/execution-projection-read-retry.test.ts @@ -0,0 +1,150 @@ +import { getTableName, type SQL } from "drizzle-orm"; +import { PgDialect } from "drizzle-orm/pg-core"; +import type { Db } from "@paperclipai/db"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { activityService } from "./activity.js"; +import { executionProjectionsForRuns } from "./execution-projection.js"; +import { logger } from "../middleware/logger.js"; + +const companyId = "00000000-0000-4000-8000-000000000001"; +const issueId = "00000000-0000-4000-8000-000000000002"; +const runId = "00000000-0000-4000-8000-000000000003"; +const now = new Date("2026-01-01T00:00:00Z"); +const tables = ["heartbeat_runs", "native_run_finalizations", "issue_thread_interactions", "issue_recovery_actions"]; +const dialect = new PgDialect(); +function disconnected(code = "CONNECTION_CLOSED") { + return new Error("Failed query", { cause: Object.assign(new Error("Connection failed"), { code }) }); +} + +function fixture(options: { + failures?: Record; + missing?: boolean; + noIssue?: boolean; + backfill?: boolean; + writeFailure?: Error; +} = {}) { + const attempts: Array<{ key: string; params: unknown[]; order: string[] }> = []; + const row = { + id: runId, companyId, runId, status: "succeeded", runtimeMode: "native", errorCode: null, + nativeIssueId: options.noIssue ? null : issueId, contextSnapshot: {}, + finishedAt: now, startedAt: now, createdAt: now, resultJson: null, usageJson: null, + scheduledRetryAt: null, scheduledRetryAttempt: 0, retryOfRunId: null, nextAction: null, + }; + const select = vi.fn((fields: Record = {}) => ({ + from(table: Parameters[0]) { + const name = getTableName(table); + const key = name === "heartbeat_runs" && "runId" in fields ? "run_list" + : name === "heartbeat_runs" && "stdoutExcerpt" in fields ? "liveness_backfill" + : name; + let condition: SQL; + let order: SQL[] = []; + let result: Promise | undefined; + const query = { + where(value: SQL) { condition = value; return query; }, + innerJoin() { return query; }, + orderBy(...values: SQL[]) { order = values; return query; }, + limit() { return query; }, + then(resolve: (value: unknown[]) => unknown, reject: (error: unknown) => unknown) { + // Re-awaiting the same failed builder cannot recover the connection. + result ??= Promise.resolve().then(() => { + const attempt = attempts.filter((read) => read.key === key).length; + attempts.push({ key, params: dialect.sqlToQuery(condition).params, + order: order.map((value) => dialect.sqlToQuery(value).sql) }); + const error = options.failures?.[key]?.[attempt]; + if (error) throw error; + if (options.missing) return []; + if (key === "heartbeat_runs" || key === "run_list") return [row]; + if (key === "liveness_backfill") return options.backfill ? [row] : []; + if (key === "issues") return [{ status: "in_progress", title: "Fixture", workMode: "standard" }]; + if (key === "issue_thread_interactions") return [{ issueId, kind: "ask_user_questions" }]; + return []; + }); + return result.then(resolve, reject); + }, + }; + return query; + }, + })); + const updateWhere = vi.fn(async () => { if (options.writeFailure) throw options.writeFailure; }); + const update = vi.fn(() => ({ set: () => ({ where: updateWhere }) })); + const transaction = vi.fn(() => { throw new Error("Must not start or replay a transaction"); }); + const db = { select, update, transaction } as unknown as Db; + return { db, attempts, select, update, updateWhere, transaction, + count: (key: string) => attempts.filter((read) => read.key === key).length }; +} + +afterEach(() => vi.restoreAllMocks()); + +describe("execution projection read retries", () => { + it.each(tables)("rebuilds only the failed %s SELECT within the existing retry budget", async (table) => { + const test = fixture({ failures: { [table]: [disconnected(), disconnected("CONNECTION_ENDED")] } }); + const result = await executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true }); + expect(result.get(runId)).toMatchObject({ phase: "waiting_for_answer", label: "Waiting for answer" }); + for (const name of tables) expect(test.count(name)).toBe(name === table ? 3 : 1); + expect(test.select).toHaveBeenCalledTimes(6); + const reads = test.attempts.filter((read) => read.key === table); + expect(reads[1]).toEqual(reads[0]); + expect(reads[2]).toEqual(reads[0]); + expect(reads[0].params).toContain(companyId); + expect(reads[0].params).toContain(table === "heartbeat_runs" || table === "native_run_finalizations" ? runId : issueId); + if (table === "issue_thread_interactions") expect(reads[0].params).toContain("pending"); + if (table === "issue_recovery_actions") expect(reads[0].order).toEqual(['"issue_recovery_actions"."updated_at" desc']); + expect(test.update).not.toHaveBeenCalled(); + expect(test.transaction).not.toHaveBeenCalled(); + }); + + it.each(tables)("preserves the last %s error after three attempts", async (table) => { + const errors = [disconnected(), disconnected(), disconnected()]; + const test = fixture({ failures: { [table]: errors } }); + await expect(executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true })).rejects.toBe(errors[2]); + for (const [index, name] of tables.entries()) { + expect(test.count(name)).toBe(index < tables.indexOf(table) ? 1 : name === table ? 3 : 0); + } + }); + + it.each(tables)("keeps the default transaction-compatible %s path single-attempt", async (table) => { + const error = disconnected(); + const test = fixture({ failures: { [table]: [error] } }); + await expect(executionProjectionsForRuns(test.db, companyId, [runId], now)).rejects.toBe(error); + expect(test.count(table)).toBe(1); + expect(test.transaction).not.toHaveBeenCalled(); + }); + + it.each([new Error("CONNECTION_CLOSED"), disconnected("28P01"), disconnected("42501"), disconnected("22P02"), disconnected("25P02")])( + "does not retry an unclassified or non-transient error", async (error) => { + const test = fixture({ failures: { issue_thread_interactions: [error] } }); + await expect(executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true })).rejects.toBe(error); + expect(test.count("issue_thread_interactions")).toBe(1); + expect(test.count("issue_recovery_actions")).toBe(0); + }, + ); + + it("preserves empty and taskless projection behavior without unnecessary reads", async () => { + const empty = fixture(); + expect(await executionProjectionsForRuns(empty.db, companyId, [], now, { retryDatabaseReads: true })).toEqual(new Map()); + expect(empty.select).not.toHaveBeenCalled(); + for (const options of [{ missing: true }, { noIssue: true }]) { + const test = fixture(options); + const result = await executionProjectionsForRuns(test.db, companyId, [runId], now, { retryDatabaseReads: true }); + expect(test.select).toHaveBeenCalledTimes(2); + expect(options.missing ? result.size : result.get(runId)?.phase).toBe(options.missing ? 0 : "completed"); + } + }); + + it.each([false, true])("the run list retries projection reads without replaying other reads or backfill writes (write fails: %s)", async (writeFails) => { + vi.spyOn(logger, "warn").mockImplementation(() => undefined); + const test = fixture({ backfill: true, writeFailure: writeFails ? disconnected() : undefined, + failures: { issue_thread_interactions: [disconnected(), disconnected()] } }); + const result = await activityService(test.db).runsForIssue(companyId, issueId); + expect(result).toMatchObject([{ runId, execution: { phase: "waiting_for_answer" } }]); + expect(test.count("issue_thread_interactions")).toBe(3); + for (const key of ["run_list", "liveness_backfill", "heartbeat_runs", "native_run_finalizations", "environment_leases", "issue_recovery_actions"]) { + expect(test.count(key)).toBe(1); + } + // One retry-exhaustion lookup plus the independent backfill's event count. + expect(test.count("heartbeat_run_events")).toBe(2); + expect(test.update).toHaveBeenCalledTimes(1); + expect(test.updateWhere).toHaveBeenCalledTimes(1); + expect(test.transaction).not.toHaveBeenCalled(); + }); +}); diff --git a/server/src/services/execution-projection.ts b/server/src/services/execution-projection.ts index cab39d73e3..ae88b4b739 100644 --- a/server/src/services/execution-projection.ts +++ b/server/src/services/execution-projection.ts @@ -9,6 +9,7 @@ import { import type { ExecutionProjection } from "@paperclipai/shared"; import { EXECUTION_CONTROL_DEADLINE_MS } from "./execution-control-deadline.js"; import { executionFailureRetryCount } from "./execution-recovery-attempt.js"; +import { retryIdempotentDatabaseOperation } from "../database-retry.js"; const text = (v: unknown) => (typeof v === "string" ? v : null); const executionRunColumns = { id: heartbeatRuns.id, @@ -48,10 +49,16 @@ export async function executionProjectionsForRuns( companyId: string, runIds: string[], now = new Date(), + options: { retryDatabaseReads?: boolean } = {}, ) { const projections = new Map(); if (!runIds.length) return projections; - const runs = await db + // Opt in only with a pooled Db outside a transaction. Issue enrichment also + // calls this helper with a transaction, whose failed reads must not replay. + const read = (query: () => Promise) => options.retryDatabaseReads + ? retryIdempotentDatabaseOperation(query) + : query(); + const runs = await read(async () => db .select(executionRunColumns) .from(heartbeatRuns) .where( @@ -59,8 +66,8 @@ export async function executionProjectionsForRuns( eq(heartbeatRuns.companyId, companyId), inArray(heartbeatRuns.id, runIds), ), - ); - const coordinators = await db + )); + const coordinators = await read(async () => db .select() .from(nativeRunFinalizations) .where( @@ -68,7 +75,7 @@ export async function executionProjectionsForRuns( eq(nativeRunFinalizations.companyId, companyId), inArray(nativeRunFinalizations.runId, runIds), ), - ); + )); const issueIds = [ ...new Set( runs @@ -77,7 +84,7 @@ export async function executionProjectionsForRuns( ), ]; const pending = issueIds.length - ? await db + ? await read(async () => db .select({ issueId: issueThreadInteractions.issueId, kind: issueThreadInteractions.kind, @@ -89,10 +96,10 @@ export async function executionProjectionsForRuns( inArray(issueThreadInteractions.issueId, issueIds), eq(issueThreadInteractions.status, "pending"), ), - ) + )) : []; const recovery = issueIds.length - ? await db + ? await read(async () => db .select({ issueId: issueRecoveryActions.sourceIssueId, cause: issueRecoveryActions.cause, @@ -112,7 +119,7 @@ export async function executionProjectionsForRuns( ]), ), ) - .orderBy(desc(issueRecoveryActions.updatedAt)) + .orderBy(desc(issueRecoveryActions.updatedAt))) : []; const coordinatorByRun = new Map(coordinators.map((row) => [row.runId, row])); for (const run of runs) { diff --git a/server/src/services/invite-grants.ts b/server/src/services/invite-grants.ts index 32c9aa42fe..b2141e0930 100644 --- a/server/src/services/invite-grants.ts +++ b/server/src/services/invite-grants.ts @@ -1,5 +1,6 @@ import { PERMISSION_KEYS } from "@paperclipai/shared"; import type { HumanCompanyMembershipRole } from "@paperclipai/shared"; +import { NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS, newStandardAgentGrantScope } from "./agent-permissions.js"; import { grantsForHumanRole } from "./company-member-roles.js"; export function grantsFromDefaults( @@ -38,22 +39,21 @@ export function grantsFromDefaults( } export function agentJoinGrantsFromDefaults( - defaultsPayload: Record | null | undefined + defaultsPayload: Record | null | undefined, + agentId: string, ): Array<{ permissionKey: (typeof PERMISSION_KEYS)[number]; scope: Record | null; }> { const grants = grantsFromDefaults(defaultsPayload, "agent"); - if (grants.some((grant) => grant.permissionKey === "tasks:assign")) { - return grants; + // Invitation approval replaces the whole grant set after agent creation. + // Retain new-agent defaults, but never widen an explicitly scoped grant. + for (const permissionKey of NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS) { + if (!grants.some((grant) => grant.permissionKey === permissionKey)) { + grants.push({ permissionKey, scope: newStandardAgentGrantScope(permissionKey, agentId) }); + } } - return [ - ...grants, - { - permissionKey: "tasks:assign", - scope: null, - }, - ]; + return grants; } export function humanJoinGrantsFromDefaults( diff --git a/server/src/services/native-runtime/native-provider-failure-evidence.ts b/server/src/services/native-runtime/native-provider-failure-evidence.ts new file mode 100644 index 0000000000..e97143aca9 --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure-evidence.ts @@ -0,0 +1,45 @@ +import { and, desc, eq, sql } from "drizzle-orm"; +import { heartbeatRunEvents, heartbeatRuns, type Db } from "@paperclipai/db"; +import { parseNativeExecutionInput, validatePrpEvent, type PrpTerminalState } from "../../vendor/paperclip-runner/index.js"; +import { nativeSha256 } from "./canonical.js"; +import { createNativeProviderFailureObservation } from "./native-provider-failure.js"; + +function record(value: unknown): Record { + return value !== null && typeof value === "object" && !Array.isArray(value) + ? value as Record : {}; +} + +/** Re-derive authority from immutable run-bound evidence, including after restart. */ +export async function readPersistedNativeModelRejection( + db: Db, + run: typeof heartbeatRuns.$inferSelect, + turnId: string | null, + terminal: PrpTerminalState, +) { + if (run.runtimeMode !== "native" || !turnId || !run.nativeSessionId || !run.runnerInstanceId || + terminal.turnTerminalState !== "failed" || terminal.runTerminalState !== "failed") return null; + let execution; + try { execution = parseNativeExecutionInput(record(run.runnerProfileJson).nativeExecutionInput); } catch { return null; } + if (execution.provider.kind !== "codex" || !execution.provider.model || + execution.binding.runId !== run.id || execution.binding.companyId !== run.companyId || + execution.binding.agentId !== run.agentId || execution.binding.issueId !== run.nativeIssueId || + execution.session.normalizedSessionId !== run.nativeSessionId || + execution.completionContract.id !== run.completionContractId || + execution.completionContract.sha256 !== run.completionContractSha256) return null; + + const [row] = await db.select().from(heartbeatRunEvents).where(and( + eq(heartbeatRunEvents.companyId, run.companyId), eq(heartbeatRunEvents.runId, run.id), + eq(heartbeatRunEvents.sourceInstanceId, run.runnerInstanceId), eq(heartbeatRunEvents.eventType, "turn.failed"), + sql`${heartbeatRunEvents.payload}->'prpEvent'->>'turnId' = ${turnId}`, + )).orderBy(desc(heartbeatRunEvents.seq)).limit(1); + if (!row) return null; + const parsed = validatePrpEvent(record(row.payload).prpEvent); + if (!parsed.ok || parsed.event.runId !== run.id || parsed.event.normalizedSessionId !== run.nativeSessionId || + parsed.event.sourceKind !== "runner" || parsed.event.sourceInstanceId !== run.runnerInstanceId || + parsed.event.sourceEventId !== row.sourceEventId || parsed.event.sourceSeq !== row.sourceSeq || + parsed.event.eventType !== row.eventType || parsed.event.turnId !== turnId || + row.sourcePayloadSha256 !== nativeSha256(parsed.event)) return null; + const observed = createNativeProviderFailureObservation(execution.provider); + observed.observe(parsed.event); + return observed.forTerminal(turnId, terminal); +} diff --git a/server/src/services/native-runtime/native-provider-failure.integration.test.ts b/server/src/services/native-runtime/native-provider-failure.integration.test.ts new file mode 100644 index 0000000000..2da7891abb --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure.integration.test.ts @@ -0,0 +1,272 @@ +import { randomUUID } from "node:crypto"; +import { eq } from "drizzle-orm"; +import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { agents, agentWakeupRequests, companies, completionContracts, createDb, heartbeatRunEvents, heartbeatRuns, issueRecoveryActions, issueThreadInteractions, issues, nativeRunFinalizations, nativeRunResults, statusDecisions, workAssessments } from "@paperclipai/db"; +import type { ControlPlanePort, NativeExecutionInputV1, PrpEvent, PrpTerminalState } from "@paperclipai/paperclip-runner"; +import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js"; +import { CONTROL_PLANE_CONFORMANCE_RESULT } from "../../vendor/paperclip-runner/testing.js"; + +const provider = vi.hoisted(() => ({ execute: vi.fn() })); +vi.mock("../../vendor/paperclip-runner/index.js", async (importOriginal) => ({ + ...(await importOriginal()), + executeNativeSession: provider.execute, +})); + +import { executePaperclipNativeSession } from "./native-session-executor.js"; +import { buildNativeCompletionContract } from "./completion-contracts.js"; +import { PaperclipControlPlanePort } from "./paperclip-control-plane-port.js"; +import { prepareNativeHeartbeatRun } from "./prepare-native-run.js"; +import { NATIVE_MODEL_REJECTION_DIAGNOSTIC, NATIVE_MODEL_REJECTION_MESSAGE } from "./native-provider-failure.js"; +import { collectRunFailureDiagnostics } from "../run-failure-diagnostics.js"; +import { finalizeNativeRun } from "./native-run-finalizer.js"; +import { readPersistedNativeModelRejection } from "./native-provider-failure-evidence.js"; +import { heartbeatService } from "../heartbeat.js"; +import { getNativeReviewAssignment, readNativeReviewAssignmentContext } from "./native-review-participant.js"; +import { claimQueuedNativeReviewRun } from "./native-review-dispatch.js"; +import { commitNativeStatusDecision, NativeStatusRaceError } from "./status-decision-committer.js"; +import { NATIVE_STATUS_ARBITER_POLICY_VERSION } from "./status-arbiter.js"; +import { issueThreadInteractionService } from "../issue-thread-interactions.js"; +import { createReleaseIssueExecution } from "../../modules/wake-queue/application/use-cases.js"; +import { createPostgresWakeQueueAdapter } from "../../modules/wake-queue/adapters/postgres.js"; + +describe("native completed failure diagnostics", () => { + let temporary: Awaited>; + let db: ReturnType; + const companyId = randomUUID(); + const agentId = randomUUID(); + const workerId = randomUUID(); + const model = "test-model"; + + beforeAll(async () => { + temporary = await startEmbeddedPostgresTestDatabase("native-model-failure-"); + db = createDb(temporary.connectionString); + await db.insert(companies).values({ id: companyId, name: "Native failure regression", issuePrefix: "NMF" }); + await db.insert(agents).values({ id: agentId, companyId, name: "Native failure agent", status: "active", adapterType: "paperclip_runner", adapterConfig: { provider: "codex", model } }); + await db.insert(agents).values({ id: workerId, companyId, name: "Review source worker", status: "active", adapterType: "paperclip_runner" }); + }); + afterAll(async () => { await temporary?.cleanup(); }); + + it.each(["committed", "duplicate", "pending-review", "superseded-review", "resolved-review", "reassigned", "successor-race", "different-turn", "unknown-error", "untrusted-source", "untrusted-run", "untrusted-session", "succeeded", "cancelled"] as const)( + "preserves accepted results and only labels the matching provider rejection (%s)", async (scenario) => { + const issueId = randomUUID(); + const runId = randomUUID(); + const [issue] = await db.insert(issues).values({ id: issueId, companyId, title: "Explain a provider rejection", status: "in_progress", workMode: "standard", assigneeAgentId: agentId }).returning(); + const [run] = await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native", nativeIssueId: issueId, invocationSource: "assignment", triggerDetail: "system", contextSnapshot: { issueId } }).returning(); + await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId)); + const native = await prepareNativeHeartbeatRun({ db, run, issue, environmentLeaseId: randomUUID() }); + const [contract] = await db.select().from(completionContracts).where(eq(completionContracts.issueId, issueId)); + const completionInput = buildNativeCompletionContract(issue, { revision: contract.revision }); + await db.insert(nativeRunFinalizations).values({ runId, companyId, issueId, phase: "observed" }); + const execution: NativeExecutionInputV1 = { + schema: "paperclip.native-execution-input.v1", + binding: { companyId, runId, issueId, agentId, executionWorkspaceId: runId }, + provider: { kind: "codex", model }, + task: { identifier: issueId, title: issue.title, description: null, prompt: "Reply to the current message.", workMode: "standard" }, + workspace: { cwd: "/tmp", repoUrl: null, repoRef: null, branchName: null }, + session: { normalizedSessionId: native.normalizedSessionId, driverKind: "codex_app_server", protocolVersion: 1, lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null } }, + completionContract: { id: contract.id, sha256: contract.canonicalSha256, schemaVersion: "paperclip.completion-contract.v1", contract: completionInput }, + interactionResponses: [], credentialBindings: [], + }; + await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeExecutionInput: execution } }).where(eq(heartbeatRuns.id, runId)); + if (scenario.endsWith("-review")) { + const sourceRunId = randomUUID(), sourceResultId = randomUUID(), assessmentId = randomUUID(); + const decisionId = randomUUID(), interactionId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId, agentId: workerId, nativeIssueId: issueId, runtimeMode: "native", status: "succeeded", completionContractId: contract.id, completionContractSha256: contract.canonicalSha256 }); + await db.insert(nativeRunResults).values({ id: sourceResultId, companyId, issueId, runId: sourceRunId, completionContractId: contract.id, serverFingerprint: sourceResultId, schemaStatus: "accepted", resultJson: {}, canonicalSha256: sourceResultId }); + await db.insert(workAssessments).values({ id: assessmentId, companyId, issueId, runId: sourceRunId, contractId: contract.id, resultId: sourceResultId, triggerKind: "native_result", triggerActorCompanyId: companyId, priorIssueStatus: "in_progress", priorStatusVersion: 0, policyVersion: "fixture", assessmentJson: {}, inputDigest: assessmentId }); + const [reviewIssue] = await db.update(issues).set({ status: "in_review", assigneeAgentId: workerId }).where(eq(issues.id, issueId)).returning(); + await db.insert(statusDecisions).values({ id: decisionId, companyId, issueId, runId: sourceRunId, assessmentId, decisionVersion: 1, policyVersion: "fixture", fromStatus: "in_progress", toStatus: "in_review", reasonCode: "explicit_review", decisionJson: { projectedStatusVersion: reviewIssue.statusVersion }, decisionDigest: decisionId, applicationState: "applied" }); + await db.update(issues).set({ lastStatusDecisionId: decisionId }).where(eq(issues.id, issueId)); + await db.insert(issueThreadInteractions).values({ id: interactionId, companyId, issueId, kind: "request_confirmation", sourceRunId, addresseeAgentId: agentId, createdByAgentId: workerId, + continuationPolicy: "wake_assignee", + status: scenario === "resolved-review" ? "accepted" : "pending", + ...(scenario === "resolved-review" ? { resolvedByRunId: runId, resolvedByAgentId: agentId } : {}), + payload: { version: 1, prompt: "Review the completed work.", target: { type: "custom", key: "native_completion_review", revisionId: decisionId } }, + }); + const contextSnapshot = { issueId, wakeReason: "native_completion_review", nativeReviewInteractionId: interactionId, nativeReviewDecisionId: decisionId }; + await db.update(heartbeatRuns).set({ contextSnapshot }).where(eq(heartbeatRuns.id, runId)); + if (scenario === "superseded-review") await db.update(issues).set({ lastStatusDecisionId: null }).where(eq(issues.id, issueId)); + const review = await getNativeReviewAssignment(db, { companyId, issueId, agentId, contextSnapshot, allowResolvedByRunId: runId }); + if (scenario === "superseded-review") expect(review).toBeNull(); + else expect(review?.interaction.status).toBe(scenario === "resolved-review" ? "accepted" : "pending"); + } + const event: PrpEvent = { + schema: "paperclip.prp.event.v1", runId, normalizedSessionId: native.normalizedSessionId, + turnId: native.turnId, sourceInstanceId: native.runnerInstanceId, sourceEventId: "model-terminal", sourceSeq: 1, + sourceKind: "runner", eventType: "turn.failed", schemaVersion: 1, priority: 0, emittedAt: new Date().toISOString(), + payload: { status: "failed", error: { codexErrorInfo: "other", message: JSON.stringify({ + type: "error", status: 400, error: { type: "invalid_request_error", message: scenario === "unknown-error" ? "Private unknown failure" : `The '${model}' model is not supported when using Codex with a ChatGPT account.` }, + }) } }, + }; + const openInput = { identity: { companyId, issueId, runId, agentId, sessionId: native.normalizedSessionId }, backendKind: "mock" as const, sourceInstanceId: native.runnerInstanceId }; + if (scenario === "duplicate") { + const prior = new PaperclipControlPlanePort(db, { companyId, issueId, runId, agentId, sessionId: native.normalizedSessionId, sourceInstanceId: native.runnerInstanceId, controlPlaneSourceInstanceId: "prior-controller", completionContractId: contract.id, completionContractSha256: contract.canonicalSha256 }); + await prior.openRun(openInput); + expect((await prior.appendEvent(event)).disposition).toBe("committed"); + } + const turnId = scenario === "different-turn" ? "later-turn" : native.turnId; + const terminal: PrpTerminalState = { + schema: "paperclip.prp.terminal.v1", turnTerminalState: scenario === "succeeded" ? "completed" : scenario === "cancelled" ? "cancelled" : "failed", + runTerminalState: scenario === "succeeded" ? "succeeded" : scenario === "cancelled" ? "cancelled" : "failed", reportedWorkDisposition: "blocked", + }; + const result = { ...CONTROL_PLANE_CONFORMANCE_RESULT, reportedWorkDisposition: "blocked" as const, + blocker: { reasonCode: "provider_rejected", owner: { kind: "user" as const, name: "Task owner" }, unblockAction: "Select a compatible connection.", scope: "task_wide" as const }, + completionClaim: { ...CONTROL_PLANE_CONFORMANCE_RESULT.completionClaim, contractRevision: completionInput.revision } }; + provider.execute.mockReset().mockImplementation(async (options: { controlPlane: ControlPlanePort }) => { + await options.controlPlane.openRun(openInput); + if (scenario === "untrusted-source") { + await expect(options.controlPlane.appendEvent({ ...event, sourceInstanceId: "wrong-runner" })).rejects.toThrow("native_event_source_binding_mismatch"); + } else if (scenario === "untrusted-run" || scenario === "untrusted-session") { + await expect(options.controlPlane.appendEvent({ ...event, + ...(scenario === "untrusted-run" ? { runId: randomUUID() } : { normalizedSessionId: "wrong-session" }), + })).rejects.toThrow("native_event_binding_mismatch"); + } else { + expect((await options.controlPlane.appendEvent(event)).disposition).toBe(scenario === "duplicate" ? "duplicate" : "committed"); + } + await options.controlPlane.completeRun({ result, terminal, turnId }); + return { result, terminal, turnId, normalizedSessionId: native.normalizedSessionId, providerSessionId: "fixture-provider-session", driverKind: "codex_app_server", driverVersion: "fixture", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null }; + }); + const adapter = await executePaperclipNativeSession({ db, execution, runnerInstanceId: native.runnerInstanceId, backend: { + descriptor: async () => { throw new Error("Unexpected provider call"); }, + openSession: async () => { throw new Error("Unexpected provider call"); }, + } }); + const classified = scenario === "committed" || scenario === "duplicate" || scenario.endsWith("-review") || scenario === "reassigned" || scenario === "successor-race"; + const reviewAuthorityEnded = scenario === "superseded-review" || scenario === "resolved-review"; + const ownerChanged = scenario === "reassigned" || scenario === "successor-race"; + expect(adapter.errorCode).toBe(classified ? "native_provider_model_rejected" : undefined); + expect(adapter.errorMessage).toBe(classified ? NATIVE_MODEL_REJECTION_MESSAGE : scenario === "succeeded" ? null : `Native session ${terminal.runTerminalState}`); + expect(adapter.resultJson?.nativeProviderFailure).toEqual(classified ? NATIVE_MODEL_REJECTION_DIAGNOSTIC : undefined); + expect(adapter.nativeFinalization).toMatchObject({ result, terminal, turnId, workspaceFinalizeStatus: "pending" }); + expect(provider.execute).toHaveBeenCalledTimes(1); + expect(execution.provider).toEqual({ kind: "codex", model }); + const [coordinator] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, runId)); + expect(coordinator).toMatchObject({ phase: "workspace_finalizing", leaseOwner: null, attempt: 1, nextAttemptAt: null }); + expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId))).toHaveLength(1); + expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(0); + expect(await db.select().from(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, runId))).toHaveLength(scenario.startsWith("untrusted-") ? 0 : 1); + const diagnostics = collectRunFailureDiagnostics({ ...run, resultJson: adapter.resultJson ?? null }, {}); + expect(diagnostics.provider).toEqual(classified ? NATIVE_MODEL_REJECTION_DIAGNOSTIC : {}); + expect(JSON.stringify(diagnostics)).not.toContain(model); + expect(JSON.stringify(diagnostics)).not.toContain("Private"); + const [persistedRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId)); + expect(persistedRun.errorCode).toBeNull(); // No adapter metadata has been saved yet. + expect(await readPersistedNativeModelRejection(db, persistedRun, turnId, terminal)).toEqual(classified ? { + errorCode: "native_provider_model_rejected", errorMessage: NATIVE_MODEL_REJECTION_MESSAGE, diagnostic: NATIVE_MODEL_REJECTION_DIAGNOSTIC, + } : null); + if (classified) { + if (scenario === "committed") { + // Neither saved display diagnostics nor a different execution binding + // can confer the right to change the finalizer's retry decision. + for (const changed of [ + { ...persistedRun, runnerProfileJson: null }, + { ...persistedRun, nativeSessionId: "foreign-session" }, + { ...persistedRun, runnerInstanceId: "foreign-source" }, + { ...persistedRun, companyId: randomUUID() }, + { ...persistedRun, completionContractSha256: "foreign-contract" }, + { ...persistedRun, runnerProfileJson: { nativeExecutionInput: { ...execution, provider: { kind: "codex", model: "different-model" } } } }, + { ...persistedRun, runnerProfileJson: { nativeExecutionInput: { ...execution, binding: { ...execution.binding, runId: randomUUID() } } } }, + ]) expect(await readPersistedNativeModelRejection(db, changed, turnId, terminal)).toBeNull(); + const [stored] = await db.select().from(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, runId)); + await db.update(heartbeatRunEvents).set({ sourcePayloadSha256: "invalid-hash" }).where(eq(heartbeatRunEvents.id, stored.id)); + expect(await readPersistedNativeModelRejection(db, persistedRun, turnId, terminal)).toBeNull(); + await db.update(heartbeatRunEvents).set({ sourcePayloadSha256: stored.sourcePayloadSha256 }).where(eq(heartbeatRunEvents.id, stored.id)); + } + // Simulate a restart after native result acceptance, before heartbeat's + // adapter update. Finalization must use durable provider evidence. + const acceptedBefore = await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId)); + if (scenario === "reassigned") { + await db.update(issues).set({ assigneeAgentId: workerId, executionRunId: null }).where(eq(issues.id, issueId)); + } else if (scenario === "successor-race") { + const [snapshot] = await db.select().from(issues).where(eq(issues.id, issueId)); + const newerRunId = randomUUID(), raceAssessmentId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: newerRunId, companyId, agentId, status: "running", nativeIssueId: issueId }); + await db.update(issues).set({ executionRunId: newerRunId }).where(eq(issues.id, issueId)); + const [changed] = await db.select().from(issues).where(eq(issues.id, issueId)); + expect(changed.statusVersion).toBe(snapshot.statusVersion); + await db.insert(workAssessments).values({ id: raceAssessmentId, companyId, issueId, runId, contractId: contract.id, resultId: acceptedBefore[0].id, triggerKind: "native_result", triggerActorCompanyId: companyId, priorIssueStatus: snapshot.status, priorStatusVersion: snapshot.statusVersion, policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, assessmentJson: {}, inputDigest: raceAssessmentId }); + await expect(commitNativeStatusDecision({ db, companyId, issueId, runId, assessmentId: raceAssessmentId, + priorStatus: snapshot.status, priorStatusVersion: snapshot.statusVersion, priorDecisionId: snapshot.lastStatusDecisionId, + requireModelRejectionOwner: { agentId, reviewContext: null }, + decision: { policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, statusAction: "blocked", toStatus: "blocked", reasonCode: "native_provider_model_rejected", unblockDescriptor: { owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }, effects: [{ kind: "bind_blocker", owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }] }, + })).rejects.toBeInstanceOf(NativeStatusRaceError); + expect(await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId))).toHaveLength(0); + } + for (let retry = 0; retry < 2; retry++) { + await finalizeNativeRun({ db, runId, workspaceFinalizeStatus: "succeeded", projectRunStatus: true }); + } + expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId))).toEqual(acceptedBefore); + const [finalRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId)); + expect(finalRun).toMatchObject({ status: "failed", errorCode: "native_provider_model_rejected", error: NATIVE_MODEL_REJECTION_MESSAGE }); + const [finalIssue] = await db.select().from(issues).where(eq(issues.id, issueId)); + expect(finalIssue.status).toBe(scenario.endsWith("-review") ? "in_review" : ownerChanged ? "in_progress" : "blocked"); + const decisions = await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId)); + expect(decisions).toHaveLength(1); + expect(decisions[0].reasonCode).toBe(reviewAuthorityEnded ? "native_review_action_finished" : ownerChanged ? "run_failed_partial_evidence_preserved" : "native_provider_model_rejected"); + const wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId)); + expect(wakes.filter(wake => wake.payload?.issueId === issueId)).toHaveLength(ownerChanged ? 1 : 0); + if (!ownerChanged) expect(await heartbeatService(db).dispatchPendingNativeStatusWakeups({ companyId })).toEqual({ scanned: 0, dispatched: 0, recovered: 0, deferred: 0 }); + expect(provider.execute).toHaveBeenCalledTimes(1); + if (scenario === "pending-review") { + const reviewContext = readNativeReviewAssignmentContext(finalRun.contextSnapshot)!; + expect(finalIssue).toMatchObject({ lastStatusDecisionId: reviewContext.nativeReviewDecisionId, executionRunId: null, checkoutRunId: null }); + const originalReview = await getNativeReviewAssignment(db, { companyId, issueId, agentId, contextSnapshot: reviewContext }); + expect(originalReview?.interaction.status).toBe("pending"); + expect(finalIssue.statusVersion).toBe(originalReview!.sourceDecision.decisionJson.projectedStatusVersion); + expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(0); + + // Exercise the real release and periodic recovery lanes, rather than + // assuming that absence of a finalizer wake prevents later retries. + const unexpected = vi.fn(async () => { throw new Error("Unexpected automatic recovery"); }); + const release = createReleaseIssueExecution({ + issueLock: createPostgresWakeQueueAdapter(db, { resolveResponsibleUserId: unexpected, getRoutineEnv: unexpected, resolveSessionBeforeForWakeup: unexpected }), + recovery: { escalateStrandedAssignedIssue: unexpected, escalateStrandedRecoveryIssueInPlace: unexpected }, + }); + expect((await release({ companyId, runId, now: new Date() })).outcome.kind).toBe("released"); + await heartbeatService(db).reconcileStrandedAssignedIssues(); + expect(unexpected).not.toHaveBeenCalled(); + expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(0); + expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId))).filter(wake => wake.payload?.issueId === issueId)).toHaveLength(0); + const [afterRecovery] = await db.select().from(issues).where(eq(issues.id, issueId)); + expect(afterRecovery).toMatchObject({ status: "in_review", statusVersion: finalIssue.statusVersion, lastStatusDecisionId: finalIssue.lastStatusDecisionId }); + + // The operator repairs configuration and explicitly retries the + // assigned reviewer. No review rebinding or status reset is needed. + await db.update(agents).set({ adapterConfig: { provider: "codex", model: "repaired-model" }, runtimeConfig: { heartbeat: { wakeOnDemand: true, maxConcurrentRuns: 1 } } }).where(eq(agents.id, agentId)); + // Occupy the execution slot to exercise real wake admission without + // starting a provider. The normal atomic review claim follows below. + const occupiedRunId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: occupiedRunId, companyId, agentId, status: "running", startedAt: new Date() }); + const retryRun = await heartbeatService(db, { runtimeEnv: {} }).wakeup(agentId, { + failedRunId: runId, source: "on_demand", triggerDetail: "manual", reason: "retry_failed_run", + requestedByActorType: "user", requestedByActorId: "review-repair-operator", + payload: { issueId }, contextSnapshot: { ...reviewContext }, + }); + expect(retryRun).toMatchObject({ status: "queued", retryOfRunId: runId, contextSnapshot: { issueId, ...reviewContext } }); + const retryRunId = retryRun!.id; + await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date() }).where(eq(heartbeatRuns.id, occupiedRunId)); + expect(await claimQueuedNativeReviewRun(db, { run: retryRun!, agentNameKey: "reviewer", claimedAt: new Date(), claimValues: { nativeIssueId: issueId, runtimeMode: "native" } })) + .toMatchObject({ id: retryRunId, status: "running" }); + expect(await getNativeReviewAssignment(db, { companyId, issueId, agentId, contextSnapshot: reviewContext, actingRunId: retryRunId, issueExecutionRunId: retryRunId })).not.toBeNull(); + await issueThreadInteractionService(db).acceptInteraction( + { id: issueId, companyId, projectId: null, goalId: null, status: "in_review" }, + reviewContext.nativeReviewInteractionId, {}, { agentId, runId: retryRunId }, + ); + const [resolved] = await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, reviewContext.nativeReviewInteractionId)); + expect(resolved).toMatchObject({ status: "accepted", resolvedByAgentId: agentId, resolvedByRunId: retryRunId }); + expect(await db.select().from(issues).where(eq(issues.id, issueId))).toMatchObject([{ status: "done", assigneeAgentId: workerId }]); + expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, runId))).toEqual(acceptedBefore); + expect(provider.execute).toHaveBeenCalledTimes(1); + await db.update(agents).set({ adapterConfig: { provider: "codex", model } }).where(eq(agents.id, agentId)); + } + } else if (scenario === "unknown-error") { + // A lookalike diagnostic flag does not alter unknown failed-result policy. + await db.update(heartbeatRuns).set({ resultJson: { nativeProviderFailure: NATIVE_MODEL_REJECTION_DIAGNOSTIC } }).where(eq(heartbeatRuns.id, runId)); + await finalizeNativeRun({ db, runId, workspaceFinalizeStatus: "succeeded" }); + const [decision] = await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId)); + expect(decision.reasonCode).toBe("run_failed_partial_evidence_preserved"); + const wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId)); + expect(wakes.filter(wake => wake.payload?.issueId === issueId)).toHaveLength(1); + } + }, + ); +}); diff --git a/server/src/services/native-runtime/native-provider-failure.test.ts b/server/src/services/native-runtime/native-provider-failure.test.ts new file mode 100644 index 0000000000..5db2942e4c --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure.test.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from "vitest"; +import type { PrpEvent, PrpTerminalState } from "../../vendor/paperclip-runner/index.js"; +import { createNativeProviderFailureObservation, NATIVE_MODEL_REJECTION_DIAGNOSTIC, NATIVE_MODEL_REJECTION_MESSAGE } from "./native-provider-failure.js"; + +const model = "test-model"; +const response = { + type: "error", status: 400, + error: { type: "invalid_request_error", message: `The '${model}' model is not supported when using Codex with a ChatGPT account.` }, +}; +const event: PrpEvent = { + schema: "paperclip.prp.event.v1", runId: "run", normalizedSessionId: "session", turnId: "turn", + sourceInstanceId: "runner", sourceEventId: "failure", sourceSeq: 1, sourceKind: "runner", + eventType: "turn.failed", schemaVersion: 1, priority: 0, emittedAt: "2026-01-01T00:00:00.000Z", + payload: { status: "failed", error: { message: JSON.stringify(response), codexErrorInfo: "other" } }, +}; +const terminal: PrpTerminalState = { + schema: "paperclip.prp.terminal.v1", turnTerminalState: "failed", runTerminalState: "failed", reportedWorkDisposition: "blocked", +}; +const observation = () => createNativeProviderFailureObservation({ kind: "codex", model }); + +describe("native provider failure observation", () => { + it("binds the structured rejection to the failed turn and emits no response or model text", () => { + const observed = observation(); + observed.observe(event); + observed.observe(event); // A byte-identical committed replay remains idempotent. + const result = observed.forTerminal("turn", terminal); + expect(result).toEqual({ errorCode: "native_provider_model_rejected", errorMessage: NATIVE_MODEL_REJECTION_MESSAGE, diagnostic: NATIVE_MODEL_REJECTION_DIAGNOSTIC }); + expect(JSON.stringify(result)).not.toContain(model); + expect(JSON.stringify(result)).not.toContain("invalid_request_error"); + expect(observed.forTerminal("later-turn", terminal)).toBeNull(); + expect(observed.forTerminal(null, terminal)).toBeNull(); + expect(observed.forTerminal("turn", { ...terminal, runTerminalState: "succeeded" })).toBeNull(); + expect(observed.forTerminal("turn", { ...terminal, turnTerminalState: "cancelled" })).toBeNull(); + }); + + it.each([ + { ...event, sourceKind: "control_plane" as const }, + { ...event, eventType: "item.completed" as const }, + { ...event, eventType: "harness.diagnostic" as const }, + { ...event, turnId: undefined }, + { ...event, payload: { ...event.payload, status: "completed" } }, + { ...event, payload: { ...event.payload, recoverable: true } }, + { ...event, payload: { status: "failed", error: { message: JSON.stringify(response), recoverable: true } } }, + ])("ignores non-authoritative or retryable evidence %#", (unrelated) => { + const observed = observation(); + observed.observe(unrelated); + expect(observed.forTerminal("turn", terminal)).toBeNull(); + }); + + it.each([ + response.error.message, "malformed JSON", " ".repeat(4_097), + JSON.stringify({ ...response, status: 401 }), + JSON.stringify({ ...response, status: "400" }), + JSON.stringify({ ...response, type: "notice" }), + JSON.stringify({ ...response, error: { ...response.error, type: "server_error" } }), + JSON.stringify({ ...response, error: { ...response.error, message: "Private arbitrary provider text" } }), + JSON.stringify({ ...response, error: { ...response.error, message: response.error.message + " Private suffix" } }), + JSON.stringify({ ...response, error: { ...response.error, message: response.error.message.replace(model, "another-model") } }), + JSON.stringify([response]), "null", + ])("leaves unknown failures unchanged %#", (message) => { + const observed = observation(); + observed.observe({ ...event, payload: { status: "failed", error: { message } } }); + expect(observed.forTerminal("turn", terminal)).toBeNull(); + }); + + it.each([{ kind: "claude", model }, { kind: "codex" }, { kind: "codex", model: "other" }])( + "does not apply a rejection to another selected provider/model (%j)", (provider) => { + const observed = createNativeProviderFailureObservation(provider); + observed.observe(event); + expect(observed.forTerminal("turn", terminal)).toBeNull(); + }, + ); +}); diff --git a/server/src/services/native-runtime/native-provider-failure.ts b/server/src/services/native-runtime/native-provider-failure.ts new file mode 100644 index 0000000000..ec7380ffc8 --- /dev/null +++ b/server/src/services/native-runtime/native-provider-failure.ts @@ -0,0 +1,60 @@ +import type { PrpEvent, PrpTerminalState } from "../../vendor/paperclip-runner/index.js"; + +export const NATIVE_MODEL_REJECTION_MESSAGE = + "The selected model is not supported by the current ChatGPT connection. Choose a supported model or a compatible AI connection."; + +export const NATIVE_MODEL_REJECTION_DIAGNOSTIC = { + provider: "codex", + category: "model_auth_incompatible", + status: 400, + authMode: "chatgpt", +} as const; + +function record(value: unknown): Record { + return value !== null && typeof value === "object" && !Array.isArray(value) + ? value as Record + : {}; +} + +/** Reconstruct only the closed diagnostic vocabulary, never provider text. */ +export function readNativeModelRejectionDiagnostic(value: unknown) { + try { + const entry = record(value); + return Object.entries(NATIVE_MODEL_REJECTION_DIAGNOSTIC).every(([key, expected]) => entry[key] === expected) + ? { ...NATIVE_MODEL_REJECTION_DIAGNOSTIC } + : null; + } catch { return null; } +} + +/** Observe only events accepted by the bound control-plane persistence port. */ +export function createNativeProviderFailureObservation(provider: { kind: string; model?: string | null }) { + let rejectedTurnId: string | null = null; + return { + observe(event: PrpEvent) { + if (provider.kind !== "codex" || !provider.model || + event.sourceKind !== "runner" || event.eventType !== "turn.failed" || !event.turnId) return; + const payload = record(event.payload); + const failure = record(payload.error); + if (payload.status !== "failed" || payload.recoverable === true || failure.recoverable === true || + typeof failure.message !== "string" || failure.message.length > 4_096) return; + let response: Record; + try { response = record(JSON.parse(failure.message)); } catch { return; } + const error = record(response.error); + // This is a provider terminal envelope, not text emitted by a tool or an + // agent. Bind the exact rejection to the configured model without storing + // its name or the response. Unknown provider errors retain their behavior. + if (response.type !== "error" || response.status !== 400 || error.type !== "invalid_request_error" || + error.message !== `The '${provider.model}' model is not supported when using Codex with a ChatGPT account.`) return; + rejectedTurnId = event.turnId; + }, + forTerminal(turnId: string | null | undefined, terminal: PrpTerminalState) { + if (!rejectedTurnId || turnId !== rejectedTurnId || + terminal.turnTerminalState !== "failed" || terminal.runTerminalState !== "failed") return null; + return { + errorCode: "native_provider_model_rejected" as const, + errorMessage: NATIVE_MODEL_REJECTION_MESSAGE, + diagnostic: { ...NATIVE_MODEL_REJECTION_DIAGNOSTIC }, + }; + }, + }; +} diff --git a/server/src/services/native-runtime/native-restart-recovery.ts b/server/src/services/native-runtime/native-restart-recovery.ts index 2fc54b97a2..96378b4bc2 100644 --- a/server/src/services/native-runtime/native-restart-recovery.ts +++ b/server/src/services/native-runtime/native-restart-recovery.ts @@ -16,6 +16,7 @@ import { redactSensitiveText } from "../../redaction.js"; import { issueRecoveryActionService } from "../issue-recovery-actions.js"; import { reportRunFailure } from "../run-failure-report.js"; import { isNativeRunnerOwnershipHeld } from "./native-runner-ownership.js"; +import { nativeRestartInterruptedTurnId, type PersistedNativeSession } from "../../vendor/paperclip-runner/index.js"; export type NativeControllerIdentity = { bootId: string; @@ -764,7 +765,8 @@ export async function claimNativeRestartRecoveries(input: { hasProviderEvidence, checkpointFailed: (checkpointRecord.terminal as Record | undefined) - ?.runTerminalState === "failed", + ?.runTerminalState === "failed" && + nativeRestartInterruptedTurnId(checkpointRecord as unknown as PersistedNativeSession) === null, providerAttempt: row.coordinator.attempt, }); const ownershipChanged = diff --git a/server/src/services/native-runtime/native-run-finalizer.ts b/server/src/services/native-runtime/native-run-finalizer.ts index b6aea881b4..fb248551b9 100644 --- a/server/src/services/native-runtime/native-run-finalizer.ts +++ b/server/src/services/native-runtime/native-run-finalizer.ts @@ -1,4 +1,5 @@ import { settleSlackConversation } from "../slack-conversation-lifecycle.js"; +import { readPersistedNativeModelRejection } from "./native-provider-failure-evidence.js"; import { dismissAutomaticCompletionReviews } from "./automatic-completion-reviews.js"; import { getNativeReviewAssignment, readNativeReviewAssignmentContext } from "./native-review-participant.js"; import { conversationNativeDecision, isConversation } from "../agent-conversations.js"; @@ -24,7 +25,7 @@ import { workspaceOperations, } from "@paperclipai/db"; import { classifyNativeEvidence } from "./evidence-classifier.js"; -import type { PrpIgnoredAttentionRequest } from "@paperclipai/paperclip-runner"; +import type { PrpIgnoredAttentionRequest, PrpTerminalState } from "@paperclipai/paperclip-runner"; import { arbitrateNativeStatus, NATIVE_STATUS_ARBITER_POLICY_VERSION, @@ -1275,7 +1276,15 @@ export async function finalizeNativeRun(input: { companyId: run.companyId, issueId: authoritativeIssue.id, agentId: run.agentId, contextSnapshot: reviewContext, allowResolvedByRunId: run.id, }) : null; + const providerModelRejection = await readPersistedNativeModelRejection( + input.db, run, resultRow.turnId, envelope.terminal as PrpTerminalState, + ); + const ownsModelRejectionDecision = + (!authoritativeIssue.executionRunId || authoritativeIssue.executionRunId === run.id) && + (reviewContext ? nativeReview?.interaction.status === "pending" + : authoritativeIssue.assigneeAgentId === run.agentId && !authoritativeIssue.assigneeUserId); const proposedDecision = resolveNativeFinalizerStatus({ + providerModelRejected: providerModelRejection !== null && ownsModelRejectionDecision, ...(reviewContext ? { nativeReviewOutcome: nativeReview ? nativeReview.interaction.status === "pending" ? "pending" as const : "resolved" as const : "stale" as const } : {}), @@ -1358,6 +1367,9 @@ export async function finalizeNativeRun(input: { priorStatusVersion: Number(authoritativeIssue.statusVersion), priorDecisionId: authoritativeIssue.lastStatusDecisionId, decision, + requireModelRejectionOwner: decision.reasonCode === "native_provider_model_rejected" + ? { agentId: run.agentId, reviewContext } + : undefined, requireBoardResponseWaitSource: decision.reasonCode === "board_response_waiting" || repairBoardResponseWait ? boardResponseWait?.source @@ -1400,6 +1412,7 @@ export async function finalizeNativeRun(input: { ); const clearExecutionFailure = input.projectRunStatus && finalizationPhase === "committed" && terminalState === "succeeded"; const finalizationMetadata = { + ...(providerModelRejection ? { nativeProviderFailure: providerModelRejection.diagnostic } : {}), finalizationPhase, ...(finalizationPhase === "committed" ? { failureCode: null, originalFailureCode: null, nextAttemptAt: null } : {}), assessmentId: assessmentRow.id, @@ -1450,6 +1463,12 @@ export async function finalizeNativeRun(input: { nativePhase: finalizationPhase, nativePhaseUpdatedAt: now, ...(clearExecutionFailure ? { error: null, errorCode: null } : {}), + ...(providerModelRejection ? { + // Recovery can finalize before heartbeat saves the adapter result. + // Retain any independently recorded execution/cleanup failure. + error: sql`coalesce(${heartbeatRuns.error}, ${providerModelRejection.errorMessage})`, + errorCode: sql`coalesce(${heartbeatRuns.errorCode}, ${providerModelRejection.errorCode})`, + } : {}), resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || ${JSON.stringify(finalizationMetadata)}::jsonb || ${clearExecutionFailure ? recoveredExecutionFailureMetadata() : sql`'{}'::jsonb`}`, diff --git a/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts b/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts index 820661f889..c73db0b848 100644 --- a/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts +++ b/server/src/services/native-runtime/native-runner-restart-recovery.integration.test.ts @@ -1018,6 +1018,30 @@ describeEmbeddedPostgres("native runner restart recovery with real processes", ( ]); }); + it.each([2, 3])("recovers a checkpointed restart interruption within provider budget %s", async (attempt) => { + const fixture = await seedRun(`INTERRUPTED-${attempt}`); + await fixture.db.update(heartbeatRuns).set({ runnerProfileJson: { sessionCheckpoint: { + driverKind: "codex_app_server", + sessionId: fixture.native.normalizedSessionId, + identity: { companyId, issueId: fixture.issueId, runId: fixture.runId, agentId, sessionId: fixture.native.normalizedSessionId }, + providerSessionId: "interrupted-provider", + activeTurnId: null, + semanticResult: null, + terminal: { runTerminalState: "failed", turnTerminalState: "failed" }, + terminalTurns: [{ turnId: "interrupted-turn", fingerprint: JSON.stringify({ + terminalState: "failed", result: null, + error: { code: "provider_turn_lost_on_restore", recoverable: true }, + }) }], + } } }).where(eq(heartbeatRuns.id, fixture.runId)); + await fixture.db.update(nativeRunFinalizations).set({ attempt }).where(eq(nativeRunFinalizations.runId, fixture.runId)); + const dispositions = await claimNativeRestartRecoveries({ + db: fixture.db, controller: successor, restartKind: "hot", runIds: [fixture.runId], + }); + expect(dispositions).toEqual([expect.objectContaining(attempt < 3 + ? { kind: "resume_dead_runner", runId: fixture.runId, providerAttempt: attempt } + : { kind: "blocked", runId: fixture.runId, reason: "execution_recovery_budget_exhausted" })]); + }); + it("terminalizes the recorded failed-checkpoint incident atomically instead of resuming it on upgrade", async () => { const fixture = await seedRun("FAILED-CHECKPOINT"); await fixture.db.update(heartbeatRuns).set({ runnerProfileJson: { sessionCheckpoint: { diff --git a/server/src/services/native-runtime/native-session-executor.ts b/server/src/services/native-runtime/native-session-executor.ts index cc217b9452..981169652e 100644 --- a/server/src/services/native-runtime/native-session-executor.ts +++ b/server/src/services/native-runtime/native-session-executor.ts @@ -9,6 +9,7 @@ import { REMOTE_CODEX_SUPPORTED_RANGE, } from "./codex-runtime-compatibility.js"; import { createNativeToolTrace, type NativeToolTrace } from "./native-tool-trace.js"; +import { createNativeProviderFailureObservation } from "./native-provider-failure.js"; import { createNativeGitHubAccess, type NativeGitHubAccess } from "./native-github-access.js"; import { resolveGitHubOperationCredentials } from "../github-operation-credentials.js"; import { bindManagedNativeCredentialTurn, completeManagedNativeCredentialTurn } from "./managed-native-credentials.js"; @@ -7805,6 +7806,7 @@ async function executePaperclipNativeSessionWithinScope( let turnStartedAtMs: number | null = null; let firstAgentEventRecorded = false; let providerUsageLimitObserved = false; + const providerFailureObservation = createNativeProviderFailureObservation(input.execution.provider); let turnCompletedAtMs: number | null = null; let runnerSessionStartupScope: NativeRunSpanScope | null = null; let agentTurnScope: NativeRunSpanScope | null = null; @@ -7861,6 +7863,7 @@ async function executePaperclipNativeSessionWithinScope( await liveQuestions.observe(event); await projectSessionGoalEvent(event); providerUsageLimitObserved ||= nativeProviderUsageLimitFromEvent(event); + providerFailureObservation.observe(event); const eventAtMs = Date.parse(event.emittedAt); const milestoneAtMs = Number.isFinite(eventAtMs) ? eventAtMs @@ -8064,6 +8067,7 @@ async function executePaperclipNativeSessionWithinScope( await liveQuestions.observe(event); await projectSessionGoalEvent(event); providerUsageLimitObserved ||= nativeProviderUsageLimitFromEvent(event); + providerFailureObservation.observe(event); const questionFallback = await materializeRuntimeQuestionFallback({ db: input.db, binding: input.execution.binding, @@ -8334,6 +8338,7 @@ async function executePaperclipNativeSessionWithinScope( trace.activate(runnerSessionStartupScope); const result = await trace.run(runnerSessionStartupScope, () => executeNativeSession({ + resumeInterruptedTurn: input.restartRecovery?.kind === "resume_dead_runner", getFreshSessionHandoff: input.getFreshSessionHandoff, onSessionAdmission: async () => { // Invalidate prior stop evidence before a backend can spawn. @@ -9203,6 +9208,7 @@ async function executePaperclipNativeSessionWithinScope( ); if (collectInstructions && !collectedByOwner) await instructionCopy!.collectStopped(); } + const providerFailure = providerFailureObservation.forTerminal(native.turnId, native.terminal); const adapterResult: AdapterExecutionResult = { exitCode: native.terminal.runTerminalState === "succeeded" ? 0 : 1, signal: null, @@ -9210,10 +9216,12 @@ async function executePaperclipNativeSessionWithinScope( errorMessage: native.terminal.runTerminalState === "succeeded" ? null - : `Native session ${native.terminal.runTerminalState}`, + : providerFailure?.errorMessage ?? `Native session ${native.terminal.runTerminalState}`, + ...(providerFailure ? { errorCode: providerFailure.errorCode } : {}), resultJson: { nativeResult: native.result as unknown as Record, nativeTerminal: native.terminal as unknown as Record, + ...(providerFailure ? { nativeProviderFailure: providerFailure.diagnostic } : {}), ...(native.goalRolloverRequired ? { goalRolloverRequired: true } : {}), planSynchronizations, }, diff --git a/server/src/services/native-runtime/status-arbiter.test.ts b/server/src/services/native-runtime/status-arbiter.test.ts index 0f3b1abad9..4162548217 100644 --- a/server/src/services/native-runtime/status-arbiter.test.ts +++ b/server/src/services/native-runtime/status-arbiter.test.ts @@ -44,6 +44,23 @@ function arbitrate( } describe("native status authority", () => { + it.each(["in_progress", "in_review"] as const)("blocks a current worker's proven model rejection without a retry (%s)", (priorIssueStatus) => { + const decision = arbitrate({ priorIssueStatus, terminalState: "failed", providerModelRejected: true }); + expect(decision).toMatchObject({ statusAction: "blocked", toStatus: "blocked", reasonCode: "native_provider_model_rejected", unblockDescriptor: { owner: "board" } }); + expect(decision.effects).toEqual([{ kind: "bind_blocker", owner: "board", action: expect.any(String) }]); + expect(arbitrate({ terminalState: "failed", providerModelRejected: false }).effects).toContainEqual(expect.objectContaining({ kind: "schedule_retry" })); + expect(arbitrate({ terminalState: "succeeded", providerModelRejected: true }).reasonCode).not.toBe("native_provider_model_rejected"); + expect(arbitrate({ terminalState: "failed", providerModelRejected: true, workspaceFinalizeStatus: "failed" }).reasonCode).toBe("finalization_failed_claim_preserved"); + expect(arbitrate({ terminalState: "failed", providerModelRejected: true, priorIssueStatus: "done" }).toStatus).toBe("done"); + for (const nativeReviewOutcome of ["stale", "resolved"] as const) { + expect(arbitrate({ terminalState: "failed", providerModelRejected: true, priorIssueStatus, nativeReviewOutcome })) + .toMatchObject({ statusAction: "preserve", toStatus: priorIssueStatus, reasonCode: "native_review_action_finished" }); + } + }); + it("preserves pending review authority without automatic recovery after model rejection", () => { + expect(arbitrate({ priorIssueStatus: "in_review", terminalState: "failed", providerModelRejected: true, nativeReviewOutcome: "pending" })) + .toMatchObject({ statusAction: "preserve", toStatus: "in_review", reasonCode: "native_provider_model_rejected", unblockDescriptor: null, effects: [{ kind: "release_checkout" }] }); + }); it("a reviewer finishes its decision without completing rejected or still-reviewed work", () => { for (const priorIssueStatus of ["in_progress", "in_review"] as const) { const decision = arbitrate({ priorIssueStatus, nativeReviewOutcome: "resolved" }); @@ -511,7 +528,7 @@ describe("native status authority", () => { expect.objectContaining({ statusAction: "blocked", toStatus: "blocked", - policyVersion: "phase6-v7", + policyVersion: "phase6-v8", reasonCode: "current_track_blocker_waiting", unblockDescriptor: { owner: "board", diff --git a/server/src/services/native-runtime/status-arbiter.ts b/server/src/services/native-runtime/status-arbiter.ts index 55e298bf5a..c4235536a6 100644 --- a/server/src/services/native-runtime/status-arbiter.ts +++ b/server/src/services/native-runtime/status-arbiter.ts @@ -1,6 +1,7 @@ import type { NativeEvidenceAssessment } from "./evidence-classifier.js"; +import { NATIVE_MODEL_REJECTION_MESSAGE } from "./native-provider-failure.js"; -export const NATIVE_STATUS_ARBITER_POLICY_VERSION = "phase6-v7"; +export const NATIVE_STATUS_ARBITER_POLICY_VERSION = "phase6-v8"; export type NativeAuthoritativeIssueStatus = | "backlog" @@ -115,6 +116,8 @@ export function arbitrateNativeStatus(input: { reviewOwnerUserId?: string | null; /** Review decisions own task state; a reviewer's finish report cannot override them. */ nativeReviewOutcome?: "resolved" | "pending" | "stale"; + /** Re-derived from the committed runner terminal and pinned execution identity. */ + providerModelRejected?: boolean; agentId: string; priorIssueStatus: NativeAuthoritativeIssueStatus; }): NativeStatusDecision { @@ -146,6 +149,31 @@ export function arbitrateNativeStatus(input: { ], }; } + if (input.providerModelRejected && input.terminalState === "failed" && + input.nativeReviewOutcome !== "stale" && input.nativeReviewOutcome !== "resolved") { + if (input.nativeReviewOutcome === "pending") { + // The pending review is bound to the worker's status decision and + // version. Preserve that authority so an explicit retry after a + // configuration repair can resolve the same review. In particular, + // do not create the normal unresolved-review recovery action. + return { + policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, + statusAction: "preserve", + toStatus: input.priorIssueStatus, + reasonCode: "native_provider_model_rejected", + unblockDescriptor: null, + effects: [{ kind: "release_checkout" }], + }; + } + return { + policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, + statusAction: "blocked", + toStatus: "blocked", + reasonCode: "native_provider_model_rejected", + unblockDescriptor: { owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }, + effects: [{ kind: "bind_blocker", owner: "board", action: NATIVE_MODEL_REJECTION_MESSAGE }], + }; + } if (input.nativeReviewOutcome) { return { policyVersion: NATIVE_STATUS_ARBITER_POLICY_VERSION, diff --git a/server/src/services/native-runtime/status-decision-committer.ts b/server/src/services/native-runtime/status-decision-committer.ts index 307efb532d..21c14f5e93 100644 --- a/server/src/services/native-runtime/status-decision-committer.ts +++ b/server/src/services/native-runtime/status-decision-committer.ts @@ -24,6 +24,7 @@ import { type NativeStatusEffect, } from "./status-arbiter.js"; import { nativeSha256 } from "./canonical.js"; +import { getNativeReviewAssignment, type NativeReviewAssignmentContext } from "./native-review-participant.js"; import { readNativeBoardResponseWaitSource, readNativeBoardResponseWaitOrigin, @@ -1535,6 +1536,7 @@ export async function commitNativeStatusDecision(input: { preMaterializedEffects?: NativeMaterializedStatusEffect[]; supersedesCommittedDecisionId?: string; requireExternalChatResponseWaitAuthorization?: { agentId: string }; + requireModelRejectionOwner?: { agentId: string; reviewContext: NativeReviewAssignmentContext | null }; requireBoardResponseWaitSource?: NativeBoardResponseWaitSource; requireBoardResponseWaitOrigin?: NativeBoardResponseWaitOrigin; reviewResponsePresentation?: { @@ -1547,6 +1549,9 @@ export async function commitNativeStatusDecision(input: { throw new Error("native_status_reason_code_required"); } const reasonCode = input.decision.reasonCode; + if (reasonCode === "native_provider_model_rejected" && !input.requireModelRejectionOwner) { + throw new Error("native_model_rejection_owner_required"); + } const publications: ActivityPublication[] = []; const terminalRunsToEmit: (typeof heartbeatRuns.$inferSelect)[] = []; const committed = await input.db.transaction(async (tx) => { @@ -1616,6 +1621,20 @@ export async function commitNativeStatusDecision(input: { ) { throw new NativeStatusRaceError(); } + if (input.requireModelRejectionOwner) { + const owner = input.requireModelRejectionOwner; + // A successor can claim execution without changing statusVersion. Keep + // this new blocking authority behind the same locked issue snapshot. + if (issue.executionRunId && issue.executionRunId !== input.runId) throw new NativeStatusRaceError(); + if (owner.reviewContext) { + const review = await getNativeReviewAssignment(tx as unknown as Db, { + companyId: input.companyId, issueId: input.issueId, agentId: owner.agentId, contextSnapshot: owner.reviewContext, + }); + if (review?.interaction.status !== "pending") throw new NativeStatusRaceError(); + } else if (issue.assigneeAgentId !== owner.agentId || issue.assigneeUserId) { + throw new NativeStatusRaceError(); + } + } if (input.requireExternalChatResponseWaitAuthorization) { let authorization; try { diff --git a/server/src/services/recovery/provider-failure-classification.test.ts b/server/src/services/recovery/provider-failure-classification.test.ts index d58bcf42a9..9343e87813 100644 --- a/server/src/services/recovery/provider-failure-classification.test.ts +++ b/server/src/services/recovery/provider-failure-classification.test.ts @@ -7,8 +7,20 @@ import { classifyContinuationFailure, } from "./service.js"; import { legacyExecutionNeedsReconciliation } from "../legacy-execution-recovery.js"; +import { isConfigurationIncompleteFailedRun } from "../../modules/wake-queue/domain/values.js"; describe("classifyAdapterFailureForRecovery", () => { + it("waits for model configuration changes instead of retrying a typed native model rejection", () => { + const run = { errorCode: "native_provider_model_rejected", error: "Fixed model configuration message", resultJson: null }; + expect(classifyAdapterFailureForRecovery(run)).toEqual({ kind: "configuration_incomplete" }); + expect(classifyContinuationFailure(run as never)).toMatchObject({ kind: "non_retryable", maxAttempts: 0 }); + expect(isConfigurationIncompleteFailedRun(run)).toBe(true); + expect(isAiAuthenticationBlocked(run as never)).toBe(false); + const unknown = { ...run, errorCode: "adapter_failed", error: "native_provider_model_rejected" }; + expect(classifyAdapterFailureForRecovery(unknown)).toBeNull(); + expect(classifyContinuationFailure(unknown as never).kind).toBe("transient_infra"); + expect(isConfigurationIncompleteFailedRun(unknown)).toBe(false); + }); it.each(["acpx_auth_required", "claude_auth_required", "codex_auth_required", "adapter_auth_missing", "refresh_token_reused", "authentication_required"])("does not automatically retry provider authentication failure %s", (errorCode) => { const classification = classifyRunLiveness({ runStatus: "failed", issue: null, errorCode, authenticationRepairRequested: true }); const run = { errorCode, ...classification }; diff --git a/server/src/services/recovery/service.ts b/server/src/services/recovery/service.ts index cfda20e5bc..2cb7ac951f 100644 --- a/server/src/services/recovery/service.ts +++ b/server/src/services/recovery/service.ts @@ -491,6 +491,7 @@ const TRANSIENT_INFRA_CONTINUATION_ERROR_CODES = new Set([ ]); const NON_RETRYABLE_CONTINUATION_ERROR_CODES = new Set([ + "native_provider_model_rejected", "provider_tool_definition_invalid", "adapter_engine_unavailable", "agent_not_invokable", @@ -626,7 +627,7 @@ export function classifyAdapterFailureForRecovery( ): AdapterFailureRecoveryClassification { // An engine prerequisite cannot be repaired by asking the same unavailable // engine to retry. Use the existing configuration-blocker path. - if (latestRun.errorCode === "adapter_engine_unavailable" || latestRun.errorCode === "provider_tool_definition_invalid") { + if (latestRun.errorCode === "adapter_engine_unavailable" || latestRun.errorCode === "provider_tool_definition_invalid" || latestRun.errorCode === "native_provider_model_rejected") { return { kind: "configuration_incomplete" }; } if ( diff --git a/server/src/services/routines.ts b/server/src/services/routines.ts index c681e73b0b..98bffd4f18 100644 --- a/server/src/services/routines.ts +++ b/server/src/services/routines.ts @@ -665,6 +665,14 @@ function mapRoutineDescriptionDocument(row: { }; } +// Match PostgreSQL UUID input, including compact/braced forms and any version. +// Do not trim or resolve prefixes: the exact input must be safe for a UUID query. +function isRoutineUuidInput(id: string): boolean { + if (/\s/.test(id)) return false; + const value = id.startsWith("{") && id.endsWith("}") ? id.slice(1, -1) : id; + return /^(?:[0-9a-f]{4}-?){7}[0-9a-f]{4}$/i.test(value); +} + export function routineService( db: Db, deps: { @@ -682,6 +690,7 @@ export function routineService( }); async function getRoutineById(id: string) { + if (!isRoutineUuidInput(id)) return null; return db .select() .from(routines) @@ -766,6 +775,7 @@ export function routineService( } async function getTriggerById(id: string) { + if (!isRoutineUuidInput(id)) return null; return db .select() .from(routineTriggers) diff --git a/server/src/services/run-failure-diagnostics.ts b/server/src/services/run-failure-diagnostics.ts index 5a83450794..c5c9d7e820 100644 --- a/server/src/services/run-failure-diagnostics.ts +++ b/server/src/services/run-failure-diagnostics.ts @@ -5,6 +5,7 @@ import { redactDiagnosticText } from "@paperclipai/adapter-utils/command-redacti import { sanitizeWorkspaceRestoreDiagnostic } from "@paperclipai/adapter-utils/workspace-restore-diagnostics"; import { redactCurrentUserText } from "../log-redaction.js"; import { redactSensitiveText, REDACTED_EVENT_VALUE } from "../redaction.js"; +import { readNativeModelRejectionDiagnostic } from "./native-runtime/native-provider-failure.js"; type Run = typeof heartbeatRuns.$inferSelect; type Context = Record; @@ -184,6 +185,8 @@ export function collectRunFailureDiagnostics(run: Run, options: RunFailureReport "stackPreview", "status", "statusCode", "requestId", ]); const provider = scalars(read(result, "terminalSessionFailure"), ["category", "title", "details"]); + const nativeProviderFailure = readNativeModelRejectionDiagnostic(read(result, "nativeProviderFailure")); + if (nativeProviderFailure) Object.assign(provider, nativeProviderFailure); const truncatedFields: string[] = []; const providerTruncation = read(read(result, "terminalSessionFailure"), "truncatedFields"); if (Array.isArray(providerTruncation)) { diff --git a/server/src/vendor/paperclip-runner/index.ts b/server/src/vendor/paperclip-runner/index.ts index 80f98ee6f0..e113899dfa 100644 --- a/server/src/vendor/paperclip-runner/index.ts +++ b/server/src/vendor/paperclip-runner/index.ts @@ -128,6 +128,7 @@ export const validatePrpStructuredRunResult = runner.validatePrpStructuredRunResult; export const NativeProviderTerminalFailure = runner.NativeProviderTerminalFailure; +export const nativeRestartInterruptedTurnId = runner.nativeRestartInterruptedTurnId; export const completeTerminatedRemoteNativeSessionCleanup = runner.completeTerminatedRemoteNativeSessionCleanup; export const completeTerminatedLocalNativeSessionCleanup = runner.completeTerminatedLocalNativeSessionCleanup; diff --git a/ui/src/api/activity.ts b/ui/src/api/activity.ts index 50ecab9a5a..5daa3701cc 100644 --- a/ui/src/api/activity.ts +++ b/ui/src/api/activity.ts @@ -16,6 +16,8 @@ export interface RunForIssue { invocationSource: string; responsibleUserId?: string | null; errorCode?: string | null; + /** Bounded provider message for a failed model rejection. */ + error?: string | null; usageJson: Record | null; resultJson: Record | null; logBytes?: number | null; diff --git a/ui/src/components/ExecutionBlockerNotice.test.tsx b/ui/src/components/ExecutionBlockerNotice.test.tsx index 3683c76883..d5d2b0a95d 100644 --- a/ui/src/components/ExecutionBlockerNotice.test.tsx +++ b/ui/src/components/ExecutionBlockerNotice.test.tsx @@ -107,6 +107,28 @@ describe("stopped task recovery notice", () => { expect(onRetried).toHaveBeenCalledOnce(); }); + it.each([ + "The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.", + "The selected model is not supported by the current ChatGPT connection. Choose a supported model or a compatible AI connection.", + ])("shows the model rejection and repair action without expanding run logs: %s", async (runError) => { + vi.mocked(activityApi.runsForIssue).mockResolvedValue([{ runId: "model-run", agentId: "agent", + status: "failed", errorCode: "native_provider_model_rejected" }] as never); + await act(async () => root.render( + + )); + await act(async () => { await new Promise(resolve => setTimeout(resolve, 10)); }); + expect(container.textContent).toContain("Model unavailable."); + expect(container.textContent).toContain(runError); + expect(container.textContent).toContain("clear the task's model override, then retry"); + expect(container.textContent).toContain("Inspect the original failure before continuing."); + expect(container.querySelector("button")?.textContent).toBe("Retry"); + }); + it.each(["native_continuation_requires_reconciliation", "uncertain_external_action"])("offers Retry for a server-admitted native failure: %s", async cause => { await act(async () => root.render( run.runId === blocker.runId && ["failed", "timed_out"].includes(run.status)); + const modelRejected = failedRun?.errorCode === "native_provider_model_rejected"; const requiresInspection = blocker.cause === "native_continuation_requires_reconciliation" || blocker.cause === "native_session_cleanup_quarantined"; const retry = useMutation({ @@ -34,7 +35,8 @@ export function ExecutionBlockerNotice({ companyId, issueId, blocker, onRetried return (
-

Recovery needed.{blocker.runError ? ` ${blocker.runError}` : ""}

+

{modelRejected ? "Model unavailable." : "Recovery needed."}{blocker.runError ? ` ${blocker.runError}` : ""}

+ {modelRejected &&

Choose a supported model or clear the task's model override, then retry.

}

{blocker.nextAction}

{Boolean(blocker.savedMessageCount) && (

{blocker.savedMessageCount} saved {blocker.savedMessageCount === 1 ? "message is" : "messages are"} waiting for recovery.

diff --git a/ui/src/components/TaskChatThread.test.tsx b/ui/src/components/TaskChatThread.test.tsx index 11ca30d76a..1a111c1167 100644 --- a/ui/src/components/TaskChatThread.test.tsx +++ b/ui/src/components/TaskChatThread.test.tsx @@ -437,7 +437,15 @@ describe.each(["legacy", "native"] as const)("%s task history readiness", (runti async (status) => { const props = { issueId: "issue-1", - comments: createLongThreadComments(), + comments: [...createLongThreadComments(), { + ...createLongThreadComments()[3], + id: "agent-answer", + authorType: "agent" as const, + authorAgentId: "agent-1", + body: "Final saved reply", + runId: "started-run", + createdAt: new Date("2026-08-25T18:00:02.000Z"), + }], onAdd: async () => {}, linkedRuns: [ retryRun, @@ -455,9 +463,17 @@ describe.each(["legacy", "native"] as const)("%s task history readiness", (runti container.querySelector('[data-testid="task-chat-history-loading"]'), ).not.toBeNull(); expect(container.textContent).toContain("Thread message 1"); - + expect(container.querySelector('[data-thread-anchor="comment-1"]')?.closest('[inert]')).not.toBeNull(); transcriptState.hydratedRunIds = new Set(["started-run"]); nativeTranscriptState.hydratedRunIds = new Set(["started-run"]); + transcriptState.transcriptByRun.set("started-run", [{ + kind: "thinking", text: "Reasoning before the reply", + ts: "2026-08-25T18:00:01.000Z", + }]); + nativeTranscriptState.transcriptByRun.set("started-run", [{ + kind: "assistant", text: "Reasoning before the reply", channel: "progress", + ts: "2026-08-25T18:00:01.000Z", + }]); render(); await act(async () => { await new Promise((resolve) => requestAnimationFrame(resolve)); @@ -467,6 +483,15 @@ describe.each(["legacy", "native"] as const)("%s task history readiness", (runti expect( container.querySelector('[data-testid="task-chat-history-loading"]'), ).toBeNull(); + expect(container.querySelector('[data-thread-anchor="comment-1"]')?.closest('[inert]')).toBeNull(); + expect(container.textContent).toContain("Thread message 1"); + if (status === "succeeded" && runtimeMode === "native") { + expect(container.textContent).toContain("Reasoning before the reply"); + expect(container.textContent).toContain("Final saved reply"); + expect(container.textContent!.indexOf("Reasoning before the reply")).toBeLessThan( + container.textContent!.indexOf("Final saved reply"), + ); + } }, ); }); @@ -1778,6 +1803,32 @@ describe("TaskChatThread runtime transcript selection", () => { expect(container.textContent).not.toContain("The runner stopped"); }); + it.each([ + "The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.", + "The selected model is not supported by the current ChatGPT connection. Choose a supported model or a compatible AI connection.", + ])("promotes a rejected model above the runner's generated failure response: %s", (error) => { + nativeTranscriptState.transcriptByRun.set("model-rejected", [{ + kind: "run_result", ts: "2026-08-25T18:00:01.000Z", + summary: "The Codex run failed before it completed.", + disposition: "needs_review", objectiveSatisfied: false, verification: [], + remainingWork: [], blocker: null, artifacts: [], + }]); + render( {}} linkedRuns={[{ + runId: "model-rejected", runtimeMode: "native", status: "failed", + errorCode: "native_provider_model_rejected", + error, + agentId: "agent-1", agentName: "Runner", adapterType: "paperclip_runner", + createdAt: "2026-08-25T18:00:00.000Z", startedAt: "2026-08-25T18:00:00.000Z", + finishedAt: "2026-08-25T18:00:02.000Z", + }]} />); + const marker = container.querySelector('[data-testid="task-chat-collapsible-marker"]'); + expect(marker?.textContent).toContain("Model unavailable"); + flushSync(() => marker!.querySelector('button[aria-expanded="false"]')!.click()); + expect(container.textContent).toContain(error); + expect(container.textContent).toContain("clear the task's model override, then retry"); + expect(container.textContent).not.toContain("after returning a final response"); + }); + it("keeps workspace contention out of the conversation's cancellation markers", () => { render( {}} linkedRuns={[{ runId: "workspace-wait", runtimeMode: "native", status: "cancelled", errorCode: "workspace_busy", diff --git a/ui/src/components/TaskChatThread.tsx b/ui/src/components/TaskChatThread.tsx index 3c70f2aafc..45690233df 100644 --- a/ui/src/components/TaskChatThread.tsx +++ b/ui/src/components/TaskChatThread.tsx @@ -1637,7 +1637,9 @@ export function TaskChatThread(props: TaskChatThreadProps) { ? "native_runner_timed_out" : "native_runner_process_exited"); const label = - code === "native_provider_approval_required" && source.status === "failed" + code === "native_provider_model_rejected" && source.status === "failed" + ? "Model unavailable" + : code === "native_provider_approval_required" && source.status === "failed" ? "Approval required" : code === "native_provider_usage_limit" && source.status === "failed" ? "Usage limit reached" @@ -1659,7 +1661,7 @@ export function TaskChatThread(props: TaskChatThreadProps) { : code === "native_provider_approval_required" ? "This operation requires approval, but this runner has no interactive approval handler. Review the operation and update the agent's permission setting before retrying." : code === "native_provider_model_rejected" - ? "The provider rejected the selected model. Check the model ID and your account's access, save the agent configuration, then retry. View the run for the provider's full error." + ? `${meta?.error || "The provider rejected the selected model."} Choose a supported model or clear the task's model override, then retry.` : code === "native_provider_usage_limit" && source.status === "failed" ? "The model provider has reached its current usage limit. Try again after the limit resets." diff --git a/ui/src/hooks/useTaskBrowsers.test.ts b/ui/src/hooks/useTaskBrowsers.test.ts index 603c84db1d..23b8e0638d 100644 --- a/ui/src/hooks/useTaskBrowsers.test.ts +++ b/ui/src/hooks/useTaskBrowsers.test.ts @@ -3,18 +3,25 @@ import { act, createElement } from "react"; import { createRoot } from "react-dom/client"; import { afterEach, beforeEach, expect, it, vi } from "vitest"; import type { TaskBrowser } from "@paperclipai/shared"; -import { useBrowserArrivals } from "./useTaskBrowsers"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { browserUseApi } from "@/api/browser-use"; +import { useBrowserArrivals, useTaskBrowsers } from "./useTaskBrowsers"; (globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true; const cleanups = new Set<() => void>(); function cleanup() { for (const unmount of cleanups) unmount(); } -function renderHook(callback: (props: P) => R, options?: { initialProps: P }) { +function renderHook(callback: (props: P) => R, options?: { initialProps: P; client?: QueryClient }) { const host = document.createElement("div"); document.body.append(host); const root = createRoot(host); let props = options?.initialProps as P; const result = {} as { current: R }; function Harness() { result.current = callback(props); return null; } - function rerender(next = props) { props = next; act(() => root.render(createElement(Harness))); } + function rerender(next = props) { + props = next; + act(() => root.render(options?.client + ? createElement(QueryClientProvider, { client: options.client }, createElement(Harness)) + : createElement(Harness))); + } function unmount() { act(() => root.unmount()); host.remove(); cleanups.delete(unmount); } cleanups.add(unmount); rerender(); @@ -26,7 +33,32 @@ const browser = (id: string): TaskBrowser => ({ error: null, createdAt: "2026-09-29T10:00:00Z", }); beforeEach(() => { localStorage.clear(); sessionStorage.clear(); }); -afterEach(() => { cleanup(); vi.restoreAllMocks(); }); +afterEach(() => { cleanup(); vi.restoreAllMocks(); vi.useRealTimers(); }); +it("starts browser polling when a chat draft becomes a saved task and stops on the next draft", async () => { + vi.useFakeTimers(); + const list = vi.spyOn(browserUseApi, "list").mockResolvedValue([]); + const client = new QueryClient(); + const taskId = "12345678-1234-4234-8234-123456789abc"; + const { rerender, unmount } = renderHook(useTaskBrowsers, { + initialProps: undefined as string | undefined, + client, + }); + await act(async () => { await vi.advanceTimersByTimeAsync(6000); }); + expect(list).not.toHaveBeenCalled(); + rerender(`chat:${taskId}`); + await act(async () => { await vi.advanceTimersByTimeAsync(6000); }); + expect(list).not.toHaveBeenCalled(); + rerender(taskId); + await act(async () => { await vi.advanceTimersByTimeAsync(1); }); + expect(list).toHaveBeenCalledExactlyOnceWith(taskId); + await act(async () => { await vi.advanceTimersByTimeAsync(3000); }); + expect(list).toHaveBeenCalledTimes(2); + rerender(`chat:${taskId}`); + await act(async () => { await vi.advanceTimersByTimeAsync(6000); }); + expect(list).toHaveBeenCalledTimes(2); + unmount(); + client.clear(); +}); it("queues simultaneous arrivals and claims only after the panel acknowledges", () => { const browsers = [browser("first"), browser("next")]; const { result, rerender } = renderHook(() => useBrowserArrivals("user", "task", browsers)); diff --git a/ui/src/hooks/useTaskBrowsers.ts b/ui/src/hooks/useTaskBrowsers.ts index 0d59497756..be8a7d27ac 100644 --- a/ui/src/hooks/useTaskBrowsers.ts +++ b/ui/src/hooks/useTaskBrowsers.ts @@ -6,7 +6,7 @@ export function useTaskBrowsers(issueId?: string) { return useQuery({ queryKey: ["task-browsers", issueId], queryFn: () => browserUseApi.list(issueId!), - enabled: Boolean(issueId), + enabled: Boolean(issueId) && !issueId?.startsWith("chat:"), refetchInterval: 3000, retry: false, }); diff --git a/ui/src/lib/issue-chat-messages.ts b/ui/src/lib/issue-chat-messages.ts index bc6e7dfef1..cd184c0042 100644 --- a/ui/src/lib/issue-chat-messages.ts +++ b/ui/src/lib/issue-chat-messages.ts @@ -59,6 +59,7 @@ export interface IssueChatLinkedRun { hasStoredOutput?: boolean; logBytes?: number | null; errorCode?: string | null; + error?: string | null; scheduledRetryAt?: string | null; nextAction?: string | null; resultJson?: Record | null;