DottaandPaperclip
d3f2a60e79
## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests verify work through the browser and public APIs. > - The classic matrix already tests several common task stories. > - Its older stories depend on local fixture setup and workspace access. > - Deployment tests need the same story checks with a separate target adapter. > - This PR shares those checks and adds five explicit journey selections. > - A deployment pass still requires evidence from that deployment. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E journey reuse across managed instances and deployed Cloud instances. **Current behavior** The deployed-target catalog covers task creation, assignment, Ask, and fresh onboarding. Plan revision and Everyday stories run through the classic fixture driver. Several story steps assume a local workspace. **Proposed behavior** Add explicit Plan revision, artifact revision, protected-action approval and decline, Skill Studio, and Stop/redirect journeys. Share the main browser sequences and content checks with the classic driver. Let target adapters provide authentication, native agents, a reachable service fixture, and owned cleanup. **Reason and benefit** A local pass cannot verify the deployed application. Shared story code lets a deployed target exercise the same behavior without copying an entire test or starting a replacement server. **Breaking changes** No product API or default matrix selection changes. Passed deployment results must satisfy the declared story checks and work-run count. Unsupported target capabilities fail before a story runs. Related public work: #17262 and #18098. Upstream prerequisite: #18230. ## What Changed - Extract shared Plan, artifact download/oracle, protected service, Skill Studio, and Stop/redirect helpers. Keep classic callers on those helpers. - Add five explicit paid journey selections with owned-task observations and native company, agent, model, and environment checks. - Include classic prompts, delivery/decision code, and the decline grader in the source-definition hash. Keep classic transient-read retries and reject deployment binding failures immediately. Match cookie directory boundaries in sensitive fixture requests. - Require complete story checks and expected run receipts before retaining a passing result. Keep failed evidence and private downloads. - Add a secret-safe request method for deployment fixture credentials. Add the missing declaration for the existing manual evidence policy module so the E2E project can typecheck. - Document target responsibilities and promotion after a real deployment pass. This source catalog does not enroll candidates in a daily plan. ## Verification - E2E TypeScript check: passed, including the review follow-up. - Review follow-up: 30 tests passed across four files, covering transient Plan reads, immediate binding rejection, cookie-path scope, agent receipts, and artifact evidence. Stop now waits for a saved, still-active native turn before the browser click and retains its startup events. Counterexamples reject claimed-but-unstarted, already-completed, interrupted, and non-native work. Shared observation and API helpers also enter the definition fingerprint. Cancellation snapshots are captured before lease cleanup. A direct terminal-boundary assertion rejects native work after cancellation even if it is already in the first snapshot; five cleanup-window counterexamples are covered. The Stop-output regression first failed for both native `item.delta` and `item.completed`; the fixed check detects both while allowing accounting-only updates. A report-catalog test first failed because the sandbox denied the tsx child IPC pipe (`listen EPERM`); the unchanged test passed outside the sandbox. - Affected journey, catalog, API, service fixture, and delivery checks: 313 tests passed across 19 files. - Workspace build: passed after rebase. - Workspace typecheck: passed. The default repository run was attempted. Repeated embedded PostgreSQL startup failures prevented local database coverage; it was stopped without claiming a full pass. Linux CI must supply that coverage. - Broad E2E support run: 4,793 passed, 23 failed, 2 skipped. Failures include existing stale image/Pi assertions and process/startup timeouts under host load. The affected tests pass separately. This is not a claim that the broad suite is green. - Upstream CI prerequisites: this branch is rebased on the merged #18230 repair. The earlier conflicting Turbo flags no longer apply. The shell-hook fix is also supplied by that upstream change. Linux shell-hook investigation: 19 focused tests and adapter TypeScript passed. The exact staged-config sequence reproduced CI's 500 in a pinned Node 24 Linux container. Node's default stdin socket at shell level zero selects Bash remote-shell startup and reads `.bashrc` instead of `BASH_ENV`. Ignored stdin and a nested shell each produce 1000. The fixture now ignores stdin; the production hook and 1000 assertion are unchanged. See [Bash startup rules](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files). - GitHub CI on the earlier head passed most suites but failed two browser tests (a fixture cleanup lock timeout and a fixture run staying queued) plus the shell-hook assertion. The lock-timeout case received one retry, which was cancelled by a later source push and does not establish recovery. These failures remain part of the verification record. - Real Cloud qualification: the private target adapter is reviewed and merged. These candidates must pass there before daily-plan promotion. Current source review: Greptile 5/5 and Paperclip Review 5/5 on `a9df1d6e8d880710fce39b87e689e07e1c8e6edc`, with zero unresolved review threads. The full [GitHub CI run](https://github.com/paperclipai/paperclip/actions/runs/40974946081) passed on that exact head, including both required aggregate checks. ## Risks Shared helper changes affect classic callers. The independent ZIP oracle and visible delivery requirements remain in place. Stop requires native acknowledgement, terminal event evidence, resource cleanup, and no later original-turn activity. The managed local target still lacks paid-agent fixtures and reports these journeys as unsupported. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, terminal use, and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
16 KiB
navigation
| navigation | ||||||
|---|---|---|---|---|---|---|
|
Questions and approvals
People answer an agent's questions and decide requests without losing the task's history. A question, plan confirmation, governance approval, and app-tool review can look similar but have different authority and continuation rules. Each decision must stay attached to its original request and survive reload.
Feature ID: questions-and-approvals.
Agent access: companion.
Sub-features
questions:questions-and-approvals/questions— single-choice, multiple-choice, custom, and free-text answers; paged forms retain answers until submission and show validation failures.dismiss-and-reopen:questions-and-approvals/dismiss-and-reopen— questions and composer requests (including completion reviews) remain in the feed after dismissal or a newer message; reopening restores the form without inventing an answer.confirmations:questions-and-approvals/confirmations— accept, decline, or request changes; stale, expired, resolved, and withdrawn requests cannot be acted on again.plan-review:questions-and-approvals/plan-review— inspect the requested plan revision before accepting or asking for changes. An ordinary planning task can proceed to execution; Agent Chat hands work to linked tasks and remains a conversation.audience:questions-and-approvals/audience— the current actor, company, named audience, and review policy govern resolution. A visible request is not necessarily actionable by that viewer.governance:questions-and-approvals/governance— pending/history views, approve/reject, revision request, and resubmission for formal board approvals.tool-review:questions-and-approvals/tool-review— approve once, always allow the displayed scope, or decline; decision status and provider execution outcome remain distinct.continuation:questions-and-approvals/continuation— one recorded response leads to the eligible waiting run or continuation; repeated clicks, reloads, and competing tabs do not repeat work.
How to get to it (user POV)
task-thread
Open a task from Tasks, search, a project, or its direct /issues/:issueId link.
Answer the question near the composer, or reopen its compact unanswered entry
in the history. Open the plan preview before using the confirmation controls.
Exercise both available task presentations when a change affects shared cards.
agent-chat
With Agent Chat enabled, choose Chat and an agent (/chats/:agentRef). Answer,
dismiss, or reopen requests in this persistent conversation. A plan handoff here
creates linked work rather than turning the conversation into implementation.
attention
Open Decisions (/decisions, including a saved queue) or an Inbox attention row.
Expand an actionable interaction inline; follow its task link to inspect context.
With combined Inbox/Tasks enabled, open the corresponding Tasks view instead.
board-approval
Open Approvals → Pending or All (/approvals/pending, /approvals/all). Use the
card action or open /approvals/:approvalId for details, comments, revision
request, and resubmission. These are formal approvals, not ordinary questions.
app-review
In a task, open Review request for an Ask first app call. The same request
appears in Apps → Review (/apps/review) and the connection's Review tab.
Use Approve & run, Always allow, or Decline on the surface under test.
skill-test
Open Skill Studio, run a skill test that produces an interaction, and inspect its output card. Skill-test authority is narrower than an ordinary company run.
external-channel
In a configured chat channel, answer a published question or confirmation as a linked sender. Open the associated Paperclip task to inspect the saved outcome. Channel-specific forms and identity linking require separate provider evidence.
Driving it
The explicit plan-revise-accept and protected-actions Product journeys cover questions-and-approvals/plan-review and questions-and-approvals/tool-review. They share the classic browser actions and saved-state checks. Cloud qualification remains separate; source registration is not a passing result.
Preconditions: follow the baseline. Use separate disposable requests for accept, decline, and revision paths. Record the actor and request revision. Prepare a waiting question through a test agent; a component fixture alone cannot prove live response delivery.
Operation checkpoints
Use these checks alongside the entry-point recipes below and the operation records in the access companion. Record the actual actor, surface, saved result and denial separately. Source review and linked tests do not establish a live pass.
| Capability ID | Action and persisted result | Denial, race and retry checks |
|---|---|---|
questions-and-approvals/questions |
Create fixed-ID single/multiple/custom/text questions; move between pages, submit, and reload the saved answer and responder. | Reject missing/invalid answers without resolving. Competing answers settle once. Distinguish cancel, withdraw, historical Done-task answers, and local dismissal; historical answers must not restart work. |
questions-and-approvals/dismiss-and-reopen |
Dismiss questions, completion reviews, confirmations, choices, and item requests without deciding; reload and reopen from history. | Confirm the server request remains pending and a new ordinary comment does not answer it. Resolve in another tab and refresh the old form. Saved-request dismissals persist on the server and suppress repeated agent demands; drafts and raw runtime dismissals remain browser-local. Protected actions still require authorization. |
questions-and-approvals/confirmations |
Use separate accept, decline/change-request, checkbox subset, suggested-task selection, and partial item-verdict requests. Inspect the actual selected items and created task IDs. | Test invalid selections, child-without-parent selection, expiry, supersession, withdrawal, and repeated/conflicting decisions. Partial verdicts persist; skip must not start a continuation. |
questions-and-approvals/plan-review |
Open the exact requested plan revision; reject one request and accept another. Inspect saved revision, work mode, owner, and the resulting continuation. | Edit after requesting review and reject stale acceptance. Test finalization readiness and bound completion review separately. In Agent Chat, inspect the later project/task creation and its source-plan provenance; acceptance alone is not a completed handoff. |
questions-and-approvals/audience |
Repeat a request for a named person, named agent, human-only resolver, and independent reviewer. Inspect actionability and final attribution. | Test creator exclusion, wrong audience/company, private task, missing or cancelled run, and changed responsible-user access. A human comment used by an agent remains subject to that agent’s resolver authority. |
questions-and-approvals/governance |
Create and inspect a formal request, comment, request revision, resubmit, then approve or reject distinct requests. Reload history and linked task state. | Test board decision authority separately from agent request/comment/resubmit authority. A repeated decision cannot repeat activation. Current pending status, payload revision, and governed side effects must agree. |
questions-and-approvals/tool-review |
Invoke a harmless Ask first action, compare task and Apps review surfaces, then exercise Approve once, Always allow, and Decline separately. | No provider call before approval. Repeat decisions and compare invocation count. Change the tool definition, revoke access, or set Off before a decision. Record the approval decision and provider outcome independently. |
questions-and-approvals/continuation |
After answering, inspect the saved response, delivery receipt, eligible run, and useful result in the original task/thread. | Reload, race two tabs, and interrupt delivery after saving. Require one continuation without repeated external actions. Reassignment, Stop, pause, budget, and external-thread boundaries remain effective. |
task-thread
Automated: interaction cards, paged task cards, and task thread cover form, receipt, and dismissal behavior. Resolution routes and response delivery cover authority and durable continuation. Run a targeted set with:
pnpm exec vitest run ui/src/components/TaskChatThread.test.tsx server/src/__tests__/question-response-delivery.test.ts
Manual: answer a multi-page question, go back, edit, then submit. Reload and confirm the saved answers and one agent continuation. On a fresh request, dismiss, reload, reopen, and verify the draft. Send a new comment while a question remains unanswered; it must stay reopenable in history. Try a denied actor and a request already resolved in another tab; expect an explicit error or settled receipt, never a second decision. For plans, change the plan revision before trying the old approval, then review the current revision.
agent-chat
Automated: the task thread suite parameterizes unanswered-question history for conversation mode on and off. This proves component parity, not a model's plan handoff.
Manual: repeat dismissal, reload, reopening, and answer submission in Chat. Approve a plan and inspect the linked execution task and copied plan. The chat history remains and the conversation does not close as a completed task.
attention
Automated: attention resolver covers preparing-request refresh without accepting it. The resolution-route tests above cover the API, not every Decisions or Inbox navigation path.
Manual: open the same pending request in an attention row and task tab. Resolve from the row; both surfaces must settle after refresh. Check an empty queue, loading, permission denial, and a stale request. Repeat with combined Inbox/Tasks on when that surface is part of the change. Full navigation parity remains a manual coverage gap.
board-approval
Automated: approval service and idempotency routes cover persistence and duplicate decisions. They do not drive the approval pages.
Manual: create disposable formal approval requests; approve one from the list, reject another from details, and request revision on a third. Resubmit that third request and verify Pending/All and detail history after reload. Confirm the recorded decision and its governed effect, not just a success toast.
app-review
Automated: connection reviews drives task/queue synchronization, approve, decline, remembered permission, provider failure, and restart with a scripted process agent and local MCP provider:
pnpm exec playwright test -c tests/e2e/connection-reviews.config.ts
Manual: dismiss and reopen a review, decide it from each entry point using fresh requests, and inspect the continuation's useful result. Always allow must show its scope; a future action with changed arguments should obey the saved rule. Provider failure must remain an approved decision with a failed execution. Real provider and model checks follow Task reviews.
skill-test
Automated: read-only interaction contract checks that fetching interactions does not mutate them. Shared card suites cover rendering. Neither proves the complete Skill Studio host or its permission matrix.
Manual: run a skill test with a question, answer through its visible output, and inspect the test result. Attempt a governed action outside the test's scope; expect denial. The complete Skill Studio browser journey is not covered by the referenced unit tests.
external-channel
Automated: chat question forms, Discord forms, and interaction publications cover serialization and delivery logic with fixtures.
Free-text and multi-question cards show their title, prompts, help, and context. Slack modals retain instructions and editable starting text. Closed single-choice questions use buttons for up to five options and an inline searchable selector for six to 100. Multiple choices use checkboxes (up to ten) or a multiple selector, then explicit Submit. Custom choices combine direct buttons with an Other modal; pattern errors appear beside the native input. Ordinary confirmations use buttons; checklists preserve defaults and bounds; item verdicts use radio choices and reasons; suggested tasks use Create, selection, and Dismiss. Connection requests offer Review connection and native Not now, without granting access from Slack.
Governed approvals, unsupported mixed forms, attachments, and oversized requests retain the full Paperclip review path. Link clicks navigate without attempting an answer. Terminal edits preserve the submitted response in the same authorized conversation and remove stale controls. Inspect the saved interaction, exactly one continuation, and useful reply independently. See the continuation regressions and Slack card tests. Local real-Slack proof for eleven formats, including native validation and public link navigation, is recorded with PR #17255. It uses a deterministic protocol fixture and does not establish deployed staging or model reliability.
Manual: use a disposable provider conversation, answer as the linked sender, and check the task's persisted result and one continuation. Repeat from an unlinked or wrong sender and with an expired card. A browser answer does not qualify Slack, Discord, Telegram, email, or iMessage; record each as not run unless its actual provider journey was driven.
Gotchas
-
Canonical question forms show field validation only after Next/Submit (including the keyboard submit shortcut) is attempted. Invalid answers remain unsent and the form stays on the relevant question; correction clears the field warning. Pagination arrows still browse without validation, drafts remain preserved, and uploads/requests in flight still disable submission.
-
Dismissing a question does not answer it. Dismissing an app review neither grants permission nor resumes the agent. Ordinary comments cannot approve it.
-
A resolved request and a successful tool execution are separate facts.
-
Do not reuse an accepted request to test rejection; use another request.
-
Agent Chat and skill tests have distinct lifecycle/authority rules despite sharing UI. A watchdog cannot acquire board authority by resolving a card.
-
An answer to an old question can queue behind a successor run; it does not automatically steer that run. See steering.
Documentation scope: Formal governance approvals
Formal board decisions and signoff have recipes; every approval payload/type and actor/host error permutation is not enumerated.