6dda8cc2b7
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final head232d3c52d0, with no unresolved findings. - Both actual-model necessary-question development cells passed on the earlier recordedc4e409d819source. This is supporting evidence for that revision, not current qualification. - Both fresh Stop development cells at6036cee2f4stopped, remained idle, resumed once, reached Done, and settled original and successor accounting. Their original machine grades remain failed: the observer counted the generated file-preparation carrier as another final reply. The corrected observer independently correlates the applied preparation receipt with the actual attachment/comment binding. Retained-wire diagnostic replay finds one final delivery for each model without changing either grade. Those original failures remain retained: https://github.com/paperclipai/paperclip/actions/runs/39644220458. - Both fresh actual-model Stop/resume cells pass all five checks at232d3c52d0: https://github.com/paperclipai/paperclip/actions/runs/39647010372. Independent inspection confirms exactly one cancelled original and one committed successor on the same task, 5.091/5.513-second quiet intervals before the user resumes, retained workspace/instructions/tool identities, and one receipt-bound final response. Each visible JSON chip opens in a tab and its Download control yields verified bytes. The rendered reference and changed label independently reconstruct the registered file hash and the original context hash. Both cleanups pass. These remain local API-key/per-turn development results, not the required staging/Slack qualification. - Other retained Stop failures include missed active boundaries, the real accounting admission failure, and a preparation observer that required nativeIssueId before runtime selection. They remain recorded at https://github.com/paperclipai/paperclip/actions/runs/39636796645 and https://github.com/paperclipai/paperclip/actions/runs/39642060165. The preparation observer now validates all present task IDs and still requires selected native execution at the active checkpoint. - The remote finalization attempt failed during image export with no disk space before a provider started. Remote crash acceptance remains unverified. - Negative calibration rejects wrong or absent ownership/evidence, stale settings, unsafe files, repeated admissions, unacknowledged Stop, premature resumption, incomplete receipts, lost context, genuine duplicate final replies, and missing delivery. A prepared-looking body cannot suppress a comment; absent, rejected, foreign, or mismatched file receipts and attachment bindings also cannot suppress it. - Core staging/Slack drivers, 72 fresh candidate passes, deployed staging proof, and recovery of the original user-visible task remain incomplete. No measured reliability percentage or completed qualification is claimed. ## Risks These are explicit development cells. Their API-key and per-turn runs do not qualify deployed subscription authentication, warm reuse, or Slack. A fast provider can miss the Stop boundary; that attempt fails. The remote crash boundary depends on independently observed public state and an owned controller. The checkpoint is opt-in through a private manifest and cannot replace the missing live observers. The two production changes affect native Stop settlement and saved-input promotion. They use the existing transaction signals and admission path. A complete receipt can settle without another provider turn. Partial accounting and all ownership, cleanup, pause, and budget gates remain authoritative. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>
11 KiB
navigation
| navigation | ||||||
|---|---|---|---|---|---|---|
|
Steering and queued messages
People can send follow-ups while an agent works, manage saved messages that have not been consumed, and choose when to steer or interrupt. Stopping a run and pausing a task are separate controls. The UI must preserve the message, explain why it is waiting, and only claim delivery after an authoritative acknowledgement.
Feature ID: steering.
Agent access: companion.
Sub-features
queue:steering/queue— a busy task saves follow-ups and shows their order and wait reason.edit-order-discard:steering/edit-order-discard— edit full Markdown, reorder, and discard eligible saved messages; ownership, revision conflicts, and already-dispatched states matter.steer:steering/steer— explicit user action sends a queued message into a compatible active run; unavailable capabilities and provider rejection remain visible.interrupt:steering/interrupt— stop the current execution and carry accepted follow-ups into the eligible continuation without delivering them twice.stop-and-pause:steering/stop-and-pause— distinguish stop-in-flight, confirmed stop, task/ancestor pause, agent pause, and budget hold. Resume must respect the actual gate.drafts:steering/drafts— switching tasks, reloading, and uncertain submission retain the correct task's draft and attachment receipts.approval-queue:steering/approval-queue— answered questions and accepted approvals have durable delivery distinct from editable ordinary comments.
How to get to it (user POV)
task-composer
Open an assigned task (/issues/:issueId). Start work, then send follow-ups while
the agent runs. Use the queued-message controls to edit, reorder, discard,
steer, or interrupt when offered. Use the composer stop/pause control separately.
Both task presentations need attention when changing their shared behavior.
agent-chat
Enable Agent Chat and open an agent conversation. Send another message during
a reply and inspect the shared queue. /new starts fresh provider context in
the same conversation; it is an ordered boundary, not a normal model prompt.
request-response
Answer an earlier question or accept a confirmation while successor work is active on the task. Inspect the resulting queued response and its available steer/interrupt control instead of assuming acceptance interrupted execution.
external-channel
Send a follow-up from a linked external chat conversation while its task is running. Use that provider's supported stop/control affordance where available; inspect the resulting task and run in Paperclip.
Driving it
Preconditions: follow the baseline. Use an agent that stays busy long enough to inspect the queue. Record the adapter, native/legacy mode, and advertised steering capability. Test supported and unsupported steering without treating an absent capability as a product failure.
task-composer
Automated: queued-message UI covers order, failure restoration, unavailable steering, stale revisions, and discard acknowledgements. Queue routes cover durable ordering, authorization, steering acknowledgements, interrupt recovery, and concurrent dispatch. Composer covers drafts and uncertain saves.
Task-detail delivery races cover a consumed message whose send response and queue snapshot arrive late. A started run's input receipt removes that message from pending, while a separate unsent message stays queued. Admission to a run that has not started is not delivery. A later retry of the same message remains visible in its new authoritative queue. Receipts retain the primary wake and coalesced source queue IDs. They clear a late snapshot even when its target names the preceding run or is null.
pnpm exec vitest run ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx server/src/__tests__/issue-queued-comments-routes.test.ts
Automated stop coverage is separate: composer stop with its configuration and ACP stop/continuation. Inspect their fixture and opt-in prerequisites; skipped native cases are not native proof. The explicit follow-up/Stop/resume journey requires a later user message before continuation. The database Stop accounting regression checks complete and late receipts without another provider turn. These are separate scopes; database success does not establish deployed or Slack acceptance.
Manual: queue three distinguishable messages. Edit one, reorder them, reload, and confirm the saved order. Discard one and confirm it never reaches execution. After a queued message enters a started run, confirm it leaves pending even if the send response arrives later. Reload and confirm it appears once in history. Steer one while busy and verify the agent receives that exact message once. Repeat with a rejected steer and a stale second-tab edit; the UI must preserve retryable content and explain the conflict. On a fresh run, interrupt with a queued follow-up and verify confirmed stop followed by one continuation. Pause the task (and separately its ancestor), preserve a draft, then resume the actual hold; no hidden send should occur while paused. Check both task presentations when available; the named component tests do not prove every host integration.
Fixture-backed recovery check: start with an agent in its completed paused
lifecycle state, then insert a new accepted deferred_issue_execution request
through the PostgreSQL recovery fixture.
Normal wake admission
rejects paused agents, so this pending state cannot be created by sending a new
message through the ordinary UI or API. Let release and periodic recovery run.
Confirm that the saved request remains pending. Resume the agent, wait for its
lifecycle to become ready, and verify one delivery. An explicit Pause cancels
older queued work; those cancelled receipts must not be replayed. A paused
request must not prevent another eligible recipient's queued work from proceeding.
agent-chat
Automated: agent chat sessions covers persistent conversation/session behavior with deterministic harness fixtures; shared composer tests cover the controls. These do not qualify live steering for every harness.
Manual: queue two follow-ups while the conversation replies. Reload, edit or
discard one, then let the other run. Verify one reply and unchanged conversation
identity. Send /new, then a new prompt; old history remains visible, and the
new turn uses fresh provider context. Verify a useful answer rather than only
the session-divider UI. Repeat the queue operations with a live capable harness
when the change touches delivery.
request-response
Automated: response delivery proves queueing/coalescing and retry-safe receipts. The queue-route suite tests separate comment and approval queues, explicit steering, and fresh-session restrictions on plan approvals.
Manual: answer an old question during a new run, confirm that it is saved and queued, and let the active run end. Verify exactly one continuation with the answer. Repeat an eligible approval with explicit steer; another approval that requires a fresh session must not offer same-turn delivery. Do not edit an approval outcome through an ordinary queued-message editor.
external-channel
Automated: chat interaction arbitration covers backend arbitration with simulated requests. It is not a provider UI test.
Manual: send two distinguishable messages through one disposable linked conversation while busy. Inspect their saved task order, run delivery, and external reply. Check a duplicate provider event and unauthorized sender. Provider-specific control semantics and real transport timing remain a manual gap; do not assume browser queue controls are available in every channel.
Gotchas
- Paperclip's queue is server-owned once saved; it is not Omnigent's unsent browser draft buffer. A stale edit must not overwrite a consumed message.
- Steer availability depends on the active runtime's capability, not its brand name. A UI row disappearing optimistically does not prove provider receipt.
- Interrupt, stop, task pause, agent pause, and budget pause have different effects. A running indicator disappearing does not prove remote work stopped.
- An interrupted run may need verified cleanup before a saved follow-up can start. Preserve and inspect the wait reason; see recovery.
- A healthy idle Agent Chat is waiting for input, not stranded unfinished work.
Storybook verification
drafts
Capability ID: steering/drafts.
Scope: ui Storybook: Task-keyed restored draft and attachment receipts; task switch/reload, failed send and uncertain-save recovery. Production: Retain correct task draft across navigation and reload; inspect authoritative saved receipt before claiming delivery; do not clear storage on every harness mount. Sources: Source: TaskChatComposer.tsx, composer-draft.ts. Tests: TaskChatComposer.test.tsx, issue-queued-comments-routes.test.ts
approval-queue
Capability ID: steering/approval-queue.
Scope: ui Storybook: Answered question/accepted approval delivery waiting, eligible steer/interrupt, failed delivery and fresh-session-only approval. Production: Keep response delivery durable and distinct from editable comments; coalesce once, enforce fresh-session restrictions and deduplicate concurrent/retried continuation. Sources: Source: TaskChatQueuedMessages.tsx, question-response-delivery.ts. Tests: TaskChatQueuedMessages.test.tsx, question-response-delivery.test.ts, issue-queued-comments-routes.test.ts