mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 12:59:13 +01:00
DottaandPaperclip
7ae81c9fa7
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Questions and review decisions must work in the conversation where a task started. > - Slack omitted question context and did not publish several review kinds. > - Some answers saved in Paperclip also lost their authorized continuation or their final Slack reply. > - A settled card and a successful run did not prove that the user received the result. > - This pull request preserves complete questions and follows the existing answer and delivery receipts through continuation. > - Users can read the request, submit an authorized decision, and receive the resumed task's result in the same conversation. ## Linked Issues or Issue Description Refs #17045 and #17218. **What happened?** Slack question cards could omit instructions and editable starting text. Checkbox reviews, item verdicts, task suggestions, and connection requests had no publication path. Complex question answers submitted in Paperclip could fail continuation authorization. Several review continuations completed without delivering their result to Slack. **Expected behavior** Show the actual question and context. Use native Slack controls when they preserve the form contract. Otherwise provide an explicit Paperclip review link. Persist one authorized response, resume through the existing delivery path, settle the card, and deliver the useful result to its originating conversation. **Steps to reproduce** 1. Start a task in Slack and ask a text or choice question with instructions and help text. 2. Submit native text and choice responses, then submit a complex form through Paperclip. 3. Accept a checklist, submit item decisions, select suggested tasks, or decline a connection request. 4. Inspect the saved answer, continuation run, original Slack card, and final Slack reply independently. ## What Changed - Preserve full prompts, context, descriptions, recommendations, help, and editable starting text. Use direct buttons for short choices, inline searchable selects for longer lists, and native checkboxes with explicit submission for multiple choices. - Add native custom-answer and validated-text modals, ordinary confirmation buttons, checklist decisions, per-item verdicts with reasons, suggested-task creation/dismissal, and connection decline. Keep governed approvals, permission grants, and forms that cannot be represented faithfully in the full Paperclip review surface. - Keep Slack-specific canonical-validation feedback in its owning platform module. Shared callbacks delegate without interpreting provider presentation. - Reuse the existing canonical interaction transactions, callback receipts, publication queue, and continuation delivery. Share ordinary confirmation handling across Telegram and Slack. Recheck current authorization and commit response receipts atomically. - Settle cards with the saved answer and remove stale controls. Ignore passive Slack link callbacks so opening a task does not display a false answer error. - Validate exact multi-select receipt mappings during native continuation. Recover automation-origin connection replies from their published prompt after validating the durable connection receipt. - Keep oversized multi-select handoffs free of unused native tokens, and retain the complete Paperclip form for 101–128 choices plus Other. Make the original-conversation audience explicit before submitting a Slack modal. - Add rollback, provider serialization, callback, continuation, and negative authorization regressions. Update the connector playbook and feature/access maps. No React or application UI changes are included. - Add a durable Q01–Q11 qualification index and provider coverage matrix. Require distinct functional, good-experience, and best-available-control assessments; real unanswered/answering/settled evidence; saved results, continuation and navigation proof; and retained failures. Mark other providers unaudited rather than assuming Slack parity. ## Verification Candidate: `e0017d446b1016a38d65382366cd1408cce33039`. The eleven-core-format live Slack gallery is pinned to `c3d458b2a06b660448940fd8cf33c5906f0058d5`; subsequent boundary fixes and the sharing notice have automated verification, not new live screenshots. - Real Slack UI testing covered all 11 formats: text, closed choice, long single select, multiple choices, choices with a custom answer, pattern-validated text, confirmation, checklist, item verdicts, suggested tasks, and connection requests. Each has unanswered/answering/answered screenshots as applicable, an inspected saved result, one continuation, and an individual experience review in the local HTML gallery. - The dedicated private QA app/channel used signed callbacks to the local candidate. A separately approved password-protected, read-only preview proved connection-review, suggested-task review, and created-task navigation. Anonymous access, writes, and credential routes were denied. No OAuth grant was attempted. - Real testing exposed and repaired a multi-select continuation failure and a misleading error on link-button clicks. Failed attempts remain in the evidence. The final navigation replay opened the exact pending connection request and declined it in Slack without granting access. Suggested-task creation produced the selected child with the correct parent and description. - Final review-fix verification: **32 form tests**, **28 provider/form-rendering tests**, **30 publication tests**, and **171 native continuation/authorization tests** passed. Earlier **3 atomic-resolution rollback**, **9 targeted callback/publication/Telegram**, and **28 review-continuation** cases passed. The final Slack/Teams modal subset also passed. The automation-origin connection cases, 101/128 custom-choice cases, and oversized multi-select continuation reproduced before their fixes and passed afterward; failing logs remain retained. These counts describe relevant runs, not a reliability percentage. - CI on `b5885b85fd` exposed a test fixture that assumed every Slack modal block was an input; the new sharing notice correctly renders a non-input block. The fixture now selects input blocks safely. The exact signed-modal correction/one-continuation test passed locally, together with 14 Slack form/modal tests. The failing CI log and its specific cause remain recorded. - `pnpm -r typecheck` and `pnpm build` passed. Server typechecking was repeated after the final source changes. Map and module-boundary checks and `git diff --check` passed. - The full `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL semaphore exhaustion on this host. Its 1,395-test chat suite found two stale native-control assertions; both were corrected and re-tested in a focused run. Focused database suites used separate disposable databases on existing local PostgreSQL. The full suite is not claimed green; new-head CI must complete before merge. - Final-head CI passed on `e0017d446b1016a38d65382366cd1408cce33039`: all test/build jobs and required `ci / verify` and `ci / e2e` gates are green ([run](https://github.com/paperclipai/paperclip/actions/runs/40115430194)). Greptile and Ponytail both reviewed this exact head at 5/5, the security scan passed, and all review threads are resolved. - All three QA app callbacks were restored to staging. Both temporary tunnels and local proxies were closed. The local endpoint was archived and its local credential copies removed. The candidate app was stopped. Limits: provider execution used a deterministic Claude protocol fixture, not an LLM. This is local real-provider UX proof, not deployed staging or broader Codie/Claude qualification. The public preview intentionally blocked ancillary artifact routes, so its destination pages could show a partial-history warning. The complete permission-grant flow and every oversized/mixed-form branch were not driven manually. Earlier fixture, test, and provider failures remain recorded. ## Risks - New reply routes require durable origin proof. Incorrect receipt matching could lose a reply or route it incorrectly. Tests cover missing, forged, mismatched, and revoked authority. - Governed forms keep their Paperclip resolution checks. This change does not grant new approval authority to Slack. - Existing completed review results can become deliverable through the repaired publication path without another provider run. - No schema, lockfile, workflow, public task-status model, or runtime-instruction change is included. This branch is not deployed. ## Model Used OpenAI Codex, GPT-6 family, with code execution and tool use. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the new head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the new head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
10 KiB
10 KiB
Chat channel question coverage index
This is the index of per-provider question qualification records. The connector playbook owns the Q01–Q11 inventory, control-selection principles, and acceptance standard. Keep that inventory and this index in sync when a channel or question experience changes. Provider capability flags alone do not establish support for a form.
Provider records
The provider list comes from CHAT_PROVIDERS in
packages/shared/src/types/chat-channels.ts. Add a record for a new provider;
update the affected record for a changed form. An unaudited row means this index
has no qualification record for it, not that the provider lacks controls or that
its existing implementation is broken.
| Provider | Q01–Q11 record | Scope and status |
|---|---|---|
| Slack | October 24, 2026 baseline | All eleven core experiences observed in real Slack against a local candidate; limited scope below. Later changes require affected-case remeasurement. |
| GitHub | Not yet indexed | Audit all applicable question paths, including comment replies and authenticated handoffs; no qualification claim here. |
| Discord | Not yet indexed | Audit all applicable question paths; no qualification claim here. |
| Microsoft Teams | Not yet indexed | Audit all applicable question paths; no qualification claim here. |
| Telegram | Not yet indexed | Audit all applicable question paths; no qualification claim here. |
| Speko | Not yet indexed | Establish applicable conversational paths and equivalents for controls; no qualification claim here. |
| AgentMail | Not yet indexed | Include email reply and authenticated handoff behavior; no qualification claim here. |
| iMessage Photon | Not yet indexed | Audit all applicable question paths; no qualification claim here. |
Do not turn an untested native path into “unsupported” or “not applicable.” Research provider capabilities before choosing a response path. A text/reply-only channel can meet the standard with an understandable, correctly bound reply or an exact authenticated handoff; it need not imitate Slack's UI.
Provider record template
Keep the record here or in a provider-specific document linked above. Use one row per inventory ID and relevant variant. Every row needs a state, including gaps; never drop difficult forms from the denominator.
Record metadata once per campaign:
- Accountable owner, date, source commit and dirty-tree fingerprint if relevant, provider/client surface, adapter/SDK versions, and redacted configuration.
- Test scope: local serialization, real-provider local candidate, live-model behavior, or deployed verification. Name a deterministic fixture as such.
- Source entry points: runtime input, composer/card, legacy payload, or canonical question set. List applicable variants and those not exercised.
- Authorized isolated bot/channel and conversation references, reproducible setup, evidence location accessible to reviewers, and cleanup outcome. Keep private conversation links and credentials out of public repository records.
Use this row schema:
| Field | Required content |
|---|---|
| Inventory ID and variant | Q01–Q11 or a newly added ID, exact payload/source entry point, and required/optional/default/size variations. |
| Provider capability and treatment | Native control, structured reply, or Paperclip handoff; official capability/limit reference and installed adapter behavior. Explain why this is the best feasible path. |
| Canonical mapping and authority | How the response maps to the saved question/decision, actor and conversation, including validation and current authorization. A review-link click is navigation, not a response or grant. |
| Functional status | Pass, fail, not run, or not applicable with reason; expected versus actual saved result, canonical continuation policy/count, artifacts, and delivery receipts. |
| Experience grades | Separate ✓/✕ for good experience and best feasible experience, with a short rationale after inspecting unanswered, answering, and settled states. Missing inspection is not run. |
| Evidence and history | Actual provider captures and link-click outcomes, durable-state correlation, failed attempts, repair/retest, and exact candidate for each attempt. |
| Remaining coverage | Untested variants, boundary/negative-test coverage, provider constraints, and any live-model or deployment gap. Name the next required check. |
Prefer a browsable per-format gallery. Keep the conclusions and coverage gaps in this durable record or the PR even if its temporary preview expires. Sensitive evidence can stay in an access-controlled artifact with an accessible reference; do not copy secrets or private tenant data into the index. Reuse existing test harnesses and publication/continuation contracts instead of inventing a provider workflow engine to fill a matrix cell.
Slack: October 24, 2026
Reference implementation and review:
PR #17255.
The local campaign's final implementation was recorded as
c3d458b2a06b660448940fd8cf33c5906f0058d5; captures were collected during its
implementation, with failures and repairs retained in the campaign gallery.
This is a historical evidence summary, not a certification of later PR heads
or staging. The PR verification section records later regression fixes/checks.
Scope: dedicated real Slack QA conversation, local server candidate, desktop Slack client, and a deterministic agent protocol fixture. Each core case had an actual unanswered/answered pair, saved result, one intended continuation, and a published final reply. Open controls and validation states were captured where applicable. The author inspected the gallery and graded each captured experience. The local gallery remains in the originating work chat; it is not a public or permanent hosted acceptance artifact. Reproduce affected cases when that private evidence is unavailable to the reviewer.
The table describes exercised paths, not every possible payload within a kind. The playbook defines the preferred behavior and broader variant requirements.
| ID | Path exercised in Slack | Observed functional result | Good / best feasible | Scope notes |
|---|---|---|---|---|
| Q01 | Native multiline text modal | Pass | ✓ / ✓ | Full prompt/context, editable starting text, saved answer retained. |
| Q02 | Direct choice buttons | Pass | ✓ / ✓ | Descriptions and recommendation visible; selected label retained. |
| Q03 | Inline searchable single-select | Pass | ✓ / ✓ | Selected the thirteenth option by searching; saved label remained visible. |
| Q04 | Checkboxes and explicit submit | Pass after repair | ✓ / ✓ | Empty submission feedback, two selected labels retained, and one continuation after fixing multi-selection authorization. Original failure retained. |
| Q05 | Choice buttons plus Other… text modal | Pass | ✓ / ✓ | Both listed and custom answers exercised; descriptions and recommendation retained. |
| Q06 | Native text modal with inline validation | Pass | ✓ / ✓ | Invalid pattern kept the draft open; corrected answer saved and continued. Not every numeric/length variant was tested live. |
| Q07 | Accept/Reject buttons | Pass | ✓ / ✓ | Ordinary acceptance exercised. Governed and required-reason variants retain the canonical review path; not all were tested live. |
| Q08 | Checklist with Save and Reject | Pass | ✓ / ✓ | Defaults and bounds visible; over-selection rejected, correction saved, exact selection retained. |
| Q09 | Native per-item radio choices and reason fields | Pass | ✓ / ✓ | Required-reason validation and atomic submission exercised; decisions and reasons readable afterward. |
| Q10 | Create task, multi-proposal selection/Create selected, and Dismiss | Pass | ✓ / ✓ | Reviewed created task title/description/parent independently; clicked review and created-task links to their correct destinations. |
| Q11 | Authenticated Review connection handoff and native Not now | Pass after repair | ✓ / ✓ | Clicked the exact review destination and exercised decline without a grant; removed a spurious error caused by the passive link callback. No OAuth/access grant attempted. |
Remaining scope: these eleven cases do not establish all large/mixed-form boundaries, every terminal state, mobile behavior, other providers, live-model judgment, or deployed staging behavior. Deterministic regressions cover additional authorization, duplicate-response, and fallback cases. Later review identified oversized multi-select continuation and large custom-choice fallback defects; their failed-before tests, fixes, and final-head verification belong to the PR record and must not be hidden by this core-case scorecard.
Cleanup: restored the QA app's original callbacks, closed temporary callback and read-only preview tunnels, and removed temporary local credential copies. The read-only preview exercised navigation, not a connection permission grant.
Lessons behind the quality bar
The PAP-701 investigation and this campaign exposed distinct failures:
- A title plus Respond hid the actual question and context. Inventory question content and its answering path, not only interaction kinds.
- The first eleven-case gallery proved saved results returned to Slack, but only three cases answered natively and only one passed the complete UX review. Functional parity and “native” are both weaker than a good experience.
- Real multi-select submission saved correctly but failed continuation authorization. Inspect durable results and subsequent delivery alongside the screenshots; a settled card is not completion.
- The connection review link reached its destination while its callback also displayed a misleading error. Click every relevant handoff/result link and inspect both sides before grading it.
- Later boundary review found gaps beyond the core gallery. Keep core-case UX evidence separate from boundary coverage, retain failures, and remeasure changed paths on the candidate that will ship.
These are reusable acceptance requirements. The eleven Slack cases, particular option counts, fixture choice, and temporary tunnel setup are campaign details; future providers must research their own controls, limits, and delivery mechanism.