mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 10:13:11 +02:00
DottaandPaperclip
e6dcd1e8e9
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can submit suggestions and complaints while they work. > - Operators currently need database access to read those messages. > - That makes it hard to find feedback from one agent or share a specific message. > - This pull request adds a standard Commentary tab in Audit. > - Operators can filter messages, browse 50 at a time, and share links to full message dialogs. ## Linked Issues or Issue Description Refs #15367, which added feedback storage and agent submission. The duplicate search found no open PR for a commentary reader. **Subsystem affected** Server, shared API contracts, and the board UI. **Problem or motivation** Agent suggestions and complaints are stored but have no board reader. Operators must query the database to inspect feedback. **Proposed solution** Add Commentary to Audit without an experimental toggle. Show previews, agent and message-type filters, text search, and 50-message pages. Opening a row updates the URL. A copied URL opens the full message with its agent, task, and run context. **Alternatives considered** Direct database queries remain useful for operators, but do not provide a normal product workflow. An experimental setting would add setup to a read-only audit page. **Roadmap alignment** This completes the operator view for the existing feedback feature. It supports human oversight and does not change agent execution or task ownership. ## What Changed - Add company-scoped, board-only list and detail endpoints with audit permission and source-task privacy checks. - Return bounded previews and counts. Fetch full bodies only when a message opens. - Add the Commentary tab, search, filters, 50-row pagination, and URL-based selection. Audit tab destinations work in both layouts, with a classic sidebar entry for Commentary. - Add production Storybook examples for populated, filtered, linked, long, paginated, empty, loading, failed, denied, mobile, and light states. - Update API documentation and the feature and agent access maps. ## Verification - Final head `abc4a251c14cfbfec007542214beec528f2b18bf`: 55 checks pass; two conditional Storybook checks are skipped. The PR workflow is successful. Greptile is 5/5 with all findings resolved. - Focused database, HTTP, submission, OpenAPI, and UI regression tests cover company isolation, task privacy, filter bounds, 50-row pagination, and copied message links. - Browser Storybook journeys cover row opening, direct links, combined filters, search, and pagination. Storybook uses synthetic API fixtures; database authorization is tested separately. - Token gates, feature maps, and Storybook catalog validation pass. - `pnpm -r typecheck`, `pnpm build`, and `pnpm build-storybook` pass on the rebased branch. All 80 focused tests and all 37 Audit tests pass, including the real App route table in both layouts, task grant/revocation through HTTP, deleted-task privacy, and URL UI regressions. The full suite passes in the parallel PR CI matrix. The local serial full-suite run was stopped before completion after checking the recorded 94-minute serial duration. Local focused tests, workspace typecheck, production build, and final server/UI checks pass. - The linked-message Storybook test initially asserted against the loading dialog. It now waits for the message body; both direct-link and row-open journeys pass. - Native and legacy deleted-task privacy regressions failed before the fix and pass afterward. The reader checks the run task binding after the task foreign key clears. - A rapid search-then-agent-filter browser regression reproduced the search overwriting the selected agent. The pending search now restarts when URL filters change. The same browser test passes with both filters applied. - CI found two existing AuditHub assertions that still expected five tabs. They now assert the six-tab model and selected-company Commentary rendering. All 37 Audit tests pass. All other test and build jobs passed before this test-only correction. - The [agent-chat attachment browser test](https://github.com/paperclipai/paperclip/actions/runs/39307881171/job/118250189664) timed out waiting for an attachment preview. The [same shard passed on the prior head](https://github.com/paperclipai/paperclip/actions/runs/39306658703/job/118245498343) with identical application code; the intervening change only updates AuditHub unit tests. The single retry passed on the same head. The precise cause of the earlier failure remains unconfirmed. ## Risks - Commentary may contain sensitive task context. The reader requires board access and audit permission, and filters source-task visibility before returning previews, counts, or full bodies. - Text search scans stored message bodies. Pages are bounded and full bodies are fetched on demand. - Offset pagination can shift if new messages arrive while paging. Ordering uses timestamp and ID for stable ties. - No database migration or agent submission behavior changes are required. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, tool use, browser interaction, and code execution. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
169 lines
9.3 KiB
Markdown
169 lines
9.3 KiB
Markdown
# Agent complaints and suggestions
|
||
|
||
Agents submit incidental feedback into the current instance's `agent_commentary`
|
||
table. Complaints preserve a raw reaction; suggestions describe an improvement.
|
||
Both are free-form, internally attributed records, not anonymous feedback.
|
||
|
||
## Agent entry points
|
||
|
||
Skill-capable legacy adapters automatically receive `complain` and
|
||
`suggestion-box` alongside the `paperclip` operational skill, including existing
|
||
agents with empty optional-skill selections. The shared helper lives in that
|
||
operational skill at `scripts/submit-agent-commentary.mjs`. It reads the body
|
||
from stdin and uses the existing `PAPERCLIP_API_URL`, `PAPERCLIP_API_KEY`,
|
||
`PAPERCLIP_COMPANY_ID`, and `PAPERCLIP_RUN_ID`. The helper uses the existing
|
||
Node.js runtime with no additional dependencies; custom runtimes can call HTTP
|
||
directly. Sandboxed runs use the callback bridge.
|
||
|
||
Native runs receive `submit_complaint({body, idempotencyKey})` and
|
||
`submit_suggestion({body, idempotencyKey})` in standard, ask, and planning modes.
|
||
Specialized completion-review runs retain their restricted tool set. Native
|
||
submission uses the bound authority without a shell or API credential.
|
||
Submit before finishing the run.
|
||
|
||
Instructions closely adapt Warp's
|
||
[complain](https://github.com/warpdotdev/common-skills/blob/main/.agents/skills/complain/SKILL.md)
|
||
and [suggestion-box](https://github.com/warpdotdev/common-skills/blob/main/.agents/skills/suggestion-box/SKILL.md)
|
||
skills, with their MIT notices included. Paperclip changes the transport,
|
||
attribution, suggestion form, and size limit. Agents submit proactively when
|
||
warranted, without routine previews or announcements, then continue working.
|
||
Answer truthfully if a user asks about feedback or submitted content. Operators
|
||
can exclude these legacy skills through the existing runtime skill policy.
|
||
Brevity and at most three suggestions per run are guidance, not server quotas.
|
||
Do not report the same incident through both paths or retry failed submissions.
|
||
|
||
## HTTP contract
|
||
|
||
`POST /api/companies/:companyId/agent-commentary` accepts an authenticated agent
|
||
and its active run (`Authorization: Bearer …`, `X-Paperclip-Run-Id: …`):
|
||
|
||
```json
|
||
{"kind":"suggestion","body":"The tool returned before persistence completed. Wait for the write before returning success.","idempotencyKey":"write-completion-1"}
|
||
```
|
||
|
||
Only these three fields are accepted. Identity and the optional task reference
|
||
are server-derived. Board callers cannot impersonate agent feedback. Native runs
|
||
use their dedicated tools rather than this legacy HTTP path.
|
||
|
||
Bodies must contain non-whitespace text and may contain up to 524,288 JavaScript
|
||
string code units, matching ordinary issue documents. No truncation is applied.
|
||
Existing 10 MiB HTTP and callback bridge request limits remain. Keys are nonempty
|
||
and at most 240 characters.
|
||
|
||
Creation returns HTTP 201 with `{id, kind, createdAt, replayed:false}`. Identical
|
||
replay in the same company/run returns HTTP 200 with the same ID and timestamp
|
||
and `replayed:true`. Conflicting key reuse returns 409. Invalid input returns 400;
|
||
invalid authority returns 401/403; storage failure returns a sanitized 503.
|
||
Authorization precedes replay. The helper accepts an optional second argument
|
||
for a stable key, generating one otherwise. It makes one request with a
|
||
10-second HTTP deadline and exits zero on failure with a content-free diagnostic.
|
||
It never follows redirects with credentials or claims an uncertain write succeeded.
|
||
|
||
## Storage and inspection
|
||
|
||
Rows contain company, agent, run, nullable issue, kind, body, retry key, payload
|
||
hash, and timestamp. Known run secrets and credential syntax are redacted before
|
||
storage; instructions remain essential because redaction cannot guarantee
|
||
secrecy. Writes and one content-free activity record commit together. Feedback
|
||
bodies are excluded from HTTP diagnostics and dedicated mutation receipts.
|
||
|
||
Browse feedback at Audit → Commentary (`/activity/commentary`). The page has
|
||
text search, agent/type filters, 50-message pages and a full-message dialog with
|
||
task/run context. Filters, page offsets and selected messages have shareable
|
||
URLs (`?entry=<message-id>` opens a message directly).
|
||
|
||
The board-only read API is `GET /api/companies/:companyId/agent-commentary`
|
||
(`agentId`, `kind`, `q`, `limit` up to 100, `offset`) and
|
||
`GET /api/companies/:companyId/agent-commentary/:id`. Both require company access
|
||
and `audit:view_agent_actions`. Source-task privacy filters previews, counts and
|
||
details. List results contain at most 280 characters
|
||
per message; the full body is loaded only on opening a message. Feedback whose
|
||
source task was deleted stays hidden, using the task binding retained on its run.
|
||
|
||
There is no notification, automatic task creation, or external forwarding.
|
||
This is separate from first-party Telemetry, OpenTelemetry
|
||
Observability, and the run log. Ordinary provider transcripts can still contain
|
||
submitted tool arguments; there is no anonymity or ephemeral-storage promise.
|
||
|
||
Authorized operators inspect the instance database, for example:
|
||
|
||
```sql
|
||
SELECT id, kind, body, agent_id, run_id, issue_id, created_at
|
||
FROM agent_commentary
|
||
WHERE company_id = '<company UUID>'
|
||
ORDER BY created_at DESC;
|
||
```
|
||
|
||
The service enforces company/run ownership. Foreign keys clear the task reference
|
||
on task deletion and cascade deletion with its run, agent, or company. Normal
|
||
logical database backups include these rows. No retention scheduler or backfill.
|
||
|
||
## Verification
|
||
|
||
Focused coverage lives in `server/src/__tests__/agent-commentary.integration.test.ts`,
|
||
`agent-commentary-helper.test.ts`, `agent-commentary-skills.test.ts`, the shared validator tests, and the HTTP logger
|
||
and sandbox callback bridge suites. These exercise real PostgreSQL and routes,
|
||
atomic audit rollback, replay races, authority changes, deletion, default mounts,
|
||
Unicode, and the exact document-body boundary.
|
||
|
||
The opt-in live smoke uses existing Codex login credentials, copied to a private
|
||
temporary home, and a disposable instance database. It calls the real legacy
|
||
adapter and production native-session executor. It incurs normal provider usage:
|
||
|
||
```sh
|
||
node cli/node_modules/tsx/dist/cli.mjs server/scripts/verify-agent-commentary-live.ts
|
||
```
|
||
|
||
To run the same checks on real Daytona sandboxes, install the standalone
|
||
provider's dependencies, provide its API key and an immutable image reference,
|
||
and specify a Linux amd64 runner built from this checkout:
|
||
|
||
```sh
|
||
pnpm --dir packages/plugins/sandbox-providers/daytona install --frozen-lockfile
|
||
PAPERCLIP_LIVE_ENVIRONMENT=daytona \
|
||
PAPERCLIP_LIVE_DAYTONA_ENV_FILE=/path/to/private.env \
|
||
PAPERCLIP_LIVE_LINUX_RUNNER=/path/to/linux-amd64/paperclip-runnerd \
|
||
PAPERCLIP_LIVE_EVIDENCE_PATH=/tmp/commentary-evidence.json \
|
||
node cli/node_modules/tsx/dist/cli.mjs server/scripts/verify-agent-commentary-live.ts
|
||
```
|
||
|
||
The private env file needs only `DAYTONA_API_KEY` and
|
||
`PAPERCLIP_E2E_DAYTONA_IMAGE` (`...@sha256:...`). Alternatively, export those
|
||
variables directly. A positional `legacy` or `native` argument selects one path.
|
||
For a current Linux runner, add an export stage to a temporary copy of the root
|
||
Dockerfile and build it with `--platform linux/amd64 --target commentary-runner-export
|
||
--output type=local,dest=/tmp/commentary-runner-linux`:
|
||
|
||
```dockerfile
|
||
FROM scratch AS commentary-runner-export
|
||
COPY --from=runner-build /app/packages/paperclip-runner/runner/target/release/paperclip-runnerd /paperclip-runnerd
|
||
```
|
||
|
||
The smoke calls the production Daytona provider hooks for creation, execution,
|
||
file transfer, private ingress, and confirmed deletion. Legacy feedback crosses
|
||
the sandbox queue callback bridge. Native feedback crosses the runner's private
|
||
WebSocket and live tool authority. The current Linux runner is staged and
|
||
verified independently of the base image. Fixture setup creates authority;
|
||
the real agent submits all feedback. This is a focused feedback transport smoke,
|
||
not a full Runner E2E catalog qualification or browser onboarding test.
|
||
|
||
Both paths passed locally and on Daytona on 2026-10-06. Each persisted one complaint and one suggestion
|
||
with company, agent, run, and task attribution; each then wrote the continuation
|
||
marker. Both exited zero, left task status unchanged, created no task comments,
|
||
and recorded two content-free activity entries. The script emits the attributed
|
||
row IDs and timestamps as evidence, optionally saves a JSON report, and deletes
|
||
its temporary data, credentials, and remote sandboxes. Remote reports are emitted
|
||
only after confirmed sandbox deletion.
|
||
|
||
The Daytona verification used base image
|
||
`ghcr.io/paperclipai/paperclip-daytona-runner@sha256:b81a86d5242088f9d832666a411f09da7438d92e99f9962ccf88ebe439cd3b32`
|
||
and the Linux runner built from `abf47b5953d14aa7a8dbf2941460b5df8a4d66c3`
|
||
(binary SHA-256 `c794141152ae2e2986da0df14b01a54e4448949e06ec4031c83bad8a9fff8bac`).
|
||
|
||
| Environment | Runner | Complaint row | Suggestion row | Submitted at (UTC) |
|
||
| --- | --- | --- | --- | --- |
|
||
| Local | Legacy Codex | `59413a00-1de2-4bb1-bcc6-9c4b54c64aa6` | `3db2364d-3e15-4f47-846f-875d3902999d` | 20:35:10 |
|
||
| Local | Native Codex | `5da22b5f-41df-4de5-8ba0-d9345ab01267` | `2d5abe17-dd41-403c-a5ee-4729f2d58921` | 20:35:26–20:35:27 |
|
||
| Daytona | Legacy Codex | `27c9d0aa-8477-409f-9da0-e8ffa48dee50` | `209681c9-d1e9-4ce1-999e-48fa07692389` | 20:33:07–20:33:09 |
|
||
| Daytona | Native Codex | `6eb001bb-4bcf-43f7-8717-f662f53dc7c3` | `77c383d8-a997-49e5-a33e-25c70e15c0b2` | 20:31:54 |
|