Files
PaperClipAI/docs/feedback-voting.md
DottaandPaperclip 3f7b00a8d0 fix: preserve public identifiers in feedback redaction (#16168)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People rate agent comments and documents to capture feedback.
> - Feedback bundles remove credentials and personal information before
export.
> - The feedback scanner treats ordinary dotted text as JWTs and
authentication terms as bearer tokens.
> - This pull request checks encoded JWT headers and preserves known
authentication terms.
> - The benefit is readable feedback context with credential masking
kept in place.

## Linked Issues or Issue Description

Refs #12309 for the related dotted-identifier false-positive report.
This PR fixes the separate feedback scanner. It does not close every
case in that report.

Related PR: #14318 adds fine-grained GitHub token coverage. It addresses
a separate credential form and is not duplicated here.

**What happened?**

Feedback exports replace `api.openai.com`, dotted filenames, and public
schema IDs with a JWT placeholder. They also replace the words after
`bearer` in phrases such as “bearer tokens.” Five-part encrypted JWTs
are only partly masked.

**Expected behavior**

Keep public identifiers and authentication terms readable. Mask complete
credential tokens. Count only text replacements in the redaction
summary.

**Steps to reproduce**

1. Pass `api.openai.com` or `Use bearer tokens.` to
`sanitizeFeedbackText`.
2. Check that the original text is returned and the redaction counts
stay empty.
3. Pass a signed JWT or an encrypted JWT to the same function.
4. Check that the full token is replaced and one JWT replacement is
counted.

**Paperclip version or commit**

Reproduced in the feedback scanner on master at
`5e8e992df2b3d71f82091b7bbf29fd482854d9f5`.

**Deployment mode**

Built from source. The defect is in the shared feedback export service
and does not depend on deployment mode.

## What Changed

- Use the existing JWT header predicate in the feedback scanner.
- Match complete three-part and five-part tokens, including
direct-encryption tokens with an empty key segment. Accept short token
segments in the predicate.
- Preserve the closed set of authentication terms already used by the
general scanner, including terms inside closing prose delimiters.
- Count only replacements that change the text.
- Add tests for readable context, credential masking, nested values, and
redaction counts.
- Document feedback text redaction.

## Verification

- Focused redaction and feedback service tests: 89 tests pass across
five suites.
- `pnpm --filter @paperclipai/adapter-utils typecheck` passes. Full
local typecheck, tests, and build were attempted. The kernel killed
compiler and test processes at the sandbox memory limit of 4 GB. Serial
retries and a capped Node heap still could not complete the server
compiler. CI runs the full repository checks.
- `git diff --check` passes.
- Latest commit `0eb8d3fb4257bb79e67fab615a8299ef69230ad8`: **55 checks
passed**, including full typecheck, tests, build, and E2E; two optional
Storybook checks were skipped. [CI
evidence](https://github.com/paperclipai/paperclip/actions/runs/38407932875).
Both Greptile findings have regression tests and fixes. [Greptile gives
this commit
5/5](https://github.com/paperclipai/paperclip/pull/16168#issuecomment-6146917423),
with both review threads resolved.

## Risks

- This changes the feedback export matching rules. Public dotted text no
longer triggers credential masking by shape alone.
- Explicit credential assignments, provider keys, structured secret
fields, email addresses, and phone numbers remain masked. Tests cover
these cases.
- Existing feedback snapshots are not rewritten. No schema change or
migration is needed.
- The encoded-header check remains a heuristic. It is not a full JWT
validator.

## Model Used

OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The
runtime identifies the model as GPT-6. It does not expose the exact
provider deployment ID or context window size. The earlier patch and
this PR preparation were AI-assisted.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-13 09:10:37 -05:00

202 lines
8.3 KiB
Markdown

# Feedback Voting — Local Data Guide
When you rate an agent's response with **Helpful** (thumbs up) or **Needs work** (thumbs down), Paperclip saves your vote locally alongside your running instance. This guide covers what gets stored, how to access it, and how to export it.
## How voting works
1. Click **Helpful** or **Needs work** on any agent comment or document revision.
2. If you click **Needs work**, an optional text prompt appears: _"What could have been better?"_ You can type a reason or dismiss it.
3. A consent dialog asks whether to keep the vote local or share it. Your choice is remembered for future votes.
### What gets stored
Each vote creates two local records:
| Record | What it contains |
|--------|-----------------|
| **Vote** | Your vote (up/down), optional reason text, sharing preference, consent version, timestamp |
| **Trace bundle** | Full context snapshot: the voted-on comment/revision text, issue title, agent info, your vote, and reason — everything needed to understand the feedback in isolation |
All data lives in your local Paperclip database. Nothing leaves your machine unless you explicitly choose to share.
When a vote is marked for sharing, Paperclip immediately tries to upload the trace bundle through the Telemetry Backend. The upload is compressed in transit so full trace bundles stay under gateway size limits. If that immediate push fails, the trace is left in a retriable failed state for later flush attempts. The app server never uploads raw feedback trace bundles directly to object storage.
## Viewing your votes
### Quick report (terminal)
```bash
pnpm paperclipai feedback report
```
Shows a color-coded summary: vote counts, per-trace details with reasons, and export statuses.
```bash
# Installed CLI
paperclipai feedback report
# Point to a different server or company
npx paperclipai feedback report --api-base http://127.0.0.1:3000 --company-id <company-id>
# Include raw payload dumps in the report
pnpm paperclipai feedback report --payloads
```
### API endpoints
All endpoints require board-user access (automatic in local dev).
**List votes for an issue:**
```bash
curl http://127.0.0.1:3102/api/issues/<issueId>/feedback-votes
```
**List trace bundles for an issue (with full payloads):**
```bash
curl 'http://127.0.0.1:3102/api/issues/<issueId>/feedback-traces?includePayload=true'
```
**List all traces company-wide:**
```bash
curl 'http://127.0.0.1:3102/api/companies/<companyId>/feedback-traces?includePayload=true'
```
**Get a single trace envelope record:**
```bash
curl http://127.0.0.1:3102/api/feedback-traces/<traceId>
```
**Get the full export bundle for a trace:**
```bash
curl http://127.0.0.1:3102/api/feedback-traces/<traceId>/bundle
```
#### Filtering
The trace endpoints accept query parameters:
| Parameter | Values | Description |
|-----------|--------|-------------|
| `vote` | `up`, `down` | Filter by vote direction |
| `status` | `local_only`, `pending`, `sent`, `failed` | Filter by export status |
| `targetType` | `issue_comment`, `issue_document_revision` | Filter by what was voted on |
| `sharedOnly` | `true` | Only show votes the user chose to share |
| `includePayload` | `true` | Include the full context snapshot |
| `from` / `to` | ISO date | Date range filter |
## Exporting your data
### Export to files + zip
```bash
pnpm paperclipai feedback export
```
Creates a timestamped directory with:
```
feedback-export-20260331T120000Z/
index.json # manifest with summary stats
votes/
PAP-123-a1b2c3d4.json # vote metadata (one per vote)
traces/
PAP-123-e5f6g7h8.json # Paperclip feedback envelope (one per trace)
full-traces/
PAP-123-e5f6g7h8/
bundle.json # full export manifest for the trace
...raw adapter files # codex / claude / opencode session artifacts when available
feedback-export-20260331T120000Z.zip
```
Exports are full by default. `traces/` keeps the Paperclip envelope, while `full-traces/` contains the richer per-trace bundle plus any recoverable adapter-native files.
Feedback text redaction preserves hostnames, dotted filenames, public schema IDs,
and authentication terms such as “bearer tokens.” Dotted text must have an encoded
JWT header to be masked as a token. Complete three-part signed tokens and
five-part encrypted tokens are masked, including tokens with short segments
and direct-encryption tokens with an empty encrypted-key segment.
Explicit credentials, secret fields, email addresses, and phone numbers remain
masked. The redaction summary counts text that was actually replaced.
```bash
# Custom server and output directory
npx paperclipai feedback export --api-base http://127.0.0.1:3000 --company-id <company-id> --out ./my-export
```
### Reading an exported trace
Open any file in `traces/` to see:
```json
{
"id": "trace-uuid",
"vote": "down",
"issueIdentifier": "PAP-123",
"issueTitle": "Fix login timeout",
"targetType": "issue_comment",
"targetSummary": {
"label": "Comment",
"excerpt": "The first 80 chars of the comment that was voted on..."
},
"payloadSnapshot": {
"vote": {
"value": "down",
"reason": "Did not address the root cause"
},
"target": {
"body": "Full text of the agent comment..."
},
"issue": {
"identifier": "PAP-123",
"title": "Fix login timeout"
}
}
}
```
Open `full-traces/<issue>-<trace>/bundle.json` to see the expanded export metadata, including capture notes, adapter type, integrity metadata, and the inventory of raw files written alongside it.
Each entry in `bundle.json.files[]` includes the actual captured file payload under `contents`, not just a pathname. For text artifacts this is stored as UTF-8 text; binary artifacts use base64 plus an `encoding` marker.
Built-in local adapters now export their native session artifacts more directly:
- `codex_local`: `adapter/codex/session.jsonl`
- `claude_local`: `adapter/claude/session.jsonl`, plus any `adapter/claude/session/...` sidecar files and `adapter/claude/debug.txt` when present
- `opencode_local`: `adapter/opencode/session.json`, `adapter/opencode/messages/*.json`, and `adapter/opencode/parts/<messageId>/*.json`, with optional `project.json`, `todo.json`, and `session-diff.json`
## Sharing preferences
The first time you vote, a consent dialog asks:
- **Keep local** — vote is stored locally only (`sharedWithLabs: false`)
- **Share this vote** — vote is marked for sharing (`sharedWithLabs: true`)
Your preference is saved per-company. You can change it any time via the feedback settings. Votes marked "keep local" are never queued for export.
## Data lifecycle
| Status | Meaning |
|--------|---------|
| `local_only` | Vote stored locally, not marked for sharing |
| `pending` | Marked for sharing, saved locally, and waiting for the immediate upload attempt |
| `sent` | Successfully transmitted |
| `failed` | Transmission attempted but failed (for example the backend is unreachable or not configured); later flushes retry once a backend is available |
Your local database always retains the full vote and trace data regardless of sharing status.
## Remote sync
Votes you choose to share are sent to the Telemetry Backend immediately from the vote request. The server also keeps a background flush worker so failed traces can retry later. The Telemetry Backend validates the request, then persists the bundle into its configured object storage.
- App server responsibility: build the bundle, POST it to Telemetry Backend, update trace status
- Telemetry Backend responsibility: authenticate the request, validate payload shape, compress/store the bundle, return the final object key
- Retry behavior: failed uploads move to `failed` with an error message in `failureReason`, and the worker retries them on later ticks
- Default endpoint: when no feedback export backend URL is configured, Paperclip falls back to `https://telemetry.paperclip.ing`
- Important nuance: the uploaded object is a snapshot of the full bundle at vote time. If you fetch a local bundle later and the underlying adapter session file has continued to grow, the local regenerated bundle may be larger than the already-uploaded snapshot for that same trace.
Exported objects use a deterministic key pattern so they are easy to inspect:
```text
feedback-traces/<companyId>/YYYY/MM/DD/<exportId-or-traceId>.json
```