DottaandClaude Opus 5.5
353945c86d
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Tool gateways let external MCP clients use a company's tools with a
gateway client token
> - Since delegated connection management (#17737), an agent can create
and revoke gateway client tokens for its responsible person
> - #20581 stopped agents from getting the one-time token value, but it
did this by making token creation person-only
> - The product decision is that agents keep their delegated authority
to create and revoke tokens
> - This pull request restores agent token creation, but the agent never
receives the token value: the person reveals the value once on the
gateway's Tokens page
> - The benefit is that an agent can do the rotation work for the
person, and the secret still never enters chat, run events, logs or tool
output
## Linked Issues or Issue Description
Refs #17737
Refs #20581
**What happened**
After #20581, `POST /api/tool-gateway/gateways/{gatewayId}/tokens`
returns 403 `gateway_token_requires_person` to every agent caller.
Agents lost the delegated authority to create gateway client tokens that
#17737 gave them. Before #20581, an agent that created a token received
the plaintext value in its tool result and could repeat it in chat.
**Expected behavior**
An agent with delegated connection management can create and revoke
gateway client tokens. The agent never receives a token value. The
responsible person gets the value once, on the gateway's Tokens page.
Person callers keep today's behavior.
**Steps to reproduce**
1. Give an agent delegated connection management for a responsible
person with `tools:admin`.
2. As the agent, call `POST
/api/tool-gateway/gateways/{gatewayId}/tokens` (for example through
`call_api`).
3. See 403 `gateway_token_requires_person`.
## What Changed
- Agent create: `POST /api/tool-gateway/gateways/:gatewayId/tokens` from
an agent with delegated authority returns 201 with token metadata only
(`id`, `tokenPrefix`, `expiresAt`, `pendingReveal: true`), plus
`tokensPath`, a `reveal_credential` `requiredUserAction` for the
responsible person, and a `nextStep` instruction. The saved row has no
usable value: its stored hash is of a value that is generated and then
discarded. Gateway authentication also rejects `pendingReveal` rows.
- Person reveal: new person-only `POST
/api/tool-gateway/gateway-tokens/:tokenId/reveal`. It generates the
value, stores its hash, clears `pendingReveal` and returns the value
once. A conditional update makes a second or concurrent reveal return
409. Only the token's responsible person can reveal it (403
`gateway_token_reveal_requires_responsible_person`). Agent callers get
403 `gateway_token_requires_person`. A token revoked before reveal
cannot be revealed.
- Why a reveal that generates the value: no stored plaintext existed to
reveal, and storing one (even encrypted) for later display adds a secret
at rest. Generating the value at reveal time keeps the existing one-time
display and stores only a hash.
- Schema: `tool_mcp_gateway_tokens.pending_reveal boolean not null
default false` (migration `0460_gateway_token_pending_reveal`). Shared
`ToolMcpGatewayToken.pendingReveal`, and `ConnectionRequiredUserAction`
gets `reveal_credential` / `gateway_token`.
- UI: the Tokens page shows a "Waiting for you to reveal" section for
pending tokens with a "Reveal token" button. The value appears in the
existing "New token — copy now" one-time block. Pending tokens show the
status "Not revealed".
- Agent gateway create: the 201 for agent callers also includes
`gatewayPath`, `tokensPath` and a `nextStep` that tells the agent to
link the saved gateway.
- The agent create-token `nextStep` and the catalog say not to revoke
the old token in the same turn: its client keeps working on it until the
person installs the new value.
- The reveal update and its `tool_gateway.token_revealed` activity entry
commit in one transaction. The Tokens page offers Reveal only for tokens
that act for the signed-in person.
- API reference: new `docs/api/tool-gateway-tokens.md` (create, reveal,
revoke) in the docs nav.
- Revoke stays allowed for agents with delegated authority (unchanged
route).
- API catalog: create-gateway, create-token, reveal and revoke
descriptions now say who acts, that agents never receive values, the
page paths, and that the old token is revoked only after the client
works with the new one.
- Agent access map: create-token policy note and a new reveal entry.
- Evals `gateways-profiles.create-gateway` and
`gateways-profiles.rotate-token`:
- Remove the keyword grader (`human|operator|admin|manage`) and the
handoff text checks. Grade saved state, rendered links and secret
absence. A plain reply or a question card is accepted.
- Remove the setup probes that called the create routes as the agent.
Under delegated authority, those probes created a real gateway
("Forbidden agent gateway"), which failed create-gateway before the
agent ran.
- create-gateway: exactly one gateway for the intended profile and
company, created by the agent (or by the person after a handoff); the
person opens it from a rendered link; attribution matches the creator;
any agent-created token is still pending reveal; no token value in chat,
cards or run events.
- rotate-token: after the agent's turn, both setup tokens still work and
any replacement the agent created is pending the person's reveal; the
person follows the rendered Tokens page link and reveals (or issues) the
value in a private browser; the new value works before the old token is
revoked; the unrelated client keeps access; no token value appears
anywhere, including a regex scan for any full `pcgw_<id>.<secret>`
value.
- The changed scenario context and outcomes are recorded in
`chat-full-catalog-testops-manifest.json` `changedFields` and pinned in
the manifest test. The inventory digest, descriptors (`oracleVersion`
v2, `sourceFiles`) and SDK public input pins are updated. The unused
`external-gateway-handoff.ts` module is removed.
- Fixture realism: the create-gateway profile now allows a real read
tool (an empty profile made the agent ask which apps to add instead of
finishing), and the rotation tokens get a neutral owner note instead of
"Disposable rotation fixture" (the agent read that note as an eval
hint).
- Shared private-browser helpers (`external-private-browser.ts`,
`external-private-browser-worker.ts`) get one optional `pendingTokenId`
input and one `revealedPending` receipt field.
## Verification
- Red: on `master` route code, the new integration tests `lets agents
create and revoke gateway client tokens without ever receiving a token
value` and `reveals an agent-created gateway token value only to its
responsible person` fail (403 `gateway_token_requires_person`).
- Green: `cd server && npx vitest run
src/__tests__/connection-management.integration.test.ts -t gateway` (25
passed). The test checks that the agent `call_api` result has no `token`
field and no token-shaped value, that the agent cannot reveal, that the
person reveals once (second reveal 409) and the value authenticates,
that agent revoke works and stops the value, that a token revoked before
reveal never gets a value, and that person create still returns the
value.
- `npx vitest run src/__tests__/tool-gateway.test.ts
src/__tests__/openapi-routes.test.ts
src/services/native-runtime/runner-api.test.ts` (2247 passed).
- `cd ui && npx vitest run
src/pages/apps/gateways/panels/TokensPanel.test.tsx
src/pages/apps/gateways/gateway-helpers.test.ts` (16 passed); `pnpm
check:token-gates` clean.
- Eval graders and catalog: `pnpm exec vitest run --config
tests/runner-e2e/vitest.config.ts` on the gateway oracle tests,
`external.test.ts`, catalog consistency/contract, TestOps
export/retained, phase/clock callers and SDK input/type tests (all
pass); `node --test` on the TestOps manifest, adapter, contract,
streaming and reconcile tests (all pass); `node
tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`
(unchanged).
- `node scripts/check-maps.mjs` passes. `tsc --noEmit` for `server`,
`ui` and `tests/runner-e2e`: no errors in changed files.
- Paid end-to-end run of both eval cases, run
[42872921890](https://github.com/paperclipai/paperclip/actions/runs/42872921890)
on 39f38a7f6c (before the last three commits): both FAILED, for
agent-behavior reasons the later commits address. Not rerun yet.
- rotate-token: the agent created the replacement (pending reveal, no
value), then revoked the old token in the same turn, despite the create
response saying to wait. Its reasoning cited the eval-looking fixture
note. The grader rejected the premature revocation. The reply linked the
Tokens page correctly.
- create-gateway: the agent created the gateway and one pending token,
then asked a question card about which apps to add to the empty profile,
with no link to the gateway. The grader rejected the missing link.
- Token values: a scan of all evidence files of both cells found no full
`pcgw_<id>.<secret>` value and no `pcgw_<uuid>` form. Agent `call_api`
payloads in run events are redacted; the integration test proves the
agent tool result has no value.
## Risks
- Behavior change for agents: token create returns metadata and a Tokens
page path, not a value. Agents that expect `token` in the response must
send the person to the Tokens page.
- Migration adds one boolean column with a default. Existing tokens get
`false` and keep working.
- A pending token that is never revealed stays as an unusable row until
someone revokes it. It cannot authenticate.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Chat feature scenarios
This is the planning catalog for what happens when a person asks an agent to use
Paperclip. It belongs to Product E2E, alongside the existing chat
and Everyday Workflows suites. These files contain human requests, assumed context
and loosely specified outcomes. Entries alone do not run models, register tests,
or contribute passing results to a report. Two task entries now have explicit
executable Product E2E cases: tasks.create-backlog and
tasks.delegate-announcement. Their results remain in the existing E2E reports.
The feature map answers what the product lets a person do. The agent access map records which APIs and actual agent entry points exist and their boundaries. This catalog asks whether a person can accomplish that work through a conversation, including discovering a prerequisite, completing a human step and returning to work. All three reuse the same capability IDs.
Start with these requests
- Hire a Claude Code engineer:
agents.hire-claude-code, with missing-account, approval-required, denied-permission and ambiguous-wording cases. - Create a website task:
tasks.website-task, with missing-project, attached-brief, ambiguous-task, privacy and ownership-conflict cases. - Archive completed inbox items:
inbox-search.archive-done, with missing authority, partial failure and undo cases. - Read Notion:
connection-setup.notion-connectedandconnection-setup.notion-missingdeliberately use the same sentence. The second requires setup in the conversation and continuation after usable access exists. Other cases cover decline, wrong account, unassigned agent and expired access.
Reading a scenario
Each area file uses catalog.schema.json and implicitly inherits defaults.yaml. Scenario context overrides those defaults. A scenario is independent: mentions of previous turns or existing resources are fixture requirements, not dependencies on running an earlier catalog entry.
| Field | Meaning |
|---|---|
schema_version |
Version of this planning format; currently 1. |
feature |
Primary feature-map filename without .md. |
status |
Always planned; execution results belong to the existing E2E system. |
scenarios[].id |
Stable, globally unique <feature>.<scenario> identifier. |
capabilities |
Full feature-map IDs exercised by this request; may span areas. |
context |
Existing records, current identity, authority, accounts, gates and significant starting conditions, plus any later human action or fault needed for the branch. |
query |
The natural-language message a person would send, including an answer or follow-up where that is the subject of the case. |
expected_outcome |
A paragraph describing the visible conversation, resulting durable state, necessary human interaction and meaningful boundaries. |
For example, two fixtures can send “Read my Notion and tell me the latest on the launch.” In the connected fixture, the answer comes from current pages and links back to them. In the unconnected fixture, the agent requests setup, the addressed person completes it, and the original work continues. A third fixture declines setup and should receive a truthful explanation without any provider read.
Context and expected outcomes are evaluator material. Do not paste them into an agent's system prompt or teach the model which API to choose. A future fixture should expose the actual product state, ordinary production instructions, the human query and any genuine attachments or previous conversation. The model must discover the available operations normally.
What coverage means here
Every current feature-map capability has at least one associated scenario. This is inventory coverage, not complete operation coverage, source verification or live acceptance. Broad capabilities can still need more CRUD, actor, transport, provider and lifecycle variations. The maps' Policy and Gaps sections remain the source for access assumptions; recheck them against implementation when promoting an entry. The YAML does not grant permissions or promise an unimplemented UI.
The default starting point is a person talking to their CEO in Agent Chat with legitimate agent permissions. CEO is not board identity. A valid expected outcome can therefore be direct action, a grounded answer, a necessary clarification, a connection or approval request, a precise human handoff, or a truthful boundary. An unsupported operation must not be “passed” because the agent says it succeeded. A vague “you can do this in settings” is also insufficient when a supported specific control or request card exists.
Some capabilities cannot originate inside an already working chat: first server installation, login, an unfinished first-agent wizard, browser preferences and developer-only labs. Their entries explicitly describe the relevant boundary or companion journey. External MCP and channel cases also name their different surface. Preserve those labels in future selection and reporting; a successful operator, browser-only or external-client step cannot certify a direct Agent Chat action. Where only guidance is currently possible, qualifying automated chat execution requires a separate product change and updated access contract.
Conditional coverage to preserve
| Condition | Representative scenarios |
|---|---|
| Account ready / missing / wrong / expired / not assigned | connection-setup.notion-* |
| Allowed / human approval / denied / wrong resolver | agents.hire-*, questions-and-approvals.wrong-reviewer |
| Unambiguous / missing required detail / ambiguous reference | tasks.website-task, questions-and-approvals.necessary-question, tasks.ambiguous-task |
| Complete list / partial mutation / undo | tasks.organize-open-work, inbox-search.archive-partial-failure, inbox-search.restore-archive |
| Same company / other company / private resource | access.other-company, access.private-resource, tasks.private-task |
| Current decision / stale decision / duplicate answer | questions-and-approvals.ambiguous-yes, questions-and-approvals.review-plan, questions-and-approvals.duplicate-answer |
| Idle / busy / steering / stop / saved continuation | steering.*, recovery.post-stop-message |
| Known result / uncertain side effect / failed accounting | recovery.unknown-email-outcome, budgets-costs.repair-cost-recording |
| Feature enabled / disabled / different surface | cases.cases-disabled, agent-chat.chat-disabled, developer-labs.gated-lab |
| Saved record / actually ready / independently usable output | runs-adapters.validate-hire, workspaces.preview-service, documents-artifacts.produce-download |
Different starting conditions should usually be separate scenarios with the same query, so one favorable fixture cannot satisfy all branches. Outcomes can mention invariant guards without turning each paragraph into a complete executable test. When implementing a branch that currently contains several human choices, split those choices into explicit cells while preserving the planning scenario's ID as traceability metadata.
Promoting a scenario later
- Read the matching product recipe and access companion. Confirm the operation, actual admitted agent surface and permission rules; keep current gaps visible. Decide whether this is an Agent Chat journey or a named companion boundary.
- Reuse or extend the existing Product E2E fixture, catalog and report paths. Everyday Workflows already cover hiring/reuse, delegation, connection decisions and delivered work. Existing chat-hardening, chat-qualification, completion-update and confirmation suites are also documented in the E2E guide. Overlap is a reuse opportunity, not evidence that this exact query and context have passed.
- Create isolated synthetic state through supported production setup surfaces. Pin the clock, identities, policy, runtime and provider profile; record feature gates and owned test connections. Keep actual secret material out of this catalog. Follow fixture and security rules.
- Send the query through the real conversation UI. Drive each specified human answer, connection callback, navigation, reconnect or fault at its actual boundary. Preserve the source chat and linked tasks across those transitions.
- Turn the outcome into independent assertions: rendered response and correct links; saved records and attribution; exact target and audience; actual file bytes or provider receipt; absence of unintended mutations; correct return to work. Validate the oracle against a plausible wrong outcome. Success prose, a tool declaration or an accepted HTTP response alone cannot pass a journey.
- Declare the ordinary E2E execution selection, environment, turn/time/cost limits, cleanup and evidence contract. Retain failures, retries, source and definition revisions, actual model/profile, usage and missing evidence in the existing report. A changed setup or model is a separate measurement, and a missing observer is not a pass. No live or paid runs are authorized by these planning files themselves.
The first useful runnable slice is the user's three core jobs plus the paired Notion branches above. Extend from that slice across this catalog without building a second runner, scheduler, scoring system or evidence store.
Maintaining the catalog
Keep area filenames aligned with the feature map and IDs stable when wording changes. Add a capability reference only when the query and outcome exercise it; linking every scenario to every related subsystem would inflate coverage. Write specific context and an observable paragraph rather than mechanically repeating a feature description. Use cross-feature references for real combined journeys.
Before handing off edits, parse every YAML file, validate area files against the
schema, check unique IDs and nonempty prose, and reconcile capability references
with the Sub-features keys in maps/feature-map/*.md. Check both missing and
unknown IDs. The schema alone cannot validate references or uniqueness across
files. defaults.yaml is shared context and is not an area-schema document.
chat-scenario-catalog.test.ts performs these structural checks in
pnpm test:e2e:runner:unit, including the 35 areas, 244 scenario IDs and 191 mapped
capabilities. Update the inventory counts intentionally when the catalog grows.
After editing map links, also run node scripts/check-maps.mjs. That existing
check validates the maps, not this YAML catalog and not live chat behavior. Keep
structural validation and any future execution evidence separate. Do not label a
scenario passed, register a paid cell or alter default suite selection merely
because its planning text has been reviewed.
Browse by feature
The feature recipe and same-named access companion define each area's behavior and authority. Each YAML file can also reference capabilities from other areas.