Merge branch 'codex/work-folders-pi-refresh' into codex/work-folders-storybook-refresh

* codex/work-folders-pi-refresh:
  fix(runner): keep healthy native sessions alive (#13261)
  feat(connections): add AgentMail inboxes and email tasks (#13256)
  fix: stop completion reviews caused by policy upgrades (#13266)
  fix(ci): avoid empty pnpm caches from lockfile refresh (#13267)
  feat: accept a base64-encoded Cloud UI snippet (#13245)
  ci: use reserved AWS capacity for post-merge cloud verification (#13257)
  Fix Codex API key authentication in tests and runs (#13260)
  refactor: remove automatic productivity reviews (#13263)
  fix: continue conversations after confirmed remote runner stop (#13254)
  ci: cache Rust dependencies used by post-merge typecheck (#13259)
  fix(ui): restore task page archive shortcut (#13253)
  feat(ui): show one rolling runner activity per commentary group (#13255)
  fix(ui): hide retry countdown after execution starts (#13258)
  test: isolate native controller takeover fixture (#13212)
This commit is contained in:
Dotta committed 2026-09-11 17:15:49 -05:00
commit 24a7ab22f6
201 files changed
+62608 -3880

No files matched your search

+1 -1
View File
@@ -150,7 +150,7 @@ Paperclip’s core identity is a **control plane for autonomous AI companies**,
Work is not done until the user can see the result: file, document, preview link, screenshot, plan, or PR.
6. **Execution visibility without log worship**
Active runs, recovery issues, productivity review states, blockers, and work products should be first-class surfaces. Raw transcripts are available when needed, but they are not the primary product surface.
Active runs, recovery issues, blockers, and work products should be first-class surfaces. Raw transcripts are available when needed, but they are not the primary product surface.
7. **Local-first, cloud-ready**
The mental model should not change between local solo use and shared/private or public/cloud deployment.
+13
View File
@@ -1582,3 +1582,16 @@ action outcomes; do not replay tool calls or reset the failed incident's automat
retry budget. Existing pause, approval, budget, ownership, and dependency gates
remain in effect. See `doc/execution-semantics.md` for admission and stop-proof
requirements.
### Experimental task-bound email
AgentMail channel connections extend the experimental conversation/task pipeline
with explicit email publication. Each owned inbox/provider thread binds one task;
external email senders do not gain board authority. Incoming correspondence uses
the assigned agent's normal execution controls. Internal task activity never
implicitly sends email. New outgoing conversations create child tasks and durable
send intents before provider contact. The board directs email work through the
normal task conversation; rich email cards show the correspondence and delivery
outcomes without a separate email composer. See
[AgentMail connections](connections/AGENTMAIL.md) for setup, transports, recovery,
authorization, and the API/CLI contract.
@@ -220,6 +220,19 @@ Examples:
- a materialization failure records the failed phase and next retry time rather
than silently dropping the side effect.
## Policy upgrades
The policy version on an assessment is audit metadata. New runs use the current
rules. A version change alone does not reassess an old run, change task status,
or ask a person to review completion. New evidence and explicit status changes
still use the existing reconciliation paths.
Reconciliation also withdraws pending review cards created solely by the old
policy-version check. It restores the previous status only if that exact decision
and status version are still current and no other review gate is pending. A later
user or agent decision takes precedence. The old assessments and decisions remain
in the audit history; cleanup does not accept or reject the agent's work.
## Diagnosing an unexpected status
Start with the terminal heartbeat run and inspect:
+77
View File
@@ -123,6 +123,36 @@ registry checks can be rerun without deploying or changing mutable npm channels.
When reverting this workflow, restore the master push trigger in
`docker-cloud.yml` in the same change so master images continue to build.
## Reserved AWS verification capacity
`AWS_POST_MERGE_CI_ENABLED=true` routes cloud source verification, artifact
waiting, readiness signals, and exact-master migrator preparation to the
`paperclip-post-merge` runner group. The separate Fleet label is
`runs-on/fleet=paperclip-post-merge-x64/env=public-ci`. Its 36 reserved slots use
the same four-vCPU, 16-GiB machines as approved PR jobs. PR capacity is reduced
to 64; image capacity stays at eight. The total ceiling remains 108 runners.
This keeps PR bursts from consuming every post-merge verification slot.
Every selector checks the canonical repository name and ID, master ref, and a
push or manual event. Reusable verification also requires `inputs.ref` to equal
that event's `github.sha`. The migrator route requires `cloud-migrator` and
`inputs.source_ref == github.sha`. Branch/tag refs, PR events, arbitrary preview
sources, and missing or disabled switches use GitHub-hosted runners. If another
merge lands before a migrator dispatch resolves master, the older source uses
GitHub-hosted runners too. npm publication always remains GitHub-hosted to keep
its trusted-publisher identity.
Before enabling the switch, deploy the separate Fleet and restrict its GitHub
runner group to repository ID `1170821064` and these workflows at
`refs/heads/master`: `cloud-readiness.yml`, `cloud-artifacts.yml`,
`release-verify.yml`, `runner-chaos-evals.yml`, and `release.yml`. Do not authorize
PR-controlled workflow versions. PR placement retains its independent pinned
workflow and six-account author/actor allowlist.
Disable the switch and rerun the whole workflow to restore GitHub-hosted
placement. Assigned jobs keep their original runners. Readiness requirements,
source checks, and npm integrity checks are unchanged.
## AWS cloud build routing
`AWS_CLOUD_BUILDS_ENABLED=true` routes the Docker cloud job to the
@@ -145,3 +175,50 @@ workflow. Changing the variable does not migrate an already assigned job.
Check the Actions job's runner name and runner group to verify placement. Record
queue time, image verification completion, and `Cloud deployable v1` separately;
source verification and the migrator still run on GitHub-hosted runners.
### Typecheck Rust dependency cache
Source verification's typecheck job builds the native Runner binary through the
server's `prepare:runner-vendor` command. It restores and saves compiled Rust
dependencies only for canonical master pushes that verify the event's exact SHA.
The `release-typecheck-v1` cache is separate from Runner verification because
those jobs compile different profiles. The pinned toolchain is selected before
cache lookup. Workspace crates and installed cargo binaries are excluded, and
all typechecks still execute. A missing or invalidated cache triggers compilation.
### pnpm dependency store cache
The Refresh Lockfile workflow does not cache the pnpm store. Its resolution-only
command does not download packages and can save an empty default-branch cache
before full install jobs finish. Jobs that install dependencies retain caching.
After deploying this correction, remove any existing empty default-branch entry
for the current lockfile key. List cache IDs, branches, and archive sizes first:
```sh
gh api --paginate 'repos/paperclipai/paperclip/actions/caches?ref=refs/heads/master&key=node-cache-Linux-x64-pnpm-&per_page=100' \
--jq '.actions_caches[] | {id, ref, key, size_in_bytes}'
```
Match the key and upload size against the cache-creation job's logs. The
September 11 incident was cache ID `7559920987`, a 216-byte archive. This guarded
command deletes only that observed entry. It leaves a populated replacement or
an entry on another branch untouched, and does nothing if the old ID is absent:
```sh
bad_cache_id=7559920987
bad_cache_key=node-cache-Linux-x64-pnpm-c3096ecb02a34aaa9782baaadafcb731510e1dba10dd661618c3a2ee91e58fa5
entries="$(gh api --paginate --slurp 'repos/paperclipai/paperclip/actions/caches?ref=refs/heads/master&per_page=100')"
if printf '%s\n' "$entries" | jq -e --argjson id "$bad_cache_id" --arg key "$bad_cache_key" '
[.[].actions_caches[] | select(.id == $id)] |
length == 1 and .[0].ref == "refs/heads/master" and
.[0].key == $key and .[0].size_in_bytes == 216
' >/dev/null; then
gh api --method DELETE "repos/paperclipai/paperclip/actions/caches/$bad_cache_id"
fi
```
A subsequent master install can populate the missing entry. Check the saved
archive size and package reuse in install logs; a cache hit alone does not prove
that the entry contains dependencies.
+19
View File
@@ -10,6 +10,25 @@ application origin and is visible to every browser that receives the UI shell.
Do not include secrets or customer data. Restart the app after changing it.
Operators must review scripts and any required CSP changes before deployment.
## Base64 variant
Delivery pipelines that write env vars through provider APIs can sit behind
web application firewalls that reject values containing raw script markup.
`PAPERCLIP_CLOUD_UI_SNIPPET_B64` carries the same snippet through them as
standard base64 of the UTF-8 HTML:
```sh
PAPERCLIP_CLOUD_UI_SNIPPET_B64="$(base64 < snippet.html)"
```
Whitespace and line wrapping in the value are tolerated. A value that is not
canonical padded base64 of UTF-8 text, or that decodes to blank, is ignored —
if the widget does not appear, check that the value round-trips through
`base64 -d`. A present `PAPERCLIP_CLOUD_UI_SNIPPET` always wins, blank
included: clearing the plain variable to blank disables injection even while
a base64 value is still deployed. Everything else about the snippet is
unchanged.
## Plain closed beta
Set the value to this standard embed, replacing `YOUR_CHAT_APP_ID` with the
@@ -0,0 +1,105 @@
# AgentMail Daytona verification — 2026-09-11
Worktree: `codex/agentmail`; isolated test drive at `http://localhost:3103`.
The original checkout remains untouched. Test mail used only the two previously
authorized inboxes, `pap15838-qa@agentmail.to` and
`attractiveforce961@agentmail.to`.
## Defects corrected
- AgentMail REST connections fell through the generic health-check branch into
local-stdio MCP validation. Both saved account credentials and inbox credentials
now validate against AgentMail's `/auth/me` API. Catalog refresh returns no MCP
tools, and invalid keys still produce a failed health result. The live Apps card
was inspected in the browser and showed Connected with no stdio error.
- Sandbox callback routing omitted task-email endpoints. It now allows assigned
inbox discovery, task-thread reads, delivery reads, and explicit sends. Server
company, inbox, task/run, and action-policy authorization remains in force.
Setup, credentials, reconnect, and operator delivery resolution stay denied.
- Shell-backed sandbox reads did not preserve ENOENT for a missing optional
Codex `auth.json`, causing cleanup to fail after a successful email send. Reads
now confirm absence in a searchable parent and return ENOENT; actual read and
transport failures still propagate. This lets existing auth copy-back treat
missing credentials as a no-op.
- Runtime instructions now document inbox discovery directly; the agent otherwise
spent time guessing that endpoint when initiating a new conversation.
## Live observations
The board used the ordinary task composer in
[AGE-10](http://localhost:3103/AGE/issues/AGE-10) to request a test email.
The agent executed the real Codex CLI in Daytona, used the sandbox callback
bridge to discover its assigned inbox and queue the send, and created
[AGE-11](http://localhost:3103/AGE/issues/AGE-11) as an email child task.
- Provider sandbox: `c2f176ca-dbde-41a6-995d-aefa4689e4c5`.
- Runtime verified by the agent: Linux, x86_64; hostname matched the sandbox.
- Run: `cd7b934a-5555-4361-b0e5-b8106c1510ce`.
- Publication: `d5b7bf41-a583-4f9f-90c0-4d21680e39c2`, **Delivered**.
- Subject: `[Paperclip E2E] Daytona sandbox — Sep 11`.
- Provider key remained in Paperclip's vault. The sandbox used its injected
Paperclip run credential, and the model key was separately vaulted.
The first fixture launches exposed an unavailable default ACP executable and a
host `service_tier` setting incompatible with the fleet image's Codex CLI. The
QA fixture explicitly selects the CLI engine and an isolated Codex home. Earlier
failed launches remain in AGE-9. The outbound send above completed, but its run
then failed during missing-auth-file cleanup; the cleanup fix is verified
separately below rather than rewriting that history.
## Cleanup verification
A fresh Daytona run in [AGE-12](http://localhost:3103/AGE/issues/AGE-12)
read the existing publication, confirmed Delivered, recorded its Linux hostname,
and completed successfully without sending another email.
- Run: `39e77902-714e-455f-90d8-8709f2d13762`, **Succeeded**.
- Sandbox: `d50c6979-de6e-4a0c-ac18-bd616a39ee1f`.
- Cleanup log: “no sandbox credential to copy back (absent auth.json); host
credential kept.” The environment lease reached Released.
## Automated checks
- 20 durable email pipeline tests passed, including health checks for account and
inbox credentials, catalog discovery, and invalid credentials.
- 56 sandbox callback bridge tests passed, including the four email routes and
denial of email administration routes.
- 28 command-managed runtime tests passed, including the missing-file contract
and propagation of real read failures.
- 4 capability inventory tests passed. Regenerated both capability indexes for
the new task-email runtime documentation and updated the expected row count.
- Server typecheck, server build, adapter-utils build, and whitespace checks passed.
## Inbound round trip
After Chrome access recovered, sent a new authorized test email from the other
inbox through AgentMail Console. WebSocket intake created
[AGE-13](http://localhost:3103/AGE/issues/AGE-13), assigned Email QA, and started
the agent in a fresh Daytona sandbox. The agent read the bound thread, explicitly
replied once, checked delivery, and marked the task Done.
- Run: `76be255b-df2e-4479-8c62-f4506f039132`, **Succeeded**.
- Sandbox/verified Linux hostname: `7bb660fa-3cff-4b26-9e10-68c884be21bb`.
- Reply publication: `efdd704c-afd3-4025-ab48-24fab6c97333`, **Delivered**.
- Incoming comment persisted at `19:05:14.750Z`; run started at
`19:05:14.920Z` (170 ms later). This interval excludes provider delivery and
does not measure model startup. The run finished at `19:06:09.481Z`.
- Exactly one incoming and one outgoing email comment, plus an internal summary.
- Visually verified the exact acknowledgement in
[the other AgentMail inbox](https://console.agentmail.to/dashboard/inboxes/attractiveforce961@agentmail.to?thread=805fd7f1-26c2-414a-b139-5fb65f490f50),
with matching reply message ID and original-message reference.
## Test cleanup
Restored Email QA's original local adapter configuration. Removed the temporary
Daytona environments, all six sandbox instances created by this test, and the
temporary vaulted Daytona/model credentials. Provider inboxes, saved AgentMail
credentials, task history, and run evidence remain available.
## Qualification limits
This run exercises the Codex CLI sandbox adapter. The native runner `task_email`
path is covered deterministically, but was not separately live-qualified in Daytona.
The Daytona inbound round trip used WebSocket intake. Earlier local-agent
WebSocket and signed-webhook qualification is documented in
[the main verification report](AGENTMAIL-VERIFICATION.md).
+122
View File
@@ -0,0 +1,122 @@
# AgentMail verification — 2026-09-11
## Environment
Worktree: `codex/agentmail`. The original checkout and its merge conflicts were
preserved. Live checks used the isolated AgentMail Test Drive company at
`http://localhost:3103`, with the experimental connections feature enabled.
Only these user-authorized inboxes exchanged test mail:
- Paperclip: `pap15838-qa@agentmail.to`, assigned to Email QA.
- Other end: `attractiveforce961@agentmail.to`, inspected in AgentMail Console.
## Live browser results
| Journey | Observed result |
| --- | --- |
| Connect from the Apps catalog | Saved personal human access, selected agent access, and a vaulted key through the real UI. |
| Give an agent an address | Used Permissions → three-step wizard → existing scoped inbox. The selected agent, review warnings, and connection persisted. |
| Trust controls | Saved Low-trust review with a root-task boundary, verified the missing-sandbox prerequisite, then explicitly restored Standard for this local QA agent. |
| Live receiving | Inbound correspondence created AGE-6. The agent explicitly replied once; the reply appeared in AgentMail Console and Paperclip recorded Delivered. |
| Signed webhook | Registered an inbox-scoped webhook. Actual signed POSTs returned 204. AGE-7 received its email, the agent replied once, and both consoles showed the exchange. |
| Reply to a completed conversation | New mail reused the same task and reopened it. |
| Restart catch-up | Sent another reply while the server health endpoint was unreachable. Startup imported it into AGE-7, woke the agent, and sent one acknowledgement in the same thread. |
| Agent-initiated new conversation | A board request in AGE-7 caused the agent to create AGE-8 with `parentId` pointing to AGE-7. One email was sent and marked Delivered; it appeared as a separate thread in AgentMail Console. |
| Internal publication boundary | Internal summaries and the outbound-only child task's “No reply sent” response produced no additional emails. |
| Cleanup | Restored WebSocket mode, removed Paperclip's test webhook, stopped the webhook-only proxy/tunnel, and removed the temporary public URL from the isolated configuration. Inbox history and vaulted test credentials remain inspectable. |
Useful live pages:
- [Saved connection permissions](http://localhost:3103/AGE/apps/78dd5c23-f60f-42ca-b30a-6f0c701b38d3/permissions)
- [Inbox settings](http://localhost:3103/AGE/apps/chat/7cdf17d6-465e-4eef-8858-2b545be64b3a/settings)
- [Inbound conversation and restart recovery: AGE-7](http://localhost:3103/AGE/issues/AGE-7)
- [Agent-created email child: AGE-8](http://localhost:3103/AGE/issues/AGE-8)
- [Other inbox in AgentMail Console](https://console.agentmail.to/dashboard/inboxes/attractiveforce961@agentmail.to)
## Timing
These are individual observations from `email.received` audit records, not a
load test or latency guarantee. Admission-to-wakeup includes durable processing
and heartbeat admission; it excludes provider delivery and subsequent model
startup/generation.
| Check | Admission to wakeup |
| --- | ---: |
| Live inbound, AGE-6 | 409 ms |
| Signed webhook, AGE-7 | 421 ms |
| Startup catch-up, AGE-7 | 585 ms |
The clean webhook run was created at `18:10:53.471Z`, started at
`18:10:53.512Z`, sent its reply at approximately `18:11:40Z`, and finished at
`18:12:05.225Z`. Model work is separate from the sub-second admission measurement.
## Fixes found by testing
- Personal credential access displayed as organization access in the generic
connection panel. AgentMail now displays the actual saved grants and installs.
- Low-trust permissions used the wrong mutation route; Standard omitted rather
than cleared the previous boundary. Both are fixed and covered by regressions.
- Email task recovery incorrectly entered restricted chat replay. Normal email
work now uses normal task recovery while retaining execution controls.
- A send/read-only key could not register a webhook. Setup now explains the
required inbox-scoped webhook permissions. A failed switch leaves the live
connection active. The user authorized a replacement scoped key for the live
webhook test.
- Graceful shutdown retained the socket lease until its crash timeout. Shutdown
now releases only this worker's socket tokens; the ownership test verifies
immediate takeover by a second worker. The final live restart became ready at
`18:30:10Z` and completed a mail check at `18:30:14Z`, with no connection error.
- A path-like attachment filename could produce a stored object key that the
storage reader rejected. Imported filenames now remove path traversal segments.
The regression covers bounded, deduplicated intake, reading stored bytes,
task-scoped attachment references, and rejecting bytes changed after queueing.
- The initial QA agent attempted to install the released CLI for an unreleased
feature. The test agent now uses the local HTTP API. Runtime documentation also
describes the direct HTTP fallback.
- The first QA instruction to leave work open omitted a valid task disposition,
triggering existing recovery controls after a successful send. Corrected QA
instructions explicitly set the requested disposition. Clean subsequent runs
completed successfully; those earlier diagnostic tasks remain inspectable.
## Automated verification
- API/provider and durable-pipeline tests: 32 passed, including signature checks,
deduplication, callback-before-response, uncertain-send handling, inbox/company
isolation, credentials, low-trust placement, and socket ownership/shutdown.
- OpenAPI contract checks passed (8 tests); the final combined run passed all 40.
- Deterministic Playwright setup and task-conversation coverage includes actual
trust-permission persistence, rich email cards, and Bcc details. Following the
board UX revision, email controls were removed and instructions use the normal
task composer. Its provider responses are mocked; it is separate from the live
browser results above.
- Trust UI tests passed (10 tests).
- Catalog regression and damaged-runner-history recovery regression passed.
- Repository typecheck and build passed; changed-package checks were repeated
after subsequent fixes. Token gates and whitespace checks passed.
- Full repository Vitest run did **not** pass. The general server group finished
with 10,584 passing tests, five failing tests, and one database-startup suite
failure. Its five individual failures subsequently passed in focused reruns
(email recovery/trust, gallery count, plugin wait, and damaged runner history).
This broad run began before the final fixes; it is not a final green result.
- Additional broad workspace and serialized-route groups encountered database
startup, hook, and adapter timeouts. The UI group had 5,923 passing tests and
five failures; rerunning its two affected files passed all 73 tests. Shared
contracts passed 727 tests and the skills catalog passed 20. Remaining broad
groups have not been rerun to completion, so this is not a PR-ready all-green
qualification.
## Limits
The account was at its inbox limit, so live setup attached an existing inbox.
Programmatic inbox creation and custom domains were not live-qualified.
Attachment transfer, invalid signatures, cross-company denial, cancellation,
duplicate callbacks, and expired idempotency windows are checked deterministically
rather than against the live provider. Low-trust execution was not run in a real sandbox; setup correctly
rejected the isolated test drive's missing sandbox runtime.
One restart-test acknowledgement arrived in the other inbox while Paperclip's
status remained Sent because its delivery receipt was missed during socket
recovery. Sent records provider acceptance; Paperclip does not fabricate a
Delivered receipt or resend the message. The later independent outbound email
received and recorded its Delivered receipt normally.
+216
View File
@@ -0,0 +1,216 @@
# AgentMail email connections
AgentMail is an experimental **channel** connection. Enable experimental chat
connections, open Apps → AgentMail, select which humans and agents may use the
credential, then enter an API key. In the saved connection’s Permissions page,
choose **Give an agent an email address**. The three-step wizard selects an agent,
creates or attaches an address, and reviews the setup. Selecting an agent outside
the current allowed list adds that agent when setup completes. Every provider thread in that inbox has one Paperclip task. Subjects are
not identifiers. The same email delivered to two connected inboxes creates two
independent tasks.
Setup accepts an AgentMail API key or the saved company credential from another
AgentMail connection. Organization and pod keys create an inbox-scoped runtime
key. An existing inbox-scoped key can connect only its own inbox. Credentials
are vaulted and resolved by the server; they are not passed to agents. An inbox
can have only one non-archived Paperclip endpoint across the instance.
Verified custom domains are selectable after checking the API key. Complete DNS
setup in [AgentMail](https://docs.agentmail.to/custom-domains). Paperclip does not
register domains or manage DNS.
The setup and Permissions page warn that an unrestricted inbox can receive mail
from anyone. Configure sender allowlists in AgentMail; Paperclip does not manage
or verify them. AgentMail controls new-message and reply lists separately. The
wizard recommends Paperclip’s existing **Low-trust review** preset and lets the
operator configure a project or root-task boundary. Incoming tasks are placed
inside that boundary. Low-trust execution also requires isolated workspaces and an active sandbox
environment selected for the agent; setup rejects an unavailable runtime. New
inbound tasks request isolated execution. The trust preset itself does not
sandbox filesystem or network access. Standard agents remain selectable with a warning.
Removing the assigned agent’s saved-connection access or revoking its credential
grant stops receiving and sending. Connection creation saves the vaulted binding,
human grants, and agent access in one database transaction.
## Receiving and task lifecycle
WebSocket is the default and needs no public HTTP URL. The server authenticates
with an Authorization header, keeping the provider key out of the connection URL
([provider handshake](https://www.agentmail.to/docs/api-reference/websockets/websockets)). The service holds a
renewable database lease, subscribes to the connected inbox, and reconnects with
backoff. Webhook mode needs the configured public HTTPS webhook base URL. Setup
registers a Paperclip-owned webhook. The raw request body is verified using Svix
before the inbox is admitted to the shared durable delivery queue.
The API key needs inbox-scoped `webhook_create`, `webhook_read`, and
`webhook_delete` permissions in addition to mail access. AgentMail's
"Send & read mail" preset alone cannot register a webhook. A rejected
registration while switching from WebSocket leaves live receiving active.
Both transports deduplicate by inbox, event kind, and provider message ID. A
per-conversation worker lease serializes work; independent conversations can
proceed concurrently. Provider messages, comments, and attachment links preserve
the provider message identity. A reply to a completed task reopens it. A cancelled
task retains new mail but does not wake its agent. Provider-classified spam,
blocked and unauthenticated mail do not start automatic work. Recognized automatic
replies can be retained in an existing conversation but do not wake an agent or
create a new task.
Activation establishes the intake cutoff. Activation, reconnect, and periodic
maintenance scan paginated message metadata and fetch eligible messages using a
receipt-time checkpoint with a five-minute overlap. Metadata scans traverse all
pages because AgentMail sorts messages by the sender's timestamp: a newly
received message can have an old Date header. Message-ID deduplication makes
repeated scans safe. Earlier messages in a newly active thread are imported as
context without separate historical wakeups. There is no automatic historical
mailbox import and no assumption of WebSocket replay.
Incoming mail wakes the selected agent through its normal task execution path,
including its configured permissions and budget controls. The external sender
is recorded in the email envelope; an email address never grants Paperclip
membership or board authority.
## Explicit email actions
Internal comments, progress, final responses, approvals, and errors never send
email. Email endpoints have an explicit publication mode; shared automatic chat
publication paths exclude them. Sending email does not close a task.
The task displays the email envelope, extracted reply text, full text context,
attachments, and delivery outcomes. Use the normal task conversation to ask the
agent to send an email or reply. There is no separate email composer or mode
switch. The agent uses an explicit email action; task messages themselves are
not sent as email. Reply uses Reply-To when present, otherwise the sender;
reply-all must be requested.
Bcc is retained in the originating envelope but is not copied to reply inputs.
Remote email images are not rendered. Attachments use Paperclip's content-type,
size, company, and task bounds.
An agent must own the inbox, be assigned the source task, and supply the running
source task's `X-Paperclip-Run-Id` at acceptance. Board actions require company
write access. Configured action policies apply to both. Authority is checked
again when the durable send executes. A new conversation creates its child task
and immutable send intent in one transaction before contacting AgentMail.
All paths below are relative to `/api`:
| Operation | Path |
| --- | --- |
| Save credential and human/agent access | `POST /companies/:companyId/email/connections` |
| Inspect a saved credential | `POST /companies/:companyId/email/connections/:connectionId/inspect` |
| List authorized inboxes | `GET /companies/:companyId/email/inboxes` |
| Inspect setup credentials (connection manager) | `POST /companies/:companyId/email/inspect` |
| Create or attach an inbox (connection manager) | `POST /companies/:companyId/email/inboxes` |
| Pause, resume, disconnect | `POST /email/inboxes/:endpointId/control` |
| Replace credentials / receiving mode | `POST /email/inboxes/:endpointId/reconnect` |
| Start an email child task or reply | `POST /companies/:companyId/email/send` |
| Read the email context of a bound task | `GET /companies/:companyId/email/tasks/:issueId` |
| Read delivery outcome | `GET /companies/:companyId/email/deliveries/:publicationId` |
| Resolve an uncertain outcome (connection manager) | `POST /companies/:companyId/email/deliveries/:publicationId/resolve` |
A new send request:
```json
{
"endpointId": "<inbox-endpoint-uuid>",
"parentIssueId": "<current-task-uuid>",
"to": ["recipient@example.com"],
"cc": [],
"bcc": [],
"subject": "Question about the proposal",
"text": "Could you clarify the delivery date?",
"attachmentIds": [],
"idempotencyKey": "<new-request-uuid>"
}
```
A reply request uses `conversationId` and `replyToMessageId` from the bound task:
```json
{
"endpointId": "<inbox-endpoint-uuid>",
"conversationId": "<email-conversation-uuid>",
"replyToMessageId": "<provider-message-id>",
"replyAll": false,
"text": "Thanks, that answers the question.",
"attachmentIds": [],
"idempotencyKey": "<new-request-uuid>"
}
```
Native runners with an active, authorized inbox receive `agentmail_inboxes`,
`agentmail_read_thread`, `agentmail_send`, and `agentmail_delivery`. The system
also installs the AgentMail skill for those agents through the normal runtime
skill path. These tools supply run authority and work independently of the
optional generic runtime API rollout. Where enabled, `search_api` and `call_api`
also expose these operations. The CLI uses the same authenticated
operations and inherits the agent run ID:
```sh
paperclipai email inboxes
paperclipai email thread "$PAPERCLIP_TASK_ID"
paperclipai email send --file email-request.json
paperclipai email reply --file email-reply.json
paperclipai email delivery '<publication-uuid>'
```
A `202` response includes task, conversation, and publication IDs immediately.
The publication progresses through queued, sent, delivered, failed, or uncertain.
Delivery callbacks update that publication and do not create new correspondence.
Retries reuse the same immutable request and provider idempotency key. The worker
stops automatic retries after 23 hours, conservatively inside AgentMail's 24-hour
deduplication window. An uncertain receipt can be resolved by matching its
provider message ID and Paperclip publication header, or by an operator confirming
that it was not sent. The latter marks it failed; any resend is a new explicit
action. Do not change an idempotency key just because a request timed out.
## Disconnect and diagnostics
Reconnect preserves inbox and task identity. Pause stops intake and sending.
Disconnect archives the local endpoint and removes its credential bindings,
unreferenced vaulted credentials, and only the webhook/runtime key created by
Paperclip. It never deletes the provider inbox or task history. If a revoked key
prevents provider cleanup, local disconnection still completes and reports that
Paperclip's provider registrations need cleanup in AgentMail.
Connection settings show state, receiving mode, catch-up time and errors. Tasks
show publication failures and uncertain delivery resolution. Delivery admission,
message processing and agent wakeup are separate from provider delivery and model
startup; live latency measurements must distinguish those stages.
## Verification and live qualification
Deterministic coverage lives in `server/src/__tests__/agentmail-api.test.ts`,
`server/src/__tests__/email-channels.integration.test.ts`, and
`tests/e2e/agentmail.spec.ts`. It exercises real database transactions with a fake
provider, plus browser setup and explicit task email actions.
Before labeling an installation live-qualified, use a disposable inbox and an
approved test recipient. In each transport mode, receive a message, verify one
task and one wake, send an explicit reply, and verify provider threading and
delivery. Also disconnect/reconnect, interrupt receiving, and verify catch-up.
Record provider message IDs and timestamps without copying credentials. Compare
the durable delivery `received_at` with the wake request time separately from
provider transit time and model startup. Automated fixtures do not constitute
live provider qualification.
Provider references: [inboxes](https://docs.agentmail.to/inboxes),
[webhook verification](https://docs.agentmail.to/webhook-verification),
[idempotency](https://docs.agentmail.to/idempotency),
[message listing](https://docs.agentmail.to/api-reference/inboxes/messages/list),
[reply API](https://docs.agentmail.to/api-reference/inboxes/messages/reply).
### Sandbox execution
AgentMail runs in the Paperclip control plane using its vaulted credentials. It
is a REST connection, not a local-stdio MCP server. The connection health check
validates the key against AgentMail; it does not launch a local command or discover
MCP tools.
Agents in Daytona and other sandbox environments use the same task email actions.
The sandbox callback bridge allows inbox discovery, bound-thread reads, delivery
reads, and explicit sends. The controller enforces company, inbox, task/run, and
action-policy checks. Mailbox setup, credential inspection, reconnect, and manual
delivery resolution remain outside that sandbox API surface. Native runners use
the assigned AgentMail tools through their run-bound tool channel. Neither path exposes the
AgentMail provider key to the sandbox.
+127 -1
View File
@@ -42,6 +42,7 @@ for the P1/P2/P3 boundary and the D7 standing rule.
- [Secret storage and lifecycle](#secret-storage-and-lifecycle)
- [Current access defaults](#current-default-access-policy)
- [Golden-path agent tutorial](#golden-path-agent-tutorial)
- [Connection UX and user journeys](#connection-ux-and-user-journeys)
- [AppDefinition field reference](#appdefinition-field-reference)
- [Troubleshooting](#troubleshooting-and-failure-classification)
- [Definition of done](#definition-of-done)
@@ -425,6 +426,122 @@ connection work or enforce a real tenant boundary. Follow these rules:
label is not enforcement. The provider, gateway, wrapper, or managed header/
query projection must enforce the boundary.
#### Connector-provided skills and tools
Connectors may contribute bundled skills with optional native tools. Keep provider-specific
instructions out of the universal Paperclip skill and provider-specific tools
out of the universal runner catalog. Use the trusted connector contribution
registry in `server/src/services/connector-runtime.ts`; AgentMail is the first
consumer. This registry describes bundled server implementations, not executable
code or skill URLs supplied by a credential or external message.
For each contribution, declare its connector key, bundled skill, namespaced tool
definitions, resource-assignment resolver, and execution handler. Use names such
as `agentmail_send` rather than extending core tools with provider-specific
branches. Existing MCP connectors continue to use their normal MCP tool catalog;
they do not need a duplicate native wrapper just to supply a skill.
**Resolve eligibility from current assignments and access.** An AgentMail account
credential alone does not give an agent email capabilities. An active inbox
assigned to that agent does, provided both the inbox connection and saved
credential access remain authorized and the experimental chat-connector flag is
on. Other connectors must define an equally concrete assignment rule. Keep every
lookup company-scoped. Revoked grants, disabled connections, removed assignments,
and experimental gates must remove the contribution. Fail closed on lookup errors.
**Install skills transparently through the existing runtime skill path.** Merge
system-managed contributions with the agent's chosen skills for each run, without
writing them into its saved skill preferences. Deduplicate multiple resources
from the same connector into one skill. Include only authorized resource context,
never provider secrets; treat resource values as data. Supply the short skill
description for discovery and keep detailed instructions in the skill. The same
resolved set must reach local CLI adapters, sandbox adapters, and native runners.
Adapters with isolated skill delivery receive the bundle. Adapters that install
into shared user directories receive the same assigned skill in the run prompt,
including resumed turns, without writing connector files into that directory.
Manual skill-sync operations must also exclude automatic connector bundles.
The agent Skills page should identify automatic contributions and explain that
assignment controls them; they are not independently enabled/disabled there.
**Bind tools to the same resolved skill assignment.** Native sessions advertise
only contributions present in their pinned runtime skill bundle. Include skill
content, resource assignments, and tool revisions in session compatibility so a
changed assignment cannot reuse stale declarations. Revalidate live assignment,
company/task/run authority, and configured action policy on every execution.
Removing a tool from discovery alone is not revocation enforcement. Retained
provider sessions and previously issued calls must fail after access is revoked.
**Avoid shared runtime contamination.** Do not install assignment-specific skills
into a company-wide or user-wide runtime home. Use immutable skill bundles and
scoped runtime directories. Codex CLI connector runs use a separate home per
agent and connector-skill revision, seeded from the selected model credential
home. Disconnecting returns to a runtime without those skills; another agent must
never inherit them. Preserve explicit model identity and normal session recovery.
Required tests cover no assignment, credential access without a resource,
authorized assignment, multiple resources with one skill, cross-company access,
revocation during a retained run, disabled flags/connections, and reassignment.
Verify skill installation and removal in both CLI/sandbox and native execution,
including tool discovery, runtime cache changes, and absence of provider secrets.
Exercise an actual connector operation through the contributed tool, not just
its declaration. Record which runtime paths were tested live versus deterministically.
#### Connection UX and user journeys
Design the whole journey, from finding the app to doing useful work with an
agent. A successful credential exchange is only one step. Describe who the
user is, where they start, what they want to accomplish, and where they will
see the result. Walk through first use, returning use, and recovery from a
failed action. For messaging connections, cover both agent-initiated work and
incoming messages that start or continue work.
**Separate connecting from assigning an agent a resource.** First configure
who can use the connection and authenticate with the provider. If the feature
also assigns a resource to a specific agent, offer a second wizard from the
connection's Permissions view after the connection is saved. Give its entry
point a prominent, concrete action name. For example, AgentMail uses “Give an
agent an email address,” followed by Agent → Email address → Review. Reuse the
saved credential; do not ask for the API key again. Use the existing numbered
step pattern, sensible defaults, Back and Cancel, and a clear completion state.
Do not add a second wizard when there is no separate assignment to configure.
Let the operator search eligible company agents, including agents not yet on
the connection's allowed list. When assigning a resource also grants connection
access, make that consequence clear and persist the grant through the existing
access machinery. Respect the operator's authority to grant access, and show
the selected agent's avatar and name.
**Use the minimum text needed to make the next action clear.** Prefer familiar
controls and precise labels over explanatory paragraphs. Remove repeated
headings, redundant access summaries, implementation details, and reassurance
that does not help the user decide or act. Keep necessary warnings, meaningful
consequences, and actionable errors. Put optional expert settings under a
collapsed Advanced disclosure. Link to provider-owned administration, such as
AgentMail allowlists, rather than rebuilding it in Paperclip.
**Keep ongoing interactions in Paperclip tasks.** Connections are where users
set up access and configuration; tasks are where they work with agents. Design
what happens after setup: how an agent invokes the connection, where incoming
work lands, how follow-ups stay associated with that work, and how users see
success or recover from failure. Avoid introducing a separate mailbox or
provider dashboard as the primary interaction surface.
Use rich cards in the task feed when they make external activity easier to
understand. An email card, for example, can show the sender, recipients, body,
attachments, and delivery state. Keep external activity distinguishable from
internal discussion; a task comment or agent progress update must not imply
that an external action occurred. Reuse existing task-feed components and
preserve one visible record per external event.
**Make interactive Storybooks for setup and actual use.** Include the catalog
card, access and credential steps, any agent-resource wizard, and the task
journeys after setup. Provide a clickable walkthrough plus focused stories for
important steps, loading, errors, and recovery. Use realistic fixtures and
clearly label simulated actions. Reuse production components as implementation
lands, and replace obsolete stories so the examples describe the current
experience. Storybooks support design review and deterministic interaction
tests; they do not replace a real-provider browser test.
### Phase 4: Add official branding before exposing the app
Every store-visible provider needs an official local mark. A letter tile is
@@ -689,7 +806,11 @@ At minimum, add or update tests in these layers:
declared.
- Finish setup resumes the exact draft using `resumeConnectionId`.
- Optional customer OAuth details stay folded when automatic OAuth exists.
- Setup success leads to the connection's Test page.
- Setup success leads to the connection's Test page, or to Permissions when a
separate agent-resource assignment is the next step. Follow the
[connection UX guidance](#connection-ux-and-user-journeys).
- Interactive Storybooks cover setup and ongoing task interactions, including
relevant failure states; the walkthrough matches the implemented journey.
- Missing images fall back at runtime, while manifest acceptance still fails
missing branding.
@@ -1213,6 +1334,11 @@ connection actually enables it.
The wizard path comes from auth mode and transport:
These paths describe authentication and provisioning. Apply the
[connection UX guidance](#connection-ux-and-user-journeys) to the user-facing
sequence: choose access before authentication, then configure any per-agent
resource through a separate wizard on the saved connection.
| Auth mode | Operator path | Stored result |
| --- | --- | --- |
| OAuth | Gallery card -> Connect -> vendor consent -> callback -> configure filters -> health/catalog -> access defaults. | OAuth token material in `company_secrets`; connection metadata redacted. |
-1
View File
@@ -130,7 +130,6 @@ Grouped by rough domain area. One line each; variants column is props-based wher
| `ExternalObjectStatusIcon.tsx` / `ExternalObjectStatusSummary.tsx` / `ExternalObjectPill.tsx` | External-object (linked PR/doc/etc.) status glyph, rollup summary, and inline pill — a third, deliberately separate status-presentation family |
| `BlockedReasonChip.tsx` | Chip explaining why a task is blocked |
| `SourceTrustBadge.tsx` / `SourceResolvedFoldBadge.tsx` / `SourceResolvedFoldCallout.tsx` | Trust/fold badges for external content sources |
| `ProductivityReviewBadge.tsx` | Review-status badge |
**KNOWN-DUPLICATES.md lead verified:** StatusIcon / inline-mention chips / task chips are intentionally three separate systems (StatusIcon+StatusGlyph = task status glyph family; `ExternalObjectStatusIcon`/`Pill`/`Summary` = a second, external-object-specific family; mention chips in `lib/mention-chips.ts` + markdown CSS = a third, generic "chip in prose" family). **Documented here per instruction, not merged.**
+19 -8
View File
@@ -499,6 +499,12 @@ Monitor policy lives under `executionPolicy.monitor` and includes:
Monitors are not recurring intervals. When a monitor fires, Paperclip clears the scheduled monitor and queues an `issue_monitor_due` wake for the assignee. If the external service is still pending, the assignee must explicitly re-arm the monitor with a new `nextCheckAt`. If the issue moves to `done`, `cancelled`, an invalid status, or a human/unassigned owner, the monitor is cleared.
The task's waiting banner and composer countdown also display automatic retries
while their run is `scheduled_retry`. Once a retry is `queued` or `running`, its
retained `scheduledRetryAt` is historical and must not produce a waiting or overdue
warning. A separately scheduled monitor remains visible. Completed and cancelled
tasks hide both waiting surfaces even if a stale schedule remains in the response.
Because `serviceName` and `notes` remain visible in issue activity and wake context, operators should keep them short and non-secret. Put enough context for the assignee to know what to inspect, but do not include signed URLs, bearer tokens, customer secrets, tenant-private identifiers, or provider links with embedded credentials.
Monitor bounds are enforced. Paperclip rejects attempts to re-arm a monitor whose `timeoutAt` or `maxAttempts` is already exhausted. When a scheduled monitor reaches an exhausted bound at trigger time, Paperclip clears it and follows `recoveryPolicy`: `wake_owner` queues a bounded recovery wake for the assignee, `create_recovery_issue` opens visible issue-backed recovery work, and `escalate_to_board` records a board-visible escalation comment/activity.
@@ -595,15 +601,16 @@ Automatic retries that can continue source work use the agent's configured model
Startup recovery and periodic recovery are different from normal wakeup delivery.
On startup and on the periodic recovery loop, Paperclip now does five things in sequence:
On startup and on the periodic recovery loop, Paperclip performs the following recovery passes:
1. reap orphaned `running` runs
2. resume persisted `queued` runs
3. reconcile stranded assigned work
4. scan silent active runs only for source-aware terminal folding and legacy cleanup; API reads classify ordinary output silence for the board UI
5. reconcile productivity reviews
The stranded-work pass closes the gap where issue state survives a crash but the wake/run path does not. The silent-run scan covers the separate case where a live process exists but has stopped producing observable output. The productivity-review pass is later and separate; it reviews unusual progression patterns on assigned source issues, not stale run handles after a source issue already has a valid disposition.
The stranded-work pass closes the gap where issue state survives a crash but the wake/run path does not. The silent-run scan covers the separate case where a live process exists but has stopped producing observable output.
Automatic productivity reviews are retired. Run counts, missing comments, and elapsed task time do not create review tasks or impose continuation holds. Bounded continuation, provider recovery, budget limits, explicit blockers, and normal review/approval stages remain in force. Existing productivity-review tasks, comments, assignments, and dependencies remain unchanged and readable; their historical origins still identify them as recovery work for recursion suppression.
### Issue-thread interaction resolution
@@ -777,7 +784,7 @@ Do not fold a run only because it is quiet. Keep the informational signal visibl
In the normal non-terminal case, critical silence remains a UI signal and does not block the source issue. In the source-resolved case, a completed source issue does not acquire a new review or blocker merely because an old run handle stayed active. Only real unresolved work should block work.
This is distinct from productivity review. Productivity review asks whether an assigned source issue has unusual progression patterns, such as no-comment terminal-run streaks, long active duration, or high churn. Source-resolved watchdog folding asks whether a stale active-run signal outlived a source issue that already reached a valid terminal disposition. One does not substitute for the other.
Source-resolved watchdog folding concerns stale active-run bookkeeping after a valid terminal disposition. It does not infer productivity from run counts, comment frequency, or elapsed task time.
Detached process cleanup is operational hygiene, not source issue liveness. Cleanup should be best-effort and auditable. If cleanup fails but the source issue is already terminal with same-run durable evidence, Paperclip should preserve the cleanup failure on the run/watchdog audit trail and route only the cleanup concern to bounded recovery when a real owner/action remains.
@@ -831,7 +838,7 @@ Shutdown, process loss, and provider failure use the existing durable failure re
Real gates still apply: company and task ownership, active provider ownership, budget limits, agent availability, dependencies, pending approval/review paths, and explicit pause holds. Native runner reattachment and finalization retain their existing ownership protocol. Process, HTTP, and gateway adapters retain their recovery rules because invoking those adapters can itself repeat an external action rather than start a conversation turn.
An operator Stop still waits for local provider termination. Stop alone never promotes deferred comments or starts an automatic continuation. Once stopped, the next explicit wake adopts pending comment IDs in order through the existing queue. A compatible saved ACP session can resume, and an unavailable or incompatible session can start fresh with the full task context. Run credentials and scratch paths remain scoped to the new run. A subtree pause requires Resume; a message does not bypass it.
An operator Stop waits for provider termination. Remote sandbox providers may return a stopped/deleted receipt after their control-plane operation completes. Paperclip binds that receipt to the company, run, and exact lease; successful file cleanup, a terminal run row, or an in-sandbox shutdown event is not sufficient. Legacy conversational runs receive their cancellation acknowledgement after all remote leases have confirmed termination. Stop alone never creates a continuation. A user message queued during remote cleanup is reconsidered when the provider confirms termination; it still passes normal admission and adopts pending comment IDs in order. Once stopped, the next explicit wake uses the same queue. A compatible saved ACP session can resume, and an unavailable or incompatible session can start fresh with the full task context. Run credentials and scratch paths remain scoped to the new run. A subtree pause requires Resume; a message does not bypass it.
Historical legacy interruption holds for conversational adapters no longer block new messages or Resume. Classification uses the run’s saved adapter invocation or continuation policy, never the agent’s current adapter settings. Missing historical adapter evidence retains the hold. A terminal row with a live predecessor process or unreleased environment lease still blocks actual admission and Resume. Retry scheduling can happen before cleanup, but grants no execution authority. Recovery folds their obsolete no-replay bookkeeping without changing task ownership, status, or automatically waking old work. The audit trail remains readable. Native integrity and ownership holds, and non-conversational adapter holds, remain enforced.
@@ -878,9 +885,13 @@ request, task history, completed work, and the interruption notice. It receives
no instruction to repeat old tool calls. Later messages cannot reset the old
incident's retry budget or create another automatic replacement for it.
The initial native admission path verifies local process identities. Missing
process identity or remote ownership without a target-aware stop proof remains a
hold; a terminal database status or a PID check on the wrong host is insufficient.
Native admission verifies local process identities for local runs. Remote runs
instead require a provider termination receipt for every lease, with successful
cleanup and no active ownership. This applies to both per-turn and warm native
runners. A stop receipt retires only the settled cleanup owner for that exact company, run, provider, and sandbox resource, without changing its checkpoint or recorded action outcomes. Independent remote sandboxes have separate cleanup gates, including when one run owns multiple sandboxes. Successful pending-cleanup retries persist the same receipt and reconsider deferred user messages; a delivery failure never reverts successful provider cleanup. A failed checkpoint does not prevent destruction of a terminal run's isolated sandbox; busy ownership still prevents it.
Missing receipts and failed cleanup retain the hold. Older providers that return
no receipt remain supported but cannot authorize remote continuation. A terminal
database status or a PID check on the wrong host is insufficient.
No historical task is automatically awakened by this change.
### Explicit Recovery Action