Files
PaperClipAI/doc/connections/AGENTMAIL-DAYTONA-VERIFICATION.md
DottaandPaperclip 2083bf6f9a feat(connections): add AgentMail inboxes and email tasks (#13256)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections give agents controlled access to external services.
> - Experimental channels already map conversations to tasks and durable
work queues.
> - Email needs inbox ownership, recipient envelopes, delivery records,
and explicit sends.
> - This pull request adds AgentMail to that infrastructure and keeps
the provider key in the server vault.
> - Agents can receive and send email from local or sandbox execution
while the board follows each conversation in its task.

## Linked Issues or Issue Description

**Problem or motivation**

Agents need dedicated email addresses. Incoming email should become
assigned work. Internal task comments and progress must never become
outgoing email by accident.

**Proposed solution**

Add experimental AgentMail connections, an inbox assignment wizard,
durable email intake and publication, task email cards, and
authenticated API, CLI, and native runtime actions. Agents use Paperclip
credentials to request sends. Paperclip owns the provider key and
enforces access and task authority.

**Alternatives considered**

A general mailbox MCP connector does not provide durable task binding or
publication boundaries. A separate mailbox application duplicates task
collaboration. The board instead directs the agent through the normal
task conversation.

**Roadmap alignment**

This extends the existing experimental connections and task
infrastructure. Product scope and interaction design were reviewed with
the maintainer. Related connection authority work: #11831 and #11818.
The duplicate search found no competing task-based AgentMail
integration.

## What Changed

- Add AgentMail catalog data, shared contracts, company-scoped email
records, and an additive migration.
- Add vaulted setup, inbox assignment, access grants, trust guidance,
and provider-side allowlist guidance.
- Support WebSocket and signed-webhook intake through a shared durable
pipeline, deduplication, catch-up, and task wakeups.
- Queue explicit new conversations and replies with immutable send
intents, idempotency, delivery state, and uncertain-send resolution.
- Show inbound and outbound email cards in normal task conversations.
Keep internal messages internal.
- Add task-scoped CLI actions and the sandbox callback routes required
for Daytona execution.
- Provide a dedicated AgentMail skill automatically only to agents with
active authorized inbox assignments. Keep email instructions out of the
universal Paperclip skill.
- Advertise connector-owned `agentmail_inboxes`,
`agentmail_read_thread`, `agentmail_send`, and `agentmail_delivery`
tools only in eligible native sessions. Recheck live authority on
execution.
- Isolate Codex CLI connector skills by agent and skill revision.
Deliver the assigned skill in the run prompt for adapters that use
shared skill directories, including resumed turns. Keep automatic skills
out of manual persistent sync. Show them as read-only and document the
pattern in the connector playbook.
- Fix AgentMail health checks that entered local-stdio validation and
optional missing Codex credential cleanup in sandboxes.
- Add API, pipeline, authorization, sandbox, browser, and Storybook
coverage.

## Verification

- Live AgentMail testing covered WebSocket intake, signed webhooks,
restart catch-up, and a full receive → task → Daytona Codex CLI →
explicit reply → Delivered round trip. The reply was verified in the
other inbox. The normal task composer also initiated an outgoing email
child task.
- The connector-skill change was verified in the browser: AgentMail
appears once as an automatic, read-only skill with its assigned address.
Disabling experimental chat connections removes it; re-enabling restores
it. A regression test covers assignment data arriving after library
data.
- Connector regression coverage passed 178 runtime utility, email
integration, skill-route, and heartbeat tests. All 17 Codex execution
tests passed, including per-agent skill isolation, model identity,
revision changes, removal, and prompt delivery without shared skill
files.
- After rebasing onto master, all 44 focused email, heartbeat, and
native-authority tests passed. All 313 native-session executor tests
passed. The UI regression suite passed all 3 tests. These test sets
overlap earlier focused runs.
- Full workspace typecheck and build passed after the rebase. Token
gates passed. Earlier focused Playwright task/setup coverage and the
Storybook build also passed.
- Native connector tool execution uses deterministic integration tests.
Live Daytona qualification used the Codex CLI adapter; the new
shared-home prompt fallback has deterministic coverage.
- The full repository suite is run by CI. The earlier unsharded local
full-suite attempt was stopped after the equivalent CI suites passed and
is not reported as a completed local run. Greptile reviewed
`7e57dc267a8446d3c906e3cc5b8abc94fb8860eb` at 5/5 with no unresolved
threads. All server, workspace, serialized server, and browser suites
passed in CI. The build job hit a five-second timeout in a runner
transport test; both variants and the full 80-test file passed locally
with unchanged timeouts. The build passed on retry on the same commit
without code or timeout changes. All required CI gates, including the
final `ci / verify` and `ci / e2e` summaries, are green on
`7e57dc267a8446d3c906e3cc5b8abc94fb8860eb`.

## Risks

- Email from external senders can start normal agent work. Setup
recommends a low-trust agent and AgentMail sender controls. Sender
addresses never grant board membership.
- Provider timeouts can leave uncertain sends. Retries retain their
idempotency key; expired windows require reconciliation or operator
resolution.
- Connector skills and native tools are assignment-dependent and require
current access. Revocation denies retained calls; assignment changes
select a new runtime context.
- Activation remains behind the experimental-channel setting. The native
runner path has deterministic coverage; live Daytona qualification used
the Codex CLI adapter.
- Schema changes are additive. Inbox ownership is unique across
companies. Disconnect preserves provider inboxes and task history.

## Model Used

OpenAI GPT-6 (Codex). Used reasoning, repository tools, code execution,
and browser testing. The exact deployment model ID and context-window
size were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-11 16:56:38 -05:00

5.7 KiB

AgentMail Daytona verification — 2026-09-11

Worktree: codex/agentmail; isolated test drive at http://localhost:3103. The original checkout remains untouched. Test mail used only the two previously authorized inboxes, pap15838-qa@agentmail.to and attractiveforce961@agentmail.to.

Defects corrected

  • AgentMail REST connections fell through the generic health-check branch into local-stdio MCP validation. Both saved account credentials and inbox credentials now validate against AgentMail's /auth/me API. Catalog refresh returns no MCP tools, and invalid keys still produce a failed health result. The live Apps card was inspected in the browser and showed Connected with no stdio error.
  • Sandbox callback routing omitted task-email endpoints. It now allows assigned inbox discovery, task-thread reads, delivery reads, and explicit sends. Server company, inbox, task/run, and action-policy authorization remains in force. Setup, credentials, reconnect, and operator delivery resolution stay denied.
  • Shell-backed sandbox reads did not preserve ENOENT for a missing optional Codex auth.json, causing cleanup to fail after a successful email send. Reads now confirm absence in a searchable parent and return ENOENT; actual read and transport failures still propagate. This lets existing auth copy-back treat missing credentials as a no-op.
  • Runtime instructions now document inbox discovery directly; the agent otherwise spent time guessing that endpoint when initiating a new conversation.

Live observations

The board used the ordinary task composer in AGE-10 to request a test email. The agent executed the real Codex CLI in Daytona, used the sandbox callback bridge to discover its assigned inbox and queue the send, and created AGE-11 as an email child task.

  • Provider sandbox: c2f176ca-dbde-41a6-995d-aefa4689e4c5.
  • Runtime verified by the agent: Linux, x86_64; hostname matched the sandbox.
  • Run: cd7b934a-5555-4361-b0e5-b8106c1510ce.
  • Publication: d5b7bf41-a583-4f9f-90c0-4d21680e39c2, Delivered.
  • Subject: [Paperclip E2E] Daytona sandbox — Sep 11.
  • Provider key remained in Paperclip's vault. The sandbox used its injected Paperclip run credential, and the model key was separately vaulted.

The first fixture launches exposed an unavailable default ACP executable and a host service_tier setting incompatible with the fleet image's Codex CLI. The QA fixture explicitly selects the CLI engine and an isolated Codex home. Earlier failed launches remain in AGE-9. The outbound send above completed, but its run then failed during missing-auth-file cleanup; the cleanup fix is verified separately below rather than rewriting that history.

Cleanup verification

A fresh Daytona run in AGE-12 read the existing publication, confirmed Delivered, recorded its Linux hostname, and completed successfully without sending another email.

  • Run: 39e77902-714e-455f-90d8-8709f2d13762, Succeeded.
  • Sandbox: d50c6979-de6e-4a0c-ac18-bd616a39ee1f.
  • Cleanup log: “no sandbox credential to copy back (absent auth.json); host credential kept.” The environment lease reached Released.

Automated checks

  • 20 durable email pipeline tests passed, including health checks for account and inbox credentials, catalog discovery, and invalid credentials.
  • 56 sandbox callback bridge tests passed, including the four email routes and denial of email administration routes.
  • 28 command-managed runtime tests passed, including the missing-file contract and propagation of real read failures.
  • 4 capability inventory tests passed. Regenerated both capability indexes for the new task-email runtime documentation and updated the expected row count.
  • Server typecheck, server build, adapter-utils build, and whitespace checks passed.

Inbound round trip

After Chrome access recovered, sent a new authorized test email from the other inbox through AgentMail Console. WebSocket intake created AGE-13, assigned Email QA, and started the agent in a fresh Daytona sandbox. The agent read the bound thread, explicitly replied once, checked delivery, and marked the task Done.

  • Run: 76be255b-df2e-4479-8c62-f4506f039132, Succeeded.
  • Sandbox/verified Linux hostname: 7bb660fa-3cff-4b26-9e10-68c884be21bb.
  • Reply publication: efdd704c-afd3-4025-ab48-24fab6c97333, Delivered.
  • Incoming comment persisted at 19:05:14.750Z; run started at 19:05:14.920Z (170 ms later). This interval excludes provider delivery and does not measure model startup. The run finished at 19:06:09.481Z.
  • Exactly one incoming and one outgoing email comment, plus an internal summary.
  • Visually verified the exact acknowledgement in the other AgentMail inbox, with matching reply message ID and original-message reference.

Test cleanup

Restored Email QA's original local adapter configuration. Removed the temporary Daytona environments, all six sandbox instances created by this test, and the temporary vaulted Daytona/model credentials. Provider inboxes, saved AgentMail credentials, task history, and run evidence remain available.

Qualification limits

This run exercises the Codex CLI sandbox adapter. The native runner task_email path is covered deterministically, but was not separately live-qualified in Daytona. The Daytona inbound round trip used WebSocket intake. Earlier local-agent WebSocket and signed-webhook qualification is documented in the main verification report.