Reproduce a blocker appearing during admission, recheck current recovery before queuing, and consume the authorizing receipt atomically even when aggregation is suppressed. Retain the same saved receipt under daily-cap retries.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Keep terminal stop receipts intact when a late process callback arrives. Reconsider saved execution-wait comments independently of recovery-action state, validate their author under the task lock, and retain admission gates and retry deduplication.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Retry decisions must respect changes to an issue owner.
> - Database tests verify this with two concurrent transactions.
> - The test started the competing operation before its fixture held the
lock.
> - That race can fail a correct source verification run and block
deployment.
> - This PR waits for lock acquisition before starting the competing
operation.
## Linked Issues or Issue Description
Refs #13257 for the deployment verification work that exposed this test
race. No duplicate fix was found.
**What happened?**
[Cloud verification job
103424158921](https://github.com/paperclipai/paperclip/actions/runs/34648268409/job/103424158921)
failed because the test did not observe a concurrent issue-row lock
waiter. The fixture and retry operation both started without an ordering
guarantee.
**Expected behavior**
The fixture must hold the issue lock before the competing retry
operation starts. The test must still prove that the retry waits for the
lock and observes the reassignment.
**Steps to reproduce**
1. Add a temporary 50 ms delay before the fixture acquires the issue
lock.
2. Run the promoteOrCancelDueRetry issue-lock test.
3. The original fixture fails with the same missing-waiter error as CI.
4. The synchronized fixture passes with that delay. The delay is not
part of this PR.
## What Changed
- Separate fixture readiness from its transaction completion promise.
- Wait for readiness at both callers before starting the retry decision.
- Propagate transaction failure during setup through Promise.race.
## Verification
- The original fixture fails under the temporary delayed-lock probe. The
fixed fixture passes the same probe.
- All 31 tests in
server/src/modules/run-dispatch/adapters/postgres.test.ts pass after
removing the probe.
- The real concurrent waiter, lock-order, and reassignment assertions
remain intact. No timeout was increased.
- Full local typecheck and build pass (221s and 50s). The full local
test command stopped in its server phase after 10,610 passes, 65 skips,
and 14 failures: 13 existing macOS skill-cache rename/permission
failures and one unchanged Telegram test assertion. The Telegram test
passes in a focused rerun. Later local phases did not run after that
failure. Current-head Greptile is 5/5 with no findings. All 32
current-head checks pass, including full Linux typecheck, build, native
verification, server suites, browser suites, and canary packaging. The
unchanged GitHub browser mock assertion passed its single failed-shard
retry.
- git diff --check passes.
## Risks
- Only test synchronization changes. Production database behavior is
unchanged.
- Awaiting the transaction itself during setup would deadlock the test.
Returning the completion promise inside an object avoids that problem.
## Model Used
OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — all 31 database adapter
tests pass; full local verification limits are disclosed above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
Preserve verified native process-stop evidence and retry saved user messages through normal continuation admission after recovery cleanup. Show the current wait reason and serialize delivery so a saved message starts one fresh turn.
Validated with 410 focused tests, typecheck, build, token gates, all PR CI checks, and Greptile 5/5.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native sessions keep a provider process and its work alive across
control-plane operations.
> - A hidden 15-minute turn deadline stopped work even when the agent
timeout was zero.
> - A one-hour runner lifetime and fixed connection lease added two more
limits.
> - Recovery also rejected goal commands because it reconstructed their
startup summary with the wrong protocol version.
> - This pull request removes implicit duration limits and renews
authenticated leases in the harness.
> - Healthy sessions can continue without model action or a user
interface change.
## Linked Issues or Issue Description
Refs #13092 and #12845. Related: #13163 covers sandbox recovery after
app restarts; this change covers session duration and lease renewal.
**What happened?**
A native Codex session stopped after 15 minutes while a tool was still
running. The agent had `timeoutSec: 0`. Recovery then rejected a
`session.goal.get` startup command with `invalid provider startup
ownership fence`.
**Expected behavior**
An unlimited session keeps working while its provider and authenticated
controller remain healthy. Lease maintenance is transparent. Explicit
timeouts, cancellation and revoked authority still take effect.
**Steps to reproduce**
Start a native session with `timeoutSec: 0` and run a tool beyond 15
minutes. Before this fix, the runtime cancels the turn. A recovery
startup that uses a goal command also exposes the protocol-version
mismatch.
## What Changed
- Honor the agent turn timeout. Zero disables the timer. Long explicit
durations use timer chunks to avoid Node timer overflow.
- Default native runner lifetime to unlimited. Keep bounded startup,
reconnect and control-operation deadlines.
- Renew leases over the authenticated connection. Persist renewal before
the reply. Validate identity, epoch and expiry. Handle duplicate
requests and a lost reply on reconnect.
- Freeze renewal during warm ownership transitions and terminal
handling.
- Validate persisted goal startup commands with protocol v2.
- Add duration, renewal, ownership, recovery and real-process regression
tests. Update runner protocol and recovery docs.
- Add no UI components or controls. Renewal requires no model output or
user action.
## Verification
Current head: `348e369c35c5da8bb8be378f4b35dcf6f40882e7`. [CI
run](https://github.com/paperclipai/paperclip/actions/runs/34649767113).
- All 32 checks pass on this head. The two Storybook checks are skipped
as expected. CI includes full build, typecheck, runner verification,
browser suites, server tests and the canary package dry run.
- Greptile reports 5/5 on this head. All review threads are resolved,
and the security scan passes.
- Passed `pnpm -r typecheck` and `pnpm build` locally.
- Passed 219 native-runtime and controller tests, including fake-clock
tests for three weeks of renewal and 30-day explicit timeouts. Six
denial tests confirm that renewal cannot extend expired, revoked or
mismatched authority.
- Passed 337 executor, cancellation and restart-recovery tests, plus 278
Rust runner-core library tests.
- Passed a real runner with a silent fake Codex provider across its
original lease expiry. Runner PID, provider PID, thread and active turn
stayed unchanged. Warm-attach recovery tests also pass.
- Passed all 83 plugin-worker tests and 159 of 161 workspace-runtime
tests locally. The two remaining assertions passed with a canonical
macOS temporary directory, as did the changed runtime fixture. The full
affected server shard passes in CI.
- An unchanged GitHub callback-ordering test failed once in CI, passed
locally in isolation, and passed its one test-shard retry. The final CI
summary is successful.
- The full local `pnpm test:run` sweep was interrupted after dependency
setup failures and load-related timeouts. Identified failing suites
passed in isolated reruns after the dependency repair. The complete test
matrix passed remotely in CI.
## Risks
- Deploy the controller and runner together to enable renewal. Older
peers keep their existing bounded lease behavior.
- Unlimited runtime permits long resource use until completion, explicit
cancellation, configured timeout or loss of valid authority.
- Lease renewal changes authenticated protocol handling. Regression
tests cover stale, revoked and mismatched authority, lost replies and
warm handoff behavior.
- Simulated multi-week tests and a real lease-boundary test do not
constitute a weeks-long production soak.
## Model Used
OpenAI GPT-6 through Codex, with repository inspection, code execution
and TypeScript/Rust test tools. The exact backend revision and
context-window size are not exposed in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections give agents controlled access to external services.
> - Experimental channels already map conversations to tasks and durable
work queues.
> - Email needs inbox ownership, recipient envelopes, delivery records,
and explicit sends.
> - This pull request adds AgentMail to that infrastructure and keeps
the provider key in the server vault.
> - Agents can receive and send email from local or sandbox execution
while the board follows each conversation in its task.
## Linked Issues or Issue Description
**Problem or motivation**
Agents need dedicated email addresses. Incoming email should become
assigned work. Internal task comments and progress must never become
outgoing email by accident.
**Proposed solution**
Add experimental AgentMail connections, an inbox assignment wizard,
durable email intake and publication, task email cards, and
authenticated API, CLI, and native runtime actions. Agents use Paperclip
credentials to request sends. Paperclip owns the provider key and
enforces access and task authority.
**Alternatives considered**
A general mailbox MCP connector does not provide durable task binding or
publication boundaries. A separate mailbox application duplicates task
collaboration. The board instead directs the agent through the normal
task conversation.
**Roadmap alignment**
This extends the existing experimental connections and task
infrastructure. Product scope and interaction design were reviewed with
the maintainer. Related connection authority work: #11831 and #11818.
The duplicate search found no competing task-based AgentMail
integration.
## What Changed
- Add AgentMail catalog data, shared contracts, company-scoped email
records, and an additive migration.
- Add vaulted setup, inbox assignment, access grants, trust guidance,
and provider-side allowlist guidance.
- Support WebSocket and signed-webhook intake through a shared durable
pipeline, deduplication, catch-up, and task wakeups.
- Queue explicit new conversations and replies with immutable send
intents, idempotency, delivery state, and uncertain-send resolution.
- Show inbound and outbound email cards in normal task conversations.
Keep internal messages internal.
- Add task-scoped CLI actions and the sandbox callback routes required
for Daytona execution.
- Provide a dedicated AgentMail skill automatically only to agents with
active authorized inbox assignments. Keep email instructions out of the
universal Paperclip skill.
- Advertise connector-owned `agentmail_inboxes`,
`agentmail_read_thread`, `agentmail_send`, and `agentmail_delivery`
tools only in eligible native sessions. Recheck live authority on
execution.
- Isolate Codex CLI connector skills by agent and skill revision.
Deliver the assigned skill in the run prompt for adapters that use
shared skill directories, including resumed turns. Keep automatic skills
out of manual persistent sync. Show them as read-only and document the
pattern in the connector playbook.
- Fix AgentMail health checks that entered local-stdio validation and
optional missing Codex credential cleanup in sandboxes.
- Add API, pipeline, authorization, sandbox, browser, and Storybook
coverage.
## Verification
- Live AgentMail testing covered WebSocket intake, signed webhooks,
restart catch-up, and a full receive → task → Daytona Codex CLI →
explicit reply → Delivered round trip. The reply was verified in the
other inbox. The normal task composer also initiated an outgoing email
child task.
- The connector-skill change was verified in the browser: AgentMail
appears once as an automatic, read-only skill with its assigned address.
Disabling experimental chat connections removes it; re-enabling restores
it. A regression test covers assignment data arriving after library
data.
- Connector regression coverage passed 178 runtime utility, email
integration, skill-route, and heartbeat tests. All 17 Codex execution
tests passed, including per-agent skill isolation, model identity,
revision changes, removal, and prompt delivery without shared skill
files.
- After rebasing onto master, all 44 focused email, heartbeat, and
native-authority tests passed. All 313 native-session executor tests
passed. The UI regression suite passed all 3 tests. These test sets
overlap earlier focused runs.
- Full workspace typecheck and build passed after the rebase. Token
gates passed. Earlier focused Playwright task/setup coverage and the
Storybook build also passed.
- Native connector tool execution uses deterministic integration tests.
Live Daytona qualification used the Codex CLI adapter; the new
shared-home prompt fallback has deterministic coverage.
- The full repository suite is run by CI. The earlier unsharded local
full-suite attempt was stopped after the equivalent CI suites passed and
is not reported as a completed local run. Greptile reviewed
`7e57dc267a8446d3c906e3cc5b8abc94fb8860eb` at 5/5 with no unresolved
threads. All server, workspace, serialized server, and browser suites
passed in CI. The build job hit a five-second timeout in a runner
transport test; both variants and the full 80-test file passed locally
with unchanged timeouts. The build passed on retry on the same commit
without code or timeout changes. All required CI gates, including the
final `ci / verify` and `ci / e2e` summaries, are green on
`7e57dc267a8446d3c906e3cc5b8abc94fb8860eb`.
## Risks
- Email from external senders can start normal agent work. Setup
recommends a low-trust agent and AgentMail sender controls. Sender
addresses never grant board membership.
- Provider timeouts can leave uncertain sends. Retries retain their
idempotency key; expired windows require reconciliation or operator
resolution.
- Connector skills and native tools are assignment-dependent and require
current access. Revocation denies retained calls; assignment changes
select a new runtime context.
- Activation remains behind the experimental-channel setting. The native
runner path has deterministic coverage; live Daytona qualification used
the Codex CLI adapter.
- Schema changes are additive. Inbox ownership is unique across
companies. Disconnect preserves provider inboxes and task history.
## Model Used
OpenAI GPT-6 (Codex). Used reasoning, repository tools, code execution,
and browser testing. The exact deployment model ID and context-window
size were not exposed in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runtime records completion assessments and task
decisions.
> - An application update can change the assessment policy version.
> - The previous code treated that version change as a reason for human
review.
> - An upgrade alone does not give the user a new decision to make.
> - This change keeps existing decisions and withdraws obsolete upgrade
review cards.
## Linked Issues or Issue Description
**What happened?**
A rules version change moved unfinished tasks into review and created a
card that said, "Review the superseding native policy assessment." The
task did not need new work or a human decision.
**Expected behavior**
New runs use the current rules. An upgrade leaves existing task
decisions alone. The saved policy version remains available in the audit
history.
**Steps to reproduce**
1. Save a native run assessment and a task decision.
2. Change the native status policy version.
3. Run finalization reconciliation without new task evidence.
4. The previous code created a review card. The corrected code keeps the
saved assessment and decision.
Related context: #13038 changed the native policy version. Searches
found no duplicate fix for upgrade-only completion reviews.
## What Changed
- Remove policy-version mismatch as a reconciliation trigger.
- Withdraw pending cards only when their source decision, effect ledger,
creator, key, and prompt match the old upgrade-only review.
- Restore the previous status only while that decision and status
version remain current and no other review gate is pending.
- Preserve answered cards, real review requests, later task changes, and
historical assessments and decisions.
- Record cleanup activity and retire any corresponding chat review
actions.
- Isolate cleanup failures so one old card cannot block other cleanup or
normal finalization.
- Update the status conformance fixture, regression tests, and
architecture documentation.
## Verification
- Passed: `pnpm exec vitest run
server/src/__tests__/native-status-arbiter-corpus.test.ts` (23 tests,
including the 53-fixture status corpus).
- Passed: `pnpm -r typecheck`.
- Passed: `git diff --check`.
- Passed: `pnpm build`.
- Passed: fresh Greptile review at 5/5 on `d24e5ecaf`, with no open
review threads.
- Passed: the 23 focused tests in the isolated full-suite environment.
- Local `pnpm test:run`: the server group finished with 10,638 passed
and one missing-fixture failure. Built the required
`fake-codex-app-server` fixture and reran the entire affected native
session-resume suite: 37/37 passed. The initial full command exited on
that server-group failure, so remaining groups are covered by CI.
- Passed: the workspace-runtime-exposure suite (25 tests, 3 platform
skips).
- Passed: all CI gates, including every test shard, browser tests,
typecheck, and build. The server shard passed on one retry after an
unrelated host-port conflict in the unchanged workspace-exposure tests.
- Cleanup tests cover repeat runs, real reviews, answered cards, later
statuses, newer decision identity, status changes back to review, other
pending requests, run scope, cleanup failures, retry, and live
publication failures.
## Risks
- Cleanup changes stored task state. It checks the exact obsolete
decision and status version under database locks before restoring
status.
- Pending approvals, interactions, and execution stages prevent
restoration out of review.
- The cleanup handles at most 100 matching cards per reconciliation
pass. It does not rewrite old decisions or accept an agent's completion
claim.
- New evidence and explicit task changes still use the existing
reconciliation paths. This change does not reevaluate old work merely
because Paperclip was updated.
## Model Used
OpenAI Codex, GPT-6. The exact deployment identifier and context window
are not exposed in this session. Used reasoning, repository inspection,
code editing, shell execution, and automated tests.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - CI installs dependencies before it verifies and builds cloud
artifacts.
> - Install jobs share a pnpm package-store cache with lockfile refresh.
> - Lockfile refresh resolves versions without downloading packages.
> - That job saved an empty cache before full install jobs could save
theirs.
> - This PR prevents lockfile refresh from publishing that empty entry.
> - Full install jobs can then populate the cache and reuse
dependencies.
## Linked Issues or Issue Description
Refs #13259 for the related cloud verification cache work. No duplicate
empty-cache fix was found.
**What happened?**
Refresh Lockfile run 34517514932 saved a 216-byte default-branch pnpm
cache at 18:58:08 UTC on September 10. Full install jobs still restore
that empty entry. The cache API reports 216 bytes for master and about
703 MB for populated entries with the same key and cache version in PR
scopes.
**Expected behavior**
A job that installs dependencies should populate the shared
package-store cache.
**Steps to reproduce**
1. Run lockfile refresh with a new lockfile cache key.
2. Its resolution-only command leaves the package store empty.
3. The Node action saves the empty archive before a full install
finishes.
4. Later jobs report a cache hit but download packages again.
**Paperclip version or commit**
Observed on master 6728e133f8 and still
present at a23ae894a5.
**Deployment mode**
GitHub Actions cloud verification and release workflows.
## What Changed
- Disable package-manager caching in Refresh Lockfile.
- Document how to remove the existing empty default-branch entry and
verify a populated replacement.
- Add regression coverage for explicit and automatic package-manager
cache selection in a resolution-only job.
## Verification
- actionlint and git diff checks pass.
- All 175 existing workflow-script tests pass. Both new regression cases
pass and fail against the original workflow, covering the explicit pnpm
cache and automatic npm cache paths. This change adds no application
behavior.
- [The cache creator
job](https://github.com/paperclipai/paperclip/actions/runs/34517514932/job/103006542158)
logs a 216-byte upload under the same key still used by cloud
verification.
- The batch-wide local full typecheck and build passed. The local full
test run reported 10,600 passed, 65 skipped, and 13 permission failures
in unchanged runtime-skill suites. These checks were not repeated in
this dependency-free worktree. Current-head Linux CI passes. The
unchanged chat and browser suites passed on their single retry; all
final checks are green. Greptile is 5/5 with all threads resolved.
- After merge, delete only the existing empty master cache entry. Verify
that a master install saves a populated archive and subsequent jobs
reuse packages. Measure the net install-time change before claiming a
latency gain.
## Risks
- Lockfile resolution can require fresh registry metadata. It does not
need a cached package store.
- The existing empty cache must be removed once; this change prevents
its recreation by this workflow.
- Cache benefits vary with download speed and archive extraction time.
## Model Used
OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — focused workflow checks
pass; the batch-wide local test limitation is disclosed above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip Cloud instances can inject a trusted operator HTML snippet
before `</body>` in served UI pages (#13168)
> - Operators deliver env vars to managed instances through
hosting-provider APIs
> - Web application firewalls in front of those APIs reject request
bodies that contain raw `<script` markup
> - The snippet value always contains a script tag, so the firewall
blocks its delivery every time
> - This pull request adds `PAPERCLIP_CLOUD_UI_SNIPPET_B64`, which
carries the same snippet as standard base64
> - The benefit is that operators can ship the snippet through
WAF-fronted delivery pipelines without firewall exceptions
## Linked Issues or Issue Description
Refs #13168 (introduced the Cloud UI snippet).
**Subsystem affected**
Server UI shell serving: `server/src/cloud-ui-snippet.ts`, used by
`static-index-html.ts` and `app.ts`.
**Current behavior**
`PAPERCLIP_CLOUD_UI_SNIPPET` is the only way to configure the Cloud UI
snippet. The value is raw HTML. Delivery pipelines that write env vars
through provider APIs can fail to deliver it: web application firewalls
classify a request body that contains `<script` as an injection attempt
and block it. Verified against Railway's GraphQL API, which sits behind
Cloudflare: a variable value with a bare `<script></script>` returns
HTTP 403 before authentication, and JSON unicode escapes (`<script`) do
not bypass the block. The snippet is exactly the kind of value that
always contains a script tag, so such pipelines cannot deliver it at
all.
**Proposed behavior**
A new optional `PAPERCLIP_CLOUD_UI_SNIPPET_B64` carries the same snippet
as standard base64 of the UTF-8 HTML. The server decodes it and injects
the result through the existing path. The plain variable wins when both
are set. Whitespace and line wrapping in the value are tolerated. A
value that is not canonical base64, or that decodes to blank, is ignored
instead of injected as garbage.
**Reason and benefit**
Operators can deliver the snippet through WAF-fronted APIs without
requesting firewall exceptions. Base64 contains no markup, so the
firewall has nothing to match. Existing deployments see no behavior
change.
**Breaking changes**
None. The new variable is optional. The existing variable is unchanged
and takes precedence.
## What Changed
- `server/src/cloud-ui-snippet.ts`: resolve the snippet from
`PAPERCLIP_CLOUD_UI_SNIPPET` first, then from base64-decoded
`PAPERCLIP_CLOUD_UI_SNIPPET_B64`; ignore non-canonical or blank-decoding
values.
- `server/src/__tests__/cloud-ui-snippet.test.ts`: cover
decode-and-inject, plain-wins precedence, whitespace tolerance,
invalid/blank values, and the self-hosted no-op.
- `doc/cloud-ui-snippet.md`: document the variant, how to produce the
value, and the ignore rules.
## Verification
- `pnpm vitest run server/src/__tests__/cloud-ui-snippet.test.ts
server/src/__tests__/static-index-html.test.ts` — 13 tests pass.
- `tsc --noEmit` in `server/` passes.
- Manual check: on a Cloud-managed instance, set
`PAPERCLIP_CLOUD_UI_SNIPPET_B64="$(base64 < snippet.html)"`, restart,
open `/`, and confirm the decoded snippet appears before `</body>`. On a
self-hosted instance, confirm no snippet is injected.
## Risks
Low risk. The injection path and the Cloud-managed gate are unchanged. A
malformed base64 value is ignored, so the failure mode is a missing
widget, not corrupted HTML. The decoded value is trusted operator
configuration, the same trust model as the plain variable.
## Model Used
- Claude Fable 5 (Anthropic, `claude-fable-5`) via Claude Code CLI,
agentic workflow with tool use (code edits, test runs, and live API
verification of the WAF behavior).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud deployments consume a verified image and exact-source
migrator.
> - An image alone is not deployable until source checks and artifact
checks pass.
> - GitHub-hosted queues delayed those checks and the final readiness
signal.
> - This PR gives trusted master work a separate concurrency allowance
on existing AWS runners.
> - Community PRs and arbitrary source inputs keep the GitHub-hosted
fallback.
## Linked Issues or Issue Description
Refs #13243.
**What existing behavior does this improve?**
Time from a master merge to the Cloud deployable v1 signal.
**Current behavior**
For merge d0b7ba4, the image was available after 7m 42s, but readiness
took 16m 05s. Typecheck queued for 6m 40s and the final readiness job
queued for 1m 46s.
**Proposed behavior**
Allow up to 36 concurrent post-merge verification and migrator jobs on
the existing four-vCPU, 16-GiB AWS runners. Workers launch on demand and
terminate after their job; no always-on worker pool or AWS Reserved
Instance purchase is introduced. Keep the combined runner ceiling
unchanged. A separate operator switch enables this route only after the
restricted runner group and Fleet exist.
**Reason and benefit**
Remove GitHub-hosted queue delays from the cloud deployment path. The
gain depends on queue pressure and which remaining job finishes last;
the observed queues are not additive savings.
**Breaking changes**
None to source verification or readiness contracts. Paid routing is
limited to canonical master push/manual events, with exact source checks
on reusable and migrator jobs.
## What Changed
- Route source verification, artifact waiting, dispatch, and readiness
jobs to the separate post-merge Fleet when enabled.
- Require source inputs to match the event's master SHA. Preview inputs
and raced older migrator dispatches stay GitHub-hosted.
- Keep npm publication on GitHub-hosted runners for trusted publishing.
- Bound AWS job timeouts below the 45-minute instance lifetime.
- Document activation, capacity reservation, and rollback.
- Exercise each actual runner selector against allowed and rejected
event/source combinations.
## Verification
- 268 routing and timeout cases pass, including unapproved PR, fork,
branch/tag, arbitrary ref, and disabled-switch cases.
- All 461 focused workflow, preview, and readiness tests pass. The 284
routing/preview cases also pass after the review fixes.
- actionlint passes for changed workflows with the existing
SC2012/SC2016/SC2129 warnings excluded.
- Full local typecheck and build pass (167s and 206s). `pnpm test:run`
completed: 10,600 passed, 65 skipped, and 13 failed in the unchanged
company-skills-service/runtime-skill-cache suites with local filesystem
permission errors. Linux CI is the required test gate; this is not a
claim of a fully passing local suite. Current-head Linux CI is green,
Greptile is 5/5, and all findings are resolved. The final Build retry
passed on a verified 60 GiB AWS runner after correcting the earlier
disk-capacity failure.
- After activation, verify a master run selects the separate group and
all readiness prerequisites pass.
## Risks
- A missing or incorrectly restricted runner group can leave eligible
jobs queued. Enable the switch only after Fleet and group verification.
- PR bursts have 64 slots after reserving 36 for post-merge work. The
image Fleet retains eight, for the same 108-runner total.
- A migrator dispatch racing a newer merge uses GitHub-hosted runners.
This preserves source trust but can retain some queue delay.
- Roll back placement by disabling AWS_POST_MERGE_CI_ENABLED and
rerunning the whole workflow.
## Model Used
OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — focused change tests
pass; full-suite local permission failures are disclosed above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
* codex/work-folders-storybook-refresh:
fix: keep supported ACPX profiles available in the Pi layer
test: wait for active Daytona transfers before teardown
