Commit Graph
231 Commits
Author SHA1 Message Date
Devin FoleyandPaperclip 21bf7ebcee feat(connectors): connect Tailscale networks to SSH environments (#20617)
## Thinking Path

> - Paperclip manages work for AI agents.
> - SSH environments run tasks on computers that users own.
> - A Paperclip server needs network access before it can reach a
private computer.
> - A reusable Tailscale connection belongs in Connectors.
> - Connected machines should become choices in the existing SSH
environment form.
> - This change connects a tailnet and resolves selected machines
through a private network client.
> - Agents retain their choice of harness and AI account.

## Linked Issues or Issue Description

**Subsystem affected**

Connectors, environment settings, SSH execution, company secret storage,
and runtime packaging.

**Problem or motivation**

Entering a Tailscale address in the SSH form does not connect the
Paperclip server to that tailnet. Requiring a separate device key for
every environment also duplicates network setup.

**Proposed solution**

Add a Tailscale Connector with **Connect Tailscale Network**. Use
brokered OAuth consent to enroll one private network client per
connection. Let SSH environments select a connected network and stable
machine identity. Preserve the enrolled node state through the company
vault so a restart does not reuse the one-time token.

**Alternatives considered**

A separate Tailscale environment type would duplicate SSH behavior and
mix network access with harness selection. This change uses the existing
`ssh` driver. Related #15648 adds OAuth client credentials and a health
check; this change uses user consent and supplies outbound task
execution. Related #11743 concerns HTTPS brokering.

**Roadmap alignment**

This extends the existing Connected Apps, environment, and Personal &
Shared AI Accounts capabilities. This PR is stacked on #21248, which
provides SSH account selection independently of Tailscale. The hosted
OAuth broker is a companion control-plane change.

## What Changed

- Preserve Cloud reauthentication in the current browser tab. Explain
canceled or failed consent in the connector and offer a retry of the
same authorized saved connection.
- Record Cloud enrollment starts initiated through the Tailscale
connector in the company activity log.
- Check install-preparation descendant liveness through the kernel
process table in the Runner support test. Keep the direct child’s strict
reaping assertion and reject any non-zombie descendant.
- Align the application image stage contract with the Tailscale client
stage after the upstream Runner-image refactor. Keep the exact stage
list and check every application stage, including `tailscale-client`,
for prohibited Runner compiler commands.
- Group Tailscale consent, discovery, daemon lifecycle, pairing, and
deletion under `server/src/services/connectors/tailscale/`, with their
service tests. Update existing route, runtime, environment, vault, and
company-deletion imports. Keep the existing contracts, authorization,
audit, and transaction owners.
- Handle early stdin closure in the Kubernetes real-shell test helper.
Stop the input stream on `EPIPE`, preserve nonzero child exits, and
reject a zero exit when input was lost. Add two real-process regression
cases. Production transfer code and confinement checks are unchanged.
- Read configured Tailscale executable paths when a client starts, after
supported `.env` loading. Keep explicit command overrides available.
- Discard pending pairing results after switching to Direct SSH or
closing the form. Retain the existing company, connection, machine, and
port checks.
- Store network credentials with an immutable server-owned purpose from
their first vault row. Exclude them from generic catalogs, bindings,
environment projection, and value reads. Use a dedicated audited server
consumer for pinned connector reads. Block unclassified legacy
serialized node state at the same value boundary. Keep SSH private-key
binding separate from connector credentials.
- Gate network queries and setup on verified local-implicit or
instance-admin access. Hide Tailscale catalog rows for other members,
show an admin requirement on direct setup navigation, and avoid device
discovery before management access is known.
- Keep failed-client cleanup in the network manager. A failed request
cannot disconnect a newer retry for the same identity; explicit
connector disconnect and enrollment retirement still stop their
networks.
- Check for readable Ed25519, ECDSA, or RSA host public keys before
pairing edits `authorized_keys`. Print a known-hosts entry for every
available key, or fail before changing authorized keys if none are
available.
- Return self-hosted Cloud enrollment to the Tailscale connector. Save
the unprefixed setup path and let the shared callback add the verified
company prefix. Keep external and unrelated destinations rejected.
- Hide pairing instructions after an SSH port change. Ignore late
key-selection results for the old port. Operators prepare again for
matching known-hosts output.
- Retire connector references before deleting a company or its vault
credential. Stop company daemons after the database commit. Keep failed
secret retirement retryable and prevent late node-state writes from
restoring deleted connections. Discard retired state without blocking
behind the vault deletion lock.
- Reload a mismatched credential before replacing a ready daemon.
Recheck the active entry and manager generation after the read. An older
request cannot stop a newer healthy socket or cross shutdown.
- Pin connector credentials to the exact vault version and SHA-256
digest recorded by trusted enrollment and node-state writes. Ignore
generic secret rotations. Advance both fields with a compare-and-set
guard. Reject unpinned or tampered credentials before network startup.
- Save an explicit Direct SSH selection and clear old Tailscale fields
after config merging. Preserve the operator's entered direct host.
- Route all Tailscale SSH runtime access through the connection service
after SSH-key loading. Pin access to the environment's saved connector
identity and recheck after device discovery. Remove standalone key-based
daemon startup and reject raw enrollment keys or credential-only network
configurations. Align the shared config type and SDK pins.
- Correct two E2E assumptions exposed during connector qualification.
Keep the two-window chat fixture stale until its own mutation. Validate
the original failed run and its permitted write-revocation timestamp in
the legacy recovery browser test.
- Add the Tailscale Connector page, catalog entry, OAuth return flow,
connected-network management, device discovery, and disconnect. Document
all six routes in OpenAPI and the API reference, including their
instance-admin requirement.
- Add company-scoped connection storage in `0480_lazy_lethal_legion`,
trusted credential version/digest fields in `0481_round_pet_avengers`,
and server-owned secret purpose/backfill in
`0482_aberrant_rumiko_fujikawa`. Generate all three migrations through
the maintained command and retain the five newest snapshots.
- Add machine pairing that generates a fresh SSH key per request, stores
its private half in the company vault, and uses the generated public
half in the install command. Do not trust editable saved metadata.
Preserve existing authorized keys, including a file without a final
newline. Keep SSH username, workspace, and trusted host keys explicit.
- Use the existing SSH driver. Resolve the machine's current address
from its stable device ID. Reject disconnected, removed, offline, and
wrong-company selections.
- Hide Network when Direct is the only option. Select a sole connected
tailnet automatically and hide its selector. Remove the requested setup
helper copy. Do not restrict the harness to Claude.
- Run a private userspace Tailscale daemon with a separate socket.
Persist its node state before execution. Restore it after restart, bound
startup, serialize reconnect cleanup, and close it during server
shutdown.
- Guard OAuth completion with a durable attempt marker. Disconnect and
newer authorization invalidate older callbacks. Close replaced daemons
and remove cancelled enrollment secrets without changing newer
connection state.
- Preserve SSH host verification and tailnet-specific session identity.
Honor the configured Tailscale executable path, including spaces and
OpenSSH percent escaping.
- Package checksum-pinned Tailscale clients and their license in the
standard runtime image. Exercise the client build stage in PR CI and
include it in image smoke checks.

## Verification

- Merge status: all CI and image checks pass, and current-head Apex is
5/5 with no unresolved comments. GitHub rejected a direct merge because
the active master ruleset requires code-owner approval for
`.github/workflows/pr-trusted.yml`. The [CODEOWNERS workflow
pattern](https://github.com/paperclipai/paperclip/blob/master/.github/CODEOWNERS#L3)
applies; eligible maintainers already have review requests. Auto-merge
is enabled with the squash method for the verified head. The PR remains
open pending that required approval; no branch policy was bypassed.
- Final revision: `fdb29b2cc366f4945b7f4c3e2b2d25070329a21b`, including
master `999709b0a6dd416af335b5debded13e1de7fcb7a`. [Final CI
verification
passed](https://github.com/paperclipai/paperclip/actions/runs/43518584814/job/132213508320).
All required checks pass, all applicable test/build gates are green, and
the branch is mergeable against master. SSH prerequisite #21248 and
Cloud broker paperclipai/paperclip-cloud#1314 are already merged.
- [Apex reviewed this exact revision at
5/5](https://github.com/paperclipai/paperclip/pull/20617#issuecomment-6776629437).
There are no unresolved review threads. The PR is classified as Critical
impact.
- Final image qualification passes for the [application Runner
path](https://github.com/paperclipai/paperclip/actions/runs/43518584522/job/132206784267),
[amd64 Runner
image](https://github.com/paperclipai/paperclip/actions/runs/43518584544/job/132207296198),
and [arm64 Runner
image](https://github.com/paperclipai/paperclip/actions/runs/43518584544/job/132207296156).
Typechecks, Runner support, native/Rust tests, provider tests,
serialized-server suites, browser shards, and the remaining applicable
CI checks pass. This is source/image CI qualification, not a production
rollout.
- The last master conflict affected only Docker stage declarations and
their boundary test. Both the checksum-pinned Tailscale stage and
upstream `runner-lockfile` guard are retained, including the upstream
`deps` inheritance. The Tailscale client stage is byte-identical. All
180 cases across five Docker/image contract suites pass locally, along
with server and UI typechecks. The maintained 543-input SDK check has no
drift. Runtime and lock pins are unchanged. Secret/PII and whitespace
scans pass.
- Seven OAuth regression cases failed before the fixes. The current-tab
test uses the real handoff helper, checks that the session is stored
before reauthentication navigation, mounts the shared return page,
resumes consent, and verifies handoff cleanup. Callback tests check
trusted saved-connection context, fixed safe outcomes, failure audit
metadata, denied membership and expired-state rejection, JSON API
errors, and no provider text in browser returns. UI retries require a
connection in the current company's catalog. All 80 OAuth
route/service/UI/shared-handoff cases pass; server and UI source/test
typechecks pass.
- [Server shard 4/12 on the prior
refresh](https://github.com/paperclipai/paperclip/actions/runs/43510727031/job/132180301436)
passed 2,784 cases and skipped 21, then failed a reassignment fixture's
`TRUNCATE companies CASCADE` cleanup with PostgreSQL `40P01`. Its
automatic retry found the surviving `REA` prefix. The unchanged test
passes all three cases locally. A controlled reader/cleanup overlap
reproduces the same deadlock, but the exact competing CI query was not
logged. Each case now has a distinct company prefix and identifier; the
owned database is removed at suite end. The three test bodies and
assertions are byte-identical. Eight local reassignment/lock-release
cases pass, and the [final server shard 4/12
passes](https://github.com/paperclipai/paperclip/actions/runs/43518584814/job/132207159392).
Production reassignment logic, deadlines, and retries are unchanged.
- The initial master integration passed 330 cases across 17 suites.
Coverage includes connector state, device discovery, network lifecycle,
pairing, routes, SSH binding and exports, environment persistence, and
connector UI. All three unpublished migrations were regenerated after
master's `0479_short_darkstar`: `0480_lazy_lethal_legion`,
`0481_round_pet_avengers`, and `0482_aberrant_rumiko_fujikawa`. Each SQL
body is byte-identical to its earlier unpublished migration. Generated
snapshots preserve existing master tables and add only the intended
Tailscale table and secret-purpose column. Exactly five snapshots
remain, 0478–0482. Numbering, safety, compatibility, and snapshot-drift
checks pass. The later Docker refresh did not alter this migration
sequence.
- Earlier attempts remain part of the verification record. On `1c020a0`,
Rust cache restoration and server shard 5/12 were interrupted by
shutdown signals, Runner shard 2/3 lost communication with no
retrievable log, and an amd64 provider download failed with npm
`ECONNRESET`. Same-head retries recovered, without establishing the
cause of the communication loss. Later superseded runs also had
cancellations with incomplete logs. These interrupted attempts are not
counted as passes. The final revision has fresh successful CI.
- Local platform limits remain recorded. The Kubernetes suite on macOS
passed 191 cases, failed 12 Linux `/proc/self/fd` transfer cases, and
skipped four Linux-only cases. Unchanged source reproduced the same 12
failures. An initial run also lacked provider dependencies until the
maintained preparation command installed them. Those local attempts are
not green qualification. Final Linux provider CI passes. Local Docker
qualification was unavailable; final image checks passed in CI.
- Before PR separation, a live tailnet/SSH Claude Code task completed on
Ubuntu ARM64 without provider login for the remote user. The selected
Paperclip subscription ran it, wrote the expected file, posted
completion, and marked the task done. File contents and private runtime
cleanup were checked. This is prior live evidence; the final revision
was not live-retested. Other providers have automated coverage only.
Hosted registration for unrelated tailnets and production concurrency
remain qualification limits below.
- The Superagent security and supply-chain reports remain neutral and
inconclusive. Earlier attempts to restart the failed security scan
returned HTTP 404. Supply-chain intelligence lacked coverage for
extracted pinned dependencies and the shared workflow reference. These
reports do not establish successful security qualification. The required
CI and review gates pass; this external coverage gap remains disclosed.

## Risks

- The Superagent security and supply-chain reports are inconclusive. The
security scan re-run request returned HTTP 404, and the supply-chain
scanner lacks coverage for the extracted dependencies and shared
workflow reference. These neutral results are not successful security
qualification.
- The server-owned secret purpose is internal database metadata, not an
HTTP-writable field. The generic secret service owns enforcement so
direct API reads and runtime projection share the same restriction.
Previously copied node state remains valid outside Paperclip until the
enrolled device is revoked in Tailscale; a local disconnect cannot stop
an external copy.
- Tailscale network configurations now require a connected Connector and
selected machine. The unreleased key-only path is removed; any saved
configuration from that path needs Connector setup.
- The connector remains a pilot. The local test used an OAuth app
registered in the consenting user's tailnet. Qualify hosted app
registration for unrelated customer tailnets before general
availability.
- This implementation supports one serving process per instance.
Concurrent replicas need exclusive ownership of the persisted network
identity before production rollout. Preserve the database and company
vault master key across restarts.
- Each machine-pairing preparation creates a new vault secret. Unused
generated keys follow the existing secret deletion lifecycle. A saved
SSH key remains available as an explicit choice.
- Apply additive migrations `0480_lazy_lethal_legion`,
`0481_round_pet_avengers`, and `0482_aberrant_rumiko_fujikawa` before
application rollout. The new credential version and digest columns are
nullable. Existing unpinned connections require fresh OAuth consent; the
migration deliberately does not trust or backfill a generic secret
value. Regenerate or renumber migrations if master advances the sequence
before merge.
- The target needs an SSH server, its chosen harness, and allowed
tailnet access. A reachable network does not grant SSH or provider
authorization. Pairing reads standard `/etc/ssh` host-public-key files;
custom host-key locations require manual known-hosts configuration.
- Roll back the application revision to remove new entry points.
Existing connection secrets and registered Tailscale nodes retain their
separate lifecycle. Do not drop the table or delete node state as part
of an application rollback.
- Production qualification still needs denied-policy, revocation,
cancellation, and deployment-overlap checks. Local Docker and a
production deployment were not performed.

## Model Used

OpenAI Codex, gpt-6.1-sol, xhigh reasoning, with local code execution
and tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used
- [x] I have checked ROADMAP.md and this extends existing connector and
environment features
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR using the feature request fields
- [x] I have not referenced internal issues, local URLs, or private
machine identifiers
- [x] My branch name describes the change
- [x] I have run focused tests locally and they pass
- [x] I have added tests for new behavior and failure paths
- [x] I have updated documentation
- [x] I have documented migration and runtime risks
- [x] All Paperclip CI gates are green
- [ ] Required code-owner approval received for the workflow change
- [x] Greptile Apex is 5/5 on the final head with no unresolved comments
- [x] I will address CI and review findings before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-13 15:15:43 -08:00
DottaandClaude Opus 5.5 2d1c2db64a feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use connected apps through MCP. Paperclip already hosts MCP
Apps views and speaks stateless MCP 2026-07-28.
> - The MCP Skills extension (SEP-2640, Final) lets an app publish
skills (instructions and files) next to its tools. Hugging Face already
serves 25.
> - Paperclip did not read them, so agents missed each app's own
workflow guidance.
> - Skills are a prompt-injection surface. The spec makes manifest
binding, verification, origin tagging and approval revocation host
requirements.
> - This pull request makes Paperclip a Skills host: discover, review,
verify, and deliver app skills to the agents that use the app.
> - The benefit is that connected apps can teach agents their workflows,
under board control.

## Linked Issues or Issue Description

**Problem**
Connected MCP servers can publish skills with the
`io.modelcontextprotocol/skills` extension. Paperclip ignored them. A
board could not see, review or approve them, and agents never received
them.

**Proposed solution**
Implement the host side of SEP-2640. List skills during catalog refresh,
record them per connection and company, bind board approval to the full
file manifest, and load approved skills into agent runs with verified,
lazily fetched content.

**Why it matters**
Apps ship workflow guidance with their tools. Without the extension,
each team must copy that guidance into company skills by hand, and it
goes stale.

Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work
this builds on).

## What Changed

- **Client capability.** Every stateless request declares
`io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The
capabilities that `server/discover` advertises are remembered beside the
protocol era.
- **Discovery.** Catalog refresh calls `skills/list` (paged) only when
the server declared the extension. A listing inside its `ttlMs` is
reused. A skills failure never fails the tool refresh. Nothing runs
while a run starts.
- **Schema.** New `tool_mcp_skills` (company, connection, skill URI;
frontmatter, sorted manifest, manifest digest, review state) and
`tool_mcp_skill_files` (verified bytes keyed by connection, URI and
digest). Both cascade with the connection.
- **Governance.** New skills wait for review. Approval binds to the
order-free digest of every `{uri, digest, size}`; a request for any
other digest gets 409. A changed manifest or frontmatter revokes
approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized
and invalid skills are declined with a reason. Approve and decline use
the company skill policy (`skills.install`, `external_package`,
`mcp-connection:<id>`).
- **Run time.** Approved skills of connections the agent can use load in
parallel with other skill preparation, inside a 3 s budget. `SKILL.md`
is read with the agent's grant and checked for size, SHA-256 and
frontmatter. A failure calls `skills/get` and takes the skill out of
use. Each skill becomes a per-run bundle on the connector skill path,
named in its app's namespace, tagged with its origin. Paperclip writes
the frontmatter: `allowed-tools` only if the board approved it.
- **Supporting files.** `GET
/api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed
files to an agent in an active run, through the skill's own connection.
The sandbox bridge allows the route.
- **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…`
(list, read instructions, approve, decline). The Skills page shows
"Skills from connected apps" with app, state, size, requested tools and
plain-language notes. It renders nothing when no app publishes skills.
- **Docs.** `doc/connections/MCP-SKILLS.md`, linked from
`doc/connections/README.md`.
- **Shared frontmatter parser.** It now decodes YAML single-quoted
scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and
would fail the frontmatter check; now all 25 match. The generated runner
copy is regenerated.

File size notes: `server/src/services/tool-gateway.ts` grows by about
200 lines. The run-time reads must use its grant resolution and
`callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting
the gateway is a separate refactor. `CompanySkills.tsx` and
`CompanySkills.production.tsx` gain 2–4 lines each; the section lives in
its own component.

## Verification

- `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake
stateless skills server and embedded Postgres. It covers: discovery only
after the server declares the extension; the capability in `_meta`;
`ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md`
at load; supporting files on read); a stale-digest approval refused; a
reorder that keeps approval; a changed or added file that revokes it;
dynamic skills declined; a same-size tampered `SKILL.md` rejected,
refreshed with `skills/get`, and taken out of use; mismatched
frontmatter rejected; reads outside the manifest refused; the same URI
and name from two servers kept apart; company scoping; a slow app
skipped within the budget and cached later; HTTP authorization (board
only, agent needs an active run, run header must match) and activity
logging.
- Red/green: with the digest check disabled, the tamper test fails; with
the manifest comparison disabled, the revocation test fails. Both pass
with the code.
- `server/src/__tests__/mcp-protocol-versions.test.ts`,
`mcp-apps.test.ts`, `connection-instructions.test.ts`,
`tool-gateway-discovery-http.test.ts`: 47 pass (two capability
assertions updated for the new extension).
- `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar
case; the runner contract test passes (23 tests).
- `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and
`ui/src/pages/CompanySkills.test.tsx` (44) pass.
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89)
passes.
- `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes
except the known missing generated Storybook catalog files.
- `pnpm verify` returned exit 2 (doc change is a global CI input);
required CI supplies broad coverage.
- Live check against `https://huggingface.co/mcp` without credentials,
through Paperclip's own services and the real egress guard: 25 skills
discovered as pending; three approved; cold load 135 ms, cached load 13
ms; a supporting file read through the manifest; the bundle tagged with
its origin.
- Storybook: `Agents and reusable capabilities/Skills and Skill
Studio/Connected app skills` (needs review, all states, tool request,
name collision, reading instructions).

## Risks

- **Migration:** two new tables only. No changes to existing tables.
Safe with the previous release running.
- **Run start:** companies without an approved app skill pay one indexed
read. With approved skills the wait is capped at 3 s; late or failing
skills are left out with a `skills.mcp_unavailable` run warning and
never fail the run.
- **Agent harness enforcement:** Paperclip cannot gate each
code-execution tool call an agent makes while following an app's skill.
Approval is explicit and per skill, the approval note says agents may
run commands the skill describes, and `allowed-tools` is never granted
without a separate tick.
- **Shared-home adapters** receive app skills like company skills. A
revoked skill leaves on the next skill sync.
- Not in this PR: per-connection opt-in for dynamic skills; serving
Paperclip's own skills from its public MCP endpoint;
`resources/directory/read`; app skills on an agent's skills tab.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs, live HTTP checks).
Extended reasoning enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:18:06 -06:00
DottaandClaude Opus 5.5 8ff45c0fa9 feat(connections): render MCP Apps views in the task chat, starting with Circleback (#21119)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents reach external apps through MCP connections. Paperclip
records each call and shows it as a tool row in the task chat.
> - MCP Apps (SEP-1865, extension `io.modelcontextprotocol/ui`, spec
2026-01-26) lets an MCP server attach an interactive `ui://` view to a
tool. Claude, Claude Code and VS Code render these views in chat.
Circleback uses them to show meeting cards.
> - Paperclip dropped the tool's `_meta.ui` at discovery and the
result's `_meta` at call time. Its gateway reported MCP App UI as
"unsupported". Circleback was reachable only through Composio, whose
toolkit has no `Display*` tools.
> - This pull request makes Paperclip an MCP Apps host. It also adds
Circleback as a native catalog connection.
> - The benefit is that a Circleback recap in a task shows the same
meeting cards as in Claude, and any other MCP server with views works
the same way.

## Linked Issues or Issue Description

**Subsystem affected**
Connections and apps (remote MCP gateway) and the task chat.

**Problem or motivation**
An agent that uses Circleback in Paperclip can only return text. The
same tools in Claude Code show interactive meeting cards
(`DisplayMeetings`). Paperclip discards the MCP Apps metadata, so it
cannot show them.

**Proposed solution**
Paperclip implements the host side of MCP Apps end to end. It keeps the
tool's view declaration and records each call that has a view. It
renders the view under the tool row in a sandboxed iframe that uses the
MCP Apps postMessage protocol. Circleback becomes a native catalog
connection, so its `Display*` tools are available.

**Alternatives considered**
Composio's `circleback_mcp` toolkit has 11 read tools and no `Display*`
tools, and its meta-tools return plain JSON, so it cannot carry views.
The spec's double-iframe sandbox proxy on a separate origin needs a
second origin that Paperclip does not have. A single opaque-origin
`srcdoc` frame with an injected CSP gives the same isolation.

**Roadmap alignment**
This extends the existing Apps catalog and MCP gateway. It adds no new
integration substrate (decision D4 in doc/connections/README.md).

**Additional context**
The standard, the design and the testing plan are in
`doc/connections/MCP-APPS.md`.

## What Changed

- **Shared:** `packages/shared/src/mcp-apps.ts` adds MCP Apps types,
`_meta.ui` and resource parsing (current and legacy keys), and a CSP
builder that keeps only well-formed `https` origins.
- **DB:** migration `0471` (`online_compatible`) adds
`tool_catalog_entries.ui_meta` and a new `tool_mcp_app_views` table.
- **Server, catalog:** discovery keeps each tool's `_meta.ui`. It is not
part of the version hash, so a view never quarantines a reviewed tool.
- **Server, gateway:**
- Paperclip advertises the MCP Apps client capability when it
initializes a session.
  - App-only tools (`visibility: ["app"]`) are hidden from agents.
- The result `_meta` is kept for the view and is not sent to the agent.
- Each successful call to a tool with a view records a view row.
Recording is best effort.
- The gateway's `initialize` now reports `paperclip/mcp-app-ui:
host_rendered`.
- The context protocol (`resources/read`) now uses the injected test
transport, like the other remote calls.
- **Server, routes:** `server/src/routes/mcp-apps.ts`
- `GET /api/issues/:issueId/mcp-app-views` lists a task's views. It
needs task read access.
- `GET /api/issues/:issueId/mcp-app-views/:viewId` returns the view
document, read live through the agent's connection.
- `POST /api/issues/:issueId/mcp-app-views/:viewId/tool-calls` runs the
view's `tools/call`. It needs `tools:use` and reaches only tools of the
same connection that allow `app`. It uses the connection test-call path,
so policy, Ask first and audit apply.
- **UI:** `ui/src/components/mcp-app/`
- The host protocol: `ui/initialize` with theme, style variables and
size limits; tool-input; tool-result; size-changed; `tools/call`;
`ui/open-link` (http and https only); teardown.
- An opaque-origin iframe (`allow-scripts`, never `allow-same-origin`)
with the CSP placed before the view's markup.
- The views of a run render with that run's reply, in the live turn and
in settled history. They are keyed by the reply's run, so history needs
no transcript.
- **Circleback:** a native catalog app (`mcp-oauth`, DCR, `user
user:write`) with:
  - the official logo;
- a reviewed risk classifier (Share* tools are writes, Delete* tools are
destructive, Display* tools are reads);
  - warnings in the sign-in step;
  - `doc/connections/CIRCLEBACK.md`.
  
Search now prefers a native app over an aggregator listing named "X
MCP".
- **Storybook:** `Tasks/MCP App views` shows the tool row with its view,
the ready state (Refresh calls a tool), loading, view unavailable, and a
tool call that needs approval.

## Verification

- `pnpm exec vitest run packages/shared/src/mcp-apps.test.ts
ui/src/components/mcp-app/`: 19 passed.
- `pnpm exec vitest run server/src/__tests__/mcp-apps.test.ts`: 3
passed, on embedded Postgres. The test connects Circleback through the
catalog with DCR OAuth against a fake Circleback MCP server. It checks:
  - `ui_meta` is stored;
  - an app-only tool is hidden from the agent;
- an agent call records one view with arguments, structured content and
`_meta`;
  - a call without a view records nothing;
  - the routes serve the view's HTML and CSP;
  - the view's tool call works and a model-only tool is refused;
  - another task, another company and an agent actor are refused.
- Circleback connector: 13 focused files, 645 tests passed. These are
the app definitions, search, aggregator fallback, connection intents,
tool policy, `AppsConnect` and `Browse`.
- Typecheck passed for `@paperclipai/shared`, `@paperclipai/server` and
`@paperclipai/ui`. `node scripts/storybook-catalog.mjs` and the
migration compatibility check also passed.
- Storybook check in the browser:
- In both themes, the view runs the handshake, renders the meeting cards
from `structuredContent`, picks up the theme variables, and resizes to
its content.
  - Refresh sends `tools/call` and re-renders the cards.
- **Live Circleback end to end** on a local instance, with a real
Circleback account:
- Connected through the catalog entry with Circleback's dynamic client
registration and OAuth.
  - Discovery stored the real view declarations:
    - `DisplayMeetings` uses `ui://circleback/meetings.v2.html`.
    - `DisplayActionItems` and `DisplayCalendarEvents` also have views.
- `ListAssigneeOptions` and `SetCalendarEventRecording` are app-only
helpers with no view. They are hidden from agents.
  - A `claude_local` agent called `DisplayMeetings` on two tasks.
- The real 3.5 MB Circleback view rendered the meeting cards with the
agent's reply.
- Clicking a card opened that meeting in Circleback through
`ui/open-link`.
- The live run found two bugs, both fixed in this PR, each with a
regression test:
    - The 1 MB tool-result limit blocked the view document.
    - Views were hidden inside the folded tool history.

## Risks

- **New migration:** a new table and a nullable column. It is
`online_compatible`.
- **Existing connections:** tools get `ui_meta` on the next catalog
refresh. Until then, no view shows and the tool row is unchanged.
- **Untrusted provider HTML:** the view runs as provider HTML in an
opaque-origin sandbox. It cannot reach Paperclip's session, DOM or API.
Its network access is limited to the origins its server declared. A
view's tool calls use the viewer's permissions and the normal policy and
Ask-first path.
- **Agent tool lists:** app-only tools are removed from them. No current
provider marks tools app-only, so no listed tool disappears.
- **Search:** the change that prefers a native app over an "X MCP"
aggregator listing affects every app.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
subagents (one subagent authored the Circleback catalog connector).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:28:05 -06:00
DottaandClaude Opus 5.5 b853cc28ae perf(server): reach readiness without waiting for backlog-sized recovery passes (#21014)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server process starts, recovers saved work, and then reports
ready on `/api/ready`. The deploy health gate waits for that signal.
> - On a busy staging workspace, one boot took 85 s to bind the listener
and 4.4 minutes to report ready. Small, idle stacks take 20–90 s.
> - Two parts of that time do not need to block readiness. The server
published several hundred UI assets one at a time before it bound the
listener. After the listener bound, it waited for the first pass of each
periodic recovery queue, and those passes grow with the backlog.
> - The volume-free data gate, which can refuse a start, ran last. By
then the server had already dispatched queued runs that a refusal would
abandon.
> - This pull request moves the refusal gate to the start, publishes the
assets 8 at a time with replayable bodies, and runs the backlog-sized
first passes after readiness, in their original order.
> - The benefit is a faster time from container start to ready, and no
work admitted before a gate that can refuse to start.

## Linked Issues or Issue Description

No public issue exists. This is the problem description (enhancement
template).

**Is your enhancement related to a problem? Please describe.**
Server boot time grows with the saved backlog. These timestamps come
from the log of one staging boot (release 87fe4d9793). The container
started at 01:16:56. The listener bound at 01:18:21, 85 s after start.
The fresh queued-run dispatch completed at 01:18:25. The full queued-run
dispatch pass completed at 01:20:14. The volume-free data gate ran at
01:21:20. Readiness came about 4.4 minutes after start. During startup
the log showed hundreds of `An error was encountered in a non-retryable
streaming request.` lines.

**Describe the solution you'd like**
Readiness waits only for the correctness gates and for the adoption of
old runs. Work that the periodic queues already retry runs after
readiness. A gate that can refuse to start runs before the server admits
any work.

**Describe alternatives you've considered**
- Move all post-listen recovery to the background. We rejected this.
Native recovery can fail closed and must still fail the start. It also
adopts old runs.
- Skip asset re-publication when the image is unchanged. We rejected
this. The collision check (never overwrite different bytes) needs a read
per asset.

**Additional context**
Startup map (line numbers are from `origin/master` 82e0393598):
- Blocks the listener: migrations and the embedded database
(`server/src/index.ts:369-741`); config and identity (`:753-774`);
backfills (`:821-833`, where
`sweepSupersededPendingRequestConfirmations` scans every pending
confirmation); `qualifyCloudConfigurationForCurrentImage` (`:956`,
synchronous package hashing); `createApp` (`:964`), which awaited serial
UI asset publication (`server/src/app.ts:1110`,
`server/src/retained-ui-assets.ts:69-105`).
- Blocks readiness after the listener: first passes of tool reviews,
gateway sessions and delivery workers (`index.ts:1529-1558`); the
terminal workspace reaper (`:1652`); native recovery and hot-restart
adoption (`:1707-1779`); the first passes of 17 recovery queues
(`:1782-1890`), including the full queued-run dispatch (`:1805-1811`);
backup registration (`:1979-1999`); the data gate and package
qualification (`:2020-2026`).
- Grows with the backlog: the queue first passes (queued runs, pending
deliveries, watchdogs, native sessions and results), the terminal
workspace sweep, the native recovery scan, and the confirmation sweep.
UI asset publication grows with the build (hundreds of assets), not the
backlog.

## What Changed

- **Volume-free data gate runs first.** `assertEphemeralServerData` now
runs right after `claimVolumeFreeControllerState` and the runtime config
load. That is before `createApp`, before the listener binds, and before
any queued-run dispatch. It used to run at the end of startup. The
package qualification gate stays at the end, because bundled plugin
installs can change its inputs.
- **UI asset publication is faster but still a gate before work.**
`createApp` starts the publication while it builds its routes. It awaits
the result before its first job, queue or run starts, which is the same
point as on master. A colliding build still refuses to start before it
admits any work (Greptile review).
- **Publication sends Buffer bodies, 8 at a time.** Each asset is read
once into a Buffer. The S3 client can replay a Buffer body; it cannot
replay a stream. The new code sends no stream bodies, so it no longer
reaches the SDK path that logs "non-retryable streaming request". After
the first failure, no new assets start. A collision found on one asset
outranks a storage outage on another (Greptile review).
- **Backlog-sized first passes run after readiness.** The 17 queue
registrations after native recovery (residual recovery through
stopped-instruction collection) now run in a tracked background
sequence, in the same order. Each pass is isolated. A pass that fails is
logged with its name, and the later passes, the housekeeping and the
scheduler still run. A queue whose first pass failed stays registered,
and the coordinator retries it. Native recovery and hot-restart adoption
still complete before ready. Idle sleep stays blocked while the sequence
runs, and shutdown waits for it like other scheduler work.
- **Queued turns keep the right status.** A new flag,
`startupQueuedRunPassOutstanding()`, stays true until the first full
queued-run pass ends. Until then the execution projection still shows
"Waiting for server restart". The flag clears even when that pass fails,
because the dispatch queue then owns those turns.
- **Startup timeline log.** At readiness the server logs the duration of
each startup step (`startup timeline at readiness`). It logs again when
the background passes finish. No OpenTelemetry collector is needed.
- **Benchmark.** A new script,
`scripts/benchmark-retained-ui-assets.ts`, measures publication against
storage with a fixed round-trip latency. It can fail when time exceeds a
budget (`RETAINED_UI_BENCH_MAX_MS`).

Why each moved step is safe:
- **Data gate (earlier).** Its inputs are the migrated database and the
owner marker. `claimVolumeFreeControllerState` still runs first.
Interaction with #20952 and #20990, which are both in this branch: those
changes made the gate tolerate
`native_harness_backups_require_migration` and
`local_instruction_copies_require_settlement` when the filesystem
carries the volume-free owner marker. That covered the case where this
boot's own dispatch and recovery wrote such run state before the late
gate ran. With the gate first, no work of this boot can write run state
before the gate: the app, its queues, run admission, native recovery and
dispatch all start after it. A test checks this order. So the gate sees
only state that a previous process left, and the owner-marker tolerance
still decides that state. This is strictly safer. Nothing relied on the
gate running after recovery: the only work that could clear a blocker
was background work that the old position never awaited. Effect: a
refused start no longer leaves dispatched runs behind.
- **UI asset publication.** The await point is unchanged, so the
gate-before-work order matches master. Only the time it takes changes.
- **Recovery queue first passes (after ready).** Each queue owns its
work and retries it on deadlines, commit wakes, and intervals. A first
pass that ends after ready behaves like a later pass. Interrupted runs
are still classified and adopted by native recovery before ready, and
they resume without a Retry. No wait is added before a turn starts:
turns can start sooner. The HTTP API already served requests during
recovery, because readiness never gated requests. Observable changes:
- The deploy gate passes earlier. A queue that throws while it registers
is now logged instead of failing the start. Each queue's `ready` already
absorbed run errors.
- `startup queued-run dispatch pass complete` can now appear after
`Server startup recovery complete`.

Left alone (safety unclear or not measured):
- The first passes of tool review recovery, gateway sessions, the
delivery workers, the GitHub workers, and the terminal workspace reaper.
These run before native recovery, and I did not prove the ordering safe.
- `qualifyCloudConfigurationForCurrentImage` before `createApp`. It
hashes plugin packages synchronously; its result is only a warning
there.
- The confirmation sweep and principal access backfill before the
listener. They scan whole tables.

## Verification

- Red/green: with `server/src/index.ts`, `app.ts` and
`retained-ui-assets.ts` reverted to master, these new tests fail. With
the change they pass:
  - `server-startup-feedback-export.test.ts`:
- refuses ineligible volume-free data before the app, the listener or
any queued-run dispatch
- reports ready while the first queued-run dispatch pass still runs,
then finishes it and the passes after it
- runs every deferred recovery pass after readiness when one throws,
logs it, and keeps the failed queue registered (fails on the first
revision of this PR, passes now)
  - `retained-ui-assets.test.ts`:
    - re-publishes with bounded parallel, replayable writes
- refuses a colliding build even when another asset's storage outage
fails first (fails with `failure ??=`, passes now)
- Regression guards that also pass: "checks volume-free data once,
before any of this boot's own startup work can write run state" and
"stops starting new assets after one collides".
- Focused suites pass after merging `origin/master`:
`server-startup-feedback-export` (131 tests), `retained-ui-assets`,
`execution-projection`, `liveness`, `heartbeat-startup-fresh-dispatch`,
`ephemeral-server-startup-run-state`, `ephemeral-server-data`,
`ephemeral-migration`, `ephemeral-server-contract` and
`static-index-html`. `tsc --noEmit -p server/tsconfig.json` is clean.
- Benchmark (measured locally, 400 synthetic assets, 25 ms simulated
round trip): `pnpm --filter @paperclipai/server exec tsx
../scripts/benchmark-retained-ui-assets.ts`

  | | serial (before) | 8 in flight (after) |
  | --- | --- | --- |
  | first publish | 10.5 s | 1.3 s |
  | restart re-publish | 21.3 s | 2.7 s |

This time is still before the listener, as on master, but it is about 8
times shorter and overlaps route construction.
- Local boot (measured, empty instance, embedded PostgreSQL, 3 boots
each, a shared and noisy machine):
  - listener to ready: before 385–427 ms; after 212–303 ms.
- Spawn to listener was 5.5–12.8 s in both cases. Module load and the
embedded database dominate it, and it does not differ.
- Not measured: the hosted saving. From the staging log, the inferred
saving is the queued-run pass and the passes after it (about 175 s:
01:18:25 to 01:21:20), plus most of the serial asset publication before
the listener. Confirm with the new `startup timeline at readiness` log
line after deploy.

## Risks

- **Medium: readiness semantics change.** The deploy gate now passes
before the first recovery passes finish. The old server can leave
earlier while the new server still dispatches its backlog. Queued turns
show "Waiting for server restart" until the dispatch pass ends, and the
queues retry on their own.
- **Low: earlier data gate.** It can refuse a start that the old order
would have admitted only if background work cleared a blocker first.
That was a race before this change.
- **Low: parallel publication against throttling storage.** Eight
requests at a time, with the existing per-asset backoff (0.5–8 s). A
persistent outage still lets the server start on local files.
- No migrations. No API contract changes.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edit, subagents for code search). Extended
reasoning on.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Related: #21010 (retry a throttled asset upload).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 05:35:47 -06:00
DottaandClaude Fable 5.2 fd5ee7e385 fix(native): continue a routine assignment or chat message whose remote Runner an upgrade lost mid-turn (#21052)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs on remote sandboxes (Daytona) use a detached Runner.
When the server is replaced during an upgrade, the new server reattaches
to that Runner
> - On staging, upgrades every 15–40 minutes lose the in-flight Runner.
The same-run reattachment then finds `process_verification` /
`process_not_running`, the run fails with
`runner_remote_process_identity_unavailable`, and the task is blocked
> - #21039 made these runs continue once in a fresh session (the turn is
recorded as having an unknown outcome and is not replayed). It covers
user wakes and automatic retries (`retryOfRunId`) only
> - A routine's system assignment (`issue_assigned`, requested by the
system) and an external chat message wake have no `retryOfRunId`. They
got `not_user_request` and stayed blocked for a person
> - This pull request treats those two wakes as owed task work, so they
continue the same way once the Runner is proven gone
> - The benefit is that an upgrade no longer leaves these tasks waiting
for a Retry click

## Linked Issues or Issue Description

Refs #21039, #21023, #20923

**What happened?**
Staging replaced its server at about 07:10Z. 17 remote native runs were
in flight. Each new controller reached its Runner 114–152 s after
detach. Recovery failed with "PRP command session.snapshot timed out".
The same-run reattachment then found the Runner process gone, and each
run failed with `native_continuation_requires_reconciliation`. #21039
continued 14 of the 17 automatically. The other 3 were a routine's
assignment (two runs) and an external chat message. They have no
`retryOfRunId`, so `requestInterruptedNativeTurnRetry` returned
`not_user_request`. Those tasks are still blocked with no retry.

**What did you expect to happen?**
The Runner is proven gone, so nothing can still run the turn. The run
should continue once in a fresh session, the same as an automatic retry
of owed work.

**Steps to reproduce**
1. Start a remote native run from a routine assignment, or from an
external chat message.
2. Replace the server while the provider turn is active, so that the
reattachment finds `process_not_running`.
3. The run fails and the task stays blocked. No automatic Retry is
requested.

## What Changed

- `server/src/services/heartbeat/restart-interrupted-turn.ts`: after the
Runner is proven gone, a system wake also continues when it is a system
assignment (`issue_assigned`), or when it is a comment or external chat
message wake whose message is still on the task (not deleted, not
written by a run). It runs under the run's responsible user, at most
once per chain of runs, like an automatic retry already does. A linked
person's external chat message continues under that person when the
message is still on the task. Monitors and other system wakes are
unchanged.
- The same file and `server/src/app.ts`: the hook now passes the chat
service's Retry callback (`prepareFailedChatRunRetry`, registered at app
startup next to the budget-resume callback). A task in a connected
conversation whose endpoint restricts external execution then continues
through the chat Retry path. Before, `requestTaskRetry` refused it
(`retry_refused`) and the task stayed blocked.
- `server/src/services/runner-lost-mid-turn-continuation.test.ts`:
production-shape cases for a system assignment, an unlinked external
chat sender, and a linked person's chat message. Negative cases: a
deleted message, a message written by a run, a deleted linked message,
and a system assignment whose Runner was not proven gone.
- `doc/execution-semantics.md`: describes the rule.

## Verification

- Red on `origin/master` with only the new tests: `cd server && npx
vitest run src/services/runner-lost-mid-turn-continuation.test.ts` fails
4 of 16. The three new continuation cases return `{ requested: false }`.
The deleted linked-message case returns `not_user_request` instead of
`message_unavailable`.
- Green with the fix: 16 of 16. Two more cases bind the task to a real
restricted Slack endpoint and conversation. With the chat callback
registered, the run continues and the Retry carries the chat action (18
of 18). Without the callback, the result is `retry_refused`, which was
the behavior before for every connected conversation.
- Related suites pass: `runner-lost-mid-turn-continuation.test.ts`,
`heartbeat-native-remote-failure-retention.test.ts`,
`workspace-recovery-retry-copyback.test.ts`,
`governed-wait-runner-loss.test.ts` (116 tests).
- `cd server && npx tsc --noEmit` passes.
- Production evidence was read from staging run events through the
read-only API: 3 of 17 losses at the 07:10Z swap took this path. After
this change the next post-upgrade audit should show an automatic
`task_retry` for every `native_continuation_requires_reconciliation` run
whose refusal is `process_not_running`, including routine assignments
and chat messages.

## Risks

- Low. The change applies only after the Runner process is proven gone
(`process_verification` / `process_not_running`), so no Runner can still
run the turn. The turn is not replayed. The fresh session continues over
the saved work, as in #21039.
- The task can contain partial tool work from the lost turn. This is the
same accepted risk as #21039.
- Previous-release story: no persisted shape changes. The rule reads the
wake's reason and comment id, which earlier releases already write. A
run an older release already settled keeps its visible Retry.
- This does not stop the Runner loss itself. That is a separate fix.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude (Anthropic) via Claude Code, with tool use (shell, file edits,
test runs, read-only staging API reads). Model ID as attributed in the
commit trailer: Claude Fable 5.2.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 02:11:28 -06:00
DottaandClaude Fable 5.2 82e0393598 fix(ui-assets): retry a throttled asset upload and never fail startup on a storage outage (#21010)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A hosted Paperclip server can run volume-free. Before it serves, it
publishes its UI build assets to shared object storage, so an older
browser tab can still load its chunks after the old container is gone
> - A build has several hundred assets. The server uploads them one
after another at startup
> - Object storage throttles a burst with `SlowDown` (HTTP 503). The
upload body is a stream, so the storage client cannot replay the
request, and the error leaves `startServer`
> - The process exits, the host restarts it, and the next start uploads
the same assets again. A workspace stayed down while four starts in a
row failed this way
> - This pull request retries a throttled asset and lets the server
start when storage stays unavailable
> - The benefit is that a storage slowdown cannot keep a server from
starting

## Linked Issues or Issue Description

Refs #20990, #20952 (other startup failures found on the same hosted
workspace).

**What happened?**

A hosted volume-free workspace received a new release. Each start logged
many `An error was encountered in a non-retryable streaming request.`
lines and then `Paperclip server failed to start` with `SlowDown: Please
reduce your request rate.` for a key under `ui-assets/v1/`. The four
starts failed after 44, 42, 79 and 90 seconds, each on a later asset
name.

**Expected behavior**

The server waits and repeats a throttled upload. If shared storage stays
unavailable, the server starts and serves its build from local files.

**Steps to reproduce**

1. Run a server with `PAPERCLIP_SHARED_UI_ASSETS=true` and S3-compatible
storage.
2. Make the storage answer `SlowDown` (HTTP 503) to one `PutObject`
during startup.
3. The server exits with `Paperclip server failed to start`.

**Paperclip version or commit**

`52cdf76bf3e0cdf29037b63bf0f5e99aa464eab7`

## What Changed

- `retainedUiAssets.publish`: each asset is published in a retry loop. A
throttled or server-side storage failure waits (0.5 s to 8 s, seven
waits at most) and repeats the asset with a fresh stream.
- `isTransientStorageError`: recognizes `SlowDown`, throttling, request
timeout and HTTP 429/500/502/503/504 answers.
- `app.ts`: when publication still fails with a transient storage error,
the server logs an error and continues. Other failures, such as a name
collision with different bytes, are still fatal.
- Tests for the retry, for the give-up path, and for the error
classification.

## Verification

- `npx vitest run src/__tests__/retained-ui-assets.test.ts` in
`server/`: 4 tests passed. The two new tests fail on `master` because
`publish` rejects on the first `SlowDown` and takes no `sleep` option.
- Not run locally: the full suite. CI supplies that coverage.
- Not verified: a live start against throttling storage.

## Risks

- A start can take up to about 31 seconds longer for each throttled
asset before it gives up.
- If storage stays unavailable, the server starts without its assets in
shared storage. An older browser tab can then fail to load a chunk after
the old container is gone. The server logs this at error level.
- Collision detection is unchanged.

## Model Used

Claude Fable 5.2 (`claude-fable-5-2`), through Claude Code with tool use
and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 21:21:27 -06:00
DottaandClaude Opus 5.5 a9a08d8169 fix(plugins): retry a plugin activation whose worker initialize timed out (#20982)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Plugins, sandbox providers such as Daytona among them, run as worker
processes that the server activates at boot. Activation sends the worker
an `initialize` RPC with a 15-second timeout.
> - When `initialize` timed out, the loader marked the plugin `error`
for the life of the server process. Only the next boot or an operator
retried it.
> - For a sandbox provider, that means every Daytona run on that server
fails. New runs fail with `lease_acquire_failed … plugin is currently
error`. Reattachments fail with "The exact sandbox lease cannot be
resumed by its verified provider". Retry hits the same dead plugin.
> - A worker that is slow or stalled for a moment at boot is transient,
not a broken plugin.
> - This pull request retries an activation whose `initialize` timed
out, with a bounded backoff. Meanwhile the server keeps serving, and
runs that need the plugin see a restarting worker, which setup retries.
Deterministic failures stay a permanent error.
> - The benefit is that a short plugin stall at boot no longer disables
a provider until the next deploy.

## Linked Issues or Issue Description

Refs #20966, #20857

**What happened?**
Chaos wave `w-20261111T232850-s9521` (cell CR4, first version) stopped
the replacement server's Daytona plugin worker while the server was
activating it. The worker's `initialize` RPC timed out after 15 s
(`INITIALIZE_TIMEOUT_MS`). `plugin-loader.ts` then called
`lifecycleManager.markError`, and the plugin stayed `error`. Task t03's
reattachment failed in `resumeRunLease`, and its Retry did not start a
new run. Task t04 failed 18 s after launch with `lease_acquire_failed`
("…plugin is currently error"), before its own fault.

**Expected behavior**
An activation timeout is retried, and the plugin ends `ready` when the
worker answers. Runs that need the plugin meanwhile are retried within
their bounded setup budget. A bad manifest, a worker that exits during
`initialize`, or a version mismatch is still a permanent error.

**Steps to reproduce**
1. Start the server.
2. Stop (SIGSTOP) a plugin worker for more than 15 s while the server
activates it.
3. Resume the worker. The plugin stays `error` until the next boot.

**Paperclip version or commit**
master at dfd2a6efca.

## What Changed

- `plugin-worker-manager.ts`: the "Worker initialize failed" error keeps
its cause, the `JsonRpcCallError` with `PLUGIN_RPC_ERROR_CODES.TIMEOUT`.
- `plugin-loader.ts`, `activatePlugin`:
- An activation whose error chain has the TIMEOUT code is retried after
5, 15, 45 and 120 s (`activationTimeoutRetryDelaysMs`).
- Before each retry, the loader tears down the partial runtime and keeps
the plugin `ready`. It sets `lastError` to "Activation timed out;
retrying in Ns (retry n of 4): …" so operators can see the state.
- A retry stops when the plugin is no longer `ready` or its worker
already runs. Shutdown clears pending retries.
- After the last retry, the loader calls `markError` with "Activation
timed out after 4 retries: …". Any other failure calls `markError` with
"Activation failed: …" at once, as before.
- `pluginLoader.requestActivationTimeoutRecovery(plugin)`: re-activates
a plugin in `error` whose `lastError` starts with "Activation timed
out", at most once per minute.
- `app.ts` exposes it through
`workerManager.requestActivationTimeoutRecovery`.
- `environment-runtime.ts`:
- Lease acquisition, for both pinned and key-resolved providers, calls
`requestActivationTimeoutRecovery` for such a plugin and reports
`worker_unavailable` (`PluginWorkerUnavailableError`, which setup
retries) instead of the terminal "plugin is currently error".
- `resumeRunLease` reports a stopped worker as
`PluginWorkerUnavailableError` when the plugin is `ready`, or is
re-activating from a timeout, instead of the generic "cannot be resumed
by its verified provider".
- This covers every plugin kind: activation retry is in the shared
loader. The on-demand path is in the sandbox-provider resolution, which
is the path a run waits on.
- `doc/plugins/PLUGIN_SPEC.md` §12 describes the rule.

## Verification

- Red on master:
`server/src/__tests__/plugin-loader-activation-timeout.test.ts` uses a
real PostgreSQL plugin registry and a bundled image fixture, and all 3
of its tests fail on master. On master, "initialize times out once, then
succeeds" stays `error` and the worker starts only once.
- Green, all 3 pass:
- A timeout followed by success ends `ready`, with a running worker and
no `markError`.
- Timeouts past the bound end `error` with the "Activation timed out
after 2 retries" message, and an on-demand request re-activates the
plugin to `ready`.
- A worker crash during `initialize` is a permanent error, with no retry
and no on-demand re-activation.
- `environment-runtime.test.ts` has new tests:
- A sandbox plugin errored by activation timeouts gives
`PluginWorkerUnavailableError` and requests re-activation. When no
re-activation is in flight, the error stays terminal.
- `resumeRunLease` on a stopped worker of a `ready` or re-activating
plugin gives `PluginWorkerUnavailableError` and sends no provider call.
- These files pass: `environment-runtime`,
`environment-run-orchestrator`, `plugin-loader-error-retry`,
`plugin-environment-driver-ready-recovery`,
`bundled-plugin-registration`, `plugin-lifecycle-restart`,
`plugin-lifecycle-delivery` and `plugin-dev-watcher`.
- `tsc --noEmit` passes for `server`.
- Chaos: cell CR6 (a plugin stalled during activation at boot) runs
before and after this change. The results are in a comment.

## Risks

- A plugin whose `initialize` keeps timing out now takes about 3 minutes
to reach `error`, instead of 15 s. During that time runs that need it
see a restarting worker and are retried within their setup budget.
- On-demand re-activation is limited to one attempt a minute per plugin,
so a stuck plugin cannot crash-loop.
- Nothing changes for deterministic activation failures.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edit, code search). Extended reasoning was
on.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:18:47 -06:00
8818402e7a feat(inbox): route requests to people and record personal outcomes (#20959)
## Thinking Path

> - Paperclip helps people manage AI agents and their work.
> - The inbox collects requests that need a human decision.
> - Teams need to know who owns each request and who may act on it.
> - A shared close operation must differ from hiding a card for one
person.
> - This pull request adds personal routing and durable outcomes to the
existing attention service.
> - People can claim or transfer work and inspect what another person
did for them.
> - Domain permissions still decide who may resolve each request.

## Linked Issues or Issue Description

**Subsystem affected**

Server, shared contracts, and database. This is the routing foundation
for a stacked inbox UI change.

**Problem or motivation**

A company-wide attention feed does not give teams a clear personal
queue. Archive, withdrawal, transfer, and decisions by another person
need distinct scopes and records.

**Proposed solution**

Resolve a request audience from its addressee, task participants, and
configured owners. Provide personal, unassigned, company, all-assignee,
and archived views. Add claim, transfer, personal archive and snooze,
shared withdrawal, receipts, and personal history.

**Alternatives considered**

A single shared inbox leaves ownership and personal hiding ambiguous.
The service keeps one canonical request and projects its audience
instead of copying requests for each person.

**Roadmap alignment**

This extends the existing Multiple Human Users and Agent Reviews and
Approvals capabilities in ROADMAP.md.

**Additional context**

Refs #19363 and #19520 for the current inbox navigation and confirmation
replacement rules. Refs #20268 for related actionable-request work. That
PR filters actionability; this stack adds audience routing and personal
outcomes.

This is part 1 of a two-PR stack. The inbox UI PR #20961 targets this
branch.

## What Changed

- Add nullable company and project decision owners and a company budget
owner.
- Add audience views and personal badge counts to the canonical
attention projections.
- Add claim, transfer, withdrawal, personal archive, snooze, and restore
operations with permission checks and activity records. Lock addressed
transfers to the current addressee or routing administrators, and
validate the recipient against canonical resolution policies.
- Add receipts and scoped history for work handled by another person.
- Keep closed-task requests and replaced confirmations out of current
cards.
- Add an idempotent migration and a concurrent receipt-history index.
- Update typed fixtures so the existing UI remains compatible with the
new contracts.

## Verification

- Server layered typechecking passes, including all three test-tree
partitions.
- Seven routing and interaction-resolution suites pass all 249 tests
against a dedicated external PostgreSQL test database on `4caecc21d8`.
Six new regressions failed before the repair; they cover takeover
followed by approval, legitimate delegation, excluded creators/review
requesters, and unauthorized claims.
- Prior focused verification covers private approval history and
receipts, transfer visibility, audience escalation, withdrawal
governance, hidden-card counts, per-recipient confirmation replacement,
and SDK source contracts.
- A synthetic three-person company passed 37 routing and lifecycle
acceptance checks. The UI stack adds browser acceptance for the shared
controls.
- The complete [GitHub Actions CI
matrix](https://github.com/paperclipai/paperclip/actions/runs/43134997331)
passes on `4caecc21d891677323a7335f23b1a015cce24c13` (Linux/Node 24:
typecheck, build, general and serialized tests, Runner support, browser
shards, and aggregate gates). Greptile is 5/5 on that exact head with no
open findings. The fresh security scan passes after explicitly
rechecking the addressee-transfer repair.
- `pnpm verify` reports exit 2 because a CI-global script changed; full
CI supplies that broader coverage. Embedded PostgreSQL could not start
on this Mac because shared-memory slots were exhausted, so local
database checks use a separate external test database. Linux CI covers
the embedded setup.

## Risks

- Personal badge counts change to the viewer's assigned requests.
Accessible feed totals and task membership keep their existing scope.
- The migration adds nullable owner fields and a dismissal fingerprint.
Its index runs outside a transaction. The migration is classified and
rerunnable.
- Routing changes the addressee used by resolution authorization. The
current addressee or a routing administrator must authorize an addressed
transfer; the recipient must satisfy writer access, canonical resolver
exclusions, and task review policy under row locks.
- The shared validator export barrel crosses 1,000 lines because it
exports the new attention schemas. Schema implementation stays in the
dedicated attention validator module.

## Model Used

OpenAI GPT-6 through Codex. Used repository tools, code execution, and
browser checks. This session does not expose a more specific runtime
model ID or its context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
(OpenAPI contracts and docs/api/attention.md)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-11 17:49:39 -08:00
Nicky LeachandPaperclip 38501b1170 Add turn-scoped attachment downloads to native Runner (#20956)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Native agents open input images with a file reader.
> - Plugin Runner tasks use an authenticated Runner connection.
> - Runner needs a way to fetch an authorized file from the tenant app.
> - The tenant grants access to the exact attachment for the active run.
> - Runner downloads and verifies it before it returns a local path.

## Linked Issues or Issue Description

Refs #19740. Builds on the attachment authorization and command
transport added in #20939.

## What Changed

- Add the PRP v3 `runner.attachment.download` command for native
workspace providers.
- Issue short-lived grants for exact authorized wake and historical
attachments. Recheck the active run, source, size, and SHA-256 at the
tenant endpoint.
- Download current-wake inputs before provider dispatch. Download
historical files when the authorized chat reader asks for them. Preserve
SSH and sandbox command staging as a fallback.
- Run HTTPS transfers off the Runner control loop. Report completion
with one PRP event per transfer. Drop pending results on cancellation
and clear files at turn settlement, authority rotation, and restart.
- Grant restricted Codex sessions read-only access to the attachment
directory.

## Verification

- 69 targeted server attachment tests passed. They cover wake and
historical downloads, authorization failures, and command fallback.
- 22 focused historical attachment tests passed, including grant
redemption under a held database lock.
- Six focused Rust attachment tests passed, including a source-pinned
Codex sandbox check. The sandbox could read the staged file but could
not modify it or read adjacent Runner state.
- The real Runner transport test confirmed one durable command and one
completion event for a transfer. Restart-continuation tests confirmed
fresh attachment paths appear before dispatch.
- Server and Runner TypeScript checks, Rust formatting, and PRP
generated schema checks passed.
- Required GitHub CI passed on `7a33752bd7`. [CI
run](https://github.com/paperclipai/paperclip/actions/runs/43122967837)
- Greptile scored the same revision 5/5 with no unresolved review
threads.
[Review](https://github.com/paperclipai/paperclip/pull/20956#issuecomment-6808582134)

## Risks

- Saved provider and Runner identities keep their existing formats. A
previous Runner falls back to command staging where the target supports
it. A restart continuation refreshes temporary attachment paths before
dispatch.
- Plugin Runner tasks require a compatible Runner ROM and outbound HTTPS
access to the configured tenant origin.
- Live agent-computer qualification remains a release check.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, shell tools, and
local test execution. Exact variant and context window are unavailable
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #123` /
`Closes #123` / `Refs #123` or (b) described the issue in-PR following
the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-11 16:28:02 -08:00
DottaandPaperclip 89ec4e7614 feat(links): add cached GitHub pull request previews (#20593)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task conversations contain links to pull requests and other work.
> - A URL alone cannot tell the operator whether a PR is open, merged,
or blocked by conflicts.
> - Fetching every link at render time wastes API calls and can expose
private data through a shared cache.
> - This pull request adds a demand-driven links service and a GitHub PR
adapter.
> - Each viewer and company has a separate cache. Repeated links share
one request and result.
> - Operators get context when they hover or focus a link. Normal links
stay useful when a provider is unavailable.

## Linked Issues or Issue Description

**Subsystem affected**

Task Markdown links, shared link metadata, and the server GitHub API
boundary.

**Problem or motivation**

PR links have compact labels and neutral icons. They do not provide a
rich hover card or a common status source across repeated mentions. A
background resolver cannot borrow a person's GitHub connection because
it has no viewer identity.

**Proposed solution**

Recognize supported links locally. Request metadata after a delayed
hover or keyboard focus. Show a card at the first line of the link,
aligned with its icon. Share the result across all mentions in the
current viewer and company. Keep the exact original navigation URL.

The first adapter supports GitHub pull requests. Open is green, merged
is purple, and merge conflicts are orange. Draft, closed, unknown,
loading and unavailable states remain quiet.

**Alternatives considered**

Eager page scans would spend API quota on links the operator does not
inspect. Company-wide caching of personal GitHub results would cross the
credential boundary. This change uses neither approach.

**Roadmap alignment**

ROADMAP.md was checked. This extends existing task links and the GitHub
resolver. It adds no deployment adapter or new background worker.
Searches of open and closed PRs for link previews and PR hover cards
found no matching implementation.

## What Changed

- Add host-owned link recognition and a normalized preview contract. The
service selects an adapter by provider.
- Add a board-only company read endpoint. It checks company access
before cache reads and sends a private, no-store response.
- Use the current viewer's eligible GitHub connection. A configured
connection that cannot supply credentials fails closed. With no managed
connection, reuse existing company credentials or public access.
Instance credentials retain the public-repository check.
- Reuse the GitHub external-object resolver for API transport and
response parsing. Add title and mergeability to its normalized snapshot
without changing its existing state contract.
- Coalesce requests by company, viewer and canonical PR URL. Bound
cached entries and concurrent work. Cache success for one minute and
failures for a short period or the provider retry window.
- Add an abortable request deadline. Timed-out work stays in the
concurrency budget until it settles. No periodic worker or idle polling
is added.
- Add a shared browser query and anchored preview card. Repeated
Markdown links observe the same result. There are no automatic retries,
window-focus refreshes, toasts or connection prompts.
- Add ten production-component Storybook examples and real
pointer/keyboard browser checks. Document the API and verification
route.

## Verification

- Linux source checkout: 233 focused tests passed in nine suites. They
cover recognition, fixed API destinations, provider states, access
failures, credential selection, viewer/company isolation, request
coalescing, expiry, cache bounds, concurrency bounds and mounted OpenAPI
coverage.
- A live public GitHub smoke returned this PR's correct title and open
state. Two equivalent URLs shared one real network request. Personal
connection access was not part of that smoke.
- A controlled HTTP server stalls a real response body. The test proves
cancellation while another lookup completes.
- UI typecheck and the server layered typecheck passed. The UI
production build passed.
- Token gates and query policy gates passed. All 30 map/catalog support
checks passed. The three new server test files also passed a focused
TypeScript check.
- All 27 focused browser checks passed on the reviewed source tree. They
cover light/dark themes, narrow layouts,
open/merged/conflict/draft/closed/loading/unavailable/failed states,
fixed positioning during pointer movement, exact navigation and keyboard
access. A real browser HTTP interception counts one deferred lookup for
two equivalent PR links.
- The first browser attempt lacked Chromium. A later combined build/test
attempt exhausted the sandbox's 8 GiB memory limit and killed Storybook.
Those attempts supply no pass evidence. Chromium is installed, and the
final browser checks run with the build stopped. One initial Storybook
dependency reload interrupted a streaming check. All eight existing
inline-link cases passed on the warmed server, followed by a complete
27-case pass.
- Synthetic Storybook pointer events did not exercise the native hover
boundary. The assertions now run through real Playwright pointer and
keyboard actions. The fixture stories retain the production components.
- Review fixes preserve the Draft label when GitHub reports both draft
and conflicts. They also preserve neutral link colors in dark mode. The
25 affected service checks pass, including the added combined-state
regression. Browser checks compare unknown, draft and closed links with
the muted theme color.
- The first CI policy run rejected a literal retry opt-out. The code now
uses the existing explicit zero-retry helper. A new test-spy type error
was also corrected. These failures are fixed; their earlier runs remain
failed.
- `pnpm verify --explain` returned exit 2 because the browser test path
selects broad CI. No full local suite is claimed. The final CI run
supplies that coverage.
- The previous head `437913c797963aa113bb82e2ae8b836969317ef9` passed CI
and received a 5/5 Greptile score. All four review threads are resolved.
Master then advanced. The current head
`ae2a1c2adbd61a621cca9bdd160561952613df6e` resolves the source-manifest
conflict against master `9b21dcbdda01296349f3cfe0a3a6891e79df6130`.
Fresh CI and review are running.
- The previous CI run found a lost class on decorated PR links and a
stale SDK source manifest. Both are fixed. The existing CommentThread
integration test and 26 focused link service tests pass. All 45 SDK
source/privacy tests pass. All eight source-manifest helper tests pass.
The manifest helper changed only source identities and added the two new
shared files. Runtime, compiler, parser and library pins stay unchanged.
- A later review found that expired timeouts could reuse a settled
promise. The service now removes settled requests from its
shared-request map and counts unfinished provider work separately. The
new regression failed before the fix and passes after it. The
concurrency-bound and real stalled-HTTP tests also pass.
- UI typecheck and token gates pass after these fixes. The browser rerun
needed Chromium in the new session. That first attempt did not exercise
the UI. The complete 27-case browser rerun passed before the master
merge with retries disabled. All 27 browser cases also passed on the
merge revision with retries disabled.

- After the master merge, all 84 focused link, OpenAPI and integration
tests pass. All 55 SDK source/privacy/manifest tests pass. UI typecheck
and token gates pass again. The source manifest retains master's
reviewed runtime pins and includes both sets of source changes. GitHub
confirms the PR can merge cleanly. The preflight helper rebuilt the
shared package and plugin SDK; Cargo is unavailable locally. Final CI
supplies Rust coverage.

- The latest master update changed the reviewed report-lock policy but
left the SDK lock hash at its previous value. The SDK source/evidence
checks reproduced the mismatch. This change sets only the SDK lock hash
to the authenticated output of the existing master policy and patch.
Compiler, parser and Zod pins stay unchanged. All 56 SDK
source/evidence/manifest tests now pass. All 84 focused
link/API/integration tests, UI typecheck and token gates also pass on
the updated source.

## Risks

- This cache is process-local. Separate server replicas can make
separate GitHub requests.
- A cached result can remain visible for its short lifetime after GitHub
changes. A later hover refreshes expired data.
- GitHub can report unknown mergeability. The UI shows conflicts only
when GitHub explicitly reports them.
- Personal preview data stays in viewer/company caches. It is not
written to company external-object records or persisted browser query
storage.
- Stories simulate the API. They do not prove a live GitHub login or
private-repository access.
- No schema migration is required.

## Model Used

OpenAI Codex, based on GPT-6. The runtime does not expose the exact
serving model ID or context-window limit. Used reasoning, repository
inspection, source editing, shell execution and browser tools. No
subagents were used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-10 19:32:51 +00:00
Nicky Leach 8809f44b5e Connect Codex and Claude login to agent computers (#20417) 2026-11-09 22:12:49 -08:00
DottaandClaude Opus 5.5 120017e39d fix(workspaces): read saved workspace files in place instead of extracting the whole archive (#20442)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - When a cloud task finishes, its workspace is saved as an encrypted
snapshot and its sandbox is released. The board's Files view then reads
that saved copy
> - Saved Files access downloaded, decrypted and extracted the whole
archive for every request, and refused archives over 512 MiB with
`too_large`
> - A finished task in a Paperclip-sized repository saves a snapshot of
about 730 MiB, so its files could not be listed or opened at all
> - The archive format already records each file's size and SHA-256, and
the snapshot records the manifest digest, so single files can be read
and verified without the rest of the archive
> - This pull request reads the saved archive in place, one verified
file at a time
> - The benefit is that users can open any file of a large finished
workspace quickly, with bounded memory and no whole-archive transfer

## Linked Issues or Issue Description

**What happened?**
In restart-chaos waves on Daytona with a real Paperclip clone, the
committed workspace snapshots were 763,944,535 bytes. After the sandbox
was released, every saved Files read returned 422 `too_large` (506 reads
in total), because `createSavedWorkspaceLoader` refuses archives over
`maxArchiveBytes` (512 MiB). The board Files view uses the same loader,
so it fails the same way.

**Expected behavior**
A user can list and open any single file of a saved workspace of
Paperclip-repository size without the server extracting or buffering the
whole archive. Memory and time stay bounded per request, and integrity
is still verified.

**Steps to reproduce**
1. Run a cloud task whose workspace is a large repository, so the final
snapshot exceeds 512 MiB.
2. Let the task finish and its sandbox be released.
3. Open the task's Files view, or call `GET
/api/issues/:id/file-resources/content?workspace=execution&path=…`: it
returns 422 `too_large`.

**Paperclip version or commit**
`master` at f73ff4ace9.

## What Changed

- `openWorkspaceSnapshotRanges`
(`remote-workspace-snapshot-storage.ts`):
  - range-reads the snapshot object's header;
- decrypts any plaintext byte range with AES-256-CTR, starting from the
AES-GCM counter block for that range;
- checks that the object size matches the wrapped key's plaintext size.
- New `saved-workspace-archive.ts`:
  - range-reads and decrypts the archive header and manifest;
- verifies the manifest's SHA-256 against the snapshot's recorded
`manifestDigest`;
- applies the restore path's manifest checks (safe paths, parent
directories, file sizes and hashes, total size) and computes each file's
offset;
  - keeps parsed manifests of recent snapshots within a 128 MiB budget.
- `remote-workspace-files.ts`: with the new `openSavedWorkspaceArchive`
option, saved stat, list, read and download use the verified manifest,
not the extracted-directory helper.
- Path normalization, denied paths, symlink and missing-path errors,
traversal, paging, truncation and text detection mirror the helper
exactly.
- A read or download decrypts only that file into a private temporary
file and verifies its SHA-256 before serving it.
- The changed-files view needs repository status, so it still uses the
bounded extraction. A saved copy beyond its limits now reports
`changed_unavailable`, a specific code, while its other views stay
available.
- `app.ts` and the support inspection route pass the new opener.
- `doc/SPEC-implementation.md` describes in-place saved reads and the
remaining changed-files limits.

## Verification

- Red before the fix, green after: `reads a saved workspace above the
extraction cap in place, one verified file at a time`. With extraction
only (master's Files service), the capped snapshot returns 422
`too_large`. With the archive reader, preview, list and a 256 KiB
download succeed through the ordinary routes. Every storage read is a
byte-range read, and no extraction directory is created.
- Parity: `answers saved stat and list exactly like the extracted
helper` compares archive-backed results with the extracted-helper path
on the same snapshot.
- Stat covers files, directories, nested and binary files, missing
paths, paths through a file, symlinks, denied names, and invalid paths.
- List covers all mode, a subdirectory, paging, queries, a scan limit, a
non-directory, and a denied directory.
- In recent mode, the entries match. Order is by path, because a saved
copy has no modification times; the extracted copy's mtimes were just
its extraction times.
- Integrity: `refuses a saved file whose stored bytes no longer match
its verified manifest`. Tampered stored bytes return 422 `file_changed`,
and no bytes are served.
- `keeps changed files on bounded extraction and names its limit…`: over
the cap, the changed view returns `changed_unavailable` and the other
views still work.
- The existing location-clearing preview test now checks for range reads
and an empty request temp directory, not for an extraction directory.
- Measured locally, with local-disk storage and a 749,006,840-byte
snapshot (88 × 8 MiB pack file, a 10 MiB file, 2,000 small files):

| path | time |
| --- | --- |
| Old: download, decrypt and extract all, once per request | 2,338 ms,
plus a full 749 MB transfer from S3 |
| New, cold: open (header and manifest range reads) | 7 ms; open and
read a small file: 13 ms |
| New, warm (cached manifest) | open 2 ms; open and read: 6 ms |
| New: list 100 entries and download a 10 MiB file | 23 ms |

- `remote-workspace-file-resources`, `remote-workspaces.integration`,
`remote-workspace-save-concurrency`, and
`remote-workspace-snapshot-retention` pass (212 tests).
- `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server` passed.
- Not yet verified: the Daytona R1 wave's file checks after release. The
chaos harness owner will run them on a build with this change.

## Risks

- Range decryption bypasses GCM's whole-object tag. Integrity instead
comes from the manifest's SHA-256 matching the digest recorded in the
database, and from each file's SHA-256 in that manifest. A file is
verified before it is served. Text detection on stat samples the first 4
KiB without verifying the whole file. It only sets a display hint, and
content is always verified before it is shown.
- Saved files carry no modification times, so the recent view of a saved
copy is ordered by path, where it was previously in effectively random
extraction order.
- The changed-files view of a saved copy over 512 MiB still reports
`changed_unavailable`, because it needs a full extraction to run `git
status`.
- No schema or API change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, local benchmarks).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 22:33:58 -06:00
DottaandPaperclip 5bad074baa feat: add individual agent interaction permissions (#20411)
Add eleven sparse incoming and outgoing restrictions, human CLI/API administration, shared mutation enforcement, and pending-delivery checks. Preserve existing defaults, agent settings, and existing authorization boundaries.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-09 20:34:04 -06:00
DottaandPaperclip ff2d463eed feat(cli): manage agent grants and keep remote login on the public origin (#20323)
## Thinking Path

> - Paperclip lets people control their AI agents and permissions.
> - Remote operators need a browser-approved CLI login.
> - Proxies can send that login's approval link to an internal host.
> - Human member commands cannot grant an agent an explicit permission
such as commentary:view.
> - This change keeps approval on the canonical public origin and adds
Board-only single-grant commands.
> - The API checks company access and permission-management authority
and records each change.
> - An operator can sign in, grant one agent permission, and preserve
the agent's other grants.

## Linked Issues or Issue Description

**What happened?**

A proxied CLI login returned an approval URL on the upstream host. The
signed-in browser was on the public host. The CLI also had no command to
add an arbitrary explicit agent grant: member commands manage human
members, while agent permissions:update exposes fixed configuration
fields.

**Expected behavior**

Approval uses the configured canonical public origin. A Board operator
with users:manage_permissions can grant or revoke one explicit agent
permission without replacing other grants or activating an inactive
membership.

**Steps to reproduce**

Configure a public auth origin. Create a CLI challenge through a proxy
that replaces forwarded-host headers. Inspect the approval URL. Then try
to grant commentary:view to an agent through the CLI.

## What Changed

- Use the existing public-origin resolver for CLI approval URLs.
- Add agent permission:grant and permission:revoke commands.
- Add PATCH /api/agents/{id}/permission-grants with strict shared
validation and Board-only OpenAPI metadata.
- Serialize each mutation on the agent row. Preserve unrelated grants
and membership state. Commit the grant and its activity record together
and publish activity after commit.
- Document remote login and single-grant management.

## Verification

- The new approval-link regression failed before the fix and passes
after it.
- All 25 CLI auth route tests pass.
- All 72 affected CLI, login and OpenAPI tests pass.
- All 5 PostgreSQL grant tests pass, including actual commentary reads
(403 before grant, 200 after grant, 403 after revoke), preservation,
audit attribution, concurrent additions, invalid inputs, inactive or
missing membership, agent self-grants, cross-company denial, and
rollback on audit failure.
- Shared and CLI typechecks and server declaration-layer checks pass.
- pnpm verify requests broad CI coverage because doc/CLI.md is a global
verification input. Greptile reviewed final head
338c58fb24 at 5/5 with no findings. All 58
non-skipped checks pass on that head. CI run:
https://github.com/paperclipai/paperclip/actions/runs/42663640985. Live
target acceptance is in progress.

## Risks

This adds an explicit permission-management API. It requires Board
authentication, company access, and users:manage_permissions.
Foreign-company agents return 404. Granting a key intentionally makes
that key unscoped in its company; revocation removes that key. Other
keys and agent configuration remain unchanged. Inactive memberships and
unapproved or terminated agents cannot gain grants. No schema change is
needed. Revert this change to remove the new management surface;
existing grants retain their normal authorization rules.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code editing, shell tools,
and browser testing. The runtime does not expose a more specific model
ID or context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-09 11:35:52 -06:00
DottaandClaude Opus 5.5 ac0d0374f2 perf(issue-detail): measure task opens on real instances (#20276)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - People open a task from their issues list to see what an agent is
doing and to reply
> - On a busy staging instance, the composer and the latest messages
take 1.1–2.0 s to appear after a click. For one task with a slow run
log, they took 13.9 s
> - Local fixtures do not reproduce this. They do not have real network
round trips, real server contention, or real object-store latency
> - This pull request adds the measurement we need: a browser harness
that times a task open on a real instance, and a Server-Timing entry on
every API response that separates server time from network time
> - It also records the staging baseline and the improvement plan
> - The benefit is that each later performance change can be measured on
real data with the same tool, before and after deploy

## Linked Issues or Issue Description

**Problem or motivation**
Opening a task from the issues list is slow on instances with real data.
The composer and the latest chat messages can take seconds to appear. We
could not measure this on staging in a repeatable way. We also could not
tell server time from network and queueing time for most API requests.

**Proposed solution**
Add a reusable browser harness for task opens, and add a `paperclip_api`
Server-Timing entry to every API response.

**Alternatives considered**
Local Playwright fixtures only. They do not reproduce real network round
trips, server contention or object-store latency, so they did not show
the delays that people see. Per-endpoint Server-Timing only. It does not
cover the many other requests that run during a task open.

**Roadmap alignment**
This is measurement for existing task-loading behaviour. It does not add
product features. The staging baseline and the measured causes are in
`doc/plans/2026-11-09-task-loading-performance.md`. Follow-up pull
requests change the loading behaviour itself, starting with #20277.

## What Changed

- Added `server/src/middleware/api-server-timing.ts`. Every `/api`
response now has a `paperclip_api;dur=<ms>` Server-Timing entry. The
value is the server time from request arrival until headers are written,
including admission waits. The entry is appended, so the existing
`paperclip_issue` phases stay first.
- Registered the middleware first in `server/src/app.ts`.
- Added `tests/perf/issue-detail/staging-click-harness.js`. It clicks a
task row and records, frame by frame, when the composer and the latest
message become visible, layout shifts, movement of the latest message,
and the requests that run before the page is ready. The summary labels
each sample instead of using the task reference. It replaces query
values and every path segment that contains a digit. A fetch that is
still running when the sample ends is listed as pending.
- Documented the harness in `tests/perf/issue-detail/README.md`.
- Added the dated plan
`doc/plans/2026-11-09-task-loading-performance.md` with the staging
baseline, the causes, and the prioritized slices.

## Verification

- `cd server && npx vitest run src/__tests__/api-server-timing.test.ts`.
3 tests pass. They cover the JSON responses, the time spent in earlier
middleware, preserving route phases, and explicit `writeHead` calls.
- `tests/perf/issue-detail/issue-detail.perf.spec.ts` reads the first
`dur=` value. That value is still `paperclip_issue`.
- The harness was used on staging to capture the baseline in the plan
document.

## Risks

- Low risk. The header has only a duration, with no request, user or
company data.
- The middleware wraps `res.writeHead` once for each request. This is
the same pattern that `on-headers` uses.
- Server-Timing is visible to same-origin scripts only. No
`Timing-Allow-Origin` header is added.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, browser automation, file editing).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 09:06:37 -06:00
DottaandClaude Opus 5.5 5a51a7685d fix(budgets): raise_budget_and_resume continues the work its budget stopped, once (#20144)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A budget hard stop cancels the scope's running work; the board
resolves it with Raise budget and resume
> - In the restart-chaos H1 scenario, the cancelled tasks did not
continue after resume and each needed a manual Retry
> - Resume is the board's explicit decision to continue, so the work it
stopped should continue once, as if the board clicked Retry
> - This pull request files that Retry automatically, keyed by the
cancelled run, once the run's stop has settled
> - The benefit is that a budget raise restores the work without manual
Retries, without repeating completed work, and without charging the
run's attempt budget

## Linked Issues or Issue Description

Refs #20143 (bug B; this PR builds on it), #20113 (bug A), #20101
(superseded: split into #20143 and this PR)

**What happened?**
Restart-chaos waves w-20261108T195944-s42 and w-20261108T210118-s9802
(H1). After `raise_budget_and_resume`, CHA-3's budget-cancelled run
(enforcement stop) and CHA-1's run (agent lifecycle pause from the
budget hold) stayed cancelled until someone clicked Retry.

**Expected behavior**
Resuming continues each task its budget stopped, exactly once, with no
attempt charged, an activity entry, and no repeat of work that already
continued. A run a person stopped stays stopped.

**Steps to reproduce**
Set a budget below spend while a task runs; let the stop cancel the run;
resolve with Raise budget and resume. Before this change, no new run is
queued.

## What Changed

- New table `budget_resume_claims` (migration 0451). One row per
cancelled run, outside the run's result JSON, so a Stop that settles
during the resume cannot erase it.
- `markBudgetCancelledWorkForResume` (called by `resolveIncident` before
the incident closes): claims every task run its budget stopped since the
incident opened. That covers runs budget enforcement fenced, including
stops still in flight, and runs the agent lifecycle pause stopped for
this budget (`pauseStop`, carried from the lifecycle operation's
`pauseCause` so a resume before the Stop keeps the link). Runs a person
stopped are excluded. Two indexed passes led by `(company_id, status)`.
- `deliverBudgetResumeContinuations`: once a claimed run's stop settles,
files the resolving board member's Retry (`requestTaskRetry`,
idempotency key `budget-resume:<run>`, with the member's session source
and the chat Retry callback). The Retry path checks, under the task
lock, that the run is still the task's latest and the task is open, so
continued or completed work is never repeated. Claim settlement and its
activity entry commit together; retryable refusals and errors retry.
- Delivery runs in the always-on budget-enforcement worker after each
pass's Stop delivery. Its schedule includes deliverable claims and the
earliest stale-claim expiry. The execution-continuation worker is a
backstop.
- `agent.paused` / `agent.resumed` logging is in #20113.
`budget.cancelled_work_resume_requested` (incident) and
`budget.cancelled_work_continued` / `_not_continued` (task) are recorded
here.

## Verification

- New `budget-resume-continuation.test.ts` (Postgres, real
`resolveIncident` → claims → `requestTaskRetry` → real `resumeTaskRetry`
into a queued run):
- each task continues once, including a blocked task; repeat delivery or
resolve saves nothing new;
- a stop still in flight at resume, whose terminal write replaces the
result JSON, is still continued;
  - a 30-task scope is fully continued;
- a lifecycle-pause stop caused by the budget is continued, and one
caused by a manual pause is not;
- continued work, done tasks, a run that finished before its stop, and
person-stopped runs (including ones enforcement also fenced) are not
continued.
- `lifecycle.test.ts`: the pause records its cause on the operation and
on stopped runs, including when the budget is resumed before the pause's
Stop. `budget-enforcement-worker.test.ts`: delivery after Stop delivery,
and a wake at claim expiry.
- 287 tests across the affected suites passed. Server typecheck
(`typecheck-layers`, low-memory): no errors.
- Not run: the H1 chaos rerun (Phase 3 runs it after merge).

## Risks

- Resume now continues budget-stopped tasks automatically. A board that
wants a task stopped can Stop it again; person-stopped runs are never
continued.
- A new table and one index (migration 0451).

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test execution). Extended reasoning
enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 22:56:28 -06:00
Dotta 3112e4c955 Verify original Dot OAuth authority for signed Cloud requests (#19819)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Dot uses a user OAuth grant to act in one company.
> - Cloud must use that original grant when it resolves a selected
user's identity.
> - A Cloud signature alone must not create user or board authority.
> - The tenant's OAuth owner must check the token and its current
restrictions.
> - This pull request adds a signed private HTTP boundary for that
check.
> - The result contains only the checked identity fields. It grants no
task or board access.

## Linked Issues or Issue Description

**Subsystem affected**

Server external-agent identity, public MCP OAuth, Cloud control
verification, and HTTP logging.

**Problem or motivation**

Cloud needs a private way to resolve the original Dot token for the
selected user, client, company and resource. The existing generic Cloud
control route creates an operator actor. That route is not the right
owner for this identity check.

**Proposed solution**

Add `POST /mcp/runner/oauth/cloud-principal`. Require the existing Cloud
signature and replay verifier. Bind the whole request to a new, closed
`mcp-dot:principal` action with a maximum 60-second lifetime.
Authenticate the original token once through the normal Dot OAuth owner.
Check current namespace authority before and after that call.

**Alternatives considered**

A Cloud-created grant or a board actor would change the authority owner.
Neither is used. The private route reuses the existing OAuth and Cloud
identity owners.

**Roadmap alignment**

This is a narrow boundary for the existing MCP and Cloud areas in
`ROADMAP.md`. It adds no work queue, public hosted profile, or provider
enablement.

Related public work: Refs #14846, #19427, #19750 and #19776. Searches
for the exact private path and action found no duplicate PR. The
matching private Cloud caller has merged. The current App head passed
its required checks and review.

## What Changed

- Add an original-token principal reader. Keep the original client,
selected user, company, resource, purpose, scope, membership and feature
restrictions.
- Add a 16 KiB raw JSON private route. Reject compressed, invalid UTF-8,
malformed and unknown-field bodies. Bind an ordered whole-body digest to
the signature and existing nonce fence.
- Compare the current stack, claim, instance, origin and connector
identity before and after OAuth. An absent owner or namespace fails
closed.
- Mount the route in normal `createApp` after hostname, runtime-claim
and standby checks, before session/actor work. Do not add it to generic
board-control routing.
- Keep private request and response logs content-free, including raw
buffers, headers, query data and reflected errors.
- Add controlled HTTP/privacy tests and a real normal-App
OAuth/Cloud-signer integration case. Document the machine-only contract
and correct the existing `assistant-connections/client-use` access
entry. No new capability ID or agent API entry is added. No schema,
lockfile, workflow, Runner, Muse or useful-task fixture changes belong
to this PR.

## Verification

- Current `70c03593`: normal module-boundary policy and all seven App
compiler programs passed. The public module factory has an explicit
portable Express return contract. Whole HTTP/log controls passed again:
30/30. The existing real paired OAuth/Cloud-signer/private HTTP case
then passed again on a fresh PostgreSQL fixture: 1/1, zero skipped. All
known child processes and five database connections closed; the final
independent SQL audit was empty.
- The original remote policy failure rejected an App import of a
module-internal path; dependent E2E/verify gates stopped on that
failure. The fix delegates through the public module API. A first local
declaration run reported TS2883 for its inferred return type; the
explicit public type fixed it. Original logs remain preserved. No
checker, compiler option, test body or authority check was weakened.

- The normal full App graph passed on the integrated source. The later
fixture-only repairs each passed the complete affected owning
program:405 entries, including 403 source roots. The remaining programs
and own dependency outputs carried only where their source was exact.
- Whole private HTTP/log files passed:30 tests, zero failures or skips.
The HTTP controls use a controlled OAuth owner with real signatures. The
log controls use the actual HTTP logger.
- The paired Cloud source passed its normal strict build and whole 26
signer/client/namespace and credential-error controls. These are
controlled caller tests, not provider tests.
- One normal-App paired case passed with a dedicated PostgreSQL
database. It used real DCR 201, PKCE, consent, token exchange and a
persisted unpaired grant; actual signed metadata publication and a
committed Cloud receipt; the real Cloud signer/client and actual App
HTTP; exactly-once authentication; refresh; current namespace
restrictions; and revocation. The Cloud registry/provider were the
existing controlled in-memory/Fake fixtures. Recorded inbound Hosts
matched, with zero metadata overflow. No grant, readiness or mirror ACK
was fabricated.
- The original principal helper's 21 controlled tests and two real-OAuth
tests passed before the HTTP slice. Those three files remained unchanged
through integration. They are separate source-scoped evidence, not an
extra combined run.
- Preserved failures: a helper import and test typing error were
corrected before the passed checks. An earlier suite failure did not
serialize its hook error and remains unexplained. Later runs captured a
missing normal managed-config initialization and HTTP 404 from the
loopback mapper. The fixture now calls the normal initialization owners
and uses the existing Host-preserving canonical HTTP adapter. All
original authority assertions and deadlines remain.
- Reproduce the local files with `pnpm --filter @paperclipai/server exec
vitest run src/modules/external-agents/cloud-principal-http.test.ts
src/modules/external-agents/cloud-principal-log.test.ts --maxWorkers=1
--no-file-parallelism`. The real integration file is
`server/src/__tests__/cloud-principal-normal-app.postgres.test.ts`; it
requires its explicit dedicated test DB and an own compiled matching
Cloud checkout.
- The normal `node scripts/check-maps.mjs` check passed for 35 areas and
192 capability IDs. This validates map structure, not business
acceptance. The later edits are documentation/map only; all executed
business source and own outputs from the paired case remained exact.
- Current-head remote CI passes: 59 successful checks and three
intentional skips. Greptile reviewed this exact head at 5/5 with no
findings or review threads. The prior Ponytail review was Ship 5/5 at
the earlier head; no current-head Ponytail turn is present, which is
nonblocking under the authorized review policy. The three
full-repository commands were not repeated locally; the focused normal
App graph and affected tests above passed, and required remote CI
supplies the broad gates. No deployed, provider-capacity, cold-wake or
full hosted lifecycle result is claimed.

## Risks

- This is a security boundary. Signature, current namespace and original
OAuth restrictions must continue to fail closed. The response gives
identity only; callers still need normal operation authority.
- The request carries the original token. The reserved route uses a
closed log projection on both accepted and refused requests. Privacy
regressions remain a material risk.
- Both repositories must agree on the action and digest contract. Older
code rejects the unsupported action; no fallback creates authority.
- This is a prerequisite private boundary. The remaining hosted
discovery, wake, capability, operation, event, retry/reactivation and
revocation lifecycle work, plus Muse, remains open. No profile or
capability is enabled here.
- No migration is needed. Rollback removes the private boundary and
paired caller; normal OAuth and Personal MCP remain with their existing
owners.
- The engineering documentation describes this private machine boundary
and its access restriction. No operator-facing feature is enabled.
Documentation for full hosted enablement remains with that later work.

## Model Used

OpenAI `gpt-6.1-sol` (Sol6.1), through Codex, produced the source and
tests with tool use and local compiler/test execution. A GPT-6-based
Codex coordinator reviewed the work. Its exact model ID is unavailable.
Context window sizes are unavailable.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-07 08:02:42 -06:00
af0fe8fe81 Keep personal MCP connections functional while idle tenants sleep (#19427)
## Thinking Path

> - Paperclip manages work for AI agents.
> - Personal MCP connections should remain enabled while an idle tenant
sleeps.
> - Unfinished OAuth, transfer and delivery work must retain its owner
and deadline.
> - Cloud can receive bounded metadata and incoming work while the
tenant remains the company-scoped authorizer.
> - Current identity, revision and delivery acknowledgments establish
when ownership has transferred.
> - This change preserves useful wake, retries and revocation while
making settled ownership inspectable without writes.

## Linked Issues or Issue Description

Refs: #19377 and #19750. Hosted companion:
[paperclip-cloud#1217](https://github.com/paperclipai/paperclip-cloud/pull/1217).

**What happened?** Enabled personal MCP grants and subscriptions could
keep an otherwise idle tenant awake. Ignoring those records would lose
discovery, authorization, delivery and revocation obligations.

**Expected behavior** Unconsumed authorization codes, device requests,
file handoffs and delivery work retain their original deadlines. Settled
connections remain enabled and become quiet after the current external
owner acknowledges responsibility. Incoming useful work resumes normal
authorization and delivery; revocation still takes effect for the
current company.

**Steps to reproduce** Complete normal PKCE/device consent and company
selection, synchronize client/device and revocation state, retain the
connection while idle, then exercise useful tools/events and revocation.
Sleep eligibility requires current incoming/deadline ownership
throughout.

## What Changed

- Add bounded signed client/device metadata and revocation
synchronization with exact revision, identity and ACK checks, startup
recovery and finite legacy-device cutover.
- Preserve company selection, authorization and operation identity.
Register grant work before its INSERT in the same transaction; retain
rollback, expiry and retry behavior.
- Require current managed identity/resource before hosted publication.
Receipt-cleared re-enable must obtain a fresh source ACK before device
creation.
- Reuse read-only settings and inspection budgets; connect MCP owners to
normal startup, drain and shutdown.
- Bound metadata pages and the actual signed request body while
preserving the default limits of other operations.
- Generate migration 0440 after the merged Legacy migration. Preserve
all 438 existing executable migrations and the journal prefix; retain
the normal five newest snapshots.
- Preserve the merged transaction-owner forwarding and regression
coverage from #19750 through normal ancestry integration.

## Verification

- Current own Shared/SDK, database, eval-kernel and Runner builds
passed, together with all seven normal application TypeScript programs.
Missing generated package exports/workspace links were restored through
normal setup; the original setup failures were retained. No source,
compiler option, test assertion or timeout was weakened.
- Normal migration generation changed only the five intended MCP tables;
its SQL is byte-identical to the reviewed MCP delta. Fresh reference
databases and both retained master/Muse histories upgraded through 0440,
preserving historical hashes, row fields, schema and idempotence. Direct
assertions cover the Legacy null default and MCP null/zero defaults. Two
selected history cases passed; the other 21 database cases were
unselected.
- All six current MCP lifecycle cases passed with zero skips: normal
PKCE approval intent, ACK recovery after outage, mutation during
publication, disable/re-enable with original expiry, company-deletion
rollback/commit and key rotation.
- All 81 checks in five current whole files passed with zero skips:
OpenAPI 34, read-only settings 2, diagnostic catalog 25, PostgreSQL
diagnostics 7 and dependency scheduling 13. PostgreSQL fixtures ran
normal migrations before test imports.
- Earlier canonical proof at App `cd1c8c6` / Cloud `9f67531` includes
real 60-second rate and three 15-second quiet windows, useful work and
three revokes. Later paired proof at App `43af514` / Cloud `914eb0f`
plus its reviewed candidate includes normal DCR, PostgreSQL metadata, a
144,017-byte signed publication, actual ACK, cap/default-413 controls
and a real 15-second quiet window. The relevant producer/signing/wire
inputs remain byte-identical in this integration; shared changes are
qualified above. These remain source-scoped observations, not a fresh
full canonical run on the integrated tree.
- All known test owners, process groups and OS sockets closed; an
independent final PostgreSQL audit found no remaining test sessions. One
dependency-test trace recorded 46 opens and 45 close events. Its cause
is unknown; this is not claimed as proof of graceful closure.
- Fresh exact-head remote checks passed: 59 successful and 4 skipped
contexts, with a complete current 5/5 Greptile review and all seven
review threads resolved. Targeted local checks do not constitute a new
whole-repository test run.

## Risks

App and Cloud must agree on identity, revision, namespace, deadlines and
revocation. Partial rollout, unavailable authority or missing ACK must
retain awake debt. Shared startup/shutdown and settings changed since
the earlier canonical profile, so historical observations carry only
within their stated source scope. No deployed Railway/Neon suspension,
Cloud alarm, savings or complete hosted Dot/Muse acceptance is claimed.

## Model Used

OpenAI `gpt-6.1-sol` produced code and verification, with a Codex
coordinator reviewing and integrating the changes. The coordinator's
exact runtime model ID and context-window sizes were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Codex <noreply@openai.com>
2026-11-07 02:12:48 -06:00
839d41aabb fix(server): retire owned plugin listeners before idle sleep (#19358)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Plugins receive durable agent lifecycle events through a registered
host.
> - The host uses a PostgreSQL LISTEN connection to wake delivery.
> - An idle report could return `none` while that host-owned connection
stayed open.
> - This change gives that listener an exclusive owner and closes it
during the current idle hold.
> - Release or expiry restores listening before journal replay.
> - Failed close and pending work continue to block sleep.

## Linked Issues or Issue Description

Refs #18988, which is now merged. This focused ten-file change now
targets master. Current integrated candidate:
`0f83005122ef1bdb1f1e08f8f58424c9a840ebb1`, with actual master
`644ee66f5d4ae8365fe8b9eca57df2be7405b5ab` preserved through normal
merges. No history was rewritten.

**What happened?**

A normal local App with an installed lifecycle plugin could return an
idle `none` report while its lifecycle LISTEN backend and socket stayed
open.

**Expected behavior**

The host must close its own listener before it reports idle safety. It
must retain other subscriptions, durable events, company scope, retry
deadlines and useful reactivation.

**Steps to reproduce**

Start the normal server with a lifecycle plugin. Create an agent and
wait for its durable acknowledgment. Stop the server, create another
agent while it is stopped, and restart with the same files. Request an
owned idle hold after ordinary replay. Check the exact listener backend
and matched Node socket before shutdown. Let the valid hold expire
naturally, then make an ordinary authorized agent update and check its
durable plugin acknowledgment.

Live executing or persistent preview services remain active. The
separate hosted preview dependency
[paperclip-cloud#478](https://github.com/paperclipai/paperclip-cloud/pull/478)
retains its own App and operator prerequisites. This PR does not claim
deployed preview or suspension acceptance.

## What Changed

- Add a private, exclusive database listener owner that inherits the
normal client policy.
- Prepare plugin lifecycle retirement after native retirement under the
same current idle hold.
- Use one delivery-eligibility helper for dispatch and idle journal
inspection. It preserves the same manifest, capability, company-disable
query, and skip rules.
- Close only the owned listener. Keep failed close ownership and
inspection failures visible.
- Resume listening on normal release or expiry, before journal replay.
- Preserve disabled-feature retry history and the original deadline on
re-enable.
- Await lifecycle disposal during normal shutdown and consumer teardown.
- Add owner regressions and document the lifecycle contract.

## Verification

**Current integrated source:**
`0f83005122ef1bdb1f1e08f8f58424c9a840ebb1`. Fresh normal Runner
TypeScript passed in 0.785 seconds. The complete normal App layered
semantic check passed in 28.284 seconds. All tracked source, the
lockfile and the 18 separate private witness inputs stayed unchanged.
Known compiler owners closed and the isolated temporary home was
removed.

**Current product qualification:** at measured integration
`896cc9b62993e6ed10418d7a7417e1ddbf8dc89c`, all three whole files passed
with no failures or skips: lifecycle delivery 40, runtime hooks 20, and
runtime assignments 24. The actual physical LISTEN and natural-expiry
controls passed. All owned clients and captured processes closed.
Independent final session checks were zero before normal deletion of
each successful test database.

**Normal full-App physical retirement:** the original case passed in
47.574 seconds. It exercised real startup, readiness, SDK callbacks,
durable ACK, foreign-company exclusion, physical stop, offline input and
cold replay. The initial unknown hold and its ordinary release remain in
the evidence. After the original 30.5-second cadence, a new valid hold
returned quiescent. The exact current cold-App LISTEN backend and its
matched Node socket closed before server stop.

**Normal full-App natural expiry:** the separate original case passed in
77.391 seconds under its original 90-second body cap. The final valid
30-second hold expired naturally. A new listener backend and matched
current App socket appeared before fresh input. An ordinary authorized
agent-name PATCH produced journal event 20, the actual SDK callback and
durable ACK. Foreign exclusion and readiness assertions passed. The
final measured hold had no manual release, replacement hold or shortened
TTL. The original preparatory unknown hold, ordinary release and
30.5-second cadence remain explicit.

Both App cases used the normal current server helpers and actual SDK
bundling/worker entry. A private compatibility change supplied the same
owned HOME at the exact existing child-fork boundary. It did not change
the product helpers, business assertions or timing limits. These
separate acceptance fixtures remain outside the focused product diff.

**Source carry to the current candidate:** all nine other owned files,
listener mechanics, actual producer paths, SDK paths and measured hold
behavior are unchanged. The sole owned intersection from newer master is
an independent environment-delete cleanup callback in `app.ts`. The
plugin witnesses do not call that route. Normal merges preserve every
unrelated master blob and file mode. The fresh current semantic checks
above qualify the integrated source. New master Native crash-recovery
and saved-copyback Retry capabilities are not claimed by these plugin
cases.

**Actual master merge target:** newer master contains retained plugin
Runner release/resume and other Native recovery fixes. The projected
combined target passed fresh normal Runner TypeScript in 0.799 seconds
and the full App semantic check in 29.715 seconds. All ten plugin-owned
blobs and modes remain exact to the published head; every unrelated
master change is preserved. The measured plugin cases create no Native
provider turn or warm Native session. The unchanged
native-first/plugin-second ordering still checks the native result
before listener retirement. Later sparse-checkpoint and Native
response-publication changes do not enter those paths. This is a scoped
source carry, not new hosted Runner acceptance.

**Failures retained:** an earlier reactivation fixture's real timer
returned one millisecond before its original wall-clock expiry. Its
private fixture now waits any positive remainder and retains the same
expiry assertion and limits. The earlier partial-composition `unknown`
result remains distinct from normal-startup proof. Current qualification
also exposed missing normal workspace dependencies and a private guard
that assumed loose Git refs. Normal owned dependency setup and an exact
loose-or-packed reader fixed those setup failures. No business
assertion, timeout, permission boundary or product sleep gate was
relaxed.

Repeat focused product coverage with a fresh external fixture database
and the normal server Vitest command, one worker:

```sh
pnpm --filter @paperclipai/server exec vitest run src/__tests__/plugin-lifecycle-delivery.test.ts --maxWorkers=1 --no-file-parallelism
pnpm --filter @paperclipai/server exec vitest run src/__tests__/agent-runtime-hooks.test.ts --maxWorkers=1 --no-file-parallelism
pnpm --filter @paperclipai/server exec vitest run src/__tests__/agent-runtime-assignments.test.ts --maxWorkers=1 --no-file-parallelism
```

Exact current head `0f83005122ef1bdb1f1e08f8f58424c9a840ebb1` has 60
successful checks, three skips and one neutral result, with no pending
or failing contexts. Greptile completed its full review at 5/5 with no
actionable issue. All root and nested review pages were read; the
earlier eligibility finding is resolved. Ponytail's latest turn stopped
before a verdict. A normal retry was requested; no current Ponytail
verdict or score is claimed. Previous `3ca86ec` CI and scores are
historical. A full local repository typecheck/test/build or browser run
is not claimed. Controlled local App/SDK/PostgreSQL proof does not
establish live provider, deployed Railway/Neon suspension, signed Cloud
alarms or global exactly-once delivery.

## Risks

- A listener close failure or uncertain inspection must continue to
block sleep.
- Release, expiry, configuration changes, and journal arrival can race
with retirement. Epoch and current-hold checks retain conservative
results.
- Delivery remains at least once across uncertain ACK boundaries. This
change does not add global exactly-once delivery.
- The database helper closes only its exclusive listener client. Other
owners and subscriptions keep their resources.

## Model Used

OpenAI `gpt-6.1-sol`, reasoning and code/tool use. The managing Codex
agent implemented shared database/composition seams and reviewed the
change; its exact model ID was not exposed. The runtime does not expose
a verified context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Codex <noreply@openai.com>
2026-11-06 22:56:47 -06:00
DottaandClaude Opus 5.5 aed053633e fix(environments): run pending sandbox cleanup when an environment is deleted (#19716)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Operators delete a sandbox environment, optionally with consent to
destroy its sandboxes (`DELETE
/api/environments/:id?destroyReusableSandboxLeases=true`)
> - A sandbox whose teardown failed or was interrupted is kept as a
`pending_cleanup` lease, and it blocks the delete
> - The cleanup sweep retries such a lease only after a cooldown of up
to 5 minutes
> - A consented delete therefore returned 409 `pending_sandbox_cleanup`
for minutes. A caller with a shorter deadline gave up, and the
environment and its sandbox stayed behind
> - This pull request makes the consented delete run one teardown
attempt for the environment's pending cleanups now, then re-check
> - The benefit is that a consented environment delete completes as soon
as the provider teardown succeeds

## Linked Issues or Issue Description

**What happened?**
After a restart chaos wave, the harness deleted the wave's Daytona
environment with `destroyReusableSandboxLeases=true`. The server
returned 409 `pending_sandbox_cleanup` every 3 seconds for the whole
120-second client deadline. A run lease had entered `pending_cleanup`
after its run failed, and the sweep's retry cooldown had not passed. The
environment was not deleted.

**Expected behavior**
With explicit consent, the delete tears down the environment's own
pending sandbox cleanups and completes when that teardown succeeds. A
teardown that fails, or one already in flight elsewhere, still blocks
the delete.

**Steps to reproduce**
1. Leave a sandbox lease of an environment in `pending_cleanup` with a
future retry time.
2. `DELETE /api/environments/:id?destroyReusableSandboxLeases=true`.
3. On master, the response is 409 `pending_sandbox_cleanup` until the
cooldown ends.

Refs #19696 (restart recovery for creation and teardown claims; this
builds on it).

## What Changed

- `heartbeat/recovery.ts`: `sweepPendingCleanupLeases` accepts
`environmentRetry: { environmentId, actorId }`. Like the existing
run-scoped explicit retry, it bypasses the cooldown for one attempt,
never takes over an in-flight attempt, and logs
`environment_lease.cleanup_retried` with reason `environment_deleted`.
It touches only that environment's leases.
- `routes/environments.ts`: when the delete has consent and the only
blockers are pending cleanup and reusable leases, it runs that retry
first, then re-reads the blast radius. The existing reusable-lease
destroy and the final guard are unchanged. The new dependency is a
narrow, optional `retryPendingEnvironmentCleanup` option; without it,
behavior is unchanged.
- `app.ts`: wires the option to the existing heartbeat service instance.

## Verification

- Red, then green:
- `environment-routes.test.ts`: "runs the environment's pending sandbox
cleanup now and deletes with explicit consent" and "keeps rejecting when
the consented pending cleanup attempt does not finish" fail before and
pass after. "does not run pending sandbox cleanup without consent or
past another blocker" passes in both.
- `heartbeat-pending-cleanup-sweep.test.ts`: "runs one attempt for a
deleted environment's cleanup during its cooldown, scoped to that
environment" fails before and passes after. It checks that another
environment's lease and an in-flight claim are untouched, and that the
activity log is written.
- `sandbox-creation-operations.test.ts`: "reclaims an issue-bound
creation after its run failed and the issue finished on a later run"
covers another chaos finding. A creation intent bound to an issue was
interrupted by SIGKILL 0.5 s after it committed, its run ended
`process_lost`, and a later run finished the issue. The test fails on
the code before #19696 and passes on master, so #19696 already covers
this case. The test is kept as a regression.
- After rebasing on master, `environment-routes`,
`heartbeat-pending-cleanup-sweep`, `agent-runner-qualification-routes`
and `sandbox-creation-operations` pass (192 tests). `tsc --noEmit`
reports no errors in the changed files.

## Risks

- A consented delete now makes provider teardown calls for pending
cleanups. Those sandboxes were already queued for destruction, and the
operator consented to destroy the environment's sandboxes.
- One explicit attempt counts toward the lease's attempt cap, the same
as a user Retry.
- The delete request now waits for those teardown attempts. Each attempt
uses the existing plugin timeouts.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file editing, test execution). Extended reasoning
enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 21:28:54 -06:00
DottaandPaperclip 6be71028ad Prepare online ephemeral enrollment through signed Cloud control (#19660)
## Thinking Path

> - Paperclip manages agent work through a control-plane web server.
> - A replacement web server needs shared durable data and the original
signing identity.
> - Existing enrollment exports that identity only after a global drain
and offline migration.
> - Online enrollment needs a separate preparation operation while agent
work continues.
> - This change adds a purpose-bound Cloud control operation with
existing data and key owners.
> - The result is an audited preparation receipt, with no permission to
stop agents or detach storage.

## Linked Issues or Issue Description

Refs #19038. Refs #19024.

**What are you trying to improve?**
Allow Cloud to prepare an already migrated tenant for volume removal
without the offline migrator or a global-idle gate.

**What problem does this solve?**
Cloud must preserve the original decision key before starting a
replacement with static ephemeral mode. A tenant session must never gain
key-export authority.

**What would the improvement look like?**
A fresh exact-action Cloud assertion obtains a private, deployment-bound
preparation receipt. Unfinished durable-data migration and invalid
replacement configuration block export.

## What Changed

- Add the signed `ephemeral-enrollment:prepare` control action and a
strict UUID-only POST route.
- Check existing durable-data eligibility and ephemeral configuration
before reading the original key. Never create a missing key.
- Audit only public preparation metadata. Keep HTTP diagnostics
secret-sensitive and the response non-cacheable.
- Bind the receipt to the attempt, stack, build commit, Railway
deployment, time, static mode and key digest.
- Always return `enrollmentReady: false`. Cloud still needs to arm a
replacement, verify old writers have stopped, repeat eligibility and
qualify native handoff before detaching the retained volume.

## Verification

- PASS: 80 focused tests across the preparation owner, signed route,
existing Cloud control, HTTP privacy and OpenAPI route coverage.
- PASS: 2 real PostgreSQL cases. Concurrent copies of one signed request
export once; the durable receipt still rejects reuse after clearing the
process fence. Normal run admission before skill pins is allowed;
missing durable skill installation remains blocked.
- Genuine prepatch RED: the same two PostgreSQL cases exported twice and
rejected ordinary admission. The fixed owner was restored automatically
after the comparison.
- PASS: server TypeScript check with workspace dependencies built.
- Fixed the first CI route-coverage failure by declaring this private
control endpoint in the existing explicit OpenAPI exclusion registry.
- Initial local HTTP execution lacked sandbox socket permission; the
reviewed run passed. Initial PostgreSQL fixtures enabled durable files
too late and reused an issue prefix; those fixture defects were
corrected before the red/green comparison.
- All required CI gates pass on
5346721847. Fresh Greptile review is 5/5
and all review threads are resolved. No deployment, volume detachment,
admission change or agent stop is performed by this PR.

## Risks

- The private response contains the original signing key. Only a fresh
stack-bound assertion for this exact action can obtain it. The Cloud
consumer must pass it directly to its existing vault owner and must not
expose it through admin responses or registry metadata.
- Eligibility is an observation. It cannot replace static writer
restrictions or qualified ownership handoff. This PR deliberately
supplies no detach or readiness authorization.
- Cloud transition integration is still required. This PR alone does not
enroll an instance.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, code execution and browser
tools. The host does not expose the exact serving model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-06 18:00:28 -06:00
DottaandPaperclip 124b2cd0ad fix(server): certify background deadlines for idle tenants (#19001)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Background services own future checks, retries, and recovery work.
> - An idle tenant can stop only when a durable wake covers its
remaining duties.
> - The current alarm catalog covers several domains but leaves these
owners blocked.
> - A saved future timestamp alone cannot prove that reconstruction or
admitted work has settled.
> - This pull request binds durable domain snapshots to the actual
registered worker revisions.
> - The tenant can hand off supported future duties and retain all
checks for uncertain or active work.

## Linked Issues or Issue Description

No matching public issue was found. Related work: [domain deadline
handoff #18656](https://github.com/paperclipai/paperclip/pull/18656),
[owned blocker diagnostics
#18632](https://github.com/paperclipai/paperclip/pull/18632), and
[completed history proof
#18646](https://github.com/paperclipai/paperclip/pull/18646). The
separate [native idle-session change
#18988](https://github.com/paperclipai/paperclip/pull/18988) covers
runtime retirement. [Far-future retry re-probes
#13534](https://github.com/paperclipai/paperclip/pull/13534) changes
recovery responsiveness after an upstream success; this change preserves
existing retry decisions.

**What happened?**

Future status-card, external-object, skill-sync, watchdog, retry, and
quota duties lacked complete owner-bound coverage in the idle deadline
proof. A completed quota task could also retain its monitor policy and
timestamp and remain blocked after the scheduler had disarmed it.

**Expected behavior**

A settled registered owner can hand off its exact supported future
deadline. Active work, dirty reconstruction, transaction uncertainty,
changed revisions, claims, unknown state, and unsupported duties keep
the tenant awake. Completed history retains its data and stops recurring
work when the owner has no remaining duty.

**Steps to reproduce**

1. Produce a supported future duty through its normal service.
2. Let the registered worker finish reconstruction and admitted work.
3. Request an owned idle report with the negotiated domain deadline
catalog.
4. Check the saved earliest deadline and generation before durable alarm
registration.
5. Stop and reconstruct the worker. Check that ordinary admission and
claims resume useful work once due.
6. Complete a quota task. Keep its saved policy and timestamp. Check
that linked unfinished work still blocks sleep and that settled history
does not.

**Paperclip version or commit**

The owned patch was qualified at
`699c8569fa34338dd75cdc5731381ec1e2efdb45`. Publication head
`491fac368ee72eaac8ae105d53e6bf6b305d8a6a` normally merges actual master
`039e0dad2c51c80251e1302a19c5566e039c357a`. All 43 owned files are
byte-identical. All 29 new upstream paths are preserved. The later
normal current-master integration is
`8c912e1ba4fc66397a7341ade59373af8075068f`, described separately below.

**Deployment mode**

Source checkout with real PostgreSQL 17 and a controlled local Node
process. No deployed alarm, tenant suspension, database suspension, or
fleet result is claimed.

## What Changed

- Add registered-worker reconciliation revisions and invalidate them on
stop.
- Keep status-card, external-object, and skill deadline interpretation
with their owners.
- Reuse the watchdog classifier against the supplied database snapshot.
- Validate the exact saved retry run and paired queued wake.
- Validate legacy quota source bindings and the current assignee or
reviewer policy.
- Preserve common issue scope, dependency, sibling, and state checks for
covered waits.
- Compose normalized owner snapshots within the existing bounded
deadline proof and recheck revisions after inspection.
- Bind actual service and worker instances from server startup. Missing
bindings preserve existing blockers.
- Retain completed quota policy and timestamps while independently
rejecting linked unfinished runs or wakes.
- Block unconsumed budget evaluation inputs through the owning queue
predicate. Accounting policy and settlement stay unchanged.
- Bind completed quota history through fixed-size UUID-array parameters,
preserve exact company/task/source associations, and guard external
references before the run-ID join.
- Batch watchdog subtrees and shared classifier facts without changing
the 64-query or two-second proof budget; preserve exact company pairing
for first-run evidence.
- Extend production-path regression controls and document supported
coverage and limits.

## Verification

Current integration head: `688abc6cac6016b9fb50e5832f81e2975ed5d49f`.
Fresh exact-head checks completed with **58 successes, three intentional
skips and one neutral review**, including aggregate `ci / verify`. The
complete current Greptile review is **5/5**. The complete current
Ponytail verdict is **Ship (4/5)**, with only optional cleanup of an
unused MCP proof input; both prior correctness findings are resolved.
All review threads and nested pages were inspected. The normal master
merges preserve the registered retry deadline owner and current startup
completion log after the same readiness await. The earlier focused
evidence below remains tied to its tested source revisions.


Current-master integration `8c912e1ba4fc66397a7341ade59373af8075068f`:

- Normally merge actual master
`46fb933b0155d183ce64a8bdc4950d1b58c926f6`. Every nonoverlapping tree
blob retains its owning parent across 14,044 paths. All 42 owner-only
files and 15 critical binding/proof/readiness files remain
byte-identical. Three automatic overlaps preserve the existing watchdog
binding, conservative unavailable-reader fallback and optional actual
owner arguments. The two scoped-deadline conflicts retain the entire
upstream resource-reservation/mid-pass admission/abort algorithm inside
the existing reconciliation revision/running/failure fence, with both
parent test sets.
- Actual TypeScript 7.0.2 server noEmit passed in 9.434 seconds, normal
source declaration layers in 12.102 seconds, and four selected
test/helper/config layers extending the actual CI configuration in 3.167
seconds. No compiler option weakening, heap override, donor declarations
or SDK replacement stub.
- Focused PG0 controls passed: 102 scheduler/proof/owner controls in
16.451 seconds, including all 21 scoped-deadline cases; 121 startup
controls; and all 107 unchanged instance-settings route cases in 6.456
seconds. The latter first hit a local prewire-guard
listener-registration race. Only the ignored guard was corrected to
register the same actual OS-bound server port immediately after native
listen returns. Closed, unknown, foreign, offhost and PG destinations
remain blocked. Original failure evidence is retained; product code and
assertions were unchanged.
- The exact current UI policy command now reports zero findings (0.362
seconds); its seven scanner controls pass. The earlier dependent PR used
an old scanner/source baseline under a newly enforcing trusted workflow
and reported 397 findings. Carrying the genuine current master resolves
that mismatch without editing unrelated UI sites or relaxing the gate.
- Every meaningful worker checked its executing PID/thread/preload hash
and matching receipt. PG0 controls opened no PostgreSQL sockets; all
owned processes exited. Original dependency-cache/DNS, ignored-harness
syntax/type and guard failures remain separate from product results.
- This is fresh semantic and focused runtime qualification, not a local
whole-repository or current-head PostgreSQL acceptance rerun. The
upstream workspace-busy parked-retry/terminal-holder notification path
remains unchanged by this merge and needs its separate current-graph
producer check. Historical six-case/scale results below stay tied to
their measured sources. A matching controlled default-cadence follow-up
is in #19191; no fresh current-head two-cycle or deployed suspension
claim follows from this merge.
- All 59 exact-head check contexts now pass (55 success, one neutral,
three intentional skips), plus the legacy security status. The complete
edited Greptile review is 5/5 for this exact commit, with all
root/nested pages inspected and no unresolved threads. Ponytail stopped
without completing its new turn; it supplied no new verdict or finding.
Prior actual findings remain addressed. Earlier scores/checks below
belong to their explicit earlier commits.


On frozen clean merged source `699c8569fa`:

- Real PostgreSQL acceptance: **6/6 passed**,83.483 seconds wall. Actual
retry producer creates the exact saved run/wake; stopped and
reconstructed owner resumes through current admission, one durable
start, real local process output and normal task completion. Includes
cancellation, drain arrival, retained quota history with queued-run/wake
negatives, watchdog review settlement and budget-input
empty/pending/consumed control using the actual owner.
- Affected coordinator, deadline, scoped-deadline, recovery and
composed-proof controls: **78/78 in five files passed**,32.114 seconds
wall, worker 1/no file parallelism.
- `pnpm exec tsc --noEmit -p server/tsconfig.json`: **passed**,17.467
seconds. Native TypeScript 7.0.2 and Node 24.22.2.
- PostgreSQL tests use one dedicated test database, matching
fixture/admin URLs, pool limit four, one worker, and no embedded
fallback. Every run recorded unchanged source hashes and a clean
checkout.
- The eight-second real-clock retry is a domain execution check. It is
below the Cloud minimum wake lead and is not a positive accepted hosted
sleep cycle.
- The completed quota-history product failure and original
fixture/protocol failures were preserved; the owning-reader fix passed a
real PostgreSQL red/green regression before the fresh merged six-case
pass.
- After runtime completion, the dedicated database had zero other
clients and no owned runtime processes. All three qualification process
PIDs exited. The coordinator independently audits and drops the
database.
- Comment-only cleanup has equal strict TypeScript parser ASTs across 15
files. The final lexical audit finds no added comments relative to
actual master. `git diff --check` passed.
- Earlier retry fixtures omitted responsible-user membership, advanced
JavaScript time past the real PostgreSQL lease clock, or produced stdout
without normal task completion. Corrected fixtures use real membership,
an eight-second real deadline, and ordinary task comment/completion. The
last case exposed legitimate disposition repair rather than duplicate
retry promotion.
- The quota and watchdog controls first ran before admitted work or the
existing 30-second reconstruction duty settled. The stronger tests
require an interim blocked report and then exact settled-owner proof.
Production deadlines were not changed. The separate
completed-quota-history defect was fixed in its reader and verified with
linked unfinished-run and wake controls.
- The status/object and skill owners reviewed the actual final startup,
route and post-backup bindings. Their domain source evidence carries
where token-equivalent. The master coordinator coalescing change was
reviewed separately. Historical domain PostgreSQL results are not
presented as fresh full-suite results on this head.

The later master merge adds retry predecessor cleanup checks under the
existing issue claim lock. The source audit confirms unchanged saved
deadlines and run/wake identities. The three actual retry
lifecycle/cancellation/drain cases passed on the batched candidate with
this parent (31.817 seconds). Those local dependency controls preceded
the current exact-head remote run. Native backup preservation and
TestOps changes from master are retained. No pass is inferred for
changed dependency behavior solely from unchanged owned files.

Review repair published as `643f1e7400`:

- The original reader returned `unknown` after seven actual watchdog
reviews settled across seven companies. The unchanged composed-report
regression passed after batching; the final reader passed in 61.841
seconds, including active queued-work and stopped-owner rejection.
- A separate two-company red/green control rejects terminal-run context
from another company as first-run grace evidence. Shared facts are
serialized once, with per-root IDs and review summaries.
- The affected recovery, proof and diagnostic unit controls passed 47/47
(14.575 seconds). Startup owner-contract fixtures passed 121/121 (10.589
seconds).
- Diagnostic PostgreSQL controls passed7/7 (12.406 seconds), including
the normal budget-input producer and its exact blocker reason. Initial
empty-state setup uses the same cleanup as per-case teardown when a
dedicated test database is reused.
- Actual repository `pnpm exec tsc --noEmit -p server/tsconfig.json`
passed 12.710 seconds with no heap override. Final source has no added
code comments; whitespace checks pass.
- The original query-budget failure, two-company discriminator, initial
cross-suite fixture failure and an interrupted incorrect local-driver
selector are retained separately. The interrupted run is not a pass.
Independent cleanup found zero owned clients or runtime processes.

Quota-scale repair published as `6696a37f8a`:

- The exact 10,000-row completed-history case reproduced
`MAX_PARAMETERS_EXCEEDED`:90,002 bound parameters exceeded the driver's
65,534 limit, and the composed report returned `unknown` despite settled
owners. Original failure and actual driver diagnostics are retained.
- Fixed pending-work queries use three typed UUID-array parameters plus
the result limit, keeping exact company/issue/source tuple bindings. The
primary run-ID column stays uncast; a guarded CASE validates the raw
external reference before conversion. The 10,000-row cap and
64-query/two-second proof budget are unchanged.
- The same 10,000-row composed-report case passed 18.860 seconds, with
unfinished run/wake rejection, malformed-reference safety and
conservative 10,001-row overflow. Its fixture derives retained history
from the actual quota producer and normal task completion, expands valid
persisted issue/source pairs, then starts the actual registered owners.
- The existing actual quota producer case passed 14.187 seconds,
including exact future deadline, retained completion history, unfinished
work, real foreign-company source references and noncanonical/malformed
references.
- Affected owner/proof controls passed 22/22 (26.444 seconds).
Production server noEmit passed 16.039 seconds.
- CI on 643 identified TS7034/TS7005 in the settled-watchdog fixture's
initially untyped collection. It is now explicitly typed. A separate
affected-entry semantic check extending the repository's actual
`server/src/__tests__/tsconfig.json` passed 15.093 seconds: six affected
tests, two owner files, ambient declarations and current imported
source, with unchanged strict/module/lib/path settings. The production
server config excludes that test directory and is not presented as
test-layer proof. No custom suppression or replacement SDK stub was
added.
- All runs were sequential with pool max 4, worker 1,
source/config/command hashes and cleared inherited
provider/LIVE/other-database variables. Cleanup found zero owned
database clients or local runtime processes. At prior head
`6696a37f8a5718cd0f79054969a0084462d41d47`, all required CI gates pass.
The exact-head inventory has 55 successes, one neutral review and three
intentional skips. Greptile is 5/5. The complete latest Ponytail verdict
confirms the quota finding is fixed; its only remaining note is optional
unused MCP proof state, not a behavior defect. All three review threads
are resolved. The earlier 643 result remains historical.

The repository-wide typecheck, tests, build and browser gates passed in
the prior `6696a37f8a5718cd0f79054969a0084462d41d47` CI run. The new
`8c912e1ba4fc66397a7341ade59373af8075068f` full CI also passes. They
were not run as a full local workspace command in the focused PostgreSQL
lease. Local evidence above uses bounded external databases and does not
claim embedded-cluster, live-provider or deployed-alarm qualification.

## Risks

- An unsupported or uncertain owner state must remain blocked. An
optimistic exemption could lose work while compute is stopped.
- Changes during reconstruction or inspection must invalidate the proof.
A memory-only timer is not durable alarm coverage.
- Current authorization, budgets, cancellation, accounting, and resource
ownership still apply on dispatch. The proof does not grant execution
authority.
- The supported writer model uses in-process transaction signals and
startup/resume reconstruction. Independent SQL writers and other
replicas need an explicit wake contract.
- Native, review, coalesced, and taskless retry variants remain
conservative until their owner cases qualify. Reviewer quota and hosted
alarm qualification remain open.
- The shared catalog remains opt-in. Legacy callers and missing owner
bindings retain their old blockers.

## Model Used

OpenAI `gpt-6.1-sol` was the configured model for the delegated service
implementations and reviews. OpenAI Codex, based on GPT-6, performed
managing review and integration. Both used repository tools and code
execution. The exact served backend snapshots and context-window limits
were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass on the final merged head
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green — exact current-head run passes
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
complete current-head review and all paginated threads inspected
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-06 13:48:55 -06:00
Nicky LeachandClaude Opus 5.5 98ebeefbc9 feat(environments): let managed config declare plugin environments (#19508)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A managed instance gets its instance-level environments from the
`PAPERCLIP_MANAGED_CONFIG` document
> - #19490 added the plugin environment rows, their marker index and
write floor, but nothing creates them yet
> - The managed config still accepts only one sandbox entry, so a
managed instance cannot get a plugin-provided environment without manual
setup
> - This pull request lets the managed config declare plugin entries
beside the sandbox entry, and provisions one locked row for each at boot
> - The benefit is that a managed instance gets its plugin environments
with no manual setup, and the rows recover by themselves when their
plugin restarts

## Linked Issues or Issue Description

Refs #19490 (the plugin environment rows this pull request provisions)

**Subsystem affected**
Server environments: the managed-config parser, the managed environment
bootstrap, and the environment routes' write floor.

**Problem or motivation**
A managed instance cannot get a plugin environment from its managed
config. The `environments` section accepts one entry, and that entry
always provisions the managed sandbox. Company settings cannot create a
plugin environment either. So each managed instance that needs one
requires manual setup.

**Proposed solution**
Let an `environments` entry declare `driver: "plugin"`. Boot provisions
one locked row for each plugin entry through the row layer from #19490.
A later pull request adds a one-time instance default for one plugin
entry, and advertises the new managed-config capability.

**Alternatives considered**
Manual setup on each instance, and a new control-plane endpoint. Both
need a step outside the deploy. The managed config already delivers the
sandbox entry.

**Roadmap alignment**
This extends the existing managed-config `environments` section, which
already provisions the managed sandbox. It adds no new product surface.

## What Changed

- `server/src/services/managed-config.ts`: an `environments` entry may
set `driver` to `"sandbox"` or `"plugin"`. An entry without `driver` is
still the sandbox entry, so today's documents parse unchanged.
- A plugin entry has `name`, optional `description`, `pluginKey` (the
plugin's manifest id), `driverKey` and optional `driverConfig`.
- At most one sandbox entry. Names are unique across the section, and
`pluginKey`–`driverKey` pairs are unique. Secret-like keys in
`driverConfig` refuse startup, as for sandbox `config`. An unknown key
or driver still refuses startup, and that includes `instanceDefault`
until the follow-up pull request.
- Plugin entries parse into a new `pluginEnvironments` list. The
`environments` list keeps only the sandbox entry, so its consumers do
not change.
  - `MANAGED_ENVIRONMENT_DRIVERS` lists the accepted drivers.
- `server/src/services/managed-environments.ts`
(`applyManagedEnvironments`):
- Boot refuses to start unless an auto-installed catalog entry installs
each plugin entry's `pluginKey`. `plugins.autoInstall` holds catalog
keys, not manifest ids, so the parser cannot check this. Boot uses the
manifest ids `createApp` already resolves from the bundled and
distribution catalogs (`app.locals.managedBundledPluginKeys`, passed in
by `server/src/index.ts`). Without that list it resolves the catalog
itself.
- Plugin entries run after the sandbox entry and reuse its readiness
handling. Boot awaits `pluginsReady`, then requires the plugin `ready`,
declaring the entry's driver, and running a worker. Otherwise boot
archives the row and re-ensures it on the worker's `ready`.
- Recovery re-ensures on every worker `ready` until one ensure succeeds
or the plugin rejects the config (HTTP 422).
- If the ensure fails because the worker died, boot archives the row and
waits for `ready`. If the worker is still running (for example the call
timed out), boot keeps the row as it is, retries once at once and on the
next `ready`, and otherwise at the next boot. No timer is added.
  - Removing an entry never archives its row.
- A plugin-only document does not conflict with
`PAPERCLIP_EXECUTION_MODE`, because plugin rows do not use the sandbox
slot.
- Plugin entries never change the instance default. The sandbox stamp
still fills only an unset or local default, so it never replaces a
plugin default.
- `server/src/routes/environments.ts`: `isPlatformSlotEnvironment`
treats a marked plugin row as a live slot while the document declares
its entry. So the marker-clear escape hatch is refused then, and allowed
again after the entry leaves the document.
- `doc/DEVELOPING.md`: new "Managed environments" section.

## Verification

- `cd server && npx vitest run src/__tests__/managed-config.test.ts`:
old documents parse unchanged. Sandbox plus plugin parses. Each of these
refuses startup: two sandbox entries, `driver: null`, duplicate names, a
duplicate pair, a secret-like key, an unknown key (including
`instanceDefault`), an unknown driver, a malformed `driverKey` or
`driverConfig`.
- `npx vitest run
src/services/managed-environments-plugin-entries.test.ts` (the
sandbox-entry tests stay in `managed-environments.test.ts`; shared
fixtures are in `managed-environments.test-support.ts`):
- A plugin entry is ensured with the validator's worker manager, after
the sandbox entry.
- Boot refuses a plugin entry whose `pluginKey` no auto-installed
catalog entry installs, both from the passed list and from its own
catalog lookup.
- A plugin that is down archives the row, and worker recovery re-ensures
it. A missing plugin, a plugin without the driver, or a worker manager
without `call` counts as unavailable.
- Recovery stays armed after a failed attempt and stops after one
success or a permanent config rejection (422). A `ready` during an
attempt queues one more.
- A worker that dies during the boot ensure archives the row and arms
recovery. A failure while the worker keeps running leaves the row as it
is and retries.
- Removing an entry does not archive. Plugin entries and the sandbox
stamp never change a plugin default.
  - An ensure failure is counted, not thrown.
- `npx vitest run src/__tests__/managed-plugin-environments.test.ts`
(embedded Postgres, real `applyManagedEnvironments`): the first boot
creates the row and the second changes nothing. A plugin that is down
archives the row, and recovery brings it back. Removing the entry does
not archive the row. The instance default stays unset. The row tests
from #19490 still pass.
- `npx vitest run src/__tests__/environment-routes.test.ts`: the marker
clear is refused while the document declares the entry, and allowed
after it leaves.
- `npx vitest run src/__tests__/instance-settings-service.test.ts
src/__tests__/instance-settings-cloud-defaults.test.ts`: pass with the
new `pluginEnvironments` field.
- `pnpm --filter @paperclipai/server typecheck` passes.
- I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm
build`. CI covers them.

## Risks

- No schema change.
- A managed-config document with a plugin entry now provisions a row at
boot. A document without one behaves as before.
- Adoption renames an unmarked row for the same plugin and driver to the
declared name and resets its driver config to the declared one. The row
keeps its id, agents and other metadata, and it is then locked like the
managed sandbox.
- Boot now refuses to start when a plugin entry names a plugin that no
auto-installed catalog entry provides. This is the same fail-closed
posture as an unknown sandbox provider.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (file edits, shell, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 17:35:13 +00:00
DottaandPaperclip 27d35314be feat: connect experimental Cursor Cloud to native Runner (#18069)
## Thinking Path

> - Paperclip helps people manage AI agents at work.
> - Native Runner owns execution and task authority.
> - Operators need an explicit native Cursor Cloud choice.
> - Tools/files need company, task, session and run scope.
> - This connects setup and lifecycle to Runner Core.
> - Legacy agents retain their selected runtime/history.

## Linked Issues or Issue Description

Depends on #18068. Managed-only API placement and API-owned Stop
containment are in #18310. Refs #5664, #11803, #17299, and #17471. No
duplicate native integration was found.

## What Changed

- Add explicit native setup, Test, saved settings and dispatch with
no-prompt account/catalog/callback checks.
- Scope MCP to company, agent, task, session and run through
authenticated HTTPS. Register verified artifacts/work products; preserve
unpriced accounting, completion and budgets.
- Connect Question/Plan continuation, cold handoff and admitted fresh
sessions; recheck waiting admission under the task lock.
- Present audited Stop intent while preserving physical containment
proof; bound typed Stop metadata. Preserve title edits through
preference/navigation readiness.
- Select one live/history representation per run to prevent duplicate
settled cards. Integrate upstream task/read-state/retry/schema contracts
and repair two test fixtures.

## Verification

Core #18068 merged normally as `3910352c923a`. Product head
`5a60f541c982` has passing repository CI/build/browser checks, including
required `ci / verify` and `ci / e2e`, plus a fresh exact-head Greptile
5/5 review with zero unresolved threads. Two non-required external app
checks remain pending: Paperclip Review is queued waiting for its
assigned agent; Superagent Security Scan is reviewing the diff. These
are not claimed completed. The normal merge endpoint enforces the
repository rules; no policy or status override is used. The dependent
Top PR is addressing its fresh deferred-Stop review finding.

Product `5a60f541c982` passed 270 UI and 20 selected
Cloud/native/disk-full tests, UI/server typechecks, tokens and maps.
Top’s clean normal merge passed tokens/maps/diff checks; its matrix is
pending. Earlier receipts retain Core’s 47 Rust, 4 compiled-native
retirement and 19 settlement controls, normal Runner build, and runtime
9af build/recursive typecheck. Reproduced loading/shutdown CI fixture
failures passed 7/7 browser and 121/121 unit tests, including a
broken-unsubscribe negative control. No timeout/oracle was relaxed.

The actual-parent restack adopts 75 mainline paths, including
schema/permission and generic runtime changes, beyond the prior two
fixture repairs and client projection fix. The full difference from
hosted `9af1a5de42a2` is therefore larger than four files; backend
source also differs. The merged config-read checks match Core391, and
the Cursor placement/runtime/relay/selection modules match frozen9af;
selected affected regressions pass. Paid proof keeps its actual 9af
source/image identity. The separately tested live/history card fix is
not claimed deployed on that host.

On 9af, original saved settings survived reopening and fresh no-prompt
Test authenticated catalog/model/binding; no new Save is claimed. Fresh
Linux admission passed. With `claude-sonnet-4-5`, Artifact downloaded
exact 27 bytes and matched 121 canonical identities in complete
S3-served history. First Stop’s strict quiet FAILED on a 10.373-second
gap despite independent cancellation/no late artifact. One targeted
unchanged-sampler retry retained all 24 samples: 35.076 seconds quiet /
maximum gap 4.584 seconds, 67 ordered canonical identities, and separate
CANCELLED/artifact-absent status after the original 180-second deadline.
The pre-reload duplicate settled card is disclosed. Exact-ID cleanup for
the three new provider agents is awaiting authorization, so full grades
and the fresh Artifact’s post-cleanup download remain blocked. A
historical deleted-provider artifact was newly downloaded on 9af;
original F226 Question/Plan/reconnect and cleanup proof keep their
identities.

Retained failures: stale fixture 132/133; E8 48/135 with 87 reset-hook
failures; Mac 17 SQL failures/skips; earlier
title/startup/custody/Stop/deletion failures with unknown causes/bounded
recoveries; `dea023` Stop’s warning/50 duplicate identities; pre-repair
ACK and original Core CI failures. WSP’s 251-assertion, 7/8-suite
attempt failed physical PostgreSQL teardown; the unchanged wrapper rerun
passed 128. Retry alone establishes no cause or full local-suite pass.
All paid attempts remain recorded; vendor dollar cost is UNKNOWN.

## Risks

Experimental: Cursor independently acknowledges neither the effective
model nor instruction digest before the first prompt. Explicit values
are submitted and checked through behavior; unpriced provider dollar
cost stays UNKNOWN.

Callbacks need authenticated reachable HTTPS. Scope, revocation,
budgets, completion and cleanup remain authoritative. Operator Stop
intent is separate from physical containment. Saved legacy agents retain
their selected runtime. Prior shared title-save/deletion failures remain
disclosed.

## Model Used

OpenAI Codex, GPT-6 family, assisted with reasoning, repository tools,
code execution, browser verification, and delegated code review. The
serving model’s exact API ID and context window are not exposed. No
value is inferred from the model catalog.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used and the limits of available model
metadata
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked public issues or described the issue in this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch describes the change and contains no internal ticket ID
- [x] I have run the scoped local tests reported above; pending gates
are stated separately
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented the risks
- [x] All required CI gates pass on the current head
- [x] Fresh Greptile is 5/5 with no unresolved findings on the current
head
- [x] I will address all reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 09:01:39 -06:00
DottaandPaperclip a92f616cd0 fix(db): attribute bounded captures to execution owners (#18634)
## Thinking Path

> - Paperclip manages agent work through shared server and database
services.
> - High query volume can spend main-thread time on SQL construction and
result decoding.
> - The existing bounded response capture counts rows and bytes, but it
cannot name the operation that executed a query.
> - Bundled callers can be missing, and measuring every parsed JSON
value adds work to the diagnostic.
> - This pull request carries a semantic execution owner into each
measured query and bounds response sizing.
> - Operators can rank query volume by owner before choosing a
performance fix.

## Linked Issues or Issue Description

Refs #17504, #17522, #17537, #17564.

**What existing behavior does this improve?**

The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic.

**Subsystem affected**

Database diagnostics and existing server execution boundaries.

**Current behavior**

The capture groups by table, operation, and builder caller. It cannot
distinguish execution owners that reuse a prepared query. Bundled
deployments need source maps for caller locations. The capture visits
every returned value and serializes parsed JSON again to estimate its
size.

**Proposed behavior**

Version 2 captures use fixed semantic owners for HTTP requests, delivery
queues, execution phases, and budget work. They count completed calls,
returned rows, errors, and completion wall time. They sample successful
results 1, 17, 33, and so on, with fixed cell and string/byte limits.
Complex values remain explicitly unmeasured. Semantic ownership works
without source maps.

**Reason and benefit**

A short census can identify which operations issue the most queries or
return the most rows. It preserves each query's execution owner through
pooled connections, concurrent operations, and prepared-query reuse. It
reduces diagnostic work without claiming per-query CPU attribution.

**Breaking changes**

Opt-in capture output moves from `entry.v1`/`summary.v1` to
`entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace
all-result byte estimates. Consumers must use the new fields and must
not extrapolate them into a total. There is no database migration or
application API change.

## What Changed

- Added `withDatabaseOperation(owner, action)` to the DB package. The
server owns semantic labels. Queries retain execution ownership in their
completion closure.
- Added fixed owners at existing HTTP, delivery, budget, and execution
timing boundaries. Delivery work replaces the context of the producer
that woke it.
- Added complete call/row/error counters and completion wall durations
within the existing bounded window. Durations include waits and are not
CPU measurements.
- Sampled and bounded primitive-value sizing. Complex and oversized
values remain unmeasured. Known owners skip repeated stack extraction.
- Preserved the deadline, group limit, paced output, query outcomes, and
existing coverage exclusions. Updated the operator documentation.
- Added real PostgreSQL owner-isolation and prepared-query tests, server
scope tests, and plain bundled-runtime proof.
- Moved projection-test and benchmark full-response sizing into an
explicit test helper. Existing payload-size thresholds remain unchanged;
these synthetic checks do not use the production sampler.

## Verification

- Final-commit DB capture suites: 43 tests passed in 13.86 seconds
against the actual PostgreSQL driver, including concurrent owners,
prepared reuse, transaction failure, a dedicated client, sampling, and a
query crossing the deadline. DB typecheck passed.
- Server attribution and existing delivery/execution/budget suites: 53
tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5
tests passed.
- Runtime bundle suite: 8 existing tests passed. The new no-source-map
owner test passed after its fixture was corrected to use a driver
thenable. An ordinary native Promise can bypass an overridden `.then`;
the real PostgreSQL tests separately cover driver integration.
- Observer-only experiment: three alternating pairs per case on Node
24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an
already-materialized synthetic JSON result, median capture CPU fell from
82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell
from 8.10 to 4.40 microseconds/call. Every result retained identity and
all calls were counted. The new diagnostic intentionally measures less
byte data; these are observer costs, not database latency or a hosted
application speedup. Disabled-case timings were below one
microsecond/call and too small for a performance conclusion.
- Final lifecycle regression: 28 unit tests and DB typecheck passed.
Closing the last capture disables and replaces owner storage;
overlapping captures retain it.
- Final-commit full recursive typecheck passed in 38.38 seconds and full
build passed in 40.81 seconds. The build stamp matches
`83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution
tests also passed 5/5 before the test-helper-only migration. Full test
acceptance will use the repository's parallel CI gates; the unsharded
local command takes hours.
- Verification failures retained: initial sandbox runs could not open
local IPC/PostgreSQL listeners; permitted reruns exercised the real
services. One old test expected JSON serialization and was updated to
assert explicit unmeasured complex values. The first bundle fixture used
the wrong promise boundary, as described above. A retained-callback
regression also demonstrated that disabling and reusing the same
AsyncLocalStorage instance could resurrect an old owner; replacing the
instance after the last capture closes fixes that case.
- The first CI run found older projection tests consuming the previous
full-response fields. Those consumers now use the full-response test
helper, preserving their byte thresholds. The two complete affected
route suites plus helper tests passed 115/115 in 73.08 seconds. A
temporary full-row query negative control then failed the unchanged
2,048-byte limit at 5,000,812 bytes and was restored; five other service
suites passed 34/34 and the affected chat cases passed 3/3. All three
previously failing CI shards passed on the corrected head. A local
skill-list benchmark smoke check completed 255 captured queries with no
in-flight queries or overflow; this was integration verification under
concurrent test load, not a timing benchmark.
- Two final-head runner jobs lost communication with GitHub before
reporting a test result. One targeted retry of those jobs and their
dependent aggregate passed. All 53 final-head checks passed, with 3
intentional skips; previously passing suites were preserved.
- Hosted capture remains pending until the reviewed revision is deployed
and a bounded window is explicitly configured.

Reproduce the focused checks with Node 24 and the repository pnpm
version:

```sh
pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts
pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs
pnpm -r typecheck
pnpm build
```

## Risks

- Owner context is active while a valid future capture window is armed,
up to 30 minutes before accounting starts. No-window and expired calls
take the direct path. Diagnostics still add bounded work while enabled.
- A nearest nested owner is reported. Broad `http.request` and
unattributed groups remain visible; this does not identify every service
callsite or SQL shape.
- The existing census excludes tagged/raw explicit-execute/streaming
paths and internal protocol commands. Its counts are not PostgreSQL
transaction totals.
- Completion wall time includes queue/database/network/scheduling waits.
Concurrent durations overlap. Sampled bytes exclude complex values,
large strings, and values beyond the sizing limits.
- Activation is startup-only. Operators must choose the window after
artifact preparation, allow rollout/warm-up time, and remove the expired
setting afterward.

## Model Used

OpenAI Codex, GPT-6 family, with code execution, repository inspection,
and parallel agents. The exact deployment model ID and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 12:12:21 -06:00
DottaandPaperclip 006d5691a6 feat(server): support volume-free web server handoff (#18518)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its web server also owns controller admission and user commands.
> - A rolling deployment starts a replacement before the old server
exits.
> - Shared database state alone does not preserve local agent files,
browser chunks, or in-process commands.
> - This pull request adds an opt-in ephemeral server contract and a
local overlap test.
> - The benefit is a web server that can relinquish traffic and local
files without losing acknowledged work.

## Linked Issues or Issue Description

Refs #18403, #18327, #18335, #18341.

**Current behavior**

The control-plane web server still has local agent-file authority and
local command targets. A new container can miss saved bytes or send a
command to the wrong process. An existing browser tab also needs chunks
from its original build.

**Proposed behavior**

Eligible servers share only PostgreSQL, object storage and stable
secrets. A replacement waits for readiness. The old server drains
admitted requests, keeps owner commands available through handoff, and
releases controller leases. Explicit maintenance adopts existing files
before the volume is removed.

**Reason and benefit**

This is the application prerequisite for rolling app-only deployments.
Schema migrations and permanent multi-replica operation are outside this
change.

## What Changed

- Store agent-file manifests in PostgreSQL and immutable bytes in S3.
Treat local roots as disposable projections. Preserve existing
permission checks and edit conflicts.
- Add an explicit migration, eligibility and object-retention command to
the production runtime bundle.
- Add `/api/ready`, admitted-request drain, browser restart signaling
and retained immutable UI chunks.
- Carry transaction-aware work hints across controllers during handoff
and active execution. Keep listeners closed when idle.
- Deliver durable Stop and runtime responses to the execution owner.
Recheck request lifecycle and access before response dispatch.
- Reject local-only configuration in ephemeral mode. Keep the feature
disabled for existing installations by default.
- Add an isolated PostgreSQL/S3/proxy/browser deployment harness with
independent server homes and retained failure evidence.

## Verification

- All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`:
workspace typecheck, build, Runner checks, server/general tests and
browser shards. Greptile reviewed the runtime changes at 5/5 with all
findings resolved; the final commit updates maps and the plan.
- Local targeted checks passed: coordinator 27; Stop/recovery 33;
PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap
Stop 1; restart recovery 75. PostgreSQL command tests simulate the
provider edge.
- Local browser proof passed 91 reads and 19 saves; reconnect took
1,013ms. The tab opened Audit from S3 after deletion of A's whole
home/build. This uses compiled workspace code and local-trusted auth,
not released containers or live agents.
- Local workspace typecheck/build passed; current server and runtime
bundle passed. The earlier broad local suite was stopped after three
hours and is not a passing receipt. Its retained failures and isolated
reruns are documented in the plan.
- Stacked #18526 adds stricter graceful-exit proof, an unready
replacement control, live Runner qualification and a Stop handoff
correction. Hosted enablement remains gated on those results and Cloud
#1096. No hosted stack has been enabled.

## Risks

- Apply migration 0422 before enabling the feature. Existing data must
pass each owner's migration and eligibility checks. Do not discard a
volume based on a single migration page.
- Durable manifests remain authoritative after the opt-in is disabled.
Rolling back to a binary that ignores them is unsupported.
- Cold directory restoration and complete shutdown timing still need
qualification before enabling a hosted stack.
- Retained UI assets must cover the supported browser and rollback
window. Cleanup is explicit; no new idle maintenance polling is
introduced.
- This change does not claim production zero downtime or authorize a
fleet-wide enablement.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window limit are not exposed in this session. Used reasoning,
code editing, shell execution and browser automation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-01 01:58:54 -06:00
DottaandPaperclip eb12cfc96b perf(github): activate connection work from events and deadlines (#18395)
Reviewed and verified at f77620b9a4.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-31 15:38:43 -05:00
DottaandPaperclip 5e4cc01073 feat(workspaces): keep remote files off the webserver disk (#18403)
Keep native sandbox workspace files independent of the webserver disk. Use encrypted saved versions and Postgres ownership so a replacement server can recover files and continue work. Enable the path by default for new eligible workspaces when durable storage is configured.

Preserve explicit opt-outs and existing host workspaces. Include safe snapshot repair, Files access, import, migration and retention. Bound saved-Files resources, preserve imported Git recovery, reject unfinished Git operations, publish immutable concurrent captures, and verify permanent object-version deletion.

Validated 471 targeted local controls, build/typechecks, all 40 CI partitions, fresh Greptile 5/5 and security review. The PR records original failed attempts and scoped retries.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-31 14:54:01 -05:00
DottaandPaperclip 8ff812cdde feat(chat): deliver proactive task messages to agent chats (#18374)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat gives each person a persistent conversation with each
agent.
> - Tasks and routines produce results outside that conversation.
> - Those runs need a safe way to bring a requested result back to the
person.
> - The server must select the agent's own chat and check the source
task's access rules.
> - This pull request adds proactive chat messages and unread
indicators.
> - People can see the result, open its source task, and reply in the
same chat.

## Linked Issues or Issue Description

**Subsystem affected**

Server, native runner tools, shared contracts, database, and UI.

**Problem or motivation**

An agent can report progress on a task or comment on an authorized task.
It cannot send a routine result to its own standing chat with the
responsible person. A morning review routine therefore cannot leave a
visible digest in that person's chat.

**Proposed solution**

Add `send_chat_message({body, idempotencyKey})` and `POST
/api/agents/me/chat/messages`. The server selects the company, agent,
and responsible person from the active run. It saves an attributed
comment and audit record in one transaction. The chat shows a
source-task link and per-person unread state.

**Alternatives considered**

A destination parameter on a generic comment tool would expose
unnecessary routing authority. A new delivery worker would add another
durable delivery path. This change uses a dedicated tool and the
existing comment and live-event paths.

**Roadmap alignment**

This extends the experimental Agent Chat capability in `ROADMAP.md`. The
existing `enableAgentChat` setting controls it. Related public work:
Refs #13282, Refs #13283, and Refs #14408. Searches for proactive chat
and unread chat found no duplicate PR.

## What Changed

- Add a focused messaging service shared by the native tool and HTTP
route. It checks current run ownership, execution mode, recipient
membership, task access, and task privacy.
- Reuse the canonical company/agent/person conversation. Save the
comment and audit record together. Keep task completion and chat
execution separate from delivery.
- Add source-task provenance and source-scoped retry keys. Identical
retries return the saved receipt. Changed content returns a conflict.
- Enforce the conversation's fixed agent at the shared comment-write
boundary. Keep normal reply cancellation and session checks.
- Add chat unread state, accessible agent dots, aggregate Chat
indicators, and visible-message read cutoffs. Refresh through existing
live events.
- Add the request-key constraint, native tool contracts, documentation,
capability maps, stories, and regression coverage.

## Verification

- Local delivery/security proof: all 38 proactive-message cases pass.
Eight credential-scope regressions and four quota regressions fail
against the prior source. The repaired delivery and canonical
cross-issue cap suites pass all 49 cases; all 31 OpenAPI tests, server
typecheck, and capability-map checks pass. The shared per-run cap covers
native and HTTP sends, mixed ordinary mutations, concurrent last-slot
contention, and retry/recovery without double charging.
- Credential tests preserve the authenticated person and authorizer;
reject bridge/test credentials and persisted skill-test runs after
task-mode changes; and recheck API-key revocation, scope, and
responsible person after destination locking. The earlier
delivery/native-authority suites passed all 74 cases before the four
quota additions.
- Four PostgreSQL contention cases cover native and HTTP sends plus
retries during a native chat mutation. All four deadlock against the
earlier source and pass with destination-before-source/run locking.
Recovery and human-reply delivery enqueue only the human comment; a
proactive digest never wakes the chat agent.
- Passed 108 companion conversation, native-authority, and OpenAPI
tests; 112 runner catalog/workflow-report tests; 47 completion-delivery
tests; and 25 project-handoff tests. Focused runner tests cover native
tool availability without generic API tools.
- All 327 affected UI tests, UI typecheck, token gates, and catalog
checks pass. The read hook retries at most three times, respects server
retry delay, coalesces only seen cutoffs, recovers on reconnect, and
cancels on scope changes.
- Real browser journeys with a deterministic process agent verify
routine-to-chat delivery, first-chat creation, unread state across
reload, the source link, secondary chat navigation, and a follow-up
reply referring to the digest. This does not qualify live GitHub
retrieval or model output quality.
- The Storybook showcase uses production components with deterministic
network fixtures. Rendered interactions pass for the landing, ordinary
read success, transient read recovery, and persistent failure after
bounded retries. Both Chat navigation variants update from two unread
chats to one while only Kodi's dot clears. Mobile and light variants
were reviewed. The existing Coordinator Recents label contrast finding
is unchanged; no Linux visual baseline archive exists.
- Current master's independent sandbox migration retains 0418.
Regenerated the feature migration as 0419 from the combined schema and
retained exactly the newest five snapshots; numbering/safety, replay,
and schema-drift checks pass.
- Workspace typecheck and build passed before PR preparation; canonical
server/UI typechecks passed again on the combined tree. Migration
`0419_slippery_impossible_man.sql` passes safety and schema drift checks
on disposable PostgreSQL, including interrupted index repair, concurrent
migrators, one-time owner read-state backfill, preserved receipts, and
two SQL replays. A default five-second local snapshot comparison timed
out; a diagnostic 30-second run passed without drift.
- Earlier failures are retained in the record: full local `pnpm
test:run` and broad shards encountered embedded-PostgreSQL startup,
timing, and provider-recovery failures. Thirteen selected
external-channel cases pass in isolation on the original and changed
sources; an unordered comment-row assertion now explicitly orders both
reads. The earlier route registry, native tool count, runner catalog,
and accessible Chat badge-name failures were fixed and passed full
remote CI on `920b3bb7db`.
- Repeated managed-skill server8 failures came from CI's newer synthetic
merge: its Cloud guard correctly rejected the old local-disk fixture.
Master independently repaired the fixture in #18385; this branch keeps
that repair unchanged. All 57 managed-skill and Cloud prerequisite cases
pass on the combined tree, and that fixture has no remaining PR diff.
- Current reviewed head `dc16f314739a6d977bed21cb1446d0e80a541578`:
Greptile 5/5 with no findings, Superagent security assessment passed,
advisory Paperclip Review 5/5 with no findings, and zero open review
threads. All 55 current-head check runs pass, with five expected skips
and the advisory review neutral at 5/5; Snyk also passes. No checks are
pending or failing. Protected squash auto-merge is enabled and is
waiting for required code-owner approval. A normal exact-head merge was
refused by the base-branch policy; no admin bypass was used. The
previous head's Superagent quota finding is fixed and covered above.

## Risks

- Chat remains company-visible. Delivery rejects a source task with
private-task, project, or ancestor restrictions.
- Read state uses a comment timestamp cutoff. The server retains
database precision and never moves a read receipt backward.
- The new unique constraint changes comment query planning. An existing
regression now uses an explicit row order. The migration builds the
unique index concurrently, then takes a brief table lock to attach it.
It uses the existing top-level migration and interrupted-index recovery
path.
- The upgrade backfills missing chat-owner read-state rows at the latest
existing agent comment. Existing read receipts remain intact. New chats
can still be unread before the human participates.
- Quota observation commits atomically with delivery. A rejected send
rolls back its attempted denial audit row and any first-chat creation;
the canonical warning remains in server logs, and previously spent quota
remains enforced. Identical/conflicting retries and receipt recovery
spend no extra slot.
- This version delivers in-app only. It adds no external-channel, push,
or email delivery.
- Local full-suite acceptance remains limited by the failures recorded
above. The feature stays behind `enableAgentChat`.

## Model Used

OpenAI Codex. PR preparation and CI work used `gpt-6.1-sol` with tool
use and code execution. The earlier implementation also used Codex, but
its exact model ID was not recorded in the visible session. The
context-window size was not provided.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass; the broad local
failures are recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 21:39:07 -05:00
DottaandPaperclip 0e15c92073 feat(skills): make managed content independent of web-server disk (#18334)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use installed skills and exact version pins during execution.
> - Managed skills still depended on mutable directories on the web
server.
> - A replacement server could lose acknowledged edits or remove an
unassigned skill whose directory was absent.
> - This pull request makes immutable versions the authority for managed
content and installed repository reads.
> - The server can rebuild delivery files from durable content after
replacement.
> - This establishes skills readiness for ephemeral web servers. General
server draining and workspace recovery remain separate work.

## Linked Issues or Issue Description

**Subsystem affected**

Skills services, shared contracts, database receipts, Skill Studio, and
runtime delivery.

**Problem or motivation**

A fresh web server must discover, read, edit, and deliver protected
installed skills using the same database, object storage, secrets, and
supported application artifacts. It must not need its predecessor's
files or process memory.

**Proposed solution**

Use `managed` ownership with a null source locator. Read installed
content by immutable version. Prepare packages before a short
publication transaction. Persist execution selections before
materialization. Recover exact pinned runtime assets and reject deletion
while active or recovering pins exist.

**Alternatives considered**

Reconstructing a mutable installed directory preserves the old
source-of-truth ambiguity. Permanent dual writes create two competing
authorities. This change uses one managed model after a bounded legacy
conversion.

**Roadmap alignment**

This improves the existing Skills Manager, Studio, version history, and
Cloud execution capabilities. The maintainer supplied the implementation
contract. This does not add a separate skills product or deploy fleet
changes.

Related work: #18082 supplies immutable package storage. #17273
addresses edits outside the API for linked local skills; this PR keeps
that source category separate. #2648 is a closed earlier
ephemeral-filesystem proposal.

## What Changed

- Add managed ownership and focused content publication and execution
materialization modules. Keep the archive format and package storage
tables.
- Publish managed create, save, delete, fork, rename, restore, and
catalog content with version checks. Add durable create receipts.
Preserve file-save receipt responses.
- Return file version identities. Keep Studio drafts after a conflict
and provide an explicit reload action. A delayed reload cannot overwrite
newer typing or navigation.
- Persist exact runtime selections, including empty selections. Rebuild
pinned assets and verify their admitted digests on a replacement server.
- Preserve source access on forks and disconnected copies. Keep the
existing workspace policy category for managed skills.
- Add bounded conversion, eligibility, and independent verification
phases. Load persisted Cloud configuration in the standalone CLI without
seeding or changing it. Reject partial current versions against
installed inventories; recover only complete saved content or the exact
source. Document storage migration blockers and the compatible rollback
floor.
- Add separate-process PostgreSQL/S3 qualification and update domain,
retention, policy, and UI regressions.
- Pass the plugin worker manager into Studio execution, admit restricted
`skill_test` mode through native Runner contracts, and preserve admitted
scope across native and REST tools. Native tests cannot create ordinary
tasks, browse company data, or use external tools.
- Select the exact durable Studio revision even when unassigned or Beta
skills is off. Persist Studio output and audit atomically with native
task completion, preserving the result on replay.

## Verification

- Final revision `13e8af480b88ba6fea5737bac4bd81104d604792` passed full
CI: all 49 workflow jobs completed successfully ([run
41127396098](https://github.com/paperclipai/paperclip/actions/runs/41127396098)).
This includes typecheck, test suites, Runner checks, browser checks and
build. Greptile review 15 is 5/5 on that same revision, with zero
unresolved threads. The PR is ready for review.
- Local workspace typecheck and build passed after integrating master.
Focused checks passed: 33 execution/completion cases, 52 Runner HTTP
cases, 16 managed-content cases, 17 Studio cases, the new real-process
CLI regression, map checks and UI token gates. The CLI regression proves
inspection does not seed missing Cloud configuration. Server typecheck
passed after that fix.
- The independent-process PostgreSQL/S3 matrix passed through normal
HTTP skill routes. It covers A → B → C replacement, lost commit
responses, upload crashes, concurrent saves, binary contents and
executable delivery. Metadata listing and warm selected delivery retain
zero object downloads. Local timing measurements are recorded; they do
not establish hosted latency.
- Hosted A → B → C passed on the owned disposable test tenant with
external Postgres, S3 and a real remote Paperclip Computer. Predecessor
temporary homes were absent. C read and executed B's acknowledged
managed save. New runs used repository v1 while v2 was installed. Hashes
matched an independently checked exact upstream commit.
- Compatible image rollback to `6518b323` passed. Studio retained the
unassigned fork's C save. A fresh remote run executed the newer managed
content and historical repository pin. The actual shell event confirmed
expected hashes, stdout, exit 0 and mode 0555.
- Runtime candidate `9200e3fcb4` deployed successfully ([Cloud run
41124147876](https://github.com/paperclipai/paperclip-cloud/actions/runs/41124147876)).
The final Studio test delivered only the unassigned fork's exact v3,
executed its script and persisted a successful output snapshot.
Independent command-event inspection confirmed all three hashes, stdout,
exit 0 and mode 0555. All 14 installations were version-backed.
Independent verification checked all 25 versions: 23 remote objects, two
bundled database snapshots, zero blockers.
- The deployed image is `9200e3fcb4`. Later master integration, the
client-only delayed-reload draft guard and the CLI bootstrap have
separate automated proof. The CLI bootstrap was also exercised against
the tenant's stored configuration and storage through the same read-only
services. This PR does not claim that the deployed image is the final PR
revision.
- Live Studio checks exposed worker wiring, admission, access scope,
exact selection and completion-publication gaps. They are fixed. Failed
attempts remain in the evidence. The full local suite exceeded 80
minutes under host contention and was stopped. Later combined/isolated
local attempts also timed out; they are not counted as passes. Full CI
covers the integrated suites. Earlier CI found an obsolete restart-scope
fixture, which was corrected and passed all 52 HTTP cases.
- Reproduction and migration:
[`doc/skill-ephemeral-web-servers.md`](doc/skill-ephemeral-web-servers.md).
Source revisions, deployments, versions, run IDs, hashes and failure
history:
[`doc/plans/2026-10-30-ephemeral-web-skills.md`](doc/plans/2026-10-30-ephemeral-web-skills.md).

## Risks

- Legacy conversion requires stopped admission and retired
filesystem-authoritative writers. Pre-conversion binaries are not
supported rollback targets afterward.
- Cloud readiness requires remote object storage. Local provider
receipts and provider mismatches remain blockers until objects are
copied and verified.
- Linked local sources retain filesystem semantics and are outside the
cloud guarantee.
- Cold reads still fetch the existing archive format. Warm runtime
preparation retains the verified-cache path.
- Bundled reconciliation preserves an installed head. Existing release
snapshots remain selectable. An older server must not replace a newer
head with its shipped files.
- No production volumes, fleet defaults, or general deployment
coordination change in this PR.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools, code
execution, and browser use. The exact backend model ID and
context-window size are not exposed in this session. No subagents were
used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 16:18:27 -05:00
DottaandPaperclip 8f822bfcf7 fix(cloud): persist configuration and load release-image packages (#18335)
Land the approved Cloud configuration and release-image package stack.

Includes the separately reviewed adapter settings (#18335), external adapters (#18333), bundled plugins (#18336), startup configuration (#18339), and migration/package checks (#18341).

Includes the final image-owned loader and linked-dependency inventory corrections.

Verified stack head: 462716170d.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-30 19:21:33 +00:00
1bceae0c91 Make import spool cleanup wake on changes and saved deadlines (#17835)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/38/changes/paperclip/38
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip manages agent work and the files that support it.
> - Resumable company imports keep uploaded parts until apply or expiry.
> - An hourly cleanup loop reads the database for every retained upload
directory.
> - Failed and applying imports can retain files without owing periodic
cleanup.
> - Existing transfer rows and file timestamps already identify real
deadlines.
> - This change schedules those deadlines and wakes cleanup after
relevant changes.

## Linked Issues or Issue Description

Refs: #17801.

**What problem does this solve?**

The hourly import sweep keeps querying retained uploads when no cleanup
is due. It also treats a failed database read as a missing transfer row.
That can cause deletion of a live upload whose directory is old.

**What is the proposed solution?**

Use the existing delivery coordinator with per-transfer deadlines. Keep
commit-aware notifications in the transfer ledger service. Keep
filesystem change notifications in the spool owner. Recover pending
cleanup from existing files and rows at startup.

**What alternatives have you considered?**

A slower periodic sweep would still query idle history. A new job table
would duplicate the existing ledger and directory timestamps. Retrying
only after successful uploads would miss partial file writes that leave
cleanup work.

This is slice 59. Parent #17801 has merged. This publication targets
master and preserves that landed prerequisite. Fresh current-head CI and
review remain required.

## What Changed

- Repair an inherited CI concurrency test. Observe exact PostgreSQL
blockers instead of assuming lock order from two 50 ms sleeps. Keep
production authorization and all fail-closed assertions unchanged.

- Refresh verified-upload activity before writing bytes. Preserve the
guarded cleanup claim without recording a part receipt before the file
exists.

- Isolate unreadable child directories so healthy cleanup continues. Use
unique, self-cleaning route fixtures for external database reruns.

- Replace the hourly timer with the existing scoped deadline worker. No
new table or independent scheduler.
- Schedule open uploads at saved activity time plus 24 hours, preserving
the strict cutoff and guarded cancellation before deletion.
- Keep failed and applying imports quiet while retaining their files.
Clean completed/cancelled leftovers immediately.
- Register run-scoped write intent before every supported ledger
mutation, including nested transactions. The helper stays inside the
existing ledger owner.
- Wake from filesystem changes, including failed writes and removals.
Reconstruct orphan deadlines from directory timestamps after restart.
- Propagate database and permission errors. Only a missing directory
proves absence. Read only the four ledger fields needed for cleanup.
- Preserve startup-only stranded-apply recovery. The cleanup worker
cannot fail live applies. Remove its filesystem subscription and join
active cleanup on shutdown.
- Record producer coverage and limits in
`doc/plans/2026-10-26-import-spool-cleanup-sleep.md`.

## Verification

Current repair source: `5d974e5eb28fc3e28b5e75e36fb18c61582348d5`. The
publication adds a test-fixture repair and audit text to the previously
tested import-spool source; runtime code is unchanged. **765 distinct
cases pass across nine complete files:** the original 293 consumer cases
remain hash-identical, and the full tool-access file passes all 472 in
20.86 seconds.

Controlled evidence: a 500 ms cleanup-query delay fails all three old
timing cases; observed lock barriers pass all three with the same delay
and unchanged policy assertions. The artificial delay is not committed.
Current-head CI's original audience-race failure is preserved. The first
local full-file run also retains one separate 15-second stdio HTTP
timeout; one fresh diagnostic full run passes all 472, without
establishing that timeout's cause or changing its limit. Fresh CI and
reviews are required on this new publication head.

Accepted production source: `87549234b125464dc7fa216227fc9bfa1d93f3d7`.
Publication head: `016b3af51a734ad0ddf574de701679a7bdc33d06`. Adopting
the actual parent squash preserves the complete accepted child tree. The
subsequent changes are the CI test fixture and its audit text.

- **293 cases pass across eight complete server files, with zero
skips:** spool worker 18; HTTP import transfers 25; transfer ledger 9;
cloud floor 8; portability 114; startup 92; coordinator 26; scoped
deadlines 1. Runtime: 12.28 seconds.
- Real filesystem and PostgreSQL cases cover zero idle SQL/directory
scans, actual deadlines, first/concurrent writes, nested
commit/rollback, restart, failure, drain/resume, root and symlink
boundaries, and joined shutdown. Three fresh PostgreSQL 17 fixtures and
isolated PAPERCLIP_HOME keep the run separate. All eight source hashes
remain exact; no assertion, timeout, or source adaptation.
- **Static gates pass:** database build; server typecheck/build; module
boundaries; migration order; profile identity. All 412 migration journal
entries are present through index/tag 0413_tired_kronos.
- **Composition:** 1,237 parent-only and eight owned-only paths remain
exact; 11,521 identical paths and two automatic shared production
results are preserved. The only manual conflict is the status ledger.
Packages and other native worker services equal the parent. Its 4,192
selected cases are prior source-scoped evidence, not added to this
count.
- **Historical failures remain recorded:** earlier trusted-runner and
Greptile service errors are not current passes. The older Ponytail Ship
verdict names another head. Fresh CI, security, and exact-head Greptile
5/5 must pass before merge.
- [Master integration and spool acceptance
audit](https://github.com/paperclipai/paperclip/blob/016b3af51a734ad0ddf574de701679a7bdc33d06/doc/plans/2026-10-26-import-spool-cleanup-sleep.md#current-master-integration-and-spool-acceptance-2026-10-29).
No full local repository, live provider, or deployed sleep claim is
made.

<details><summary>Earlier revision verification and failure
history</summary>

Receiver failure fairness: a controlled 1,100 ms batch reproduced a
30-second delay for healthy due receivers. The worker now supplies its
existing per-item deadline through `nextRunAtOnError`. All 20 receiver
lifecycle tests pass on the owner and combined tip
`31c12f1d68ff4a51e06fc2be2abf2c2aee9f9d3a`; server production
typechecking passes on the owner. The failed receiver retains its own
backoff, and settled receivers return to zero recurring SQL. The
five-second assertion budget is unchanged. See [the owning
audit](https://github.com/paperclipai/paperclip/blob/7435a19f6d18f14466175d72b3b7d893ed39acd0/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#healthy-receiver-progress-after-a-slow-failed-batch).
Current-head CI and review remain required.

Approved delegated Test-tab calls now evaluate caller and target
policies in the existing claim transaction. Real HTTP regressions first
showed a missing caller charge and shared-limit self-rejection. All 78
connection-management cases and 92 gateway cases now pass on the owning
branch, along with server production typechecking. All 78
connection-management cases also pass at combined tip
`d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover
shared charges, separate charges, rollback on denial, duplicate
approval, and rejection of a later authorized call. Intermediate
counter/identity fixture failures and a reused-database Slack collision
are recorded in [the owning
audit](https://github.com/paperclipai/paperclip/blob/1509a56a97f2ffb2ab59a9a27f6d7be16ab46569/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes).
The earlier combined checks below remain evidence for their stated
revisions. Current-head CI and review remain required.

Inherited voice-cleanup verification: the controlled slow-provider
regression and all 13 lifecycle tests passed on its owner and combined
tip `ab22d8ed0c09028315607414b83b375e5cc0a60a`. That repair changed only
the fixture clock advancement and its audit note.

Current head: `7435a19f6d18f14466175d72b3b7d893ed39acd0`. This remains a
single-duty PR stacked on #17801.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/47ba0a93c4b01484ad58d16bbeb5a372c76b5a0d/doc/plans/2026-10-24-recurring-server-sleep-stack.md),
[2026-10-26-import-spool-cleanup-sleep.md](https://github.com/paperclipai/paperclip/blob/47ba0a93c4b01484ad58d16bbeb5a372c76b5a0d/doc/plans/2026-10-26-import-spool-cleanup-sleep.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.


</details>

## Risks

- Expiry and active apply must keep their guarded ledger claims. A stale
read must never authorize file deletion after a concurrent resume.
- Database and permission failures remain unresolved work. They cannot
be interpreted as missing rows or empty directories.
- Only supported ledger and spool owner writers emit these local
notifications. Independent processes or SQL writers need ingress;
startup recovery alone cannot wake stopped compute.
- Cloud sleep blockers remain. No deployed suspension, external alarm
coverage, live provider qualification, or cost savings is claimed.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, code execution, real
PostgreSQL and filesystem tests, and GitHub tools. The exact runtime
model identifier and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes:`
/ `Refs:` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Codex <noreply@openai.com>
2026-10-29 07:42:14 -05:00
829f9e7eaf fix(server): settle Muse terminal assignments from committed changes (#17793)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/12/changes/paperclip/12
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its external-agent module owns Muse assignments, operation receipts,
and worker holds.
> - Running assignments and retained terminal history must not cause an
empty maintenance scan every five seconds.
> - Terminal run changes and operation results already have durable
database transactions.
> - This pull request wakes the existing maintenance worker from those
commits and reconstructs work at startup.
> - The worker retries only concrete settlement debt. It retains
qualification deadlines and authority checks.

## Linked Issues or Issue Description

Refs: #17772. This is slice 57. Parent #17772 has merged.
Related: #17000, #17525, #17752, #17758. The publication targets master
and preserves the merged parent.

**What existing behavior does this improve?**

Muse terminal assignment settlement and the existing Muse maintenance
worker.

**Current behavior**

A running assignment can keep maintenance scanning every five seconds.
Terminal reconciliation rewrites unchanged history. Native finalization
also calls that reconciliation during periodic replay. Discovery
excludes a revoked original binding even when its assignment still needs
settlement.

**Proposed behavior**

Committed run changes and operation receipts wake the existing worker
with the original company, run or binding identity. Startup reconstructs
actionable work from existing rows. Maintenance processes bounded
batches and disarms after settlement. Default live and manual callbacks
remain. The periodic callback delegates this one duty.

**Reason and benefit**

Configured but idle instances stop querying for this obligation. Late
receipts, restarts, drain and historical bindings still have recovery
paths. No table or independent timer is added.

**Breaking changes**

No schema, endpoint or provider protocol changes. Explicit manual
reconciliation retains its unbounded default. This does not alter Cloud
sleep admission or claim that all instance work can sleep.

## What Changed

- Add an owner predicate shared by discovery and locked settlement.
Fenced private-worker uncertainty remains held without an automatic
retry loop.
- Preserve original binding generation and company/agent/run identity,
including revoked bindings. Fence only the terminal assignment during
reconciliation.
- Extend existing maintenance with scoped hints, bounded batches,
concrete-debt retry, startup reconstruction and quiet drain behavior.
- Register notifications inside existing terminal, projection,
cancellation, operation-result and send-failure transactions. Keep
qualification and OAuth signals explicitly global when coalesced with
scoped changes.
- Serialize only four domain IDs. Real native-finalizer tests caught and
prevent a circular-object serialization failure at this boundary.
- Use existing restore/worktree suppression and settings reactivation.
Delegate both periodic finalizer branches while retaining resource
callbacks and default live/manual settlement.
- Add the producer audit and regression evidence in
`doc/plans/2026-10-26-muse-terminal-settlement-sleep.md`.

## Verification

Accepted production source: `584015018ed4a9d57d2b0f192b75847503b54e53`.
Publication head: `305014b030d6c5eb891c70bc57036f4192d4fdad`. Normal
adoption of the actual parent squash preserves the complete accepted
child tree. Only audit text, a migration-count clarification, and the
proved Photon test extension follow.

- **3,715 distinct cases in 52 complete server files pass**, plus **20
selected chat cases** (3,735 selected total). The main 49-file batch
passes 3,008; process recovery passes 410; review response passes 273;
the full Photon file passes 24. The other 1,794 chat cases were not run
locally.
- **Quiet producer proof:** moving only the Muse fanout outside its
parent guard produces six candidate SELECTs and fails the regression.
Keeping the guard yields zero and still settles a real terminal run. The
same test hash spans the controlled red/green pair.
- **Photon review evidence:** repeated active callbacks produce zero
settlement hints/candidate SQL. Three concurrent attention recoveries
produce exactly one of each. The existing locked attention guard was
correct; production needed no fix. The parent security check accepted
the resolved finding before merge. This PR keeps the additional
settlement queue in the existing regression.
- **Static checks pass:** database build; server typecheck/build; module
boundaries; migration order; current Copilot identity. Packages equal
the accepted parent byte for byte. No new live provider qualification is
claimed.
- **Fixtures and failures:** 34 fresh PostgreSQL 17 fixtures have 412
applied migration receipts through tag 0413_tired_kronos, plus fresh
per-call helper fixtures. All 52 original test hashes restore exactly.
The additional Photon test hash matches its passing proof. Earlier
remote chat passive-idle failure remains recorded; both passive/missing
cases pass in the new selection. Parent handoff timeout/fresh-file-pass
accounting remains in its audit. No assertion or timeout was weakened.
- [Integration and quiet producer
audit](https://github.com/paperclipai/paperclip/blob/305014b030d6c5eb891c70bc57036f4192d4fdad/doc/plans/2026-10-26-muse-terminal-settlement-sleep.md#current-master-integration-and-quiet-producers-2026-10-29).
Fresh CI, security, and exact-head Greptile 5/5 must pass before merge.

<details><summary>Earlier revision verification and failure
history</summary>

Receiver failure fairness: a controlled 1,100 ms batch reproduced a
30-second delay for healthy due receivers. The worker now supplies its
existing per-item deadline through `nextRunAtOnError`. All 20 receiver
lifecycle tests pass on the owner and combined tip
`31c12f1d68ff4a51e06fc2be2abf2c2aee9f9d3a`; server production
typechecking passes on the owner. The failed receiver retains its own
backoff, and settled receivers return to zero recurring SQL. The
five-second assertion budget is unchanged. See [the owning
audit](https://github.com/paperclipai/paperclip/blob/f7477933469fdc960f555425dce66eff25a94e54/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#healthy-receiver-progress-after-a-slow-failed-batch).
Current-head CI and review remain required.

Approved delegated Test-tab calls now evaluate caller and target
policies in the existing claim transaction. Real HTTP regressions first
showed a missing caller charge and shared-limit self-rejection. All 78
connection-management cases and 92 gateway cases now pass on the owning
branch, along with server production typechecking. All 78
connection-management cases also pass at combined tip
`d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover
shared charges, separate charges, rollback on denial, duplicate
approval, and rejection of a later authorized call. Intermediate
counter/identity fixture failures and a reused-database Slack collision
are recorded in [the owning
audit](https://github.com/paperclipai/paperclip/blob/bbd31dea5ef3a4875950a87b4636af8b6bde858e/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes).
The earlier combined checks below remain evidence for their stated
revisions. Current-head CI and review remain required.

Inherited voice-cleanup verification: the controlled slow-provider
regression and all 13 lifecycle tests passed on its owner and combined
tip `ab22d8ed0c09028315607414b83b375e5cc0a60a`. That repair changed only
the fixture clock advancement and its audit note.

Current head: `f7477933469fdc960f555425dce66eff25a94e54`. This remains a
single-duty PR stacked on #17772.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/ff196791db6dada2dc04ea9db0d58c5c3beb4a41/doc/plans/2026-10-24-recurring-server-sleep-stack.md),
[2026-10-26-muse-terminal-settlement-sleep.md](https://github.com/paperclipai/paperclip/blob/ff196791db6dada2dc04ea9db0d58c5c3beb4a41/doc/plans/2026-10-26-muse-terminal-settlement-sleep.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.


</details>

## Risks

- In-process signals do not capture independent SQL or other replicas.
Supported external events still need ingress.
- Pending assignments and operation receipts retain their original
binding identity. A new agent configuration cannot take their ownership.
- Genuine qualification, OAuth, and recovery deadlines remain
obligations. They must not be discarded merely to make the worker quiet.
- The parent coarse-hint guard prevents recurring Muse discovery; real
company/binding authority changes retain their signals.
- No full local repository suite, full chat suite, live Muse or Copilot
qualification, deployed sleep, database suspension, external alarm
coverage, or cost savings is claimed. Existing Cloud blockers remain.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, code execution, real
PostgreSQL tests, and GitHub tools. The exact runtime model identifier
and context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Codex <noreply@openai.com>
2026-10-29 06:18:40 -05:00
DottaandPaperclip 7afee8f062 fix(server): schedule direct GitHub webhook recovery from saved deadlines (#17666)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/218/changes/paperclip/218
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub callbacks can fail before the app records an inbox row.
> - Direct GitHub integrations need bounded recovery from provider
delivery history.
> - The chat timer checks for due recovery every second, even without an
eligible integration.
> - Existing recovery checkpoints already store the next scan time and
runtime epoch.
> - This pull request schedules those saved obligations and wakes on
committed configuration changes.
> - It removes the final chat maintenance interval while preserving
authenticated recovery.

## Linked Issues or Issue Description

**What happened?**

The app queries due GitHub recovery endpoints every second. This still
runs when no direct GitHub integration needs recovery. Other chat
maintenance duties already have independent workers in the preceding
stack.

**What did you expect?**

Reconstruct existing recovery deadlines at startup and after committed
changes. Run only when an eligible endpoint reaches its saved deadline.
Make no recurring SQL or HTTP calls for unconfigured, disabled, or
Cloud-relayed integrations.

**Steps to reproduce**

Start an instance without an active direct GitHub integration. Leave it
idle. The chat maintenance interval continues to call
processFailedGitHubWebhookDeliveries each second.

Follows merged #17664. Related #15981 owns the original GitHub webhook
ingress/recovery separation. #17662 preserves review-check recovery,
which is a separate obligation. No duplicate scheduling migration was
found in the related-PR search.

## What Changed

- Put deadline reconstruction, scopes, and registration in the GitHub
provider owner. Read existing recovery checkpoints in pages of 100.
Reload current eligibility before each dispatch.
- Dispatch batches of five through the existing shared helper. Preserve
company fairness and the default limit of 25 for other consumers.
- Notify inside configuration, credential, checkpoint,
installation-lifecycle, and Cloud-registration transactions. A signed
installation deletion reproduced a missing wake before the fix.
- Detect credential-only epoch changes during a future backoff. Preserve
the existing floor reset without adopting old events into new
credentials.
- Gate every new recovery HTTP call on dispatch admission and shutdown.
Join in-flight work and retain its existing outcome/uncertainty
settlement. Other queues remain independent.
- Remove the final chat reconciliation interval and its unused
coordinator. Update the old coordinator fixtures; preserve the voice
retention worker acceptance.
- Restore known-unsent redelivery claims after drain/shutdown, guarded
by exact attempt identity. Preserve uncertain outcomes after transport
starts and never overwrite a newer receipt.
- Keep all existing redelivery, source, authority, lease, receipt, and
callback checks. Add no table, migration, or external wake contract.

## Verification

Current source: `ab00fc5b30d51d6c85891536599e7bdfd0c822fe`.

- **751 distinct cases pass** at
`3d7f163bc5259b60f9bd19b7fc90b402f0a6e925`: 274 complete GitHub review
cases, 294 other GitHub-named cases, 30 publication cases, 145
voice/retention cases, and eight startup cases.
- All 19 new recovery worker cases and all six stop-before-transport
variants pass. Coverage includes configured eligibility, callback-only
admission, committed configuration changes, outer transaction
settlement, restart/drain/standby, saved deadlines, fair batches,
independent queues, held HTTP, joined shutdown, known-unsent
restoration, and newer-receipt fencing.
- The first workflow filter omitted the separate recovery group. The
saved case list exposed the gap. The second selection ran the remaining
GitHub cases in a new database. Passed-name sets are disjoint. The two
commands filtered 1,504 and 1,484 unrelated cases respectively. No full
chat or repository-suite claim is made.
- Database build, server typecheck/build, module boundaries, and
migration order pass. Five fresh PostgreSQL fixtures receive the
unchanged parent migration history through 0403. This slice adds no
database migration or table.
- The parent browser-fixture repair is carried with both complete
patches preserved. At predecessor
`836acef0370d42323fdfdbd6d62a805b2469f1d4`, entire server, package and
UI source trees and owned hashes match the 751-case acceptance. The
current follow-up adds three test cases and audit evidence; production
source remains unchanged. The parent has merged.
- [The owning
audit](https://github.com/paperclipai/paperclip/blob/ab00fc5b30d51d6c85891536599e7bdfd0c822fe/doc/plans/2026-10-25-github-webhook-recovery-sleep-audit.md)
preserves earlier failures and exact acceptance scope. Fresh exact-head
Greptile and all CI gates remain required.

## Risks

- Direct GitHub endpoints still require the existing one-minute external
history cadence, with five-minute failure/claim backoff and provider
backoff up to one day. Cloud relay endpoints are excluded. This does not
promise network-idle direct integrations.
- The existing history, envelope, attempt, and lease limits remain.
Provider history never substitutes for a genuine authenticated callback.
- Supported producers share the current database owner's process-local
signals. Independent SQL writers and other server processes need their
own integration. Unknown transaction outcomes remain held by the
existing reconciliation protocol.
- Already-started HTTP keeps its existing timeout and is joined during
shutdown. No later request starts after drain or stop; durable attempt
uncertainty is preserved.
- Host-sleep eligibility and Cloud alarm coverage stay unchanged. Local
worker tests do not prove host/database suspension or fleet savings.
- Each remaining sleep slice needs its own current review and CI checks
before merge.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


Parent startup-test correction: #17664 fixes its legacy coordinator
boundary marker. This slice removes that coordinator and timer entirely,
so its existing absence assertions supersede that obsolete inspection.
All other parent and upstream changes are preserved. At predecessor
`836acef0370d42323fdfdbd6d62a805b2469f1d4`, complete server, package and
UI trees match the 751-case acceptance; all eight startup cases also
pass after the explicit conflict resolution. Fresh review and CI are
required for this new head.


## Required review finding: orphan assignment investigation

The reported orphan endpoint cannot be committed through the existing
schema: assigned_agent_id is NOT NULL, its agent FK is ON DELETE
RESTRICT, and a validated nondeferrable composite FK enforces the same
company. A first reproduction failed at fixture mutation with PostgreSQL
23502 before reaching the worker; that failure remains recorded. Three
new real-PostgreSQL cases verify NULL/missing/deleted assignments are
rejected, then prove the preserved valid endpoint recovers once, saves
its next deadline, and makes no SQL reads during a 15-second
pre-deadline interval. All 22 recovery-scheduling cases pass (19
previous plus three new); unselected cases are not counted. The
preceding style-only commit preserved production behavior and
non-comment tokens, and the prior 751-case evidence retains its original
scope (754 distinct cases across both selections). A new review must
assess this evidence before merge; the old conditional verdict is not
waived. Actual parent squash ancestry is carried with whole-tree
identity before this test-only follow-up.


Repository style follow-up removes seven new code-comment lines from the
recovery ingress owner and constraint tests. Their explanations remain
in the audit. Both files retain identical non-comment TypeScript token
streams, preserving all guards, assertions and deadlines. Production
eligibility SQL is unchanged. Fresh current-head CI and reviews remain
required.


## CI test corrections

The original strict deletion assertion reproduced CI's PostgreSQL-18
failure locally: the forbidden delete reports 23001 (restrict
violation), versus 23503 on PostgreSQL 17. The corrected assertion
accepts only these integrity codes for deletion, retains exact codes for
NULL/missing assignments, and verifies the original agent and assignment
remain. All 22 recovery cases pass on embedded PostgreSQL 18; the same
three integrity probes pass on PostgreSQL 17.

A delayed real announcement response reproduced the attachment E2E's
filechooser timeout: the dismiss handler tried to click outside an
already-open modal menu. It now closes the menu, dismisses through the
UI, and restores that menu so the original action resumes. Controlled
late-announcement acceptance passes; the temporary fault is removed. All
11 complete attachment receipt journeys pass, retries zero and original
assertions/budgets unchanged, using a fresh supported external
PostgreSQL-17 database with canonical isolated
onboarding/server/UI/storage. An earlier embedded-cluster startup
failure occurred before tests and remains recorded. Production server
typecheck passes; production source and schema are unchanged. Fresh
current-head CI and review are required.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 20:05:31 -05:00
scotttongandPaperclip 71f0c0c99e feat: add opt-in Coordinator workspace for chat, tasks, and decisions (#17983)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/147/changes/paperclip/147
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People coordinate work through conversations, tasks, and requests
for input.
> - These surfaces must share task state and preserve the context of
each decision.
> - The existing workspace makes people switch surfaces to follow that
context.
> - This pull request adds an optional Coordinator workspace with linked
conversations, tasks, documents, and decisions.
> - Each person can enable it in Settings > Experimental and return to
the classic workspace.
> - The benefit is a coherent path from a request to an assigned task
and reviewed work.

## Linked Issues or Issue Description

**Subsystem affected**
UI, server, shared contracts, and database.

**Problem or motivation**
People need a consistent way to start tasks from agent conversations,
answer questions, review work, and return to the source context.
Decisions and task state must agree across these surfaces.

**Proposed solution**
Add a personal, default-off workspace experiment. Keep one conversation
per agent. Add task proposals, compact task and decision cards,
contextual document panels, and a Needs You queue with search, filters,
sorting, grouping, and history.

**Alternatives considered**
Replacing the classic workspace for everyone would remove the opt-out
path. Keeping separate decision state in each surface would create
conflicting answers. This change uses the existing canonical task and
interaction records instead.

**Roadmap alignment**
This supports the existing agent chat, task management, and human review
capabilities. It is an experimental presentation choice. The user
requested this isolated implementation after design review. No Discord
discussion is claimed.

## What Changed

- Add per-person workspace preferences, a default-off setting, and an
additive migration.
- Add Coordinator navigation, pinned destinations, task lists, agent
switching, and contextual panels.
- Link task suggestions and creation to the source conversation. Default
new task ownership to the current agent unless the user specifies
another owner.
- Share pending questions, decision receipts, exact review targets, and
history across tasks, chat, and Needs You.
- Preserve task access, assignment authorization, revision binding, and
required output review gates.
- Add compact document links, relative timestamps, composer controls,
and reduced-motion support for task and decision transitions.
- Add component stories, regression coverage, and capability-map
documentation.

## Verification

- Final head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, based on
master `8bc051245f77a47bee0f8ba45896b8bfa5922e0a`. Verified October 29,
2026 at 00:10 UTC: **MERGEABLE, 56 checks passed, four conditional
skips, no pending or failed checks**. [Current CI
run](https://github.com/paperclipai/paperclip/actions/runs/40790639688)
passed without a rerun on this head.
- Fresh Greptile **5/5**, zero new comments and zero open review
threads; Superagent passed with no suspicious changes.
[Checks](https://github.com/paperclipai/paperclip/pull/17983/checks).
- Local workspace typecheck/build passed after the latest rebase. The
final account-read patch passed UI typecheck/build and all 12 related
preference/settings tests, including a delayed real-client request
across account changes. Earlier scoped checks cover review
deduplication/replay, decision deep links, cross-surface state, access
boundaries, and migration replay.
- Local database/browser rechecks after the latest rebase were limited
by exhausted macOS shared-memory IDs: `initdb` failed before database
creation. The local run passed 31 UI/snapshot tests and skipped 24
database tests; those skips and the blocked local browser start are not
claimed as passes. The independent Linux CI database and browser checks
passed. No running dev database or host setting was changed.
- Upstream migration 0403 is preserved. Preferences and durable review
retry aliases are generated as replay-safe migrations
`0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`; five newest
snapshots are retained. Every detected conflict was rebased and pushed
before waiting for CI/review, as requested.
- The PR is unmerged. The active master ruleset still requires
code-owner approval for the changed Paperclip skill; no approval or rule
change was performed.

<details>
<summary>Earlier verification failures, integration history, and
evidence</summary>

These entries preserve prior checkpoints and failures; the current
acceptance result is above.

- Focused UI and server checks from the isolated build are recorded in
the implementation plan.
- Browser checks covered the opt-in shell, navigation, task/document
links, and decision card states. No new live provider trial is claimed
for this handoff.
- Current acceptance head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`,
rebased onto master `8bc051245` immediately after its GitHub
review-repair migration took 0403. Preferences are generated as
idempotent `0404_tense_silver_sable.sql`, preserving upstream
SQL/journal/snapshot history and five newest snapshots. GitHub reports
MERGEABLE after the immediate lease-protected push. Scott's updated
instruction is to rebase and push every conflict resolution first,
without waiting for CI or Greptile; verification follows the push.
- The final reviewed feature tree was consolidated for repeatable
single-commit rebases. The complete previous commit history remains
reachable through local recovery ref
`codex/coordinator-recovery-before-rebase-20261028` (resolved snapshot
`0e1bbdf4e`). The actual rebase completed, and its tree hash exactly
matches that resolved snapshot. The force-with-lease required the known
remote head `0e6ef7c44`, protecting against concurrent branch changes.
No PR merge or permission/runner setting change occurred. Fresh
CI/review are in progress for the current head.
- `2001132fa` received Greptile 5/5 with one nonblocking account-switch
read finding. Current head detaches path-coalesced preference GETs and
disables HTTP caching, retaining React Query's account-scoped sharing. A
delayed real-client GET regression proves account B can load its saved
choice before account A responds, and A's late response only updates A's
cache. Preference/settings tests 12/12 and UI typecheck passed; its
review thread is resolved. Fresh current-head CI/reviews are pending.
The prior CI chat shard 3 ended with “The operation was canceled” and no
assertion failure ([job
123171595631](https://github.com/paperclipai/paperclip/actions/runs/40789688333/job/123171595631),
`/tmp/pr17983-2001132-chat3.log`); no code/timeout change is attributed
to that cancellation.
- `843e823b4` reached fresh Greptile 5/5 with zero comments/open threads
and Superagent pass. All browser tests including the repaired YAML shard
passed, and 54 checks passed. General server shard 9 ended with “The
operation was canceled” and no failed assertion; its test step was
marked cancelled (job
[123164350893](https://github.com/paperclipai/paperclip/actions/runs/40787501761/job/123164350893),
log `/tmp/pr17983-843e-server9.log`). A single unchanged-head rerun was
started after the workflow completed, but master moved while it ran. A
local nested-clone check timed out in its 30-second database-startup
hook and then failed cleanup before the 20 database cases ran; this is
retained as a local verification limit
(`/tmp/pr17983-843e-nested-clone-check.log`), not claimed as a pass or
proof of the CI cancellation cause.
- Following Scott's instruction, the new conflict was rebased and pushed
immediately without waiting for that rerun. Upstream's
`0403_natural_cerise.sql` is preserved; regenerated preferences and
retry-alias migrations are `0404_tense_silver_sable.sql` and
`0405_harsh_avengers.sql`, replay-safe with five newest snapshots
retained. The YAML helper conflict combines upstream's click and header
behavior with the editor-selection guard and injected regression.
Recovery ref `codex/coordinator-recovery-843e823b4` preserves pre-rebase
work. Current head is MERGEABLE. Full workspace typecheck/build passed;
UI and snapshot checks passed 31 tests, but 24 database tests were
skipped by the local support probe, and the local browser server could
not initialize PostgreSQL. A direct disposable `initdb` diagnostic
established the host limitation: `could not create shared memory
segment: No space left on device` (`shmget`, shared-memory IDs
exhausted, not disk space). No dev database or host setting was changed.
Diagnostics: `/tmp/pr17983-2001132-initdb-probe.log`,
`/tmp/pr17983-2001132-db-probe.log`; checks:
`/tmp/pr17983-2001132-tests.log`, `/tmp/pr17983-2001132-browser.log`,
`/tmp/pr17983-2001132-types.log`, `/tmp/pr17983-2001132-build.log`.
These local skipped/blocked checks are not claimed as passes; fresh CI
on independent runners is in progress.
- `d18f683cd` reached fresh Greptile 5/5 with zero comments/open threads
and Superagent pass, but [E2E shard
7](https://github.com/paperclipai/paperclip/actions/runs/40785966243/job/123159719925)
failed parsing an Advanced YAML clipboard read beginning with “Skip to
Main Content.” The log is retained at `/tmp/pr17983-d18f-e2e7.log`.
Current head hardens the test helper to verify editor focus and
selection, clear stale clipboard content, and require the YAML header at
the start of a fresh copy before parsing. Saved-state assertions and
timeouts are unchanged. All six Advanced YAML browser journeys passed on
a disposable local instance (41.0s); the exact lost-response journey
also passed with a one-time injected surrounding-page copy, verifying
rejection/reacquisition (17.6s). The precise browser focus/clipboard
timing in CI remains unproven; the observed invalid-read path is now
guarded and deterministically exercised. Logs:
`/tmp/pr17983-yaml-clipboard-scope-browser.log`,
`/tmp/pr17983-yaml-clipboard-injected-browser.log`. Fresh current-head
CI and reviews are pending.
- `19e6dcb0b` passed Superagent and Greptile 5/5, with one nonblocking
deep-link finding. Current head preserves attention links using
`?decisionId=`: the requested pending decision is rendered even beyond
the first 40 cards, focused and scrolled into view, then only that
parameter is consumed so later refreshes do not refocus. Other URL
parameters remain. Needs You tests 10/10 and UI typecheck passed; its
review thread is resolved. The preceding post-rebase regression run
passed 45/45 tests. Fresh current-head CI/reviews are pending.
- `6d772b7cc` reached Greptile 5/5 with zero comments/open threads and
Superagent pass. While 32 checks had passed, master moved and caused a
documentation conflict in `maps/feature-map/onboarding.md`. Rebased
immediately, preserving both onboarding paragraphs. The complete
four-commit branch was pushed as `19e6dcb0b` before waiting on
CI/review, per Scott's instruction. A transient Git index lock
interrupted the first continue; the command sequence briefly pushed
intermediate `34e298c07`, then the completed rebase was immediately
pushed. All feature and security/retry fixes are present; local recovery
ref `codex/coordinator-recovery-6d772b7cc` preserves the pre-rebase
history. The only tree differences from that ref are upstream's two
smoke-test files and the combined documentation paragraph. GitHub
reports MERGEABLE. Fresh current-head verification is in progress;
prior-head review results are not current acceptance.
- `21efa3120` passed Superagent but Greptile scored 4/5 for a valid
retry-key persistence bug: an acknowledged duplicate key could reopen a
resolved review. Current head adds an additive, replay-safe
`0404_light_swarm.sql` migration for indexed, task-scoped aliases from
acknowledged retry keys to the canonical interaction. Alias registration
and retry lookup occur under the existing task lock; payload/actor
mismatch still conflicts, and each canonical review permits at most 100
aliases. Original and acknowledged keys replay the terminal result
rather than creating a new gate. Five newest migration snapshots are
retained. Service/routes tests 165/165 passed; final quality review plus
migration replay/snapshot checks 25/25 passed; server typecheck passed.
The P1 thread is resolved in code; fresh current-head reviews and CI are
pending.
- `69c2c7543` passed 55 checks and Greptile 5/5 with no open Greptile
findings, but Superagent check
[123140675812](https://github.com/paperclipai/paperclip/pull/17983/checks?check_run_id=123140675812)
correctly flagged unbounded pending quality-card creation. Current head
fixes that under the existing task-row lock: equivalent requests with
fresh idempotency keys reuse the pending card; conflicting requests for
an unresolved deliverable return the existing interaction ID; pending
cards are capped at ten per task across all reviewers. Distinct outputs
remain independent, and already-answered items in a partially pending
batch can receive a fresh review. Added real-database concurrency,
cross-card duplicate, partial-answer, and hard-cap regressions. Quality
review 22/22 tests and final server typecheck passed; earlier
service/routes 163/163 tests passed before the final partial-answer
refinement. One added assertion initially expected a service HTTP error,
but duplicate keys are rejected earlier by the existing shared schema;
the assertion now verifies that validation. Fresh current-head
CI/reviews are pending; the action-required security result is not
treated as passed.
- `c9c575e5f` completed all 56 checks successfully with four conditional
skips, Greptile 5/5, and Superagent pass. Its two P2 review findings are
fixed in the current head: independently bound quality approvals now
each remain visible in attention; changing/clearing URL-backed
assignee/status filters updates the URL and survives task navigation.
Both review threads are resolved. Regression evidence: attention service
34/34 tests passed, CoordinatorTasks 20/20 passed, UI and server
typechecks and the full workspace build passed on `69c2c7543`. The first
new UI test run caught an assertion racing React Router's transition;
waiting for the navigation before asserting passes without weakening the
navigation/remount checks. These older-head green checks are not claimed
for the new fix commit.
- Preceding `0e6ef7c44` completed all 56 checks successfully with four
conditional skips, fresh Greptile 5/5 and no open findings, and passed
Superagent. A fifth migration-metadata collision arrived before the
final Runner job finished. Those green results are preserved as
prior-head evidence, not current acceptance.
- Previous fully verified migration head:
`0e6ef7c44696a377f35e69521b93587576374bd0`, integrating master
`f8226b051` after the fourth migration-number collision. Master added
terminal GitHub-response migration 0401 while the final native check
ran; upstream 0401 SQL/journal/snapshot is preserved unchanged, and
preferences are generated as idempotent `0402_yielding_blink.sql` on
top. Exactly the five newest snapshots remain. GitHub reports MERGEABLE
after the push. All 1,155 relevant
migration/review/interaction/native-execution/remote-assets tests,
workspace typecheck/build, migration guards and whitespace checks pass.
Fresh exact-head CI/Greptile/Superagent are pending.
- Immediately preceding head `36c67cf03` completed all 56 checks
successfully with four conditional skips, fresh Greptile 5/5 and no open
findings, plus passed Superagent security. No retry was needed for that
head's CI. Master advanced and introduced only a migration-metadata
conflict while its final native check ran, so that green result does not
establish acceptance of the current integration. Four consecutive
collisions (0398, 0399, 0400, 0401) during verification are retained as
concrete integration churn; a brief pause of competing migration merges
was requested, without assuming it happened or changing any repo
setting. The PR has not been merged.
- Prior fully verified head: `36c67cf03`, on the clean `30059c0ed`
integration of master `efd65c112`. The last Greptile P2 is fixed: Needs
attention retains the participant-agent scope through authoritative
server membership queries for each pending task, including historical
participation when another agent currently owns the task. It does not
approximate participation with current assignment or enumerate every
company task. The actual view-switch regression and all 31 focused
task/routing/collection tests, UI typecheck/build, token gates and
whitespace checks pass. Greptile completed a fresh 5/5 review on this
exact head (review 19) with zero new comments and no open review
threads. CI and Superagent remain in progress; no current-head CI
failure has been reported.
- Head `30059c0ed` received Greptile 5/5 with that one nonblocking
participant finding, now fixed. Superagent passed that head.
Current-base integration restored the ephemeral RunsOn fleet after a
superseded workflow retained queued aggregate jobs: normal cancellation
was insufficient; the single force-cancel request was interrupted but
readback confirmed that obsolete run cancelled and current CI began on
the trusted fleet. No cancellation retry or runner/security/workflow
changes were made. Its partial CI was superseded by the necessary
participant fix and is not current acceptance.
- Previous compatibility/base-sync head:
`30059c0ededc0eab4e04e8299a884f36db8b292f`, integrating master
`efd65c112` and the Inbox/workspace compatibility fixes. GitHub reports
MERGEABLE after the push. Coordinator keeps Inbox standalone under More
without rewriting the instance's Combined Inbox preference; opting out
restores that preference. Saved merged-Inbox URLs resolve to their
original owner, task status presets remain usable, and canonical
repeated `workspace` parameters retain scoped, deduplicated pagination
(including Needs attention). All 192 broader
routing/Inbox/sidebar/privacy/review tests and the final 16 task-page
tests pass, as do workspace typecheck/build, token gates and whitespace
checks. The two review threads are resolved. Fresh exact-head CI,
Greptile and Superagent results are pending.
- The preceding `d5b9be5bd` remained mergeable but its CI routing gate
intentionally fell back to GitHub-hosted runners: “stacked base merge
commit does not contain the current base branch.” All 45 downstream jobs
stayed queued. Current master was merged cleanly to refresh the pinned
base; no runner/security settings or workflow logic were changed.
Greptile scored that preceding head 4/5 with the two compatibility
findings fixed here. Its automatic Superagent scan was inconclusive at
checkout; the documented retry was still pending when superseded. Those
results do not establish acceptance of this current head.
- Previous approval/migration head:
`d5b9be5bd3b8d6b0437e42bf48138377f34fa67d`, integrating master
`ba5bba86d` and the stale-output approval correction. GitHub reports
MERGEABLE after the push. The fresh P1 finding is fixed: every submitted
item now validates and locks its bound output inside the resolution
transaction, whether or not it carries a required-quality-review
binding. Real database regressions verify that an advanced document
revision or deleted attachment refuses the entire batch without
recording approvals, and current ordinary output reviews still resolve.
All 272 combined migration/review/interaction tests pass; workspace
typecheck/build passed. Fresh exact-head Greptile and CI are pending.
The automatic Superagent scan ended inconclusive at checkout; one
documented on-demand retry is pending. These are explicit verification
limits, not passed security results. The P1 thread is resolved.
- Master consumed the next migration slot three times during this
verification pass: confirmation recovery took 0398, continuation
recovery took 0399, and GitHub acknowledgement recovery took 0400. Each
upstream SQL/journal/snapshot is preserved unchanged. Preferences are
now generated as idempotent `0401_white_lenny_balinger.sql` from
upstream 0400; exactly five snapshots remain, and replay coverage
locates this migration by table content. This is recorded concurrent
integration churn, not a claim that older green results apply to the
latest base.
- Head `ed3b68da9` fixed the two history/URL findings with 26 focused
regression tests, server/UI typechecks, workspace build and token gates
passing. Greptile then scored that head 4/5 with the stale-output P1
corrected in the current head. Its partial CI is superseded, not counted
as current acceptance.
- Previous filter-fix head: `ed3b68da9`, based on the `8458d39d9` merge
of master `063cfd438`. The two fresh Greptile findings are fixed:
history outcome filters now match recorded per-item verdicts in SQL
before pagination (mixed reviews remain in both applicable outcomes),
and same-page URL assignee changes/back/forward navigation update that
filter without clearing independent filters. All 26 focused
database/navigation/collection tests, server/UI typechecks, workspace
build, token gates and whitespace checks pass. The two review threads
are resolved. Fresh exact-head CI, Greptile and Superagent results are
pending.
- The `8458d39d9` migration integration also passed all 328 targeted
migration/interaction/confirmation tests, workspace typecheck/build and
Superagent security scan. Greptile scored that head 4/5 with the two
functional gaps fixed in the current head; those prior-head checks are
not current acceptance.
- Migration integration head:
`8458d39d9721f4cb312c2390070a0609f8cddd1a`, integrating master
`063cfd43813cabf18417605fb9eefb9aacf95c8d`. GitHub reports MERGEABLE
after the push. Master advanced twice during this conflict-resolution
pass: first its confirmation-recovery migration took 0398, then its
continuation-recovery migration took 0399. Both upstream migrations and
journal entries remain unchanged; Coordinator preferences are now
generated as idempotent `0400_sad_guardian.sql` from upstream's latest
snapshot. Exactly the five newest snapshots remain. The replay
regression now locates the table migration by content instead of
hardcoding a renumbered filename.
- All 328 targeted question/review/confirmation/migration tests passed
again after the second merge. Workspace typecheck/build and fresh
exact-head CI/reviews are in progress. The preceding `0c46f46d6` passed
workspace typecheck/build and Superagent security scan, but was
superseded because master consumed migration 0399 while those checks
ran. That prior-head result is not acceptance of the current head. No
production settings or runner/security permissions were changed, and the
PR has not been merged.
- Previous integration head: `0c46f46d6f2b23417ab872ce3cb6ff2673647287`,
integrating master `287b04d72f1b9fca1412e05435c75b8302ba0710`. GitHub
reports MERGEABLE after the push. The renewed conflicts were
`0398_snapshot.json`, `_journal.json`, and the top imports in
`issue-thread-interactions.ts`. Upstream's `0398` confirmation-recovery
index and journal history are preserved unchanged; Coordinator
preferences were regenerated as idempotent `0399_bouncy_miek.sql`, with
replay coverage updated and exactly the five newest snapshots retained.
Both terminal confirmation notifications and Coordinator task
assignment/required review behavior are preserved.
- All 328 targeted interaction/review/confirmation/migration tests pass,
as do workspace typecheck, map validation and whitespace checks. The
workspace build and fresh exact-head CI/Greptile/Superagent checks are
pending. The first migration-generation attempt hit the sandbox's local
IPC restriction before generating files; the unchanged authorized local
generation completed successfully. Prior-head green checks below do not
establish acceptance of this merge.
- Previous acceptance head: `8b7dcf739ed15afc51351fdfa0e0e5fbc75ab788`,
with master `1aa3a21ac` integrated and GitHub reporting MERGEABLE.
Greptile completed a fresh 5/5 review on this exact head with zero new
actionable findings and no open review threads. The [fresh CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/40752864319)
completed SUCCESS on attempt 2 at 19:40:08 UTC, including all eight E2E
shards. Final completed check set: 55 succeeded, four conditional checks
skipped, zero pending/queued/running checks, and one
neutral/inconclusive Superagent scan. The auxiliary commitperclip review
and isolated native Runner compilation/cache-reuse check both passed.
This is not a claim that Superagent produced a passed security verdict.
GitHub still reports MERGEABLE against current master
`60b8089a44b4eaa37e627adbfdf2f130bda13335`.
- Current-head E2E: all eight shards and the aggregate passed, including
the previously failing YAML browser journey. Serialized server shard 9/9
failed: the protected YAML projection test exceeded its original
15-second timeout; its late update request then reached the next test's
reset mock (the captured payload is the prior test's model/label
update). All 88 tests in that exact unchanged file passed locally in
38.03 seconds, with no timeout or assertion changes. The original
slow-request cause remains unproven. CI failure log:
`/tmp/pr17983-8b7-serialized9.log`; local recovery:
`/tmp/pr17983-adapter-yaml-repro.log`. The single bounded unchanged
shard rerun passed, and the workflow completed SUCCESS. Rerun log:
`/tmp/pr17983-8b7-serialized9-rerun.log`. The initial failure is
retained; a passing retry does not establish its original slow-request
cause.
- Greptile scored preceding head `4873704be` 5/5 with one valid P2:
opened task decision records missed company-scoped live invalidation.
The current head uses the existing company decision prefix for
resolution and effect-completion events, and resets private record
payloads on access changes. Its thread is resolved. All 107 focused
record/live-update tests, UI typecheck/build, token checks and
whitespace checks pass.
- CI on `4873704be` failed E2E shard 7/8 in the Advanced YAML
lost-creation-response journey: its clipboard helper selected page text
starting with “Skip to Main Content” instead of YAML. The exact
unchanged journey passed in a disposable local browser instance (1 test,
17.1 seconds). No assertion, timeout or production behavior was
weakened; the original failure cause remains unproven. Failure log:
`/tmp/pr17983-487-e2e7.log`; local check:
`/tmp/pr17983-agent-yaml-repro.log`. The full unchanged Advanced YAML
browser suite also passed locally (6 tests, 37.8 seconds). Fresh
current-head CI subsequently passed this journey and all other E2E
shards.
- Both the automatic and one supported on-demand Superagent scan on
`4873704be` ended neutral/inconclusive: “Pinned PR scan merge-base
checkout failed or timed out.” These do not count as a passed security
verdict. The current head's automatic scan and one documented on-demand
retry also returned the identical neutral checkout failure, with zero
annotations or further diagnostics. This is an external scanner blocker,
not a passed security verdict. No security settings or permissions were
changed.
- Recorded decision links now render and focus the full source-task
record in both Classic and Coordinator, including recorded choice,
submitted inputs, and the existing effect-result resolver. Fresh
terminal records take precedence over older open-decision cache entries.
Context and account checks keep mismatched or inaccessible records out
of the DOM; no decision-history tab was added to tasks or chat. The
shared company live-run feed now drives live-only filtering, including
ended runs and terminal tasks with stale run entries. All 213 focused
tests, UI typecheck/build, token and whitespace checks pass; the final
focus follow-up passed all 188 task-detail tests and UI typecheck. The
first new navigation test failed because its test host omitted company
context; the host was corrected before the passing run.
- Previous integration head: `375c5e89516705050d94bab99336212841b2e6e1`
merges master `1aa3a21ac` after four additional upstream commits. The
sole conflict was competing imports in `server/src/services/issues.ts`;
both quality-review policy and inbound wake-notice notifications are
preserved. The PR-specific review import now sits beside
execution-policy imports instead of the frequently changing first line.
GitHub mergeability is rechecked after the push.
- On the combined master tree, workspace typecheck/build, all 153
affected UI tests, map validation, and whitespace checks pass. Four
server suites passed 77 tests; the fifth suite's module-import hook
timed out while broad verification was running. Its unchanged isolated
run passed all 41 tests in 7.2 seconds with the original timeout. This
records recovery without claiming the original import-timeout cause is
proven. CI on `375c5e895` passed every substantive job, including all
eight E2E shards and Superagent; Greptile found the two functional gaps
fixed in the current head. A short-lived `669d90876` commit added the
main fixes before the focused navigation follow-up.
- Immediately preceding head `9d9737b49` completed all 56 checks
successfully with four conditional skips, Greptile 5/5 and no open
findings, plus a successful Superagent scan. The signoff E2E shard
passed on its single accepted rerun; its initial cancellation/timing
failure is retained below. Master moved again while those checks ran,
requiring this final integration rather than treating the prior head's
green result as current acceptance.
- Previous head `9d9737b496911a6054d0ea22d1b42b28b115e9ab` includes
master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`; that head was
subsequently conflicted by moving master. The master merge preserved
durable account-bound writes and Coordinator UX. Workspace
typecheck/build, 416 focused UI tests, and 142 affected comment-route
tests passed. A stable full UI run on preceding head `938ad1665` passed
all 9,833 tests across 836 files; its final full workspace build also
passed.
- Greptile found nested proposal project inheritance and Classic
bound-revision links on `91be74952`. Both were fixed with 142
interaction-route tests, 113 focused UI tests, and UI/server typechecks.
Its next review identified a valid draft-preservation bug in the new
Classic loading UI. Head `9d9737b49` keeps loading/error notices outside
editors and annotations while preserving dirty drafts; 115 focused
review UI tests pass, including pending/failed revision loads and
continued draft editing. All related threads are resolved. Greptile
completed a fresh 5/5 review of
`9d9737b496911a6054d0ea22d1b42b28b115e9ab` with no open findings.
- CI on `938ad1665` exposed an integration regression with the newly
merged durable startup saves: the personal preference gate blocked the
task page before its first session lookup. The guarded title editor now
mounts while that lookup is pending, with presentation-specific tools
still gated. All 216 affected component tests pass. The unchanged
five-test `disconnect-write-recovery.spec.ts` browser suite passes
against a newly launched disposable local-trusted PostgreSQL instance on
port 33219, including the exact failed startup-save journey. No live
provider, hosted authentication or production proof is claimed. The
helper-owned test server exited and the port has no listener. UI
typecheck, built-browser assets, token and whitespace checks pass.
- Initial durable-write merge tests found duplicate imports and unscoped
draft fixtures, fixed before 416 focused tests passed. A sandbox
typecheck could not create its IPC socket; the supported rerun passed.
The first revision-link test used a wrong Return to latest button
assertion, corrected before the 113-test pass. The startup E2E
reproduction initially still failed because an outer shell gate also
blocked the page; the subsequent fix passed all five cases without
changing the test or its timeouts. A focused loading-header assertion
initially used an incorrect fixture title; correcting it preceded the
216-test pass. These failures remain recorded.
- Earlier routing gates fell back to GitHub-hosted runners while master
was moving or conflicted. Superseded queued runs were cancelled, then
the `938ad1665` routing gate explicitly selected the ephemeral RunsOn
fleet. Its CI actively executed and completed most tests; only the
diagnosed startup E2E shard and aggregate failed, alongside the valid
Greptile finding. Auxiliary GitHub-hosted workflows also completed
successfully on earlier heads, so queueing alone is not an
administrative blocker. Fresh [CI on
`9d9737b49`](https://github.com/paperclipai/paperclip/actions/runs/40744012678)
is the acceptance target; the previously failing startup E2E shard now
passes. A different signoff E2E shard failed during executor
resubmission when its fixture run was cancelled as
`issue_continuation_waiting_on_review`. The exact unchanged journey
passed locally (1 test, 46 seconds), and the preceding head with
identical server/signoff code passed that CI shard. This supports a
bounded rerun but does not establish the timing failure's root cause.
After the workflow finished, GitHub accepted one rerun of that failed
shard and its dependent aggregate; both passed, completing all checks on
that head. The failure log is retained at `/tmp/pr17983-9d9737-e2e7.log`
and the local reproduction at `/tmp/pr17983-signoff-repro.log`. No
assertion or timeout was weakened.
- On the first merge, full workspace typecheck/build, all 9,622 UI
tests, 267 tests across seven affected server suites, two migration
checks, map validation, token gates, and the PR diff whitespace check
passed. Greptile then found duplicate local task-search filtering and
lost completed item-review detail. Both are fixed, with 42 passing
focused UI tests and passing UI typecheck/build.
- A subsequent Greptile review found attachment confirmations without
working links and mobile history without a visible host. Confirmation
layouts now derive the authenticated immutable-content endpoint from the
attachment ID, and Needs You uses the existing full-screen mobile sheet.
All 119 focused tests pass, including missing/stale display URLs, both
review layouts, and both mobile history entry buttons. UI
typecheck/build and token checks pass. Full workspace typecheck/build
also pass after the second master merge.
- Greptile gave head 22f69c304 a 5/5 score with one non-blocking
desktop-history reopening finding. Head 6dd52b6d3 fixed it, passed all
nine history tests, UI typecheck/build, token gates, and the full
9,679-test UI suite, then received a fresh 5/5 review with zero open
findings. These are prior-head results; review on the master-sync commit
is pending.
- The chat and workspace CI jobs on head 22f69c304 were interrupted by
runner shutdown signals and cancellation before completing their tests.
Their logs do not report failing assertions. Those interrupted jobs are
not claimed as passes. The new head starts fresh runs.
- The first merge CI had a Slack session-sync idle assertion fail after
an unexpected database read. Its exact isolated reproduction passes, and
the chat CI shard passed on head 618c5a2a3. The original failure's cause
remains undetermined. The broader local reproduction, run concurrently
with full verification, instead encountered 21 timeout failures across
its two test chunks; it is not claimed as a passing full local server
run. A concurrent local UI run had eight failures (seven timeouts and
one delayed autosave assertion). The complete UI run on the final head
passed all 9,679 tests across 830 files with two workers; no test
timeout or assertion was weakened. This establishes recovery without
claiming that every earlier timeout's cause has been proved. An earlier
intermediate UI run also overlapped the second master merge and failed
the newly added Advanced-sidebar expectation, so it is not treated as
stable-tree evidence. The newly merged UI dependencies initially
required refreshing the local install from the upstream package
manifest; the generated local lockfile update is not committed, per
repository policy.
- Superagent returned neutral/inconclusive scans on the preceding heads
because its pinned merge-base checkout failed or timed out. The
available retry endpoint returned 404. These are not successful security
verdicts. The fresh scan on `91be74952` passed (PR scan passed; no
suspicious PR changes detected). The `938ad1665` scan again returned
neutral/inconclusive with the same pinned merge-base checkout error; the
check-suite rerun endpoint returned 404. The fresh `9d9737b49` scan
completed successfully at 18:22 UTC with no suspicious changes detected.
Earlier neutral scans are not counted as successful security verdicts.
- Prior head `57902152ac3218096fca669d33ff18400a2a86ae` passed 9,618 UI
tests across 826 files, received Greptile 5/5 with no open findings, and
completed CI with 56 passes and four conditional skips. These are
historical results, not acceptance of the latest merge commit.
- The first full build found generated capability-contract drift. The
generated outputs were refreshed. Initial sandbox runs could not open
local IPC sockets; verification was rerun with local test access.
- Current master was merged. The workspace preference migration was
regenerated after the existing migration history.
- The UI run first found two outdated expectations: early custom-answer
validation and the document access error copy. Both were corrected, and
the full UI suite then passed.
- The local server run found three failures in unchanged tests. The Muse
test passes with Python 3.13 instead of the host's Python 3.9. The two
workspace tests pass when their test process uses the expected master
default branch. These environment settings do not change application
behavior or global machine settings. The unsharded server run was
stopped after those reproductions; CI supplies the full sharded server
result. No successful full local server run is claimed.
- Skill inventory generation needs an external eval checkout, which is
absent locally. The existing generator functions refreshed only the live
skill inventory and its derived files. The checked-in eval inventory was
preserved. The inventory validation gate passes.
- Greptile found a dropped MCP review requirement, missing per-item
output links, generic item-review summaries, and a history-refresh
failure. The fixes have 32 passing database-backed server tests and
focused rendered-card tests. A follow-up partial-review finding was
corrected by forwarding the canonical mutation status before settling
the card. Attachments open through the authenticated immutable-content
route in both card layouts. The final focused UI set has 117 passing
tests, including these regressions.
- Both browser shards that first failed expected a disabled-chat
message. The copy was restored, and those shards passed in CI on the
next commit. The two journeys also passed locally. One local attempt
first timed out before feature disable while waiting for the provider's
initial message; its isolated reproduction passed. That earlier timing
failure remains recorded and is not presented as an explained
infrastructure failure.
- Serialized server shard 9 first failed during pnpm setup with
ECONNRESET, before dependencies or tests ran. Its single rerun passed
setup and all tests. The build and production runtime boot passed on the
first attempt, but that run's public-install step timed out during Pi
setup. The single build-job rerun passed compilation, runtime boot, and
clean public-install verification. Those setup scripts are unchanged by
this PR. The successful retry establishes recovery; the earlier Pi
timeout's underlying cause remains undetermined. The aggregate CI gate
is green: [workflow
run](https://github.com/paperclipai/paperclip/actions/runs/40637789695).

</details>

## Risks

- This touches shared task, interaction, and completion paths.
Regression tests must cover classic and experimental behavior,
authorization, duplicate answers, and stale review targets.
- The additive preference table must migrate before the setting is used.
The setting defaults off.
- The experiment changes navigation and message presentation. Canonical
task and document records remain the source of truth.
- The initial task brief and saved plan can repeat information. This
change fixes chronology; a broader redundancy policy remains a design
decision.

## Model Used

OpenAI GPT-6 Codex with reasoning and tool-assisted code editing, tests,
and browser verification. Earlier iterations used Codex models in this
task; their exact model IDs are not available in the current context.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used
- [x] I have checked ROADMAP.md
- [x] I have searched GitHub for duplicate or related PRs
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change
- [x] I have run relevant tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open recommendations or follow-ups
- [x] I will address review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 17:20:01 -07:00
DottaandPaperclip 8bc051245f fix(server): wake stale GitHub review repair from committed changes (#17664)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/215/changes/paperclip/215
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub checks and task admission can finish in either order.
> - A check can reject an old head before admission creates its review
row.
> - The chat maintenance loop repeatedly scans for that late review.
> - Existing cancellation and delivery receipts already identify the
repair.
> - This pull request wakes repair from committed changes and leaves
settled history quiet.

## Linked Issues or Issue Description

**What happened?**

GitHub chat maintenance repeatedly scans all review projections. Empty
instances and retained history incur database work even when no repair
is owed.

**What did you expect?**

A committed stale-head cancellation or admitted review should wake
repair. Only the exact delivery's unassessed queued/running review
should become superseded. When no repair remains, this duty should stop
making database calls.

**Steps to reproduce**

Configure a GitHub chat endpoint and leave it idle after all reviews
settle. The previous maintenance callback continues to call
repairSupersededReviews. A rapid push can also cancel a check before
admission inserts its review, which explains why recovery must remain.

Refs #17662, the merged review-check publication parent. Related #17661
and #17656 migrate terminal responses and acknowledgements. This slice
owns only late stale-head projection repair.

## What Changed

- Move the repair predicate and execution into the GitHub provider
owner. Match the original admitted delivery and cancellation receipt.
Recheck state and assessment in the update.
- Register a scoped worker with the existing coordinator. Use keyset
pages and fair bounded dispatch. Keep retries only for unresolved repair
or failed reconstruction.
- Signal before cancellation, automatic receipt insertion,
automatic-review binding, and check-driven review-state writes. Preserve
existing transaction ownership and outer commit behavior.
- Remove the periodic provider wrapper and publication-maintenance
callback. Preserve explicit manual recovery and independent
missed-webhook recovery.
- Add two concurrent, idempotent partial indexes. Use a guarded delivery
UUID lookup and a bounded exact-receipt lookup. Add no tables.
- Add ten real-database lifecycle and producer cases. Document the
producer audit, failure history, and remaining limits.

- Keep YAML browser reads scoped to the editor after focus changes, with
the original journey assertions and deadlines.

## Verification

Current source: `0886a0229b1c4f68d2f582da86aa1f248e9124f3`.

- At `dd6bca0485c3444d6e3d0ee10ec4aa651e801392`, **287 distinct cases
pass**: 274 complete GitHub workflow cases and 13 startup cases. The
GitHub command filters out 1,485 unrelated cases. The workflow suite
uses a new, pre-migrated PostgreSQL database.
- Coverage includes both orders of cancellation and admission, outer
commit and rollback, restart, drain, standby, failed-update retry,
concurrent assessment under an observed row lock, shutdown during that
wait, more than one candidate page, manual history, and settled-history
quiet.
- Database build, server typecheck/build, module boundaries, and target
migration order pass.
- Canonical migration `0403_natural_cerise.sql` preserves both original
index definitions and concurrent idempotent creation. All 401 parent
migration receipts remain unchanged during upgrade. One new receipt is
added. Replay adds none. Both indexes are ready and valid on fresh and
upgraded databases. The five newest snapshots remain.
- The normal parent merge preserved the prepared tree. The later browser
fixture repair changes only its central read helper and verification
note. Entire server, package, and UI trees remain identical to the
accepted source.
- [The owning
audit](https://github.com/paperclipai/paperclip/blob/0886a0229b1c4f68d2f582da86aa1f248e9124f3/doc/plans/2026-10-25-github-review-projection-sleep-audit.md)
records the exact evidence and earlier failures. Earlier query-plan
measurements retain their historical revision scope.
- Fresh current-head CI and Greptile review remain required. No deployed
sleep, full repository-suite, or fleet-savings claim is made.

Failure accounting: browser shard 7 failed twice on
`1453a4447c0d4d33f713ae3e00734c7950f95426` because the YAML helper read
whole-page clipboard text while main was focused. Three isolated
unchanged local passes did not resolve that failure. A controlled native
focus-loss probe reproduced the same error before the repair and passed
after it. The helper now clears stale clipboard data, focuses the
editor, and waits within the existing expectation budget for a fresh
YAML copy. All six complete YAML journeys pass with zero retries in 50.9
seconds. Journey assertions and deadlines are unchanged. The temporary
fault is removed. The exact CI focus-change trigger remains unknown.
[Both CI
attempts](https://github.com/paperclipai/paperclip/actions/runs/40777754306)
remain part of the verification record. Fresh new-head checks are
required.

Ponytail follow-up: the 4/5 review of `b3c81ba` was conditional on
fixing a stale startup-test slice marker, so the merge was held.
Boundary assertions reproduced the invalid -1 end index. The test now
uses the existing timer declaration and checks both bounds; all 13
startup/hostname cases pass. The optional no-op coordinator cleanup is
already in #17666. The six YAML journeys and all production code remain
unchanged. Fresh review and CI are required for this corrected head.

## Risks

- Missing a future producer signal could delay discovery until restart.
Supported writers share the current database owner's process-local
notification scope. Independent SQL writers and other server processes
need their own integration.
- The migration builds indexes concurrently. The existing migration
runner handles interruption and invalid index recovery.
- Direct GitHub missed-webhook discovery remains periodic. This change
does not make the whole chat service idle.
- Existing Cloud wake coverage and sleep blockers remain unchanged.
These tests do not prove deployed host or database suspension.
- Each remaining sleep slice needs its own review and CI checks before
merge.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 18:42:54 -05:00
DottaandPaperclip 65fa1cd617 fix(server): schedule GitHub review checks from committed changes (#17662)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/213/changes/paperclip/213
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub review checks report the state of admitted agent work.
> - The chat maintenance loop searches for check work even when none
remains.
> - Existing action rows already record unfinished publication and retry
deadlines.
> - Committed changes can wake those rows through the shared
coordinator.
> - An unresolved check must still detect a remote head change when a
webhook is missed.
> - This pull request schedules only outstanding checks and leaves
settled history quiet.

## Linked Issues or Issue Description

**What happened?**

The GitHub chat maintenance callback repeatedly queries review-check
actions. Empty queues and completed history still incur database reads.
Local state changes wait for the maintenance callback.

**Expected behavior**

Committed check creation, review binding, run changes, and
publication-lease release wake the worker. Existing retry and
remote-head deadlines apply only to unfinished checks. A settled worker
makes no recurring database or provider calls.

**Steps to reproduce**

Configure a GitHub bot with no outstanding checks, or finish all of its
checks. Leave the instance idle. The old maintenance callback continues
to call the check service.

Refs #17661, the merged terminal-response parent. Related #17656
migrates response acknowledgements. Related #16385 changes branch-policy
admission; this PR preserves that separate policy boundary. Late
stale-head projection repair remains a separate follow-up slice.

## What Changed

- Move the existing check-state reader into the GitHub provider owner.
Preserve current authority, run selection, assessment ownership, and
check replacement behavior.
- Register a scoped worker with the existing coordinator and scheduler.
Existing received and retryable-failed actions remain durable truth. No
new table is added.
- Signal check enqueue, task review receipts, manual review
insertion/binding, run lifecycle changes, and publication-lease release
through existing transaction ownership.
- Preserve the saved failure backoff, eight-attempt policy, and
competing lease expiry. Keep five-second remote-head reconciliation only
while a received check remains unresolved.
- Check admission before each HTTP request. Shutdown leaves durable work
recoverable without spending a provider failure attempt.
- Remove check dispatch from the periodic GitHub callback. Keep the
separate late-projection repair and the manual combined recovery method
compatible.
- Add an idempotent concurrent partial index for unfinished check
selection. Retain the newest five generated snapshots.
- Add eleven real-database worker cases and startup wiring coverage.
Document producer coverage and the external reconciliation limit.

## Verification

Current reviewed source: `2bba08403b195bf646eda48992b46d6f5cebf756`.

- Normal merges preserve the complete review-check patch, the landed
terminal-response parent, and current master changes. No landed SQL
migration or journal entry changes.
- At `6995f84b240a14a7b8660e1cf095b3aa1e1e3168`, **331 distinct tests
pass**: 264 complete GitHub workflow cases, 30 completion cases, eight
issue-tree producer cases, 16 login-cleanup cases, and 13 startup cases.
The GitHub command filters out 1,485 unrelated cases. Login cleanup uses
its real embedded PostgreSQL fixture.
- Combined-master acceptance: **277 distinct tests pass** at
`9c538aed1251fd7f7cecb829a0ecbce6110fb1ed`: 264 complete GitHub workflow
cases and 13 startup cases. The GitHub command filters out 1,485
unrelated cases. A new PostgreSQL database receives all migrations
through 0402..
- Migration `0402_ancient_malice.sql` preserves the original partial
index definition and concurrent idempotent creation. All 400 prior
migration receipts stay unchanged during upgrade. One new receipt is
added. Replay adds no receipt. The index is ready and valid on fresh and
upgraded PostgreSQL databases. The five newest snapshots are retained.
- A rollback-only query probe adds 10,000 processed actions and one
pending action. The actual parameterized candidate predicate selects the
one pending row through the partial index, with two shared buffers and
0.007 ms execution time. This measures candidate discovery, not all
lease or provenance reads.
- Database build, server typecheck and build, module boundaries,
migration safety and migration order pass. Audit: [GitHub review-check
scheduling](https://github.com/paperclipai/paperclip/blob/2bba08403b195bf646eda48992b46d6f5cebf756/doc/plans/2026-10-25-github-review-check-sleep-audit.md).
- Fresh checks and Greptile review are required for this head. Prior
scores do not count. These checks do not prove deployed host suspension
or fleet savings.

## Risks

- Unresolved queued/running checks still make remote-head requests at
the existing five-second deadline. GitHub webhook receipt does not prove
future pushes cannot be missed. This PR does not claim active reviews
become network-idle.
- Run and configuration hints rebuild outstanding checks for the
affected company. Enqueue, review, and lease hints narrow to the
affected endpoint/repository/PR. Startup and scope reconstruction read
existing pending rows; they are not constant-cost operations.
- Delivery admission failures remain discoverable through the unresolved
check's external reconciliation deadline. Not every shared delivery
mutation gains a new immediate signal.
- Notifications use the existing process-local database ownership
contract. Independent SQL writers and other replicas need their own
supported integration.
- Provider markers, publication leases, and saved actions preserve
recovery after uncertainty. This change does not expand Cloud alarm
coverage or prove deployed host/database suspension.
- Each remaining sleep slice needs its own review and CI checks before
merge.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 17:08:27 -05:00
DottaandPaperclip f8226b0515 fix(server): wake GitHub terminal responses from committed changes (#17661)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/210/changes/paperclip/210
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip coordinates agent work and presents results in chat.
> - GitHub acknowledgements already persist the response action and
provider receipt.
> - A terminal run can still owe a final response after acknowledgement.
> - Periodic maintenance searches those rows even when no relevant state
changed.
> - Committed run, issue, presentation, lease and publication changes
identify when recovery can advance.
> - The owner should recover from those events and real deadlines, then
become quiet.

## Linked Issues or Issue Description

Refs #17656, which is merged. This existing terminal-response slice
targets master.

**Problem:** A saved GitHub response can remain nonfinal while its run,
native presentation, publication or source is unresolved. Removing
maintenance without covering those producers loses recovery. Polling
settled history keeps the tenant active.

**Expected behavior:** Committed supported changes wake the terminal
owner. It reconstructs existing actions, preserves provider authority
and pending publication ownership, waits at saved deadlines where
needed, and becomes quiet after settlement.

## What Changed

- Register terminal GitHub response work with the shared scoped deadline
coordinator, using existing action/delivery/publication/run rows.
- Notify response availability from supported run milestones, native
identity, recovery resolution, issue updates and tree/workspace
restoration writes. Preserve outer transaction commit/rollback.
- Keep GitHub response interpretation in its platform owner. Use the
bounded run projection for both direct authorization and scheduling,
including native presentation guards and deliberate passive outcomes.
- Retain the existing publication lease, markers, source/principal
checks and final receipt precedence. A publication owner or uncertain
outcome keeps its existing authority; the worker does not replay it.
- Remove only recurring terminal-response recovery, preserving the
manual path and neighbouring duties. Add no table or scheduler service.
- Regenerate the original sparse terminal-work index as
`0401_yummy_ultimates.sql`, concurrently and idempotently through the
existing top-level migration runner. Preserve all prior SQL/journal
entries and the five newest snapshots.

## Verification

Published head: `88a209e28b969b5229e9a621aeddfd3cbae9b0cd`, integrated
with landed #17656 (`ba5bba86dcdde32b14d4d7dec1b51898e5f6f22d`) and
master `52b86c2032a189b0ea259c5b0cb7158b07ef788c`.

- **74 distinct current cases pass:** 17 terminal-response
lifecycle/producer cases (1,721 other chat cases skipped), all six
queued-comment consumption regressions for the new upstream query
optimization, all 30 heartbeat completion cases, all eight real/unit
issue-tree control cases and all 13 startup cases. Database build,
server typecheck/build, module boundaries and four fresh migrations
through 0401 pass.
- The earlier **304-case** prepared-parent matrix passes at
`aa72f32641b2e2c680342cd49477272349aca9db`: all 253 GitHub workflow
cases, 30 completion cases, eight issue-tree cases and 13 startup cases.
It retains that revision scope. Repeated cases are not added to the
current 74.
- The terminal matrix covers finalized-history quiet, deleted Board
sources, pending presentation/passive outcomes, outer rollback/commit,
paused/new runs, restart/drain/standby, committed native identity, lease
waits, full-page unrelated history, uncertain publication ownership and
tree/workspace/exhausted-recovery producers.
- A populated 0400 database upgrades to 0401 with all 399 prior
migration receipts identical, exactly one added, and none added on
replay. The terminal index is valid/ready on fresh and upgraded
databases. Its SQL matches the original draft after whitespace
normalization; generation keeps snapshots 0397–0401.
- Normal integration `3c8339a541bf68bc9957b9d5a8df80d3a7c3452d`
preserves the complete 24-path owned binary patch and all five new
master paths. The landed parent tree also equals the previously
published parent tree byte for byte. Both normal merge parents and the
exact composition tree are checked. No force push or rebase occurs.
- [The owner
audit](https://github.com/paperclipai/paperclip/blob/88a209e28b969b5229e9a621aeddfd3cbae9b0cd/doc/plans/2026-10-25-github-terminal-response-sleep-audit.md#acceptance-on-the-landed-github-parent-and-current-master)
preserves the original producer, quietness, failure-isolation,
skipped-suite and presentation-wait results with their scopes. Current
passes do not erase earlier failures or turn skips into passes.
- The audit-only follow-up leaves all owned source/test/migration hashes
unchanged. **Fresh exact-head CI and Greptile 5/5 with no actionable
findings remain required.** No full local repository suite, live GitHub
tenant or deployed sleep result is claimed.

## Risks

- Process-local notifications cover supported application writers and
shared pools. Independent SQL writers and other replicas gain no new
wake guarantee.
- Native terminal status alone does not authorize fallback publication.
Pending presentation and existing publication ownership remain separate
obligations.
- Stop can leave an in-flight provider request settling; admission
fences stop a subsequent request. Unknown outcomes retain their existing
recovery rules.
- Index creation can take time on large histories. Concurrent/idempotent
migration and existing replay recovery remain in charge.
- Worker quiet does not establish host/database suspension. Cloud alarm
coverage and sleep admission remain unchanged.

## Model Used

OpenAI Codex, using reasoning, repository inspection, code execution and
real PostgreSQL tests. The exact model identifier and context-window
size are not exposed in this session.

## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 16:51:57 -05:00
DottaandPaperclip ba5bba86dc fix(server): wake GitHub acknowledgements from committed changes (#17656)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/206/changes/paperclip/206
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip coordinates agent work and presents its progress in chat.
> - GitHub deliveries save acknowledgement actions and provider
receipts.
> - Periodic maintenance currently searches for unfinished
acknowledgements even when idle.
> - Committed delivery, lease, response and presentation changes
identify when work can advance.
> - The existing owner should wake on those changes and recover only at
saved deadlines.
> - Completed history should become quiet without losing unfinished
provider effects.

## Linked Issues or Issue Description

Refs #17653, which is merged. This existing GitHub acknowledgement slice
targets master.

**Problem:** A saved GitHub acknowledgement can wait for a publication
lease, fast-response result or native presentation decision. Polling
empty or unchanged work keeps the tenant active. Removing maintenance
without covering these producers can strand the acknowledgement.

**Expected behavior:** Committed changes wake the scoped owner. It
reconstructs existing actions on startup, retries at saved deadlines,
parks unchanged presentation waits and becomes quiet when settled.

## What Changed

- Register GitHub acknowledgement work with the shared scoped deadline
coordinator. Reuse existing action, delivery, lease and fast-response
rows.
- Add transaction-aware response-availability notifications at supported
writes. Keep GitHub protocol details in its platform owner and shared
notifications limited to IDs.
- Share the bounded latest-request/run projection between scheduling and
direct response authorization. An absent native presentation decision
parks the action until its real writer commits.
- Retain publication leases, source/principal authority, version
fencing, response markers and final receipt precedence. Check admission
before each new provider request; let in-flight work settle. Stop does
not consume the provider retry budget.
- Remove only recurring acknowledgement recovery and keep its manual
path. Add no work table or scheduler service.
- Regenerate the three original indexes as
`0400_absent_phantom_reporter.sql`: pending acknowledgement IDs,
fast-response delivery lookup and publication comment/destination
lookup. Each uses concurrent/idempotent creation and the existing
top-level migration runner. Preserve all landed SQL/journal entries and
the five newest snapshots.

## Verification

Published head: `09ca00b9f029838434ec042cda691345d67b493e`, on landed
continuation parent `063cfd43813cabf18417605fb9eefb9aacf95c8d`.

- **87 distinct current cases pass:** 18 real PostgreSQL acknowledgement
lifecycle/presentation cases, all 26 fast-response service cases, all 30
heartbeat completion cases and all 13 startup cases. Database build,
server typechecking/build, module boundaries and three fresh migrations
through 0400 pass. The current Runner bundle rebuild also passes after
upstream provider-event changes.
- The lifecycle matrix covers empty/history quiet, lease release,
restart, drain/standby, fast-response failure/expiry/publication/unknown
outcome, streaming quarantine, outer rollback/commit, saved provider
retry, stop between requests and receipt state. The real presentation
writer proves quiet while pending, rollback, committed reactivation and
final quiet.
- The earlier **305-case** matrix (236 full GitHub workflow cases, 26
fast-response service cases, 30 completion cases and 13 startup cases)
passes at `337f272853d4f8a51b5b9300900a46eb4e13f698`. It retains its
prepared-parent scope. Repeated cases are not added to current 87.
- A populated 0399 database upgrades to 0400 with all 398 prior
migration receipts unchanged, one new receipt and no additional receipt
on replay. All three indexes are valid/ready on fresh and upgraded
databases. Their SQL definitions match the original draft after
whitespace normalization; migration history and retained snapshot bytes
are preserved.
- Normal integration `9b3607e159153e00ccdfe538d52b591bd38d6499`
preserves the full 28-path owned binary patch and all 20 unrelated
master paths, including native retry, provider-event and evaluation
workflow changes. Both normal merge parents and the exact tree are
checked. No force push or rebase occurs.
- [The owner
audit](https://github.com/paperclipai/paperclip/blob/09ca00b9f029838434ec042cda691345d67b493e/doc/plans/2026-10-25-github-chat-maintenance-sleep-audit.md#acceptance-on-the-actual-landed-continuation-parent)
retains the original missing-notification, JSON precedence, receipt
lifecycle, external-database fixture and migration-registry failures.
Earlier 69/228/broad counts keep their original scope. Skips and failed
revisions are not relabelled as current passes.
- The final audit-only edit leaves all owned source/test/migration
hashes unchanged. **Fresh exact-head CI and Greptile 5/5 with no
actionable findings remain required.** No full local repository, live
GitHub tenant or deployed sleep result is claimed.

## Risks

- Process-local transaction hints cover supported application writers
and shared pools. Independent SQL writers and other replicas do not gain
a new wake guarantee.
- Provider effects remain guarded by the existing publication lease and
current source/principal authority. Unknown outcomes retain their
existing recovery semantics.
- Presentation waits depend on their supported milestone/transaction
producers. A missing producer can defer recovery until another supported
change or restart.
- Index creation can take time on large histories and runs concurrently
outside a transaction. Existing migration recovery handles replay; prior
SQL is unchanged.
- Worker quiet is distinct from host/database suspension. Cloud wake
coverage and sleep admission remain unchanged; later maintenance duties
are separate slices.

## Model Used

OpenAI Codex, using reasoning, repository inspection, code execution and
real PostgreSQL tests. The exact model identifier and context-window
size are not exposed in this session.

## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 16:06:42 -05:00
DottaandPaperclip 063cfd4381 fix(server): wake interaction continuations from committed responses (#17653)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/204/changes/paperclip/204
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip coordinates agent work and human approvals.
> - Human decisions already persist the continuation action that resumes
an agent.
> - A response can commit before its callback or publication finishes.
> - Periodic maintenance repeatedly discovers those rows, including when
idle.
> - Committed writes should wake the existing owner; saved retries and
claims supply real deadlines.
> - Startup reconstructs durable work, and settled history becomes
quiet.

## Linked Issues or Issue Description

Refs #17651, which is merged. This existing continuation slice targets
master and removes its duty from periodic chat maintenance.

**Problem:** A saved `interaction_wakeup` action can survive a committed
human response whose callback crashes. Without producer coverage,
removing maintenance strands the continuation. Repeated empty
maintenance also keeps the database active.

**Expected behavior:** The outer transaction's settlement wakes the
continuation owner. It handles ready conversation publications,
dispatches under existing authority, preserves interrupted/retry work,
and becomes quiet after settlement.

## What Changed

- Register continuation recovery with the shared scoped deadline
coordinator. Reconstruct existing issued/processing actions and retain
the existing retry and stale-claim deadlines.
- Notify from both enqueue producers and owner-local claim/settlement
writes. Preserve outer commit/rollback and avoid moving provider or
heartbeat calls into database transactions.
- Give the existing publication queue an optional
company/endpoint/conversation scope. The continuation owner gives ready
cards in that conversation a bounded delivery opportunity before
claiming the action; FIFO, capacity, ambiguity and provider authority
remain with the queue.
- Carry admission/cancellation through publication preparation and check
before every new send. Join in-flight requests and retain pending
actions/publications through drain, standby, shutdown and resume.
- Remove only recurring continuation recovery; keep the manual API. Add
no table. A concurrent partial index on existing pending continuation
actions and matching company/id keyset paging prevent retained history
from being scanned. Parent #17651's indexed confirmation discovery and
current master are retained.

## Verification

Published head: `a1d16f84b3fae361d5d648670a5964486370c3a3`, on actual
landed terminal-confirmation parent
`ee67d2ece9bda157dfad9bdf25175b1d004ed341`.

- **75 distinct current cases pass:** 32 selected real PostgreSQL chat
cases (14 continuation lifecycle/producer/stop cases, one new
continuation query-cost case, and 17 parent confirmation/query-cost
cases; 1,671 skipped), and all 43 cases in complete
interaction-publication and app/startup suites. Database build, server
typecheck/build, module boundaries and fresh migrations through 0399
pass. The original landed-parent matrix passed 74 cases before the
additional query-cost repair; those are not added again.
- The new real-SQL regression seeds 20,000 completed/unrelated actions
and one pending processing claim. Before repair it examines 20,345 rows
and hits 20,421 buffers. The final startup and company-scoped queries
must examine at most two action rows and fewer than 20 action buffers;
empty scoped recovery must also stay below 20 buffers. Planner settings
are unchanged. Migration `0399_ambitious_onslaught.sql` is
concurrent/idempotent and uses the existing top-level migration runner;
prior SQL/journal entries and the five newest snapshots are preserved.
- The matrix covers callback recovery, concurrent responses, malformed
prefixes, more than one page, saved retries/stale claims, outer
rollback/commit, deferred/future/paused publication states, restart,
drain and stop before each new provider request. Quiet assertions
inspect real selects/transactions across virtual idle time.
- Normal integration `db5a84c0f8b73a0ce863f4d2fbaee0a115cf044f`
preserves the complete 12-path owned binary patch and all 12 unrelated
current-master paths, including managed Cloud sandbox changes. A
verified semantic base resolves squash-ancestry conflicts; both normal
parents and the exact tree are checked. The subsequent index/keyset
repair is a deliberate additional change, tested separately; the final
patch is not claimed identical to the original 12-file implementation.
No force push or rebase occurs.
- The same 74 pre-index cases passed on the earlier prepared and actual
landed parents. The new query-cost regression failed before its repair,
then passes on the repaired source. Earlier 57-case and broad acceptance
retain their original revision scopes; repeated cases are not added to
the current count. The audit-only follow-up leaves all owned server
source/test hashes unchanged.
- [The owner
audit](https://github.com/paperclipai/paperclip/blob/a1d16f84b3fae361d5d648670a5964486370c3a3/doc/plans/2026-10-25-interaction-continuation-sleep-audit.md#acceptance-on-the-actual-landed-terminal-confirmation-parent)
retains the original missing-signal regression, the skipped standalone
publication suite, and the review's stop-before-send failures with their
corrections. Skips and earlier failing revisions are not relabelled as
current passes.
- **Fresh exact-head CI and Greptile 5/5 without unresolved actionable
findings remain required.** The former remote head's reviews are stale.
No full local repository suite, live provider or deployed sleep result
is claimed.

## Risks

- Process-local transaction hints cover supported application writers
and shared pools. Independent SQL writers and other replicas do not gain
a new wake guarantee.
- The publication queue's existing FIFO, capacity, retry/ambiguity and
authorization contracts still govern effects. In-flight provider
requests can settle after Stop; the fence prevents a subsequent request
from starting.
- Missing a supported producer can defer recovery until another change
or startup. Producer and rollback tests cover this slice's supported
paths.
- Worker quiet is separate from host/database suspension. External alarm
coverage and Cloud sleep gates are unchanged.

## Model Used

OpenAI Codex, using repository inspection, code execution and real
PostgreSQL tests. The exact model identifier and context-window size are
not exposed in this session.

## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 15:45:18 -05:00
DottaandPaperclip ee67d2ece9 fix(server): wake terminal confirmation recovery from committed changes (#17651)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/202/changes/paperclip/202
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip persists confirmation clicks and interaction decisions
before dispatch completes.
> - A crash after a decision commits can leave a processing confirmation
action.
> - Periodic publication maintenance repeatedly scans those actions even
when idle.
> - Saved decisions and the supported board/admin writers already
identify when settlement can change.
> - This owner reconstructs durable actions, reconciles real changes and
then becomes quiet.

## Linked Issues or Issue Description

This existing terminal-confirmation slice follows merged #17649. It
removes only terminal-confirmation recovery from periodic chat
maintenance while retaining the explicit manual API and existing durable
action/interaction/publication rows.

**Problem:** An accepted response can commit before its dispatch
settles. Removing periodic recovery without covering board decisions,
expiry and ownership changes would strand processing actions. Pending
human decisions must remain recoverable without polling indefinitely.

**Expected behavior:** Committed supported changes wake the scoped
owner. It reconciles terminal responses once, parks unresolved human
decisions, preserves concurrent commits/restart/drain and becomes quiet
when settled.

## What Changed

- Register terminal-confirmation work with the existing delivery
coordinator and scoped lifecycle. Reconstruct processing action IDs,
bound selection pages, preserve concurrent wakes and keep current
authorization in the existing interaction owner.
- Add the existing transaction-aware confirmation notification topic at
real resolution writes and callback settlement. Reuse milestone signals
instead of adding per-caller recovery wrappers.
- Cover ordinary and checkbox confirmations, suggestions and connection
intents through their owning service. Cover board accept/reject,
cancellation/withdrawal, opt-in comment supersession, expiry,
reassignment and already-satisfied connection resolution.
- Preserve task/actor authority, one-use callback tokens and immutable
provider publication semantics. Explicitly fail malformed missing
bindings instead of silently retaining unrecoverable processing actions.
- Remove only this duty from recurring publication maintenance. Add no
table or scheduler infrastructure. A concurrent partial index on
existing action rows prevents recovery discovery from scanning completed
history.

## Verification

Published repair head: `a35e78d076c1b618bc643391b7520fa6952fc996`,
targeting master after normal integration of
`644937c93150c423f12674992c7ea1e8b5dbc98c`. [Detailed acceptance and
preserved
failures](https://github.com/paperclipai/paperclip/blob/a35e78d076c1b618bc643391b7520fa6952fc996/doc/plans/2026-10-25-terminal-confirmation-sleep-audit.md#bound-retained-history-reads--october-28-review-repair).

- **New query-cost regression:** Capture the actual SQL emitted by
startup and company-scoped confirmation discovery. Seed 20,000
completed/unrelated action rows and one pending human decision. Before
the index, the regression fails because recovery examines 20,009 action
rows. After the partial index, startup and scoped discovery must examine
at most two action rows and fewer than 20 table/index buffers; the
no-pending scoped path must also stay below 20 buffers. Planner settings
are unchanged, and returned IDs remain correct.
- Match startup keyset ordering to `(company_id, id)`, the partial index
order. The index alone still let PostgreSQL choose the primary key and
scan 20,206 rows for `ORDER BY id`; the final query fixes this without
forcing a planner setting or adding a second index. Fixed kind/status
predicates stay literal for prepared-plan index eligibility.
- Add the generated `0398_yielding_random.sql` migration, using `CREATE
INDEX CONCURRENTLY IF NOT EXISTS` and the existing top-level migration
runner. Keep the five newest generated snapshots and preserve every
prior SQL/journal entry. No work table is introduced.
- **110 distinct cases pass:** 17 final confirmation
lifecycle/query-cost cases (1,671 skipped), seven real Runner descriptor
variants (295 skipped), and all 86
app/startup/bundled-plugin/plugin-database cases. Server/database
typechecking and builds, migration safety/numbering, module boundaries
and a fresh PostgreSQL migration through 0398 pass. The final test-only
vacuum precondition is followed by another 17-case confirmation pass and
server typecheck. Counts are not added across repeated runs. Normal
integration of master `644937c93150c423f12674992c7ea1e8b5dbc98c`
preserves the complete owned patch and all 18 unrelated master paths.
- Previous landed-parent acceptance: 29 distinct cases (16 confirmation
lifecycle/producer cases with 1,671 skipped, plus all 13 startup cases)
passed on `32e64a7e9ea5b457e54ca7b268be90e1e155b451`; the earlier
80-case matrix remains scoped to
`c1e8542b588a4fa59b62f9829541d3daea4f1ddc`. Counts are not added across
repeated runs.
- Actual producer cases prove parking/quiet, caller-owned outer
transaction rollback, commit-driven automatic settlement, restart/drain
and concurrent commits. Checkbox, suggestion, connection-intent and
Telegram/Slack callback paths retain existing authority and immutable
publication semantics.
- **Fresh exact-head CI and Greptile 5/5 with no unresolved actionable
findings are required.** The preceding head's Runner CI failed its
timeout-attachment durable-suspension case (1 failed/1,955 passed/4
skipped); that failure is retained and not counted as a pass. The new
head must pass this job. No full local repository, live provider or
physical suspension claim is made.

The owner audit retains the earlier missing-board-producer regressions
and their corrections, including the invalid supersession fixture that
assumed a new checkbox expires an older one. Existing behavior was
preserved by using the supported opt-in human-comment path. Prior
scheduler/backoff, route and plugin test failures remain at their
original revisions; no old pass is substituted for current remote gates.

## Risks

- Process-local notifications cover supported application writers and
their shared pools, not independent SQL writers or other replicas.
- Processing actions waiting on a human response are deliberately
parked. Supported interaction resolution/expiry writers must use the
notification boundary; a missing producer delays settlement until
restart.
- Invalid bindings have explicit failure evidence. Source, actor, task
and publication authority remain checked by their existing owners.
- Following continuation/GitHub duties remain separate. Cloud sleep
gates and external alarms are unchanged; local worker quiet does not
establish physical host/database suspension or fleet savings.

## Model Used

OpenAI GPT-6 / Codex, with reasoning, repository inspection, code
execution, and PostgreSQL tests. The exact runtime model identifier and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 15:00:31 -05:00
DottaandPaperclip f4f02880ca fix(server): wake failed retry notices from committed changes (#17649)
## Thinking Path

> - Paperclip keeps exact failed chat retries in durable action and wake
receipts.
> - Deferred and declined retries owe safe notices to their original
conversation.
> - Periodic publication maintenance repeatedly inspects those receipts
when idle.
> - Committing receipt, authority, lifecycle and lease changes already
identifies when eligibility can change.
> - A scoped owner reconstructs durable obligations, drains bounded
candidates, retains contention, then becomes quiet.

## Linked Issues or Issue Description

This existing failed-retry notice slice follows #17647. It removes the
failed-retry notice duty from periodic publication maintenance while
preserving the explicit manual API. Existing action, receipt,
publication and source rows remain authoritative; no new table or
scheduler is added.

**Problem:** Removing the scan alone loses notices after real admission,
cancellation, early refusal, requester or agent restoration, and
relevant lease settlement. Millisecond cursor truncation can also repeat
denied history.

**Expected behavior:** Committed supported writes wake the scoped notice
owner, which uses current authorization and precise cursors. Real
retries survive failure, contention, restart and drain; settled work has
no recurring SQL.

## What Changed

- Extract failed-retry projection under the existing authorization
owner. Preserve current receipt/source authority and provider transport
authorization.
- Register bounded scoped work using the existing delivery
coordinator/scheduler. Preserve exact PostgreSQL cursor precision,
contention and a wake arriving during reconciliation.
- Extend the shared transaction-aware notification boundary to
failed-retry notices. Cover admission, conditional denial, early
refusal, requester changes, and agent lifecycle transitions to an
invokable status. Two shared receipt hooks and their budget fixture
correction already landed in the preceding inbound slice; preserve them
once.
- Notify when relevant native lease cleanup completes or a reusable
lease is retired, preserving claim fencing and caller-owned
transactions.
- Avoid the discovery scan identified in #17647: known task scopes
require zero discovery SQL; company/startup discovery uses a single
DISTINCT projection from the same candidate predicate. Bounded current
authorization still runs afterward.
- Remove only this duty from periodic publication maintenance, retain
manual behavior and startup reconstruction, and add real
producer/lifecycle/query-budget regressions with an audit.

## Verification

Published head: `fa95a1f2fb9d546aa62dbd3b43aa714f5c88bd35`, targeting
master after #17647 merged at
`1aa3a21ac41f602977638cf0be875b01a243e69f`.

- **42 distinct current combined-master cases pass:** 21
retry-notice/native-lease/restored-input producer cases (1,651 skipped),
plus all 21 app/startup and task-drain admission-release cases. Server
typechecking, build and module boundaries pass. The final follow-up adds
acceptance documentation only; all 17 owned server source/test hashes
are unchanged.
- The prior **404-case** matrix passes at prepared revision
`c2c53b839497be6f5e28aa2cd5ba50cfd1d76d4a`: 259 across six complete
consumer/lifecycle/environment suites, 20 notice/lease cases, 53
ordinary-notice and 72 Retry/budget cases. Those chat selections skip
1,651/1,618/1,599 cases. A subsequent 21-case combined producer check
passes after the parent restoration repair. These earlier runs retain
their revision scope.
- The normal two-parent integration
`43d0f84ea56845b8fc4946a7533ecbf7f07b679a` preserves the complete
19-path owned binary patch and all 23 unrelated master paths/deletions.
The semantic merge base is the verified landed parent head
`41f7e2c9e351efa6cfa5ef699c0e69df7425012b`; squash ancestry otherwise
causes historical conflicts. Both parent commits and the exact resolved
tree are verified; no rebase or force push occurs.
- Three scope-discovery regressions fail before the fix. The first SQL
projection fails with PostgreSQL 42702 (ambiguous company column), then
passes after using a distinct company alias. Known task hints have zero
discovery SQL; company/startup discovery has one query returning only
distinct task scopes. Existing bounded projection/current authorization
remain intact.
- Current producer tests cover commit/rollback, parked authority
restoration, real agent resume, early refusal, requester changes, native
lease cleanup/replacement, drain/restart/shutdown, precise cursors and
contention. Parent shared admission/denial/restored-input notifications
are preserved once. No polling backstop or storage is added.
- [The owner
audit](https://github.com/paperclipai/paperclip/blob/fa95a1f2fb9d546aa62dbd3b43aa714f5c88bd35/doc/plans/2026-10-25-failed-retry-notices-sleep-audit.md#final-landed-parent-acceptance--october-28)
records current commands, fixtures, failure history and limits. Fresh
exact-head remote checks and Greptile 5/5 are pending; old-head reviews
do not count.

The owner audit preserves prior failures and their original revision
scope. The old remote head's Runner receipt timeout, cancelled gate, and
review error are not current passes; fresh exact-head checks and
Greptile 5/5 are required. No full local repository suite, live provider
tenant, external wake alarm, physical suspension or savings claim is
made.

### Final remote gates

Exact published head `fa95a1f2fb9d546aa62dbd3b43aa714f5c88bd35` has
Greptile 5/5 with no actionable findings, successful current main CI and
ready-triggered security scan. The final dependency/quality review
workflow `40747725096` passed after waiting for a GitHub-hosted runner.
No Ponytail verdict is present.

Two automatic main CI requests and two identical-head quality-review
requests were created one second apart. The older main run was
automatically cancelled; only the proven duplicate, unstarted quality
request was cancelled manually. Their same-head, same-workflow
replacements both finish successfully. The merge guard verifies these
replacements, all current checks and current required-status integration
rules before excluding the superseded cancellation contexts. No actual
failing review or current check is waived and no protection bypass is
used.

## Risks

- Process-local signals cover supported writers using this boundary, not
independent SQL or other replicas.
- Lease completion now reads/locks the exact row before updating.
Reusable lease retirement wakes company scope because its previous issue
can differ. Claims and transaction ownership remain enforced.
- Broad discovery can inspect history inside PostgreSQL; DISTINCT
reduces returned rows and round trips, without claiming constant
database cost. Task hints are wake requests, so bounded reconciliation
handles already exhausted scopes.
- Native filesystem activation belongs to execution authorization and is
not an eligibility input for this notice projection. Other
notice/maintenance duties and Cloud sleep gates remain separate.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, local code inspection,
shell execution, and PostgreSQL testing. The exact serving model
identifier and context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 14:14:28 -05:00
DottaandPaperclip 1aa3a21ac4 fix(server): wake inbound queue notices from committed changes (#17647)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Chat users receive a notice when accepted input waits for execution.
> - Periodic publication maintenance scans those receipts even when
nothing changes.
> - Admission, cancellation, source removal, and authority changes
already define when a notice can change.
> - Existing action, receipt, publication, and provider-link rows
preserve that work across restart.
> - This PR gives ordinary inbound queue notices a commit-driven owner
using the shared scheduler.
> - Settled notices stop querying while unfinished batches and
contention retain recovery.

## Linked Issues or Issue Description

This ordinary inbound-notice slice follows merged #17895 and now targets
master. Related extraction: #16187. Failed-run retry notice projection
remains a separate following slice (#17649); its two shared
receipt-writer notifications move forward here because ordinary input
can attach to those receipt owners.

**What happened?**

Publication maintenance scans accepted input to project queued,
not-started, and removed-source notices. Empty scans keep touching the
database. Its cursor also loses PostgreSQL timestamp precision, so a
full page of suppressed candidates can repeat.

**Expected behavior**

Committed changes should wake the notice owner. The worker should
reconstruct saved candidates after restart and stop scanning when
settled. It must retain a wake across drain, failed selection,
contention, and a commit during reconciliation.

**Steps to reproduce**

Start an otherwise idle instance with chat history. Observe the ordinary
inbound notice scan in periodic publication maintenance. For the cursor
defect, retain a full page of denied candidates with microsecond
timestamps before an eligible notice.

## What Changed

- Register an ordinary inbound-notice owner with the existing delivery
coordinator and scheduler. Bound each pass to 100 candidates, preserve
precise continuation cursors and earlier-page contention, and reset
scoped discovery after committed changes.
- Notify from real admission, failed action settlement, source removal,
task/tree/agent/budget/company cancellation, signed Slack Stop, and
deferred queue cancellation/failure. Preserve pending-status and
absent-run guards so concurrent promotion wins.
- Cover task privacy/grants, project membership/visibility and current
authority through their persistence owners. Keep receipt ownership,
current authorization, provider policy and closed notice text in the
existing chat service.
- Cover shared retry-owned receipts through the existing durable
authorization callback and a notifying transaction around conditional
denial retirement. These small producer fixes already exist in #17649;
no per-caller recovery wrappers are added. Retain its matching
budget-refusal fixture correction.
- Remove only ordinary notices from periodic maintenance. Preserve the
separate failed-retry pass at 50 and keep the explicit combined manual
API.
- Preserve timestamp precision, fairness, restart, drain/resume,
rollback and quiet behavior. Add two real producer transaction
regressions and update the audit. Add no storage or scheduler
infrastructure.

## Verification

Current publication head: `41f7e2c9e351efa6cfa5ef699c0e69df7425012b`,
after normal merge of master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`
(#17960). The integration preserves both complete added/removed patches,
all non-overlap upstream blobs/deletions, and task-before-comment lock
order before the notice source lock/intent. Full binary identity is not
claimed for the overlapping files. The final follow-up changes
acceptance documentation only.

- **202 distinct combined-master cases pass on
`1720b685d9d5fb9cb4377223fdaa0a9fae2aff9c`:** 148 across the real
PostgreSQL receipt concurrency and two comment route suites, plus 54
notice/FIFO/producer cases (1,598 skipped). Current server
typechecking/build and module boundaries pass; all 21 owned source/test
hashes are unchanged after the documentation follow-up.
- The earlier CI voice-reconstruction quiet assertion failed once after
105 fetches. All 15 voice lifecycle cases pass unchanged locally; no
assertions or production code were relaxed and no CI retry was used. The
original failure remains in the audit, and current-head full CI must
pass.
- **Earlier restoration repair (head
`28c898e93504318056d7a6ab2bbc5eafa6b639ae`): 126 distinct selected cases
pass (54 ordinary-notice/producer cases with 1,598 skipped, plus 72
separate Retry/budget cases with 1,580 skipped).** Fix the blocking
[restored input stays
silent](https://github.com/paperclipai/paperclip/pull/17647#discussion_r4404007181)
finding. A failed Retry's earlier receipt wake can drain before
`releaseUnadmittedTaskRetryInput` restores ordinary receipts in a later
transaction. The restoring transaction now awaits the existing
company/task notification before its guarded write.
- The new causal regression builds a durable unadmitted failed-Retry
owner, uses the real input-claim owner, consumes an earlier committed
wake, and proves two minutes of worker quiet with no notice. The real
release transaction then restores the ordinary row, emits intent/commit
settlement, and reactivates the coordinator to create its queued notice.
It fails before the hook with the restored row visible but no intent.
This is real PostgreSQL producer/consumer boundary proof with seeded
Retry-refusal state; the separate compatibility selection exercises
existing supported GitHub Retry flows. It is not a claimed complete
public-request failure walkthrough.

- **Earlier discovery-repair acceptance (revision
`4a713edfa06b9d3dcc0e81e900006fbdb973b2d9`): 267 distinct selected cases
pass:** 142 complete cancellation/app/admission consumer cases, 53
ordinary-notice cases, and 72 separate retry/budget cases. The
selections skip 1,598 and 1,579 other cases. The final master
integration leaves this complete patch byte-identical; the new master
paths concern provider workspace/session reuse and E2E reports.
- Fix the valid P2 [discovery rereads denied
history](https://github.com/paperclipai/paperclip/pull/17647#discussion_r4386889825),
despite the previous head's 5/5 summary. Task hints now need zero
discovery SQL; company/startup discovery returns DISTINCT company/task
identifiers with one query. Bounded projection still checks current
authority and candidates. Three real database regressions with 250
denied actions and a second company fail before the fix and pass
afterward, checking actual query and returned-row counts. Broad SQL may
still inspect candidate history; no constant database-cost claim is
made.
- Both new producer tests fail before their fixes and pass afterward.
One observes intent before a callback-owned receipt write and settlement
after outer commit. The other uses real heartbeat admission/recovery and
a PostgreSQL row lock to prove exact company/task intent exists before
conditional denial retirement; the outside connection still sees
deferred status until commit.
- The reader can select a retry-owned deferred receipt for later
ordinary input, and receipt resolution follows its owner. Disabling
coalescing on the retry's own request does not exclude that later path.
The two transferred notifications close the actual writer gaps; current
authorization remains enforced by existing consumers. This is
producer/transaction proof, not a claimed complete mixed-ingress
walkthrough.
- The moved callback exposes an old test mock that authorized against
the root DB. The existing #17649 fixture correction authorizes and
writes in one transaction; original requester, comment, skipped-status
and budget assertions remain. Both the initial invalid lock fixture and
the corrected missing-intent failures are preserved in the audit.
- Updated shared dependency build, server typechecking/build and
module-boundary checks pass. Because master also changed agent
configuration, upgrade admission and runner contracts, the same 264
cases were rerun on combined revision
`875a6ee8d211cba7f0499aace7fcea7f6880c4a2` with new isolated databases.
Repeated checks are not 528 different tests. Tests under `src/__tests__`
were executed separately from production typechecking.
- [Current acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/41f7e2c9e351efa6cfa5ef699c0e69df7425012b/doc/plans/2026-10-25-inbound-wake-notices-sleep-audit.md#shared-retry-receipt-producer-gap-resolved-october-28)
records current scope, exact commands, actual fixtures, source hashes
and remaining limits. Earlier baseline/full-suite counts retain their
original revision scope.
- Server typechecking/build and module boundaries pass after the current
restoration hook. The earlier combined-master compilation at
`661f2f6da8c6c22cdcc25edccd57502ee6fa8c4b` retains its original scope.
- **Current exact-head CI is green and Greptile is 5/5.** The old head's
review error is not a pass. No full local repository matrix, live
provider tenant, external alarm, physical suspension or savings claim is
made. Following notice duties retain their separate owners.

### Final merge verification

- Reviewed PR head remains `41f7e2c9e351efa6cfa5ef699c0e69df7425012b`:
exact-head Greptile 5/5, all 55 check contexts clean, no unresolved
review threads, and no Ponytail verdict present.
- One Runner shard initially failed descriptor suspension and a
five-second OpenCode fixture deadline. The same package passed on the
preceding head. The first local attempt was invalid because required
Rust fixture binaries were absent; after building them, all eight
relevant cases pass with unchanged timeouts. One failed-job retry then
passes 1,956 cases (four skipped). No source/assertion/time-budget
change or second CI retry was used.
- Current master `0e365768f032a472807b5720da8431f92d2f07ce` adds manual
monitor and task-drain fixes. Its routes file overlaps but merges
cleanly. Local normal-merge validation commit
`2720b79c769cb609ad4d392512331e67b910ff3f` preserves both complete
added/removed patches and every unrelated master blob/deletion. This is
not raw binary identity for overlapping files.
- **156 distinct cases pass on that exact combined tree:** all 102
monitor/drain/policy cases plus 54 notice cases (1,598 skipped). Types
and build pass. An initial competing-cold-check run failed setup hooks
(38 passed, two hook failures, 62 skipped); the same four suites pass
alone with all original hook/test timeouts and assertions intact. The
failed run remains recorded, not counted as full acceptance.
- This merge proof changes no published source or review head. The
protected pinned-head merge guard recomputes both patches and requires
the exact tested master/tree/head, successful validation receipts and
unchanged live merge gates. Previous acceptance counts retain their
original revision scope; no local full-repository or deployed-sleep
claim is made.

Master then advanced to `75754ffb7af2b7ee7dce1e915a145faa82a8fb2a` with
#17976 (five UI/e2e/map files only). The entire `server` and `packages`
trees are identical to the validated composition. Both complete patches
and all newly landed UI files are preserved in the final clean merge
tree; the protected gate checks these exact source-root hashes and
changed paths. No additional UI-test result is claimed.

## Risks

- Missing a supported producer could delay a notice until restart. Tests
cover real service and API producers; durable rows remain authoritative.
Notifications cover supported application writers in the owning process,
not arbitrary external SQL writers.
- Privacy invalidation can affect descendants, so those changes use
company scope. Ordinary admission and publication changes use task
scope. Settled history causes no recurring scan.
- Provider output remains subject to current authority and receipt
ownership. A cancelled request without a visible queued notice does not
acquire a new failure message.
- This establishes worker quiet, not host or database suspension. Cloud
alarms and conservative sleep gates do not change. Other maintenance
remains in place.
- Remaining stacked slices still need their own integration, review, and
CI. This slice merges only under the user’s explicit campaign
authorization after current-head gates pass.

## Model Used

OpenAI Codex, GPT-6 family, with repository inspection and code
execution tools. The exact serving variant and context-window size are
not exposed in this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 13:34:38 -05:00
DottaandPaperclip 1ed0af4186 fix(server): wake native chat progress from committed changes (#17639)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Chat users receive coarse progress updates while native runs
execute.
> - A recurring chat pass scans saved run events even when no progress
changes.
> - This traffic prevents an idle database from staying idle.
> - Events, lineage changes, and publication state already identify when
projection can change.
> - This PR gives native progress its own commit-driven worker and
schedules only unfinished work.
> - Idle progress stops scanning while restart and recovery preserve
delivery.

## Linked Issues or Issue Description

Refs #17606. This is slice 29 of the recurring-server-work stack, based
on `codex/sleep-28-chat-run-milestones`. It owns native chat progress
only. Related historical optimization: #16845. The parent and this PR
must land in stack order.

**What happened?**

Native chat progress is still called by the one-second chat
reconciliation loop. It repeatedly selects run events with no new work.
Its cursor also truncates PostgreSQL timestamp precision, which can
repeat a rejected full page forever.

**Expected behavior**

Committed events and eligibility changes should wake the projection
owner. Existing rows should reconstruct work after restart. A settled
worker should have no recurring database activity. Full batches, lock
contention, and errors must retain recovery.

**Steps to reproduce**

Start an otherwise idle instance with chat history. Observe the native
progress lane in the recurring chat coordinator. For the cursor defect,
create a full page of suppressed candidates sharing a microsecond
timestamp and one eligible later candidate. The old cursor fails to
advance.

## What Changed

- Extract native progress from lifecycle milestones into its owning
service. Keep the public combined API and budget compatible.
- Register a scoped worker with the existing delivery coordinator and
scheduler. Remove native progress from the recurring chat coordinator.
- Notify before allowlisted event writes in both persistence owners.
Preserve outer commit, rollback, duplicate detection, and the ordinary
non-progress log fast path.
- Extend the existing lineage/authority notification boundary once.
Failed-publication replay and operator retry carry their issue scope.
First working publications signal progress so commit ordering cannot
strand an already-inspected event.
- Limit notification payloads to company/task IDs so wider caller inputs
never serialize database handles or private fields. Load task-binding
fields only for progress event kinds; other native/recovery events
retain their narrow lock query.
- Preserve current safe phrases, cadence, phase deduplication, provider
policy, authorization, and final-output precedence.
- Retry concrete issue/run lock contention. Disarm deleted or terminal
candidates. Preserve scoped dirty work across batches, failures, drain
and shutdown.
- Preserve PostgreSQL timestamp precision in pagination. Add a bounded
regression for the former infinite rejected-page scan.
- Document the producer audit, lifecycle contract, failures, and
remaining limits in
`doc/plans/2026-10-25-native-chat-progress-sleep-audit.md`.

## Verification

Published head: `b7ee820e49fd06c780b36b79d9b1c0883557fb73`, targeting
master after #17606 merged at
`5f0da562dbfdf3229f3cd4bb3dc742ab541986b4`. A normal ancestry merge
preserves the complete 18-path native-progress patch byte-for-byte
relative to its verified parent, with all six unrelated current master
paths intact. No new production change was introduced during final
reconciliation.

- **163 distinct selected cases pass on the verified unchanged patch:**
137 across eight complete suites (native progress, atomic event append,
all five milestone worker suites, and app admission wiring), plus 26
real chat integration cases selected by `native progress`. The other
1,595 chat integration cases were skipped, not passed.
- The full worker suites cover restart, committed events, trace
reprojection, duplicate/no-progress events, first-working publication
ordering, concrete lock contention, issue scope, backlog, drain/resume,
failures and shutdown. Settled history leaves no recurring SQL. The real
chat cases verify operator retry and replay, microsecond pagination,
authority, cadence, phase deduplication, final/question precedence and
redaction.
- The sole earlier parent conflict preserves both this slice's
issue-scoped retry wake and the exact voice conversation scope. The
current parent's milestone producers, onboarding completion guard,
identity query/read-scope coverage, migration 0397 and voice
cleanup/retention repairs remain present.
- Server production typechecking, build and module-boundary checks pass.
The selected server tests live under the excluded `src/__tests__` path
and were executed separately. Each database owner uses a new isolated
app-migrated PostgreSQL fixture.
- [Current-parent acceptance and failure
ledger](https://github.com/paperclipai/paperclip/blob/b7ee820e49fd06c780b36b79d9b1c0883557fb73/doc/plans/2026-10-25-native-chat-progress-sleep-audit.md#current-milestone-parent-acceptance-october-28)
records current counts and scope. Older combined checks and failures
retain their original revision scope and are not added to 163. No new
table, dependency, workflow, migration, or scheduler contract is
introduced.
- **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No
Ponytail review is present, which is an explicitly authorized
nonblocker; no failing verdict is waived. Older head/parent reviews are
not current passes. Actual failing Ponytail findings must be fixed.
Missing/stopped/paused turns without a verdict are authorized
nonblockers; active unfinished reviews retain the original per-head
timer with a 25-minute monitor cutoff respecting the 30-minute maximum.
- No complete local repository matrix, live provider tenant, external
wake, physical host/database suspension or savings claim is made.
Following stack duties remain separate.

## Risks

- Signals are process-local under the existing single active writer
contract. Independent SQL writers or another active process need an
explicit wake integration.
- Allowlisted direct log writes now use a transaction. Native bound runs
register transaction intent; unrelated direct event types retain their
atomic-statement path.
- The shared eligibility boundary wakes both lifecycle and native
projection. It does not decide provider policy or copy event payloads.
- The existing 24-hour selection window, event-time cadence and
authority checks remain. A suppressed event is not retried just because
wall-clock time passes.
- This establishes worker quiet, not whole-instance suspension. Cloud
alarm coverage and conservative sleep gates do not change. Other chat
maintenance remains on its existing cadence.
- This actual-master reconciliation still requires fresh exact-head
review and CI before protected merge. Subsequent slices retain their own
gates.

## Model Used

OpenAI Codex, GPT-6 family, with repository inspection, code execution
and browser-test tools. The exact serving variant and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 11:22:29 -05:00
DottaandPaperclip 5f0da562db fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected chats show safe notices when a run starts, waits,
finishes, or fails.
> - A recurring scan projected these notices even when no eligible work
existed.
> - Eligibility depends on committed run state, conversation lineage,
and current authority.
> - Existing database rows can reconstruct that work after restart.
> - This PR wakes a scoped worker from those writes and removes
lifecycle polling.
> - Native progress events keep their separate existing owner until the
next migration.

## Linked Issues or Issue Description

**What happened?**

The chat maintenance timer repeatedly scanned run lifecycle state. Empty
and settled instances still queried the database. A run-status event
alone cannot replace the scan: prompt receipts, continuation receipts,
identity changes, and restored authority can make a run eligible later.

**What did you expect?**

Committed changes wake projection. Startup reconstructs eligible work. A
completed pass stops until another relevant change arrives. Failed work
retains recovery without scanning settled history forever.

**Steps to reproduce**

Configure an automatic chat endpoint. Finish its work, or leave it
unused. Observe recurring lifecycle projection queries from the chat
maintenance callback.

Stacked on #17592. Related foundation: #15625. This PR owns lifecycle
milestones only.

## What Changed

- Preserve upstream coalesced Cloud membership writes together with
locked membership restoration and its committed chat notification.

- Signal milestone eligibility from the real Slack account-link
transaction.

- Register the lifecycle worker with the existing delivery coordinator.
Remove lifecycle projection from the periodic chat callback. Keep native
progress in a separately named lane and preserve the explicit combined
API.
- Reconstruct company/issue scopes from the existing selector. Drain
bounded batches with company fairness. Retry failures, retain writes
through drain/resume, and disarm settled work.
- Signal from supported run, presentation, interaction, receipt,
lineage, conversation, recovery, and authority writers. Preserve their
transaction ownership, including nested savepoints and uncertain
settlement. No new table is added.
- Preserve provider authorization, authored-final suppression,
responsible-user continuity, deduplication, the 24-hour selection
window, and raw-output restrictions. Keep provider interpretation in its
owner.
- Preserve PostgreSQL timestamp precision in lifecycle pagination.
Rejected submillisecond candidates must not repeat the same page and
starve later notices.
- Document producer coverage and alternate-writer dispositions in
`doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`.

## Verification

Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting
master after #17592 merged at
`7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates
the actual landed parent. The entire 77-path milestone patch is
binary-identical to the locally verified preparation across that
ancestry reconciliation; both unrelated master paths are preserved. The
four earlier producer/test integration conflicts retain scoped milestone
and voice notices, current onboarding completion admission, and expanded
identity query/read-scope coverage.

- **330 distinct selected cases pass on the verified unchanged patch:**
305 across 14 complete suites (all five milestone worker suites, run
identity, board claim, Cloud provisioning, auth session routes, Cloud
actor authentication, stale-lock recovery, GitHub credentials, project
Git authorization, and app admission wiring), plus 25 selected real chat
producer cases. The remaining 1,590 chat integration cases were skipped,
not passed.
- The real producer cases cover saved milestone discovery, GitHub and
Slack identity changes, account-link rollback/reconfirmation, ordinary
ingress reactivation, post-close action/delivery receipt settlement, all
ten authority-restoration paths including Cloud synchronization, and
changed native review metadata.
- Worker coverage verifies outer commit, rollback, uncertain-write
recovery, restart, bounded batching/fairness, timestamp precision,
retries, drain/resume, and settled SQL quiet. Authorization and existing
publication deduplication remain with their owning services.
- Server production typechecking, build, and module-boundary checks
pass. The tests above were executed separately. Production typechecking
excludes `src/__tests__` and does not replace those runs. Each database
owner uses its own newly created and app-migrated PostgreSQL fixture.
- [Current-parent acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28)
records commands, counts, producer scope, conflict dispositions, and
limits. Earlier combined-stack counts and failures keep their original
revision scope; local acceptance does not erase old CI failures.
- No new migration, queue table, dependency, workflow, or shared
scheduler contract is introduced. Parent migration 0397, bounded consent
recovery, voice cleanup/retention repairs, and runner checkpoint
restoration remain present.
- **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No
Ponytail review is present, which is an explicitly authorized
nonblocker; no failing review is waived. Older reviews and parent checks
are not current passes. Actual failing Ponytail findings must be fixed;
an absent/stopped/paused turn without a verdict is an authorized
nonblocker. Active unfinished Ponytail uses its original request/start
timer, with a 25-minute monitor cutoff respecting the user's 30-minute
maximum.
- No complete local repository matrix, live provider tenant, external
alarm, physical host/database suspension, or fleet savings claim is
made. Native progress remains a separate following slice.

## Risks

- A future eligibility writer must notify the existing projection
boundary. Independent SQL writers or another active server process are
outside this process-local signal contract.
- Authority and lineage remain sensitive. The worker uses the existing
selectors and publication policy; it does not grant execution or
disclosure authority.
- Native progress and other publication maintenance still have recurring
work. This PR removes one reason, not the entire chat maintenance timer.
- Worker quiet does not prove database suspension, host sleep, or
external alarm coverage. Cloud sleep blockers remain unchanged.
- Merge eligibility requires passing current-head CI and Greptile 5/5
after this actual-master reconciliation. Subsequent stack slices retain
their own gates.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window size are not exposed. Used repository inspection,
reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub
tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available in this session)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run the documented focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 11:07:54 -05:00
DottaandPaperclip 7d75ae0569 fix(server): recover Teams attempts from saved claim deadlines (#17592)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Teams file delivery saves a claim before each provider operation.
> - A crashed operation can leave that claim unfinished.
> - Periodic maintenance scans for abandoned claims even when none
exist.
> - This PR wakes recovery from committed changes and the saved claim
deadline.
> - Empty work becomes quiet while unknown effects remain safe to
review.

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Teams abandoned file-transfer recovery. This is slice 27, stacked on
consent expiry #17590.

**Current behavior**
The remaining Teams maintenance callback scans every 30 seconds for
expired transport claims. It runs from shared publication maintenance.

**Proposed behavior**
The Teams owner reconstructs existing claim deadlines at startup and
reloads a transfer after its committed update. At expiry it clears the
claim and preserves the existing unknown-outcome or conflict state.
Normal receipt settlement removes the deadline.

**Reason and benefit**
Remove the last periodic Teams maintenance scan without resending a
possibly delivered card or file.

**Breaking changes**
No schema or public API change. Explicit diagnostic recovery methods
remain compatible. Consent expiry stays with its separate parent worker.

## What Changed

- Add a Teams-owned worker for consent_sending, uploading,
file_info_sending and conflict rows with a saved attempt expiry.
- Notify from the existing versioned update before writes. Preserve
transaction ownership and shared uncertain-write reconciliation.
- Expose a narrow per-transfer recovery method that uses the existing
lock order, integrity checks and atomic projection.
- Remove the periodic Teams maintenance callback and its
publication-maintenance invocation.
- Add real claim, receipt, conflict, backlog, restart, precision,
failure, drain and shutdown tests. Wire the owner through existing app
admission gates.
- Retain the existing signoff cancelled-run selection test and GitHub
inline retry-window race assertion.
- Document supported writers, remaining sleep limits and the failure
ledger in `doc/idle-teams-attempt-recovery.md` and its linked audit.

## Verification

Published head: `24c54399909105984e2a10f5869064984f69c472`, targeting
master after consent #17590 merged at
`4a89456bb3f9aef87cfe0e922e8b9d1a777a6c7e`. A normal ancestry merge
reconciles the landed squash with the previously verified child. The
complete 13-path binary patch is identical before and after
reconciliation, and all 30 unrelated master paths are preserved. This
adds no new worker behavior; the 239-case local evidence below belongs
to the verified unchanged patch. All 58 current-head checks are now
clean, including the complete required CI run. Greptile is 5/5 at this
exact head. The current Ponytail turn is paused without a verdict, which
is an explicitly authorized nonblocker; no failing review is waived.

The complete 13-path slice preserves the reviewed attempt implementation
and inherits the parent's report migration 0397, bounded consent worker
and retention CI test repair. No new schema, queue table, shared
scheduler contract, workflow or lockfile change is added by this duty.

- **239 distinct selected cases pass:** 221 across five complete
transfer/publication/consent/runtime/app-wiring suites, 16 selected real
Board/Teams-authority/GitHub-retry integration cases, and two delegated
Slack policy cases. Other chat integration cases (1,579) and
connection-management cases (81) were skipped, not passed. Each database
owner uses a new isolated PostgreSQL database.
- Real Board publication saves an active attempt deadline; its confirmed
receipt retires recovery, and SQL stays quiet afterward. The inherited
consent producer and authority cases still pass. Lifecycle coverage
exercises claim stages, unknown-outcome quarantine without repeating
provider I/O, conflicts, rollback, restart, precision, backlog/fairness,
failure recovery, drain and shutdown. The changed GitHub metadata/retry
race passes without duplicate assessments.
- The old server10 CI failure, run `40557802358` job `122390238707`, was
a delegated Slack invitation rate-window assertion (2,122 passed/one
failed/15 skipped in that shard). The verified parent already contains
its Date-only fixture fix. Both linked and invitation cases now pass
their full caller/target/shared-policy scenarios on a fresh server
fixture; no duplicate policy change or new owned file was added. That
old failure is preserved rather than called a pass.
- Server production typecheck, build and module-boundary checks pass.
Test files were executed; production typechecking does not statically
check them. The existing test-only signoff cancelled-run selection fix
is retained but was not executed locally in this acceptance; fresh
remote E2E gates remain required.
- [Current-parent acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-25-teams-attempt-recovery-sleep-audit.md#current-consent-parent-preparation-and-acceptance-october-28)
records current scope, commands, counts and limits. Earlier 235-case and
combined-stack counts belong to their stated revisions and are not added
to 239. The [stack
ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-24-recurring-server-sleep-stack.md)
retains inherited failure history. No complete local repository suite,
live Teams/Slack tenant, external wake, physical suspension or savings
claim is made.
- **Exact-head Greptile 5/5 and all 58 checks are clean.** Actual
failing Ponytail findings must be repaired; a missing/stopped/paused
turn without a verdict is an authorized nonblocker. An active no-verdict
turn waits at most 30 minutes from its original request/start (25-minute
monitor cutoff). Older reviews and parent checks are not treated as
current passes.

## Risks

- Recovery must not infer that an expired claim means a provider effect
did not happen. It retains unknown outcomes and does not resend.
- Late receipts must still match current claim identity. Current-state
rechecks and existing protocol tests preserve this fence.
- Supported deployment uses one active writer process through its owning
DB client. Independent SQL and additional processes need a separate wake
contract.
- Cloud external alarm coverage and sleep eligibility are unchanged.
Other chat duties still run periodically until their own migrations.
- Voice-report parent review repairs are propagated: explicit
single-active-writer scope and an indexed report window. Thirty
report-worker/startup tests pass on the combined child. Fresh remote
checks, older stack findings and final master reconciliation remain
required.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, tool use and
code execution. The exact serving model ID and context-window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 10:51:16 -05:00
DottaandPaperclip 4a89456bb3 fix(server): schedule Teams consent expiry from saved deadlines (#17590)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Teams file delivery saves consent and transfer state in the
database.
> - Unused consent must expire even when no new message arrives.
> - The maintenance loop currently checks this state every 30 seconds.
> - This PR schedules each existing consent deadline and wakes on
committed changes.
> - Empty or settled consent work stops querying while expiry remains
recoverable.

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Teams unused file-consent expiry. This is slice 26 of the
recurring-server migration stack, following merged #17586 and targeting
master.

**Current behavior**
The Teams maintenance callback scans for expired unused consent and
abandoned transport attempts together, even when consent work is empty.

**Proposed behavior**
The Teams owner reconstructs unused consent at startup and reloads one
transfer after a committed write. The shared scheduler runs its saved
expiry. Abandoned attempts keep their current recovery until a separate
slice migrates them.

**Reason and benefit**
Remove unnecessary consent queries without losing expiry, atomic
publication projection, or recovery after restart.

**Breaking changes**
No schema or API change. The explicit combined recovery method remains
available. Cloud sleep eligibility and external alarms remain unchanged.

## What Changed

- Add a Teams-owned scoped deadline worker for the three unused consent
phases. Registered reconstruction yields after one 100-row page per
scope refresh and resets cursors on registration; the explicit
diagnostic inventory still returns every eligible row.
- Signal before insertion and the existing versioned update inside the
owning transaction. Nested writes wait for outer settlement.
- Extract the existing locked per-transfer recovery operation. Preserve
authority checks, atomic projection, bounded lock waits and
unknown-outcome quarantine.
- Remove unused consent from periodic maintenance selection and
execution. Keep abandoned attempts as a separate obligation.
- Register the worker through existing app lifecycle gates. Add
lifecycle, producer, production wiring and precision regressions.
- Document supported writers, remaining sleep limits and the failure
ledger in `doc/idle-teams-consent-expiry.md` and its linked audit.

## Verification

Published head: `9e84cfab1f795279c8cec413a946631f1d781bd2`. This
follow-up changes only the inherited voice-retention test and its CI
failure ledger; all production behavior is unchanged from `36f43922`. It
normally merges actual landed master/report parent
`8049fbf51d0701ca6b242e4b1198e5449f7661b2`. The verified 12-path Teams
patch was byte-identical across the actual-parent ancestry integration.
This test-only CI follow-up adds one existing retention test path,
bringing the final scope to 13 paths. Every unrelated master change,
report documentation fix, and migration 0397 is preserved. No database,
shared-worker, workflow or lockfile change belongs to this slice.

- **223 distinct selected cases pass:** 209 across five complete
transfer/publication/consent/runtime/app-wiring suites, plus 14 selected
real Board-producer and Teams-authority integration cases. The other
1,580 integration cases were skipped, not passed. The protocol and
production consumers use separate new PostgreSQL databases migrated
through the application migrator.
- A real 105-transfer regression failed against the original worker
because one registered startup pass selected 105 rows instead of
yielding at 100. The corrected worker uses the existing paged shared
contract. Restarting the same service midway through discovery selects
the first 100 again, restores all 105 saved deadlines and expires them
at the original deadline. SQL is quiet before the deadline and after
settlement, with no consent-card or upload requests. The one-shot
inventory still returns all 105 rows.
- A guarded mutation removing only cursor reset fails at five rows
instead of 100 after restart. The original production source was
restored before the complete suites. The initial regression fixture
incorrectly faked network timers before its first SQL connection and
timed out; that setup failure is retained separately from the corrected
105-versus-100 regression. No assertion or timeout was weakened.
- The real Board producer registers the worker before publication,
commits the transfer wake, preserves the saved expiry, remains SQL-quiet
until due and atomically projects expiry to a cancelled publication
without another provider request. Existing authority and consent
transitions remain covered.
- Complete lifecycle coverage retains all three consent phases,
precision rounding, rollback and held outer commit, exact-row hints,
expiry recheck, failed projection retry, corrupt-row fairness,
drain/resume, shutdown and separation from abandoned-attempt
maintenance. Production server typecheck, build and module boundaries
pass. Server production typechecking does not statically check test
files; those files were executed.
- [Current bounded reconstruction and failure
ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#bounded-reconstruction-and-current-parent-acceptance-october-28)
records exact scope, counts, guarded mutation, corrected timer setup,
earlier fixture failures and limitations. Earlier 222-case and
combined-stack results belong to their stated historical heads; they are
not added to 223 or claimed rerun here. The [stack
ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-24-recurring-server-sleep-stack.md)
retains inherited failure history. No complete local repository-suite,
live Teams tenant, Cloud wake, host/database suspension or savings claim
is made.
- **Fresh exact-head Greptile 5/5 and all CI are required.** Actual
failing Ponytail findings must be repaired. A missing/stopped/paused
turn without a verdict is an authorized nonblocker; an active no-verdict
review waits at most 30 minutes from its original request/start
(25-minute monitor cutoff). Older scores and parent checks are not
accepted as current passes.

- **Preserved inherited CI failure:** prior head `36f43922` had
exact-head Greptile/Ponytail 5/5 but CI run `40708566195` server6 job
`122887395220` failed the retention fairness assertion (2,291 passed/one
failed/one skipped in that shard). A fixed final UUID reproduces the
wrong expectation that an undiscovered page-two row is already deleted.
The corrected test controls time, expects that row to remain before
discovery and requires its deletion in the next fair batch, then all
remaining history and idle SQL. The existing five-second wait budget and
global timeout are unchanged. All 13 retention lifecycle cases pass;
bypassing company rotation fails the corrected assertion, and the
original shared helper is restored byte-for-byte. Earlier 223-case Teams
acceptance remains valid for unchanged Teams source and is not relabeled
as a new complete repository run. [Full CI repair
ledger](https://github.com/paperclipai/paperclip/blob/9e84cfab1f795279c8cec413a946631f1d781bd2/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#inherited-retention-ci-fairness-repair-october-28).
Fresh CI and exact-head reviews are required for this test-only
follow-up; old scores are stale.

- **Current merge gates:** all 58 checks are now clean for this exact
head and Greptile is 5/5 with no actionable findings. The current
Ponytail turn is paused without a verdict, an authorized nonblocking
state. Workflow `40712464720` attempt 1 independently hit npm
`ECONNRESET` during the serialized8 job's tool self-installer, before
its tests ran. Exactly one targeted job retry was requested after the
original workflow finished; attempt 2 completed successfully on the
unchanged head. No code, dependencies, workflow, timeout or review timer
changed for that retry. The original setup failure is retained in its
[job
log](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122901243926);
the successful replacement is [job
122910350074](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122910350074).

## Risks

- Missed supported producer signals could delay expiry until restart.
Real insertion, cancellation, consent transitions, caller-owned
transactions and Board publication are covered.
- Corrupt or locked rows must retain retry without starving healthy
work. Paging, fairness, atomic rollback, redaction, drain/resume and
shutdown tests cover these paths.
- Independent SQL writers and other processes need their own
notification contract. No persistent DB listener is added.
- Attempt recovery still scans periodically until the next slice. A
local deadline is not a durable Cloud alarm. This PR does not claim
whole-instance sleep.
- This slice is reconciled with its landed parent; current-head remote
checks/review and live-master composition proof still gate merge. Later
stack duties remain separate PRs.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, tool use and
code execution. The exact serving model ID and context-window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 10:32:30 -05:00
DottaandPaperclip 8049fbf51d fix(server): schedule voice call reports from saved deadlines (#17586)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Voice calls retain reports and project guest transcripts into intake
tasks.
> - Chat maintenance repeatedly scans recent calls even when no report
is due.
> - Empty scans keep the tenant database active.
> - Existing session and report rows already contain the work and its
deadlines.
> - This pull request moves report refresh to the shared coordinator and
removes the final voice maintenance lane.
> - Idle voice configuration and old call history no longer cause report
scans.

## Linked Issues or Issue Description

Refs #17583, #17580, #17578, #17576. This is slice 25 of the
recurring-server sleep stack. It now follows merged #17583 and targets
master.

**What existing behavior does this improve?**

Voice call-report refresh and guest transcript projection.

**Current behavior**

The chat interval scans terminal calls for reports. It repeats the scan
even when there are no eligible calls or all saved deadlines are in the
future.

**Proposed behavior**

Committed session transitions and signed callbacks wake a scoped worker.
Startup reconstructs existing rows. Later refreshes use saved deadlines.
The worker becomes quiet after all eligible windows expire.

**Reason and benefit**

Stop empty database scans without losing late transcript and billing
updates. Keep the existing 24-hour refresh policy, provider response
ordering, and guest-task trust restrictions.

## What Changed

- Add a voice-owned report worker using the existing scoped deadline
helper. No queue table or field is added; the existing session table
receives a report-window index.
- Save each attempt and its retry before provider I/O. Compare that
ownership on settlement so an older response cannot overwrite a later
callback or claim.
- Preserve available reports when a response is incomplete or stale.
Preserve their finite refresh window.
- Route voice-change and reply call-ended writes through
transaction-aware report signals.
- Preserve guest transcript locks and use the observed document revision
when updating an existing transcript.
- Register the worker in app startup. Remove voice reconciliation from
the chat interval. Keep explicit one-shot diagnostics.
- Document producer scope, lifecycle behavior, tests, and external wake
limits.

## Verification

Published head: `758cfe15c3e6283d38ba512c8416ccce4d200862`. The
follow-up after `c28de398` changes only two documentation files; the
verified production, migration and test source is unchanged. It normally
merges landed retention/master
`980775e0e956eecdc28c73c9b4aca0407ad45e1a`. The 20 owned paths remain
the complete slice, including officially regenerated migration 0397, its
snapshot and required pruning of snapshot 0392. The non-database
reviewed patch was preserved during ancestry integration; the only
subsequent source edit makes one existing report test’s provider
responses call-specific. All production report behavior and unrelated
master changes are preserved. No shared worker, workflow or lockfile
change belongs to reports. All 58 current-head checks are now clean,
including the previously failing approval shard; exact-head Greptile is
5/5 with no new actionable issues. Ponytail’s current turn stopped
without a verdict, an authorized nonblocking state. The prior
conditional documentation requirement was fixed; old CI failures remain
recorded below.

- **Prior-head evidence at `9bf20467df`: 303 distinct targeted cases
passed**, not rerun wholesale for this integration: all 132 voice
integration cases; 170 cases across 11 complete report, retention,
cleanup, recovery, company-removal, shared-worker and startup/hostname
suites; and one complete snapshot/schema drift case. This includes 15
report, 13 retention and 14 cleanup lifecycle cases. Voice integration
and the four other database owners use separate initially empty
databases and migrate themselves. Report cases use their dedicated
database after the setup repair.
- Five real connection saves over 205 sessions with saved future report
deadlines selected 1,025 session rows before the notification repair.
They now select zero and preserve the original deadline. The report
notification helper accepts only exact session scopes. Configuration,
authority and connection hints retain their existing live
recovery/reply/cleanup behavior but do not rebuild report history. Real
session terminal writes, callback hints, report claims and receipts
still reactivate reports.
- Database/server typechecks and builds, migration numbering/safety, and
module-boundary checks pass. Final server types/build also pass after
integrating the retention review repairs. The parent 205-admission,
five-configuration-save test still observes zero retained-history rows
and preserves its deadline.
- The old eager reader selected all 105 saved-report headers before
yielding and failed the 100-row budget. Registered reconstruction now
uses the existing paged contract and yields at 100 rows per scope
refresh. A partial scan can restart on the same owner and reconstruct
every saved deadline. At due time all 105 report IDs are fetched exactly
once. Future deadlines remain quiet between actual retries.
- The company-fairness case places a small company deterministically on
page two and requires it in the next 25-work batch after discovery, with
all 105 reports processed before the failed provider's retry. Guarded
mutations catch a missing cursor reset and bypassed company rotation.
Original source was restored before final acceptance.
- At prior head `9bf20467df`, migration 0396 was generated with `pnpm
db:generate` on the parent 0395 lineage. It creates the partial
`(ended_at, id)` report-window index concurrently and replayably through
the existing migration runner. All 394 historical SQL files and the
journal prefix are unchanged; snapshots 0392 through 0396 remain. Fresh
application migration and two replay invocations succeed with a valid
index. The real plan case retains 20,001 old sessions, 205 recent
sessions, precision-safe keysets, and fewer than 1,000 rows removed by
filtering on each page.
- Existing real hangup/callback/reply-error producers, callback
ownership fencing, monotonic report results, saved retry before provider
contact, guest transcript projection after a document lock, transaction
rollback/nesting, drain and shutdown remain covered. No table, field or
provider protocol is added. Available reports retain the existing finite
24-hour refresh policy.
- [Current-parent acceptance and failure
ledger](https://github.com/paperclipai/paperclip/blob/9bf20467dfd50b81848e1c8dd7ddc905d206418f/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#current-parent-reconstruction-and-migration-acceptance-october-28)
records the failed fresh fixture, eager read, fairness-clock attempts,
guards, commands, counts and limits. Older 174/138-case and
workspace-wide results belong to their stated heads. No new complete
local repository suite, live provider, Cloud wake, host/database
suspension or savings claim is made.
- [Report configuration-scope
repair](https://github.com/paperclipai/paperclip/blob/9bf20467dfd50b81848e1c8dd7ddc905d206418f/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#configuration-scope-review-repair-october-28)
records the current 303 cases and measured rescan repair. The prior
`80207ba6bb` head had Ponytail 5/5; that review is stale after this
change. Fresh exact-head Greptile 5/5 and all CI remain required. Actual
failing Ponytail findings must be fixed; a stopped or paused turn
without a verdict is an authorized nonblocker. Active no-verdict review
waits are capped at 30 minutes from the original request/start
(25-minute monitor cutoff). Older scores and parent checks are not
inferred as current passes.

- **Current-master verification: 185 distinct cases pass**: final
complete voice integration 132, complete report/cleanup/retention
15+24+13, and snapshot/schema drift 1. Each database owner used its own
empty fixture database. The first combined run had 183 passing cases and
one report-test failure; it is preserved in the audit. A deliberately
earlier-ended competing call reproduces the old one-shot mock consuming
the intended response. The corrected test selects responses by provider
session ID, verifies independent costs for both calls, then rejects
stale target data. The isolated repair and full 132-case suite pass;
production behavior and timeouts are unchanged. These 185 cases are not
added to the older 303 count.
- **Current migration lineage:** official `pnpm db:generate` after
actual master creates `0397_known_madrox.sql`; the reviewed
concurrent/idempotent report-index SQL is retained byte-for-byte. All
395 existing SQL files and journal entries are unchanged; snapshots
0393–0397 remain and the new snapshot links parent 0396. The application
migrator boots a fresh database, repairs a deliberately invalid index
during two concurrent replay invocations, records one history row, and
replays again successfully. Database build, server build/typecheck,
migration checks and module boundaries pass. [Full current-master
acceptance and failure
ledger](https://github.com/paperclipai/paperclip/blob/c28de398fef074f650eff532952ab6f5563739a8/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#landed-master-migration-and-acceptance-october-28).

- **Review follow-up and preserved CI timeout:** Ponytail’s prior
`c28de398` review was 4/5 advisory, with no must-fix production defect
and a request to fix the stale design-doc migration reference before
shipping. That document now names migration 0397 and links current
185-case acceptance; older 302/303-case evidence is explicitly
historical. The optional suggestions about one-shot diagnostic inventory
cost and inherited test-clock helpers remain disclosed. CI run
`40703490340` failed the unchanged approval-queue backoff test on its
existing 15-second timeout and then an activity-log FK in teardown
(2,097 pass/one fail/one skip in that shard). The complete unchanged
28-case approval suite passes on a new isolated database; the exact CI
timing cause remains unproven. No timeout, workflow or production logic
was changed, and no old-head rerun or failure waiver is claimed. This
documentation head requires fresh CI and exact-head Greptile. [Review
and timeout
ledger](https://github.com/paperclipai/paperclip/blob/758cfe15c3e6283d38ba512c8416ccce4d200862/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#review-documentation-and-preserved-remote-timeout-october-28).

## Risks

- Process-local commit signals do not observe independent SQL writers or
another server process.
- Available reports still refresh for 24 hours. The provider's complete
flag does not promise immutable final data.
- Transcript projection retries remain within the existing report
window. This change does not add indefinite projection repair.
- One-shot diagnostic `reconcile(limit)` inventories the eligible window
before slicing; it remains a manual diagnostic cost, not recurring idle
work. Some inherited test helpers advance fake time during assertion
retries, an optional reliability follow-up noted by review.
- A local deadline cannot wake a stopped host. Cloud alarms and sleep
eligibility remain unchanged. No deployed sleep or cost saving is
claimed.
- This PR depends on the preceding stack. Final master reconciliation
and remote review gates remain required.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, code editing,
and local test execution. The exact serving model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 09:41:15 -05:00
DottaandPaperclip 980775e0e9 fix(server): schedule inbound voice history retention by deadline (#17583)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Incoming voice calls leave history after provider cleanup is
confirmed.
> - The chat maintenance tick repeatedly deletes old admission history.
> - Each eligible row already contains the dates needed for its
retention deadline.
> - This pull request schedules that deadline and wakes on committed
changes.
> - Idle retention stops querying while the existing seven-day policy
remains enforced.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Inbound voice history retention. This PR follows merged #17580, which
owns physical call cleanup, and now targets master. Call reports remain
a separate migration.

**Current behavior**

Voice maintenance scans admission history repeatedly. It deletes
confirmed, terminal, unbound admissions when both expiry and last update
are at least seven days old.

**Proposed behavior**

Load existing retention deadlines at startup. Wake from the existing
admission transaction notifications. Delete at the later timestamp plus
seven days. Lock and recheck current eligibility before deletion.

**Reason and benefit**

Configured idle endpoints and future history do not cause recurring
retention SQL. Failed deletion retains a retry. Restart reconstructs the
same saved deadlines.

## What Changed

- Add a voice-owned retention worker using the existing scoped deadline
helper and coordinator. Yield after 100 history rows per scope refresh
and reset reconstruction on registration. Keep complete one-shot
inventory reads. No table, field, or migration is added.
- Extend `notifyVoiceInbound` to notify retention before the same
committed writes that already notify cleanup. Callers need no new
recovery wrapper.
- Preserve confirmation, terminal-state, session, task, and seven-day
eligibility conditions. Round PostgreSQL submillisecond deadlines up.
- Lock and recheck the current row before deletion. New bindings or
moved dates invalidate stale candidates.
- Remove history pruning from recurring voice maintenance. Keep the
explicit one-shot diagnostic path. Leave call reports for their own
slice.
- Document producer ownership, retry/drain behavior, tests, and external
wake limitations.

## Verification

Published head: `53f2fafcb64d104887eb4455c292b48e6dd10253`. It includes
the normal merge of actual landed master
`71b448b9cb841e6fa18fe883585430c5081245da` after cleanup #17580. That
ancestry integration at `768c650365` preserved the complete binary diff
of all 13 reviewed retention paths. This follow-up changes only the
existing cleanup fairness test and its failure ledger, bringing the
final scope to 14 paths; all production source is unchanged from that
integration. All unrelated master changes, cleanup repair, migration
`0396_tan_paladin.sql`, and gateway clock repair are preserved. Database
files, shared worker, workflows and lockfile match actual master. The
earlier 282-case/type/build evidence below belongs to unchanged
retention source at its stated head; current repair acceptance is 37
distinct complete cleanup and retention lifecycle cases. All 58
current-head checks are now clean, including the Canary Dry Run;
Greptile is exact-head 5/5 with no new actionable issues. The current
Ponytail turn stopped without a verdict, which is an authorized
nonblocking state. The earlier failing CI remains preserved below.

- **282 distinct targeted cases pass**: all 127 voice integration cases
and all 155 cases in ten complete retention, cleanup, recovery,
company-removal, shared-worker, startup and hostname suites. These
include 13 retention and 14 cleanup lifecycle cases. Voice integration,
parent recovery and company removal start on separate empty databases
and migrate themselves. Retention uses its dedicated database after the
setup repair; cleanup uses its already-migrated dedicated database.
- Ponytail's prior exact-head 4/5 review found configuration-driven
history rescans. Five real connection saves selected 1,025 rows over 205
future admissions before repair. Retention now receives only admission
hints and startup reconstruction; the same saves select zero admission
rows and retain the original deadline. Cleanup still receives its full
configuration and authority scopes. Restrictive binding foreign keys
prevent implicit unbinding outside the admission-write boundary.
- Successful deletion now returns its confirmed result without reloading
the deleted row. A real SQL-count case and a guarded negative probe
cover that path. Changed or not-yet-due history still reloads current
eligibility and its deadline.
- Server typecheck, build and module-boundary checks pass. No schema,
driver, shared scheduler contract or provider protocol changes are added
by this slice.
- A real 105-row future-history case failed before the reader repair
because startup selected 105 rows rather than yielding at 100. The
existing paged shared-worker contract now bounds each scope refresh. The
same-owner restart case reconstructs every deadline and deletes all 105
eligible rows at the original deadline. It verifies SQL quiet before
that deadline and after settlement. Removing only cursor reset fails at
five rows instead of 100 after restart; guarded source was restored
before final acceptance.
- Existing cases retain the later-of-two-timestamps seven-day policy,
submillisecond rounding, current-state confirmation and binding checks,
concurrent binding lock, rollback/outer commit, first write during
drain, failure retry and shutdown. Real signed ingress, cleanup
confirmation and maintenance-lane removal remain covered by the complete
voice suite.
- The fresh retention fixture first failed on missing schema and now
applies the application migrator. The initial parent merge duplicated a
test import and failed to parse before any voice case ran. Both failures
are retained in the audit; the corrected complete suites pass.
- Prior head `daca270a99` had a failed server (4/12) check, run
40557790710/job 122386307340. Its actual log reports a runner shutdown
and cancellation before completion. This is incomplete acceptance, not a
pass or a proven product flake. New-head CI must pass independently.
- [Current-parent acceptance and failure
ledger](https://github.com/paperclipai/paperclip/blob/68b62b6d13f28ac5f17f35b98adf5488752f9b97/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#current-parent-paging-and-restart-acceptance-october-28)
records the exact scope, commands, counts and preserved failures.
Earlier 218/133/134-case and full-workspace results belong to their
stated heads. No new full local repository suite, live provider, Cloud
wake, host/database suspension or savings claim is made.
- [Notification review repair and current 282-case
evidence](https://github.com/paperclipai/paperclip/blob/68b62b6d13f28ac5f17f35b98adf5488752f9b97/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#configuration-rescan-review-repair-october-28)
records the new source. The prior head had green CI and a 4/5 Ponytail
review; neither is accepted for this new commit. Fresh exact-head
Greptile 5/5 and all CI remain required. Completed actual failing
Ponytail findings must be repaired; a stopped or paused turn without a
verdict is not a merge blocker under the user’s authorization. An active
Ponytail review waits at most 30 minutes from its original request
(25-minute monitor cutoff).

- **Preserved current-master CI failures:** run `40697141057` failed
server shard 7/12 in cleanup’s fairness test and Canary Dry Run on an
npm `rollup-4.66.0.tgz` 404. A fixed final admission UUID reproduces the
same fairness assertion: the second company was sometimes beyond the
first 100-row page, so expecting it among the first 25 calls was
invalid. The corrected test controls time and proves service within the
next 25-call batch after discovery, all 105 successful calls before the
failed provider’s saved retry, exactly one retry at its deadline, and
idle SQL after settlement. Bypassing company rotation fails the
corrected assertion; the shared helper was restored byte-for-byte. Both
complete cleanup (24) and retention (13) suites pass after restoration.
An intermediate 23/24 run incorrectly demanded idle before
reconstruction finished; the audit preserves that failure and its
corrected pending-intent wait. No production behavior, timeouts,
dependencies or workflow gates changed. The exact npm metadata and
tarball now return 200; new-head canary CI must pass independently.
Prior Greptile/Ponytail 5/5 on `768c650365` are stale for this new head.
[Full CI repair
ledger](https://github.com/paperclipai/paperclip/blob/53f2fafcb64d104887eb4455c292b48e6dd10253/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#current-master-ci-repair-october-28).

## Risks

- Supported admission writers must use the existing transaction
notification boundary. Independent SQL writers and additional processes
need a separate wake contract.
- Retention never deletes unconfirmed cleanup or session/task-bound
history. Those records remain with their owners.
- The in-process deadline does not wake stopped compute. This PR does
not extend Cloud alarms or relax sleep eligibility, and does not claim
deployed host/database suspension.
- Call-report reconciliation and other chat maintenance still need
separate migrations.

## Model Used

OpenAI Codex with reasoning, shell, and code-editing tools. The exact
model version and context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green for this new head
- [x] Greptile is 5/5 on this new head with no unresolved actionable
findings
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 08:57:58 -05:00
DottaandPaperclip 71b448b9cb fix(server): schedule inbound voice cleanup from durable obligations (#17580)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Incoming voice calls need approval, expiry, and safe provider
cleanup.
> - The remaining voice maintenance tick scans admissions every second,
including idle endpoints.
> - Admission status does not prove that the provider call has ended.
> - Existing admission rows can retain the cleanup owner, confirmation,
and retry deadline.
> - This pull request replaces the cleanup scan with committed changes
and actual deadlines.
> - Idle cleanup stops querying while unfinished calls keep a
recoverable path to termination.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Inbound voice admission cleanup. Its reply-delivery parent #17578 has
landed; this PR targets master. Live-session recovery is in #17576. It
preserves separate ownership for those services.

**Current behavior**

The voice maintenance tick scans expired admissions even when no cleanup
is due. A session can commit before the separate approved-admission
write. Expiry can then terminate that authorized call. A rejected guest
call can be labeled ended before provider termination succeeds.
Seven-day retention can delete that unfinished cleanup.

**Proposed behavior**

Wake cleanup from committed admission and session changes. Schedule the
original expiry or saved retry. Claim current state under the admission
lock before provider I/O. Commit approval with session reservation. Keep
unconfirmed termination durable and transfer bound calls to live-session
recovery.

**Reason and benefit**

Configured idle endpoints and confirmed history cause no recurring
admission cleanup SQL or provider requests. Approval, retries, crash
recovery, and cleanup remain supported. History retention and call
reports stay separate migration items.

## What Changed

- Recover terminal and rejected legacy admissions whose session binding
was lost. Copy known termination or bind a proven matching live session
for cleanup without granting approval.
- Register inbound cleanup with the shared coordinator and scoped
deadline worker. Yield after each 100-row scope page and reset cursors
on registration. Remove only cleanup from periodic voice maintenance.
Keep complete manual reconciliation.
- Add commit-aware notifications at admission, approval, rejection,
guest binding, pending-tool expiry, lifecycle confirmation, and session
termination writes.
- Commit authorized session binding and admission approval together.
Recheck cleanup ownership under the same admission lock. Recover older
interrupted bindings only from matching session evidence.
- Add nullable cleanup confirmation, next-attempt, and provider-account
fields to the existing admission table. No new queue table is added.
Generated migration 0396 backfills only provable historical termination
in committed 1,000-row keyset batches and adds a concurrent partial
index.
- Save the retry before provider contact. An unconfirmed end receipt
retains cleanup. Bound calls remain owned by live-session recovery.
- Verify the signed provider account before using a rotated key for
unbound cleanup. A different account cannot inspect or end the saved
call. Legacy admissions require their original credential fingerprint.
- Prevent history pruning and company deletion from erasing unfinished
cleanup. Company deletion locks endpoints before the voice owner's
check, matching ingress serialization.
- Document producer coverage, ownership, deadlines, verification, and
remaining sleep limits.

## Verification

Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply
parent #17578 has landed. A normal merge integrates actual master
`e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated
changes, and regenerates this slice’s migration after landed migration
0395. Fresh exact-head reviews and CI are required.

- The previous Ponytail 3/5 finding is repaired: legacy terminal
admissions no longer enter approval-only repair, and rejected admissions
with valid matching live sessions retain their rejection and persist the
binding needed for session-scoped cleanup confirmation. Five regressions
failed with ownership conflicts before the repair. Five
conflicting-session regressions prevent an unrelated leg from being
claimed. No cleanup operation grants approval or starts a provider call.
- **24 complete cleanup lifecycle cases and 122 complete voice-store
integration cases pass** on the repair. After actual-master integration
and migration regeneration, the complete 24-case cleanup suite passes
again on a new empty PostgreSQL 17 database, and the database migration
integration test passes separately. The cleanup owner, voice-store test
and admission schema are byte-identical to that tested repair; the
cleanup fixture only changes the migration filename. Migration
numbering/safety, database build, production server typecheck and build
pass.
- Migration `0396_tan_paladin.sql` was generated with `pnpm
db:generate`. Every landed SQL migration and the complete parent journal
prefix are unchanged. Generated snapshots retain exactly 0392–0396,
pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and
committed 1,000-row historical backfill are retained byte for byte under
the new generated filename, registered for top-level execution. The real
application migrator boots a fresh database, repairs an interrupted
concurrent index, serializes replays and verifies eligible-history
backfill across actual transactions.
- The preceding published head
`48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but
failed CI server shard 9/12: one existing connected MCP rate-limit test
expected two real-clock calls to share a fixed 60-second bucket. A
controlled window rollover reproduced its exact failure. That existing
fixture now freezes only `Date` around the same-window 429 assertion,
verifies acceptance after advancing one window, and restores the real
clock in `finally`. All 92 gateway acceptance cases pass. Production
rate limiting, timeouts and assertions are unchanged. This adds one
existing CI fixture to the slice; the original failed run remains
recorded.
- The first regression invocation skipped the suite because it used the
wrong database variable; that is not a pass. The corrected invocation
reproduced five ownership conflicts. The first fixed cleanup run passed
22 and failed two new low-level deletion test calls on a
retained-session foreign key; the regressions now check the cleanup
deletion guard at its owning boundary, with production deletion ordering
unchanged. The final full suite passes all 24.
- The first complete voice-store invocation passed 121 and timed out on
one transcript case at the unchanged 15-second limit. That case passed
in isolation on a fresh database. One subsequent complete invocation,
with builds finished, passed all 122 without source, assertion or
timeout changes. The underlying timeout cause is unproven; the original
failure remains recorded. A new PostgreSQL startup also failed from
exhausted host semaphore capacity; fresh databases on the already-owned
server were used without removing shared IPC resources.
- Prior source `b8b12199b9` passed 266 distinct cases and carried three
guarded mutation proofs; those older results are not added to this
repair’s case count. Its CI run 40641168596 failed E2E shard 3 when
legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub
initially rejected a retry while the run was active, then accepted one
targeted retry after completion; attempt 2 passed on that same old head.
The lock holder and underlying cause remain unproven. The unchanged Node
policy checker’s older failure on ignored generated Copilot manifests
and passing tracked-input mirror are also retained in the audit; no
workflow, checker, vendor manifest or ignore rule was modified.
- [Current repair and failure
record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28).
Current-head CI and Greptile 5/5 gate merge. A completed failing
Ponytail verdict must be fixed; missing or interrupted reviews do not
block, and active pending review waits are capped at 30 minutes under
the maintainer’s explicit instruction. No new full local repository
suite, real provider, Cloud alarm, host/database suspension or savings
claim is made.

## Risks

- The migration distinguishes admission status from physical
termination. Legacy ended guest rejections without proof remain cleanup
obligations. Missing original credentials can keep those old obligations
unresolved.
- Future writers must use the existing notification boundary.
Independent SQL writers and multiple processes need their own wake
contract.
- Pending inbound cleanup now blocks company deletion instead of
allowing cascade deletion to lose the provider obligation. Existing
voice-history deletion restrictions remain. This deliberately includes
an active anonymous inbound ring and an old call whose termination
cannot be verified. An operator must stop ingress and settle the call
before deletion; this slice adds no abandonment override and never
treats missing credentials as proof of termination. This is the retained
cleanup ownership policy, not a claim that every deletion attempt can
complete immediately.
- History retention and report processing still run in the remaining
maintenance lane. This PR does not extend Cloud wake coverage or claim
measured host/database suspension.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 08:30:15 -05:00