Commit Graph
2669 Commits
Author SHA1 Message Date
DottaandPaperclip 8373d78679 fix(recovery): register missing native preservation on Retry (#17800)
## Thinking Path

> - Paperclip lets people manage agents and their saved work.
> - Native runs can leave a prepared remote workspace after a control
deadline.
> - The deadline repair in #17788 records preservation for future
failures.
> - Older failed runs can still have an active lease without a
preservation owner.
> - Retry correctly refuses another executor, but nothing can finish the
missing preservation.
> - This pull request registers the exact prepared source through the
existing recovery services when the user retries.
> - The bounded maintenance worker can then stop the provider and save
its files before normal admission continues.

## Linked Issues or Issue Description

**What happened?**

A historical failed native run has a prepared workspace and an active
retained allocation. It has no workspace preservation record. Ordinary
Retry cannot prove that the old executor stopped and has no maintenance
owner to produce that proof.

**Expected behavior**

A deliberate Retry registers missing preservation for the exact unowned
source. Existing recovery proves termination and saves the files.
Existing admission then checks current user authority, effects, pause
and budget gates before one successor can enter the queue.

**Steps to reproduce**

1. Persist a failed native run with a prepared workspace, an active
lease and a terminal, unowned finalization.
2. Omit the workspace preservation record, as an older deadline handler
did.
3. Call the normal task Retry route. Before this fix, preservation
remains absent and Retry cannot continue.

**Paperclip version or commit**

The regression fails on `46de15ff4b99bbee59c38f32f4f30b80ad5bbd24`,
which already includes #17788.

Related: Refs #17788 (future deadline failures), #17741 (committed
recovery wakes), #17733 (connected Retry authority). #17773 adds source
diagnostics and does not repair this missing registration.

## What Changed

- Use native workspace preservation inside the existing issue-locked
Retry transaction. A live finalization owner remains held.
- Adopt a valid unowned preservation obligation through the existing
bounded maintenance service. Bind it to the deliberate Retry intent
without resetting an active generation.
- Add real PostgreSQL regressions for exact, mismatched and owned
sources. Verify that registration cannot create a termination receipt or
a successor.
- Exercise the HTTP route, worker, real temporary file merge and queue
admission with the stop reply held. Duplicate requests admit one queued
turn after preservation.
- Extend connected GitHub and Telegram Retry coverage for missing
historical preservation. Document the contract and access boundaries.

## Verification

- Red: the new historical-state regression fails on the base commit
because `workspaceRestoreRecovery` remains absent. The mismatched and
owned negative controls pass.
- Green: the three focused historical-state cases pass with the fix
against isolated local PostgreSQL 17.
- The local HTTP journey restores saved files and retains newer host
edits. It admits no successor while the provider stop reply is held. It
admits one successor after verified preservation. The real GitHub and
Telegram admission paths wait behind that recovery.
- `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs` and
`git diff --check` pass.
- Green: 152 tests across four relevant suites, including all connected
Retry safety cases. The six new cases pass again on final head
`b8900801a3181fa673ea45f1129f968bbb08d1fd`.
- Final-head GitHub checks: 55 successful, two intentionally skipped
Storybook checks, and one neutral Paperclip bot review. Greptile is 5/5
with no unresolved threads. The Paperclip bot stopped before finishing
and supplied no approval.
- The full local command imported an earlier fixture revision before its
assertions were corrected. It reported those two stale connected-fixture
assertions and later hit the host PostgreSQL semaphore limit again. It
was stopped after final-head CI completed. It is not counted as a full
local pass; the complete CI test lanes passed on the final head.
- The first local rerun hit the Mac PostgreSQL semaphore limit before
assertions. Verification moved to an isolated disposable PostgreSQL
container. Early assembled fixtures also exposed the existing
destruction-cleanup gate; the retained-workspace fixture now uses its
actual `keep_running` disposition. This fix does not weaken cleanup.

## Risks

- Retry now registers maintenance for historical native failures. Exact
descriptor, allocation and coordinator checks remain owned by the
existing preservation service. Physical recovery still requires its
existing exclusive owner and provider stop receipt. Registration alone
grants no execution authority.
- No database migration or provider protocol changes. No model, trust,
assignment, retry-budget or external identity changes.
- Local recovery uses real database, filesystem, HTTP, worker and
admission code. Only the provider lifecycle boundary is simulated. No
hosted allocation or live model turn has been verified for this new
change.
- The workspace recovery test file crosses 1,000 lines because this case
shares its real database, filesystem and assembled route fixture.
Keeping it there avoids duplicating that fixture or extracting unrelated
production behavior.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools and code
execution. The exact model identifier and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 16:22:36 -05:00
DottaandPaperclip 2ee36a4c0a test: add chat feature scenarios and real task workflow evals (#17794)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat lets a person assign work through a conversation.
> - Feature and API maps do not prove that a natural request produces
the right saved result.
> - Product E2E already owns isolated fixtures, browser runs, evidence
and cleanup.
> - This pull request adds a chat scenario catalog and two real task
workflows in that framework.
> - Independent checks verify the task, its author, the saved output and
the reply in chat.
> - The benefit is a small executable foundation for broader chat
coverage.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

It adds real task creation and delegation coverage for Agent Chat.

**Current behavior**

Existing maps describe features and agent access. Existing chat evals
cover specific protocols and handoffs. They do not connect a broad
human-request catalog to independent saved-state checks.

**Proposed behavior**

Keep 244 planned scenarios across 35 feature areas. Promote two requests
into the explicit-only `chat-features` suite. One saves a backlog task
without starting work. The other delegates an announcement and returns
the saved result to the original chat.

**Reason and benefit**

A successful tool call or confident reply does not prove the requested
result exists. These evals check public API state, attribution and
browser access after a real model run.

**Breaking changes**

No API or data schema changes. Chat instructions now require actual
relative task links. The local E2E private-state scanner recognizes
Codex account ownership metadata while continuing to reject credentials
and model-visible identity leaks.

**Additional context**

Related prior work: #13969 and #13762. This improves verification of the
existing Agent Chat roadmap item. It does not add a new core product
feature.

## What Changed

- Add YAML scenarios with capability IDs, starting context, user
requests and expected outcomes. Validate all area files against their
schema, global ID uniqueness and exact feature-map capability coverage.
- Add two local native Codex subscription cells. Send the exact catalog
request once through Chromium.
- Seed only prerequisites through public APIs. Use shipped agent
instructions and explicit disposable budget policies for unpriced
subscription usage.
- Check task ownership, creation audit, no premature execution, current
worker-authored output, automatic completion delivery and usable links
after reload.
- Keep private Codex account ownership metadata distinct from bearer
credentials. Scan rollout message content, all tokens and published
evidence as before.
- Teach normal chat instructions to use relative task links instead of
invented hosts. Update tests, maps and the runbook.
- Repair existing E2E contract drift: include the active Copilot
identity in the image digest and update Pi profile expectations. Keep
global catalog cardinality in its owning catalog test.

Product E2E owns the new flow and grader. Its existing launch and report
paths remain in use. The shared scanner change is limited to
owner-private temporary Codex runtime metadata. Other callers retain the
full secret scan. No UI, endpoint, schema or permission behavior
changes.

## Verification

- `node scripts/check-maps.mjs` passed: 35 areas and 191 capability IDs.
- Product E2E typecheck and focused tests passed before rebase: 225
Vitest tests and 137 Node tests. Conversation service tests passed
40/40. Five existing runner turn-binding regressions passed.
- Historical live measurement used real Chromium, server, database and
native Codex with no automatic retries. The initial same-definition
batch passed 5/6 cells. Backlog passed 3/3; delegation passed 2/3. One
CEO startup failed because the runner did not receive a provider turn
identity. The cause is unresolved.
- An unchanged follow-up passed delegation three times and backlog
twice. Combined evidence is backlog 5/5 and delegation 5/6. Each attempt
passed cleanup. Later passes do not erase the earlier failure or prove a
runner repair.
- Earlier failed revisions exposed an invented task hostname, private
account metadata false positives and missing funding for unpriced
subscription usage. Those led to the specific changes above; the failed
records remain retained locally.
- The runbook records campaign identifiers and definition provenance.
Generated result artifacts are not committed. Historical live results
predate this branch rebase; fresh PR-source live evidence and final
validation are recorded below.
- Full workspace typecheck and production build passed on the rebased
branch. Product E2E typecheck passed.
- The full Product E2E support suite first reproduced four baseline
failures: two missing active Copilot image-identity inputs and two stale
Pi/catalog expectations. The narrow contract fixes above resolved them.
The full rerun passed 4,282 Vitest tests plus 137 Node tests, with two
skipped. Image checks verify mutation changes the hash, unchanged input
stays stable, and an input closure without Copilot is unaffected.
Original failed logs remain retained.
- The repo-wide local Vitest run has reported two failures in unchanged
service files. The Workspace connection test passed an isolated recheck
and its full file passed 469/469 unchanged. The recovery test hit a
15-second timeout; its full file passed 89/89 unchanged. The run later
hit explicit PostgreSQL `semget` resource exhaustion and was interrupted
after those observed failures; no final aggregate counts or full local
pass are claimed. Its complete available log is retained. No assertion
or timeout was weakened. Matching remote server CI passed on the same
source.
- On clean PR head `39ad80b523ce194c90aafafd077d48dd9a36b4c5`, backlog
passed in `local-2026-10-26T19-24-13-014Z`. The parallel delegation
fixture failed during PostgreSQL initdb, before browser or model
execution; its failed cleanup grade is retained. A disposable standalone
initdb probe then passed. No semaphore sets or kernel settings were
changed before that probe/recheck. After confirming no remaining fixture
process, its private temporary files were removed.
- One separate delegation recheck passed in
`local-2026-10-26T19-29-24-800Z`, with one attempt, zero automatic
retries and successful cleanup. Both passing current-source cells use
definition
`3f2c9ba2d03e3dcddde47ddab3b43bc52de977f25e256fceae0a24c9128e204b`. The
earlier bootstrap failure and the historical provider startup failure
remain distinct. These local results omit automatic source-SHA metadata;
the clean checkout SHA is recorded here from the execution session.
- The later broad local run reported `could not create semaphores: No
space left on device`. A bounded cleanup removed 32 current-user
PostgreSQL sets with 17 members, marker 537, a dead recorded creator and
zero waiters. No live process or kernel limit was changed by that
cleanup. The test process group was subsequently stopped; no broad rerun
is planned. This direct later resource failure does not prove the
precise cause of the earlier live initdb failure.
- Review follow-up adds only static catalog validation and documentation
of the existing five-minute journey bound. The executable definition is
unchanged. Focused catalog/oracle tests passed 28/28; no redundant live
run is claimed.
- Final head `30fcb36ac9ae1f0270a589dbfafcec91ca6a03f3` passed all
required and underlying CI checks in [run
40391402347](https://github.com/paperclipai/paperclip/actions/runs/40391402347),
including `ci / verify` and `ci / e2e`. Greptile and Paperclip Review
both scored this head 5/5, with all review threads resolved. The static
follow-up leaves every executable definition input byte-identical to the
live-tested source at `39ad80b`.

Normal squash auto-merge is enabled. Actual merge is pending eligible
code-owner approval for the owned dependency manifest; no protection is
bypassed.

## Risks

- Live model behavior can vary. One missing turn-start event remains
unexplained; this PR does not claim provider reliability or broad
catalog qualification.
- Subscription costs are unpriced. Explicit fixture funding admits that
usage. Run counts and deadlines remain bounded, but the dollar limit
cannot cap unknown cost.
- Account identity exceptions apply only to known private Codex runtime
files. Token reuse, unknown files and model-visible rollout content
remain blocking findings.
- Deterministic checks establish saved state, output structure, required
brief facts and access. They do not score all prose quality. Initial
samples also require human inspection.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, code execution, browser and
repository tools. The precise model revision and context window are not
exposed to this session. Live eval agents used `gpt-5.6-sol`; that is
the evaluated model, not a claim about the authoring model.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run local tests; focused suites pass and the interrupted
full-run failures are documented above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 16:19:39 -05:00
Nicky LeachandPaperclip 4ff493fb40 feat(plugins): include managed skills for selected environments (#17774)
## Thinking Path

> - Paperclip manages agents and their execution environments.
> - Plugins can vendor company skills and provide environments.
> - Environment tools need their instructions before the provider
starts.
> - Persistent computers also need those skills before any run begins.
> - This change adds environment requirements to normal skill selection
and lifecycle synchronization.
> - Plugin names prevent ambiguity, and required skills take delivery
precedence without deleting company data.

## Linked Issues or Issue Description

**Problem or motivation**

An environment can provide desktop tools while its agent receives no
instructions for those tools. A first-run-only import also leaves idle
computers without required skills. Same-name company skills can make
provider discovery ambiguous.

**Proposed solution**

Let plugin environment drivers declare their managed `skillKeys`.
Install missing requirements during lifecycle reconciliation or new-run
preparation. Deliver updates through the durable runtime event journal.
Give managed imports stable plugin names and report shadowed
assignments.

**Alternatives considered**

ROM links do not make a skill a normal Paperclip selection. Polling
duplicates the existing lifecycle delivery queue. Replacing a same-name
company record would destroy independent data and assignments.

**Roadmap alignment**

This extends the shipped Plugin system and Skills Manager. The existing
related-PR search found no duplicate contract.

## What Changed

- Validate `environmentDrivers[].skillKeys` as unique references to the
same plugin's managed skills. Require a ready, enabled plugin with
`skills.managed` authority.
- Resolve company-scoped skills before version capture. Preserve
explicit pins, company edits, and saved recovery selections. Serialize
concurrent first installs.
- Allow lifecycle callers to request missing-skill installation through
`agents.getRuntime`. Ordinary task and recovery reads do not install new
declarations. Pass task-selected environment IDs through the SDK.
- Add SQL migration `0378_environment_skill_runtime_hooks`. Required
skill edits, effective default changes, environment changes, and plugin
requirement changes enqueue durable runtime events. Reuse lifecycle
delivery and retries; add no polling.
- Import managed defaults as `plugin-<short-plugin-id>-<skill-slug>`,
including the SKILL.md name. Preserve canonical keys. Keep same-name
company records separate without counter suffixes.
- Required environment skills win delivery over same-name company
assignments. Preserve saved assignments and library content. Record
shadowed and winning keys in run warnings, run snapshots, and plugin
runtime state. Removing the environment requirement restores the
ordinary assignment.
- Document the contract for `driver: "plugin"`. Other drivers and
plugins without requirements keep existing behavior. Existing managed
copies retain edits and names until reset.

## Verification

- Latest targeted local checks: environment skills (5), managed skills
(9), heartbeat execution configuration (27), and SDK worker RPC host
(23): **65 tests passed** (including the targeted schema-drift check).
Disposable PostgreSQL exercised the migration, first install before any
run, concurrent lifecycle reads, implicit edit events, inherited
defaults, rollback, plugin requirement changes, shadowing, preserved
records, version capture, and recovery.
- Review follow-up also verifies case-insensitive/runtime-name aliases,
recovery of a task-mentioned shadowed skill, preserved quoted/multiline
YAML, and zero runtime events for repeated local-folder settings writes.
- Migration numbering and safety checks passed. Whitespace and
secret/PII scans passed.
- Earlier targeted manifest validation passed (43 tests). No validator
behavior changed in this follow-up.
- Broad tests, typecheck, build and end-to-end verification passed in
[GitHub
CI](https://github.com/paperclipai/paperclip/actions/runs/40387647219)
for current head `ce57f2a4d`. Greptile is 5/5 with no unresolved
threads. The [commitperclip review
check](https://github.com/paperclipai/paperclip/actions/runs/40387646208)
also passed after restoring its required duplicate-search checkbox.
- Earlier attempts found a SQL alias collision, an unparenthesized CASE
expression, and fixture assumptions about bundled catalog rows and YAML
quoting. These were corrected before the passing checks. The first broad
CI run also exposed a schema-drift check that assumed every journal
entry had a schema snapshot; it now reads the latest actual snapshot, so
SQL-only migrations need no invented snapshot. A test callback
replacement also affected an unrelated fixture; it was restored before
the final passing execution-configuration suite. Sandbox runs could not
start PostgreSQL; the reported database tests used local socket access.

## Risks

- Apply migration 0378 before using the updated lifecycle contract. It
adds trigger functions and event producers; it does not rewrite agent
assignments or skill content.
- Required environment skills take precedence over same-name
assignments, including task-selected skills. Both library records
remain. Run warnings identify omissions.
- Company edits are preserved. Plugin upgrades do not reset installed
content or rename existing copies. Explicit reset adopts current
defaults.
- Namespaced library slugs can be shared by distinct canonical records.
Callers must use IDs or canonical keys when a slug is ambiguous.
- No live provider request or deployment was performed. The downstream
plugin must be built against this Core revision. Persistent-home
synchronization retains its existing edit-conflict and retry behavior.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code
execution. The exact model identifier and context-window size are not
exposed in this task context.

## Checklist

- [x] Thinking path and issue description included
- [x] Model and available capabilities recorded
- [x] Roadmap and related PRs checked
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] Public references only; no internal issue IDs or instance URLs
- [x] Targeted local tests pass; broad verification delegated to GitHub
CI
- [x] New behavior covered and documentation updated
- [x] Current-head CI green and Greptile 5/5 with no unresolved comments

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 14:11:35 -07:00
DottaandPaperclip 9ae4c25e7e perf(runner): reuse compatible protected provider packs (#17749)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native Runner starts provider runtimes in remote execution
environments.
> - Sandbox images already contain qualified provider packs.
> - Startup required the image pack to match the entire current app
release manifest.
> - An unrelated app commit could therefore cause pack verification,
transfer, and installation again.
> - This pull request selects protected packs by their provider
interface and platform.
> - The benefit is no extra pack probe, directory scan, version command,
link, or upload on the compatible image path.

## Linked Issues or Issue Description

**What happened?**

Starting a remote task can reject a compatible image pack because its
source revision or unrelated provider differs from the controller. Pack
replacement added about 16 seconds in the investigated startup. Startup
also repeated full content checks.

**Expected behavior**

Use an already qualified, protected pack when its selected provider
interface and target platform are compatible. Keep the selected artifact
identity for process launch and recovery.

**Steps to reproduce**

1. Install a qualified provider pack in a sandbox image.
2. Start a controller built from another app commit.
3. Start a remote OpenCode task. The old complete-manifest comparison
rejects the image pack even when the provider interface still works.

Related repairs: #16493 reduced transfer cost. #17512 retained exact
artifact authority during reattachment. This change preserves that
recovery rule and changes fresh-launch pack selection.

## What Changed

- Add explicit OpenCode and ACPX interface revisions to produced pack
manifests. App Git SHAs and dependency pins remain provenance.
- Read bounded protected-installation metadata in the existing final
runner metadata query. Accept only root-owned paths that the execution
user cannot write, including ancestors and manifest. Reject writable
aliases.
- Select and prepare the pack in server-owned environment setup before
constructing the existing transport. Bind provider launch paths and
digests from the selected installation. Save that identity for exact
reattachment.
- Keep full content verification for uploads, writable installations,
and legacy packs. Reuse successful verification within one startup. Keep
older attached runners compatible with the optional metadata request.
- Seal image packs and document the one-time image refresh that enables
reuse across later compatible releases. Keep selected ACPX agent
qualification checks.

## Verification

- Red/green regression: the old code rejects a protected pack from
another app revision with `manifest mismatch`; the fixed path selects it
without executing verification commands.
- Native startup regression: select the installed pack's own Node digest
with one existing runner metadata request and no pack verification or
version commands.
- Targeted pack, artifact, recovery, execution-driver, and Rust metadata
tests pass. Incompatible interfaces/platforms, malformed identities,
writable roots, path aliases, and exact reattachment are covered.
- Final revision `91dc1676ff67bdbe5866c8586daac10240530cca`: workspace
typecheck and build pass. All 53 current-head checks are successful,
with no pending or failed checks. Greptile is 5/5 with no open findings.
- The broad local `pnpm test:run` attempt reported a workspace-runtime
conflict test failure; its isolated rerun passed. The broad run was
stopped after that failure. Full CI now passes, including the server
test shards. This is not a claim that the full local run passed.
- Linux filesystem evidence: the production metadata functions read a
real root-owned preinstalled pack as a non-root user in 16 microseconds
median and 20 microseconds p95 across 100 warm reads. This measures
metadata admission, not model or hosted response latency.
- All 910 affected server tests and 57 Copilot attestation tests pass
locally on the final implementation. Full CI covers the unchanged Runner
transport. No provider qualification hash was refreshed.
- Built the actual Linux daemon at
`416e7ec7425d0d3565b4a5e1808002d650a60414` and ran its metadata command
as UID 65534. The protected installation returned its real manifest with
`immutable: true`; writable alias and missing-pack cases returned
`immutable: false`. This is metadata/permission evidence, not provider
execution.
- Hosted rollout and a hosted end-to-end latency measurement have not
run.

## Risks

- Existing images need one rebuild to publish the compatibility
declaration. Packs without it keep the legacy content-verification path.
- Fast admission relies on image publication producing a qualified
protected tree. Root execution and task-writable installations retain
full verification.
- Bump an interface revision when required runner/pack behavior changes
incompatibly. Deploy a new image to roll out compatible runtime fixes or
upgrades.
- This removes provider-pack preparation delay. Other runner startup,
context, model, and finalization costs remain separate.

## Model Used

OpenAI Codex, configured model `gpt-6.1-sol`, reasoning effort `xhigh`,
with tool use and code execution. The context-window size is not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 14:43:12 -05:00
DottaandPaperclip 8c702d59db feat(evals): package a portable Runner controller runtime (#17506)
Package a verified portable Runner eval controller with complete bounded file receipts, copied dependency closure, private evidence replay and symlink-safe CLI entrypoints. Preserve fail-closed cost holds, native trace receipts and all parent security repairs.

Verified head: 1c892ff464.
Validation: 72 synthetic source/portable tests pass after current-master sync. Full current-head CI passed (40386180906); Greptile reviewed this exact head at5/5, with zero unresolved findings.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-26 14:32:46 -05:00
DottaandPaperclip 5061ab7b35 perf(ui): speed up task and inbox cold opens (#17756)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task lists and inboxes are the main entry points to that work.
> - Cold opens fetch overlapping task lists, task descriptions, and run
details before useful rows settle.
> - Independent responses can insert rows above content that a person
has started to read.
> - This pull request loads a small task index first and adds details
only for visible rows.
> - It keeps row order and geometry stable while those details arrive.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Cold opens of Tasks and Inbox, including the shared navigation badge.

**Subsystem affected**

Server list reads, shared response contracts, and the React UI.

**Current behavior**

The badge fetches full task bodies. Mine fetches several overlapping
lists. Historical diagnostics and row metadata add work before the first
useful list. A fast failed-run response can appear before task rows,
then move down when tasks arrive.

**Proposed behavior**

Load the selected inbox source and its required context together. Use an
index without task descriptions or recovery diagnostics. Fetch
diagnostic details for up to 50 visible rows. Keep skeletons, recovery
indicators, external status, and live metadata in fixed space.

**Reason and benefit**

Reduce work on a cold open and prevent loading-induced row movement. On
the same 100 local fixture tasks, the compact response was 262,441
decoded bytes and the index was 110,909 bytes. This comparison excludes
compression and is not a page latency benchmark.

**Breaking changes**

Existing full and compact responses remain available. New index/row
projections are opt-in and marked as partial data. Mine keeps its
existing most-recent-100 display limit, now selected on the server.
Existing filters and group preferences remain in the client. No database
migration.

**Additional context**

Refs: #17593. Related: #13095, #17428, #13803. This change addresses
collection overfetch and layout stability. It does not replace the
blocked-inbox query work or the database activity-index work.

## What Changed

- Add authorized index and row projections, with bounded task and parent
ID batches.
- Replace badge list downloads with unread IDs and aggregate counts.
Preserve actor privacy, dismissals, and local archive behavior.
- Load only the selected inbox source, plus bounded parent and
failed-run context. Return only relevant failures from the bounded run
window.
- Batch hierarchy reads, including at most one authorized child per
collapsed parent in one request. Collapsed markers do not consume the
expanded row budget, and partial descendant counts are not shown as
totals. Enrich visible rows without replacing membership, ordering,
titles, or read state.
- Reserve row geometry on desktop and mobile. Reveal the initial inbox
only after its sources settle. Preserve legacy task-list reads.
- Start collection module downloads during access checks. Keep partial
rows out of the task-detail cache.
- Keep ETags and concurrent request sharing for the new projections, but
skip the short server response TTL so archive undo sees fresh state.

## Verification

- On final head `ef6afb2a56`, the full CI workflow passed, including
build, typecheck, server/workspace/runner suites, and all eight browser
shards. A server shard reported **1,489 passing tests** before runner
shutdown canceled the job; its single infrastructure retry passed. Both
Greptile and Paperclip Review are **5/5**, with zero unresolved review
threads.
- The optional external **Entire Gates** check remains `unassessed`
because it has no active assessed Change for this comparison. Entire is
not set up in this checkout. The repository ruleset requires `ci /
verify` and `ci / e2e`; both passed. No gate configuration was changed.

- Passed workspace typecheck after the final API batching change. The
full production build also passed on the final commit; the exact-retry
E2E run built the shipped UI.
- Passed token gates and map validation. The final batch-preview pass
has 91 passing route/hierarchy/component tests, including privacy,
request bounds, 30 collapsed parents in one read, and a 120-child parent
alongside a sibling.
- Passed the 166-test focused UI/inbox/badge run, 47 additional UI
regression tests, and 34 issue-list route tests. The latest regressions
cover low-trust agent badge IDs, cold company selection, 30
description-only Mine matches with archive actions, retrying optional
row details, and collapsed/expanded siblings.
- The full local `pnpm test:run` spanned the rebase and reported
failures in five suites before it was stopped; it is not claimed as a
complete pass. A fresh serial rerun of all five suites passed
**452/452** tests (tool gateway, native resume, project repositories,
workspace recovery, and runner selection). Full current-commit coverage
is checked by CI.
- Real browser: delayed index and row-detail responses on Tasks and
Mine. The first placeholder and first row had the same origin and
44-pixel height. Twelve rows on each page had zero position, height, or
title-width change after recovery details arrived.
- Real browser: reviewed long titles and recovery status at a 390 by 844
viewport. Task text stays on one line and recovery uses a compact
indicator.
- Three fresh document loads per page on final commit `260893bf92`, with
HTTP cache retained on a warm local development server: Tasks
1.031–1.190 seconds; Mine 0.999–1.193 seconds. Browser-control overhead
is included. These values must not be compared directly with staging
latency or the earlier local samples taken during a full test run.
- The browser archive test exposed stale results after Undo. The new
projections now bypass the server TTL; immediate-mutation regression
tests pass. The final browser retry passed: Archive followed by Undo
restored exactly one row and the badge count.

- CI exposed a hook-order crash when company selection resolved after
the first inbox render. The hook now runs before the empty-company
return. All seven affected exact-retry and legacy-continuation browser
scenarios pass locally after the fix.
- Real browser review regressions: 97 description-only source matches
retained Archive actions; deliberately failed optional row reads
displayed a retry notice while all 12 observed row positions and heights
stayed unchanged. Retry restored the recovery indicator. Network fault
injection was removed.

- The final production build and workspace typecheck passed. Final
focused verification passed 153 UI/helper tests and 26 heartbeat privacy
route tests. Final review regressions cover newer diagnostics arriving
during the index read, reloading diagnostics when the same task ID
changes version, optional source failure/retry without hiding available
rows, and required external-status filter failure. The latest-failures
route test verifies bounded selection, agent filtering, success
suppressing an older failure, and private-run redaction.
- Real browser: deliberately failed the run source, then retried it.
Twelve available task rows kept the same position and 44-pixel height
before, during, and after retry. No loading skeleton replaced them.
Fault injection was removed.
- Hierarchy review decision: the cold-open barrier stays because
inserting ancestors or expanded children after revealing index rows
would cause layout shifts. Existing hierarchy context now survives
subsequent requests, with a regression proving a new ancestor read does
not blank the list. The bounded recursive tree read can still add
latency for unusually deep expanded trees.

- Tasks now handles a failed external-status filter with an explicit
error and Retry, suppressing a false empty state. All 52 Tasks component
tests pass, including failure-to-success retry. One existing large-tree
test timed out while workspace typechecking ran concurrently; the whole
component suite passed when rerun without that competing work. Workspace
typecheck, the full production build, and token gates passed for this
fix.

## Risks

- Saved filters and grouping still operate on the existing bounded
collection. This is not a new global server pagination model.
- Fixed row slots reduce available title width. Long titles remain
accessible through task navigation.
- Recovery and related diagnostics arrive after core rows. Their
responses cannot replace ordering fields, and stale versions are
rejected.
- Index/row requests now revalidate against current data instead of a
short server TTL. They still share concurrent reads and use ETags.
- This has not been deployed by this pull request. Staging latency must
be measured after deployment under comparable conditions.

## Model Used

OpenAI Codex, GPT-6 family. The exact runtime model ID and
context-window size are not exposed in this session. Used code editing,
terminal execution, and real-browser diagnostics.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 14:29:20 -05:00
DottaandPaperclip 46de15ff4b fix(recovery): preserve native workspaces after control deadlines (#17788)
## Thinking Path

> - Paperclip lets people manage agents and their saved work.
> - Native runs can prepare a remote workspace before provider
execution.
> - A finalization deadline can stop a run outside its normal exception
path.
> - That handler marked the run failed without recording its prepared
files for recovery.
> - Retry then waited for a lease whose preservation had no durable
owner.
> - This pull request records the exact source in the deadline
transaction through the existing recovery service.
> - Existing containment and file recovery can then settle the source
before another executor starts.

## Linked Issues or Issue Description

**What happened?**

A native run failed at its control deadline with a prepared workspace
and no accepted result. Its environment lease remained active. The run
had no workspace preservation record. A saved Retry waited for remote
cleanup without a preservation operation to finish that cleanup.

**Expected behavior**

Record unfinished files when the deadline marks a native run failed. Use
the existing stop-and-retain worker. Keep accepted results, exact source
binding, and current task ownership authoritative.

**Steps to reproduce**

1. Persist a native run with a prepared workspace, an active lease, and
an expired control deadline.
2. Reconcile the deadline twice.
3. Inspect the source lease and the run's workspace recovery record.

**Paperclip version or commit**

Reproduced against `a67a18c171816b097f46eae78fc58c7e55c9c565`.

**Deployment mode**

Hosted native Runner with a remote sandbox. The regression uses real
local PostgreSQL and no provider mocks.

Related: Refs #17741 for committed Retry delivery. Refs #17073 for
remote process-exit verification. This change records file preservation
after a deadline; it does not change either policy.

## What Changed

- Call the existing native workspace preservation service in the
transaction that fails an abandoned run.
- Preserve files after reassignment without changing the current task
owner or state.
- Add real-database cases for prepared, finalized, mismatched,
reassigned, and accepted-result states. Repeated reconciliation retains
one source obligation and creates no successor.
- Document the deadline behavior and its existing access boundaries.

## Verification

- Red: on unchanged production code, the prepared-workspace case failed
because its recovery record was null. Finalized and mismatched controls
passed.
- Green: all 10 deadline reconciliation tests pass. The existing native
workspace recovery suite passes all 48 cases.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- Map checks, module-boundary checks, and `git diff --check` passed.
- Current-head CI passes on `23bf2aa4b50d2c4b74a9ee2ee47034ae542d25c2`:
all GitHub Actions CI lanes, including grouped/serialized tests, Runner
verification, typecheck, build, E2E, and Canary Dry Run, succeeded. Two
Storybook-only checks were skipped. Greptile is 5/5 with no findings.
Superagent Security Scan also passed. Entire Gates reports passing
checks/findings and fails only because no reviewer has approved trail
#10.
- The first server shard 11/12 attempt passed all 1,750 assertions but
reported an unhandled early process-exit rejection in the unchanged
workspace-exposure fixture. Its focused local suite passed (28 passed, 3
skipped); one CI rerun passed.
- Full local `pnpm test:run` was started, then interrupted after the
complete CI lanes passed to avoid duplicate work. No full local suite
pass is claimed. Local targeted tests, typecheck, and build passed.
- On the affected staging instance, the existing deployed preservation
helper registered the exact failed source. Normal Retry then drove
actual sandbox stop, file recovery, and a persisted successful recovery
receipt. A successor reached a separate remote process-fingerprint
failure before provider execution. The automatic deadline change is not
deployed; task execution recovery remains incomplete.
- Initial local setup attempts did not reach product assertions: skipped
dependency lifecycle scripts left native PostgreSQL library links
absent, and the restricted shell blocked localhost binding. The
package's link hydration and an authorized isolated listener resolved
those setup prerequisites. One new negative-control assertion also
required null-safe access; the final red run retained only the intended
behavioral failure.

## Risks

- File preservation must not certify provider termination. The existing
stop receipt, source binding, copyback, and admission checks remain in
force.
- Preservation failure rolls back the deadline transaction, leaving its
durable deadline eligible for another bounded attempt.
- Reassignment preserves old files without changing the new owner's
task. Accepted results retain their current finalizer.
- No schema, credential, provider, or polling change. Historical failed
runs still require scoped recovery; this patch does not replay them.

## Model Used

OpenAI Codex based on GPT-6, with reasoning, repository inspection, code
execution, and browser tools. The exact serving model ID and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted suites; full
local suite interrupted after CI passed)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (CI lanes pass; external Entire
reviewer-approval gate remains pending)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 14:18:53 -05:00
DottaandPaperclip 063a498b54 Bound native OpenCode assigned-tool catalog projection (#17740)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents use assigned connection tools through a governed
gateway.
> - OpenCode received the gateway catalog directly, outside the bounded
server tool projection.
> - A large catalog can exceed a provider grammar compiler's capacity
even when it fits the transport limits.
> - Synthetic requests reproduced the Fireworks grammar timeout on the
configured DeepSeek model.
> - This pull request gives OpenCode two assigned-tool discovery and
call schemas through the existing server relay.
> - The gateway keeps the same grants, validation, approvals,
revocation, and audit rules.

## Linked Issues or Issue Description

**What happened?**

A native OpenCode run could fail during provider grammar compilation
before its first assigned-tool call. OpenCode advertised the full
assigned gateway catalog. A synthetic catalog of 741 tool schemas
reproduced `Grammar processing took too much time` on Fireworks through
OpenRouter with `deepseek/deepseek-v4.1-flash-1017`. The same model and
backend accepted a compact two-tool discovery/call catalog. This
reproduces the mechanism; it does not replay a historical private
request.

**Expected behavior**

Assigned-tool discovery must keep provider grammar size bounded. Every
call must retain the run's original gateway binding and permissions.

**Steps to reproduce**

1. Give a native OpenCode run a large assigned connection catalog.
2. Use a provider that compiles tool schemas into a constrained output
grammar.
3. Observe the full catalog in the provider tool surface before this
change.
4. After this change, observe only `paperclip_search_assigned_tools` and
`paperclip_call_assigned_tool` for assigned tools. Discover and call a
permitted tool through its pinned alias.

**Paperclip version or commit**

Source checkout based on current `master`.

**Deployment mode**

Native local or sandbox execution with OpenCode 1.18.34.

Related work: Refs #14258 (discovery paging), #16678 (local adapter MCP
launch), and #17688 (gateway capability advertisement). This change
covers native OpenCode catalog projection.

## What Changed

- Route native OpenCode assigned tools through the existing server-owned
authenticated relay.
- Use bounded discovery/call schemas for OpenCode, including small
catalogs. Preserve transport capacity checks.
- Keep assigned gateway credentials on the controller. Ignore gateway
and credential values in tool arguments.
- Preserve exact declared MCP operation names through admission and
driver dispatch. Resolve transport prefixes only against the closed
catalog. Reject unknown or shortened names.
- Refresh former OpenCode direct-catalog and shortened-name checkpoints.
Keep the existing fingerprints for other providers.
- Add local and remote wiring, catalog capacity, alias, scope,
work-mode, approval, and revocation tests.
- Allow the native test files to use a validated dedicated external test
database when embedded PostgreSQL is unavailable.
- Document the assigned-tool execution contract.

- Bound OpenCode discovery pages to 32 KiB and large-schema chunks to
16K characters. Keep other harness budgets unchanged.
- Test medium pages and escaped large schemas through the actual HTTP
MCP bridge on the explicit Runner testing surface.

## Verification

- Red before the catalog fix: OpenCode did not create the server relay,
and forced compact delivery advertised 1 or 741 direct schemas.
- Red before the alias repair: HTTP MCP changed declared operation
names, and the driver sent `search_assigned_tools` instead of the
authorized `paperclip_search_assigned_tools`.
- Current repair tests: 112 OpenCode/proxy tests pass, plus one actual
Rust/proxy/HTTP MCP regression. Two checkpoint regressions fail with the
former scoped revision and pass with the new revision. Other providers
keep their prior fingerprints.
- The combined QA candidate with this exact PR head passes all 916
native tests and all 113 transport tests on supported Node 24.22. An
earlier Node 26.4/macOS run passed the 113 assertions but had an
uncaught networking error. That non-clean run remains recorded; its
stack matches [the upstream Undici
defect](https://github.com/nodejs/undici/issues/5544). No production
workaround or error suppression was added.
- `pnpm -r typecheck`, `pnpm build`, and `git diff --check` pass on
`bc41ba72be2195eabeb0d7727227ee0cd02e8532`. The maps structural check
passes.
- The local `pnpm test:run` attempt stopped after host embedded
PostgreSQL initialization failures. The relevant database suites pass
with isolated PostgreSQL 18. All repository CI gates, including
aggregate `ci / verify`, pass on the current head.
- Synthetic live provider comparison: 741 invented schemas produced the
exact Fireworks grammar timeout in 20.9 seconds. Two compact invented
schemas produced a discovery call in 585 ms on the same model and
backend.
- The first hosted QA attempt exposed the alias failure described above.
On the patched source, a real local Rust runner → proxy → OpenCode
1.18.34 → exact DeepSeek smoke completed discovery, one synthetic
gateway read, and `paperclip_finish`, then emitted `turn/completed`. It
used the production compact projection and unchanged Rust allowlist.
- Two earlier local smoke attempts exposed missing terminal authority
and completion-contract fields in the smoke harness. Those failures
remain recorded. The corrected ordinary bindings produced the clean
result.
- Fresh Greptile: 5/5 on `bc41ba72be2195eabeb0d7727227ee0cd02e8532`,
with no new actionable findings or unresolved threads.
- Hosted QA publication and lifecycle acceptance remain pending. The
separate deployed-agent `Paperclip Review` check is also pending; it
follows a task that failed on the earlier deployed runtime.

- Local OpenCode/OpenRouter acceptance at `bc41ba72`: 530 synthetic
tools were authorized and installed. A simple model-ID reply completed
in 17.3 seconds with no app calls. A discovery task completed in 37.6
seconds, used the search/call relay, made one upstream call, and
returned the exact synthetic value. This is local source acceptance, not
a staging sandbox timing comparison.
- The large-result review finding has a failing-before/passing-after
HTTP MCP regression. Five medium schemas now page without invalid JSON.
A large schema with escaped text reconstructs exactly through bounded
chunks. All 35 assigned-tool tests and the server typecheck pass after
this follow-up.
- Final head `d84ce4856a6e65a35ad5fb8b561492598b525f4e`: all CI gates
pass. Greptile and Paperclip Review both rate the change 5/5. No review
threads remain open. The final full build, server typecheck, and 35
assigned-tool tests pass locally.

## Risks

- OpenCode uses discovery before it calls an assigned tool, including
for small catalogs. This can add a model turn.
- An existing OpenCode provider checkpoint rotates once because it
retains the former tool declarations. Paperclip task history remains
available.
- The historical filtered catalog was not captured. The synthetic
comparison proves the failure mechanism, not the exact cardinality of an
earlier private request.
- No model, provider route, permission, or automatic retry policy
changes.

## Model Used

The bounded-result follow-up also used OpenAI Codex (GPT-6 family). The
exact backend model ID and context-window size were not exposed for that
session. It used reasoning, code execution, and tool use.

OpenAI `gpt-6.1-sol` in Codex, with reasoning, tool use, code execution,
and source inspection. The runtime does not expose the context-window
size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 10:49:23 -05:00
DottaandPaperclip 5823e9a664 fix(runner): preserve disabled Codex web search (#17744)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native Codex agents use the saved agent settings to prepare each
provider launch.
> - Onboarding saves web search as off, but native configuration
conversion removes that setting.
> - Codex 0.160.0 includes a hosted web-search tool by default, even in
an authentication probe that asks it not to use tools.
> - A gateway that permits only priced function tools can reject that
request before budget admission.
> - This change preserves the explicit restriction through native
execution and disables web search for authentication probes.

## Linked Issues or Issue Description

**What happened?**

A native Codex authentication probe can include hosted web search.
Normal task execution also loses a saved `search: false` setting. A
restricted Responses gateway can reject these requests before it makes a
model call. Related repairs: Refs #17445, Refs #17529. The roadmap and
related PRs were checked; no duplicate repair was found.

**Expected behavior**

Authentication probes check the selected account and model without
hosted web search. A saved search-off setting remains effective in
native task launches, prepared launches, and session recovery. Existing
behavior for an absent setting remains unchanged.

**Steps to reproduce**

1. Create a Codex agent with search off and choose Paperclip Runner.
2. Use a Responses gateway that rejects hosted tools.
3. Inspect the request from the native authentication probe or task
launch. Before this fix, the pinned Codex SDK includes `web_search`
despite the saved setting.

## What Changed

- Preserve an explicit search-off setting when native agent
configuration is built and converted.
- Carry the restriction in the durable native input and provider launch
requirements.
- Apply the restriction to authentication probes and normal launch
paths, including custom arguments and recovery.
- Reject prepared launches that discard the required restriction. Keep
gateway billing rules unchanged.
- Admit only the restrictive literal value in the strict plugin task
API. Exercise the real plugin setup path.
- Add regression coverage and document the policy.
- Add the immutable Copilot v60 source identity required by the changed
shared files. Preserve v59 and all original live evidence. Copilot
authentication probes keep their prior launch requirements.

## Verification

- Regression cases were run against the unfixed source and failed at the
configuration and launch boundaries.
- An isolated capture using the real Codex 0.160.0 app-server reproduced
hosted-tool rejection. Disabling web search reached a mocked budget
reservation. The helper and child processes permitted only loopback
network access and used synthetic credentials. It made zero upstream
calls.
- Focused checks passed: 284 server cases, 61 native form cases, 198
backend/security/launch/parser cases, and 36 authentication/launch
cases. A separate complete transport/launch/parser run passed 393 cases.
These groups overlap.
- Head `617b98f` passed workspace typecheck/build and 28/28 plugin
contract and actual Runner probe tests with no skips. Final rebased head
`20ff0c1` passed all current-head CI checks, including full isolated
test lanes, and fresh Greptile and Paperclip reviews at 5/5.
- The additive Copilot identity update passed 134
compatibility/policy/protocol checks and nine authentication probe
checks. The real pinned Codex SDK capture was repeated after the final
policy change: all five cases passed with zero upstream calls.
- Two local full-suite attempts stopped after native PostgreSQL loader
failures. Seven local dependency aliases were restored, and an isolated
cluster initialized successfully. A third run later encountered host
PostgreSQL semaphore exhaustion and was interrupted before
source-profile regeneration. Its zero exit code after SIGINT is
explicitly not a full-suite pass. Final full-suite verification runs in
isolated CI; the local attempts remain recorded.
- The first CI run exposed a missing strict plugin API field. Its five
failures were reproduced locally, then passed after repair. The next run
passed every underlying job except the source-identity guard. That guard
now binds an additive identity with v59 retained. Final-head CI and both
fresh reviews passed; addressed review threads are resolved.
- The offline capture runs on Darwin ARM64. It does not prove the
contents of a prior unretained Linux request or complete Cloud
onboarding acceptance.

## Risks

- The restriction must survive prepared launches and resumed sessions.
Regression tests cover those boundaries.
- Existing absent settings retain their prior behavior. This change does
not add native search-on support or loosen gateway admission.
- Older durable inputs without the optional restriction must remain
valid.
- Copilot now uses profile v60 for exact launch admission. Historical
v59 inputs remain readable. The new declaration adds no live Copilot
qualification and does not change its executable or admission flags.
- A deployed Cloud acceptance run is still required after this source
repair.

## Model Used

- OpenAI GPT-6 through Codex, with reasoning, code execution, and
delegated review. The exact backend model identifier and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 10:29:51 -05:00
DottaandPaperclip 0373f9f261 fix(recovery): wake retained workspace preservation after Retry (#17741)
## Thinking Path

> - Paperclip lets people manage AI agents and their saved work.
> - A stopped run can owe workspace preservation before saved input can
continue.
> - Retry saved a recovery request but did not wake the worker that owns
preservation.
> - Older recovery records could also take the bounded scan slots before
a due request.
> - A second Retry could change the revision while the physical owner
was still saving files.
> - This pull request wakes a separate worker after commit, selects due
work first, and keeps the active owner's revision.
> - Saved work can progress without replaying the stopped provider turn
or blocking other finalization.

## Linked Issues or Issue Description

**What happened?**

An operator approved a saved design and selected Retry after native
workspace finalization failed. The API accepted one canonical request.
The source remained scheduled at zero preservation attempts for more
than 30 minutes. No new provider turn started. A local regression also
found that repeated Retry during export invalidated the physical owner's
completion revision.

**Expected behavior**

Committed Retry must wake bounded source preservation. Due recovery must
progress ahead of passive cleanup obligations and unrelated history.
Duplicate requests must keep one physical attempt. Saved input must
still pass normal admission after verified recovery.

**Steps to reproduce**

1. Retain a terminal run with an exhausted workspace recovery and saved
authorized input.
2. Let the delivery coordinator become quiet, then request Retry.
3. Observe the recovery state and physical preservation receipt. Repeat
Retry while the export is held.

**Paperclip version or commit**

Observed on served commit `4b89eec644d6e1a99d58926e77ae58927c3fa87b`.
Local baseline: `a1f77821b9cd5c9c846fdf7a0fa5cc620ad2b6bb`.

Related work: Refs #17053, #17267, #17094. This change repairs explicit
Retry delivery. It is separate from the run instruction projection in
#17724 and historical unsafe-export discovery in #17687.

## What Changed

- Notify a dedicated workspace recovery duty from the transaction that
saves Retry, recovery admission, adoption, or recovery progress.
- Reconstruct work and saved retry deadlines at startup. Fill each free
export slot independently, track physical work for idle drain, and join
the dedicated worker's exports on shutdown.
- Drain one bounded pass through completed cleanup and saved-input
obligations. Restart that pass after committed changes without
repeatedly rechecking paused input.
- Share source eligibility between deadline selection and candidate
consumption. Keep resolved files-only obligations and ignore unrelated
resolved or cancelled history.
- Select due unfinished preservation before passive historical records.
Rotate past source locks held by other workers and retain the indexed
source lookup.
- Bind the canonical Retry intent without changing an active physical
attempt's generation or revision. Preserve that binding across later
progress writes.
- Add real producer, physical preservation, startup, deadline, idle, and
independent finalization regressions. Document the delivery contract.

## Verification

- Red controls: Retry without a notification stayed scheduled; duplicate
Retry invalidated the active export; whole-batch waiting stranded a
third Retry; a scan without cursor rotation starved a source behind two
actual advisory locks. Removing unfinished-first selection left HTTP
Retry scheduled behind five real recovered sources still owing cleanup.
The fixture explicitly verifies their eligibility with the worker's
actual query filter.
- Further red controls: the previous worker visits only two of three
eligible completed sources. Disabling the committed-change cursor reset
misses a new earlier source arriving while two callbacks are held. Both
pass with the final worker.
- Green: the public Retry HTTP route, registered production worker, real
file copyback, duplicate canonical receipts, and public recovery receipt
reload passed on an owned local fixture. Its agent is paused to keep
this files-only check from starting a provider turn.
- Full workspace recovery suite passed 56 tests, including the
foreign-lock regression.
- Execution-control lifecycle suite passed 24 tests. It covers idle SQL,
stored deadlines, cancelled and resolved history, independent
finalization, and shutdown. One, three, and seven paused completed
sources are each visited in a bounded pass, then stay deferred without a
successor or SQL during 90 seconds of fake-clock idle time. A committed
earlier source is evaluated when it arrives during two held callbacks.
- Both GitHub and Telegram `automatic_running` continuation regressions
passed with the new canonical intent binding.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- `node scripts/check-maps.mjs` and `node
scripts/check-module-boundaries.mjs` passed.
- Local `pnpm test:run` could not complete: embedded PostgreSQL
repeatedly failed to start. The owned run was stopped after this setup
failure repeated. Latest-head CI supplies broad coverage.
- Earlier setup failures included a localhost sandbox denial and loss of
the owned disposable Docker database. These were separate from the
product red controls above.
- Local database tests use an explicitly named disposable PostgreSQL
`_test` database. That opt-in database is reset before the suite.
Default embedded test behavior is unchanged.
- Exact-head CI is green at `ef7dcf1ba1060864c9d216c33b51f4d9f8f57fbd`;
Greptile and Ponytail both rate it 5/5, with no unresolved review
threads. Earlier CI caught the indexed discovery assertion and two exact
chat intent-binding regressions; all three were fixed and pass locally
and in CI. A separate chat shard lost contact with its self-hosted
runner without a product assertion result. A later shared runner
shutdown interrupted several jobs; one infrastructure rerun passed
before the final test-and-worker follow-up triggered a fresh green run.
- No staging deployment is part of this PR. The original approval
remains saved; deployed recovery still needs acceptance after rollout.

## Risks

- More recovery writes now wake a bounded maintenance duty. Completed
history must not keep it active. Idle and deadline tests cover this
boundary.
- Resolved files-only obligations must remain eligible without granting
replay. The startup regression checks that path.
- Provider stop proof, copyback ownership, source binding, retry
budgets, and normal execution admission remain required. There is no
schema or public API change.

## Model Used

- OpenAI Codex based on GPT-6, with code execution and browser tools.
The exact serving model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 10:28:12 -05:00
DottaandPaperclip 338bf75761 fix(chat): admit task Retry after GitHub reconnect (#17733)
## Thinking Path

> - Paperclip manages agents and their tasks.
> - Task Retry must preserve saved input and enforce current access.
> - A GitHub reconnect invalidates old runtime callbacks.
> - Retry reused that old callback authority and rejected valid new
intent.
> - Retry also replayed an older failed request when a follow-up was
already saved.
> - This change admits a deliberate Retry under current authority and
transfers saved input once.

## Linked Issues or Issue Description

Refs #16965, #17053 and #17317. These repairs preserve work and contain
old executors. This change fixes the subsequent chat admission boundary.
A search found no overlapping open reconnect/admission fix.

**What happened?**

Ordinary task Retry returned `chat_failed_run_retry_not_authorized`
after the same GitHub App reconnected. The endpoint was healthy, but the
saved request still carried its old runtime generation. A newer queued
follow-up also made Retry reject the older failed request instead of
continuing the follow-up. No successor was admitted.

HTTP qualification found two related defects. Concurrent clicks could
omit the canonical chat action from a response. Signed-in board dispatch
also passed an incomplete task resource to the permission resolver,
which rejected a permitted mutation.

**Expected behavior**

A deliberate Retry checks current authority for the same App, sender,
repository, conversation and task. It continues saved input once. A
later reconnect, changed App, revoked access or changed source prevents
dispatch. Old runtime callbacks remain invalid.

**Steps to reproduce**

1. Admit a GitHub request and record a failed native run.
2. Save a follow-up while execution is blocked.
3. Reconnect the same GitHub App.
4. Click ordinary task Retry.
5. Before this change, admission returns 409. Without reconnect, the
newer input alone also causes 409.

**Paperclip version or commit**

Baseline source: `41c42895fa862c586ff841f8c50200d93ffc83a4`.

**Deployment mode**

Source checkout on macOS. Regressions use real PostgreSQL transactions
and the assembled chat, HTTP route and heartbeat admission services.
Provider boundaries are simulated. No hosted recovery is claimed.

## What Changed

- Record the admitting App identity for new GitHub deliveries.
Historical deliveries can use their original working-comment binding.
Reauthorize deliberate Retry under a pinned current generation and
credential-reference fingerprint.
- Select the complete saved-input queue from the failed run's actual
conversation before replaying the older failed batch. Keep exact-run
Retry bounded while ordinary task Retry supports larger queues. Transfer
its receipts with the dispatch transaction. Recover the same ownership
after restart or a lost response. Release unadmitted input after a
definitive rejection.
- Preserve canonical chat action IDs across concurrent clicks. Supply
current task status and assignment fields to board mutation checks.
- Keep exact-run Retry, stale webhook, containment, source integrity,
publication, pause and budget checks. Update recovery contracts and
verification recipes.

## Verification

- Red: all three new PostgreSQL regressions failed on unchanged
production code. Reconnect failed at chat authorization. Newer input
failed at the obsolete-source guard. The combined case also failed.
- Green: 61 selected chat integration cases passed. These include 27 new
cases and 34 existing cases for containment, retained-source recovery,
exact batches and source revocation.
- A further red/green PostgreSQL regression reproduces the 51-message
overflow identified during review. All 51 messages reach one successor,
including a later Retry of that successor. A separate 52-message
consumed-batch regression also passes.
- A competing conversation retains its own deferred message. Distinct
automatic policy contexts are checked independently; revoking only the
later policy blocks the whole unadmitted batch. Batch validation still
rejects a later message with a changed App binding or deleted source. A
real PostgreSQL query capture reduced the 51-message preparation from
1,103 calls to fewer than 180.
- All 16 durable task Retry tests passed.
- The real HTTP route returns 202 and admits one queued successor for
concurrent deliveries. Fault injection covers rollback during input
handoff, response loss after dispatch commit, and permission loss when
resuming an existing durable wait. A wait only adopts input after
current admission succeeds; a one-shot settlement marker keeps status
projection read-only. A new deliberate Retry works after another
reconnect or a failed successor.
- Current-App, repository, sender, source edit/delete, reassignment,
endpoint pause and board-permission controls passed. Historical
deliveries remain unchanged.
- Final-head CI: 55 checks passed, three expected skips. The unchanged
minute-bucket rate-limit assertion passed locally against PostgreSQL and
on its one CI rerun. Greptile reviewed the final commit at 5/5 with no
unresolved threads.
- Workspace typecheck and full build passed. Map alignment, module
boundaries, UI token gates and `git diff --check` passed.
- Hosted task recovery and a live provider turn remain unverified. The
local tests hold agent capacity to inspect admission without starting a
model or publishing a GitHub response.

## Risks

- A deliberate GitHub Retry can admit old saved input under current
connection authority. It requires the same App binding and current
access. It pins that admission; it does not rewrite the historical
delivery or make stale callbacks valid.
- The durable chat queue has an optional admission callback. It
transfers input only when the transaction contains a valid queued
execution or saved wait. Other chat consumers have no callback and
retain their behavior.
- No migration, Cloud dependency, deployment, configuration change or
workspace cleanup is included.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, source inspection, code
execution and GitHub tools. The exact serving variant and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 10:26:45 -05:00
DottaandPaperclip a1f77821b9 perf(tasks): show recent conversation before historical activity (#17593)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A task page must show the instruction and current conversation
quickly.
> - The current page waits for related task data and historical run
transcripts.
> - These reads grow with task history, even when the reader only needs
the latest answer.
> - This pull request gives cold opens a small initial view and loads
older history on demand.
> - Messages and runs observed while the page stays open remain visible.
> - The benefit is a readable task before historical diagnostics finish
loading.

## Linked Issues or Issue Description

Related: #14667 and #16229. The first changes the reveal gate and scroll
anchors. The second adds general history pagination. This PR adds a core
task read and a folded recent-conversation view. It preserves the
default full-read API contracts. The shared files will need coordination
if either related PR lands first.

**What existing behavior does this improve?**

Cold opening an ordinary task with a long conversation or large run
logs.

**Current behavior**

Task enrichment and historical transcripts delay the readable
conversation. A cold open can fetch several megabytes of tool activity
before revealing saved messages.

**Proposed behavior**

Show the task instruction, recent saved replies, latest human input, and
the latest run's conversation data first. Fetch older messages and full
run activity when the reader chooses **More messages**. Retain new
messages while the page stays open.

## What Changed

- Add an authorized core issue view that skips related enrichment. Keep
enrichment on the existing detail read. Share task mutation, rollback
and invalidation updates with the core cache.
- Add a bounded recent-comment read. Preserve the first human
instruction when no description exists, including legacy rows with a
NULL author type. Keep company and task access checks and comment
redaction.
- Add optional recent activity and run-summary views. Fetch original
input receipts for visible comments separately so delivered messages
stay out of the queue. Reconcile watched unfinished runs after a
reconnect beyond the preview window, removing deleted runs after
reconciliation.
- Filter the latest native run's initial events to assistant messages,
request lifecycles and terminal results. Full event reads remain
available on expansion. Legacy fallback reads use a small tail extended
to the first complete record. HTTP cancellation aborts the storage read.
- Retain observed messages across cursor catch-up and new runs. Apply
local cancellation and deletion to both caches and their task aliases.
Refetch locally seeded history pages on expansion. Preserve expansion
across identifier and UUID links. Keep deep links, classic tasks and
Agent Chat on their existing full-history path.
- Update API documentation, design guidance, capability maps and
regression coverage.

## Verification

- Final acceptance on `9ea0b15ae44d9f6d2c9a9f824b4835f299f92979`: all CI
gates passed, including server/UI tests, all eight browser shards,
runner checks, full typecheck and build. [Final CI
run](https://github.com/paperclipai/paperclip/actions/runs/40326990328).
GitHub reports 55 successful checks, two intentionally skipped checks,
and one neutral Paperclip Review check described below. No checks are
pending or failed; Greptile is 5/5 and no review threads remain open.
- October 26 merge update: merged current `master`. The only conflict
was in task-page imports; both the compact-history hook and the new
live-update hooks remain. All CI tests and build jobs passed on
`044fd6733d`. Fresh review found canceled-comment cache drift, legacy
human-comment filtering, send-then-expand truncation, and partial legacy
log records. These defects were fixed in `af3d66a5bd`.
- Review-fix verification: 470 tests pass across ten affected suites,
including all six PostgreSQL cold-open tests without a helper override.
Cancellation/delete and send-then-expand regression tests failed before
the fixes and pass afterward. Coverage includes task identifier/UUID
reopen, stale in-flight reads, legacy NULL author types, local-file and
object-storage tails cut inside UTF-8 text, route forwarding, and
request cancellation. Full workspace typecheck and production build,
token gates, map validation and diff checks pass.
- Follow-up review found that a failed expanded legacy read lost its
record-alignment state on retry. `9ea0b15ae4` commits the expanded
window only after a successful read. The new regression fails before the
fix and passes afterward; all 19 transcript tests, UI typecheck, UI
build and token gates pass. Greptile reviewed this commit at 5/5 with no
actionable findings, and all review threads are resolved.
- The final Paperclip Review check ended neutral because its execution
did not complete; no passing Paperclip assessment is claimed. Its
earlier four concrete findings were fixed and regression-tested.
Greptile and the security scans completed successfully on the final
commit.
- The preceding `af3d66a5bd` CI run was canceled when superseded by the
retry fix. Its server shard 11 check briefly reported failure, but the
job API records cancellation and its test step passed all 1,750 tests.
This superseded run is not used as final CI acceptance.
- 549 focused tests pass on the final implementation across nine task
UI, activity-route, OpenAPI and PostgreSQL suites. These cover pending
history, desktop/mobile visibility, expansion, retained runs, message
bursts, cancellation recovery and queued-comment removal in both views.
- Legacy expansion keeps the latest answer visible while fetching and
deduplicating older output.
- Five PostgreSQL tests pass for core access denial, recent comments,
missing-description instructions, event paging and bounded activity/run
reads. macOS could not start another PostgreSQL cluster because it
exhausted its semaphore limit. These tests used a separate disposable
database on an existing local server through a temporary test setup
override. The helper was restored afterward.
- Full workspace typecheck and build pass with the final state fixes, as
do token gates and capability-map validation.
- Token gates, capability-map validation and `git diff --check` pass.
- Browser proof used a disposable task with 80 comments and six large
run histories. The cold view showed the instruction and recent exchange.
**More messages** revealed older comments. Navigation away and back
preserved expansion.
- Local HTTP sample: core 50 ms, recent comments 24 ms, recent run
summaries 17 ms. The latest native conversation response was 839 JSON
characters / 8 ms; its full event response was 1,099,005 / 86 ms.
Browser content paint was 1.34 seconds with development assets cached.
These are local observations, not staging latency claims.
- The first CI run passed every job except four board-send browser cases
and the review threshold. Those browser fixtures only mocked the old
comments endpoint; they now serve both recent and full reads, retaining
all message and attachment assertions. Both initial review findings were
fixed with regression tests. The next CI run passed every job. The
deeper review identified core-cache mutation, older delivery receipt and
retained run-status gaps; those are now fixed with regression coverage.
Final review is 5/5 with zero unresolved findings. Final-head CI
initially failed one unchanged runner shutdown test
(`runnerd-codex-transport.test.ts`, timeout-attachment case) with
`NativeSessionCloseUnrecoverableError`; all other CI jobs passed. The
same runner code passed on 971a762f80. A targeted local reproduction
also failed during provider drain. The failure is outside the changed
task loading paths; its underlying cause remains unresolved. The single
failed-job retry passed on the same head; this demonstrates recovery,
not a root-cause fix. All CI gates passed on the previous implementation
head `9004f94f28`: [CI run and
retry](https://github.com/paperclipai/paperclip/actions/runs/40192334629).
- `pnpm test:run` was attempted and stopped after repeated embedded
PostgreSQL startup failures. The native session suite recorded 15
database-related failures; other database suites could not start. No
complete local suite pass is claimed.

## Risks

- The compact view intentionally omits older diagnostics until
expansion. Existing open questions and approvals still use the current
interaction and run paths.
- Previously observed comment and activity rows remain cached while
open. The newest preview and incoming cursor pages refresh; this does
not add a general historical-edit synchronization protocol.
- Recent views are optional. Existing full API callers retain their
response shape and behavior. No migration is required.
- A legacy tail can exceed the requested byte limit by the prefix of its
first log record. Backward reads stop at the preceding record boundary
and honor cancellation.
- Staging has not been deployed or qualified with this change. Instance
startup latency is separate from task history loading.
- The related open PRs touch the same task-history files.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository editing, shell
tools and browser testing. The exact serving model identifier and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — 549 affected task tests;
the full local suite and unrelated runner failure are documented above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 08:11:39 -05:00
DottaandPaperclip 4b89eec644 fix: check selected workspace projects before offering retry (#17552)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task recovery explains access failures before a user retries saved
work.
> - A task project and its selected workspace are separate settings.
> - Moving a task can retain files from a project that the agent cannot
access.
> - The recovery notice checked only the task project or a past failure
record.
> - This pull request checks the projects required by the current
workspace selection.
> - The notice now offers Retry only after those project checks pass.

## Linked Issues or Issue Description

Follow-up to #17509. Related: #13230 concerns workspace binding
consistency; this fix changes recovery presentation only.

**What happened?**

A task could show “Project access is ready” after moving to an allowed
project while keeping a repository from a denied project. Typed failures
could also keep the notice blocked after an authorized workspace
replacement.

**Expected behavior**

Use the current task, workspace, and source project requirements. Honor
a valid pending selection for the next admission. Keep stale, archived,
or missing selections blocked without offering a project-permission
repair.

**Steps to reproduce**

1. Assign a low-trust agent to a task whose selected source belongs to a
denied project.
2. Persist a project access failure.
3. Move the task to an allowed project without changing its selected
files.
4. Reload the task. The notice must still explain the access block.
5. Select an allowed source through the workspace API. Reload the task.
The notice must offer an explicit retry without starting work.

**Paperclip version or commit**

Reproduced after #17509 on master. The regression failed on the old
projection and passed with this change.

## What Changed

- Batch the current workspace project requirements for tasks with
project access failures.
- Check retained workspace and source projects, plus valid pending
workspace selections.
- Keep unknown historical denial projects unnamed. Do not expose
unrelated project identifiers.
- Keep unavailable workspace selections in the non-retryable failure
state. Adding project permissions cannot repair these selections.
- Add real database regressions for project moves, pending and applied
repairs, stale revisions, archived pending selections, and
foreign-company workspaces.
- Update the recovery contract and verification recipe.

## Verification

- Red-green regression with the real issue update service and execution
access check.
- Focused tests: 17 tests passed across `agent-project-access.test.ts`
and `task-interruption-presentation.test.ts` after rebasing on current
master.
- Browser: the retained source stays blocked when the task project is
allowed. Manage project access opens Permissions / Trust with Allowed
projects and Add project.
- The normal workspace API accepted an allowed pending selection.
Returning to the task showed “Project access is ready” and Retry.
Permissions and run count did not change.
- Full repository typecheck and build passed before the final
presentation adjustment. The final adjustment passed server typecheck,
server build, and all 17 focused tests. The stale, archived, and
foreign-workspace cases failed before this adjustment. Map and
module-boundary checks passed.
- Browser: an archived pending workspace shows the existing failure
notice without Retry or a project-permission action.
- `pnpm test:run` was attempted and stopped after 15 native runtime
database tests failed because embedded PostgreSQL could not initialize
on this host. The focused database tests passed on isolated PostgreSQL
18 in Docker. All technical CI checks passed on `a6e9f977`. Fresh CI and
review are required for the presentation adjustment. One Runner shard
passed on its single rerun after a provider shutdown failure. A fresh
local Runner build also passed that case and its reuse sibling. The
prior Paperclip Review could not publish its assessment. Its saved
presentation finding is addressed. Merge waits for a fresh published
review and all checks on the current head.

## Risks

- Presentation must follow workspace selection precedence. Tests cover
retained bindings, valid pending changes, stale changes, and company
isolation.
- This adds at most three bounded reads for tasks with a project access
failure. It adds no reads for this purpose on unaffected tasks.
- Dispatch remains the authorization authority. This change does not
grant access, move files, or start execution. No migration is required.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and browser tools. The exact serving model identifier and
context window were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 07:30:27 -05:00
Devin FoleyandPaperclip 1fbbef45a8 perf: omit unused descriptions from recovery candidates (#17695)
Keep unused task descriptions out of the recurring candidate read. Preserve the task content, state gates, and full current-task reload before repair.

Verified 86 focused recovery tests, root typecheck, server build, a failing base counterfactual, all 55 CI checks, and fresh Apex 5/5 review with no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 21:50:48 -07:00
Devin FoleyandPaperclip 6bdcb70244 perf: reduce unused task payloads in recovery state checks (#17694)
Use the existing disposition-state fields for source and child task checks. Preserve full action evidence and the current full-task reload before repair.

Verified 61 focused recovery tests, root typecheck, server build, the base counterfactual, full CI, and fresh Apex 5/5 review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 21:32:21 -07:00
Devin FoleyandPaperclip 7703220d38 perf(github): narrow response authority reads (#17686)
Read current GitHub response identity and policy fields without transferring saved message bodies, instructions, or automatic policy wrappers. Preserve company predicates, shared locks, publication leases, malformed input handling, and stored content.

Verify workspace typecheck, server build, 414 selected GitHub tests, three focused browser cases, all 55 checks, and Apex 5/5. Preserve the original browser cleanup deadlock failure; always restore experiment settings while reporting cancellation errors.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 20:52:09 -07:00
Devin FoleyandPaperclip 6336c34041 perf(chat): narrow bound-conversation authorization reads (#17684)
Read current identity, policy, and setup credentials without transferring unused chat content. Preserve company predicates and all three lock modes.

Verified workspace and server typechecks, the server build, 246 selected tests, all CI checks, and Apex 5/5. The setup cases use fixed ordered receipt and processing timestamps.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 20:04:45 -07:00
DottaandPaperclip 5cea9c7c0e fix(ui): show eligible wake targets when resuming tasks (#17521)
## Thinking Path

> - Paperclip manages agents and their assigned tasks.
> - A task pause controls execution without changing task status.
> - Resume can remove that pause and request work from eligible
assignees.
> - The preview counted blocked tasks and former executors as wake
targets.
> - The release route correctly skipped those targets, so the dialog
promised work that did not start.
> - Resume also trusted stale pause snapshots after tasks were reopened.
> - This PR aligns preview with execution eligibility and rechecks
current task state when releasing a pause.

## Linked Issues or Issue Description

**What happened?**

Pause an assigned blocked task, then open Resume. The dialog says that
one task will resume and offers to wake one agent. Applying it only
removes the pause. The route skips blocked and backlog tasks. A former
executor can also inflate the wake count after reassignment.

**Expected behavior**

The dialog must distinguish removing a pause from starting work. Its
wake count must use the current assignees and statuses accepted by the
release route. Blocked work must stay blocked.

**Steps to reproduce**

1. Open an assigned blocked task.
2. Choose Pause work, then Resume task.
3. Inspect the wake count and apply Resume work.

Related runtime recovery work is tracked separately in #17097 and
#17439. This PR does not change remote execution recovery.

## What Changed

- Share the resume status list between preview and release. Count only
current assignees on todo, in-progress, and in-review tasks for resume
wakes.
- Explain that Resume removes the pause. Show the existing preview
warnings, including tasks that remain idle.
- Disable and uncheck the wake option when no agents qualify. Send the
same choice to the server.
- Reevaluate previously skipped pause members after reopening or
releasing another hold. Retain active-pause and unsafe-execution guards.
- Add real PostgreSQL and HTTP-route regressions plus browser coverage
for both task interfaces. Keep draft, ancestor-pause, and
Stop/reconciliation coverage.
- Update the task coordination maps. Add an opt-in dedicated test
database URL for hosts where embedded PostgreSQL cannot start.

## Verification

- Final head `9548eb16736f632cc111a5c13a9a994ec1d95a13`: all 55
completed checks pass, with two optional Storybook checks skipped. [CI
40188094904](https://github.com/paperclipai/paperclip/actions/runs/40188094904)
passes, including typecheck, build, all general/serialized suites, and
all eight E2E shards. Greptile is 5/5 on this commit; the reopened-task
finding is fixed and resolved.
- Red: the real PostgreSQL regression receives three agents instead of
the one eligible current assignee on the original code.
- Green: all 27 tree-control service, route, and unit tests pass. They
cover current assignees, blocked/backlog exclusions, reopened tasks,
company boundaries, remaining pause holds, and execution blockers.
- The reopened-task route regression fails on the previous
implementation: skipped members never reach preflight/wake, and an
unsafe replay releases the hold instead of returning 409. All cases pass
with the fix.
- Browser: all five applicable cases pass together (90 seconds):
blocked-work resume in both interfaces, draft/ancestor-pause cases in
both interfaces, and the process
Stop/pause/reconciliation/restart/cancel lifecycle. The optional
native-runner fixture is skipped locally because its fixture binary is
not configured.
- All 157 task-page component tests pass on the prior commit; subsequent
changes affect the server route and Stop fixture only.
- `pnpm -r typecheck` and `pnpm build` pass before rebase, and CI
typecheck/build passed on the previous PR commit. Server typecheck
passes again after the route fix. The final-head CI run also passes
these full checks.
- `node scripts/check-maps.mjs`, `node scripts/check-token-gates.mjs`,
and `git diff --check` pass.
- Full `pnpm test:run` was attempted but did not complete: this host
cannot initialize embedded PostgreSQL, and unrelated workspace-runtime
tests timed out. The focused database tests and E2E suite use disposable
PostgreSQL 18 instead. All general-test and serialized-test CI shards
passed on the previous PR commit; the final-head CI run also passes
every test shard.
- The first CI run caught the existing Stop test trying to check a
disabled wake option for blocked legacy work. It now asserts zero wake
targets and retains the no-execution-before-reconciliation checks.
- Earlier E2E attempts exposed fixture problems: assignment started the
fixture before it was paused, and the classic interface had two matching
editor nodes. The fixture now enables dispatch after the hold exists and
scopes the final assertion to the task composer. No product assertion
was removed.

## Risks

Resume preview is a snapshot. The release route still checks current
status, assignment, and execution blockers. It does not bypass approval,
budget, or recovery gates. Pause, cancel, and restore keep their
existing affected-agent semantics. The test database override only
accepts a dedicated database name ending in `_test`; it runs migrations
before the suite and closes its clients afterward.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, browser
control, and code execution. The runtime does not expose a more specific
model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-25 21:49:52 -05:00
Devin FoleyandPaperclip a8e63f0e2a perf(server): avoid repeated delivery scans in GitHub review repair (#17669)
Resolve the exact delivery through its primary key before looking up stale-head cancellations. Backport the predicate from #17664 while preserving the current worker lifecycle and receipt checks.

Verified 372 selected tests, workspace typechecking, the server build, green CI, and an Apex 5/5 review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 17:42:54 -07:00
Devin FoleyandPaperclip 5a28f7acf8 fix(server): include restored monitor proof in assessment identity (#17665)
Include validated preserved monitor proof in the immutable assessment input so restored receipts or completion times can produce a new status decision at the same deadline. Preserve accepted results, earlier decisions, provider attempts, monitor deadlines, and idempotent replay.

Validate two baseline regressions, 201 targeted tests on current master, workspace typecheck, server build, full CI, and Apex 5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 16:06:36 -07:00
Devin FoleyandPaperclip 9f3a5456ac perf(server): bound run reads in authentication and conversation recovery (#17663)
Read compact run identity during authentication and compact turn state during conversation recovery. Preserve fresh checks, durable steering and reply evidence, locking, and complete stored payloads.

Validate 184 targeted PostgreSQL and lifecycle tests, workspace typecheck, server build, full CI, and Apex 5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 15:53:18 -07:00
Naeem AkhtarandPaperclip f9581e533e perf(server): find the native event cursor through the source seq index (#16140)
Find the contiguous native cursor in PostgreSQL while preserving gaps, late arrivals, source isolation, and event replay. Verify a long-history response budget against current master.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 15:35:36 -07:00
Naeem AkhtarandPaperclip 2a45fd95df Reduce newer-run history reads for attention (#15697)
Replace broad newer-run responses with bounded company-scoped existence probes. Read keyless history once per unique agent and preserve existing matching and millisecond rules. Keep all stored run data.

Verify with 31 real PostgreSQL attention tests, workspace typecheck, server build, response-budget counterfactuals, full green CI, and Apex 5/5 review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 14:15:03 -07:00
Devin FoleyandPaperclip f0b597172b perf(skills): verify bundled releases with current inventory digests (#17652)
Use current scoped database inventory fingerprints and the current on-disk expectation before reusing bounded snapshot verification. Preserve full validation, seeding and access checks when content, expectation or row existence changes.

Verified177 skill and authorization tests, workspace typechecking, server build, all green CI and exact-head Apex5/5. Prior staging release measured and saved Legacy data fingerprints preserved.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 13:37:11 -07:00
Devin FoleyandPaperclip 67aa87a93b perf(recovery): skip unchanged retained workspace evidence writes (#17650)
Guard the retained-evidence update with its existing JSONB comparison. Reuse the full first result when evidence is unchanged, while preserving new evidence delivery, source-bound retention, cleanup, and recovery holds.

Verified with 199 recovery and retention tests, full workspace typechecking, the server build, green CI, and exact-head Apex5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 12:58:50 -07:00
Devin FoleyandPaperclip 1949e726f8 perf(recovery): read only legacy terminal control fields under lock (#17646)
Select only runtime mode, status, complete result evidence and the delivery token from the locked legacy terminal-state read. Keep filters, row lock, guards, writes, full returned rows, source retention and recovery holds.

The synthetic regression reduces a 699,454-byte read to under 4 KB while preserving full stored and returned data. All CI checks, 236 combined tests, workspace typecheck and server build pass. Apex 5/5 after correcting a documentation contradiction, with no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 11:46:29 -07:00
Devin FoleyandPaperclip 82c998233c perf(recovery): omit unused launch payloads from reconciliation reads (#17645)
Project unused launch context and native execution input out of two preliminary recovery reads. Preserve full results, unknown controls, complete legacy profiles, current filters, native rejection and full locked terminalization.

Apex 5/5 with no unresolved comments and all CI checks green. Ten combined local suites pass (370 tests), plus workspace typecheck and server build. Hosted traffic effects require staging verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 11:06:22 -07:00
Devin FoleyandPaperclip 332e4c5df9 fix(native): settle completed results that retain a valid monitor (#17641)
Preserve a valid saved monitor after its successful source releases execution and checkout. Recheck accepted result identity, receipts, ownership and governance under the task lock. Notify the existing scheduler once and preserve accepted live and recovered replies.

Apex 5/5 after fixing its reply-presentation finding. All CI checks, 191 focused tests, 293 broader native tests, workspace typecheck and server build pass. Hosted effects require staging verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 10:47:02 -07:00
Devin FoleyandPaperclip 19ec787e5b perf(recovery): release native process identity without saved payload reads (#17622)
Keep locked process and terminal-state evidence plus JavaScript-compatible checkpoint presence. Preserve every host, provider, lease, coordinator and stop-receipt check without transferring saved prompts or checkpoint bodies.

53 focused tests, server typecheck, build, all 55 CI checks, and exact-head Apex 5/5 pass.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 10:06:27 -07:00
Devin FoleyandPaperclip b65897f9bf perf(recovery): read workspace contract without native launch prompts (#17621)
Project native input to the complete schema, binding, provider, and workspace evidence used by execution-blocker checks. Preserve absent and malformed fields and retain all saved inputs and recovery holds.

149 focused tests and 123 workspace-recovery tests pass. Server typecheck, build, all 55 CI checks, and exact-head Apex 5/5 pass.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 09:53:27 -07:00
Devin FoleyandPaperclip e302f2098f perf(wake-queue): omit unused native launch context from release reads (#17619)
Read native wake-release control context without saved launch payloads. Keep stored input, full legacy context, cancellation evidence, and release decisions. PostgreSQL response-budget regression, 785 lifecycle tests, typecheck, build, and all 55 CI checks pass. Apex reviewed the exact head at 5/5 with no findings.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 09:40:28 -07:00
Devin FoleyandPaperclip b792165ba4 Expire native runner continuity archives and quarantine (#17616)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runner sessions keep durable state under
`<instance>/runtime/paperclip-runner/durable-sessions/`
> - When a session starts a fresh provider session, the retired state is
moved into `<session>/continuity-breaks/<epoch>-<uuid>/`; when state
loses authority, it is moved into
`quarantine/<session>.<reason>.<epoch>.<pid>.<uuid>/`
> - Nothing ever removes either, and each can hold a full Codex home of
several gigabytes
> - A sleep-aware sweep already expires failover backup copies in the
same tree
> - This pull request extends that sweep to remove both kinds of
evidence after a retention window, while keeping anything recovery may
still read
> - The benefit is that native runner disk use stops growing without
bound

## Linked Issues or Issue Description

Bug report described in-PR (bug report template).

**What happened?**
On one instance, one day of native runs produced 20 continuity-break
archives totaling 4.8 GB, one session alone holding 4 GB. The quarantine
folder held 12 entries totaling 2.1 GB, the oldest a month old. Neither
is read again by normal operation (continuity-break archives are written
by `archiveContinuityState` and never read), and nothing deletes them.
They were removed by hand to free space and will regrow.

**Expected behavior**
Evidence older than a bounded window is removed automatically, as
failover backup copies already are, without removing anything a live
session or recovery path still needs.

**Steps to reproduce**
1. Run a native Codex session that hits a continuity break (fresh
provider session), or whose state is quarantined (for example
`identity_indeterminate`).
2. Inspect `durable-sessions/<session>/continuity-breaks/` and
`durable-sessions/quarantine/`.
3. Wait any length of time: the entries stay forever.

**Paperclip version or commit:** `master` at e335265568.
**Deployment mode:** any with native runner sessions.
**Agent adapter(s) involved:** native runner (Codex, and any provider
whose state lands in these folders).

## What Changed

-
`server/src/services/native-runtime/native-harness-backup-expiration.ts`:
the existing tick also retires evidence.
- **Continuity-break archives** (`<epoch ms>-<uuid>` under each
session): removed when older than the window. Age is the epoch in the
name, or the folder mtime when that epoch is in the future.
`archiveContinuityState` refuses to archive state with warm-transition
evidence, and no code path reads `continuity-breaks`.
- **Quarantine entries**
(`<session>.<identity_indeterminate|identity_mismatch>.<epoch
ms>.<pid>.<uuid>`): removed by the same age rule. Native cleanup source
archives (`<session>.identity_indeterminate.cleanup.<uuid>`) are
referenced from recovery history, have no epoch, and keep their original
inode's mtime, so they are always kept.
- Every entry is skipped while its session is executing, initializing,
warm, or closing in this process; this is checked again just before
removal. Symlinked entries and symlinked
`continuity-breaks`/`quarantine` folders are reported as `unsafe_path`
and never followed.
- Removal renames to `.expired-<uuid>` in the same folder and then
deletes; later sweeps finish interrupted deletions. An empty
`continuity-breaks` folder is removed.
- Archives, quarantine entries, and leftover deletions count toward
`maxEntries` and `maxRemovals`, and stop on `shouldContinue`. Paging
resumes inside a session's archives and then in quarantine, so a bounded
or dry-run batch always makes progress and never recounts a session's
backup.
- New result fields: `evidenceCandidates`, `evidenceCandidateBytes`,
`evidenceRemoved`, `evidenceReclaimedBytes`, and skip reasons
`recent_evidence` and `cleanup_archive`. The service also exposes
`noteEvidence(writtenAt)`.
- `server/src/services/native-runtime/native-runner-evidence-signal.ts`
(new): a process-local signal. `native-session-executor.ts` raises it
with the entry's epoch right after `archiveContinuityState` creates an
archive and after `quarantineRunnerdStateRoot` moves state into
quarantine; `server/src/index.ts` subscribes the sweep and unsubscribes
on shutdown.
- `server/src/config.ts`:
`PAPERCLIP_NATIVE_RUNNER_EVIDENCE_RETENTION_DAYS` (default `7`, minimum
`1`).
- `server/src/index.ts`: passes the window and logs a batch when it
found evidence candidates.
- `doc/recoverable-disk-cleanup.md`, `doc/DEVELOPING.md`: document the
variable and behavior.

**Sleep-aware background work**
(`rules/sleep-aware-background-work.mdc`):
- *Trigger:* each entry's own expiry deadline (its epoch plus the
window), plus a direct call from the executor when it writes new
evidence. No timer, interval, or polling loop is added; the pass runs
inside the existing sweep's tick and keeps its drain, warm-standby, and
shutdown handling.
- *Source of truth:* the filesystem entries themselves. After a restart
the first sweep reconstructs the next deadline from them.
- *Pending-work predicate / next deadline:* after a full pass the sweep
keeps the earliest due time of the entries it retained; ticks before
that time skip the evidence pass entirely (no readdir, no SQL). A busy
session, a failed removal, or a full batch makes it due on the next
sweep while that obligation remains.
- *Quiet condition:* no evidence, or only evidence not yet due. The pass
never queries the database, so retained evidence never causes recurring
SQL. Dry run reports each candidate once rather than every hour.

No code comments were added.

## Verification

- `pnpm exec vitest run src/__tests__/native-harness-backup-expiration`
(in `server/`, embedded Postgres): 52 passed. New cases:
- old archives and quarantine entries are removed, recent ones kept; the
empty `continuity-breaks` folder is removed;
- age comes from the name, with mtime used for a future epoch;
non-archive folders are ignored;
- a busy session keeps its evidence, including when it becomes busy
after the first check, and it is retried on the next tick;
- dry run reports candidates and bytes, changes nothing (including
leftovers), and reports each candidate once until new evidence comes
due;
- symlinked entries and symlinked evidence folders are skipped without
touching the target;
- an old quarantine entry is removed even when its session's issue is
still open; cleanup source archives are kept;
- the pass stays quiet until retained evidence is due, and a
`noteEvidence` signal schedules newly written evidence;
- inspected archives and leftovers count against `maxEntries`, and
leftover removal stops on `shouldContinue` and resumes next tick;
- a due pass stopped by drain before any session stays pending;
one-entry batches finish without recounting a resumed session;
- interrupted `.expired-*` deletions are finished; paging across ticks
with `maxRemovals`, in normal and dry run, with no recount of the backup
when resuming.
- `native-session-executor.test.ts`: quarantining a legacy root raises
the evidence signal with the epoch in the entry name.
- `native-harness-backup-expiration-config`: default, explicit, minimum,
and fallback parsing.
- **Mutation checks**, each failing at least one test: removing the age
check, the future-epoch fallback, either busy check, the busy retry
deadline, the recent-entry deadline, the due gate, the `noteEvidence`
update, merging the pass deadline, the entry and both folder symlink
checks, the dry-run return, the dry-run leftover guard, leftover
removal, the leftover budget check and its retry deadline, counting
archives against `maxEntries`, the cleanup-archive skip, the
empty-folder removal, the resume cursor, the per-archive budget check,
the resume skip of the backup pass, not charging a resumed session,
keeping the pass active across drain, and the executor's quarantine
signal.
- `pnpm exec vitest run config native-harness
native-workspace-sync-retention native-session-executor server-startup`
(in `server/`): 1272 passed on the previous commit; `config
native-harness native-workspace-sync-retention server-startup`: 450
passed on the head commit.
- `pnpm run typecheck` (in `server/`): pass.
- Not run: server build and boot smoke. The continuity-archive signal
call is exercised only by a runner integration test that needs the
runnerd binary.

## Risks

- **Quarantine as a fail-closed fence:** recovery
(`recoverQuiescentRunnerdState`) and failed-run retry
(`nativeFailedRunRetryStateIsSafe`) treat a quarantined entry for a
session as a reason not to bootstrap fresh. Once such an entry ages out,
a later run of that session can start a fresh provider session instead
of failing with `quarantine_owner_unverified`. This applies only after
the window, matching what backup expiry already accepts for closed work.
- **Signal coverage:** the signal is process-local. Evidence written by
another process sharing the same state directory is picked up when the
next retained deadline is due or after a restart, not immediately.
- **Legacy-path quarantine:** entries quarantined from legacy roots
carry a legacy digest, so the in-process busy check cannot match them;
they are removed by age only.
- **Remote sandboxes:** archives moved aside on a remote runner
(`<root>.continuity-break-<token>`) are outside this sweep.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code
with tool use (shell, file edits, tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-25 09:20:30 -07:00
Devin FoleyandPaperclip 34409a92c4 fix(native-runtime): stop failover backups from carrying throwaway provider state (#17618)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runner sessions on managed sandboxes keep a failover backup
of provider state, so a replacement sandbox can resume the session
> - Each backup copies the provider's runtime directory, with a
per-directory sha256 manifest, minus `excludeEntries`
> - Codex already excludes credentials and re-downloadable plugin
caches; OpenCode excluded nothing, and ACPX excluded only per-agent
credentials
> - OpenCode also leaked a temporary HOME on every launch, because
removing it failed on the read-only skill snapshot inside and the error
was swallowed
> - So every backup carried every earlier launch's HOME (with its npm
cache) plus re-installable plugin `node_modules`
> - This pull request removes those homes correctly, sweeps homes left
by earlier versions, and excludes the throwaway entries from OpenCode
and ACPX backups
> - The benefit is much smaller backups and live session directories,
while session history is still backed up and restored

## Linked Issues or Issue Description

Bug report described in-PR (bug report template).

**What happened?**
On one instance, current native failover backups totalled 24 GB across
about 300 sessions. For OpenCode sessions:
- `opencode/<session>/home-XXXXXX/`: 170 per-launch HOME directories
across 67 sessions, 9.3 GB. Each held `.npm` (about 111 MB of npm
cache), `.claude` (the skill snapshot) and `.local`.
- `opencode/<session>/config/opencode/node_modules`: 68 sessions, 4.3 GB
of OpenCode plugin installs.

One ACPX session also carried `acpx/<session>/data/pnpm/store` (229 MB
in the current backup, several GB in older copies).

Cause of the leaked homes: `startRuntime` in the OpenCode driver creates
a fresh `mkdtemp` HOME for each launch and seals the assigned skills
inside it (directories `0555`, files `0444`). `close()` and the
startup-failure paths removed it with `rm(..., { recursive: true, force:
true }).catch(() => undefined)`. That fails with `EACCES` on the sealed
directories, and the failure was ignored, so each launch left its HOME
behind.

**Expected behavior**
Per-launch homes are removed when the runtime closes, and backups hold
only state needed to resume a session. Caches that the provider
re-creates are left out.

**Steps to reproduce**
1. Run an OpenCode native session with at least one assigned skill on a
managed sandbox, over several turns or runs.
2. Inspect `<session>/` in the OpenCode runtime directory: one
`home-XXXXXX` per launch remains.
3. Inspect `failover-backups/current/opencode/<session>/`: the same
homes and `config/opencode/node_modules` are in the backup.

**Paperclip version or commit:** `master` at e335265568.
**Deployment mode:** any with native OpenCode/ACPX sessions on managed
sandboxes.
**Agent adapter(s) involved:** OpenCode (native), ACPX.

## What Changed

- OpenCode driver
(`packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.ts`):
- `removeIsolatedHome` unseals the skill snapshot with the existing
`releaseMaterializedNativeRuntimeSkills`, then removes the HOME. Close
and every startup-failure path use it. It only unseals when `.claude` is
a real directory, not a symlink.
- Per-launch homes are now created under `<session>/homes/home-XXXXXX`,
so one exact path can be excluded.
- When a session opens, `home-*` directories left at the session root by
earlier versions are removed. History is not in them: OpenCode keeps it
in `XDG_DATA_HOME` (`<session>/data/opencode/opencode.db`), and HOME
only holds caches, state locks and the skill copy.
- Exports `opencodeRuntimeSessionDirectoryName` (the existing
session-directory naming) and `OPENCODE_ISOLATED_HOMES_DIRECTORY` for
the server.
- `resolveNativeHarnessPersistenceProfile`
(`server/src/services/native-runtime/native-session-executor.ts`):
- OpenCode excludes `<session>/homes` and
`<session>/config/opencode/node_modules`.
- ACPX (every agent) excludes `acpx/<session>/home/.npm` and
`acpx/<session>/data/pnpm/store`, in addition to the existing per-agent
entries.
- `excludeEntries` are matched as exact relative paths (wildcards are
rejected), so the session directory is computed in the same way as the
existing ACPX entries.
- Exposes the two new exports through the server's runner vendor shim.

No code comments were added.

## Verification

- `pnpm exec vitest run src/drivers/opencode
src/drivers/runtime-context-materializer` (in
`packages/paperclip-runner/`): 108 passed. The 1 failure, `uses provider
structure rather than prose length to select a post-tool answer`, also
fails on unmodified `master`.
- The existing skill-tree test, with and without a startup retry, now
asserts that homes are created under `homes/` and that `homes/` is empty
after close.
- New test: a sealed legacy `home-*` at the session root is removed when
the session reopens, and `data/opencode/opencode.db` is kept.
- `pnpm exec vitest run src/services/native-runtime/
src/__tests__/native-harness-backup-expiration.test.ts` (in `server/`):
5080 passed. `plugin-runner-probe` could not run locally because of a
cargo manifest-path problem in this environment; it is unrelated to this
change.
- Profile tests cover the new OpenCode and ACPX exclusions, including a
session id that needs sanitizing.
- New checkpoint/restore round-trip tests for OpenCode and ACPX. They
use `syncRemoteRunnerDirectoryOut` for the backup and
`stageRemoteRunnerDirectory` for the replacement. Excluded caches are
absent from both the backup and the replacement, while history, config
and workspace files are restored and the live directory is unchanged.
- **Mutation checks**, each failing its test:
- removing the skill unsealing (homes leak again; 3 runner tests fail);
  - removing the legacy sweep (1 test fails);
  - emptying the OpenCode/ACPX exclusion lists (4 server tests fail).
- Checked by hand against the bundled OpenCode 1.18.34 binary, with the
driver's environment:
- launching with a fresh HOME and no `config/opencode/node_modules`
still lists earlier sessions;
  - OpenCode reinstalls the plugin dependencies on launch.
- `pnpm run typecheck` in `server/` and `packages/paperclip-runner/`:
pass.
- Not run: server build and boot smoke.

## Risks

- **Rollout:** existing backups still contain legacy `home-*`
directories. After a restore, the sweep removes them when the session
opens, and the next backup no longer carries them.
- **First launch after restore:** OpenCode reinstalls its plugin
dependencies and npm repopulates its cache. This needs network access to
the npm registry, the same as a brand-new session.
- **Crash leftovers:** a HOME left behind by a hard crash (the runner
killed before `close()`) stays under `homes/` until the session
directory is removed. It is still excluded from backups. Homes under
`homes/` are not swept on open, to avoid touching a HOME that a
concurrently running runtime might still use.
- **pnpm store:** dropping the pnpm store from ACPX backups means the
next `pnpm install` in a replacement sandbox downloads packages again.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code
with tool use (shell, file edits, tests, and running the real OpenCode
binary to check what state it reads on resume).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-25 08:51:39 -07:00
Devin FoleyandPaperclip 7ebf98eafd fix(recovery): avoid duplicate terminal run status notifications (#17614)
Retain pending status deliveries when Native or legacy terminal recovery projects the same state. Preserve completion time on unchanged Native terminal status. Real state, metadata, error, explicit patch, and retained-file changes still notify. The publisher acknowledgement, accepted results, retry deadlines, recovery holds, ownership checks, and workspace cleanup remain intact.

Correct the queued Retry browser fixture race by releasing its old-process guard at the exact Retry request, requiring a successful response, and retaining fixture cleanup after failure.

Validated by 159 focused tests, 653 wider Native recovery tests, six real Chromium/PostgreSQL browser cases, five queued Retry repeats, build/typecheck, exact-head Apex 5/5, and green required CI. The original detached-button CI timeout is documented in the PR. The additional full local repository Vitest command is still running and is not claimed complete.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 08:22:53 -07:00
Devin FoleyandPaperclip bf9a8551c3 perf(issues): filter queued comment wakes by task (#17613)
Apply the existing typed task identity filter in PostgreSQL before returning queued-comment wake controls. Preserve queue selection, ordering, payloads, and current run ownership checks.

All 84 route tests, build, typecheck, exact-head Greptile 5/5, and required CI pass. The old source fails the new deferred and queued byte budgets; corrected reads preserve the complete response and all saved rows.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 07:51:28 -07:00
Devin FoleyandPaperclip a09ef344b3 perf(agents): avoid duplicate configuration reads (#17612)
Reuse one complete company roster for list responses and narrow other-agent eligibility checks to their five consumed fields. Preserve full selected-agent configuration, terminated-manager evidence, current spend, authorization, and fresh reads.

Verified with 170 affected tests, four rejected regression mutations, build and typecheck, exact-head Greptile 5/5, and green required CI.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 07:38:18 -07:00
DottaandPaperclip aec86c794b fix(runner): preserve native work through controller outages (#17594)
Keep remote native work under its authenticated connection lease through controller outages. Hold durable ACPX tools through reconnection, align the native Claude HTTP MCP client budget, and wait for the recovered handler before dispatch. Preserve Stop, cancellation, exact identity and artifact checks.

Verified with real-process regressions and a real Claude Opus 5.5 Daytona workflow: 125 seconds offline during useful work, original process and lease retained, no failed native tool calls, queued follow-up completed, independently verified ZIP and committed workspace receipts. All current-head CI gates pass and Greptile is 5/5 with all findings resolved.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 09:01:24 -05:00
Devin FoleyandPaperclip 3536b3649f Read only task creation provenance fields (#17604)
Narrow saved-run provenance reads to current identity, scope, and typed legacy bindings. Preserve binding precedence, current viewer checks, and complete stored runs. Real PostgreSQL and route tests pass 23 cases; legacy full-context and type-coercion regressions are caught. Exact-head required CI is green; Greptile is 5/5 with Low impact and no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 06:30:25 -07:00
Devin FoleyandPaperclip 8c6bd5a83b perf(runner): skip wake payloads in ordinary authority reads (#17600)
Omit saved wake payloads from ordinary unlocked Native authority reads. Load the current full wake for mail restrictions. Preserve stored rows, locked mutation reads, and current authorization predicates.

Verified with full build and typecheck, 2,360 affected tests, real PostgreSQL byte and response checks, counterfactual performance and mail authorization failures, complete exact-head CI, and Greptile 5/5 Medium with no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 05:36:43 -07:00
Devin FoleyandPaperclip 2e47c42fad perf(runner): omit unused launch context from tool authority reads (#17591)
Omit nine unused named launch values from the unlocked Native authority query. Preserve complete wake payloads, other context, result receipts, stored input, and locked execution and mutation readers.

Verified with full build and typecheck, 2,359 affected tests across 20 suites, real PostgreSQL byte/retention/cancellation checks, a counterfactual, complete exact-head CI, and Greptile 5/5 Medium with no unresolved comments. An existing cleanup scheduling test passed on CI retry after its suite passed locally with both base and branch source.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 04:21:50 -07:00
Devin FoleyandPaperclip bd77ffe78d perf(connections): scope profile rule and binding reads (#17589)
Read current company/agent profile bindings and rules matching the exact catalog rows being evaluated. Preserve all five selector kinds, null application matches, exclusions, and the existing access evaluator without a cross-call cache.

Verified with full build and typecheck, 652 affected tests across ten suites, all 18 strengthened effective-assignment tests, exclusion mutation and byte counterfactuals, complete exact-head CI, and Greptile 5/5 Medium with no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 04:12:14 -07:00
Devin FoleyandPaperclip b7dc594bfa perf(recovery): narrow saved-message execution-wait reads (#17588)
Read current ownership, startup, cancellation, task-binding, and adapter evidence without transferring unused saved profiles, launch context, or output. Preserve normal continuation admission, recovery holds, and locked mutation reads.

Verified with full build and typecheck, 537 affected tests across 12 suites, a counterfactual real PostgreSQL byte regression, complete exact-head CI, and Greptile 5/5 Medium with no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 03:54:59 -07:00
Devin FoleyandPaperclip 35d5b0686a perf(runner): narrow current tool authority reads (#17587)
Select the current run fields used by Native tool authority without transferring unused saved profiles or output. Preserve full context, receipts, authorization predicates, and locked mutation reads.

Verified with full build and typecheck, 2,358 affected tests across 20 suites, a counterfactual real PostgreSQL byte regression, complete exact-head CI, and Greptile 5/5 Medium with no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 03:45:25 -07:00
Devin FoleyandPaperclip c4c4f7eecb perf(environments): reuse provider schemas after content validation (#17581)
Add a separate schema-only provider lookup with current plugin identity and SHA-256 content validation in one statement. Retain bounded schema definitions, return independent objects, and preserve current full driver and secret access paths.

Build, repository typecheck, 485 affected tests, the supported-host database tests, and the complete required CI matrix passed. Greptile scored this exact head 5/5 at medium impact with no unresolved findings.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 03:20:45 -07:00
Devin FoleyandPaperclip 3f36947e0c perf(realtime): batch task and run authorization per event (#17579)
Use one fresh authorization batch for each live event task and run binding. Preserve current membership, grant, company, and payload disclosure checks.

Build, typecheck, 70 affected local tests, and the complete required CI matrix passed. Greptile scored the exact head 5/5 at medium impact with no unresolved findings. The separate full local suite remains in progress.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 02:43:17 -07:00
Devin FoleyandPaperclip 50797ea03c perf(issues): narrow queued comment wake display reads (#17570)
Read only typed queue display inputs from saved wake payloads and only runtime mode from queued runs. Preserve complete rows for locked edits, discard, steering, and interruption.

Validation: 93 focused route and policy tests, full build and repository typecheck. PostgreSQL fixtures preserve identical queue responses and saved input with less than 4 KB of read values; restoring the full wake read fails at about 1.8 MB. Exact-head Greptile 5/5 (medium), all CI checks green, and zero unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 00:41:14 -07:00
Devin FoleyandPaperclip 7494b8dd4c perf(auth): batch current run history read decisions (#17569)
Reuse the existing authorization batch owner for current run-history read decisions after company-scoped existence checks. Preserve missing-task denial, per-request freshness, and the decide-only adapter path.

Validation: 97 focused tests, full build and repository typecheck. A real PostgreSQL fixture returns identical decisions with 4 queries instead of 327; the original path fails the query-bound regression. Exact-head Greptile 5/5 (medium), all CI checks green, and no unresolved comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 00:35:07 -07:00
Devin FoleyandPaperclip 0d73505098 perf(git): narrow operation identity snapshot reads (#17567)
Select only the run context used by GitHub operation credential checks. Preserve the default complete identity capture for Muse and other consumers, current trust checks, grants, and saved run evidence.

Validation: 72 focused GitHub, identity, Slack, chat-project, and project-route tests; full build and typecheck; exact-head Greptile 5/5 (medium), all CI checks green, and zero unresolved comments. Real PostgreSQL tests prove current revocation and response-size bounds. No live external GitHub push was performed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 00:31:41 -07:00
Devin FoleyandPaperclip f978a521ff perf(heartbeat): narrow workspace operation access reads (#17565)
Read only the current run access fields and typed workspace binding for workspace-operation polling. Keep the current privacy decisions, operation checks, and complete run storage.

Verified focused run-state, live-event, and real database privacy-route tests, full build and typecheck, all CI checks, and exact-head Greptile 5/5 with no unresolved threads.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 00:21:15 -07:00
Devin FoleyandPaperclip 75dbd3050b perf(git): filter managed identity connection reads (#17563)
Apply the existing exact GitHub source marker predicate in the company-scoped connection query. Keep the JavaScript guard and every current install, grant, consent, identity, and credential check unchanged.

Validation: 74 focused tests, full build and typecheck, exact-head Greptile 5/5 (medium), full green CI, and no unresolved threads. Both marker locations, malformed JSON types, changed/disabled connections, and retained unrelated configuration are covered. The unfiltered counterfactual fails both row-count regressions.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 00:09:56 -07:00