Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-03 17:51:58 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,165 Commits 1,392 Branches 2,228 Tags
82c033ed127d07862e6765aa4b69a3cf339bde81
Commit Graph
2968 Commits
Author SHA1 Message Date
DottaandPaperclip 82c033ed12 test(e2e): cover uploaded input and personal inbox triage (#18643) 
## Thinking Path

> - Paperclip lets people assign work to agents and manage their tasks.
> - Product E2E journeys verify the saved outcome through the browser
and normal APIs.
> - Uploaded input and personal inbox actions need repeatable coverage.
> - An agent answer must match the input file, and an inbox action must
retain its saved state.
> - This change adds two journeys with independent evidence checks.
> - A blocked fixture action must use the normal human-owned unblock
contract.
> - Registration stays separate from deployed qualification and daily
admission.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Product E2E coverage for file attachments, saved task documents, and
personal inbox read and archive actions.

**Current behavior**

The journey catalog lacks an uploaded-input calculation and a persisted
personal inbox triage path. A successful agent run alone cannot prove
either outcome.

**Proposed behavior**

Upload a generated CSV through the task composer. Require an exact saved
answer and unchanged input bytes. Read and archive one owned task
through the inbox, reload each state, and verify that a new Blocked
event brings it back without agent work.

## What Changed

- Add a file-input journey with a random reference held only in the
uploaded CSV.
- Check exact result values, document authorship, native run binding,
original input bytes, and persistence after reload.
- Add a browser inbox journey with an independently authenticated human
author, saved read and archive state, unchanged work status, and
activity resurfacing checks.
- Reject wrong, missing, duplicate, and foreign evidence in assertion
controls.
- Register both cases and document their scope and privacy rules.
- Supply a current-human unblock owner and action for the archived-task
fixture. Test that exact payload through the production route and retain
the bare-blocked422 negative control.
- Include the fixture constructor in the source definition hash and both
new cases in the exact catalog assertion.
- Reject collected inbox receipts with unexpected runs, providers,
models, or native/legacy execution modes.

## Verification

- The frozen source is `cdb6659fb4e4b67e9321bc3fd963cf11e9b8b2f3`. It
includes one fixed master integration at
`fd0d61b8e329efc89ff541dc2cac44e7c7a37f78`. The single map conflict
retains both coverage paragraphs.
- The production HTTP route rejects the original bare blocked payload
with422 and accepts the exact human-owned fixture payload with200.
Neither control wakes an agent. The full related route file passes
all200 tests.
- The frozen current head passes recursive workspace typecheck, runner
typecheck, full build, map checks, and diff checks. Its full related
journey/catalog suite passes723 tests, the production route suite
passes200, and the separate Node support suite passes347. The retained
contract file passes all38 tests. Fresh Greptile review is 5/5 on the
frozen current head. Current-head hosted CI passes after one authorized
failed-jobs-only recheck. The original Runner close test failed; the
unchanged, properly cargo-prepared focused local control passed in2.99s,
and the hosted recheck then passed. The cause remains unknown; no Runner
code, test timing, or assertion changed. The original hosted diagnostic
and the initial invalid local invocation without required fake binaries
remain recorded. The ordinary local full-test run is in progress.
- The unrestricted local runner support run on the fixed-master
integration is not green:5993 tests passed,12 failed,3 skipped. The
unchanged failures cover onboarding cleanup expectations, Pi metadata,
workflow-security expectations, and an expired test allowance. An
untouched743 baseline reproduces the same non-PG failures; their
source/test blobs also match pre-feature40b807. This change does not
extend an allowance or alter paid admission. The
original5983-pass/15-failure run remains recorded. Restoring the local
Postgres package's own library aliases resolved its two startup
failures. The feature-related catalog omission is corrected.
- The uploaded-file case passed its first complete deployed QA attempt:
one configured native run, all six independent checks, original
evidence, complete cleanup, and no active operations.
- The first inbox attempt stopped at the agent comment execution-context
guard. That guard stays intact. The fixture uses an approved,
independently authenticated QA human.
- The next original inbox attempt verified protected-secret retrieval,
human attribution, and session closure, then failed its unread
prerequisite. The product unread defect was repaired in #18670.
- The fresh deployed inbox attempt passed external-human activity, saved
read state, archive persistence, and unchanged task status after the
product rollout. It then stopped at the fixture's bare blocked PATCH
before any resurfacing assertion. The response status was discarded;422
is an explicit contract inference independently reproduced through the
production route. That failed attempt remains retained. A fresh full
inbox qualification on the current frozen head now passes all six checks
and all five phases; the normal human comment receipt, closed human
session, zero agent work, complete cleanup, and absence of active
operations are verified in the original evidence. Daily admission
remains a separate coordinator action.
- The retained-result review gap failed a negative receipt control
before the repair. All38 retained journey contract tests now pass. The
two old unread advisory threads are resolved only after the fresh full
deployed inbox pass.
- Original failures remain recorded, including the earlier unchanged
environment-runtime assertion and unchanged runner/occupied-port startup
failures. Closed operation codes and failure screenshots retain the
failing step without raw browser errors. Actual retry qualification
remains separate from source tests.

## Risks

The target must supply the declared external-human fixture. Inbox
control selectors may differ across deployed UI revisions. File
processing depends on the configured native agent. Exact saved-state and
source checks must remain required when those dependencies fail. The
change adds test journeys and does not change production behavior or
schemas.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code execution, and browser
tools. The runtime does not expose a more specific model identifier or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (relevant suites; broader
baseline failures are listed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 18:12:16 -06:00
Nicky Leach 3a7bb4dac6 perf(runner): activate shared retained startup context (#18405) 2026-11-02 15:12:57 -08:00
DottaandPaperclip 58308a3e63 perf(apps): scope sidebar review counts to one connection (#18688) 
## Thinking Path

> - Paperclip helps people manage AI agents and their governed app
access.
> - The Apps sidebar displays a Review badge for one connection.
> - It polls the full company attention service every 30 seconds to get
two counts.
> - That service also loads every profile, rule and active catalog
entry.
> - This pull request adds an authorized connection read for only those
counts.
> - The same badge now avoids unrelated profile evaluation and database
result decoding.

## Linked Issues or Issue Description

**Current behavior**

The contextual sidebar polls company-wide attention. It needs only one
connection's signed pending action count and whether any catalog entries
are quarantined.

**Proposed behavior**

Both sidebar variants poll a connection-scoped count endpoint with
existing connection read authority. The full attention API stays
unchanged.

**Reason and benefit**

The retained 200-profile fixture performs seven service queries and
returns 80,115 rows for this badge. The new reader performs one query
and returns one row. This reduces local service CPU and avoids
unnecessary decoded profile/rule data. Related merged evaluator fixes
are already in the baseline: Refs #18673, Refs #18681. A search found no
existing PR for this scoped count reader. ROADMAP.md contains no
duplicate planned work.

## What Changed

- Add the narrow shared DTO, typed client, guarded REST/OpenAPI route
and catalog-owned reader.
- Count independent aggregates with both company boundaries and exact
signed-argument semantics. Keep archived counts at zero.
- Update both contextual sidebar variants. Keep the exact badge, 30
second interval and existing attention-prefix invalidation.
- Add PostgreSQL differential tests, authenticated HTTP tests, sidebar
tests, a repeatable benchmark and access documentation.

## Verification

- Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, including both
merged evaluator fixes. Service treatment: `0827f7d0b3`; route/fixture
refinement: `230c42ee15`; final local existence gate: `390a218254`.
- Passed: focused PostgreSQL differential test; assembled HTTP test with
board session and agent JWT; 14 sidebar component tests; token gates;
map checks; diff check.
- Fixture: 200 profiles, 76,000 rules, 1,900 active tools, 695 removed
references, 12 agents, a sibling connection and 2,000 unrelated
historical invocations. Complete normalized two-count results are equal
on both paths.
- Quiet local measurement: seven alternating warm pairs after two warmup
pairs, Node 24.22.2, macOS, real disposable PostgreSQL. Legacy/new
service CPU median: 213.891/1.680 ms; ranges:
209.329–228.552/0.515–6.137 ms. Wall median: 156.867/1.248 ms; ranges:
151.739–162.289/1.115–1.386 ms. Queries: 7/1. Returned rows: 80,115/1.
- Run `cd server && node --expose-gc --import
./node_modules/tsx/dist/loader.mjs
scripts/benchmark-connection-review-counts.ts`. See
`doc/connection-review-counts.md` for scope and failure history.
Byte-volume accounting runs outside timing; no HTTP payload reduction is
claimed.
- Initial raw scalar SQL failed with ambiguous columns. Typed subqueries
fixed it and passed real PostgreSQL equality. Initial fixture agents
lacked permissions and correctly received 403. The final fixture grants
explicit authority and tests revocation. An initial benchmark included
byte serialization in timing; the reported run excludes it.
- Passed required local `pnpm -r typecheck` and `pnpm build`. The
initial sandbox typecheck could not open the tsx IPC socket; the
escalated rerun passed. Full monolithic `pnpm test:run` was not run
locally: repeating a 15–25 minute suite serially across both performance
PRs adds no distinct evidence. Broad suite acceptance uses final-head
CI, which runs the same general/serialized test partitions and the
native Runner lane. Final-head CI is in progress.
- Passed built-static Chromium journey on an owned local-trusted server
and disposable PostgreSQL: Review 3 appears, only scoped counts are
requested by this sidebar, resolving requests/clearing quarantines then
reloading removes the badge, and archived counts are zero. The first ad
hoc launcher omitted required decision-service options and failed at
startup; using the normal no-op wake callback fixed that harness.
- Greptile identified an invalid resolved-request fixture label.
Corrected `denied` to the declared `rejected`, then reran the real
PostgreSQL differential test. The count SQL and non-pending exclusion
are unchanged.
- The first final-head CI failed the static cross-tenant
existence-oracle guard because the new explicit resource company
assertion had no nearby `hasCompanyAccess` gate. Existing management
admission already rejects foreign resources with 404. The route now also
follows the standard local two-step gate. The unchanged guard plus
authenticated HTTP negative cases passed (3 tests). No allowlist or test
was weakened.
- These local service measurements do not prove hosted performance. No
deployment is included.

## Risks

Low risk. New read admission reuses existing current-authority checks.
The old attention API is unchanged. Counts still require database work
over relevant pending/catalog rows. No cache, schema migration or
telemetry is added. Roll back this PR if the badge differs or scoped
reads deny previously permitted users.

## Model Used

OpenAI `gpt-6.1-sol` (Sol 6.1), with reasoning, tool use and code
execution. The host does not expose the configured context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 17:12:22 -06:00
commitperclip[bot] fd0d61b8e3 fix: Compare native finaliser lease expiry at millisecond precision when releasing an expired owner (#17214) 
> [!NOTE]
> Opened by Prospector's automated issue-fix pipeline. A maintainer
reviews and decides; nothing here merges on its own.

## Thinking Path

- Issue #16715 reports a defect in paperclipai/paperclip.
- Root cause, as the fixing agent read it: Lease renewal writes
'lease_expires_at' from the database clock ('now() + interval'), so the
stored value has microseconds. 'agentExecutionsHaveStopped' reads that
value into a millisecond JS 'Date' and then releases the row with an
exact 'eq' on it. The update never matches, so the stop check returns
false on every tick and the agent stays in 'pausing'.
- This pull request: Compare native finaliser lease expiry at
millisecond precision when releasing an expired owner

## What Changed

- `server/src/services/agent-execution-stop.ts`: The release update now
truncates the stored lease expiry to milliseconds before comparing it
with the value it read, using the same 'date_trunc('milliseconds', ...)
= ...::timestamptz' pattern as 'chat-channels/activity.ts'. The other
guards stay the same: owner, generation, phase and the null result. This
was the only exact-match comparison on a lease expiry in 'server/src'.

## Verification

- Reproducing test(s), added in this PR:
`server/src/modules/agent-lifecycle/__tests__/lifecycle.test.ts`
- They fail on the base commit and pass with the fix, each twice in a
fresh sandbox.
- 3 independent attempts wrote 3 reproductions; this fix passes every
one of them.
- The repository's compile command passes with the fix applied.
- The full test suite passes except for the 10 file(s) that already fail
on the base commit.

## Risks

- Written by an automated pipeline, not a person: review it as you would
an outside contribution.
- Risk tier of the touched paths: 2.
- The scope reviewer found it also changes, beyond the report:
agentExecutionsHaveStopped for an expired finaliser row whose
lease_expires_at is already millisecond-precise (written from a JS Date)
- The scope reviewer found it also changes, beyond the report: Finaliser
row whose lease was renewed or taken over between the select and the
update
- The scope reviewer found it also changes, beyond the report: Rows with
null leaseExpiresAt

## Model Used

- codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code;
OpenAI, via the Codex CLI), writing and checking the change in an
isolated sandbox.
- opus (Anthropic Claude, via Claude Code), reviewing it independently.

## Linked Issues

Fixes #16715

Other pull requests that name #16715:
- #17204 (open): fix(heartbeat): stop loading full run rows and stale
pids when pausing an agent
- #16745 (open): fix(agent-lifecycle): release an expired native
finalizer lease at millisecond precision

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes
#' / 'Refs #' OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip'
URLs)
- [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...')
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

<!-- prospector:issue-fix v1 issue=16715 base=c7ea5c0216b8
report=1a4e13ecd7c6cdb6 -->

Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com>
 2026-11-02 15:01:52 -08:00
DottaandPaperclip 554a770c6f perf(apps): scope connection profile configuration reads (#18687) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Apps Permissions page reads saved access profiles for one
connection.
> - The page fetched every company profile, rule, and binding.
> - It used only the app profile and eligible task-granted profiles for
that connection.
> - This pull request adds a saved configuration read owned by the
profile service.
> - The route keeps current connection and per-profile management
checks.
> - The benefit is fewer database rows, less Node work, and a smaller
response for the same displayed configuration.

## Linked Issues or Issue Description

**Subsystem affected**

Server profile configuration reads and the Apps Permissions and Review
data caller.

**Problem or motivation**

One connection page loaded and serialized unrelated company profiles. In
the local fixture, this produced a 32.35 MB response with 76,001 rule
rows.

**Proposed solution**

Select the displayed connection's app profile and exact eligible
connection-intent profiles before loading their entries and bindings.
Return an explicit configuration DTO with complete selected arrays. Keep
summary and pending-review work on the existing full API.

**Alternatives considered**

Filtering the full response in the browser retains the database and Node
cost. Truncating the full API would break other clients. Filtering
entries to the connection would lose selectors and bindings needed by
saves.

**Roadmap alignment**

This improves the existing governed Apps capability. It does not add a
new access policy. Related merged work: #18673 and #18681. Related
harness cleanup: #18683. Public PR searches found no duplicate
connection configuration projection.

## What Changed

- Add `ToolProfileConfiguration` and `GET
/api/tool-connections/{connectionId}/profile-configuration`.
- Select `app:<connectionId>` at any status and exact active, correctly
bound `connection_intent` profiles. Run the existing per-profile
visibility filter before configuration queries.
- Preserve complete ordered entries and bindings. Keep all save
behavior, empty-binding install fallback, and the existing full profile
API.
- Switch AppDetail and AgentConnectionAccess to the configuration DTO.
Use a distinct cache key below the existing profiles prefix so mutation
and reconnect invalidations still refresh it.
- Add real PostgreSQL differential and authenticated HTTP tests, UI
caller coverage, a reproducible benchmark, and access-map documentation.

## Verification

Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, after both #18673
and #18681. Final head: `b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22`.
Current implementation and reproduction details are in [the performance
note](https://github.com/paperclipai/paperclip/blob/b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22/doc/performance/2026-11-02-connection-profile-configuration.md).

- AppDetail: 79 tests passed. Existing access/action saves, install
fallback, grant removal, errors, and refresh cases use the new caller.
- Real PostgreSQL differential: passed. Full ordered selected
configuration equals the old full-list selection. Cases cover extra
bindings, sibling selectors, wrong source/key/status/binding, null and
non-string agent metadata, fresh mutations, empty app bindings,
visibility filtering, and company scope.
- Production HTTP authentication: API key and run JWT cases passed.
Hidden personal accounts omit spanning profiles. Private connection
reads fail. Restored ownership restores visibility. Revoked management
grants deny fresh calls. Final UID canonicalization and foreign-company
404 assertions passed on the current head.
- Map structural check, token gate, and `git diff --check`: passed.
- Required local `pnpm -r typecheck` and `pnpm build`: passed on the
current head. `pnpm test:run` was started once. Its first chat shard
passed (337 tests, 151.87 seconds); the second shard was intentionally
interrupted after the complete final-head CI suite passed. This partial
local attempt is not a full local test pass. The same stable-runner
general and serialized partitions and Runner lanes passed in [final-head
CI](https://github.com/paperclipai/paperclip/actions/runs/41538805231),
including aggregate `ci / verify`. No local assertion failures were
observed before interruption.
- Final-head focused acceptance: four files, 85 tests passed. Full token
gates, maps, and module boundaries passed.
- Actual browser acceptance passed on the current checkout with isolated
real PostgreSQL and a local trusted board. An action saved as Off
persisted after reload. Both app agent bindings stayed intact. Removing
an agent-only task grant persisted after reload. Requests used only the
narrow profile endpoint. No browser exceptions occurred. No responses
were mocked. No provider calls or hosted changes were made. The helper
stopped the owned runtime and database after the check.
- Final exact-head checks: 54 success, one clean neutral Paperclip
Review, three skipped, zero pending or failures. Legacy Snyk status:
success. No inline review comments.
- Greptile reviewed exact head
`b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22`: 5/5, no findings, completed
successful check.

Five alternating unprofiled pairs ran with real PostgreSQL on
macOS/arm64, Node v24.22.2, and forced GC. The fixture has 210 profiles
(200 heavy profiles plus ten eligibility cases), 76,001 rules, 1,900
active tools, 695 removed tools, and 12 agents. The selected
configuration has two profiles and 1,391 rules. Each pair asserts
complete ordered selected DTO equality.

| Measurement | Full list | Configuration read |
| --- | --- | --- |
| Service wall ms, median [min, max] | 200.927 [193.319, 213.390] |
6.414 [5.508, 6.973] |
| Node CPU ms, median [min, max] | 262.751 [246.727, 271.573] | 27.525
[22.576, 29.308] |
| Select queries / returned rows | 7 / 78,139 | 3 / 1,397 |
| Actual response JSON bytes | 32,353,534 | 592,233 |
| Serialization wall ms, median [min, max] | 90.714 [88.715, 99.304] |
1.644 [1.637, 1.745] |
| Serialization CPU ms, median [min, max] | 90.644 [88.569, 96.778] |
1.654 [1.637, 1.807] |

Reproduce: `node --expose-gc --import
./server/node_modules/tsx/dist/loader.mjs
server/scripts/benchmark-connection-profile-configuration.ts 5`.

The benchmark measures the service and actual response JSON separately.
It excludes HTTP and authorization work through an identity visibility
callback. Node CPU does not include PostgreSQL CPU. No hosted or
foreground-under-load improvement is claimed.

Preserved failures: offline install lacked a locked tarball; frozen
online install passed. Sandbox localhost and tsx IPC restrictions
blocked initial DB/typecheck attempts; permitted retries use the same
checks. The first HTTP suite needed the unbuilt plugin SDK; its owning
build helper fixed setup. Shared typecheck caught a missing barrel
export, now fixed. A broad typecheck inherited the host's unpinned
nested pnpm executable; the owned stalled process was stopped and
restarted with pinned pnpm 9. The complete current-head CI suite and
aggregate verify passed. The separate Paperclip agent review published
5/5 with no findings at this head and completed with a clean neutral
conclusion. An added foreign-company assertion expected 403; the
existing guard correctly returned 404. Only the test expectation
changed. The first real browser attempt saved and reloaded the action
correctly, but grant deletion returned 422: the stress fixture had a
company binding, which makes that profile the company default. The UI
showed the existing deletion error. The second browser seed used the
production agent-only intent shape and passed removal and reload. This
does not change the stress fixture or product deletion gates. Initial
external browser script attempts also needed an ESM extension and the
correct dependency path. Logs, first-failure screenshots, traces, and
result files are retained in the local evidence directory.

## Risks

- SQL selection must stay aligned with connection-intent eligibility.
Differential fixtures cover the exact current predicate.
- A selected connection can still have many rules. This change preserves
all selected configuration.
- Per-profile authorization still reads current selected resources and
can add work. Connection visibility does not bypass that check.
- No migration, default-allow, effective-access, policy-review,
provider, or deployment behavior changes. The old endpoint remains
available. Rollback restores the old AppDetail caller and removes the
additive endpoint.

## Model Used

OpenAI Codex, `gpt-6.1-sol`, with code execution and repository tools.
The host does not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:55:56 -06:00
DottaandPaperclip 00fea0da52 refactor(perf): reuse database test setup for profile benchmark (#18683) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tool profiles govern the tools that agents can use.
> - Profile listing and app attention shared nested selector scans that
consumed the server CPU.
> - #18681 and #18673 replaced those scans with indexes for each
computation.
> - Their performance benchmarks need consistent setup and reproducible
per-service measurements.
> - This pull request uses the shared helper entry point and retains the
exact measurement harness and results.
> - Reviewers can inspect both the CPU savings and the limits of the
evidence.

## Linked Issues or Issue Description

Refs #18681 and #18673.

**What existing behavior does this improve?**

The consistency of benchmark setup and the reproducibility of evidence
for the two merged CPU fixes.

**Current behavior**

The profile-review benchmark imports the database starter directly. The
matching benchmark uses the shared server test-helper entry point. On
the current base, that helper re-exports the database starter, which
already supports `PAPERCLIP_TEST_POSTGRES_ADMIN_URL`. The previous
committed benchmark measures both services together; the per-service
measurements and separate HTTP/Inspector runs lacked a committed
harness.

**Proposed behavior**

Use the same test-helper entry point in both benchmarks. Retain the
compile, per-service timing, Inspector, and HTTP-probe programs with
their exact commands and raw samples. Preserve the existing embedded and
admin-server database behavior.

**Reason and benefit**

A shared entry point keeps benchmark setup consistent. The committed
harness lets reviewers reproduce and compare each real owning service
before and after the merged fixes. The failure-path checks verify that
result-write errors still release the database and probe resources.

**Breaking changes**

None. This change affects benchmark setup and documentation.

## What Changed

- Import the database starter from the repository test helper in
`benchmark-profile-review-projection.ts`.
- Record separate CPU and elapsed measurements for profile listing and
app attention.
- Retain the actual compile, per-service measurement, Inspector capture,
and separate-process HTTP probe harness.
- Always tear down the test database, HTTP server, and probe even when
result writing fails.
- Retain all timing and responsiveness samples as sanitized JSON.
- State the runtime, fixture, source revisions, and limits of the local
comparison.

## Verification

- 24 focused tests passed across selector matching, pending projection,
review-service freshness, and effective assignments.
- Four selected real database-backed service/route tests passed for
summaries, review changes, auto-allow behavior, and company isolation.
The other 494 tests in that file were not selected.
- Injected an `EISDIR` result-write failure into both profiling programs
on an owned local admin server. Each child exited with the expected
error and removed its disposable database. The responsiveness child also
terminated its probe/server handles.
- The changed benchmark completed with a configured local PostgreSQL
admin URL. Its full responses matched the baseline, and query/row counts
matched.
- The exact documented compile, benchmark, and responsiveness commands
completed from the committed harness at
`ca458baddc6461fb6c888984246e329e2bb4bdfe`. The JSON records that build
revision and the four owner source hashes.
- Five alternating samples per service and implementation used Node
24.22.2 with compiled ES modules. No runtime TypeScript loader ran in
those measurements.
- Median process CPU from the committed harness: profile listing 3,164
ms to 385 ms; app attention 2,466 ms to 221 ms. Both versions returned
identical ordered results and made seven queries per service.
- Separate Inspector profiles confirmed that the repeated scope and
summary scans were removed. Three separate-process HTTP probe trials per
service showed less event-loop blocking. These are local synthetic
service results, not hosted page-load acceptance.
- Initial local setup failed on missing embedded PostgreSQL library
aliases and an unbuilt plugin SDK. Restoring local aliases and building
the SDK resolved those setup failures. The original failed setup logs
were retained; the 28 selected tests then passed.
- The merged summary fix CI run
[41535304968](https://github.com/paperclipai/paperclip/actions/runs/41535304968)
hit two 15-second timeouts in the unchanged attention-retention history
cases. The server shard passed unchanged on one failed-jobs-only retry.
Original and retry logs are retained. The compared runs used different
runners and synthetic merge trees, so the original timeout cause is not
established. No timeout or assertion was changed.
- Full repository typecheck, test matrix, and build passed in CI on
final head `2f484ed4284f3fa632d8e73d9be084827d1c987e`: 54 successful
check runs, three expected skips, one non-blocking Paperclip review, and
a successful Snyk status. No failures or pending checks remain. Those
broad commands were not rerun locally for this benchmark-tooling
follow-up; the 28 selected local regressions and the actual
profiling/failure-path programs ran locally.
- Fresh Greptile is 5/5 on that exact head. Both review P2 threads are
resolved. The Paperclip review's current-base documentation note is
corrected in this description; its optional tooling suggestions remain
outside this bounded change.

## Risks

- The test helper creates a unique disposable database on the configured
admin server and removes it afterward. The operator must use a database
server where those test operations are appropriate.
- Synthetic data and shared-host activity affect absolute timings. The
measurements do not establish a production latency percentage or
capacity limit.
- Production behavior is covered by the two linked merged fixes. This
follow-up changes only benchmark setup and documentation.

## Model Used

OpenAI GPT-6 through Codex, with code execution and repository tools.
The exact runtime model revision and context window were not exposed in
this session. The original linked performance fixes record their own
model details.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:50:58 -06:00
DottaandPaperclip d0ffa179b4 perf(tools): index profile selectors per resolution (#18673) 
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Core resolves tool profiles before it prepares runtime MCP
assignments.
> - A selected profile can contain a rule for each individual tool.
> - The resolver and profile summary scan these rules for each catalog
row.
> - This repeats millions of exact comparisons for a large selected
profile.
> - This pull request indexes each selector kind once per computation.
> - The benefit is less server CPU with the same current access
decisions and result contract.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Effective tool assignment and profile summary computation for large
selected profiles.

**Current behavior**

Each catalog row scans the profile's includes and excludes. The work
grows with catalog size times rule count. The runtime assignment path
also computes a profile summary with the same scans.

**Proposed behavior**

Build five selector sets for each include or exclude group. Match
catalog rows with exact set membership. Retain all current database
reads and the full result contract.

**Reason and benefit**

Reduce repeated JavaScript comparisons. On the synthetic PostgreSQL
fixture below, median process CPU per resolution falls from 50.068 ms to
10.129 ms. This is independent of
[#18371](https://github.com/paperclipai/paperclip/pull/18371), which
removes unused administrative results and narrows instruction
descriptors. This PR retains those results and does not include that
change.

## What Changed

- Index application, connection, catalog-entry, tool-name, and risk
selectors in the profile owner.
- Use indexed matching in effective access resolution and profile
summaries.
- Add a differential test against the original single-rule matcher.
Cover null applications, exact special-character names, unknown
selectors, and empty rule groups.
- Add a PostgreSQL regression for per-profile exclusion precedence, the
union of multiple effective profiles, and current rule removal.
- Add a real PostgreSQL resolver benchmark and document reproduction and
limits.

## Verification

Baseline: `410b0e27015495ab8951768b5b0b94727de33785`. Treatment:
`895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`. Node 24.22.2 on macOS arm64.
Source modules run with `tsx`; these are local service measurements, not
production-build or hosted acceptance.

The real owning resolver ran against disposable PostgreSQL with 1,890
active catalog tools, 1,575 individual include/exclude rules, one
deny-by-default effective profile, and 12 agents. The installation
dependency returns an empty list because the fixture has no installed
connections. There are no provider calls. Full returned assignments,
profile summaries, rules, bindings, and names are identical to baseline.

After two warmups per version, seven alternating samples ran ten
resolutions each. Other workers paused CPU measurements and builds.
Unrelated host work was not controlled.

| Per resolution | Baseline median (range) | Change median (range) |
| --- | --- | --- |
| Process CPU | 50.068 ms (46.380–58.880) | 10.129 ms (8.089–11.072) |
| Elapsed time | 50.362 ms (44.799–106.408) | 8.853 ms (7.719–18.590) |
| Queries | 6 | 6 |
| Returned database rows | 3,480 | 3,480 |

Median process CPU falls 79.8%; median elapsed time falls 82.4%. The
long baseline elapsed sample had no matching process CPU increase. Query
and row counts do not decrease. Small or broadly allowed profiles can
save less. This does not establish whole-run latency or hosted
performance.

Reproduce with `server/scripts/benchmark-tool-profile-matching.ts`;
baseline module preparation is in `doc/mcp-discovery-performance.md`.

Passed on the treatment source:

```sh
node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-matching.test.ts server/src/__tests__/tool-effective-assignments.test.ts server/src/services/native-runtime/runtime-context.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts
```

All 45 tests in four suites passed. `git diff --check` passed. The first
PostgreSQL test attempt failed at setup because the sandbox blocked its
loopback listener (`EPERM`); the authorized unsandboxed retry passed. No
behavior failure was discarded.

At measured head `895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`, CI
typecheck, build, canary, Runner checks, and all eight browser shards
passed. Server general shard 8/12 failed an unchanged
background-quietness assertion in `heartbeat-run-dispatch-work.test.ts`:
expected 235 select calls, observed 236 after advancing fake timers. Its
cause remains unverified; no baseline reproduction was run.

That head also failed the module-boundary policy because the benchmark
inserted synthetic agents outside the test fixture tree. Follow-up head
`e20165b6eb` moves the fixture body byte-for-byte into
`server/src/__tests__/helpers/tool-profile-matching-fixture.ts`,
following the existing listing benchmark pattern. The production matcher
and benchmark measurement loop are unchanged. `node
scripts/check-module-boundaries.mjs`, a Node 24/tsx fixture import
check, and `git diff --check` pass. The benchmark was not rerun for this
extraction. New-head CI and review are pending. Greptile gave the
measured head 5/5 with no actionable issues. No deployment or live
runtime mutation was performed.

## Risks

Exact equality must remain identical to the original matcher, especially
for null application selectors and exclusions. The differential and
PostgreSQL tests cover these cases. Sets add linear temporary memory in
rule count. They live only for the current computation. Current
company-scoped state reads, profile precedence, revocation, and runtime
assignment digest checks remain in place. Revert this commit to roll
back.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code
execution, tests, and local PostgreSQL benchmarking. The context window
size is not available in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:13:18 -06:00
Nicky LeachandPaperclip a9b18e1cbe fix(runner): preserve cleanup ownership across startup failures (#18658) 
## Thinking Path

> - Paperclip controls which run owns an agent task.
> - A plugin can continue accepted work after Core stops waiting.
> - A late task-creation reply can therefore arrive after startup
cancellation.
> - Core must keep that work owned until its resource identity and
cleanup outcome are known.
> - This PR supplies the cleanup and recovery support before shared
startup is enabled.
> - The dependent activation PR can then reuse transport data within a
bounded startup context.

## Linked Issues or Issue Description

Refs #18405. This is the independently mergeable prerequisite for that
stacked activation PR.

**What existing behavior does this improve?**

Cleanup of explicitly recorded startup work that outlives its caller.

**Current behavior**

Caller timeout, worker exit, or failed cleanup-intent persistence can
leave work without a complete recovery path.

**Proposed behavior**

Retain the cleanup obligation, use existing Stop and recovery queues,
and release ownership only after the exact task and accepted work have
stopped. Ordinary calls keep their existing startup payloads and
budgets. This PR does not create startup contexts or advertise transport
reuse.

**Reason and benefit**

Separate the failure-handling contract from activation so each can be
reviewed on its own. This support PR has **no standalone TTFT speedup**.

## What Changed

- Add the startup cleanup marker, exact ownership checks, verified
completion, and acknowledgement helpers.
- Connect recorded cleanup obligations to existing Stop, deadline
reconciliation, restart recovery, and retry scheduling.
- Retain typed cleanup responsibility when a database write fails. Keep
the nonterminal run from being treated as ordinary completed cleanup.
- Observe late acquisition results and save resource identities before
retiring them. Retain real host writes and received results across
worker exit or persistence failure.
- Queue abandoned replacement tasks through the existing creation
journal and cleanup machinery. Require the replacement's own Stop proof.
- Accept an explicit startup context in the internal bridge for lifetime
tracking. Automatic context creation, marker production, capability
advertisement, transport capture, and executor activation belong to
#18405.
- Add PostgreSQL and worker IPC regressions, including legacy payload
assertions.

## Verification

Initial split source `8fb482d8be9e70101b0cfdd0830f41cf5b8e3cf4` had tree
`95508bd1859e991670cad8afd4de8d2167009ac5`. **190 distinct targeted
tests pass**, along with server typechecking. The selected checks cover
real PostgreSQL cleanup/ownership, worker IPC, the bridge and creation
journal, ordinary/distributed Stop, restart and deadline recovery,
idle-sleep behavior, and the unchanged legacy warm-task suite.
Independent recovery and worker reviews found no blocking issue.

The first correction, `b6342d42042601b376da38dc8e1dba9ca80e9539`, fixes
the replacement cleanup finding with the existing startup-cleanup
marker, scoped to contextual abandoned task creations. The replacement
was never prepared, so preparation-stop proof is not required. No
dispatch is fabricated and no normal-startup query is added. The actual
queue/sweep/driver/receipt/cancellation regression fails before the fix
and passes after it: void, running or foreign-task status retains
ownership; exact-task stopped status permits completion even when
Release returns void. **64 affected targeted cases and server
typechecking pass**. Independent review found no blocker.

Current source is `f53af72af76f263096c3c2e788506e6faef9474f`. It fixes
both recovery findings from the prior head's Apex 3/5: orphan recovery
now preserves pending or completed-but-not-cancelled cleanup through its
existing final-write fence, and exact late identities can reach verified
cleanup after reconciliation or restart. The regression selection went
from four failing assertions to six passing cases. Affected selections
cover 217 distinct cases; the exact final six-case regression and server
typecheck pass. Independent reviewers found no blocker in either fix.
Current-head standard review is 5/5 and all CI checks pass (54
successes, three expected skips, plus Snyk). Apex also completed at
**5/5** on this exact head, with zero unresolved threads. The requested
independent availability audit is complete on this head. It found an
activation-dependent operability gap: an acquisition response that is
never received can leave an unknown creation identity with no supported
task inspection/reconciliation path, while cleanup keeps the run owner
reserved. This does not occur in ordinary lower-only calls. The contract
and recovery policy need follow-up before activation; the external
review scores do not resolve that availability tradeoff.

The split preserves the accepted implementation at `21db7b3851` after
its clean merge with frozen master
`946735d03a55c1fb4cc5434d6bf1d87742daa9df`. The initial dependent
activation PR restored exact aggregate tree
`f68c6895f87c7b93ae03e6324d955d2659a99dce`. Both subsequent cleanup
corrections are integrated into upper head
`27978f294a02da3992585c7f4daa73cd8fcf99f8`; its current aggregate tree
is `a0b66c33f5e22745f034f45d24d213f5c210cf16`. Lower-only payload tests
cover the absence of capability advertisement and transport projection.

Only targeted local tests and typechecking are used. Full regression
coverage runs in GitHub CI. No deployment or ROM qualification is part
of this change.

The lower audit covered connected worker, creation-journal, Stop,
restart, deadline and orphan-recovery paths. A new isolated
real-PostgreSQL characterization reproduced the unknown-ID hold through
actual recovery, cleanup verification and run-control cancellation. It
models a lost acquisition response; it is not a live provider or
complete IPC test. No source edits were made by the audit. The
upper/composition audit remains required after its review cycle.

## Risks

- Recording a replacement result now commits with the existing cleanup
notification. This conditional replacement path adds
transaction/notification responses; common retained startup is
unchanged.
- This adds cleanup and recovery state handling. A missing receipt or
permanently inaccessible old backend can keep ownership held
indefinitely. Temporary outages retry; an ordinary retry is not evidence
that a deleted or replaced backend stopped.
- A caller timeout does not undo work already accepted by a plugin.
Unknown resource creation remains unresolved until its identity/outcome
is established. Current task creation has no supported
inspect-by-operation or operator reconciliation API for a permanently
lost identity; repeated Stop alone cannot resolve it.
- The support code understands records produced by the dependent
activation PR. It does not enable the shared startup policy by itself.
- Existing legacy callers must retain their behavior; context-only
automatic replacement cleanup remains explicitly scoped.

## Model Used

OpenAI Codex, GPT-6 family. The exact model ID and context window are
not exposed in this session. Tool-assisted implementation, targeted
testing, and independent subagent review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used, with available version and
capability details
- [x] I have checked ROADMAP.md and confirmed this is the requested
split of existing work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue with the required fields
- [x] I have not referenced internal issues or private links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] Targeted local tests pass on this source
- [x] I have added or updated relevant tests
- [x] I have updated relevant documentation
- [x] I have documented the risks
- [x] All current-head GitHub CI gates are green
- [x] Current-head standard Greptile and Apex are 5/5 with no unresolved
findings
- [x] I will address all review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 14:13:00 -08:00
DottaandPaperclip b4315943e8 fix(heartbeat): keep remote retry workspace strategy stable (#18660) 
## Thinking Path

> - Paperclip preserves agent work across separate execution services.
> - A remote retry can create a new isolated Git workspace.
> - Later admissions compare its saved configuration with the current
policy.
> - Fresh preparation also creates a temporary controller-side strategy.
> - Remote preparation saved that temporary strategy in its fingerprint.
> - The next unchanged turn then reported a workspace strategy
replacement.
> - This change saves the admitted remote strategy so unchanged reuse
stays stable.

## Linked Issues or Issue Description

Related preparation work: #18543 and #17924. I searched open and merged
workspace/session repairs and public workspace-strategy issues. None
covers this fresh remote retry case.

**What happened?**

After Retry created an isolated remote Git workspace, each unchanged
follow-up reported `workspace_config_freshness` with `action: replace`
and `changedCategories: [strategy]`. It retained the same workspace. The
stored fingerprint contained `project_primary`, which was only a
temporary controller provisioning choice.

**Expected behavior**

Save the configured remote strategy. An unchanged follow-up must reuse
the same workspace without a false replacement report. A real strategy,
base-ref, mode or environment change must retain its replacement checks.

**Steps to reproduce**

1. Prepare a fresh isolated remote Git retry with a `git_worktree`
strategy.
2. Complete its runtime handoff.
3. Admit another turn with the same workspace policy.
4. Inspect the recorded workspace freshness decision.

**Paperclip version or commit**

Reproduced on `b9e94b85a69bf679818eeef9ecf4655554a89c6b`. The regression
also fails on the unchanged preparation code in the current base.

**Deployment mode**

Authenticated server with remote workspace authority and a sandbox
execution target.

## What Changed

- Keep the admitted strategy when a fresh workspace uses remote
authority.
- Retain the temporary controller strategy for existing host-side seed
preparation.
- Extend the PostgreSQL preparation test through a fresh admission and
its next ordinary admission. Assert the same workspace, current writer
and stable fingerprint, with no false drift operation.
- Cover a genuinely configured `project_primary` fresh remote Git retry
followed by a real strategy change, preserving its replacement decision
and stored fingerprint.
- Document the execution rule and the separate live verification recipe.

## Verification

- The final regression was run before and after the production change.
Before: the next admission recorded a strategy replacement. After:
unchanged reuse records no drift and retains the workspace and
fingerprint.
- All 312 targeted cases pass on
`48b3e548afa42e740871feae2763d88fd5a00431` across workspace preparation,
workspace/session policy and inherited session preparation. These
include actual strategy/base-ref changes and the rule that explicit
reuse must not hide real replacement-class drift.
- All eight remote preparation cases also pass with the compatibility
regression: a genuinely configured `project_primary` fresh Git retry
creates a `git_worktree` row, and its later real strategy change retains
the replacement decision and saved fingerprint.
- `node scripts/check-maps.mjs` and `git diff --check` pass.
- Full recursive typecheck and `pnpm build` pass locally on this head.
- The [full current-head CI
matrix](https://github.com/paperclipai/paperclip/actions/runs/41524105868)
passes: typecheck/build, all general server, chat, workspace and
serialized test groups, plus Runner and E2E checks. The workflow and
`run-vitest-stable.mjs` partition the complete suite.
- The duplicate local `pnpm test:run` passed all 1,828 chat cases and
was intentionally stopped during general-server after the same complete
CI matrix passed. This is not recorded as a full local-suite pass.
- Greptile and Paperclip Review both assessed this head at 5/5 with no
new findings. Paperclip Review accepted the compatibility boundary after
the counterexample; its check conclusion is neutral. The addressed
review thread is resolved.
- The regression uses real PostgreSQL and preparation policy. Runtime
acquisition is controlled at its external boundary. The new fix has not
been deployed; live provider resumption remains unverified for this
patch.

## Risks

- Existing incorrect fingerprints are not rewritten. The same stored
shape can come from a genuinely configured `project_primary` remote Git
retry; a Git workspace row alone cannot prove the old admission policy.
Rewriting it would hide a real policy change. After preserving and
inspecting retained work, an explicitly admitted fresh recovery after
rollout establishes a correct new fingerprint. Ordinary reuse of the old
workspace does not migrate it. Normal recovery gates remain required.
- Real policy changes still use the current replacement guard. No
schema, credential or authority change is included.

## Model Used

OpenAI GPT-6 Codex with reasoning, repository editing, shell execution,
PostgreSQL tests and browser inspection. The host does not expose the
exact model variant or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:11:30 -06:00
DottaandPaperclip 482706e7dd perf(tools): index pending profile review scopes (#18681) 
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The Apps UI lists tool profiles and shows new tools that need
review.
> - Both services project new-tool review for each profile.
> - The projection scans include rules and exact rules for each catalog
row.
> - This repeats millions of comparisons when many selected profiles
exist.
> - This pull request builds scope and selector indexes once per
projection.
> - The benefit is less server CPU with the same fresh review results.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

New-tool review projection in profile listings, profile review, and app
attention.

**Current behavior**

Each newer catalog row scans the profile's included scopes. Included
catalog entries require a map lookup for each scope comparison. Rows in
scope then scan all rules again to omit tools that were already
reviewed. Profile listing and app attention repeat this work for every
profile.

**Proposed behavior**

Index covered application and connection scopes once. Expand included
catalog entries to their existing connection and non-null application.
Index exact selectors across all rules once. Keep the ordered catalog
filter and returned metadata.

**Reason and benefit**

A recent main-thread profile identified this scope-matching path as a
large CPU consumer. The local reproduction below confirms multi-second
CPU work in the actual owning services. This PR removes the repeated
scans without caching authorization or changing database reads.

Related [#18673](https://github.com/paperclipai/paperclip/pull/18673)
indexes effective access and profile summaries. It leaves this
pending-review path unchanged. This PR targets the separate
pending-review mechanism and has no dependency on that PR. The selector
helper is identical so the owners can share it when both changes land.

## What Changed

- Build covered application and connection sets for included scopes.
- Build exact selector sets for all rules, including exclusions, to omit
already reviewed tools.
- Preserve null application selectors, missing catalog references,
review watermarks, status checks, output order, and names.
- Add differential tests against the former scope/matching evaluator.
- Add PostgreSQL checks through all three affected service methods,
including current catalog and rule changes.
- Add a reproducible PostgreSQL benchmark of complete profile-list and
app-attention results.

## Verification

Baseline: `1f8f2e195f4897cd1c237bdc068de95780ab0c3c`. Treatment:
`fd59c0183ce64c44696f79604516f4d51ca5662b`. Node 24.22.2 on macOS arm64.
The production projection file at this baseline is identical to the file
at `410b0e27015495ab8951768b5b0b94727de33785`.

The benchmark exercises the real
`createToolProfileService(...).listProfiles` and
`createToolAppAttention(...)` methods against disposable PostgreSQL. It
uses 200 selected profiles, 1,900 active tools, 76,000 rules, and 12
agents. Profiles alternate between 65 and 695 rules. Half reference 695
removed catalog entries; half select current tools. This tests both the
slow failed scope scans and the slow exact-rule scans. No provider calls
or hosted traffic are generated.

The complete ordered results from both services match baseline. After
one equality/warmup invocation per version, five samples alternate
implementation order. Forced GC runs before each measured pair of
service calls. Other workers paused owned builds and benchmarks;
unrelated host activity was not controlled.

| One profile listing plus one attention listing | Baseline median
(range) | Change median (range) |
| --- | --- | --- |
| Process CPU | 5,406.512 ms (5,221.533–5,648.262) | 1,770.116 ms
(1,596.232–2,045.617) |
| Elapsed time | 5,279.231 ms (4,932.419–5,537.105) | 1,532.414 ms
(1,499.199–1,610.428) |
| Queries | 14 | 14 |
| Returned database rows | 156,215 | 156,215 |

Median process CPU falls **67.3%** and median elapsed time falls
**71.0%**. Profile summaries retain their old nested matching in this
comparison, so the result isolates pending-review indexing. Query counts
and row counts do not decrease. This is local source/tsx service proof.
It does not establish hosted improvement, whole-run latency, or HTTP
tail latency.

Reproduce with `server/scripts/benchmark-profile-review-projection.ts`;
baseline-module setup is documented in
`doc/mcp-discovery-performance.md`.

Passed:

```sh
node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-pending-projection.test.ts server/src/__tests__/tool-profile-review-service.test.ts server/src/__tests__/tool-profile-entry-order.test.ts
node scripts/check-module-boundaries.mjs
```

All five tests in three suites passed. The differential tests cover all
selector kinds, include/exclude effects, null scope behavior, missing
references, review watermarks, and exact special-character names.
PostgreSQL checks preserve current pending counts after exclusions,
catalog removal, and review; another company remains separate. `git diff
--check` passes.

The first new integration assertion used `summary` instead of the
existing attention response's `totals` field. It failed before the
mutation checks. The test assertion was corrected; no production code
changed for that correction. The final run above passed.

Server typecheck was attempted but did not pass local setup. The
packaged command did not reach TypeScript because a global pnpm child
tried to fetch and verify the project's pnpm version through an
unavailable registry. A direct Node 24 TypeScript invocation then
reported missing built `@paperclipai/paperclip-runner` and related
package declarations, with cascading type errors in unchanged consumers.
No error points to the changed projection. The prerequisite build was
not expanded into a full workspace build. Full local unsharded tests and
build were not run; final-head CI must supply broad acceptance. No merge
or deployment was performed.

## Risks

Scope expansion must preserve existing semantics. An individual catalog
rule covers its connection and only a non-null application. An explicit
null application selector still matches null applications. Removed
references contribute no scope when absent from the supplied catalog.
The oracle and PostgreSQL tests cover these boundaries.

The indexes add linear temporary memory in rule count and live only for
one projection. Current company-scoped database reads and review
mutations remain in their owners. Exclusions remain explicitly reviewed;
they do not create new review items. Revert this commit to roll back.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code
execution, tests, and local PostgreSQL benchmarking. The context window
size is not available in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:10:42 -06:00
DottaandPaperclip 2b374caba0 fix(recovery): preserve unfinished remote exports before Retry (#18678) 
## Thinking Path

> - Paperclip manages agent work and preserves each task's saved files.
> - A completed native result can still need workspace export.
> - Remote workspaces now use a different saved descriptor from legacy
copyback.
> - The execution hold checked only the legacy descriptor.
> - Task Retry could admit another run before the original export
finished.
> - A later failure could then supersede the original recovery record.
> - This change keeps unfinished remote exports fenced and permits
explicit historical file recovery.
> - Recovery keeps human review and newer execution authority intact.

## Linked Issues or Issue Description

**What happened?**

A native run completed successfully, but its remote workspace export
failed. The recovery action used
`native_workspace_sync_out_retry_exhausted`. Task Retry created a second
task run while the original files still needed preservation. The newer
failure then superseded the original action. A board file-recovery
request could queue that cancelled record, but the worker excluded it
from discovery.

**Expected behavior**

Keep Retry behind the unfinished export. Save the exact source and
finalize the existing accepted result once. Preserve pending human
review. An explicit request for a superseded source must save its files
without applying an old result over newer work.

**Steps to reproduce**

1. Save an accepted successful native result with a prepared
`remoteWorkspace` descriptor.
2. Fail its final export and retain its exact sandbox lease.
3. Select Task Retry before preservation completes.
4. Observe another task run before the original workspace is saved.
5. Supersede the original action and request file recovery for its exact
source. Observe queued work that the worker does not select.

**Paperclip version**

Observed at `1f8f2e195f4897cd1c237bdc068de95780ab0c3c`. The regression
is present on current master before this patch.

**Deployment mode**

Authenticated hosted instance with native Paperclip Runner and remote
sandbox workspaces.

Related: #18403 introduced the remote workspace path. #18644 reduces
export scratch disk use. #17916 covers a separate legacy warm-copyback
policy case. This change repairs remote export admission and explicit
historical preservation.

## What Changed

- Include prepared remote descriptors in the existing native export
hold.
- Retain remote source identity in the blocker and continuation read
projections.
- Let the existing worker select a cancelled export-repair record only
after an explicit board file-recovery request. Keep the current action
and workspace binding unchanged.
- Add real PostgreSQL, HTTP, filesystem and Git regressions. Update
execution semantics and recovery maps.

## Verification

- Red before the admission fix: HTTP Retry creates two task runs instead
of one.
- Red before the historical-source fix: the explicit recovery stays
scheduled and saves no snapshot.
- All 588 focused tests pass across the workspace recovery, recovery
source, explicit continuation and native export retry suites. The three
new integration cases verify duplicate request identity, original result
finalization, restored bytes and Git HEAD, pending human review,
current-writer exclusion, and preservation without older-result
commitment over newer work.
- `pnpm -r typecheck`, `pnpm build`, map validation and `git diff
--check` pass.
- Full hosted CI passes on `c598718379fa939ec539e8e91a15a94d90321b59`,
including all server, workspace, serialized, Runner and browser lanes
([run](https://github.com/paperclipai/paperclip/actions/runs/41529739604)).
Greptile reviewed that commit at 5/5 with no findings; there are no
unresolved review threads. Paperclip Review also published a complete
5/5 assessment for that exact commit with zero findings. Its check
conclusion is advisory/neutral, titled `5/5`. The first attempt lacked a
structured assessment; one normal completion request recovered the saved
analysis and published the validated assessment.
- The local `pnpm test:run` passed all six chat partitions, then was
stopped during the general server group after equivalent full CI
completed. Its partial log is retained; it is not reported as a
completed local full-suite pass.
- The test provider boundary uses real temporary files and the
production archive path. It simulates provider resume/stop calls. The
deployed fix and original hosted task recovery remain unverified until
merge and rollout.
- The older local Node 24.11.0 runtime failed to load the installed tsx
worker. Node 26.4.0 passed the isolated worker check. Required
repository checks use an isolated Node 24.22.2 runtime. The earlier
failure logs remain recorded.

## Risks

- Recognizing another unfinished export can keep previously admitted
work waiting. This is intentional until persistence is verified.
- Explicit historical recovery can resume the exact stopped sandbox to
read files. It retains source, allocation, ownership, writer and
bounded-attempt checks. It starts no provider turn.
- A newer run still prevents commitment of an older accepted result.
This change does not infer that a cancelled run had no external effects.
- No schema, credential, deployment or fleet configuration changes.

## Model Used

OpenAI Codex, GPT-6 family. The runtime does not expose the exact model
ID or context-window size. Reasoning, repository inspection, tool use
and code execution assisted this change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:03:09 -06:00
DottaandPaperclip 919ddbf46d fix(workspaces): read committed files before live allocations (#18677) 
## Thinking Path

> - Paperclip keeps task files in managed workspaces.
> - A final remote save commits a durable snapshot and clears writer
authority.
> - The old allocation can remain recorded while its transport becomes
unavailable.
> - Files still tried that live allocation before the accepted saved
version.
> - This PR reads the validated save when authority is clean and no
writer owns it.
> - Dirty work and active writers still require the live allocation.

## Linked Issues or Issue Description

Related: #18403 introduced remote Files access. #18665 fixes a separate
completion-settlement issue and is included in the base.

**What happened?**

A file preview returned `422 workspace_unavailable` after a remote task
had saved its files. Its clean authority still had a live location. The
same original bytes were readable from the committed save after that
location cleared. The exact original live transport failure is unknown.

**Expected behavior**

Once the workspace is clean and has no writer, Files reads its exact
current committed save. A retained location must not make accepted files
depend on remote command execution.

**Steps to reproduce**

Create a real committed workspace archive. Keep its allocation active in
the recorded authority. Clear dirty state and writer authority. Make
live execution unavailable, then request the ordinary file preview
route. Before this fix it returns 422. After this fix it returns the
saved bytes without a provider command.

## What Changed

- Select the saved-file loader before consulting a live location when
authority is clean and has no writer.
- Keep exact company, workspace, snapshot, archive, and authority-change
checks.
- Keep the live path for dirty work and active writers. Never substitute
their older saved bytes.
- Cover browse, resolve, preview, download, rejection controls, and
saved-copy cleanup through real routes and storage.
- Document the selection rule and update its feature and access-map
recipes.

## Verification

- Red: the new ordinary-route regression expected HTTP 200 but received
422 against the old service.
- A separate compatibility control reproduced the optional-loader
regression. Valid live reads still work when the loader is absent;
no-location reads still fail, and a configured invalid save never falls
back to live bytes.
- Green: 98 ordinary file-route and authorization tests passed with real
PostgreSQL and archive storage.
- All 46 opt-in remote workspace lifecycle tests passed against one
owned local disposable PostgreSQL database. Its database, server, and
files were cleaned after the run. No live S3 or provider flags were
enabled.
- Full workspace typecheck passed after the optional-loader
compatibility repair. Full build passed locally before that narrow
repair. All normal CI checks pass on the current head, including build
and server suites. The separate Paperclip Review job and the full local
test suite remain in progress.
- Feature/access-map validation and `git diff --check` passed.
- A read-only staging check preserved and verified the original saved
bytes. It did not rerun the agent or qualify the failed journey. The
fixed active-location path has local regression evidence; deployed
acceptance is pending.

## Risks

A clean workspace now uses the existing bounded saved-archive loader
while a location remains recorded. Each Files request can retrieve and
extract that archive. Existing disk, size, concurrency, cancellation,
and cleanup bounds remain. A dirty workspace or current writer still
uses live files and fails closed when they are unavailable. This change
neither releases compute nor changes access permissions. It does not
identify the original provider RPC subreason.

## Model Used

OpenAI GPT-6 through Codex with reasoning, code execution, and browser
tools. The runtime does not expose a more specific model identifier or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 16:00:04 -06:00
DottaandPaperclip 0c48f5e727 test(runner): cover document guidance for file-only objectives (#18669) 
## Thinking Path

> - Completion guidance can link a saved task document even when the
user requested only a file.
> - The validator loads document publication proof when the objective
requests a document or the completion report cites one.
> - The success tests added in #18661 all requested both a document and
a CSV.
> - Those tests therefore could not detect loss of the cited-document
branch.
> - This test requests only a CSV and resubmits the actual guidance link
with a registered file receipt.

## Linked Issues or Issue Description

Addresses the [coverage warning on
#18661](https://github.com/paperclipai/paperclip/pull/18661#discussion_r4447457009).

**What happened?** The earlier success cases exercised the
document-requested condition. The independently cited-document condition
lacked regression coverage.

**Expected behavior:** A file-only objective can cite a verified
saved-document guidance link alongside the required registered file.

**Steps to reproduce:** Publish a task document, register a CSV, request
only the CSV, obtain the saved-document href from completion guidance,
and submit both pieces of evidence. Removing only the cited-document
lookup must make this case fail.

**Paperclip version or commit:** Current master after #18661.

**Deployment mode:** Local source tests with real PostgreSQL and native
publication receipts; the remote CSV reader is controlled. No deployment
is part of this follow-up.

## What Changed

- Allow the existing registered-file test helper to accept a file-only
objective.
- Add one success case using real document publication, file
registration, and the server-generated guidance href.
- Keep production source and the existing negative cases unchanged.

## Verification

- All 27 native completion feedback tests pass on current production
code.
- The new case fails when only `|| citesTaskDocument` is temporarily
removed from publication lookup. The production file is restored exactly
afterward. This demonstrates a coverage gap; it does not establish a new
production defect.
- Server typecheck and `git diff --check` pass.
- Full repository CI passed, including both required E2E and verify
aggregate gates. Greptile scored the current head 5/5 with no unresolved
findings. The separate optional Paperclip review is queued and has
supplied no finding. The preceding production PR records its separate
local full-suite timeout and isolated passing retry; this follow-up uses
the focused local suite and full CI rather than repeating that long
local run.

## Risks

Low risk: only the existing test file and its fixture helper change. The
new case depends on real document/file publication contracts, including
the CSV receipt requirement.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with reasoning, code editing, shell
execution, and automated tests. The context-window size is not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 15:33:46 -06:00
DottaRudy CelekliPaperclip
 		45e91f097c fix(inbox): requalify existing unread fix (#18670) 
## Thinking Path

> - Paperclip helps people manage AI agents and their work.
> - The personal inbox must keep new comments unread until the person
reads or answers them.
> - A comment updates task recency, even when another person writes it.
> - The assignee unread query treats that task timestamp as a read
receipt.
> - [rudycelekli](https://github.com/rudycelekli) already repaired this
in #15085, including a reviewed deleted-comment safeguard.
> - That PR now has merge conflicts and checks from an older base. This
PR ports and requalifies the existing unread fix on current master.
> - The port adds real human-comment and API regressions. The original
contribution remains open and unchanged.

## Linked Issues or Issue Description

Fixes: #15079
Refs: #15085

This is an attributed maintainer port of #15085 at
`fe6a486c9ca21cd3a6ea0cef9764c372c53f4572`. The commit includes the
verified original contributor credit. This is not an independent claim
to the original repair.

## What Changed

- Port the stable creation-time fallback for a human assignee in SQL and
derived user context. Explicit reads and own replies still take
precedence.
- Port the reviewed safeguard that excludes deleted external comments.
Keep the historical touch from a deleted own reply.
- Exercise real comment writes, task recency updates, existing read
receipts, read/unread toggles, later assignment, company isolation, and
comment deletion through native PostgreSQL and ordinary API projections.

## Verification

- Native PostgreSQL regressions fail before the repair: real comment
writes leave the assignee falsely read, and later assignment hides
existing comments.
- A separate deletion regression fails before the reviewed safeguard: a
deleted external comment remains the latest external activity.
- The frozen head `0dadf85611cbfdc7a762cb82b03ed64519be5434` passes all
41 focused inbox, source-visibility, request, archive, and user-context
tests on native PostgreSQL. No tests were skipped.
- `pnpm -r typecheck` and `pnpm build` pass on that same frozen head
with Node 24 and pnpm 9.15.4. The current-head CI is green, including
all general and serialized test shards, build, typecheck, Runner checks,
and eight E2E shards. [CI
run](https://github.com/paperclipai/paperclip/actions/runs/41525545915).
- The local full-test attempt was stopped before the one master rebase.
It is incomplete and is not a local full-suite pass. The full
current-head CI completed successfully.
- Fresh Greptile review is 5/5 on
`0dadf85611cbfdc7a762cb82b03ed64519be5434`, with no actionable findings
or open review threads.

## Risks

- Assigned tasks with older external comments can appear unread. A later
explicit read or own reply still clears them.
- This changes no schema, API fields, company membership rules, or query
joins.
- Legacy compact-response cache policy is unchanged. Immediate mutation
checks use the ordinary, rows, and index projections.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository inspection,
code editing, and command execution. This session does not expose a more
specific backend model identifier or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
documentation change is needed to restore the existing unread contract)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 15:26:28 -06:00
DottaandPaperclip b4a66027b7 fix(server): deliver chat completions after nested workspace saves (#18665) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat returns completed worker tasks to the original
conversation.
> - Delivery must retain its wake receipt when execution gates refuse
admission.
> - A workspace save can contain a nested snapshot operation.
> - The old checks lost the receipt and treated that saved snapshot as
unfinished work.
> - This pull request preserves the receipt and recognizes a successful
owning save.
> - The benefit is that completed work can reach the chat through the
normal queue.

## Linked Issues or Issue Description

Refs #18652. Related: #17206 covers failed finalize recovery. This
repair handles a successful finalize with an enclosed snapshot and does
not change failed-save recovery policy.

**What happened?**

A delegated worker finished and saved its result. The original chat
received no update. The dependency gate compared operation start times.
A nested snapshot started after its enclosing finalize and therefore
left a permanent hold. The completion wake also lost its exact
idempotency receipt when the queue merged a refused wake into a generic
dependency wait. The outbox reported `Completion wake returned no run or
durable receipt`.

**Expected behavior**

A successful finalize covers a completed operation enclosed by that save
when company, workspace, issue, and run match. Completion wakes that
disable coalescing keep their own durable receipt even when admission
waits. Dependency, budget, Stop, and pause gates still apply.

**Steps to reproduce**

1. Hand a task from a chat to a worker and make the chat wait on that
task.
2. Finish a workspace finalize that starts before its nested snapshot
and ends after the successful snapshot.
3. Mark the worker Done and deliver its completion outbox entry.
4. On the parent revision, the dependency stays blocked and the
completion receipt is absent.

**Paperclip version or commit**

Reproduced against `53f480d1b3` with real disposable Postgres rows and
the normal queue/outbox services.

## What Changed

- Honor `allowRunCoalescing: false` when recording execution waits.
- Accept an enclosed successful operation only when the successful
finalize has the same company, workspace, issue, and run and covers its
whole recorded interval.
- Preserve the existing rule that a strictly later successful workspace
finalize clears earlier work.
- Add failure controls for foreign runs, unfinished snapshots, failed
saves, snapshots that finish too late, and later operations.
- Exercise the real outbox and queue together. Hold provider dispatch at
the existing drain boundary and verify server-recorded Done facts reach
the admitted turn.
- Document receipt identity and workspace-save coverage.

## Verification

- Before repair: the two causal regressions fail and five safety
controls pass. The missing exact receipt and empty wakeable-dependent
list match the observed failure.
- After repair: those seven cases pass without assertion changes.
- The full affected set passed 288 tests and failed one existing
scheduling test. Its unchanged Node24 baseline also failed. A
closed-phase probe showed the fake adapter finished successfully at
3.453 seconds, after the incidental 3-second poll. The test now joins
the existing execution promise and retains its status and dependency
assertions under the unchanged Vitest timeout. The synchronized full
dependency suite passes all 13 cases.
- `pnpm -r typecheck` and `pnpm build` pass on pinned Node24/pnpm9.
`pnpm test:run` is running. The combined real outbox/queue case passes.
Fresh Greptile is 5/5 on `d7cee24b7032c7fdf171bfd1246cd90d41db1cba`.
Exact-head normal CI and required `ci / verify` and `ci / e2e` pass.
- The original scheduling failures on Node26 and Node24, the
unchanged-source baseline failure, and the closed-phase probe are
retained in the validation record. No model, deadline, production
assertion, or paid attempt was retried to change the failure grade.
- Current-head CI attempt1 passed all product and workspace coverage but
failed the unchanged serialized retry-worker case at
`issue-scheduled-retry-routes.test.ts:606` (first batch not observed
before the poll deadline;410 other shard tests passed). The exact
unchanged case passes locally on Node24 with real Postgres. The one
failed-jobs-only retry passed on the same head; the original grade is
retained.
- The full local command completed all1828 chat integration tests. Its
general stage remains running and has retained failures in unchanged
`workspace-runtime.test.ts` (30-second local port-conflict fixture) and
`muse-control.test.ts` (current binding authentication fixture). These
are not counted as passing local full-suite evidence. Their current-head
hosted coverage passed.
- The separate advisory Paperclip Review app is paused and has no code
finding. Its in-progress state is preserved and is not counted as a
pass; it is not a required repository status check.
- The live Cloud completion journey remains failed and unqualified. It
must pass at the deployed fixed revision before daily-plan admission. No
QA oracle, model, budget, or run limit changes are included.

## Risks

- The workspace coverage query must not clear a save from another owner
or new work after finalization. The guard checks the full interval and
exact owner identity, with negative controls.
- Refused noncoalescible automatic wakes retain separate receipts.
Replaceable automatic signals still merge. No new polling or provider
retry path is added.
- No migration or API change. Deployment and live acceptance are
separate remaining gates.

## Model Used

- OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The
exact deployment variant and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 15:06:26 -06:00
DottaandPaperclip cd12432947 fix(runner): recognize verified task document evidence (#18661) 
## Thinking Path

> - Paperclip manages AI agents and keeps their results attached to
tasks.
> - Native tools return links to saved task documents.
> - Completion feedback asks agents to include those links.
> - The file-delivery validator treated every such relative link as a
workspace path.
> - A valid document and CSV result could therefore fail completion when
it cited the supplied document link.
> - This change derives document links from the same verified
publication projection for guidance and validation.
> - Requested files still need their own verified delivery evidence.

## Linked Issues or Issue Description

**What happened?**
The native document tool and completion feedback produce a link such as
`/EX/issues/EX-1#document-output`. When a task also requests a
downloadable CSV, citing that link causes completion to reject it as a
workspace-only file. This occurs even when the document is published and
the file has a valid current-run delivery receipt.

**Expected behavior**
Accept a server-derived document link only when it names the current
company/task and a currently attached revision with a matching native
publication receipt. Keep the separate requested-file requirement and
all foreign, stale, missing, and workspace-file guards.

**Steps to reproduce**
1. Save a task document through the native `write_document` tool.
2. Publish a CSV through `register_deliverable`.
3. Ask completion feedback for the final-response guidance.
4. Cite its returned document link alongside the file receipt in the
next completion report.
5. The old validator rejects the valid document link as a workspace-only
file.

**Paperclip version or commit**
Current source checkout. The regression exercises the real native
document producer and completion validator.

**Deployment mode**
Local source verification on macOS arm64 with Node 24.22.1, pinned pnpm
9.15.4, locked dependencies, temporary PostgreSQL, and a controlled
remote file-reader boundary. No live provider or hosted deployment is
included.

## What Changed

- Derive the canonical document href in the existing
company/task/current-revision publication query.
- Use that same verified href in completion guidance and reference
validation.
- Keep file registration and hash/size receipt requirements separate
from document evidence.
- Cover evidence, artifact, and criterion references; foreign
companies/tasks; bogus keys; local paths; and lost current-revision
publication proof.
- Exercise actual native document writes and CSV registration, including
attachment download query and fragment links.
- Update the execution contract and the document feature/access maps.

## Verification

- Red before the production edit: native completion generated a real
saved-document link, then rejected the same link alongside a valid
published file. The failure was the workspace-only-file guard.
- Green: 210 tests across native completion feedback, explicit output
validation, and the existing native file-handoff suite pass. These use
real PostgreSQL and native tool receipts; the remote file reader is
controlled.
- A saved document link alone still fails the explicit CSV requirement.
A document whose current revision lacks its publication receipt is
rejected. Foreign task/company links, missing keys, malformed links, and
workspace paths remain rejected.
- Full workspace typecheck, server typecheck, and production build pass.
- `node scripts/check-maps.mjs`, `node
scripts/check-module-boundaries.mjs`, and `git diff --check` pass. Map
checks establish structure only.
- Full `pnpm test:run` is running separately. There is no full-suite
verdict yet. CI and review are pending.
- Hosted acceptance and any recovery of existing tasks remain separate
steps. This PR does not change final-response recovery or scheduling.

## Risks

- An overly broad URL exception could admit foreign or unpublished
output. The validator accepts only exact hrefs from the authenticated
publication projection.
- Document evidence could hide a missing requested file. A separate
regression keeps that delivery gate effective.
- The shared projection adds current task/company joins. Existing
document feedback and publication tests cover those consumers.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with reasoning, code editing, shell
execution, and automated tests. The context-window size is not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 15:02:00 -06:00
DottaandPaperclip aa6c9c3f25 perf(storage): overlap streamed S3 multipart uploads (#18659) 
## Thinking Path

> - Paperclip manages AI agents and preserves their work.
> - Large workspace and skill objects use the shared S3 multipart
uploader.
> - The uploader waits for each part acknowledgement before it sends the
next part.
> - Storage request latency therefore accumulates once per part.
> - This change overlaps up to four streamed part requests.
> - One source reader preserves byte order and backpressure.
> - Failure cancels and settles active requests before cleanup.

## Linked Issues or Issue Description

Related: #18082 added the shared streamed multipart uploader. #18644
fixes temporary disk use during workspace capture. It changes a separate
stage.

**What happened?**
The shared S3 multipart uploader sends large object parts in sequence.
The code waits for each `UploadPartCommand` before it consumes the next
part. A controlled HTTP test confirms that only one complete part
reaches the endpoint while its acknowledgement is held.

**Expected behavior**
Upload independent parts concurrently within a fixed limit. Preserve
exact bytes, ordered completion receipts, source backpressure,
cancellation, and full snapshot verification.

**Steps to reproduce**
1. Upload a large stream through the S3 multipart provider.
2. Hold each successful part acknowledgement at a disposable storage
endpoint.
3. Observe that the old uploader sends only one complete body until its
acknowledgement returns.
4. Repeat with this change. Four complete bodies can reach the endpoint
before the first acknowledgement.

**Paperclip version or commit**
Current source checkout. The affected uploader was introduced in #18082.

**Deployment mode**
Local source verification on macOS arm64 with Node 24.22.1, pinned pnpm
9.15.4, and locked dependencies. The HTTP tests use synthetic data and a
disposable loopback endpoint.

## What Changed

- Send up to four part requests at once through bounded streams.
- Keep one source reader and retain the existing adaptive part-size
policy.
- Store successful receipts by part number before completion.
- Cancel a stalled asynchronous source read without waiting for the
source to produce more bytes.
- Preserve the first error. Cancel sibling requests and settle them
before multipart abort.
- Cover exact bytes, out-of-order acknowledgements, backpressure, the
concurrency limit, source errors, cancellation, and the real S3 HTTP
client.
- Document the transfer bound. Snapshot encryption, publication, and
full object verification retain their existing behavior.

## Verification

- Failing before the change: the held-acknowledgement regression
expected four complete part bodies and received one. The same regression
now passes.
- All 15 focused multipart and storage-diagnostic tests pass. The real
AWS S3 SDK sends four exact 8 MiB bodies through a disposable HTTP
endpoint before any acknowledgement. Completion retains source order.
- Controlled transfer comparison: one synthetic 64 MiB stream, eight 8
MiB parts, real S3 SDK and loopback HTTP, each acknowledgement delayed
100 ms. Original: 904 ms, peak one request. Fixed: 247 ms, peak four.
Both transfers verified every part hash and completion order. This is
local boundary evidence, not a hosted latency claim.
- A second safety regression reproduced cancellation waiting
indefinitely on a stalled asynchronous source. Abort-aware reads now
settle within the existing bounded cleanup budget.
- All nine existing encrypted snapshot storage tests pass after
correcting missing library aliases in the local embedded-Postgres
dependency. The failed setup attempt had no test verdict.
- Full workspace typecheck and production build pass. The rebased server
typecheck also passes.
- The first full `pnpm test:run` attempt stopped during chat test
collection with `No test found ... in lines ...` before a suite verdict.
The chat test and line-selection scripts are unchanged from master. A
retry after local Postgres dependency setup passed its first chat
shards, then was stopped to avoid redundant host load after current-head
CI passed every type/build/general/serialized/runner group. The
unrelated narrow-banner E2E job failed its first attempt and passed one
bounded retry; all current-head GitHub CI jobs are now green. There is
no full local verdict. The separate Paperclip native review concluded
neutral after reporting that work was paused; it supplied no substantive
code review.
- Commands: `pnpm -r typecheck`, `pnpm build`, and `pnpm --filter
@paperclipai/server exec vitest run --configLoader runner
src/__tests__/storage-s3-multipart.test.ts
src/__tests__/storage-s3-failure-diagnostics.test.ts
src/services/remote-workspace-snapshot-storage.test.ts`.
- Deployment and an end-to-end hosted acceptance run remain separate
steps.

## Risks

- Four requests per upload increase concurrent storage traffic. Multiple
simultaneous uploads multiply that bound.
- Shared source consumption could reorder bytes. One producer and
exact-content tests protect that boundary.
- Failed siblings could outlive multipart cleanup. Cancellation and
settlement tests protect cleanup order.
- Capture and full read-back verification still take time. This change
only targets serial multipart latency.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with reasoning, code editing, shell
execution, and automated tests. The context-window size is not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 15:01:34 -06:00
Devin FoleyandPaperclip 49b3804830 fix(connections): load permissions and show AI account identity (#18653) 
## Thinking Path

> - Paperclip manages AI agents and their access to saved connections.
> - The Permissions page loads account details and permitted agent
choices.
> - Company-wide AI accounts repeat retained installation checks for
every candidate agent.
> - This makes target checks grow with the square of the roster size.
> - The browser waits without a deadline, and connection names do not
identify the provider login.
> - This pull request batches direct AI checks and bounds loading with
recovery controls.
> - Account details load separately and show a provider email or an
explicit profile-permission limit.

## Linked Issues or Issue Description

**What happened?**

Opening a saved AI account can leave Permissions on “Loading AI
account…” for minutes. The account itself can be connected. The existing
capability batch excludes AI connections, so a company-wide install
repeats checks for every installed agent for every candidate. Saved
names and Paperclip ownership also do not establish which provider login
a credential uses.

**Expected behavior**

Direct AI accounts reuse retained-target checks within one descriptive
request. The page shows loaded account controls or a bounded failure
with Retry. Permissions shows the provider email and organization when
the selected credential permits profile access. An unavailable profile
must not block maintenance or disconnect a working inference credential.
Writes still authorize current permissions.

**Steps to reproduce**

1. Save an AI account in a company with several agents.
2. Install the account company-wide.
3. Open its Permissions page.
4. Delay the account-list or grants response to exercise the loading
failure.

The maintained benchmark reproduces database work with synthetic local
data. Refs #17945, which added batching for non-AI connections. Searches
found no open PR for this AI-account loading fix.

## What Changed

- Include direct AI accounts in the existing board capability batch.
Referenced pool members, delegated actors, complex selectors, and
mutation authorization keep their existing evaluator.
- Add opt-in per-caller GET deadlines to the API client. A timed-out
caller detaches the stuck coalesced request so Retry starts a fresh
fetch. Other callers retain their own cancellation and results.
- Apply a 15-second deadline to AI account lists and grants reads. The
permissions view retries transient failures once and reports other
failures immediately.
- Share the parent Permissions grants query with AI account details.
Keep loaded controls visible after a background error and offer Retry.
- Add a grant-scoped identity GET and OpenAPI contract. The service
checks current membership and credential audience before it reads the
selected credential. It projects only the provider email and
organization; provider IDs, tokens, and raw errors stay private.
- Show account details independently of maintenance controls. Anthropic
profile denial has an explicit message and preserves connected status.
Transient profile failures have a separate Retry action. Observations
are cached for five minutes under the company, connection, grant, and
grant revision.
- Extend real PostgreSQL and authenticated HTTP coverage,
client/component regressions, map guidance, and the existing benchmark's
AI mode.
- Address the Apex file-size finding by moving gallery identity lookup
into the existing app display helper module. `AppDetail.tsx` is 984
lines; lookup behavior is unchanged.

## Verification

Local source checkout on macOS, Node 25.6.1, pnpm 9.15.4, and disposable
PostgreSQL 18.1. Provider calls use fixtures. No hosted deployment or
provider qualification is claimed.

- Passed `pnpm -r typecheck` and `pnpm build`.
- Passed all 87 tests in `connection-management.integration.test.ts` and
all 33 tests in `connection-install-capabilities.test.ts`.
- Passed 196 AI connection, identity-provider, and OpenAPI tests:
`ai-connections.test.ts`, `ai-account-identity.test.ts`, and
`openapi-routes.test.ts`.
- Passed 107 focused UI/client tests: `client.test.ts`,
`ManagedAiConnectionDetails.test.tsx`,
`AiConnectionAccountDetails.test.tsx`, and `AppDetail.test.tsx`.
- After the Apex extraction, passed UI typecheck/build and 106 component
tests across `AppDetail.test.tsx`, `AppNotConnected.test.tsx`,
`ManagedAiConnectionDetails.test.tsx`, and
`AiConnectionAccountDetails.test.tsx`.
- The authenticated HTTP tests compare complete grants responses across
human roles, for tool and AI connections. They verify a real install
mutation succeeds before revocation and fails afterward.
- A company-wide personal AI fixture checks 20 targets instead of 400.
Pool-member permission loss still denies the complete pool description.
- UI states covered: shared cached grants, stuck account/grants fetches,
one automatic retry, terminal failure, manual recovery, authorization
failure, background failure with retained controls, and unmount
cancellation. Client tests also cover stalled response bodies and
independent coalesced callers.
- Identity coverage includes actual bearer-key/JWT reads, private-user
and cross-company denial before provider access, revoked credentials,
provider 401/403/429/500, and grant-change isolation. A real local HTTP
fixture stalls the response body, verifies the shared 15-second
deadline, and keeps unrelated HTTP requests responsive. The test
shortens the native deadline to 100 ms. Profile permission denial leaves
the saved account connected.
- Passed `pnpm check:token-gates`, `node scripts/check-maps.mjs`, `node
scripts/check-module-boundaries.mjs`, and `git diff --check`.
- The full local `pnpm test:run` started with the loading-only change.
All six chat shards passed. It recorded one failure in the unchanged
`heartbeat-retry-scheduling.test.ts` quota-rejection case. That exact
case passed a focused rerun on the final head. The initial failure cause
remains unconfirmed and its local log is retained; this is not a full
local-suite pass. The redundant local run was stopped with SIGINT after
final-head CI completed successfully.
- [Final-head
CI](https://github.com/paperclipai/paperclip/actions/runs/41515556725)
passed on `ead8d276d4a2023614fff54b3e9a858f28c3178b`: typecheck, build,
all 12 server shards, all 3 chat shards, workspace suites, all 9
serialized suites, all 8 end-to-end shards, Runner checks, and the
aggregate verify gate. Security checks passed. Optional Storybook checks
were skipped by workflow policy. GitHub reports the branch mergeable
without conflicts.
- Apex reviewed head `ead8d276d4a2023614fff54b3e9a858f28c3178b` at 5/5,
confirmed the extraction resolves its earlier file-size finding, and has
no unresolved review threads.

The existing benchmark ran for the loading-only change now recorded in
commit `5f725b36fa52002545241a5a8b7a10602f0cc0ce`, with `--purpose=ai
--baseline-ref=026fd9e3452`. The identity follow-up leaves that batch
source unchanged. Two warmups and five alternating samples exercise each
of eight fixture shapes. Complete positional capabilities, reasons, and
operations match the baseline in every sample.

| AI fixture | SQL calls before → after | Target reads before → after |
| --- | ---: | ---: |
| 1 agent, company install | 9 → 9 | 1 → 1 |
| 32 agents, company install | 5,248 → 164 | 1,024 → 32 |
| 100 agents, company install | 50,400 → 504 | 10,000 → 100 |
| 100 agents, partial install | 13,175 → 503 | 2,575 → 100 |

These counts measure the real service over local PostgreSQL. They do not
predict hosted page latency. To reproduce the standalone benchmark, use
a checkout of commit `5f725b36fa52002545241a5a8b7a10602f0cc0ce` and run
the command below. Its isolation check intentionally rejects the later
identity follow-up's additional runtime changes.

```sh
node --import ./server/node_modules/tsx/dist/esm/index.mjs scripts/benchmark-connection-install-capabilities.ts --purpose=ai --baseline-ref=026fd9e3452
```

Run the final-head regression suites from the repository root:

```sh
cd server
pnpm exec vitest run src/__tests__/connection-management.integration.test.ts src/__tests__/connection-install-capabilities.test.ts src/__tests__/ai-connections.test.ts src/services/ai-account-identity.test.ts src/__tests__/openapi-routes.test.ts
cd ../ui
pnpm exec vitest run src/api/client.test.ts src/components/ai-connections/ManagedAiConnectionDetails.test.tsx src/components/ai-connections/AiConnectionAccountDetails.test.tsx src/pages/apps/AppDetail.test.tsx
```

## Risks

Descriptive permissions reuse retained facts within one request. A
concurrent change can leave a displayed choice stale; the next request
and every write still authorize fresh. Pools with referenced members
retain their existing cost.

The account and grants clients now bound each caller's wait. A browser
abort does not cancel an authorization read already admitted by the
server. The independent profile request uses a fixed Anthropic origin, a
bounded body, no redirects, and a 15-second deadline. Inference-only
credentials can return no email; that is reported without guessing from
a Paperclip owner or changing connection health. Profiles are fetched
only for the caller's authorized selected grant. No ambient CLI
identity, persistent identity cache, background polling, or
billing-identity change is added. Other API reads keep their existing
behavior. No migration is required. Staging has not been redeployed.

## Model Used

OpenAI `gpt-6.1-sol` through the Codex harness in T3 Code, with xhigh
reasoning, code editing, execution, and test tools. The host does not
expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 12:29:03 -08:00
DottaandPaperclip 33fc7fcddc fix(db): clean up owned PostgreSQL test servers (#18627) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Database, server, and CLI tests use real PostgreSQL fixtures.
> - Each fixture must own its server until PostgreSQL has stopped.
> - The old teardown returned after five seconds even when its server
remained alive.
> - A startup failure could then mark the host unsupported and skip
database coverage.
> - This pull request waits for owned shutdown and makes supported-host
startup failures fail collection.
> - A shared-server command reduces process creation while keeping each
fixture's database isolated.

## Linked Issues or Issue Description

Refs: #7756 for related development-process ownership. That
development-server issue remains outside this test-fixture fix.
Related PR: #16799 collects macOS startup diagnostics. This PR changes
test shutdown and failure behavior.

**What happened?**

Repeated test runs can exhaust macOS PostgreSQL semaphore resources. The
fixture abandoned slow shutdown and swallowed stop errors. The support
probe converted ordinary startup failures into skipped integration
coverage.

**Expected behavior**

Tests must stop their PostgreSQL servers, retain files if stop fails,
and report resource exhaustion as a failure. Fixtures on a shared server
must not close another database's clients.

**Steps to reproduce**

Start a fixture whose PostgreSQL stop exceeds five seconds, or make a
supported-host startup fail with semaphore exhaustion. The former helper
returned from teardown or reported unsupported coverage. The new
regression tests exercise both paths without exhausting host resources.

**Paperclip version or commit**

Source checkout based on current master.

**Deployment mode**

Local source development and CI tests.

## What Changed

- Add explicit ownership for test PostgreSQL children. Wait for fast
shutdown, then request immediate PostgreSQL shutdown if needed. Keep
cleanup idempotent and fail when exit cannot be confirmed.
- Publish reusable templates only after clean shutdown. Do not publish a
crashed or immediately stopped cluster.
- Stop owned children on POSIX worker signals and process exit. Preserve
test directories when shutdown fails. Remove conflicting dependency exit
hooks.
- Retry port collisions only. Fail collection for resource exhaustion,
unavailable configured servers, and other startup failures on supported
platforms. Preserve skips for explicitly unsupported platforms.
- Move administrative-server support into the common fixture used by
database, server, and CLI tests. Close clients at database scope and
migrate cloned templates before use.
- Add a directly invoked shared-server runner to own one test server and
run suites against separate disposable databases. Preserve child exit
status and support an existing disposable server.
- Unify cancellation: stop and wait for the test process group before
PostgreSQL teardown. Add nested-process coverage, real worker-exit
coverage, shutdown regressions, shared-database isolation checks, and
setup documentation. Replace ignored heartbeat completion polls with the
existing execution drain before asserting persisted outcomes.
- Finish owned cluster-directory cleanup after signal-driven
cancellation. Preserve prepared templates across shared-fixture tests.
Remove stopped directories asynchronously, using native removal on POSIX
with a Node fallback.
- Align database and CLI fixture hook budgets with the existing
30-second server budget. This lets the ten-second shutdown protocol
finish and report errors. The idle-sleep suite uses the existing
90-second fixture budget for its unusually large final directory
cleanup.

## Verification

- The semaphore-exhaustion regression failed against the old helper
because it resolved to `supported: false` instead of rejecting.
- All 37 focused tests pass on the final runtime code (`e4de00ab40`).
They include real worker exit, SIGTERM, nested-process SIGINT/SIGTERM
cancellation with PostgreSQL available during child cleanup, failed
template shutdown, unsupported platforms, stale SQL templates, and
directory removal. A child exiting with status 7 makes the shared-server
runner return status 7 after cleanup.
- Full local `pnpm -r typecheck` and `pnpm build` pass on the final
runtime code (`e4de00ab40`). Subsequent changes only adjust server-test
lifecycle waits, the idle-sleep cleanup budget, and documentation; the
server typecheck also passes with those final tests.
- A fresh shared-server run on `90b65fd8ad` passed all 68 database files
and 297 tests. Earlier preparation runs had two ten-second teardown
timeouts and a stale-template failure; the final helper migrates cloned
templates and permits the full shutdown budget. These earlier failures
remain accounted for here.
- The local full repository attempt stopped on a Discord receiver-lease
test after 338 passing chat tests. That exact case passed in isolation.
This full-suite attempt is not a pass.
- On `90b65fd8ad`, [server shard
6](https://github.com/paperclipai/paperclip/actions/runs/41496495253/job/125452949369)
and its one unchanged retry reported two 15-second cached-input
heartbeat timeouts and a 30-second idle-sleep teardown timeout. All ten
cached-input cases then passed locally with their original deadlines. A
local parallel run of the full files reported eight behavioral/hook
failures and two suite teardown timeouts (281 tests passed).
- Serial idle-sleep runs passed all 225 test assertions but overran the
30-second final hook. Tracing measured client closure in 4 ms,
PostgreSQL exit after 5 seconds, and synchronous directory removal
taking another 35 seconds. Asynchronous Node removal and native removal
also exceeded that hook under the full workload. Cleanup now remains
asynchronous and awaited, and only this heavy suite's final hook uses
the existing 90-second fixture budget. No PostgreSQL shutdown deadline
or test assertion was weakened. The serial run then passed the complete
idle-sleep file, including all 225 tests and teardown. Its heartbeat
file passed 63/64 cases, exposing an ignored three-second completion
poll. After replacing that poll with an execution drain, another timer
case exposed the same pattern (63/64 passed). All remaining ignored
completion polls now await execution drain without changing test
deadlines. The complete heartbeat file then passed all 64 tests. The
intentional asserted lock-contention poll is unchanged. The final server
typecheck passes, and a process audit found no PostgreSQL server left by
the completed local runs.
- On final head `2cf316d3f571e4516890cba743ecffeb9703cce3`, the
previously failing server shard 6, database lane, full typecheck, and
build passed. The [first CI
attempt](https://github.com/paperclipai/paperclip/actions/runs/41506358136)
then lost self-hosted runners in the canary and serialized-server lanes,
with additional jobs cancelled. The server-2 lane had one 30-second
timeout in the unchanged workspace-recovery HTTP Retry test (2,282 other
tests passed). That exact test passed locally with its original deadline
(28.78 seconds including imports). One unchanged retry of only failed
and cancelled jobs completed successfully on the exact final commit
(workflow attempt 2). The workspace-recovery timeout case and every
interrupted lane passed; successful first-attempt jobs were retained.
The additional Paperclip Review check remains queued, waiting for its
assigned agent; it has no unresolved GitHub review threads. Greptile
completed review on this exact final head at 5/5, with zero unresolved
review threads. All four original review findings and their threads are
resolved; additional Windows shutdown, cancellation-directory, and
template-reuse findings are fixed.
- `pnpm check:module-boundaries` and `git diff --check` passed. Post-run
process checks found no PostgreSQL servers left by the completed
verification runs.

## Risks

- Immediate PostgreSQL shutdown can require crash recovery. This
fallback applies to disposable test clusters and runs only after fast
shutdown times out.
- No process can handle `SIGKILL` or machine power loss. Host resource
cleanup remains a separate backstop.
- Directory cleanup on cancellation applies to the shared-server
runner's own cluster. Directly terminated test workers stop their
PostgreSQL process but retain disposable files for inspection. The
process-exit guard cannot await file removal.
- The two-second `pg_ctl -W` deadline bounds signal dispatch. It is
separate from the five-second and thirty-second server-exit waits. A
control-command failure fails cleanup and retains its directory; it does
not report successful cleanup or force-delete live files.
- Shared fixtures require a disposable PostgreSQL server and a role that
can create databases. The directly invoked shared command is opt-in; the
existing `pnpm test` command remains available.
- These changes do not stop unrelated development servers or delete
host-wide IPC resources. Clean shutdown uses the owned `pg_ctl` binary
on every supported platform. Its real control path was tested on macOS.
A Windows host was not available for live verification; POSIX signal
tests skip on Windows.

## Model Used

- OpenAI Codex, based on GPT-6. The runtime does not expose the exact
serving variant or context-window size. The agent used reasoning, shell
execution, code editing, and GitHub tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 14:24:48 -06:00
DottaandPaperclip 53f480d1b3 fix(native): preserve warm continuity and deferred review replies (#18631) 
## Thinking Path

> - Paperclip manages AI agents and their saved work.
> - Product E2E tests must check saved personal files across separate
turns.
> - The server records file collection in the durable `instruction_save`
run-log event.
> - The warm test read optional terminal result metadata instead and
rejected a recorded successful save.
> - This pull request reads the complete event stream for the current
run and uses its latest save receipt.
> - The prompt now specifies the exact nine bytes required by the
temporary-file transfer count.
> - Human review actions select the actual interaction card, which can
share its interaction ID with a navigation button.
> - A later real three-turn run exposed a missing reply after workspace
finalization moved to its durable recovery owner.
> - Reply recovery handled completed tasks but omitted ordinary tasks
that awaited review.
> - This pull request also publishes their recorded final reply once
while preserving the pending review.

## Linked Issues or Issue Description

**What happened?**

The [retained live continuity
attempt](https://github.com/paperclipai/paperclip/actions/runs/41477302876/job/125420001074)
saved the correct memory file and unchanged 8 MiB binary. Its durable
run log contained a successful `instruction_save` receipt. The oracle
read `resultJson.instructionSave`, which was absent, and failed before
turn two. The temporary file also had a trailing LF. Its prompt allowed
that result, while the byte-count check required nine bytes.

**Expected behavior**

The oracle must use the documented authoritative save event. A missing
receipt, a later failed save, incorrect transfer counts, or another
run's events must fail. The prompt must specify the bytes that its
independent check requires.

**Steps to reproduce**

1. Run `daytona-warm-continuity.runner-codex.daytona.warm-three-turn`
with the configured protected credentials and immutable image.
2. Complete a native turn whose terminal result has no `instructionSave`
field.
3. Compare the oracle output with the same run's successful
`instruction_save` event.

A subsequent [three-turn
attempt](https://github.com/paperclipai/paperclip/actions/runs/41495337334/job/125451304553)
passed every personal-file assertion and reused the same warm process.
Its first final reply was recorded as a completed agent-message event
but was absent from comments. The log shows
`native_workspace_finalization_busy`; the durable recovery path excluded
`needs_review` replies. The final message matcher correctly failed. This
failure remains recorded.

The observed application revision was
`aab87ce70744660a4a21c77278b0ede267c04720`. Related work: #14695 and
#18494. The retained failed result keeps its original grade and
definition.

## What Changed

- Read all run-event pages through the existing bounded collector.
- Require events to belong to the requested run and select the latest
`instruction_save` event.
- Retain strict byte counts and reject missing, failed, pending or
superseded receipts.
- Specify nine ASCII bytes and no trailing newline for the temporary
file.
- Scope human review actions to the interaction card and its exact
interaction ID.
- Advance the warm suite definition to version 5 and document its
evidence source.
- Recover recorded final replies for ordinary native tasks with an
accepted `needs_review` result and terminal state.
- Preserve pending reviews, existing reply selection,
company/turn/digest checks and external governed-response authorization.
- Update the task feature and agent access maps.

## Verification

- Real PostgreSQL regression: deferred review reply discovery failed
before the production repair (expected one candidate; found zero). The
final-message event and pending review were present.
- Recovery cases cover one publication, no repeated provider run,
unchanged review, and rejection of absent/wrong-turn final messages, bad
digest and suppressed comments.

- Red: seven new boundary regressions failed against the old receipt
lookup.
- Green: 169 focused oracle, event pagination and catalog checks passed.
Product E2E typecheck passed after rebuilding local package
declarations.
- Retained-evidence replay confirms that the new lookup finds the real
successful save event. The original attempt remains failed because its
temporary file included an LF.
- Recursive typecheck passed before the final recovery change; server
typecheck passed afterward. The recovery and control-plane suites passed
all 112 tests. Full build and map checks passed.
- The broad local test run reached one unrelated 30-second timeout in
`attention-retention-work.test.ts` (maintained-review/confirmation
batching). The same unchanged case then passed on one targeted retry
(39.9 seconds including module load). The original failure remains
retained. The broad run also reported one unchanged workspace-conflict
case, which passed on its targeted retry (14.85 seconds including module
load). Both original failures remain retained. The redundant broad local
run was stopped after current-head CI completed all required gates: 55
successful checks and 3 selection skips. No all-green local broad-run
claim is made.
- The separate artifact-transfer cell already passed on the recorded
application revision: independent source and delivered bytes matched at
407,485 bytes, with a completed task, succeeded native run and cleared
execution locks. This does not establish a three-turn continuity pass.

- [Real native three-turn
acceptance](https://github.com/paperclipai/paperclip/actions/runs/41499102994/job/125465795210)
**passed** at `3b207a42943e9eeb371d4f2d5623de72093eb6d5`, suite
definition 5. Turn one exercised `committed_task_response_recovered`.
All three completed final replies were persisted once and in order. All
three strict personal-file receipts passed. The unchanged 8 MiB binary
survived; later transfers copied only 36 and 54 bytes. The lease was
created, resumed, and resumed with the same runner PID/start time,
runner instance, native session and provider session. Final task was
`done`, all three runs succeeded, and checkout/execution locks were
null.
- The aggregate startup summary has no separate provider-process PID
proof. The raw run records independently prove the stable runner process
and provider session stated above. This single native cell does not
qualify the legacy cell or a full provider matrix.
- The passing campaign reused the signed immutable runner image. It
required no disk expansion. Earlier campaign image builds used the
recorded temporary 120 GiB mitigation; default 60 GiB build capacity
remains unqualified. All scoped completed probes were removed and
independently returned 404; org CPU/disk usage returned to zero. This
does not resolve unknown historical resources under a different
credential identity.
- Current-head Greptile is 5/5; review-thread pagination is complete
with zero threads. All 55 current-head checks are successful; 3
selection checks are skipped. The paid campaign, result enforcement and
report publication all completed successfully.

## Risks

- Publishing a reply must not approve the work. Recovery only copies the
accepted turn’s completed final message. It preserves the pending review
and does not start an executor.
- Conversation and external governed-response paths retain their
existing separate authorization.

The latest save event can record failure after an earlier success. The
oracle rejects that case. Event pagination, missing receipts and
cross-run evidence also fail closed. The exact temporary-file byte count
remains required. The review selector still requires the intended
interaction ID and enabled action on its actual card. The production
change is limited to ordinary internal-task reply recovery. It still
requires an accepted result, a matching successful terminal disposition,
the completed final message, and existing publication authority.

## Model Used

OpenAI Codex, based on GPT-6, with shell and code tools. The exact
deployment variant and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 14:19:48 -06:00
DottaandPaperclip 0fd82e386e feat(evals): stream completed chat catalog results to TestOps (#18647) 
## Thinking Path

> - Paperclip lets people manage AI agents and inspect their work.
> - The chat catalog uses real Product E2E cases and reports their
original grades to TestOps.
> - The current report matrix waits for every selected worker.
> - A large selection can hide completed results for hours.
> - Each case can use the existing private reader and safe exporter
after its own worker ends.
> - This PR gives each case a separate upload and readback job, then
verifies final coverage without another upload.
> - People can inspect completed cases while the rest of the selection
runs.

## Linked Issues or Issue Description

Refs #18542, #18625.

**What existing behavior does this improve?**

The existing chat catalog reports results to TestOps only after all
selected workers finish.

**Current behavior**

One slow case holds back the report and upload jobs for all other cases.

**Proposed behavior**

Each finished case gets its own protected reader, safe projection,
upload and backend verification. The final job checks the whole
selection through read-only requests. It does not upload results again.

**Reason and benefit**

A person can inspect a completed grade and its safe evidence receipts
while other cases run. This changes future runs. It does not change an
in-flight run.

## What Changed

- Add one same-commit reusable workflow with separate worker,
private-reader and TestOps-upload jobs. Each job uses only its own
protected environment. Callers pass empty named secret placeholders and
do not inherit secrets.
- Reuse the original exporter and grade checks. Freeze each case's three
Allure JSON files once. Allow known sibling results during per-case
readback. Keep final exact selection coverage.
- Submit each bundle at most once. Reconcile uncertain responses by
readback. Retain only closed publication, failure and upload-intent
receipts. A rerun attempt still fails the existing attempt-one admission
guard.
- Preserve failed original grades, failed cleanup and missing evidence.
No missing original can become a passing result.
- Keep external-input cases in the serial lane. Small native-only
selections allow two cases at once. Small mixed selections allow one
native and one external case. Larger selections retain eight native
cases and one external case.
- Add source, privacy and failure controls to the existing unit roster.
Document the reporting flow and runner-group admission requirement.
- Fix an existing corruption-test fixture that could write the same byte
back to a random IV. Read and flip one bit instead. Keep the storage
rejection and unpublished-plaintext assertions unchanged.

## Verification

- 152 focused Node controls pass across the shared client, API,
selection, CLI and streaming tests. The retained parent controls show
the missing multipart and partial-readback behavior, plus the small
external-selection error before its fix.
- The focused workflow security control passes. Actionlint passes for
both workflows. Changed JavaScript files pass syntax checks. Git diff
checks pass.
- Local controls use synthetic data and mocked HTTP. They make no
provider call and do not start an app, database or native build.
- Previous head `b51406df` passed PR CI (48 jobs pass and one is
skipped). The review-driven test-only successor `0c65cc21` then failed
one unchanged snapshot-storage corruption test, plus the final CI
summary. The latest head includes its narrow fixture repair. Its first
CI attempt separately hit runner shutdown/lost-communication failures,
two setup/timeouts and a heartbeat-cancel cleanup HTTP 500/lock timeout;
those observations are retained without claiming their underlying cause.
One same-head failed-and-cancelled-only recovery now passes at
`bf9c3e8f`: 48 jobs pass and one is skipped, including required `ci /
verify` and `ci / e2e`. All 31 already-passing jobs retained their
original execution records; all 15 recovered leaves and both dependent
summaries passed. The first attempt remains recorded; the recovery does
not establish the underlying causes of those failures. A small AWS run
after delivery must prove that one case and both attachments are visible
before another original case ends. That live proof is pending; local
controls do not prove vendor ingestion or environment-secret resolution.
- Retained earlier fixture-calibration failures remain recorded. The
final integrated Node control run is green. A deterministic Node
crypto/filesystem control also reproduces the no-op corruption mutation
with a valid IV byte equal to `X`, then passes after flipping that byte.
This control uses no app or database. The failing CI byte was not
captured, so the exact historical byte remains unknown.

## Risks

- The called workflow needs its exact master ref in the existing
selected-workflow runner group. No IAM role change is needed.
- Environment-only secrets must resolve in the called jobs. The caller
passes no actual model or reporting key. The small live proof must
confirm this behavior.
- TestOps does not document an idempotency key for this upload route. An
uncertain write is never resent automatically. If readback cannot verify
it, the workflow fails with retained closed receipts.
- The existing job run can already be marked finished by the selection
watch. The live proof must record its actual stage and prove subsequent
uploads are accepted. This PR does not reopen, stop or close that job
run.
- Backend ingestion does not prove that an already-open TestOps page
refreshes automatically.
- The corruption-test repair changes no production storage or
cryptography. New-head AWS CI must qualify the existing database-backed
owner test.
- Each case adds short protected reader and upload jobs. The app source,
original case clocks, graders, model, budgets, raw-evidence privacy
boundary and zero eval retries stay unchanged. The versioned stream
concurrency group serializes new streaming runs; an old immutable batch
run can continue in its original group.

## Model Used

OpenAI `gpt-6.1-sol`, with tool use and code execution. The session does
not expose a context-window size. Independent source peers used the same
model family.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 14:18:00 -06:00
DottaandPaperclip 946735d03a Persist task sorting in URLs and show newest routine tasks first 
Save task sort choices in URLs across task views. Open routine tasks by creation time, newest first, with stable sorting before pagination.

Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com>
 2026-11-02 13:32:40 -06:00
DottaandPaperclip 5a07c30c1a fix(workspaces): reduce temporary disk use during remote capture (#18644) 
## Thinking Path

> - Paperclip manages AI agents and preserves their completed work.
> - Native remote finalization must save workspace files before it
releases the writer.
> - Portable capture copied the working tree and Git history, repacked
the history, and then wrote an archive.
> - These temporary copies can exhaust a small sandbox after normal
development work.
> - A completed provider turn then fails finalization and file
preservation repeats the same failure.
> - This change streams checked working files and keeps only an
independent Git copy in scratch.
> - The save still rejects changed sources and incomplete output.

## Linked Issues or Issue Description

Related: #18403 added portable remote workspace storage. #18172 handles
disk pressure in Daytona directory transfer. This failure occurs
earlier, while the portable archive helper creates its source archive.
That earlier repair does not cover this path.

**What happened?**
A completed native run failed workspace finalization three times. Its
file-preservation retries also failed. In a retained 10 GB Linux sandbox
with about 1.5 GB free, the deployed helper failed during `git repack -a
-d` with `Out of diskspace`. The working files used about 603 MB, and
Git used about 444 MiB.

**Expected behavior**
Capture a coherent workspace when free space can hold one full archive
plus an independent Git copy. Preserve working files, staged changes,
unpushed commits, and existing recovery gates.

**Steps to reproduce**
1. Use a disposable Linux sandbox with a Git repository and generated
working files.
2. Leave enough free space for an archive and a Git copy, but not a
second working tree and another repack.
3. Run the portable capture helper. The old helper fails while creating
temporary data.
4. Run the fixed helper against the same source and free-space limit.
Verify each archived file hash.

The small regression uses real file serialization and a controlled
filesystem capacity failure at archive creation. It failed with `ENOSPC`
before the production edit and passes afterward.

**Paperclip version or commit**
The observed staging server served
`42c64d4ec4b9b0898556fcdf6ba304c4285a7819`. The repair is based on
`a92f616cd0`.

**Deployment mode**
Hosted staging. Linux x64, Node 25.9.0, Daytona sandbox. Local tests use
a macOS source checkout.

## What Changed

- Stream working files from checked file handles. Keep only normalized
Git metadata in scratch.
- Remove the second repack after a mirror clone that already uses
`--dissociate` and `--no-hardlinks`.
- Check each streamed file's hash, size, mode, and filesystem identity.
Recheck the source inventory and Git state before returning a receipt.
- Remove incomplete archives after failure or cancellation. Keep
pre-existing archive files intact.
- Add space-limit, concurrent-change, cancellation, and existing-file
safety cases. Update execution semantics and workspace verification
recipes.

## Verification

- Red before the production change: the capacity regression failed with
`ENOSPC`. Green afterward: 54 archive and transport tests pass.
- The existing tests restore ordinary, linked, and borrowed Git
repositories after their sources disappear. They retain staged,
unstaged, and unpushed work. Credential and path guards also pass.
- Live comparison on the same retained sandbox: the deployed helper
failed after 19.4 seconds with Git disk exhaustion. The fixed helper
completed in 35.3 seconds. It produced a 1,081,872,803-byte archive with
24,111 entries. An independent reader checked every archived file hash
and the manifest digest. The source commit stayed unchanged. Diagnostic
scratch was removed.
- This live probe exercised the changed capture boundary. It did not
deploy a server, upload a production snapshot, finalize the failed run,
or replay its provider turn. Full task recovery still needs rollout and
the normal export recovery path.
- `node scripts/check-maps.mjs` and `git diff --check` pass.
- Current-head CI passes: [PR
workflow](https://github.com/paperclipai/paperclip/actions/runs/41499985180),
commit `d42700f7f4b0bef14a5969a44c2da0b2513e0ed2`. Greptile rates the
same head 5/5 with no actionable findings or unresolved threads.
- The independent Ponytail advisory also rates this exact head 5/5 with
no findings. Its structured check remains queued because the automatic
request was bound to the earlier head. A [fresh current-head review
request](https://github.com/paperclipai/paperclip/pull/18644#issuecomment-6585296011)
was accepted and queued. Its predecessor has a successful workspace
export but remains active in finalization, so the new review cannot
start yet. This review-service publication gate must complete before
claiming every check is green.
- `pnpm -r typecheck` and `pnpm build` pass. The final server-only
typecheck also passes after the test additions.
- Supported-runtime verification on Node 24.11.0: all 54
archive/transport tests and all 46 existing workspace lifecycle tests
pass. The lifecycle suite uses a disposable real Postgres database and
local filesystem transport with encrypted snapshot storage. It checks
save/restore after losing the original allocation, ownership, retention,
and file continuity. This is separate from the live Daytona capture
probe.
- The first extra lifecycle attempt could not start its local Postgres
binary because the installed Darwin package lacked library aliases.
Local dependency setup was repaired before the passing run. No
production or repository code changed for that setup.
- The full local `pnpm test:run` on the host default Node 26.4.0 stops
in chat-test collection with `No test found ... in lines ...`; it has no
suite verdict. A clean retry repeats the failure. The unchanged
`a92f616cd0` test wrapper reproduces it with the same installed
dependencies, after correcting an initial baseline setup that lacked
built package entries. This is a local test-runner limitation, not a
full local pass. Current-head CI remains the full-suite gate.
- One focused run timed out while deeply comparing 4 MB buffers under
concurrent build load. Complete SHA-256 comparisons retain the content
check without that assertion cost. The capacity case then passes in
under one second, and the final 54-test run passes. The disk allowance
and production code did not change.

## Risks

- Streaming reads the live source instead of a copied working tree. File
identity, content hashes, mode checks, the final inventory read, and Git
signatures must reject concurrent changes. Regression tests exercise
these checks.
- Capture still needs space for the full archive and normalized Git
metadata together. It cannot export from a completely full disk.
- Archive framing and restore validation stay compatible. The change has
no database migration or new execution authority.
- The incident remains unrecovered until the fixed server performs
normal workspace finalization. No deployment or merge is included in
this PR.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
editing, shell tools, browser diagnostics, and automated tests. The
exact serving variant and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 13:22:23 -06:00
DottaandPaperclip 026fd9e345 fix(server): keep controller identity independent of Git probes (#18649) 
Read the existing boot timestamp without synchronous Git diagnostics. Keep the voice-retention precision test clock fixed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
 2026-11-02 13:11:41 -06:00
DottaandPaperclip a92f616cd0 fix(db): attribute bounded captures to execution owners (#18634) 
## Thinking Path

> - Paperclip manages agent work through shared server and database
services.
> - High query volume can spend main-thread time on SQL construction and
result decoding.
> - The existing bounded response capture counts rows and bytes, but it
cannot name the operation that executed a query.
> - Bundled callers can be missing, and measuring every parsed JSON
value adds work to the diagnostic.
> - This pull request carries a semantic execution owner into each
measured query and bounds response sizing.
> - Operators can rank query volume by owner before choosing a
performance fix.

## Linked Issues or Issue Description

Refs #17504, #17522, #17537, #17564.

**What existing behavior does this improve?**

The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic.

**Subsystem affected**

Database diagnostics and existing server execution boundaries.

**Current behavior**

The capture groups by table, operation, and builder caller. It cannot
distinguish execution owners that reuse a prepared query. Bundled
deployments need source maps for caller locations. The capture visits
every returned value and serializes parsed JSON again to estimate its
size.

**Proposed behavior**

Version 2 captures use fixed semantic owners for HTTP requests, delivery
queues, execution phases, and budget work. They count completed calls,
returned rows, errors, and completion wall time. They sample successful
results 1, 17, 33, and so on, with fixed cell and string/byte limits.
Complex values remain explicitly unmeasured. Semantic ownership works
without source maps.

**Reason and benefit**

A short census can identify which operations issue the most queries or
return the most rows. It preserves each query's execution owner through
pooled connections, concurrent operations, and prepared-query reuse. It
reduces diagnostic work without claiming per-query CPU attribution.

**Breaking changes**

Opt-in capture output moves from `entry.v1`/`summary.v1` to
`entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace
all-result byte estimates. Consumers must use the new fields and must
not extrapolate them into a total. There is no database migration or
application API change.

## What Changed

- Added `withDatabaseOperation(owner, action)` to the DB package. The
server owns semantic labels. Queries retain execution ownership in their
completion closure.
- Added fixed owners at existing HTTP, delivery, budget, and execution
timing boundaries. Delivery work replaces the context of the producer
that woke it.
- Added complete call/row/error counters and completion wall durations
within the existing bounded window. Durations include waits and are not
CPU measurements.
- Sampled and bounded primitive-value sizing. Complex and oversized
values remain unmeasured. Known owners skip repeated stack extraction.
- Preserved the deadline, group limit, paced output, query outcomes, and
existing coverage exclusions. Updated the operator documentation.
- Added real PostgreSQL owner-isolation and prepared-query tests, server
scope tests, and plain bundled-runtime proof.
- Moved projection-test and benchmark full-response sizing into an
explicit test helper. Existing payload-size thresholds remain unchanged;
these synthetic checks do not use the production sampler.

## Verification

- Final-commit DB capture suites: 43 tests passed in 13.86 seconds
against the actual PostgreSQL driver, including concurrent owners,
prepared reuse, transaction failure, a dedicated client, sampling, and a
query crossing the deadline. DB typecheck passed.
- Server attribution and existing delivery/execution/budget suites: 53
tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5
tests passed.
- Runtime bundle suite: 8 existing tests passed. The new no-source-map
owner test passed after its fixture was corrected to use a driver
thenable. An ordinary native Promise can bypass an overridden `.then`;
the real PostgreSQL tests separately cover driver integration.
- Observer-only experiment: three alternating pairs per case on Node
24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an
already-materialized synthetic JSON result, median capture CPU fell from
82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell
from 8.10 to 4.40 microseconds/call. Every result retained identity and
all calls were counted. The new diagnostic intentionally measures less
byte data; these are observer costs, not database latency or a hosted
application speedup. Disabled-case timings were below one
microsecond/call and too small for a performance conclusion.
- Final lifecycle regression: 28 unit tests and DB typecheck passed.
Closing the last capture disables and replaces owner storage;
overlapping captures retain it.
- Final-commit full recursive typecheck passed in 38.38 seconds and full
build passed in 40.81 seconds. The build stamp matches
`83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution
tests also passed 5/5 before the test-helper-only migration. Full test
acceptance will use the repository's parallel CI gates; the unsharded
local command takes hours.
- Verification failures retained: initial sandbox runs could not open
local IPC/PostgreSQL listeners; permitted reruns exercised the real
services. One old test expected JSON serialization and was updated to
assert explicit unmeasured complex values. The first bundle fixture used
the wrong promise boundary, as described above. A retained-callback
regression also demonstrated that disabling and reusing the same
AsyncLocalStorage instance could resurrect an old owner; replacing the
instance after the last capture closes fixes that case.
- The first CI run found older projection tests consuming the previous
full-response fields. Those consumers now use the full-response test
helper, preserving their byte thresholds. The two complete affected
route suites plus helper tests passed 115/115 in 73.08 seconds. A
temporary full-row query negative control then failed the unchanged
2,048-byte limit at 5,000,812 bytes and was restored; five other service
suites passed 34/34 and the affected chat cases passed 3/3. All three
previously failing CI shards passed on the corrected head. A local
skill-list benchmark smoke check completed 255 captured queries with no
in-flight queries or overflow; this was integration verification under
concurrent test load, not a timing benchmark.
- Two final-head runner jobs lost communication with GitHub before
reporting a test result. One targeted retry of those jobs and their
dependent aggregate passed. All 53 final-head checks passed, with 3
intentional skips; previously passing suites were preserved.
- Hosted capture remains pending until the reviewed revision is deployed
and a bounded window is explicitly configured.

Reproduce the focused checks with Node 24 and the repository pnpm
version:

```sh
pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts
pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs
pnpm -r typecheck
pnpm build
```

## Risks

- Owner context is active while a valid future capture window is armed,
up to 30 minutes before accounting starts. No-window and expired calls
take the direct path. Diagnostics still add bounded work while enabled.
- A nearest nested owner is reported. Broad `http.request` and
unattributed groups remain visible; this does not identify every service
callsite or SQL shape.
- The existing census excludes tagged/raw explicit-execute/streaming
paths and internal protocol commands. Its counts are not PostgreSQL
transaction totals.
- Completion wall time includes queue/database/network/scheduling waits.
Concurrent durations overlap. Sampled bytes exclude complex values,
large strings, and values beyond the sizing limits.
- Activation is startup-only. Operators must choose the window after
artifact preparation, allow rollout/warm-up time, and remove the expired
setting afterward.

## Model Used

OpenAI Codex, GPT-6 family, with code execution, repository inspection,
and parallel agents. The exact deployment model ID and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 12:12:21 -06:00
DottaandPaperclip 5f0bb518b0 fix(runner): handle blank assigned-tool schema references (#18637) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents discover assigned app tools through a governed
gateway.
> - OpenCode receives search and call tools instead of the full app
catalog.
> - A blank optional schema reference selected schema lookup instead of
search.
> - Agents received an unknown-tool error and could abandon valid app
access.
> - This pull request treats blank references as search and explains
invalid schema requests.
> - Agents can find their assigned tools while the gateway keeps its
access checks.

## Linked Issues or Issue Description

**What happened?**

An agent called `paperclip_search_assigned_tools` with
`{"query":"github","schemaTool":""}`. The server tried to find a tool
with an empty name. It returned `assigned_mcp_tool_unknown` instead of
search results. The agent then reported that its GitHub tools were
missing.

**Expected behavior**

A blank optional schema reference must permit normal tool discovery. A
bad schema reference must explain how to search again.

**Steps to reproduce**

1. Assign an app tool to a native agent that uses on-demand discovery.
2. Search with `{"query":"github","schemaTool":""}`.
3. Observe the unknown-tool error instead of the assigned tool.

**Paperclip version or commit**

Reproduced from source before this fix, on master at d26628e35a.

**Deployment mode**

Native OpenCode with assigned MCP tools. The shared search code also
serves other native providers.

Related implementation: Refs #17740. A search for `schemaTool` found no
duplicate issue or pull request.

## What Changed

- Treat empty and whitespace-only schema references as normal search.
Accept a zero schema offset in this mode.
- Explain how to correct unknown references and positive offsets without
a reference.
- Test discovery, invocation, and revocation through the authenticated
OpenCode MCP bridge.
- Update tool descriptions, developer guidance, and the execution maps.

## Verification

- The three new regression cases failed before the fix.
- `pnpm exec vitest run
server/src/services/native-runtime/assigned-mcp-tools.test.ts`: 43 tests
passed after the fix.
- Existing tests cover schema chunks, work-mode restrictions,
revocation, approvals, and pinned tool catalogs.
- `node scripts/check-maps.mjs`: passed.
- `pnpm build`: passed.
- `pnpm -r typecheck`: passed.
- Full CI passed on commit `26d20fc19d846d46d7656a9f0160db5e81479dbc`:
55 successful checks, one neutral review check, and three skipped
checks. Nothing is pending or failing. This includes all CI test shards,
Runner verification, build, typecheck, security checks, and the canary
dry run.
- The duplicate local `pnpm test:run` completed its first five chat
integration shards without failures. I stopped the remaining local run
after the complete CI suite passed. The full local suite did not finish;
the complete-suite result is from CI.
- Greptile and the Paperclip code review each gave commit
`26d20fc19d846d46d7656a9f0160db5e81479dbc` a 5/5 score with no findings.
There are no unresolved review threads.
- The initial focused test could not collect tests until the fresh
worktree built its plugin SDK dependencies. `pnpm --filter
@paperclipai/plugin-sdk ensure-build-deps` resolved that setup
requirement.

## Risks

Low risk. Blank references and zero schema offsets now select normal
search. Nonblank references still require an exact authorized tool name.
Discovery and calls keep their existing gateway checks. The change needs
no database migration.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, tool use, and code
execution. The host does not expose the exact backend model ID or
context window size in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 12:07:28 -06:00
DottaandPaperclip 60a99307b3 fix(server): settle cleanup after reusable sandbox destruction (#18633) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A web-server replacement must preserve work and let completed
cleanup become idle.
> - Several task turns can use one remote sandbox through recorded lease
handoffs.
> - The last owner can destroy that sandbox while an earlier owner still
waits for its own termination receipt.
> - A completed agent-file cleanup can also retain a retry deadline from
a competing callback.
> - This pull request resolves confirmed destruction through the
existing handoff contract and settles obsolete cleanup retries.
> - The benefit is correct cleanup recovery without deleting history or
weakening sleep checks.

## Linked Issues or Issue Description

Refs: #18554. Related work: #14695 handles warm agent-file ownership;
#13291 handles interrupted conversation continuation. This PR fixes
cleanup recovery after confirmed final destruction. It does not change
provider process reuse or start a new run.

**What happened?**

After ten qualified staging web-server replacements, a bounded
database-idle check failed. Completed native runs still owned pending
agent-file cleanup. Normal workspace archive confirmed final sandbox
destruction. An earlier lease in the same recorded handoff chain
remained unresolved. Two completed cleanup receipts also kept stale
retry deadlines.

**Expected behavior**

Resolve an earlier owner's cleanup only when the complete recorded
handoff chain leads to an exact provider-confirmed destruction receipt.
Clear a retry when its cleanup is already complete. Keep real or
uncertain work pending.

**Steps to reproduce**

1. Complete a remote run, reuse its sandbox for a second run, then
archive the completed workspace.
2. Recover agent-file collection for the first run after the final lease
confirms destruction.
3. Separately, reconstruct a completed cleanup receipt that still has a
retry deadline.
4. Observe that both records remain unfinished without this fix.

**Paperclip version or commit**

Observed on `1a6017d6460267ad89d7e1784e2b521c28d8e9ac`. The changed
source was unchanged in master
`97c4dc969e13364464765ffbde74955005bf44d3`, which is this branch's base.

**Deployment mode**

A managed ephemeral web server with PostgreSQL, object storage and a
remote sandbox. Regression tests use local PostgreSQL and no live
provider or model calls.

## What Changed

- Resolve remote termination at read time through the existing strict
reusable-lease handoff validator. Require one final owner and a valid
destruction receipt. Preserve historical lease rows.
- Use the same proof in agent-directory cleanup and remote-stop
inspection. Keep stopped, ambiguous, broken and cross-company chains
blocked.
- Reconcile completed cleanup receipts with stale retry deadlines
through the existing deadline worker. Compare the receipt before
clearing a deadline.
- Add recovery, refusal, restart quietness and concurrent-obligation
regressions.
- Document the cleanup contract and preserve deployment evidence, failed
checks and remaining acceptance work in the existing plan.

## Verification

- Both owning service suites passed 169 tests before the final receipt
comparison was added.
- The seven-case selection passed again after the release-race review
fix: `pnpm exec vitest run
server/src/__tests__/agent-directory-working-copies.test.ts -t 'stale
retry|destroyed successor chain|stale-deadline settlement'`.
- Repository-wide `pnpm -r typecheck` and `pnpm build` passed on final
review fix `7935d99bed`. The full `pnpm test:run` is still running.
Latest-head CI has 55 successful checks, three skips, and one neutral
review result. Greptile is 5/5. The Paperclip reviewer’s race finding is
fixed; its optional query-deduplication suggestion is documented and the
fresh lock-boundary checks are retained.
- The reviewer identified an early return after a lost receipt
comparison. The strengthened race regression failed before the fix and
passed afterward: release now acts on the returned authoritative row,
completes the new cleanup obligation, and preserves canonical files.
- The first focused attempt failed the new stale-deadline test because
the public release guard skipped settlement. That guard was fixed. The
original failure is retained. A sandboxed typecheck could not fetch pnpm
signature metadata; the normal network-enabled command passed without
changing verification policy.
- Prior deployed evidence: ten Railway replacements, 12,521 reads, 1,566
verified saves, 34 matching file hashes, no HTTP failures, actual
browser reconnects of 1.16–1.29 seconds, and one remote execution
retained across replacement. Two harness false alarms were separately
adjudicated without redeploying those steps.
- That deployment evidence predates this PR. Current-image natural idle,
automatic sleep/wake browser return and the isolated production canary
remain unverified. Production has not changed.

## Risks

- Cleanup authority must not cross a company or allocation boundary. The
existing validator checks the entire recorded chain; stopped successors
and unexpected owners remain unresolved.
- Confirmed destruction permits no-save cleanup. It does not claim that
uncollected agent files were saved.
- Read-time resolution adds a scoped lease-history query only for
expired reusable leases without direct termination proof. No new
recurring worker or polling cadence is added.
- The deadline update must not erase new work. It requires the same
cleanup receipt and preserves concurrent changes.
- No schema or public API changes are included.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
execution and test tools. The precise backend revision and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 11:53:44 -06:00
DottaandPaperclip 0710c56842 fix: retry saved chat input after a successful run (#18621) 
## Thinking Path

> - Paperclip manages AI agents and the work they receive.
> - Connected conversations save incoming requests before they can run.
> - A request can remain queued when the previous turn finishes
successfully.
> - Task Retry sent that case through the failed-run retry guard, which
rejected the successful run.
> - This change resumes the saved request through its existing admission
owner.
> - The operator can continue saved work while preserving the prior
result and current permissions.

## Linked Issues or Issue Description

**What happened?**

Task Retry returned a conflict for a restricted connected conversation
when its latest run succeeded and newer input remained saved. A GitHub
review can reach this state when a new PR head arrives during the
previous review. The prior review cannot publish against the new head,
but its completed turn is not a failed run that can be replayed.

**Expected behavior**

Retry should resume the saved request with its original source identity
and current authorization. It must preserve the previous successful
result and must not replay that turn.

**Steps to reproduce**

1. Start a review through a restricted GitHub endpoint.
2. Send a PR synchronize event while that review runs and retain its
deferred input receipt.
3. Let the original run finish successfully with the task still open.
4. Click Retry. Before this change, the failed-run guard rejects the
request. With this change, the saved input creates one successor through
normal admission.

Related work: #17725, #16965, #15033.

## What Changed

- Select the newest saved input by its canonical chat action and wake
receipt.
- Store its receipt ID in the task retry intent and resume it through
the existing chat owner.
- Preserve caller permissions, source authorization, task state, agent
state, cleanup waits, and duplicate-click handling.
- Exclude saved-input intents from generic board-message replay. Keep
board messages in separate queued receipts and allow independently saved
input after successful board turns.
- Add signed webhook, HTTP endpoint, accepted-result preservation, and
refusal tests. Update execution semantics and capability maps. Stabilize
the existing Teams lifecycle test with explicit database result
ordering.

## Verification

- Current-head [CI
passes](https://github.com/paperclipai/paperclip/actions/runs/41482727909):
54 successful checks, one neutral review, three expected skips, and a
successful Snyk status. Greptile gives commit `53f02e9105` 5/5 with no
unresolved review threads.
- Reproduced the original HTTP 409 before the fix. Reproduced both
review findings with three failing cases before fixing them.
- Passed 25 focused and related GitHub retry cases, 52 ordinary queue
tests, and three native startup cases against disposable PostgreSQL.
Earlier related verification also passed 83 GitHub retry cases and 109
recovery/receipt cases.
- Passed `pnpm -r typecheck`, `pnpm build`, `node
scripts/check-maps.mjs`, and the module boundary check. The final server
typecheck passed after the review fixes.
- The full local `pnpm test:run` attempt stopped on an existing Slack
queue deadline assertion with a 1 ms database/host clock difference.
That case passed in isolation. A second attempt was stopped when review
changes superseded its source. Full local completion is not claimed; all
chat, server, browser, build, and runner CI jobs passed on the final
commit.
- The runner CI job needed one rerun after a two-second startup timing
assertion failed. That case and its two related cases passed locally;
the rerun passed without code changes.
- Live GitHub review publication with this patch is not verified. The
signed provider fixture uses mocked GitHub responses. This branch is not
deployed.

## Risks

- This changes how Retry dispatches deferred external input after a
successful run. It relies on the existing inbound owner to recheck
source authority before execution.
- Tests cover paused connections and agents, changed source bindings,
completed tasks, pending cleanup, duplicate clicks, and preservation of
the accepted result.
- No schema migration or public API shape change is required.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code execution, and browser
tools. The exact deployed model ID and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 11:46:46 -06:00
DottaandPaperclip d26628e35a test(server): stop voice report waits from crossing refresh deadlines (#18628) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Voice calls retain reports that background work refreshes for a
finite period.
> - Tests must check backlog processing without starting later
refreshes.
> - The backlog test advances fake time while real database work is
still running.
> - On a slow runner, this starts refreshes and breaks the expected
request count.
> - This pull request advances time only to scheduled passes before the
first refresh.
> - The test can check each report once and still prove that the worker
becomes quiet.

## Linked Issues or Issue Description

Refs #18607. This is a separate test failure from that PR's task-list
avatar changes.
The test was added in #17586 and is unchanged by #18607.

**What happened?**

[Server shard
5/12](https://github.com/paperclipai/paperclip/actions/runs/41472540522/job/125369899753)
failed in `voice-call-reports.test.ts`, in "bounds report reconstruction
and resets a partial scan when the same owner restarts".
The test expected zero active work but found two active holds. Its logs
contain 170 requests for 105 sessions, including 65 second attempts.
The final `ci / verify` failure follows from this failed shard.

**Expected behavior**

Process all 105 reports once. Wait for each pass to settle before
advancing to another scheduled pass. Keep the clock before the first
60-second refresh deadline.

**Steps to reproduce**

1. Run the named test against a disposable PostgreSQL database.
2. Temporarily delay the first provider response by 3,250 ms with
`node:timers/promises`.
3. Use the same 15-second wait budget for the old and new versions.
4. The old polling makes 315 requests across three attempts per session
and fails. The new polling makes 105 requests, one per session, and
passes.

**Paperclip version or commit**

Observed at PR head `7f1d137a53489e2a45ba05e506b5ba17b51d5882`.

**Deployment mode**

GitHub Actions. Local verification uses disposable PostgreSQL 17.

## What Changed

- Keep fake time fixed while a report pass waits on database work.
- Advance to the coordinator's next scheduled pass only before the first
report refresh.
- Disable implicit fake-time advancement in the assertion polling.
- Allow 15 seconds for the 105-report wait and 30 seconds for the full
fixture test.
- Retain all assertions for bounded reconstruction, restart, exact
session IDs, one attempt per report, and no later database or provider
activity.

## Verification

- Controlled comparison with the same injected delay and 15-second wait:
old polling fails with 315 requests; fixed polling passes with 105
requests. The temporary delay is removed from this change.
- All 52 tests pass across `voice-call-reports.test.ts`,
`scoped-deadline-work.test.ts`, and `delivery-work-coordinator.test.ts`.
- `git diff --check` passes.
- The initial embedded PostgreSQL run could not start because its macOS
package lacks a `libzstd.1.dylib` alias. The passing tests use a
separate disposable PostgreSQL 17 cluster.
- The full 15-test voice-report suite passes again after rebasing onto
current master.
- `pnpm -r typecheck` passes.
- `pnpm test:run` was started, then stopped during chat-suite startup
after no tests completed in that invocation. It has no complete result.
Its cause is not established.
- `pnpm build` passes.
- Greptile gives 5/5 with no inline findings on
`caa6db568deb9df98437d1a532cc345e9f5b16dc`. Its check completed
successfully for that exact commit.
- The first CI attempt passed all 12 server shards, including the
voice-report regression. Three other jobs were interrupted: Runner
Vitest 2/2 received a runner shutdown signal; Typecheck + Release
Registry and E2E shard 5/8 have GitHub annotations reporting lost runner
communication. These are incomplete checks, not passing retries. [First
attempt](https://github.com/paperclipai/paperclip/actions/runs/41488319181/attempts/1).
- The single retry passed for all three interrupted jobs. The final `ci
/ e2e` and `ci / verify` gates passed. [Successful second
attempt](https://github.com/paperclipai/paperclip/actions/runs/41488319181/attempts/2).
- Final commit `caa6db568deb9df98437d1a532cc345e9f5b16dc`: all 56
reported checks are terminal and clean. The two optional Storybook
checks are skipped. GitHub reports the PR as clean and mergeable.

## Risks

- Low risk. The change affects one integration test. Production code and
report refresh behavior are unchanged.
- The test still has a finite wall-clock timeout. A database that cannot
process the fixture within that budget will fail.

## Model Used

OpenAI GPT-6 through Codex, with repository tools and code execution.
The runtime does not expose a more specific model ID or context-window
size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 11:39:37 -06:00
DottaandPaperclip 40b8075916 feat(ui): show scheduled task waits as Monitoring (#18624) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task indicators tell an operator whether work is active, waiting, or
complete.
> - An agent can yield until a saved monitor checks CI or another
external result.
> - The task currently keeps the In Progress icon between those runs.
> - Changing the stored status would affect monitor admission and
recovery.
> - This pull request adds a read-only Monitoring display state with a
blue clock.
> - Operators can see and filter scheduled waits while execution keeps
its existing rules.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Task status indicators, task collection filters, board columns, and
search facets.

**Current behavior**

A task that waits for a scheduled monitor shows the same In Progress
icon as active work.

**Proposed behavior**

Show Monitoring with a static blue clock when an agent-owned task has a
saved ordinary monitor and no active run, execution lock, review
participant, or active recovery action. Return to the normal task
indicator when the check starts. Preserve review, blocked, terminal, and
quota-recovery states.

**Reason and benefit**

Operators can distinguish scheduled waits from current agent work
without reading the task history.

**Breaking changes**

None to stored task statuses or execution rules. Task reads gain
optional displayStatus. Collections and search gain a separate read
filter. Existing status filters keep their lifecycle meaning.

Related historical work: #4396 covered a monitoring label exemption.
This change uses the saved monitor and a read projection.

## What Changed

- Add one SQL projection for task detail, compact lists, summaries,
relations, search results, and display-status counts.
- Show the blue clock in shared task icons and chips, task detail and
breadcrumbs, properties, references, list rows, recent navigation, and
agent task cards.
- Add Monitoring to task filtering, grouping, sorting, board columns,
and search facets. Prevent a drag from creating this derived state.
- Keep stored lifecycle statuses, monitor scheduling, checkout,
recovery, permissions, and agent status filters unchanged.
- Add database and UI regression coverage, full-page Storybook examples,
and product and access-map documentation.

## Verification

- Passed `pnpm -r typecheck`.
- Passed `pnpm build`.
- Passed `pnpm check:token-gates` and the map/catalog structural checks.
- Ran the new database suite on a disposable PostgreSQL 17 database. It
covers read surfaces, filter pagination/counts, queued/running checks,
rescheduling, completion, retained monitors, review, and recovery holds.
- Ran existing search, list, board, sidebar, task status, and filter
tests. Corrected board queries use displayStatus and eight columns.
- The full local `pnpm test:run` stopped in a chat integration shard: 12
failures (four timeouts and eight FIFO publication assertions). That run
spanned a rebase. A focused rerun on the current source passed all 21
selected cases without assertion or timeout changes. The original cause
is not established; this does not claim the full local suite passed.
Final-head required CI passed (ci / verify and ci / e2e, run
41488267209).
- Passed 82 tests for task API summaries, recovery precedence, search,
and optimistic updates; 54 tests for status cards, database reads, and
board moves; and 27 tests for blocker relations and recent-task cache
updates. One intermediate database setup hit its 20-second hook timeout;
the serial rerun passed.
- Greptile is 5/5 on d003ea672d. All
review threads are resolved. The separate optional Paperclip Review
ended early and returned neutral; GitHub rejected a direct rerun request
with HTTP 404. It is not counted as a completed review.
- Review fixes cover escalated recovery, is:open, optimistic status
changes, blocker-summary projection, and recent-task dedupe.
- Reviewed 15 screenshots from production pages and components with
isolated Storybook API fixtures: task detail/properties/breadcrumb,
list, groups, status filter, board, search/results/filter counts,
reference pills, sibling navigation, agent cards, picker, dashboard,
inbox, profile, watched status-card tasks, light theme, and mobile. This
does not claim deployed staging verification.

## Risks

- Read projections must stay consistent across full, compact, summary,
and search responses. Tests cover those paths.
- Monitoring is not a writable lifecycle status. The board does not
accept drops into its Monitoring lane or onto a monitored card. This
prevents a drag from implying a scheduled check. Move work into In
Progress through that lane instead.
- The read projection adds run/recovery existence checks. Existing
heartbeat_runs_company_issue_created_idx and
issue_recovery_actions_company_source_status_idx indexes cover the
company/task lookup. It adds no worker, timer, recurring query,
migration, or provider call. No production load benchmark is claimed.
- Older clients can ignore displayStatus. Existing lifecycle status
filters keep their behavior.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository tools, code execution,
and browser tools. The exact backend model identifier and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 11:35:30 -06:00
DottaandPaperclip b9e94b85a6 feat(observability): trace Daytona lease acquisition and resume (#18626) 
## Thinking Path

> - Paperclip manages work performed by AI agents.
> - The server prepares a remote execution environment before the agent
can respond.
> - OpenTelemetry traces show the time spent in each preparation phase.
> - A retained Daytona sandbox can take minutes to resume from an
archive.
> - The provider worker previously exposed no child timings for this
operation.
> - This pull request adds bounded lease phase spans under the existing
worker trace.
> - Operators can separate provider startup from lookup, teardown waits,
and workspace verification.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Daytona environment acquisition and resume tracing. A long worker
invocation currently hides the operation that consumed its time. Related
tracing foundation: #10999. No duplicate open PR was found.

**Subsystem affected**

The Daytona sandbox provider and the server's existing plugin span
validation boundary.

**Proposed behavior**

Show lookup, start or recovery, teardown waits, workspace verification,
and fresh acquisition setup as timed child spans. Group verification
steps beneath one parent. Record the bounded provider state before
start, including archived.

**Reason and benefit**

An operator can identify which step delayed the first agent response.
SDK startup includes its internal readiness wait; this change does not
claim to measure provider queue time separately.

**Breaking changes**

The opt-in trace schema gains fixed span names and one closed state
attribute. Execution, authorization, resource ownership, deadlines, and
retry policies keep their existing contracts.

## What Changed

- Add lease phase spans through the existing invocation tracer.
- Add the fixed span names and state values to host validation. Define
the state key in the shared startup attribute contract.
- Preserve the exact work result or error if tracing fails. Never repeat
work through the tracing wrapper.
- Test worker receipts for archived and warm resume, recovery,
concurrency, failure, disabled tracing, and cleanup.
- Document the waterfall and its attribution limits.

## Verification

- Daytona plugin suite: 525 passed, 7 skipped. The expanded tracing
suite separately passes all 12 cases.
- Host span validation and instrumentation tests: 46 passed, 3 skipped.
- Standalone Daytona TypeScript check passes.
- Module boundary and whitespace checks pass.
- Full recursive typecheck and full build pass.
- The full local Vitest attempt at
b007a8e64f stopped in chat shard 1: 337
passed, 2 failed, 1478 skipped. One retry-notice case exceeded 15
seconds. The next case then reported that fake timers were no longer
installed. A focused rerun of all four retry-notice cases passed. This
is not a full local suite pass.
- Final head 8e1f81523b passes CI and
Greptile 5/5, with no unresolved review threads. The CI chat shards pass
despite the preserved local timeout attempt.
- The expanded lease tracing suite passes 15 tests; host/shared tracing
suites pass 66 tests. Disabling the wait flag makes the new teardown
test fail, and restoring it passes. Standalone plugin typecheck passes.
- The tests drive the real plugin worker RPC and invocation tracer with
a controlled provider fixture. They validate receipts and host parent
IDs. They do not establish deployment or Honeycomb ingestion. Inspect a
new deployed run to verify the exported waterfall.

## Risks

- A new provider phase must be added to the closed host allowlist or it
appears as other.
- SDK-internal start and readiness work remains one measured operation.
- Older workers without withSpan run normally without these new detail
spans.
- Receipt budgets and clock validation can omit detail from very large
or invalid receipts.

## Model Used

OpenAI Codex, GPT-6, with repository tools and code execution. The exact
runtime model ID and context window were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 11:32:03 -06:00
DottaandPaperclip 3f7811eef0 fix(runner): keep requests responsive during retired backup cleanup (#18619) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote native runs keep durable harness backups for continuation and
recovery.
> - Publishing a new backup retires earlier backup directories in the
server process.
> - The synchronous permission walk and recursive removal block
unrelated requests.
> - This pull request makes that cleanup asynchronous and keeps
publication waiting for completion.
> - The benefit is continued request progress while large retired
backups are removed.

## Linked Issues or Issue Description

**What happened?**

Publishing a backup with many small files blocks the Node event loop
during directory permission repair and removal. A local comparison with
20,000 retired files reproduced 1.37–1.95 second event-loop stalls. This
is a pause candidate. A short hosted profile has not established it as
the dominant source of sustained CPU use.

**Expected behavior**

Unrelated requests can progress during cleanup. Publication still waits
for complete removal, preserves rollback on a failed lease stamp, and
leaves live files and symlink targets unchanged.

**Steps to reproduce**

Run the benchmark in `doc/performance/native-backup-publication.md`. It
compiles the actual publication service from a pinned baseline and the
working tree, prepares immutable retired backups, and sends foreground
HTTP probes from a separate process.

**Paperclip version or commit**

Measured baseline: `774ddc72713d6aa7cb516691c71f1fc23bcfa5ff`. The
measured treatment source is committed in this branch. Current head:
`0a69cf2d43c2f9fcf032d96c1dceb56163bf21ba`. The runtime patch is
unchanged from the measured source; later commits only strengthen the
test and benchmark evidence paths.

**Deployment mode**

Local compiled service on macOS arm64 with Node v24.22.2. No hosted
rollout has occurred.

Related work: #18380 changed retention scheduling. #13676 included the
existing immutable-backup cleanup behavior. Neither change makes
publication cleanup asynchronous. The GitHub duplicate search also found
#11929, which concerns database dumps rather than native harness
backups. No matching open event-loop cleanup issue was found.

## What Changed

- Use asynchronous filesystem operations for retired-backup cleanup.
Visit directories serially and skip unnecessary metadata calls for file
entries.
- Await cleanup before publication proceeds, during rollback, after
successful rotation, and in executor finalization.
- Exercise immutable current and previous backups in the publication
test. Preserve the existing symlink-isolation and rollback tests. Add a
real separate-process HTTP probe that must complete while publication is
still active.
- Add a repeatable service benchmark and document its fixture, metrics,
and acceptance limits.

## Verification

Draft status: focused correctness tests, the local comparison, and
repository typecheck pass. The complete local test gate is incomplete.
The build passed. Complete CI, hosted acceptance, and final-head review
remain pending.

- Final focused Vitest run: 2 files, 4 tests passed in 649 ms with the
normal server configuration. The regression warms a separate Node
client, waits for its HTTP request during the stamp callback, queues the
response for the next event-loop turn, and then allows final
retired-tree removal to start. The async treatment serves that response
before publication completes. The original synchronous cleanup fails the
same final regression in a separate source fixture. That expected
control failure is retained.
- Syntax and `git diff --check` passed.
- `pnpm -r typecheck` passed on head `332c57b88c` in 152.18 seconds,
including the native Runner build. Validation uses Node v24.22.2 and the
repository-pinned pnpm v9.15.4 with isolated dependencies.
- `pnpm test:run` was intentionally stopped after 1,390.24 seconds (exit
143). All six chat shards passed, covering all 1,817 parameterized cases
across their exact partitions. The general server phase was advancing
but incomplete. The repository timing manifest estimates another roughly
two hours for the serial remainder, so validation moved to final-head
parallel CI. This is not a full local test pass. The command began on
`332c57b88c`; later changes only refine the regression and benchmark,
and the final regression has its own focused pass.
- Benchmark failure/metadata checks pass in a tiny isolated Git fixture:
an injected worker exit 17 is reported in 0.44 seconds without hanging,
and the result includes a staged source change when the unstaged diff is
empty.
- `pnpm build` passed in 41.80 seconds on `9f7e4e669b`. The final commit
only releases a pending test probe on failure; runtime and build sources
are unchanged. An injected client exit verifies that final test failure
path exits promptly in 0.97 seconds instead of hanging.
- Three alternating baseline/treatment pairs used fresh fixtures with
100 directories × 100 files in each of two retired backups. Each
publication removed 20,000 files. A separate process sent HTTP probes at
fixed 5 ms intervals, including overdue arrivals.
- Every run verified the replacement manifest, complete cleanup, and
unchanged symlink-target contents and mode. All 2,395 HTTP requests
completed without errors. The largest measured generator delay was 8.8
ms.

| Metric | Baseline median (range), ms | Treatment median (range), ms |
| --- | ---: | ---: |
| Foreground request p95 | 1,357.7 (1,287.4–2,014.0) | 1.8 (1.7–25.9) |
| Maximum event-loop delay | 1,439.7 (1,371.5–1,954.5) | 19.9
(17.4–40.7) |
| Publication completion | 1,429.1 (1,360.2–1,952.9) | 1,439.5
(897.9–1,649.6) |
| Process CPU during publication | 1,194.1 (1,041.8–1,225.5) | 2,423.1
(1,995.7–2,514.7) |

The treatment served 180–329 requests while publication was active. The
baseline served zero during that interval. Process CPU includes
foreground work that the baseline cannot serve until cleanup completes.
The result supports responsiveness, with a CPU tradeoff. It does not
establish lower total CPU or a hosted latency gain.

Reproduce with Node v24.22.2:

```sh
pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-harness-backup-cleanup.test.ts src/services/native-runtime/native-harness-backup-publication.test.ts
node server/scripts/benchmark-native-backup-publication.mjs --baseline-ref 774ddc7271 --output /tmp/native-backup-publication.json
```

The benchmark records the baseline SHA, treatment HEAD and source diff.
Raw local evidence remains outside the repository. A first small smoke
run hit a sandbox restriction on loopback listeners; the authorized
retry resolved it. The smoke also exposed load-generator interval drift.
Fixed-clock arrivals replaced that generator before the reported
measurements.

Pending gates: complete current-head CI, final-head Greptile review, and
the real run/checkpoint/recovery journey. The full local test run
remains explicitly incomplete. The four concrete review findings were
fixed. The prior head received 5/5; final-head review remains pending.

Local evidence remains outside Git:
`/private/tmp/backup-publication-measurement-20261102.json`,
`/private/tmp/native-backup-focused-final.log`,
`/private/tmp/native-backup-typecheck.log`,
`/private/tmp/native-backup-test.log`,
`/private/tmp/native-backup-test-interruption.json`,
`/private/tmp/native-backup-build.log`, and
`/private/tmp/native-backup-harness-check.json`. The harness-check
receipt points to its isolated metadata and worker-failure logs. The
final client-failure and baseline-control logs are
`/private/tmp/backup-benchmark-check-gprcfiiz/failed-client-cleanup.log`
and
`/private/tmp/backup-benchmark-check-gprcfiiz/final-probe-baseline-control.log`.
These local paths are retained for the author; the commands above let
reviewers reproduce the synthetic proof.

## Risks

- Asynchronous removal adds filesystem-pool work and used more process
CPU in the fixture. Check resource pressure as well as responsiveness
before rollout.
- Filesystem operations can now interleave with unrelated work. The
existing session owner still controls publication. Cleanup remains
awaited; no deletion is detached and no scheduler is added.
- Errors must still fail publication or restore the prior backup as
before. Corruption checks and destructive lease authorization are
unchanged.
- This synthetic macOS fixture does not represent every Linux volume or
backup shape. It does not qualify a deployed fix.
- Rollout acceptance should confirm the serving SHA, checkpoint
completion, continuation/recovery, error rate, and loop delay under
matched backup load. Revert if cleanup or recovery fails.

## Model Used

OpenAI Codex, GPT-6 family, with tool use and code execution. The exact
served model ID and context-window size were not exposed to this agent.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 17:00:42 +00:00
AlexFrontegg 8424a51018 fix(redaction): scan long letter/digit runs in linear time (#16999) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip removes secrets from run logs, tool results and
diagnostics with `redactSensitiveText` before it stores or shows them.
> - The JSON field-name patterns in that function start with an
unanchored `[A-Za-z0-9_-]*`. Each start position inside a long run of
letters and digits scans to the end of the run, so the cost grows with
the square of the run length.
> - Base64 of text is one long run of letters and digits. A 40 KB base64
blob of source code in a shell command takes about 5 s to redact, and a
large Gmail RAW tool result does not finish.
> - The redaction runs on the main event loop, so one such chunk blocks
every request and every run on the instance.
> - This pull request makes these scans start only at the start of a
run. The scans become linear, and the redacted output does not change.
> - The benefit is that large base64 payloads no longer freeze the
server, and every secret that was redacted before is still redacted.

## Linked Issues or Issue Description

Fixes #15393. The tool content guard runs `redactSensitiveText` on tool
results. A Gmail RAW body is base64url of a MIME message, which is one
long letter/digit run. `validateToolContent` on a Gmail-shaped result
with a 68 KB `raw` field takes 59 s on master and 19 ms with this
change; a 5.4 MB `raw` field takes 0.65 s.

Thanks @ts-beat for the clear report in #15393.

Refs #12309 (false positives in the same patterns; this pull request
does not change which text matches).

I searched issues and pull requests for "redaction",
"redactSensitiveText", "quadratic", "backtracking", "hangs the server"
and "100% CPU". #15393 reports the hang. No open pull request fixes it.

**Subsystem affected:** secret redaction (`server/src/redaction.ts`,
`packages/adapter-utils/src/command-redaction.ts`), used by run logs,
tool content guards and run diagnostics.

**Problem:** `compactRunLogChunk` redacts each run log chunk before it
truncates the chunk. A shell command such as `printf '%s' '<base64>' |
base64 -d > file.ts` with base64 of source text makes one long
letter/digit run. Base64 of ASCII text contains `+` or `/` only where
every third byte is `>`, `?`, `~` or DEL, so random base64 does not show
the problem. One `redactSensitiveText` call on such a tool-call line
takes (local measurement):

| Blob | Before | After |
| --- | --- | --- |
| 4 KB | 66 ms | 1.1 ms |
| 8 KB | 272 ms | 0.5 ms |
| 16 KB | 1.1 s | 1.0 ms |
| 40 KB | 5.4 s | 2.5 ms |
| 1 MB | not measured | 36 ms |

The `\b`-anchored command patterns (`KEY=value`, JWT, `Authorization`)
and the CLI option pattern have the same quadratic cost on
dash-separated runs, for example base64url data.

**Proposed solution:** start the unanchored field-name scans only where
no name character precedes them (`(?<![A-Za-z0-9_-])`). A prefixed name
such as `myApiKey`, `x_auth_token` or `GITHUB_TOKEN` still matches,
because the scan from the start of the run absorbs the prefix.

**Alternatives considered:**

- Truncate or cap the input before redaction. Rejected: a cut can split
a secret and leave part of it unredacted, and the scan stays quadratic
below the cap.
- Run the scan in a worker thread with a deadline, as #15393 suggests.
That limits the damage, but the CPU cost stays, and a timeout must then
drop or block the content. It is still useful as an additional defense.

## What Changed

- `server/src/redaction.ts`: `JSON_SECRET_FIELD_TEXT_RE` and
`ESCAPED_JSON_SECRET_FIELD_TEXT_RE` start a field name only where no
name character precedes it. The lookbehind comes after the optional
opening quote, so quoted keys behave as before. The `Authorization` word
scan uses the same start instead of `\b`.
- `packages/adapter-utils/src/command-redaction.ts`: the `KEY=value`
pattern uses the same start. The JWT pattern starts only at the first
word character of a run. A CLI option name can no longer span `--`, so
each `--` starts its own short scan.
- Redaction still runs before truncation. `compactRunLogChunk` does not
change.

Why the output does not change: when an old pattern could match from the
middle of a run, it could also match from the start of that run, because
the `[A-Za-z0-9_-]*` prefix absorbs the extra characters. The engine
tries the start of the run first. So the old patterns never matched from
the middle of a run, and the new start condition only skips positions
that cannot produce a match. The old JWT pattern also always matched
from the first word character of a run. A CLI option name always ends at
the end of its run, so the match from the last `--` in the run ends at
the same place.

## Screenshots

No UI change.

## Verification

- New tests:
- prefixed secret field names in plain, single-quoted and escaped JSON
are still redacted, also when the name is glued to the end of a long
run;
- a 48 KB base64-of-source tool call through `compactRunLogChunk` stays
intact, the secret in the same command is redacted, and the call
finishes within 1 s;
- `redactSensitiveText` and `redactCommandText` handle 48-120 KB
letter/digit, dash-separated, `--`-separated and base64url runs within 1
s, and redact the secret that follows each run;
- `x--token`, `--x--token`, `a-b--api-key=`, `prefix-MY_API_KEY=` and
`--<jwt>` are still redacted, and `my-token is here` is not changed.
- Without the change, the timing tests fail: 9.3 s, 9.8 s, 32 s and 60 s
against the 1 s budget. The correctness tests pass with and without the
change.
- Mutation check: I reverted each of the six pattern changes one at a
time. Each revert makes at least one test fail.
- Differential fuzzing: 1.1 million random inputs built from secret
names, quotes, escaped quotes, dashes, dots, colons and JWT parts gave
the same output from the old and new `redactSensitiveText`,
`redactCommandText` and `redactDiagnosticText`.
- `cd server && npx vitest run <files>`: all 33 server test files that
use the redaction helpers, with embedded Postgres, 1730 passed.
- `npx vitest run --project @paperclipai/adapter-utils
packages/adapter-utils/src/command-redaction.test.ts`: 25 passed.
- `npx vitest run --project @paperclipai/adapter-utils`, complete suite:
80 of 82 files passed in the parallel run. The 2 other files
(`github-launcher-environment`, `sandbox-managed-runtime`) had 5 s
timeouts in git-based tests and passed on rerun.

## Risks

- Lookbehind assertions need ES2018. `redactCommandText` also runs in
the UI bundle. Node and current browsers support lookbehind, and the
server (`server/src/log-redaction.ts`) and the UI
(`ui/src/components/task-chat/task-chat-attachments.ts`) already use it.
- A change to a redaction pattern could leak a secret. The reasoning
above, the unchanged existing tests and the differential fuzzing show
the same output as before.
- Not changed here, found while checking: `redactDiagnosticText`
backtracks exponentially on an escaped JSON secret value made of many
escaped backslashes with no closing quote, and `redactSensitiveText`
redacts a value only up to its first quote of either kind
(`{"apiKey":"abc\"tail"}` keeps `tail"`). Both behave the same before
and after this pull request.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with
tool use (shell, code search, and test runs) and extended thinking. A
human directed the design choices.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
 2026-11-02 10:49:08 -06:00
DottaandPaperclip 0748b90062 fix(server): stop goal-less native runs before board task closure (#18616) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A task owns its active run and its tool access.
> - A board user can close a task while that run is still working.
> - The task update route stopped native runs only when they had an
active session goal.
> - Providers without goals kept running after the route cleared task
ownership.
> - This pull request stops those native runs before a board closure
releases ownership.
> - The benefit is an orderly stop instead of tool denials and failed
finalization.

## Linked Issues or Issue Description

**What happened?**

A board user marked a task Done while its native OpenCode run was
active. The route cleared its execution lock without stopping the
provider. Later tool calls failed with
`paperclip_runner_tool_binding_not_authorized`. The final wake fence
then failed with `native_current_wake_comments_binding_changed`, leaving
workspace finalization in retry. Cancelled also released ownership
before its later Stop call.

**Expected behavior**

A board closure must confirm native Stop before it releases task
ownership. A failed Stop must leave the task open. The provider does not
need to support session goals.

**Steps to reproduce**

1. Start a native task with a provider that has no active session goal.
2. Mark the task Done while its run is active.
3. Observe that the old route updates the task without cancelling the
run.

## What Changed

- Reuse the existing audited cancellation path before a board user marks
an active native task Done or Cancelled.
- Keep the existing 409 response when Stop is unconfirmed.
- Record the board user in the Stop metadata. The existing
acknowledged-Stop policy then leaves queued input and recovery quiet
until a later explicit wake.
- Inspect additional runs only when a task has an execution owner; tasks
without a run lock keep their existing completion path.
- Keep native semantic completion, legacy completion, and goal control
behavior.
- Add route regressions for stop ordering, failed Stop, and legacy
completion.
- Update the execution contract and task maps.

## Verification

- Red before the fix: three regressions failed. Done had no Stop;
Cancelled stopped after the update; an unconfirmed Stop allowed Done.
- A second red proved that a Stop receipt without the route's operator
metadata was not recognized by the real recovery policy. Both cases pass
after the metadata repair.
- Final local result on macOS arm64 / Node.js 26.4.0: **348 tests pass
across eight suites**. These cover the route, native self-completion
authority, cancellation, acknowledged Stop, recovery queues, dependency
wakeups, and completion telemetry.
- CI found seven sibling route failures from an unnecessary active-run
lookup on tasks without execution ownership. Those cases failed locally
before the correction and pass afterward.
- Earlier concurrent local runs hit a module-load hook timeout and one
watchdog socket hang-up. The isolated watchdog case and the complete
final eight-suite run pass with a 90-second startup budget. No assertion
was removed.
- `node scripts/check-maps.mjs` and `git diff --check`: pass.
- `pnpm build` and `pnpm -r typecheck`: pass. Server build and typecheck
also pass after the Stop metadata change; the final server build
includes the lookup correction.
- `pnpm test:run`: attempted; the unchanged chat shard selector failed
before executing its selected tests (`No test found ... in lines ...`).
This is separate from the PostgreSQL startup failure. CI supplies
full-suite verification.
- PostgreSQL integration and live-provider acceptance did not run
locally. Embedded PostgreSQL failed before app startup with `could not
create semaphores: No space left on device`. No usable isolated local
database runtime was available. CI supplies the PostgreSQL regression
environment. The full live OpenRouter workflow remains unverified.
- The first final-head CI pass hit a 15-second timeout in the unchanged
`attention-retention-work.test.ts` blocker-history fixture. Cleanup then
deadlocked and later inserts reused IDs. This service test does not call
the changed route. The original log is [retained
here](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289220).
The one allowed rerun also failed on the same commit: the `running` case
exceeded 15 seconds again ([rerun
log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125393915025)).
The rerun otherwise passed 2,346 tests. No retention source or assertion
was changed. This repeatable failure is the remaining CI gate; the PR is
**not ready to merge**. The next step is to isolate that retention
fixture/query in a usable PostgreSQL environment and repair its cause
before rerunning CI.
- Greptile reviewed final commit
`b3cb41af2d911e9ced219e846cada3ecec835dbf` at **5/5**, with no
unresolved review threads. Greptile is green. CI is blocked by the
repeated retention test timeout described above.
- CI native cancellation evidence: all 16 tests in
`server/src/__tests__/native-cancellation-request.integration.test.ts`
pass ([job
log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289222)).
- CI PostgreSQL wake-queue evidence: all 96 tests in
`server/src/modules/wake-queue/adapters/postgres.test.ts` pass ([job
log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289470)).

## Risks

Board closure now waits for native Stop even when the provider has no
goal. A failed or uncertain Stop keeps the task open. The change uses
the existing cancellation and workspace-preservation path. It adds no
retry loop, credential access, schema change, or publication authority.
Direct service updates and agent semantic completion retain their
existing paths.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, shell
execution, and browser inspection. The exact serving snapshot ID and
context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 10:41:44 -06:00
Nicky LeachandPaperclip f8a888a072 feat(runner): define bounded startup lifetime contracts (#18490) 
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Native sessions and plugin workers have separate startup lifetimes.
> - A caller timeout does not prove that a worker or an accepted host
write has stopped.
> - Shared startup code needs an exact deadline and an actual settlement
contract.
> - This PR adds those contracts and tests them before any production
caller uses them.
> - The dependent integration in #18405 will use the contracts with
verified cleanup.
> - This split lets reviewers assess the lifetime rules separately from
runtime activation.

## Linked Issues or Issue Description

Refs #18405. This PR extracts the foundation from that change. The
integration will depend on this PR.

**Problem or motivation**

A bounded startup must distinguish the caller's wait from actual worker
completion. A late worker reply or an accepted host request can outlive
the original RPC promise. A shorter timeout must not discard that
responsibility or authorize a replacement run.

**Proposed solution**

Add one Core execution context, optional Runner startup deadlines, and
worker settlement and retirement APIs. Test the contracts with real
child-process IPC. Keep production context creation, transport
projection, capability negotiation, durable cleanup, and Stop/recovery
integration in #18405.

**Alternatives considered**

One combined PR mixed the lifetime contract with its activation and
cleanup policy. Activating a shorter deadline before the complete
transport and cleanup path is unsafe. Splitting only the SDK
declarations would not provide a useful behavioral review boundary.

**Roadmap alignment**

This supports the existing plugin, sandbox execution, and bounded
recovery capabilities. It adds no new product workflow, endpoint,
migration, provider or runtime dependency.

## What Changed

- Add `NativeExecutionContext`. It retains the selected input, binds one
transport target, rejects work after cancellation or expiry, and tracks
actual worker settlement. Expiry alone does not complete pending work.
- Add worker startup groups. A caller timeout retains the group's
responsibility. Retirement revokes obsolete invocation scopes, waits for
accepted host handlers, and retires only the verified group's
bookkeeping. Each cleanup wait is bounded; an unfinished handler remains
pending for a later retry.
- Add an optional Runner startup signal and absolute deadline for
session open and retained attachment. Clean up abort listeners after
synchronous failures. Existing callers keep their default timeout path.
- Expose the remaining confirmed controller-lease budget without a new
query or renewal timer.
- Add optional SDK context and preparation-retirement fields. No
production bridge sends or advertises them in this slice.
- Keep the required lease mock update with the new `remainingMs()`
contract so this foundation typechecks independently.

Production activation files, including the native executor and
environment task bridge, match the current master base. No production
caller constructs `NativeExecutionContext` or passes the new Runner
startup option. This PR adds no durable cleanup producer and claims **no
standalone TTFT improvement**. The dependent integration must enable the
context, transport and verified cleanup together.

## Verification

Initial split verification on
`8ebc8a2f9c6c02e90f920356138b3742a4067b28`, based on
`9a3f395737e7a92faf99971459979061d423531a`:

- From `server`: `../node_modules/.bin/vitest run
src/services/native-runtime/native-execution-context.test.ts
src/services/native-runtime/native-execution-context-worker.test.ts
src/services/native-runtime/native-execution-lease.test.ts` — **17
passed**. Includes real child IPC, caller timeout with late reply,
worker death, accepted host-handler draining, obsolete cleanup
invocation rejection, unrelated work preservation, context scope,
expiry, and confirmed lease budget.
- From `packages/paperclip-runner`: `../../node_modules/.bin/vitest run
src/native-session-runtime.test.ts` — **188 passed**. Includes existing
callers without the new option and new
synchronous-failure/expired-deadline cases for fresh and retained
sessions.
- Direct TypeScript checks for `packages/plugins/sdk/tsconfig.json` and
`packages/paperclip-runner/tsconfig.json` — passed. Built only their
shared/eval TypeScript prerequisites.
- From `server`: `../node_modules/.bin/tsc --noEmit` — passed.
- Checked the partition against the reviewed combined candidate. The
twelve selected files match exactly. The thirteenth file carries only
the required test-mock update. The production executor, bridge and
target wiring remain equal to the base.
- `git diff --check`, added-line secret/private-data scan, and
added-code-comment scan — passed. No generated files or run artifacts
are committed.
- The first old-base split passed its focused tests but failed server
typechecking because the unchanged executor test mock lacked
`remainingMs()`. The original failure is retained. Moving that one-line
mock update into this foundation resolved the split dependency; the
final current-master checks above pass.
- Full local CI suites, Rust builds, deployments and provider calls were
not run, as requested. GitHub CI will provide full regression coverage.
Current-head GitHub CI, standard review and Apex are complete; exact
acceptance is recorded below.

Review follow-up `d7635f4d280dab757a3666703737bcda955c9361` fixes three
reproduced lifetime edge cases. A retiring startup group rejects new
calls and shares one retirement promise; failed bounded retirement
remains retryable. Transport identity uses a fixed field tuple rather
than object-property order. Finishing a context rejects pending lookup
waits without aborting an already-started backend or pretending the
worker completed. All three new regressions failed before the fix; the
updated foundation selection passes **20 tests**, including actual IPC
and failed-retirement retry. Server typechecking and added-line checks
pass. These replace the earlier 17-case selection, rather than adding 20
unique cases to it. The original failures and passing reruns are
retained.

Apex follow-up `cb8860bab5ab9d6357a352e11f276be0d0cdb26d` keeps removed
worker handles reachable while they own startup calls, accepted host
handlers or retirement work. Drain/retirement includes active and
retained handles; completion callbacks remove settled handles. Stop and
replacement remove only the exact handle they observed. This adds no
database/network operation, polling loop or generic shutdown wait. The
SDK, Runner, context and transport bridge are unchanged.

Seven additional IPC regressions failed on the prior manager and pass
after the fix, covering removal/replacement, Stop-all, host-handler
settlement, bounded retirement failure/retry, pruning, and replacement
ownership. The updated foundation selection passes **27 tests**, with
server typechecking passing; these supersede the earlier 20-case
selection. A separate real-PostgreSQL/child-IPC characterization uses
the integration's actual SDK host, storage registry and run-control path
with the exact corrected manager bytes. Before the fix, cleanup released
run ownership while an accepted host write was blocked, then the late
write restored obsolete metadata. After the fix, ownership and cleanup
remain pending until the write completes; cleanup then removes the stale
metadata before releasing ownership. The original failure, corrected
result and source snapshots are preserved. Final head
`cb8860bab5ab9d6357a352e11f276be0d0cdb26d` passes GitHub CI (53
successful checks, 2 expected skips, Snyk success), standard Greptile
**5/5** and fresh Apex **5/5**. All review threads are resolved and the
PR is conflict-free.

## Risks

The shared Runner helper now observes synchronous operation failures
through its asynchronous cleanup path. The affected runtime file's 188
tests pass, but full regression coverage still depends on CI.

The new lifetime APIs require honest settlement signals. A future caller
must not treat a deadline, worker crash or Stop intent as verified
retirement. Production activation remains dependent on the complete
integration and its cleanup tests in #18405. This foundation introduces
no shorter production startup budget or new cleanup policy by itself.

The context's transport projection is process-local. It must not be
persisted or used as a general authorization cache. No production caller
supplies it in this PR. There is no database migration, new dependency,
or deployment requirement for merging this foundation.

## Model Used

OpenAI Codex, based on GPT-6, assisted with repository inspection, the
split, tests and command execution. The exact serving model ID and
context-window size were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal issue
identifier
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes — this
description documents the inactive contracts; operational activation
documentation remains with #18405
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green on `cb8860bab5`
- [x] Current-head standard Greptile and Apex are 5/5 with no unresolved
findings
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 08:22:42 -08:00
DottaandPaperclip 412834a860 test: overlap long recovery and maintenance waits (#18544) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - CI verifies recovery, execution authority, and background work
before changes merge.
> - Diff selection saves work, but a selected file still runs its cases
in sequence.
> - The three longest files in a recent full master run each took over
eight minutes.
> - Many of these cases must observe real lease expiry or idle periods.
> - This pull request gives independent cases separate fixtures so those
periods can overlap.
> - The benefit is a shorter test path with the same assertions and
observation windows.

## Linked Issues or Issue Description

Refs #18464.

The latest measured successful master [Cloud readiness
run](https://github.com/paperclipai/paperclip/actions/runs/41300933157)
spent 581 seconds in issue recovery, 569 seconds in native restart
recovery, and 512 seconds in maintenance server routes. File-level
selection cannot divide those long serial files. Shortening their lease
or idle windows would weaken the regressions they protect.

## What Changed

- Split issue recovery into four files with separate database and worker
fixtures. Keep all 18 cases.
- Split native restart coverage into recovery, lease renewal,
preparation, checkpoint cost, and disk-loss files. Keep all original 79
cases, including the existing container-qualified cases. The master
update also retains its newly added recovery cases.
- Close each native case's database clients after pending failure
reports drain. The shortened file must not rely on idle timeouts to stay
below PostgreSQL's connection cap.
- Run the native Runner lane with two isolated fork workers. Register
every new file in the lane and verify that the partition still includes
every suite.
- Give each maintenance case an owned database. Run at most two cases
together with test-local assertion contexts. Keep both real 90-second
quiet windows and all 17 cases.
- Preserve serial execution when any explicit shared
`PAPERCLIP_*_TEST_DATABASE_URL` is present. An admin URL that creates a
separate database per fixture can run concurrently.
- Refresh shard weights from the protected master run. Mark the initial
split-file weights as conservative estimates until master measures them.
- Anchor the attention-retention fake clock to PostgreSQL time. Its
fixed clock started trailing real database timestamps after 12:00 UTC on
November 1, causing an archive-deadline test to time out. Keep its
deadline assertions and timeout unchanged.
- Document the fixture and concurrency rules. Leave runner authorization
and workflows unchanged.

## Verification

- Final head `c9a4ff11b3ba81277341294c7721b34fb8705ede`: [full PR CI
passed](https://github.com/paperclipai/paperclip/actions/runs/41477848686),
including typecheck, build, all test shards, E2E, and the final
verification gate. The rollup has 55 successful checks, one neutral
review check, four expected skips, and no pending or failed checks.
Greptile reviewed this exact head at 5/5 with no actionable findings.
- November 2 master update: merge commit
`c9a4ff11b3ba81277341294c7721b34fb8705ede` resolves the native recovery
import conflict while retaining the new busy-row and remote continuity
cases from master. All 38 test-runner checks and the three new busy-row
cases passed locally.
- Retention verification after the master update: the first local full
run passed 52 cases and timed out at 15 seconds in the paged-backlog
case. That case passed in isolation. The unchanged master suite passed
all 53 cases (82.95 seconds), and a full PR rerun also passed all 53
(76.65 seconds). The initial timeout cause remains unproven; no timeout
or assertion was weakened.
- Previously verified head `fd288a8e8c88df54dcdca7b7e42366b019182782`:
[full PR CI
passed](https://github.com/paperclipai/paperclip/actions/runs/41312625439).
There are 55 successful checks, one neutral review check, four expected
skips, and no pending or failed checks. Greptile reviewed this head at
5/5 with no actionable findings.
- Issue recovery: 18/18 passed before and after the split on the same
local machine and PostgreSQL fixture path. Wall time changed from
**630.6 seconds to 319.7 seconds** (49% less). All case titles match.
- Maintenance also passed all 17 cases on AWS: structured file cost
dropped from **511.8 to 282.6 seconds** across the linked master run and
the first PR run. These are different CI runs, so the matched local
comparison below is the stronger isolated measurement.
- Maintenance: 17/17 passed before and after with the restored pinned
dependencies. Wall time changed from **488.1 seconds to 317.2 seconds**
(35% less). All 120 assertion lines remain. Both real 90-second windows
also passed a final targeted run.
- Native restart: 71 passed and eight existing container-only skips
before and after the split. Wall time changed from **632.1 seconds to
346.8 seconds** (45% less). All 79 cases and 254 assertion lines remain.
- `pnpm -r typecheck`: passed.
- `pnpm build`: passed.
- Extra TypeScript checking of the moved recovery and maintenance tests
passed. The standard server config excludes `src/__tests__`. A separate
check of attention-retention fixtures reports existing type errors; the
unchanged baseline has identical diagnostics, with none introduced by
the clock fix.
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/ci/test-selection*.test.mjs`: 61 passed.
- `pnpm test:run`: stopped in the second chat partition: 335 passed and
four failed (two GitHub retry-state assertions and two Teams timeouts),
after 339 passed in the first partition. These chat tests are unchanged.
All four pass in a focused run (19.1 seconds). The full-suite
ordering/load cause remains unproven. All three CI chat jobs passed. The
previously failing cases passed there in 1.3, 1.3, 2.4, and 2.2 seconds;
see [the chat job
log](https://github.com/paperclipai/paperclip/actions/runs/41309691650/job/124857580714).
The attention-retention timeout repeated in CI and in an isolated local
run. Its fake clock was fixed at November 1 noon, which had fallen
behind real PostgreSQL timestamps. Anchoring it to the database clock
passes all 53 cases; the failed case now takes 425 ms.
- The native CI lane passed all 162 runnable cases with eight existing
skips after client cleanup: **650.7 seconds on the earlier master run to
391.0 seconds on this PR**.
- CI found native connection exhaustion after the long waits moved out
of the main file. The same failure reproduced locally at a
100-connection cap. Per-case client cleanup fixes the shortened file at
that cap, and all five native files now pass together at that cap (71
passed, eight existing skips).
- Early attempts exposed a missed fixture reference and a missed
reporting setup hook during extraction; both were restored. The first CI
policy run also required moving the new native fixture under test
helpers; that boundary check now passes locally. A separate auxiliary
command picked up the host pnpm version and removed dependencies. The
original maintenance baseline reproduced the same missing-package
failures, so that failed attempt is excluded from timing comparisons. A
later unchanged-baseline run passed all 17 cases. The pinned install was
restored before the passing maintenance run.

The audit also covered heartbeat process recovery, deferred receipts,
chat, workspace recovery, attention, and connection management. Their
shared fake clocks/provider mocks or existing partitioning need
different changes. The 16,400-row attention case and the large
checkpoint stream remain intact because their size protects real limits.

## Risks

- Two isolated processes use more memory and database connections than
one. Concurrency is capped at two. Shared database overrides force
serial execution.
- Moving setup into helpers can change test lifecycle order. The split
suites keep the same case assertions and use independent process state;
focused runs and the full CI suite check the move.
- Split-file weights are estimates based on case timings from the linked
run plus repeated setup cost. The next protected-master refresh replaces
them. Weights never permit tests to be skipped.
- Local timing varies with host load. CI measurements after merge are
needed to establish the overall PR wall-time improvement.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, tool use,
and code execution. The exact backend revision and context-window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (changed suites and focused
reruns; the full local-run caveat is recorded above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 10:19:24 -06:00
DottaandPaperclip baa4a2e64e fix(runner): restore Codex auth before source recovery (#18612) 
Restore disposable Codex authentication and safe provider configuration before cold source recovery. Preserve source receipts and conversation state until reconciliation, and rebuild assigned tools from the admitted launch binding.

Validate remote launch destinations, clear stale authentication when no selected credential exists, and atomically publish SSH file transfers without following retained links. Red/green recovery regressions, a real two-turn Codex/MCP probe, and all current PR checks pass.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
 2026-11-02 10:14:24 -06:00
Nicky LeachandPaperclip aab87ce707 Prove workspace delivery before inspecting cleanup candidates (#18493) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Completed tasks leave workspaces that need safe cleanup.
> - Providers without ancestry inspection currently need a recorded
merged PR before cleanup can start.
> - A squash merge changes commit identity, so direct ancestry cannot
prove that delivery.
> - This change proves delivery from the finalized HEAD and captured
destination before storage inspection.
> - Fresh storage checks and a second delivery check still protect
removal.
> - This reuses the existing worker and receipts. It adds no table or
timer.

## Linked Issues or Issue Description

Refs #18410. Related: #13400 addresses workspace identity during
inspection; this change adds destination-aware delivery discovery.

## What Changed

- Resolve direct ancestry or discover an associated merged GitHub PR
without a recorded work product. Require exact HEAD, destination
repository and branch, and a reachable integration commit.
- Use the latest settled finalization receipt as a candidate. Reject
missing, failed, pending, conflicting, or changed evidence.
- Inspect storage only after positive remote proof for providers that
disable ancestry inspection. Recheck HEAD, branch, source identity and
delivery before removal intent.
- Add an optional provider capability for a specific PR recovery hint
and an optional prepared destination receipt. Older providers keep their
existing protocol.
- Wake cleanup when finalization writes settle or gain a workspace
binding. Keep remote-only retries for concrete unresolved delivery. Park
missing or unsupported candidate identity.
- Reuse the source project's explicit GitHub consent for delivery checks
through the existing authorization service. Revalidate the bound
repository and consent before removal; missing or revoked consent
retains storage without switching identities. Public sources remain
anonymous.
- Update the contracts, documentation, access map, and targeted
regressions.

## Verification

- Initial-head GitHub CI passed at
`5acd4c818be6f1764f8319f42891f9956c357c26`, including full build and
typecheck. Standard Greptile is 5/5 with no open threads. The first Apex
review scored 3/5 and found startup latency from remote proof lookups
and removal retry identity changes across provider capability changes.
Both corrections are pushed in `9d1a38df3d`, with the qualified
destination-ref length fix. Red regressions reproduced each issue.
Latest targeted checks passed: cleanup/worker 26, provider integration
41, SDK contract 8. Head `9d1a38df3d` passed standard and Apex reviews
at 5/5 with no open threads, plus all CI including full build/typecheck.
Canary Dry Run initially failed on an npm 404 fetching
`rollup-4.67.3.tgz`; the exact tarball subsequently returned HTTP 200,
and one targeted retry passed without source changes. Aggregate
verification also passed. After #18403 merged, master introduced
conflicts in shared workspace cleanup and metadata handling. Resolved
them in `7700832536`, preserving remote snapshot cleanup and Git
delivery-proof cleanup. Targeted merge verification: 93 tests passed in
the first run (59 opt-in skips), then all 16 remote lifecycle tests
passed with explicit database opt-in; map checks passed. The isolated
database is stopped. The merge head received standard review 5/5. Its CI
found two failures in unchanged test cleanup paths: native runtime
temporary-directory ENOTEMPTY and agent-chat cancellation lock timeout.
Those failures remain recorded and require successful final-head
verification. The subsequent project-consent change has targeted
coverage: authorization36, resolver33 and cleanup19 tests passed,
including revoked consent, anonymous public access and source-binding
races. The positive consent test failed before the fix. Current head
`c772679e05` has green GitHub CI, including full build/typecheck and
both previously failing test jobs, plus standard Greptile 5/5 with no
open threads. Neither cleanup failure required changes to its test path.
Apex also completed at 5/5 on this exact head (check `124651548532`,
summary updated 2026-10-31 20:45 UTC), with no open review threads. The
PR remains draft. Earlier failures remain recorded above.

- Final focused run on the rebased branch: **182 tests passed across six
files**, covering the resolver, delivery cleanup, provider bridge,
finalization notifications, existing workspace service and SDK
contracts. The disposable test database was stopped afterward.
- Targeted resolver tests: 33 passed. These cover ancestry, discovered
squash PRs, wrong HEAD/base/repository, removed integration commits,
bounded responses, cancellation and credential isolation.
- Targeted cleanup integration tests: 17 passed. Provider bridge
integration: 36 passed. SDK contract tests: 7 passed. Finalization
notification tests: 9 passed.
- A negative control that removed the final delivery check failed the
race regression as expected. The check was restored.
- Read-only live GitHub probe: the new resolver discovered PR #18410
from HEAD `3f5ade4cd42b707040e9a82fdaed343e3d736a09`, without PR hints.
It proved integration commit `6484f96c28bfb11a793ce4be6cc5ac15643a5a02`
reachable from observed master
`5bc029bf6c1db6647e96acea323f3723fd1d3eba`. GitHub also advertised the
exact original HEAD at `refs/pull/18410/head`.
- `node scripts/check-maps.mjs` passed. Structural checks do not prove
deployment behavior.
- Local tests are targeted by user request. Full build, typecheck and
test suite run in GitHub CI. A standalone server typecheck could not
complete because local Runner build declarations were absent; no
changed-file diagnostics were reported.
- Initial local execution hit a stalled pnpm wrapper, sandbox socket
denial and stale generated SDK declarations. Direct installed binaries,
the authorized isolated test database and rebuilt SDK resolved those
setup failures. No deployment was performed.

## Risks

- Destructive cleanup still requires fresh storage safety and lifecycle
checks. Remote proof alone never authorizes removal. A persisted removal
intent preserves existing recovery semantics.
- Legacy workspaces without an unambiguous destination or completion
HEAD remain retained. This does not guess the original default branch.
- Private repositories use the explicit standing GitHub consent on the
bound source project, with current membership, grant, repository access
and trust checks. No separate company token is required when that
consent is valid. Missing or revoked consent and unavailable or
rate-limited access retain storage; the worker does not borrow a task or
agent identity.
- A known undelivered commit keeps the existing five-minute retry
because external GitHub changes have no guaranteed ingress to this lane.
This can keep cleanup scheduled. Missing evidence parks instead.
- Providers need the new recovery capability to clean a squash merge
whose original HEAD is reachable only through its PR ref. Providers
without it remain conservative.
- The resolver currently supports GitHub.com. Existing local and
other-provider ancestry paths stay in place.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools and
parallel code agents. The exact runtime model ID and context-window size
were not exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 07:59:17 -08:00
DottaandPaperclip 71a20453e1 fix(workspaces): honor inherited remote workspace bindings (#18611) 
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Task creation can bind a child to its parent's execution workspace.
> - Native sandbox runs reserve that workspace before the provider
starts.
> - Remote reservation rejected inherited isolated workspaces and
operator branches because the original source task differed.
> - This check stopped valid child tasks before their existing access
and writer checks could run.
> - This pull request uses the persisted binding and existing
source-access checks to admit that continuation.
> - The benefit is that authorized child tasks can continue saved work
while the single-writer lock stays effective.

## Linked Issues or Issue Description

**What happened?**

A child inherited an active isolated remote workspace through normal
task creation. Its next run failed during setup with `Workspace
ownership changed.` No provider work started. The reservation guard
treated the original `sourceIssueId` as exclusive ownership, although
task creation had persisted a valid child binding.

**Expected behavior**

A bound child or follow-up can reserve the workspace after the prior
writer saves. Current task/run access, source visibility, trust, company
scope, closed-workspace state, and writer checks must still apply. The
original source task remains provenance.

**Steps to reproduce**

1. Create a task with an active saved remote isolated workspace or
operator branch.
2. Create a child without an explicit workspace override.
3. Confirm that the child inherited the execution workspace ID.
4. Start its native sandbox run. The old reservation guard rejects it
before provider execution.

**Paperclip version or commit**

Reproduced on source based on
`06bc8db75c71142ed108e0f331394c5187015d6a`. The same guard is present at
`1a6017d6460267ad89d7e1784e2b521c28d8e9ac`.

Related: #17156 proposes separate workspaces for child tasks. This
repair preserves the current inheritance contract and enforces the
remote writer lock. #16911 addresses child bindings across repositories.

## What Changed

- Remove the original-task equality restriction for active remote
workspaces. Keep the existing persisted-binding and retained-source
authorization path. Explicit task-root selection follows the existing V1
filesystem contract; task-assignment grants do not define filesystem
permissions.
- Keep closed workspace rejection and unfinished-writer checks.
- Add database-backed regressions for inheritance, replay, competing
children, an unbound task, private source data, company isolation, and
closed records.
- Extend real save/restore tests for inherited isolated workspaces and
operator branches. Keep shared-workspace discovery coverage.
- Update the development guide and workspace maps.

## Verification

- Red: before the repair, the inherited isolated/operator cases fail
with `Workspace ownership changed.` The shared case passes. The unfixed
suite has five failures and four passes.
- Green: 11 reservation cases pass with real PostgreSQL 18 in an
isolated Docker container.
- An earlier run of the four affected workspace/preparation/trust suites
passed 191 tests. The review follow-up passes all 15 selected
reservation/save/restore cases. Inherited cases remove the original
files, use a distinct allocation and target, restore the saved file,
append child work, and independently read that work from the child's new
snapshot. Shared-source discovery stays covered. The filter excludes 42
other integration cases.
- `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and
`git diff --check` pass. A direct server typecheck also passes after the
final test edit.
- `pnpm test:run` stopped in its first chat shard: 338 passed, one
Discord receiver-lease setup failure, 1,478 outside the shard. The
failing test passes in isolation on both the unchanged baseline and the
fix. Its suite-order failure is not explained by that retry and remains
recorded. The unchanged baseline also fails three other Discord
publication cases in that chat shard, while its next chat shard passes.
This does not reproduce or explain the original lease failure; both
results remain recorded. CI is complete on `e697f6e1b8`: 54 successful
check runs, one successful legacy status, three skipped checks, and one
neutral check. No check is pending or failed. Greptile gives this commit
5/5 with no remaining findings. The security reviewer accepted the
V1-contract explanation; both review threads are resolved.
- Live sandbox acceptance remains unverified. The available test
credentials could not resolve the required sandbox snapshot. The
provider rejected creation before workspace execution, and a scoped
follow-up found no allocation. No live model turn, deployment, or
original-task recovery is claimed.
- Local environment: macOS arm64, Node 26.4.0, pnpm 9.15.4. Native
PostgreSQL startup hit the host semaphore limit; tests use disposable
Docker databases. No shared database was reset.

## Risks

- A bound child can now write after the prior writer releases the
workspace. Incorrect binding or access checks could expose source data.
Regressions cover unbound, private-source, and foreign-company
rejection.
- Concurrent children must not become two writers. A real transaction
race grants exactly one reservation; saved-file tests keep the second
writer blocked until save completes.
- No schema or migration changes. This PR does not change task-creation
inheritance, requested models, credentials, trust, approval gates, or
recovery budgets.

## Model Used

- OpenAI Codex, based on GPT-6. The exact serving model ID and context
window are not exposed in this session. Used reasoning, tool use, code
execution, and browser diagnostics. No sub-agents were used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — the targeted workspace
suites pass; the full local chat-suite failure and baseline comparison
are documented above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 09:36:05 -06:00
DottadevinfoleyPaperclip
 		06bc8db75c fix(runner): install verified API artifacts in execution workspaces (#18494) 
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Native agents use authorized API tools to read saved results.
> - Copying a large response as model-generated text can lose bytes.
> - The execution workspace needs an exact copy of the saved artifact.
> - This change copies the authorized bytes without shell API
credentials.
> - The runtime verifies size and SHA-256 before it reports success.
> - A live evaluation checks capture origin, installation and delivered
bytes.

## Linked Issues or Issue Description

**What happened?**

A native agent lost one byte while copying a 24576-byte API text page.
Existing capture and paging controls passed. The runtime had no verified
saved-artifact-to-workspace copy tool. A separate historical screening
refusal does not establish capture corruption.

**Expected behavior**

An authorized native run can install the complete saved artifact. The
stored and installed bytes must match in size and SHA-256. A conflicting
file must remain unchanged. A lost publication receipt must not cause an
unsafe repeated write.

**Steps to reproduce**

Run
`server/src/services/native-runtime/artifact-transfer.integration.test.ts`
on baseline `c46b8f35add9255fc71be294d27f5a7865bcb47f`. Its capability
assertion fails because `download_artifact` is absent. Apply this PR and
run the same regression with the filesystem and API controls. It
verifies 388131 synthetic bytes through the authority and installation
boundary.

Refs #18429. Related #17393, #17086 and #17285 cover paging, transfer
foundations and remote output reads. This PR adds the inbound
installation contract.

## What Changed

- Add `download_artifact` for supported standard-mode native runs, with
a 10 MiB limit.
- Reuse current company, task, run, asset-content and responsible-user
access checks. Check authority again before publication.
- Bind installation to the current execution lease. Stage through
confined Linux descriptors and publish with an atomic create-only link.
Keep API credentials out of execution commands.
- Verify bytes at storage and execution boundaries. Reject unsafe paths,
symlinks, different existing files, truncation and corruption. Reconcile
an unknown publication through verification only; an identical explicit
retry remains safe.
- Add API documentation, access-map entries and content-free audit
records.
- Preserve only the pinned Daytona SDK numeric HTTP status when failed
creation cleanup is unconfirmed. Keep provider exception messages and
headers private.
- Keep only the finished runner binary and portable provider pack in
their builder layers. Remove Rust targets/downloads and the Node install
workspace/store before export. Report tree sizes.
- Add one explicit Codex/Daytona Product E2E cell. Its independent
oracle checks the server capture audit, redacted tool receipts, stored
source, delivered attachment and browser link after reload. Uploaded
substitutes and one-byte loss fail.

## Verification

- Merge update at `6766b46553cab5b9787ade87a002eccaf50e68fc`: merge
current master and preserve both explicit suites in the catalog. Focused
catalog, image and oracle checks: 122 pass. Filesystem and authority
integration: 20 pass. Maps, module boundaries and whitespace checks
pass. New-head CI and Greptile are running. Full local build and
typecheck remain unavailable because cargo is absent.

- Behavioral red/green: the absent-tool regression failed before
implementation and passed afterward. The initial missing-package attempt
is recorded as setup failure.
- Linux filesystem and real PostgreSQL/authority integration: 19
filesystem tests and 1 authority test pass. Existing API authority,
capture and paging controls: 70 pass, 2 skipped. Provider transport in
these integration controls is simulated.
- Qualification calibration: 118 focused tests pass. The real
persisted-event redaction case and a byte-identical uploaded substitute
each failed before their oracle repair. The unchanged controls then
passed. Harness typecheck passes.
- Full recursive typecheck and build pass locally on the integrated
production source at `086f7a2901`. Later commits change only
qualification tests and documentation.
- Structural map/module checks and whitespace checks pass.
- Full local `pnpm test:run` was attempted by the implementation worker
and stopped under its 8 GiB memory limit, with no full-suite verdict.
Current-head CI must supply the full test result.
- The complete local Product E2E support suite has 13 failures. The
unchanged pre-qualification baseline reproduces all 13: host PostgreSQL
semaphore capacity, stale timeout/profile/workflow inventory
expectations, and expired fixed-date allowance fixtures. This is not a
full local suite pass.
- Earlier CI at `086f7a2901` failed a browser test during cleanup
cancellation with a database lock timeout. It remains recorded. All
browser shards passed on both later fully checked heads.
- Two qualification campaigns were canceled during setup before provider
execution while correcting the oracle. They provide no live product
verdict.
- Previous-head CI at `28fa001457f59df1c300c016dc569049e72d0061`: 53
checks pass, 3 intentional selection skips; no failed or pending checks.
- Real image/export regression: campaign 41272351780 compiled the Linux
runner and passed root/unprivileged CLI and transport checks, then
failed BuildKit export with `no space left on device` (exit 102,
filesystem 99.99%). No provider cell ran. Removing the Rust build trees
alone was insufficient: campaign 41273604815 also failed during
provider-pack export with exit 102. Commit `5bbb549c54` also
consolidates dependency installation and pack construction so the
disposable workspace/store can be removed in the same layer. All 122
image/oracle/catalog checks pass. Image and remote-pack publication then
passed in campaign 41275196741 after cleanup changes plus a scoped
60-to-120 GiB expansion of only its delete-on-termination builder disk.
The VM terminated and the expanded volume was deleted. This does not
establish that cleanup alone makes the default 60 GiB builder
sufficient.
- The Daytona admission probe then failed before agent creation, with
unconfirmed provider cleanup. The queued continuity case was canceled
because it shares this prerequisite. No artifact-transfer or continuity
success is claimed.
- Status-diagnostic regression: the original generic wrapper failed the
HTTP-status assertion. After the repair, the full plugin suite passes
501 tests, with 7 skips; its standalone TypeScript check passes.
Controls reject invalid status values, avoid accessors, preserve the
original create status and exclude credential canaries.
- Previous-head Greptile: 5/5; its summary names
`28fa001457f59df1c300c016dc569049e72d0061`. All review threads are
resolved.
- Live artifact transfer and three-turn warm continuity: NOT VERIFIED.
The diagnostic case in
https://github.com/paperclipai/paperclip/actions/runs/41277598289
confirmed Daytona HTTP 401 before agent creation. The configured
`DAYTONA_API_KEY` in the existing `runner-e2e-paid` environment needs
verification or refresh. The queued continuity control was not run after
its shared admission prerequisite failed. No further provider attempt is
planned until the credential changes. Unconfirmed provider cleanup
remains recorded.

## Risks

- Source-provenance changes can repeat the provider dependency install.
The combined build layer trades install-cache granularity for lower
retained disk use.
- Linux descriptor semantics and Node are required on the execution
host. Unsupported providers and restricted runtime profiles fail closed.
- Storage reads buffer at most 10 MiB. A provider outage can leave a
reserved staging file. Partial bytes never become the destination.
- A lost publication receipt can leave a complete file with an unknown
result. Recovery verifies the existing file without installation.
Conflicts are never overwritten.
- Artifact contents remain untrusted. The copy does not bypass screening
or authorize downstream execution.
- Live qualification uses the governed Codex API profile. It does not
establish every model, subscription profile, platform or historical
failure cause.

## Model Used

OpenAI GPT-6 in Codex assisted with diagnosis, code, tests and
qualification. The desktop runtime does not expose a more specific model
deployment ID or context-window size. The model used reasoning, source
inspection, tools and code execution. The native implementation agent is
configured with OpenAI `gpt-6.1-sol`.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 08:27:09 -06:00
DottaandPaperclip 67948173f4 fix(sentry): report saved-message task attempt failures (#18541) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tasks retain user messages when an execution attempt cannot finish.
> - The task interruption projection drives the saved-message failure
notice.
> - That state can occur without a new terminal run transition, which
the existing Sentry reporter requires.
> - This pull request reports that failed task state through the
existing Sentry capture path.
> - Operators can find the failure through the same tag and saved run
links.

## Linked Issues or Issue Description

Refs #13446 and #16300. The related PR search found no open
implementation of this change.

**What happened?**

A task can show “Couldn’t finish this attempt. Your message is saved.”
without a matching run-failure Sentry event. A stale saved-message wait
can produce this state after the previous run succeeded.

**Expected behavior**

When run-failure reporting is enabled, this failed task state also
reports with `event_kind=run_failure`.

**Steps to reproduce**

1. Enable the backend Sentry DSN and
`PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` or `all`.
2. Leave a saved-message wait on an unfinished task after its last run
has ended.
3. Read the task after the wait grace period expires.
4. The task shows the failure notice. Before this change, that read does
not report it to Sentry.

**Paperclip version or commit**

Base commit: `e122478368`.

## What Changed

- Report the system-owned failed presentation from the existing batch
task reader. The pure projection remains unchanged.
- Reuse `captureRunFailure()` with `event_kind=run_failure`,
`failure_source=task_interruption`, and
`error_code=task_attempt_failed`.
- Preserve the actual saved run status and company, agent, task and run
references. Load the run agent's adapter through a company-scoped join
in the existing run query, including after reassignment. Add no database
query or background timer.
- Send fixed error text. Do not read chat bodies, prompts or raw run
diagnostics for this event.
- Suppress repeated reports for the same company/task/run/intent with a
bounded process-local cache. Failed or disabled captures do not consume
the report.
- Document the enabled modes, event fields, duplicate behavior and cache
limits.

## Verification

- Regression before implementation: the two new Sentry tests failed
because the capture entry point was absent; the existing 38 Sentry tests
passed.
- Focused verification: 332 tests passed across
`run-failure-sentry.test.ts`, `task-interruption-query.test.ts`,
`task-interruption-presentation.test.ts`, and
`run-failure-report.test.ts`. The reassignment regression first
reproduced the missing adapter, then passed after the query correction.
- Command: `PAPERCLIP_TEST_POSTGRES_ADMIN_URL=<local-test-admin-url>
pnpm exec vitest run server/src/__tests__/run-failure-sentry.test.ts
server/src/__tests__/task-interruption-query.test.ts
server/src/services/task-interruption-presentation.test.ts
server/src/services/__tests__/run-failure-report.test.ts`.
- PostgreSQL tests used isolated databases on a local PostgreSQL 17
server, with automatic cleanup. Initial embedded starts failed because
installation scripts had not restored library links; after repair, the
host semaphore limit prevented new clusters. The existing
external-database helper supplied the working test path.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- `git diff --check` passed.
- `pnpm test:run` was attempted. It stopped in the unchanged chat
integration shard collection step with “No test found” for line filters.
The complete local suite has not passed.
- Sentry capture assertions use the SDK test double. Live Sentry
ingestion and a deployed browser journey were not tested.

## Risks

- An enabled instance can send more Sentry events when task reads
discover retained failures. It can emit a task-state event alongside the
separate terminal run event. The source tag distinguishes them.
- Deduplication is process-local and retains 1,000 reports. Restarts,
multiple replicas or cache eviction can report a retained failure again.
- No schema, permission, execution or UI changes. The existing opt-in
reporting gate stays in place.

## Model Used

OpenAI GPT-6 through Codex, with tool use, code editing and test
execution. The exact runtime model ID and context window are not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-02 08:17:22 -06:00
TonioandClaude Opus 5.5 8ea240218d feat(ui): bring the Refine-v2 layout to the agent page (#18378) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The agent page is where an operator reads and changes one agent.
> - The page grew one tab at a time. Ten tabs sit in one flat list, the
header holds many buttons, and the audit links leave the agent.
> - A design pass, called Refine-v2, made a calmer layout for this page.
It ran in a prototype on fixture data only.
> - This pull request moves the Refine-v2 layout and behavior into the
live agent page. Where the prototype and the live page differ, the live
behavior stays. For example, the floating save bar stays on screen.
> - The benefit is a shorter navigation, edits in place, and audit data
that stays inside the agent.

## Linked Issues or Issue Description

Refs #13716
Refs #9166

Both pull requests mount plugin tabs or slots on the agent page. They
touch the same files, so one of the three needs a rebase after the
others merge.

**Subsystem affected**

The agent detail page in `ui/` (`ui/src/pages/AgentDetail.tsx`, its
sidebar, header actions, and the shared agent config form).

**Problem or motivation**

The agent page is hard to scan. The navigation shows ten items in one
list. The header shows Run now, Pause, Clear error, and Assign task as
separate buttons. The name, title, and manager are form fields deep in
the Harness tab. Activity, Runs, Costs, and Budgets open company-wide
pages, so the operator loses the agent context. Each tab uses a
different card and table style.

**Proposed solution**

Use one layout system for every agent tab. Group the navigation into
Agent and Audit. Edit the name and title in the header. Move the work
actions into the actions menu. Show audit views inside the agent. Keep
the persistent save bar and add a status dot to it.

**Alternatives considered**

We considered a flag that lets operators opt in to the new layout. We
did not use a flag, because two layouts on one page double the test
surface. We also considered moving the save button to the end of the
scrolling content, as the prototype did. We kept the floating save bar,
because it is always visible.

**Roadmap alignment**

`ROADMAP.md` has no item for the agent page. This change is a UI
refinement of an existing page. It adds one optional field to an
existing API request and no database schema change.

## What Changed

- **Navigation.** The agent sidebar shows two groups. Agent has
Overview, Identity, Harness / Runtime, Secrets & variables, Permissions
/ Trust, and Advanced. Audit has Activity, Runs, Costs, and Budgets.
Skills, Tools, Channels, and API Keys leave the navigation. Their URLs
still open. The Instructions tab is now Identity, at `/identity`. The
old `/instructions` URL redirects.
- **Header.** The agent character shows at a larger size. The name and
title edit in place. Enter or blur saves. Escape cancels. An error
restores the old value. A rename updates the URL. While a tab has
unsaved changes, the header refuses the edit and asks the operator to
save or discard first. The third line shows the title and a link to the
manager. A crown button to the right of the name replaces the "Set as my
primary" link and the separate primary indicator. It is outlined and
dimmed until the agent is your primary, then fills with a short pop.
Replacing an existing primary asks for confirmation first, and the pop
plays after the dialog closes. Reduced motion skips the pop. Run now,
Pause or Resume, Clear error, and Agent history move into the actions
menu. Assign task is the primary button. For an agent that still needs
setup, Finish setup is the primary button and Assign task moves into the
menu.
- **Agent history.** A new dialog in the actions menu replaces the
Revisions entry. Each row shows a plain-language summary, the person or
agent who made the change, the date, and Restore. Restore asks for
confirmation. A failed restore shows its reason inside the confirmation.
- **Overview.** Four metric tiles show activity this week, runs this
week, costs this month, and budget used. Each tile links to its audit
view. When the list behind a count is capped, the tile shows a lower
bound, for example 200+. The activity tile follows live updates.
Profile, Skills, and Recent tasks panels follow. The Profile panel lists
direct reports under their label in at most two rows, then a "+N more"
button that expands the list. The Latest run card is removed.
- **Identity.** The file preview shows the raw text. A pencil button
switches to a plain text editor. Copy and Delete are icon buttons. A
chip next to the file name says "Managed by you" or "Managed
externally". Per-file history and unsaved edits stay.
- **External instructions.** "Advanced settings" opens Root path and
Entry file fields and a Load profile button. Load profile asks the
server to switch only if the root directory and the entry file already
exist. If either is missing, the server refuses and the panel shows the
reason. The entry file can have any name inside the root.
- **Instructions bundle API.** `PATCH /agents/:id/instructions-bundle`
accepts an optional `requireExistingEntry` flag (`packages/shared`
validator, `server` service, `ui` client). With the flag, the server
checks the root directory and entry file before it creates or copies
anything. Without the flag, the behavior does not change.
- **Harness / Runtime.** The config form gets a `panels` layout: Harness
and Execution side by side, then the runtime test, then environment
variables. The identity and run-policy sections are hidden from this
tab.
- **Secrets & variables and Advanced.** These tabs use the same filled
panels and table headers.
- **Permissions / Trust.** The trust presets show as selectable cards.
An Allowed connectors panel shows up to three connector logos and links
to Apps, filtered to installed connectors. Capabilities sit in their own
panel. Changes on this tab save at once, with no save bar.
- **Audit inside the agent.** Activity, Runs, Costs, and Budgets render
inside the agent page. Runs is locked to the current agent. A failed
budget incident action shows its reason under the incident.
- **Save bar.** The floating save bar stays. It shows an amber dot for
unsaved changes and a green dot with "Changes saved" after a save.
Discard stays.
- **Leave guard.** If the form has unsaved changes and the operator
clicks a link, a dialog asks to keep editing or discard the changes.
Browser back and the header actions keep the native confirm.
- **Apps.** `/apps?source=installed` opens the Apps page on the
Installed filter.
- **Tokens.** New `--agent-detail-*` tokens in `ui/src/index.css` hold
every spacing and fill value. Each fill is a mix of foreground into
background, so one value works in light and dark themes.
- **Tests and maps.** Tests cover the navigation model, sidebar, header
menu, Overview layout, Identity editor, and helpers. The
agent-configuration recipe in `maps/feature-map/agents.md` describes the
new page.

## Verification

- `cd ui && npx vitest run`: 849 test files and 10,081 tests pass.
- `cd server && npx vitest run
src/__tests__/agent-instructions-routes.test.ts
src/__tests__/agent-instructions-service.test.ts`: 40 tests pass.
- `npx playwright test --config tests/e2e/playwright.config.ts
tests/e2e/onboarding.spec.ts
tests/e2e/chat-adapters-ui-providers.spec.ts`: 26 tests pass.
- `cd ui && pnpm typecheck` passes. This includes the Storybook catalog
check. The `server` and `packages/shared` typechecks pass.
- `cd ui && pnpm build` passes.
- `pnpm run check:token-gates` reports all gates clean.
- `node scripts/check-maps.mjs` and `node scripts/storybook-catalog.mjs
--check` pass.
- Manual check on a local instance, in light and dark themes:
1. Open an agent. Confirm the sidebar shows the Agent and Audit groups.
2. Click the agent name in the header. Type a new name and press Enter.
Confirm the URL changes. Rename it back.
3. Open the actions menu. Open Agent history. Confirm the rows and the
Restore confirmation.
4. Open Identity. Click the pencil. Change a line. Confirm the save bar
shows the amber dot. Click a sidebar link. Confirm the discard dialog.
Click Keep editing, then save. Confirm the green dot.
5. Open Harness / Runtime, Secrets & variables, and Permissions / Trust.
Confirm the panels and the trust cards.
6. Open Runs under Audit. Confirm the list shows only this agent and the
page stays inside the agent.

## Risks

- This changes a high-traffic page for every operator at once. There is
no flag to switch back.
- Run policy and Reports to are no longer editable on the agent page.
The header shows the manager as a link only.
- Skills, Tools, Channels, and API Keys are reachable by URL only.
Operators who use them from the sidebar must use the URL.
- The pull request conflicts with #13716 and #9166 in `AgentDetail.tsx`
and the agent sidebar.
- Storybook stories for the agent page and `DESIGN.md` are not updated
in this pull request.
- The new `requireExistingEntry` flag is opt-in. Existing API callers
keep the old behavior, which creates a missing external directory and
entry file.
- I ran two e2e spec files locally, not the full `pnpm test:e2e` or
`pnpm test:release-smoke`. CI runs the full e2e suite. I did not run the
repo-wide `pnpm -r typecheck`, because the local toolchain has no Rust.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code, with
reasoning and tool use (shell, file edits, and a browser for visual
checks). A person directed every design decision.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-01 22:27:09 -08:00
DottaandPaperclip 1a6017d646 fix(server): resume suspended sessions after controller replacement (#18554) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A web server replacement must preserve a completed conversation as
well as active execution.
> - A warm remote Runner can suspend before the replacement admits
another turn.
> - The replacement restores its controller checkpoint from PostgreSQL,
but lacks the Runner suspension receipt.
> - It therefore tries to adopt a live process and rejects the safely
suspended source.
> - This change restores the verified receipt and preserves retained
launch files while the old authority is reconciled.
> - Later turns can keep the same provider conversation after the old
web server filesystem is deleted.

## Linked Issues or Issue Description

Refs #18552, #18543, #18366.

**What happened?**

The real rolling-deployment journey passed both HTTP handoffs, with 765
reads and 78 verified saves. The first three agent turns kept their
native and provider sessions. After the second replacement, the fourth
continuation failed with `remote_warm_session_not_idle`. The remote
Runner was suspended with an empty outbox and no pending terminal. A
focused regression reproduced that exact failure. After restoring its
suspension receipt, the same test exposed a stale launch-file read
during source recovery.

**Expected behavior**

A replacement can resume the same provider conversation from a settled
suspended Runner. It must validate the physical sandbox and all
persisted identity fields. A live source must still pass the existing
adoption checks.

**Steps to reproduce**

Enable the Linux Runner fixture for
`native-runner-restart-recovery.integration.test.ts`. Run the
`diskless-warm-suspended` cases. The fixture completes and gracefully
suspends each source, deletes the controller projections, and admits a
later turn from another process. It does not write a synthetic
suspension receipt.

## What Changed

- Reuse the existing per-turn suspended-receipt reader before warm
admission chooses live adoption or suspended recovery.
- Keep the receipt read retryable when a failed remote read leaves only
the controller projection.
- Leave every non-suspended warm state to the existing live-adoption
checks. A preliminary backpressure observation cannot reject admission
before that owner inspects the current Runner state.
- Preserve current sandbox binding, settled-outbox, exact-identity,
local-evidence and admission checks.
- Pass the structured launch specification to remote asset preparation.
Source reconciliation keeps the old authority's retained launch files; a
current-authority launch still stages its current files.
- Add two Linux Runner/PostgreSQL regressions for two controller-cache
deletions and three turns on one provider thread, including an
interrupted receipt read.
- Update the deployment qualification record, including the failed live
evidence and the successful staging database-idle observation.

## Verification

- Before the fix, the focused regression fails with the exact live
`remote_warm_session_not_idle` error.
- Both new regressions pass with real PostgreSQL and Linux Runner. The
provider is deterministic; this is not a live model qualification.
- The first receipt-only fix failed on the deleted controller skills
cache. The completed fix passes that same scenario without recreating
fixture launch assets.
- All 1,067 surrounding launch supervision, warm recovery, launch-asset
and controller recovery tests pass. The exact `cc23019691` server build
passes. All ten Linux continuity cases pass, including live adoption,
graceful suspension, per-turn resume, changed configuration and
failed-read retry. Each case checks provider-thread identity and exact
turn counts. The live Codex/Daytona journey passes on the same clean
source and exact build: 518 reads, 53 saves and 53 verified saves with
zero probe failures; both predecessor homes deleted; the active
execution completes once; all four turns preserve the provider
conversation and native receipts; saved files and the final command
output are independently checked. No automatic retries. Child credential
and object-storage bucket cleanup pass. Browser reconnection remains a
separate hosted acceptance check.
- The review refinement has a failing-before regression: only the
preliminary read reports transient backpressure; the subsequent adoption
reads the real Runner state. It and both suspension cases pass on the
final source (three cases), with server typecheck. The existing idle
check remains unchanged. An initial retry hit a fixture port left by the
intentional pre-fix failure; a fresh container resolved that
infrastructure error.
- Independent cleanup verification found two archived setup sandboxes
left by the live harness. Both were deleted and confirmed absent. The
failed automatic-cleanup check remains recorded.
- No claim of the ten-deploy Railway campaign or production canary yet.

## Risks

- A cold warm-session admission can now make one bounded remote state
read when local lifecycle evidence is absent. Its failure stops
admission and remains retryable.
- This path cannot replace contradictory local evidence, transfer
authority, or treat a suspended process as a live one.
- Source recovery must keep the previous authority's launch files until
reconciliation completes. The launch identity selects that behavior;
ordinary launches still prepare current assets.
- No schema migration or public API change.

## Model Used

OpenAI GPT-6 Codex, with reasoning, repository editing, shell execution
and integration-test tools. The exact runtime model variant and
context-window size are not exposed in this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

CI qualification note: first final-head run `41322376054` passed the
native/runtime and remaining suites but failed the unrelated
`automatic_message_on_reactivation` browser fixture at its
pre-reactivation deferred-queue count (0 rather than 1). The unchanged
test passed locally against isolated PostgreSQL (1 test, 43.6 s);
inspection found an asynchronous company-pause/fixture-seeding timing
window. Original CI logs and local evidence are retained. The single
failed-job retry passed; all final-head CI gates are green. Both reviews
are 5/5 and no review threads remain unresolved. The first local
embedded-Postgres launch failed before tests during cluster
initialization; the isolated external-Postgres test passed, and its
disposable database was removed after correcting the private launcher
cleanup call.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 10:08:51 -06:00
DottaandPaperclip c0550e7010 fix(server): preserve sessions and defer busy overlap recovery (#18552) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A web server replacement must preserve remote execution and later
task continuation.
> - The active-run handoff now survives deletion of the departing server
home.
> - The next admission still compared disposable instruction and skill
paths as configuration.
> - A native receipt could also settle without the old adapter returning
its terminal metadata.
> - A replacement can also collide with a run row locked by the serving
controller.
> - This change preserves durable session identity and terminal
evidence, and defers locked recovery work so overlap can reach
readiness.

## Linked Issues or Issue Description

Refs #18543, #18366, #9977, #18118.

**What happened?**

A real Codex/Daytona journey replaced both control-plane servers and
deleted their homes. Reads, saves, saved files and active execution
passed. A later useful answer used a new provider session. All four run
records lacked the native terminal contract. The whole qualification
therefore failed. A later attempt also exposed startup failure: the
replacement waited for a row lock held by the serving controller and
aborted on its one-second lock timeout.

**Expected behavior**

An unchanged managed instruction entry and immutable skill version
retain the session across server cache relocation. Actual content and
configuration changes still reset it. A committed native receipt retains
its terminal contract even when the originating adapter never returns. A
busy recovery row defers through the existing scheduler while the
replacement reaches readiness.

**Steps to reproduce**

Run the `rolling-web-deployment.runner-codex.daytona.recover-controller`
Product E2E cell with separate controller homes and shared
PostgreSQL/S3. The focused regressions reproduce the failures without a
model call.

## What Changed

- Read the managed instruction entry digest from the company-scoped
durable file head. Do not restore or read the cache for this check.
- Omit disposable instruction paths only when that durable authority
exists. Fingerprint selected immutable skill versions in both runtime
skill inputs.
- Preserve the existing local and external instruction path checks, and
unversioned skill identity.
- Publish the accepted native terminal contract in the same transaction
as the canonical result receipt.
- Skip locked recovery rows and schedule the same run for retry. Avoid
an audit write that would block on the same row. No ownership is
transferred while busy.
- Keep replacement readiness independent of an in-progress skill
selection; migration and cache eviction still require exact pins.
Missing durable skill versions still refuse startup.
- Project only the instruction digest in SQL instead of loading the
entire agent-file manifest per admission.
- Add real PostgreSQL lock contention tests for the issue, run and
coordinator, including the heartbeat startup caller and later claim.
- Limit native test teardown to its owned database on a shared test
host. The old behavior also fails on clean master.
- Add cache-loss, scope, changed-content, changed-version and
finalization regressions. Update the deployment qualification record.

## Verification

- Both focused regressions failed before the source changes.
- All 249 tests pass across `heartbeat-workspace-session`,
`agent-file-durability`, and `paperclip-control-plane-port`.
- Server typecheck and exact-revision build pass.
- Assembled admission ran 59 passing cases but failed during
native-suite teardown. The teardown failure reproduced on clean master.
All 16 native resumption cases and teardown pass after the
database-scope fix.
- All three row-lock regressions fail with the pre-fix lock timeout and
pass with deferred recovery. The full recovery suite passes 74 cases,
including the startup caller; eight Docker-specific cases were not
enabled.
- The real attempt at `d2165dec5d` failed when B aborted on the run
lock. Its credential and bucket were removed; all three provider
sandboxes were independently confirmed absent. The next real attempt at
`aae8a53d01` failed a different readiness gate while a run was preparing
skill pins. That race now has a failing-before regression; all 33
readiness, durable-file and managed-skill tests pass after the fix. Its
owned resources are cleaned. The `a15a12651c` live attempt passed both
web replacements: 765 reads, 78 saves and 78 verified saves with zero
HTTP probe failures. The first three turns preserved the provider
session and native terminal contract. The fourth continuation failed
because a safely suspended remote Runner was treated as live/idle; that
distinct recovery gap is being handled in a follow-up. All three owned
sandboxes were independently confirmed absent. No claim of full Railway
qualification yet.

## Risks

- The new fingerprint shape may reset a session once when upgrading from
older fingerprint metadata. It is stable across subsequent cache
replacements.
- A versioned skill is identified by its immutable selected version;
materialization and admission still enforce availability. Unversioned
skills retain their existing path boundary.
- A native terminal describes the accepted execution result. Workspace
and issue settlement remain separate authorities.
- This does not yet prove the ten-deploy staging campaign, natural
database autosuspension or the production canary.

## Model Used

OpenAI GPT-6 Codex, with reasoning, repository editing, shell execution
and live integration-test tools. The exact runtime model variant and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 09:14:34 -06:00
DottaandPaperclip 7751fc3107 fix(server): complete volume-free controller admission and recovery (#18543) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its volume-free web server must admit remote warm execution without
depending on a local disk.
> - Fresh run rows retain the initial legacy label during workspace
preparation.
> - The native runtime selector commits the execution mode later.
> - The earlier guard rejected valid remote runs before that selection.
> - This PR checks storage during preparation and the resolved lifecycle
during runtime selection.
> - Agent-file recovery also has to reconstruct disposable cache paths
from the retained remote directory.
> - The deployment test removes the complete predecessor home, including
immutable caches, to enforce that contract.

## Linked Issues or Issue Description

Refs #18518, #18529, #18533.

**What happened?**

A real Codex/Daytona deployment test failed during setup because the
volume-free workspace guard checked the initial run label. After fixing
admission, controller retirement first failed on an immutable skill
cache. Removing that cache exposed a production recovery failure: B
rejected the saved agent-directory path from A, even though the remote
execution identity was unchanged. The next live attempt recovered that
directory and transferred the same runner, but adoption then tried to
recreate provider launch assets from the deleted controller home.

**Expected behavior**

Allow remote workspace preparation before native selection. Reject
actual legacy and per-turn execution at the runtime boundary. Delete the
entire stopped predecessor home, including immutable caches, without
following links to shared dependencies. B must reconstruct its local
agent-file projection from the exact registered remote lease while
preserving the save baseline and unsaved remote edits. Reattaching an
already-running provider must not prepare a new launch or require the
old launch files. The full backend must recover the admitted
instructions from the matching durable run checkpoint.

**Steps to reproduce**

Run the explicit
`rolling-web-deployment.runner-codex.daytona.recover-controller` Product
E2E cell with a complete server/UI build, disposable PostgreSQL/S3, and
an owned Daytona fixture. The new boundary regressions also reproduce
the premature guard and missing runtime rejection without a model call.

**Paperclip version or commit**

The failures were observed on
`247552fe0719dcc92f7b42c50b658be6f624986d`. This branch is based on
`e122478368`.

## What Changed

- Validate Runner and remote workspace authority before runtime
selection without trusting the initial run label.
- Reject resolved legacy execution and local-source fallback, preserving
the native warm-policy check before admission.
- Rebind remote agent-file caches on ephemeral controllers and retrieve
their validated contents without uploading over the retained execution.
Preserve pending saves when the remote collector is unavailable.
- Restore over-quota remote directories without applying save quotas to
run admission; ordinary saves still enforce quotas.
- Skip provider launch asset materialization when adopting an existing
Runner. A real-process regression deletes the old skill/instruction
sources before reattachment.
- Recover admitted system instructions through Runner-owned checkpoint
validation for the same run, session and context digest. Fresh launches
still compose current instructions.
- Make the attention-retention test advance past its database-assigned
deadline, fixing a clean-master clock mismatch.
- Serialize remote collection and cleanup across controllers with an
operation-scoped Postgres advisory lock; fence stale cache-path updates.
- Keep synthetic deployment probe tasks in a separate disposable company
so they cannot block the business completion oracle.
- Share the launcher's directory cleanup with rolling retirement. Reject
a symlink root and skip linked descendants.
- Test admission with real PostgreSQL transactions and cleanup with real
read-only directories and external symlinks.
- Update developer documentation and the canonical qualification record.
Broader deployment acceptance remains in progress.

## Verification

- `pnpm -r typecheck` and `pnpm build` passed.
- All 156 workspace preparation, runtime selection and ephemeral
contract tests passed. Two new regressions failed before the source fix.
- Seven focused Product E2E cleanup/replacement tests and `pnpm
test:e2e:runner:typecheck` passed.
- `pnpm test:run` stopped at an unrelated 15-second chat integration
timeout after 338 passes. That exact test passed in isolation without
increasing its timeout. The later full-suite shards did not run.
- `pnpm test:e2e:runner:unit` reported 5,898 passes and 13 failures.
Clean master reproduced 11 failures in unchanged onboarding
expectations, operating receipts, Pi catalog assertions, workflow
bootstrap counts and embedded PostgreSQL startup. The other two workflow
assertions were fixed by the intervening master commit. These failures
are retained, not counted as passes.
- All 122 workspace-preparation tests pass after the reviewed
host-fallback fix. The first broader attempt exposed a test environment
leak; restoring the ephemeral flag between cases resolved it.
- Agent-directory and instruction regressions passed 179 tests. Five
focused cache-loss cases pass: different home, same path on an empty
filesystem, foreign lease, concurrent controller, and missing collector.
- The live fixture at `a18f6b85a3` gracefully stopped A and deleted its
entire home, then failed the 420-second remote reattachment deadline on
the saved local agent-directory path. That failure is retained; exact
owned sandbox cleanup was independently confirmed. The next live attempt
at `e14ca5f3e9` restored the cache and transferred the same runner, but
failed while rematerializing launch assets from A. Both failures are
retained and their owned sandboxes confirmed absent. No active-overlap,
ten-deploy, or production-canary acceptance is claimed.
- All 71 related Runner adoption/recovery cases pass, including the new
removed-context regression (failed before the source fix).
- The revised full agent-directory suite passed 110 cases and failed
three cleanup timing cases. All 20 cleanup/cache-loss cases then passed
in isolation with unchanged timeouts, including over-quota recovery.
- Server and Runner TypeScript checks pass after the latest corrections;
the live retry remains pending.

- The live test at `67714af9ff` adopted the same remote execution after
A was deleted, then failed while the full backend read the retired
instruction bundle. All owned sandboxes were independently confirmed
absent. Its regression now passes with actual backend recovery and no
new thread or turn. All 59 backend factory tests and focused checkpoint
identity tests pass. Runner and server TypeScript checks pass; the exact
committed server build passes. A fresh paid-model retry followed.
- The attention-retention CI timeout reproduced on clean master. The
deadline correction passes all 53 tests in that file with unchanged
timeouts.

- The real `cd690de4d3` run transferred the same runner/session after
deleting A, completed, and completed the next revision. During handoff,
372 reads and 38 saves had zero failures; all 38 saves were verified.
The full attempt was interrupted because the probe's open task
contaminated the business company oracle. This remains incomplete
evidence. The scope regression fails before the fixture fix, then
passes; all nine deployment support tests and E2E typecheck pass. All
three owned sandboxes were confirmed absent and temporary
credentials/storage were cleaned up. A clean corrected attempt is
running.

## Risks

- The runtime guard must still reject legacy and per-turn execution
before dispatch. Tests cover these refusals and a valid resolved warm
policy.
- Directory permission changes apply only during cleanup of stopped,
test-owned homes. Traversal does not follow symlinks. This changes no
production cache permissions.
- Cache reconstruction must not change remote execution authority or
rebase the original save baseline. Tests verify foreign-lease rejection,
preserved receipts, and absence of local-to-remote upload.
- Postgres locking is scoped to active remote file work and adds no idle
polling; local/durable-server locking remains unchanged.
- No schema or public API changes. External conversation exemptions
remain unchanged.

## Model Used

OpenAI GPT-6 Codex, with reasoning, repository editing, shell execution
and live integration-test tools. The exact runtime model variant and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Latest real qualification at f5915fdc33: both empty-home replacements
completed with 523 reads, 53 saves and 53 verified saves, zero probe
failures. The active remote execution survived and completed once; saved
project bytes and the later useful answer passed. The full workflow
remains FAILED: the later continuation reset its native/provider session
and the public run row lacked the native terminal contract. These newly
exposed compatibility/projection gaps are tracked in the next
implementation slice; this PR is not a claim of full Railway
qualification. All owned Daytona sandboxes were independently confirmed
absent after cleanup.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 08:24:14 -06:00
DottaandPaperclip 247552fe07 test(server): avoid deep traversal in durable file assertions (#18533) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud web servers restore agent files from shared storage after
replacement.
> - The durability test checks restoration, corruption, cancellation and
retry.
> - Its generic equality assertions traverse every byte in many large
buffers.
> - The post-merge check for #18531 exceeded the 15-second test timeout.
> - This change uses native buffer equality and keeps every byte check.

## Linked Issues or Issue Description

Refs #18531.

The post-merge Cloud-source verification run timed out in the new
cold-folder test. Other shards passed. The unchanged test passed
locally. This change removes avoidable assertion cost; it does not
increase test or production timeouts.

## What Changed

- Use `Buffer.equals` for the cold-folder content assertions.
- Keep the concurrency, cancellation, cleanup, old-projection and retry
assertions.
- Record packaged-image proof, the hosted deployment state and the
original CI failure in the existing plan.

## Verification

- Eight durability tests passed against a real local PostgreSQL fixture.
Object storage uses controlled asynchronous streams.
- The same test file took 10.03 seconds before and 5.90 seconds after
this change. The changed cold-folder case took 561 ms after the change.
- The first local attempt could not start embedded PostgreSQL and ran no
tests. The existing external test database enabled the comparison.
- Original failed CI:
https://github.com/paperclipai/paperclip/actions/runs/41296253723 . That
failure remains in the qualification record.
- Server typecheck and build passed. All latest-head PR checks passed,
including the full test, typecheck, build and browser gates. One
cancelled runner job was retried after its log showed an infrastructure
shutdown; the retry passed. This PR changes no production code or
database schema.

## Risks

- A failed content check now reports a boolean mismatch instead of a
large byte diff. Equality still covers buffer length and every byte.
- Local timing is evidence for the fixture, not a general CI performance
guarantee.
- Hosted active-agent and rollout acceptance remain incomplete as
recorded in the plan.

## Model Used

OpenAI Codex, based on GPT-6. The exact runtime snapshot and
context-window size are not exposed. Used code editing, shell execution
and test analysis.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 05:13:30 -06:00
DottaandPaperclip 1d348bbef0 fix(server): restore durable agent folders with bounded concurrency (#18531) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A replacement web server restores agent files from shared storage.
> - The old restore reads each object in sequence.
> - Modest network latency makes a 1,024-file folder exceed its
deadline.
> - This pull request uses at most eight concurrent restore streams.
> - Every file must pass its size and hash checks before the folder is
published.

## Linked Issues or Issue Description

Refs #18518 and #18526. Local deployment qualification found that a 64
MiB folder could not be restored after the old server filesystem was
removed.

## What Changed

- Restore at most eight files concurrently within the existing file
authority service.
- Cancel sibling transfers on failure. Await all workers before removing
the temporary folder.
- Keep the existing deadline, file-transfer database pool, authorization
and publication rules.
- Add regression coverage for bounded concurrency, corruption, settled
cancellation, retained previous bytes and retry.
- Document the behavior and update the deployment plan with local
qualification evidence.

## Verification

- Real PostgreSQL and S3-compatible storage: 1,024 files of 64 KiB each,
plus instructions, with 25 ms added to each object GET. Delete the
entire predecessor home and disable local adoption before reading
through the successor.
- Before: the read failed after 30.065 seconds. After: the same case
passed in 4.180 seconds and fetched 1,025 objects. All owned test
resources were cleaned.
- The durable-file suite passed eight tests against PostgreSQL. Server
TypeScript compilation passed. All 56 latest-head CI checks are complete
and green or intentionally skipped, including full typecheck, build,
unit and browser suites. Greptile is 5/5 with no review findings or open
threads.
- The plan separately records a released-container overlap proof: 130
reads, 26 saves, zero failures, two deleted predecessor containers and a
retained authenticated session. That image predates this restore change.

## Risks

- A cold folder can use eight simultaneous streams instead of one. Two
file-transfer database connections bound the simultaneous restore
callers. Streams verify bytes without loading entire objects into
memory.
- The 30-second deadline is unchanged. The measured size and latency do
not qualify every folder at the maximum storage limits.
- No schema, API or UI changes. Paid-model and hosted deployment
acceptance remain separate work.

## Model Used

OpenAI Codex, GPT-6. Tool use, code execution and repository review. The
runtime does not expose a more specific model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 04:09:47 -06:00
DottaandPaperclip d941ed3879 feat(server): verify existing-volume ephemeral enrollment (#18529) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud web servers can now keep durable state in PostgreSQL and S3.
> - Existing servers still have files and a decision signing key on
their volume.
> - The volume must stay attached until every data owner verifies its
migration.
> - This pull request adds a dedicated maintenance process for that
transition.
> - Cloud can preserve the original key and remove the volume only after
a matching receipt.

## Linked Issues or Issue Description

Refs #18518 and #18526. Existing-volume enrollment is a required step in
the zero-downtime web deployment work. This is the Core maintenance
side; the Cloud transition is a separate private change.

## What Changed

- Package a standalone maintenance entry that starts no normal app
routes or execution workers.
- Coordinate the existing agent-file, run-evidence and skill migration
owners. Follow every page and verify shared bytes before final
eligibility.
- Require stopped writers. Refuse active execution, missing local bytes,
local workspaces and external instructions.
- Read the existing decision key without generating a replacement.
Preserve normal key creation for ordinary server startup.
- Require an attempt token for status and export. Export the original
key only after successful migration. Keep status and errors free of
secret material.
- Document the Cloud ownership boundary, deadline, retry and retention
requirements.

## Verification

- Full workspace typecheck and build passed before the review fixes. The
final server TypeScript compilation passed after correcting Drizzle's
optional predicate type.
- Production runtime bundling passed: 2,702 inputs, 122 chunks,
including the maintenance executable.
- The focused suite passed 21 cases using PostgreSQL and real
S3-compatible storage. It covers multi-page migration of 26 agents,
exact reads after deleting the predecessor home, corrupt-object
rejection, active/missing-data refusal, and the bundled plain-Node
process with authenticated export and clean SIGTERM exit.
- Regression cases refuse unsaved terminal working copies, require
explicit stopped/cleaned receipts for unavailable copies, clean 101
expired traces through the owning service, and enforce the deadline
after migration succeeds.
- Initial integration attempts exposed fixture cleanup ordering
problems; corrected runs pass. CI also caught a Drizzle predicate type
error, now fixed and compiled locally.
- All repository CI gates passed on a4db955403, including typecheck,
build, browser suites and serialized tests. Greptile reviewed that head
at 5/5; all four review threads are resolved. Hosted enrollment and
active-agent continuity remain separate acceptance work.

## Risks

- This is an explicit maintenance entry. The caller must confirm all
prior deployments stopped and keep normal tenant routing closed through
detachment.
- A timeout blocks export but an owner may finish its current bounded
operation before observing cancellation. Cloud must confirm process exit
before detaching. SIGTERM has a ten-second hard exit deadline.
- The authenticated export contains the original signing key. The Cloud
caller must put it only in the stack vault and must not log the body.
- Local-workspace and external-instruction migrations remain their
existing owners' responsibility. Enrollment fails closed until those
data are eligible.
- No database schema changes. Ordinary server startup is unchanged
unless this maintenance entry is explicitly launched.

## Model Used

OpenAI Codex, GPT-6. Tool use, code execution and repository review. The
runtime does not expose a more specific model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 03:56:44 -06:00
DottaandPaperclip 73cd8eb9a5 fix(server): deliver answers promptly during web handoff (#18530) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its web servers can overlap while Railway replaces one deployment.
> - An answer can reach the replacement while the original server owns
the native execution.
> - A competing delivery claim can make the owner miss the first
notification.
> - The normal retry interval then delays the answer by 30 seconds.
> - This pull request gives pending answers a one-second retry in
ephemeral mode and verifies delivery between separate processes.

## Linked Issues or Issue Description

Refs #18518 and #18526. This closes a measured native-question delivery
delay during control-plane web deployment overlap.

## What Changed

- Register question delivery through its owning service. Use a
one-second retry for ephemeral servers and retain the configured cadence
for ordinary servers.
- Keep the existing durable claims, response identity, authorization and
continuation behavior.
- Add a two-process PostgreSQL regression for a claim held by the
non-owning server. Verify one command, settlement, idle behavior and
reactivation by another committed answer.
- Keep the assembled startup fixture on the real registration function.
Update the deployment plan with current evidence and remaining
acceptance work.

## Verification

- The same local two-process race took 30.157 seconds before the fix and
1.197 seconds after it. The command sink is simulated; PostgreSQL,
notification transport, interaction response, delivery claim and worker
are real.
- The cross-process regression and assembled startup suite passed 122
tests. No further sweeps or pending inspections occur after settlement.
A second answer wakes the owner through PostgreSQL without a test-owned
sweep.
- Server TypeScript compilation passed. All PR CI gates passed on
d1d426c0dc, including full tests, typecheck, build, browser shards and
serialized suites. Greptile reviewed that head at 5/5 with no findings.
- Live provider continuation, hosted staging deployments and production
acceptance are still separate work.

## Risks

- An unresolved answer causes more frequent reconciliation in ephemeral
mode. Settled history does not keep the worker active. Existing claim
fencing and bounded error attempts are unchanged.
- A 30-second abandoned claim still needs its existing stale-claim
recovery. This change addresses released claims, not proof of provider
liveness.
- The test simulates the provider command sink. It cannot establish that
a paid model consumed the answer or that a remote execution survived a
deploy.
- No schema, API or UI changes.

## Model Used

OpenAI Codex, GPT-6. Tool use, code execution and repository review. The
runtime does not expose a more specific model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 03:39:48 -06:00
DottaandPaperclip 16787e4167 Qualify rolling web deployments and preserve Stop through handoff (#18526) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The web server owns request handling and remote execution
coordination.
> - Deployment must start a ready replacement while the old server still
serves.
> - An outage test does not prove that two live servers can overlap
safely.
> - This pull request adds a separate Product E2E cell for that
boundary.
> - The existing run, file, recovery, and retained-session checks still
apply.

## Linked Issues or Issue Description

Refs #18518. The runtime prerequisite is merged; this PR qualifies its
ephemeral server and deployment contracts.

## What Changed

- Add the explicit `rolling-web-deployment` Codex Daytona workflow.
- Start independent A/B/C homes and compiled builds behind a test
ingress.
- Wait for readiness before routing, then drain and delete the old home.
- Observe active authority before handoff and use the existing read-only
recovery oracle.
- Require ordinary reads and saves to succeed throughout replacement.
Read back every accepted save exactly once.
- Preserve the five-minute outage suites and their existing validators.
- Add a model-free supervisor smoke and an unready replacement control.
- Require graceful exit before deleting an old home; forced cleanup
cannot satisfy the oracle.
- Preserve a durable Stop through lease release/expiry until the
recovered execution owner acknowledges it. Permit only fenced
reattachment of the original execution for Stop cleanup; submit no new
turn.
- Exclude events saved by A during shutdown from evidence of B making
progress.

## Verification

- E2E TypeScript check passed.
- Affected harness checks: 143 passed. Catalog, fixture and report
checks: 120 passed.
- Actual supervisor smoke passed two replacements: 436 reads, 45 saves,
zero request failures, and confirmed owned resource cleanup.
- B became ready in 18.7 seconds; C became ready in 109.2 seconds. A
prior C timeout remains failed evidence.
- Fresh browser proof: 11 checks passed, 246 reads, 50 acknowledged
saves, zero failures, 1,058ms reconnect, both old homes deleted and
cleanup confirmed.
- Stop lease-gap regression: three PostgreSQL delivery tests passed with
a simulated provider edge. Current server TypeScript build/vendoring
passed.
- Full harness: 5,814 passed, 17 failed, four skipped. Two shutdown
races passed the focused file; two report timeouts passed an isolated
42-test rerun. Onboarding/workflow-security/Pi/date-expired fixtures are
unchanged on inspected master; the host embedded PostgreSQL fixture
remains a setup limitation. This is not a green full local suite.
- CI at `172509b8be` passed all gates. Greptile is 5/5 with every review
thread resolved. The earlier access-service race failure passed its
isolated regression and the subsequent full CI.
- Final review corrections: six focused executor cases, three PostgreSQL
command-delivery cases, 19 harness cases and server/E2E typechecks
passed.
- Three live attempts failed before agent creation: two harness setup
gaps were fixed, then provider authentication failed. The saved OpenAI
key returns 401; a refresh is pending. The Daytona probe reported
cleanup uncertain, and no sandbox with that attempt label was found
through the authenticated staging manager. No live model execution is
claimed.
- Live Runner overlap, released-container qualification and hosted
deployment proof remain pending. Merging this opt-in qualification lane
does not claim those results.

## Risks

- The source lane shares workspace dependencies and does not establish
container packaging correctness.
- The live workflow needs an explicitly owned S3 bucket, PostgreSQL,
pinned remote image and provider credentials.
- Qualification cleanup stops all owned processes after failure. A
failed replacement must not be reported as a pass.

## Model Used

OpenAI Codex, GPT-6. Tool use, code execution and repository review. The
runtime does not expose a more specific model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-01 02:53:27 -06:00