mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-15 03:51:22 +02:00
ddcd746eb6a01f2bbe9b4adeec238ba35bde5adb
5049
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
ddcd746eb6 |
feat(runner): qualify native Codex platforms and cache hosted builds
Extract the tested runtime and installed-package prerequisite without changing agent creation defaults or experimental gates. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7d9b93aee2 |
docs: establish paired feature and agent access maps (#16025)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors use the feature map to understand and verify product capabilities. > - Agents need APIs and usable entry points within their permission boundaries. > - Product recipes and agent access need separate records with shared capability IDs. > - This pull request establishes that contract and gives maintenance ownership of both maps. > - The benefit is repeatable access review without changing product behavior or permissions. ## Linked Issues or Issue Description Refs: #15983, which added the portable verification skills and updated the feature map. This is its contract-only follow-up. Related: #15288 added the original feature map. **Issue type** Missing documentation and maintenance contract. **Where is the issue?** The product map, its missing agent access companions, and the verification skills. **What's wrong?** There is no shared convention for API presence, agent entry points, permission boundaries, access gaps, or evidence. An API declaration does not prove that a permitted native or remote agent can use it. **Suggested fix** Keep the product and agent access maps as siblings under `maps/`. Reuse capability IDs. Record representative source inventory separately from operation review and live proof. Keep both maps aligned through the maintenance skill. ## What Changed - Move the existing product map to `maps/feature-map/`. Preserve its recipes and Poteto format. Update relative links and add companion links. - Add `maps/agent-access-map/` with 35 area files and 171 matching capability IDs. Seed representative API operations, agent paths, policies and known gaps from the reviewed source inventory. Mark entries as `baseline`, `source-only`, and live verification `not-run`. - Define the shared contract in `maps/README.md`. Track API presence, agent availability, policy, evidence and review depth separately. Require operation enumeration before marking a broad capability operation-reviewed. - Extend `maintain-verification-skill` to reconcile both maps and inspect actual route policy and production tool admission. Update creation, verification, notices and contributor references to use the shared contract. - Add `scripts/check-maps.mjs` and ten regression tests. Check matching areas/IDs, indexes, references, contract fields, API method/path examples, justified exceptions and evidence-state consistency. - Keep the coverage inventory unchanged. Preserve skill invocation policies and both MIT license blocks. No product API, permission, UI, runtime helper, migration, dependency or workflow changes. ## Verification - Passed: `node scripts/check-maps.mjs` reports 35 paired areas and 171 matching capability IDs. All local map references resolve. - Passed: `node --test scripts/check-maps.test.mjs` runs ten tests. Fixtures cover missing companions, renamed/orphan/duplicate IDs, broken references, missing operation fields, empty exceptions, missing API examples, empty policy fields, and verified evidence without passed verification. - Passed: metadata validation for all three verification skills. Skill and contributor links resolve. Invocation policies and both license blocks are unchanged. - Compared relocated recipes with current master. Contents differ only in relative links and companion links. `coverage.json` is byte-for-byte unchanged. Master's updated read-only interaction test reference is retained. - Passed: `git diff --check`. The branch starts from current master and contains only this contract change. - Repository-wide typecheck, tests and build were not run locally for this documentation and Node structural-check scope. Live agent access was not exercised. Those gaps are explicit in the baseline entries; the structural check does not certify behavior. CI and automated review are pending. ## Risks - The companions are representative source baselines. They are not complete operation inventories or passed agent journeys. - The checker cannot establish actual permissions, native tool admission, operation completeness or truthful live evidence. Maintenance must inspect source and verify with the actual permitted agent context. - External links to the old product-map directory may need updates. Repository references have been updated. - Existing access gaps and board/user restrictions remain recorded. This contract does not change authority or implement missing paths. ## Model Used OpenAI GPT-6 through Codex assisted with this change. The session does not expose the exact serving model ID, configured context-window size, or reasoning setting. Capabilities used: source analysis, file edits, shell commands, and structural regression tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
4b766b8729 |
fix(acpx): record turn activity at the moment a stop fires (#15962)
Record stop-time ACP turn diagnostics (last-event age, oldest pending tool age, pending tool kinds) so wall-clock timeout events show whether the turn was stalled. Diagnostics only. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>canary/v2026.1012.0-canary.12 |
||
|
DottaandPaperclip
|
0d0e138dad |
refactor(chat): extract channel status, receipts, drafts and authority workflows (#15995)
## Thinking Path > - Paperclip manages AI agents and their work. > - Chat channels connect that work to provider conversations. > - Provider workflows still fill the shared service. > - Their authority and recovery rules need clear homes. > - This PR lands five reviewed extractions into channel folders. > - Shared authorization, state and worker lifetime stay explicit. ## Linked Issues or Issue Description Refs #15976, #15996, #15999, #16001 and #16002. This root PR now contains all five extractions from batch three. The four upper PRs were merged into their parents. The combined tree matches reviewed stack head `789e918b5af49ec1958076ce4d22d31006205302` exactly. ## What Changed - Extract Slack session status and accepted-file receipt recovery. - Extract Telegram publication drafts and Stop handling. - Extract Photon events and native responses. - Extract Teams file-transfer authority. - Reduce `chat-channels.ts` from 29,897 to 26,461 lines (3,436 lines, 11.5%). - Preserve function bodies, runtime-map identity, leases and transaction boundaries. ## Verification - All 117 selected public-service cases passed before and after their respective extractions. - All 30 moved declarations and remaining service statements preserve normalized syntax across the five slices. - Server typecheck, module boundaries and whitespace checks passed. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. Combined-head CI and review are pending on `5de1b0b99d71049a72440e8c131bf4bb30bdf3f3`. - Fixtures exercise assembled services and disposable databases, not live third-party providers. - Cumulative local `pnpm test:run` on `789e918b5af49ec1958076ce4d22d31006205302` exited 1: 32,414 passed, 125 skipped, one failure. `company-skills-routes.test.ts:1691` (`stars, forks, and comments on skills through company-scoped endpoints`) failed with `socket hang up`; that test and its route are unchanged by this stack. The same 58-case suite then passed in isolation without code changes. The original command did not reach 111 later serialized suites; its failure is retained, so the full-local-test checklist remains unchecked. ```sh pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts -t 'Slack session|Slack turns to Idle|signed Slack session stop|denies guest Slack stops' ``` - Combined-head CI attempt 1: five runners received a shutdown signal at 19:13:08 UTC (server shard 10/12, serialized shards 2/9 and 7/9, E2E shards 3/8 and 7/8). These jobs stopped during execution without a failing test assertion. [Original workflow](https://github.com/paperclipai/paperclip/actions/runs/38293566028). The affected jobs will receive one unchanged retry after the other jobs finish. ## Risks - Changes cross durable provider recovery and authority boundaries. The moved bodies preserve syntax, and 117 focused public-service cases passed before and after their extractions. - Shared workers, timers, runtime maps and generic authorization remain in the service. - The full local test run failed in an unchanged company-skills route test; its result and the unrun suites are recorded above. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cc88b61c09 |
feat: add portable verification skills and refresh the feature map (#15983)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors use the feature map to select verification recipes. > - The repository needs separate skills to create a verifier, maintain recipes, and run checks. > - Shared recipes must match current routes, controls, and prerequisites. > - This pull request adds portable verification skills and updates the existing map. > - The benefit is a repeatable verification workflow with clear ownership and evidence limits. ## Linked Issues or Issue Description Refs: #15288, which added the existing feature map. The related test cleanup in #15978 changes test ownership; this PR changes skill instructions and recipes. **Issue type** Missing documentation and outdated verification instructions. **Where is the issue?** The project-local skills in `.agents/skills/` and the recipes in `feature-map/`. **What's wrong?** The repository lacks the portable creation and maintenance skills. The verifier mixes live verification with map editing. Some recipes describe old controls or omit required flags, identities, and provider setup. **Suggested fix** Import the skills and their complete references from Paperclip's pstack fork. Keep the existing map in its current directory. Let the verifier read recipes and report drift. Let the maintenance skill update the map. ## What Changed - Add `create-verification-skill` and `maintain-verification-skill` from the portable fork of Poteto's pstack. Include invocation policies, all feature-map examples, self-contained attribution notices and the full MIT license in each skill folder. - Use the existing `feature-map/` directory and Poteto's recipe format. Omit the fork's separate runtime skill and runtime-skill dependencies because Paperclip supplies execution. - Add `verify-paperclip` with executable launch, readiness, onboarding, and cleanup helpers. Use isolated temporary data and separate evidence directories. - Make `verify-paperclip` consume the map and report recipe drift. Give map editing to `maintain-verification-skill`. - Review all 35 feature files and correct 30 recipes. Clarify routes, flags, current controls, persistence scope, and the limits of linked tests. - Remove the unrelated Conference Room flag change from the onboarding smoke helper. - Add a standard-library ownership test. It proves that doctor and stop reject a foreign checkout without stopping its live server. A delayed-exit fixture also proves cleanup waits for runtime exit. - Keep audit screenshots and scratch reports outside the repository. No product code, migrations, lockfile, or workflow changes. ## Verification - Passed after rebase: metadata validation for all three skills, Python syntax and executable checks, JavaScript syntax, 35-entry map/index consistency, relative links, license preservation, and `git diff --check`. - Ran one isolated live audit before rebase. Checked task persistence, document upload/download bytes, goal hierarchy, project edits, search, local process Agent Chat, question replies, queued-message discard/stop, and a harmless loopback MCP call. - Checked local settings and draft flows. Synthetic cost events reached a small budget limit and paused the test agent. These checks do not prove external model quality or provider billing. - Checked cleanup of the audit server, database, local MCP fixture, and workspace service inventory. - Passed: `pnpm -r typecheck`. - Greptile: 5/5 on commit `4344ac4df7`. The cleanup finding is fixed and its review thread is resolved. A fresh review is pending for the notice relocation. - Passed: `python3 -m unittest discover -s .agents/skills/verify-paperclip/tests -v`. Both tests pass: the foreign fixture stayed alive after rejection, and owned cleanup waited for delayed runtime exit. - Passed: `pnpm build`. - All GitHub CI checks were green on commit `4344ac4df7`. The cloud-auth browser shard passed after one retry for a disposed mocked API response. The notice relocation passed skill metadata, relative-link, and license-preservation checks; fresh CI/review is pending. - The full local `pnpm test:run` did not complete. It was interrupted after nearly 30 minutes once the complete CI suite passed on the same commit. Exit code: 130. The new helper safety tests passed locally. The local full-suite checklist item stays unchecked. - Full live coverage is incomplete. Authenticated multi-user checks and external provider journeys need their own identities, credentials, and targets. - The corrected onboarding helper could not be re-tested after removing the flag mutation. Two fresh launches failed before server startup. Direct PostgreSQL initialization reported exhausted shared-memory IDs or limits. The host had free disk space. No host-wide limits or unrelated instances were changed. ## Risks - Recipe drift can return as the UI changes. The map records verification scope, not proof of a passed journey. - The new helpers assume a source checkout with installed dependencies, Python, Node, and `lsof`. The onboarding helper also needs Playwright. - The local audit found observations that need separate investigation: an action test returned fixture data while the connection control showed Off and native profiles existed; a failed status-card setup task left the card showing Setting up. Workspace service start also did not reach a running state. This PR does not change product behavior or redefine those outcomes as success. - The main audit trace did not finalize during browser shutdown. Screenshots, text captures, and iteration results were retained. - Live evidence was captured before rebase. The onboarding helper re-test and full provider/auth qualification remain incomplete. ## Model Used OpenAI GPT-6 through Codex assisted with this change. The session does not expose the exact serving model ID, configured context-window size, or reasoning setting. Capabilities used: source analysis, read-only source subagents, file edits, shell commands, and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1012.0-canary.11 |
||
|
|
8227485372 |
perf(ci): cache the Runner's release crate artifacts keyed by their inputs (#15932)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Its trusted PR workflow builds the Runner's Rust binary in five
jobs: Typecheck + Release Registry, Build, Canary Dry Run, and the
`rust` and `vitest 2/2` lanes of Verify Paperclip Runner. The `Build`
job is the slowest check in almost every green run (461s, 489s and 515s
in runs 38180574768, 38182717052 and 38183149920 on 2026-10-11).
> - Each of those jobs restores master's `release-runner-v2` dependency
cache. That cache holds only third-party crates. `rust-cache` removes
workspace crates when it saves, and a fresh checkout invalidates cargo's
mtime fingerprints for workspace crates in any case. So cargo compiles
`paperclip-runner-core` in release mode from scratch on every run. The
job logs show one `Compiling` line and 28s to 49s for it.
> - 18 of the 239 commits on `master` in the seven days before this
change touched the crate's inputs. The other 221 compiled the same
sources to the same binary again.
> - This pull request adds a second cache that holds only that crate's
release artifacts (about 70 MB). The key is made from the compiler, the
checkout root and a hash of every input the build reads. The master Rust
lane writes it. A pull request whose merge tree has byte-identical
inputs restores it and marks the artifacts newer than the checkout, so
cargo keeps them. Every other pull request misses and compiles as
before.
> - The benefit is about 28s to 49s less on `Build`, which is the run's
critical path, and the same on Typecheck, Canary Dry Run and two Runner
lanes, for about nine of ten pull requests.
## Linked Issues or Issue Description
No GitHub issue exists for this. The description below follows the
enhancement template.
**What existing behavior does this improve?**
The time the trusted PR workflow spends to compile the Runner's own
crate on every run.
**Subsystem affected**
CI / build
**Current behavior**
Every PR job that builds the Runner binary compiles
`paperclip-runner-core` in release mode from scratch. The compile takes
28s to 49s on the `runs-on` fleet (one `Compiling paperclip-runner-core`
line, then `Finished release profile` in runs 38169436532 to 38183149920
on 2026-10-11). The shared Rust cache cannot help: it holds only
third-party crates, and a fresh checkout gives every source a new mtime,
which makes cargo discard the crate's fingerprints.
**Proposed behavior**
The master Rust lane in `release-verify.yml` saves the crate's release
artifacts under a key made from the compiler, the checkout root and a
hash of every input the release build reads. The PR jobs that build the
binary restore that entry read-only. On a hit, a script sets the mtime
of every restored artifact to now, so cargo sees every source as older
than the build and keeps the artifacts. `pnpm build` then prints
`Finished release profile` in under a second for that crate. On a miss,
cargo compiles the crate exactly as before.
**Reason and benefit**
About 28s to 49s off `Build`, the slowest check in almost every green
run, and the same off the Typecheck, Canary Dry Run and two Verify
Paperclip Runner lanes. The saving applies to every pull request whose
merge tree does not change the crate's inputs, which was 221 of 239
`master` commits in the week before this change.
**Breaking changes**
None. The checks are unchanged. A miss or an eviction costs the compile
time it costs today and nothing else.
**Additional context**
This is a workflow change. `pr.yml` calls `pr-trusted.yml@master`, so
this pull request's own run uses the current reader steps, and the
writer runs only on a master push. The saving becomes visible on the
first PR runs after the merge and after the next master push has written
the entry.
## What Changed
- `scripts/runner-release-crate-cache.mjs` (new): the one place that
defines the cached path list (`CACHED_PATHS`), the hashed inputs
(`HASHED_INPUTS`) and the files a restore must contain (`REQUIRED`). The
`scope` subcommand prints the platform, the `rustc --version --verbose`
release, commit and host, and a hash of the checkout root. The
`mark-current` subcommand sets the mtime of every restored artifact to
now and prints a `paperclip.runner-release-crate.v1` JSON line with the
file count and bytes. If the restored set lacks any of the seven files
cargo's freshness check reads or checks for (the fingerprint hash and
dep-info of each unit, the lib's rlib and rmeta, the binary under
`deps`), it removes the restored files, prints `action: discarded`
naming the missing file, and exits 0, so cargo compiles as on a miss.
- `.github/workflows/pr-trusted.yml`: after the `release-runner-v2`
restore in `typecheck_release_registry`, `build` and `canary_dry_run`,
and in the `rust` and `vitest 2/2` lanes of `verify_paperclip_runner`
(new matrix key `release_crate: true`), three steps: compute the key
scope, `actions/cache/restore` the crate under
`runner-release-crate-v1-<scope>-<hashFiles(...)>` with no
`restore-keys`, and run `mark-current` when `cache-hit == 'true'`. The
`hashFiles` list covers the toolchain pin, the manifests and lockfile,
the crate sources, the protocol tree and the three fixture trees the
test modules include (two under the Runner package and
`tests/runner-e2e/fixtures` at the repository root).
- `.github/workflows/release-verify.yml`: in the Rust lane of an
own-repository master push, after the debug warm-up: compute the key
scope, look the key up (`lookup-only: true`), and on a miss run `cargo
clean --release -p paperclip-runner-core` and `pnpm build:binary` (the
exact command the readers run, so the entry carries no test-profile
variants), then `actions/cache/save` the same path list under the same
key.
- `scripts/__tests__/runner-release-crate-cache.test.mjs` (new, 10
tests, in `test:release-registry`): the path list and hashed inputs are
the expected ones and exist; every `include_str!` and `include_bytes!`
in the crate sources resolves under a hashed input, whether its argument
is a literal or a `concat!` of literals, `env!("CARGO_MANIFEST_DIR")`
and macro metavariables, and any other form fails the test by name (this
guard found the `test/fixtures` tree during development and the
`tests/runner-e2e/fixtures` tree in review); the parser itself is tested
on a synthetic crate for each supported form and six rejected ones;
every PR reader restores after the dependency cache and marks before its
build command with the script's exact path list and key and no
`restore-keys`; the Verify lanes are gated by `release_crate`; the
writer runs after the checks with the lookup, the clean and build, and
the save under the readers' exact path list and key; the scope changes
with the compiler and the checkout root and survives a missing `rustc`;
`mark-current` makes every artifact newer than the checkout, leaves a
dependency's artifacts alone, discards a restore missing any one of the
seven required files (one case each, checking the named reason), and
still marks a restore missing any one of the seven files cargo does not
need.
## Review fixes
Greptile's two findings on the first commit were both valid and are
fixed in the second commit:
- **Incomplete caches looked usable.** `REQUIRED` checked four files,
but cargo's freshness check needs seven. Measured with cargo 1.97.1 on
the real crate (restore, touch the artifacts, delete one file, build):
without the lib's fingerprint hash, dep-info, rlib or rmeta it
recompiles the crate (24s to 27s); without the bin's fingerprint hash,
dep-info or the binary under `deps` it relinks (1.5s); without either
`.json` fingerprint record, either rustc `.d` file, the uplifted binary
or its `.d` it reports the crate fresh in 0.15s and recreates the
uplifted binary from `deps`, and the staging step then copies it.
`REQUIRED` now lists exactly the seven files, the glob for the binary
under `deps` excludes the `.d` beside it, and the test removes each of
the seven in turn (discarded, naming it) and each of the other seven in
turn (still marked).
- **Composed include paths escaped the guard.** The regex matched only
`include_str!("literal")` and skipped `include_str!(concat!(...))`,
which the crate uses four times (30 of 34 includes were checked). The
guard now parses every include argument and resolves the `concat!`
forms; one of them, in `provider_events.rs`, reads
`tests/runner-e2e/fixtures/native-completion/terminal-tool-carrier.json`
at the repository root, outside every hashed input. It is test-only, so
today's release binary does not read it, but the key is meant to cover
every include whatever its build, so `tests/runner-e2e/fixtures/**` (9
files, 80 KB) joins `HASHED_INPUTS` and the six key embeddings in the
two workflows. With the first commit's input list, the new guard fails
on exactly that include.
- `doc/RELEASE-AUTOMATION-SETUP.md`: a "Runner release crate cache"
section under the Runner verification dependency cache, with the key,
the writer, the readers and the trust boundary.
- `package.json`: the new test file in `test:release-registry`.
## Verification
- `node --test scripts/__tests__/runner-release-crate-cache.test.mjs`:
10 tests pass, three consecutive runs.
- The second commit, with the real toolchain on the real crate (`rustc
1.97.1`, cold build 2m20s, the crate's artifacts 74 MB): every source
touched to a fresh-checkout mtime and the archived artifacts restored,
cargo compiles (`Compiling paperclip-runner-core`, 24.8s and 25.1s, the
two controls); restored and marked by `mark-current`, `Finished release
profile in 0.16s` with no `Compiling`, and the same on a second build.
Then, for each of the fourteen files cargo wrote for the two units, one
at a time: restored, marked and that file deleted, cargo recompiles
without any of the seven `REQUIRED` files and reports fresh without any
of the other seven; restored with that file deleted and run through
`mark-current`, the seven required ones print `action: discarded` naming
the file and the others print `marked`. A restore holding only the seven
required files builds fresh in 0.15s, cargo recreates the uplifted
binary, and `stage-runner-binary.mjs` copies it into `dist/bin`.
- `node --test '.github/scripts/tests/*.test.mjs'`: 378 of 381 pass. The
3 failures are in `docker-canary-promotion.test.mjs` and fail the same
way on an unmodified `master` checkout in this sandbox (the stand-in
docker writes no `calls.jsonl`).
- `node --test scripts/__tests__/release-verify-workflow.test.mjs
scripts/__tests__/e2e-shard.test.mjs
scripts/__tests__/run-vitest-stable-shard.test.mjs`: 99 of 101 pass; the
2 failures spawn `pnpm`, which is not on this sandbox's PATH, and fail
the same way on `master`.
- Both workflows parse as YAML. No job has a duplicate step name or step
id.
- The mechanism, with the real toolchain (`rustc 1.97.1`, the package's
pin) on a copy of the crate: a cold `cargo build --release -p
paperclip-runner-core --bin paperclip-runnerd` (2m23s here). Then the
artifact set was archived with `tar`, removed, every source re-copied so
each had a new mtime, and the set restored:
- restored, not marked: `Compiling paperclip-runner-core`, 24.6s (the
control; this is what CI does today);
- restored and marked by `mark-current`: `Finished release profile in
0.15s`, no `Compiling`; a second build is the same;
- one Rust source edited after a marked restore: `Compiling`, 24.0s;
- one protocol schema edited after a marked restore: `Compiling`, 23.9s
(the protocol files enter cargo's dep-info as absolute paths and are
hashed into the key);
- the rlib removed from a restored set: `mark-current` prints `action:
discarded, reason: incomplete: the lib rlib is missing` and removes the
rest, and cargo compiles.
- Not run here: the workflow steps themselves. `pr.yml` pins
`pr-trusted.yml@master`, so this pull request's own `Build` job runs the
current reader steps, and the writer runs only on a master push. The
first PR runs after the merge and after the next master push show the
`Restore the Runner release crate` step's result and the
`paperclip.runner-release-crate.v1` line in their `Build` logs.
## Risks
- Correctness: a reader can only restore artifacts that master built
from inputs byte-identical to its own merge tree, because every input
the build reads is in the key, and the key has no prefix match. A pull
request cannot write. The trust boundary is unchanged: master writes, a
pull request restores master's entry.
- An input the build reads that is outside the hashed patterns would let
a stale artifact pass. The release dep-info lists the crate sources and
the protocol schemas and fixtures; all are hashed. The guard test
resolves every `include_str!` and `include_bytes!` in the crate, literal
or `concat!`, and fails if one lands outside the hashed inputs or uses a
form it cannot resolve, so a new include form cannot slip past it by not
matching a regex.
- GitHub-hosted runners check out under `/home/runner/work`, the
`runs-on` fleets under `/home/runner/_work`. Cargo records the included
protocol files as absolute paths, so an entry from one root is useless
under the other. The checkout root is in the key, so the other root
misses instead of restoring something cargo then discards.
- A cache that does not keep mtimes on restore changes nothing: the
artifacts are then already newer than the checkout, and `mark-current`
sets them to now in any case.
- Cost: one new entry of about 70 MB per distinct input set, written
only by the master Rust lane on a miss, plus one `cargo clean` and
`build:binary` (about 40s to 50s) on that lane when the inputs changed.
The readers pay one restore of about 70 MB (two to three seconds at the
fleet's observed restore speed) and a sub-second `mark-current`.
- An incomplete entry (a writer that saved a partial set) is discarded
by `mark-current` and costs one compile, the same as a miss.
## Model Used
- Claude (Anthropic), model id `claude-fable-5-1` (Claude Fable 5.1),
extended thinking, with tool use (shell, file editing, GitHub CLI, a
local Rust toolchain for the experiment) in a Paperclip agent session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
|
7df517d18f |
perf(build): reuse a current Runner build in the server's prepare:runner-vendor step (#15926)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its CI runs a `Build` job on every pull request. That job runs `pnpm build` and then verifies the public npm install. Since the server test shards got faster, `Build` is the slowest check in most runs (419-497s on the six fully green `master` runs created 2026-10-11 19:46Z to 20:17Z). > - Inside `pnpm build`, the server package is the last package to build. Its first step, `prepare:runner-vendor`, runs the full build of `@paperclipai/paperclip-runner` so the server can vendor `packages/paperclip-runner/dist`. In `pnpm -r build` that package was built seconds earlier by the same command. The second build took 12-16s on those six runs and sits on the critical path of the job. > - The step cannot just be removed. A standalone `pnpm --filter @paperclipai/server build` (the Dockerfile's path) must produce a current vendored Runner on its own. > - This pull request keeps that guarantee and skips only the build that is provably redundant. The Runner's `build` records a digest of its inputs, captured before its first step and confirmed after its last, together with a manifest of every file it left under `dist`. The server's `prepare:runner-vendor` reuses the build when the digest matches, the stamp is for this platform, and every recorded output is still there at its recorded size. Any other state runs the same full Runner build as before. > - The benefit is 12-16s off the `Build` job and off every other `pnpm build` (the Canary Dry Run's `release.sh` build, local builds), with no change to what a standalone server build produces. The PR's own first run measured it: the server step reused the build in 56ms where `master` runs rebuilt for 11-15s. ## Linked Issues or Issue Description No GitHub issue exists for this. The description below follows the enhancement template. **What existing behavior does this improve?** The server build's `prepare:runner-vendor` step, which builds `@paperclipai/paperclip-runner` so the server can vendor its `dist`. **Subsystem affected** Build and CI (`server/package.json`, `packages/paperclip-runner/package.json`, the `Build` job). **Current behavior** `prepare:runner-vendor` runs the Runner's full `build` every time. Inside `pnpm -r build`, the Runner was already built by the same command 20 to 25 seconds earlier, with every check passing. On six fully green `master` runs (2026-10-11, 19:46Z to 20:17Z) the repeated build took 11.8s to 15.9s, measured from the `server build$ pnpm run prepare:runner-vendor` log line to the Runner's `Finished release profile` line. The server package builds last, so that time is on the critical path of `Build`, which is the slowest check in most pull request runs. **Proposed behavior** - The Runner's `build` starts with `node scripts/record-build-inputs.mjs --begin` and ends with `node scripts/record-build-inputs.mjs --finish`. `--begin` removes the previous stamp and records a digest of every build input before the first step reads anything. `--finish`, after every step and check has passed, recomputes the digest and writes `packages/paperclip-runner/node_modules/.cache/paperclip-runner-build-inputs.json` only if the inputs are still the ones the build began with, together with the path and size of every file under `dist`. A stamp therefore exists only for a build that completed every step on inputs that did not move while it ran. - The server's `prepare:runner-vendor` becomes `node scripts/prepare-runner-vendor.mjs`. It reuses the build if the stamp is for this platform and architecture, every recorded output is still there at its recorded size, and the recomputed digest matches. In every other case it removes the stamp and runs `pnpm --filter @paperclipai/paperclip-runner build`, the exact command the step ran before. - `PAPERCLIP_RUNNER_VENDOR_REBUILD=1` forces the rebuild. **Reason and benefit** 12-16s off the `Build` job and off every other `pnpm build` (the Canary Dry Run's `release.sh` build, local builds), with no change to what a standalone `pnpm --filter @paperclipai/server build` produces. **Breaking changes** None. The rebuild path runs the same command as before. A fresh checkout, a changed input, a missing or truncated output, an edit that landed during the build, or a stamp from another platform always rebuilds. ## What Changed - `packages/paperclip-runner/scripts/build-inputs.mjs` (new): defines the input set, the digest, the output manifest and the reuse decision. Inputs are every file under `packages/paperclip-runner` except `dist`, `dist-browser`, `runner/target`, `node_modules` and dot-directories; every file under `packages/paperclip-eval-kernel` except `dist` and `node_modules` (the Runner's `build:typescript` builds it first); every file under the root `patches` directory (`check:acpx-profiles` hashes `acpx@0.13.1.patch`, and `build-verified-provider-entrypoints.mjs` copies the `pi-acp` and `brace-expansion` patches into `dist/cli/pi-provision-inputs`); and the root `package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml` and `tsconfig.base.json`. The toolchain pins are inside the package (`rust-toolchain.toml`, `runner/Cargo.lock`) and the TypeScript version is in the lockfile, so both are covered. A symlink is hashed as its target path and is not followed. Outputs are every file under `dist`, the tree the server copies wholesale, recorded with their sizes. Exports `beginRunnerBuild`, `recordRunnerBuild` and `assessRunnerBuild` (returns `current` plus a stable reason code: `no_stamp`, `schema_mismatch`, `platform_mismatch`, `missing_output`, `outputs_changed`, `inputs_changed`, `inputs_unchanged`). - `packages/paperclip-runner/scripts/record-build-inputs.mjs` (new): the two phases, `--begin` and `--finish`, added as the first and last step of the Runner's `build` in `packages/paperclip-runner/package.json`. Each build chain has its own pending record under `node_modules/.cache/paperclip-runner-build-inputs.pending/`, named after the shell pnpm runs the chain in (the parent of both phases; `--finish` also accepts a record named after its own pid, for a shell that execs the last command of the chain), so two Runner builds running at once in one checkout do not consume each other's record. `--finish` without a pending record for its chain is an error, so the chain cannot silently lose its first step; `--finish` after an edit that landed mid-build prints `not recorded: inputs changed while the build ran` and leaves no stamp. Records of builds that never finished are pruned by the next `--begin` after a day. - `server/scripts/prepare-runner-vendor.mjs` (new): the reuse-or-rebuild step. It resolves the pnpm that is running it through `npm_execpath` with the same rule as the CLI's `resolvePnpmInstallInvocation` (a JavaScript entry runs through this Node, a native executable runs directly, otherwise `pnpm` on PATH). It prints one `paperclip.runner-vendor.timing.v1` line with `action`, `reason`, `check_ms` and `build_ms`, so the saving is measured on this PR's own `Build` job. - `server/package.json`: `prepare:runner-vendor` now runs that script. The `build` and `typecheck` scripts are unchanged. - `server/src/__tests__/server-package-build-script.test.ts`: pins the new indirection and the command behind it (imported from the script, so the Dockerfile's standalone path stays pinned), and a new test asserts the Runner's `build` starts with `--begin`, ends with `--finish`, and has exactly those two record steps. - Tests: `packages/paperclip-runner/scripts/build-inputs.test.mjs` (node:test, added to `test:typescript:prep`) and `server/scripts/prepare-runner-vendor.test.mjs` (vitest, in the server lane). - `doc/DEVELOPING.md`: a "Reused Runner build in the server build" section under Test Commands. ## Review findings on the first head, all fixed in [ |
||
|
|
0ea8ebd040 |
fix(test): make the vitest home teardown remove read-only runtime-context bundles (#15931)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Its CI runs the server test suite as twelve `General tests (server
(n/12))` shards on every pull request. Each shard worker gets its own
throwaway `PAPERCLIP_HOME` from
`server/src/__tests__/setup-supertest.ts`, which removes it with a
recursive `rmSync` at worker exit.
> - That removal has been failing now and then with `EACCES, Permission
denied: /tmp/.../paperclip-vitest-home-...` after every test file in the
shard passed (four shard jobs on 2026-10-11 between 14:10Z and 22:10Z,
on four different branches, each attributed to
`native-session-resumption.test.ts` because that file was running in the
worker at exit). Vitest reports it as an unhandled error and fails the
shard, so the job has to be re-run, which costs a full shard (four to
six minutes) on the run's critical path.
> - The directory it cannot remove is one a native session leaves
behind: runtime-context bundles are published under the instance root
(`$PAPERCLIP_HOME/instances/<id>/runtime-context-assets/bundles/<digest>`)
and every directory in them is made read-only by design
(`makeDirectoriesReadOnly` in
`services/native-runtime/runtime-context.ts`). A file inside a read-only
directory cannot be unlinked, so the recursive removal stops with
`EACCES`. Whether such a bundle is under the worker's home at exit
depends on timing: suites set and restore `PAPERCLIP_HOME` for their own
fixtures, and a heartbeat that is still finishing when the worker exits
can publish its bundle under whichever home is current.
> - The teardown is the right place to fix this: the bundle is read-only
on purpose, and the test home is throwaway. On a permission error the
teardown now makes every directory in the tree writable and removes it
again.
## Linked Issues or Issue Description
No GitHub issue exists for this. The description below follows the bug
report template.
**What happened?**
A `General tests (server (n/12))` shard fails after all its test files
passed:
```
Vitest caught 1 unhandled error during the test run.
Error: EACCES, Permission denied: /tmp/pv-k2snZS/t/paperclip-vitest-home-qANo1F
❯ Object.rmSync node:fs:1282:18
❯ process.<anonymous> server/src/__tests__/setup-supertest.ts:32:31
This error originated in "server/src/__tests__/native-session-resumption.test.ts" test file.
Test Files 74 passed (74)
Tests 3226 passed | 1 skipped (3227)
Errors 1 error
```
Seen on 2026-10-11 in the first attempt of run 38146108750 (server
(11/12)), run 38150787810 (server (11/12)), run 38178832013 (server
(12/12), branch `fix/windows-opencode-utf8`) and run 38179110043 (server
(11/12), branch `fix/sentry-timeout-assertion-event-id`). The re-run of
the same job passes every time.
**Expected behavior**
A shard whose tests all passed is green. The per-worker test home is
removed at exit whatever a finishing run left in it.
**Steps to reproduce**
1. Create a directory tree with a read-only directory inside it, the way
`makeDirectoriesReadOnly` publishes a runtime-context bundle: `mkdir -p
home/bundle && touch home/bundle/SKILL.md && chmod 555 home/bundle`.
2. Remove it the way the setup file did: `node -e
"require('fs').rmSync('home', { recursive: true, force: true })"`.
3. The removal throws `EACCES` and leaves the tree behind. The new
`removeTestHome` removes it.
**Paperclip version or commit**
`master` at
|
||
|
|
a96329dfa2 |
fix(test): stop the Sentry timeout assertion matching a random event id (#15929)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `Sentry SDK contract` workflow, whose one job installs the real `@sentry/node` outside the workspace and runs `server/src/__tests__/run-failure-sentry-real-sdk.test.ts` to prove that run-failure reports never carry provider output or credentials. > - One of its assertions checks that the captured events do not contain the provider's stderr canary `<html>504 Gateway Time-out private-timeout-credential-canary</html>`, by matching the serialized events against `/private-timeout-|<html>|504|Timed out after/`. > - The bare `504` alternative also matches a Sentry event id. On pull request #15926 (a build-script change that does not touch this code) the job failed because that run's event id was `f353d184e5504ee39db611f2e60f49ff`; the re-run passed. About 0.7% of random 32-character hex ids contain `504`, and the test serializes two events, so roughly one job in seventy fails for no reason. > - This pull request narrows that alternative to `504 Gateway`, which still catches the canary with or without its HTML wrapper and cannot match a hex id. ## Linked Issues or Issue Description No GitHub issue exists for this. The description below follows the bug report template. **Pre-submission checklist** - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip (or can reproduce on `master`). - [x] I have confirmed the error originates in Paperclip itself — not in my agent adapter, API provider, or local configuration. **What happened?** The `Sentry SDK contract` job (`Real Sentry SDK isolation`) failed on pull request #15926 at head |
||
|
Devin FoleyandClaude Fable 5.1
|
f8798aac1e |
docs(release): align 2026.1006.0-beta.0 stable notes header with v2026.1012.0 (#16000)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100). --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The release pipeline promotes a soaked beta to stable. The stable job publishes `releases/beta/v<beta-version>.md` from `master` as the GitHub Release body, pinned at dispatch time. > - Beta `2026.1006.0-beta.0` (source `22a3ea3414e9039a538fbb0374b3cfa7fc4371eb`, 130 commits after v2026.1005.0) has soaked for more than three days. The stable version for 2026-10-12 is `v2026.1012.0`. > - The curated notes from #15388 were written at beta time. They guess `v2026.1009.0` and `2026-10-09`, which did not ship. > - A pre-dispatch audit confirmed the commit count, the PR cross-check, the migration range, the environment defaults, and the Contributors handles. Only the header, the date, the intro version, and the contributor count are stale. > - This pull request corrects those four lines before the stable dispatch, so the public release page shows the version that ships. > - The benefit is that the GitHub Release body and the canonical notes file agree with the npm version and the git tag. ## Linked Issues or Issue Description **Issue type** Docs: release notes. **Where is the issue?** `releases/beta/v2026.1006.0-beta.0.md` on `master`. **What's wrong?** The notes carry a guessed stable version and date from beta time: - The H1 reads `# Paperclip v2026.1009.0` and the next line reads `> Released: 2026-10-09`. No `v2026.1009.0` stable shipped. The stable that promotes this beta on 2026-10-12 resolves to `v2026.1012.0`. - The intro paragraph names `v2026.1009.0` for the same reason. - The Contributors line says 8 contributors. `git shortlog -sn v2026.1005.0..22a3ea3414e9039a538fbb0374b3cfa7fc4371eb` lists 10 authors. Excluding `github-actions[bot]` gives 9 (this count includes `dependabot[bot]`, as the v2026.916.0 notes did). Paperclip Cloud changes in the range (#15314, #15340) stay out of these notes. The notes do not cite them today, and this change does not add them. **Suggested fix** Change the version in the H1 and the intro paragraph to `v2026.1012.0`, change the Released line to `2026-10-12`, and change the contributor count to 9. Change nothing else. ## What Changed - Line 1: `# Paperclip v2026.1009.0` → `# Paperclip v2026.1012.0`. - Line 3: `> Released: 2026-10-09` → `> Released: 2026-10-12`. - Line 5: the intro paragraph now starts `Paperclip v2026.1012.0 carries 130 commits`. - Contributors: `from 8 contributors` → `from 9 contributors`. - No other lines changed. The 130-commit count, the cited PR numbers, the `0294`–`0305` migration range, the environment defaults, and the external handles were already correct. ## Verification - Docs only. `scripts/release.sh stable --print-version --date 2026-10-12` on `master` prints `2026.1012.0`, which matches the new header. - `git rev-list --count v2026.1005.0..22a3ea3414e9039a538fbb0374b3cfa7fc4371eb` is 130, which matches the notes. - All 89 PR numbers cited in the notes are in that range. The 40 uncited in-range PRs are 23 Dependabot bumps, one lockfile refresh, six test-only changes, five release-notes and contributor-docs changes, one CODEOWNERS change, two evals changes, one release-smoke test change, and two Paperclip Cloud changes (#15314, #15340). - Migrations `0294`–`0305` match `packages/db/src/migrations` in the range. `0298` is the only migration that updates existing rows. - Defaults were read at the beta source: `--listen-port` defaults to 43127 in `paperclip-runnerd.rs`; silent-run thresholds are 5 and 15 minutes in the sandbox runtime; remote Codex accepts `0.149.0` up to but excluding `0.161.0` in `codex-runtime-compatibility.ts`; pins are Codex `0.160.0`, Claude Agent SDK `0.3.286`, OpenCode `1.18.34`; Sonnet 5.5 needs Claude Code `2.1.284` or later. - `git shortlog -sn` over the range gives 9 contributors excluding `github-actions[bot]`. The external handles are `@baozdemir`, `@valentinpalkovic`, and `@abderbejaoui`, which the notes already list. - `git diff --stat` shows one file, 4 insertions, 4 deletions. ## Risks Low. Documentation only. This must merge before the stable dispatch, because the stable job pins the `master` revision of this file at dispatch time. ## Model Used Anthropic Claude Fable 5.1 (`claude-fable-5-1`) via Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (docs only; no tests apply) - [x] I have added or updated tests where applicable (none apply) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (running) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (running) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>canary/v2026.1012.0-canary.10 |
||
|
|
2145925f71 |
fix: use Hermes adapter API URL env in prompt (#9425)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Hermes adapter builds the command environment and renders the prompt that tells Hermes how to call back into Paperclip. > - Adapter configuration can provide `PAPERCLIP_API_URL` through `config.env`, which is often the most precise API base for the spawned worker. > - The current prompt rendering path does not read that adapter env value when deciding which Paperclip API base to show. > - This can leave Hermes with prompt instructions that point at a less appropriate process-level or public URL, even though the adapter env contains the intended API base. > - This pull request makes prompt rendering honor the adapter env value before falling back to broader process defaults. > - The benefit is a more consistent Hermes prompt without changing command execution behavior. ## Linked Issues or Issue Description Adapter: Hermes Agent or provider Hermes local adapter. Why this adapter is useful The Hermes adapter lets Paperclip run Hermes-backed agents while preserving Paperclip run context and callback instructions. How the agent is invoked Paperclip spawns the configured Hermes command and passes a rendered prompt plus adapter-provided environment variables. What happened? When `PAPERCLIP_API_URL` is supplied through adapter env, the Hermes subprocess receives it, but the prompt can still render its API base from a process-level fallback instead of that configured adapter env value. Expected behavior If an explicit `paperclipApiUrl` option is not configured, the prompt should prefer `config.env.PAPERCLIP_API_URL` when present, then fall back to the existing defaults. Related context: supersedes NousResearch/hermes-paperclip-adapter#118 for Paperclip's bundled Hermes adapter. ## What Changed - Added a small env-value resolver for raw string values and `{ value: string }` wrappers. - Builds a prompt-specific config that uses `config.env.PAPERCLIP_API_URL` when `paperclipApiUrl` is not explicitly set. - Added a focused command-resolution test that verifies the rendered prompt API base uses the adapter env URL instead of the process fallback. ## Verification - `pnpm --filter @paperclipai/hermes-paperclip-adapter test:command-resolution` - `pnpm --filter @paperclipai/hermes-paperclip-adapter typecheck` - `pnpm --filter @paperclipai/hermes-paperclip-adapter build` ## Risks Low risk. This only affects prompt rendering for Hermes adapter runs where `paperclipApiUrl` is unset and `config.env.PAPERCLIP_API_URL` is present. Explicit `paperclipApiUrl` values keep precedence, and existing fallback behavior remains in place when the env value is absent. ## Model Used OpenAI Codex, GPT-5.5, tool-enabled coding session with local shell, Git, and GitHub CLI validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: zocomputer <help@zocomputer.com> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
Devin Foley
|
4f399cec9f |
fix(docker): skip the full-tree ownership walk on routine boots (#15992)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Docker image keeps all instance state under `PAPERCLIP_HOME`,
and deployments mount a volume there
> - Before it drops privileges, the entrypoint makes sure that the
`node` user owns the home (#10530)
> - The ownership probe is a first-mismatch `find` over the whole tree.
When ownership is correct, `-quit` never fires, so every boot stats
every file on the volume
> - On homes with many agent worktrees and `node_modules` trees, this
walk takes minutes before Node starts. That is close to common 300s
healthcheck windows
> - This pull request keeps a cheap shallow probe on every boot. It runs
the full-tree probe only once per volume and node UID/GID, and records
the result in a marker file
> - The benefit is that routine boots no longer scale with the file
count, and the original root-owned-volume fix stays in place
## Linked Issues or Issue Description
Refs #10431 (the open follow-up about the remaining cost of the
ownership walk). Related: #10530 (the change that added the full-tree
probe).
**What happened?**
On a hosted multi-instance deployment, we measured the time from
container start to the first app log line. Instances with many files on
the volume (agent worktrees with `node_modules`) took about 190-235s.
Small instances took 5-22s. One 30 GB volume with few files took 24s, so
the cost follows the file count, not the volume size. Almost all of this
time is the `find "$home_dir" \( ! -user node -o ! -group node \) -print
-quit` walk in `scripts/docker-entrypoint.sh`. When ownership is already
correct, `-quit` never fires, so the walk stats every inode on every
boot.
**Expected behavior**
When ownership is already correct, the entrypoint pre-start phase is
short and does not depend on the number of files in `PAPERCLIP_HOME`. A
fresh root-owned volume, a UID/GID remap, and an upgrade from an older
image are still repaired before the privilege drop.
**Steps to reproduce**
1. Run the image with a volume on `/paperclip` that holds a few hundred
thousand files, all owned by `node:node`.
2. Restart the container.
3. Compare the container start time with the first server log line. The
gap grows with the file count, even though no `chown` runs.
**Paperclip version or commit**
`master` at
|
||
|
DottaandPaperclip
|
a2ac5c2715 |
refactor(chat): isolate Telegram, Slack, GitHub and Discord workflows (#15976)
## Thinking Path > - Paperclip manages AI agents and their work. > - Chat channels connect that work to external conversations. > - The shared chat service still contains provider-specific workflows. > - Those workflows are hard to inspect in a 34,000-line file. > - This stack extracts Telegram, Slack, GitHub and Discord workflows into their channel folders. > - Shared authorization and service lifetime remain explicit. ## Linked Issues or Issue Description Refs #15823, #15834, #15836, #15844 and #15845. Root of the second behavior-preserving channel-folder refactor stack. The dependent PRs #15979, #15981, #15985 and #15987 have merged into this branch; this PR lands all five extractions on master. ## What Changed - Move Telegram maintenance and subscription repair into `chat-channels/telegram/maintenance.ts`. - Move Slack session stopping and task starts into `chat-channels/slack/session-stop.ts` and `task-start.ts`. - Move GitHub webhook intake and recovery into `chat-channels/github/webhook-ingress.ts`. - Move Discord command registration and handling into `chat-channels/discord/commands.ts`. - Share type-only runtime records while preserving shared authorization, transactions, live lease checks and worker ownership. - Reduce the original service from 34,603 to 29,897 lines, removing 4,706 lines (13.6%). ## Verification - All 205 selected integration cases passed before and after their respective extractions: Telegram 41, Slack session stop 2 multi-step scenarios, GitHub 84, Slack task start 25 and Discord 53. Each dependent PR records its focused reproduction command. - Server typecheck, module boundaries and whitespace checks passed. - Per-slice syntax comparisons passed for moved declarations and remaining service statements, allowing only the documented live callback substitutions. - Tests exercise the public service with disposable databases and fixture provider transports. They do not claim live third-party execution. - Cumulative workspace typecheck and build passed on the final stack. - Current-head CI passed on the combined stack commit `d5b37dbe512e3e199932b6608cdc17639eda555a`; Greptile is 5/5 with no unresolved threads. - The [original serialized shard](https://github.com/paperclipai/paperclip/actions/runs/38266518817/job/114870022249) timed out in unchanged `adapter-auth-signal-routes.test.ts`. The exact case passed locally without changes, and the [single job retry](https://github.com/paperclipai/paperclip/actions/runs/38266518817/job/114874019933) passed. The original timeout remains recorded. Reproduce the original Telegram slice’s focused suite: ```sh pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts -t 'Telegram durable private draft Stop|Telegram maintenance|Telegram webhook|Telegram.*(pause|remove)|Telegram.*commands' ``` - Cumulative local `pnpm test:run` exited 1: the first server group had **18,770 passed, 105 skipped, and one timeout** in unchanged `server/src/services/workspace-git-snapshot-streaming.test.ts:56` (300,000 ms on macOS). The fail-fast runner did not reach its later workspace and serialized groups. An unchanged isolated rerun passed using the existing timeout; this does not turn the original full run green. The test and its Git utility code are outside this refactor; all current-head CI jobs passed. - Reproduce that diagnostic alone: `pnpm exec vitest run server/src/services/workspace-git-snapshot-streaming.test.ts`. ## Risks - Credential lease ownership and provider-confirmation ordering must remain unchanged. Existing rotation, lease-loss, recovery and durable Stop cases pass. - The channel factories create no workers or timers and preserve live runtime, shutdown and gateway-ownership reads. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
a9830414c7 |
Preserve repository dependencies when importing GitHub skills (#15731)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its Skills library imports reusable instructions from GitHub. > - Some skills depend on shared files and sibling skills outside their folders. > - Directory-only imports lose those dependencies. > - This pull request keeps repository files together and lets users select which skills appear in the library. > - Agents receive the original files and paths through the existing skill delivery system. ## Linked Issues or Issue Description Refs #14713. Related: #13656. **Current behavior** GitHub imports copy independent skill directories. Shared scripts, policies, helper skills, and credit files can be omitted. One shared-file error can block every skill. **Proposed behavior** New imports keep repository files together at the scanned commit. Show all skills with their folders and select every skill by default. Collapse individual package files. Omit unsupported or oversized files with a warning. Import available content without repair choices or an extra confirmation. Keep warnings on the saved source. **Reason and benefit** Skills can read their shared repository files after Paperclip delivers them to an agent. Original instructions stay unchanged. Users can inspect included files and understand omissions before and after import. ## What Changed - Store immutable company-scoped repository snapshots and associate them with skill versions. Regenerate the migration after merging master. - Preserve original bytes, binary encoding, and executable modes. Shared-file changes version each usable selected skill. Pinned versions retain their original files. - Deliver a discovery wrapper with the original tree under `.paperclip-repository/`. Preserve invocation policy when present. Fork and create-from-existing retain repository files. - Keep repository files together for new UI imports. Show every skill, select all by default, and collapse package files. Remove the packaging-mode menu. Keep the mobile import footer visible. - Warn about symlinks, submodules, oversized files, missing references, and invalid optional manifests. A malformed skill remains selected with a Needs setup warning. Usable skills still publish, and its original file remains repository support content. Source warnings persist through refresh. Unsafe execution and secret-disclosure findings omit the affected files before snapshot publication. - Expose saved repository files in the installed skill viewer, including binary encoding and executable flags. Reads use the immutable company-scoped snapshot; relative instruction links resolve from their original repository directory. - Audit delivered inventories and derive trust from all repository files. Accept evaluation prose while retaining dynamic-execution findings. Read blobs in one bounded Git batch and check budgets before reading support files. Imports never execute setup commands. - Add scanner, database, delivery, UI, and browser regressions. Update Storybooks, screenshots, and documentation. ## Verification - 69 focused scanner, Git snapshot, delivery, and PostgreSQL lifecycle tests passed. They cover partial imports, warning persistence, missing references, binary files, executable modes, copies, pinned versions, cancellation, company isolation, and batched Git reads. - 110 import, skill-detail, selection-tree, saved-file link, and Markdown renderer tests passed. A file-switch regression failed before the resolver fix and passed afterward. Browser walkthroughs cover the all-selected open tree, repository preview, non-blocking warnings, missing references, setup problems, warning persistence, and mobile layout. - Full workspace typecheck and build passed. Runner E2E TypeScript checks and design-token gates passed. The full serial local suite was attempted and stopped after over an hour. The existing 40,000-file Git streaming stress test failed after 303 seconds on macOS; the broad local suite is not counted as passed. All 55 applicable PR checks passed on the final commit, including the full sharded CI suite. Two optional Storybook workflows were skipped; the five local Storybook walkthroughs passed. - On October 12, live campaigns imported the private Poteto repository through the production GitHub Sources UI, selected architect and bro, and invoked them with native Codex on local and Daytona. The source connection was disabled before invocation. Independent filesystem inspections matched every supported Git blob, size, and executable mode. Both tasks completed and cleanup passed. Campaigns: `local-2026-10-12T15-07-39-504Z` (both environments), `local-2026-10-12T16-08-39-682Z` (local), and `local-2026-10-12T16-03-33-938Z` (Daytona). Each of the two selected skills received all 234 supported repository files. Later changes only affect the installed-file viewer and documentation. This qualifies skill delivery and instruction reads, not every Poteto workflow or adapter. - A manual source-checkout test drive imported a synthetic private GitHub repository with a symlink, an oversized file, a missing reference, and malformed skill metadata. Available skills published with their shared checklist. The walkthrough found duplicate warning text after import; the final UI fix removes that duplicate. The file viewer exposes saved repository dependencies and resolves authored relative links without changing the instructions. A follow-up live refresh omitted unsafe helper files with warnings, preserved every safe file byte, and survived a server restart. Screenshots below use the production UI with Storybook fixtures. They do not replace the live GitHub and runtime checks. <details> <summary>All skills visible and selected by default</summary>  </details> <details> <summary>Import stays available with warnings</summary>  </details> <details> <summary>Repository file inspector</summary>  </details> <details> <summary>Mobile import</summary>  </details> ## Risks - Each selected skill carries a repository copy in the runtime. The database stores each snapshot once. Runtime transfer and storage increase with the number of assigned skills. - Runtime budgets remain: 9,990 repository entries, 1 MiB per file, and 61 MiB total. File omissions are recorded as warnings. Source-access or download failures retain installed versions. - Import success does not prove that every skill can run. Environment dependencies, credentials, delegation tools, and schedules still need configuration. Imported setup scripts are not executed. - Live runtime coverage is native Codex on local and Daytona. Other adapters use the shared delivery path but were not exercised in these campaigns. The roadmap marks Skills Manager as shipped. This extends the existing importer. Related pull requests were checked. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code execution, automated tests, and browser tools. The exact deployment identifier and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting mergecanary/v2026.1012.0-canary.8 |
||
|
DottaandPaperclip
|
ce08872dfb |
refactor(heartbeat): extract instruction working-copy lifecycle (#15984)
## Thinking Path > - Paperclip runs agent tasks and preserves their managed agent files. > - Heartbeat must prepare a private copy, collect stopped-provider edits, and retain ownership during warm-session handoff. > - That lifecycle is spread across preparation, adapter callbacks, and cleanup. > - A per-run controller can own the copy and receipt while the executor retains teardown ordering. > - This PR moves that coordination into a dedicated module. > - Live getters preserve updated state and partially acquired resources. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of heartbeat instruction preparation. This is the second and final PR in this batch, originally stacked on #15980. That dependency is now merged; this PR is rebased onto its squash merge with an identical patch and identical touched-file contents. Related-PR search found no duplicate extraction. **Current behavior** After #15980, `heartbeat.ts` has 4,215 lines and 173,657 bytes. It coordinates instruction-copy preparation, save receipts, warm handoffs, and deferred cleanup inline. **Proposed behavior** `heartbeat/instruction-working-copy.ts` owns the per-run lifecycle in 233 lines. `heartbeat.ts` becomes 4,055 lines and 162,520 bytes. The executor retains the outer release order, unavailable-copy reporting, and environment/service cleanup policy. **Reason and benefit** The copy, save receipt, and warm ownership are readable together. The remaining executor names preparation and dispatch phases without obscuring teardown decisions. **Breaking changes** None intended. All 140 named public heartbeat exports remain unchanged. ## What Changed - Extract copy preparation, save/checkpoint callbacks, warm reservation/adoption, prompt guidance, and the deferred-release closure. - Read the executor's current run row through a callback; expose current copies, receipts, and acquired cleanup handles through live getters. - Keep cleanup handles available after preparation failure, and preserve release-before-retry on invalidated warm reuse. - Add four public-path characterization tests with real disposable PostgreSQL and files, plus four direct boundary tests. - Add isolated browser acceptance with a real process that edits instructions, completes its task, and reads the saved bytes on its next task. - Document the boundary and opt-in acceptance command. ## Verification - **Before extraction:** four new public-path tests pass on the original implementation, as do 108 existing instruction-copy/directory/native-cleanup tests. The two-task browser journey also passes on the baseline server. - **After extraction:** all 112 baseline tests pass; all four new boundary tests pass; 12 selected native-session ownership/retirement cases pass (the other 670 cases were outside this focused selection). - **Browser acceptance:** the fresh extracted-code server passes the full two-task journey (25.9 seconds): real child-process edits, saved/unchanged receipts, canonical bytes through the agent-file API, next-task restore, completed task UI, and copy cleanup. - `pnpm -r typecheck`, final server typecheck, and `pnpm build` pass. Module-boundary and public-export checks pass. - Normalized syntax comparison confirms all moved bodies and the reconstructed executor match the base, accounting explicitly for live run getters and delegation. - Final rebased head: `a08872c97e587a21ffc0fc6b78f34b81e484e7b0`. Combined entry-point and boundary regressions pass again (12/12); fresh Greptile review is 5/5 with no findings. All 55 current-head checks pass (two configured skips), including the hosted full-suite tests, build, typecheck, and all eight browser shards. There are no open review threads. ## Risks - Acquisition can succeed before preparation throws. Boundary tests verify that both the reservation and a transferred copy remain visible to the cleanup owner. - Callbacks can replace copy/save state during execution. Public tests verify saved receipts; direct tests verify the current run row, warm checkpoint receipt, and permission-root preservation. - Missing edit authorization and cleanup failure must remain nonfatal. Characterization tests revoke actual membership after dispatch admission and inject a cleanup error while preserving the saved result. - The browser uses a test-launcher-only instruction-capable wrapper around the real process adapter. Production adapter capabilities are unchanged. No live provider calls are made; native warm ownership is covered by focused module and native-session tests rather than a live provider browser run. ## Model Used OpenAI Codex, based on GPT-6, assisted with reasoning, code edits, tool execution, and verification. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1e0cfcb4fc |
test: retire redundant suites and consolidate owner coverage (#15978)
Retire duplicate ACPX and Codex proof after migrating unique assertions to their behavioral owners. Consolidate repeated fixtures and exact case matrices while preserving lifecycle, protocol, company, credential, approval, and ordering contracts. Remove the unused runtime-controls compatibility path and document test ownership and preservation evidence. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
94e4218310 |
refactor(heartbeat): extract runtime service coordination (#15980)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat runs start workspace services and receive service reports from adapters. > - Service context updates and readiness notices are spread through the executor. > - These steps share a clear responsibility, while process and lease ownership already live elsewhere. > - This pull request puts service coordination in one module. > - The executor retains the same dispatch and cleanup order. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of heartbeat execution. This is the first of two final focused extractions after #15973. Related #15653 changes runtime callback origins; this PR preserves that policy. **Current behavior** `heartbeat.ts` has 4,372 lines and 178,459 bytes. It validates service intents, starts services, persists adapter reports and primary URLs, and posts readiness notices inline. **Proposed behavior** `heartbeat/runtime-services.ts` owns those steps in 212 lines. `heartbeat.ts` becomes 4,215 lines and 173,657 bytes. Process registries and leases stay in `workspace-runtime.ts`. The executor still releases services in its original cleanup path. **Reason and benefit** Service coordination has named preparation and result phases. The execution path is easier to read without changing service lifetime or moving cleanup policy. **Breaking changes** None intended. All 140 named public heartbeat exports remain available, including `postWorkspaceReadyComment`. ## What Changed - Move service-intent validation, startup, adapter-result persistence, and readiness notices into one module. - Preserve mutable run context, primary URL selection, notice error handling, and release ordering. - Add four public-path tests using disposable PostgreSQL, real HTTP service processes, and a fixture adapter. - Extend browser recovery acceptance to fetch the actual service URL, check the readiness notice, and verify listener cleanup. - Update the module map and acceptance instructions. ## Verification - Before extraction: all five tests pass, including the four new characterization cases and existing notice formatting test. - After extraction: all 17 tests pass across service coordination, notice formatting, output recording, and runtime authorization. - Structural comparison confirms that all moved bodies and the reconstructed executor match the baseline. Module boundary checks pass. - `pnpm -r typecheck` and `pnpm build` pass. - Extended isolated browser acceptance: **2/2 pass** (3.7 minutes), covering actual service readiness, recovery, bounded failure, and cleanup. At `8e33e2032de681082f128470eb8ab0711b382afe`, all 55 current-head checks pass (two configured skips), including hosted full-suite tests. Greptile is 5/5 with no findings or review threads. ## Risks - Service context must be persisted before adapter dispatch and before run completion. The tests inspect both phases and preserve the first service URL when adapter reports arrive. - Notice failures must remain nonfatal. The public-path test checks both startup and result notices. - Shared services must survive run cleanup; ephemeral services must stop after success or failure. Tests use actual processes and HTTP requests. - The fixture supplies UUID service report IDs, as required by the database. The existing generated-ID fallback for reports without IDs is outside this extraction. - Test adapters and services are local fixtures; these tests make no live provider calls. ## Model Used OpenAI Codex, based on GPT-6, assisted with reasoning, code edits, tool execution, and verification. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
914b70572c |
refactor(heartbeat): extract execution configuration (#15973)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat turns task, agent, environment, and account settings into an executable run. > - The executor still assembles scoped credentials and skill configuration inline between environment selection and session preparation. > - That phase has a clear input/output boundary, but credential acquisition must remain visible to cleanup even when later setup fails. > - This pull request extracts configuration assembly and immediately notifies the executor of acquired resources and redaction state. > - The executor retains admission, dispatch, and cleanup, with the same statement ordering and explicit busy-account deferral. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of heartbeat execution. This PR follows #15969 and #15966, both now merged. The branch is rebased onto `master`; `git range-diff` confirms that the extraction patch is unchanged. **Current behavior** `heartbeat.ts` has 4,561 lines and 188,950 bytes. Configuration assembly mixes workspace overrides, secrets, managed AI preparation, and skill delivery into the executor. **Proposed behavior** `execution-configuration.ts` owns this preparation phase in 269 lines and 16,204 bytes. The executor calls it between environment and session preparation and retains immediate cleanup ownership. `heartbeat.ts` becomes 4,372 lines and 178,459 bytes. **Reason and benefit** The execution path exposes another named phase, with tests that exercise its effective configuration, resource lifetime, failure diagnostics, and busy-account result through public heartbeat dispatch. Related work: #11827 changes pre-dispatch secret refresh and #15355 changes GitHub delegation; this extraction preserves current policies. **Breaking changes** None intended. Public exports remain available, and the original helpers keep their modules. ## What Changed - Move configuration merging, scoped secret resolution, managed AI acquisition, runtime skills, connector instructions, and managed identity preparation into one typed module. - Notify the executor immediately when failure-redaction values, managed credentials, and identity redaction become available. Returning resources only after all preparation succeeds would lose cleanup on partial failure. - Preserve the live failure-secret getter for later runtime environment changes. Return an explicit deferred result for busy accounts. - Add four real-database characterization tests through `heartbeat.wakeup`, using actual credential files and cleanup with a fixture adapter. - Update the developer module map. ## Verification - Before extraction: all 45 tests pass across the new public-path tests and existing project-environment tests. - After extraction: 47 tests pass across execution configuration, project environment, and runtime skills. No skips. - Two additional real-process managed identity tests pass; nine unrelated cases are deliberately filtered out. - `pnpm -r typecheck` and `pnpm build` pass. - Normalized syntax comparison confirms the moved body and reconstructed executor match after accounting for immediate ownership callbacks and the explicit prepared/deferred result. - Isolated browser acceptance: 2/2 pass (3.9 minutes). A real process worker completes after Git recovery, with persisted stdout and matching final output byte counts. Persistent failures stop after three attempts. Both outcomes survive reload. The disposable server was shut down after verification. - All eleven new public-path characterization cases across the three-PR stack pass together on the final commit. - All 140 named public exports from the batch baseline remain available. - CI initially failed the legacy-continuation message browser case: periodic recovery escalated the issue, then queue admission cancelled the run with `execution_reconciliation_required` before configuration preparation. The unchanged predecessor shard passes, and the exact case passes locally (5 seconds in the browser). The original failing case passed on its single rerun (8.2 seconds), but GitHub then shut down the runner during a different test (`The runner has received a shutdown signal`). The separate infrastructure restart completed: 24 browser cases passed, with two existing optional cases skipped. Assertions and timeouts are unchanged. - Greptile: 5/5 on `27ab7e3df64c157ba1f00c5757db812ff7a4e5aa`, with no actionable findings. Hosted full-suite CI passes on this exact commit: all 54 check runs and the additional commit status are successful. - Merge preparation: all eleven characterization tests pass after rebasing onto `master`. Fresh full CI passes for `1758673f87eaa76a8b2c520eac4b631ada5d9ba1`: 55 successful checks and two optional Storybook skips, including all eight browser shards, typecheck, tests, and build. Greptile is 5/5 on this exact commit with zero review threads. All heartbeat source, tests, documentation, and browser assertions are identical to the previously reviewed final stack. ## Risks - A failure after AI acquisition must still remove the disposable credential home. The new public-path failure test proves actual filesystem cleanup and diagnostic redaction before the external reporting sink. - The baseline exposed an existing gap: arbitrary exception text in the local `run.error` field can contain a credential even though the failure reporter redacts it. This behavior is unchanged and merits a separate fix; the test does not claim the local error field is sanitized. - Account-busy deferral must return without dispatch while retaining its scheduled retry. The new test checks the durable run states. - Browser acceptance uses a real process fixture and disposable data, without live provider calls. ## Model Used OpenAI Codex, based on GPT-6, assisted with reasoning, code edits, tool execution, and verification. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
37913e4907 |
refactor(heartbeat): extract run output recording (#15969)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat runs stream adapter output and persist execution results. > - Output recording currently sits inside the executor beside admission and cleanup. > - Log handles, sequence numbers, progress writes, and excerpt buffers share one attempt lifetime. > - This pull request moves that state and its callbacks into a per-attempt recorder. > - The executor keeps dispatch and finalization visible, while finalization reads live output values. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of heartbeat execution. This PR follows #15966, which is now merged. The branch is rebased onto `master`; `git range-diff` confirms that its extraction patch is unchanged. **Current behavior** `heartbeat.ts` has 4,740 lines and 195,258 bytes. It directly owns log handles, excerpt buffers, output sequence numbers, progress flushes, and adapter event callbacks. **Proposed behavior** `run-output.ts` owns the per-attempt recorder. The main file has 4,561 lines and 188,950 bytes; the recorder has 209 lines and 7,757 bytes. **Reason and benefit** Output state has one explicit lifetime and interface. The shared log store and finalization policy retain their original owners. Related open output work includes #12539 and #15524; this PR does not change redaction policy. **Breaking changes** None intended. Public flush constants and log formatting exports remain available from `heartbeat.ts`. ## What Changed - Move log initialization, redaction, excerpts, progress persistence, live delivery, and adapter callbacks into `createHeartbeatRunOutput`. - Preserve the acquired log handle if later setup fails. Bind callbacks to the execution-start run row. - Pass live excerpt getters to success and failure completion so output arriving during finalization remains visible. - Add three real-database public-path tests. Extend browser acceptance to inspect persisted worker stdout and output byte counts. - Update the module map. ## Verification - Before extraction: 23 tests pass, including all three new output characterization cases. - After extraction: 45 tests pass across output, log formatting, event sequencing, and run completion. No skips. - `pnpm -r typecheck` and `pnpm build` pass. - Normalized syntax comparison confirms the moved bodies and reconstructed executor match. Reviewed the factory ownership, callback binding, getter wiring, and constant re-exports separately. - Isolated browser acceptance: 2/2 pass, including persisted worker stdout and matching final byte counts. The process worker completes after recovery; persistent timeouts stop after three attempts. Both terminal states survive reload. - Hosted full-suite CI passes on `d32c8fc40e72b9766bdd9e191d777e8dc554e3f2`; all 55 reported checks are successful. Greptile is 5/5 with zero review threads. - The initial chat shard timed out waiting for GitHub mention admission. Its fixture mocks heartbeat wakeup and never calls the output recorder. Both parameterized cases passed locally; the unchanged shard passed on one rerun. No test or timeout changes. - Merge preparation: all seven session-preparation and run-output regression tests pass after rebasing onto `master`. Fresh full CI passes for `60e2a49ce2879645a9f618a801e0ba0903ede161`: 55 successful checks and two optional Storybook skips, including all eight browser shards, typecheck, tests, and build. Greptile is 5/5 on this exact commit with zero review threads. ## Risks - A stale excerpt snapshot could lose output that arrives while finalization awaits I/O. Passing-before/passing-after tests exercise this for both successful and failed runs. - Per-attempt state must not become service-wide. Each executor constructs its own recorder and keeps the existing shared store. - Browser acceptance uses a real process fixture and disposable data; it makes no live provider calls. ## Model Used OpenAI Codex, based on GPT-6, assisted with reasoning, code edits, tool execution, and verification. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1012.0-canary.7 |
||
|
DottaandPaperclip
|
d497d1ccec |
refactor(chat): extract failed-run retry coordination (#15845)
## Thinking Path > - Paperclip manages agent work and sends results back to chat. > - A failed chat run can be retried only while its original source remains authorized. > - That decision includes persisted source evidence and native runtime state. > - Retry claims also share in-flight promises and depend on service shutdown. > - This PR groups retry authorization, execution and notices in one module. > - The service retains the live admission, runtime and shutdown boundaries. ## Linked Issues or Issue Description Refs #15844, #15836, #15834 and #15823. Stack slice 5 of 5, based on `codex/chat-publication-replacement`. Merge each predecessor first, then retarget this PR. Do not merge the stack out of order. Stack order: #15823 (provider messages) → #15834 (resource reconciliation) → #15836 (identity links) → #15844 (publication replacement) → #15845 (failed-run retries). GitHub native stack merging preserves this order and automatically restacks the remaining branches. ## What Changed - Extract failed-run retry admission and coordination while keeping the in-flight map, cursor, error class and authority registrations scoped to one chat service. - Move the exact Teams uncertain-file continuation proof into `microsoft-teams/file-transfer-source.ts`. - Pass the existing transaction, explicit source identifiers and denial factory. Return only the proven publication ID. - Keep committed-result, complete source-batch, ancestor and other-uncertain-effect checks in shared `failed-run-retry.ts`. - Keep native execution recovery together; its provider references describe runner backends, not chat channels. - The retry coordinator is 1,825 lines and the Teams proof is 149 lines. Restack on revised #15844 and record the reassessment. ## Verification - Baseline: 59 retry cases and six negative Teams authority cases passed. The combined 65 cases passed after the move. - Cases cover retry ownership, service restart/lifetime, ambiguous Teams receipts, sibling unknown effects, changed source authority and active attempts. - Teams guard, joins, locks, attempt checks and selected ID match after explicit input substitutions. The shared coordinator retains its state and cleanup ownership. - Server typecheck, module-boundary and whitespace checks passed. - Integration uses disposable databases and fixture provider transports; no live third-party execution is claimed. - Fresh Greptile review is 5/5 on `4f7151409717458717633aa270441a8faf33664e`, with a successful check and no open review threads. - Final-stack `pnpm -r typecheck` and `pnpm build` passed at `57ec5ba94aecd860471f9b14db46f99120c7c790`. - All CI checks passed on `57ec5ba94aecd860471f9b14db46f99120c7c790`. - Full local `pnpm test:run` stopped in the general-server group: 18,763 passed, 105 skipped, and one test timed out after 20 seconds in unchanged `workspace-runtime.test.ts` ("does not accept an occupied allocated port when listener ownership is unavailable"). The same case passed in an isolated rerun without code, assertion or timeout changes (1 passed, 175 excluded). The original timeout is retained as a failed full run; later workspace and serialized groups were not reached. The full local gate remains unverified. - Merge verification on the restacked tree: 118 focused identity, publication, Photon and retry integration tests passed; server typecheck and module-boundary checks passed. GitHub’s rebased trees exactly match the previously reviewed file contents. - All current-head required checks passed on `4f7151409717458717633aa270441a8faf33664e`: full CI; Greptile 5/5 with no unresolved threads. ## Risks - Authorization must remain fail-closed across committed output and retry admission. The helper proves only the Teams transfer; the caller still checks the complete source and result in the same transaction. - In-flight promises, error-class identity, registration cleanup, cursor state and the live shutdown getter must remain per-service. Their code is unchanged by this channel-folder move. - This PR depends on #15844. Merge and retarget in stack order. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5a72061ccc |
refactor(chat): extract publication replacement decisions (#15844)
## Thinking Path > - Paperclip publishes agent progress and results back to chat. > - Some updates replace an earlier message while others need a new message. > - The current outbound link and task generation determine that authority. > - These decisions are spread through the large chat service. > - This PR groups publication replacement and supersession decisions in one module. > - Reviewers can follow those decisions without reading provider transport code. ## Linked Issues or Issue Description Refs #15836, #15834 and #15823. Stack slice 4 of 5, based on `codex/chat-identity-links`. Merge the predecessor first, then retarget this PR. Do not merge the stack out of order. Stack order: #15823 (provider messages) → #15834 (resource reconciliation) → #15836 (identity links) → #15844 (publication replacement) → #15845 (failed-run retries). GitHub native stack merging preserves this order and automatically restacks the remaining branches. ## What Changed - Extract shared publication replacement and supersession queries from the large service. - Keep exact outbound ownership, run/wake scope, consumed progress, interaction replacement and retained-source authorization together. - Put the iMessage typing-versus-progress policy in `imessage-photon/publications.ts`. - Preserve endpoint lookup timing and all ownership query bodies. The multi-provider close allowlist remains shared dispatch metadata. - Restack onto the revised #15836 and record why this boundary remains mostly shared. ## Verification - The same 49 existing publication and Photon integration cases passed before and after the provider-policy move. Other cases were excluded by the filter. - Cases cover close-owned progress, ambiguous delivery, interleaved Slack control, Telegram confirmation, FIFO replacement and Photon channel behavior. - Server typecheck, module-boundary and whitespace checks passed. Only the iMessage predicate/reason moved; ownership queries are unchanged. - Historical server shard 2 hit a cleanup deadlock; its unchanged [isolated rerun passed](https://github.com/paperclipai/paperclip/actions/runs/38134431077/job/114457999409). [Original failure](https://github.com/paperclipai/paperclip/actions/runs/38134431077/job/114455332022). - Tests use disposable databases and fixture provider transports, not live third-party execution. - Fresh Greptile review is 5/5 on `8e2da97b6dc11eb446128b61c789edb529c50d5a`, with a successful check and no open review threads. - Final-stack `pnpm -r typecheck` and `pnpm build` passed at `57ec5ba94aecd860471f9b14db46f99120c7c790`. - Historical pre-restack CI failed on `ci / General tests (server (11/12))` and its aggregate `ci / verify`. All 1,255 tests passed, but worker-exit cleanup in unchanged `server/src/__tests__/setup-supertest.ts:32` failed with EACCES removing a temporary `paperclip-vitest-home-*` directory. The [original job](https://github.com/paperclipai/paperclip/actions/runs/38256549828/job/114836065382) and the [single unchanged rerun](https://github.com/paperclipai/paperclip/actions/runs/38256549828/job/114839735273) failed the same way. No further reruns were requested. After GitHub automatically restacked the branches, the full cumulative run on #15845 passed this server shard. This middle PR uses the repository’s policy-only stack CI scope; the top PR runs all suites. - The final full-local-suite outcome for this cumulative stack is recorded in #15845; do not infer that gate from the focused tests above. - Merge verification on the restacked tree: 118 focused identity, publication, Photon and retry integration tests passed; server typecheck and module-boundary checks passed. GitHub’s rebased trees exactly match the previously reviewed file contents. - All current-head required checks passed on `8e2da97b6dc11eb446128b61c789edb529c50d5a`: policy CI (middle-stack scope; full cumulative suites passed on #15845); Greptile 5/5 with no unresolved threads. ## Risks - Splitting shared ownership queries per channel would duplicate authorization rules. They remain in the shared module. - Preserve the issue-to-run lock order and retained-source callback. The factory creates no worker or mutable state. - This PR depends on #15836. Merge and retarget in stack order. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
de79750939 |
refactor(chat): extract identity linking (#15836)
## Thinking Path > - Paperclip connects provider identities to company members. > - A confirmed identity determines whose permissions apply to chat messages. > - Token consumption and membership checks must remain atomic. > - These operations currently share the large chat channel service. > - This PR moves identity linking into one module with explicit runtime callbacks. > - Reviewers can inspect the complete identity-link flow together. ## Linked Issues or Issue Description Refs #15834 and #15823. Stack slice 3 of 5, based on `codex/chat-resource-reconciliation`. Merge the predecessor first, then retarget this PR. Do not merge the stack out of order. Stack order: #15823 (provider messages) → #15834 (resource reconciliation) → #15836 (identity links) → #15844 (publication replacement) → #15845 (failed-run retries). GitHub native stack merging preserves this order and automatically restacks the remaining branches. ## What Changed - Extract company-scoped identity linking into shared `identity-links.ts`. - Move Slack connect receipt queries, access requests and confirmation-notice staging into `slack/identity-links.ts`. - Move GitHub identity fields into `github/identity-links.ts`. - Preserve shared token consumption, link locks, membership checks and revocation. - Stage the Slack notice inside the original confirmation transaction. Schedule delivery after commit. Keep runtime and public URL callbacks live. - Restack on the revised resource PR #15834 and record the architectural reassessment in the plan. ## Verification - Four existing service integration cases passed before and after. The Slack connect case covers discovery, preview, access requests, duplicate requests, member confirmation, provider notice, consumed tokens and endpoint isolation. - Receipt queries, the access-request transaction and notice payload match the previous implementation with explicit parameter substitutions. - Server typecheck, module-boundary and whitespace checks passed. - These are local database and fixture transport tests, not live Slack execution. - Fresh Greptile review is 5/5 on `7b44962f09fe8043c77ed40e05f761f31c64b33c`, with a successful check and no open review threads. - Final-stack `pnpm -r typecheck` and `pnpm build` passed at `57ec5ba94aecd860471f9b14db46f99120c7c790`. - All CI checks passed on `62ee045a4728da2d756be3d2e40d13e592a0b5a9`. - The [original workspace shard](https://github.com/paperclipai/paperclip/actions/runs/38256203686/job/114834873234) timed out waiting for the CLI import/export test server health endpoint. Its [single unchanged rerun passed](https://github.com/paperclipai/paperclip/actions/runs/38256203686/job/114838954081). The original timeout remains part of the evidence. - The final full-local-suite outcome for this cumulative stack is recorded in #15845; do not infer that gate from the focused tests above. - Merge verification on the restacked tree: 118 focused identity, publication, Photon and retry integration tests passed; server typecheck and module-boundary checks passed. GitHub’s rebased trees exactly match the previously reviewed file contents. - All current-head required checks passed on `7b44962f09fe8043c77ed40e05f761f31c64b33c`: full CI; Greptile 5/5 with no unresolved threads. ## Risks - Confirmation must retain transaction and token ownership. The Slack helper receives the existing transaction and stages the same effect; scheduling remains after commit in the shared coordinator. - This PR depends on #15834. Merge predecessors and retarget before merging this PR. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
848e659bc6 |
refactor(heartbeat): extract session preparation (#15966)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat execution prepares a provider session before dispatch. > - The same executor also owns workspace setup, admission, cleanup, and recovery. > - Session freshness and compaction form a separate responsibility with explicit inputs and results. > - This pull request moves those two session phases into one module and keeps their order around workspace preparation. > - The benefit is a shorter executor with unchanged session behavior. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of heartbeat execution. This continues the extraction series after #15872. **Current behavior** `heartbeat.ts` has 4,989 lines and 206,650 bytes. Session config fingerprints, resume selection, credential compatibility, and compaction are embedded in the executor. **Proposed behavior** The executor calls `prepareHeartbeatSession` before workspace preparation and `resolveHeartbeatAdapterSession` after it. `heartbeat.ts` has 4,740 lines and 195,258 bytes. The new cohesive module has 385 lines and 19,518 bytes. **Reason and benefit** A reviewer can find session policy in one module. Runtime acquisition, admission gates, and cleanup remain in the executor. Related open work includes #15787 and #15142; this PR does not change either policy. **Breaking changes** None intended. Existing public helper exports remain available. ## What Changed - Move 267 lines into `server/src/services/heartbeat/session-preparation.ts` with typed task, config, workspace, and session inputs. - Keep config fingerprints and context mutations on the same objects. Return compaction and adapter-session results to existing consumers. - Add four real-PostgreSQL characterization tests through heartbeat wakeup and adapter dispatch. - Document the two session preparation phases. ## Verification - Before extraction: four new characterization tests pass; 180 existing workspace/session tests pass. - After extraction: 205 tests pass across session preparation, workspace/session policy, managed-account session compatibility, and accepted-plan refresh. No skips. - `pnpm -r typecheck` and `pnpm build` pass. - Normalized syntax comparison: both moved blocks and the reconstructed executor, including public exports, match the base. - Isolated browser acceptance: 2/2 pass. A real process worker completes after automatic recovery; persistent timeouts stop after three attempts. Both terminal states survive reload. - All hosted CI gates pass on `d33f8506a7`, including the complete test shards, typecheck, build, and browser suites. The duplicate local `pnpm test:run` was interrupted after hosted CI passed; it is not claimed as a full local pass. - Greptile: 5/5 on this head, with no review threads. ## Risks - Session freshness can alter provider context if a dependency is omitted. Typed inputs and passing-before/passing-after dispatch tests cover unchanged resumes, model changes, tool refresh, and compaction. - The module acquires no runtime resource. Workspace preparation still occurs between the two phases. - Live external providers are not exercised; browser acceptance uses a real process fixture in a disposable app. ## Model Used OpenAI Codex, based on GPT-6, assisted with reasoning, code edits, tool execution, and verification. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3a213ab987 |
refactor(chat): extract resource reconciliation and availability (#15834)
## Thinking Path > - Paperclip manages AI agents and their work. > - Chat resources control which provider destinations can admit that work. > - Inventory updates and lifecycle callbacks must preserve company scope and operator choices. > - These rules share a large service with runtime and credential ownership. > - This PR extracts resource reconciliation and its availability transaction. > - Reviewers can inspect resource changes together while the service retains runtime control. ## Linked Issues or Issue Description Refs #15823 (merged). Slice 2 of 5 now targets `master`. Remaining order: #15834 → #15836 → #15844 → #15845. ## What Changed - Extract shared resource inventory, audited selection and availability settlement from the large service. - Put GitHub repository parsing, rename/transfer repair, webhook recovery and listing in `github/resources.ts`. - Put Telegram group migration in `telegram/resources.ts`, using the coordinator's existing transaction and timestamp. - Put Slack labels, Teams IDs and Photon participant parsing in their channel folders. - Keep lease assertions, delivery settlement and the resource upsert between Telegram migration steps in the shared coordinator. - The shared module is 598 lines; provider resource modules total 409 lines. Update the dated plan with this reassessment. ## Verification - On updated master, 15 existing resource integration cases and one Telegram migration case passed before this folder move. The same 16 cases passed after. Other cases were excluded by the filter. - Eight moved declarations and three transaction blocks match the prior implementation. The GitHub provider guard remains in the caller. - Server TypeScript, module-boundary and whitespace checks passed. Rebuilt plugin SDK types after rebasing onto current master. - Historical browser shard 8 timed out on inbox retry; its unchanged [rerun passed](https://github.com/paperclipai/paperclip/actions/runs/38134430238/job/114457529315). [Original failure](https://github.com/paperclipai/paperclip/actions/runs/38134430238/job/114455332214). - Tests use local disposable databases and fixture provider transports. No live third-party execution is claimed. - Fresh Greptile review is 5/5 on `e914f532ca66c429125492c1723df38afc6e13d7`, with a successful check and no open review threads. - Final-stack `pnpm -r typecheck` and `pnpm build` passed at `57ec5ba94aecd860471f9b14db46f99120c7c790`. - All CI checks passed on `e914f532ca66c429125492c1723df38afc6e13d7`. - The final full-local-suite outcome for this cumulative stack is recorded in #15845; do not infer that gate from the focused tests above. ## Risks - Resource identity and migrations depend on lock and write order. Provider modules use the original transaction; statement order and credential fencing remain unchanged. - Shared selection and availability still contain provider dispatch predicates. They coordinate common state transitions and should remain shared. - Full local tests and current-revision CI remain required for a green handoff. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fb21a3f30a |
refactor(chat): extract provider message normalization (#15823)
## Thinking Path > - Paperclip manages AI agents and keeps their work attached to company tasks. > - Chat providers deliver messages and corrections into those tasks. > - The chat channel service has grown to 39,155 lines. > - Provider message normalization has no database or worker ownership of its own. > - This pull request moves that responsibility into one module while preserving the service body. > - Reviewers can now find provider identities, timestamps, and edit revisions together. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of inbound chat message normalization in the server. **Subsystem affected** Server chat channels. **Current behavior** Provider message parsers and identity rules share a 39,155-line file with credentials, runtime ownership, database admission, and publication workers. **Proposed behavior** Move provider message normalization into `chat-channels/provider-messages.ts`. Keep service exports and runtime behavior unchanged. **Reason and benefit** This creates a clear first boundary for an incremental refactor. A dated plan ranks the next four larger extractions. No duplicate chat refactor was found in the GitHub title search. **Breaking changes** None intended. No schema, endpoint, transport, or public service API changes. Stack order: #15823 (provider messages) → #15834 (resource reconciliation) → #15836 (identity links) → #15844 (publication replacement) → #15845 (failed-run retries). Merge in this order; retarget each child after its predecessor lands. ## What Changed - Move provider message parsing into `chat-channels/telegram/messages.ts`, `slack/messages.ts`, `discord/messages.ts`, `microsoft-teams/messages.ts` and `github/messages.ts`. - Keep shared actor normalization, actor data and the inbound text limit in `chat-channels/message-lifecycle.ts`. These six modules total 566 lines / 19,534 bytes. - Import provider modules directly. Follow this provider-folder convention for future extractions. - Preserve exact actor IDs, edit hashes, timestamp provenance, and Telegram message identity rules. - Narrow two input types to the Teams parser method and the retained delivery fields actually read. - Reduce `chat-channels.ts` from 39,155 lines / 1,509,090 bytes to 38,643 lines / 1,491,703 bytes. - Record five ranked extraction candidates and verification evidence in the dated plan. ## Verification - The original extraction passed complete CI and 5/5 review at `cbba30a526`. Those results are historical; the updated head is `90ac517e6c310c8cf194dbbd668006f740e58f3a`. - Structural comparison: all 21 moved declarations match the base after normalizing exports and two type-only changes. All remaining non-import declarations, including the complete service body, match the base. - Provider-folder update: the same 48 public service integration cases passed locally before and after the move on PR #15823. They cover Slack/Discord revision identity, Teams edits, GitHub edits, Telegram rich messages and provider chronology. - All 22 message declarations retain identical bodies; the service changes imports only. The four dependent extractions rebased without conflicts. - Final-stack workspace `pnpm -r typecheck` passed. Module-boundary and whitespace checks passed. - Final-stack `pnpm build` passed. The fresh `pnpm test:run` was interrupted after PostgreSQL startup failures recurred. An isolated `native-status-arbiter-corpus.test.ts` run failed during database initialization after five attempts; all 45 tests skipped. All 32 host shared-memory slots were occupied. The full local gate remains unverified; only the task-owned run was stopped. - Greptile rated updated head `90ac517e6c310c8cf194dbbd668006f740e58f3a` 5/5 with no inline review threads. - Updated-head CI passed: 53 successful checks and two skipped checks, including all chat integration shards, typecheck, build and browser suites. - Tests use disposable databases and fixture provider transports. No live third-party execution is claimed. ## Risks - A wrong module import or changed revision identity could affect message admission. Existing service-path tests and the declaration comparison cover those risks. - Authentication, company-scoped transactions, leases, background workers, and shutdown remain in the original service. - The service closure remains large. Resource reconciliation is the recommended next extraction. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, local code execution, and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate planning or review agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1012.0-canary.6 |
||
|
DottaandPaperclip
|
ce15fe7c73 |
refactor(heartbeat): extract task context preparation (#15872)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat execution prepares task context before it selects an environment and starts an adapter. > - This preparation still occupied 319 lines inside the executor after the earlier extractions. > - It combines continuation summaries, comments, prompt variants, communication guidance, and redaction. > - This pull request moves that responsibility into one typed module under `services/heartbeat/`. > - The executor keeps admission, resource ownership, dispatch, and cleanup ordering together. > - The result makes the preparation phase easier to find and test without changing execution behavior. ## Linked Issues or Issue Description Refs #15825, #15807. This continues the incremental heartbeat extraction series. A search found no duplicate task-context extraction. **What existing behavior does this improve?** Maintenance and review of heartbeat task context preparation. **Current behavior** The executor assembles prompts and trust-filtered context inline. This hides its execution phases among preparation details. **Proposed behavior** Call `prepareHeartbeatTaskContext` at the same point in execution. Preserve statement order, prompt contents, public exports, and the shared run context object. ## What Changed - Move task context assembly to `server/src/services/heartbeat/task-context.ts`. Inputs group task state, policy, and two existing service methods. Results preserve the continuation and wake values used by later runtime selection and dispatch. - Keep company-scoped loading, quarantined-content filtering, secret redaction, and revision hashing in their original order. Add no execution resource acquisition. - Add four public-path integration tests before the move. Cover prompt variants and redacted revisions, quarantined summaries/comments, a mixed-company wake batch, and stale caller fields. - Extend the real process worker in the opt-in browser suite to check persisted prompt variants and the description revision before it completes its task. - Update the module map and acceptance-test documentation. - Reduce `heartbeat.ts` from 5,314 lines / 220,784 bytes to 4,989 lines / 206,650 bytes. The new module has 376 lines / 15,912 bytes. ## Verification - Before the move: 11 existing prompt tests and all four new characterization tests passed against the inline implementation. - Normalized syntax comparison passed for all 319 moved lines and the remaining orchestrator, including public exports. Imports and the typed boundary were reviewed separately. - `pnpm -r typecheck` passed. - `pnpm build` passed. - The clean post-move focused run passed all 26 tests in five files, with no skips: task-context characterization, task-markdown, Slack invitation, accepted-plan workspace refresh, and run-preparation. - The local `pnpm test:run` attempt was interrupted after database failures while the Mac exhausted its shared-memory slots. All 15 affected native warm-ownership database cases passed in an isolated rerun after the browser instance stopped; the other 666 cases in that file were excluded by the test-name filter. The interrupted command is not a full-suite pass. Full CI passed on `6bf86cfdc07e7fe34015f7b719218639f4232510`: 55 successful checks and two optional Storybook checks skipped. Greptile reviewed the latest commit at 5/5; its one earlier worker-test finding is fixed and resolved. - Browser E2E passed both scenarios on the final code: persistent timeouts stop after three runs and survive reload; one timeout recovers without manual Retry, verifies task-context prompt variants/revision and preserved workspace files in a real process, then completes through the run-scoped API. The first recovery attempt exposed an incorrect new test assertion because the fixture title equals its description. The worker now checks the description section specifically; the corrected recovery scenario passed in 55.9 seconds. The original failure trace was preserved. These tests use a deterministic process adapter and make no live model calls. ## Risks This is a behavior-preserving extraction. The main risks are changed closure bindings, missing return values, or changed preparation order. Typed inputs, public-path tests, and syntax comparison cover those risks. The run context is still the same mutable object. The helper acquires no leases or processes. It introduces no schema, API, or prompt policy changes. The first post-move database run skipped cases due to host resource exhaustion. All 26 focused cases passed on the subsequent clean run. Browser E2E passed after the worker assertion fix. The full local suite remains incomplete. All 15 native warm-ownership database cases passed on their isolated rerun; The full CI test suite passed on the final commit. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code execution, and test tools. The host does not expose the exact deployment model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
762d680daf |
feat(ui): upload files directly in task artifacts (#15964)
## Thinking Path > - Paperclip helps people manage agent work and retain its context. > - Task documents and attachments keep that context with the task. > - The Artifacts panel showed output but could not accept a new file. > - Uploading through a message can require resuming a paused task. > - Existing document and attachment APIs can save files directly. > - This PR adds direct uploads and a drop target to the task panel. > - People can add context while keeping work paused and unassigned. ## Linked Issues or Issue Description **What existing behavior does this improve?** The task Artifacts panel and its existing document and attachment APIs. **Subsystem affected** ui/ — React board UI. No server, schema, or API changes. **Current behavior** The Artifacts panel cannot upload a file. It also excludes files uploaded by people. The message composer can require a task to resume before it submits attachments. **Proposed behavior** Use the upload icon or drop files anywhere in the open Artifacts panel. The target includes the header, padding, and existing cards. Files save directly to the task without a message or a pause change. **Reason and benefit** People can prepare a handoff on a paused, unassigned task without starting agent work. Markdown becomes a task document. Other supported files use attachment storage. **Breaking changes** Files from people now appear in Artifacts alongside agent output. Promoted attachments still appear once. A repeated Markdown filename creates a separate document instead of replacing an existing revision. Existing APIs and limits stay the same. **Additional context** Related upload API work: Refs #7472 and Refs #7248. This PR adds the missing task-panel entry point. ROADMAP.md lists Artifacts and Work Products as available; this is a scoped UI improvement. ## What Changed - Add an upload icon with no visible label or background. - Accept file drops across the full task panel, including the desktop portal header and mobile tabs. - Show per-file progress and inline Retry and Dismiss actions. - Import Markdown under a new document key and retain existing revisions. - Include files from people in Artifacts and dedupe promoted attachments. - Add component and task-journey stories for empty, populated, loading, error, mobile, and drag-over states. - Add real browser tests for persistence, pause preservation, duplicate document names, full-panel drops, and partial failure recovery. ## Verification - Five Playwright tests passed against an isolated instance started by the source `test-drive` command. The document API, attachment storage, pause holds, downloads, and task UI were real. Only the explicit upload failure was injected. - The browser tests verified desktop and mobile uploads, document reading, persistence after reload, no task comments or runs, header drops, preservation of original and hidden internal documents, reserved filename handling, and upload retry. - 163 focused UI tests passed. - `pnpm -r typecheck` passed. - `pnpm build` passed. - `pnpm check:token-gates` passed. - The full Vitest suite is running after repair of an exhausted local PostgreSQL shared-memory pool. The initial run failed to initialize PostgreSQL. Six unattached segments had no living creator or last-user process; removing them restored free slots. - CI and the current-head Greptile review are pending. ## Risks - Artifacts now includes user files, so existing tasks can show more cards. - Uploads retain the server's authorization, file-type, size, and revision checks. No migration is needed. - Binary attachment uploads use the existing API. A transport failure with an unknown save outcome can retain the existing API's retry ambiguity. ## Model Used OpenAI GPT-6 through Codex. The exact runtime snapshot ID and context-window size were not exposed to this session. The model used repository inspection, code editing, shell tools, and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
88b7bbb5e2 |
Stabilize task lifecycle browser fixtures (#15878)
## Thinking Path > - Paperclip manages agents and their tasks. > - Browser and runner tests check task recovery and user actions. > - Some fixtures act before a prior request or run has settled. > - Those races can report a product failure before the intended assertion runs. > - This PR binds each check to the exact request, run, or geometry snapshot it needs. > - The tests retain their existing budgets and negative assertions. ## Linked Issues or Issue Description Refs: #15813, #15804. The draft stack is master → [fixture prerequisite #15878](https://github.com/paperclipai/paperclip/pull/15878) → [runtime prerequisite #15880](https://github.com/paperclipai/paperclip/pull/15880) → [Boat core #15813](https://github.com/paperclipai/paperclip/pull/15813) → [UI #15804](https://github.com/paperclipai/paperclip/pull/15804). GitHub confirms 5, 63, 97, and 75 changed files respectively, each below the 100-file limit. Existing commits and PR URLs remain intact. [Historical evidence](https://github.com/paperclipai/paperclip/tree/5909bfac9fa41405beb3fe8a65253dc28ba52835/doc/plans/assets/2026-10-10-boat) remains pinned to its original commit; current proof files stay with the UI PR. This is the test-only prerequisite extracted from the experimental Boat work. It contains five existing test-file changes. It does not add the Boat adapter or change task admission. ## What Changed - Wait for the exact signoff fixture run and each replacement run to succeed before applying the next stage decision. - Wait for the inbox Retry response for the exact failed run before navigating away. - Compare task planning time with the first run that actually started, instead of assuming the first list item started. - Read the touch target, container, and scroll position in one browser evaluation. Keep strict containment and native gesture checks. - Bind the native recovery fixture to its disposable fake Codex executable so installed provider binaries cannot override it. ## Verification Draft branch: `codex/boat-fixture-prerequisite`, commit `e318850909c0777565b43644a4d8114c6006da60`. Base: master `e733bddc45e00efb1819fffa8677f7b0d07802c5`. Five changed files. The isolated fixture-plus-runtime stack passed 24 browser cases. Six legacy recovery cases could not reach their assertions because the fixture requires embedded PostgreSQL metadata. A normal embedded PostgreSQL attempt then failed during cluster setup on this host. These are not six passing cases. No test seam, timeout increase, or product-database fixture was used. The fixture-only commit has not yet had a separate complete local browser run. All six legacy cases subsequently passed on this exact fixture commit in PR CI browser shard 8 (20/20 total, job114470999277). The original local setup failures remain recorded. The full-stack typecheck/build results belong to the assembled fixture-plus-runtime tree and are not claimed as independent qualification of this smaller commit. Review the exact run/request filters and the unchanged rejection assertions. Run the four changed Playwright specs and the native-session-resume suite through the repository's normal fixture setup. At 13:03:24 UTC, this exact head had 53 successful checks and two skips, with no pending checks or failures. All eight browser shards passed. Greptile is 5/5 on the exact head with no unresolved threads. [Exact-head legacy browser proof](https://github.com/paperclipai/paperclip/blob/9eacca2c5321bb63f061eef4bb775b6951d5846d/doc/plans/assets/2026-10-10-boat/evidence/prerequisite-legacy-browser-ci-proof.json). Feature acceptance stays in the downstream report. Deployment29 verified `05f3725af838b5a074707df1d6ed1e193bdfe49e` at 12:23:23.944 UTC. Native Claude cold execution passed at 12:39:57.782 after 14m56s of first-use preparation. Warm continuation passed at 12:42:09.943 with the same owner, boot and session identities. Normal retirement and archive followed. The ordinary saved post-idle turn then failed: its primary attempt reported the closed `LINK_COUNT` module fence at `turn_ownership_verified` (3,711ms), and same-run recovery settled failed at 12:52:09 with identity/authority errors. Natural archive completed at 12:52:21. Numeric link counts and their direction are not yet proven. A narrower failure-only diagnostic and a bounded read-only probe are under review; neither establishes a fix for this live failure. Prior staging27/28 failures remain preserved. Local complete lifecycle passes and staging Codex/computer/HMR passes remain separate source-bound evidence. No session, quarantine, provider stop or migration history was reset. These live observations do not qualify this prerequisite by themselves. ## Risks - Waiting for the wrong run could hide a task race. The fixtures match task, agent, and replacement-run identity before proceeding. - A geometry helper could weaken touch coverage. This change preserves containment and gesture assertions and only makes their inputs one snapshot. - The six legacy browser cases passed in exact-head CI; their local PostgreSQL setup limitation remains recorded. No live Boat acceptance result follows from this test-only change. Documentation impact: no product command or behavior changes. This description records fixture scope and validation limits. ## Model Used OpenAI Codex (GPT-6 family) assisted with code, test execution, and review. The host does not expose the exact primary model ID or context window. The work used tools for source inspection and local execution. Prior architecture work used `gpt-6-astra`; an independent design review used `gpt-6.1-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal issue id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
github-actions[bot]andlockfile-bot
|
b189f586d2 |
chore(lockfile): refresh pnpm-lock.yaml (#15960)
Auto-generated lockfile refresh after dependencies changed on master. This PR only updates pnpm-lock.yaml. Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
db2f00947b |
fix(runner): include Claude ACP in the server package (#15912)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server ships a copy of Paperclip Runner for native agent execution. > - The Claude ACP bridge is a runner dependency, but it was absent from the server package. > - The runner resolves this bridge dynamically, so the import-based packaging check missed it. > - A Claude session then failed before startup and showed a generic connection error. > - This pull request ships the qualified bridge and reports a missing runtime with repair steps. ## Linked Issues or Issue Description **What happened?** A server install with the vendored runner could not load `@agentclientprotocol/claude-agent-acp/package.json`. The runtime test suggested checking the model or provider connection. The task reported an interrupted runner session. **Expected behavior** The server package includes its qualified Claude ACP runtime. A missing bundled runtime reports an installation error and explains how to repair it. **Steps to reproduce** 1. Use a server install with the vendored runner and no Claude ACP bridge. 2. Select the Paperclip Runner ACP harness and the Claude provider. 3. Run the runtime test or open a session. 4. Observe a missing-module failure before Claude starts. **Paperclip version or commit** Reproduced on `e733bddc45`. This branch is based on current master. **Deployment mode** Self-hosted server built from source. Related: #15224 upgrades the Claude bridge. This PR fixes its omission from the server package and keeps the current qualified version. A future bridge upgrade must update the server pin too. ## What Changed - Add Claude ACP `0.73.0` to the server dependencies and bundled dependencies. - Pin the qualified Claude SDK and bridge dependencies in the npm bundle producer graph. Publish its native packages as consumer optional dependencies instead of shipping the producer executable. - Bind source-built vendored runners to the shared pnpm dependency store while retaining the server manifest as the provider authority. Keep published npm and self-rooted deployment roots unchanged. - Include dynamically resolved Claude and Codex bridges in the vendor dependency check. - Preserve the missing-package category through the sidecar, Rust transport, and native session executor. Stop automatic retries for this installation failure. - Show **Runtime missing** in the runtime test and task thread. Explain the reinstall or rebuild, server restart, and runtime test steps. - Add regression tests and document the repair procedure. ## Verification - Red/green comparison with the production sidecar: the same missing-package setup returns a raw require-stack error on the original code and an actionable, non-retryable missing-runtime error on this branch. - The rebuilt server's vendored Claude installation probe passes using the source-server authority selected by the launcher, without starting a provider or making a model request. - Focused server verification: 830 tests passed. Runner installation and sidecar error tests: 83 passed, with 6 platform-specific skips. Packaging tests: 28 passed. Rust transport tests: 7 passed. - Source/deployment authority tests: 3 passed. Red/green packaging tests include a Linux-produced Claude SDK bundle, actual npm pack, and closed executable admission on Linux x64 and macOS ARM64/x64. - Full UI suite: 8,113 tests passed. - `pnpm -r typecheck` and `pnpm build` passed on the rebased branch. - Full current-head CI passed, including all general and serialized test lanes, Runner Rust/Vitest verification, typecheck, build, and all eight e2e shards. Greptile reviewed `d6a2741834ff922956560f2f689382827bbaf9e0` at 5/5 with no unresolved threads. - Local full-suite limitation: an earlier parallel attempt hit Postgres startup failures and timeouts; the affected server tests and a database fixture passed in isolation. The subsequent serial `pnpm test:run` was stopped while still running after complete CI passed. Focused local tests, full UI tests, typecheck, and build completed as listed above. - Token gates and `git diff --check` passed. ## Risks - Bundling the Claude bridge increases the server package size. The producer graph retains the qualified SDK version and existing isolation patch. Native platform declarations move to the published server manifest, while the runtime keeps its exact version, declared-slot, no-follow, and executable digest checks. - The new failure category is permanent. Operators must repair the server installation before they retry the task. - Existing failed runs retain their recorded error. The clearer message applies to new runtime tests and runs. - No database schema or migration changes. ## Model Used OpenAI GPT-6 through Codex. The system reports the model family; the exact deployment ID and context-window size are not exposed in this session. Used reasoning, browser inspection, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1012.0-canary.5 |
||
|
|
1b4c803816 |
fix(costs): preserve usage attribution and explain unavailable run costs (#15948)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The cost ledger records usage and controls agent budgets. > - Some runs lose the selected model or API billing category before their usage reaches the ledger. > - The dashboard reports missing costs but does not identify the affected runs or the available actions. > - This pull request preserves usage attribution and adds a compact view of cost gaps grouped by agent. > - Operators can inspect runs, open agent settings, and record saved costs without starting new agent work. ## Linked Issues or Issue Description Refs #15634, #11938, #15879. This follows the merged subscription reporting in #15634. The older proposal in #11938 addresses part of ACP model attribution. This change also rejects conflicting models across a turn, fixes credential-backed billing capture, and explains cost gaps in the existing dashboard. The Pi receipt recovery work in #15879 covers a separate adapter recovery path. This PR preserves unresolved receipts and does not recover provider usage or reprice past entries. **What happened?** ACP results use the requested model even when the runtime selects a model. Native and OpenCode API runs can report unknown billing despite a matching API credential. The dashboard shows notices without links to the affected runs. These gaps make API and subscription totals harder to interpret. **Expected behavior** Record the model selected by the runtime. Identify API billing only from a matching credential. Keep costs unresolved when reliable usage or pricing is absent. Show the affected agents and runs. Offer an accounting retry only when saved evidence is ready. **Steps to reproduce** 1. Run an ACP agent with its model left unset, or use a matching API credential with a native or OpenCode agent. 2. Inspect the recorded model and billing category for the run. 3. Open Costs Overview for a period with unpriced usage or terminal runs awaiting receipts. 4. Inspect the notice and try to find the affected run or the relevant agent settings. **Paperclip version or commit** Current master at `657e71584a0`. The patch is rebased on that commit. **Deployment mode** Self-hosted server and local development. The cost detail tests use a disposable PostgreSQL database. ## What Changed - Preserve the runtime-selected ACP model in checkpoints and final results. Conflicting models across one turn remain unpriced. - Share credential-backed API billing checks across native and OpenCode results. Keep unknown billing when the provider and credential do not match. - Add company-scoped cost detail and accounting retry endpoints. Board viewers can read details but cannot retry. Agents and foreign companies are denied. - Limit each detail category to 25 entries. Use one consistent database snapshot and sparse indexes matching the filters and sort order. Add the unknown-billing count to the existing summary query. - Replace the notice with an expandable count line. Group entries alphabetically by agent. Show the saved blob icon, link each name to configuration, and put run and retry links beside the date. - Preserve estimate and unavailable-price labels on Providers and Billers; keep retry confirmation after the resolved notice disappears. - Fetch details only while expanded. Retain the last loaded details after a refresh error. Retry saved accounting only; do not rerun the agent. - Add capture, database, authorization, UI, and Storybook coverage. Update the implementation contract and cost reporting guide. ## Verification - `pnpm -r typecheck` passed after rebase. - `pnpm build` and `pnpm build-storybook` passed after rebase. - `pnpm check:token-gates` and `git diff --check` passed. - 1,086 focused tests passed across 17 files. These cover ACP, native and OpenCode capture, PostgreSQL reports, route authorization, OpenAPI, Costs UI, and existing accounting regressions. - Desktop and mobile browser checks passed during UI review. They cover lazy expansion, alphabetical groups, loaded blob icons, configuration and run links, keyboard retry, and layout overflow. - `pnpm test:run` was attempted with command-scoped `init.defaultBranch=master` for Git fixtures. It was stopped after a native session recovery failure reproduced on unchanged base `657e71584a0` (`archives a damaged prior epoch before completing guarded same-task replacement with real runnerd`). An MCP upload case failed in the broad run but passed in isolation on both the base and PR branch. These local limitations do not affect the 1,086 passing focused tests. - Current-head CI is green, including all unit test lanes, serialized server suites, runner checks, typecheck, build, policy, and eight browser shards. Apex re-review is 5/5 on `56de60effdb`; all four original findings are addressed and every review thread is resolved. - PostgreSQL applies migration `0333` in the integration fixtures. EXPLAIN ANALYZE on the production detail statements uses the new sparse indexes behind 5,000 healthy records in each table. - Ten provider/biller card cases cover missing and estimated prices; the retry test covers row removal and notice unmount. A live Storybook browser check verifies the confirmation and updated count. - Run `pnpm storybook`. Open **Costs / Subscriptions / Cost Data Quality Details** to inspect the production page with representative fixtures. ## Risks - Model and billing metadata change for future runs. Estimates still require complete usage and a supported price. No past cost amount is rewritten. - A missing final receipt remains unresolved for budget checks. The display changes after fifteen minutes; that time limit never authorizes settlement. - Accounting retry is a write action. It retains the existing company checks, receipt fences, idempotency, and audit logging. - Detail reads are bounded and run only after expansion. They poll every 30 seconds while open. No provider probe or new summary query is added. - Migration `0333` adds two partial indexes; it changes no stored cost amounts or table columns. Index creation scans the existing tables and may briefly block writes during deployment. No lockfile or workflow change is included. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code editing, browser checks, and test execution. The exact serving model ID, reasoning setting, and context window were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1012.0-canary.4 nightly/v2026.1012.0-nightly.0 beta/v2026.1012.1-beta.0 |
||
|
|
4611d6e95a |
fix(restore): retry sandbox copy-back before holding the task for repair (#15920)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Legacy sandbox runs copy the agent's workspace files back to the host when the run settles. > - When this copy-back fails one time, the server keeps the sandbox and puts a board-owned repair hold on the task. > - The hold blocks new messages. Only an operator API runbook can clear it. The board shows that runbook text, so users do not know what to do. > - Many copy-back failures are transient: a dropped transfer, a provider error, or a merge lock that another restore holds. > - This pull request retries those failures automatically while the run still has its merge baseline and before any host file changes. > - The benefit is that most failures repair themselves, and the remaining holds explain the situation in plain language. ## Linked Issues or Issue Description Refs #14590 (related: no-replay recovery holds) **What happened?** A legacy sandbox run finished its work, but one workspace copy-back attempt failed. The task then showed "Recovery needed" with operator-only steps ("verify its exact stop receipt … record workspaceRepairEvidence"). Messages from the user stayed saved and were not delivered. **What did you expect?** Paperclip retries a transient copy-back failure automatically. If a repair is still necessary, the task says clearly what happened and who must act. **Steps to reproduce** 1. Run an agent in a legacy sandbox environment. 2. Make the workspace copy-back fail one time, for example with a dropped transfer or a held merge lock. 3. Open the task. It is blocked behind a repair hold, and new messages wait. **Additional context** Separately, distinct live events can render toasts with the same title and body (for example, two runs of the same agent that both succeed). The toast list showed both, and the desktop app forwards each one as a notification. ## What Changed - `packages/adapter-utils/src/sandbox-managed-runtime.ts`: retry the workspace copy-back up to three times in total, with a 2 s and then a 5 s backoff. A retry occurs only when the failure is `restore_failed` or `restore_lock_timeout` and no host workspace file was written. The merge now marks the first host write in `beforeApply`, which runs under the merge lock before any write. - Unsafe archives, permission failures, and failures after the merge started writing are never retried. Each retry uses a new temp directory and a new imported Git ref. - New optional `workspaceRestoreRetryDelaysMs` input, so tests can shorten the backoff. Nested repository restores receive the same value. - `ui/src/components/ExecutionBlockerNotice.tsx`: for a workspace repair hold, the notice now says what happened in plain language. The stored operator runbook text moves under "Technical details". - `ui/src/context/ToastContext.tsx`: also dedupe toasts on their visible tone, title, and body inside the existing 3.5 s window. - `doc/workspace-restore-locks.md`: document the automatic retry. ## Verification - `pnpm --filter @paperclipai/adapter-utils exec vitest run src/sandbox-managed-runtime.test.ts src/workspace-restore` (218 tests pass). New tests cover: - a retry after a dropped transfer, - a retry after a merge-lock timeout, - a stop after three attempts, - no retry for an unsafe archive, a permission failure, or a failure after the merge started writing. - The existing step-diagnostic test now uses failures that persist across retries. It still expects one diagnostic per failed step. - `pnpm --filter @paperclipai/ui exec vitest run src/context/ToastContext.test.tsx src/components/ExecutionBlockerNotice.test.tsx src/context/LiveUpdatesProvider` (120 tests pass). - `tsc --noEmit` for adapter-utils passes. `node scripts/check-token-gates.mjs` is clean. - Not fully run: the full adapter-utils suite has two failures in `github-launcher*.test.ts`. They read the managed GitHub credentials of the local sandbox environment and do not touch restore code. UI `tsc -b` reports five errors in `PluginOrganizationSwitcher.tsx`, which is not changed here, because the plugin SDK is not built in a fresh install. ## Risks - A copy-back that fails before the merge can take about 7 s longer before it reports a failure. Lock timeouts can take up to about 97 s in total (three 30 s lock waits plus backoff). Settlement already waits for the copy-back. - The retry logic depends on `beforeApply` running before the first host write in `mergeDirectoryWithBaseline`. That is true today, and a comment marks it. - A Git bundle import before the merge can leave unreferenced Git objects after a retry. They are harmless, and the existing ref cleanup still runs on each attempt. - Content-based toast dedupe can hide a second toast with identical text inside 3.5 s. That is the intended behavior. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`, Anthropic) in the Claude Agent SDK, with tool use (shell, file edit, code search) and a subagent for code exploration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip Agent <agent@paperclip.ing> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>canary/v2026.1012.0-canary.3 |
||
|
|
657e71584a |
fix(daytona): record provider state when lease creation hits its deadline (#15946)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can run in Daytona sandboxes. The Daytona plugin acquires a
lease before each run.
> - Some acquisitions time out. The report then says
`environmentAcquisitionPhase=create` and that the whole 300 s budget was
used.
> - The Daytona SDK create timeout is equal to the acquisition budget.
The outer timer always fires first. Thus the report never shows the
state that Daytona left the sandbox in.
> - Without that state, we cannot choose a fix. A stuck snapshot pull, a
slow start, and a missing allocation need different fixes.
> - This pull request adds one bounded, read-only provider lookup at the
create deadline. It records a fixed-enum provider state in the existing
acquisition diagnostic.
> - The benefit is that the next timeout identifies where the provider
stopped. Retry, budget, and cleanup behavior do not change.
## Linked Issues or Issue Description
Refs #15736 (bounded Daytona acquisition failure observations). This
pull request extends those observations.
**What happened?**
A Daytona lease acquisition timed out with `Daytona lease acquisition
timed out; allocation cleanup is pending`. The run-failure report gave
`environmentAcquisitionPhase=create`,
`environmentAcquisitionElapsedMs=300004`, and
`environmentAcquisitionBudgetMs=300000`. The report did not show the
Daytona sandbox state. The SDK create timeout is equal to the outer
budget, so the outer timer always fires first. The SDK never reports the
stuck state.
**Expected behavior**
When creation is still pending at the deadline, the report should show
where the provider left the allocation. Examples are `pulling_snapshot`,
`starting`, `error`, or `absent`.
**Steps to reproduce**
1. Configure a Daytona environment.
2. Start a run when Daytona does not finish create and start within the
acquisition budget.
3. Inspect the setup-failure report. Only the phase, the elapsed time,
and the budget are present.
**Paperclip version or commit**
`master` at
canary/v2026.1012.0-canary.2
|
||
|
|
ba04968b10 |
fix(server): report failing workspace jobs as 422 instead of 500 (#15941)
A non-zero workspace job exit is the operator job's own result. Map it to 422 workspace_job_failed with the exit code instead of a 500 that the error handler reports to Sentry as a server crash. A job without a command is also 422. Other workspace command failures are unchanged. Co-Authored-By: Amy (Opus) <claude@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>canary/v2026.1012.0-canary.1 |
||
|
|
f331d386dc |
fix(ui): attribute stackless unhandled rejections with event and lifecycle facts (#15940)
Diagnostics only for Sentry PAPERCLIP-FRONTEND-G: record trusted/synthetic unhandledrejection origin, promise kind, and coarse page lifecycle facts without reading rejected values. Co-Authored-By: Amy (Opus) <claude@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bfe7c6357d |
fix(ui): stack confirmation prompt and hide raw custom target keys (#15895)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents ask the board for decisions with `request_confirmation`
interaction cards
> - The compact task-chat card puts the prompt and the target pill in
one `justify-between` flex row
> - An agent can send a `custom` target with only a machine `key` (for
example `github:org/repo:pull:16`) and no `label`
> - The card shows that raw key as a mono pill that does not shrink, so
the card splits about 50/50 and the prompt is squeezed into the left
half
> - This pull request stacks the prompt above the pill and hides custom
targets that have no human label
> - The benefit is a readable, full-width confirmation prompt without
meaningless identifier pills
## Linked Issues or Issue Description
**What happened?**
A pending confirmation card split its body into two columns. The prompt
used the left half. A pill with the raw text
`github:paperclipai/paperclip-desktop:pull:16` used the right half. The
line breaks in the prompt were also lost.
**Steps to reproduce**
1. Create a `request_confirmation` interaction with a multi-line
`prompt` and `target: { "type": "custom", "key":
"github:org/repo:pull:16" }` (no `label`).
2. Open the task chat for the issue.
3. See that the prompt and the key pill share one row, split about
50/50.
**Expected behavior**
The prompt uses the full card width and keeps its line breaks. A custom
target key is an identifier for idempotency and stale detection, so it
does not show as display text. A target pill shows only when the target
has a human label.
## What Changed
- `TaskChatCompactInteractionCard`: stack the prompt full width with the
target pill below it. This applies to the confirmation and the
checkbox-confirmation cards.
- `TaskChatCompactInteractionCard`: keep line breaks in the prompt
(`whitespace-pre-line`, `break-words`).
- `TaskChatCompactInteractionCard` and `IssueThreadInteractionCard`:
show no target chip for a `custom` target without a `label`. Labeled
custom targets (for example connection authorization) and
`issue_document` targets do not change. The stale-target chevron shows
only when both chips show.
- Add regression tests for the stacked layout, the hidden unlabeled key,
and the labeled link pill. The tests cover the compact confirmation and
checkbox-confirmation cards, and the thread card pending chip and
stale-target expiry (with labeled, unlabeled, and mixed custom targets,
and the chevron rule).
- `CONTRIBUTING.md`: add a "Keep Screenshots Out of the Repo" section.
Contributors attach review screenshots to the PR on GitHub. They do not
commit them to the repository.
## Verification
- `cd ui && npx vitest run
src/components/task-chat/TaskChatInteractionCard.test.tsx
src/components/IssueThreadInteractionCard.test.tsx`: 2 files, 100 tests
pass. If you revert the `isDisplayableRequestConfirmationTarget` guard,
3 of the new thread-card tests fail.
- `node scripts/check-token-gates.mjs`: all gates clean.
- `pnpm --filter @paperclipai/ui typecheck`: no errors in changed files.
The only errors come from `@paperclipai/plugin-sdk/ui`, which had no
built `dist` in the local workspace. These errors are not related to
this change.
- Manual: create a confirmation as in the steps above. The prompt is
full width and no key pill shows. Add `label: "PR #16"` and an `href`. A
`PR #16` link pill shows below the prompt.
### Screenshots
These are Storybook renders of the compact task-chat card at 720 px. The
three cards are: a confirmation with an unlabeled custom target (the
reported case), a confirmation with a labeled custom target, and a
checkbox confirmation with an unlabeled custom target.
The images load from an earlier commit of this PR. A later commit
deletes them, so they are not part of the merged change.
| Before (light) | After (light) | After (dark) |
| --- | --- | --- |
|
| 
| 
|
## Risks
- Low risk. This is a UI-only change. No schema or API change.
- Agents that send a custom target without a label lose the visible
pill. The key was never readable display text. To show a pill, agents
set `label`.
## Merge Note
The first commit was authored as Paperclip. When you squash-merge, keep
this trailer in the squash body:
Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com>
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code agent
harness with tool use (shell, file edit). Standard context window.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
canary/v2026.1011.0-canary.12
|
||
|
Devin FoleyandPaperclip
|
d49cbf4c4f |
Permit idle sleep with consumed retries and passive worktrees (#15907)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Idle sleep must wait for accepted work and durable cleanup.
> - The retry dispatcher keeps its due date after it promotes a retry.
> - Finished retries and passive local directories can therefore look
like pending work.
> - This pull request requires completion proof and permits known
passive directory records.
> - It retains the records and keeps real work and unknown state awake.
## Linked Issues or Issue Description
Related: #15897 and #15851. The open runtime-service work in #13413
retains its separate service and ingress guarantees. This patch does not
change those checks.
**What happened?**
Idle sleep treats a consumed retry date as pending work. It also blocks
on every retained execution workspace, including a local directory
without runtime activity.
**Steps to reproduce**
1. Promote a scheduled retry, then finish its run without accounting
debt. Retain its scheduled retry date.
2. Retain a local primary directory or Git worktree without runtime
configuration, commands, or cleanup due.
3. Request an owned idle safety report. The report returns `present` for
this history.
**Paperclip version or commit**
Reproduced on master at
|
||
|
|
2223b7de9e |
perf(ci): stop building the standalone packages inside the public-install verifier (#15908)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `Build` check. Its last step is the public npm install verifier, `scripts/verify-grok-npm-install.mjs`, which stages the eighteen public packages reachable from the server and the CLI, packs them, and installs them in a clean consumer. > - Before it stages anything, the verifier runs `scripts/build-standalone-public-packages.mjs`. That script installs and compiles the seven sandbox-provider plugins that live outside the pnpm workspace: one network `pnpm install --ignore-workspace` and one `tsc` per plugin. The line was copied from the build block of `release.sh` when the verifier was written in #14024. > - release.sh needs that build because it publishes those plugins. The verifier does not for anything it stages. Its package closure follows `workspace:` specs from the server and the CLI, and a `workspace:` spec can only name a workspace package, so none of the eighteen packages it packs is a standalone package and no tarball reads that build's output. > - One optional path at the end of the verifier does read it: qualification-package retention, which `docker-runner-check.yml` enables with `PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR` after running only `pnpm build`, packs the Daytona plugin from its `dist`. The PR `Build` job never sets that variable. > - The timing line from #15868 and #15888 puts the verifier's staging phase at 75.8s on the latest run, of which the staged sources, the bundled installs and the packs account for 46.5s. The rest, about 29s, is the work before staging, and the standalone build is almost all of it. `Build` at 433-530s is the slowest check in most PR runs once the server shards drop, so that time sits on the run's critical path. > - This pull request runs the standalone build only when retention is requested, keeps the `ui-dist` copy the server package needs, and times both on the timing line so what is left before staging is visible. > - The benefit is a `Build` job about 21-28s shorter, with no change to what the verifier installs or checks, no loss of coverage for the standalone build because Canary Dry Run still runs it through `release.sh`, and no change to the retention path that reads it. ## Linked Issues or Issue Description Replaces #15892, which GitHub closed automatically when the branch of #15888 (its base at the time) was deleted on merge. Same two commits, rebased onto `master` on top of #15888 ( |
||
|
Devin FoleyandPaperclip
|
0ba88d2195 |
fix(opencode-local): preserve deadlines in Daytona process sessions (#15905)
## Thinking Path > - Paperclip manages work done by AI agents. > - The OpenCode adapter can run its CLI in a provider sandbox. > - A long CLI run must remain observable until it exits or the host stops it. > - The legacy main command does not request a provider process session. A launch without an active run step uses one synchronous command request. > - That request can lose its response before the resolved execution budget ends. > - This PR requests the existing asynchronous session path for the main Daytona OpenCode command. It also carries the resolved command deadline through the host and provider contracts. The host still applies the effective capability gate. > - The change removes that synchronous request from eligible sandbox runs. ## Linked Issues or Issue Description Related reporting fix: #15789. A search of open and closed GitHub issues and PRs found no duplicate transport fix. **What happened?** A hosted Linux legacy OpenCode run retained a gateway-timeout response about one hour after CLI startup. It had a four-hour resolved budget and recent agent output. The earlier reporting fix correctly kept the configured policy separate from the timeout message. The original response owner and specific timer were not retained, so this PR does not claim to prove that provider cause. **Expected behavior** A Daytona target that supports persistent process sessions should dispatch the long agent command asynchronously. Paperclip should observe its output and exit through the existing session path. A single command HTTP request should not need to stay open for the full agent run. **Steps to reproduce** 1. Use legacy OpenCode with a Daytona sandbox target and a four-hour execution budget. 2. Launch outside an active measured step. The current main CLI call does not request a persistent process session. 3. In the SDK fixture, make the synchronous command request return a provider timeout before the budget ends. Keep the asynchronous session healthy for more than one hour. 4. The regression tests require session dispatch, incremental output, one command dispatch, a real exit result, and session cleanup. 5. Set a shorter command deadline and stall dispatch, a status read or a log snapshot. The command must return a timeout at its own deadline. Healthy output must not extend that deadline. **Paperclip version or commit** Observed source: `e733bddc45e00efb1819fffa8677f7b0d07802c5`. PR base: `4b59e2a2ecc67e98b336b8eb4c09959681672d54`. **Deployment mode** Hosted Linux server, legacy OpenCode adapter, Daytona sandbox. The host gate retains the old path when a provider cannot keep process sessions. ## What Changed - Add an optional sandbox session choice to the shared process helper. Leave the choice absent for existing callers. - Request that session only at the main Daytona OpenCode CLI invocation. Keep probes, preparation and other providers on their existing paths. - Carry a separate optional command deadline through the shared runner, host service and plugin SDK. Existing session callers can omit it. Zero leaves the command deadline disabled. - Bound dispatch, live output and recovery observations with one command timer. Clear it on settlement. Fence late stream callbacks. Remove deadline listeners after each read and clear interrupted polling waits. Preserve a known exit code if only the final logs time out. A timeout result does not claim that the process stopped. - Add adapter and process-helper regression coverage. Test the existing Daytona session for a virtual run beyond one hour, without command replay, with release cleanup. - Document transport selection and the limits of the timeout evidence. ## Verification - `pnpm -r typecheck` passed on Node 25.6.1 and pnpm 9.15.4 with the command deadline changes. - `pnpm build` passed with the updated contracts. The separately installed Daytona package also passes its own typecheck and build against the pinned SDK 0.203.0 and TypeScript 5.9.3. - Five focused process, adapter, host and provider suites passed all 349 tests. They cover a run beyond one hour, five stalled observation phases, healthy-stream deadlines, zero/absent budgets, late callbacks, session release, SSH and capability fallback. - The real environment-runtime forwarding tests passed: 19 selected tests, including the separate command budget and forced session flag in the worker RPC. - An earlier full local `pnpm test:run` began before a clean build and overlapped the correction. That mixed-version run was stopped and is not passing evidence. After the build, the two previously uncollected suites pass (1,659 tests). Two unchanged workspace fixtures fail with this machine's Git default branch `main`; both pass with a test-process-only `master` override. One unchanged native recovery fixture fails on both this branch and an independently installed and built checkout of base `4b59e2a2ecc67e98b336b8eb4c09959681672d54`, with the same terminal-result mismatch. It is not introduced by this change. The full local suite is not claimed green. These limitations do not replace fresh hosted CI. - Final head `8e96638cdef8ca6e91f7a42a52c83a075180c137` passes all hosted checks: 54 passed and 2 expected Storybook skips. Greptile reviewed this exact head at 18:12 UTC and scored it 5/5. Both actionable findings are fixed and resolved. GitHub reports no conflicts and a clean merge state. Earlier-head results are not used as final-head proof. - The standalone provider test command passes 373 tests, with 14 expected live-probe skips. - SDK tests use a controlled virtual clock. No live provider experiment, task replay, manual deployment, or public PR merge was performed. ## Risks The main legacy OpenCode command uses Daytona's existing session route when the effective capability gate permits it. A quiet log stream can use the existing status and snapshot fallback. Those observations add provider reads. The new optional command budget is separate from each read's timeout and returns a timeout without the host RPC's extra 30-second allowance. Existing session callers that omit it keep their previous lifetime. Callback bridge control commands retain their explicit session bypass. The capability gate retains the previous path for unsupported targets. Other providers, local and SSH execution keep their existing path. Retry policy, Stop acknowledgement and lease ownership rules stay the same. The SDK exposes no cancellation handle for an abandoned stream; existing Stop and lease teardown still own command termination. This fix does not establish the original gateway owner or prove recovery in a live environment. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact backend identifier and context window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed paths; the independently reproduced local baseline failure is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1ac48aff8d |
perf(ci): pack the verifier's staged packages concurrently (#15888)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `Build` check. Its last step is the public npm install verifier, `scripts/verify-grok-npm-install.mjs`, which stages the eighteen public packages, packs them, and installs them in a clean consumer. > - The verifier packs the eighteen packages one after another with `npm pack`. Each pack is a Node start-up plus gzip of that package's files. The timing line added in #15868 shows this loop at 35.6-37.9s on four `Build` runs, the largest part of the staging phase now that the bundled installs overlap. > - `Build` at 433-530s is the slowest check in most PR runs once the server shards drop, so its staging phase sets the wall-clock time of the run. > - Each pack reads only its own `package-<index>` directory and writes one tarball. The runner has four cores and the packs are CPU-bound. There is no reason to run them one after the other. > - This pull request runs the packs as concurrent children through the prefetch helper, one per core, bundled packages first, each into its own directory, and puts the results back into dependency order before every tarball is checked. > - The benefit is a shorter `Build` job, and a timing line that reports the packs' serial cost next to their wall time, so the saving is measured on this PR's own run instead of estimated. ## Linked Issues or Issue Description Refs #15868, #15861. This branch is stacked on the branch of #15868, which is stacked on #15861, because the same execution branch holds both open PRs. The diff here is this PR's one commit. After #15868 merges, this PR is retargeted to `master` and rebased. **What existing behavior does this improve?** The `Build` PR check. Its public npm install verifier packs the eighteen public packages in one serial loop. **Subsystem affected** CI and release scripts (`scripts/verify-grok-npm-install.mjs`, `scripts/prefetch-child.mjs`). **Current behavior** After the bundled installs are joined, the verifier runs `npm pack --pack-destination <root>` for each of the eighteen packages in turn, finds the new tarball by listing the shared root, and checks it with `tar -tzf`. `stage_pack_ms` was 35.8s, 37.1s, 35.6s and 37.9s on the four runs of #15868. The parent is idle apart from the one pack it waits on, and three of the four cores are idle. **Proposed behavior** Start the packs as children through `startPrefetch`, at most one per core (`os.availableParallelism()`), bundled packages first because they make the largest tarballs. Each child packs into its own `pack-<index>` directory, so the tarball a package produced is read from there instead of guessed from a shared directory. Register each child for cleanup as it starts. Put the results back into dependency order, then check every tarball with `tar -tzf` as before, then copy them into the assets directory as before. Report `stage_pack_check_ms`, `pack_concurrency`, `pack_serial_ms` and `pack_longest_ms` on the timing line, and keep `stage_pack_ms` as the whole step. **Reason and benefit** With four slots the step takes about the longest single pack or a quarter of the serial sum, whichever is larger. The expected saving is 20-25s of the 36-38s, measured on this PR's own `Build` job by `pack_serial_ms` against `stage_pack_ms`. **Measured on this PR's first run ([38146108750](https://github.com/paperclipai/paperclip/actions/runs/38146108750), `Build` 433s, verifier 214s):** `stage_pack_ms` 29.9s against 35.6-37.9s before, so the step shed 6-8s, not the 20-25s estimated. `pack_serial_ms` is 44.4s with `pack_concurrency` 4, and `pack_longest_ms` is 28.3s: one pack (the server with its 513 MB bundled closure, gzip level 9 inside npm) runs for almost the whole step, so the step is bounded by that pack and not by the slot count. The seventeen other packs fit inside it. The serial sum is 7-9s higher than the old serial step, which is the four processes contending for the runner's cores. `stage_packages_ms` 75.8s against 80.4-89.6s on #15868's runs; `image_wait_ms` and `grok_wait_ms` stay 0. **Breaking changes** None. The same packages are packed with the same `npm pack` arguments, the tarballs are checked the same way, and the consumer receives them in the same order. ## What Changed - `scripts/pack-staged-packages.mjs` (new): `packStagedPackages` launches the packs in the order given with at most `concurrency` children at once, through the prefetch helper, and resolves with each entry's tarball path and wall time in the same order. Every child is passed to `register` as it starts. The first failure rejects with the tail of that child's log and nothing further is launched. A child that leaves anything other than one tarball in its directory is a failure. `packConcurrency` is one per core. `packTimings` gives the timing-line keys. - `scripts/verify-grok-npm-install.mjs`: each entry gets its `index` and its own `pack-<index>` destination. The pack loop is replaced by one `packStagedPackages` call, launched bundled-first and registered into the `prefetches` list as each child starts, so an earlier failure stops any pack still running before the scratch root is removed. The results are sorted back into dependency order, every tarball is checked in that order, and the copy into the assets directory is unchanged. Only the lifecycle sentinel is still packed in-process. - Timing line: `stage_pack_check_ms`, `pack_concurrency`, `pack_serial_ms` and `pack_longest_ms` are added. `stage_pack_ms` still covers the whole step, so the before and after numbers compare directly. - `scripts/__tests__/pack-staged-packages.test.mjs` (new): five tests with a local stand-in for npm and no network. The default concurrency and the pack arguments; four packs with two slots run the two slow packs together, start a third only once a slot is free, receive the exact `npm pack` argument list, register every child, and come back in entry order with their tarballs and timing keys although the fast packs finish first; a failed pack rejects with its log tail, launches nothing more, and leaves the running pack to be stopped; a pack that produces two tarballs or none is a failure; and no entries resolve at once. - `scripts/__tests__/grok-public-install-sandbox.test.mjs`: the guard test on the verifier source now checks the import, the bundled-first launch with cleanup registration, the reorder before the checks, the checks before the copy, that only the sentinel is packed in-process, that each pack has its own directory, and the new timing keys. - `package.json`: the new test file is in `test:release-registry`. ## Verification - `node --test scripts/__tests__/pack-staged-packages.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/prefetch-child.test.mjs`: 35 tests pass (30 on #15868's head plus the five new ones). The new file passes three consecutive runs. - `node --check` passes on both scripts. - With the real npm 11.12.1 on Node 25.9: two small packages packed through `packStagedPackages` with two slots produce one tarball each in their own `pack-<index>` directory, with the expected `package/` listing from `tar -tzf`. - The verifier itself needs Docker and a network, which this workspace does not have. This PR's own `Build` job runs the branch's verifier; its timing line carries `stage_pack_ms`, `pack_serial_ms`, `pack_longest_ms` and `pack_concurrency`, which is the measurement. Before, on #15868's four runs: `stage_pack_ms` 35.8s / 37.1s / 35.6s / 37.9s. - `docker-runner-check.yml` runs the same script after `pnpm build` and needs no change. ## Risks - Four `npm pack` processes run at once and share the host npm configuration and cache. `npm pack --ignore-scripts` reads the package directory and writes one tarball; it does not write to the cache beyond its per-process log file. If a shared-state race appears, the failing pack's log tail is in the error and the fix is to lower the concurrency. - Memory and CPU: four Node processes instead of one on the four-core `Build` runner, for about thirty seconds. The largest pack (the server with its bundled dependencies) is the same process as before. - The pack output moves from the CI log into per-pack log files in the scratch root. A failure reports the last 2000 characters of the failing pack's log; a successful pack's output is discarded with the scratch root. - The tarball checks (`tar -tzf`) now run after all packs instead of after each pack. A bundled Codex payload is still rejected before any tarball reaches the consumer; the only change is that the error arrives a few seconds later. - Low risk otherwise. The bundled installs, the consumer install, and every probe are unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.1 (Anthropic), model ID `claude-fable-5-1`, run as a Paperclip agent through the Claude Code local adapter with tool use (shell, file edits, GitHub CLI). Reasoning mode: extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (run 38146108750: `server (11/12)` failed on its first attempt on a teardown `EACCES` while removing the per-worker `PAPERCLIP_HOME` after all 74 files and 1257 tests had passed, in a file this PR does not touch; the re-run of that job passed and every check is green) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>canary/v2026.1011.0-canary.11 |
||
|
|
3d3b8d4371 |
fix(external-objects): distinguish hidden GitHub repos from genuinely-missing objects (#15906)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Issues reference external objects such as GitHub pull requests. The instance resolves their status and shows it in the UI. > - GitHub returns 404 both when a pull request or issue does not exist and when the repository exists but the configured credentials cannot see it. GitHub hides private repos behind a 404 on purpose. > - The resolver treated every 404 as a terminal not-found verdict. It froze references to inaccessible repos forever, and marked them "Not found (Stale)". > - This pull request probes the repository metadata to tell the two cases apart and keep frozen objects re-checkable. > - The benefit is accurate statuses instead of permanent "Not found" labels for repos the instance cannot see. ## Linked Issues or Issue Description No public GitHub issue exists for this report. **What happened?** - The issue properties panel and markdown decoration showed many GitHub pull request and issue references as "Not found (Stale)" even though the objects exist. - GitHub returns 404 for objects the configured credentials cannot access, not only for objects that are truly gone. - The resolver treated every 404 as a terminal not-found verdict, so those references were frozen and showed a misleading "stale" liveness. - Detected-but-unresolved GitHub objects were labeled "Not yet resolved" in inline markdown, which did not match the pill label "Not yet refreshed". **Expected behavior** - A reference to a repo the credentials cannot see should surface as authorization-required, not as a permanent "Not found". - A truly missing object should still be "Not found". - A terminal object (merged, closed, or confirmed not-found) should not be labeled "Stale", because it is never refreshed again. - Inline markdown statuses should match the pill statuses. **Steps to reproduce** 1. Configure GitHub credentials with no access to a private repo. 2. Add a comment or issue description that links to a pull request in that repo. 3. Wait for the external object resolver to run. 4. Observe the reference labeled "Not found (Stale)". **Paperclip version or commit** - `master` at `e733bdd` (before this change). **Deployment mode** - Self-hosted server. ## What Changed - Added a repository visibility probe to the GitHub external object provider. - A 404 for an object now probes the repository metadata with the same credentials: - Repository visible: the object genuinely is gone. Keep the not-found snapshot. - Repository hidden: return an authorization-required result instead of a terminal not-found verdict. - Probe failed (network, rate limit, 5xx): return an unreachable result and retry later. - A rate-limited repository probe (403 with `x-ratelimit-remaining: 0`) is treated as uncertain, not as missing access, so it retries instead of showing a misleading "Requires auth". - Re-check previously frozen GitHub not-found objects on their normal schedule so they self-heal once credentials or repo visibility improve. A frozen object is only re-checked until a probe confirms the repo is visible; once confirmed deleted, it is terminal and no longer polled. - Cleared any terminal verdict when a refresh fails, so a failed refresh does not freeze an object. - Kept terminal objects from displaying the "stale" liveness label. - Matched the inline markdown status label to the pill status label, so detected-but-unresolved GitHub objects say "Not yet refreshed" instead of "Not yet resolved". - Skipped the liveness suffix in markdown links and status icons when it duplicates the status label (e.g. no more "Requires auth (Requires auth)"). - Cached repository visibility probes per company+repo + credential fingerprint, so many objects in the same repo share one probe per refresh cycle instead of each issuing its own — and a verdict obtained under one token is never reused by a later refresh running under a different credential. ## Verification - Added provider tests for visible, hidden, failed, and rate-limited repository probes, per-company/per-repo probe caching, and probe-verified not-found snapshots. - Added service tests for terminal-object liveness, for re-checking frozen GitHub not-found objects, and for ensuring probe-verified not-found objects are never re-queued. - Added UI tests asserting unresolved GitHub objects render as "Not yet refreshed" (not "Not yet resolved") and that failed-first-refresh labels do not repeat the liveness text. - Ran `npx vitest run src/__tests__/external-objects-service.test.ts` in `server/`: 34 passed (incl. provider + service suites). - Ran the external-object markdown/pill/icon suites in `ui/`: 97 passed. - `ci / Typecheck + Release Registry` and all other CI gates pass on this head. Local `tsc` could not run to completion in the execution sandbox (the sandbox memory cap killed it), so CI typecheck is the authoritative gate. ## Risks - Low risk. The change is confined to the GitHub external object provider and the external object service. - A repo that genuinely does not exist and a private repo both return 404 for the probe. Both now surface authorization-required or a retry instead of an immediate not-found freeze, which stays re-checkable and self-heals. - Re-checking frozen not-found objects adds at most one repository probe per affected object until the probe produces a definitive verdict, after which the object is terminal and no longer polled. Probe results are cached per company+repo+credential, so a credential rotation always re-probes instead of inheriting a stale verdict. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: DeepSeek via OpenRouter, model ID `openrouter/~deepseek/deepseek-v4-flash-latest`, running inside the Paperclip opencode adapter (`opencode_local`). - The model produced and assisted with the reasoning and code changes; the execution workspace, test suite, and git/PR steps above were run by the same agent run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details — the branch name is assigned by the managed execution workspace and the harness requires keeping it - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the current head (all `ci / *`, Greptile, Socket, and review checks pass) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
026bdcd046 |
perf(ci): run the verifier's bundled-dependency installs concurrently (#15868)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `Build` check. Its last step is the public npm install verifier, `scripts/verify-grok-npm-install.mjs`, which stages the eighteen public packages and installs them in a clean consumer. > - The verifier stages the packages in one serial loop. Three packages bundle dependencies (`@paperclipai/server`, `@paperclipai/adapter-utils`, `@paperclipai/db`). Each one runs a host-side `npm install` of its bundled closure in-process, one after the other, and nothing in the loop overlaps anything else. The staging phase took 102-116s on recent `Build` jobs. > - `Build` at 433-530s is the slowest check in most PR runs once the server shards drop, so its staging phase sets the wall-clock time of the run. > - The three installs write only their own directories and read nothing from each other. There is no reason to run them one after the other. > - This pull request starts the three installs together as child processes and copies the other fifteen packages while they run. Packing stays serial and in order. > - The benefit is a shorter `Build` job, and a timing line that reports each install, so the saving is measured on this PR's own run instead of estimated. ## Linked Issues or Issue Description Refs #15861. This branch is stacked on the branch of #15861 because the same execution branch holds that open PR. The diff here is this PR's three commits. After #15861 merges, this PR is retargeted to `master` and rebased. **What existing behavior does this improve?** The `Build` PR check. Its public npm install verifier stages the eighteen public packages in one serial loop. **Subsystem affected** CI and release scripts (`scripts/verify-grok-npm-install.mjs`, `scripts/prepare-bundled-package.mjs`, `scripts/prefetch-child.mjs`, `scripts/prefetch-supervisor.mjs`). **Current behavior** The staging loop calls `prepareBundledPackage` in-process for the three packages that bundle dependencies. Each call runs a host-side `npm install` of that package's bundled closure, applies the repo patches, and rewrites the manifest, before the loop moves to the next package. The timing line reports the whole staging phase as one number, 102-116s on recent runs. The three installs are the only network work in that phase. **Proposed behavior** Stage the three bundled packages' sources first, start their installs at the same time as children of the script's own CLI entry (the same path `release.sh` uses), copy the other fifteen packages while the installs run, join the installs, and then pack every package in dependency order as before. Report `stage_sources_ms`, `stage_bundled_ms`, `stage_pack_ms`, `bundled_wait_ms` and one `stage_bundled_<package>_ms` per install on the timing line. **Reason and benefit** The staging phase shortens by the two shorter installs. Measured on this PR's first run (below): the three installs sum to 25.9s and take 16.4s of wall time together, so the overlap removes about 9.5s from the phase, and the timing line now shows that packing (35.8s) and the standalone-package build before the loop (about 27s) are the larger remaining costs. The original estimate of 30-50s was too high because it assumed the installs were about half of the phase; they are a quarter. **Breaking changes** None. The packed tarballs are produced by the same script, in the same order, and are checked the same way. ## What Changed - `scripts/stage-bundled-packages.mjs` (new): `startBundledInstalls` starts one child per bundled package through the prefetch helper from #15849, with the child's output in a log file. `bundledInstallTimings` reads each child's wall time for the timing line. `bundledPackageSlug` maps a package name to its log and timing key. - `scripts/verify-grok-npm-install.mjs`: the staging loop is split. The bundled packages' sources are staged first and their installs start together. The other packages are copied while the installs run. The install handles are pushed into the `prefetches` list at once, so an earlier failure stops any install still running before the scratch root is removed. The join comes before the first `npm pack`. The pack loop walks every package in `needed` order and checks each tarball as before. `prepareBundledPackage` is no longer imported. - Timing line: `stage_sources_ms`, `stage_bundled_ms`, `stage_pack_ms`, `bundled_wait_ms` and `stage_bundled_server_ms`, `stage_bundled_adapter_utils_ms`, `stage_bundled_db_ms` are added. `stage_packages_ms` and the download keys are unchanged, so the before and after numbers compare directly. - `scripts/__tests__/stage-bundled-packages.test.mjs` (new): five tests with local Node children and no network. The slug mapping; the children run the CLI entry `release.sh` runs and never an in-process call; two installs run at the same time, each with its own source and target, and their logs and timing keys are correct (each child exits only after the other has started, so serial children would fail); a failed install rejects with its own log tail while the other is still running and is then stopped; and stopping the installs after one fails also stops the stand-in npm process the other wrapper is blocked on. - `scripts/__tests__/grok-public-install-sandbox.test.mjs`: one guard test on the verifier source. The bundled sources are staged before the installs start, the installs are registered for cleanup before the other packages are copied, the join comes before the pack loop, the pack loop is in order, there is still exactly one package pack loop, and the timing keys are reported. - `package.json`: the new test file is in `test:release-registry`. ### Review fix (second commit): stop a prefetch's whole process tree Greptile pointed out that `stopPrefetches` signalled only the Node wrapper of each install, while the wrapper is blocked in `execFileSync('npm', ['install', ...])`. After one install failed, the other wrapper died and its npm process kept writing into the scratch root the verifier was about to remove. - `scripts/prefetch-child.mjs`: on POSIX every prefetch child is started `detached`, so it leads its own process group. `stopPrefetches` signals the group (`process.kill(-pid, ...)`) and waits until no member of the group exists, with `SIGKILL` after the grace period and a second grace period before it gives up so cleanup cannot hang. On Windows the child is signalled directly as before. - `scripts/__tests__/prefetch-child.test.mjs`: two tests. A stopped prefetch's grandchild is gone after `stopPrefetches` returns, and a `SIGINT` to a separate parent process reaches the grandchild while the parent still exits with `SIGINT`. Both are skipped on Windows. - `scripts/__tests__/stage-bundled-packages.test.mjs`: one test, the cancellation case Greptile asked for. The stand-in `prepare-bundled-package` blocks in `execFileSync` on a stand-in npm that announces its pid; after the other install fails, `stopPrefetches` must leave that pid gone. All three new tests fail against the previous helper (`grandchild <pid> outlived stopPrefetches`, `outlived the parent's SIGINT`, `stand-in npm <pid> outlived stopPrefetches`) and pass with the fix. ### Review fix (third commit): the parent runs no signal handler Greptile pointed out that the second commit forwarded the terminal's signals to the detached groups through a JavaScript handler in the parent. The verifier spends most of its time blocked in `execFileSync` (the standalone builds, every `npm pack`), where no handler can run, and installing the handler also replaced the parent's immediate default termination. So a Ctrl-C during a synchronous phase did nothing until that command returned, and the detached installs and downloads kept running meanwhile. Confirmed with the new test against the second commit's helper: the parent survived the `SIGINT`, exited with code 1 once the synchronous command had died, and its handler never ran. - `scripts/prefetch-supervisor.mjs` (new): every prefetch command now runs under this small Node script as the leader of its own process group. The supervisor holds a pipe from the parent on stdin (which the parent writes to only to request a stop where it cannot signal the group; see the fourth commit). The kernel closes it when the parent exits for any reason, including a signal delivered while the parent is blocked, or `SIGKILL`, and the supervisor then stops its group (`SIGTERM`, `SIGKILL` after the grace period) and says so in the log. It passes group-wide signals on to the command and stays alive until the command exits, so its own exit reports the command's exit code or signal. Run by hand with a terminal or `/dev/null` on stdin it watches nothing. - `scripts/prefetch-child.mjs`: `startPrefetch` spawns the supervisor with the command, with stdin as the pipe. The forwarding handlers are gone; the parent installs no signal handler, so its default termination stays immediate. `stopPrefetches` is unchanged. - `scripts/__tests__/prefetch-child.test.mjs`: the first test asserts the parent has no `SIGINT`/`SIGTERM`/`SIGHUP` listener after a prefetch starts. The "signal to the parent" test now runs for `SIGINT` and `SIGKILL` and checks the supervisor's log line. One new test gives a separate parent its own process group, blocks it in `execFileSync` on a 20s command, and sends `SIGINT` to that group the way a terminal sends Ctrl-C: the parent must die of `SIGINT` within two seconds and the prefetch's grandchild must be gone. Against the previous helper the parent exits with code 1 instead, the grandchild outlives `SIGKILL`, and the handler-count assertion fails. ### Review fix (fourth commit): a deliberate stop on Windows goes through the pipe Greptile pointed out that on Windows, where `startPrefetch` now returns the supervisor, `stopPrefetches` called `child.kill('SIGTERM')` on it. Windows has no process groups and `kill` there ends a process outright, so the supervisor died without running its handler and its command kept running after cleanup returned. - `scripts/prefetch-child.mjs`: without process groups, the graceful stop no longer signals the supervisor. `stopPrefetches` writes a stop request to the pipe the supervisor already watches for the parent's exit (`stdin.end('stop\n')`). Only the `SIGKILL` escalation targets the supervisor itself, after it has had its grace period. POSIX keeps the group signal, so `stopPrefetches` and the existing stop and failure tests are unchanged there. - `scripts/prefetch-supervisor.mjs`: data on stdin is a stop request and EOF is the parent's exit; both end the group the same way (`SIGTERM`, `SIGKILL` after the grace period) and the log line names which one it was. - `scripts/__tests__/prefetch-child.test.mjs`: one new test writes the stop request to a running prefetch's pipe instead of calling `stopPrefetches`. The supervisor must exit with `SIGTERM`, the command's grandchild must be gone, and the log must say `stop requested` and not `parent exited`. The previous supervisor fails it on the log line, since it treated the request as the parent's exit. The Windows kill itself cannot be reproduced on the Linux runners, so the Windows branch of `signalTree` is verified by that supervisor test plus reading its two lines. The stand-in that ignores `SIGTERM` is now skipped on Windows, where `kill` cannot be ignored. ## Verification - `node --test scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs`: 30 tests pass (was 19 before this PR, 24 after the first commit, 27 after the second, 29 after the third). The three files pass three consecutive runs. - `node --check` passes on all four scripts. - The verifier itself needs Docker and a network, which this workspace does not have. This PR's own `Build` job runs the branch's verifier. Measured on all four commits' runs ([38132868803](https://github.com/paperclipai/paperclip/actions/runs/38132868803): `Build` 428s, `pnpm build` 112s, verifier step 214s; [38136912842](https://github.com/paperclipai/paperclip/actions/runs/38136912842): `Build` 447s; [38139871529](https://github.com/paperclipai/paperclip/actions/runs/38139871529): `Build` 432s; [38142762538](https://github.com/paperclipai/paperclip/actions/runs/38142762538): `Build` 457s, `pnpm build` 123s, verifier step 231s): | Key | Run 1 | Run 2 | Run 3 | Run 4 | Before | | --- | --- | --- | --- | --- | --- | | `stage_packages_ms` | 80.4s | 84.9s | 81.5s | 89.6s | 102-116s on master runs 38112764439, 38111191543, 38110932348, 38110914319, 38110897745, 38127611057; 84.5s on #15849's run on a runner as fast as these | | `stage_sources_ms` | 1.4s | 0.7s | 0.9s | 0.9s | (inside the number above) | | `stage_bundled_server_ms` / `adapter_utils` / `db` | 16.3s / 5.0s / 4.6s | 14.9s / 3.6s / 3.3s | 16.4s / 5.2s / 4.7s | 17.1s / 4.4s / 3.8s | ran one after another, 21.8-25.9s together | | `stage_bundled_ms` (wall, all three) | 16.4s | 15.0s | 16.5s | 17.2s | 21.8-25.9s | | `bundled_wait_ms` | 15.3s | 14.7s | 16.1s | 16.7s | — | | `stage_pack_ms` | 35.8s | 37.1s | 35.6s | 37.9s | (inside the number above) | | `consumer_install_ms` | 94.4s | 97.8s | 95.7s | 99.8s | 97-134s | | `image_prefetch_ms` / `image_wait_ms` | 33.7s / 0ms | 31.8s / 0ms | 34.7s / 0ms | 33.7s / 0ms | unchanged | | `grok_prefetch_ms` / `grok_wait_ms` | 2.1s / 0ms | 2.1s / 0ms | 2.0s / 0ms | 2.1s / 0ms | unchanged | The overlap removed about 7-10s from the staging phase on each of the four runs (21.8-25.9s of installs in 15.0-17.2s of wall time). The third and fourth commits changed no timing: both waits stay at 0ms and the staging phase sits inside the band of the first two runs. The rest of the phase is the standalone-package build and the server UI dist prep before the loop (about 27-32s) and the eighteen serial `npm pack` plus `tar -tzf` checks (36-37s). - `docker-runner-check.yml` runs the same script after `pnpm build` and needs no change. ## Risks - Three `npm install` processes share the host npm cache at the same time. npm's cache (`cacache`) is designed for concurrent writers, and the three installs have disjoint target directories. If a shared-cache race appears, the failing install's log tail is in the error and the fix is to serialize again. The first CI run showed none. - The install output moves from the CI log into per-install log files in the scratch root. A failure reports the last 2000 characters of the failing install's log; a successful install's output is discarded with the scratch root. - Memory: three concurrent installs on the `Build` runner instead of one. The closures are small (the server's eight bundled packages are the largest), so this stays well under the runner's memory. - Process groups: a detached child no longer receives the terminal's signals on its own. The supervisor stops its group when the parent is gone, which it learns from the kernel closing the stdin pipe, so this holds for a Ctrl-C while the parent is blocked in synchronous work and for `SIGKILL` alike. The parent keeps Node's default signal handling. If the supervisor itself is killed with `SIGKILL` before its command, the command is orphaned and finishes on its own (each install ends within seconds; the image pull within a minute), the same as any orphaned grandchild before this PR. - One extra Node process per prefetch (five per verifier run). Each costs about 40ms to start, inside the time the parent spends staging anyway. - Low risk otherwise. Packing, the tarball checks, the consumer install, and every probe are unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.1 (Anthropic), model ID `claude-fable-5-1`, run as a Paperclip agent through the Claude Code local adapter with tool use (shell, file edits, GitHub CLI). Reasoning mode: extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on all four commits - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (on the fourth commit Greptile reviewed 8 files and added no comments; all three earlier threads are resolved) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <bender-fable@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>canary/v2026.1011.0-canary.10 |
||
|
|
85f4205de3 |
perf(test): reuse a migrated embedded Postgres template across suites (#15861)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Pull requests to this repository run a large CI matrix. The slowest
check sets the time a contributor waits for a green run.
> - After #15853, the twelve `server (n/12)` shards are the slowest
check in every fully green PR run created since its merge (518s to 548s;
`Build` is 482s to 504s). Each shard is about 60s of setup and 454s to
490s of Vitest, with one worker, so its wall time is the sum of its
suites.
> - From two shard logs, 22 to 24 of the 71 to 74 suites in a shard
start an embedded Postgres cluster. Each one runs `initdb`, starts the
cluster, and applies all 331 migrations before its first test. The
smallest such suite takes 6s with one or two trivial tests. On top of
that, the `getEmbeddedPostgresTestSupport` probe starts and stops a
second throwaway cluster at import time in 19 of those suites, which is
why their per-file import overhead is 3.5s to 4.1s against 0.9s to 1.0s
for suites without Postgres.
> - A cluster data directory that was shut down cleanly can be copied
and started again as an independent cluster, and the migrated schema is
the same for every suite.
> - This pull request builds one migrated cluster per host, publishes
its data directory as a template under the OS temp directory, and starts
every later suite from a copy of it. The probe reuses the template
instead of starting a cluster.
> - The benefit is about 7s to 8s per Postgres suite on the CI runners
(measured 3.2s to 0.35s per start and 0.75s to 0.03s per probe locally),
about 120s to 170s per server shard, and a faster local test loop for
every suite that uses the fixture.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The `server (n/12)` PR checks are the slowest check in every fully green
PR run since 2026-10-11 05:27Z. Their Vitest step runs suites one at a
time, and about a third of each shard's time is embedded Postgres setup
that produces the same migrated cluster every time.
**Subsystem affected**
Test infrastructure: the embedded Postgres test fixture in
`@paperclipai/db` (`packages/db/src/test-embedded-postgres.ts`), used by
349 server suites, the db package suites, and the CLI suites.
**Current behavior**
`startEmbeddedPostgresTestDatabase` runs `initdb`, starts the cluster,
creates the database, and applies all 331 migrations for every suite.
`getEmbeddedPostgresTestSupport` starts and stops a second throwaway
cluster once per test file, because the module-level cache does not
survive Vitest's per-file isolation. Measured locally: 3.2s per suite
start plus 0.4s cleanup, and 0.75s per probe. On the CI runners the
smallest Postgres suite takes 6s and the probe adds 2.5s to 3s of import
time per file.
**Proposed behavior**
The first caller on a host builds one migrated cluster, stops it, and
renames its data directory to
`$TMPDIR/paperclip-embedded-postgres-templates-<uid>/<key>`. Every later
caller copies the template (about 90 MB) and starts Postgres on the
copy, then still runs `ensurePostgresDatabase` and
`applyPendingMigrations`, which are no-ops on a current template. The
probe treats a published template as proof that the host can run the
cluster. The key hashes the embedded-postgres version, the initdb flags,
the user, and the name and content of every migration file plus the
journal, so a changed migration builds a new template.
`PAPERCLIP_TEST_POSTGRES_TEMPLATE=0` bypasses the cache. Any template
failure falls back to the fresh path.
A template is a Postgres data directory, and starting it executes its
configuration, so the cache is private to the current OS user: the root
name carries the uid, the root is created with mode 0700, and both the
root and each template are used only when `lstat` shows a real directory
(not a symlink) owned by the current user with no group or world write
bit. A root that fails those checks (for example one pre-created by
another user in a shared `/tmp`) is left alone and the process uses
fresh clusters.
**Reason and benefit**
The slowest check decides the wall-clock time of every PR run. Measured
locally, a template start takes 0.3s to 0.4s instead of 3.2s, cleanup
0.15s instead of 0.43s, and the probe 0.03s instead of 0.75s. Four
server suites that passed in both modes dropped by 3.4s to 6.0s each;
the db `backup-lib` suite dropped from 42.7s to 20.9s. The CI runners
are about twice as slow, so the expected saving per server shard is 120s
to 170s.
**Breaking changes**
None. The returned database is fully migrated in both paths, each suite
still gets its own cluster and data directory, and a failure on the
template path falls back to the existing fresh path.
## What Changed
- `packages/db/src/test-embedded-postgres.ts`: adds the template cache.
`computeEmbeddedPostgresTemplateKey` hashes the inputs that determine a
migrated cluster. `buildEmbeddedPostgresTemplate` starts a fresh cluster
through the existing bounded retry, creates the database, applies
migrations, waits for the real `stop()` (30s bound), and publishes the
data directory with an atomic rename; a concurrent builder that loses
the rename discards its copy.
`startEmbeddedPostgresFromTemplateWithRetry` copies the template into a
fresh temp directory, sets mode 0700, and starts the cluster with the
same per-attempt port and directory rotation as the fresh path.
`startEmbeddedPostgresTestDatabase` tries the template first and falls
back to the fresh path with one warning per process.
`probeEmbeddedPostgresSupport` returns supported when a template exists
or can be built, and keeps its throwaway-cluster path for the case where
the template could not be published. Templates for other keys are pruned
after seven days. The connection strings and initdb flags move into
shared constants; no other behavior changes.
- `packages/db/src/test-embedded-postgres.test.ts`: two unit tests for
the key (stable for identical inputs; changes with migration content, a
new file, flags, version, or user; hashes this package's real migrations
directory) and two real-cluster tests (the published template is
complete and private, a suite starts from it in `template` mode with
migrations up to date, a table written to one copy is not visible in the
next copy, and `PAPERCLIP_TEST_POSTGRES_TEMPLATE=0` runs the `fresh`
path). The real-cluster tests skip on hosts without embedded Postgres,
like every other fixture suite.
- `doc/DEVELOPING.md`: a short section under Test Commands describing
the template, its key, and the bypass variable.
**Review fixes (second commit), one per finding:**
- Private cache root (Greptile P1, Superagent P2). The root is
`$TMPDIR/paperclip-embedded-postgres-templates-<uid>` (user name on
Windows), created with mode 0700. `prepareTemplateRootDir` and
`isUsableTemplate` both reject, via `lstat`, anything that is a symlink,
not a directory, owned by another uid, or writable by group or others; a
rejected root logs one warning and the process uses fresh clusters. A
root created by the first commit with a wider mode is narrowed to 0700
if this user owns it.
- Template failures never skip suites (Greptile P1).
`getEmbeddedPostgresTemplateDir` no longer rejects. A build or root
failure resolves `null` with one warning, and
`probeEmbeddedPostgresSupport` then runs the throwaway-cluster probe, so
only a cluster that cannot start reports `supported: false`. A migration
that cannot apply fails the suite on the fresh path instead of being
hidden by `describe.skip`.
- Idle-based pruning (Greptile P2). Each reuse refreshes the template's
timestamp with `utimes`, so the seven-day prune measures idle time. A
template that disappears after a process resolved its path is not
retried five times: `startEmbeddedPostgresFromTemplateWithRetry` checks
the template before each copy and throws at once, the suite falls back
to a fresh cluster, and the fallback forgets the path so the next caller
in that process rebuilds the template.
- The template-path test enables the cache for itself and restores the
original `PAPERCLIP_TEST_POSTGRES_TEMPLATE` value afterwards (Greptile
P1), so the suite passes under `PAPERCLIP_TEST_POSTGRES_TEMPLATE=0` as
well.
- Eight new tests: `isUsableTemplate` accepts a private cluster
directory and rejects a missing `PG_VERSION`, a leftover
`postmaster.pid`, a symlink, and (POSIX) group- or world-writable modes;
a world-writable pre-created root and a symlinked root both fall back to
fresh clusters, publish nothing, log one warning, and still report
`supported: true` from the probe; a ten-day-old template is refreshed on
reuse; a pruned template is rebuilt on the next start; and a template
build that fails after the cluster started falls through to the fresh
probe. The tests that need an empty cache point `os.tmpdir()` at a
scratch directory (`TMPDIR`, `TEMP` and `TMP` together, so the redirect
holds on Windows as well) and restore it.
**Review fixes (third commit,
|
||
|
Devin FoleyandPaperclip
|
4b59e2a2ec |
fix(server): allow idle sleep with saved AI auth and verified lease history (#15897)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Automatic idle sleep must preserve accepted work, cleanup and recovery. > - Saved AI login connections are configuration without independent workers. > - Failed ephemeral leases can remain after their resources have stopped. > - Detached subscription observations and quota refreshes can outlive a response. > - This pull request distinguishes inert configuration and verified history from unfinished work. > - The benefit is safe idle eligibility while keeping credentials and audit history. ## Linked Issues or Issue Description **What happened?** An active saved AI login blocks idle sleep after its login and agent runs finish. A failed ephemeral lease also blocks after successful cleanup and provider termination. The connection guard currently hides detached quota and subscription work that can outlive the request that started it. **Expected behavior** An idle instance can sleep with saved AI credentials and verified failed-lease history. Accepted refresh work, recovery journals, unfinished login sessions, pending cleanup and run accounting must still block. **Steps to reproduce** 1. Save an AI connection with purpose `ai`, transport `runtime_auth` and status `active`. 2. Finish login and all agent work, then acquire an owned idle drain. 3. Read the idle safety report. The saved connection alone reports background work. 4. Repeat with a failed ephemeral lease whose exact provider termination and finished run are recorded. Its history also reports background work. **Paperclip version or commit** Based on `0bbd586a2f6`, after updates from master. **Deployment mode** Single-process managed or self-hosted server using the owned idle-drain protocol. Related work: #15522, #15761 and #15851. Searched open and closed pull requests for related idle-sleep changes. The open accounting recovery work in #15879 is separate; this change does not settle or discard accounting debt. ## What Changed - Exempt active saved AI runtime-auth configuration from the connection sleep check. Other connection types and unknown states keep their existing checks. - Track detached subscription discovery and provider observations until their actual promises settle. - Track underlying quota work through an HTTP timeout, including an OAuth exchange and the save of its replacement credential. - Inspect the credential-recovery directory without reading credential bytes. Journals, probes, temporary files, unexpected entries and read failures keep sleep blocked. Empty company directories left by successful recovery permit sleep. - Exempt a failed ephemeral lease only after successful cleanup, a release timestamp, an exact company/run/lease/resource termination receipt, and a finished run without retry or accounting debt. Export and cleanup intents still block. - Use the same failed-lease proof when checking finished login-session cleanup. - Add PostgreSQL and filesystem regression tests and document the behavior. ## Verification - Four focused suites passed: 186 tests covering durable guard decisions, local recovery, detached subscription work and quota timeouts. - `pnpm build` passed. - `pnpm -r typecheck` passed. - `pnpm --filter @paperclipai/server build` and a final server TypeScript check passed after the directory-scanner cleanup. - The final directory-scan recheck passed all 18 tests. - The original-base `pnpm test:run` attempt found three local failures. It was stopped before completion to update the branch after two master changes. All three also reproduce on unchanged master `e733bddc45e`: two Git default-branch fixtures in `workspace-runtime.test.ts` and one native runner recovery fixture in `native-session-resume.test.ts`. - Both Git fixtures pass with a command-scoped `init.defaultBranch=master` setting. The native fixture fails with the same result on both branches. No global Git setting was changed. - All CI gates passed on the final head `ff63497d9bef40f5f00567bc729fb17536cfe136`: [complete CI run](https://github.com/paperclipai/paperclip/actions/runs/38156312722). The only skipped checks are the two path-gated Storybook jobs. - Greptile Apex scored the final head 5/5 with no actionable findings and no unresolved review threads. The branch has no merge conflicts with master. - After the master update, `pnpm build` and `pnpm -r typecheck` passed again. All four focused suites passed again: 186 tests. - Staging deployment and live sleep/wake verification are pending the Core merge and the verified private Cloud build. No production rollout has been started. ## Risks - The failed-lease exception is deliberately narrow. Older records without exact termination evidence or a complete run receipt continue to block. - Pending OAuth recovery blocks sleep until it is recovered. No credential, history, cleanup obligation or accounting receipt is deleted. - The existing owned drain, ingress fence, local-work generation checks, plugin drain and backup checkpoint remain required. - No schema migration or provider mutation is introduced. Roll back to the previous application release to restore the conservative guard behavior. ## Model Used OpenAI GPT-6 (Codex), with repository inspection, code editing, test execution and read-only provider analysis. No additional model version or context-window size was supplied by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved comments - [x] I will address all reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1011.0-canary.9 |
||
|
Devin FoleyandPaperclip
|
0bbd586a2f |
fix(server): wait for workspace restore before shutdown lease release (#15857)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Server shutdown must stop agent execution and preserve its files. > - Legacy adapters export workspace and instruction changes during settlement. > - Shutdown released their sandbox leases before this export finished. > - The provider closed command access, so the later Git export failed. > - This change stops an owned adapter through its cancellation control and waits for settlement before release. > - A bounded timeout retains the exact sandbox source behind the existing repair gate. ## Linked Issues or Issue Description **What happened?** A server restart interrupted a remote legacy run. Shutdown began releasing its sandbox lease while the adapter was still stopping its provider and restoring files. The sandbox provider closed command admission. The adapter's later Git export failed with `workspaceRestoreFailure: restore_failed`, leaving the task held for workspace repair. **Expected behavior** Shutdown keeps the exact lease available through provider stop, instruction collection, and workspace restore. Cleanup follows adapter settlement. A restore that exceeds the bounded deadline keeps its exact source for repair before later execution. **Steps to reproduce** 1. Start an owned legacy adapter with a sandbox workspace. 2. Send SIGTERM to the server while the adapter is active. 3. Delay adapter settlement until after the shutdown terminal write. 4. On the base branch, shutdown releases the lease without cancelling or joining the adapter. Its later export can encounter closed provider admission. Related: #14311 defines the provider-stop boundary for file collection. #15042 closes run admission during shutdown. This fix orders sandbox release after the adapter's existing file settlement. ## What Changed - Cancel each owned legacy adapter and join all affected runs within one shared 30-second window. - Commit a durable fence on the exact sandbox before exposing interruption. Other cleanup workers defer while export is pending. - Permit the exact aborted owner's stop-only request so shutdown does not block provider termination. - Reload current execution evidence after settlement. Release leases even when the adapter's ordinary terminal write wins the shutdown race. - Claim the current fence before recording timeout retention. An old snapshot cannot create a repair hold after successful settlement. - Recover fenced sources after controller loss, including released stop receipts. Block reuse or destruction of the physical allocation while file recovery is required, including direct resume and destroy calls with historical lease snapshots. Convert failed ordinary cleanup into stop-only recovery after its existing attempt settles. - Allow 90 seconds for shutdown in shipped Compose services. Preserve native runner restart ownership and unregistered process behavior. - Add database regressions for restore order, timeout, concurrent cancellation, another cleanup worker, controller loss, stale snapshots, stop-only settlement, and completion races. Document the shutdown order. ## Verification - The original ordering regression failed on the base implementation. - Latest recovery and environment runtime suites: 190 tests passed. The other four focused suites passed 136 tests, covering run control, shutdown, file repair, server startup, and environment runtime. - Server typecheck passed. Full `pnpm -r typecheck` and `pnpm build` passed on the latest head. - `pnpm test:run` was run locally. Its general-server group passed 18,559 tests and failed seven, with one unhandled error. The long run started before the review fixes, then loaded newer regression cases against older transformed modules. The current focused suites pass in a fresh process. The damaged-prior-epoch native-session-resume failure also reproduces on unchanged base commit |
||
|
Devin FoleyandPaperclip
|
5847f047c7 |
fix(connections): retire requests for already available tools (#15856)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connection cards let users provide the access an agent needs. > - Users can complete setup in Apps while an earlier card stays pending. > - A stale card can then cancel recovery despite valid existing access. > - This change verifies existing access and expires the obsolete request. > - It records a system outcome and sends one durable continuation. > - Approval, identity, permission, and workspace recovery gates remain in place. ## Linked Issues or Issue Description **What happened?** An ordinary tool connection request stayed pending after the addressed user connected the service in Apps. A later conversation retry was cancelled with `issue_waiting_for_response`. **Expected behavior** A request expires when the same user and requesting agent already have usable access. The agent gets a durable tool refresh. The system records no new human approval. **Steps to reproduce** 1. Have an agent request a tool connection on an active task. 2. Connect the service through Apps with the addressed user's existing grant, agent install, and allowed tools. 3. Let a conversation run stop and retry. The pending card still blocks the retry on the base branch. Related: #12345 introduced the connection request flow. This fix uses its existing access checks and delivery queue. ## What Changed - Verify company, task ownership, user identity, membership, health, grants, installs, and task tool policy before expiring an ordinary request. - Preserve AI account, channel, additional permission, and upstream provider requests on their existing paths. - Save the system outcome, audit event, and continuation in one transaction. Notify an idle delivery worker when a request commits. - Recheck access and other questions or approvals before delivery, retry, final dispatch, and native projection. Refresh system observations for an authorized replacement account. - Rearm delivery when access changes before provider work starts. Verify the host's admission receipt against execution evidence before allowing retry without a repair incident. - Require the current delivery claim when marking an attempt complete. An old completion cannot overwrite a rearmed continuation. - Add embedded-Postgres regressions for permissions, policy, concurrency, restart, replacement accounts, both dispatch gates, and the delivery race. Keep genuine workspace repair holds blocking. ## Verification - Latest-head connection, wake-context, dispatch, and legacy recovery suites: 124 tests passed. The connection suite includes 44 real-Postgres cases. - Server typecheck passed. Full `pnpm -r typecheck` and `pnpm build` passed on the latest head. - `pnpm test:run` was run locally. Its general-server group passed 18,583 tests and failed 20. That long run started before the review fixes, then loaded new test cases against modules transformed before those fixes. The affected latest-head suites pass in a fresh process. - One full-run failure also reproduces on unchanged base commitcanary/v2026.1011.0-canary.8 |
||
|
DottaandPaperclip
|
e733bddc45 |
perf(chat): stop polling provider action and session sync queues (#15822)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat stores pending provider actions in durable database rows. > - Three action queues still scan every second when empty. > - These scans keep idle databases active. > - This pull request wakes those queues after a commit or at their next deadline. > - Empty queues stop querying while existing recovery rules remain in place. ## Linked Issues or Issue Description Refs #15808, #15802, #15625. Related sleep work: #15750, #15757. **What existing behavior does this improve?** Provider messages and notices, accepted GitHub webhook receipts, and Slack session status synchronization. **Current behavior** Chat maintenance scans these three queues every second. It also rechecks paused Slack sync rows every 30 seconds. **Proposed behavior** Reuse the transaction-aware action workers and existing chat_actions rows. Wake after commits. Restore unfinished work on startup. Use stored retry and stale-claim deadlines. Stop scanning when empty. Park paused Slack sync rows until an endpoint change. ## What Changed - Add three topics to the existing action worker registration. No tables, migrations, triggers, or external services. - Signal producers, claims, retry updates, terminal settlements, and operator resolution inside their owning transaction. Provider I/O remains outside transactions. - Share pending-state and deadline predicates between dispatch and scheduling. Exclude terminal history and uncertain provider sends. - Preserve GitHub receipt attempt fences and provider-send quarantine. Preserve Slack's 30-minute refresh while a session is processing. - Remove these workers from periodic app maintenance. Update the sleep inventory and add regression coverage. ## Verification - Passed: 13 focused tests across scheduler, app wiring, and real PostgreSQL integration. Covers actual commit wakes with deferred fast paths disabled, GitHub retries and startup recovery, stale-send quarantine, Slack publication-to-sync delivery, paused deadlines, and idle queues. - Passed: pnpm typecheck. - Passed: full chat integration and related coordinator suites, **1,274 tests across five files**. - Passed: pnpm build. - Greptile: **5/5**, no actionable findings, completed on `dc284cd275e5f6df624d60b27954421f2bcdbee2`. - Hosted CI: **54 checks passed**, two intentionally skipped Storybook checks, no failures or pending checks on `dc284cd275e5f6df624d60b27954421f2bcdbee2`. Provider responses are simulated. PostgreSQL, transactions, service dispatch, and scheduling are real. No live Slack or GitHub App was used. ## Risks - Writers must emit the existing process-local transaction signal. Independent database clients and direct SQL still need explicit integration or startup recovery. - Accepted GitHub receipts are covered. Finding callbacks GitHub never delivered still uses the separate provider recovery lane. - Other chat maintenance and persistent transports still run. This does not establish whole-instance sleep. - Existing claim, revision, credential, and authorization checks remain authoritative during concurrent foreground and scheduled dispatch. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1011.0-canary.7 |
||
|
DottaandPaperclip
|
30e338fba6 |
refactor(heartbeat): extract environment selection (#15825)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat execution selects an environment before it prepares a workspace and starts an agent. > - Workspace preparation now has its own module, but its selection policy still occupies the executor. > - Environment defaults, recovery bindings, chat isolation, and workspace contention must retain their order. > - This pull request moves that selection phase behind one typed function. > - The executor keeps dispatch gates and lease cleanup. Tests cover the public path and the new boundary. ## Linked Issues or Issue Description Refs #15807. This continues the heartbeat extraction series after workspace preparation. **Current behavior** The 5,599-line heartbeat service contains a 301-line environment selection phase inside run execution. **Proposed behavior** Move that phase to `heartbeat/environment-selection.ts`. Keep the same policy, statement order, errors, and public exports. **Reason and benefit** Readers can find selection policy in one module and follow the executor's phases without reading each policy branch. ## What Changed - Add `selectHeartbeatEnvironment` with typed run, task, policy, configuration, and service inputs. Return the values needed by configuration and workspace preparation. - Preserve native recovery bindings, environment precedence, managed sandbox enforcement, Kubernetes provisioning, native chat isolation, and shared workspace contention. - Reduce `heartbeat.ts` from 5,599 lines / 234,599 bytes to 5,314 lines / 220,784 bytes. The new module is 402 lines / 18,215 bytes. - Add six public-path characterization cases before moving production code and six boundary cases. Repair plugin-test teardown to drain active execution and call the database cleanup helper. - Assert the selected local environment in the browser fixture's real worker and document the new boundary. ## Verification - Before the move: 71 tests passed, including all six new public-path cases. Three plugin tests skipped when the host could not start their database probe. - After the move: all 112 focused tests passed across eight files, with no skips. These cover environment policy, local and plugin execution, contention, recovery bindings, workspace preparation, chat isolation, and low-trust directories. - Passed `pnpm -r typecheck` and `pnpm build`. - A normalized TypeScript AST comparison confirms the 301 moved lines and remaining executor statements are unchanged, apart from the new call and imports. - Browser acceptance: the task UI recovered automatically and the worker verified its selected local environment, preserved uncommitted files, and completed the task. The persistent-failure case passed in isolation: three total runs, then a blocked task that remained blocked after reload. - The first persistent-failure browser attempt missed the retry banner within five seconds, despite a persisted scheduled retry. Its unchanged isolated rerun passed. The trace is retained; the intermittent UI failure is not yet attributed to a root cause. No assertion or timeout was relaxed. - Greptile reviewed `20953dd164decb565bc8d85081aad31a23fd33e5`: 5/5, no actionable findings. - Hosted CI passed on `20953dd164decb565bc8d85081aad31a23fd33e5`: 52 successful checks and two intentionally skipped Storybook checks, with no pending or failing checks. This includes all general and serialized test groups, Runner verification, eight browser shards, typecheck, and build. - The duplicate local `pnpm test:run` was interrupted after hosted CI completed. It is not reported as a local full-suite pass. Local completed checks are the 112 focused tests, typecheck, build, and browser cases described above. ## Risks - This selection phase enforces execution restrictions. A wrong input or import could select the wrong environment or change retry behavior. Public-path tests cover refusal, redirect, contention, and dispatch; boundary tests cover recovery and isolation. - The helper can provision a managed environment row and add concurrency notes to the supplied task context. It acquires no execution lease. The executor retains its catch/finally boundaries and resource ownership. - Kubernetes selection is tested with real database state and bootstrap settings. These checks do not claim execution against a live Kubernetes cluster or hosted sandbox provider. - No schema, migration, API contract, or intended behavior change. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code editing, compiler checks, database integration tests, and browser automation. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1011.0-canary.6 |
||
|
|
de9ab8e7e0 |
perf(runner): compile the TypeScript surface once per build (#15853)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Pull requests to this repository run a large CI matrix. The slowest check sets the time a contributor waits for a green run. > - After #15849 hid the verifier's downloads behind package staging, the `Build` lane is still the slowest check in five of the eight fully green PR runs created after that merge (535s to 581s; the top server shard is 529s to 560s in the other three). `Build` is about 100s of setup, 160s to 176s of `pnpm build`, and 265s to 312s of the public install verifier. > - Inside `pnpm build`, every package after `@paperclipai/shared` waits on the Runner: its `build` script takes 76s to 80s, and for 65s of that it is the only job running. The chain runs `build:typescript` three times: once directly, then again inside `check:semantic-contracts` and `check:runner-workflow-traceability`, which rebuild first so they work standalone. The server's `prepare:runner-vendor` then runs the whole Runner build a second time, so the job pays for four redundant TypeScript passes. > - Each pass is two `tsc` projects, the eval-kernel rebuild, and the schema checks, 7s to 8s on the CI runner. The dist they rebuild is the dist the first pass just wrote; nothing between the passes changes it. > - This pull request calls the two check scripts directly after the chain's own `build:typescript`, the way `generate-replay-goldens.mjs --check` already does in the same line. The standalone `check:*` scripts keep their rebuild. > - The benefit is about 30s off the `Build` job's critical path, about 15s off the Typecheck job and every Docker or release build that runs the server build, with no change to what is checked. ## Linked Issues or Issue Description **Describe the enhancement** The `Build` PR check is the slowest check in most green PR runs. Its `pnpm build` step compiles the Runner's TypeScript surface four times: three passes inside one Runner `build`, and a second full Runner `build` from the server's `prepare:runner-vendor`. Only the first pass produces a dist that the later steps read; the other three rewrite the same output. **Why is this needed** The slowest check decides the wall-clock time of every PR run. In the eight fully green PR runs created after 2026-10-11 04:10Z, `Build` topped five of them at 535s to 581s. The Runner build sits alone on the `pnpm build` critical path for about 65s, and the server build repeats it. From the job logs of four of those runs, the second and third `build:typescript` passes cost 7s to 8s each, in both Runner builds, so about 30s per job. **Proposed solution** In the Runner's `build` script, replace `pnpm run check:semantic-contracts && pnpm run check:runner-workflow-traceability` with `node scripts/generate-semantic-contracts.mjs --check && node scripts/check-runner-workflow-traceability.mjs`. Both scripts read `dist/`, which `build:typescript` wrote earlier in the same chain, and the steps between them are `--check` runs that write nothing to `dist/`. Add a guard test so the chain compiles TypeScript exactly once and both checks still run after it. **Alternatives considered** - Skip the server's `prepare:runner-vendor` rebuild when `pnpm -r build` already built the Runner. That rebuild is what makes `pnpm --filter @paperclipai/server build` correct standalone (the Dockerfile uses it), and the server's build-script test pins the command. A freshness check would be a larger semantic change; this PR shrinks the rebuild instead of removing it, and the next cycle can measure what remains. - Make `tsc` incremental across the passes. It would still re-run the schema checks and the eval-kernel build each time, and would add build-info files to the package. - Change `check:semantic-contracts` and `check:runner-workflow-traceability` themselves to stop rebuilding. They are documented standalone commands (`pnpm --dir packages/paperclip-runner check:semantic-contracts` in the capability docs) and must work from a stale dist; unchanged here. ## What Changed - `packages/paperclip-runner/package.json`: the `build` script runs `node scripts/generate-semantic-contracts.mjs --check` and `node scripts/check-runner-workflow-traceability.mjs` directly, after its own `build:typescript`, instead of the `check:semantic-contracts` and `check:runner-workflow-traceability` wrappers that rebuild first. Every other step, its order, and the standalone `check:*` scripts are unchanged. - `server/src/__tests__/server-package-build-script.test.ts`: a new test in the suite that already pins the server's Runner vendoring reads the Runner's `build` script, resolves every `pnpm run` step, and asserts that exactly one step compiles TypeScript (directly or through a wrapper), that both check scripts are in the chain, and that both come after `build:typescript`. It fails against the previous script with `expected [ 'build:typescript', …(2) ] to deeply equal [ 'build:typescript' ]`. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/server-package-build-script.test.ts`: 9 pass (1 new). With the `package.json` change stashed, the new test fails as described above and the other 8 pass. - In `packages/paperclip-runner`, the new chain without the cargo step (`check:protocol-manifest`, `build:typescript`, `check:capability-contract`, `check:capability-inventory`, `check:protocol-coverage`, `generate-semantic-contracts.mjs --check`, `check-runner-workflow-traceability.mjs`, `generate-replay-goldens.mjs --check`, `generate-semantic-action-catalog.mjs --check`) exits 0: `Runner stress traceability passed: 44 findings, 12 workflows, 1 explicit exclusion.` and `Replay golden snapshots and parity summaries are current.` Cargo is not installed on the host that produced this PR, so `build:binary` was not run locally; it is unchanged and runs in this PR's `Build` job. - `pnpm run build:typescript` alone takes 5.8s to 5.9s on this host, so each removed pass is in the range the CI logs show. - This PR's own `Build` job runs `pnpm build` from the branch checkout, so its log shows the new chain in both Runner builds. Expected on that job: `pnpm build` about 130s to 146s (was 160s to 176s on the four measured runs), `Build` about 505s to 551s (was 535s to 581s). **Measured on the four post-#15849 `master` runs that motivated this PR** (`Build` job logs): | Run | `pnpm build` | Runner build in `pnpm -r build` | Extra `build:typescript` passes in it | Runner rebuild in server build | Extra passes in it | | --- | --- | --- | --- | --- | --- | | 38112764439 | 169s | 76s | 04:51:52 and 04:52:00 (about 14s) | 04:53:12 to 04:53:41 | 04:53:26 and 04:53:34 (about 15s) | | 38111191543 | 176s | 79s | 04:22:40 and 04:22:48 (about 14s) | 04:24:04 to 04:24:34 | 04:24:18 and 04:24:26 (about 15s) | | 38110914319 | 160s | 75s | 04:19:07 and 04:19:14 (about 14s) | 04:20:25 to 04:20:53 | 04:20:39 and 04:20:46 (about 14s) | | 38110897745 | 172s | 80s | 04:15:16 and 04:15:25 (about 15s) | 04:16:41 to 04:17:11 | 04:16:55 and 04:17:03 (about 15s) | ## Risks - Low risk. The same two checks run against the same dist, in the same position in the chain. The only difference is that the dist is not rewritten twice between them. - If a future step between `build:typescript` and the two checks starts writing to `dist/`, the checks would read that output instead of a fresh rebuild. The steps there today (`check:capability-contract`, `check:capability-inventory`, `check:protocol-coverage`) are `--check` runs that only compare generated spec files. - The standalone `check:semantic-contracts` and `check:runner-workflow-traceability` scripts still rebuild first, so documented usage is unchanged. - The guard test lives in the server's suite because that suite already pins the Runner vendoring contract and runs in the server shards on every PR. ## Model Used - Claude (Anthropic), model ID `claude-fable-5-1`, running as a Paperclip agent through the Claude local adapter with tool use (shell, file edits, GitHub CLI). The agent collected the CI data, made the change, ran the tests, and wrote this description. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>nightly/v2026.1011.0-nightly.0 canary/v2026.1011.0-canary.5 |
||
|
Devin FoleyandPaperclip
|
b36960b758 |
fix(server): permit verified question-delivery history to sleep (#15851)
Treat completed question-response deliveries as history only after their exact same-company handoff finishes. Preserve missing evidence, retries, accounting debt, and idle-sleep fences. Add a concurrent all-state question history index with interrupted-build repair. Keep migrations serialized without waiting snapshots that deadlock concurrent index builds. Validated by 118 focused sleep tests, six migration/snapshot tests, full repository typecheck, all CI gates, and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>canary/v2026.1011.0-canary.4 |
