Commit Graph
983 Commits
Author SHA1 Message Date
DottaandPaperclip 2554c93d0c merge: reuse canonical credential quorum fixture repair
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 23:32:08 -05:00
DottaandPaperclip 271d434c17 test(runner): reserve owned credential quorum fixtures
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 23:28:19 -05:00
DottaandPaperclip cf40bdc4b5 merge: retain canonical held-authorization fixture correction
Co-Authored-By: Paperclip <noreply@paperclip.ing>

* commit '021df20762d96bc58d90a1dfa0fdb884393b1d5d':
  test(runner): wait for held recovery authorization entry
2026-10-12 22:52:50 -05:00
DottaandPaperclip 021df20762 test(runner): wait for held recovery authorization entry
Preserve the original recovery error and five-second bound while waiting for the actual before-bootstrap callback. Keep every authority, process and durable-state assertion.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 22:51:22 -05:00
DottaandPaperclip f34a1c81a9 merge: compose current canonical runner prerequisite
Co-Authored-By: Paperclip <noreply@paperclip.ing>

* commit 'dae808fe02694d13d322c535944910813b21f6d6':
  perf(chat): stop polling failed run retries (#15871)
  docs(release): stable notes for the 2026.1012.1-beta.0 soak (v2026.1015.0) (#16053)
  Support test Cloud connector identities (#16033)
  fix(inbox): keep automatic recovery cleanup out of personal participation (#16063)
2026-10-12 22:32:47 -05:00
DottaandPaperclip dae808fe02 fix(runner): preserve bounded recovery fixture synchronization
Merge current master and reuse the durable-resume test envelope correction. Wait for actual callback entry under the existing recovery deadline and preserve original early errors.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 22:29:13 -05:00
DottaandPaperclip 12113e431e merge: reconcile task-workspace canonical prerequisite
Preserve Claude v7 effort, lease ownership, historical readers, and Pi25 source binding while composing canonical16a and task-workspace masterf329. Protocol/surfaces noemit and all generation checks pass.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 21:52:28 -05:00
DottaandPaperclip e168512f7b perf(chat): stop polling failed run retries (#15871)
Stop one-second scans of failed chat-run retries. Wake the existing retry queue after transaction commits and at saved retry or stale-claim deadlines. Preserve claim fences, authorization, recovery, and idempotent agent admission.

Resolve conflicts with the current chat-service extraction by applying the retry signals and shared deadline predicate in the failed-run retry module.

Verified with pnpm typecheck, pnpm build, all 1,328 chat and coordinator regression tests, and current-head CI. Greptile reviewed the final head at 5/5 with no unresolved threads.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 19:49:47 -07:00
DottaandPaperclip 16a53254c4 merge master and retain bounded install failure evidence
Preserve guarded test-home cleanup and both Storybook source reviews. Keep the installer deadline and all artifact/lifecycle/runtime assertions intact while retaining its failed npm progress.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 21:44:39 -05:00
56a0bf0d49 Support test Cloud connector identities (#16033)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its Cloud connector uses an environment-bound identity to call a
hosted broker.
> - The identity parser and enrollment flow accept development, staging,
and production.
> - A separate test broker needs its own environment identity.
> - This pull request accepts test identities and maps the test broker
to that environment.
> - Known hosted brokers still reject identities from other
environments.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Cloud connector configuration and self-host enrollment.

**Current behavior**

The app rejects an identity with environment `test`. Self-host
enrollment infers development for the test broker.

**Proposed behavior**

Accept `test` identities at `https://test.paperclip.app`. Infer test
during self-host enrollment. Reject development, staging, and production
identities at that broker. Continue to reject test identities at the
production and staging brokers.

**Reason and benefit**

Branch testing needs a separate broker so test identities and requests
stay in their selected environment. The existing identity, enrollment,
and signature flow can provide that isolation with a new allowed
environment value.

**Breaking changes**

Existing environment values and broker checks keep their current
behavior. Older app versions cannot use test connector identities.

## What Changed

- Add `test` to managed and self-hosted connector environment types and
validation, and to the board API enrollment status type.
- Infer test from its hosted broker and enforce the matching
environment.
- Test enrollment persistence, signed request environment/audience
claims, and broker mismatch rejection.
- Document test broker configuration in the development guide.

## Verification

- Connector, enrollment, and board API-client suites passed: 63 tests
across three files. Board UI typecheck and token gates passed after
synchronizing its enrollment response type.
- `pnpm -r typecheck` and `pnpm build` passed, including after rebasing
onto current master.
- `pnpm test:run` completed its general server stage: 17,867 passed, 105
skipped, and 9 failed across 5 files. The runner stops at that failing
stage, so it did not run subsequent workspace/serialized stages locally.
All nine failures also reproduce on the unchanged base commit in a
separate checkout. Five skill-related cases resolve an unrelated
ancestor skills directory; two workspace cases accept a remote main
branch that their fixture expects to be absent; two permission-cleanup
cases return macOS `ENOTEMPTY` instead of the expected permission error.
- Current-head GitHub CI passed, including full general and serialized
test suites, typecheck, build, runner checks, and browser e2e. Greptile
is 5/5 with no unresolved comments.
- `git diff --check` and Gitleaks passed.

## Risks

The new environment value is additive. The test broker must support test
identities before use. This change preserves existing signature
handling, managed identity overrides, and rejection of mismatched
identities at known hosted brokers. A local full-suite pass is not
claimed; the nine failures above also occur without this change.

## Model Used

OpenAI Codex, GPT-6, with reasoning and tool execution. The exact
serving model ID and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Dotta <34892728+cryppadotta@users.noreply.github.com>
2026-10-12 19:32:33 -07:00
83e13db8fa fix(inbox): keep automatic recovery cleanup out of personal participation (#16063)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The personal inbox uses issue activity to identify a person's
participation.
> - Issue reads can cancel stale recovery actions and record that
cleanup as the reader's action.
> - This adds unrelated tasks to the inbox and refreshes their activity
timestamps.
> - A workspace owner hold can also alternate between creation and
cancellation after a task finishes.
> - This pull request corrects cleanup attribution, excludes historical
read cleanup, and retains one exact owner hold.
> - Agents can read authorized issue activity to diagnose the same
problem themselves.

## Linked Issues or Issue Description

Refs #15742 excludes logs by the execution-recovery actor from activity
timestamps. This PR covers reader-attributed stale recovery cleanup,
personal inbox participation, exact workspace owner holds, and sandbox
issue activity access. I searched open issues and PRs for recovery inbox
and activity bridge fixes. I found no duplicate for this defect.

**What happened?**

Loading issue lists cancels stale recovery actions under the reader's
identity. The personal inbox treats those rows as user participation.
One completed issue had hundreds of such rows and no edits or comments
by the affected reader. A retained native workspace owner repeatedly
creates a verification action that read projections then cancel. Sandbox
agents cannot inspect the issue activity because the reverse bridge
rejects the route.

**Expected behavior**

Automatic cleanup belongs to the system and does not add an issue to a
reader's inbox. Historical cleanup rows remain auditable but do not
count as participation or recent activity. An unresolved physical
workspace owner keeps one stable verification action. Agents can inspect
activity only for issues their identity may read.

**Steps to reproduce**

1. Finish a task whose native run retains an unverifiable workspace
owner receipt.
2. Repeat workspace recovery checks and GET requests for issue detail,
compact lists, and recovery actions.
3. Observe new recovery action IDs and reader-attributed cancellations
in the activity history.
4. Query the personal inbox for that reader and observe the unrelated
task.
5. From a sandbox agent, request GET /api/issues/:id/activity and
observe Route not allowed.

**Paperclip version or commit**

Observed at f5346c05b9. Regression tests
use current master behavior.

**Deployment mode**

Authenticated hosted instance with native and legacy sandbox agents. The
inbox defect is independent of the adapter.

## What Changed

- Attribute automatic stale recovery cleanup to the recovery system
actor.
- Exclude exact historical read cleanup rows from personal inbox
membership, unread counts, and both activity timestamp queries. Keep all
audit rows and explicit user recovery decisions.
- Retain owner verification holds only while the exact company, run,
issue, and owner fingerprint still match. Apply the same rule to reads
and background recovery.
- Make repeated checks for the same owner hold leave its identity,
attempts, and timestamps unchanged. Keep physical takeover blocked.
- Allow GET issue activity through the sandbox bridge. Keep the existing
company and issue-read authorization and reject broader audit routes.
- Add regressions and update the runtime API and recovery documentation.
Refresh generated source references for the documentation edits. Update
the paired activity, inbox, and recovery maps.

## Verification

- Focused Vitest verification: 315 tests pass across five files. The
updated terminal owner hold regression also passes. The files cover
issue service, recovery routes, activity authorization, sandbox callback
bridge, and native workspace finalization.
- `pnpm -r typecheck` and `pnpm build` pass locally. The full general
and serialized unit test matrix passes in CI on the latest commit. The
duplicate local `pnpm test:run` was stopped after about 30 minutes once
that full CI coverage passed; it did not report a local full-suite pass.
- `node scripts/check-maps.mjs` passes for all paired areas and
capability IDs.
- Apex is 5/5 on commit 44c3787888. Its
access map finding is fixed and resolved. All CI checks are green on
this commit.
- The new regression repeats reads and background sweeps for done and
cancelled tasks. It checks one unchanged hold, zero physical copyback,
zero personal inbox participation, and system cleanup after the exact
receipt is cleared.
- Historical cleanup tests preserve explicit recovery decisions,
nullable metadata, unread counts, and the original audit rows.
- Authorization tests permit an authorized agent and reject
cross-company or issue-boundary reads. Bridge tests reject write
methods, extra path segments, and company-wide audit routes.

## Risks

Existing inbox items can disappear if their only participation evidence
was automatic read cleanup. No data migration or audit deletion is
needed. An unresolved workspace owner remains visible after task
completion; this preserves the physical writer fence. A stale or cleared
owner does not retain the hold. Issue activity access expands only the
sandbox transport allowlist and still uses existing controller
authorization.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
editing, shell execution, and test tools. The session identifies the
model family as GPT-6. A more specific runtime model ID and context
window size are not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Amy (Opus) <noreply@anthropic.com>
2026-10-12 19:25:15 -07:00
DottaandPaperclip f329b24ee1 feat: give tasks durable project-optional workspaces (#15749)
Give tasks durable company-scoped workspace bindings independent of project assignment, channel defaults and authorized repository preparation. Reuse verified input seeds and sparse output recovery while preserving trust, privacy and controller provisioning boundaries.

Full CI and E2E, Superagent, legacy checks and Greptile Apex 5/5 passed on reviewed head 753680b61c. All review findings are resolved; the PR records focused verification and historical acceptance limits.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 20:45:18 -05:00
DottaandPaperclip 48ffca9438 merge: reconcile published native prerequisite and current master
Preserve canonical startup ownership, scoped evidence, current Storybook IDs, and the Codex form controls while carrying Claude effort and immutable source bindings.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 20:40:11 -05:00
DottaandPaperclip bb765e9009 chore(runner): apply contribution rules with immutable source attestations
Retain Pi profile 24 and bind the updated sandbox source in profile 25. Provider closures and qualification boundaries are unchanged.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 20:34:26 -05:00
DottaandPaperclip 64c074fb8d docs: record runtime qualification and catalog review
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 20:33:39 -05:00
DottaandPaperclip 9f59ff36cf merge: reconcile runtime qualification with current master
Co-Authored-By: Paperclip <noreply@paperclip.ing>

* origin/master:
  fix(server): keep classified provider quota runs out of Sentry (#16029)
  refactor(chat): extract delivery, reaction, and setup boundaries (#16024)
  fix(heartbeat): answer an unsettled adapter Stop with a 409 instead of a 500 (#16032)
  refactor(storybook): organize folders and navigation by feature map (#16049)
  test: remove repeated setup and strengthen regression proof (#16022)
  feat: maintain Storybook through a downstream feature catalog (#16046)
  ci: route PR admission through reserved AWS capacity (#16042)
  docs(release): canonicalize v2026.1012.0 release notes (#16045)
  ci: add protected routing workflow for reserved AWS capacity (#16039)
  docs: add PR rules for file size and code comments (#16036)
  Add managed Slack setup with an independent own-app path (#15679)
2026-10-12 20:02:47 -05:00
DottaandPaperclip d69d94a0f8 docs: record startup ownership and production readiness gates
Keep feature and access maps synchronized with the scoped cancellation repair, and retain the exact source and unresolved installed/staging qualification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 20:02:06 -05:00
DottaandPaperclip 0ff31b2b76 fix(runner): version Pi attestation for owned settings initialization
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 18:29:22 -05:00
DottaandPaperclip 64bd6e9e07 refactor(chat): extract delivery, reaction, and setup boundaries (#16024)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Chat channels connect that work to provider conversations.
> - One shared service contains callback, delivery, reaction, and setup
logic.
> - These responsibilities have distinct contracts and existing
public-path tests.
> - This PR moves ten cohesive responsibilities into focused modules.
> - The service retains shared runtime ownership and scheduling.
> - Provider behavior is easier to locate and review.

## Linked Issues or Issue Description

Refs #15995, #16026, #16027, #16030, #16031, #16035, #16037, #16038,
#16041, #16043.

This PR consolidates the reviewed extraction stack. The plan is
`doc/plans/2026-10-12-chat-channel-delivery-boundaries-refactor.md`.

## What Changed

- Move Slack callback verification into the Slack folder.
- Move Discord Gateway events and root-thread admission/recovery into
the Discord folder.
- Extract durable receipt reactions and inbound reaction
admission/replay.
- Extract inbound attachment ingestion and outbound integrity-checked
hydration. Preserve the public hydration export.
- Extract durable provider effects. Keep Telegram callback notices in
the Telegram folder.
- Extract conversation/activity reads and setup qualification. Keep
Slack and Teams projections in their channel folders.
- Preserve the merged managed Slack setup behavior, shared workers,
credential leases, and transaction boundaries.
- Reduce `chat-channels.ts` from 26,493 lines / 1,048,769 bytes to
21,460 lines / 862,284 bytes. This removes 5,033 lines from the service.

## Verification

- Each extraction passed its selected existing tests before and after
through public service paths.
- On updated master, all 55 selected Slack cases passed before
extraction. All 194 selected chat/managed-Slack cases passed on the
final stack tip.
- `pnpm -r typecheck`, `pnpm build`, module-boundary checks, and
whitespace checks passed on that tip.
- The normalized syntax audit verifies 69 moved declarations and all
retained statements, including their relative order, against updated
master.
- The eight earlier stack merges retain the exact validated repository
tree. The combined root equals the validated tip merged with current
master. All chat-channel files remain byte-identical to the validated
tip. Current combined-root CI and review are pending.
- Full local verification remains unverified. The unchanged
`workspace-git-snapshot-streaming.test.ts:56` stress test timed out at
300,000 ms in a clean run and again in isolation. The full run was
stopped after the reproduced failure. No timeout or assertion was
weakened.
- These checks use assembled services, real local databases, and
provider fixtures. They do not prove live third-party execution.

Reviewers can run the public-path suites:

```sh
pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts server/src/services/chat-outbound-attachment-hydration.test.ts server/src/__tests__/slack-development-manager.test.ts
```

## Risks

- Factory wiring must preserve live values, shared state, and
initialization order. Identity-link construction retains deferred access
to the provider-effects callback.
- Credential fences, authorization transactions, and delivery settlement
must keep their original ordering. Syntax parity and public-path
integration cases cover these boundaries.
- Full local verification has the recorded Git-streaming timeout. The
combined PR must pass master’s required CI before merging into master.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and repository tools. The
exact runtime model ID and context-window size are not exposed in this
session. No separate agent was used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 22:46:25 +00:00
DottaandPaperclip 88bf4c6c1d refactor(storybook): organize folders and navigation by feature map (#16049)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The feature map describes product areas and verification entry
points.
> - Storybook has 223 story suites that show production components,
proposals, and retained history.
> - The parent PR maps those suites to features, but their folders and
sidebar still use unrelated groups.
> - This pull request puts the existing suites under feature folders and
uses the feature map's names and order in the sidebar.
> - Explicit meta IDs preserve existing story links while the titles
change.
> - Contributors can find stories through the same product structure
without losing examples.

## Linked Issues or Issue Description

Refs #16046. This is the requested follow-up. Parent #16046 is merged.
This PR now targets master, with its single follow-up commit rebased
onto the parent squash merge. The Storybook implementation is unchanged
by that rebase. Related open Storybook PRs were checked; this change
organizes existing suites and adds no product capability.

## What Changed

- Move all 223 story files into folders named for their primary feature
area. Keep useful scenario groups below the feature's sidebar label.
- Order the sidebar by the eight feature-map sections. Put Overview
first and shared Foundations last. Leave areas with no stories out of
the sidebar.
- Move 19 support files into `ui/storybook/support/` and update relative
imports and active documentation.
- Preserve all 2,210 story IDs and the single docs ID through explicit
CSF meta IDs. Keep all entry names and story exports.
- Record stable IDs and sidebar titles in the downstream catalog.
Validate feature folders, title prefixes, and IDs. Update the
maintenance skill and review evidence.

## Verification

- Storybook build passed. Built index comparison against parent
`75bd3c977cd828d5222823850339e8204f40eb47` found the same 2,211 entries,
IDs, and names.
- UI typecheck passed (`ui/node_modules/.bin/tsc -b` from `ui`, invoked
as `./node_modules/.bin/tsc -b`).
- UI Vitest project: 728 files and 8,176 tests passed.
- Catalog checker and its nine tests passed. Feature-map checks, token
gates, and the maintenance-skill validator passed.
- Chromium: nine existing play cases and one render case reached
`storyFinished: success` with no page exceptions. The guide rendered,
sidebar sections followed map order, and a search for `steering/queue`
opened its preserved story link.
- Source comparison found only meta title/ID changes, rebased relative
references, and updated fixture path text in story files.
- To inspect: build Storybook, start `node
scripts/serve-storybook-static.mjs --port 6136`, and open the Overview
guide. Expand feature folders and use existing story links.
- The complete repository suite was not repeated for this organization
change. The parent records a backend timeout that passed in isolation,
without a complete broad-suite pass. Visual comparisons remain blocked
by the unpublished empty baseline. The separate fixture-server story was
not run.

## Risks

- Open branches that add stories under old paths will need to choose a
feature folder and update the catalog. Cross-feature suites have one
primary home and retain related capability references.
- Sidebar paths change; story and docs IDs remain stable. Automated
references to source paths may need the new location.
- This is one requested follow-up with 258 changed files, mostly moves.
It exceeds the usual review-size guideline because splitting it would
leave the sidebar and folder convention partly migrated.
- Browser verification samples ten stories, not all 2,210. Existing
production/prototype classifications and review gaps remain unchanged.
- Required CI gates and a 5/5 review passed on the original head. Fresh
checks and review are running for the rebased head. Catalog validation
and all nine checker tests passed after rebase; UI, map, skill, and
checker contents are identical to the original reviewed head.

## Model Used

- OpenAI Codex, GPT-6. The exact runtime model ID and context-window
size are not exposed in this session. Used reasoning, repository
inspection, code execution, and local browser automation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 22:29:22 +00:00
DottaandPaperclip a583630e1c docs: record network policy and mapped acceptance evidence
Keep the source-bound fixture regression, runner preservation proof, required map synchronization and pending full gates in the existing dated record.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 17:24:50 -05:00
DottaandPaperclip dabb8c5daf test: remove repeated setup and strengthen regression proof (#16022)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its tests protect company boundaries, approvals, durable tasks,
credentials and the board UI.
> - Several owner suites repeat large database fixtures and render
frames.
> - Some tests repeat stronger proof or pass when the behavior they
describe is broken.
> - This pull request shares exact setup, removes proven duplicate cases
and repairs weak observations.
> - It preserves distinct security, lifecycle, concurrency and
user-facing contracts.
> - The benefit is 5,027 fewer test lines and stronger regression
checks.

## Linked Issues or Issue Description

Refs #15978. This continues the merged test-audit cleanup in [PR
#15978](https://github.com/paperclipai/paperclip/pull/15978), which
removed 11,976 test/support lines. The savings in this pull request are
additional.

Related open work: [PR
#13460](https://github.com/paperclipai/paperclip/pull/13460) changes
legacy OAuth parent/version lock ordering. This pull request retains
those production paths and the existing lock-order test. Its secrets
changes compact fixtures and repair the two AWS wrapper exclusion
observations.

Scoped title searches found no open duplicate audit or
fixture-consolidation PR and no matching audit issue. The problem is
described here using the enhancement template.

**What existing behavior does this improve?**

The maintenance and regression value of the existing server, UI, CLI,
shared and wiki-plugin test suites.

**Subsystem affected**

Cross-cutting test code. Two UI modules also lose unused test inspection
exports.

**Current behavior**

Owner tests repeat database rows, service inputs, HTTP setup and render
frames. Some cases repeat stronger assertions. Two AWS lock tests record
only cleanup events, so an AWS provider write can happen inside the held
lock without failing those observations. Several other negative tests
accept or do not observe the wrong result. Worktree fixtures also use a
temporary path alias that conflicts with the existing canonical-path
checks on macOS.

**Proposed behavior**

Use small local helpers that preserve exact inputs, fresh allocations,
callbacks, query order and await positions. Remove only cases with named
stronger keepers. Tighten negative assertions and observe actual AWS
provider entry while a confirmed lock holder remains parked. Resolve
temporary fixture roots to their canonical paths.

**Reason and benefit**

Remove 5,027 net test lines and 15 redundant registered executions.
Remove 11 lines of unused UI inspection exports. Keep every distinct
company, actor, approval, credential, persistence, stale-write and
concurrency contract. No wall-time speedup is claimed for setup
compaction.

**Breaking changes**

None for application behavior, APIs or stored data. The removed exports
expose private UI test state and have no remaining callers.

## What Changed

- Consolidate repeated runtime, workspace, portability, route,
tool-access, policy, gateway and state-service fixtures.
- Share repeated UI render frames and CLI/wiki-plugin setup while
retaining the original actions and expected results.
- Remove 15 repeated case executions only where the audit records an
exact stronger keeper.
- Repair weak HTTP/UI negative assertions and two AWS account-home lock
observations. Keep the existing durable results and cleanup behavior.
- Remove the unused `__inflightGetCount` and `__secondTickInternals`
inspection exports. Keep observable request coalescing and ticker
lifecycle checks.
- Canonicalize temporary worktree fixture paths without weakening
production path checks.
- Record the owner decisions, savings, independent preservation reviews
and validation in [the dated audit
report](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-rounds.md)
and [the compact control
evidence](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-controls.json).

## Verification

- Complete owner and retained-sibling baselines passed before each audit
cutover. Independent reviewers checked the full original tests, actual
helper bodies, exact inputs, receipt provenance, async order, fresh
allocations and case registrations.
- All 14 UI/route/wiki mutations failed at their intended assertions:
ten UI/API/ticker controls, one route control and three wiki
persistence/receipt controls. Each mutated production source was
restored to its exact original bytes.
- Both old AWS lock tests passed with their respective AWS-only lock
bypass. Both repaired tests failed at the intended assertions with the
same bypasses. These give 16 intended failures and two passing weak
controls across 18 executions. Each source was restored exactly.
- Final complete state-owner/sibling run: 329/329 passed at code
revision `64370831ebe191e5ce092f85f79b6fb661df7e16`.
- Final `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build`:
passed at the same code revision.
- Delivery head `58c5c4aa241c85f4665a3159c192888bbb2a5cfa` includes the
evidence commit and master `4b766b872993045a2e787404f0475503683e5ded`.
All 27 audited test files and both application files match the locally
verified code revision byte-for-byte. Current-head CI validates the
integrated branch, including the incoming master changes.
- All eight production-owner SHA guards for the 16 deliberate failing
mutation controls still match the integrated delivery checkout. The
mutation receipts therefore apply to those current owners.
- Full default `pnpm test:run`: **failed** with exit 1 at frozen code
revision `64370831ebe191e5ce092f85f79b6fb661df7e16`; HEAD and tracked
source bytes remained unchanged. The first general-server group reported
878 passing files, four failing files and six skipped files; 17,846
passing cases, 24 failing cases and 105 skipped cases. Later groups did
not run. Failures were `http-log-redaction` (socket hangup),
`public-mcp` (initial 15-second timeout followed by hosted token
“Refresh before subscribing” failures), `remove-test-home` (macOS
`ENOTEMPTY` where permission errors were expected, plus cleanup
failure), and `workspace-git-snapshot-streaming` (300-second timeout).
All four failing test files and the relevant cleanup/MCP production
owners are unchanged by this audit. Full local success is not claimed.
- Focused complete-owner diagnosis at the same frozen revision: 149/152
passed, three failed, zero skipped. `public-mcp` passed 82/82 and
`http-log-redaction` passed 66/66 unchanged. The two `remove-test-home`
cases reproduced macOS `ENOTEMPTY` against an expected `EACCES`/`EPERM`
contract; the 32 MiB workspace snapshot case reproduced its existing
300,000 ms timeout. HEAD and tracked files stayed clean. Exact blob
comparison confirmed all four test files and the relevant cleanup/MCP
owners match both inspected master
`4b766b872993045a2e787404f0475503683e5ded` and the frozen local
revision. The three remaining local failures are unresolved; no timeout,
retry or assertion was weakened.
- Current-head CI: [workflow
38299262149](https://github.com/paperclipai/paperclip/actions/runs/38299262149)
has passed runner selection and created all downstream
test/typecheck/build gates; those gates are queued, with no failures
reported. Greptile completed successfully at
`58c5c4aa241c85f4665a3159c192888bbb2a5cfa` with [5/5 and zero
comments](https://github.com/paperclipai/paperclip/pull/16022#issuecomment-6131457592).
All review threads are resolved or absent. This pull request will remain
open after verification.

Reviewers can reproduce the required workspace checks:

```sh
pnpm -r typecheck
pnpm check:token-gates
pnpm test:run
pnpm build
```

Browser suites were not run. Their sources and user journeys are
unchanged by this audit.

## Risks

- A fixture helper can hide a specialized input or change object
freshness. Full-program expansion checks and independent reviews
preserve the exact old inputs and execution order.
- Removing duplicate proof can remove a distinct contract by mistake.
The audit records the keeper for each deletion; all distinct security
and lifecycle cases remain.
- Concurrency checks can miss progress during a fixed delay. The AWS
repairs record actual provider entry, confirm holder entry, use measured
entry windows, require exact final event order and drain both operations
in `finally`.
- The full local default run failed. Focused diagnosis reproduced two
macOS cleanup failures and one existing snapshot timeout in unchanged
owners; these remain unresolved. Current-head CI is pending. No test
deadline, retry policy, skip or CI shard was weakened to obtain a pass.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-6, reasoning, repository inspection, code
execution and delegated audit/review. The runtime does not expose the
exact served model identifier or context-window size; those values are
unavailable and are not inferred from the model catalog.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 22:23:11 +00:00
DottaandPaperclip 9e721264e9 feat: maintain Storybook through a downstream feature catalog (#16046)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Storybook supports review of its production UI and proposed designs.
> - Its 223 story files have no consistent link to product capabilities
or record of component and mock provenance.
> - Old compositions can drift while they still look like current
product examples.
> - This pull request adds two maintenance skills and a catalog that
consumes the existing feature map.
> - The first audit preserves all stories and fixes proven fixture
drift.
> - Contributors can now find examples by feature and see what still
needs review.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Storybook maintenance, feature discovery, and the accuracy of isolated
UI examples.

**Current behavior**

Stories are already centralized under `ui/storybook/`, but their sidebar
has 51 top-level groups. Production components, local prototypes, and
historical designs are not consistently distinguished. The old guide
uses manual coverage claims. GitHub searches found no duplicate
feature-map catalog change.

**Proposed behavior**

Keep existing story locations and IDs. Add a downstream catalog keyed to
stable feature capabilities. Record product status, rendered subject,
data boundary, and declared review dependencies separately. Use
dedicated maintenance and review skills to reconcile drift and report
evidence.

**Reason and benefit**

The catalog makes existing examples reusable without treating mock
behavior or old designs as production acceptance. The feature map
remains independent of Storybook.

## What Changed

- Add `maintain-storybook` and `review-storybook` skills with
contributor guidance.
- Inventory all 223 story files. Generate a feature index and provide
searchable navigation in the existing Overview story. Track direct
subjects through relative imports and local wrappers, including the
primitives demonstrated by foundation suites.
- Add a checker for inventory, capability references, duplicate IDs, and
declared-source drift, with eight regression tests. The parser test
covers real nested CSF files, export filters, explicit IDs, and play
counts.
- Preserve historical designs and classify mixed wrappers. Record the
initial audit and remaining parity work.
- Align the question-dismiss/send fixture with production behavior. Add
dismissal and confirmation-reopen regressions and repair the editor
selector.
- Exclude three exported fast-response fixtures from story discovery and
add the missing render-time callback spy.

## Verification

- Passed: `node scripts/storybook-catalog.mjs`, `node --test
scripts/storybook-catalog.test.mjs` (8 tests), and `node
scripts/check-maps.mjs`.
- Passed: validators for both new skills, `pnpm check:token-gates`,
`pnpm -r typecheck`, and `pnpm build`.
- Passed: final Storybook build; nine selected browser `play` cases;
three fast-response render cases; feature-guide search/link navigation;
question-fixture reset on reload.
- Built-index comparison preserved all 2,208 original story IDs and
added only two new regression cases.
- Passed: `pnpm exec vitest run --project @paperclipai/ui` (728 files,
8,176 tests).
- The full `pnpm test:run` has reported a 15-second timeout in the
untouched backend case `connects advertised Workspace gmail.read without
mutating definitions in authenticated`. The exact test passed standalone
without changing code, expectations, or timeouts. The remaining broad
run was stopped after this failure. The full suite is not verified.
- Visual comparison is blocked by the existing empty baseline manifest.
The managed-accounts API/DB fixture was not started. See
`maps/storybook-map/REVIEW.md` for exact scope.

## Risks

- This is an initial inventory and targeted audit. 188 suites still
require deeper state and fixture parity review.
- Review hashes cover declared dependencies. They do not follow all
transitive production imports or prove semantic correctness.
- The catalog requires upkeep when stories, mapped recipes, or declared
subjects change. Shared mocks still need feature-by-feature contract
review.
- No production application source, database schema, or upstream
feature-map contract changes.

## Model Used

OpenAI GPT-6 via Codex. Used reasoning, shell tools, file editing, and
Chromium browser verification. The exact runtime model ID and context
window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 17:20:00 -05:00
DottaandPaperclip eee0534154 docs: record Codex configuration review and restart proof
Retain source-bound regression and preview evidence, current pending gates, and the independent prerequisite split.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 16:57:55 -05:00
DottaandPaperclip f5346c05b9 ci: route PR admission through reserved AWS capacity (#16042)
## Thinking Path

> - Paperclip uses CI to verify changes before release.
> - Trusted PR tests can run on isolated AWS workers.
> - Their runner-selection job currently waits for GitHub-hosted
capacity.
> - A large hosted test queue therefore blocks admission to AWS.
> - A dedicated routing workflow lets us reserve capacity for metadata
validation.
> - This PR moves the routing job to reserved AWS capacity after the
bootstrap workflow and pool are available.

## Linked Issues or Issue Description

**Issue type**

CI reliability enhancement.

**Where is the issue?**

The Select trusted runner job in PR CI.

**What's wrong?**

The routing job competes with a large hosted test queue. Its sampled
median wait was about 20 minutes. AWS tests cannot begin before this job
finishes.

**Suggested fix**

Add a protected reusable routing workflow. Deploy its dedicated AWS
pool, then switch the PR caller in a follow-up PR. No overlapping open
PR or issue was found.

## What Changed

- Call ci-routing.yml on protected master from pr-trusted.yml.
- Check the canonical repository and approved PR author/sender before
allocating reserved AWS metadata capacity. Unapproved PRs continue to
use hosted routing.
- Permit approved manual routing canaries on protected master.
- Use a validated roster snapshot for placement. Invalid JSON, missing
settings, stale snapshots, and unapproved identities select hosted
routing without a placement-time JSON parse.
- Preserve the full in-job identity, rerun actor, repository, head,
merge, and stack checks.
- Update stack-scope and placement regression tests and document the
pool and rollback.

## Verification

- The routing scripts and placement expression tests passed (44 tests),
routing-plus-shard tests passed (56 tests), and the existing
.github/scripts suite passed (381 tests).
- Existing routing allow/deny scenarios passed with the extracted script
(16 cases).
- actionlint passed with the dedicated custom runner label declared in
its local configuration.
- git diff --check passed.
- Full pnpm typecheck/test/build cannot run in this worktree because
dependencies are not installed. The change is limited to workflow YAML
and Node tests.
- The dedicated pool is deployed with two warm On-Demand workers and a
four-worker ceiling. The routing qualification passed: it started about
five seconds after workflow creation, selected AWS, and finished while
the hosted gate in the same PR remained queued.
- Bootstrap https://github.com/paperclipai/paperclip/pull/16039 is
merged. The deployed group now authorizes only ci-routing.yml on
protected master, and its master canary passed.

## Risks

- The maintainer explicitly approved a queued-CI admin bypass after
current-head Greptile 5/5 and focused verification.

- Activation requires the deployed routing Fleet and its group to
authorize the routing workflow on protected master. Keep the bootstrap
dependency explicit.
- The group must authorize only ci-routing.yml on protected master and
canonical repository ID 1170821064.
- The routing workflow runs no checkout or PR code and has read-only
permissions. The test admission checks are unchanged.

## Model Used

- OpenAI GPT-6 via Codex, with reasoning, shell tools, and file editing.
The exact runtime model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused routing and
workflow tests; full pnpm commands are blocked by missing worktree
dependencies)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] Required CI status checked; the maintainer explicitly approved
admin bypass for queued CI after clean current-head review and focused
verification
- [x] Greptile is 5/5 on 993fbe695a with
no open findings or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 16:47:57 -05:00
DottaandPaperclip 5415f33d4e docs: record final environment metadata review closure
Keep source identities, real PostgreSQL race controls, runner preservation and pending current-source/cloud gates inspectable in the existing dated record.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 16:42:35 -05:00
DottaandPaperclip 64d0746f8a test(agents): distinguish measured and unknown sandbox defaults
Keep the discovery fixture truthful for unknown platforms and add a measured GNU2.39 ARM64 control. Record existing managed fixture, cost reporting and route qualification evidence without claiming missing live acceptance.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 16:30:25 -05:00
DottaandPaperclip 65700c6908 fix(agents): preserve legacy policy and require measured sandbox defaults
Keep explicit Legacy permission choices through automatic selection, import editing, draft reset and policy display. Use company-scoped measured immutable-image facts for automatic sandbox defaults without allocating resources during creation or discovery.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 16:19:00 -05:00
DottaandPaperclip a1bfe8f5be Merge reviewed runtime and measured-platform fixes before Codex defaults
Co-Authored-By: Paperclip <noreply@paperclip.ing>

* codex/runner-native-platform-prerequisites:
  fix(runner): close review gaps in setup and platform qualification
2026-10-12 16:14:48 -05:00
DottaandPaperclip 2a19f3721d fix(runner): close review gaps in setup and platform qualification
Read inert exact-source target data before selecting hosted native jobs. Bound registration teardown and retain unconfirmed recovery state. Record company-scoped sandbox platform facts only for immutable OCI images after confirmed cleanup; preserve existing connection and runner behavior.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 16:14:34 -05:00
DottaandPaperclip f0e12c8519 Add managed Slack setup with an independent own-app path (#15679)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Slack connects a person to an agent through an installed app and a
verified identity.
> - Independent installations must remain usable without Paperclip Cloud
or manager credentials.
> - Approved Cloud deployments can use Slack manager grants to create
and install each agent's app.
> - Both paths should share credential storage, identity checks,
recovery, and the existing runtime.
> - This pull request adds gated managed setup and keeps independent app
setup under Advanced.
> - It also makes ongoing connection settings and personal test messages
easier to use.

## Linked Issues or Issue Description

**Subsystem affected**

Slack setup, connection settings, tenant credential storage, and the
Cloud connector handoff.

**Problem or motivation**

Approved deployments need an installation flow without pasted
configuration tokens. Open-source users must retain their independent
Slack app flow. A connection manager also needs a simple way to check a
bot from Settings.

**Proposed solution**

Use the managed path when the instance is configured to support it.
Choose agent and Connect are the two visible stops. Continue starts
authorization or installation. Use your own app remains under Advanced
before creation. Save personal manager grants by company, Paperclip
user, Slack user, workspace, and manager app. Reuse only verified
identities for test messages. Unknown identities must complete a private
Slack connection challenge before linking and receiving the test DM.

**Alternatives considered**

A shared bot would change each agent's Slack identity. Requiring Slack
CLI or a hosted service for self-hosted users would remove independent
installation.

**Roadmap alignment**

This maintainer-requested change improves Connected Apps and
experimental Agent Chat. It adds provisioning and management controls to
the existing Slack runtime.

**Additional context**

The companion Cloud broker profile is a separate deployment
prerequisite. Managed setup remains gated. Existing-app adoption is
deferred.

## What Changed

- Extract Slack registration, provider requests, completion,
customer-owned setup, managed setup, and personal manager grants into
focused services. Preserve the compatibility facade and independent
removal cleanup.
- Add company-scoped manager grants and a registration reference through
an additive, idempotent migration after current master. Keep runtime
credentials separate from app-registration secrets.
- Add authenticated manager authorization, sealed handoff, serialized
refresh and revocation, app creation, managed installation, and
installer-bound OAuth fallback.
- Preserve creation intent, block automatic uncertain recreation, save
credentials before downstream checks, and validate app, workspace, bot,
scopes, and installer identity.
- Add the two-stop production wizard and ordered stories. Keep the
own-app, manual, existing-app, and legacy-draft paths.
- Simplify Settings. Open Slack configuration targets the saved app. The
test-message action has its own row. Remove the old manual-avatar
section, channel-message section, and broken permissions link.
- Add paginated account discovery and a private connection challenge for
unknown accounts. Expired invitations stop waiting and offer a new
private invitation. Save send receipts before provider dispatch and
preserve confirmed delivery when activity logging fails.
- Reinstall the saved managed app on explicit reconnect, replace revoked
credentials, and resume completion. Persist reconnect intent and
duplicate-request receipts without creating another app.
- Update provider browser tests, recovery tests, secret redaction, setup
documentation, and the agent prompt. Preserve current master's durable
verification scheduling.

## Verification

- Full workspace typecheck and repository build passed on `56d389d47a`.
The subsequent reconnect retry fix in `288364eada` also passed UI
typecheck and UI build.
- 114 focused Slack registration cases and 13 production managed-wizard
cases passed, including fresh-token reconnect, retry after installation
failure, and duplicate reconnect requests.
- All nine affected Slack browser cases passed against the production UI
and a disposable fixture server. They cover managed setup, automatic
setup, own-app selection, refresh, recovery, mobile behavior, and
connection settings.
- Eight focused real-database test-message cases passed. They cover a
private ownership challenge, saved grant reuse, account pagination,
revoked and conflicting links, workspace mismatch, disabled DMs, revoked
membership, audit failure, ambiguous dispatch, and secret canaries.
- UI token gates and the production Storybook build passed.
- Greptile reviewed exact head
`288364eada5d9c114fad991ff634612639d06672` at 5/5 with no actionable
findings or unresolved review threads.
- The full repository test suite and current-head CI are running. Their
final results will replace this line before merge.
- Earlier live feature previews created two bots in the authorized
Paperclip Slack workspace and reused one personal manager grant. Local
managed Continue created and installed a bot without a configuration
token or a separate Add to Slack action. Avatars, welcome DMs, signed
callbacks, and direct replies were observed. A Settings test DM also
arrived in the signed-in user's Slack account.
- Earlier staging thread qualification required manual publication
recovery for one bot. Current-head live qualification remains required
before enabling the production default. Fixture results do not establish
Slack approval behavior in other workspaces.

## Risks

- Personal manager tokens can create apps. They stay in scoped vault
entries. The manager client secret stays at the broker. Company,
actor/session, origin, and app identity checks apply throughout setup.
- The additive migration must also work on qualification instances that
applied earlier branch migrations. Table, column, index, and constraint
creation is idempotent.
- Creation and message timeouts can leave uncertain outcomes. Saved
request IDs prevent automatic duplicate dispatch.
- Unknown Slack identities must receive and complete their private
challenge while signed in. Account selection alone does not grant
Paperclip permissions. Existing revoked or conflicting links are not
overwritten.
- A development-only manager transport uses an explicitly configured
private credential file for local qualification. Production ignores this
transport; no manager credentials are bundled with open source.
- Managed rollout requires Slack approval, the companion broker
deployment, configured secrets, and final live qualification. Removing a
connection does not delete its Slack app.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser
tools. The session does not expose the exact serving model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 15:34:02 -05:00
Dotta d3916151f6 Merge the standalone runtime prerequisite into Codex defaults
Co-Authored-By: Paperclip <noreply@paperclip.ing>

* codex/runner-native-platform-prerequisites:
  feat(runner): qualify native Codex platforms and cache hosted builds
  docs: establish paired feature and agent access maps (#16025)
  fix(acpx): record turn activity at the moment a stop fires (#15962)
  refactor(chat): extract channel status, receipts, drafts and authority workflows (#15995)
  feat: add portable verification skills and refresh the feature map (#15983)

# Conflicts:
#	doc/plans/2026-10-06-default-agent-runner.md
#	packages/paperclip-runner/README.md
#	server/src/__tests__/dev-runner-native-binary.test.ts
2026-10-12 15:16:51 -05:00
DottaandPaperclip ddcd746eb6 feat(runner): qualify native Codex platforms and cache hosted builds
Extract the tested runtime and installed-package prerequisite without changing agent creation defaults or experimental gates.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 15:15:01 -05:00
DottaandPaperclip 7d9b93aee2 docs: establish paired feature and agent access maps (#16025)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Contributors use the feature map to understand and verify product
capabilities.
> - Agents need APIs and usable entry points within their permission
boundaries.
> - Product recipes and agent access need separate records with shared
capability IDs.
> - This pull request establishes that contract and gives maintenance
ownership of both maps.
> - The benefit is repeatable access review without changing product
behavior or permissions.

## Linked Issues or Issue Description

Refs: #15983, which added the portable verification skills and updated
the feature map. This is its contract-only follow-up. Related: #15288
added the original feature map.

**Issue type**

Missing documentation and maintenance contract.

**Where is the issue?**

The product map, its missing agent access companions, and the
verification skills.

**What's wrong?**

There is no shared convention for API presence, agent entry points,
permission boundaries, access gaps, or evidence. An API declaration does
not prove that a permitted native or remote agent can use it.

**Suggested fix**

Keep the product and agent access maps as siblings under `maps/`. Reuse
capability IDs. Record representative source inventory separately from
operation review and live proof. Keep both maps aligned through the
maintenance skill.

## What Changed

- Move the existing product map to `maps/feature-map/`. Preserve its
recipes and Poteto format. Update relative links and add companion
links.
- Add `maps/agent-access-map/` with 35 area files and 171 matching
capability IDs. Seed representative API operations, agent paths,
policies and known gaps from the reviewed source inventory. Mark entries
as `baseline`, `source-only`, and live verification `not-run`.
- Define the shared contract in `maps/README.md`. Track API presence,
agent availability, policy, evidence and review depth separately.
Require operation enumeration before marking a broad capability
operation-reviewed.
- Extend `maintain-verification-skill` to reconcile both maps and
inspect actual route policy and production tool admission. Update
creation, verification, notices and contributor references to use the
shared contract.
- Add `scripts/check-maps.mjs` and ten regression tests. Check matching
areas/IDs, indexes, references, contract fields, API method/path
examples, justified exceptions and evidence-state consistency.
- Keep the coverage inventory unchanged. Preserve skill invocation
policies and both MIT license blocks. No product API, permission, UI,
runtime helper, migration, dependency or workflow changes.

## Verification

- Passed: `node scripts/check-maps.mjs` reports 35 paired areas and 171
matching capability IDs. All local map references resolve.
- Passed: `node --test scripts/check-maps.test.mjs` runs ten tests.
Fixtures cover missing companions, renamed/orphan/duplicate IDs, broken
references, missing operation fields, empty exceptions, missing API
examples, empty policy fields, and verified evidence without passed
verification.
- Passed: metadata validation for all three verification skills. Skill
and contributor links resolve. Invocation policies and both license
blocks are unchanged.
- Compared relocated recipes with current master. Contents differ only
in relative links and companion links. `coverage.json` is byte-for-byte
unchanged. Master's updated read-only interaction test reference is
retained.
- Passed: `git diff --check`. The branch starts from current master and
contains only this contract change.
- Repository-wide typecheck, tests and build were not run locally for
this documentation and Node structural-check scope. Live agent access
was not exercised. Those gaps are explicit in the baseline entries; the
structural check does not certify behavior. CI and automated review are
pending.

## Risks

- The companions are representative source baselines. They are not
complete operation inventories or passed agent journeys.
- The checker cannot establish actual permissions, native tool
admission, operation completeness or truthful live evidence. Maintenance
must inspect source and verify with the actual permitted agent context.
- External links to the old product-map directory may need updates.
Repository references have been updated.
- Existing access gaps and board/user restrictions remain recorded. This
contract does not change authority or implement missing paths.

## Model Used

OpenAI GPT-6 through Codex assisted with this change. The session does
not expose the exact serving model ID, configured context-window size,
or reasoning setting. Capabilities used: source analysis, file edits,
shell commands, and structural regression tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 14:52:28 -05:00
DottaandPaperclip bb7591dfd0 fix: retain matching-host runner development overrides
Use the controller override only for matching remote OS and CPU, preserving explicit remote priority and strict target header verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 14:34:41 -05:00
4b766b8729 fix(acpx): record turn activity at the moment a stop fires (#15962)
Record stop-time ACP turn diagnostics (last-event age, oldest pending tool age, pending tool kinds) so wall-clock timeout events show whether the turn was stalled. Diagnostics only.

Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:33:59 -07:00
DottaandPaperclip 0d0e138dad refactor(chat): extract channel status, receipts, drafts and authority workflows (#15995)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Chat channels connect that work to provider conversations.
> - Provider workflows still fill the shared service.
> - Their authority and recovery rules need clear homes.
> - This PR lands five reviewed extractions into channel folders.
> - Shared authorization, state and worker lifetime stay explicit.

## Linked Issues or Issue Description

Refs #15976, #15996, #15999, #16001 and #16002. This root PR now
contains all five extractions from batch three. The four upper PRs were
merged into their parents. The combined tree matches reviewed stack head
`789e918b5af49ec1958076ce4d22d31006205302` exactly.

## What Changed

- Extract Slack session status and accepted-file receipt recovery.
- Extract Telegram publication drafts and Stop handling.
- Extract Photon events and native responses.
- Extract Teams file-transfer authority.
- Reduce `chat-channels.ts` from 29,897 to 26,461 lines (3,436 lines,
11.5%).
- Preserve function bodies, runtime-map identity, leases and transaction
boundaries.

## Verification

- All 117 selected public-service cases passed before and after their
respective extractions.
- All 30 moved declarations and remaining service statements preserve
normalized syntax across the five slices.
- Server typecheck, module boundaries and whitespace checks passed.
- Cumulative `pnpm -r typecheck` and `pnpm build` passed. Combined-head
CI and review are pending on `5de1b0b99d71049a72440e8c131bf4bb30bdf3f3`.
- Fixtures exercise assembled services and disposable databases, not
live third-party providers.
- Cumulative local `pnpm test:run` on
`789e918b5af49ec1958076ce4d22d31006205302` exited 1: 32,414 passed, 125
skipped, one failure. `company-skills-routes.test.ts:1691` (`stars,
forks, and comments on skills through company-scoped endpoints`) failed
with `socket hang up`; that test and its route are unchanged by this
stack. The same 58-case suite then passed in isolation without code
changes. The original command did not reach 111 later serialized suites;
its failure is retained, so the full-local-test checklist remains
unchecked.

```sh
pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts -t 'Slack session|Slack turns to Idle|signed Slack session stop|denies guest Slack stops'
```

- Combined-head CI attempt 1: five runners received a shutdown signal at
19:13:08 UTC (server shard 10/12, serialized shards 2/9 and 7/9, E2E
shards 3/8 and 7/8). These jobs stopped during execution without a
failing test assertion. [Original
workflow](https://github.com/paperclipai/paperclip/actions/runs/38293566028).
The affected jobs will receive one unchanged retry after the other jobs
finish.

## Risks

- Changes cross durable provider recovery and authority boundaries. The
moved bodies preserve syntax, and 117 focused public-service cases
passed before and after their extractions.
- Shared workers, timers, runtime maps and generic authorization remain
in the service.
- The full local test run failed in an unchanged company-skills route
test; its result and the unrun suites are recorded above.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and repository tools. The
exact runtime model ID and context-window size are not exposed in this
session. No separate agent was used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 19:26:54 +00:00
DottaandPaperclip e79254b2fa fix: preserve saved setup and match sandbox runner targets
Share actual-target artifact selection across remote Codex setup and execution. Keep unchanged saved setup choices outside new-selection rules while retaining runtime and authentication validation. Update the served platform guide.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 14:12:14 -05:00
DottaandPaperclip 665c155472 Merge master and preserve qualified runner paths
Retain both hosted cache scopes. Use shared test-home teardown and restore owned directory permissions for the observed Mac ENOTEMPTY failure while keeping unrelated errors visible.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 13:42:01 -05:00
DottaandPaperclip a5b16834c8 fix: pin ARM qualification helper to workflow source
Keep frozen product and reusable workflow tooling identities separate. Preserve clean-consumer and package integrity checks.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 13:26:56 -05:00
8227485372 perf(ci): cache the Runner's release crate artifacts keyed by their inputs (#15932)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Its trusted PR workflow builds the Runner's Rust binary in five
jobs: Typecheck + Release Registry, Build, Canary Dry Run, and the
`rust` and `vitest 2/2` lanes of Verify Paperclip Runner. The `Build`
job is the slowest check in almost every green run (461s, 489s and 515s
in runs 38180574768, 38182717052 and 38183149920 on 2026-10-11).
> - Each of those jobs restores master's `release-runner-v2` dependency
cache. That cache holds only third-party crates. `rust-cache` removes
workspace crates when it saves, and a fresh checkout invalidates cargo's
mtime fingerprints for workspace crates in any case. So cargo compiles
`paperclip-runner-core` in release mode from scratch on every run. The
job logs show one `Compiling` line and 28s to 49s for it.
> - 18 of the 239 commits on `master` in the seven days before this
change touched the crate's inputs. The other 221 compiled the same
sources to the same binary again.
> - This pull request adds a second cache that holds only that crate's
release artifacts (about 70 MB). The key is made from the compiler, the
checkout root and a hash of every input the build reads. The master Rust
lane writes it. A pull request whose merge tree has byte-identical
inputs restores it and marks the artifacts newer than the checkout, so
cargo keeps them. Every other pull request misses and compiles as
before.
> - The benefit is about 28s to 49s less on `Build`, which is the run's
critical path, and the same on Typecheck, Canary Dry Run and two Runner
lanes, for about nine of ten pull requests.

## Linked Issues or Issue Description

No GitHub issue exists for this. The description below follows the
enhancement template.

**What existing behavior does this improve?**

The time the trusted PR workflow spends to compile the Runner's own
crate on every run.

**Subsystem affected**

CI / build

**Current behavior**

Every PR job that builds the Runner binary compiles
`paperclip-runner-core` in release mode from scratch. The compile takes
28s to 49s on the `runs-on` fleet (one `Compiling paperclip-runner-core`
line, then `Finished release profile` in runs 38169436532 to 38183149920
on 2026-10-11). The shared Rust cache cannot help: it holds only
third-party crates, and a fresh checkout gives every source a new mtime,
which makes cargo discard the crate's fingerprints.

**Proposed behavior**

The master Rust lane in `release-verify.yml` saves the crate's release
artifacts under a key made from the compiler, the checkout root and a
hash of every input the release build reads. The PR jobs that build the
binary restore that entry read-only. On a hit, a script sets the mtime
of every restored artifact to now, so cargo sees every source as older
than the build and keeps the artifacts. `pnpm build` then prints
`Finished release profile` in under a second for that crate. On a miss,
cargo compiles the crate exactly as before.

**Reason and benefit**

About 28s to 49s off `Build`, the slowest check in almost every green
run, and the same off the Typecheck, Canary Dry Run and two Verify
Paperclip Runner lanes. The saving applies to every pull request whose
merge tree does not change the crate's inputs, which was 221 of 239
`master` commits in the week before this change.

**Breaking changes**

None. The checks are unchanged. A miss or an eviction costs the compile
time it costs today and nothing else.

**Additional context**

This is a workflow change. `pr.yml` calls `pr-trusted.yml@master`, so
this pull request's own run uses the current reader steps, and the
writer runs only on a master push. The saving becomes visible on the
first PR runs after the merge and after the next master push has written
the entry.

## What Changed

- `scripts/runner-release-crate-cache.mjs` (new): the one place that
defines the cached path list (`CACHED_PATHS`), the hashed inputs
(`HASHED_INPUTS`) and the files a restore must contain (`REQUIRED`). The
`scope` subcommand prints the platform, the `rustc --version --verbose`
release, commit and host, and a hash of the checkout root. The
`mark-current` subcommand sets the mtime of every restored artifact to
now and prints a `paperclip.runner-release-crate.v1` JSON line with the
file count and bytes. If the restored set lacks any of the seven files
cargo's freshness check reads or checks for (the fingerprint hash and
dep-info of each unit, the lib's rlib and rmeta, the binary under
`deps`), it removes the restored files, prints `action: discarded`
naming the missing file, and exits 0, so cargo compiles as on a miss.
- `.github/workflows/pr-trusted.yml`: after the `release-runner-v2`
restore in `typecheck_release_registry`, `build` and `canary_dry_run`,
and in the `rust` and `vitest 2/2` lanes of `verify_paperclip_runner`
(new matrix key `release_crate: true`), three steps: compute the key
scope, `actions/cache/restore` the crate under
`runner-release-crate-v1-<scope>-<hashFiles(...)>` with no
`restore-keys`, and run `mark-current` when `cache-hit == 'true'`. The
`hashFiles` list covers the toolchain pin, the manifests and lockfile,
the crate sources, the protocol tree and the three fixture trees the
test modules include (two under the Runner package and
`tests/runner-e2e/fixtures` at the repository root).
- `.github/workflows/release-verify.yml`: in the Rust lane of an
own-repository master push, after the debug warm-up: compute the key
scope, look the key up (`lookup-only: true`), and on a miss run `cargo
clean --release -p paperclip-runner-core` and `pnpm build:binary` (the
exact command the readers run, so the entry carries no test-profile
variants), then `actions/cache/save` the same path list under the same
key.
- `scripts/__tests__/runner-release-crate-cache.test.mjs` (new, 10
tests, in `test:release-registry`): the path list and hashed inputs are
the expected ones and exist; every `include_str!` and `include_bytes!`
in the crate sources resolves under a hashed input, whether its argument
is a literal or a `concat!` of literals, `env!("CARGO_MANIFEST_DIR")`
and macro metavariables, and any other form fails the test by name (this
guard found the `test/fixtures` tree during development and the
`tests/runner-e2e/fixtures` tree in review); the parser itself is tested
on a synthetic crate for each supported form and six rejected ones;
every PR reader restores after the dependency cache and marks before its
build command with the script's exact path list and key and no
`restore-keys`; the Verify lanes are gated by `release_crate`; the
writer runs after the checks with the lookup, the clean and build, and
the save under the readers' exact path list and key; the scope changes
with the compiler and the checkout root and survives a missing `rustc`;
`mark-current` makes every artifact newer than the checkout, leaves a
dependency's artifacts alone, discards a restore missing any one of the
seven required files (one case each, checking the named reason), and
still marks a restore missing any one of the seven files cargo does not
need.

## Review fixes

Greptile's two findings on the first commit were both valid and are
fixed in the second commit:

- **Incomplete caches looked usable.** `REQUIRED` checked four files,
but cargo's freshness check needs seven. Measured with cargo 1.97.1 on
the real crate (restore, touch the artifacts, delete one file, build):
without the lib's fingerprint hash, dep-info, rlib or rmeta it
recompiles the crate (24s to 27s); without the bin's fingerprint hash,
dep-info or the binary under `deps` it relinks (1.5s); without either
`.json` fingerprint record, either rustc `.d` file, the uplifted binary
or its `.d` it reports the crate fresh in 0.15s and recreates the
uplifted binary from `deps`, and the staging step then copies it.
`REQUIRED` now lists exactly the seven files, the glob for the binary
under `deps` excludes the `.d` beside it, and the test removes each of
the seven in turn (discarded, naming it) and each of the other seven in
turn (still marked).
- **Composed include paths escaped the guard.** The regex matched only
`include_str!("literal")` and skipped `include_str!(concat!(...))`,
which the crate uses four times (30 of 34 includes were checked). The
guard now parses every include argument and resolves the `concat!`
forms; one of them, in `provider_events.rs`, reads
`tests/runner-e2e/fixtures/native-completion/terminal-tool-carrier.json`
at the repository root, outside every hashed input. It is test-only, so
today's release binary does not read it, but the key is meant to cover
every include whatever its build, so `tests/runner-e2e/fixtures/**` (9
files, 80 KB) joins `HASHED_INPUTS` and the six key embeddings in the
two workflows. With the first commit's input list, the new guard fails
on exactly that include.
- `doc/RELEASE-AUTOMATION-SETUP.md`: a "Runner release crate cache"
section under the Runner verification dependency cache, with the key,
the writer, the readers and the trust boundary.
- `package.json`: the new test file in `test:release-registry`.

## Verification

- `node --test scripts/__tests__/runner-release-crate-cache.test.mjs`:
10 tests pass, three consecutive runs.
- The second commit, with the real toolchain on the real crate (`rustc
1.97.1`, cold build 2m20s, the crate's artifacts 74 MB): every source
touched to a fresh-checkout mtime and the archived artifacts restored,
cargo compiles (`Compiling paperclip-runner-core`, 24.8s and 25.1s, the
two controls); restored and marked by `mark-current`, `Finished release
profile in 0.16s` with no `Compiling`, and the same on a second build.
Then, for each of the fourteen files cargo wrote for the two units, one
at a time: restored, marked and that file deleted, cargo recompiles
without any of the seven `REQUIRED` files and reports fresh without any
of the other seven; restored with that file deleted and run through
`mark-current`, the seven required ones print `action: discarded` naming
the file and the others print `marked`. A restore holding only the seven
required files builds fresh in 0.15s, cargo recreates the uplifted
binary, and `stage-runner-binary.mjs` copies it into `dist/bin`.
- `node --test '.github/scripts/tests/*.test.mjs'`: 378 of 381 pass. The
3 failures are in `docker-canary-promotion.test.mjs` and fail the same
way on an unmodified `master` checkout in this sandbox (the stand-in
docker writes no `calls.jsonl`).
- `node --test scripts/__tests__/release-verify-workflow.test.mjs
scripts/__tests__/e2e-shard.test.mjs
scripts/__tests__/run-vitest-stable-shard.test.mjs`: 99 of 101 pass; the
2 failures spawn `pnpm`, which is not on this sandbox's PATH, and fail
the same way on `master`.
- Both workflows parse as YAML. No job has a duplicate step name or step
id.
- The mechanism, with the real toolchain (`rustc 1.97.1`, the package's
pin) on a copy of the crate: a cold `cargo build --release -p
paperclip-runner-core --bin paperclip-runnerd` (2m23s here). Then the
artifact set was archived with `tar`, removed, every source re-copied so
each had a new mtime, and the set restored:
- restored, not marked: `Compiling paperclip-runner-core`, 24.6s (the
control; this is what CI does today);
- restored and marked by `mark-current`: `Finished release profile in
0.15s`, no `Compiling`; a second build is the same;
  - one Rust source edited after a marked restore: `Compiling`, 24.0s;
- one protocol schema edited after a marked restore: `Compiling`, 23.9s
(the protocol files enter cargo's dep-info as absolute paths and are
hashed into the key);
- the rlib removed from a restored set: `mark-current` prints `action:
discarded, reason: incomplete: the lib rlib is missing` and removes the
rest, and cargo compiles.
- Not run here: the workflow steps themselves. `pr.yml` pins
`pr-trusted.yml@master`, so this pull request's own `Build` job runs the
current reader steps, and the writer runs only on a master push. The
first PR runs after the merge and after the next master push show the
`Restore the Runner release crate` step's result and the
`paperclip.runner-release-crate.v1` line in their `Build` logs.

## Risks

- Correctness: a reader can only restore artifacts that master built
from inputs byte-identical to its own merge tree, because every input
the build reads is in the key, and the key has no prefix match. A pull
request cannot write. The trust boundary is unchanged: master writes, a
pull request restores master's entry.
- An input the build reads that is outside the hashed patterns would let
a stale artifact pass. The release dep-info lists the crate sources and
the protocol schemas and fixtures; all are hashed. The guard test
resolves every `include_str!` and `include_bytes!` in the crate, literal
or `concat!`, and fails if one lands outside the hashed inputs or uses a
form it cannot resolve, so a new include form cannot slip past it by not
matching a regex.
- GitHub-hosted runners check out under `/home/runner/work`, the
`runs-on` fleets under `/home/runner/_work`. Cargo records the included
protocol files as absolute paths, so an entry from one root is useless
under the other. The checkout root is in the key, so the other root
misses instead of restoring something cargo then discards.
- A cache that does not keep mtimes on restore changes nothing: the
artifacts are then already newer than the checkout, and `mark-current`
sets them to now in any case.
- Cost: one new entry of about 70 MB per distinct input set, written
only by the master Rust lane on a miss, plus one `cargo clean` and
`build:binary` (about 40s to 50s) on that lane when the inputs changed.
The readers pay one restore of about 70 MB (two to three seconds at the
fleet's observed restore speed) and a sub-second `mark-current`.
- An incomplete entry (a writer that saved a partial set) is discarded
by `mark-current` and costs one compile, the same as a miss.

## Model Used

- Claude (Anthropic), model id `claude-fable-5-1` (Claude Fable 5.1),
extended thinking, with tool use (shell, file editing, GitHub CLI, a
local Rust toolchain for the experiment) in a Paperclip agent session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 11:25:31 -07:00
7df517d18f perf(build): reuse a current Runner build in the server's prepare:runner-vendor step (#15926)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Its CI runs a `Build` job on every pull request. That job runs `pnpm
build` and then verifies the public npm install. Since the server test
shards got faster, `Build` is the slowest check in most runs (419-497s
on the six fully green `master` runs created 2026-10-11 19:46Z to
20:17Z).
> - Inside `pnpm build`, the server package is the last package to
build. Its first step, `prepare:runner-vendor`, runs the full build of
`@paperclipai/paperclip-runner` so the server can vendor
`packages/paperclip-runner/dist`. In `pnpm -r build` that package was
built seconds earlier by the same command. The second build took 12-16s
on those six runs and sits on the critical path of the job.
> - The step cannot just be removed. A standalone `pnpm --filter
@paperclipai/server build` (the Dockerfile's path) must produce a
current vendored Runner on its own.
> - This pull request keeps that guarantee and skips only the build that
is provably redundant. The Runner's `build` records a digest of its
inputs, captured before its first step and confirmed after its last,
together with a manifest of every file it left under `dist`. The
server's `prepare:runner-vendor` reuses the build when the digest
matches, the stamp is for this platform, and every recorded output is
still there at its recorded size. Any other state runs the same full
Runner build as before.
> - The benefit is 12-16s off the `Build` job and off every other `pnpm
build` (the Canary Dry Run's `release.sh` build, local builds), with no
change to what a standalone server build produces. The PR's own first
run measured it: the server step reused the build in 56ms where `master`
runs rebuilt for 11-15s.

## Linked Issues or Issue Description

No GitHub issue exists for this. The description below follows the
enhancement template.

**What existing behavior does this improve?**

The server build's `prepare:runner-vendor` step, which builds
`@paperclipai/paperclip-runner` so the server can vendor its `dist`.

**Subsystem affected**

Build and CI (`server/package.json`,
`packages/paperclip-runner/package.json`, the `Build` job).

**Current behavior**

`prepare:runner-vendor` runs the Runner's full `build` every time.
Inside `pnpm -r build`, the Runner was already built by the same command
20 to 25 seconds earlier, with every check passing. On six fully green
`master` runs (2026-10-11, 19:46Z to 20:17Z) the repeated build took
11.8s to 15.9s, measured from the `server build$ pnpm run
prepare:runner-vendor` log line to the Runner's `Finished release
profile` line. The server package builds last, so that time is on the
critical path of `Build`, which is the slowest check in most pull
request runs.

**Proposed behavior**

- The Runner's `build` starts with `node scripts/record-build-inputs.mjs
--begin` and ends with `node scripts/record-build-inputs.mjs --finish`.
`--begin` removes the previous stamp and records a digest of every build
input before the first step reads anything. `--finish`, after every step
and check has passed, recomputes the digest and writes
`packages/paperclip-runner/node_modules/.cache/paperclip-runner-build-inputs.json`
only if the inputs are still the ones the build began with, together
with the path and size of every file under `dist`. A stamp therefore
exists only for a build that completed every step on inputs that did not
move while it ran.
- The server's `prepare:runner-vendor` becomes `node
scripts/prepare-runner-vendor.mjs`. It reuses the build if the stamp is
for this platform and architecture, every recorded output is still there
at its recorded size, and the recomputed digest matches. In every other
case it removes the stamp and runs `pnpm --filter
@paperclipai/paperclip-runner build`, the exact command the step ran
before.
- `PAPERCLIP_RUNNER_VENDOR_REBUILD=1` forces the rebuild.

**Reason and benefit**

12-16s off the `Build` job and off every other `pnpm build` (the Canary
Dry Run's `release.sh` build, local builds), with no change to what a
standalone `pnpm --filter @paperclipai/server build` produces.

**Breaking changes**

None. The rebuild path runs the same command as before. A fresh
checkout, a changed input, a missing or truncated output, an edit that
landed during the build, or a stamp from another platform always
rebuilds.

## What Changed

- `packages/paperclip-runner/scripts/build-inputs.mjs` (new): defines
the input set, the digest, the output manifest and the reuse decision.
Inputs are every file under `packages/paperclip-runner` except `dist`,
`dist-browser`, `runner/target`, `node_modules` and dot-directories;
every file under `packages/paperclip-eval-kernel` except `dist` and
`node_modules` (the Runner's `build:typescript` builds it first); every
file under the root `patches` directory (`check:acpx-profiles` hashes
`acpx@0.13.1.patch`, and `build-verified-provider-entrypoints.mjs`
copies the `pi-acp` and `brace-expansion` patches into
`dist/cli/pi-provision-inputs`); and the root `package.json`,
`pnpm-lock.yaml`, `pnpm-workspace.yaml` and `tsconfig.base.json`. The
toolchain pins are inside the package (`rust-toolchain.toml`,
`runner/Cargo.lock`) and the TypeScript version is in the lockfile, so
both are covered. A symlink is hashed as its target path and is not
followed. Outputs are every file under `dist`, the tree the server
copies wholesale, recorded with their sizes. Exports `beginRunnerBuild`,
`recordRunnerBuild` and `assessRunnerBuild` (returns `current` plus a
stable reason code: `no_stamp`, `schema_mismatch`, `platform_mismatch`,
`missing_output`, `outputs_changed`, `inputs_changed`,
`inputs_unchanged`).
- `packages/paperclip-runner/scripts/record-build-inputs.mjs` (new): the
two phases, `--begin` and `--finish`, added as the first and last step
of the Runner's `build` in `packages/paperclip-runner/package.json`.
Each build chain has its own pending record under
`node_modules/.cache/paperclip-runner-build-inputs.pending/`, named
after the shell pnpm runs the chain in (the parent of both phases;
`--finish` also accepts a record named after its own pid, for a shell
that execs the last command of the chain), so two Runner builds running
at once in one checkout do not consume each other's record. `--finish`
without a pending record for its chain is an error, so the chain cannot
silently lose its first step; `--finish` after an edit that landed
mid-build prints `not recorded: inputs changed while the build ran` and
leaves no stamp. Records of builds that never finished are pruned by the
next `--begin` after a day.
- `server/scripts/prepare-runner-vendor.mjs` (new): the reuse-or-rebuild
step. It resolves the pnpm that is running it through `npm_execpath`
with the same rule as the CLI's `resolvePnpmInstallInvocation` (a
JavaScript entry runs through this Node, a native executable runs
directly, otherwise `pnpm` on PATH). It prints one
`paperclip.runner-vendor.timing.v1` line with `action`, `reason`,
`check_ms` and `build_ms`, so the saving is measured on this PR's own
`Build` job.
- `server/package.json`: `prepare:runner-vendor` now runs that script.
The `build` and `typecheck` scripts are unchanged.
- `server/src/__tests__/server-package-build-script.test.ts`: pins the
new indirection and the command behind it (imported from the script, so
the Dockerfile's standalone path stays pinned), and a new test asserts
the Runner's `build` starts with `--begin`, ends with `--finish`, and
has exactly those two record steps.
- Tests: `packages/paperclip-runner/scripts/build-inputs.test.mjs`
(node:test, added to `test:typescript:prep`) and
`server/scripts/prepare-runner-vendor.test.mjs` (vitest, in the server
lane).
- `doc/DEVELOPING.md`: a "Reused Runner build in the server build"
section under Test Commands.

## Review findings on the first head, all fixed in
[2d57542](https://github.com/paperclipai/paperclip/pull/15926/commits/2d575425977f87b4612222d0a1961bc9cc5ba569)

Greptile scored the first head 2/5 with three P1 findings. All three
were valid.

| Finding | Fix |
| --- | --- |
| **Patch edits leave stale output.** The root patch files the build
reads were not in the digest. | Every file under the root `patches`
directory is an input. Tests assert the patches are in the input list
and that editing the pi-acp patch, the acpx patch, or adding a patch
changes the digest. |
| **Missing helpers pass the build.** Only four outputs were checked, so
a `dist` that lost `dist/cli/acpx-runtime-sidecar.cjs` still counted as
current. | `--finish` records the path and size of every file under
`dist`; reuse requires all of them intact and reports `outputs_changed`
naming the first missing or resized file. Schema bumped to
`paperclip.runner-build-inputs.v2` so an older stamp is rejected. Tests
cover a deleted helper, a truncated helper, and the server step
rebuilding on `outputs_changed`. |
| **New edits count as compiled.** The digest was hashed after the
build, so a source edited while `build:binary` ran was recorded as
compiled. | The build is bracketed: `--begin` captures the digest before
the first step, `--finish` compares after the last and writes nothing if
it moved. Tests cover the mid-build edit, the undisturbed build, and the
CLI in both orders. |

## Review finding on the second head, fixed in the third commit

Greptile scored the second head 4/5 with one P1, valid: two Runner
builds running at once in one checkout (server `build` beside
`typecheck`) shared one pending record, so the second `--begin`
overwrote it, the first `--finish` consumed it, and the second
`--finish` threw `no pending build inputs` after every step of its build
had passed. Reproduced on the second head's module: two
`beginRunnerBuild`, then the first `recordRunnerBuild` records and the
second throws.

| Finding | Fix |
| --- | --- |
| **Overlapping builds fail each other.** One pending file for every
build chain. | One record per chain, named after the shell pnpm runs the
chain in: `--begin` sees it as its parent, and so does `--finish`,
unless the shell execs the last command of the chain (bash can), in
which case `--finish` is that shell and finds the record under its own
pid. Pending records of builds that never finished are pruned after a
day. An edit between two overlapping starts leaves no stamp whichever
build finishes last: the build that began before the edit reports
`inputs_changed_during_build` and removes the stamp the other wrote, so
a stamp always means every build that wrote `dist` read the inputs it
names. Tests: both finishing orders with and without an edit, the
pruning, a `--finish` that cannot use another chain's record, and two
real `sh -c` chains (plus one under `bash`) held open at the same time,
both recording. On the real repo, two `pnpm --filter
@paperclipai/paperclip-runner build` chains started 8s apart with a
cargo stand-in both recorded (`files=1223 outputs=1434`) and left the
pending directory empty. |

## Measured on this PR's runs

The `Build` job ran `pnpm build` from the branch, so both builds of the
Runner in that job used the new chain.

First head (c23faab, run 38175281058), against the four fully green
`master` runs created 20:41Z to 21:37Z the same evening:

| | PR run 38175281058 | `master`, four runs |
| --- | --- | --- |
| Runner build inside `pnpm -r build` | 55.3s | 47.3-62.0s |
| Server step `prepare:runner-vendor` |
`action:reuse,reason:inputs_unchanged,check_ms:56,build_ms:0` | full
Runner rebuild, 11.0-14.8s |
| Server package build | 29.7s | 32.3-43.5s |
| `pnpm build` step | 114s | 102-135s |
| `Build` job | 411s | 409-474s |

Second head (2d57542, run 38178686999), against the two fully green
`master` runs created in the same half hour (38178951607 and
38180574768):

| | PR run 38178686999 | `master`, two runs |
| --- | --- | --- |
| Runner build inside `pnpm -r build` | 53.9s (`begin:` 22:03:04,
`recorded:` 22:03:57, digest 460e02616081 over 1231 files and 1434
outputs) | 62.2s, 64.6s |
| Server step `prepare:runner-vendor` |
`action:reuse,reason:inputs_unchanged,check_ms:109,build_ms:0` | full
Runner rebuild, 14.5s, 15.2s |
| Server package build | 27.4s | 43.5s, 44.8s |
| `pnpm build` step | 106s | 91s, 95s to the Runner's end plus the rest;
127s, 135s in all |
| `Build` job | 405s | 461s, 479s |

The rebuild is gone from the server build on both heads, and the check
costs 56-109ms where `master` rebuilds for 11-15s; the rest of the
spread between runs is runner speed on the `runs-on` fleet. The third
commit changes nothing on the reuse path: it only changes where the
pending record lives between `--begin` and `--finish`.

## Verification

- `node --test packages/paperclip-runner/scripts/build-inputs.test.mjs`:
11 tests, three consecutive runs clean. Two builds at once in one
checkout each keep their own pending record and both record, in either
finishing order; with an edit between their starts the build that began
before it leaves no stamp whichever finishes last. A build's start
prunes records older than a day and no others. A `--finish` cannot use
another chain's record. Two chains run by separate `sh -c` shells at the
same time (and one under `bash`) both record and leave the pending
directory empty. The input set is exactly the sources, the eval kernel,
the root patches and the root config, and skips outputs and caches. The
output manifest lists every file under `dist` with its size and nothing
outside it. The digest is stable across reads, does not change when
`dist`, `dist-browser`, `node_modules`, `.vite` or `runner/target`
change, and does change for a Runner source, a Rust source, an
eval-kernel source, the pi-acp patch, the acpx patch, a new patch, the
lockfile, the base tsconfig, a new file and a rename. A symlink changes
the digest but its target's files are not read. A build is current only
with a matching v2 stamp, the same platform and architecture, the four
named outputs present, and every recorded output intact (a deleted or
truncated helper is `outputs_changed`; a file the build did not produce
may appear). An input edited between `--begin` and `--finish` leaves no
stamp and removes the pending record; the next build records normally.
`--finish` without `--begin`, or after a build that left a required
output missing, throws and leaves no stamp. The CLI brackets the build
for the package it is pointed at, reports a mid-build edit, and rejects
`--finish` first, both flags, neither flag, and unknown arguments.
- `vitest run scripts/prepare-runner-vendor.test.mjs
src/__tests__/server-package-build-script.test.ts` in `server/`: 21
tests. The decision reuses a current build, rebuilds a stale one, and
rebuilds a current one only when `PAPERCLIP_RUNNER_VENDOR_REBUILD=1`.
The pnpm resolver covers a JavaScript entry, a native executable, a
Windows executable, no `npm_execpath`, and npm's `npm_execpath`. With an
injected build command: a current stamp runs nothing and prints
`action:reuse`; changed inputs run the exact command from `server/` with
the stale stamp already removed; a lost
`dist/cli/acpx-runtime-sidecar.cjs` rebuilds with
`reason:outputs_changed`; a failing build returns its status and leaves
no stamp. End to end, the CLI with a stand-in `pnpm` on PATH that
brackets its build the way the real one does rebuilds on the first call,
reuses on the second, rebuilds after a source change, rebuilds after a
lost output, and reuses again. The guard test asserts
`prepare:runner-vendor` is the new script, the rebuild command is still
`pnpm --filter @paperclipai/paperclip-runner build`, the server build
still starts with the step, and the Runner's `build` starts with
`--begin` and ends with `--finish` with exactly two record steps; it
fails against the previous build script on its first assertion.
- Real repo, this checkout, with `cargo` and `pnpm` stand-ins on PATH
(no Rust toolchain here; the stand-in writes the one file
`stage-runner-binary.mjs` copies): `pnpm --filter
@paperclipai/paperclip-runner build` ran the complete chain through the
real nested pnpm (every TypeScript step and check, the stand-in cargo,
the binary staging, the two binary-reading checks) and recorded
`files=1223 bytes=18735059 outputs=1434`; the stamp is 128 KB. `pnpm run
prepare:runner-vendor` in `server/` then printed
`action:reuse,reason:inputs_unchanged,check_ms:222,build_ms:0`.
Appending a newline to `patches/pi-acp@0.0.33.patch` made the assessment
`inputs_changed`; truncating or deleting
`dist/cli/acpx-runtime-sidecar.cjs` or
`dist/cli/opencode-app-server-proxy.cjs` made it `outputs_changed` in
1-5ms; restoring each made it current again. `--begin`, an edit to
`src/index.ts`, then `--finish` printed `not recorded: inputs changed
while the build ran` and left no stamp.
- The whole check on the real set (1223 inputs, 18 MB; 1434 outputs)
takes 165-227ms on this host and took 56ms on the CI runner; the output
manifest part is 1-5ms.
- Not run here: the real `cargo build`, and `pnpm build` end to end.
This PR's `Build` job runs `pnpm build` from the branch, so its log
carries the Runner's `begin:` and `recorded:` lines and the server
build's `paperclip.runner-vendor.timing.v1` line with the measured
result.

## Risks

- A build input outside the digest would let a stale Runner be vendored.
The digest covers the whole Runner package except its outputs, the eval
kernel the Runner builds first, the root patches, and the root files the
Runner's tsconfig and dependency graph read. The lockfile covers
dependency versions, and the pins inside the package cover the Rust
toolchain. An input missed here is a bug to fix by adding it to the set;
the fallback direction is always a rebuild, never a skip.
- The whole `patches` directory is an input, although the Runner build
reads three of its fourteen files. An edit to an unrelated patch
therefore costs one extra Runner rebuild, which is the safe direction
and avoids a hand-kept list of file names that could drift from the
scripts that read them.
- The stamp lives in `node_modules/.cache`, which pnpm does not manage
and `.gitignore` and `.dockerignore` already exclude. It is about 128 KB
with the output manifest. The Docker build has no stamp and no prior
Runner build, so it rebuilds exactly as before.
- A stamp is written only after the Runner's `build` passed every step
on inputs that did not change while it ran. The Runner's first step and
the prepare step both remove any existing stamp before a rebuild starts,
so a build that stops partway, or a build command that fails before its
first step, leaves no stamp.
- `node scripts/record-build-inputs.mjs --finish` on its own is an
error; the script is only meant to run as the two ends of the `build`
chain, and the guard test pins that.
- The pending record is keyed by the pid of the shell that runs the
chain. pnpm runs every script through a shell (`sh -c` on POSIX, `cmd`
on Windows), and both phases are children of that shell; a shell that
execs the last command of the chain is covered by matching the finishing
process's own pid too. A chain run some other way, with the two phases
under different parents, fails loudly at `--finish` rather than
recording. Two builds running at once still write the same `dist`, which
no stamp can make safe; what the record guarantees is that a stamp is
written only by a build whose own inputs did not move.
- The platform and architecture are part of the comparison, so a stamp
from another machine is never trusted; an older v1 stamp is rejected by
schema.
- The Typecheck job builds the Runner once, through this step, with no
prior `pnpm -r build`, so it gains nothing and loses nothing: there is
no stamp and it rebuilds as before.
- The step now spawns pnpm through `npm_execpath` instead of relying on
`pnpm` being on PATH, because under corepack the script shell has no
`pnpm`. Outside a pnpm script it still falls back to `pnpm` on PATH.

## Model Used

- Claude (Anthropic), model id `claude-fable-5-1` (Claude Fable 5.1),
extended thinking, with tool use (shell, file editing, GitHub CLI) in a
Paperclip agent session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 11:25:24 -07:00
DottaandPaperclip 0690801f13 test: qualify Linux ARM with stock sandbox prerequisites
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 13:13:19 -05:00
Devin Foley 4f399cec9f fix(docker): skip the full-tree ownership walk on routine boots (#15992)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Docker image keeps all instance state under `PAPERCLIP_HOME`,
and deployments mount a volume there
> - Before it drops privileges, the entrypoint makes sure that the
`node` user owns the home (#10530)
> - The ownership probe is a first-mismatch `find` over the whole tree.
When ownership is correct, `-quit` never fires, so every boot stats
every file on the volume
> - On homes with many agent worktrees and `node_modules` trees, this
walk takes minutes before Node starts. That is close to common 300s
healthcheck windows
> - This pull request keeps a cheap shallow probe on every boot. It runs
the full-tree probe only once per volume and node UID/GID, and records
the result in a marker file
> - The benefit is that routine boots no longer scale with the file
count, and the original root-owned-volume fix stays in place

## Linked Issues or Issue Description

Refs #10431 (the open follow-up about the remaining cost of the
ownership walk). Related: #10530 (the change that added the full-tree
probe).

**What happened?**
On a hosted multi-instance deployment, we measured the time from
container start to the first app log line. Instances with many files on
the volume (agent worktrees with `node_modules`) took about 190-235s.
Small instances took 5-22s. One 30 GB volume with few files took 24s, so
the cost follows the file count, not the volume size. Almost all of this
time is the `find "$home_dir" \( ! -user node -o ! -group node \) -print
-quit` walk in `scripts/docker-entrypoint.sh`. When ownership is already
correct, `-quit` never fires, so the walk stats every inode on every
boot.

**Expected behavior**
When ownership is already correct, the entrypoint pre-start phase is
short and does not depend on the number of files in `PAPERCLIP_HOME`. A
fresh root-owned volume, a UID/GID remap, and an upgrade from an older
image are still repaired before the privilege drop.

**Steps to reproduce**
1. Run the image with a volume on `/paperclip` that holds a few hundred
thousand files, all owned by `node:node`.
2. Restart the container.
3. Compare the container start time with the first server log line. The
gap grows with the file count, even though no `chown` runs.

**Paperclip version or commit**
`master` at a9830414c7

**Deployment mode**
Docker production image (any platform that mounts a volume on
`PAPERCLIP_HOME`)

## What Changed

- `scripts/docker-entrypoint.sh`: the ownership probe is now tiered.
- Every boot runs a shallow probe (`find -maxdepth 3`) over the home and
its first three levels. A fresh root-owned mount and a UID/GID remap
both change the owner of the home directory, so this probe catches them.
- The full-tree probe runs only when
`PAPERCLIP_HOME/.paperclip-ownership-verified` is missing or records a
different node UID/GID (format `v1 <uid>:<gid>`). This catches deep
mismatches, including the first boot after an upgrade from an older
image.
- Any mismatch repairs the whole tree with `chown -R node:node`, as
before. After a clean probe or a repair, the entrypoint writes the
marker as the `node` user (through `gosu`), so the marker never trips
the shallow probe. A failed marker write is ignored: the only cost is a
full probe on the next boot. If `find` exits non-zero without a mismatch
(for example, an I/O error), the walk is incomplete, so the entrypoint
does not write the marker and the next boot probes again.
- `scripts/docker-entrypoint.sh`: timestamped log lines at entrypoint
start, around the full probe and the repair (with durations), and right
before the `exec` of the server. You can now measure the pre-start phase
from the container log.
- `server/src/__tests__/docker-entrypoint.test.ts`: the `find` stub now
tells the shallow probe from the full probe. New tests cover each path
(see Verification).
- `doc/DOCKER.md`: documents the marker, how to force a full check, and
the new log lines.

## Verification

- `cd server && npx vitest run src/__tests__/docker-entrypoint.test.ts
src/__tests__/container-init-reaping.test.ts
src/__tests__/docker-build-stamp.test.ts`: 40 passed. New cases:
- correct ownership + current marker → shallow probe only, no full walk,
no `chown`
- current marker + deep-only mismatch → full walk skipped (this is the
documented trade-off)
- no marker (upgrade) → one full walk, no `chown`, marker written as `v1
1000:1000`
- no marker + deep mismatch → full walk finds it, `chown -R`, marker
written
- root-owned top level with a current marker → `chown -R` without a full
walk
- full walk fails without a mismatch → no `chown`, no marker, retry
logged
- marker for another UID/GID → full walk runs again, marker rewritten as
`v1 1001:1001`
- timestamped `starting`, `ownership repair finished in Ns`, and
`starting app as node` lines, in order, before the command runs
- The new tests fail against the old script. The 11 existing tests pass
unchanged.
- `cd server && npx tsc --noEmit`: passes.
- `shellcheck -s sh scripts/docker-entrypoint.sh`: clean. `dash -n`:
syntax OK.
- Manual run with the real `find` (BSD find, script run under `dash`)
against a 200k-file tree. Only `id`/`usermod`/`groupmod`/`chown`/`gosu`
were stubbed, and `node` was mapped to the local user and group:
- boot 1, no marker, one deep group mismatch → full probe, `chown -R`,
marker written
  - boot 2, marker present → no full probe, no `chown`
- boot 3, no marker, clean tree → full probe, no `chown`, marker written
  - boot 4, marker present, clean → no full probe, no `chown`
  - boot 5, top-level group mismatch with marker → `chown -R`
- I did not build the image or measure boot time in a container. Docker
was not available on the machine I used. On a warm local disk the full
walk is sub-second, so the local timings say nothing about cold network
block storage.

## Risks

- Once the marker is current, a mismatch deeper than three levels is not
detected automatically. Examples: files written as root by `docker exec`
or a restore run as root, after the first verified boot. Before this
change, those files were repaired on the next boot. The workaround is in
`doc/DOCKER.md`: delete the marker and restart. A remap, a fresh volume,
and a root-owned top level are still detected on every boot.
- On the first boot after an upgrade, each existing volume pays one full
walk (the same cost as today). Later boots skip it.
- The marker is a new dotfile in `PAPERCLIP_HOME`. Backups and copies
that include the home carry it along. This is safe, because the marker
records the UID/GID that it verified, and a different UID/GID forces a
new full probe.
- The non-root path (restricted PodSecurity, OpenShift) does not change,
apart from two new stdout log lines.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`, 1M context) via Claude Code, with
tool use (shell, file editing). Default reasoning mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-10-12 10:22:43 -07:00
DottaandPaperclip a200495cc7 fix: preserve TOML table semantics during Codex startup trust
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 12:14:38 -05:00
DottaandPaperclip a2ac5c2715 refactor(chat): isolate Telegram, Slack, GitHub and Discord workflows (#15976)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Chat channels connect that work to external conversations.
> - The shared chat service still contains provider-specific workflows.
> - Those workflows are hard to inspect in a 34,000-line file.
> - This stack extracts Telegram, Slack, GitHub and Discord workflows
into their channel folders.
> - Shared authorization and service lifetime remain explicit.

## Linked Issues or Issue Description

Refs #15823, #15834, #15836, #15844 and #15845. Root of the second
behavior-preserving channel-folder refactor stack. The dependent PRs
#15979, #15981, #15985 and #15987 have merged into this branch; this PR
lands all five extractions on master.

## What Changed

- Move Telegram maintenance and subscription repair into
`chat-channels/telegram/maintenance.ts`.
- Move Slack session stopping and task starts into
`chat-channels/slack/session-stop.ts` and `task-start.ts`.
- Move GitHub webhook intake and recovery into
`chat-channels/github/webhook-ingress.ts`.
- Move Discord command registration and handling into
`chat-channels/discord/commands.ts`.
- Share type-only runtime records while preserving shared authorization,
transactions, live lease checks and worker ownership.
- Reduce the original service from 34,603 to 29,897 lines, removing
4,706 lines (13.6%).

## Verification

- All 205 selected integration cases passed before and after their
respective extractions: Telegram 41, Slack session stop 2 multi-step
scenarios, GitHub 84, Slack task start 25 and Discord 53. Each dependent
PR records its focused reproduction command.
- Server typecheck, module boundaries and whitespace checks passed.
- Per-slice syntax comparisons passed for moved declarations and
remaining service statements, allowing only the documented live callback
substitutions.
- Tests exercise the public service with disposable databases and
fixture provider transports. They do not claim live third-party
execution.
- Cumulative workspace typecheck and build passed on the final stack.
- Current-head CI passed on the combined stack commit
`d5b37dbe512e3e199932b6608cdc17639eda555a`; Greptile is 5/5 with no
unresolved threads.
- The [original serialized
shard](https://github.com/paperclipai/paperclip/actions/runs/38266518817/job/114870022249)
timed out in unchanged `adapter-auth-signal-routes.test.ts`. The exact
case passed locally without changes, and the [single job
retry](https://github.com/paperclipai/paperclip/actions/runs/38266518817/job/114874019933)
passed. The original timeout remains recorded.

Reproduce the original Telegram slice’s focused suite:

```sh
pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts -t 'Telegram durable private draft Stop|Telegram maintenance|Telegram webhook|Telegram.*(pause|remove)|Telegram.*commands'
```

- Cumulative local `pnpm test:run` exited 1: the first server group had
**18,770 passed, 105 skipped, and one timeout** in unchanged
`server/src/services/workspace-git-snapshot-streaming.test.ts:56`
(300,000 ms on macOS). The fail-fast runner did not reach its later
workspace and serialized groups. An unchanged isolated rerun passed
using the existing timeout; this does not turn the original full run
green. The test and its Git utility code are outside this refactor; all
current-head CI jobs passed.
- Reproduce that diagnostic alone: `pnpm exec vitest run
server/src/services/workspace-git-snapshot-streaming.test.ts`.

## Risks

- Credential lease ownership and provider-confirmation ordering must
remain unchanged. Existing rotation, lease-loss, recovery and durable
Stop cases pass.
- The channel factories create no workers or timers and preserve live
runtime, shutdown and gateway-ownership reads.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and repository tools. The
exact runtime model ID and context-window size are not exposed in this
session. No separate agent was used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 12:03:23 -05:00
DottaandPaperclip 598e190cb3 test: bind installed native payloads and await saved state
Isolate actual Codex resolver fixtures from global bridge lookup and preserve host qualification blockers with literal evidence.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 11:45:22 -05:00
Dotta a9830414c7 Preserve repository dependencies when importing GitHub skills (#15731)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its Skills library imports reusable instructions from GitHub.
> - Some skills depend on shared files and sibling skills outside their
folders.
> - Directory-only imports lose those dependencies.
> - This pull request keeps repository files together and lets users
select which skills appear in the library.
> - Agents receive the original files and paths through the existing
skill delivery system.

## Linked Issues or Issue Description

Refs #14713. Related: #13656.

**Current behavior**

GitHub imports copy independent skill directories. Shared scripts,
policies, helper skills, and credit files can be omitted. One
shared-file error can block every skill.

**Proposed behavior**

New imports keep repository files together at the scanned commit. Show
all skills with their folders and select every skill by default.
Collapse individual package files. Omit unsupported or oversized files
with a warning. Import available content without repair choices or an
extra confirmation. Keep warnings on the saved source.

**Reason and benefit**

Skills can read their shared repository files after Paperclip delivers
them to an agent. Original instructions stay unchanged. Users can
inspect included files and understand omissions before and after import.

## What Changed

- Store immutable company-scoped repository snapshots and associate them
with skill versions. Regenerate the migration after merging master.
- Preserve original bytes, binary encoding, and executable modes.
Shared-file changes version each usable selected skill. Pinned versions
retain their original files.
- Deliver a discovery wrapper with the original tree under
`.paperclip-repository/`. Preserve invocation policy when present. Fork
and create-from-existing retain repository files.
- Keep repository files together for new UI imports. Show every skill,
select all by default, and collapse package files. Remove the
packaging-mode menu. Keep the mobile import footer visible.
- Warn about symlinks, submodules, oversized files, missing references,
and invalid optional manifests. A malformed skill remains selected with
a Needs setup warning. Usable skills still publish, and its original
file remains repository support content. Source warnings persist through
refresh. Unsafe execution and secret-disclosure findings omit the
affected files before snapshot publication.
- Expose saved repository files in the installed skill viewer, including
binary encoding and executable flags. Reads use the immutable
company-scoped snapshot; relative instruction links resolve from their
original repository directory.
- Audit delivered inventories and derive trust from all repository
files. Accept evaluation prose while retaining dynamic-execution
findings. Read blobs in one bounded Git batch and check budgets before
reading support files. Imports never execute setup commands.
- Add scanner, database, delivery, UI, and browser regressions. Update
Storybooks, screenshots, and documentation.

## Verification

- 69 focused scanner, Git snapshot, delivery, and PostgreSQL lifecycle
tests passed. They cover partial imports, warning persistence, missing
references, binary files, executable modes, copies, pinned versions,
cancellation, company isolation, and batched Git reads.
- 110 import, skill-detail, selection-tree, saved-file link, and
Markdown renderer tests passed. A file-switch regression failed before
the resolver fix and passed afterward. Browser walkthroughs cover the
all-selected open tree, repository preview, non-blocking warnings,
missing references, setup problems, warning persistence, and mobile
layout.
- Full workspace typecheck and build passed. Runner E2E TypeScript
checks and design-token gates passed. The full serial local suite was
attempted and stopped after over an hour. The existing 40,000-file Git
streaming stress test failed after 303 seconds on macOS; the broad local
suite is not counted as passed. All 55 applicable PR checks passed on
the final commit, including the full sharded CI suite. Two optional
Storybook workflows were skipped; the five local Storybook walkthroughs
passed.
- On October 12, live campaigns imported the private Poteto repository
through the production GitHub Sources UI, selected architect and bro,
and invoked them with native Codex on local and Daytona. The source
connection was disabled before invocation. Independent filesystem
inspections matched every supported Git blob, size, and executable mode.
Both tasks completed and cleanup passed. Campaigns:
`local-2026-10-12T15-07-39-504Z` (both environments),
`local-2026-10-12T16-08-39-682Z` (local), and
`local-2026-10-12T16-03-33-938Z` (Daytona). Each of the two selected
skills received all 234 supported repository files. Later changes only
affect the installed-file viewer and documentation. This qualifies skill
delivery and instruction reads, not every Poteto workflow or adapter.
- A manual source-checkout test drive imported a synthetic private
GitHub repository with a symlink, an oversized file, a missing
reference, and malformed skill metadata. Available skills published with
their shared checklist. The walkthrough found duplicate warning text
after import; the final UI fix removes that duplicate. The file viewer
exposes saved repository dependencies and resolves authored relative
links without changing the instructions. A follow-up live refresh
omitted unsafe helper files with warnings, preserved every safe file
byte, and survived a server restart.

Screenshots below use the production UI with Storybook fixtures. They do
not replace the live GitHub and runtime checks.

<details>
<summary>All skills visible and selected by default</summary>

![Default repository
import](https://github.com/paperclipai/paperclip/blob/codex/skill-repository-packages/docs/pr-screenshots/pr-15731/repository-import.png?raw=true)

</details>

<details>
<summary>Import stays available with warnings</summary>

![Import
warnings](https://github.com/paperclipai/paperclip/blob/codex/skill-repository-packages/docs/pr-screenshots/pr-15731/repository-warning-details.png?raw=true)

</details>

<details>
<summary>Repository file inspector</summary>

![Repository
files](https://github.com/paperclipai/paperclip/blob/codex/skill-repository-packages/docs/pr-screenshots/pr-15731/repository-files.png?raw=true)

</details>

<details>
<summary>Mobile import</summary>

![Mobile
import](https://github.com/paperclipai/paperclip/blob/codex/skill-repository-packages/docs/pr-screenshots/pr-15731/repository-mobile.png?raw=true)

</details>

## Risks

- Each selected skill carries a repository copy in the runtime. The
database stores each snapshot once. Runtime transfer and storage
increase with the number of assigned skills.
- Runtime budgets remain: 9,990 repository entries, 1 MiB per file, and
61 MiB total. File omissions are recorded as warnings. Source-access or
download failures retain installed versions.
- Import success does not prove that every skill can run. Environment
dependencies, credentials, delegation tools, and schedules still need
configuration. Imported setup scripts are not executed.
- Live runtime coverage is native Codex on local and Daytona. Other
adapters use the shared delivery path but were not exercised in these
campaigns.

The roadmap marks Skills Manager as shipped. This extends the existing
importer. Related pull requests were checked.

## Model Used

OpenAI Codex, based on GPT-6, with repository inspection, code
execution, automated tests, and browser tools. The exact deployment
identifier and context-window size were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-10-12 11:30:20 -05:00
DottaandPaperclip 123a188374 fix: separate Codex login and ARM host qualification
Keep daemon ABI admission separate from CLI login and preserve restricted Docker installation checks before native host startup.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 11:20:36 -05:00
DottaandPaperclip 695506ad39 perf: cache native builds and verify Linux runtime targets
Preserve actual target selection, record the GNU libc baseline, and retain primary installed-startup failures for ARM qualification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 10:53:24 -05:00