2 Commits

Include renames

Author	SHA1	Message	Date
Devin FoleyandPaperclip	b70641f23f	feat(plugins): support image catalogs and persistent application overlays (#13646) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Plugins extend the application without adding each integration to Core. > - A downstream image needs a way to supply prebuilt plugins. > - Some plugin UI must stay mounted as users move between pages. > - This change adds an image catalog and a persistent application slot. > - Operators can upgrade or remove these plugins through their image and configuration. ## Linked Issues or Issue Description **Subsystem affected** Plugin packaging, activation and application UI. **Problem or motivation** The built-in plugin catalog is fixed in Core source. Downstream images cannot add entries through an explicit catalog. Existing page slots also cannot preserve a small application overlay across route changes. **Proposed solution** Read a bounded catalog of prebuilt plugins from the image. Verify its files before importing manifests. Use the existing managed selection and plugin lifecycle. Add an `appShellOverlay` slot with account and company cleanup. **Alternatives considered** A downstream fork adds merge work. Script injection provides no plugin lifecycle. A separate runtime download system adds a second distribution channel. **Roadmap alignment** This extends the existing plugin system. Related PR #9006 covers runtime install replication; this change covers immutable image contents. PR #12555 covers CLI scaffolding. Neither provides this catalog or application slot. The maintainer requested this work directly. ## What Changed - Validate catalog identities, confined paths, package versions and bundle hashes before importing code. - Apply image selection to persisted plugin installs, including removal and rollback. Adopt the verified image path from existing npm/local installs and bind runtime worker/UI entrypoints to verified package declarations. - Mount application overlays in both UI shells. Preserve route state and clear it on account, company and onboarding changes. - Restrict service-worker offline storage/fallback to hashed public assets in a separate cache namespace; exclude application HTML and extension/API data, including after worker restart. - Document the packaging contract, trust model and rollback requirements. ## Verification - Passed `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates`. Affected server/UI typechecks and builds, plus token gates, passed again after rebasing onto current master; the 124 focused tests also passed after rebase. - Latest focused verification: 124 tests in nine files passed for catalog/reconciliation/loader, overlay lifecycle, Layout and service-worker policy. The broader UI/shared/SDK run passed 7,204 tests in 690 files with canonical `TMPDIR`. - Real disposable Core/PostgreSQL: catalog install, selection removal, 0.1.0→0.1.1→0.1.0, same-version npm/legacy-path adoption, and preservation of disabled status passed. Added permissions entered `upgrade_pending`, withheld UI across restart, and activated only after explicit operator enable. - Real Chromium: desktop/mobile layout, route draft retention and Escape/focus passed with mocked extension responses. A persistent browser restart retained public hashed-asset offline fallback while refusing seeded legacy/current private entries and legacy HTML. - Full `pnpm test:run`: 12,539 passed; 17 failed across six existing files, stopping later phases. macOS read-only directory renames fail in runtime-skill-cache and company-skills-service; email tests require an absent local AgentMail fixture. Native runner/comment-redaction passed in isolation after temporary Rust setup; agent-conversations also passed in isolation. No unrelated source was changed to hide failures. - After rebase, two unchanged chat timing tests failed in CI and passed locally in isolation. Their CI shard passed on its single retry. All other current-head CI jobs passed on the initial run; review is 5/5 with no unresolved threads. - No live deployment or external plugin service was used. ## Risks - Plugins are trusted code. The catalog detects packaging errors; it does not authenticate an untrusted image builder. - Invalid catalogs fail startup. Images must contain the catalog and bundles together, with stable directories. - A host older than this contract lacks the activation guard. Disable added plugins and remove their configuration keys before reverting to it. - Offline navigation now returns 503 instead of replaying cached application HTML. Only public build assets have offline fallback. - Rolling back an unapproved permission change retains the approval gate; review the current manifest and explicitly enable it. A reduced permission set cannot establish prior approval or prior enabled status. - Plugin data migrations need their own rollback policy. This change retains installed records and does not reverse migrations. ## Model Used - OpenAI GPT-6 (Codex), model ID `gpt-6`, with repository inspection, code execution and browser verification. The runtime does not expose an exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (relevant suites; broad macOS server-run exceptions are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (fresh run on 488b3754ae; chat shard passed its single retry) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (fresh review on 488b3754ae) - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-19 09:18:10 -07:00
Devin Foley	627eef7cbd	fix(plugins): retry errored plugins at boot instead of leaving them dead (#12054) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins extend the server with sandbox providers, tools, and jobs; a loader activates them at boot > - When activation fails, the loader marks the plugin `error` and skips it on every later boot > - Activation failures are often environmental — missing package dependencies, a stale build output, a module that moved under a pull — and the fix lands on disk without any write to the plugin row > - The plugin therefore stays dead forever, and every feature behind it (sandbox destroys, cleanup sweeps, probes) silently stops working until an operator flips the row by hand > - This pull request makes `loadAll` retry errored plugins once per boot: flip to `ready`, attempt activation, and re-record the error if the attempt fails > - The benefit is that a plugin recovers on the next boot after its environment is fixed, with no manual database or lifecycle intervention ## Linked Issues or Issue Description **What happened?** Several sandbox-provider plugins sat in `error` status for weeks after a transient activation failure (a module resolution error from an older checkout state). The boot loader only loads plugins in `ready` status, so it never retried them. Environments backed by those providers lost sandbox destroys, cleanup sweeps, and probes with no visible signal other than the stale `last_error`. **Expected behavior** A plugin whose activation failure has been fixed on disk recovers on the next server boot. A plugin that still fails stays in `error` with a fresh error message. **Steps to reproduce** 1. Install a plugin whose worker cannot start (for example, delete one of its dependencies), then boot the server. The plugin lands in `error` status. 2. Restore the dependency. 3. Restart the server. Before this change, the plugin stays in `error` forever. After this change, the boot retries it and the plugin activates. ## What Changed - `server/src/services/plugin-loader.ts`: `loadAll` also fetches plugins in `error` status, flips each to `ready`, and activates it with the normal batch. The flip runs before activation because the `error` status only legally transitions to `ready` or `uninstalled`; a retry that failed while still in `error` could not re-mark itself. A failed flip logs a warning and never aborts the boot load. The stale comment at the `markError` site now describes the retry. - `server/src/__tests__/plugin-loader-error-retry.test.ts`: covers the flip-then-retry flow, the failed-flip isolation, and the empty case. ## Verification - `npx vitest run server/src/__tests__/plugin-loader-error-retry.test.ts server/src/__tests__/bundled-plugins.test.ts server/src/__tests__/plugin-lifecycle-restart.test.ts server/src/__tests__/cloud-image-bundled-plugins.test.ts` - Manual: mark an installed plugin's status to `error`, restart the server, and observe the loader log line `retrying plugins that failed activation on a previous boot` followed by a successful activation (or a fresh `last_error` if the plugin is genuinely broken). ## Risks - A genuinely broken plugin now costs one bounded activation attempt per boot (the attempts run in parallel with the ready batch under `Promise.allSettled`). It cannot crash-loop within a running process, and it returns to `error` with a fresh message. - The flip clears `last_error` before the attempt. If the process dies between the flip and the activation, the row is `ready` with no error text; the next boot simply loads it as a ready plugin. - Operators who relied on `error` as a manual "keep this off" latch should use the `disabled` status, which this change does not touch. ## Model Used - Claude (Anthropic) — Fable 5, model id `claude-fable-5`, extended thinking, agentic tool use via Claude Code CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-23 16:52:42 -07:00