3 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	96727b3cfe	fix(server): schedule plugin jobs from saved deadlines (#17518) ... ## Thinking Path > - Paperclip manages agents and plugin work. > - Plugins can declare cron jobs with saved deadlines. > - The existing scheduler queries for due jobs every 30 seconds, even when none exist. > - Empty queries keep an otherwise idle database active. > - This PR uses the shared scheduler and committed job changes to discover work. > - Existing job and run rows preserve deadlines and execution history. ## Linked Issues or Issue Description **What happened?** The plugin scheduler on the base of this stack runs empty periodic SQL. Its registration path also advances overdue timestamps. An empty replacement manifest can leave removed schedules active. **Expected behavior** No recurring SQL when no eligible jobs exist. Configured jobs wait for their saved deadline. Changes reactivate scheduling after commit. Work remains protected through drain and shutdown. **Steps to reproduce** Start the plugin scheduler with no jobs. After startup, observe queries during a three-minute idle window. The new PostgreSQL regression failed on the baseline and passes with this change. **Paperclip version or commit** Stacked on #17510 at `4d15c08428`. Deployment: source checkout, local PostgreSQL, controlled plugin RPC. Related: #7905 covers empty-manifest sync; this migration requires that behavior on both activation paths. #11571 proposes company-scoped jobs; this PR preserves the existing instance-scoped job contract. ## What Changed - Install the concurrency test barrier before due job notifications can dispatch. - Make dispatch the sole reservation owner and preserve retained or concurrently committed results when unregistering a plugin. - Replace the private interval with the existing shared deadline worker. - Signal job definition and run changes inside their owning transactions. No new tables. - Wake from plugin registration and worker status events. Retain explicit scheduler wakes received during drain. - Preserve overdue work and initialize only missing or changed schedules. - Recheck claims under a job-row lock. Keep plugin calls outside transactions and refill capacity on completion. - Commit completion and schedule advancement together. Retry failed persistence without repeating the plugin side effect. - Track admitted calls through shutdown and synchronize empty manifests. ## Verification This PR now targets master after #17510 merged as `4e27a82a4265c01f3ab4f05aaecc010a0026de75`. The normal landed-parent merge preserves the entire verified `df78ca3b5e` tree byte for byte; only final publication docs change afterwards. The PR still differs in its original seventeen paths and has no database schema or migration change. Primary test revision: `36be0bd73363680d7666d94ff5f1dd28b27f4235`, on parent `ba80060407`. Final integration of parent `51668b8240` preserves all plugin/runtime/shutdown sources from that proof and passes all 63 cases in complete plugin, approval-seeding and scoped-deadline suites at `56ce7c1d2a`, plus server types/build. Only six parent seeding/doc files changed; this is not a rerun of all eleven suites. Only the original seventeen plugin-scheduling paths differ from that parent; all other tracked files, including migration history, match it. - **247 cases pass in eleven complete suites:** plugin scheduler, shutdown, warm standby, plugin environment-driver readiness, delivery coordinator, both retention workers, approval seeding, decision expiry, scoped deadlines and startup. Server typecheck and production build pass. - The new real PostgreSQL lock test failed before the repair: shutdown remained pending while the completion update waited for a held run row. After repair, shutdown reports the statement timeout while the lock remains held; retry after release commits the saved result with exactly one plugin invocation. All 34 scheduler cases pass. - Existing regressions preserve real committed schedule changes, saved deadlines, manual/scheduled reservation, retained success and failure results during unregister, completion/unregister lock order, independent progress, rollback/outer commit, quiet configured states, drain/resume and shutdown cleanup. - The parent's complete seeding/producer run passed 522 cases at its stated revision. Earlier duplicate reservations, overwritten retained results, hourly-clock fixture failures and combined-stack CI/setup failures remain recorded in `doc/idle-plugin-jobs.md`, its dated audit and the stack ledger; they are not erased or recounted as this candidate's tests. - The completion timeout is transaction-local and defaults to thirty seconds per PostgreSQL statement. It does not bound every shutdown/network operation or make plugin side effects safe to replay. No full local repository-suite or deployed host-sleep claim is made. Commands: server Vitest for the eleven suites above; server typecheck/build. Fresh exact-head CI, Greptile and Ponytail reviews remain required before merge. ## Risks - This uses process-local notifications. Independent SQL writers and other replicas need their own wake integration. - A run left unresolved after process loss is not automatically replayed: the external effect may have occurred. Existing lifecycle/operator reconciliation must resolve it. Live-process settlement failures retry without replay. - Job definitions remain instance-scoped. This PR does not implement company-scoped job dispatch. - Cloud's plugin sleep blockers remain. Internal deadlines do not add durable external wake coverage or prove deployed sleep. - The shared deadline helper now retains direct lifecycle wakes through drain; existing coordinator and decision deadline consumers are tested. ## Model Used OpenAI Codex, GPT-6-based agent, with repository tools, code execution, and GitHub access. The exact serving model version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 21:07:10 -05:00
DottaandPaperclip	5db5734774	fix(plugins): scope operational reads to the caller's companies (#17435) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One instance hosts many companies, and company boundaries are the main isolation contract > - Plugin worker logs, job runs and webhook deliveries store a `company_id` attribution (NULL for instance scope) > - The plugin diagnostics routes checked board org access, but they did not filter those rows by the caller's companies > - A board member of one company could therefore read another company's log messages, run errors and delivery errors > - This pull request scopes those reads to the caller's companies and keeps the instance-wide view for instance administrators > - The benefit is that plugin diagnostics obey the same tenant boundary as the rest of the plugin routes ## Linked Issues or Issue Description **What happened?** A board user who is a member of only company A could read plugin operational rows attributed to company B. `GET /api/plugins/:pluginId/logs` returned log rows with `company_id` values outside the caller's companies. `GET /api/plugins/:pluginId/jobs/:jobId/runs` returned other companies' run rows, including their error text and run logs. `GET /api/plugins/:pluginId/dashboard` returned other companies' recent job runs and webhook deliveries. The routes only required board org access and filtered on plugin id or job id. **Expected behavior** A board member sees only operational rows for companies the member can access. An instance administrator (and the local implicit board) keeps the instance-wide view, which includes instance-scope rows where `company_id` is NULL. **Steps to reproduce** 1. Create two companies, A and B, on an authenticated instance. 2. Install a plugin and produce a log row, a job run and a webhook delivery for company B. 3. Sign in as a board user who is a member of company A only. 4. Call `GET /api/plugins/:pluginId/logs`, `GET /api/plugins/:pluginId/jobs/:jobId/runs` and `GET /api/plugins/:pluginId/dashboard`. Before this change, the responses included company B rows. After this change, they do not. **Paperclip version or commit** Reproduced on `2e6d3f7` (`master`, 2026-10-24). The report came from a scoped security audit of `52a66af29ddccf1fcac25f1f6efa9e9958e11fbd`. **Deployment mode** Self-hosted server, authenticated mode. The defect does not depend on the deployment mode, but `local_trusted` always yields an instance-admin actor and hides it. ## What Changed - Added `pluginOperationalReadCompanyIds()` in `server/src/routes/plugins.ts`. It returns `null` for instance admins and the local implicit board, and the actor's company ids for every other board actor. - Scoped `GET /api/plugins/:pluginId/logs` with `inArray(pluginLogs.companyId, companyIds)`. - Scoped `GET /api/plugins/:pluginId/jobs/:jobId/runs` through a new optional `companyIds` argument on `pluginJobStore.listRunsByJob()`. - Scoped the dashboard's recent job runs through the same optional argument on `pluginJobStore.listRunsByPlugin()`, and scoped its webhook delivery query. - Added `server/src/__tests__/plugin-operational-reads-tenant-scope.test.ts`. It starts embedded Postgres, seeds two companies plus instance-scope rows, and asserts the single-company view and the instance-admin view for all three routes. ## Verification - Red first: with the test added and the route unchanged, 3 of 4 cases failed and the response contained the company B rows. - Green after the fix: `pnpm exec vitest run server/src/__tests__/plugin-operational-reads-tenant-scope.test.ts` passed (4 tests). - Related suites: `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-tenant-isolation.test.ts server/src/__tests__/plugin-install-route-security.test.ts` passed (59 tests). - Typecheck: `pnpm --filter @paperclipai/server typecheck` passed. ## Risks - Behavior change for non-admin board members: they no longer see instance-scope rows (`company_id` NULL) or rows for companies they cannot access. Instance administrators keep the full view. For a single-company instance the visible rows are the same, except instance-scope rows. - No schema or migration change. No API shape change. Row-level filtering only. - A plugin that writes diagnostics without a `company_id` will show fewer rows to non-admin users. That is the intended tenant boundary. ## Model Used - Provider: OpenRouter - Model: `deepseek/deepseek-v4.1-flash-1017` (DeepSeek V4.1 Flash) - Harness: OpenCode, with tool use, code execution, and multi-agent delegation - Reasoning: enabled (thinking mode) - Context window: not exposed by the harness ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 09:28:17 -07:00
Dotta	80cdbdbd47	Add plugin framework and settings UI	2026-03-13 16:22:34 -05:00