mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
824b25654ce6bcbe9c3b13dab37aca0c2d60247b
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
124b2cd0ad |
fix(server): certify background deadlines for idle tenants (#19001)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Background services own future checks, retries, and recovery work. > - An idle tenant can stop only when a durable wake covers its remaining duties. > - The current alarm catalog covers several domains but leaves these owners blocked. > - A saved future timestamp alone cannot prove that reconstruction or admitted work has settled. > - This pull request binds durable domain snapshots to the actual registered worker revisions. > - The tenant can hand off supported future duties and retain all checks for uncertain or active work. ## Linked Issues or Issue Description No matching public issue was found. Related work: [domain deadline handoff #18656](https://github.com/paperclipai/paperclip/pull/18656), [owned blocker diagnostics #18632](https://github.com/paperclipai/paperclip/pull/18632), and [completed history proof #18646](https://github.com/paperclipai/paperclip/pull/18646). The separate [native idle-session change #18988](https://github.com/paperclipai/paperclip/pull/18988) covers runtime retirement. [Far-future retry re-probes #13534](https://github.com/paperclipai/paperclip/pull/13534) changes recovery responsiveness after an upstream success; this change preserves existing retry decisions. **What happened?** Future status-card, external-object, skill-sync, watchdog, retry, and quota duties lacked complete owner-bound coverage in the idle deadline proof. A completed quota task could also retain its monitor policy and timestamp and remain blocked after the scheduler had disarmed it. **Expected behavior** A settled registered owner can hand off its exact supported future deadline. Active work, dirty reconstruction, transaction uncertainty, changed revisions, claims, unknown state, and unsupported duties keep the tenant awake. Completed history retains its data and stops recurring work when the owner has no remaining duty. **Steps to reproduce** 1. Produce a supported future duty through its normal service. 2. Let the registered worker finish reconstruction and admitted work. 3. Request an owned idle report with the negotiated domain deadline catalog. 4. Check the saved earliest deadline and generation before durable alarm registration. 5. Stop and reconstruct the worker. Check that ordinary admission and claims resume useful work once due. 6. Complete a quota task. Keep its saved policy and timestamp. Check that linked unfinished work still blocks sleep and that settled history does not. **Paperclip version or commit** The owned patch was qualified at `699c8569fa34338dd75cdc5731381ec1e2efdb45`. Publication head `491fac368ee72eaac8ae105d53e6bf6b305d8a6a` normally merges actual master `039e0dad2c51c80251e1302a19c5566e039c357a`. All 43 owned files are byte-identical. All 29 new upstream paths are preserved. The later normal current-master integration is `8c912e1ba4fc66397a7341ade59373af8075068f`, described separately below. **Deployment mode** Source checkout with real PostgreSQL 17 and a controlled local Node process. No deployed alarm, tenant suspension, database suspension, or fleet result is claimed. ## What Changed - Add registered-worker reconciliation revisions and invalidate them on stop. - Keep status-card, external-object, and skill deadline interpretation with their owners. - Reuse the watchdog classifier against the supplied database snapshot. - Validate the exact saved retry run and paired queued wake. - Validate legacy quota source bindings and the current assignee or reviewer policy. - Preserve common issue scope, dependency, sibling, and state checks for covered waits. - Compose normalized owner snapshots within the existing bounded deadline proof and recheck revisions after inspection. - Bind actual service and worker instances from server startup. Missing bindings preserve existing blockers. - Retain completed quota policy and timestamps while independently rejecting linked unfinished runs or wakes. - Block unconsumed budget evaluation inputs through the owning queue predicate. Accounting policy and settlement stay unchanged. - Bind completed quota history through fixed-size UUID-array parameters, preserve exact company/task/source associations, and guard external references before the run-ID join. - Batch watchdog subtrees and shared classifier facts without changing the 64-query or two-second proof budget; preserve exact company pairing for first-run evidence. - Extend production-path regression controls and document supported coverage and limits. ## Verification Current integration head: `688abc6cac6016b9fb50e5832f81e2975ed5d49f`. Fresh exact-head checks completed with **58 successes, three intentional skips and one neutral review**, including aggregate `ci / verify`. The complete current Greptile review is **5/5**. The complete current Ponytail verdict is **Ship (4/5)**, with only optional cleanup of an unused MCP proof input; both prior correctness findings are resolved. All review threads and nested pages were inspected. The normal master merges preserve the registered retry deadline owner and current startup completion log after the same readiness await. The earlier focused evidence below remains tied to its tested source revisions. Current-master integration `8c912e1ba4fc66397a7341ade59373af8075068f`: - Normally merge actual master `46fb933b0155d183ce64a8bdc4950d1b58c926f6`. Every nonoverlapping tree blob retains its owning parent across 14,044 paths. All 42 owner-only files and 15 critical binding/proof/readiness files remain byte-identical. Three automatic overlaps preserve the existing watchdog binding, conservative unavailable-reader fallback and optional actual owner arguments. The two scoped-deadline conflicts retain the entire upstream resource-reservation/mid-pass admission/abort algorithm inside the existing reconciliation revision/running/failure fence, with both parent test sets. - Actual TypeScript 7.0.2 server noEmit passed in 9.434 seconds, normal source declaration layers in 12.102 seconds, and four selected test/helper/config layers extending the actual CI configuration in 3.167 seconds. No compiler option weakening, heap override, donor declarations or SDK replacement stub. - Focused PG0 controls passed: 102 scheduler/proof/owner controls in 16.451 seconds, including all 21 scoped-deadline cases; 121 startup controls; and all 107 unchanged instance-settings route cases in 6.456 seconds. The latter first hit a local prewire-guard listener-registration race. Only the ignored guard was corrected to register the same actual OS-bound server port immediately after native listen returns. Closed, unknown, foreign, offhost and PG destinations remain blocked. Original failure evidence is retained; product code and assertions were unchanged. - The exact current UI policy command now reports zero findings (0.362 seconds); its seven scanner controls pass. The earlier dependent PR used an old scanner/source baseline under a newly enforcing trusted workflow and reported 397 findings. Carrying the genuine current master resolves that mismatch without editing unrelated UI sites or relaxing the gate. - Every meaningful worker checked its executing PID/thread/preload hash and matching receipt. PG0 controls opened no PostgreSQL sockets; all owned processes exited. Original dependency-cache/DNS, ignored-harness syntax/type and guard failures remain separate from product results. - This is fresh semantic and focused runtime qualification, not a local whole-repository or current-head PostgreSQL acceptance rerun. The upstream workspace-busy parked-retry/terminal-holder notification path remains unchanged by this merge and needs its separate current-graph producer check. Historical six-case/scale results below stay tied to their measured sources. A matching controlled default-cadence follow-up is in #19191; no fresh current-head two-cycle or deployed suspension claim follows from this merge. - All 59 exact-head check contexts now pass (55 success, one neutral, three intentional skips), plus the legacy security status. The complete edited Greptile review is 5/5 for this exact commit, with all root/nested pages inspected and no unresolved threads. Ponytail stopped without completing its new turn; it supplied no new verdict or finding. Prior actual findings remain addressed. Earlier scores/checks below belong to their explicit earlier commits. On frozen clean merged source `699c8569fa`: - Real PostgreSQL acceptance: **6/6 passed**,83.483 seconds wall. Actual retry producer creates the exact saved run/wake; stopped and reconstructed owner resumes through current admission, one durable start, real local process output and normal task completion. Includes cancellation, drain arrival, retained quota history with queued-run/wake negatives, watchdog review settlement and budget-input empty/pending/consumed control using the actual owner. - Affected coordinator, deadline, scoped-deadline, recovery and composed-proof controls: **78/78 in five files passed**,32.114 seconds wall, worker 1/no file parallelism. - `pnpm exec tsc --noEmit -p server/tsconfig.json`: **passed**,17.467 seconds. Native TypeScript 7.0.2 and Node 24.22.2. - PostgreSQL tests use one dedicated test database, matching fixture/admin URLs, pool limit four, one worker, and no embedded fallback. Every run recorded unchanged source hashes and a clean checkout. - The eight-second real-clock retry is a domain execution check. It is below the Cloud minimum wake lead and is not a positive accepted hosted sleep cycle. - The completed quota-history product failure and original fixture/protocol failures were preserved; the owning-reader fix passed a real PostgreSQL red/green regression before the fresh merged six-case pass. - After runtime completion, the dedicated database had zero other clients and no owned runtime processes. All three qualification process PIDs exited. The coordinator independently audits and drops the database. - Comment-only cleanup has equal strict TypeScript parser ASTs across 15 files. The final lexical audit finds no added comments relative to actual master. `git diff --check` passed. - Earlier retry fixtures omitted responsible-user membership, advanced JavaScript time past the real PostgreSQL lease clock, or produced stdout without normal task completion. Corrected fixtures use real membership, an eight-second real deadline, and ordinary task comment/completion. The last case exposed legitimate disposition repair rather than duplicate retry promotion. - The quota and watchdog controls first ran before admitted work or the existing 30-second reconstruction duty settled. The stronger tests require an interim blocked report and then exact settled-owner proof. Production deadlines were not changed. The separate completed-quota-history defect was fixed in its reader and verified with linked unfinished-run and wake controls. - The status/object and skill owners reviewed the actual final startup, route and post-backup bindings. Their domain source evidence carries where token-equivalent. The master coordinator coalescing change was reviewed separately. Historical domain PostgreSQL results are not presented as fresh full-suite results on this head. The later master merge adds retry predecessor cleanup checks under the existing issue claim lock. The source audit confirms unchanged saved deadlines and run/wake identities. The three actual retry lifecycle/cancellation/drain cases passed on the batched candidate with this parent (31.817 seconds). Those local dependency controls preceded the current exact-head remote run. Native backup preservation and TestOps changes from master are retained. No pass is inferred for changed dependency behavior solely from unchanged owned files. Review repair published as `643f1e7400`: - The original reader returned `unknown` after seven actual watchdog reviews settled across seven companies. The unchanged composed-report regression passed after batching; the final reader passed in 61.841 seconds, including active queued-work and stopped-owner rejection. - A separate two-company red/green control rejects terminal-run context from another company as first-run grace evidence. Shared facts are serialized once, with per-root IDs and review summaries. - The affected recovery, proof and diagnostic unit controls passed 47/47 (14.575 seconds). Startup owner-contract fixtures passed 121/121 (10.589 seconds). - Diagnostic PostgreSQL controls passed7/7 (12.406 seconds), including the normal budget-input producer and its exact blocker reason. Initial empty-state setup uses the same cleanup as per-case teardown when a dedicated test database is reused. - Actual repository `pnpm exec tsc --noEmit -p server/tsconfig.json` passed 12.710 seconds with no heap override. Final source has no added code comments; whitespace checks pass. - The original query-budget failure, two-company discriminator, initial cross-suite fixture failure and an interrupted incorrect local-driver selector are retained separately. The interrupted run is not a pass. Independent cleanup found zero owned clients or runtime processes. Quota-scale repair published as `6696a37f8a`: - The exact 10,000-row completed-history case reproduced `MAX_PARAMETERS_EXCEEDED`:90,002 bound parameters exceeded the driver's 65,534 limit, and the composed report returned `unknown` despite settled owners. Original failure and actual driver diagnostics are retained. - Fixed pending-work queries use three typed UUID-array parameters plus the result limit, keeping exact company/issue/source tuple bindings. The primary run-ID column stays uncast; a guarded CASE validates the raw external reference before conversion. The 10,000-row cap and 64-query/two-second proof budget are unchanged. - The same 10,000-row composed-report case passed 18.860 seconds, with unfinished run/wake rejection, malformed-reference safety and conservative 10,001-row overflow. Its fixture derives retained history from the actual quota producer and normal task completion, expands valid persisted issue/source pairs, then starts the actual registered owners. - The existing actual quota producer case passed 14.187 seconds, including exact future deadline, retained completion history, unfinished work, real foreign-company source references and noncanonical/malformed references. - Affected owner/proof controls passed 22/22 (26.444 seconds). Production server noEmit passed 16.039 seconds. - CI on 643 identified TS7034/TS7005 in the settled-watchdog fixture's initially untyped collection. It is now explicitly typed. A separate affected-entry semantic check extending the repository's actual `server/src/__tests__/tsconfig.json` passed 15.093 seconds: six affected tests, two owner files, ambient declarations and current imported source, with unchanged strict/module/lib/path settings. The production server config excludes that test directory and is not presented as test-layer proof. No custom suppression or replacement SDK stub was added. - All runs were sequential with pool max 4, worker 1, source/config/command hashes and cleared inherited provider/LIVE/other-database variables. Cleanup found zero owned database clients or local runtime processes. At prior head `6696a37f8a5718cd0f79054969a0084462d41d47`, all required CI gates pass. The exact-head inventory has 55 successes, one neutral review and three intentional skips. Greptile is 5/5. The complete latest Ponytail verdict confirms the quota finding is fixed; its only remaining note is optional unused MCP proof state, not a behavior defect. All three review threads are resolved. The earlier 643 result remains historical. The repository-wide typecheck, tests, build and browser gates passed in the prior `6696a37f8a5718cd0f79054969a0084462d41d47` CI run. The new `8c912e1ba4fc66397a7341ade59373af8075068f` full CI also passes. They were not run as a full local workspace command in the focused PostgreSQL lease. Local evidence above uses bounded external databases and does not claim embedded-cluster, live-provider or deployed-alarm qualification. ## Risks - An unsupported or uncertain owner state must remain blocked. An optimistic exemption could lose work while compute is stopped. - Changes during reconstruction or inspection must invalidate the proof. A memory-only timer is not durable alarm coverage. - Current authorization, budgets, cancellation, accounting, and resource ownership still apply on dispatch. The proof does not grant execution authority. - The supported writer model uses in-process transaction signals and startup/resume reconstruction. Independent SQL writers and other replicas need an explicit wake contract. - Native, review, coalesced, and taskless retry variants remain conservative until their owner cases qualify. Reviewer quota and hosted alarm qualification remain open. - The shared catalog remains opt-in. Legacy callers and missing owner bindings retain their old blockers. ## Model Used OpenAI `gpt-6.1-sol` was the configured model for the delegated service implementations and reviews. OpenAI Codex, based on GPT-6, performed managing review and integration. Both used repository tools and code execution. The exact served backend snapshots and context-window limits were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass on the final merged head - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — exact current-head run passes - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — complete current-head review and all paginated threads inspected - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
453f5fbb78 |
perf(external-objects): schedule scoped refresh obligations (#18384)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
c32ecf6657 |
fix(server): schedule only outstanding merge confirmations (#18382)
Reviewed and verified at
|
||
|
Devin Foleyanddevinfoley
|
cfed739f81 |
fix(external-objects): stop reporting GitHub rate limits as unreachable (#17792)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/11/changes/paperclip/11
<!-- entire-trail-link-end -->
## Thinking Path
> - Paperclip is the open source app that people use to manage AI agents
for work.
> - Paperclip shows the status of external objects, such as GitHub pull
requests and issues.
> - A background sweep refreshes these objects and resolves a GitHub
credential.
> - The sweep read a credential only from a company secret with a
well-known name.
> - An instance without that secret made anonymous GitHub API calls.
Anonymous calls are limited to 60 per hour.
> - When the limit was reached, the sweep showed "unreachable" for
objects that it could not refresh.
> - A server environment token is instance-wide, so it must not expose a
private repository that a company cannot see.
> - This pull request reads a server token for public repositories,
reports a rate limit as its own state, and shows a warning on the pill.
> - The benefit is an accurate status, fewer misleading failures, and no
cross-company private data.
## Linked Issues or Issue Description
No public GitHub issue exists. The sections below describe the bug.
**What happened?**
The board showed "unreachable" for many GitHub links, even after a
related fix was deployed.
**Expected behavior**
The board should show "unreachable" only for real network failures. It
should show a rate limit as its own state and retry later.
**Steps to reproduce**
1. Use an instance with no GitHub token company secret and no server
token.
2. Let the external-object sweep make more than 60 anonymous GitHub
requests in one hour.
3. Open a task with GitHub links.
**Paperclip version or commit**
master (commit
|
||
|
|
3d3b8d4371 |
fix(external-objects): distinguish hidden GitHub repos from genuinely-missing objects (#15906)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Issues reference external objects such as GitHub pull requests. The instance resolves their status and shows it in the UI. > - GitHub returns 404 both when a pull request or issue does not exist and when the repository exists but the configured credentials cannot see it. GitHub hides private repos behind a 404 on purpose. > - The resolver treated every 404 as a terminal not-found verdict. It froze references to inaccessible repos forever, and marked them "Not found (Stale)". > - This pull request probes the repository metadata to tell the two cases apart and keep frozen objects re-checkable. > - The benefit is accurate statuses instead of permanent "Not found" labels for repos the instance cannot see. ## Linked Issues or Issue Description No public GitHub issue exists for this report. **What happened?** - The issue properties panel and markdown decoration showed many GitHub pull request and issue references as "Not found (Stale)" even though the objects exist. - GitHub returns 404 for objects the configured credentials cannot access, not only for objects that are truly gone. - The resolver treated every 404 as a terminal not-found verdict, so those references were frozen and showed a misleading "stale" liveness. - Detected-but-unresolved GitHub objects were labeled "Not yet resolved" in inline markdown, which did not match the pill label "Not yet refreshed". **Expected behavior** - A reference to a repo the credentials cannot see should surface as authorization-required, not as a permanent "Not found". - A truly missing object should still be "Not found". - A terminal object (merged, closed, or confirmed not-found) should not be labeled "Stale", because it is never refreshed again. - Inline markdown statuses should match the pill statuses. **Steps to reproduce** 1. Configure GitHub credentials with no access to a private repo. 2. Add a comment or issue description that links to a pull request in that repo. 3. Wait for the external object resolver to run. 4. Observe the reference labeled "Not found (Stale)". **Paperclip version or commit** - `master` at `e733bdd` (before this change). **Deployment mode** - Self-hosted server. ## What Changed - Added a repository visibility probe to the GitHub external object provider. - A 404 for an object now probes the repository metadata with the same credentials: - Repository visible: the object genuinely is gone. Keep the not-found snapshot. - Repository hidden: return an authorization-required result instead of a terminal not-found verdict. - Probe failed (network, rate limit, 5xx): return an unreachable result and retry later. - A rate-limited repository probe (403 with `x-ratelimit-remaining: 0`) is treated as uncertain, not as missing access, so it retries instead of showing a misleading "Requires auth". - Re-check previously frozen GitHub not-found objects on their normal schedule so they self-heal once credentials or repo visibility improve. A frozen object is only re-checked until a probe confirms the repo is visible; once confirmed deleted, it is terminal and no longer polled. - Cleared any terminal verdict when a refresh fails, so a failed refresh does not freeze an object. - Kept terminal objects from displaying the "stale" liveness label. - Matched the inline markdown status label to the pill status label, so detected-but-unresolved GitHub objects say "Not yet refreshed" instead of "Not yet resolved". - Skipped the liveness suffix in markdown links and status icons when it duplicates the status label (e.g. no more "Requires auth (Requires auth)"). - Cached repository visibility probes per company+repo + credential fingerprint, so many objects in the same repo share one probe per refresh cycle instead of each issuing its own — and a verdict obtained under one token is never reused by a later refresh running under a different credential. ## Verification - Added provider tests for visible, hidden, failed, and rate-limited repository probes, per-company/per-repo probe caching, and probe-verified not-found snapshots. - Added service tests for terminal-object liveness, for re-checking frozen GitHub not-found objects, and for ensuring probe-verified not-found objects are never re-queued. - Added UI tests asserting unresolved GitHub objects render as "Not yet refreshed" (not "Not yet resolved") and that failed-first-refresh labels do not repeat the liveness text. - Ran `npx vitest run src/__tests__/external-objects-service.test.ts` in `server/`: 34 passed (incl. provider + service suites). - Ran the external-object markdown/pill/icon suites in `ui/`: 97 passed. - `ci / Typecheck + Release Registry` and all other CI gates pass on this head. Local `tsc` could not run to completion in the execution sandbox (the sandbox memory cap killed it), so CI typecheck is the authoritative gate. ## Risks - Low risk. The change is confined to the GitHub external object provider and the external object service. - A repo that genuinely does not exist and a private repo both return 404 for the probe. Both now surface authorization-required or a retry instead of an immediate not-found freeze, which stays re-checkable and self-heals. - Re-checking frozen not-found objects adds at most one repository probe per affected object until the probe produces a definitive verdict, after which the object is terminal and no longer polled. Probe results are cached per company+repo+credential, so a credential rotation always re-probes instead of inheriting a stale verdict. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: DeepSeek via OpenRouter, model ID `openrouter/~deepseek/deepseek-v4-flash-latest`, running inside the Paperclip opencode adapter (`opencode_local`). - The model produced and assisted with the reasoning and code changes; the execution workspace, test suite, and git/PR steps above were run by the same agent run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details — the branch name is assigned by the managed execution workspace and the harness requires keeping it - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the current head (all `ci / *`, Greptile, Socket, and review checks pass) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
185515c97b |
fix(external-objects): refresh PR status labels (#10704)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The issue properties panel can show external objects such as GitHub pull requests. > - Those objects are resolved by external-object providers and then displayed as compact status labels. > - A GitHub pull request could remain in the fallback `unknown` state and appear as `Not yet resolved`. > - That label is confusing when the object is known but has not been refreshed yet. > - This pull request refreshes due external objects from the heartbeat scheduler and improves the unknown-status copy. > - The benefit is a properties panel that moves from pending refresh to the real pull request state without a manual refresh. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. I searched for related public issues and pull requests using the terms `Not yet refreshed`, `external objects refresh`, and `external PR status`, and did not find a duplicate implementation. **What happened?** The issue properties panel could show a GitHub pull request as `Not yet resolved` even when the referenced pull request was valid. The object stayed stale unless a manual refresh path ran. **Expected behavior** A known external object should show pending-refresh copy while it waits for provider data. When the scheduler refreshes it, the properties panel should show the provider status such as open, merged, or closed. **Steps to reproduce** 1. Create or view an issue that references a GitHub pull request. 2. Open the issue properties panel. 3. Observe the external object row before a manual refresh has run. **Paperclip version or commit** Current `master` before this pull request. **Deployment mode** Local dev and self-hosted server. **Installation method** Built from source. **Agent adapter(s) involved** Not adapter-specific. **Database mode** Not database-related. **Access context** Board view. **Privacy checklist** I reviewed this description and did not include logs, credentials, private URLs, internal issue IDs, or PII. ## What Changed - Added a heartbeat scheduler tick that refreshes due external objects for active companies. - Kept manual external-object refresh behavior on the same service path. - Changed display copy so known provider objects use liveness labels such as `Not yet refreshed`, while fresh unknown provider statuses show `Status unavailable`. - Added server and UI tests for scheduled refresh and label behavior. ## Verification - `corepack pnpm install --frozen-lockfile` - `pnpm check:token-gates` - `pnpm exec vitest run server/src/__tests__/external-objects-service.test.ts server/src/__tests__/server-startup-feedback-export.test.ts ui/src/components/ExternalObjectPill.test.tsx ui/src/components/IssueProperties.test.tsx ui/src/lib/external-objects.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/ui build` - `pnpm run typecheck:build-gaps` - GitHub PR checks passed on head `0e7fcd30` - Greptile reported 5/5 on head `0e7fcd30` with no unresolved review threads Notes: - I ran recursive typecheck and build first. Both hit container resource limits with exit 137 during concurrent package work, so I reran the affected server and UI targets separately. - An unrelated workspace-runtime auto-port test fails in this container with a PID ownership mismatch. It is outside the files changed here. ## Risks Low to medium risk. The scheduler does more periodic external-object work, so the main risk is extra provider refresh load. The implementation bounds the work to active companies, due non-terminal objects, and 50 objects per company per tick. The path also stays behind the external-objects experimental setting. ## Model Used OpenAI GPT-5 Codex in the Codex execution environment, with shell and GitHub CLI tool use. The runtime did not expose a more specific internal model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2dbaf4a7fa |
External object references across issue surfaces (#8512)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI agents, issues, approvals, comments, and work products. > - The involved subsystem is issue context: markdown links, issue properties, related work, lists, filters, inbox/sidebar status, and plugin-provided external context. > - The gap is that URLs to external systems currently remain mostly plain links, so humans and agents must manually open them to understand status, identity, and liveness. > - This matters because external work objects such as GitHub issues and pull requests are part of the operational state of a Paperclip company. > - The implementation keeps core provider-neutral: shared contracts, storage, sync, routes, and UI surfaces live in core while providers can contribute detection and status resolution. > - This pull request adds the external object reference foundation, GitHub provider support, issue-surface rendering, filters, sidebar/list/inbox signals, and test/story coverage. > - The benefit is that linked external work becomes inspectable Paperclip context without hardcoding every provider directly into the UI. ## Linked Issues or Issue Description No public GitHub issue exists for this work. Feature request: - Problem: URLs in Paperclip issues, comments, documents, and related surfaces do not expose provider status or object identity inline. - Proposed behavior: detect supported external object URLs, persist normalized references, refresh provider status, and render concise status-aware links across issue surfaces. - Users affected: board users, agents, and maintainers who triage issues containing external work links. - Acceptance: external object references are company-scoped, provider-extensible, visible in key issue surfaces, filterable where relevant, and covered by focused shared/server/UI tests. Related PR search: - No open duplicate PRs found for `external object references`. - Closed related prior attempt: #4556. ## What Changed - Added shared external-object contracts, validators, status/liveness helpers, and plugin protocol declarations. - Added database schema and additive migrations for external objects, source mentions, and display metadata. - Added server services/routes for detecting, syncing, summarizing, refreshing, and resolving external objects across issues, documents, comments, projects, and plugins. - Added a GitHub external-object provider plus plugin SDK authoring docs. - Wired UI presentation across markdown links, comments, issue chat, documents, properties, related work, issue rows, filters, inbox/sidebar badges, and Storybook stories. - Rebasing cleanup: moved the branch onto current `master`, repaired stale worktree provision config, hardened environment-sensitive tests/mocks, and removed committed screenshot artifacts from the PR branch to keep the reviewable file set below tool limits. ## Verification - `pnpm exec vitest run packages/shared/src/external-objects.test.ts server/src/__tests__/external-object-routes.test.ts server/src/__tests__/external-objects-service.test.ts ui/src/components/ExternalObjectPill.test.tsx ui/src/lib/external-objects.test.ts` passed after rebasing: 5 files, 56 tests. - Historical branch verification before this PR creation included `pnpm test:run`, `pnpm -r typecheck`, and `pnpm build`; this PR body does not claim those were rerun after the final rebase. ## Risks - Medium: this adds a new cross-surface sync path on issue/document/comment writes. The implementation uses safe sync wrappers so external-object failures warn instead of blocking core mutations. - Medium: the migrations introduce new tables and indexes. They are additive and company-scoped. - Medium: provider-specific URL parsing can miss or misclassify edge cases. Shared canonicalization tests and provider tests cover current GitHub shapes. - Low: UI badge/filter behavior could add visual noise for object-heavy issues; component tests and Storybook stories cover the intended surfaces. > Roadmap checked: `ROADMAP.md` references the plugin system as the current extension path and does not list a duplicate core feature. Related long-range docs discuss external references, work products, preview URLs, and plugin extension points; this PR implements the scoped external-object reference foundation. ## Model Used OpenAI Codex, GPT-5 coding-agent runtime, with shell and GitHub CLI tool use. Reasoning mode: medium. Exact deployed runtime model ID and context window were not exposed in the environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |