Commit Graph
3 Commits
Author SHA1 Message Date
Nicky LeachandPaperclip aab87ce707 Prove workspace delivery before inspecting cleanup candidates (#18493)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Completed tasks leave workspaces that need safe cleanup.
> - Providers without ancestry inspection currently need a recorded
merged PR before cleanup can start.
> - A squash merge changes commit identity, so direct ancestry cannot
prove that delivery.
> - This change proves delivery from the finalized HEAD and captured
destination before storage inspection.
> - Fresh storage checks and a second delivery check still protect
removal.
> - This reuses the existing worker and receipts. It adds no table or
timer.

## Linked Issues or Issue Description

Refs #18410. Related: #13400 addresses workspace identity during
inspection; this change adds destination-aware delivery discovery.

## What Changed

- Resolve direct ancestry or discover an associated merged GitHub PR
without a recorded work product. Require exact HEAD, destination
repository and branch, and a reachable integration commit.
- Use the latest settled finalization receipt as a candidate. Reject
missing, failed, pending, conflicting, or changed evidence.
- Inspect storage only after positive remote proof for providers that
disable ancestry inspection. Recheck HEAD, branch, source identity and
delivery before removal intent.
- Add an optional provider capability for a specific PR recovery hint
and an optional prepared destination receipt. Older providers keep their
existing protocol.
- Wake cleanup when finalization writes settle or gain a workspace
binding. Keep remote-only retries for concrete unresolved delivery. Park
missing or unsupported candidate identity.
- Reuse the source project's explicit GitHub consent for delivery checks
through the existing authorization service. Revalidate the bound
repository and consent before removal; missing or revoked consent
retains storage without switching identities. Public sources remain
anonymous.
- Update the contracts, documentation, access map, and targeted
regressions.

## Verification

- Initial-head GitHub CI passed at
`5acd4c818be6f1764f8319f42891f9956c357c26`, including full build and
typecheck. Standard Greptile is 5/5 with no open threads. The first Apex
review scored 3/5 and found startup latency from remote proof lookups
and removal retry identity changes across provider capability changes.
Both corrections are pushed in `9d1a38df3d`, with the qualified
destination-ref length fix. Red regressions reproduced each issue.
Latest targeted checks passed: cleanup/worker 26, provider integration
41, SDK contract 8. Head `9d1a38df3d` passed standard and Apex reviews
at 5/5 with no open threads, plus all CI including full build/typecheck.
Canary Dry Run initially failed on an npm 404 fetching
`rollup-4.67.3.tgz`; the exact tarball subsequently returned HTTP 200,
and one targeted retry passed without source changes. Aggregate
verification also passed. After #18403 merged, master introduced
conflicts in shared workspace cleanup and metadata handling. Resolved
them in `7700832536`, preserving remote snapshot cleanup and Git
delivery-proof cleanup. Targeted merge verification: 93 tests passed in
the first run (59 opt-in skips), then all 16 remote lifecycle tests
passed with explicit database opt-in; map checks passed. The isolated
database is stopped. The merge head received standard review 5/5. Its CI
found two failures in unchanged test cleanup paths: native runtime
temporary-directory ENOTEMPTY and agent-chat cancellation lock timeout.
Those failures remain recorded and require successful final-head
verification. The subsequent project-consent change has targeted
coverage: authorization36, resolver33 and cleanup19 tests passed,
including revoked consent, anonymous public access and source-binding
races. The positive consent test failed before the fix. Current head
`c772679e05` has green GitHub CI, including full build/typecheck and
both previously failing test jobs, plus standard Greptile 5/5 with no
open threads. Neither cleanup failure required changes to its test path.
Apex also completed at 5/5 on this exact head (check `124651548532`,
summary updated 2026-10-31 20:45 UTC), with no open review threads. The
PR remains draft. Earlier failures remain recorded above.

- Final focused run on the rebased branch: **182 tests passed across six
files**, covering the resolver, delivery cleanup, provider bridge,
finalization notifications, existing workspace service and SDK
contracts. The disposable test database was stopped afterward.
- Targeted resolver tests: 33 passed. These cover ancestry, discovered
squash PRs, wrong HEAD/base/repository, removed integration commits,
bounded responses, cancellation and credential isolation.
- Targeted cleanup integration tests: 17 passed. Provider bridge
integration: 36 passed. SDK contract tests: 7 passed. Finalization
notification tests: 9 passed.
- A negative control that removed the final delivery check failed the
race regression as expected. The check was restored.
- Read-only live GitHub probe: the new resolver discovered PR #18410
from HEAD `3f5ade4cd42b707040e9a82fdaed343e3d736a09`, without PR hints.
It proved integration commit `6484f96c28bfb11a793ce4be6cc5ac15643a5a02`
reachable from observed master
`5bc029bf6c1db6647e96acea323f3723fd1d3eba`. GitHub also advertised the
exact original HEAD at `refs/pull/18410/head`.
- `node scripts/check-maps.mjs` passed. Structural checks do not prove
deployment behavior.
- Local tests are targeted by user request. Full build, typecheck and
test suite run in GitHub CI. A standalone server typecheck could not
complete because local Runner build declarations were absent; no
changed-file diagnostics were reported.
- Initial local execution hit a stalled pnpm wrapper, sandbox socket
denial and stale generated SDK declarations. Direct installed binaries,
the authorized isolated test database and rebuilt SDK resolved those
setup failures. No deployment was performed.

## Risks

- Destructive cleanup still requires fresh storage safety and lifecycle
checks. Remote proof alone never authorizes removal. A persisted removal
intent preserves existing recovery semantics.
- Legacy workspaces without an unambiguous destination or completion
HEAD remain retained. This does not guess the original default branch.
- Private repositories use the explicit standing GitHub consent on the
bound source project, with current membership, grant, repository access
and trust checks. No separate company token is required when that
consent is valid. Missing or revoked consent and unavailable or
rate-limited access retain storage; the worker does not borrow a task or
agent identity.
- A known undelivered commit keeps the existing five-minute retry
because external GitHub changes have no guaranteed ingress to this lane.
This can keep cleanup scheduled. Missing evidence parks instead.
- Providers need the new recovery capability to clean a squash merge
whose original HEAD is reachable only through its PR ref. Providers
without it remain conservative.
- The resolver currently supports GitHub.com. Existing local and
other-provider ancestry paths stay in place.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools and
parallel code agents. The exact runtime model ID and context-window size
were not exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 07:59:17 -08:00
DottaandPaperclip 5f0da562db fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected chats show safe notices when a run starts, waits,
finishes, or fails.
> - A recurring scan projected these notices even when no eligible work
existed.
> - Eligibility depends on committed run state, conversation lineage,
and current authority.
> - Existing database rows can reconstruct that work after restart.
> - This PR wakes a scoped worker from those writes and removes
lifecycle polling.
> - Native progress events keep their separate existing owner until the
next migration.

## Linked Issues or Issue Description

**What happened?**

The chat maintenance timer repeatedly scanned run lifecycle state. Empty
and settled instances still queried the database. A run-status event
alone cannot replace the scan: prompt receipts, continuation receipts,
identity changes, and restored authority can make a run eligible later.

**What did you expect?**

Committed changes wake projection. Startup reconstructs eligible work. A
completed pass stops until another relevant change arrives. Failed work
retains recovery without scanning settled history forever.

**Steps to reproduce**

Configure an automatic chat endpoint. Finish its work, or leave it
unused. Observe recurring lifecycle projection queries from the chat
maintenance callback.

Stacked on #17592. Related foundation: #15625. This PR owns lifecycle
milestones only.

## What Changed

- Preserve upstream coalesced Cloud membership writes together with
locked membership restoration and its committed chat notification.

- Signal milestone eligibility from the real Slack account-link
transaction.

- Register the lifecycle worker with the existing delivery coordinator.
Remove lifecycle projection from the periodic chat callback. Keep native
progress in a separately named lane and preserve the explicit combined
API.
- Reconstruct company/issue scopes from the existing selector. Drain
bounded batches with company fairness. Retry failures, retain writes
through drain/resume, and disarm settled work.
- Signal from supported run, presentation, interaction, receipt,
lineage, conversation, recovery, and authority writers. Preserve their
transaction ownership, including nested savepoints and uncertain
settlement. No new table is added.
- Preserve provider authorization, authored-final suppression,
responsible-user continuity, deduplication, the 24-hour selection
window, and raw-output restrictions. Keep provider interpretation in its
owner.
- Preserve PostgreSQL timestamp precision in lifecycle pagination.
Rejected submillisecond candidates must not repeat the same page and
starve later notices.
- Document producer coverage and alternate-writer dispositions in
`doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`.

## Verification

Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting
master after #17592 merged at
`7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates
the actual landed parent. The entire 77-path milestone patch is
binary-identical to the locally verified preparation across that
ancestry reconciliation; both unrelated master paths are preserved. The
four earlier producer/test integration conflicts retain scoped milestone
and voice notices, current onboarding completion admission, and expanded
identity query/read-scope coverage.

- **330 distinct selected cases pass on the verified unchanged patch:**
305 across 14 complete suites (all five milestone worker suites, run
identity, board claim, Cloud provisioning, auth session routes, Cloud
actor authentication, stale-lock recovery, GitHub credentials, project
Git authorization, and app admission wiring), plus 25 selected real chat
producer cases. The remaining 1,590 chat integration cases were skipped,
not passed.
- The real producer cases cover saved milestone discovery, GitHub and
Slack identity changes, account-link rollback/reconfirmation, ordinary
ingress reactivation, post-close action/delivery receipt settlement, all
ten authority-restoration paths including Cloud synchronization, and
changed native review metadata.
- Worker coverage verifies outer commit, rollback, uncertain-write
recovery, restart, bounded batching/fairness, timestamp precision,
retries, drain/resume, and settled SQL quiet. Authorization and existing
publication deduplication remain with their owning services.
- Server production typechecking, build, and module-boundary checks
pass. The tests above were executed separately. Production typechecking
excludes `src/__tests__` and does not replace those runs. Each database
owner uses its own newly created and app-migrated PostgreSQL fixture.
- [Current-parent acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28)
records commands, counts, producer scope, conflict dispositions, and
limits. Earlier combined-stack counts and failures keep their original
revision scope; local acceptance does not erase old CI failures.
- No new migration, queue table, dependency, workflow, or shared
scheduler contract is introduced. Parent migration 0397, bounded consent
recovery, voice cleanup/retention repairs, and runner checkpoint
restoration remain present.
- **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No
Ponytail review is present, which is an explicitly authorized
nonblocker; no failing review is waived. Older reviews and parent checks
are not current passes. Actual failing Ponytail findings must be fixed;
an absent/stopped/paused turn without a verdict is an authorized
nonblocker. Active unfinished Ponytail uses its original request/start
timer, with a 25-minute monitor cutoff respecting the user's 30-minute
maximum.
- No complete local repository matrix, live provider tenant, external
alarm, physical host/database suspension, or fleet savings claim is
made. Native progress remains a separate following slice.

## Risks

- A future eligibility writer must notify the existing projection
boundary. Independent SQL writers or another active server process are
outside this process-local signal contract.
- Authority and lineage remain sensitive. The worker uses the existing
selectors and publication policy; it does not grant execution or
disclosure authority.
- Native progress and other publication maintenance still have recurring
work. This PR removes one reason, not the entire chat maintenance timer.
- Worker quiet does not prove database suspension, host sleep, or
external alarm coverage. Cloud sleep blockers remain unchanged.
- Merge eligibility requires passing current-head CI and Greptile 5/5
after this actual-master reconciliation. Subsequent stack slices retain
their own gates.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window size are not exposed. Used repository inspection,
reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub
tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available in this session)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run the documented focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 11:07:54 -05:00
Nicky Leach baec3abd67 Authorize background project Git with explicit Connector consent (#17798) 2026-10-26 17:22:49 -07:00