Commit Graph
2 Commits
Author SHA1 Message Date
DottaandPaperclip 623f1dc289 fix(server): schedule secret-proposal expiry from committed changes (#17446)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can propose secrets and bindings for approval.
> - Pending proposals already contain their expiry and encrypted value.
> - The server scans for expired proposals on every maintenance tick,
even when none exist.
> - Committed proposal changes and saved expiry times provide the
required triggers.
> - This PR uses those triggers and preserves scrubbing, approval
checks, and recovery.
> - Empty and completed proposal history no longer causes recurring
expiry queries.

## Linked Issues or Issue Description

Refs #17443. This is the second PR in the sleep migration stack and
targets its branch.

**What problem does this solve?**
The periodic proposal scan contacts PostgreSQL when no proposal needs
maintenance. That obstructs idle database behavior across tenant
instances.

**What is the proposed solution?**
Use transaction-aware notifications for creation and resolution.
Reconstruct the earliest pending expiry at startup. Run cleanup at that
deadline, or retry while reconciliation remains unfinished.

**Why is this useful?**
It removes an idle scan while retaining security checks and physical
ciphertext cleanup. No queue table or migration is required.

## What Changed

- Added intent notifications before secret/binding creation, approval,
terminal transitions, and linked interaction resolution. The existing
outer transaction owns settlement.
- Kept expiry selection, ciphertext scrubbing, dependent rejection, and
authorization in the secret-proposal service.
- Registered a startup and deadline worker in place of the aggregate
maintenance sweep. Empty/terminal states disarm; overdue batches and
failures retain retries.
- Reused shared scoped deadline dispatch for secret expiry. The service
owns company-filtered SQL and expiry policy. Custom-image expiry
continues to use `registerDeadlineWork`.
- Added real PostgreSQL producer, outer-transaction, concurrent-write,
expiry, restart, interaction resolution, and drain/resume tests. Updated
startup wiring fixtures and documented the supported writer/lifecycle
contract.

## Verification

- Landing reconciliation at `c10eb1bcf752b058713e05d3345e5cf38dd3fa72`:
160 tests pass across the complete secret-proposal, scoped deadline,
company batch, coordinator, startup, custom-image service and
custom-image worker suites; no cases skipped in the accepted runs.
Server typecheck passes. This integrates master after #17443 landed;
fresh remote CI and Ponytail review are required before merge.

Current head: `5bbe17ad9ebc4573b08ed1a9323644ef9608ca09`. This remains a
single-duty PR stacked on #17443.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-08-instance-sleep-and-scheduler-inventory.md](https://github.com/paperclipai/paperclip/blob/5bbe17ad9ebc4573b08ed1a9323644ef9608ca09/doc/plans/2026-10-08-instance-sleep-and-scheduler-inventory.md),
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/5bbe17ad9ebc4573b08ed1a9323644ef9608ca09/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Notifications cover supported writers sharing the database signal
owner. Independent SQL writers and other process roots require a
separate ingress contract; startup recovery alone is not live replica
support.
- The shared coordinator and scoped worker also serve later stack
duties. Their defaults remain intact; existing voice, chat, native, and
startup consumers are included in integrated verification.
- Startup now checks overdue proposals immediately. During drain,
physical cleanup waits; approval still rejects expired proposals
synchronously.
- Each due company pass remains bounded to 100 rows. Failed scopes back
off from 30 seconds to five minutes; healthy overdue work keeps
draining. Self-generated write hints coalesce and do not erase a failed
sweep's backoff.
- Heartbeat-disabled behavior and Cloud sleep-safety gates remain
unchanged. This does not establish deployed suspension or external wake
coverage. Tool review and gateway token cleanup remain separate slices.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 14:36:28 -05:00
DottaandPaperclip 5982246771 fix(server): schedule custom-image setup expiry from committed changes (#17443)
## Thinking Path

> - Paperclip manages AI agent work and its execution environments.
> - Interactive image setup sessions must expire even when users leave.
> - The server currently scans these sessions on every maintenance tick.
> - Empty scans keep the instance and database active.
> - Existing session rows already record status and expiry.
> - This pull request uses transaction signals and the shared scheduler
for those rows.
> - Empty and completed setup history no longer create recurring cleanup
queries.

## Linked Issues or Issue Description

**What happened?**

The server checks custom-image setup expiry every heartbeat maintenance
tick, even when no active session exists.

**What did you expect?**

Session writes should update the next cleanup deadline. Existing
provider cancellation and cleanup ownership must remain intact. Empty
state should stay quiet.

**Steps to reproduce**

Run an instance with heartbeat scheduling enabled and no custom-image
setup sessions. Observe repeated setup-expiry reads.

Refs #17145. This is the first PR in the recurring server sleep stack.
Its base is master. The full requested scope and verification contract
are recorded in `doc/plans/2026-10-24-recurring-server-sleep-stack.md`.
Follow-up PRs target their predecessor and each own a separate reason
for recurring work.

## What Changed

- Carry the historical cleanup retry-observation fixture repair and its
existing external test-database hook at the stack foundation. Runtime
scheduling is unchanged.

- Carry the previously verified login-cleanup fixture settlement
correction at the stack foundation so every intermediate CI run receives
it. This is test-only maintenance; custom-image expiry behavior is
unchanged.

- Add a custom-image expiry worker to the existing delivery coordinator.
Remove its maintenance-tick scan.
- Signal initial creation, provider refresh, cancellation/failure, and
promotion inside their owning transaction.
- Keep the active-status predicate, earliest saved expiry, and provider
behavior in the custom-image service. Shared code gains only a queue
topic.
- Preserve startup recovery, bounded retries, concurrent write
generations, and drain deferral. No tables or external alarm changes.
- Add PostgreSQL lifecycle tests and repair this suite's database
cleanup callback. A dedicated external test URL permits disposable
PostgreSQL when the host cannot start another embedded cluster.

## Verification

Current head: `3507c6be1e054fdd93353d5dcf77d4899b44518f`. This remains a
single-duty PR stacked on master.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-08-instance-sleep-and-scheduler-inventory.md](https://github.com/paperclipai/paperclip/blob/3507c6be1e054fdd93353d5dcf77d4899b44518f/doc/plans/2026-10-08-instance-sleep-and-scheduler-inventory.md),
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/3507c6be1e054fdd93353d5dcf77d4899b44518f/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Signals cover supported service producers sharing the DB owner.
Independent SQL and another process require the existing
restart/reconciliation contract.
- Failed provider cancellation retains the existing failed-session and
historical-cleanup behavior. This PR changes scheduling, not teardown
authority.
- Active rows without an expiry preserve the existing behavior; no
deadline is invented. Cloud sleep eligibility remains conservative.
- The worker remains in the existing heartbeat-enabled startup path. It
does not change disabled-scheduler behavior or prove deployed
suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Review follow-up: Added a real producer commit during a paused deadline
read. Updated startup assertions for coordinator ownership and removed
added code comments. The three affected suites pass (102 tests). The
separate login-cleanup suite also passes locally (16 tests); its prior
CI consume test timed out, and the fresh complete CI run passed.
External PostgreSQL tests now use the repository guard for disposable
databases ending in `_test`.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 14:17:01 -05:00