Files
PaperClipAI/docs/api/routines.md
T
DottaandClaude Fable 5.2 33fac97cad feat(routines): ship the redesigned routine page and let live webhooks be tested again (#22205)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Routines start agent work on a schedule, from a webhook, or on
demand
> - The routine detail page split one routine across eight sections
behind a section sidebar (overview, triggers, variables, secrets,
delivery, runs, activity, history)
> - People had to hunt through sections to see what a routine does and
to change it, and a finished webhook could not be tested again without
starting the routine
> - This pull request makes the approved two-column routine page, one
Edit dialog and one Webhook dialog the real routed page, and lets a live
webhook go back into test mode
> - The benefit is one page that answers what the routine does, when it
runs, who runs it and what it made, with every change in one dialog

## Linked Issues or Issue Description

**Problem**
The routine detail page spreads one routine over eight sections and a
section sidebar. Changing a routine means visiting several sections,
each with its own save bar. A webhook that is already live cannot be
tested again: the API only accepts `setupPending: false`, so testing a
live webhook starts the routine.

**Proposed solution**
Ship the reviewed design (stories `routine-page-design`,
`routine-edit-dialog-design`, `routine-webhook-dialog-design`) as the
routed page in both UI shells, wired to the real API. Let `PATCH
/api/routine-triggers/:id` accept `setupPending: true` for a live
webhook so the next events are checked but never start the routine.

**Alternatives considered**
Keep the sections and restyle them. The owner rejected this: the section
sidebar should go.

## What Changed

- `/routines/:id` renders `RoutinePage` in both shells
(`RoutineDetail.production.tsx` is removed). Status switch, Repeats
rows, default agent, responsible user, project, Asks for, Secrets, Run
rules, Managed by plugin, Instructions with Show all, copy and the
comment count, and History.
- History is paged: 20 tasks per page from `GET
/companies/:id/issues?originKind=routine_execution&originId=…&sortField=created&sortDir=desc&limit=21&offset=…`,
with a real Load more. Coalesced runs show as "folded into a task".
- A failed refetch keeps the loaded page on screen (`useQueryView`); a
History failure shows `QueryErrorState` in place.
- Header: Edit, "…" (Version history, Audit log, Archive / Restore) and
Run now. In the streamlined desktop shell they sit in the breadcrumb
toolbar; elsewhere above the page.
- Version history (`/routines/:id/history`, restore and restored-webhook
keys kept) and Audit log (`/routines/:id/activity`,
`RoutineAuditActivity`) open as sub-views with a way back. Old section
addresses and `?tab=` redirect to `/routines/:id`.
- Edit dialog saves title, instructions, variables, secrets, agent,
project and responsible user in one request against the revision the
dialog opened with (save conflict shows "changed while you were
editing"). The first schedule is created, changed or removed after that.
Priority and delivery settings are never sent, so saved values stay.
- Webhooks are created, changed, tested and removed (with Undo) right
away from the Edit dialog, with the Webhook dialog nested inside it.
Closing the Webhook dialog ends setup or test mode. A webhook left in
test mode shows "Testing · events don’t run" on the page and offers
Finish testing.
- Responsibility (only with `enableResponsibilities`): the page shows
direct and through-project responsibilities from the link lookup; the
dialog links or unlinks the direct one. With the flag off nothing is
fetched or shown.
- The routines list creates routines with the same dialog in create
mode, filed in the open folder, and opens the new routine page.
- Server: `setupPending: true` on a live webhook enters test mode
(planned by a pure `planTriggerUpdate`, which also gives the activity
action) (422 for other trigger kinds; 409 for a disabled or removed
webhook or an archived routine). Test deliveries keep the last real
delivery as `lastWebhookDelivery.previous`, and finishing testing
restores it. Activity `routine.trigger_test_started`.
- Additions to the reviewed design, from review, so no capability is
lost: the schedule popover has Pause/Resume; standing-check and API rows
open a small popover with Pause/Resume and Remove; removing a trigger
from the page leaves a "… removed. Undo" line under Repeats. Stories:
`StandingCheckOpened`, `ScheduleTriggerPaused(Opened)`,
`RemovedWithUndo`.
- Save is several requests (routine, schedule, responsibility). It
changes only the schedule the form opened with, records each finished
step, and a partial failure says what was saved and that saving again
finishes it. The dialog cannot be dismissed while saving. Failed reads
show a Retry state, never empty choices.
- Removed: the routine section sidebar and contextual sidebar, the
trigger wizard, trigger cards, save bar, section components and their
tests.
- Storybook: the three routine stories moved to `stories/routines` as
current production stories, a new story mounts the routed page against a
fixture API, and stories that used the old page were updated or removed.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/routines-e2e.test.ts` (17/17) and
`src/__tests__/routines-service.test.ts` (105/105).
- `pnpm exec vitest run` on the routine page, routines list, routine
form, webhook instructions, Layout, shell navigation, audit and editor
tests: 275 tests pass.
- `cd ui && npx tsc -p tsconfig.json --noEmit`, `node
scripts/check-token-gates.mjs`, `node scripts/check-maps.mjs`, `node
scripts/storybook-catalog.mjs --check`: pass.
- `pnpm verify` exits 2 (global CI input changed); required CI supplies
the broad coverage.
- Manual: an isolated worktree instance (`pnpm paperclipai worktree init
--empty`, `pnpm dev`) with a seeded routine. Headless Playwright: an old
`/triggers` address redirects, no section sidebar, History shows 20 then
23 after Load more, the schedule popover opens, pause and resume, Edit →
Add webhook → created key → Finish, title save, Test webhook → Finish
testing, Version history, Audit log shows "Started testing a webhook",
phone width has no horizontal overflow, and Create routine from the list
opens the new page. No console errors or failed API calls.

## Risks

- Behaviour change: the section pages are gone. Old links redirect to
the single page.
- The delivery-setting editors (concurrency, catch-up, activity gate)
and priority are removed from the UI. Saved values stay, and the API
still accepts them.
- The create dialog no longer offers a folder picker or delivery
settings. A new routine goes into the folder the list is showing and
takes the default settings.
- Replace key in the Webhook dialog acts at once, as in the reviewed
design. The old confirmation is gone.
- No migration. `lastWebhookDelivery.previous` is an optional field
inside the existing jsonb column.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits, headless browser) and subagents for the
server change and the Storybook move.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-18 13:13:15 -06:00

16 KiB
Raw Blame History

title, summary
title summary
Routines Recurring task scheduling, triggers, and run history

Routines are recurring tasks that fire on a schedule, webhook, or API call and create a heartbeat run for the assigned agent.

List Routines

GET /api/companies/{companyId}/routines

Returns all routines in the company.

Get Routine

GET /api/routines/{routineId}

Returns routine details including triggers.

Routine and trigger resource IDs must be complete UUIDs from the API response. Short prefixes and malformed IDs return the same 404 as a missing resource. UUID lookup accepts PostgreSQL's existing input forms, including uppercase, compact and braced UUIDs; it does not trim whitespace or resolve prefixes. Company access and routine-assignee permissions still apply. Public trigger IDs in webhook URLs are separate opaque identifiers, not resource UUIDs.

Create Routine

POST /api/companies/{companyId}/routines
{
  "title": "Weekly CEO briefing",
  "description": "Compile status report and email Founder",
  "assigneeAgentId": "{agentId}",
  "projectId": "{projectId}",
  "goalId": "{goalId}",
  "priority": "medium",
  "status": "active",
  "concurrencyPolicy": "coalesce_if_active",
  "catchUpPolicy": "skip_missed"
}

Agents can only create routines assigned to themselves. Board operators can assign to any agent.

Fields:

Field Required Description
title yes Routine name
description no Human-readable description of the routine
assigneeAgentId yes Agent who receives each run
projectId yes Project this routine belongs to
goalId no Goal to link runs to
parentIssueId no Parent issue for created run issues
priority no critical, high, medium (default), low
status no active (default), paused, archived
concurrencyPolicy no Behaviour when a run fires while a previous one is still active
catchUpPolicy no Behaviour for missed scheduled runs

Concurrency policies:

Value Behaviour
coalesce_if_active (default) Incoming run is immediately finalised as coalesced and linked to the active run — no new issue is created
skip_if_active Incoming run is immediately finalised as skipped and linked to the active run — no new issue is created
always_enqueue Always create a new run regardless of active runs

Catch-up policies:

Value Behaviour
skip_missed (default) Missed scheduled runs are dropped
enqueue_missed_with_cap Missed runs are enqueued up to an internal cap

Update Routine

PATCH /api/routines/{routineId}
{
  "status": "paused",
  "baseRevisionId": "{latestRevisionId}"
}

All fields from create are updatable. baseRevisionId is optional for backward compatibility; when provided, stale values return 409 Conflict with the current revision id. Agents can only update routines assigned to themselves and cannot reassign a routine to another agent.

When env contains a variable that an active plugin environment sets on its own tasks (protectedEnvironmentKeys), create and update still save it. The response then includes configurationWarnings entries with code protected_environment_variable, a field such as env.KEY, and a message that names those environments.

List Revisions

GET /api/routines/{routineId}/revisions

Returns append-only routine definition revisions newest first. Snapshots include routine fields and safe trigger metadata only; webhook secret values and secretId are never returned.

Restore Revision

POST /api/routines/{routineId}/revisions/{revisionId}/restore

Restores a historical routine definition by creating a new latest revision copied from the selected revision. Historical revision rows, routine run history, and activity history are preserved. If restoring a deleted webhook trigger requires recreating it, the response can include one-time replacement secret material for that trigger.

Add Trigger

POST /api/routines/{routineId}/triggers

Three trigger kinds:

Schedule — fires on a cron expression:

{
  "kind": "schedule",
  "cronExpression": "0 9 * * 1",
  "timezone": "Europe/Amsterdam"
}

Webhook — fires on an inbound HTTP POST to a generated URL:

{
  "kind": "webhook",
  "signingMode": "hmac_sha256",
  "replayWindowSec": 300
}

Signing modes: bearer (default), hmac_sha256, github_hmac, and none. The replay window applies to hmac_sha256 only: 30–86400 seconds (default 300). Creating a webhook returns trigger and one-time secretMaterial containing webhookUrl and webhookSecret. Save the secret before closing the dialog. Routine details retain the URL; rotate the secret if its value is lost.

API — fires only when called explicitly via Manual Run:

{
  "kind": "api"
}

A routine can have multiple triggers of different kinds.

Update Trigger

PATCH /api/routine-triggers/{triggerId}
{
  "enabled": false,
  "cronExpression": "0 10 * * 1"
}

Delete Trigger

DELETE /api/routine-triggers/{triggerId}

Rotate Trigger Secret

POST /api/routine-triggers/{triggerId}/rotate-secret

Generates a new signing secret for webhook triggers. The previous secret is immediately invalidated.

Manual Run

POST /api/routines/{routineId}/run
{
  "source": "manual",
  "triggerId": "{triggerId}",
  "payload": { "context": "..." },
  "idempotencyKey": "my-unique-key"
}

Fires a run immediately, bypassing the schedule. Concurrency policy still applies.

A run an agent starts through this endpoint is always recorded with source manual; the server ignores a source value an agent sends. Board callers may send manual or api.

triggerId is optional. When supplied, the server validates the trigger belongs to this routine (403) and is enabled (409), then records the run against that trigger and updates its lastFiredAt. Omit it for a generic manual run with no trigger attribution.

Fire Public Trigger

POST /api/routine-triggers/public/{publicId}/fire

Fires a webhook trigger from an external system without a Paperclip login. Send Content-Type: application/json and a JSON object. The trigger authenticates the request using its own secret; an agent or board API key is not a substitute.

Mode Headers and signature
bearer Authorization: Bearer <webhookSecret>
hmac_sha256 X-Paperclip-Timestamp (Unix seconds or milliseconds) and X-Paperclip-Signature: sha256=<hex>; HMAC-SHA256 over the timestamp string, a dot, and the exact body bytes
github_hmac X-Hub-Signature-256: sha256=<hex>; HMAC-SHA256 over the exact body bytes, without a timestamp. X-Paperclip-Signature is also accepted. Configure GitHub to send JSON.
none No signature. Anyone with the generated URL can fire the trigger; keep it private.

For a bearer trigger:

curl --fail-with-body "$WEBHOOK_URL" \
  -H "Authorization: Bearer $WEBHOOK_SECRET" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: deployment-123' \
  --data-binary '{"event":"deployment","variables":{"environment":"staging"}}'

For a timestamped HMAC trigger, sign and send the same bytes:

import { createHmac } from "node:crypto";
const body = JSON.stringify({ variables: { environment: "staging" } });
const timestamp = String(Math.floor(Date.now() / 1000));
const signature = createHmac("sha256", process.env.WEBHOOK_SECRET)
  .update(`${timestamp}.`).update(body).digest("hex");
const response = await fetch(process.env.WEBHOOK_URL, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-Paperclip-Timestamp": timestamp,
    "X-Paperclip-Signature": `sha256=${signature}`,
  },
  body,
});
console.log(response.status, await response.json());

202 returns the routine run, including its status and linked task. The task runs asynchronously, so acceptance does not mean the agent has finished. Concurrency policy can coalesce or skip a delivery while work is active. Payload fields or a nested variables object supply declared routine variables; nested values take precedence. The full payload is retained in run history. For app-webhook triggers, the sender's JSON is also copied into the execution task. Normal webhook intake does not apply a provider-specific field allowlist or a general payload scrubber. Send only data suitable for company-readable routine history; authentication and creation provenance do not certify the payload's privacy. See the Sentry tutorial for source-side scrubbing and setup checks.

For bearer, GitHub HMAC, and unsigned triggers, send a stable Idempotency-Key when retrying a delivery to receive the original run without creating another task. Timestamped HMAC rejects an identical signed delivery with 409, even inside the replay window; stale timestamps and invalid secrets/signatures return 401. Disabled triggers and paused/archived routines return 409. Missing required variables return 422; non-JSON media types return 415 and invalid JSON objects return 400. Rotating a secret immediately invalidates the old one.

Cloud installations use their canonical public origin for generated URLs. Self-hosted installations should set PAPERCLIP_PUBLIC_URL to their HTTPS origin. The reverse proxy must forward this POST endpoint and its authorization, signature, timestamp, and idempotency headers without requiring a browser login. For local HTTPS testing, proxy an isolated test instance through Tailscale Serve; use Funnel only if the sender is outside the tailnet. Existing routine and agent execution controls still apply, including the isolated-worktree execution gate.

Verified creation provenance

The receiving routine and its run are recorded as the new task's direct origin. Paperclip can also retain the run that invoked the routine. Authenticated agent calls to the routine run API record a validated caller run. For an ordinary public webhook, optionally send X-Paperclip-Source-Run-Token containing a current, standard Paperclip run JWT from the same instance and company. The trigger still authenticates with its own secret. Invalid, expired, foreign, or stopped caller credentials return 401 before dispatch.

The source reference is stored as createdByRunId on the routine run, separate from triggerPayload. Sender JSON and plain run-ID headers cannot establish trusted caller ancestry. Without the source credential, a public webhook starts a new external receiving-routine origin. This metadata grants no additional execution or source-read authority. A source reference remains recorded after run retention; unresolved ancestry is unknown, never assumed independent.

Routine-wide execution receipts

Send X-Paperclip-Idempotency-Scope: routine with a stable Idempotency-Key to deduplicate execution receipts across webhook triggers on one routine. The default is trigger. Keep the key and scope stable on retries. Routine scope uses a deterministic receipt identity, so deleting the original trigger does not let the same occurrence start another task. Scope does not change ordinary concurrency, pause, company, permission, or budget controls.

Timestamped HMAC always uses trigger-scoped anti-replay receipts and rejects a routine-scope request with 400. Setup checks create no routine runs; their content-free test receipts remain associated with the setup trigger.

For a worked example that uses the normal webhook with Sentry, see Repair Failed Runs from Sentry.

List Runs

GET /api/routines/{routineId}/runs?limit=50

Returns recent run history for the routine. Defaults to 50 most recent runs.

Agent Access Rules

Agents can read all routines in their company but can only create and manage routines assigned to themselves:

Operation Agent Board
List / Get ✅ any routine ✅
Create ✅ own only ✅
Update / activate ✅ own only ✅
Add / update / delete triggers ✅ own only ✅
Rotate trigger secret ✅ own only ✅
Manual run ✅ own only ✅
Reassign to another agent ❌ ✅

Routine Lifecycle

active -> paused -> active
       -> archived

Archived routines do not fire and cannot be reactivated.

Routine detail navigation

The routine detail page keeps Runs and Activity in the routine sidebar. Runs lists the execution issues for that routine using the shared issue list, including issue status, priority, assignee, and search controls. Activity shows the routine, trigger, and run event timeline without leaving the routine page. The overview links to these same local tabs.

Webhook setup and connection checks

Create a webhook trigger with setupPending: true to configure it safely. While setup is pending, authenticated deliveries return 202 with { "status": "test_received", "test": true, "routineStarted": false, "linkedIssueId": null }. They never create a routine run, task, or agent wakeup. This state survives refreshes and server restarts, and connection checks also work while the routine is paused. Invalid authentication still returns 401.

Routine detail exposes setupPending and lastWebhookDelivery (status, receivedAt, and test) so the wizard can show live connection feedback. The secret is only returned at creation or rotation; it is never stored in browser draft storage or included in routine detail.

Finish setup with PATCH /api/routine-triggers/{id} and { "setupPending": false }. Future deliveries use normal routine dispatch and still respect the routine's enabled state. Test events are not dispatched on activation. Retries with the same Idempotency-Key, GitHub X-GitHub-Delivery, or timestamp-HMAC replay key remain test receipts after activation. Send a unique delivery ID per event so a sender's retries can be recognized. Requests without a delivery ID are new events after activation.

For compatibility, API-created triggers without setupPending: true are immediately live.

To test a live webhook again, send PATCH /api/routine-triggers/{id} with { "setupPending": true }. Only enabled, non-removed webhook triggers on a routine that is not archived can enter test mode: other trigger kinds return 422, and a disabled or removed webhook or an archived routine returns 409. While testing, every authenticated delivery, including real events from the sender, is recorded as a test and never starts the routine. Each test delivery's lastWebhookDelivery.previous holds the last real delivery from before testing (or null). Sending { "setupPending": false } returns the webhook to live and restores that last real delivery as lastWebhookDelivery. Events received while testing are not replayed, and retries of them with the same delivery ID stay test receipts.

Trigger cards support removal with Undo. PATCH with { "archived": true } excludes a trigger from routine detail and scheduling, and rejects its webhook deliveries. Setting archived back to false restores the same URL and credentials. DELETE remains the permanent deletion API.

The webhook wizard and saved trigger editor warn about localhost, private-network addresses, Tailscale hostnames, and HTTP URLs without blocking setup. HTTPS does not imply public access: Tailscale Serve is private, while Funnel can expose the same hostname publicly. These warnings inspect the URL only; they do not resolve DNS or test internet reachability. Use the HTTPS setup guide to configure public access when the sender is outside your network.