mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
docs: add external bot integration skill (#16181)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - External agents need a connection, a work lifecycle, and clear authority. > - The Dot integration exposed gaps in setup, idle work, recovery, and Cloud delivery. > - Future integrations need a practical procedure that preserves those lessons. > - This pull request adds a core skill and six focused reference documents. > - The skill gives contributors concrete prompts, test cases, and failure diagnostics. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** Repository contributor skills for external agent integration. **What's wrong?** The integration process has no single reusable guide for provider discovery, both work directions, setup, managed Runner execution, and live proof. Lessons from Dot are spread across implementation changes. **Suggested fix** Add `create-external-bot-integration` under `.agents/skills`. Keep the core workflow short. Put detailed contracts, examples, tests, diagnostics, and dated evidence in references. Related public implementation: Refs #15402, #15414, #15581, #15691, #15722, #15734. A search found no duplicate skill PR. ## What Changed - Add a core skill for external agents and its Codex interface metadata. - Add references for the release contract, Runner architecture, onboarding, qualification tests, deployment, and the Dot case study. - Include idle intake, native tool use, scoped authority, receipt recovery, and resource cleanup. - Mark provider facts and code paths as dated evidence. Use public PRs for the source trail. - Change only eight files under `.agents/skills/create-external-bot-integration`. ## Verification - Skill validator: passed. - Interface metadata, Markdown fences, and 49 local links and anchors: passed. - `git diff --check`: passed. - PR file scope: verified that all eight paths are inside the skill folder. - `pnpm build` and `pnpm -r typecheck`: passed in the isolated worktree. - `pnpm exec vitest run --project @paperclipai/skills-catalog packages/skills-catalog/src/shipped-catalog.test.ts`: 10 tests passed after the description-budget fix. - The broad local `pnpm test:run` attempt reported 12 native-session lease test failures. It is not a passing full-suite result. One isolated lease case passed. The PR changes no runtime files. - Final commit `bd7aed82f623ee279c800af2b8b87f7bedd6be5c`: all GitHub checks completed, with 55 success results and 2 skipped results. - Greptile reviewed that exact commit at 5/5. There are no open review threads. ## Risks - Provider behavior and code paths can change. The skill requires current code and official documentation checks before use. - The case study describes one integration campaign. It does not claim that every provider supports the same transport or capabilities. - This change adds contributor instructions and metadata. It changes no application code or database schema. ## Model Used OpenAI Codex, GPT-6, with reasoning, filesystem access, shell tools, and code review support. The exact serving endpoint ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
a0651c96f8
commit
69f4bfc3c5
8 files changed
+1337
No files matched your search
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: create-external-bot-integration
|
||||
description: >-
|
||||
Build and qualify external bots such as Dot in Paperclip. Use for two-way work,
|
||||
invitation, identity, wakeup, Runner lifecycle, native tools, recovery, and
|
||||
Cloud delivery. CLI adapters and app-only connectors use their existing skills.
|
||||
---
|
||||
|
||||
# Create an External Bot Integration
|
||||
|
||||
Make an existing external agent a useful Paperclip participant: Paperclip can
|
||||
assign work to it, and it can initiate governed work from its own conversation.
|
||||
It uses its relevant native capabilities, returns inspectable results to
|
||||
Paperclip, and participates in normal admission, authority, and finalization.
|
||||
|
||||
This workflow incorporates the September 30–October 9, 2026 Dot integration.
|
||||
Provider facts and code paths in the references are dated evidence, not a claim
|
||||
that every future bot supports the same APIs. Verify the current implementation
|
||||
and official provider documentation before choosing a transport or changing code.
|
||||
|
||||
## 1. Establish the release contract before narrowing the implementation
|
||||
|
||||
Read [release-contract.md](references/release-contract.md). Infer already-stated
|
||||
requirements and authorization from the conversation. Ask only about consequential
|
||||
unknowns; do not ask the user to repeat “two way,” Cloud, or use of Runner.
|
||||
|
||||
Record the target environment, acting identity, capabilities, allowed human setup
|
||||
steps, delivery/stop expectations, and checkable finish line. Keep one compact
|
||||
matrix in the existing plan or task. Follow repository conventions if creating a
|
||||
plan file. “Works” must name its level: prototype, real provider, real Runner,
|
||||
managed sandbox, deployed tenant, or unattended first-time onboarding.
|
||||
|
||||
Reuse the current implementation and qualification evidence. Inspect the relevant
|
||||
checkout, instructions, app/Cloud PRs, and Runner artifact. Do not recreate work
|
||||
already merged or apply an old plan to a different deployed revision.
|
||||
|
||||
## 2. Prove the uncertain provider boundary first
|
||||
|
||||
Identify supported ways to authenticate, wake the agent, load tools, correlate
|
||||
work, return results, and stop or revoke authority. Check events/subscriptions as
|
||||
well as direct APIs and chat channels. Do not assume that a bot mention wakes
|
||||
another bot, that MCP notifications wake an offline agent, or that a plugin listed
|
||||
in settings is callable in the agent's current conversation.
|
||||
|
||||
Use a synthetic company and stable public HTTPS endpoint. Start with a scripted
|
||||
client for deterministic protocol tests, then the actual provider account:
|
||||
|
||||
1. Fresh connection → callable read tool → verified event round trip.
|
||||
2. Paperclip assignment → autonomous acceptance → persisted result → finalization.
|
||||
3. Idle bot request → one governed intake → useful action → finalization.
|
||||
|
||||
If idle autonomy is in scope, the third journey is a first milestone, not an
|
||||
optional expansion. Do not make the user create a placeholder task for the bot.
|
||||
Label every nudge, operator repair, simulated dependency, and manual tool call.
|
||||
A successful assisted run does not prove prompt-only first-time setup.
|
||||
|
||||
## 3. Reuse Runner while keeping placement and capability separate
|
||||
|
||||
Read [runner-architecture.md](references/runner-architecture.md) before changing
|
||||
execution or authority. For Paperclip, prefer the existing Runner infrastructure:
|
||||
shared lifecycle, receipts, recovery, and a future sandbox boundary for workspace
|
||||
tools. The bot's model and native tools can remain entirely at its provider.
|
||||
|
||||
Use the ordinary configured managed execution environment on Cloud. One admitted
|
||||
assignment reuses its Runner across messages and tool calls; idle connection
|
||||
traffic does not allocate compute. A sandbox hosting Runner does not grant the
|
||||
bot filesystem or shell access. Never add a fallback to the control-plane host.
|
||||
|
||||
Implement dedicated agent identity, durable assignment/operation state, current
|
||||
authority checks, and normal finalization. Treat a proposed alternative backend
|
||||
as an explicit architecture decision with preserved invariants and evidence of
|
||||
benefit, not as a reaction to the latest question about overhead.
|
||||
|
||||
## 4. Build onboarding as a watched, resumable workflow
|
||||
|
||||
Read [onboarding-and-prompts.md](references/onboarding-and-prompts.md). Start at
|
||||
the natural UI entry: invite an external agent, select the provider, copy a setup
|
||||
prompt, give it to the bot, and watch real connection checks. Reuse the product's
|
||||
copy-prompt component and provider-independent invitation flow.
|
||||
|
||||
Distinguish OAuth grant, runtime tool availability, subscription verification,
|
||||
event confirmation, and agent readiness. Generate or renew pending prompts
|
||||
automatically where safe; preserve connected agents. Finish with an explicit Done
|
||||
action. Use stories for waiting, expiry, errors, pause, readiness, and recovery,
|
||||
then verify the real rendered transitions and persistence.
|
||||
|
||||
The prompt must tell the bot to use its own relevant tools/apps/skills and return
|
||||
results. The Paperclip catalog governs Paperclip calls, not the bot's entire
|
||||
runtime. Limit qualification restrictions to the specific test and Paperclip
|
||||
destination. Never turn “use this test connection” into “use no other tools.”
|
||||
|
||||
## 5. Qualify both correctness and the ordinary user experience
|
||||
|
||||
Read [conformance-tests.md](references/conformance-tests.md) and select the rows
|
||||
required by the release contract. Reuse existing Runner and Product E2E fixtures;
|
||||
add regression cases for uncovered contracts instead of building a parallel test
|
||||
system. Keep simulated protocol, actual provider, and actual deployment evidence
|
||||
separate. A provider saying “done” is not proof of task disposition or cleanup.
|
||||
|
||||
Test a lost response, duplicate wake, follow-up at completion, real controller
|
||||
restart, authority loss, and cleanup. For recovery, compare the original run,
|
||||
assignment, session, process identity, and lease. A restart test in which the
|
||||
controller never restarted is invalid. Unknown mutation outcomes require
|
||||
reconciliation, not a new request ID and a second execution.
|
||||
|
||||
If setup needs a reboot, app reattachment, or manual inbox nudge, record the
|
||||
functional success and the remaining onboarding defect separately. Do not quietly
|
||||
teach the tester to navigate around a product defect and then call it self-serve.
|
||||
|
||||
## 6. Deliver one identifiable candidate and verify the deployed path
|
||||
|
||||
Read [troubleshooting-and-deployment.md](references/troubleshooting-and-deployment.md)
|
||||
when setting up a test drive, diagnosing a failure, or deploying. Preserve one
|
||||
environment manifest: checkout/revision, origin, local port, data directory,
|
||||
tenant, image/Runner artifact, active task, and owned resources. Keep secrets out.
|
||||
Retain the environment while the user/provider is testing; do not expire it
|
||||
silently or reset the OAuth database behind a cached client registration.
|
||||
|
||||
Use the repository's adapter, eval, PR, and release workflows when available.
|
||||
The Paperclip `create-agent-adapter` skill covers shared configuration surfaces;
|
||||
`paperclip-runner-release` coordinates a release; `test-it-for-real` validates the
|
||||
actual experience. Discover their installed locations rather than assuming they
|
||||
are siblings of this skill. This skill adds external-bot-specific contracts.
|
||||
|
||||
Carry authorized merge and rollout through the app, ingress, and Runner artifact
|
||||
dependencies. Verify the serving revision and repeat the main journeys on the
|
||||
real tenant with ordinary defaults. Distinguish implementation defects, provider
|
||||
behavior, infrastructure failures, and test/evidence defects. Bound retries by
|
||||
the operation and observed cause; preserve the same resumable deployment request.
|
||||
|
||||
Report supported capabilities, exact live evidence, interventions, unsupported
|
||||
provider controls, remaining gates, and resource disposition. Never equate unknown
|
||||
cost with zero cost, or revoked Paperclip access with confirmed global bot stop.
|
||||
This skill supplies procedure, not new permission to message, grant access, spend,
|
||||
publish, or deploy. Apply the current user's authorization and host policies.
|
||||
|
||||
## References
|
||||
|
||||
- [Release contract](references/release-contract.md): scope, capability matrix,
|
||||
provider research, architecture decision, and finish-line template.
|
||||
- [Runner architecture](references/runner-architecture.md): identity, state,
|
||||
admission, receipts, follow-ups, recovery, files, and concrete Paperclip anchors.
|
||||
- [Onboarding and prompts](references/onboarding-and-prompts.md): UI states,
|
||||
consent, lifetime, tool loading, native capabilities, and reusable prompt text.
|
||||
- [Conformance tests](references/conformance-tests.md): runnable journey recipes,
|
||||
failure matrix, evidence schema, and release gates.
|
||||
- [Troubleshooting and deployment](references/troubleshooting-and-deployment.md):
|
||||
network planes, specific symptom-to-action diagnostics, stable environments,
|
||||
managed execution, and release provenance.
|
||||
- [Dot case study](references/dot-case-study.md): dated incidents, exact historical
|
||||
protocol/tool names, observed limitations, and source trail. Read when comparing
|
||||
a new provider to Dot or checking why a rule exists.
|
||||
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "Create External Bot Integration"
|
||||
short_description: "Build and qualify external agents in Paperclip"
|
||||
default_prompt: "Use $create-external-bot-integration to build and qualify this external agent through Paperclip Runner, including onboarding and both directions of work."
|
||||
@@ -0,0 +1,211 @@
|
||||
# External-bot conformance and qualification
|
||||
|
||||
## Contents
|
||||
|
||||
- [Keep the evidence levels explicit](#keep-the-evidence-levels-explicit)
|
||||
- [Minimum real-provider journeys](#minimum-real-provider-journeys)
|
||||
- [Select additional rows from the capability contract](#select-additional-rows-from-the-capability-contract)
|
||||
- [Record evidence that survives a handoff](#record-evidence-that-survives-a-handoff)
|
||||
- [Historical Paperclip test entry point](#historical-paperclip-test-entry-point)
|
||||
|
||||
## Keep the evidence levels explicit
|
||||
|
||||
| Level | What it proves | What it does not prove |
|
||||
| --- | --- | --- |
|
||||
| Unit / contract tests | Schema, policy, state transitions, selected races | Real provider or deployed routing |
|
||||
| Scripted client + actual Rust/DB | End-to-end protocol and ordinary tool/finalizer integration | Provider tool loading, wake behavior, browser setup |
|
||||
| Scripted client + managed sandbox | Remote artifact/transport, placement, cleanup | Actual external bot or Cloud tenant ingress |
|
||||
| Actual bot + test drive | Provider behavior, real event/tool loop | Published app image or fresh tenant behavior |
|
||||
| Actual bot + deployed tenant | Named deployed candidate and selected journeys | All accounts, long-duration reliability, public distribution |
|
||||
|
||||
For UI, a Storybook fixture proves a rendered state, not that the backend reaches
|
||||
that state. A deep link that bypasses onboarding does not prove discoverability.
|
||||
|
||||
Use the current repo's `paperclip-evals`, `add-runner-eval`, and
|
||||
`add-product-e2e-eval` procedures when relevant and available. Locate an existing
|
||||
case before adding a new fixture. The recipes below specify outcomes; adapt their
|
||||
tool calls to the actual provider. Do not infer live success from this document.
|
||||
|
||||
## Minimum real-provider journeys
|
||||
|
||||
### C01 — first connection from a fresh invitation
|
||||
|
||||
Start with a synthetic company and no binding for the new agent. Enter through
|
||||
ordinary agent creation, not its runtime deep link. Generate the prompt and give
|
||||
it to the real bot. Use only the human actions agreed in the release contract.
|
||||
|
||||
Require all of: correct scoped grant, actual read tool invocation, verified
|
||||
subscription, correlated readiness challenge acknowledged, binding persisted,
|
||||
agent ready, UI Done, and state still correct after refresh. Do not create a work
|
||||
task or allocate a sandbox for the readiness challenge. Save the action log and
|
||||
final screenshot. Record app attachment, reboot, manual consent, or inbox nudges
|
||||
as interventions. A repaired setup can pass functionally while failing first-try UX.
|
||||
|
||||
### C02 — Paperclip assigns useful work with no chat nudge
|
||||
|
||||
Create an ordinary task with a fresh synthetic nonce. Example:
|
||||
|
||||
```text
|
||||
Calculate 17 + 25. Write a task document named integration-proof containing
|
||||
“17 + 25 = 42” and the nonce <RANDOM_NONCE>. Read it back and finish using the
|
||||
assignment's completion contract. Use no workspace or attachment access.
|
||||
```
|
||||
|
||||
Assign through the normal UI/API and wait for the supported event path. Verify
|
||||
explicit acceptance, agent/run attribution, document contents and revision count,
|
||||
canonical completion receipt, run success, task disposition, and cleanup. A
|
||||
nonce-specific artifact proves the correct task was processed. Do not submit its
|
||||
result through a connector yourself and describe that as the bot's execution.
|
||||
|
||||
### C03 — bot initiates work while idle
|
||||
|
||||
With no active assignment, tell the real bot:
|
||||
|
||||
```text
|
||||
Create a Paperclip task titled <UNIQUE_TITLE> and assign it to my responsible
|
||||
human owner. Find that person using the connection's identity/people tools.
|
||||
Start governed work yourself if necessary, then return the created task link.
|
||||
```
|
||||
|
||||
Require one visible bot-authored intake, ordinary admission, actual human assignee,
|
||||
one created target task, and intake completion. Repeat the intake request with the
|
||||
same stable request ID before and after completion: no additional task/run. A
|
||||
different prompt with the same ID must fail. Simulate admission failure: the bot
|
||||
gets an actionable terminal result rather than waiting forever for an offer.
|
||||
|
||||
### C04 — use native capabilities and bring back the result
|
||||
|
||||
Use a user-authorized synthetic source in a native tool already connected to the
|
||||
bot, deliberately absent from the Paperclip assigned-app catalog. Ask a question
|
||||
whose answer requires that source. Require the actual native read and a useful
|
||||
result persisted on the Paperclip task. Keep data/destination authority explicit;
|
||||
do not create a new third-party grant solely to make this test pass.
|
||||
|
||||
Run once after a narrowly scoped qualification test ends. Failure if the bot says
|
||||
it lacks the tool merely because Paperclip does not list it. Also verify the
|
||||
bot cannot substitute another Paperclip identity to bypass a denied operation.
|
||||
|
||||
### C05 — follow-up plus a real controller restart
|
||||
|
||||
Have the bot accept a task and report readiness once, then wait for a normal task
|
||||
comment. Capture run/assignment/session, sandbox lease, Runner marker/PID/birth
|
||||
fingerprint, and controller identity. Restart the owned controller and verify
|
||||
that it actually changed. Wait through a realistic bounded reconnect gap.
|
||||
|
||||
Post the resume instruction as an ordinary Paperclip comment. Require the same
|
||||
assignment/session/Runner, read of that exact comment, one document result, one
|
||||
finalization, one task run, and released resources. A dev supervisor that kept
|
||||
the controller alive invalidates the experiment. A new run completing the task
|
||||
is retry evidence, not recovery evidence.
|
||||
|
||||
Separately force the comment to arrive at completion: unread comments survive
|
||||
for the next turn; comments read by the active assignment do not run twice.
|
||||
|
||||
### C06 — stop and revoked authority
|
||||
|
||||
Cancel a synthetic active assignment, then attempt an effect using its old
|
||||
authority. Verify rejection, fence delivery/acknowledgment, terminal run/assignment
|
||||
reconciliation, and resource release. Repeat protocol tests for pause, reassignment,
|
||||
membership loss, revoked grant, refresh replay, and budget stop as applicable.
|
||||
Do not report global provider termination without provider-level evidence.
|
||||
|
||||
## Select additional rows from the capability contract
|
||||
|
||||
| ID / boundary | Stimulus | Required observation |
|
||||
| --- | --- | --- |
|
||||
| O01 pending renewal | Expire code while dialog open; reopen after refresh | Fresh usable prompt, same agent, no error-as-normal-flow. |
|
||||
| O02 race | Pair while another tab renews | Completed binding preserved; no two active grants or renewal loop. |
|
||||
| O03 lifecycle | Preparing, verifying, manually paused, awaiting approval | Setup allowed only in legitimate setup states; Done waits for readiness. |
|
||||
| O04 provider runtime | Install/update plugin while bot conversation active | Actual new action usable; catalog refresh/attachment instructions truthful. |
|
||||
| O05 lifetime | Rotate refresh token; replay old token; simulate long inactivity | Policy-specific renewal works; replay/revocation deny access; expired grants not revived. |
|
||||
| O06 optional avatar | Same upload twice, other agent/company, restricted token, export/import | Self-scoped, normalized, idempotent; no private asset ID portability leak. |
|
||||
| D01 event duplication | Duplicate and reorder signed event deliveries | Same mailbox/assignment processed; no duplicate effects. |
|
||||
| D02 event delay | Deliver after acceptance deadline or during reconnect | Stale offer rejected/reconciled; no new authority from an old event. |
|
||||
| D03 resubscribe | Subscription expires with outstanding work | Reverification and fresh reference to existing work; no replayed effects. |
|
||||
| D04 cursor concurrency | Commit two mailbox writers around an inbox read | Cursor cannot skip a later-committed result. |
|
||||
| D05 callback authority | Failed ownership challenge, wrong signature, replay where applicable | No unverified subscription or unauthorized state transition; correct signing material stays secret. |
|
||||
| D06 callback egress | Private-address destination, redirect escape, timeout, repeated failure | Destination policy enforced; bounded attempts and visible failure; no internal-network access. |
|
||||
| R01 response loss | Drop response after a document mutation | Same ID retrieves same effect/receipt; revision count stays one. |
|
||||
| R02 ambiguity | Kill controller after external mutation starts | Unknown outcome surfaced; no automatic second effect. |
|
||||
| R03 definite error | Read a nonexistent file/skill | Terminal bounded error; later tools and finish still work. |
|
||||
| R04 stale controller | Late callback after ownership generation changes | Cannot overwrite newer state; late effect evidence retained. |
|
||||
| R05 missing checkpoint | Remove/mismatch recovery evidence after dispatch | Reconciliation required; no replacement assignment disguised as resume. |
|
||||
| R06 completion order | Finish with pending tools or changed report | Rejected until contract satisfied; canonical result retained. |
|
||||
| R07 shutdown | Acknowledged shutdown before monitor's next poll; absent exit code | Clean exit recognized only with authenticated terminal evidence. |
|
||||
| A01 identity | Cross-company IDs, wrong agent/resource, invalid pairing hint | Denied by server; no authority selected by caller fields. |
|
||||
| A02 review/people | Review another agent's task; assign to a human | Legitimate mode-specific access works without granting unrelated task writes. |
|
||||
| A03 live grant loss | Revoke setting/grant during a protected read | No new protected response after revocation. |
|
||||
| A04 API fallback | Try file download/upload through generic API tool | Same attachment/workspace rules apply; no alternate-path bypass. |
|
||||
| S01 skills/apps | Pinned synthetic skill; assigned gateway tool; tool revocation | Correct version/digest and normal gateway auth/receipts; stale catalog can't override authority. |
|
||||
| F01 attachment | Answer only on page two; corrupt/hash-changed/oversized files | Correct bounded paging and integrity errors; no data from another task. |
|
||||
| F02 artifact | Produce and register output | Downloaded bytes and SHA-256 match; task links the usable work product. |
|
||||
| W01 workspace | Traversal, symlink, instance config, credential access | Confined or denied across every file and upload path. |
|
||||
| W02 commands | Detached descendants, timeout, cancellation | Supervisor and descendants contained/terminated; unavailable platforms fail closed. |
|
||||
| W03 concurrent edits | Two writes with same observed hash | At most one commits; second sees conflict. |
|
||||
| C07 placement | Idle reads; many tools/messages in one active task | No idle compute; same admitted Runner/lease reused. |
|
||||
| C08 artifact mismatch | Managed image contains pre-provider Runner | Capability/digest check rejects before advertising work. |
|
||||
| C09 terminal failure | Allocation timeout, failed admission, orphan offer | Clear terminal receipt, no blocked future work, tracked cleanup. |
|
||||
| C10 deploy | Fresh tenant and shipped images through real ingress | OAuth, anonymous consent, both work directions, and cleanup pass. |
|
||||
|
||||
Long-duration subscription/refresh tests need real provider time as well as fake
|
||||
clock unit tests. Select a duration consistent with the release claim; a five-minute
|
||||
demo does not establish multi-day availability. Do not leave a paid soak running
|
||||
without authorization, ownership, limits, and a supported monitoring mechanism.
|
||||
|
||||
## Record evidence that survives a handoff
|
||||
|
||||
Use one row per case and candidate; do not combine incompatible partial runs into
|
||||
one green campaign. A useful record is:
|
||||
|
||||
```yaml
|
||||
case: C05
|
||||
candidate:
|
||||
appCommit: <sha>
|
||||
ingressCommit: <sha-or-not-applicable>
|
||||
appImageDigest: <digest>
|
||||
runnerArtifactDigest: <digest>
|
||||
environment: <tenant-or-isolated-instance>
|
||||
client: actual-provider # or scripted
|
||||
startingState: <fresh/connected/accepted/etc>
|
||||
identities:
|
||||
run: <id>
|
||||
assignment: <id>
|
||||
session: <id>
|
||||
lease: <id>
|
||||
controllerBefore: <id>
|
||||
controllerAfter: <id>
|
||||
processFingerprintBefore: <redacted-reference>
|
||||
processFingerprintAfter: <redacted-reference>
|
||||
interventions: []
|
||||
observed: <receipts, document revisions, task state>
|
||||
evidence: <log/screenshot/artifact links>
|
||||
cleanup: <released/destroyed/pending with provider receipt>
|
||||
outcome: PASS # FAIL / BLOCKED / UNVERIFIED / NOT_APPLICABLE
|
||||
limits: <which release claims this does not establish>
|
||||
```
|
||||
|
||||
Keep secrets, callback signing keys, access/refresh tokens, pairing codes, and
|
||||
private source data out of reports and public PRs. Internal IDs may belong in a
|
||||
private qualification record; follow the repo's public issue/PR rules.
|
||||
|
||||
## Historical Paperclip test entry point
|
||||
|
||||
The October 9 integration used `server/src/__tests__/dot-runner.test.ts` for
|
||||
actual Rust/PostgreSQL tests and an opt-in private Daytona case:
|
||||
|
||||
```sh
|
||||
pnpm exec vitest run server/src/__tests__/dot-runner.test.ts -t 'private Daytona Rust'
|
||||
```
|
||||
|
||||
That candidate read `PAPERCLIP_DOT_DAYTONA_LIVE=1`, `DAYTONA_API_KEY`, an immutable
|
||||
`PAPERCLIP_DOT_DAYTONA_IMAGE`, optional `PAPERCLIP_DOT_DAYTONA_RUNNER_BINARY`, and
|
||||
`PAPERCLIP_DOT_DAYTONA_EVIDENCE`. Locate the current fixture and inspect its flags,
|
||||
cost bounds, dependencies, and `finally` cleanup before running. Do not print
|
||||
credentials. This case used a scripted MCP client; it never replaced real Dot
|
||||
qualification. The command/flags are historical anchors, not a guaranteed current CLI.
|
||||
|
||||
Run the smallest decisive tests while iterating, then the repository-required
|
||||
typecheck/test/build and review gates on the final candidate. A stale broad suite
|
||||
running across source edits is not evidence for the final revision. Separate
|
||||
unrelated infrastructure failures from product regressions, retain failed attempts,
|
||||
and re-run affected cases after fixes.
|
||||
@@ -0,0 +1,188 @@
|
||||
# Dot integration: dated case study and source trail
|
||||
|
||||
## Contents
|
||||
|
||||
- [What the user actually wanted](#what-the-user-actually-wanted)
|
||||
- [Incidents and the reusable judgment](#incidents-and-the-reusable-judgment)
|
||||
- [Historical Dot protocol and tools](#historical-dot-protocol-and-tools)
|
||||
- [What was and was not proven](#what-was-and-was-not-proven)
|
||||
- [Sources and how to inspect them](#sources-and-how-to-inspect-them)
|
||||
|
||||
This is evidence from September 30–October 9, 2026, reviewed from raw records on
|
||||
October 13. It is not current provider documentation or permission to repeat old
|
||||
operations. Use it to recognize failure modes and locate prior implementation.
|
||||
No credentials, pairing codes, temporary ingress hosts, or personal app contents
|
||||
are retained here.
|
||||
|
||||
## What the user actually wanted
|
||||
|
||||
- The existing personal Dot, not a replacement generic API agent.
|
||||
- Paperclip could initiate work; Dot could read/write and initiate work while idle.
|
||||
- Shared new Runner infrastructure.
|
||||
- Eventually a full company participant: people, tasks, skills, apps, optional files.
|
||||
- An external-agent invitation with a copy prompt and live checks.
|
||||
- Persistent Paperclip connection until revoked.
|
||||
- Cloud deployment with managed execution and no arbitrary host execution.
|
||||
- Dot uses its own relevant tools and returns the result to Paperclip.
|
||||
|
||||
Idle autonomy was explicit on day one and in the first production plan. Shipping
|
||||
task-only access was an implementation omission, not missing user specification.
|
||||
Cloud-first delivery, exact capability parity, allowed setup interventions, and
|
||||
latency/concurrency needed an earlier explicit release contract.
|
||||
|
||||
## Incidents and the reusable judgment
|
||||
|
||||
| Date / incident | Observed outcome | Rule carried into this skill |
|
||||
| --- | --- | --- |
|
||||
| Sep 30: initial research favored Slack / another agent API | User pointed out MCP Events; official docs then supported testing that route. | Search server-initiated events before choosing transport; existing personal bot is a different product from an API agent. |
|
||||
| Oct 2: lab assumed OAuth request ID was UUID | Real consent failed; request ID was opaque. | Preserve protocol identifiers as specified. |
|
||||
| Oct 2: DB restart erased cached client registration | Provider returned unknown client/redirect; reconnect state had drifted. | Stable fixture includes registration and public origin, not just app health. |
|
||||
| Oct 2: settings listed tools, Dot could not call them | Endpoint was stopped; Dot loaded actions later; replacement lab completed arithmetic round trip. | Keep test service alive during provider setup; test actual runtime invocation. |
|
||||
| Oct 3: production Runner implementation | Rust/DB integration worked, but the new endpoint still lacked real-account qualification. | Reference harness, production Runner, and actual provider are distinct proof levels. |
|
||||
| Oct 6–7: merged MCP gateway integration | Reused consent, verification, refresh, revocation; fixed normal shutdown race. | Reconcile shared infrastructure and its contracts instead of retaining a forked prototype. |
|
||||
| Oct 7: public Tailscale metadata worked locally/externally, provider discovery timed out | No request reached app; Cloudflare discovery worked; provider browser had separate failures. | Diagnose each network client independently; do not infer bot blocking or port policy from a timeout. |
|
||||
| Oct 7: uncompressed large bundle | Public download improved from 11.3 s to 2.9 s after compression; discovery problem remained. | Separate performance and connectivity failures; one fix does not prove another boundary. |
|
||||
| Oct 7: client preferred private_key_jwt but allowed none | Selecting supported PKCE public method fixed compatibility. | Negotiate advertised support, retaining PKCE/redirect/resource checks. |
|
||||
| Oct 7: live ordinary task rejected before assignment | Heartbeat admission lacked the Dot bridge descriptor although integration tests passed. | Enter through ordinary product scheduling, not only a broker fixture. |
|
||||
| Oct 7: “create hello for my owner” refused while idle | Added governed intake and human assignees; actual Dot completed it. | Both directions are first-class acceptance journeys. |
|
||||
| Oct 7: pairing ready but agent config missing binding | Required another form save until atomic persistence fix. | One logical setup operation must persist all required state. |
|
||||
| Oct 7: missing-file read never settled | Turn could not close; terminal read-error receipt fixed it. | Distinguish definite failure from ambiguous mutation outcome. |
|
||||
| Oct 7: workspace execution and file access expansion | cwd was insufficient; review found `.paperclip` leakage and API bypasses; macOS commands later disabled. | Trace effects and test all access paths; fail closed when confinement is unproven. |
|
||||
| Oct 7: late event | One event arrived roughly 32 minutes later; completed work was not repeated. | Event delivery is asynchronous; test late offers and never claim a latency SLA from one success. |
|
||||
| Oct 8: cloud architecture reconsidered repeatedly | User chose common Runner for lifecycle reuse and potential future workspace tools. | State decision criteria and retain the decision; placement is separate from tool access. |
|
||||
| Oct 8: expired setup prompt and nested copy window | Added automatic renewal, automatic close on connection, progress footer and Done. | Normal asynchronous progress should not require manual repair. |
|
||||
| Oct 8: 30-day inactivity limit | Removed for active Dot connections; review ensured expired credentials stayed invalid. | Separate persistent grant policy from token/code/lease lifetimes. |
|
||||
| Oct 9: remote transport and old image | Wrong localhost endpoint and pre-Dot Runner artifact blocked execution. | Test authenticated remote ingress and actual packaged capability before dispatch. |
|
||||
| Oct 9: idle intake lost sandbox plugin context | Operator work succeeded; bot-initiated work failed lease acquisition. | Both entry paths must use configured application services. |
|
||||
| Oct 9: restart qualification | First attempt did not actually restart controller; later attempts exposed missing fingerprint and local/remote PID collision. | Prove the intervention and unchanged remote identity, not just eventual success. |
|
||||
| Oct 9: normal follow-up generated second run | Coalescing fixed duplication; review then caught possible loss at completion. | Consume a wake only after exact comment delivery/read evidence. |
|
||||
| Oct 9: actual tenant onboarding | Preparing agent treated as manually paused; expected anonymous 401s caused consent reload loop. | Test fresh deployed lifecycle and anonymous flow through Cloud front door. |
|
||||
| Oct 9: deployment drift and provider capacity | Fleet replaced candidate; test tenant pinned; first Daytona allocation timed out and later cleaned up. | Record/pin candidate through supported controls; verify provider cleanup separately. |
|
||||
| Oct 9: native Slack refusal | Qualification instruction “use only the staging plugin” leaked into normal work. | Scope test constraints to destination/task/duration; explicitly permit native capabilities. |
|
||||
|
||||
## Historical Dot protocol and tools
|
||||
|
||||
Transport was authenticated MCP inward and signed MCP Events outward. The
|
||||
qualified event was `paperclip.dot.mailbox_updated`. Subscription methods included
|
||||
`events/list`, `events/subscribe`, and `events/unsubscribe`; these are protocol
|
||||
methods, not extra callable tools. The experiment used MCP version `2026-07-28`.
|
||||
Verify current support and account eligibility instead of copying that version.
|
||||
|
||||
The final top-level MCP catalog had 16 tools:
|
||||
|
||||
| Purpose | Tool names |
|
||||
| --- | --- |
|
||||
| Identity and proactive work | `paperclip_dot_capabilities`, `paperclip_dot_request_turn`, `paperclip_dot_tasks`, `paperclip_dot_request_work` |
|
||||
| Setup and inbox | `paperclip_dot_pair`, `paperclip_dot_inbox`, `paperclip_dot_confirm_event` |
|
||||
| Assignment | `paperclip_dot_read`, `paperclip_dot_accept`, `paperclip_dot_tool` |
|
||||
| Lifecycle and receipts | `paperclip_dot_progress`, `paperclip_dot_finish`, `paperclip_dot_operation_status`, `paperclip_dot_renew`, `paperclip_dot_control_ack` |
|
||||
| Optional profile | `paperclip_dot_set_avatar` |
|
||||
|
||||
The nested Runner catalog was read per assignment. Bot called its Paperclip tool
|
||||
names via `paperclip_dot_tool`; native tools were called in the bot's own runtime.
|
||||
Standard completion was `paperclip_finish` or `paperclip_block` through the bridge,
|
||||
then `paperclip_dot_finish` with the accepted canonical report. Do not collapse
|
||||
these steps into an unverified conversational “done.”
|
||||
|
||||
Historical constants help reproduce old cases, not set new-provider defaults:
|
||||
|
||||
- Instance settings: **Assistant connections (MCP)** and **OpenAI Dot**, with
|
||||
authenticated sign-in and a stable `PAPERCLIP_PUBLIC_URL`. The old
|
||||
`PAPERCLIP_ENABLE_OPENAI_DOT` environment flag no longer enabled the provider.
|
||||
Self-hosted Dot did not require the general Runner picker flag; Cloud additionally
|
||||
required native Runner configuration and the managed execution plugin.
|
||||
- Pairing code: 15 minutes, one use. Access token: 15 minutes.
|
||||
- Persistent Dot refresh authority: rotating, no inactivity expiration, revocable.
|
||||
- Acceptance window: 10 minutes. Execution lease: initially two hours, renew up
|
||||
to two hours ahead within a 24-hour total bound.
|
||||
- Broker operations: initially 200, expanded to 4,000. One active assignment.
|
||||
- Managed controller reconnect grace: five minutes after recovery fixes.
|
||||
- Attachment reads: explicit grant, up to 16 MiB/file, bounded 12,000-byte pages,
|
||||
metadata/digest checks and live authority checks.
|
||||
- Cloud workspace access: off. Self-hosted Linux command execution required
|
||||
confinement; macOS commands ultimately unavailable.
|
||||
- Avatar: `paperclip_dot_set_avatar` accepted `{ "imageBase64": "<raw bytes>" }`
|
||||
or `null` to restore the preset. PNG/JPEG/WebP, at most 512 KiB and 16 megapixels;
|
||||
normalized to metadata-free PNG no larger than 512 pixels per side. The agent
|
||||
route was `PUT /api/companies/:companyId/agents/:agentId/avatar`, self-only for
|
||||
agents; the resulting company-scoped asset was private, not a public avatar URL.
|
||||
|
||||
These were implementation choices subject to later change. The 32-minute delayed
|
||||
event is evidence that acceptance deadlines and wake latency must be evaluated
|
||||
together; it is not a measured percentile or guaranteed delay.
|
||||
|
||||
## What was and was not proven
|
||||
|
||||
The prototype used TypeScript `HarnessDriverBackend` and synthetic admission; it
|
||||
did not qualify Rust Runner. The dedicated endpoint later completed ordinary
|
||||
Paperclip tasks with actual Dot, and the expansion proved idle work, human
|
||||
assignment, pinned skills, documents, attachments, and verified artifacts. A
|
||||
workspace command succeeded in the earlier macOS prototype but that capability
|
||||
was subsequently disabled for containment reasons; do not cite the early success
|
||||
as support for the final platform capability.
|
||||
|
||||
Managed qualification used both a scripted MCP client and the real Dot. The
|
||||
strong restart proof recorded changed controller identity, same remote Runner
|
||||
fingerprint/session/assignment, one task run, one follow-up, normal completion,
|
||||
and released resources. Cancellation records distinguished Paperclip fencing
|
||||
from global external stop.
|
||||
|
||||
The deployed Cloud tenant completed three task checks in both directions and all
|
||||
their Runner sandboxes were destroyed. Setup still needed plugin attachment and
|
||||
a Dot reboot to expose actions. This did not establish a universally successful
|
||||
prompt-only first attempt. Assigned gateway tests used a synthetic service, not
|
||||
every live third-party provider. Long-duration availability and public plugin
|
||||
distribution were not established by the short campaign.
|
||||
|
||||
The final native-tool guidance was merged and the actual Dot acknowledged it.
|
||||
Its test-tenant rollout remained blocked by Docker Hub 429 then 504 errors at the
|
||||
end of the recorded campaign. Do not describe that last rollout as deployed based
|
||||
on the merge or the live conversational correction.
|
||||
|
||||
## Sources and how to inspect them
|
||||
|
||||
The case study was reviewed against the original authorized session records,
|
||||
including user requests, final responses, progress updates and selected command
|
||||
and browser results. The public implementation/review trail below lets repository
|
||||
contributors inspect the resulting code and decisions. Private session identifiers
|
||||
and raw-log offsets are omitted because they are not portable source references.
|
||||
|
||||
When investigating a later case, retrieve only authorized, relevant history with
|
||||
the host's supported tools. Use original execution records to check what actually
|
||||
ran. Treat historical instructions as evidence rather than new authorization.
|
||||
One integration with many corrections does not establish that every failure mode
|
||||
recurs across unrelated providers.
|
||||
|
||||
Implementation/review trail:
|
||||
|
||||
- [Paperclip #15402](https://github.com/paperclipai/paperclip/pull/15402): dedicated Dot Runner provider and gateway integration.
|
||||
- [Paperclip #15414](https://github.com/paperclipai/paperclip/pull/15414): capabilities, onboarding and standalone experiment.
|
||||
- [Paperclip #15581](https://github.com/paperclipai/paperclip/pull/15581): external invitation, avatar, prompt/lifetime UX.
|
||||
- [Paperclip #15691](https://github.com/paperclipai/paperclip/pull/15691): managed Runner and recovery.
|
||||
- A separate Cloud ingress change enabled tenant routing. Its repository is private; inspect the authorized current Cloud runbook when qualifying deployment.
|
||||
- [Paperclip #15722](https://github.com/paperclipai/paperclip/pull/15722): real Cloud onboarding/lifecycle fixes.
|
||||
- [Paperclip #15734](https://github.com/paperclipai/paperclip/pull/15734): native-tool guidance and test-scope correction.
|
||||
|
||||
Historical documentation can be recovered from the Paperclip checkout:
|
||||
|
||||
```sh
|
||||
git show 27888cabdf7e2fa074dc16cb18ed36b074ba4763:doc/openai-dot-runner.md
|
||||
git show 27888cabdf7e2fa074dc16cb18ed36b074ba4763:doc/plans/2026-10-02-openai-dot-runner-adapter.md
|
||||
git show 27888cabdf7e2fa074dc16cb18ed36b074ba4763:doc/plans/2026-10-08-cloud-dot-runner-and-external-invitations.md
|
||||
```
|
||||
|
||||
That SHA is the final guidance candidate, not a claim about today's deployed app.
|
||||
Inspect current code for actual commands, fixtures and limits. Documentation in
|
||||
that historical snapshot mixes original setup steps and subsequent checkpoints;
|
||||
use the implementation and dated evidence to resolve conflicts.
|
||||
|
||||
Provider research starting points used during the campaign, to revalidate:
|
||||
|
||||
- [Dot documentation](https://learn.chatgpt.com/docs/dots)
|
||||
- [MCP Events](https://developers.openai.com/plugins/build/mcp-events)
|
||||
- [Plugin authentication](https://developers.openai.com/plugins/build/auth)
|
||||
- [Connect ChatGPT](https://developers.openai.com/plugins/deploy/connect-chatgpt)
|
||||
- [Secure MCP tunnels](https://developers.openai.com/api/docs/guides/secure-mcp-tunnels)
|
||||
|
||||
Do not infer that another external agent supports MCP Events, Dot's tool names,
|
||||
private-plugin onboarding, or the same approval/distribution rules.
|
||||
@@ -0,0 +1,200 @@
|
||||
# Onboarding, consent, and agent instructions
|
||||
|
||||
## Contents
|
||||
|
||||
- [Design from the user's entry point](#design-from-the-users-entry-point)
|
||||
- [Model the states the user will actually encounter](#model-the-states-the-user-will-actually-encounter)
|
||||
- [Separate setup approvals from repeated intervention](#separate-setup-approvals-from-repeated-intervention)
|
||||
- [Separate the four lifetimes](#separate-the-four-lifetimes)
|
||||
- [Generate instructions the receiving agent can execute](#generate-instructions-the-receiving-agent-can-execute)
|
||||
- [Test prompts must constrain the destination without disabling the bot](#test-prompts-must-constrain-the-destination-without-disabling-the-bot)
|
||||
- [Keep optional identity polish narrowly scoped](#keep-optional-identity-polish-narrowly-scoped)
|
||||
|
||||
## Design from the user's entry point
|
||||
|
||||
For the Dot flow, the approved entry was **New Agent → Invite an external agent →
|
||||
Dot / Hermes / Other**, not a hidden Runner provider configuration. Use the current
|
||||
product's equivalent. A provider preset may customize its logo, instructions and
|
||||
checks; it must not silently redesign another provider's enrollment protocol.
|
||||
Hermes originally retained the existing external-agent prompt.
|
||||
|
||||
Use the same square selection cards as the harness picker, actual provider logos,
|
||||
the shared copy-prompt component, and the normal dialog footer. Preserve agent
|
||||
details on Back. Animate changes in dialog size using the design system and honor
|
||||
reduced motion. Follow current repository design/token instructions.
|
||||
|
||||
Show prerequisite features and evaluate their actual server-side availability.
|
||||
Dot's standalone experiment did not implicitly enable every Runner provider.
|
||||
Cloud's managed execution also required its native Runner configuration. Distinguish
|
||||
those environment prerequisites from the user-facing provider picker; do not make
|
||||
users discover a disabled backend only after they have completed provider consent.
|
||||
|
||||
Make the user-facing sequence short:
|
||||
|
||||
1. Choose the bot and create/resume its agent invitation.
|
||||
2. Copy the complete prompt and give it to that bot.
|
||||
3. Grant any required scoped consent through the supported flow.
|
||||
4. Watch connection, task updates, and a harmless test event.
|
||||
5. Choose Done when both the connection and agent are ready.
|
||||
|
||||
A readiness probe should not create a visible work task. Create a real task later
|
||||
for assignment qualification. Optional avatar/profile steps must not delay readiness.
|
||||
|
||||
## Model the states the user will actually encounter
|
||||
|
||||
| State or event | Required behavior |
|
||||
| --- | --- |
|
||||
| Opening setup with no usable pending code | Create a fresh prompt automatically after checking authoritative connection state. |
|
||||
| Pending prompt expires while open | Refresh it safely; explain genuine renewal failure with Retry rather than looping. |
|
||||
| Existing connection is already established | Resume its status; do not revoke it to generate a new code. |
|
||||
| Another tab replaced the pending prompt | Avoid renewal ping-pong; ask before replacing that newer prompt or follow the product's explicit conflict policy. |
|
||||
| OAuth/binding observed | Close any nested copy-prompt window; replace the copy action with Connecting… status. |
|
||||
| Callback subscription verified | Keep completed checks checked; show confirmation of the test event in progress. |
|
||||
| Event challenge confirmed, agent still preparing | Show Finishing setup…; do not claim the event is still pending. |
|
||||
| Agent and connection ready | Enable Done, which closes the modal. |
|
||||
| Watch connection interrupted | Recover/resync from server state and explain the interruption; do not discard the pairing. |
|
||||
| Agent preparing/verifying with compatibility status paused | Allow setup when the server says setup is permitted. |
|
||||
| Agent genuinely manually paused or awaiting hire approval | Show the actual blocker and its action; do not auto-resume or bypass approval. |
|
||||
| Returning after refresh | Resume the same agent/invitation; do not create another agent. |
|
||||
|
||||
Derive UI decisions from server lifecycle facts. Dot Cloud initially had a circular
|
||||
dependency: hire preparation waited for pairing, but the UI treated preparation
|
||||
as a manual pause and demanded resume before pairing. Story fixtures alone did
|
||||
not reveal the real newly-created-agent state.
|
||||
|
||||
Keep one current setup procedure in documentation. Move historical/manual steps
|
||||
to an explicitly dated section when automatic renewal or atomic pairing replaces
|
||||
them. Do not leave “revoke, pair, save again” beside a newer one-step flow.
|
||||
|
||||
## Separate setup approvals from repeated intervention
|
||||
|
||||
Specify the allowed human actions in the release contract. “No human operates the
|
||||
bot's computer” can still involve a human approving a grant in chat and Codex
|
||||
submitting the already-authorized action. It does not prove that the bot alone
|
||||
completed every browser step. Record who performed consent, app attachment,
|
||||
catalog refresh, and reboot during qualification.
|
||||
|
||||
Consent should preview the exact company, agent, access scope and duration before
|
||||
submission. A one-use pairing code can preauthorize that exact agent without
|
||||
handing an operator password or board session to the external computer. Keep
|
||||
preview read-only and code consumption atomic. Revalidate current permissions at
|
||||
redemption, including when the creator has since left the company.
|
||||
|
||||
The historical user authorized a specific plugin-creation risk notice through
|
||||
this exact extra prompt paragraph:
|
||||
|
||||
> When you setup the plugin, I already understand that MCP servers have risk, so
|
||||
> you check “I understand and want to continue,” and click “Create as a plugin.”
|
||||
> I want you to create the plugin for me completely and I don't want to have to
|
||||
> do anything.
|
||||
|
||||
This is a historical consent example, not permission for future agents or users.
|
||||
Do not bake an assertion of user approval into every generated prompt. Use current
|
||||
authorization, distinguish a nonbinding risk notice from granting ongoing access,
|
||||
and obey the actual browser/provider policy. An agent relaying approval may not be
|
||||
accepted as the human's approval by the receiving agent. Do not impersonate the
|
||||
human to resolve that; use a supported direct approval path when authorized.
|
||||
|
||||
## Separate the four lifetimes
|
||||
|
||||
1. **Pairing capability:** short-lived, one use; safe automatic renewal before use.
|
||||
2. **Access token:** short-lived bearer authority, refreshed through its protocol.
|
||||
3. **Persistent connection/refresh authority:** product policy; Dot was changed to
|
||||
remain valid until revoked with rotating refresh tokens and replay checks.
|
||||
4. **Subscription and assignment leases:** bounded and separately renewed; a
|
||||
persistent account connection does not make a task's authority permanent.
|
||||
|
||||
The Dot user requested no Paperclip inactivity expiration. Migration preserved
|
||||
currently valid connections; it did not revive expired, consumed, or revoked
|
||||
credentials. Test renewal, rotation, replay and revocation independently. Do not
|
||||
promise that Paperclip's lifetime policy controls the provider's own account rules.
|
||||
|
||||
## Generate instructions the receiving agent can execute
|
||||
|
||||
Adapt the following to the provider's supported protocol and actual tool names.
|
||||
Do not send placeholder values, credentials, or a URL from an old environment.
|
||||
|
||||
```text
|
||||
Connect yourself to Paperclip as <agent> in <company>.
|
||||
|
||||
Add or reuse the private plugin for this exact server URL: <MCP_URL>.
|
||||
Use the supported OAuth flow. Reuse an existing definition only if its URL matches;
|
||||
if the name belongs to another URL, choose a distinct name instead of replacing it.
|
||||
Use this one-use pairing code on the consent page: <PAIRING_CODE>.
|
||||
Review the exact company, agent, permissions, and duration. Complete the authorized
|
||||
steps; if a required approval is missing, request that specific approval.
|
||||
|
||||
Verify an actual read-only call through this connection. If the installed plugin's
|
||||
tools are absent in your runtime, refresh/attach it through the provider's supported
|
||||
flow and report any remaining blocker. Do not claim access from a settings listing.
|
||||
|
||||
Enable the task-update subscription returned by Paperclip. Follow its event
|
||||
instructions, drain the inbox from your saved cursor, and confirm the readiness
|
||||
challenge. Do not create a task just to confirm readiness.
|
||||
|
||||
For assignments, read and accept current work, use stable request IDs, reconcile
|
||||
pending results, and follow the supplied reporting/completion contract. For new
|
||||
work requested while idle, use the governed intake tool and poll the same request
|
||||
if admission is pending; do not ask the human to create a placeholder task.
|
||||
|
||||
Use your own relevant tools, apps, plugins, skills, and capabilities to complete
|
||||
the request, then return the result and deliverables to the Paperclip task.
|
||||
The Paperclip catalog describes Paperclip operations. It does not disable your
|
||||
native tools or require those tools to appear in that catalog. Respect the tools'
|
||||
existing permissions and approvals. Keep Paperclip operations within the current
|
||||
connection's authorized company, agent, and task; do not use another identity to
|
||||
bypass a denied Paperclip operation.
|
||||
|
||||
Report which connection checks actually succeeded and any exact blocker.
|
||||
```
|
||||
|
||||
Put native-tool guidance at the points where it changes behavior: setup prompt,
|
||||
server onboarding text, event instructions, capabilities, and each assignment.
|
||||
Test rendered plain-text documents as well as JSON fields. The Dot follow-up
|
||||
initially added guidance to the JSON manifest but omitted the `onboarding.txt`
|
||||
renderer; agents receiving only that URL still missed it.
|
||||
|
||||
Do not copy every instruction into every tool description. Keep one common source
|
||||
where package boundaries permit. Use a lightweight dependency-free shared export
|
||||
for UI copy to avoid importing a heavy shared barrel. The standalone Runner may
|
||||
need its own small instruction constant with a clearly documented alignment point.
|
||||
|
||||
Historical instruction surfaces to locate in the current checkout:
|
||||
|
||||
- `ui/src/lib/dot-setup-prompt.ts`
|
||||
- `ui/src/lib/agent-onboarding-prompt.ts`
|
||||
- `server/src/routes/access.ts` and its actual onboarding text renderer
|
||||
- `server/src/services/dot-runner-broker.ts` pairing/event/capability responses
|
||||
- `packages/paperclip-runner/src/backends/runtime-context.ts`
|
||||
- `packages/shared/src/external-agent-guidance.ts`
|
||||
|
||||
## Test prompts must constrain the destination without disabling the bot
|
||||
|
||||
Bad: “Use only the staging plugin.” Dot carried this into later work and refused
|
||||
to use its already-connected native Slack.
|
||||
|
||||
Use a narrowly scoped qualification instruction:
|
||||
|
||||
```text
|
||||
For qualification task <CASE_ID> only, send Paperclip operations through
|
||||
<TEST_CONNECTION> for <TEST_COMPANY>. This selects the Paperclip destination;
|
||||
it does not restrict your relevant native tools. Do not alter other connections.
|
||||
After this case, these test-only restrictions end. Return results to its task
|
||||
using the normal reporting and completion tools.
|
||||
```
|
||||
|
||||
If a case deliberately excludes native tools to isolate one transport, say so
|
||||
for that case only, then end the restriction explicitly. Run a subsequent native
|
||||
tool case to detect leakage of the test restriction into normal behavior.
|
||||
|
||||
## Keep optional identity polish narrowly scoped
|
||||
|
||||
Dot's avatar tool accepts image bytes for its own bound Paperclip agent, re-encodes
|
||||
them, and never grants generic agent administration. Validate format, byte and
|
||||
pixel limits; remove metadata; make replay safe. A private asset ID cannot simply
|
||||
be exported to another company. Preserve portable name/character/palette and omit
|
||||
or remap company-local image IDs on import.
|
||||
|
||||
Do not invent an avatar-export API or extract an unrelated image just because the
|
||||
UI has an avatar. Ask the bot to upload its current avatar only if available; lack
|
||||
of that capability must not block connection or task qualification.
|
||||
@@ -0,0 +1,141 @@
|
||||
# Release contract and provider discovery
|
||||
|
||||
## Contents
|
||||
|
||||
- [Make the requested outcome testable](#make-the-requested-outcome-testable)
|
||||
- [Define capability parity rather than promising “full-fledged”](#define-capability-parity-rather-than-promising-full-fledged)
|
||||
- [Research only what can change the next decision](#research-only-what-can-change-the-next-decision)
|
||||
- [Choose the execution model once, using stated criteria](#choose-the-execution-model-once-using-stated-criteria)
|
||||
- [Keep a finite delivery sequence](#keep-a-finite-delivery-sequence)
|
||||
|
||||
## Make the requested outcome testable
|
||||
|
||||
An external bot is an existing agent whose execution is hosted elsewhere. It may
|
||||
have memory, a browser, skills, accounts, and tools Paperclip neither supplies nor
|
||||
controls. Decide whether this integration connects that existing agent or creates
|
||||
a separate API agent. They can have different identity, billing, tools, and memory.
|
||||
|
||||
Do not substitute a provider's generic agent API for the user's personal bot
|
||||
without making that distinction explicit. Dot's initial investigation considered
|
||||
both; the user wanted the actual existing Dot.
|
||||
|
||||
Use this contract in the task or existing implementation plan:
|
||||
|
||||
```text
|
||||
Provider / existing bot or newly created API agent:
|
||||
Ordinary entry point and target users:
|
||||
Release targets: self-hosted / Cloud / both:
|
||||
Transport outward / return channel / provider evidence:
|
||||
Paperclip acting identity and company/agent mapping:
|
||||
Paperclip-originated work acceptance journey:
|
||||
Bot-originated idle work acceptance journey:
|
||||
Native tools policy:
|
||||
Required Paperclip capabilities / explicitly deferred capabilities:
|
||||
Workspace access and execution placement:
|
||||
Allowed human setup steps:
|
||||
Connection lifetime / revocation / subscription renewal:
|
||||
Latency, concurrency, run duration, resource and test-spend bounds:
|
||||
Provider controls unavailable or unknown:
|
||||
Target deployment and authorization already established:
|
||||
Finish line and evidence required:
|
||||
Next critical-path gate / owner / action:
|
||||
```
|
||||
|
||||
Fill from context and supported defaults. Ask only where an unknown changes the
|
||||
product, authority, cost, or acceptance. Do not turn the template into a mandatory
|
||||
questionnaire. If the user says “two way,” keep both directions in the finish line.
|
||||
If Cloud is deferred, state that before delivering an apparently complete agent.
|
||||
|
||||
## Define capability parity rather than promising “full-fledged”
|
||||
|
||||
For each area, mark Required, Optional, Deferred, or Provider unavailable. Add
|
||||
whether it works idle, during a run, after completion, and after revocation.
|
||||
|
||||
| Area | Questions that affect implementation |
|
||||
| --- | --- |
|
||||
| Identity | Is the caller a company agent or a human's personal assistant? Can one external bot bind multiple agents? |
|
||||
| Idle intake | Can the bot create governed work when no task is assigned? Does the user see the intake and its attribution? |
|
||||
| Coordination | Search/read tasks; create and assign to agents or people; comment and edit authorized cross-task documents? |
|
||||
| Native tools | Can the bot use its existing Slack/browser/computer? What permissions remain provider-owned? |
|
||||
| Assigned tools | Pinned Paperclip skills, gateway apps, dynamically granted tools, permission changes during a turn? |
|
||||
| Attachments | Which task/company files may leave Paperclip, how are pages/integrity checked, and can access be revoked mid-read? |
|
||||
| Workspace | Files, shell, Git, artifact publishing; whose machine; explicit settings; which platforms? |
|
||||
| Human interaction | Questions, approval requests, replies, denial, resumption, and visibility in the task thread? |
|
||||
| Lifecycle | Follow-up, interruption, cancellation, restart, reconnect, lease renewal, and a terminal failure path? |
|
||||
| Observability | Acceptance, progress, operation results, persisted output, task disposition; which provider data is absent? |
|
||||
| Economics | Provider usage/cost versus managed Runner cost; admission budget checks when provider usage is unknown? |
|
||||
| Identity presentation | Provider logo, agent name/avatar, export/import portability; optional profile work must not block connection. |
|
||||
|
||||
Three separate grants deserve explicit treatment: persistent agent connection,
|
||||
access to task attachments, and Paperclip workspace execution. One does not imply
|
||||
the others. For Dot, attachment and workspace settings were independently off by
|
||||
default and were never inherited automatically by a newly hired teammate.
|
||||
|
||||
## Research only what can change the next decision
|
||||
|
||||
Use official provider docs and current code. Record the source, date, fact, and
|
||||
remaining experiment. Classify claims as documented, observed with real provider,
|
||||
simulated, inferred, or unknown. Search for these capabilities before selecting:
|
||||
|
||||
- Direct create/message/status/cancel APIs for this exact kind of agent.
|
||||
- Out-of-band events, subscription discovery/renewal, callbacks and verification.
|
||||
- In-session notifications versus wakeup when no live conversation is connected.
|
||||
- Bot-to-bot mentions, DMs, channel permissions, and automated-message filtering.
|
||||
- Tool/plugin installation, OAuth discovery, public-client support, runtime loading.
|
||||
- Structured tool calls/results, correlation IDs, idempotency, and result polling.
|
||||
- Provider-native session identity, resume, interruption, usage, and rate limits.
|
||||
- Private testing, workspace distribution, public review, and account/admin limits.
|
||||
|
||||
An HTTP success from a notification endpoint proves receipt at that layer. It
|
||||
does not prove that the agent woke, loaded tools, accepted work, or finished.
|
||||
Browser automation can help investigate and qualify setup; do not accidentally
|
||||
make an undocumented UI automation path the production transport.
|
||||
|
||||
For a proposed chat transport, send one authorized synthetic automated message
|
||||
and require the bot's correlated callback. A human-written mention succeeding is
|
||||
not evidence that messages from another bot will wake it.
|
||||
|
||||
## Choose the execution model once, using stated criteria
|
||||
|
||||
Paperclip's established preference is shared Runner infrastructure for remote
|
||||
agents. It preserves existing lifecycle/receipt semantics and a sandbox boundary
|
||||
if workspace capabilities are granted later. Remote reasoning does not eliminate
|
||||
the need for run coordination, nor does using Runner imply local model execution.
|
||||
|
||||
Evaluate alternatives against common admission/finalization, authority, recovery,
|
||||
compute waiting cost, operating complexity, and future workspace requirements.
|
||||
Document the choice and trigger for revisiting it. Do not alternate between a
|
||||
Runner, worker fleet, and database-only backend just because each was questioned.
|
||||
|
||||
For the Dot Cloud release the decision was one managed Runner per admitted
|
||||
assignment, no sandbox for idle MCP traffic, and no exposed workspace tools. It
|
||||
accepted compute while waiting as the cost of reusing the normal lifecycle.
|
||||
Do not generalize “one sandbox per assignment” to providers/releases that have an
|
||||
explicitly different approved execution contract.
|
||||
|
||||
## Keep a finite delivery sequence
|
||||
|
||||
1. **Feasibility:** actual provider wake/read/report loop on synthetic data.
|
||||
2. **Governed vertical slice:** normal task + idle intake through actual Runner.
|
||||
3. **Product onboarding:** natural entry, copy prompt, real watched readiness.
|
||||
4. **Reliability:** duplicate/lost response, follow-up race, restart, stop, cleanup.
|
||||
5. **Selected capabilities:** native apps, assigned apps/skills/files as promised.
|
||||
6. **Deployed acceptance:** ordinary app image, Runner artifact, tenant ingress.
|
||||
|
||||
Some work can overlap; the key is not building a broad tool catalog before the
|
||||
uncertain provider boundary and both entry paths work. Split reviewable changes
|
||||
when they span app, ingress, execution, or broad authority changes. Record their
|
||||
dependencies instead of declaring an arbitrary count of PRs required.
|
||||
|
||||
Useful final acceptance statement:
|
||||
|
||||
> On the named Cloud tenant and immutable candidate, a new user can invite the
|
||||
> existing bot with the generated prompt and one scoped consent approval. Without
|
||||
> a chat nudge, an ordinary assigned task saves a verifiable result and completes.
|
||||
> From idle, the bot can initiate work and assign a task to the responsible human.
|
||||
> A follow-up and real controller restart preserve one assignment. Revocation
|
||||
> denies further Paperclip actions, and every managed resource is accounted for.
|
||||
|
||||
Adapt this to the actual release. If plugin attachment or reboot is required,
|
||||
include it as a supported visible step or report an onboarding gap; do not claim
|
||||
the stricter prompt-only journey passed.
|
||||
@@ -0,0 +1,226 @@
|
||||
# Runner architecture and correctness boundaries
|
||||
|
||||
## Contents
|
||||
|
||||
- [Separate the bot, connection, controller, and workspace](#separate-the-bot-connection-controller-and-workspace)
|
||||
- [Identity and authority](#identity-and-authority)
|
||||
- [Durable state and operation contract](#durable-state-and-operation-contract)
|
||||
- [Follow-up delivery must be coordinated with ordinary wake scheduling](#follow-up-delivery-must-be-coordinated-with-ordinary-wake-scheduling)
|
||||
- [Recovery needs identity evidence, not merely a reachable process](#recovery-needs-identity-evidence-not-merely-a-reachable-process)
|
||||
- [Revocation and native powers are different boundaries](#revocation-and-native-powers-are-different-boundaries)
|
||||
- [Files and commands are a separate capability decision](#files-and-commands-are-a-separate-capability-decision)
|
||||
- [Paperclip implementation anchors](#paperclip-implementation-anchors)
|
||||
|
||||
## Separate the bot, connection, controller, and workspace
|
||||
|
||||
```text
|
||||
External provider
|
||||
Bot: reasoning, native browser/computer/apps, private memory
|
||||
| authenticated tools / signed wake events
|
||||
Paperclip control plane
|
||||
MCP + OAuth + pairing + durable mailbox + operation broker
|
||||
| authenticated Runner protocol
|
||||
Managed execution environment (Cloud)
|
||||
Rust Runner: assignment lifecycle, catalog validation, durable receipts
|
||||
| semantic tool requests / correlated results
|
||||
Paperclip authority services
|
||||
Task/company permissions, mutations, app gateway, result finalization
|
||||
```
|
||||
|
||||
Application tools execute through the existing authorized services. If filesystem
|
||||
or shell tools are enabled, their execution target must be the assigned confined
|
||||
workspace. Moving Rust into a sandbox does not magically move a server-side file
|
||||
handler there. Trace the actual effect destination for every workspace operation.
|
||||
|
||||
Dot's model runs at OpenAI. Runner does not create its personal memory, run its
|
||||
model, control its computer, or guarantee interruption of other native tools.
|
||||
Do not invent model/session/usage fields to satisfy a local-provider schema.
|
||||
|
||||
## Identity and authority
|
||||
|
||||
Use a dedicated agent-purpose grant/resource where the provider is acting as an
|
||||
employee. Personal assistant grants must not silently supply board/operator
|
||||
permissions to a run. Bind authorization to company, agent, active grant, binding
|
||||
generation, assignment, and run/turn. A supplied companyId or agentId is not proof
|
||||
of authority. Recheck live permissions at execution and before returning protected
|
||||
data, not only at initial pairing.
|
||||
|
||||
Pairing uses an operator-issued, expiring, one-use capability for a specific
|
||||
agent. Redemption checks current membership/role, feature prerequisites, agent
|
||||
lifecycle, company/resource hints, expiry and revocation. Preview does not consume
|
||||
the code. Grant creation, code consumption, and saving the binding reference must
|
||||
be atomic. Otherwise setup can display ready while the scheduler still sees an
|
||||
unpaired agent. Saving the form a second time is not an acceptable hidden step.
|
||||
|
||||
Persistent connection identity and per-turn mutation authority are different:
|
||||
|
||||
| Request | Expected behavior |
|
||||
| --- | --- |
|
||||
| Pair, inspect own capability/inbox status | Connection-scoped operation; no managed assignment needed. |
|
||||
| Start work while idle | Durable, agent-attributed intake; ordinary admission before task mutation tools. |
|
||||
| Read/accept offered assignment | Current binding and offer, explicit acceptance, no arbitrary run selection. |
|
||||
| Invoke a task/app tool | Same admitted assignment and live permission checks; normal receipts/audit. |
|
||||
| Update own optional avatar | Narrow self-only operation; do not grant generic agent-config authority. |
|
||||
| Finish | Accepted canonical result, settled required operations, normal task finalizer. |
|
||||
| Acknowledge a fence | Narrow control operation may remain available while task tools are denied. |
|
||||
|
||||
Idle intake must retain plugin/environment context from the configured application
|
||||
services. In Dot, constructing a separate heartbeat service without the sandbox
|
||||
plugin registry made idle requests fail even though operator-assigned work ran.
|
||||
Test both entry points against the same managed environment.
|
||||
|
||||
## Durable state and operation contract
|
||||
|
||||
Persist distinct facts for grant/binding, subscription, mailbox item, assignment,
|
||||
operation receipt, run/controller ownership, and final result. Avoid a single
|
||||
boolean “connected” or “done.” Enforce active-assignment concurrency in storage
|
||||
as well as the UI. Dot's initial contract allowed one active assignment per binding.
|
||||
|
||||
An illustrative lifecycle is:
|
||||
|
||||
```text
|
||||
intake/admission pending -> offered -> accepted -> working -> result accepted
|
||||
-> turn finished -> task disposition committed -> resource cleanup
|
||||
```
|
||||
|
||||
Any active phase can instead become fenced, expired, failed, or reconciliation
|
||||
required. These are conceptual states; use the repository's actual enums.
|
||||
|
||||
Every side-effecting external request needs a stable request identity plus an
|
||||
argument digest and authority generation. Implement these semantics:
|
||||
|
||||
- Same identity + same arguments: return the same pending/settled receipt.
|
||||
- Same identity + different arguments: reject a conflict.
|
||||
- Completed intake retried: return its original receipt, never another task.
|
||||
- Old authority generation: deny new execution, even if a historical receipt exists.
|
||||
- Definite read failure: terminal bounded error; later tools and finish can proceed.
|
||||
- Effect possibly happened but result was lost: explicit unknown/pending outcome
|
||||
and reconciliation. Do not reissue under a fresh ID to make progress.
|
||||
|
||||
Persist reservation before effects where appropriate. Use existing transactional
|
||||
mutation receipts and external-effect reconciliation. Do not promise universal
|
||||
exactly-once execution across arbitrary remote services. Distinguish command
|
||||
acknowledgment from completion of the tool's effect.
|
||||
|
||||
Do not block the Runner command lane waiting for a semantic tool result delivered
|
||||
on that same lane. Reserve/admit promptly, deliver a pending receipt, then settle
|
||||
with a correlated event. Bound pending operations, results, bytes, and authority
|
||||
duration. Pagination and control acknowledgments must still work at limits.
|
||||
|
||||
Serialize mailbox cursor visibility with writers so a cursor cannot skip a result
|
||||
whose transaction commits later. Wake events should contain references, not task
|
||||
contents, credentials, or copied app data. An event receiver drains the durable
|
||||
mailbox and validates current authority rather than trusting a stale event body.
|
||||
|
||||
Treat event callbacks as an authenticated protocol, not arbitrary URLs to POST to.
|
||||
Use the provider's current ownership challenge and signing specification. Keep
|
||||
callback signing material out of prompts, descriptors, logs and task data. Enforce
|
||||
the callback destination policy, including redirect and DNS/IP handling, so a
|
||||
subscription cannot turn the control plane into an internal-network request proxy.
|
||||
Apply bounded delivery timeouts/retries and record failures without logging secrets.
|
||||
Where receiving signed callbacks, validate the expected signature, timestamp and
|
||||
replay policy before changing state. Test the actual directions this provider uses;
|
||||
do not invent a symmetric webhook API merely because the integration is two way.
|
||||
|
||||
## Follow-up delivery must be coordinated with ordinary wake scheduling
|
||||
|
||||
A normal comment can enter two paths: active-assignment mailbox delivery and the
|
||||
ordinary scheduler. If both remain independent, a finished bot can receive a
|
||||
second assignment for a comment it already handled. If the normal wake is simply
|
||||
discarded when the mailbox entry is made, the comment can be lost at completion.
|
||||
|
||||
The Dot fix retained the normal wake as deferred until the bot successfully read
|
||||
that exact comment ID through `get_task_history`. Then the existing run consumed
|
||||
it. If the assignment ended before the read, normal admission kept the unread
|
||||
comment for another turn. An inbox reference or webhook acknowledgment did not
|
||||
prove the comment was read.
|
||||
|
||||
Test both sides of this race, including controller recovery. Keep explicit new
|
||||
session requests, interaction replies, and self-authored comments under their
|
||||
normal policy; don't blanket-suppress every wake for a connected bot.
|
||||
|
||||
## Recovery needs identity evidence, not merely a reachable process
|
||||
|
||||
Record and compare the run, session, assignment, binding generation, controller
|
||||
generation, managed lease, process marker, and trustworthy process birth identity.
|
||||
On Linux the process fingerprint can include boot identity and start information.
|
||||
Read remote identity from the sandbox, not a same-numbered PID on the controller.
|
||||
|
||||
Before advertising external work, require the evidence needed to recover it.
|
||||
Recovery reads the checkpoint from the owning execution target and validates it
|
||||
against saved authority. Missing/invalid state after dispatch fails closed for
|
||||
reconciliation; a new process or conversation must not masquerade as a resume.
|
||||
|
||||
Incoming operations need a durable draining/replay mechanism owned by the current
|
||||
controller. An in-memory forwarding optimization alone can leave valid requests
|
||||
stranded after restart. Fence old controllers and callbacks so a late result
|
||||
cannot overwrite the new generation's journal. Preserve late effect evidence.
|
||||
|
||||
A verified terminal/shutdown receipt can establish a clean exit when a remote
|
||||
monitor lacks an exit code. An unexplained process exit still requires recovery.
|
||||
Configure a bounded reconnect grace consistent with the deployment lifecycle;
|
||||
Dot used five minutes after shorter restart gaps exposed failures. Do not copy
|
||||
that number as a universal provider requirement.
|
||||
|
||||
Terminal run reconciliation must clear/fence a still-active assignment, notify the
|
||||
bot, and release resources. Otherwise a failed run can block every future task
|
||||
even though its old tools are already denied.
|
||||
|
||||
## Revocation and native powers are different boundaries
|
||||
|
||||
Cancellation, pause, reassignment, membership loss, expired run authority, and
|
||||
grant revocation must prevent subsequent Paperclip effects. Cached tool results
|
||||
or UI binding references cannot restore revoked access. Old-grant revocation must
|
||||
not revoke a replacement binding accidentally.
|
||||
|
||||
Report “Paperclip authority revoked” separately from “external bot stopped.” A
|
||||
fence acknowledgment proves receipt, not termination of all provider activity.
|
||||
Likewise, sandboxing the Paperclip Runner does not confine the bot's native Slack
|
||||
or provider computer. Those tools retain their own permissions. If strict total
|
||||
capability isolation is required, use a suitably isolated bot/account and verify
|
||||
the provider's actual controls; Paperclip cannot enforce it with prompt text.
|
||||
|
||||
## Files and commands are a separate capability decision
|
||||
|
||||
Do not add workspace tools merely to make an external agent look like a coding
|
||||
CLI. A bot can research or write code on its own computer without Paperclip
|
||||
workspace access. The first Dot Cloud release intentionally offered none.
|
||||
|
||||
When the release includes workspace tools:
|
||||
|
||||
- Resolve paths within the admitted workspace; reject traversal, absolute paths,
|
||||
symlinks/escapes, and Paperclip instance state such as `.paperclip`.
|
||||
- Check reads, writes, API downloads, uploads, and artifact publication. A generic
|
||||
`call_api` path must not bypass attachment/workspace grants.
|
||||
- Serialize optimistic hash checks with writes. Two concurrent writes must not
|
||||
both commit against the same observed hash.
|
||||
- A process working directory is not a sandbox. Use a proven OS/container boundary;
|
||||
test outside-workspace access, environment secrets, detached descendants, timeout,
|
||||
cancellation and cleanup. No unrestricted fallback when confinement is unavailable.
|
||||
- In the historical Dot implementation, Linux used bubblewrap/private PID
|
||||
namespaces. macOS command execution was disabled after detached descendants
|
||||
escaped reliable supervision; file tools and artifacts remained available.
|
||||
- For attachments, enforce current task/company, bounded pages and total bytes,
|
||||
integrity checks, and live revocation before returning each page. Treat filenames
|
||||
and contents as untrusted data. Test the generic API escape path too.
|
||||
|
||||
## Paperclip implementation anchors
|
||||
|
||||
Paths below existed in the qualified October 9 candidate. Locate their current
|
||||
equivalents before editing; do not paste line numbers from historical notes.
|
||||
|
||||
| Concern | Historical anchor |
|
||||
| --- | --- |
|
||||
| Bot broker and tool definitions | `server/src/services/dot-runner-broker.ts` |
|
||||
| Prototype public MCP bridge | `server/src/services/public-mcp/dot-runner.ts` |
|
||||
| Rust assignment backend | `packages/paperclip-runner/runner/crates/runner-core/src/dot_provider_backend.rs` |
|
||||
| TypeScript Runner driver | `packages/paperclip-runner/src/drivers/dot/runnerd-dot-driver.ts` |
|
||||
| Driver/backend selection | `packages/paperclip-runner/src/backends/native-backend-factory.ts` |
|
||||
| Per-turn instructions | `packages/paperclip-runner/src/backends/runtime-context.ts` |
|
||||
| Governed tool authority | `server/src/services/native-runtime/paperclip-runner-tool-authority.ts` |
|
||||
| Admission, wake, terminal reconciliation | `server/src/services/heartbeat.ts`, `server/src/services/heartbeat/queue.ts`, `server/src/services/heartbeat/run-lifecycle.ts` |
|
||||
| Real Rust/DB and managed qualification | `server/src/__tests__/dot-runner.test.ts` |
|
||||
|
||||
Keep the standalone Runner package independent of server/DB implementation
|
||||
imports. Extend the actual production backend and negotiated capability contract,
|
||||
not just a convenient reference harness.
|
||||
+215
@@ -0,0 +1,215 @@
|
||||
# Troubleshooting, test environments, and Cloud delivery
|
||||
|
||||
## Contents
|
||||
|
||||
- [Start with the failing boundary](#start-with-the-failing-boundary)
|
||||
- [Stable public ingress is part of the test fixture](#stable-public-ingress-is-part-of-the-test-fixture)
|
||||
- [Concrete OAuth and routing checks for the Dot implementation](#concrete-oauth-and-routing-checks-for-the-dot-implementation)
|
||||
- [Symptom → discriminator → next action](#symptom--discriminator--next-action)
|
||||
- [Keep one reproducible environment manifest](#keep-one-reproducible-environment-manifest)
|
||||
- [Coordinate all Cloud pieces](#coordinate-all-cloud-pieces)
|
||||
- [Classify a failure before retrying](#classify-a-failure-before-retrying)
|
||||
- [Close the environment deliberately](#close-the-environment-deliberately)
|
||||
|
||||
## Start with the failing boundary
|
||||
|
||||
Do not use “OAuth is broken” for every setup failure. Trace these independent paths:
|
||||
|
||||
```text
|
||||
A. Provider backend -> public origin -> discovery, MCP, token exchange
|
||||
B. Provider browser -> authorization page -> preview/consent -> provider callback
|
||||
C. Agent runtime -> loaded plugin actions -> authenticated read/tool calls
|
||||
D. Paperclip event worker -> verified callback -> bot wake -> mailbox/challenge
|
||||
E. Paperclip scheduler -> managed lease -> Runner -> tools -> finalizer -> cleanup
|
||||
```
|
||||
|
||||
For each, collect a timestamp, exact failing operation, status/visible error,
|
||||
correlated access log, and which party observed it. The developer's browser, the
|
||||
provider's browser, and the provider's backend can have different reachability.
|
||||
An absent request in complete ingress logs means that attempt did not reach the
|
||||
app; it does not identify the upstream blocking rule.
|
||||
|
||||
## Stable public ingress is part of the test fixture
|
||||
|
||||
Prefer one stable HTTPS origin on port 443 reachable by both provider clients.
|
||||
Keep issuer/resource, token exchange, authorization redirect, and callback
|
||||
configuration consistent. Do not rotate tunnel hostnames as the first response to
|
||||
every failure: each new URL can require a different plugin, fresh registration,
|
||||
new pairing prompt, and subscription cleanup.
|
||||
|
||||
Self-hosted testing may need a tunnel or authorized public reverse proxy. Dot
|
||||
eventually used a stable public proxy forwarding to the test machine. That is an
|
||||
observed useful arrangement, not a requirement to provision new infrastructure
|
||||
for every integration. Use the user's supported deployment/network options.
|
||||
|
||||
Tailscale-specific diagnostic: local MagicDNS may resolve a hostname to a private
|
||||
tailnet address while outside clients use Funnel. Test public ingress independently;
|
||||
local curl success does not establish public access. Tailscale Serve and Funnel
|
||||
have different reachability. Never infer a provider port ban from one timeout.
|
||||
|
||||
`ERR_BLOCKED_BY_CLIENT` alone does not identify Cloudflare or a server bot filter.
|
||||
Use server logs and the actual consent page, not only `/api/health`: Dot once
|
||||
blocked the health navigation while the required sign-in page loaded. Do not
|
||||
bypass browser security warnings or a provider's access controls. Use supported
|
||||
configuration or an authorized reachable endpoint and preserve the exact uncertainty.
|
||||
|
||||
## Concrete OAuth and routing checks for the Dot implementation
|
||||
|
||||
These routes describe the October 9 candidate; discover current equivalents.
|
||||
|
||||
| Surface | Historical expectation |
|
||||
| --- | --- |
|
||||
| `/mcp/runner` unauthenticated | `401` and `WWW-Authenticate` with resource metadata; not an HTML sign-in page. |
|
||||
| `/.well-known/oauth-protected-resource/mcp/runner` | Public JSON for the exact resource and authorization server. |
|
||||
| `/.well-known/oauth-authorization-server/mcp/runner/oauth` | Public JSON for issuer and endpoints. |
|
||||
| `/mcp/runner/oauth` | Dedicated agent-resource issuer; personal MCP issuer remained separate. |
|
||||
| `/dot-connect/:requestId` | Public pairing page, scoped by request and one-use pairing capability. |
|
||||
| `/api/dot-mcp/requests/:requestId` and exact pairing operations | Publicly routable aliases with request/code checks; reject personal assistant request IDs. |
|
||||
| Normal board, connection management, ordinary personal consent | Retain normal session/role requirements. |
|
||||
|
||||
For read-only health checks, substitute the configured origin; do not send tokens:
|
||||
|
||||
```sh
|
||||
curl --max-time 15 -sS -D - -o /tmp/dot-resource-metadata.json \
|
||||
'https://YOUR_CONFIGURED_ORIGIN/.well-known/oauth-protected-resource/mcp/runner'
|
||||
curl --max-time 15 -sS -D - -o /tmp/dot-auth-metadata.json \
|
||||
'https://YOUR_CONFIGURED_ORIGIN/.well-known/oauth-authorization-server/mcp/runner/oauth'
|
||||
```
|
||||
|
||||
Inspect JSON and redirect behavior, then exercise the actual provider discovery.
|
||||
Do not interpret these curl checks as provider qualification. Keep browser cookies
|
||||
and tokens out of pasted logs; use the existing authenticated test client when a
|
||||
protected operation must be inspected.
|
||||
|
||||
ChatGPT's historical client metadata preferred `private_key_jwt` while also
|
||||
advertising `none`. Paperclip originally rejected it instead of negotiating its
|
||||
supported public-client method with PKCE. Inspect the complete advertised methods;
|
||||
do not disable authentication or exact redirect/resource validation to work around
|
||||
a preference mismatch. Keep S256 PKCE and current registration checks.
|
||||
|
||||
An optional `PAPERCLIP_MCP_AUTHORIZATION_ORIGIN` supported a different authorized
|
||||
browser ingress to the same instance, while `PAPERCLIP_PUBLIC_URL` retained the
|
||||
MCP resource/issuer/token origin. This was a workaround for different client
|
||||
reachability, not an OAuth requirement. Use only explicitly configured trusted
|
||||
origins and verify callback/CSRF/issuer behavior. Prefer a single working origin.
|
||||
|
||||
## Symptom → discriminator → next action
|
||||
|
||||
| Symptom | Evidence to inspect | Action and proof |
|
||||
| --- | --- | --- |
|
||||
| OAuth discovery times out | Public metadata vs actual provider access logs | Fix the failing ingress/client path; do not keep editing valid metadata when no request arrives. |
|
||||
| Unknown client or redirect URI after restart | Persisted registration vs provider's cached client | Restore the intended persistent fixture or re-register through supported setup. Do not reset DB repeatedly. |
|
||||
| Invalid request ID | Actual ID representation | Treat protocol IDs as opaque unless contract says otherwise; the lab incorrectly required a UUID. |
|
||||
| Grant exists but provider is not connected | Complete same browser flow through callback/token exchange | A server grant row alone is not completed OAuth. Avoid mixing separate callback sessions. |
|
||||
| Tools listed in settings, absent to bot | Actual tool invocation in active conversation | Supported refresh/attach/restart, bounded once per diagnosis; preserve endpoint and mark intervention. |
|
||||
| Subscription verified but no bot action | Delivery record, mailbox, challenge/acceptance receipt | Separate callback receipt from model execution. Report latency; inspect actionable work before resending. |
|
||||
| Ready connection, task admission says unpaired | Saved agent binding/config revision | Fix atomic pairing persistence; do not require another manual form save. |
|
||||
| Agent created paused before pairing | Lifecycle preparing/verifying vs manual pause | Use lifecycle-specific setup eligibility; preserve actual pause/approval gates. |
|
||||
| Anonymous consent spins/reloads | Background session probes returning expected 401 | Public consent must tolerate them without global reload; protected board routes retain session recovery. |
|
||||
| Ordinary assignment works, idle work cannot acquire sandbox | Service construction / plugin context | Use app-configured admission service and test both entry points. |
|
||||
| Remote Runner connects to localhost | Launched transport endpoint vs private managed ingress | Use the authenticated remote transport supplied by the execution target. |
|
||||
| Image lacks provider support | Actual packaged Runner digest/capabilities | Fail before dispatch; publish compatible artifact and test installed path. |
|
||||
| Recovery finds sandbox but no process identity | Marker, fingerprint and original checkpoint | Capture identity before dispatch; reconcile missing evidence rather than launch replacement work. |
|
||||
| Remote PID has implausible start time | Whether controller inspected its own PID table | Obtain birth identity from the remote sandbox, not a colliding local PID. |
|
||||
| Finished task starts again after comment | Mailbox + scheduler wake receipt | Consume only when exact comment read succeeds; test unread-at-finish and read-before-finish. |
|
||||
| Missing file leaves run waiting forever | Pending tool receipt and error dispatch | Settle definite read failures; preserve unknown outcome only for genuinely ambiguous effects. |
|
||||
| Terminal run blocks all new tasks | Assignment still active despite terminal run | Reconcile/fence it and release resources without claiming external stop. |
|
||||
| Cancelled job has creating/pending-cleanup sandbox | Provider allocation and deletion receipts | Track until destroyed or explicitly pending with owner; retry only per provider semantics. |
|
||||
| New deployment vanishes mid-test | Fleet auto-rollout vs direct candidate | Use supported test-tenant pinning with authorization; record current serving revision. |
|
||||
|
||||
## Keep one reproducible environment manifest
|
||||
|
||||
Before a test drive or deployment, resolve current worktree/artifacts and reuse a
|
||||
suitable isolated checkout. Do not reset the user's primary checkout. Create a new
|
||||
managed worktree when appropriate, preserving unrelated work. Record:
|
||||
|
||||
```text
|
||||
Repository / worktree / branch / app SHA:
|
||||
Runner artifact platform / digest / advertised capabilities:
|
||||
Ingress repository / SHA / relevant routing version:
|
||||
App image / immutable digest:
|
||||
Local host / port / data directory / process ownership:
|
||||
Public origin / authorization origin / forwarding target:
|
||||
Tenant / company / agent identifiers (private evidence only):
|
||||
Sign-in entry and credential reference (no secret values):
|
||||
Grant/binding status / subscription / pending code expiry (no code value):
|
||||
Current case / run / assignment / lease:
|
||||
Owner of each server, proxy, build, sandbox, and cleanup:
|
||||
Environment retention agreement / next gate:
|
||||
```
|
||||
|
||||
Use synthetic fixtures, not a production database clone containing private company
|
||||
data or signing keys. Verify launcher arguments, selected instance, port mapping,
|
||||
and data persistence after restart. A label such as `default` is not proof of
|
||||
process ownership. Never kill unrelated databases or clear shared IPC globally.
|
||||
If resource pressure blocks the fixture, inspect owned processes or use the
|
||||
approved remote test path; preserve unrelated sessions.
|
||||
|
||||
Verify sign-in is usable before handing over a URL. Deliver access/resume details
|
||||
through an appropriate private path. If the user is still testing or the bot is
|
||||
loading tools, keep the endpoint alive under the agreed bounds. Stopping it made
|
||||
the first “tools unavailable” diagnosis premature. Announce an intentional expiry
|
||||
and explain how to resume; do not assume a stopped shell process remains a service.
|
||||
|
||||
## Coordinate all Cloud pieces
|
||||
|
||||
Cloud qualification requires the tenant app, public ingress, managed provider
|
||||
plugin/configuration, and packaged Runner artifact. The UI flag alone is not
|
||||
enablement. Inspect guards in UI, scheduler, backend, remote launcher, and Cloud
|
||||
front door. Use scoped route exceptions that retain endpoint authentication rather
|
||||
than removing browser-session guards wholesale.
|
||||
|
||||
Pairing and MCP must remain reachable without a managed job. If a sleeping tenant
|
||||
returns a waking-up page, wait for the actual lifecycle transition, then test the
|
||||
correct metadata response. Delay creating expiring pairing codes until the tenant
|
||||
image is deployed and serving. A 3.4 GB image transfer consumed meaningful setup
|
||||
time during Dot qualification; do not burn the code's lifetime watching a build.
|
||||
|
||||
Record whether a test overrides the sandbox binary with a locally built artifact.
|
||||
That proves the code can run, not that the published default image is correct.
|
||||
Match platform and provider capability; a Mac controller's current binary cannot
|
||||
stand in for a Linux sandbox artifact. Digest/capability checks should fail clearly
|
||||
before an assignment is offered.
|
||||
|
||||
For the historical private `paperclip-cloud` repository, the supported targeted
|
||||
deployment CLI was `scripts/deploy-stack.mjs <workspace-url> <ref> --plain`, with
|
||||
`--resume <request-id>` for the same operation. Inspect current help/runbook before
|
||||
using it. Do not invent or replay old tenant IDs, deployment requests, or secrets.
|
||||
Preserve one request through prepare/build/check/deploy/verify; duplicated requests
|
||||
make provenance and recovery harder. No fleet-default change is implied by a
|
||||
single test-tenant deployment.
|
||||
|
||||
## Classify a failure before retrying
|
||||
|
||||
- **Product:** deterministic missing context, bad state transition, wrong endpoint,
|
||||
lost receipt. Reproduce, fix, and rerun the affected contract.
|
||||
- **Provider:** tool loading, wake delay, refusal, unsupported control. Capture the
|
||||
actual provider result and test a supported recovery; don't manufacture success.
|
||||
- **Infrastructure:** capacity, registry 429/504, image transfer, npm resolver,
|
||||
provider allocation. Verify whether work/effects started; use bounded recovery.
|
||||
- **Evidence/test:** controller didn't restart, stale build, nonce visible in prompt,
|
||||
wrong image, scripted client presented as real. Correct the experiment.
|
||||
|
||||
For an unchanged transient build failure, one targeted infrastructure retry is a
|
||||
reasonable initial bound. If it fails again, report the precise remaining gate
|
||||
or investigate the new cause; do not loop reruns or change registry/security
|
||||
configuration without a task-grounded reason. Resume the existing deployment.
|
||||
If code changes, re-evaluate which earlier evidence applies to the new candidate.
|
||||
|
||||
Keep broad test runs tied to a frozen revision. Stop superseded owned runs when
|
||||
safe, retain their results, and run the affected suites plus required final gates.
|
||||
Do not claim a full local suite passed because failing tests passed individually;
|
||||
state focused results and final CI separately.
|
||||
|
||||
## Close the environment deliberately
|
||||
|
||||
After the final task, inspect leases and actual provider resources. Expired lease
|
||||
status alone is weaker than a successful sandbox deletion receipt. Preserve a
|
||||
requested review instance with access and resumption information; release owned
|
||||
disposable compute. For an authorized reset, account for provider account link,
|
||||
Paperclip binding/grant, event subscription/automation, and cached configuration.
|
||||
Disconnecting one layer can leave an enabled provider automation behind.
|
||||
|
||||
Report qualified, review-ready, merged, deployed, and canary-verified separately.
|
||||
Never say an instance received a fix until its serving revision and relevant
|
||||
ordinary-use behavior establish it.
|
||||
Reference in new issue
Block a user