0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-28 12:04:41 +01:00

Files

T

DottaandPaperclip 16b7db35ff Shorten planning skills and measure task decomposition (#15296 )

## Thinking Path

> - Paperclip manages work for AI agents.
> - Planning guidance helps agents choose owners and dependencies.
> - The runtime skill favors few tasks, but the catalog skill requires a
child-task breakdown.
> - Both add repeated process instructions that can distract from the
requested outcome.
> - This change keeps the ownership and dependency rules and removes the
required matrix and repeated checklist.
> - A bounded Product E2E comparison measures saved outcomes and task
handoffs before qualification.

## Linked Issues or Issue Description

Refs #11057. Related measurement work: #15218.

**What existing behavior does this improve?**
Planning and delegation through the runtime plan-to-tasks and bundled
task-planning skills.

**Current behavior**
The two skills contain about 1,900 words and conflicting guidance on
whether plans require child tasks.

**Proposed behavior**
Keep cohesive work with one owner. Split only for a real owner, parallel
output, dependency, independent review, or follow-up lifecycle. Preserve
existing authorization and planning mechanics.

## What Changed

- Shorten both skills to about 400 words combined. Preserve their keys
and installed-version behavior.
- Remove the duplicate operational-skill pointer and regenerate affected
source metadata.
- Add twelve explicit Product E2E cells: four scenarios with current,
short and disabled planning skills.
- Use the current task composer and actual create-response ID; calibrate
public skill APIs and browser creation without providers.
- Eliminate an observed collision in chat-test company prefixes with a
per-suite sequence.
- Grade saved documents, exact author/run attribution, child count,
prerequisite execution order, review boundaries and completion handoffs.
- Retain current skill bytes and report source, selections, run
accounting and failures.

## Verification

- `pnpm test:e2e:runner:typecheck`: pass.
- `pnpm test:e2e:runner:unit`: 1,287 Vitest tests and 128 Node checks
pass.
- `pnpm test:e2e:runner -- --list --suite plan-task-guidance`: twelve
local Codex cells.
- Archived current skills match master
`72ff3a9f27e581a27acb49771e8658bbb0bbaa47` exactly.
- Corrected fixture: three real public-API/database calibrations pass
with zero provider runs; all 35 evaluator checks and Product E2E
typecheck pass.
- Setup campaign
[37399550253](https://github.com/paperclipai/paperclip/actions/runs/37399550253)
was canceled after source review found unsupported bundled edits and
automatic core reinstallation. Its paid-cell step was skipped: zero
provider runs, no behavioral grade.
- The next setup
[37401094799](https://github.com/paperclipai/paperclip/actions/runs/37401094799)
failed before task creation on the old title-field selector: zero actual
runs, original FAIL retained, cleanup passed. A real browser/API
calibration of the new helper passes with paused non-provider agents and
zero runs.
- Full local typecheck/build pass. Full local tests retain one unchanged
five-minute Git streaming timeout (also fails isolated), 9,591 passes
and 5,796 skips. CI's chat failure was a proven random fixture-prefix
collision; five affected cases pass after the test-only repair.
- Paid behavior comparison and new-head CI/review remain pending. This
PR remains a draft.

## Risks

- The shorter text may change delegation decisions. Live outcomes are
not yet qualified.
- The initial comparison uses one profile and one attempt per cell. It
cannot establish cross-model reliability or cost trends.
- Disabled means unassigned company-owned copies; the company library
remains discoverable. This does not qualify global removal, automatic
accepted-plan wiring changes, or installed-copy migration.
- Skill availability does not prove a model read or cognitively used it.
- No provider/tool protocol, permission, timeout or runtime lifecycle
behavior changes in production.

## Model Used

OpenAI Codex (GPT-6), with repository inspection, code editing and tool
use. The exact backend model ID and context-window size are not exposed
in this session. The declared eval model is native Codex `gpt-5.6-sol`.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-06 09:36:01 -05:00

30 KiB

Raw Blame History

Capability Capability Contract

This generated contract is a self-contained derivative of the Paperclip skill, its seven references, the Paperclip Evals corpus, and the legacy MCP tool surface. It does not import or contact the Paperclip control plane.

The skill/reference inventory and eval cases are the only normative behavior sources. Paperclip does not use the legacy MCP calls as a production capability surface; all MCP names below are traceability aliases folded into normative eval rows. Their disposition, grants, assertions, and evidence contract are inherited from the target row rather than classified independently.

Baseline Counts

Skill/reference headings: 160
Eval cases: 106 across 16 groups
Total normative rows: 266
Legacy MCP aliases folded into normative rows: 42

Eval group	Cases
hb	5
co	6
st	8
cm	6
se	4
su	4
bl	5
dp	3
ix	9
ap	6
ar	4
er	9
rf	22
mh	4
rs	3
wk	8

Regeneration

pnpm --dir packages/paperclip-runner generate:capability-inventory imports the canonical baselines and rewrites every generated file.
pnpm --dir packages/paperclip-runner check:capability-inventory validates counts, uniqueness, normative dispositions, one-to-one MCP folds, required fields, and generated-file drift without requiring the external eval repository.

Skill / Reference Rows

Legacy MCP Alias Index

This is a compatibility/traceability index, not a tool catalog. “Inherited disposition” is shown only to make the normative target easy to audit.

Legacy MCP name	Folded into normative row	Inherited disposition	Source anchor
paperclipMe	eval:hb-inbox-lite-01	control_plane_owned	packages/mcp-server/src/tools.ts:290
paperclipInboxLite	eval:hb-inbox-lite-01	control_plane_owned	packages/mcp-server/src/tools.ts:296
paperclipListAgents	eval:rf-api-mgr-heartbeat-01	optional_agent_tool	packages/mcp-server/src/tools.ts:302
paperclipListSkills	eval:rf-cskill-audit-01	optional_agent_tool	packages/mcp-server/src/tools.ts:308
paperclipGetAgent	eval:rf-api-mgr-heartbeat-01	optional_agent_tool	packages/mcp-server/src/tools.ts:314
paperclipListIssues	eval:se-q-filters-01	optional_agent_tool	packages/mcp-server/src/tools.ts:323
paperclipGetIssue	eval:se-get-issue-01	optional_agent_tool	packages/mcp-server/src/tools.ts:338
paperclipGetHeartbeatContext	eval:hb-context-01	control_plane_owned	packages/mcp-server/src/tools.ts:344
paperclipListComments	eval:se-get-issue-01	optional_agent_tool	packages/mcp-server/src/tools.ts:353
paperclipGetComment	eval:hb-wake-comment-01	control_plane_owned	packages/mcp-server/src/tools.ts:366
paperclipListIssueApprovals	eval:ap-board-approval-01	optional_agent_tool	packages/mcp-server/src/tools.ts:373
paperclipListDocuments	eval:dp-base-revision-01	always_agent_tool	packages/mcp-server/src/tools.ts:379
paperclipGetDocument	eval:dp-base-revision-01	always_agent_tool	packages/mcp-server/src/tools.ts:385
paperclipListDocumentRevisions	eval:dp-base-revision-01	always_agent_tool	packages/mcp-server/src/tools.ts:392
paperclipListProjects	eval:rf-wf-project-setup-01	optional_agent_tool	packages/mcp-server/src/tools.ts:402
paperclipGetProject	eval:rf-wf-project-setup-01	optional_agent_tool	packages/mcp-server/src/tools.ts:408
paperclipGetIssueWorkspaceRuntime	eval:rf-iws-start-url-01	optional_agent_tool	packages/mcp-server/src/tools.ts:417
paperclipControlIssueWorkspaceServices	eval:rf-iws-start-url-01	optional_agent_tool	packages/mcp-server/src/tools.ts:423
paperclipWaitForIssueWorkspaceService	eval:rf-iws-target-restart-01	optional_agent_tool	packages/mcp-server/src/tools.ts:440
paperclipListGoals	eval:su-parent-goal-01	optional_agent_tool	packages/mcp-server/src/tools.ts:466
paperclipGetGoal	eval:su-parent-goal-01	optional_agent_tool	packages/mcp-server/src/tools.ts:472
paperclipListApprovals	eval:ap-approval-wake-01	optional_agent_tool	packages/mcp-server/src/tools.ts:478
paperclipCreateApproval	eval:ap-board-approval-01	optional_agent_tool	packages/mcp-server/src/tools.ts:487
paperclipGetApproval	eval:ap-approval-wake-01	optional_agent_tool	packages/mcp-server/src/tools.ts:496
paperclipGetApprovalIssues	eval:ap-approval-wake-01	optional_agent_tool	packages/mcp-server/src/tools.ts:502
paperclipListApprovalComments	eval:ap-approval-deny-01	optional_agent_tool	packages/mcp-server/src/tools.ts:508
paperclipCreateIssue	eval:su-parent-goal-01	optional_agent_tool	packages/mcp-server/src/tools.ts:514
paperclipUpdateIssue	eval:st-done-comment-01	always_agent_tool	packages/mcp-server/src/tools.ts:521
paperclipCheckoutIssue	eval:co-body-contract-01	control_plane_owned	packages/mcp-server/src/tools.ts:528
paperclipReleaseIssue	eval:er-release-01	control_plane_owned	packages/mcp-server/src/tools.ts:540
paperclipAddComment	eval:cm-multiline-01	always_agent_tool	packages/mcp-server/src/tools.ts:546
paperclipSuggestTasks	eval:ix-suggest-tasks-01	always_agent_tool	packages/mcp-server/src/tools.ts:553
paperclipAskUserQuestions	eval:ix-questions-01	always_agent_tool	packages/mcp-server/src/tools.ts:565
paperclipRequestConfirmation	eval:ix-confirmation-plan-01	always_agent_tool	packages/mcp-server/src/tools.ts:577
paperclipRequestCheckboxConfirmation	eval:ix-checkbox-01	always_agent_tool	packages/mcp-server/src/tools.ts:589
paperclipUpsertIssueDocument	eval:dp-plan-doc-01	always_agent_tool	packages/mcp-server/src/tools.ts:601
paperclipRestoreIssueDocumentRevision	eval:dp-base-revision-01	always_agent_tool	packages/mcp-server/src/tools.ts:612
paperclipLinkIssueApproval	eval:ap-board-approval-01	optional_agent_tool	packages/mcp-server/src/tools.ts:627
paperclipUnlinkIssueApproval	eval:ap-board-approval-01	optional_agent_tool	packages/mcp-server/src/tools.ts:636
paperclipApprovalDecision	eval:ap-approval-wake-01	optional_agent_tool	packages/mcp-server/src/tools.ts:646
paperclipAddApprovalComment	eval:ap-approval-deny-01	optional_agent_tool	packages/mcp-server/src/tools.ts:668
paperclipApiRequest	eval:rf-api-404-report-01	optional_agent_tool	packages/mcp-server/src/tools.ts:677

Capability	Primary disposition	Source anchor
skill:skills/paperclip/SKILL.md:paperclip-skill:10	optional_agent_tool	skills/paperclip/SKILL.md:10
skill:skills/paperclip/SKILL.md:terminology:14	optional_agent_tool	skills/paperclip/SKILL.md:14
skill:skills/paperclip/SKILL.md:authentication:20	control_plane_owned	skills/paperclip/SKILL.md:20
skill:skills/paperclip/SKILL.md:conversation-tasks:32	optional_agent_tool	skills/paperclip/SKILL.md:32
skill:skills/paperclip/SKILL.md:server-verified-external-chat-turns:49	control_plane_owned	skills/paperclip/SKILL.md:49
skill:skills/paperclip/SKILL.md:the-heartbeat-procedure:89	optional_agent_tool	skills/paperclip/SKILL.md:89
skill:skills/paperclip/SKILL.md:generated-artifacts-and-work-products:198	always_agent_tool	skills/paperclip/SKILL.md:198
skill:skills/paperclip/SKILL.md:status-quick-guide:246	control_plane_owned	skills/paperclip/SKILL.md:246
skill:skills/paperclip/SKILL.md:monitors-and-watchers-say-only-what-you-actually-scheduled:256	optional_agent_tool	skills/paperclip/SKILL.md:256
skill:skills/paperclip/SKILL.md:delegating-review-tasks:269	always_agent_tool	skills/paperclip/SKILL.md:269
skill:skills/paperclip/SKILL.md:managing-a-user-s-inbox:280	control_plane_owned	skills/paperclip/SKILL.md:280
skill:skills/paperclip/SKILL.md:issue-dependencies-blockers:288	control_plane_owned	skills/paperclip/SKILL.md:288
skill:skills/paperclip/SKILL.md:requesting-board-approval:313	optional_agent_tool	skills/paperclip/SKILL.md:313
skill:skills/paperclip/SKILL.md:issue-thread-interactions:334	optional_agent_tool	skills/paperclip/SKILL.md:334
skill:skills/paperclip/SKILL.md:standalone-decisions:363	optional_agent_tool	skills/paperclip/SKILL.md:363
skill:skills/paperclip/SKILL.md:mcp-tool-approval-gates:467	optional_agent_tool	skills/paperclip/SKILL.md:467
skill:skills/paperclip/SKILL.md:niche-workflow-pointers:509	optional_agent_tool	skills/paperclip/SKILL.md:509
skill:skills/paperclip/SKILL.md:cases:519	optional_agent_tool	skills/paperclip/SKILL.md:519
skill:skills/paperclip/SKILL.md:company-skills-workflow:524	optional_agent_tool	skills/paperclip/SKILL.md:524
skill:skills/paperclip/SKILL.md:routines:535	optional_agent_tool	skills/paperclip/SKILL.md:535
skill:skills/paperclip/SKILL.md:issue-workspace-runtime-controls:546	optional_agent_tool	skills/paperclip/SKILL.md:546
skill:skills/paperclip/SKILL.md:proposing-credentials-safely:553	optional_agent_tool	skills/paperclip/SKILL.md:553
skill:skills/paperclip/SKILL.md:reading-granted-secrets:560	optional_agent_tool	skills/paperclip/SKILL.md:560
skill:skills/paperclip/SKILL.md:critical-rules:586	optional_agent_tool	skills/paperclip/SKILL.md:586
skill:skills/paperclip/SKILL.md:comment-style-required:607	always_agent_tool	skills/paperclip/SKILL.md:607
skill:skills/paperclip/SKILL.md:update:639	optional_agent_tool	skills/paperclip/SKILL.md:639
skill:skills/paperclip/SKILL.md:planning-required-when-planning-requested:649	optional_agent_tool	skills/paperclip/SKILL.md:649
skill:skills/paperclip/SKILL.md:key-endpoints-hot-routes:680	optional_agent_tool	skills/paperclip/SKILL.md:680
skill:skills/paperclip/SKILL.md:searching-issues:709	optional_agent_tool	skills/paperclip/SKILL.md:709
skill:skills/paperclip/SKILL.md:full-reference:719	optional_agent_tool	skills/paperclip/SKILL.md:719
skill:skills/paperclip/SKILL.md:conversational-confirmation-answers:723	always_agent_tool	skills/paperclip/SKILL.md:723
skill:skills/paperclip/references/api-reference.md:paperclip-api-reference:1	optional_agent_tool	skills/paperclip/references/api-reference.md:1
skill:skills/paperclip/references/api-reference.md:response-schemas:9	optional_agent_tool	skills/paperclip/references/api-reference.md:9
skill:skills/paperclip/references/api-reference.md:agent-record-get-api-agents-me-or-get-api-agents-agentid:11	optional_agent_tool	skills/paperclip/references/api-reference.md:11
skill:skills/paperclip/references/api-reference.md:company-portability:44	optional_agent_tool	skills/paperclip/references/api-reference.md:44
skill:skills/paperclip/references/api-reference.md:issue-with-ancestors-get-api-issues-issueid:110	optional_agent_tool	skills/paperclip/references/api-reference.md:110
skill:skills/paperclip/references/api-reference.md:issue-update-response-patch-api-issues-issueid:196	optional_agent_tool	skills/paperclip/references/api-reference.md:196
skill:skills/paperclip/references/api-reference.md:blocker-diagnostics-get-api-issues-issueid-diagnostics-blockers:238	control_plane_owned	skills/paperclip/references/api-reference.md:238
skill:skills/paperclip/references/api-reference.md:wake-diagnostics-get-api-issues-issueid-diagnostics-wakes:277	control_plane_owned	skills/paperclip/references/api-reference.md:277
skill:skills/paperclip/references/api-reference.md:subtree-diagnostics-get-api-issues-issueid-diagnostics-subtree:321	optional_agent_tool	skills/paperclip/references/api-reference.md:321

30 KiB Raw Blame History