Files
PaperClipAI/ui/src/lib/issue-reference.test.ts
T
DottaandPaperclip a386a59998 Reduce repeated native completion guidance and preserve final replies (#15151)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents receive task constraints and completion tools from
Paperclip.
> - Completion tools already define the procedure for reporting a
result.
> - Repeated procedure text adds instructions to each full task turn.
> - The final reply must still explain a blocker and link a saved
document.
> - This pull request removes repeated procedure text and keeps these
visible outcome requirements explicit.
> - A document receipt supplies the exact link, and stricter evals check
the persisted reply and browser navigation.

## Linked Issues or Issue Description

Refs: #14961. Related: #14948 and #15007.

**What happened?**

Native task envelopes repeat completion procedure text. A reduced
envelope needs explicit final-reply requirements. The `write_document`
receipt also lacks a canonical document link.

**Expected behavior**

Keep the completion tools as the source of procedure details. Require
one accepted completion result before the final reply. A blocked reply
must explain the reason, owner and unblock action. A document reply must
contain a working link to the saved document.

**Steps to reproduce**

1. Run the native assigned-skill document case and native blocker case.
2. Inspect the run-attributed provider final and its persisted comment.
3. Check the blocker explanation or open the final reply's document
link.

## What Changed

- Remove repeated completion procedure text from the native task
constraints and backend instructions.
- Keep explicit blocker and document-link requirements in full task
turns.
- Return a company/task-scoped `documentHref` from `write_document`.
Preserve the link in the idempotent mutation receipt.
- Repeat canonical links for this run's current saved revisions in
accepted completion feedback. Give blocked providers final-response
guidance for the cause, owner and unblock action.
- Keep internal document/comment anchors when Markdown issue links load
cached issue details.
- Add a manual six-cell comparison suite with strict source, build,
default-instruction and budget admission.
- Capture eighteen shared runnerd RPC projections and six direct
OpenCode HTTP projections across start, resume and continuation phases,
using scripted local transports and no provider execution.
- Apply v3 checks only to the manual instruction comparison; preserve v2
checks for the existing native completion suite. Check the actual
persisted blocker reason and exact saved-document link. Click the
rendered document link and check the original content marker in the
classic document card or the new document tab.
- Forward exact OpenCode finishing calls through the controller. Wait
for acceptance, keep accepted feedback and concrete rejection text, and
reject malformed responses. Preserve ordinary dynamic-tool response
handling.
- Settle the completion decision and tool response before mapping a
racing idle/error/abort event or handling explicit close/interruption.
Reject a concurrent finishing call before controller admission.
- Add a provider-free regression through real runnerd, the OpenCode
proxy and a fake provider. Reject the first completion, accept the
corrected report in the same turn, and propose one result.
- Keep all original verdicts unchanged. Treat replay under new checks as
separate diagnostics.

## Verification

- `pnpm -r typecheck` and `pnpm build` pass locally.
- Native document-authority tests pass, including company/run
authorization and idempotent replay.
- Native runtime-context, backend and measurement tests pass.
- Final-answer calibration, protocol scoring, source-admission and
catalog tests pass. Wrong reasons, absent links and wrong link targets
fail.
- `pnpm test:e2e:runner:typecheck` passes. Discovery lists exactly six
single-attempt local cells with the declared models.
- Exported `prepareNativeInstructionPreflight` then
`verifyNativeInstructionPreflight` pass on this clean committed source.
They build locally and make zero provider calls.
- Corrective live confirmation is incomplete. Source 3a7349d passed both
Claude and both Codex cases. OpenCode saved the correct document but
omitted its final link; its blocker case was canceled before paid
execution. Preserve this failure. The e171282 confirmation was stopped
during build after fresh review found a completion-settlement race; it
executed zero providers. Source c3e0cb303 fixes that race. Two affected
OpenCode cases await fresh review and one bounded confirmation; earlier
results remain attributed to their original source.
- OpenCode proxy parsing, driver, factory and input tests: 81 pass
across retained focused runs, including six settlement races.
Evaluator/scoring/admission checks: 122 pass. The real proxy regression
passes. Fresh local prepare then verify passes with 18 shared and 6
direct scripted captures, fresh SDK/Rust builds and zero providers.
- The full local suite recorded two failures: a webhook timeout and a
Git-scan load count mismatch. Both files pass in isolation with
unchanged assertions/time budgets; preserve the original failure log.
Fresh c3e0cb303 CI and review are pending. This PR remains draft.

## Risks

- Final-answer wording can vary by provider. The checks cover the
declared release-access blocker and saved document fixture, not general
answer quality.
- A single trial does not establish general equivalence, cause, speed,
cost or live resume behavior.
- `documentHref` is an additive receipt field. It points to the current
saved document, not an immutable historical revision. Replaying an older
receipt does not fabricate a new link.
- The correction adds four production paths for document receipts,
accepted completion feedback and UI navigation, plus four OpenCode
controller/proxy paths, beyond the original three instruction paths.
Completion rejection must remain repairable; the production-boundary
regression covers it.
- Preserve the frozen comparison context for live measurement. A
merge-tree check against current master is clean. Do not relabel earlier
live results as results from a later source tree.

## Model Used

- OpenAI Codex, GPT-6 family. The exact serving model ID and context
window are unavailable in this session. Capabilities used: reasoning,
code editing, shell execution, test authoring and evidence review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-05 08:10:14 -05:00

162 lines
6.5 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { parseIssuePathIdFromPath, parseIssueReferenceFromHref, remarkLinkIssueReferences } from "./issue-reference";
type TreeNode = { type: string; value?: string; url?: string; children?: TreeNode[] };
function paragraph(value: string): TreeNode {
return { type: "root", children: [{ type: "paragraph", children: [{ type: "text", value }] }] };
}
function paragraphChildren(tree: TreeNode): TreeNode[] {
return tree.children?.[0]?.children ?? [];
}
describe("issue-reference", () => {
it("extracts issue ids from company-scoped issue paths", () => {
expect(parseIssuePathIdFromPath("/PAP/issues/PAP-1271")).toBe("PAP-1271");
expect(parseIssuePathIdFromPath("/PAP/issues/pap-1272")).toBe("PAP-1272");
expect(parseIssuePathIdFromPath("/issues/pc1a2-7")).toBe("PC1A2-7");
expect(parseIssuePathIdFromPath("/PC1A2/issues/pc1a2-7")).toBe("PC1A2-7");
expect(parseIssuePathIdFromPath("/issues/PAP-1179")).toBe("PAP-1179");
expect(parseIssuePathIdFromPath("/issues/:id")).toBeNull();
});
it("keeps anchors and queries in navigation, outside the issue lookup ID", () => {
for (const suffix of ["#document-output", "#comment-123", "?view=chat#document-output"]) {
expect(parseIssuePathIdFromPath(`/PAP/issues/pap-1271${suffix}`)).toBe("PAP-1271");
expect(parseIssueReferenceFromHref(`/PAP/issues/pap-1271${suffix}`)).toEqual({
issuePathId: "PAP-1271", href: `/issues/PAP-1271${suffix}`,
});
expect(parseIssueReferenceFromHref(`issue://PAP-1271${suffix}`)).toEqual({
issuePathId: "PAP-1271", href: `/issues/PAP-1271${suffix}`,
});
}
});
it("does not treat full issue URLs as internal issue paths", () => {
expect(parseIssuePathIdFromPath("http://localhost:3100/PAP/issues/PAP-1179")).toBeNull();
expect(parseIssuePathIdFromPath("http://remote.example.test:3103/PAPA/issues/PAPA-115#comment-850083f3-24de-43e7-a8cd-bc01f7cc9f0d")).toBeNull();
});
it("does not treat GitHub issue URLs as internal Paperclip issue links", () => {
expect(parseIssuePathIdFromPath("https://github.com/paperclipai/paperclip/issues/1778")).toBeNull();
expect(parseIssueReferenceFromHref("https://github.com/paperclipai/paperclip/issues/1778")).toBeNull();
});
it("ignores placeholder issue paths", () => {
expect(parseIssuePathIdFromPath("/issues/:id")).toBeNull();
expect(parseIssuePathIdFromPath("http://localhost:3100/issues/:id")).toBeNull();
expect(parseIssueReferenceFromHref("/issues/:id")).toBeNull();
});
it("normalizes bare identifiers, relative issue paths, and issue scheme links into internal links", () => {
expect(parseIssueReferenceFromHref("pap-1271")).toEqual({
issuePathId: "PAP-1271",
href: "/issues/PAP-1271",
});
expect(parseIssueReferenceFromHref("pc1a2-7")).toEqual({
issuePathId: "PC1A2-7",
href: "/issues/PC1A2-7",
});
expect(parseIssueReferenceFromHref("/PAP/issues/pap-1180")).toEqual({
issuePathId: "PAP-1180",
href: "/issues/PAP-1180",
});
expect(parseIssueReferenceFromHref("issue://PAP-1310")).toEqual({
issuePathId: "PAP-1310",
href: "/issues/PAP-1310",
});
expect(parseIssueReferenceFromHref("issue://:PAP-1311")).toEqual({
issuePathId: "PAP-1311",
href: "/issues/PAP-1311",
});
});
it("normalizes exact inline-code-like issue identifiers", () => {
expect(parseIssueReferenceFromHref("PAP-1271")).toEqual({
issuePathId: "PAP-1271",
href: "/issues/PAP-1271",
});
});
it("preserves absolute Paperclip issue URLs so origin, port, and hash are not lost", () => {
expect(parseIssueReferenceFromHref("http://localhost:3100/PAP/issues/PAP-1179")).toBeNull();
expect(parseIssueReferenceFromHref("http://remote.example.test:3103/PAPA/issues/PAPA-115#comment-850083f3-24de-43e7-a8cd-bc01f7cc9f0d")).toBeNull();
});
it("ignores literal route placeholder paths", () => {
expect(parseIssueReferenceFromHref("/issues/:id")).toBeNull();
expect(parseIssueReferenceFromHref("http://localhost:3100/api/issues/:id")).toBeNull();
});
describe("known-prefix gating", () => {
it("links a bare identifier whose prefix is known", () => {
expect(parseIssueReferenceFromHref("PAP-1271", new Set(["PAP"]))).toEqual({
issuePathId: "PAP-1271",
href: "/issues/PAP-1271",
});
});
it("matches the prefix case-insensitively", () => {
expect(parseIssueReferenceFromHref("pap-12", new Set(["PAP"]))).toEqual({
issuePathId: "PAP-12",
href: "/issues/PAP-12",
});
});
it("does not link a bare identifier whose prefix is unknown (e.g. a Jira key)", () => {
expect(parseIssueReferenceFromHref("JIRA-456", new Set(["PAP"]))).toBeNull();
});
it("stays permissive when no prefix set is supplied", () => {
expect(parseIssueReferenceFromHref("FOO-1")).toEqual({
issuePathId: "FOO-1",
href: "/issues/FOO-1",
});
});
it("stays permissive when the prefix set is empty", () => {
expect(parseIssueReferenceFromHref("FOO-1", new Set())).toEqual({
issuePathId: "FOO-1",
href: "/issues/FOO-1",
});
});
it("never gates explicit issue:// scheme references", () => {
expect(parseIssueReferenceFromHref("issue://ACME-9", new Set(["PAP"]))).toEqual({
issuePathId: "ACME-9",
href: "/issues/ACME-9",
});
});
it("never gates explicit /issues/ path references", () => {
expect(parseIssueReferenceFromHref("/ACME/issues/ACME-9", new Set(["PAP"]))).toEqual({
issuePathId: "ACME-9",
href: "/issues/ACME-9",
});
});
});
describe("remarkLinkIssueReferences", () => {
it("links only known-prefix tokens and leaves foreign keys as text", () => {
const tree = paragraph("See PAP-1 and JIRA-2 today.");
remarkLinkIssueReferences({ knownPrefixes: ["PAP"] })(tree);
const children = paragraphChildren(tree);
expect(children).toEqual([
{ type: "text", value: "See " },
{ type: "link", url: "/issues/PAP-1", children: [{ type: "text", value: "PAP-1" }] },
{ type: "text", value: " and JIRA-2 today." },
]);
});
it("links every identifier when no prefixes are supplied (legacy permissive)", () => {
const tree = paragraph("See PAP-1 and JIRA-2.");
remarkLinkIssueReferences()(tree);
const links = paragraphChildren(tree).filter((node) => node.type === "link");
expect(links.map((node) => node.url)).toEqual(["/issues/PAP-1", "/issues/JIRA-2"]);
});
});
});