mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 15:55:46 +01:00
test(evals): check the wrong-resolver reply links Alex's request and task-link labels match their targets (#21028)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The chat eval catalog runs real chat journeys against real agents
and grades saved product state and rendered links, never reply wording
> - Full chat-eval cohort 4 had two cases whose graders passed replies
with a structural flaw that the grader could see but did not check
> - `connection-setup.wrong-person-resolves` promises that chat explains
whose action is required, but its grader had no reply check at all
> - `navigation-preferences.find-project-view` passed a reply with a
link labelled COM-2 that opened COM-1
> - This pull request adds two structural reply checks, with calibration
tests for valid, reworded-valid and broken replies
> - The benefit is that a reply that does not point to the request, or
that sends a person to the wrong task, fails the eval
## Linked Issues or Issue Description
**What happened?**
- `connection-setup.wrong-person-resolves`: another member said "I will
finish Alex's connection request for them". The product refused that
member (403 on setup options, phase, complete and decline) and kept
Alex's card pending. The grader checked only that. The reply's content
was not checked in any form.
- `navigation-preferences.find-project-view`: the saved reply contained
`[COM-2 Launch open website](/COM/issues/COM-1)`. The grader passed it.
**Expected behavior**
- The wrong-resolver reply points the person at Alex's actual pending
request. The card on that page says "Waiting for Alex", which is whose
action is required. The grader checks the link from saved state, and the
browser step follows the reply's own link to that card.
- Every task link in the find-project-view reply opens the task that its
visible identifier names.
**Steps to reproduce**
1. Run the two cases (OpenCode with
`openrouter/deepseek/deepseek-v4.1-flash-1017`) on master at f1aef0cb58.
2. Read the saved replies and the grader checks in the evidence.
## What Changed
- `external-wrong-resolver.ts`: new check
`reply-links-alex-pending-request`. The reply must render a same-origin
link to the related task that holds Alex's card (`/<company>/issues/<id
or identifier>`). `verifyResult` now follows that rendered link (not a
constructed URL) and confirms the "Waiting for Alex" card. Setup records
the company prefix. Oracle `paperclip.chat-catalog.external.v2` -> `v3`.
Reply wording is still not graded.
- `chat-rendered-links.ts`: new shared helper
`mislabeledIssueLinks(text, identifierFor?)`. It returns inline Markdown
links whose label names a task identifier (same prefix as the target)
other than the task the link opens. Labels without such a token, and
non-task targets, are not judged. Task-id targets are compared through
an optional resolver.
- `identity-navigation-preferences.ts`: find-project-view adds
`task-links-match-their-labels` with that helper. Oracle
`paperclip.chat-catalog.identity.v1` ->
`paperclip.chat-catalog.identity.find-project-view.v2`;
`chat-rendered-links.ts` joins its source files.
- `chat-catalog-descriptors.json`: the two descriptors carry the new
oracle versions and source files.
- The scenario YAML does not change, so the testops manifest has no
`changedFields` entry.
## Verification
- `npx vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/chat-catalog-domains/external-wrong-resolver.test.ts
tests/runner-e2e/chat-rendered-links.test.ts
tests/runner-e2e/chat-catalog-domains/identity.test.ts`: 92 tests pass.
- Calibration, wrong resolver: valid (links CMP-1), reworded valid
(links by task id, absolute same-origin link) pass; no link, another
task, another company prefix, another origin, and a sub-path fail.
Red/green: with the master grader, the "does not point at Alex's
request" test fails.
- Calibration, labels: valid, reworded (label without an identifier,
document anchor, task-id target resolved), and a different-prefix token
pass; the recorded cohort-4 link `[COM-2 Launch open
website](/COM/issues/COM-1)` fails. The helper flags exactly that link
in the retained cohort-4 reply.
- Retained cohort-4 wrong-resolver reply links `/COM/issues/COM-1` (the
related task), so it passes the new link check. Its wording ("completing
it there is the right move") is a guidance problem, fixed in #21027, not
something saved state can grade.
- `npx vitest run --config tests/runner-e2e/vitest.config.ts
chat-campaign chat-catalog`: 3856 tests pass. `node --test
tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 pass.
`tsc -p tests/runner-e2e/tsconfig.json` passes.
- A paid run of both cases with the product fixes is linked in a comment
below.
## Risks
- A valid reply that names Alex but links nothing now fails
wrong-person-resolves. The case contract says chat explains whose action
is required; the card is where Paperclip shows it, so pointing to it is
the structural form of that promise.
- The label check could flag a label that intentionally names two tasks
of the same company and links only one. That reply would still be
misleading, so failing it is acceptable.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed
1 parent
51d77648bc
commit
e879a24293
7 files changed
+62
-10
No files matched your search
@@ -2680,7 +2680,7 @@
|
||||
"../../server/src/services/connection-intents.ts",
|
||||
"chat-scenarios/connection-setup.yaml"
|
||||
],
|
||||
"oracleVersion": "paperclip.chat-catalog.external.v2",
|
||||
"oracleVersion": "paperclip.chat-catalog.external.v3",
|
||||
"expectedRunCount": 2,
|
||||
"expectedHumanMessageCount": 1,
|
||||
"minimumRunCount": 2,
|
||||
@@ -3670,9 +3670,10 @@
|
||||
"chat-catalog-domains/identity-support.ts",
|
||||
"chat-catalog-domains/internal-support.ts",
|
||||
"chat-catalog-domains/identity-navigation-preferences.ts",
|
||||
"chat-rendered-links.ts",
|
||||
"chat-scenarios/navigation-preferences.yaml"
|
||||
],
|
||||
"oracleVersion": "paperclip.chat-catalog.identity.v1",
|
||||
"oracleVersion": "paperclip.chat-catalog.identity.find-project-view.v2",
|
||||
"expectedRunCount": 1,
|
||||
"turnTimeoutMs": 120000,
|
||||
"attemptTimeoutMs": 300000,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import {describe,it,expect} from "vitest";
|
||||
import {gradeWrongResolver} from "./external-wrong-resolver.js";
|
||||
function valid(){const original={id:"card",kind:"connection_intent",status:"pending",addresseeUserId:"alex",sourceRunId:"alex-run",payload:{serviceSlug:"notion"}};return {scenarioId:"connection-setup.wrong-person-resolves",companyId:"company",leadId:"ceo",reply:"Only Alex can authorize their own Notion connection request. They need to finish that setup themselves.",phaseReplies:[],connections:[],profiles:[],secrets:[],endpoints:[],interactions:[],activity:[],links:[],humanQueries:[],runs:[{id:"alex-run",contextSnapshot:{issueId:"related"},agentId:"ceo",status:"succeeded",runtimeMode:"native"}],state:{original,after:{...original,result:null as unknown,resolvedByUserId:null as string|null,resolvedByAgentId:null as string|null},related:{id:"related"},alexId:"alex",otherId:"other",relatedVisible:true,otherSessionConfirmed:true,denied:{setupOptions:403,phase:403,complete:403,decline:403},waitingVisible:true,connectControlAbsent:true,relatedStatusUnchanged:true,alexSourceRunCount:1,canonicalSourceDifferent:true}};}
|
||||
function valid(){const original={id:"card",kind:"connection_intent",status:"pending",addresseeUserId:"alex",sourceRunId:"alex-run",payload:{serviceSlug:"notion"}};return {scenarioId:"connection-setup.wrong-person-resolves",origin:"http://paperclip.test",companyId:"company",leadId:"ceo",reply:"Only Alex can authorize their own Notion connection request on [CMP-1](/CMP/issues/CMP-1). They need to finish that setup themselves.",phaseReplies:[],connections:[],profiles:[],secrets:[],endpoints:[],interactions:[],activity:[],links:["/CMP/issues/CMP-1"],humanQueries:[],runs:[{id:"alex-run",contextSnapshot:{issueId:"related"},agentId:"ceo",status:"succeeded",runtimeMode:"native"}],state:{original,after:{...original,result:null as unknown,resolvedByUserId:null as string|null,resolvedByAgentId:null as string|null},related:{id:"related",identifier:"CMP-1"},alexId:"alex",otherId:"other",relatedVisible:true,otherSessionConfirmed:true,denied:{setupOptions:403,phase:403,complete:403,decline:403},waitingVisible:true,connectControlAbsent:true,relatedStatusUnchanged:true,alexSourceRunCount:1,canonicalSourceDifferent:true,companyPrefix:"CMP"}};}
|
||||
describe("actual addressed connection resolver boundary",()=>{
|
||||
it("accepts only a genuine pending Alex request with an independently denied different human",()=>expect(gradeWrongResolver(valid()).passed).toBe(true));
|
||||
it("rejects impersonation, unauthorized completion, account creation and released waiting work",()=>{
|
||||
@@ -17,8 +17,11 @@ describe("actual addressed connection resolver boundary",()=>{
|
||||
(o:ReturnType<typeof valid>)=>{o.state.waitingVisible=false;},
|
||||
]){const o=valid();mutate(o);expect(gradeWrongResolver(o).passed).toBe(false);}
|
||||
});
|
||||
it("grades the refused routes and pending card, not the reply's words",()=>{
|
||||
for(const reply of ["I cannot authorize Alex's account. Only Alex can finish their own connection request.","Alex has to connect Notion from the card on [CMP-2](/CMP/issues/CMP-2)."]){const o=valid();o.reply=reply;expect(gradeWrongResolver(o).passed).toBe(true);}
|
||||
it("grades the refused routes, pending card and linked request, not the reply's words",()=>{
|
||||
for(const [reply,links] of [["That request is addressed to Alex, so Alex has to finish it from [their card](/CMP/issues/related).",["/CMP/issues/related"]],["Alex has to connect Notion from the card on [CMP-1](http://paperclip.test/CMP/issues/CMP-1).",["http://paperclip.test/CMP/issues/CMP-1"]]] as const){const o=valid();o.reply=reply;o.links=[...links];expect(gradeWrongResolver(o).passed).toBe(true);}
|
||||
expect(gradeWrongResolver(undefined).passed).toBe(false);
|
||||
});
|
||||
it("rejects a reply that does not point at Alex's actual pending request",()=>{
|
||||
for(const links of [[],["/CMP/issues/CMP-2"],["/OTHER/issues/CMP-1"],["https://elsewhere.test/CMP/issues/CMP-1"],["/CMP/issues/CMP-1/comments"]]){const o=valid();o.links=links;expect(gradeWrongResolver(o).passed).toBe(false);}
|
||||
});
|
||||
});
|
||||
@@ -16,8 +16,13 @@ export function gradeWrongResolver(value:unknown){
|
||||
c.add("all-addressed-resolution-surfaces-deny-other-person",!!denied&&["setupOptions","phase","complete","decline"].every(key=>denied[key]===403)&&o.state.waitingVisible===true&&o.state.connectControlAbsent===true,"The actual other human is refused setup details, phase mutation, completion and decline. Their visible card exposes waiting identity rather than a usable authorization control.");
|
||||
c.add("pending-work-and-identity-preserved",!!before&&!!after&&after.id===before.id&&after.kind===before.kind&&after.sourceRunId===before.sourceRunId&&after.payload?.serviceSlug===before.payload.serviceSlug&&after.addresseeUserId===before.addresseeUserId&&after.status==="pending"&&!after.result&&!after.resolvedByUserId&&!after.resolvedByAgentId&&o.state.relatedStatusUnchanged===true&&o.state.alexSourceRunCount===1&&o.connections.length===0&&o.state.canonicalSourceDifferent===true,"Unauthorized responses neither release the pending task nor create a connection, readdress the card, or schedule another Alex-source run.");
|
||||
// Reply wording is not graded: the refused resolution routes and the unchanged pending card prove the boundary held.
|
||||
c.add("reply-links-alex-pending-request",!!related&&typeof o.state.companyPrefix==="string"&&linksRelatedRequest(o,related,o.state.companyPrefix),"The reply links Alex's actual pending request in this company, where the card shows whose action is required.");
|
||||
return c.result();
|
||||
}
|
||||
export function relatedRequestHref(o:{links:string[];origin:string},related:Entity,companyPrefix:string):string|undefined{
|
||||
return o.links.find(link=>{try{const url=new URL(link,o.origin),parts=url.pathname.split("/").filter(Boolean);return url.origin===o.origin&&parts.length===3&&parts[0]===companyPrefix&&parts[1]==="issues"&&[related.id,related.identifier].some(value=>value===decodeURIComponent(parts[2]!));}catch{return false;}});
|
||||
}
|
||||
function linksRelatedRequest(o:{links:string[];origin:string},related:Entity,companyPrefix:string){return relatedRequestHref(o,related,companyPrefix)!==undefined;}
|
||||
export const wrongResolverExternalCases=[definition("connection-setup.wrong-person-resolves",async ctx=>{
|
||||
const browser=ctx.input.page.context().browser(),secrets=ctx.input.privateSecrets;if(!browser||!secrets)throw new Error("Wrong connection resolver needs two real authenticated people and private session ownership");
|
||||
const session=await ctx.input.api.get<{user:{id:string}}>("/api/auth/get-session");if(!session.user?.id||session.user.id==="local-board")throw new Error("Actual signed-in other member is required");
|
||||
@@ -32,10 +37,10 @@ export const wrongResolverExternalCases=[definition("connection-setup.wrong-pers
|
||||
const cards=prepared.cards.filter(card=>card.kind==="connection_intent"&&card.payload.serviceSlug==="notion");if(cards.length!==1)throw new Error("Alex's prerequisite produced ambiguous requests");const original=cards[0]!;
|
||||
const before=await ctx.input.api.get<Entity>(`/api/issues/${related.id}`),chat=await openStandingChat(ctx);
|
||||
await ctx.input.api.patch(`/api/issues/${chat.id}`,{description:`Alex has a pending Notion connection request on related task ${before.identifier??before.id}. The current person can view that task as another company member.`});
|
||||
return {original,related:before,alexId:alex.id,otherId:session.user.id};
|
||||
return {original,related:before,alexId:alex.id,otherId:session.user.id,companyPrefix:ctx.input.fixtures.company.issuePrefix};
|
||||
}catch(error){await alex.dispose();people.delete(ctx);throw error;}
|
||||
},gradeWrongResolver,{
|
||||
oracleVersion:"paperclip.chat-catalog.external.v2",authentication:"human-session",expectedRunCount:2,minimumRunCount:2,expectedHumanMessageCount:1,attemptTimeoutMs:540000,
|
||||
oracleVersion:"paperclip.chat-catalog.external.v3",authentication:"human-session",expectedRunCount:2,minimumRunCount:2,expectedHumanMessageCount:1,attemptTimeoutMs:540000,
|
||||
sourceFiles:["chat-catalog-domains/external-wrong-resolver.ts","chat-catalog-domains/identity-auth.ts","chat-catalog-domains/identity-private-browser.ts","chat-catalog-domains/identity-private-browser-worker.ts","chat-catalog-domains/identity-wizard.ts","chat-catalog-domains/identity-support.ts","chat-catalog-domains/external-support.ts","chat-catalog-domains/internal-support.ts","../../server/src/routes/connection-intents.ts","../../server/src/services/connection-intents.ts","chat-scenarios/connection-setup.yaml"],
|
||||
async act(ctx,state){
|
||||
await waitForReply({...ctx,allRuns:async()=>(await ctx.allRuns()).filter(run=>run.contextSnapshot?.issueId===ctx.issue()?.id)});
|
||||
@@ -55,6 +60,6 @@ export const wrongResolverExternalCases=[definition("connection-setup.wrong-pers
|
||||
state.after=after;state.relatedStatusUnchanged=task.status===related.status&&task.assigneeAgentId===related.assigneeAgentId;state.alexSourceRunCount=o.runs.filter(run=>run.contextSnapshot?.issueId===related.id).length;state.canonicalSourceDifferent=ctx.issue()?.id!==related.id;
|
||||
return {...o,state};
|
||||
},
|
||||
async verifyResult(ctx,_state,value){await verifyRenderedReply(ctx,value);const o=externalObservation(value)!;const source=ctx.input.page.url();try{await ctx.input.page.goto(`/${ctx.input.fixtures.company.issuePrefix}/issues/${(o.state.related as Entity).id}`,{waitUntil:"domcontentloaded"});await expect(ctx.input.page.getByTestId(/^connection-(?:intent|authorization)-waiting$/).filter({hasText:"Waiting for Alex"})).toBeVisible();}finally{await ctx.input.page.goto(source,{waitUntil:"domcontentloaded"});}},
|
||||
async verifyResult(ctx,_state,value){await verifyRenderedReply(ctx,value);const o=externalObservation(value)!;const href=relatedRequestHref(o,o.state.related as Entity,String(o.state.companyPrefix));if(!href)throw new Error("The reply rendered no link to Alex's pending request to follow");const source=ctx.input.page.url();try{await ctx.input.page.goto(href,{waitUntil:"domcontentloaded"});await expect(ctx.input.page.getByTestId(/^connection-(?:intent|authorization)-waiting$/).filter({hasText:"Waiting for Alex"})).toBeVisible();}finally{await ctx.input.page.goto(source,{waitUntil:"domcontentloaded"});}},
|
||||
async cleanup(ctx){try{await people.get(ctx)?.dispose();}finally{people.delete(ctx);}},
|
||||
})];
|
||||
@@ -1,4 +1,5 @@
|
||||
import { chatCatalogOriginalPhaseDeadline } from "../chat-catalog-contract.js";
|
||||
import { mislabeledIssueLinks } from "../chat-rendered-links.js";
|
||||
import { expect } from "@playwright/test";
|
||||
import { checks, entity, hasLink, project, task, verifyInternalResult } from "./internal-support.js";
|
||||
import { identityDefinition, identityObservation, waitForReply, probeAgentPermission } from "./identity-support.js";
|
||||
@@ -29,6 +30,8 @@ export function gradeNavigation(value: unknown) {
|
||||
c.add("navigation-is-read-only", !o.activity.some(row => row.agentId === o.leadId && [p.id, t.id].includes(row.entityId ?? "")), "The agent does not rewrite records to make the requested view appear correct.");
|
||||
if (o.scenarioId === "navigation-preferences.find-project-view") {
|
||||
c.add("actual-project-link", hasLink(o, "projects", p), "The rendered reply links the exact Launch project.");
|
||||
c.add("task-links-match-their-labels", mislabeledIssueLinks(o.reply, id => o.tasks.find(row => row.id === id)?.identifier ?? undefined).length === 0,
|
||||
"Every task link in the saved reply opens the task its visible identifier names.");
|
||||
c.add("open-task-view", o.state.openTaskVisible === true && o.state.closedTaskVisible === false,
|
||||
"Following the project link shows the actual open task and excludes the completed task after selecting the supported open filter.");
|
||||
c.add("personal-preferences-not-agent-mutated", o.state.agentPreferenceReadStatus === 403 && o.state.preferencesBefore !== undefined
|
||||
@@ -72,7 +75,8 @@ export const navigationIdentityCases = [
|
||||
await ctx.input.evidence("identity-project-navigation.json", { projectId: p.id, openTaskId: entity(state, "task").id,
|
||||
openTaskVisible: state.openTaskVisible, closedTaskVisible: state.closedTaskVisible, url: ctx.input.page.url() });
|
||||
await ctx.input.page.goto(source, { waitUntil: "domcontentloaded" });
|
||||
}, verifyResult: verifyInternalResult,
|
||||
}, verifyResult: verifyInternalResult, oracleVersion: "paperclip.chat-catalog.identity.find-project-view.v2",
|
||||
sourceFiles: ["chat-catalog-domains/identity-support.ts", "chat-catalog-domains/internal-support.ts", "chat-catalog-domains/identity-navigation-preferences.ts", "chat-rendered-links.ts", "chat-scenarios/navigation-preferences.yaml"],
|
||||
}),
|
||||
identityDefinition("navigation-preferences.quick-search", async ctx => {
|
||||
const state = await fixture(ctx); const browser = ctx.input.page.context().browser();
|
||||
|
||||
@@ -119,6 +119,16 @@ describe("identity catalog independent oracle calibration", () => {
|
||||
expect(definition.grade(undefined).passed).toBe(false); expect(definition.grade({ ...good, activity: undefined }).passed).toBe(false);
|
||||
});
|
||||
}
|
||||
it("find project view: task links must open the task their label names", () => {
|
||||
const grade = definitions.find(row => row.scenarioId === "navigation-preferences.find-project-view")!.grade;
|
||||
const linked = valid("navigation-preferences.find-project-view"); (linked.tasks[0] as { identifier?: string }).identifier = "ACORN-1";
|
||||
linked.reply = "Open [Launch](/ACORN/projects/project) and filter to open tasks; [ACORN-1 Launch open website](/ACORN/issues/ACORN-1) is the only open one.";
|
||||
expect(grade(linked).passed).toBe(true);
|
||||
const reworded = structuredClone(linked); reworded.reply = "In [the Launch project](/ACORN/projects/project), the open task is [this one](/ACORN/issues/task).";
|
||||
expect(grade(reworded).passed).toBe(true);
|
||||
const mislabeled = structuredClone(linked); mislabeled.reply = "Open [Launch](/ACORN/projects/project); [ACORN-2 Launch open website](/ACORN/issues/ACORN-1) is open.";
|
||||
expect(grade(mislabeled).passed).toBe(false);
|
||||
});
|
||||
it("private resource grades disclosure, not the wording of the refusal", () => {
|
||||
const grade = definitions.find(row => row.scenarioId === "access.private-resource")!.grade;
|
||||
const reworded = valid("access.private-resource"); reworded.reply = "I can't open that one from this chat. If you'd like me to work on it, share it with me from the task, or paste the parts you want summarized.";
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
// @vitest-environment jsdom
|
||||
import { readFileSync } from "node:fs";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { collectConversationLinks, conversationLinkBindings, readConversationDomLinks, readLatestConversationReplyRender, waitForLatestConversationReplyRender, waitForSavedTurnReplyRender, type ConversationLinkInput } from "./chat-rendered-links.js";
|
||||
import { collectConversationLinks, conversationLinkBindings, mislabeledIssueLinks, readConversationDomLinks, readLatestConversationReplyRender, waitForLatestConversationReplyRender, waitForSavedTurnReplyRender, type ConversationLinkInput } from "./chat-rendered-links.js";
|
||||
|
||||
const card = '<div data-thread-item-kind="project_created" data-thread-anchor="project-created:project"><article aria-label="Project created: Launch"><div><p>Project created</p><a href="/RUN/projects/project">Launch</a></div></article></div>';
|
||||
const thread = (content = card) => `<div data-testid="task-chat-thread"><div data-testid="task-chat-history-content"><div data-testid="task-chat-scroller"><div id="comment-human" data-thread-item-kind="human">Create a project called Launch for the new website.</div>${content}<div id="comment-reply" data-thread-item-kind="agent">Created the Launch project. No repositories are connected yet.</div></div></div></div>`;
|
||||
@@ -144,3 +144,16 @@ describe("waits for a finished run's reply to be saved, then rendered, before it
|
||||
expect(Date.now() - started).toBeLessThan(1_500);
|
||||
});
|
||||
});
|
||||
|
||||
describe("task links whose visible identifier names another task", () => {
|
||||
it("accepts links whose label names the task they open, or names no task", () => {
|
||||
expect(mislabeledIssueLinks("Open [COM-1](/COM/issues/COM-1) in [Launch](/COM/projects/launch).")).toEqual([]);
|
||||
expect(mislabeledIssueLinks("The [open COM-1 task](/COM/issues/com-1#document-plan) and [this one](/COM/issues/COM-2).")).toEqual([]);
|
||||
expect(mislabeledIssueLinks("[COM-1 Launch open website](/COM/issues/task-uuid)", id => id === "task-uuid" ? "COM-1" : undefined)).toEqual([]);
|
||||
expect(mislabeledIssueLinks("[SHA-256 notes for COM-1](/COM/issues/COM-1)")).toEqual([]);
|
||||
});
|
||||
it("rejects a link whose label names a different task than it opens", () => {
|
||||
expect(mislabeledIssueLinks("Open tasks: [COM-2 Launch open website](/COM/issues/COM-1)... correction:")).toEqual(["[COM-2 Launch open website](/COM/issues/COM-1)"]);
|
||||
expect(mislabeledIssueLinks("[COM-2](/COM/issues/task-uuid)", id => id === "task-uuid" ? "COM-1" : undefined)).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
@@ -20,6 +20,22 @@ export function markdownLinkTargets(text: string): string[] {
|
||||
...[...text.matchAll(/<((?:https?:\/\/|\/)[^>\s]+)>/g)].map(match => match[1]!),
|
||||
];
|
||||
}
|
||||
export function mislabeledIssueLinks(text: string, identifierFor: (pathId: string) => string | undefined = () => undefined): string[] {
|
||||
const identifier = /^[A-Za-z][A-Za-z0-9]*-\d+$/;
|
||||
return [...text.matchAll(/\[((?:[^\[\]]|\[[^\]]*\])*)\]\(\s*<?([^)\s>]+)>?[^)]*\)?/g)].flatMap(match => {
|
||||
const label = match[1]!, target = match[2]!;
|
||||
let pathname: string;
|
||||
try { pathname = decodeURIComponent(new URL(target, "http://paperclip.invalid").pathname); } catch { return []; }
|
||||
const parts = pathname.split("/").filter(Boolean); const index = parts.indexOf("issues"); const pathId = index >= 0 ? parts[index + 1] : undefined;
|
||||
if (!pathId) return [];
|
||||
const opened = (identifier.test(pathId) ? pathId : identifierFor(pathId))?.toUpperCase();
|
||||
if (!opened) return [];
|
||||
const prefix = opened.split("-")[0]!;
|
||||
const named = [...label.matchAll(/(?<![\w-])([A-Za-z][A-Za-z0-9]*-\d+)(?![\w-])/g)].map(token => token[1]!.toUpperCase())
|
||||
.filter(token => token.split("-")[0] === prefix);
|
||||
return named.length > 0 && !named.includes(opened) ? [match[0]] : [];
|
||||
});
|
||||
}
|
||||
/** A link target that still holds a template placeholder such as <prefix> or {companyId}; it opens no real page. */
|
||||
export function unresolvedLinkTarget(href: string): boolean {
|
||||
let decoded = href;
|
||||
|
||||
Reference in new issue
Block a user