mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat answers a person's requests with a chat agent that reads API results and the chat guidance > - A decision card can be human-only and addressed to one person, and the teams catalog lists ready-made teams with their requirements > - When an agent tried to accept another person's card, the denial said only "human-only", so the agent told the asker to accept the card themselves > - When an agent recommended a catalog team, the guidance asked only for roles, so the agent left out the required skills > - This pull request makes the denial name the addressee as the only resolver, and asks agents to list a team's roles, required skills and configuration inputs > - The benefit is that chat answers send people to the right person and show what a team needs before install ## Linked Issues or Issue Description Refs #20206, #20220, #20239 **What happened** Alex wrote "I approve Riley's request." in Agent Chat. The pending card is human-only and addressed to Riley. The agent tried to record the decision and got `403 interaction_human_only` with only "This issue-thread interaction is human-only". The reply correctly said the card stays pending, but it then told Alex to "accept or decline that request" on the task. Alex cannot do that; only Riley can. In a second chat, a person asked "Do we have a ready-made team for building a website?". The agent recommended Product Engineering and its roles. It did not name the four required skills that the catalog returned in `requiredSkills`. **Expected behavior** When an agent cannot resolve a card that is addressed to another person, the reply names that person as the only one who can resolve it. The reply does not tell the asker to act on the card. When an agent recommends a catalog team, the reply names the team's roles, every required skill and every configuration input. It says that a catalog listing does not mean the accounts are configured. **Steps to reproduce** 1. As user Alex, create a `request_confirmation` card with `resolverPolicy: "human_only"` and `addresseeUserId` set to user Riley on Alex's chat task. 2. In that chat, write "I approve Riley's request." 3. Read the agent run. `resolve-from-comment` returns 403 `interaction_human_only` without the addressee. The reply tells Alex to accept or decline the card. 4. In a new chat, ask "Do we have a ready-made team for building a website?". 5. Read the reply. It names Product Engineering and its roles, but not its required skills. ## What Changed - `server/src/services/issue-thread-interaction-resolution.ts`: when an agent is denied on a human-only card that is addressed to a person other than the verified replying user, the denial says the card is addressed to that person, that only they can accept or decline it, that the denial is final, and that the agent must not ask the asker to act on the card. The details include `addresseeUserId`. Cards without an addressee keep the old message. - `server/src/routes/issues.ts`: the route adds the addressee's display name to that denial (`details.addresseeName` and the message). If the name lookup fails, the generic addressee text stays. - `server/src/services/agent-conversations.ts`: the teams catalog paragraph asks agents to name a recommended team's roles (`agentSlugs`), every required skill (`requiredSkills`, by ref) and every configuration input (`envInputs`), and to say that a listing does not mean accounts are configured. - Tests: evaluator unit tests, a route-level test for the named denial, and a directive assertion. ## Verification - `cd server && npx vitest run src/services/issue-thread-interaction-resolution.test.ts src/__tests__/agent-conversations.test.ts src/__tests__/issue-thread-interaction-routes.test.ts src/__tests__/confirmation-comment-resolution.test.ts`: 4 files, all pass. - `tsc --noEmit -p server/tsconfig.json`: passes. - The change is cherry-picked onto the eval branch `chat-eval/case-reply-completeness`. The live Runner E2E cells `questions-and-approvals.wrong-reviewer` and `teams.find-team` run there. ## Risks - Low risk. The resolver decision does not change: the same requests are allowed and denied. Only the denial text and details change for agents on addressed human-only cards. - The denial reads one user row for the display name. A failed read falls back to the generic text. - The chat directive is slightly longer, so each chat prompt uses a few more tokens. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
249 lines
11 KiB
TypeScript
249 lines
11 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import {
|
|
addressedHumanOnlyDenialMessage,
|
|
evaluateIssueThreadInteractionResolverAudience,
|
|
issueThreadInteractionAttentionAgentAllowed,
|
|
} from "./issue-thread-interaction-resolution.js";
|
|
|
|
const agent = { type: "agent", agentId: "agent-1", runId: "run-1" } as const;
|
|
|
|
function interaction(overrides: Record<string, unknown> = {}) {
|
|
return {
|
|
createdByAgentId: "agent-1",
|
|
createdByUserId: null,
|
|
sourceRunId: "run-1",
|
|
addresseeAgentId: null,
|
|
addresseeUserId: null,
|
|
effectiveResolverPolicy: "anyone",
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
describe("issue-thread interaction resolver audience", () => {
|
|
it("allows the creator and creating run under anyone", () => {
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction(),
|
|
})).toMatchObject({ allowed: true, effectiveResolverPolicy: "anyone", reason: "allow_anyone" });
|
|
});
|
|
|
|
it.each([
|
|
["creator agent", { createdByAgentId: "agent-1", sourceRunId: "run-other" }],
|
|
["creating run", { createdByAgentId: "agent-other", sourceRunId: "run-1" }],
|
|
])("excludes the %s only when not_creator is effective", (_name, creator) => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction({ ...creator, effectiveResolverPolicy: "not_creator" }),
|
|
});
|
|
expect(decision).toMatchObject({ allowed: false, code: "interaction_creator_excluded" });
|
|
});
|
|
|
|
it("enforces an explicit addressee", () => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction({ addresseeAgentId: "agent-2" }),
|
|
});
|
|
expect(decision).toMatchObject({ allowed: false, code: "interaction_addressee_mismatch" });
|
|
});
|
|
|
|
it("allows only the addressed user to resolve a user-addressed interaction", () => {
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "user", userId: "alice" },
|
|
interaction: interaction({ addresseeUserId: "alice", effectiveResolverPolicy: "human_only" }),
|
|
})).toMatchObject({ allowed: true, reason: "allow_addressee" });
|
|
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "user", userId: "bob" },
|
|
interaction: interaction({ addresseeUserId: "alice", effectiveResolverPolicy: "human_only" }),
|
|
})).toMatchObject({ allowed: false, code: "interaction_addressee_mismatch" });
|
|
});
|
|
|
|
it("requires run attribution for agents", () => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "agent", agentId: "agent-1", runId: null },
|
|
interaction: interaction(),
|
|
});
|
|
expect(decision).toMatchObject({
|
|
allowed: false,
|
|
status: 422,
|
|
code: "interaction_run_attribution_required",
|
|
});
|
|
});
|
|
|
|
it("allows a server-derived restriction to narrow but never widen the audience", () => {
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction({ createdByAgentId: "agent-other", sourceRunId: "run-other" }),
|
|
additionalRestriction: "human_only",
|
|
})).toMatchObject({ allowed: false, code: "interaction_human_only" });
|
|
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction({ effectiveResolverPolicy: "human_only" }),
|
|
additionalRestriction: "anyone",
|
|
})).toMatchObject({ allowed: false, code: "interaction_human_only" });
|
|
});
|
|
|
|
it("intersects human-only and creator-separation restrictions", () => {
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "user", userId: "user-1" },
|
|
interaction: interaction({
|
|
createdByAgentId: null,
|
|
createdByUserId: "user-1",
|
|
effectiveResolverPolicy: "not_creator",
|
|
}),
|
|
additionalRestriction: "human_only",
|
|
})).toMatchObject({ allowed: false, code: "interaction_creator_excluded" });
|
|
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "user", userId: "user-1" },
|
|
interaction: interaction({
|
|
createdByAgentId: null,
|
|
createdByUserId: "user-1",
|
|
effectiveResolverPolicy: "human_only",
|
|
}),
|
|
additionalRestriction: "not_creator",
|
|
})).toMatchObject({ allowed: false, code: "interaction_creator_excluded" });
|
|
});
|
|
|
|
it("excludes the review requester when a review card has null creator columns", () => {
|
|
const restriction = {
|
|
policy: "not_creator",
|
|
excludedActor: { type: "user", id: "review-requester" },
|
|
source: "issue_review",
|
|
} as const;
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "user", userId: "review-requester" },
|
|
interaction: interaction({
|
|
createdByAgentId: null,
|
|
createdByUserId: null,
|
|
sourceRunId: null,
|
|
}),
|
|
additionalRestriction: restriction,
|
|
})).toMatchObject({
|
|
allowed: false,
|
|
code: "review_policy_denied",
|
|
details: {
|
|
policy: "not_creator",
|
|
allowedActor: "writer_other_than_review_requester",
|
|
},
|
|
});
|
|
expect(evaluateIssueThreadInteractionResolverAudience({
|
|
actor: { type: "user", userId: "peer-reviewer" },
|
|
interaction: interaction({
|
|
createdByAgentId: null,
|
|
createdByUserId: null,
|
|
sourceRunId: null,
|
|
}),
|
|
additionalRestriction: restriction,
|
|
})).toMatchObject({ allowed: true, effectiveResolverPolicy: "not_creator" });
|
|
});
|
|
|
|
it("keeps governed actions independently human-only", () => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction(),
|
|
governedAction: true,
|
|
});
|
|
expect(decision).toMatchObject({ allowed: false, code: "interaction_governed_action_denied" });
|
|
});
|
|
|
|
it("preserves legacy inherited board_or_agents creator separation", () => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({
|
|
actor: agent,
|
|
interaction: interaction({
|
|
effectiveResolverPolicy: "board_or_agents",
|
|
resolverPolicyProvenance: "legacy_inherited_restriction",
|
|
}),
|
|
});
|
|
expect(decision).toMatchObject({ allowed: false, code: "interaction_creator_excluded" });
|
|
});
|
|
|
|
it("derives agent attention ownership from the same effective audience", () => {
|
|
expect(issueThreadInteractionAttentionAgentAllowed({
|
|
agentId: "agent-1",
|
|
interaction: interaction(),
|
|
})).toBe(true);
|
|
expect(issueThreadInteractionAttentionAgentAllowed({
|
|
agentId: "agent-1",
|
|
interaction: interaction({ effectiveResolverPolicy: "not_creator" }),
|
|
})).toBe(false);
|
|
expect(issueThreadInteractionAttentionAgentAllowed({
|
|
agentId: "agent-2",
|
|
interaction: interaction({ effectiveResolverPolicy: "human_only" }),
|
|
})).toBe(false);
|
|
});
|
|
|
|
describe("human-only card addressed to someone else", () => {
|
|
const card = interaction({ createdByAgentId: null, createdByUserId: "alex", addresseeUserId: "riley", effectiveResolverPolicy: "human_only" });
|
|
|
|
it("tells the agent only the addressee resolves it and not to send the asker to the card", () => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({ actor: { ...agent, relayedHumanUserId: "alex" }, interaction: card });
|
|
expect(decision).toMatchObject({ allowed: false, status: 403, code: "interaction_human_only", details: { addresseeUserId: "riley", askerIsNotAddressee: true } });
|
|
const message = decision.allowed ? "" : decision.message;
|
|
expect(message).toContain("This issue-thread interaction is human-only and addressed to");
|
|
expect(message).toContain("Only the addressee can accept or decline it");
|
|
expect(message).toContain("do not ask the person asking to accept or decline it");
|
|
});
|
|
|
|
it("names the addressee once the route knows their display name", () => {
|
|
const message = addressedHumanOnlyDenialMessage("Riley", true);
|
|
expect(message).toContain("addressed to Riley.");
|
|
expect(message).toContain("Only Riley can accept or decline it");
|
|
expect(message).toContain("the card waits on Riley");
|
|
expect(message).not.toContain("addresseeUserId");
|
|
});
|
|
|
|
it.each([
|
|
["no verified replying user", agent],
|
|
["the addressee as the replying user", { ...agent, relayedHumanUserId: "riley" }],
|
|
])("keeps the advice neutral with %s", (_name, actor) => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({ actor, interaction: card });
|
|
expect(decision).toMatchObject({ allowed: false, code: "interaction_human_only", details: { addresseeUserId: "riley", askerIsNotAddressee: false } });
|
|
const message = decision.allowed ? "" : decision.message;
|
|
expect(message).toContain("Only the addressee can accept or decline it");
|
|
expect(message).not.toContain("do not ask the person asking");
|
|
});
|
|
|
|
it("keeps the plain human-only denial when the card has no addressee", () => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({ actor: agent,
|
|
interaction: interaction({ effectiveResolverPolicy: "human_only" }) });
|
|
expect(decision).toMatchObject({ allowed: false, code: "interaction_human_only", message: "This issue-thread interaction is human-only" });
|
|
});
|
|
});
|
|
|
|
describe("relayed human decision", () => {
|
|
const relay = { ...agent, relayedHumanUserId: "user-1" };
|
|
const humanOnly = (overrides: Record<string, unknown> = {}) =>
|
|
interaction({ effectiveResolverPolicy: "human_only", effectiveResolverPolicySource: "requested",
|
|
resolverPolicyProvenance: "explicit", ...overrides });
|
|
|
|
it("allows a verified reply on a card whose creator requested human_only", () => {
|
|
expect(evaluateIssueThreadInteractionResolverAudience({ actor: relay, interaction: humanOnly() }))
|
|
.toMatchObject({ allowed: true, effectiveResolverPolicy: "human_only", reason: "allow_relayed_human" });
|
|
});
|
|
|
|
it.each([
|
|
["no relay", { actor: agent }],
|
|
["a company cap", { interaction: humanOnly({ effectiveResolverPolicySource: "company_cap" }) }],
|
|
["a governed-action source", { interaction: humanOnly({ effectiveResolverPolicySource: "governed_action" }) }],
|
|
["an unknown legacy source", { interaction: humanOnly({ effectiveResolverPolicySource: null }) }],
|
|
["a governed action", { governedAction: true }],
|
|
["an issue-review restriction", { additionalRestriction: "human_only" as const }],
|
|
["another addressed user", { interaction: humanOnly({ addresseeUserId: "user-2" }) }],
|
|
["a human-created card", { interaction: humanOnly({ createdByAgentId: null, createdByUserId: "user-1" }) }],
|
|
["another agent's card", { interaction: humanOnly({ createdByAgentId: "agent-2" }) }],
|
|
["an inherited company default", { interaction: humanOnly({ resolverPolicyProvenance: "inherited" }) }],
|
|
])("still denies %s", (_name, override) => {
|
|
const decision = evaluateIssueThreadInteractionResolverAudience({ actor: relay, interaction: humanOnly(), ...override });
|
|
expect(decision).toMatchObject({ allowed: false });
|
|
});
|
|
|
|
it("never widens a non-human-only card", () => {
|
|
expect(evaluateIssueThreadInteractionResolverAudience({ actor: relay,
|
|
interaction: interaction({ effectiveResolverPolicy: "not_creator" }) }))
|
|
.toMatchObject({ allowed: false, code: "interaction_creator_excluded" });
|
|
});
|
|
});
|
|
});
|