mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 08:28:48 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat gives each person a persistent conversation with each
agent.
> - Tasks and routines produce results outside that conversation.
> - Those runs need a safe way to bring a requested result back to the
person.
> - The server must select the agent's own chat and check the source
task's access rules.
> - This pull request adds proactive chat messages and unread
indicators.
> - People can see the result, open its source task, and reply in the
same chat.
## Linked Issues or Issue Description
**Subsystem affected**
Server, native runner tools, shared contracts, database, and UI.
**Problem or motivation**
An agent can report progress on a task or comment on an authorized task.
It cannot send a routine result to its own standing chat with the
responsible person. A morning review routine therefore cannot leave a
visible digest in that person's chat.
**Proposed solution**
Add `send_chat_message({body, idempotencyKey})` and `POST
/api/agents/me/chat/messages`. The server selects the company, agent,
and responsible person from the active run. It saves an attributed
comment and audit record in one transaction. The chat shows a
source-task link and per-person unread state.
**Alternatives considered**
A destination parameter on a generic comment tool would expose
unnecessary routing authority. A new delivery worker would add another
durable delivery path. This change uses a dedicated tool and the
existing comment and live-event paths.
**Roadmap alignment**
This extends the experimental Agent Chat capability in `ROADMAP.md`. The
existing `enableAgentChat` setting controls it. Related public work:
Refs #13282, Refs #13283, and Refs #14408. Searches for proactive chat
and unread chat found no duplicate PR.
## What Changed
- Add a focused messaging service shared by the native tool and HTTP
route. It checks current run ownership, execution mode, recipient
membership, task access, and task privacy.
- Reuse the canonical company/agent/person conversation. Save the
comment and audit record together. Keep task completion and chat
execution separate from delivery.
- Add source-task provenance and source-scoped retry keys. Identical
retries return the saved receipt. Changed content returns a conflict.
- Enforce the conversation's fixed agent at the shared comment-write
boundary. Keep normal reply cancellation and session checks.
- Add chat unread state, accessible agent dots, aggregate Chat
indicators, and visible-message read cutoffs. Refresh through existing
live events.
- Add the request-key constraint, native tool contracts, documentation,
capability maps, stories, and regression coverage.
## Verification
- Local delivery/security proof: all 38 proactive-message cases pass.
Eight credential-scope regressions and four quota regressions fail
against the prior source. The repaired delivery and canonical
cross-issue cap suites pass all 49 cases; all 31 OpenAPI tests, server
typecheck, and capability-map checks pass. The shared per-run cap covers
native and HTTP sends, mixed ordinary mutations, concurrent last-slot
contention, and retry/recovery without double charging.
- Credential tests preserve the authenticated person and authorizer;
reject bridge/test credentials and persisted skill-test runs after
task-mode changes; and recheck API-key revocation, scope, and
responsible person after destination locking. The earlier
delivery/native-authority suites passed all 74 cases before the four
quota additions.
- Four PostgreSQL contention cases cover native and HTTP sends plus
retries during a native chat mutation. All four deadlock against the
earlier source and pass with destination-before-source/run locking.
Recovery and human-reply delivery enqueue only the human comment; a
proactive digest never wakes the chat agent.
- Passed 108 companion conversation, native-authority, and OpenAPI
tests; 112 runner catalog/workflow-report tests; 47 completion-delivery
tests; and 25 project-handoff tests. Focused runner tests cover native
tool availability without generic API tools.
- All 327 affected UI tests, UI typecheck, token gates, and catalog
checks pass. The read hook retries at most three times, respects server
retry delay, coalesces only seen cutoffs, recovers on reconnect, and
cancels on scope changes.
- Real browser journeys with a deterministic process agent verify
routine-to-chat delivery, first-chat creation, unread state across
reload, the source link, secondary chat navigation, and a follow-up
reply referring to the digest. This does not qualify live GitHub
retrieval or model output quality.
- The Storybook showcase uses production components with deterministic
network fixtures. Rendered interactions pass for the landing, ordinary
read success, transient read recovery, and persistent failure after
bounded retries. Both Chat navigation variants update from two unread
chats to one while only Kodi's dot clears. Mobile and light variants
were reviewed. The existing Coordinator Recents label contrast finding
is unchanged; no Linux visual baseline archive exists.
- Current master's independent sandbox migration retains 0418.
Regenerated the feature migration as 0419 from the combined schema and
retained exactly the newest five snapshots; numbering/safety, replay,
and schema-drift checks pass.
- Workspace typecheck and build passed before PR preparation; canonical
server/UI typechecks passed again on the combined tree. Migration
`0419_slippery_impossible_man.sql` passes safety and schema drift checks
on disposable PostgreSQL, including interrupted index repair, concurrent
migrators, one-time owner read-state backfill, preserved receipts, and
two SQL replays. A default five-second local snapshot comparison timed
out; a diagnostic 30-second run passed without drift.
- Earlier failures are retained in the record: full local `pnpm
test:run` and broad shards encountered embedded-PostgreSQL startup,
timing, and provider-recovery failures. Thirteen selected
external-channel cases pass in isolation on the original and changed
sources; an unordered comment-row assertion now explicitly orders both
reads. The earlier route registry, native tool count, runner catalog,
and accessible Chat badge-name failures were fixed and passed full
remote CI on `920b3bb7db`.
- Repeated managed-skill server8 failures came from CI's newer synthetic
merge: its Cloud guard correctly rejected the old local-disk fixture.
Master independently repaired the fixture in #18385; this branch keeps
that repair unchanged. All 57 managed-skill and Cloud prerequisite cases
pass on the combined tree, and that fixture has no remaining PR diff.
- Current reviewed head `dc16f314739a6d977bed21cb1446d0e80a541578`:
Greptile 5/5 with no findings, Superagent security assessment passed,
advisory Paperclip Review 5/5 with no findings, and zero open review
threads. All 55 current-head check runs pass, with five expected skips
and the advisory review neutral at 5/5; Snyk also passes. No checks are
pending or failing. Protected squash auto-merge is enabled and is
waiting for required code-owner approval. A normal exact-head merge was
refused by the base-branch policy; no admin bypass was used. The
previous head's Superagent quota finding is fixed and covered above.
## Risks
- Chat remains company-visible. Delivery rejects a source task with
private-task, project, or ancestor restrictions.
- Read state uses a comment timestamp cutoff. The server retains
database precision and never moves a read receipt backward.
- The new unique constraint changes comment query planning. An existing
regression now uses an explicit row order. The migration builds the
unique index concurrently, then takes a brief table lock to attach it.
It uses the existing top-level migration and interrupted-index recovery
path.
- The upgrade backfills missing chat-owner read-state rows at the latest
existing agent comment. Existing read receipts remain intact. New chats
can still be unread before the human participates.
- Quota observation commits atomically with delivery. A rejected send
rolls back its attempted denial audit row and any first-chat creation;
the canonical warning remains in server logs, and previously spent quota
remains enforced. Identical/conflicting retries and receipt recovery
spend no extra slot.
- This version delivers in-app only. It adds no external-channel, push,
or email delivery.
- Local full-suite acceptance remains limited by the failures recorded
above. The feature stays behind `enableAgentChat`.
## Model Used
OpenAI Codex. PR preparation and CI work used `gpt-6.1-sol` with tool
use and code execution. The earlier implementation also used Codex, but
its exact model ID was not recorded in the visible session. The
context-window size was not provided.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass; the broad local
failures are recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
194 lines
14 KiB
TypeScript
194 lines
14 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import { and, eq, isNull, sql } from "drizzle-orm";
|
|
import { agentApiKeys, agents, companies, companyMemberships, heartbeatRuns, issueComments, issues, nativeRunResults, type Db } from "@paperclipai/db";
|
|
import { agentApiKeyScopeSchema, isUuidLike, sendChatMessageSchema, type AgentApiKeyScope } from "@paperclipai/shared";
|
|
import { badRequest, conflict, forbidden, notFound, tooManyRequests } from "../errors.js";
|
|
import { agentRunWritesRevoked, assertAgentRunWriteAllowed } from "../agent-run-cancellation.js";
|
|
import { authorizationService, canActorReadIssuePrivacy, type AuthorizationActor } from "./authorization.js";
|
|
import { persistActivity, publishActivity, type ActivityPublication } from "./activity-log.js";
|
|
import { instanceSettingsService } from "./instance-settings.js";
|
|
import { issueService } from "./issues.js";
|
|
import { issueTreeControlService } from "./issue-tree-control.js";
|
|
import { resolveActorSourceTrustForIssue } from "./source-trust.js";
|
|
import { resolveTaskWatchdogMutationScope } from "./task-watchdog-scope.js";
|
|
import { readNativeReviewAssignmentContext } from "./native-runtime/native-review-participant.js";
|
|
import { nativeRunKeyScope } from "./native-runtime/native-skill-test-scope.js";
|
|
import { crossIssueInfluenceLimitError, observeCrossIssueInfluence } from "./cross-issue-influence-limit.js";
|
|
|
|
type Actor = { companyId: string; agentId: string; runId: string; agentApiKeyId?: string | null;
|
|
keyScope?: AgentApiKeyScope | null; onBehalfOfUserId?: string | null;
|
|
authorizingUserId?: string | null; source?: AuthorizationActor["source"] };
|
|
export interface ChatMessageReceipt {
|
|
commentId: string;
|
|
conversationId: string;
|
|
chatUrl: string;
|
|
sourceTaskId: string;
|
|
replayed: boolean;
|
|
}
|
|
|
|
function runTask(run: typeof heartbeatRuns.$inferSelect) {
|
|
return run.nativeIssueId ?? run.issueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId;
|
|
}
|
|
|
|
export async function sendAgentChatMessage(
|
|
db: Db,
|
|
actor: Actor,
|
|
value: unknown,
|
|
options: { authorizeNative?: (tx: Db) => Promise<unknown>; receiptOnly?: boolean } = {},
|
|
): Promise<ChatMessageReceipt | null> {
|
|
const parsed = sendChatMessageSchema.safeParse(value);
|
|
if (!parsed.success) throw badRequest("Chat messages require body and idempotencyKey only", parsed.error.issues);
|
|
if (![actor.companyId, actor.agentId, actor.runId].every(isUuidLike)) throw forbidden("An authenticated task run is required");
|
|
const credentialScope = agentApiKeyScopeSchema.safeParse(actor.keyScope ?? { kind: "standard" });
|
|
if (!credentialScope.success) throw forbidden("The agent credential scope is invalid");
|
|
if (credentialScope.data.kind !== "standard") throw forbidden("Scoped test and task bridge credentials cannot publish agent chat messages");
|
|
const input = parsed.data;
|
|
const contentHash = createHash("sha256").update(input.body).digest("hex");
|
|
const publications: ActivityPublication[] = [];
|
|
const result = await db.transaction(async transaction => {
|
|
const tx = transaction as unknown as Db;
|
|
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`issue-privacy-tree:${actor.companyId}`}, 0))`);
|
|
const runWhere = and(eq(heartbeatRuns.id, actor.runId), eq(heartbeatRuns.companyId, actor.companyId), eq(heartbeatRuns.agentId, actor.agentId));
|
|
const [initial] = await tx.select().from(heartbeatRuns).where(runWhere);
|
|
const sourceId = initial && runTask(initial);
|
|
if (typeof sourceId !== "string" || !isUuidLike(sourceId)) throw forbidden("Chat delivery requires a task-bound run");
|
|
const sourceQuery = () => tx.select().from(issues).where(and(eq(issues.id, sourceId), eq(issues.companyId, actor.companyId)));
|
|
const [initialSource] = await sourceQuery();
|
|
if (!initialSource) throw forbidden("This run no longer owns the source task");
|
|
let source = initialSource;
|
|
const contextQuery = () => tx.select({ run: heartbeatRuns, agent: agents }).from(heartbeatRuns)
|
|
.innerJoin(agents, and(eq(agents.id, heartbeatRuns.agentId), eq(agents.companyId, heartbeatRuns.companyId)))
|
|
.where(runWhere);
|
|
const validateContext = (context: { run: typeof heartbeatRuns.$inferSelect; agent: typeof agents.$inferSelect } | undefined) => {
|
|
if (!context || context.run.status !== "running" || agentRunWritesRevoked(context.run)
|
|
|| runTask(context.run) !== sourceId || source.assigneeAgentId !== actor.agentId
|
|
|| source.executionRunId !== actor.runId || (source.checkoutRunId && source.checkoutRunId !== actor.runId)
|
|
|| ["paused", "terminated", "pending_approval", "error"].includes(context.agent.status)) {
|
|
throw forbidden("This run no longer owns the source task");
|
|
}
|
|
if (source.conversationAgentId) throw forbidden("Reply normally in this conversation; send_chat_message is for execution tasks");
|
|
if (nativeRunKeyScope({ contextSnapshot: context.run.contextSnapshot, issueId: source.id, workMode: source.workMode }).kind === "skill_test") {
|
|
throw forbidden("Skill-test runs cannot publish agent chat messages");
|
|
}
|
|
if (source.workMode !== "standard" || ["done", "cancelled"].includes(source.status)
|
|
|| readNativeReviewAssignmentContext(context.run.contextSnapshot)) {
|
|
throw forbidden("Chat delivery requires an ordinary task in standard execution mode");
|
|
}
|
|
return context;
|
|
};
|
|
const [initialContext] = await contextQuery();
|
|
const context = validateContext(initialContext);
|
|
if (!(await instanceSettingsService(tx).getExperimental()).enableAgentChat) throw notFound("Agent Chat is disabled");
|
|
const userId = context.run.responsibleUserId;
|
|
if (!userId) throw forbidden("This task has no responsible human to receive the message");
|
|
if (actor.onBehalfOfUserId !== undefined && actor.onBehalfOfUserId !== userId) throw forbidden("The authenticated responsible human no longer matches this run");
|
|
const assertCurrentKey = async (lock: boolean) => {
|
|
if (!actor.agentApiKeyId) return;
|
|
const query = tx.select().from(agentApiKeys).where(and(eq(agentApiKeys.id, actor.agentApiKeyId),
|
|
eq(agentApiKeys.companyId, actor.companyId), eq(agentApiKeys.agentId, actor.agentId), isNull(agentApiKeys.revokedAt)));
|
|
const [key] = lock ? await query.for("share") : await query;
|
|
const currentScope = key && agentApiKeyScopeSchema.safeParse(key.scopeConfig ?? { kind: "standard" });
|
|
if (!key || !currentScope?.success || JSON.stringify(currentScope.data) !== JSON.stringify(credentialScope.data)
|
|
|| (key.responsibleUserId?.trim() || null) !== (actor.onBehalfOfUserId ?? userId)) {
|
|
throw forbidden("The agent key was revoked or its authority changed");
|
|
}
|
|
};
|
|
await assertCurrentKey(false);
|
|
const [membership] = await tx.select({ id: companyMemberships.id }).from(companyMemberships).where(and(
|
|
eq(companyMemberships.companyId, actor.companyId), eq(companyMemberships.principalType, "user"),
|
|
eq(companyMemberships.principalId, userId), eq(companyMemberships.status, "active"),
|
|
)).for("share");
|
|
if (!membership) throw forbidden("The responsible human is no longer an active company member");
|
|
const authActor: AuthorizationActor = { type: "agent", ...actor, keyId: actor.agentApiKeyId, keyScope: credentialScope.data,
|
|
source: actor.source ?? "agent_jwt", onBehalfOfUserId: actor.onBehalfOfUserId ?? userId };
|
|
const authorize = async (issueId: string, action: "issue:read" | "issue:comment") => {
|
|
const decision = await authorizationService(tx).decide({ actor: authActor, action,
|
|
resource: { type: "issue", companyId: actor.companyId, issueId } });
|
|
if (!decision.allowed) throw forbidden(decision.explanation);
|
|
};
|
|
await authorize(source.id, "issue:read");
|
|
if (!(await canActorReadIssuePrivacy(tx, { type: "none" }, source))) {
|
|
throw forbidden("Restricted task content cannot be sent to a company-visible chat");
|
|
}
|
|
const svc = issueService(tx);
|
|
let conversation = await svc.getConversation(actor.companyId, actor.agentId, userId);
|
|
if (!conversation && !options.receiptOnly) {
|
|
conversation = await svc.create(actor.companyId, {
|
|
title: `Chat with ${context.agent.name}`, assigneeAgentId: actor.agentId,
|
|
conversationAgentId: actor.agentId, conversationUserId: userId,
|
|
conversationState: "waiting", status: "in_review", createdByAgentId: actor.agentId,
|
|
responsibleUserId: userId, trustExplicitResponsibleUserId: true,
|
|
});
|
|
const opened = await persistActivity(tx, { companyId: actor.companyId, actorType: "agent", actorId: actor.agentId,
|
|
agentId: actor.agentId, runId: actor.runId, action: "issue.conversation_opened", entityType: "issue", entityId: conversation.id,
|
|
details: { agentId: actor.agentId, conversationUserId: userId } });
|
|
publications.push(opened.publication);
|
|
}
|
|
if (conversation) await tx.select({ id: issues.id }).from(issues).where(and(
|
|
eq(issues.id, conversation.id), eq(issues.companyId, actor.companyId),
|
|
)).for("update");
|
|
const [lockedSource] = await sourceQuery().for("update");
|
|
if (!lockedSource) throw forbidden("This run no longer owns the source task");
|
|
source = lockedSource;
|
|
if (options.authorizeNative) await options.authorizeNative(tx);
|
|
const [lockedContext] = await contextQuery().for("update");
|
|
const current = validateContext(lockedContext);
|
|
if (current.run.responsibleUserId !== userId) throw forbidden("The responsible human changed before this message was saved");
|
|
await assertCurrentKey(true);
|
|
if ((await resolveTaskWatchdogMutationScope(tx, { ...actor, type: "agent" })).kind !== "none") {
|
|
throw forbidden("Chat delivery requires an ordinary task in standard execution mode");
|
|
}
|
|
await assertAgentRunWriteAllowed(tx, actor.companyId, actor);
|
|
if (await issueTreeControlService(tx).getActivePauseHoldGate(actor.companyId, source.id)) throw forbidden("This task is paused");
|
|
const [finished] = await tx.select({ id: nativeRunResults.id }).from(nativeRunResults).where(and(
|
|
eq(nativeRunResults.companyId, actor.companyId), eq(nativeRunResults.runId, actor.runId), eq(nativeRunResults.schemaStatus, "accepted"),
|
|
)).limit(1);
|
|
if (finished) throw forbidden("Send the message before finishing the run");
|
|
if (!(await instanceSettingsService(tx).getExperimental()).enableAgentChat) throw notFound("Agent Chat is disabled");
|
|
await authorize(source.id, "issue:read");
|
|
if (!conversation) return null;
|
|
await authorize(conversation.id, "issue:comment");
|
|
const recipientDecision = await authorizationService(tx).decide({ actor: { type: "board", userId, source: "session", ignoreInstanceAdmin: true },
|
|
action: "issue:read", resource: { type: "issue", companyId: actor.companyId, issueId: conversation.id } });
|
|
if (!recipientDecision.allowed) throw forbidden("The responsible human cannot access this chat");
|
|
const requestId = `chat:${source.id}:${createHash("sha256").update(input.idempotencyKey).digest("hex")}`;
|
|
const [prior] = await tx.select().from(issueComments).where(and(eq(issueComments.companyId, actor.companyId),
|
|
eq(issueComments.issueId, conversation.id), eq(issueComments.authorAgentId, actor.agentId), eq(issueComments.clientRequestId, requestId)));
|
|
const [company] = await tx.select({ prefix: companies.issuePrefix }).from(companies).where(eq(companies.id, actor.companyId));
|
|
const receipt = (commentId: string, replayed: boolean): ChatMessageReceipt => ({ commentId, conversationId: conversation!.id,
|
|
chatUrl: `/${company.prefix}/chats/${actor.agentId}`, sourceTaskId: source.id, replayed });
|
|
if (prior) {
|
|
if (prior.metadata?.chatMessage?.contentHash !== contentHash) {
|
|
if (options.receiptOnly) return null;
|
|
throw conflict("Chat message retry key was already used with different content");
|
|
}
|
|
return receipt(prior.id, true);
|
|
}
|
|
if (options.receiptOnly) return null;
|
|
const influence = await observeCrossIssueInfluence(tx, { companyId: actor.companyId, agentId: actor.agentId,
|
|
runId: actor.runId, responsibleUserId: userId, targetIssueId: conversation.id,
|
|
targetIssueIdentifier: conversation.identifier, kind: "comment" });
|
|
if (influence && !influence.allowed) {
|
|
const denial = crossIssueInfluenceLimitError(influence, { actorLabel: current.agent.name,
|
|
assigneeLabel: current.agent.name, issueIdentifier: conversation.identifier });
|
|
throw tooManyRequests(denial.error, denial.details);
|
|
}
|
|
const sourceTrust = await resolveActorSourceTrustForIssue({ db: tx, issue: source,
|
|
actor: { actorType: "agent", actorId: actor.agentId, agentId: actor.agentId, runId: actor.runId } });
|
|
const comment = await svc.addComment(conversation.id, input.body, { agentId: actor.agentId, runId: actor.runId }, {
|
|
clientRequestId: requestId, sourceTrust, proactiveChatMessage: { sourceTaskId: source.id, userId },
|
|
metadata: { version: 1, chatMessage: { sourceTaskId: source.id, sourceTaskIdentifier: source.identifier!, sourceRunId: actor.runId, contentHash },
|
|
sections: [{ rows: [{ type: "issue_link", issueId: source.id, identifier: source.identifier, label: "Source task" }] }] },
|
|
}, tx);
|
|
const activity = await persistActivity(tx, { companyId: actor.companyId, actorType: "agent", actorId: actor.agentId,
|
|
agentId: actor.agentId, runId: actor.runId, agentApiKeyId: actor.agentApiKeyId,
|
|
action: "issue.comment_added", entityType: "issue", entityId: conversation.id,
|
|
details: { commentId: comment.id, identifier: conversation.identifier, conversationUserId: userId,
|
|
conversationAgentId: actor.agentId, sourceTaskId: source.id } });
|
|
publications.push(activity.publication);
|
|
return receipt(comment.id, false);
|
|
});
|
|
for (const publication of publications) publishActivity(publication);
|
|
return result;
|
|
}
|