fix(chat): post the complete final reply, not a 4,000-character run-log preview (#19971)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People talk to agents in chat. The agent's final message becomes the
chat reply.
> - The runner cuts every completed agent message to a 4,000-character
run-log preview. The server then posts that preview as the reply.
> - A long answer, for example a list of 501 open tasks, reaches the
person cut off with "…[truncated]".
> - PR #19959 fixed only the case where the accepted summary starts with
the exact preview text. It missed runs where the final prose differs
from the summary.
> - This pull request keeps the complete final answer in the runner, and
makes the server never prefer a preview over a complete source.
> - The benefit is that the person receives the agent's complete reply.

## Linked Issues or Issue Description

Supersedes #19959.

**What happened**

A chat agent wrote a final answer of about 4,985 characters (all 501
tasks). The posted chat reply was a 4,000-character preview that ended
with "…[truncated]".

**Expected behavior**

The person receives the agent's complete final message. A preview of a
message is never posted when a complete text exists.

**Steps to reproduce**

1. Ask a chat agent (OpenCode or Codex) to list a large set of open
work, so that its final answer is longer than 4,000 characters.
2. Let the run finish.
3. Read the chat reply. It ends with "…[truncated]" at 4,000 characters.

Root cause:

-
`packages/paperclip-runner/runner/crates/runner-core/src/provider_events.rs`:
the codex/OpenCode facade projection of `item/completed` and the ACPX
`AssistantMessage` projection cut the final answer with
`bounded_text(text, MAX_TEXT_CHARS)` (4,000 characters).
- `durable/state.rs` `sanitize_value` caps every string again at 4,096
bytes before the event goes to the server.
- `server/src/services/heartbeat-run-summary.ts`
`resolveHeartbeatRunResponse` gives an explicit `final` agent message
top precedence, so it posts the preview.

## What Changed

- Runner (`provider_events.rs`): a completed `final_answer` agent
message keeps its complete redacted text, up to a new 32,000-character
`MAX_FINAL_AGENT_MESSAGE_CHARS` bound. Progress prose and in-flight
items keep the 4,000-character preview.
- Runner (`durable/state.rs`): `preserve_final_agent_message_text` keeps
the complete redacted text of a completed final agent message when it
fits in half of the transport frame. Otherwise the marked preview stays,
so a large message never fails the frame check.
- Server (`heartbeat-run-summary.ts`):
`readCompletedAssistantMessageCandidate` marks a stored preview as
`bounded`. The resolver never posts a bounded preview while a complete
source exists. It posts a marked adapter final response or the accepted
semantic summary instead
(`bounded_final_agent_message_replaced_by_semantic_summary`). It posts
the marked preview only when no complete text exists
(`bounded_final_agent_message_without_complete_source`). This also fixes
replies from runners that are already deployed.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-run-summary.test.ts`: 86 passed. Six new tests:
a complete final message longer than 4,000 characters is posted in full;
the old exact-prefix case posts the complete accepted answer; a preview
whose prose differs from the summary posts the complete summary (also
with chat precedence); a marked adapter final response wins over a
preview; the marked preview is posted only when nothing complete exists;
a stored preview candidate is marked `bounded`. Five of the six fail
against the `master` resolver.
- `cargo test -p paperclip-runner-core --lib -- final_answer
complete_final`: 3 passed. Both new runner tests fail when the fix is
reverted.
- `cargo test -p paperclip-runner-core --lib`: 581 passed, 3 failed. The
failures are `task_memory::tests::monitor_*` (OOM-kill monitor timing
tests). They do not touch the changed code.
- `cargo test -p paperclip-runner-core --test acpx_event_projection
--test acpx_rich_events --test local_integrity_boundary_golden --test
acpx_provider_events`: all passed. `cargo fmt --check` passes.
- Evidence from the failed eval run: the `item.completed` final
`agentMessage` text was exactly 4,000 characters (3,988 retained plus
the marker). The `run.result.proposed` summary was complete at 4,985
characters.

## Risks

- Larger durable events: a final answer can now be up to 32,000
characters (at most about 128 KB). The outbox keeps the preview when the
text exceeds half of the frame size, so the frame check does not fail.
- Redaction stays the same. The complete text goes through
`redact_sensitive_text_values`, which the preview also used.
- The server treats any final message that ends with "…[truncated]" as a
preview. If an agent writes that text itself, the server posts the
complete accepted summary instead. That summary is also the agent's own
complete answer.
- The runner part reaches users only after a runner release. The server
part fixes replies at once when an accepted summary exists.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-07 23:01:00 -06:00
1 parent 4aed3056db
commit fda58d8cc1
4 files changed
+330 -5

No files matched your search

@@ -591,6 +591,12 @@ impl DurableState {
&payload,
sanitize_value(&payload),
)?;
let sanitized_payload = preserve_final_agent_message_text(
event_type.as_str(),
&payload,
sanitized_payload,
config.max_frame_bytes,
);
// Restore authoritative tool arguments before checking protocol identities.
// Their contents are execution data, not a redacted diagnostic preview.
let sanitized_payload =
@@ -1968,6 +1974,54 @@ fn preserve_bounded_display_content(
Ok(sanitized)
}
/// A completed final answer is the reply the person receives. The generic
/// diagnostic preview cap would turn it into a cut-off copy, so keep its
/// complete redacted text when it fits comfortably inside one transport frame.
/// Otherwise the bounded preview remains, still marked as truncated.
fn preserve_final_agent_message_text(
event_type: &str,
original: &Value,
mut sanitized: Value,
max_frame_bytes: usize,
) -> Value {
if event_type != "item.completed"
|| !matches!(
original.get("kind").and_then(Value::as_str),
Some("agentMessage" | "assistant_message")
)
|| original.get("channel").and_then(Value::as_str) != Some("final")
{
return sanitized;
}
let Some(text) = original.get("text").and_then(Value::as_str) else {
return sanitized;
};
let mut candidate = sanitized.clone();
let Some(object) = candidate.as_object_mut() else {
return sanitized;
};
object.insert(
"text".to_owned(),
Value::String(redact_sensitive_text_values(text)),
);
// Size the serialized payload, not the raw text: JSON escaping can double
// the text, and the secure frame hex-encodes the encrypted envelope,
// doubling it again. Keep the bounded preview unless the complete payload,
// its envelope and the secure-frame wrapper all fit in one frame.
let Ok(serialized_bytes) = serde_json::to_vec(&candidate).map(|bytes| bytes.len()) else {
return sanitized;
};
let budget = (max_frame_bytes / 2).saturating_sub(FINAL_AGENT_MESSAGE_FRAME_HEADROOM_BYTES);
if serialized_bytes > budget {
return sanitized;
}
candidate
}
/// Plaintext reserved for the event envelope fields around a final answer
/// payload and for the secure-frame wrapper.
const FINAL_AGENT_MESSAGE_FRAME_HEADROOM_BYTES: usize = 4_096;
pub(crate) fn validate_semantic_tool_input(
operation_id: &str,
input: &Value,
@@ -4425,6 +4479,96 @@ mod tests {
}
}
#[test]
fn keeps_the_complete_final_answer_instead_of_a_display_preview() {
let config = config(PathBuf::from("unused"));
let mut state = DurableState::new(&config);
let answer = format!("token=secret-value {}", "Every open task. ".repeat(400));
assert!(answer.len() > 4_096);
for (channel, complete) in [("final", true), ("progress", false)] {
state
.enqueue_event(
&config,
"item.completed",
EventPriority::P1,
json!({"kind": "agentMessage", "channel": channel, "text": answer}),
)
.unwrap();
let text = state
.outbox
.last()
.unwrap()
.envelope
.pointer("/payload/payload/text")
.and_then(Value::as_str)
.unwrap()
.to_owned();
assert!(text.starts_with("token=[REDACTED] "), "{channel}");
assert!(!text.contains("secret-value"), "{channel}");
assert_eq!(!text.ends_with("…[truncated]"), complete, "{channel}");
if complete {
assert_eq!(text, answer.replace("secret-value", "[REDACTED]"));
}
}
// A frame too small for the complete answer keeps the marked preview.
let mut small_frame = config.clone();
small_frame.max_frame_bytes = 8_000;
let mut small = DurableState::new(&small_frame);
small
.enqueue_event(
&small_frame,
"item.completed",
EventPriority::P1,
json!({"kind": "agentMessage", "channel": "final", "text": answer}),
)
.unwrap();
assert!(small.outbox[0]
.envelope
.pointer("/payload/payload/text")
.and_then(Value::as_str)
.unwrap()
.ends_with("…[truncated]"));
}
#[test]
fn final_answer_frame_check_counts_json_escaping_and_hex_encoding() {
let mut frame = config(PathBuf::from("unused"));
frame.max_frame_bytes = 64 * 1024;
frame.max_outbox_bytes = 1024 * 1024;
let final_text = |answer: &str| {
let mut state = DurableState::new(&frame);
state
.enqueue_event(
&frame,
"item.completed",
EventPriority::P1,
json!({"kind": "agentMessage", "channel": "final", "text": answer}),
)
.unwrap();
let envelope = state.outbox[0].envelope.clone();
// The wire frame hex-encodes the encrypted envelope.
let envelope_bytes = serde_json::to_vec(&envelope).unwrap().len();
assert!(
envelope_bytes * 2 < frame.max_frame_bytes,
"{envelope_bytes}"
);
envelope
.pointer("/payload/payload/text")
.and_then(Value::as_str)
.unwrap()
.to_owned()
};
// 20,000 raw bytes, but about 40,000 once JSON-escaped: keep the preview.
let quotes = "\"".repeat(20_000);
assert!(final_text(&quotes).ends_with("…[truncated]"));
// The same length of plain prose fits and is kept complete.
let prose = "a".repeat(20_000);
assert_eq!(final_text(&prose), prose);
}
#[test]
fn diagnostic_redaction_still_bounds_retained_text() {
let output = redact_text(&format!("token=secret-value {}", "x".repeat(4_096)));
@@ -11,6 +11,10 @@ use crate::stable_identity::{
};
const MAX_TEXT_CHARS: usize = 4_000;
/// A completed final answer is the reply the person receives, not a run-log
/// preview. It keeps its complete (redacted) text up to this transport-safe
/// bound; the durable outbox applies its own frame check.
pub(crate) const MAX_FINAL_AGENT_MESSAGE_CHARS: usize = 32_000;
#[derive(Clone, Debug, PartialEq)]
pub struct NormalizedProviderEvent {
@@ -373,7 +377,7 @@ pub fn project_acpx_state_event(
"kind": "agentMessage",
"status": "completed",
"channel": "final",
"text": bounded_text(text, MAX_TEXT_CHARS),
"text": final_agent_message_text(text),
}),
)
}
@@ -616,6 +620,19 @@ fn plan_proposal_markdown(value: &str) -> String {
.collect()
}
/// Redacts a completed final answer without the run-log display preview cap.
fn final_agent_message_text(value: &str) -> String {
let redacted = redact_sensitive_text_values(value);
if redacted.chars().count() <= MAX_FINAL_AGENT_MESSAGE_CHARS {
return redacted;
}
let marker = "…[truncated]";
let retained_chars = MAX_FINAL_AGENT_MESSAGE_CHARS.saturating_sub(marker.chars().count());
let mut bounded: String = redacted.chars().take(retained_chars).collect();
bounded.push_str(marker);
bounded
}
fn bounded_text(value: &str, max_chars: usize) -> String {
let redacted = redact_text(value);
if redacted.chars().count() <= max_chars {
@@ -1458,6 +1475,10 @@ pub fn normalize_codex_notification_in(
"itemId": item_id,
"kind": bounded_text(item_type, 160),
"status": provider_status(string(provider_item.get("status")), completed),
// Only an explicit `final_answer` phase marks the reply. Legacy
// builds omit `phase` on every agentMessage, commentary included,
// so an empty phase stays a progress preview. (The plan-proposal
// branch above accepts an empty phase because it matches content.)
"channel": if item_type == "agentMessage" && provider_phase == "final_answer" {
"final"
} else if item_type == "agentMessage" {
@@ -1465,7 +1486,13 @@ pub fn normalize_codex_notification_in(
} else {
"detail"
},
"text": provider_item.get("text").and_then(Value::as_str).map(|value| bounded_text(value, MAX_TEXT_CHARS))
"text": provider_item.get("text").and_then(Value::as_str).map(|value| {
if completed && item_type == "agentMessage" && provider_phase == "final_answer" {
final_agent_message_text(value)
} else {
bounded_text(value, MAX_TEXT_CHARS)
}
})
.or_else(|| (item_type == "reasoning").then(|| reasoning_summary_text(provider_item)).flatten()),
});
// Preserve only actual terminal invocation identity in the compatibility
@@ -2932,6 +2959,50 @@ mod tests {
}
}
#[test]
fn keeps_a_complete_final_answer_longer_than_the_display_preview() {
let answer = format!(
"**Open work — 501 tasks**\n\n{}",
(1..=501)
.map(|n| format!("RUN-{n}"))
.collect::<Vec<_>>()
.join(", ")
);
assert!(answer.chars().count() > MAX_TEXT_CHARS);
let completed = normalize_codex_notification(
"item/completed",
&json!({"item": {"id": "msg-1", "type": "agentMessage", "phase": "final_answer", "text": answer}}),
);
assert_eq!(completed[0].payload["channel"], "final");
assert_eq!(completed[0].payload["text"].as_str(), Some(answer.as_str()));
// Progress prose and in-flight items stay display previews.
for (method, phase) in [
("item/completed", "commentary"),
("item/started", "final_answer"),
] {
let events = normalize_codex_notification(
method,
&json!({"item": {"id": "msg-2", "type": "agentMessage", "phase": phase, "text": answer}}),
);
let text = events[0].payload["text"].as_str().unwrap();
assert_eq!(text.chars().count(), MAX_TEXT_CHARS, "{method} {phase}");
assert!(text.ends_with("…[truncated]"));
}
// A legacy agentMessage without a phase is never treated as the final answer.
let legacy = normalize_codex_notification(
"item/completed",
&json!({"item": {"id": "msg-3", "type": "agentMessage", "text": answer}}),
);
assert_eq!(legacy[0].payload["channel"], "progress");
let text = legacy[0].payload["text"].as_str().unwrap();
assert_eq!(text.chars().count(), MAX_TEXT_CHARS);
assert!(text.ends_with("…[truncated]"));
let oversized = "x".repeat(MAX_FINAL_AGENT_MESSAGE_CHARS + 1);
let bounded = final_agent_message_text(&oversized);
assert_eq!(bounded.chars().count(), MAX_FINAL_AGENT_MESSAGE_CHARS);
assert!(bounded.ends_with("…[truncated]"));
}
#[test]
fn emits_complete_long_plan_proposals_as_rich_events() {
let body = format!("# Plan\n{}", "- step\n".repeat(1_000).trim_end());
@@ -387,6 +387,80 @@ describe("resolveHeartbeatRunResponse", () => {
},
};
describe("run-log previews of the final message", () => {
const complete = "**Open work — 501 tasks**\n\n### Morgan\n" +
Array.from({ length: 501 }, (_, i) => `RUN-${i + 1}`).join(", ");
const preview = complete.slice(0, 3_988) + "…[truncated]";
const accepted = (summary: string) => ({
nativeResult: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", summary },
});
it("posts a complete final message longer than the run-log preview bound", () => {
expect(complete.length).toBeGreaterThan(4_000);
expect(resolveHeartbeatRunResponse({
resultJson: accepted("Short summary."),
finalAgentMessage: { text: complete, sourceEventId: "event-1", channel: "final" },
})).toMatchObject({ text: complete, decision: { chosenSource: "final_agent_message" } });
});
it("posts the complete accepted answer when the preview is its exact prefix", () => {
for (const channel of ["final", "unknown"] as const) {
expect(resolveHeartbeatRunResponse({
resultJson: accepted(complete),
finalAgentMessage: { text: preview, sourceEventId: "event-1", channel },
})).toMatchObject({
text: complete,
decision: {
chosenSource: "semantic_result_summary",
reasonCodes: ["bounded_final_agent_message_replaced_by_semantic_summary"],
},
});
}
});
it("posts the complete accepted answer when the final prose differs from it", () => {
const different = "Here is every open task, grouped by owner.\n\n" + complete;
for (const preferFinalResponseOverExistingComment of [false, true]) {
expect(resolveHeartbeatRunResponse({
resultJson: accepted(different),
preferFinalResponseOverExistingComment,
finalAgentMessage: { text: preview, sourceEventId: "event-1", channel: "final" },
})).toMatchObject({ text: different, decision: { chosenSource: "semantic_result_summary" } });
}
});
it("never prefers a preview over a marked adapter final response", () => {
expect(resolveHeartbeatRunResponse({
resultJson: { ...accepted("summary"), finalResponse: { disposition: "final", text: complete } },
finalAgentMessage: { text: preview, sourceEventId: "event-1", channel: "final" },
})).toMatchObject({ text: complete, decision: { chosenSource: "adapter_final_response" } });
});
it("posts the marked preview only when no complete text exists", () => {
expect(resolveHeartbeatRunResponse({
resultJson: {},
finalAgentMessage: { text: preview, sourceEventId: "event-1", channel: "final" },
})).toMatchObject({
text: preview,
decision: {
chosenSource: "final_agent_message",
reasonCodes: ["bounded_final_agent_message_without_complete_source"],
},
});
});
it("marks a stored preview candidate as bounded", () => {
expect(readCompletedAssistantMessageCandidate({
seq: 3,
prpEvent: { sourceEventId: "event-3", payload: { kind: "agentMessage", channel: "final", text: preview } },
})).toMatchObject({ text: preview, bounded: true });
expect(readCompletedAssistantMessageCandidate({
seq: 4,
prpEvent: { sourceEventId: "event-4", payload: { kind: "agentMessage", channel: "final", text: complete } },
})).not.toHaveProperty("bounded");
});
});
it("applies comment, explicit provider, adapter, terminal fallback, and semantic precedence", () => {
const resultWithAdapterFinal = {
...resultJson,
+39 -3
View File
@@ -65,6 +65,16 @@ function splitProposedPlanBlock(value: string): { plan: string | null; rest: str
};
}
/**
* Marker the runner appends when it cuts provider text to a run-log display
* preview. Text ending with it is an incomplete copy, never a final reply.
*/
export const RUN_EVENT_TEXT_BOUND_MARKER = "…[truncated]";
export function isBoundedRunEventText(value: unknown): boolean {
return typeof value === "string" && value.endsWith(RUN_EVENT_TEXT_BOUND_MARKER);
}
/**
* The posted reply leaves a proposal to its Plan document. A stored message
* preview can be cut inside the block; its cut-off remainder is hidden too.
@@ -72,7 +82,7 @@ function splitProposedPlanBlock(value: string): { plan: string | null; rest: str
function withoutProposedPlanBlock(value: string): string {
const { plan, rest, unclosedStart } = splitProposedPlanBlock(value);
if (plan !== null) return rest;
if (!value.endsWith("…[truncated]") || unclosedStart === null) return value;
if (!isBoundedRunEventText(value) || unclosedStart === null) return value;
return value.slice(0, unclosedStart).trim();
}
@@ -194,6 +204,8 @@ export interface CompletedFinalAgentMessageCandidate {
text: string;
sourceEventId: string | null;
channel: "final" | "unknown";
/** The run log kept only a display preview of this message. */
bounded?: boolean;
}
/**
@@ -323,6 +335,7 @@ export function readCompletedAssistantMessageCandidate(input: {
text,
sourceEventId: readCommentText(prpEvent.sourceEventId),
channel: payload.channel,
...(isBoundedRunEventText(payload.text) ? { bounded: true } : {}),
};
}
@@ -489,6 +502,7 @@ export function resolveHeartbeatRunResponse(input: {
sourceEventId: string | null;
channel: "final" | "unknown";
reasonCode?: string;
bounded?: boolean;
} | null;
}): ResolvedHeartbeatRunResponse {
const resultJson = record(input.resultJson);
@@ -510,7 +524,15 @@ export function resolveHeartbeatRunResponse(input: {
}),
};
}
const finalAgentText = readCommentText(input.finalAgentMessage?.text);
const rawFinalAgentText = readCommentText(input.finalAgentMessage?.text);
// A run-log preview is an incomplete copy of the agent's message. It never
// outranks a complete source; it is posted only when no complete text exists.
const boundedFinalAgentText =
rawFinalAgentText &&
(input.finalAgentMessage?.bounded === true || isBoundedRunEventText(rawFinalAgentText))
? rawFinalAgentText
: null;
const finalAgentText = boundedFinalAgentText ? null : rawFinalAgentText;
const explicitProviderFinal = input.finalAgentMessage?.channel === "final";
const compatibleTerminalAssistant =
input.finalAgentMessage?.channel === "unknown";
@@ -568,7 +590,21 @@ export function resolveHeartbeatRunResponse(input: {
text: semanticSummary,
decision: decision("semantic_result_summary", {
commentAction: "create",
reasonCodes: ["accepted_semantic_result_summary"],
reasonCodes: boundedFinalAgentText
? ["bounded_final_agent_message_replaced_by_semantic_summary"]
: ["accepted_semantic_result_summary"],
}),
} satisfies ResolvedHeartbeatRunResponse;
}
if (boundedFinalAgentText && !isStructuredSemanticResultText(boundedFinalAgentText)) {
// No complete source exists. The visible marker says the text was cut.
return {
text: boundedFinalAgentText,
decision: decision("final_agent_message", {
sourceEventId: input.finalAgentMessage?.sourceEventId,
commentAction: "create",
reasonCodes: ["bounded_final_agent_message_without_complete_source"],
}),
} satisfies ResolvedHeartbeatRunResponse;
}