mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-11 23:36:51 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The CLI and server both update Paperclip values in `.env` files > - The server preserved operator content, but the CLI rebuilt the complete file > - A CLI rerun could remove comments, custom values, ordering, and newline style > - Both paths need one editor with one value encoding and duplicate key policy > - The final integration also needs one regression test across the related setup and sync safety mechanisms > - This pull request moves the editor to the shared package and adds cross-cutting rerun-survival coverage > - The benefit is safe setup and worktree repair reruns that preserve operator edits ## Linked Issues or Issue Description **What happened?** The CLI rebuilt the complete `.env` file when it wrote a managed Paperclip value. This action removed comments, blank lines, custom keys, original quoting, and the original newline style. **Expected behavior** Paperclip must update only the managed assignments. It must preserve all unrelated bytes. It must skip the file replacement when all managed values are current. **Steps to reproduce** 1. Add comments, custom keys, quoted values, and CRLF newlines to the Paperclip `.env` file. 2. Run a CLI path that calls the agent JWT secret setup. 3. Observe that the old writer replaces the complete file. **Paperclip version or commit** The problem exists on `master` before this pull request. Related public context: Refs #437. ## What Changed - Add one shared line-preserving `.env` editor for the CLI and server. - Define minimal and JSON value encodings in the shared helper. - Update every stale duplicate of a managed key and preserve current duplicate encodings. - Preserve comments, ordering, blank lines, unknown keys, export prefixes, trailing comments, and newline style. - Write changed files through a same-directory temporary file and atomic rename. - Limit CLI updates to non-empty `PAPERCLIP_*` entries. - Skip the write when all managed values are current. - Add shared, CLI, and server regression coverage. - Refresh the branch after the related config, sandbox, and skill safety changes landed. - Add a cross-cutting integration test for config, env-file, managed-sandbox, and managed-instructions rerun survival. ## Verification - `pnpm exec vitest run packages/shared/src/env-file.test.ts packages/shared/src/config-schema.test.ts cli/src/__tests__/agent-jwt-env.test.ts cli/src/__tests__/config-store.test.ts server/src/__tests__/config-file.test.ts server/src/__tests__/worktree-config.test.ts` passes 39 tests. - `pnpm exec vitest run server/src/__tests__/rerun-survival.integration.test.ts` passes 4 tests. - `pnpm -r typecheck` passes on the previous head. GitHub CI reruns it on the refreshed head. - The previous head passed the complete general, serialized, workspace, and E2E matrix. GitHub CI reruns that matrix on the refreshed head. - `pnpm build` passes on the previous head. GitHub CI reruns it on the refreshed head. ## Risks - Low risk. The production change only affects managed `.env` assignments. - Existing managed assignments can keep their original quoting when their decoded values are current. - Changed CLI values keep the prior minimal encoding policy. Changed server values keep the prior JSON encoding policy. - Duplicate managed assignments now follow one explicit rule: Paperclip updates each stale occurrence. - The master refresh had one import-block conflict. The resolution keeps both the config merge imports and the env-file imports. - The added integration file is test-only. It has no database, API, or UI contract effect. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex from the GPT-5 family produced this change with reasoning, tool use, and code execution. The runtime did not expose the exact model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
182 lines
5.5 KiB
TypeScript
182 lines
5.5 KiB
TypeScript
import fs from "node:fs";
|
|
import path from "node:path";
|
|
import { randomUUID } from "node:crypto";
|
|
|
|
export type EnvValueEncoding = "minimal" | "json";
|
|
|
|
export type UpdateEnvFileContentsOptions = {
|
|
valueEncoding?: EnvValueEncoding;
|
|
};
|
|
|
|
type EnvLine = {
|
|
contents: string;
|
|
ending: string;
|
|
};
|
|
|
|
const ENV_ASSIGNMENT_PATTERN = /^(\s*(?:export\s+)?)([A-Za-z_][A-Za-z0-9_]*)(\s*=\s*)(.*)$/;
|
|
const MINIMAL_ENV_VALUE_PATTERN = /^[A-Za-z0-9_./:@-]+$/;
|
|
|
|
export function encodeEnvValue(value: string, encoding: EnvValueEncoding = "json"): string {
|
|
if (encoding === "minimal" && MINIMAL_ENV_VALUE_PATTERN.test(value)) {
|
|
return value;
|
|
}
|
|
return JSON.stringify(value);
|
|
}
|
|
|
|
function trailingEnvCommentStart(rawValue: string): number | null {
|
|
let quote: "\"" | "'" | null = null;
|
|
let escaped = false;
|
|
|
|
for (let index = 0; index < rawValue.length; index += 1) {
|
|
const character = rawValue[index];
|
|
if (escaped) {
|
|
escaped = false;
|
|
continue;
|
|
}
|
|
if (quote === "\"" && character === "\\") {
|
|
escaped = true;
|
|
continue;
|
|
}
|
|
if (quote !== null) {
|
|
if (character === quote) quote = null;
|
|
continue;
|
|
}
|
|
if (character === "\"" || character === "'") {
|
|
quote = character;
|
|
continue;
|
|
}
|
|
// dotenv treats every unquoted # as the start of a comment, even when it
|
|
// immediately follows the assignment separator or the value.
|
|
if (character !== "#") continue;
|
|
|
|
let commentStart = index;
|
|
while (commentStart > 0 && /\s/.test(rawValue[commentStart - 1] ?? "")) {
|
|
commentStart -= 1;
|
|
}
|
|
return commentStart;
|
|
}
|
|
|
|
return null;
|
|
}
|
|
|
|
function parseComparableEnvValue(rawValue: string): string | null {
|
|
const commentStart = trailingEnvCommentStart(rawValue);
|
|
const encodedValue = rawValue.slice(0, commentStart ?? rawValue.length).trim();
|
|
|
|
if (encodedValue === "") return "";
|
|
if (encodedValue.startsWith("\"") && encodedValue.endsWith("\"")) {
|
|
try {
|
|
const parsed = JSON.parse(encodedValue);
|
|
return typeof parsed === "string" ? parsed : null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
if (encodedValue.startsWith("'") && encodedValue.endsWith("'")) {
|
|
return encodedValue.slice(1, -1);
|
|
}
|
|
return encodedValue;
|
|
}
|
|
|
|
function splitEnvLines(contents: string): EnvLine[] {
|
|
const lines: EnvLine[] = [];
|
|
const newlinePattern = /\r\n|\n|\r/g;
|
|
let lineStart = 0;
|
|
let match: RegExpExecArray | null;
|
|
|
|
while ((match = newlinePattern.exec(contents)) !== null) {
|
|
lines.push({
|
|
contents: contents.slice(lineStart, match.index),
|
|
ending: match[0],
|
|
});
|
|
lineStart = newlinePattern.lastIndex;
|
|
}
|
|
|
|
if (lineStart < contents.length || contents.length === 0) {
|
|
lines.push({ contents: contents.slice(lineStart), ending: "" });
|
|
}
|
|
|
|
return lines;
|
|
}
|
|
|
|
/**
|
|
* Updates only the named assignments and preserves every unrelated byte.
|
|
*
|
|
* Duplicate managed keys are intentional: every occurrence is checked and any
|
|
* stale occurrence is updated. Occurrences that already decode to the desired
|
|
* value retain their original quoting and spacing.
|
|
*/
|
|
export function updateEnvFileContents(
|
|
contents: string,
|
|
entries: Record<string, string>,
|
|
options: UpdateEnvFileContentsOptions = {},
|
|
): string {
|
|
const valueEncoding = options.valueEncoding ?? "json";
|
|
const missingEntries = new Map(Object.entries(entries));
|
|
const lines = splitEnvLines(contents);
|
|
|
|
for (const line of lines) {
|
|
const match = line.contents.match(ENV_ASSIGNMENT_PATTERN);
|
|
if (!match) continue;
|
|
|
|
const [, prefix, key, separator, rawValue] = match;
|
|
const value = entries[key];
|
|
if (value === undefined) continue;
|
|
|
|
missingEntries.delete(key);
|
|
if (parseComparableEnvValue(rawValue) === value) continue;
|
|
|
|
const commentStart = trailingEnvCommentStart(rawValue);
|
|
const trailingComment = commentStart === null ? "" : rawValue.slice(commentStart);
|
|
line.contents = `${prefix}${key}${separator}${encodeEnvValue(value, valueEncoding)}${trailingComment}`;
|
|
}
|
|
|
|
if (missingEntries.size > 0) {
|
|
const newline = lines.find((line) => line.ending.length > 0)?.ending ?? "\n";
|
|
const insertedLines = Array.from(missingEntries, ([key, value]) => ({
|
|
contents: `${key}=${encodeEnvValue(value, valueEncoding)}`,
|
|
ending: newline,
|
|
}));
|
|
const trailingBlankIndex = lines.at(-1)?.contents === "" ? lines.length - 1 : lines.length;
|
|
|
|
if (trailingBlankIndex === lines.length && lines.length > 0) {
|
|
const previousLine = lines.at(-1)!;
|
|
if (previousLine.ending === "") {
|
|
previousLine.ending = newline;
|
|
insertedLines.at(-1)!.ending = "";
|
|
}
|
|
}
|
|
lines.splice(trailingBlankIndex, 0, ...insertedLines);
|
|
}
|
|
|
|
return lines.map((line) => `${line.contents}${line.ending}`).join("");
|
|
}
|
|
|
|
/** Writes a changed env file through a same-directory temporary file and rename. */
|
|
export function writeEnvFileAtomicallyIfChanged(
|
|
filePath: string,
|
|
previousContents: string | null,
|
|
nextContents: string,
|
|
): boolean {
|
|
if (previousContents === nextContents) return false;
|
|
|
|
const directory = path.dirname(filePath);
|
|
fs.mkdirSync(directory, { recursive: true });
|
|
const temporaryPath = path.join(directory, `.${path.basename(filePath)}.${process.pid}.${randomUUID()}.tmp`);
|
|
|
|
try {
|
|
fs.writeFileSync(temporaryPath, nextContents, { encoding: "utf8", mode: 0o600, flag: "wx" });
|
|
fs.renameSync(temporaryPath, filePath);
|
|
fs.chmodSync(filePath, 0o600);
|
|
} catch (error) {
|
|
try {
|
|
fs.unlinkSync(temporaryPath);
|
|
} catch {
|
|
// The temporary file may not have been created or may already have been renamed.
|
|
}
|
|
throw error;
|
|
}
|
|
|
|
return true;
|
|
}
|