Files
PaperClipAI/packages/shared/src/env-file.ts
DottaandPaperclip 9485ffea70 fix(config): preserve env files during managed updates (#10980)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The CLI and server both update Paperclip values in `.env` files
> - The server preserved operator content, but the CLI rebuilt the
complete file
> - A CLI rerun could remove comments, custom values, ordering, and
newline style
> - Both paths need one editor with one value encoding and duplicate key
policy
> - The final integration also needs one regression test across the
related setup and sync safety mechanisms
> - This pull request moves the editor to the shared package and adds
cross-cutting rerun-survival coverage
> - The benefit is safe setup and worktree repair reruns that preserve
operator edits

## Linked Issues or Issue Description

**What happened?**

The CLI rebuilt the complete `.env` file when it wrote a managed
Paperclip value. This action removed comments, blank lines, custom keys,
original quoting, and the original newline style.

**Expected behavior**

Paperclip must update only the managed assignments. It must preserve all
unrelated bytes. It must skip the file replacement when all managed
values are current.

**Steps to reproduce**

1. Add comments, custom keys, quoted values, and CRLF newlines to the
Paperclip `.env` file.
2. Run a CLI path that calls the agent JWT secret setup.
3. Observe that the old writer replaces the complete file.

**Paperclip version or commit**

The problem exists on `master` before this pull request.

Related public context: Refs #437.

## What Changed

- Add one shared line-preserving `.env` editor for the CLI and server.
- Define minimal and JSON value encodings in the shared helper.
- Update every stale duplicate of a managed key and preserve current
duplicate encodings.
- Preserve comments, ordering, blank lines, unknown keys, export
prefixes, trailing comments, and newline style.
- Write changed files through a same-directory temporary file and atomic
rename.
- Limit CLI updates to non-empty `PAPERCLIP_*` entries.
- Skip the write when all managed values are current.
- Add shared, CLI, and server regression coverage.
- Refresh the branch after the related config, sandbox, and skill safety
changes landed.
- Add a cross-cutting integration test for config, env-file,
managed-sandbox, and managed-instructions rerun survival.

## Verification

- `pnpm exec vitest run packages/shared/src/env-file.test.ts
packages/shared/src/config-schema.test.ts
cli/src/__tests__/agent-jwt-env.test.ts
cli/src/__tests__/config-store.test.ts
server/src/__tests__/config-file.test.ts
server/src/__tests__/worktree-config.test.ts` passes 39 tests.
- `pnpm exec vitest run
server/src/__tests__/rerun-survival.integration.test.ts` passes 4 tests.
- `pnpm -r typecheck` passes on the previous head. GitHub CI reruns it
on the refreshed head.
- The previous head passed the complete general, serialized, workspace,
and E2E matrix. GitHub CI reruns that matrix on the refreshed head.
- `pnpm build` passes on the previous head. GitHub CI reruns it on the
refreshed head.

## Risks

- Low risk. The production change only affects managed `.env`
assignments.
- Existing managed assignments can keep their original quoting when
their decoded values are current.
- Changed CLI values keep the prior minimal encoding policy. Changed
server values keep the prior JSON encoding policy.
- Duplicate managed assignments now follow one explicit rule: Paperclip
updates each stale occurrence.
- The master refresh had one import-block conflict. The resolution keeps
both the config merge imports and the env-file imports.
- The added integration file is test-only. It has no database, API, or
UI contract effect.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex from the GPT-5 family produced this change with reasoning,
tool use, and code execution. The runtime did not expose the exact model
ID or context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-07 00:54:43 -05:00

182 lines
5.5 KiB
TypeScript

import fs from "node:fs";
import path from "node:path";
import { randomUUID } from "node:crypto";
export type EnvValueEncoding = "minimal" | "json";
export type UpdateEnvFileContentsOptions = {
valueEncoding?: EnvValueEncoding;
};
type EnvLine = {
contents: string;
ending: string;
};
const ENV_ASSIGNMENT_PATTERN = /^(\s*(?:export\s+)?)([A-Za-z_][A-Za-z0-9_]*)(\s*=\s*)(.*)$/;
const MINIMAL_ENV_VALUE_PATTERN = /^[A-Za-z0-9_./:@-]+$/;
export function encodeEnvValue(value: string, encoding: EnvValueEncoding = "json"): string {
if (encoding === "minimal" && MINIMAL_ENV_VALUE_PATTERN.test(value)) {
return value;
}
return JSON.stringify(value);
}
function trailingEnvCommentStart(rawValue: string): number | null {
let quote: "\"" | "'" | null = null;
let escaped = false;
for (let index = 0; index < rawValue.length; index += 1) {
const character = rawValue[index];
if (escaped) {
escaped = false;
continue;
}
if (quote === "\"" && character === "\\") {
escaped = true;
continue;
}
if (quote !== null) {
if (character === quote) quote = null;
continue;
}
if (character === "\"" || character === "'") {
quote = character;
continue;
}
// dotenv treats every unquoted # as the start of a comment, even when it
// immediately follows the assignment separator or the value.
if (character !== "#") continue;
let commentStart = index;
while (commentStart > 0 && /\s/.test(rawValue[commentStart - 1] ?? "")) {
commentStart -= 1;
}
return commentStart;
}
return null;
}
function parseComparableEnvValue(rawValue: string): string | null {
const commentStart = trailingEnvCommentStart(rawValue);
const encodedValue = rawValue.slice(0, commentStart ?? rawValue.length).trim();
if (encodedValue === "") return "";
if (encodedValue.startsWith("\"") && encodedValue.endsWith("\"")) {
try {
const parsed = JSON.parse(encodedValue);
return typeof parsed === "string" ? parsed : null;
} catch {
return null;
}
}
if (encodedValue.startsWith("'") && encodedValue.endsWith("'")) {
return encodedValue.slice(1, -1);
}
return encodedValue;
}
function splitEnvLines(contents: string): EnvLine[] {
const lines: EnvLine[] = [];
const newlinePattern = /\r\n|\n|\r/g;
let lineStart = 0;
let match: RegExpExecArray | null;
while ((match = newlinePattern.exec(contents)) !== null) {
lines.push({
contents: contents.slice(lineStart, match.index),
ending: match[0],
});
lineStart = newlinePattern.lastIndex;
}
if (lineStart < contents.length || contents.length === 0) {
lines.push({ contents: contents.slice(lineStart), ending: "" });
}
return lines;
}
/**
* Updates only the named assignments and preserves every unrelated byte.
*
* Duplicate managed keys are intentional: every occurrence is checked and any
* stale occurrence is updated. Occurrences that already decode to the desired
* value retain their original quoting and spacing.
*/
export function updateEnvFileContents(
contents: string,
entries: Record<string, string>,
options: UpdateEnvFileContentsOptions = {},
): string {
const valueEncoding = options.valueEncoding ?? "json";
const missingEntries = new Map(Object.entries(entries));
const lines = splitEnvLines(contents);
for (const line of lines) {
const match = line.contents.match(ENV_ASSIGNMENT_PATTERN);
if (!match) continue;
const [, prefix, key, separator, rawValue] = match;
const value = entries[key];
if (value === undefined) continue;
missingEntries.delete(key);
if (parseComparableEnvValue(rawValue) === value) continue;
const commentStart = trailingEnvCommentStart(rawValue);
const trailingComment = commentStart === null ? "" : rawValue.slice(commentStart);
line.contents = `${prefix}${key}${separator}${encodeEnvValue(value, valueEncoding)}${trailingComment}`;
}
if (missingEntries.size > 0) {
const newline = lines.find((line) => line.ending.length > 0)?.ending ?? "\n";
const insertedLines = Array.from(missingEntries, ([key, value]) => ({
contents: `${key}=${encodeEnvValue(value, valueEncoding)}`,
ending: newline,
}));
const trailingBlankIndex = lines.at(-1)?.contents === "" ? lines.length - 1 : lines.length;
if (trailingBlankIndex === lines.length && lines.length > 0) {
const previousLine = lines.at(-1)!;
if (previousLine.ending === "") {
previousLine.ending = newline;
insertedLines.at(-1)!.ending = "";
}
}
lines.splice(trailingBlankIndex, 0, ...insertedLines);
}
return lines.map((line) => `${line.contents}${line.ending}`).join("");
}
/** Writes a changed env file through a same-directory temporary file and rename. */
export function writeEnvFileAtomicallyIfChanged(
filePath: string,
previousContents: string | null,
nextContents: string,
): boolean {
if (previousContents === nextContents) return false;
const directory = path.dirname(filePath);
fs.mkdirSync(directory, { recursive: true });
const temporaryPath = path.join(directory, `.${path.basename(filePath)}.${process.pid}.${randomUUID()}.tmp`);
try {
fs.writeFileSync(temporaryPath, nextContents, { encoding: "utf8", mode: 0o600, flag: "wx" });
fs.renameSync(temporaryPath, filePath);
fs.chmodSync(filePath, 0o600);
} catch (error) {
try {
fs.unlinkSync(temporaryPath);
} catch {
// The temporary file may not have been created or may already have been renamed.
}
throw error;
}
return true;
}