Polish changelog fragments with claude -p at release time

- Replace `renderGroupedChanges` with `polishFragmentsWithClaude` that pipes fragments through `claude -p --model sonnet` to merge related items, drop housekeeping noise, and produce user-facing release notes
- Internal fragments kept in CHANGELOG.md under a `<details>` collapse; dropped from GitHub release notes entirely
- CI no longer auto-runs `changelog:build` on tag-based releases — fails fast with a clear error if `changes/*.md` fragments are still pending; build locally and commit before tagging
- Add `runClaude` dep-injection seam to test surface; add failure-mode coverage (missing binary, empty output, missing headers, missing `<details>` wrapper)
- Delete implemented design doc; update `changes/README.md` and `docs/RELEASING.md` with claude CLI prerequisite and new workflow
This commit is contained in:
2026-05-02 19:51:43 -07:00
parent baabdb6d30
commit 27f5b2bb58
9 changed files with 586 additions and 318 deletions
+3 -4
View File
@@ -351,12 +351,11 @@ jobs:
id: version
run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> "$GITHUB_OUTPUT"
- name: Build changelog artifacts for release
- name: Guard against pending changelog fragments
run: |
if find changes -maxdepth 1 -name '*.md' -not -name README.md -print -quit | grep -q .; then
bun run changelog:build --version "${{ steps.version.outputs.VERSION }}"
else
echo "No pending changelog fragments found."
echo "::error::Pending changelog fragments detected. Run 'bun run changelog:build --version ${{ steps.version.outputs.VERSION }}' locally and commit the polished CHANGELOG.md before tagging. CI no longer auto-builds the changelog because the polish step requires the local 'claude' CLI."
exit 1
fi
- name: Verify changelog is ready for tagged release
+5
View File
@@ -0,0 +1,5 @@
type: internal
area: release
- Replaced the changelog renderer with a `claude -p` polish pass that merges related fragments, drops PR housekeeping, and writes user-friendly release notes. CHANGELOG.md keeps internal items in a collapsed `<details>` block; the GitHub release notes drop them entirely.
- Removed the release CI auto-build for pending `changes/*.md` fragments. Tag-based release runs now fail fast with a clear error if fragments are still pending; build the changelog locally with `bun run changelog:build` (which requires the `claude` CLI on PATH) and commit before tagging.
+6
View File
@@ -31,6 +31,12 @@ Rules:
- `README.md` is ignored by the generator
- if a PR should not produce release notes, apply the `skip-changelog` label instead of adding a fragment
How fragments turn into a release:
- At release time, `bun run changelog:build` (and `bun run changelog:prerelease-notes`) pipes every pending fragment through `claude -p` to merge related items, drop noise, and rewrite into a clean user-facing release body. Write fragments as raw, informative notes — don't worry about polished prose, deduping across PRs, or line-by-line phrasing. The polish step handles all of that.
- `internal` fragments stay in `CHANGELOG.md` (inside a collapsed `<details>` block) but are dropped from the GitHub release notes entirely.
- The polished `CHANGELOG.md` and `release/release-notes.md` are committed and reviewed before tagging — edit the Markdown by hand if Claude misses something.
Prerelease notes:
- prerelease tags like `v0.11.3-beta.1` and `v0.11.3-rc.1` reuse the current pending fragments to generate `release/prerelease-notes.md`
+18 -6
View File
@@ -2,16 +2,28 @@
# Releasing
## Prerequisites
- `claude` (Claude Code CLI) installed, on `PATH`, and authenticated.
`changelog:build` and `changelog:prerelease-notes` invoke
`claude -p --model sonnet` to merge and rewrite `changes/*.md` fragments into
a polished, user-facing release body. Either OAuth login (`claude /login`) or
`ANTHROPIC_API_KEY` works. Install from <https://claude.com/claude-code> if
you don't already have it.
## Stable Release
1. Confirm `main` is green: `gh run list --workflow CI --limit 5`.
2. Confirm release-facing docs are current: `README.md`, `changes/*.md`, and any touched `docs-site/` pages/config examples.
3. Run `bun run changelog:lint`.
4. Bump `package.json` to the release version.
5. Build release metadata before tagging:
5. Build release metadata before tagging (this calls `claude -p` locally):
`bun run changelog:build --version <version> --date <yyyy-mm-dd>`
- Release CI now also auto-runs this step when releasing directly from a tag and `changes/*.md` fragments remain.
6. Review `CHANGELOG.md` and `release/release-notes.md`.
- The polished `CHANGELOG.md` and `release/release-notes.md` are committed
before tagging. Release CI no longer auto-builds the changelog; it fails
fast if `changes/*.md` fragments are still present on a tag-based run.
6. Review `CHANGELOG.md` and `release/release-notes.md`. Edit by hand if Claude
missed something — the committed Markdown is what ships.
7. Run release gate locally:
`bun run changelog:check --version <version>`
`bun run verify:config-example`
@@ -31,7 +43,7 @@
1. Confirm release-facing docs and pending `changes/*.md` fragments are current.
2. Run `bun run changelog:lint`.
3. Bump `package.json` to the prerelease version, for example `0.11.3-beta.1` or `0.11.3-rc.1`.
4. Run the prerelease gate locally:
4. Run the prerelease gate locally (this calls `claude -p` locally):
`bun run changelog:prerelease-notes --version <version>`
`bun run verify:config-example`
`bun run typecheck`
@@ -51,8 +63,8 @@ Notes:
- Pass `--date` explicitly when you want the release stamped with the local cut date; otherwise the generator uses the current ISO date, which can roll over to the next UTC day late at night.
- `changelog:check` now rejects tag/package version mismatches.
- `changelog:prerelease-notes` also rejects tag/package version mismatches and writes `release/prerelease-notes.md` without mutating tracked changelog files.
- `changelog:build` generates `CHANGELOG.md` + `release/release-notes.md` and removes the released `changes/*.md` fragments.
- In the same way, the release workflow now auto-runs `changelog:build` when it detects unreleased `changes/*.md` on a tag-based run, then verifies and publishes.
- `changelog:build` generates `CHANGELOG.md` + `release/release-notes.md` (both polished by `claude -p`) and removes the released `changes/*.md` fragments. The CHANGELOG keeps internal notes inside a `<details><summary>Internal changes</summary>` collapse; the release notes drop them entirely.
- The release workflow no longer auto-runs `changelog:build`. If pending `changes/*.md` fragments are present on a tag-based run, CI exits with a clear `::error::` pointing at the local fix. Run `bun run changelog:build --version <version>` locally, commit the polished output, then tag.
- Do not tag while `changes/*.md` fragments still exist.
- Prerelease tags intentionally keep `changes/*.md` fragments in place so multiple prereleases can reuse the same cumulative pending notes until the final stable cut.
- If you need to repair a published release body (for example, a prior versions section was omitted), regenerate notes from `CHANGELOG.md` and re-edit the release with `gh release edit --notes-file`.
@@ -1,232 +0,0 @@
# AI-Polished Changelog Design
Date: 2026-05-02
Status: Approved, awaiting implementation plan
## Problem
Today every user-visible PR drops a fragment under `changes/`. The
fragments are written by whoever shipped the PR, so the prose tone,
granularity, and noise level vary wildly. By the time a stable release
is cut, the assembled CHANGELOG section can run 40+ bullets and lean
heavily on internal area labels (`Overlay:`, `Launcher:`, etc.). End
users skim it, miss the actual story of the release, and the GitHub
release body inherits the same clutter.
## Goal
Replace the mechanical `renderGroupedChanges` step in
`scripts/build-changelog.ts` with a single non-interactive `claude -p`
call that:
- Merges related fragments into coherent feature-level bullets.
- Drops PR-housekeeping noise that no user benefits from reading.
- Reorganizes by user-visible feature, not by `area:` prefix.
- Produces a tight `## v<version>` body suitable for both
`CHANGELOG.md` and the GitHub release notes.
The fragment file format and authoring workflow stay the same. Only
the rendering step changes.
## Decisions
1. **Pipeline:** Replace the current bullet renderer. `changelog:build`
and `changelog:prerelease-notes` always invoke `claude -p`. There
is no legacy fallback path.
2. **Review:** Write straight to `CHANGELOG.md` /
`release/release-notes.md` / `release/prerelease-notes.md`. The
release engineer reviews the diff before tagging; no extra
confirmation prompt.
3. **CI:** Local-only. The release workflow no longer auto-runs
`changelog:build` when fragments are pending; instead it fails the
tagged release with a clear message asking the user to run the
build locally and commit the result. CI does not need
`claude` on PATH or any Anthropic credentials.
4. **Polish depth:** Heavy rewrite plus dedupe. Claude is allowed to
merge bullets, drop trivial ones, and reorder by feature.
5. **Internal section:** Dropped from release notes entirely. Kept in
`CHANGELOG.md` inside a `<details><summary>Internal changes</summary>`
collapse so the historical record survives without taking over the
page.
6. **Prereleases:** Same polish path. Beta and RC notes look as clean
as stable.
7. **Claude invocation:**
`claude -p --bare --model sonnet --permission-mode bypassPermissions --output-format text`
over stdin. `--bare` skips hooks, MCP, auto-memory, and CLAUDE.md
discovery so the call is self-contained and reproducible.
8. **Failure mode:** Hard fail. Missing `claude` binary, non-zero
exit, empty/short output, or output missing the required section
headers all abort the build.
## Architecture
### Affected files
- `scripts/build-changelog.ts` — primary edit. Replace
`renderGroupedChanges` with `polishFragmentsWithClaude`. Wire it
into `writeChangelogArtifacts`, `writePrereleaseNotesForVersion`,
and any other path that currently calls `renderGroupedChanges`.
- `scripts/build-changelog.test.ts` — add `runClaude` to the dep
injection surface, stub it in unit tests, add new coverage for
failure modes and mode='changelog' vs mode='release-notes'.
- `.github/workflows/release.yml` (or wherever the auto-build lives)
— remove the auto-`changelog:build` fallback; add a guard that
fails the run if `changes/*.md` exist on a tag-based release.
- `src/release-workflow.test.ts`,
`src/prerelease-workflow.test.ts` — update if they exercise the
CI auto-run path.
- `docs/RELEASING.md` — document the new local-only build step and
the `claude` PATH requirement.
- `changes/README.md` — note that fragments will be merged and
rewritten, so authors should write raw notes rather than polished
prose.
- `changes/<n>-ai-changelog-polish.md` — fragment for this change
itself (`type: internal`, `area: release`).
### New function: `polishFragmentsWithClaude`
```
polishFragmentsWithClaude(
fragments: ChangeFragment[],
options: {
mode: 'changelog' | 'release-notes',
version: string,
date?: string, // changelog mode only
deps?: { runClaude?: (prompt: string) => string },
},
): string
```
- Filters out `internal` fragments when `mode === 'release-notes'`.
- Serializes the surviving fragments into the prompt format below.
- Invokes `claude` (via `deps.runClaude` or the default
`execFileSync` wrapper).
- Validates the output contains the expected section headers; throws
on failure.
- Returns the Markdown body that will be inserted under the
`## v<version>` heading.
The existing `prependReleaseSection`, `extractReleaseSectionBody`,
`writeReleaseNotesFile`, and `generateDocsChangelog` functions stay
exactly as they are — they consume the polished body the same way
they consumed the rendered body.
### Prompt input format
```
MODE: changelog | release-notes
VERSION: 0.13.0
DATE: 2026-05-02 (changelog mode only)
FRAGMENT changes/291-character-dictionary-selection.md
type: added
area: dictionary
breaking: false
- Added CLI and in-app AniList selection for character dictionary mismatches...
- Added launcher support through `subminer dictionary --candidates`...
FRAGMENT changes/293-interjection-annotation-filter.md
type: fixed
area: tokenizer
breaking: false
- Stopped standalone `あ` interjections from receiving subtitle annotation metadata...
...
```
### Prompt output contract
Claude is instructed to emit Markdown only, no preamble, conforming to:
- Sections in this order, omitting empty ones:
`### Breaking Changes`, `### Added`, `### Changed`,
`### Fixed`, `### Docs`.
- For `mode === 'changelog'`, an additional terminal section:
```
<details>
<summary>Internal changes</summary>
### Internal
- …
- …
</details>
```
- Bullets lead with a feature name (e.g. `Playlist browser:`,
`Windows overlay:`) — not with the raw `area:` slug.
- Bullets may merge multiple fragments, but breaking changes must
retain their substance and stay in `### Breaking Changes`.
- Bullets that document only PR-housekeeping or CodeRabbit follow-ups
are dropped unless they describe a user-visible behavior change.
After parsing, the body is passed unchanged to `prependReleaseSection`,
which wraps it under the standard `## v<version> (<date>)` heading.
### Determinism and review
Claude is non-deterministic. The mitigation is the existing release
workflow: the polished CHANGELOG/release-notes are committed before
tagging, so what's in the repo is what ships. The release engineer
diffs the commit before pushing.
## Testing
`scripts/build-changelog.test.ts` already exposes a `deps` injection
seam for fs operations. Extend it with `runClaude?: (prompt: string)
=> string`. Tests stub it to return canned Markdown; the default
implementation (used in production) wraps `execFileSync('claude', …)`.
New cases to cover:
- Mode `'release-notes'` filters `internal` fragments before sending
to Claude.
- Mode `'changelog'` includes `internal` fragments and the
`<details>` wrapper expectation is propagated through to the prompt.
- Missing `claude` binary throws a clear error.
- Non-zero exit code from `claude` throws.
- Empty / whitespace-only output throws.
- Output missing the expected section headers throws.
- Prerelease path uses release-notes mode and writes
`release/prerelease-notes.md` with the existing disclaimer.
- The polished body still flows correctly through
`prependReleaseSection`, `extractReleaseSectionBody`, and
`generateDocsChangelog`.
`release-workflow.test.ts` and `prerelease-workflow.test.ts` are
updated to cover the new "fragments pending on a tag" failure mode.
## Documentation
- `docs/RELEASING.md` updates the stable and prerelease sections to
state that `changelog:build` and `changelog:prerelease-notes` invoke
`claude` locally, list the PATH requirement, and remove the note
about CI auto-running `changelog:build`.
- `changes/README.md` adds a short note: fragments will be merged and
rewritten by Claude during release, so write raw notes — don't
polish them.
## Out of Scope (YAGNI)
- No caching of polished output between runs.
- No diff-and-confirm interactive prompt.
- No SDK fallback if the CLI is missing.
- No `--no-polish` escape hatch. Revert the commit if you genuinely
need the legacy renderer back.
- No change to the fragment authoring schema or the `pr-check`
enforcement.
## Open Risks
- **Claude regresses on a release.** Mitigated by the
commit-before-tag review step. If the polish is bad, the release
engineer edits `CHANGELOG.md` directly and re-commits before
tagging.
- **`--bare` mode behavior changes upstream.** Pinning to `--model
sonnet` and `--bare` reduces drift, but Claude Code is still an
external tool. If invocation flags change, the release engineer
will see a hard failure and can patch the script.
- **CI surface area increases.** Removing the auto-`changelog:build`
fallback means a tag pushed before running the local build will
fail CI. The error message must clearly point at the fix.
+362 -17
View File
@@ -13,6 +13,75 @@ function createWorkspace(name: string): string {
return fs.mkdtempSync(path.join(baseDir, `${name}-`));
}
type RunClaudeArgs = { input: string; args: string[] };
function recordingRunClaude(responder: (input: string) => string): {
runClaude: (input: string, args: string[]) => string;
calls: RunClaudeArgs[];
} {
const calls: RunClaudeArgs[] = [];
return {
calls,
runClaude(input, args) {
calls.push({ input, args });
return responder(input);
},
};
}
function modeFromPrompt(input: string): 'changelog' | 'release-notes' | null {
// Anchor to start-of-line so we don't accidentally match the instructions text,
// which mentions "MODE: changelog" and "MODE: release-notes" mid-sentence.
const match = /^MODE: (changelog|release-notes)$/m.exec(input);
return (match?.[1] as 'changelog' | 'release-notes') ?? null;
}
function fragmentTypesInPrompt(input: string): string[] {
return input
.split(/\r?\n/)
.filter((line) => line.startsWith('type: '))
.map((line) => line.slice('type: '.length).trim());
}
function defaultPolishedBody(input: string): string {
const mode = modeFromPrompt(input);
const types = fragmentTypesInPrompt(input);
const sections: string[] = [];
const has = (t: string) => types.includes(t);
const hasBreaking = /^breaking: true$/m.test(input);
if (hasBreaking) {
sections.push('### Breaking Changes\n- Polished: breaking change.');
}
if (has('added')) {
sections.push('### Added\n- Polished: added entry.');
}
if (has('changed')) {
sections.push('### Changed\n- Polished: changed entry.');
}
if (has('fixed')) {
sections.push('### Fixed\n- Polished: fixed entry.');
}
if (has('docs')) {
sections.push('### Docs\n- Polished: docs entry.');
}
if (mode === 'changelog' && has('internal')) {
sections.push(
'<details>\n<summary>Internal changes</summary>\n\n### Internal\n- Polished: internal entry.\n\n</details>',
);
}
if (sections.length === 0) {
sections.push('### Changed\n- Polished: empty fallback.');
}
return sections.join('\n\n');
}
function defaultStubClaude() {
return recordingRunClaude(defaultPolishedBody);
}
test('resolveChangelogOutputPaths stays repo-local and never writes docs paths', async () => {
const { resolveChangelogOutputPaths } = await loadModule();
const workspace = createWorkspace('with-docs-repo');
@@ -62,10 +131,12 @@ test('writeChangelogArtifacts ignores README, groups fragments by type, writes r
);
try {
const stub = defaultStubClaude();
const result = writeChangelogArtifacts({
cwd: projectRoot,
version: '0.4.1',
date: '2026-03-07',
deps: { runClaude: stub.runClaude },
});
assert.deepEqual(result.outputPaths, [path.join(projectRoot, 'CHANGELOG.md')]);
@@ -77,18 +148,28 @@ test('writeChangelogArtifacts ignores README, groups fragments by type, writes r
assert.equal(fs.existsSync(path.join(projectRoot, 'changes', '002.md')), false);
assert.equal(fs.existsSync(path.join(projectRoot, 'changes', 'README.md')), true);
const changelog = fs.readFileSync(path.join(projectRoot, 'CHANGELOG.md'), 'utf8');
assert.match(
changelog,
/^# Changelog\n\n## v0\.4\.1 \(2026-03-07\)\n\n### Added\n- Overlay: Added release fragments\.\n\n### Fixed\n- Release: Fixed release notes generation\.\n\n## v0\.4\.0 \(2026-03-01\)\n- Existing fix\n$/m,
assert.equal(
stub.calls.length,
2,
'expected one Claude call per output (changelog + release notes)',
);
assert.deepEqual(
stub.calls.map((call) => modeFromPrompt(call.input)),
['changelog', 'release-notes'],
);
const changelog = fs.readFileSync(path.join(projectRoot, 'CHANGELOG.md'), 'utf8');
assert.match(changelog, /^# Changelog\n\n## v0\.4\.1 \(2026-03-07\)\n\n/);
assert.match(changelog, /### Added\n- Polished: added entry\./);
assert.match(changelog, /### Fixed\n- Polished: fixed entry\./);
assert.match(changelog, /## v0\.4\.0 \(2026-03-01\)\n- Existing fix\n$/);
const releaseNotes = fs.readFileSync(
path.join(projectRoot, 'release', 'release-notes.md'),
'utf8',
);
assert.match(releaseNotes, /## Highlights\n### Added\n- Overlay: Added release fragments\./);
assert.match(releaseNotes, /### Fixed\n- Release: Fixed release notes generation\./);
assert.match(releaseNotes, /## Highlights\n### Added\n- Polished: added entry\./);
assert.match(releaseNotes, /### Fixed\n- Polished: fixed entry\./);
assert.match(releaseNotes, /## Installation\n\nSee the README and docs\/installation guide/);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
@@ -159,10 +240,12 @@ test('writeStableReleaseArtifacts reuses the requested version and date for chan
);
try {
const stub = defaultStubClaude();
const result = writeStableReleaseArtifacts({
cwd: projectRoot,
version: '0.4.1',
date: '2026-03-07',
deps: { runClaude: stub.runClaude },
});
assert.deepEqual(result.outputPaths, [path.join(projectRoot, 'CHANGELOG.md')]);
@@ -260,10 +343,12 @@ test('writeChangelogArtifacts renders breaking changes section above type sectio
);
try {
const stub = defaultStubClaude();
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.5.0',
date: '2026-04-06',
deps: { runClaude: stub.runClaude },
});
const changelog = fs.readFileSync(path.join(projectRoot, 'CHANGELOG.md'), 'utf8');
@@ -276,8 +361,14 @@ test('writeChangelogArtifacts renders breaking changes section above type sectio
assert.notEqual(fixedIndex, -1, 'Fixed section should exist');
assert.ok(breakingIndex < changedIndex, 'Breaking Changes should appear before Changed');
assert.ok(changedIndex < fixedIndex, 'Changed should appear before Fixed');
assert.match(changelog, /### Breaking Changes\n- Config: Renamed `foo` to `bar`\./);
assert.match(changelog, /### Changed\n- Config: Renamed `foo` to `bar`\./);
const changelogCall = stub.calls.find((call) => modeFromPrompt(call.input) === 'changelog');
assert.ok(changelogCall, 'expected at least one changelog-mode Claude invocation');
assert.match(
changelogCall.input,
/breaking: true/,
'breaking metadata should reach the prompt verbatim',
);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
@@ -384,9 +475,11 @@ test('writePrereleaseNotesForVersion writes cumulative beta notes without mutati
);
try {
const stub = defaultStubClaude();
const outputPath = writePrereleaseNotesForVersion({
cwd: projectRoot,
version: '0.11.3-beta.1',
deps: { runClaude: stub.runClaude },
});
assert.equal(outputPath, path.join(projectRoot, 'release', 'prerelease-notes.md'));
@@ -403,13 +496,13 @@ test('writePrereleaseNotesForVersion writes cumulative beta notes without mutati
assert.equal(fs.existsSync(path.join(projectRoot, 'changes', '001.md')), true);
assert.equal(fs.existsSync(path.join(projectRoot, 'changes', '002.md')), true);
assert.equal(stub.calls.length, 1, 'prerelease should issue exactly one Claude call');
assert.equal(modeFromPrompt(stub.calls[0]!.input), 'release-notes');
const prereleaseNotes = fs.readFileSync(outputPath, 'utf8');
assert.match(prereleaseNotes, /^> This is a prerelease build for testing\./m);
assert.match(
prereleaseNotes,
/## Highlights\n### Added\n- Overlay: Added prerelease coverage\./,
);
assert.match(prereleaseNotes, /### Fixed\n- Launcher: Fixed prerelease packaging checks\./);
assert.match(prereleaseNotes, /## Highlights\n### Added\n- Polished: added entry\./);
assert.match(prereleaseNotes, /### Fixed\n- Polished: fixed entry\./);
assert.match(prereleaseNotes, /## Installation\n\nSee the README and docs\/installation guide/);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
@@ -434,16 +527,15 @@ test('writePrereleaseNotesForVersion supports rc prereleases', async () => {
);
try {
const stub = defaultStubClaude();
const outputPath = writePrereleaseNotesForVersion({
cwd: projectRoot,
version: '0.11.3-rc.1',
deps: { runClaude: stub.runClaude },
});
const prereleaseNotes = fs.readFileSync(outputPath, 'utf8');
assert.match(
prereleaseNotes,
/## Highlights\n### Changed\n- Release: Prepared release candidate notes\./,
);
assert.match(prereleaseNotes, /## Highlights\n### Changed\n- Polished: changed entry\./);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
@@ -536,3 +628,256 @@ test('writePrereleaseNotesForVersion rejects empty prerelease note generation wh
fs.rmSync(workspace, { recursive: true, force: true });
}
});
test('writeChangelogArtifacts surfaces a clear error when claude is missing from PATH', async () => {
const { writeChangelogArtifacts } = await loadModule();
const workspace = createWorkspace('claude-missing');
const projectRoot = path.join(workspace, 'SubMiner');
fs.mkdirSync(path.join(projectRoot, 'changes'), { recursive: true });
fs.writeFileSync(path.join(projectRoot, 'CHANGELOG.md'), '# Changelog\n', 'utf8');
fs.writeFileSync(
path.join(projectRoot, 'changes', '001.md'),
['type: added', 'area: overlay', '', '- A change.'].join('\n'),
'utf8',
);
// The production defaultRunClaude wrapper translates ENOENT into this friendly
// message; we simulate that contract here so the test exercises the propagation
// path through polishFragmentsWithClaude rather than re-implementing the
// execFileSync mock.
const enoent = (): string => {
throw new Error(
"claude CLI not found on PATH. Install Claude Code (https://claude.com/claude-code) and ensure 'claude' is on your PATH before running changelog:build.",
);
};
try {
assert.throws(
() =>
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.5.0',
date: '2026-04-06',
deps: { runClaude: enoent },
}),
/claude CLI not found on PATH/,
);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
});
test('writeChangelogArtifacts rejects empty claude output', async () => {
const { writeChangelogArtifacts } = await loadModule();
const workspace = createWorkspace('claude-empty');
const projectRoot = path.join(workspace, 'SubMiner');
fs.mkdirSync(path.join(projectRoot, 'changes'), { recursive: true });
fs.writeFileSync(path.join(projectRoot, 'CHANGELOG.md'), '# Changelog\n', 'utf8');
fs.writeFileSync(
path.join(projectRoot, 'changes', '001.md'),
['type: added', 'area: overlay', '', '- A change.'].join('\n'),
'utf8',
);
try {
assert.throws(
() =>
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.5.0',
date: '2026-04-06',
deps: { runClaude: () => ' \n ' },
}),
/claude returned empty output/,
);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
});
test('writeChangelogArtifacts rejects claude output missing required section headers', async () => {
const { writeChangelogArtifacts } = await loadModule();
const workspace = createWorkspace('claude-no-headers');
const projectRoot = path.join(workspace, 'SubMiner');
fs.mkdirSync(path.join(projectRoot, 'changes'), { recursive: true });
fs.writeFileSync(path.join(projectRoot, 'CHANGELOG.md'), '# Changelog\n', 'utf8');
fs.writeFileSync(
path.join(projectRoot, 'changes', '001.md'),
['type: added', 'area: overlay', '', '- A change.'].join('\n'),
'utf8',
);
try {
assert.throws(
() =>
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.5.0',
date: '2026-04-06',
deps: { runClaude: () => 'Sure, here is your changelog: it is great.' },
}),
/missing the expected section heading/,
);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
});
test('writeChangelogArtifacts rejects changelog-mode output that omits the Internal <details> wrapper when internal fragments are present', async () => {
const { writeChangelogArtifacts } = await loadModule();
const workspace = createWorkspace('claude-no-details');
const projectRoot = path.join(workspace, 'SubMiner');
fs.mkdirSync(path.join(projectRoot, 'changes'), { recursive: true });
fs.writeFileSync(path.join(projectRoot, 'CHANGELOG.md'), '# Changelog\n', 'utf8');
fs.writeFileSync(
path.join(projectRoot, 'changes', '001.md'),
['type: added', 'area: overlay', '', '- A user-facing change.'].join('\n'),
'utf8',
);
fs.writeFileSync(
path.join(projectRoot, 'changes', '002.md'),
['type: internal', 'area: release', '', '- An internal note.'].join('\n'),
'utf8',
);
const noDetailsResponder = (input: string): string => {
if (modeFromPrompt(input) === 'changelog') {
return '### Added\n- Polished: added.\n\n### Internal\n- Polished: internal (no details wrapper).';
}
return defaultPolishedBody(input);
};
try {
assert.throws(
() =>
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.5.0',
date: '2026-04-06',
deps: { runClaude: noDetailsResponder },
}),
/<details><summary>Internal changes<\/summary> wrapper/,
);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
});
test('writeChangelogArtifacts filters internal fragments from the release-notes Claude prompt', async () => {
const { writeChangelogArtifacts } = await loadModule();
const workspace = createWorkspace('release-notes-internal-filter');
const projectRoot = path.join(workspace, 'SubMiner');
fs.mkdirSync(path.join(projectRoot, 'changes'), { recursive: true });
fs.writeFileSync(path.join(projectRoot, 'CHANGELOG.md'), '# Changelog\n', 'utf8');
fs.writeFileSync(
path.join(projectRoot, 'changes', '001.md'),
['type: added', 'area: overlay', '', '- A user-facing change.'].join('\n'),
'utf8',
);
fs.writeFileSync(
path.join(projectRoot, 'changes', '002.md'),
['type: internal', 'area: release', '', '- An internal CI tweak.'].join('\n'),
'utf8',
);
try {
const stub = defaultStubClaude();
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.5.0',
date: '2026-04-06',
deps: { runClaude: stub.runClaude },
});
const changelogCall = stub.calls.find((call) => modeFromPrompt(call.input) === 'changelog');
const releaseNotesCall = stub.calls.find(
(call) => modeFromPrompt(call.input) === 'release-notes',
);
assert.ok(changelogCall, 'expected a changelog-mode invocation');
assert.ok(releaseNotesCall, 'expected a release-notes-mode invocation');
assert.deepEqual(
fragmentTypesInPrompt(changelogCall.input).sort(),
['added', 'internal'],
'changelog mode keeps internal fragments',
);
assert.deepEqual(
fragmentTypesInPrompt(releaseNotesCall.input),
['added'],
'release-notes mode drops internal fragments',
);
const releaseNotes = fs.readFileSync(
path.join(projectRoot, 'release', 'release-notes.md'),
'utf8',
);
assert.doesNotMatch(releaseNotes, /<details>/);
assert.doesNotMatch(releaseNotes, /### Internal/);
const changelog = fs.readFileSync(path.join(projectRoot, 'CHANGELOG.md'), 'utf8');
assert.match(changelog, /<details>[\s\S]*<summary>Internal changes<\/summary>/);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
});
test('writeChangelogArtifacts strips <details> blocks from release notes when reusing an existing CHANGELOG section', async () => {
const { writeChangelogArtifacts } = await loadModule();
const workspace = createWorkspace('reuse-existing-section');
const projectRoot = path.join(workspace, 'SubMiner');
const existingChangelog = [
'# Changelog',
'',
'## v0.4.1 (2026-03-07)',
'### Added',
'- Polished: previously committed.',
'',
'<details>',
'<summary>Internal changes</summary>',
'',
'### Internal',
'- Polished: internal note.',
'',
'</details>',
'',
].join('\n');
fs.mkdirSync(path.join(projectRoot, 'changes'), { recursive: true });
fs.writeFileSync(path.join(projectRoot, 'CHANGELOG.md'), existingChangelog, 'utf8');
fs.writeFileSync(
path.join(projectRoot, 'changes', '001.md'),
['type: added', 'area: overlay', '', '- Stale fragment.'].join('\n'),
'utf8',
);
try {
const stub = defaultStubClaude();
writeChangelogArtifacts({
cwd: projectRoot,
version: '0.4.1',
date: '2026-03-08',
deps: { runClaude: stub.runClaude },
});
assert.equal(
stub.calls.length,
0,
'no Claude calls should fire when the section already exists',
);
const releaseNotes = fs.readFileSync(
path.join(projectRoot, 'release', 'release-notes.md'),
'utf8',
);
assert.match(releaseNotes, /## Highlights\n### Added\n- Polished: previously committed\./);
assert.doesNotMatch(releaseNotes, /<details>/);
assert.doesNotMatch(releaseNotes, /### Internal/);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
}
});
+185 -52
View File
@@ -2,6 +2,8 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { execFileSync } from 'node:child_process';
type RunClaude = (input: string, args: string[]) => string;
type ChangelogFsDeps = {
existsSync?: (candidate: string) => boolean;
mkdirSync?: (candidate: string, options: { recursive: true }) => void;
@@ -10,8 +12,11 @@ type ChangelogFsDeps = {
rmSync?: (candidate: string) => void;
writeFileSync?: (candidate: string, content: string, encoding: BufferEncoding) => void;
log?: (message: string) => void;
runClaude?: RunClaude;
};
type PolishMode = 'changelog' | 'release-notes';
type ChangelogOptions = {
cwd?: string;
date?: string;
@@ -41,13 +46,6 @@ const RELEASE_NOTES_PATH = path.join('release', 'release-notes.md');
const PRERELEASE_NOTES_PATH = path.join('release', 'prerelease-notes.md');
const CHANGELOG_HEADER = '# Changelog';
const CHANGE_TYPES: FragmentType[] = ['added', 'changed', 'fixed', 'docs', 'internal'];
const CHANGE_TYPE_HEADINGS: Record<FragmentType, string> = {
added: 'Added',
changed: 'Changed',
fixed: 'Fixed',
docs: 'Docs',
internal: 'Internal',
};
const SKIP_CHANGELOG_LABEL = 'skip-changelog';
function normalizeVersion(version: string): string {
@@ -217,54 +215,179 @@ function readChangeFragments(cwd: string, deps?: ChangelogFsDeps): ChangeFragmen
});
}
function formatAreaLabel(area: string): string {
return area
.split(/[-_\s]+/)
.filter(Boolean)
.map((segment) => segment.charAt(0).toUpperCase() + segment.slice(1))
.join(' ');
}
// We deliberately don't pass --bare here. --bare skips OAuth/keychain reads and
// requires ANTHROPIC_API_KEY, which most Claude Code users don't have set up.
// The polish prompt is self-contained and doesn't need tools, so loading the
// user's hooks/MCP/CLAUDE.md is harmless overhead.
const CLAUDE_CLI_ARGS = [
'-p',
'--model',
'sonnet',
'--permission-mode',
'bypassPermissions',
'--output-format',
'text',
];
function renderFragmentBullet(fragment: ChangeFragment, bullet: string): string {
return `- ${formatAreaLabel(fragment.area)}: ${bullet.replace(/^- /, '')}`;
}
const SECTION_HEADER_PATTERN = /^### (Breaking Changes|Added|Changed|Fixed|Docs|Internal)$/m;
function renderGroupedChanges(fragments: ChangeFragment[]): string {
const sections: string[] = [];
const POLISH_PROMPT_INSTRUCTIONS = `You are formatting a software release changelog for end users of SubMiner, an Electron app for Japanese sentence mining.
const breakingFragments = fragments.filter((fragment) => fragment.breaking);
if (breakingFragments.length > 0) {
const bullets = breakingFragments
.flatMap((fragment) =>
fragment.bullets.map((bullet) => renderFragmentBullet(fragment, bullet)),
)
.join('\n');
sections.push(`### Breaking Changes\n${bullets}`);
}
You will receive a list of FRAGMENT entries below. Each fragment has metadata (type, area, breaking) and one or more bullet points written by the engineer who shipped that change. Your job is to merge, dedupe, and rewrite these fragments into a polished, user-facing release body.
for (const type of CHANGE_TYPES) {
const typeFragments = fragments.filter((fragment) => fragment.type === type);
if (typeFragments.length === 0) {
continue;
## Output Rules
1. Output Markdown ONLY. No preamble, no commentary, no closing remarks. Start directly with the first section heading.
2. Use these section headings, in this order, omitting any that have no bullets:
### Breaking Changes
### Added
### Changed
### Fixed
### Docs
3. In MODE: changelog only, append a final section after Docs:
<details>
<summary>Internal changes</summary>
### Internal
- …
</details>
Do not include the Internal section at all in MODE: release-notes; internal fragments will not be present in the input for that mode.
4. Each bullet should:
- Lead with a short feature/area name in title case followed by a colon, e.g. "Playlist browser:", "Windows overlay:", "Stats dashboard:". Pick the name from the fragment's bullet content, not the raw 'area:' slug.
- Be written in user-facing language. Drop implementation jargon, internal class names, file paths, and PR numbers.
- Be merged with related bullets when possible. If five fragments all touch Windows overlay z-order/focus/restore, write one or two bullets that summarize the overall improvement instead of five.
- Drop bullets that only describe PR housekeeping, CodeRabbit follow-ups, or test-only changes that don't affect users.
- Preserve the substance of every breaking change in ### Breaking Changes. Do not soften or omit them.
5. Do not invent features. Every bullet must be grounded in the input fragments.
6. Do not include the version heading (## v...) — that wrapper is added by the caller.
The input begins below.
`;
function defaultRunClaude(input: string, args: string[]): string {
try {
return execFileSync('claude', args, {
input,
encoding: 'utf8',
maxBuffer: 10 * 1024 * 1024,
stdio: ['pipe', 'pipe', 'inherit'],
});
} catch (error) {
const err = error as NodeJS.ErrnoException;
if (err.code === 'ENOENT') {
throw new Error(
"claude CLI not found on PATH. Install Claude Code (https://claude.com/claude-code) and ensure 'claude' is on your PATH before running changelog:build.",
);
}
const bullets = typeFragments
.flatMap((fragment) =>
fragment.bullets.map((bullet) => renderFragmentBullet(fragment, bullet)),
)
.join('\n');
sections.push(`### ${CHANGE_TYPE_HEADINGS[type]}\n${bullets}`);
throw new Error(`claude CLI invocation failed: ${err.message}`);
}
return sections.join('\n\n');
}
function buildReleaseSection(version: string, date: string, fragments: ChangeFragment[]): string {
function serializeFragmentsForPrompt(
fragments: ChangeFragment[],
mode: PolishMode,
version: string,
date?: string,
): string {
const header: string[] = [`MODE: ${mode}`, `VERSION: ${version}`];
if (date) {
header.push(`DATE: ${date}`);
}
const fragmentBlocks = fragments.map((fragment) => {
const relativePath = fragment.path.replace(/^.*?(changes\/.*)$/u, '$1');
return [
`FRAGMENT ${relativePath}`,
`type: ${fragment.type}`,
`area: ${fragment.area}`,
`breaking: ${fragment.breaking}`,
...fragment.bullets,
].join('\n');
});
return [...header, '', ...fragmentBlocks].join('\n\n');
}
function validatePolishedOutput(
output: string,
mode: PolishMode,
hasInternalFragments: boolean,
): string {
const trimmed = output.trim();
if (!trimmed) {
throw new Error('claude returned empty output for changelog polish.');
}
if (!SECTION_HEADER_PATTERN.test(trimmed)) {
throw new Error(
`claude output is missing the expected section heading (### Added/Changed/Fixed/Docs/Breaking Changes). Got:\n${trimmed.slice(0, 400)}`,
);
}
if (mode === 'changelog' && hasInternalFragments) {
if (!/<details>[\s\S]*<summary>[^<]*Internal[^<]*<\/summary>/m.test(trimmed)) {
throw new Error(
'claude output is missing the expected <details><summary>Internal changes</summary> wrapper for the Internal section.',
);
}
}
return trimmed;
}
function polishFragmentsWithClaude(
fragments: ChangeFragment[],
options: {
mode: PolishMode;
version: string;
date?: string;
deps?: ChangelogFsDeps;
},
): string {
const { mode, version, date } = options;
const runClaude = options.deps?.runClaude ?? defaultRunClaude;
const filtered =
mode === 'release-notes'
? fragments.filter((fragment) => fragment.type !== 'internal')
: fragments;
const hasInternalFragments =
mode === 'changelog' && fragments.some((fragment) => fragment.type === 'internal');
if (filtered.length === 0) {
throw new Error(
mode === 'release-notes'
? 'No user-facing changelog fragments found in changes/ (only internal fragments are present, which are dropped from release notes).'
: 'No changelog fragments found in changes/.',
);
}
const prompt =
POLISH_PROMPT_INSTRUCTIONS + serializeFragmentsForPrompt(filtered, mode, version, date);
const output = runClaude(prompt, CLAUDE_CLI_ARGS);
return validatePolishedOutput(output, mode, hasInternalFragments);
}
function stripDetailsBlocks(body: string): string {
return body.replace(/<details>[\s\S]*?<\/details>\s*/gm, '').trim();
}
function buildReleaseSection(
version: string,
date: string,
fragments: ChangeFragment[],
deps?: ChangelogFsDeps,
): string {
if (fragments.length === 0) {
throw new Error('No changelog fragments found in changes/.');
}
return [`## v${version} (${date})`, '', renderGroupedChanges(fragments), ''].join('\n');
const polished = polishFragmentsWithClaude(fragments, {
mode: 'changelog',
version,
date,
deps,
});
return [`## v${version} (${date})`, '', polished, ''].join('\n');
}
function ensureChangelogHeader(existingChangelog: string): string {
@@ -392,7 +515,11 @@ export function writeChangelogArtifacts(options?: ChangelogOptions): {
log(`Removed ${fragment.path}`);
}
const releaseNotesPath = writeReleaseNotesFile(cwd, existingReleaseSection, options?.deps);
const releaseNotesPath = writeReleaseNotesFile(
cwd,
stripDetailsBlocks(existingReleaseSection),
options?.deps,
);
log(`Generated ${releaseNotesPath}`);
return {
@@ -402,7 +529,7 @@ export function writeChangelogArtifacts(options?: ChangelogOptions): {
};
}
const releaseSection = buildReleaseSection(version, date, fragments);
const releaseSection = buildReleaseSection(version, date, fragments, options?.deps);
const nextChangelog = prependReleaseSection(existingChangelog, releaseSection, version);
for (const outputPath of outputPaths) {
@@ -411,11 +538,13 @@ export function writeChangelogArtifacts(options?: ChangelogOptions): {
log(`Updated ${outputPath}`);
}
const releaseNotesPath = writeReleaseNotesFile(
cwd,
extractReleaseSectionBody(nextChangelog, version) ?? releaseSection,
options?.deps,
);
const releaseNotesBody = polishFragmentsWithClaude(fragments, {
mode: 'release-notes',
version,
date,
deps: options?.deps,
});
const releaseNotesPath = writeReleaseNotesFile(cwd, releaseNotesBody, options?.deps);
log(`Generated ${releaseNotesPath}`);
for (const fragment of fragments) {
@@ -645,7 +774,7 @@ export function writeReleaseNotesForVersion(options?: ChangelogOptions): string
throw new Error(`Missing CHANGELOG section for v${version}.`);
}
return writeReleaseNotesFile(cwd, changes, options?.deps);
return writeReleaseNotesFile(cwd, stripDetailsBlocks(changes), options?.deps);
}
export function writePrereleaseNotesForVersion(options?: ChangelogOptions): string {
@@ -664,7 +793,11 @@ export function writePrereleaseNotesForVersion(options?: ChangelogOptions): stri
throw new Error('No changelog fragments found in changes/.');
}
const changes = renderGroupedChanges(fragments);
const changes = polishFragmentsWithClaude(fragments, {
mode: 'release-notes',
version,
deps: options?.deps,
});
return writeReleaseNotesFile(cwd, changes, options?.deps, {
disclaimer:
'> This is a prerelease build for testing. Stable changelog and docs-site updates remain pending until the final stable release.',
+6 -3
View File
@@ -36,9 +36,12 @@ test('release workflow verifies a committed changelog section before publish', (
assert.match(releaseWorkflow, /bun run changelog:check/);
});
test('release workflow builds changelog artifacts when fragments are pending', () => {
assert.match(releaseWorkflow, /Build changelog artifacts for release/);
assert.match(releaseWorkflow, /changelog:build --version/);
test('release workflow guards against pending changelog fragments instead of auto-building them', () => {
assert.match(releaseWorkflow, /Guard against pending changelog fragments/);
assert.match(releaseWorkflow, /::error::Pending changelog fragments detected/);
assert.match(releaseWorkflow, /Run 'bun run changelog:build --version/);
assert.doesNotMatch(releaseWorkflow, /Build changelog artifacts for release/);
assert.doesNotMatch(releaseWorkflow, /bun run changelog:build --version "\$\{\{ steps\.version/);
});
test('release workflow verifies generated config examples before packaging artifacts', () => {
+1 -4
View File
@@ -2,10 +2,7 @@ import assert from 'node:assert/strict';
import test from 'node:test';
import { SPECIAL_COMMANDS } from '../../config/definitions';
import {
describeSessionHelpCommand,
formatSessionHelpKeybinding,
} from './session-help.js';
import { describeSessionHelpCommand, formatSessionHelpKeybinding } from './session-help.js';
test('session help describes sub-seek commands as subtitle-line navigation', () => {
assert.equal(describeSessionHelpCommand(['sub-seek', 1]), 'Jump to next subtitle');