mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-05-04 00:41:33 -07:00
Add design doc for AI-polished changelog workflow
- Capture decisions from brainstorming: replace bullet renderer with `claude -p`, write straight to disk, hard-fail on missing/failed claude, drop internal section from release notes but keep collapsed in CHANGELOG.md - Document prompt input/output contract, affected files, test plan, and CI guard that fails tag-based releases when changelog fragments are still pending - Set scope boundaries (no caching, no SDK fallback, no `--no-polish` escape hatch)
This commit is contained in:
@@ -0,0 +1,232 @@
|
||||
# AI-Polished Changelog Design
|
||||
|
||||
Date: 2026-05-02
|
||||
Status: Approved, awaiting implementation plan
|
||||
|
||||
## Problem
|
||||
|
||||
Today every user-visible PR drops a fragment under `changes/`. The
|
||||
fragments are written by whoever shipped the PR, so the prose tone,
|
||||
granularity, and noise level vary wildly. By the time a stable release
|
||||
is cut, the assembled CHANGELOG section can run 40+ bullets and lean
|
||||
heavily on internal area labels (`Overlay:`, `Launcher:`, etc.). End
|
||||
users skim it, miss the actual story of the release, and the GitHub
|
||||
release body inherits the same clutter.
|
||||
|
||||
## Goal
|
||||
|
||||
Replace the mechanical `renderGroupedChanges` step in
|
||||
`scripts/build-changelog.ts` with a single non-interactive `claude -p`
|
||||
call that:
|
||||
|
||||
- Merges related fragments into coherent feature-level bullets.
|
||||
- Drops PR-housekeeping noise that no user benefits from reading.
|
||||
- Reorganizes by user-visible feature, not by `area:` prefix.
|
||||
- Produces a tight `## v<version>` body suitable for both
|
||||
`CHANGELOG.md` and the GitHub release notes.
|
||||
|
||||
The fragment file format and authoring workflow stay the same. Only
|
||||
the rendering step changes.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. **Pipeline:** Replace the current bullet renderer. `changelog:build`
|
||||
and `changelog:prerelease-notes` always invoke `claude -p`. There
|
||||
is no legacy fallback path.
|
||||
2. **Review:** Write straight to `CHANGELOG.md` /
|
||||
`release/release-notes.md` / `release/prerelease-notes.md`. The
|
||||
release engineer reviews the diff before tagging; no extra
|
||||
confirmation prompt.
|
||||
3. **CI:** Local-only. The release workflow no longer auto-runs
|
||||
`changelog:build` when fragments are pending; instead it fails the
|
||||
tagged release with a clear message asking the user to run the
|
||||
build locally and commit the result. CI does not need
|
||||
`claude` on PATH or any Anthropic credentials.
|
||||
4. **Polish depth:** Heavy rewrite plus dedupe. Claude is allowed to
|
||||
merge bullets, drop trivial ones, and reorder by feature.
|
||||
5. **Internal section:** Dropped from release notes entirely. Kept in
|
||||
`CHANGELOG.md` inside a `<details><summary>Internal changes</summary>`
|
||||
collapse so the historical record survives without taking over the
|
||||
page.
|
||||
6. **Prereleases:** Same polish path. Beta and RC notes look as clean
|
||||
as stable.
|
||||
7. **Claude invocation:**
|
||||
`claude -p --bare --model sonnet --permission-mode bypassPermissions --output-format text`
|
||||
over stdin. `--bare` skips hooks, MCP, auto-memory, and CLAUDE.md
|
||||
discovery so the call is self-contained and reproducible.
|
||||
8. **Failure mode:** Hard fail. Missing `claude` binary, non-zero
|
||||
exit, empty/short output, or output missing the required section
|
||||
headers all abort the build.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Affected files
|
||||
|
||||
- `scripts/build-changelog.ts` — primary edit. Replace
|
||||
`renderGroupedChanges` with `polishFragmentsWithClaude`. Wire it
|
||||
into `writeChangelogArtifacts`, `writePrereleaseNotesForVersion`,
|
||||
and any other path that currently calls `renderGroupedChanges`.
|
||||
- `scripts/build-changelog.test.ts` — add `runClaude` to the dep
|
||||
injection surface, stub it in unit tests, add new coverage for
|
||||
failure modes and mode='changelog' vs mode='release-notes'.
|
||||
- `.github/workflows/release.yml` (or wherever the auto-build lives)
|
||||
— remove the auto-`changelog:build` fallback; add a guard that
|
||||
fails the run if `changes/*.md` exist on a tag-based release.
|
||||
- `src/release-workflow.test.ts`,
|
||||
`src/prerelease-workflow.test.ts` — update if they exercise the
|
||||
CI auto-run path.
|
||||
- `docs/RELEASING.md` — document the new local-only build step and
|
||||
the `claude` PATH requirement.
|
||||
- `changes/README.md` — note that fragments will be merged and
|
||||
rewritten, so authors should write raw notes rather than polished
|
||||
prose.
|
||||
- `changes/<n>-ai-changelog-polish.md` — fragment for this change
|
||||
itself (`type: internal`, `area: release`).
|
||||
|
||||
### New function: `polishFragmentsWithClaude`
|
||||
|
||||
```
|
||||
polishFragmentsWithClaude(
|
||||
fragments: ChangeFragment[],
|
||||
options: {
|
||||
mode: 'changelog' | 'release-notes',
|
||||
version: string,
|
||||
date?: string, // changelog mode only
|
||||
deps?: { runClaude?: (prompt: string) => string },
|
||||
},
|
||||
): string
|
||||
```
|
||||
|
||||
- Filters out `internal` fragments when `mode === 'release-notes'`.
|
||||
- Serializes the surviving fragments into the prompt format below.
|
||||
- Invokes `claude` (via `deps.runClaude` or the default
|
||||
`execFileSync` wrapper).
|
||||
- Validates the output contains the expected section headers; throws
|
||||
on failure.
|
||||
- Returns the Markdown body that will be inserted under the
|
||||
`## v<version>` heading.
|
||||
|
||||
The existing `prependReleaseSection`, `extractReleaseSectionBody`,
|
||||
`writeReleaseNotesFile`, and `generateDocsChangelog` functions stay
|
||||
exactly as they are — they consume the polished body the same way
|
||||
they consumed the rendered body.
|
||||
|
||||
### Prompt input format
|
||||
|
||||
```
|
||||
MODE: changelog | release-notes
|
||||
VERSION: 0.13.0
|
||||
DATE: 2026-05-02 (changelog mode only)
|
||||
|
||||
FRAGMENT changes/291-character-dictionary-selection.md
|
||||
type: added
|
||||
area: dictionary
|
||||
breaking: false
|
||||
- Added CLI and in-app AniList selection for character dictionary mismatches...
|
||||
- Added launcher support through `subminer dictionary --candidates`...
|
||||
|
||||
FRAGMENT changes/293-interjection-annotation-filter.md
|
||||
type: fixed
|
||||
area: tokenizer
|
||||
breaking: false
|
||||
- Stopped standalone `あ` interjections from receiving subtitle annotation metadata...
|
||||
|
||||
...
|
||||
```
|
||||
|
||||
### Prompt output contract
|
||||
|
||||
Claude is instructed to emit Markdown only, no preamble, conforming to:
|
||||
|
||||
- Sections in this order, omitting empty ones:
|
||||
`### Breaking Changes`, `### Added`, `### Changed`,
|
||||
`### Fixed`, `### Docs`.
|
||||
- For `mode === 'changelog'`, an additional terminal section:
|
||||
|
||||
```
|
||||
<details>
|
||||
<summary>Internal changes</summary>
|
||||
|
||||
### Internal
|
||||
- …
|
||||
- …
|
||||
|
||||
</details>
|
||||
```
|
||||
|
||||
- Bullets lead with a feature name (e.g. `Playlist browser:`,
|
||||
`Windows overlay:`) — not with the raw `area:` slug.
|
||||
- Bullets may merge multiple fragments, but breaking changes must
|
||||
retain their substance and stay in `### Breaking Changes`.
|
||||
- Bullets that document only PR-housekeeping or CodeRabbit follow-ups
|
||||
are dropped unless they describe a user-visible behavior change.
|
||||
|
||||
After parsing, the body is passed unchanged to `prependReleaseSection`,
|
||||
which wraps it under the standard `## v<version> (<date>)` heading.
|
||||
|
||||
### Determinism and review
|
||||
|
||||
Claude is non-deterministic. The mitigation is the existing release
|
||||
workflow: the polished CHANGELOG/release-notes are committed before
|
||||
tagging, so what's in the repo is what ships. The release engineer
|
||||
diffs the commit before pushing.
|
||||
|
||||
## Testing
|
||||
|
||||
`scripts/build-changelog.test.ts` already exposes a `deps` injection
|
||||
seam for fs operations. Extend it with `runClaude?: (prompt: string)
|
||||
=> string`. Tests stub it to return canned Markdown; the default
|
||||
implementation (used in production) wraps `execFileSync('claude', …)`.
|
||||
|
||||
New cases to cover:
|
||||
|
||||
- Mode `'release-notes'` filters `internal` fragments before sending
|
||||
to Claude.
|
||||
- Mode `'changelog'` includes `internal` fragments and the
|
||||
`<details>` wrapper expectation is propagated through to the prompt.
|
||||
- Missing `claude` binary throws a clear error.
|
||||
- Non-zero exit code from `claude` throws.
|
||||
- Empty / whitespace-only output throws.
|
||||
- Output missing the expected section headers throws.
|
||||
- Prerelease path uses release-notes mode and writes
|
||||
`release/prerelease-notes.md` with the existing disclaimer.
|
||||
- The polished body still flows correctly through
|
||||
`prependReleaseSection`, `extractReleaseSectionBody`, and
|
||||
`generateDocsChangelog`.
|
||||
|
||||
`release-workflow.test.ts` and `prerelease-workflow.test.ts` are
|
||||
updated to cover the new "fragments pending on a tag" failure mode.
|
||||
|
||||
## Documentation
|
||||
|
||||
- `docs/RELEASING.md` updates the stable and prerelease sections to
|
||||
state that `changelog:build` and `changelog:prerelease-notes` invoke
|
||||
`claude` locally, list the PATH requirement, and remove the note
|
||||
about CI auto-running `changelog:build`.
|
||||
- `changes/README.md` adds a short note: fragments will be merged and
|
||||
rewritten by Claude during release, so write raw notes — don't
|
||||
polish them.
|
||||
|
||||
## Out of Scope (YAGNI)
|
||||
|
||||
- No caching of polished output between runs.
|
||||
- No diff-and-confirm interactive prompt.
|
||||
- No SDK fallback if the CLI is missing.
|
||||
- No `--no-polish` escape hatch. Revert the commit if you genuinely
|
||||
need the legacy renderer back.
|
||||
- No change to the fragment authoring schema or the `pr-check`
|
||||
enforcement.
|
||||
|
||||
## Open Risks
|
||||
|
||||
- **Claude regresses on a release.** Mitigated by the
|
||||
commit-before-tag review step. If the polish is bad, the release
|
||||
engineer edits `CHANGELOG.md` directly and re-commits before
|
||||
tagging.
|
||||
- **`--bare` mode behavior changes upstream.** Pinning to `--model
|
||||
sonnet` and `--bare` reduces drift, but Claude Code is still an
|
||||
external tool. If invocation flags change, the release engineer
|
||||
will see a hard failure and can patch the script.
|
||||
- **CI surface area increases.** Removing the auto-`changelog:build`
|
||||
fallback means a tag pushed before running the local build will
|
||||
fail CI. The error message must clearly point at the fix.
|
||||
Reference in New Issue
Block a user