feat(release): prepare v0.19.5 with subtitle pipeline improvements

- Rework ASS, YouTube, overlapping, and secondary subtitle handling
- Keep overlay, mining, sidebar, and stats aligned with visible cues
- Harden release-note validation and package Linux thumbnail support
This commit is contained in:
2026-09-02 09:27:06 -07:00
278 changed files with 16719 additions and 1837 deletions
@@ -69,17 +69,26 @@ interface SubtitleCue {
startTime: number; // seconds
endTime: number; // seconds
text: string; // plain text, decoded from the source format
source?: 'canonical-ass'; // recovered authored text for generated ASS animation
animationStartTime?: number; // full generated-frame envelope; entrance/exit frames
animationEndTime?: number; // run past the authored timing, live matching uses this
}
```
**Supported formats:**
- SRT/VTT: Regex-based parsing of timing lines + text content between timing blocks.
- ASS: Parse `[Events]` section, extract `Dialogue:` lines, read the field order from the `Format:` row, and take everything after the Text field index as the text (Text can itself contain commas).
- ASS: Parse the `[Events]` section, read the field order from the `Format:` row, and extract timed `Dialogue:` lines. Timed `Comment:` lines are normally ignored, but can supply canonical authored text when they match a nearby generated animation from the same style and actor. Text can itself contain commas.
**ASS decoding.** The parser is where ASS text is decoded, once, via `assToPlainText()` in `src/core/services/ass-text.ts`. That decoder mirrors mpv's `ass_to_plaintext` so a cue read from a file reads identically to the same line arriving live on `sub-text`: `{...}` override blocks are markup, `\pN … \p0` vector drawing runs are dropped rather than shown as text, `\N`/`\n`/`\h` are the only escapes (`\{`, `\}` and `\\` are not), and an unclosed `{` is rendered verbatim. Every layer downstream — renderer, timing tracker, tokenizer, tokenization cache keys — receives plain text and uses `normalizePlainSubtitleText()` for whitespace only, so nothing decodes the same string twice and one authored line always maps to one cache key.
**Duplicate collapsing.** Typeset scripts emit one `Dialogue:` event per animation frame, plus layered copies of the same line. The parser collapses identical text over an identical span unconditionally, and collapses contiguous same-text runs of at least three events when the run looks like an animation. For ASS that means shared style and actor plus authoring evidence: a temporal tag (`\t`, `\move`, `\k`/`\kf`/`\ko`/`\K`, or anything wrapped in `\t(...)`), an animated `Effect` column (`Karaoke`, `Banner`, `Scroll`), or override values that change across the run. Static tags shared by every event (`\pos`, an identical `\clip`) are not evidence. SRT/VTT carry no such metadata, so there collapsing needs at least five contiguous events all under 0.1s — the frame timing left behind by ASS-to-SRT conversion. The parser keeps this authoring metadata (style, actor, layer, `Effect`, parsed override commands, source order) private; `parseSubtitleCues()` returns only `SubtitleCue`.
ASS scripts can also redraw one complete lyric for two or more long color/highlight phases. Those flush-timed phases collapse separately from short animation frames when they share text, style, actor, and layer and carry direct animation evidence, such as temporal tags or changing non-spatial overrides. Spatial command changes do not prove a phase, so separately positioned signs remain distinct.
**Canonical animation recovery.** Some ASS producers keep the readable lyric or sign as a timed `Comment:` and generate hundreds of `Dialogue:` frames containing repeated glyphs or changing clip regions. Others retain the complete line as brief `Dialogue:` events around the generated fragments. A complete event is promoted only when nearby dialogue from the same style and actor forms a proven animation cluster and reconstructs its entire text in source order. The generated frames are then replaced by one cue marked `source: 'canonical-ass'`. This source marker lets the live primary-subtitle path prefer the clean authored text and timing for display, sidebar history, immersion recording, and mining, while unmatched editor notes and alternative translations remain ignored. Secondary selection advances to an entering canonical cue at its generated animation start when the preceding authored cue ends before the new authored span. Unrelated simultaneous cues that continue through the new span remain visible.
**Font texture cleanup.** A clipped repeated-glyph run or frequent changes to secondary alpha marks a texture seed. Clipped runs do not need a font override because some signs build their masks from ordinary `l` glyphs. The parser removes short clipped pieces that share a no-font seed's style and timing, or pieces that share a font seed's style, timing, and font even when the actor changes. It also removes positioned text layers with at least `E0` global alpha when they overlap a seed in the same style. Opaque authored sign text stays publishable when the texture switches fonts or actors around it.
#### Prefetch Service Lifecycle
1. **Activation trigger:** When a subtitle track is activated (or changes), check if it's external via MPV's `track-list` property. If `external === true`, read the file via `external-filename` using the existing `loadSubtitleSourceText` infrastructure.
+3 -1
View File
@@ -23,7 +23,9 @@ Trend charts now consume one chart-oriented backend payload from `/api/stats/tre
- lookup rate trends
- watch-time by day-of-week/hour
- vocabulary-backed:
- new-words trend
- new-words trend reads permanent daily lexical rollups
- rollup rows count only vocabulary-visible tokens and normalize mixed legacy timestamp units
- a persisted rollup version invalidates stale materializations and triggers an atomic background rebuild
## Metric Semantics
+91 -4
View File
@@ -3,7 +3,7 @@
# Subtitle Overlay Priming
Status: active
Last verified: 2026-08-04
Last verified: 2026-08-19
Owner: Kyle Yasuda
Read when: debugging subtitle state or blank Linux/X11 overlay windows when the visible overlay is shown or recreated
@@ -69,14 +69,97 @@ coming and prefetching would otherwise idle for the rest of the cue.
## Live Cue Delivery
- Primary live text first resolves recovered canonical ASS animations. Otherwise, when
every live mpv line matches an active parsed cue, it uses the parsed cue text so exact
full-span style layers appear once instead of repeating for fill, border, blur, shadow,
or equivalent whitespace variants. Any unmatched live line keeps the complete live
stack, preserving dialogue or signs that overlap a lyric.
- A tokenization cache miss emits the plain cue synchronously. Tokenization remains serialized so
live work does not contend for Yomitan state.
- The initial `time-pos`, explicit renderer seeks, and later seek-like jumps reprocess mpv's
current raw `sub-text` after the new playback time is stored. Explicit intent matters because
adjacent subtitle jumps can be shorter than the general seek-distance threshold. This corrects
ASS cleanup when mpv delivered the destination subtitle before the destination timestamp.
- Renderer `sub-seek` commands use the active parsed cue list when available. Simultaneous cues
share one boundary, overlapping lyrics advance from the latest active boundary, and mpv's native
command remains the fallback when no parsed destination exists. This prevents generated karaoke
frames from consuming next/previous subtitle presses.
- Subtitle sidebar selections seek past the preceding sanitized cue's overlapping exit span when
the selected cue has enough time remaining. This keeps direct row selection on the requested
karaoke line while clamping the seek inside that cue.
- If startup paints raw text before embedded ASS parsing finishes, parsed cue arrival may replace
that provisional line. The one-prime-per-media guard still suppresses identical repeats.
- If a newer cue arrives while an older line is still tokenizing, the newer plain cue or empty
clear payload is emitted immediately. The older tokenization result is dropped before it can
replace the current cue.
- The current cue upgrades in place when its tokens and annotations are ready. This can reflow text
or character images, but cue visibility does not wait for that work.
## Secondary Subtitle Flow
- `secondary-sub-text` remains the immediate fallback, so unreadable subtitle sources, remote URLs,
and still-extracting embedded tracks appear without waiting for file resolution. Embedded-track
extraction runs for local and network-mounted files alike (demuxing reads the whole container,
about 10 seconds per GB on gigabit, under a generous timeout); only true remote URLs skip it,
having no on-disk container to demux.
- The live fallback also suppresses per-glyph typesetting walls: when many simultaneous
one-glyph lines are present (generated karaoke lettering flattened into live text), those
lines and their short syllable companions are dropped while concurrent dialogue lines stay.
This keeps the overlay clean while extraction is still in flight and for sources that never
produce parsed cues.
- Parsed secondary text and the live fallback remove exact repeated lines at any length. A
flattened-line identity also removes long dialogue/sign repetitions that differ only in
whitespace or terminal punctuation, while distinct simultaneous short lines remain separate.
- `secondary-subtitle-track.ts` resolves `secondary-sid` against mpv's track list. External tracks
are read directly; supported embedded text tracks are extracted through the same ffmpeg-backed
source resolver used by primary subtitle prefetching.
- The selected source is parsed with `parseSubtitleCues()`, including metadata-aware ASS duplicate
and animation collapse. Playback `time-pos` selects the active parsed cue after applying
`secondary-sub-delay`.
- Fragment reconstruction marks tall multi-row positioned parts as a grid only when they read
like tiling: a couple of texts repeated across many fragments, the same text re-shown at one
spot over time (countdown/animation frames), or scattered single glyphs. Secondary text omits
those grids instead of flattening a translated table or schedule into one synthetic line.
Wrapped lyric rows, CC-style dialogue blocks, and reconstructed single-line karaoke remain
eligible for display.
- The resolved text is stored in `mpvClient.currentSecondarySubText` before it is broadcast. The
overlay, mining, timing tracker, and immersion statistics therefore consume the same secondary
text when a readable source is available.
- Simultaneous parsed cues use whitespace-insensitive identity, so ASS layers that vary only
between ordinary, hard, or ideographic spaces appear once.
- Simultaneous ASS lines are flattened in top-to-bottom positioned order, falling back to their
authored source order when no usable position exists.
- Half-size kana positioned directly above a same-timed kanji caption is treated as ASS
furigana. The parser omits it from published cues but retains hidden matching metadata so
mpv's raw live text can be reconciled without displaying or mining the reading. The
timing tracker (clipboard copy, recent-line mining) and immersion recorders run the same
reconciliation on the `sub-start`/`sub-end` sample, so they record what the overlay shows.
- Broadcast-caption rows that spell one utterance across several same-timed positioned events
(same style, layer, and vertical band, stacked at most two text rows apart) are joined into
one cue with a single line break, so `preserveLineBreaks` treats them like an authored `\N`,
and the recorders above see the whole sentence. A row continues the one above it when that row
is a bare speaker label, ends without terminal punctuation, or leaves a ≪…≫ / ⸨…⸩ span open; a
lower row that opens its own label or span always starts a new cue, which keeps two speakers
sharing the screen on separate lines. The pass runs only on scripts that read as broadcast
captions (a meaningful share of events carry speaker labels or ≪…≫ / ⸨…⸩ spans) and only on
rows containing Japanese, because fansub typesetting stacks positioned rows for signs, chat
bubbles, and headlines where that punctuation convention does not hold.
- Fragment-only ASS karaoke is reconstructed per style before publication. Explicit spaces
survive concatenation. Latin fragment typesetting with no literal spaces also recovers word
boundaries represented only by materially larger horizontal `\pos` or `\move` gaps within that
line. Unpositioned fragments stay compact instead of gaining guessed spaces between syllables.
Short runs qualify only when overlapping positioned events also show changing overrides or
repeated layer copies; an English or romaji style name alone never turns ordinary dialogue into
a lyric.
- Recovered canonical ASS text remains active for the generated animation envelope. For
reconstructed lyric styles, the longest-lived active line wins over brief entrance and exit
fragments from the same style.
- Media and `secondary-sid` changes clear the previous parsed state before refreshing the source;
track-list changes refresh without discarding an unchanged source. Observed
`secondary-sub-delay` changes retime the active parsed cue without rereading the file. If loading,
extraction, or parsing fails, the controller returns to live mpv text and the renderer's
conservative short stack heuristic remains the final display fallback.
## Emitted State
- `emitSubtitle(payload)` maps to `emitSubtitlePayload(payload)`. Overlay windows and annotation
@@ -84,8 +167,8 @@ coming and prefetching would otherwise idle for the rest of the cue.
- The basic subtitle websocket receives the immediate plain cue only. Because its serialized
payload discards annotations, the later upgrade would be an identical duplicate and is skipped
when text and cue timing match.
- Secondary priming reads mpv `secondary-sub-text`, stores it in
`mpvClient.currentSecondarySubText`, and broadcasts `secondary-subtitle:set` to overlay windows.
- Secondary priming reads mpv `secondary-sub-text` and routes it through the secondary track
controller. A parsed active cue replaces the live text when the selected source is readable.
- If secondary `requestProperty` fails, the primary flow stays complete and only a debug line is
written.
@@ -129,7 +212,11 @@ coming and prefetching would otherwise idle for the rest of the cue.
path, empty or stale bounding shapes produced invisible or clipped subtitles even though the
overlay window remained mapped above mpv.
- Pointer pass-through should continue to use `setIgnoreMouseEvents(true, { forward: true })` and
the Linux cursor-poll fallback, not bounding-shape clipping.
the Linux cursor-poll fallback, not bounding-shape clipping. Note that on Windows click-through
must go through `applyOverlayClickThrough()` (`src/core/services/overlay-click-through.ts`),
which omits `forward: true` there: Electron implements forwarding with a global low-level mouse
hook that lags mouse input system-wide whenever the main thread stalls; the Windows cursor poll
handles overlay wake-up instead.
- Visible-overlay show/reset marks Linux pointer passthrough state dirty even when the logical
interaction state is already inactive. The next cursor-poll tick must still reapply
`setIgnoreMouseEvents(true, { forward: true })`; otherwise a newly shown Electron overlay can keep