fix(subtitles): widen halfwidth katakana where subtitle text enters the app (#295)

* fix(subtitles): widen halfwidth katakana where subtitle text enters the app

Broadcast-derived Japanese subs write katakana halfwidth (アンタ, ダキニ), so
overlay lines mixed squashed and normal-width text. Normalize live mpv
sub-text and parsed cue text to fullwidth so display, lookups, mining,
and stats all see one form. The tokenizer reuses the shared helper.

* fix(subtitles): fold stray voiced marks and widen before cue dedup

- A lone halfwidth ゙ after fullwidth kana (ガ) now composes to ガ
  instead of leaving a combining mark that broke name matching.
- Parsed cue text is widened before duplicate merging so ガク and ガク
  on the same span collapse into one sidebar row.
- The YouTube track-load refresh widens the sub-text it reads directly.

* fix(stats): widen halfwidth katakana in stored lines and mined cards (#296)

Lines recorded before ingest widening still hold halfwidth katakana.
Occurrence and sentence-search queries widen line text on read, and the
mine-card route widens the sentence and word it receives, so the
dashboard and cards mined from history match newer rows. Word identity
columns are left as stored since the dashboard queries by them.

* fix(stats): match katakana in either width in sentence search

- Add narrowFullwidthKana, derived from widenHalfwidthKana so the two stay in sync
- Match subtitle text against the original, widened, and narrowed forms of each term so a search finds older lines stored as halfwidth
This commit is contained in:
sudacode authored and GitHub committed 2026-10-13 22:26:01 -07:00
1 parent bf4e5bb609
commit 01613f1be5
16 files changed
+285 -26

No files matched your search

@@ -7,6 +7,7 @@ import { resolveAnimatedImageLeadInSeconds } from '../../../anki-integration/ani
import { clampMediaEndTime } from '../../../anki-integration/media-duration.js';
import { MediaGenerator } from '../../../media-generator.js';
import { statsJson } from '../../../types/stats-http-contract.js';
import { widenHalfwidthKana } from '../../utils/halfwidth-kana.js';
import {
resolveRetimedSecondarySubtitleTextFromSidecar,
resolveSecondarySubtitleTextFromSidecar,
@@ -35,8 +36,11 @@ export function registerStatsMiningRoutes(app: Hono, options?: StatsMiningRouteO
const sourcePath = typeof body?.sourcePath === 'string' ? body.sourcePath.trim() : '';
const startMs = typeof body?.startMs === 'number' ? body.startMs : NaN;
const endMs = typeof body?.endMs === 'number' ? body.endMs : NaN;
const sentence = typeof body?.sentence === 'string' ? body.sentence.trim() : '';
const word = typeof body?.word === 'string' ? body.word.trim() : '';
// Older stats rows can carry halfwidth katakana; cards get the fullwidth form that
// live mining produces, and the word is widened alongside so highlighting still hits.
const sentence =
typeof body?.sentence === 'string' ? widenHalfwidthKana(body.sentence.trim()) : '';
const word = typeof body?.word === 'string' ? widenHalfwidthKana(body.word.trim()) : '';
const bodySecondaryText =
typeof body?.secondaryText === 'string' ? body.secondaryText.trim() : '';
const videoTitle = typeof body?.videoTitle === 'string' ? body.videoTitle.trim() : '';