SubMiner/changes/name-match-span-merge.md at db4139ba0b9531edc662d39c4b1152fc86e9014b

sudacode/SubMiner

Fork 0

mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-07-27 04:49:49 -07:00

Files

T

sudacode a2e49b369b

fix(tokenizer): merge scanner metadata per token instead of all-or-nothi

- Replace hasSameTokenSpans + full-discard with mergeScannerTokensIntoParseTokens
- Grafts isNameMatch/frequencyRank/etc onto matching parse spans; filler chunks degrade only themselves
- Fixes name annotations dropping for entire subtitle lines containing unmatched interjections

2026-07-08 22:26:34 -07:00

341 B

Raw Blame History

type: fixed area: overlay

Fixed character-name annotations dropping for an entire subtitle line when it contained any chunk the dictionary scanner could not match (e.g. an interjection like やほっ before a name): scanner metadata is now merged per token into the parseText segmentation instead of being discarded on any span mismatch.

341 B Raw Blame History

341 B

Raw Blame History