The greedy name pre-pass asked the Yomitan backend at every Japanese
position, because a character name can begin mid-token. With the character
dictionary enabled that roughly doubled the round trips per line (measured
10 -> 21 on a 23-char line).
SubMiner generates the character dictionary, so the cached snapshots already
list every form a character entry can be matched by (term and reading). Those
forms are installed into the scan runtime once per media and the pre-pass now
probes only positions where one of them starts, compared after kana
normalization so a katakana name still matches a hiragana reading form. The
overhead drops to zero (21 -> 10, the same as with the dictionary disabled).
Fail-safe: with no candidate list (no media id, no cached snapshot, failed
install) the pre-pass keeps its exhaustive behavior, so stale character data
costs speed rather than a missing name. Halfwidth katakana positions bypass
the filter since kana normalization does not fold them.
The candidate lookup is consulted per subtitle line, so it caches its snapshot
directory signature for 5s; dictionary writes still call invalidate().
- extract the injected scan runtime (helpers, install script, call-script
builder) into tokenizer/yomitan-scan-runtime-script.ts; the host module drops
from ~2700 to ~1900 lines
- append the kana run to the reading as well as the surface when an unparsed
run extends the previous token, so the reading keeps covering the surface and
the known-word reading fallback stays enabled (bumps scan runtime version)
- stop annotateMs before character-image resolution so the stage timing
measures the annotation stage only
- drop the duplicate parseText full parse per line; the termsFind scanner walk
is now authoritative and emits its own unparsed filler runs (parseText kept
only as error fallback)
- install scan helpers once per parser window (__subminerYomitanScan) instead
of re-shipping ~500 lines of script per subtitle line
- persist termsFind results across lines in a window-scoped LRU keyed by
substring, invalidated via a cache epoch on dictionary/settings changes
- skip lookups at punctuation/whitespace positions and cap the shrinking-window
retry ladder at 4 lookups per position
- build tokenizer runtime deps once (JLPT lookup cache never hit before; mecab
availability check ran per line)
- stop restarting the prefetch run on every subtitle change; resume prefetch
only after the tokenized payload lands, not on provisional raw emits
- add per-stage debug timings (scanMs/mecabMs/frequencyMs/annotateMs)
sudacode
