Compare commits

..

13 Commits

Author SHA1 Message Date
sudacode 3c24597724 fix(character-dictionary): source name parts from alias, not blank nativ
- test: exercise the case where nativeName is empty and parts must come from alternativeNames' space split instead
2026-08-04 17:58:39 -07:00
sudacode d878d8bf4f fix(character-dictionary): drop single-letter mob name disambiguators
- AniList mob names like 女子A/"Joshi A" romanize their trailing
  disambiguator letter into a single kana term (A → ア) that collides
  with common interjections (あ〜); filter these out unless kanji
- bump CHARACTER_DICTIONARY_FORMAT_VERSION to 20 to invalidate cached
  dictionaries built with the old term set
- add tests covering the dropped disambiguator and a kept single-kanji
  name part
2026-08-04 17:42:29 -07:00
sudacode 8683961967 placeholder
{"subject": "test(subtitles): fix tests for provisional emit and payload signature", "body": "- Expect the provisional plain-text emit alongside the refresh's empty emit\n- Match emitSubtitlePayload regex against its updated parameter signature"}
2026-08-04 02:53:33 -07:00
sudacode c9baaeea17 fix(subtitles): re-annotate invalidated text during autoplay priming
Autoplay priming called onSubtitleChange after a cache miss, which only asks
whether the text is new. When the miss came from an invalidation (mining a
card while the line is on screen) the text was unchanged, so nothing was
scheduled and the line stayed unannotated for as long as it was displayed.
refreshCurrentSubtitle checks the cache generation as well, so it re-tokenizes
for the new generation; the resume fallback is kept for the case where it
genuinely has nothing to do.

refreshCurrentSubtitle also returned false for empty text while a run was in
flight, even though that run goes on to emit the empty subtitle. It now
reports the pending emit so callers do not release the prefetch pause early.

The priming tests now drive the real subtitle processing controller instead of
a stub. The previous stub encoded the wrong assumption about unchanged text
and so could not catch either bug.
2026-08-04 01:56:44 -07:00
sudacode 2003efa235 fix(subtitles): release prefetch pause across all priming paths
The repeated-subtitle pause leak was only fixed for ordinary subtitle
changes. Startup autoplay priming and visible-overlay priming pause the same
way and also ignored whether any tokenization was scheduled, so a cache miss
on text the controller already holds (mining a card while the line is on
screen) left prefetching idle for the rest of the cue.

Pause and release are now one operation via pausePrefetchUntilEmit, and both
controller entry points report whether an emit is expected. A repeat arriving
while a run is already in flight keeps the pause, since that run still emits.

Also invalidate the character dictionary lookups centrally from the sync
completion handler instead of at three manager call sites. Ordinary selection
sync never invalidated them, so a stale non-null name candidate list could
skip a newly added name for up to five seconds; a missing list falls back to
the exhaustive scan, but a stale one does not. Ordering matters: the
invalidation runs before the subtitle refreshes so they re-tokenize against
the new dictionary content.

Docs: subtitle-overlay-priming no longer claims every subtitle change calls
onSeek().
2026-08-04 01:56:44 -07:00
sudacode f43674cc39 fix(subtitles): release prefetch pause on repeated subtitle events
onSubtitleChange paused prefetching unconditionally, but the processing
controller returns early when the text matches what it already has. Nothing
is tokenized, so nothing is emitted, so the resume that rides on the emit
never fires and prefetching idles for the rest of the cue. The reachable
trigger is a repeat arriving after a cache invalidation, such as mining a card
while the same line is still on screen.

The controller now reports whether it scheduled processing and the caller
resumes when it did not, so every pause has a matching resume.

Also harden the character-name candidate prefilter: Yomitan collapses emphatic
sequences before matching, so a stretched spelling still resolves to its entry
(ミナァァト matches ミナト). The candidate match now skips small kana and
prolonged marks, which only widens the probe set and so cannot drop a name.
2026-08-04 01:56:44 -07:00
sudacode b0a2ce6e8a perf(tokenizer): skip character-name lookups where no name can start
The greedy name pre-pass asked the Yomitan backend at every Japanese
position, because a character name can begin mid-token. With the character
dictionary enabled that roughly doubled the round trips per line (measured
10 -> 21 on a 23-char line).

SubMiner generates the character dictionary, so the cached snapshots already
list every form a character entry can be matched by (term and reading). Those
forms are installed into the scan runtime once per media and the pre-pass now
probes only positions where one of them starts, compared after kana
normalization so a katakana name still matches a hiragana reading form. The
overhead drops to zero (21 -> 10, the same as with the dictionary disabled).

Fail-safe: with no candidate list (no media id, no cached snapshot, failed
install) the pre-pass keeps its exhaustive behavior, so stale character data
costs speed rather than a missing name. Halfwidth katakana positions bypass
the filter since kana normalization does not fold them.

The candidate lookup is consulted per subtitle line, so it caches its snapshot
directory signature for 5s; dictionary writes still call invalidate().
2026-08-04 01:56:44 -07:00
sudacode 030c94934e refactor(tokenizer): address review feedback on Yomitan scan runtime
- extract the injected scan runtime (helpers, install script, call-script
  builder) into tokenizer/yomitan-scan-runtime-script.ts; the host module drops
  from ~2700 to ~1900 lines
- append the kana run to the reading as well as the surface when an unparsed
  run extends the previous token, so the reading keeps covering the surface and
  the known-word reading fallback stays enabled (bumps scan runtime version)
- stop annotateMs before character-image resolution so the stage timing
  measures the annotation stage only
2026-08-04 01:56:44 -07:00
sudacode e7cef039f3 perf(tokenizer): single-pass Yomitan scan with install-once runtime and cross-line cache
- drop the duplicate parseText full parse per line; the termsFind scanner walk
  is now authoritative and emits its own unparsed filler runs (parseText kept
  only as error fallback)
- install scan helpers once per parser window (__subminerYomitanScan) instead
  of re-shipping ~500 lines of script per subtitle line
- persist termsFind results across lines in a window-scoped LRU keyed by
  substring, invalidated via a cache epoch on dictionary/settings changes
- skip lookups at punctuation/whitespace positions and cap the shrinking-window
  retry ladder at 4 lookups per position
- build tokenizer runtime deps once (JLPT lookup cache never hit before; mecab
  availability check ran per line)
- stop restarting the prefetch run on every subtitle change; resume prefetch
  only after the tokenized payload lands, not on provisional raw emits
- add per-stage debug timings (scanMs/mecabMs/frequencyMs/annotateMs)
2026-08-04 01:56:44 -07:00
sudacode fe4dacc1e7 fix(overlay): show plain subtitle line immediately on tokenization cache miss (#184) 2026-08-04 01:55:52 -07:00
sudacode b08cd0db35 fix(streaming): keep subtitle tokenization prefetch warm for full episodes (#183) 2026-08-03 21:22:18 -07:00
sudacode bffb1c5982 fix(logging): surface subtitle processing debug/warn logs (#182) 2026-08-03 20:44:39 -07:00
sudacode 5b8848518a feat(subsync): add reference and target subtitle track picker (#181) 2026-08-03 01:00:14 -07:00
69 changed files with 4757 additions and 1984 deletions
+7 -9
View File
@@ -26,7 +26,7 @@
"eslint": "^10.8.0",
"prettier": "^3.8.1",
"typescript": "^5.9.3",
"undici": "7.28.0",
"undici": "7.29.0",
},
},
},
@@ -36,8 +36,9 @@
"overrides": {
"@xmldom/xmldom": "0.8.13",
"app-builder-lib": "26.15.3",
"brace-expansion": "5.0.8",
"brace-expansion": "5.0.9",
"electron-builder-squirrel-windows": "26.15.3",
"fast-uri": "3.1.5",
"form-data": "4.0.6",
"ip-address": "10.2.0",
"js-yaml": "4.3.0",
@@ -46,6 +47,7 @@
"picomatch": "4.0.4",
"tar": "7.5.21",
"tmp": "0.2.7",
"undici": "7.29.0",
},
"packages": {
"@discordjs/collection": ["@discordjs/collection@2.1.1", "", {}, "sha512-LiSusze9Tc7qF03sLCujF5iZp7K+vRNEDBZ86FT9aQAv3vxMLihUvKvpsCWiQ2DJq1tVckopKm1rxomgNUc9hg=="],
@@ -266,7 +268,7 @@
"boolean": ["boolean@3.2.0", "", {}, "sha512-d0II/GO9uf9lfUHH2BQsjxzRJZBdsjgsBiW4BvhWk/3qoKwQFjIDVN19PfX8F2D/r9PCMTtLWjYVCFrpeYUzsw=="],
"brace-expansion": ["brace-expansion@5.0.8", "", { "dependencies": { "balanced-match": "^4.0.2" } }, "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg=="],
"brace-expansion": ["brace-expansion@5.0.9", "", { "dependencies": { "balanced-match": "^4.0.2" } }, "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg=="],
"buffer-from": ["buffer-from@1.1.2", "", {}, "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ=="],
@@ -404,7 +406,7 @@
"fast-levenshtein": ["fast-levenshtein@2.0.6", "", {}, "sha512-DCXu6Ifhqcks7TZKY3Hxp3y6qphY5SJZmrWMDrKcERSOXWQdMhU9Ig/PYrzyw/ul9jOIyh0N4M0tbC5hodg8dw=="],
"fast-uri": ["fast-uri@3.1.4", "", {}, "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw=="],
"fast-uri": ["fast-uri@3.1.5", "", {}, "sha512-gHwA1O9LDIcKunMKhObS/HimwtehO1nPUECKAu5TpKgaO19fcWEl4bliWe1jWxVFvIXztJjjQ4L8XQ1EU9f7Jw=="],
"fdir": ["fdir@6.5.0", "", { "peerDependencies": { "picomatch": "^3 || ^4" }, "optionalPeers": ["picomatch"] }, "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg=="],
@@ -714,7 +716,7 @@
"typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
"undici": ["undici@7.28.0", "", {}, "sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA=="],
"undici": ["undici@7.29.0", "", {}, "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw=="],
"undici-types": ["undici-types@7.16.0", "", {}, "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw=="],
@@ -754,8 +756,6 @@
"@discordjs/rest/discord-api-types": ["discord-api-types@0.38.43", "", {}, "sha512-sSoBf/nK6m7BGtw65mi+QBuvEWaHE8MMziFLqWL+gT6ME/BLg34dRSVKS3Husx40uU06bvxUc3/X+D9Y6/zAbw=="],
"@discordjs/rest/undici": ["undici@6.27.0", "", {}, "sha512-YmfV3YnEDzXRC5lZ2jWtWWHKGUm1zIt8AhesR1tens+HTNv+YZlN/dp6G727LOvMJ8xjP9Be7Y2Sdr96LDm+pg=="],
"@discordjs/util/discord-api-types": ["discord-api-types@0.38.43", "", {}, "sha512-sSoBf/nK6m7BGtw65mi+QBuvEWaHE8MMziFLqWL+gT6ME/BLg34dRSVKS3Husx40uU06bvxUc3/X+D9Y6/zAbw=="],
"@electron/asar/commander": ["commander@5.1.0", "", {}, "sha512-P0CysNDQ7rtVw4QIQtm+MRxV66vKFSvlsQvGYXZWR3qFU0jlMKHZZZgw8e+8DSah4UDKMqnknRDQz+xuQXQ/Zg=="],
@@ -808,8 +808,6 @@
"node-gyp/env-paths": ["env-paths@2.2.1", "", {}, "sha512-+h1lkLKhZMTYjog1VEpJNG7NZJWcuc2DDk/qsqSTRRCOXiLjeQ1d1/udrUGhqMxUgAlwKNZ0cf2uqan5GLuS2A=="],
"node-gyp/undici": ["undici@6.27.0", "", {}, "sha512-YmfV3YnEDzXRC5lZ2jWtWWHKGUm1zIt8AhesR1tens+HTNv+YZlN/dp6G727LOvMJ8xjP9Be7Y2Sdr96LDm+pg=="],
"node-gyp/which": ["which@6.0.1", "", { "dependencies": { "isexe": "^4.0.0" }, "bin": { "node-which": "bin/which.js" } }, "sha512-oGLe46MIrCRqX7ytPUf66EAYvdeMIZYn3WaocqqKZAxrBpkqHfL/qvTyJ/bTk5+AqHCjXmrv3CEWgy368zhRUg=="],
"pkijs/@noble/hashes": ["@noble/hashes@1.4.0", "", {}, "sha512-V1JJ1WTRUqHHrOSh597hURcMqVKVGL/ea3kv0gSnEdsEZ0/+VyPghM1lMNGc00z7CIQorSvbKpuJkxvuHbvdbg=="],
+4
View File
@@ -0,0 +1,4 @@
type: fixed
area: logging
- Background startup now respects the configured logging level when `--log-level` is not explicitly provided.
@@ -0,0 +1,4 @@
type: internal
area: dependencies
- Patched three high-severity dependency advisories flagged by `bun audit`: `undici` (cross-user information disclosure via degenerate private cache directives), `brace-expansion` (denial of service via unbounded intermediate arrays), and `fast-uri` (host confusion via backslash authority introducer).
@@ -0,0 +1,7 @@
type: fixed
area: streaming
- Jellyfin playback now seeds the subtitle tokenization prefetch straight from the subtitle file it downloads, instead of waiting on an mpv track-selection event that could be missed or coalesced and leave a whole episode tokenizing line by line.
- Streamed media no longer drops its parsed subtitle cues when the active subtitle track briefly cannot be resolved, such as when cycling onto a subtitle track embedded in the stream.
- Subtitle prefetching now runs to the end of a file instead of stopping as soon as the tokenization cache fills, so the back half of an episode no longer gets tokenized line by line during playback. Previously the cache was also never cleared between episodes, so the stall carried over to every later title in a session.
- Raised the tokenization cache from 256 to 2500 lines. It is now purely a memory bound rather than a limit on how much gets prefetched, and it leaves room for lines that repeat across episodes so openings and endings stay warm between titles.
@@ -0,0 +1,7 @@
type: changed
area: subsync
- The subsync modal now lets you pick both sides of an alass run: the reference subtitle (correct timing, defaults to the loaded secondary subtitle track) and the out-of-sync subtitle that gets retimed (defaults to the active primary track).
- alass can now use the loaded video file itself as the reference (audio-based, local files only). It is offered in the reference list but is never the default.
- The out-of-sync subtitle picker also applies to ffsubsync, so a track other than the active primary one can be retimed.
- Retiming the secondary subtitle track now reloads the synced result back into the secondary slot and leaves the primary track selected, instead of replacing the primary subtitle.
@@ -0,0 +1,4 @@
type: fixed
area: overlay
- Subtitle lines no longer wait for tokenization to finish before appearing, even when the previous line is still being processed. On a tokenization cache miss, the plain line is shown immediately at its cue time and upgrades in place once tokens and annotations are ready; stale results cannot replace newer cues, and the basic plain-text websocket no longer receives a duplicate event for the annotation-only upgrade. A failed tokenization is no longer cached as the plain line, so a repeated line gets another chance at annotations instead of staying plain for the rest of the session.
@@ -0,0 +1,15 @@
type: changed
area: subtitles
- Subtitle tokenization no longer runs a duplicate full `parseText` pass per line: the termsFind scanner walk is now the only tokenizer and emits its own hoverable filler runs for unmatched text (parseText is kept only as an error fallback). This roughly halves the dictionary work per line.
- The Yomitan scanning helpers are now installed once per parser window (`__subminerYomitanScan`) instead of re-shipping and re-parsing a ~500-line script for every subtitle line; each line only evaluates a tiny call.
- termsFind lookups are cached across subtitle lines in a window-persistent LRU keyed by substring, so repeated particles and verb forms stop costing backend round trips. The cache invalidates on dictionary/settings changes and window reloads.
- The scanner walk now skips lookups at punctuation and whitespace positions (latin letters and digits still look up, e.g. Tシャツ) and caps the shrinking-window retry ladder at four extra lookups per position.
- Tokenizer runtime dependencies are built once instead of per line, fixing a JLPT lookup cache that never hit (it was keyed on a per-call closure identity and leaked a Map per line) and a `which mecab` availability check that re-ran synchronously on every line when MeCab is absent.
- Subtitle changes no longer restart the prefetch run per line (which discarded in-flight tokenization work); prefetch now only pauses for the live line and restarts on real seeks, cache invalidation, or option changes. Prefetch also stays paused across a provisional raw-subtitle emit and resumes only after the tokenized payload lands, so it never competes with the on-screen line for the parser window.
- Added per-stage debug timings (`scanMs`, `mecabMs`, `frequencyMs`, `annotateMs`) to the subtitle tokenization pipeline log.
- Fixed a reading that stopped covering its surface when an unmatched kana run extended the preceding token (for example a trailing る on 待ち合わせ), which silently disabled the known-word reading fallback for those tokens.
- Subtitle prefetching no longer stays paused for the rest of a cue when the same subtitle text is reported twice and there is nothing to tokenize. This covers the startup and overlay priming paths as well as ordinary subtitle changes.
- Character name and image lookups are now refreshed centrally whenever a character dictionary sync changes its content, so a newly added name can no longer be skipped by a stale candidate list.
- A subtitle that was on screen when its annotations were invalidated (by mining a card, for example) is now re-annotated instead of staying plain for the rest of the line.
- Character name annotations no longer cost a dictionary lookup at every position in a line. The scanner now knows which name forms the current title's character dictionary actually contains and only checks where one can start, which removes the whole overhead of having the character dictionary enabled (measured: 21 lookups per line down to 10, the same as with it disabled). Titles with no cached character data keep the previous exhaustive scan, so a missing snapshot costs speed rather than a missing name.
+2 -2
View File
@@ -1196,9 +1196,9 @@ See [TsukiHime Integration](/tsukihime-integration) for the modal workflow, lang
### Subtitle Sync
Sync the active subtitle track from the overlay picker using `alass` or `ffsubsync`. Both are **optional external tools** that must be installed separately and available on your `PATH` (or configured via the path options below).
Sync a subtitle track from the overlay picker using `alass` or `ffsubsync`. The picker lets you choose which track gets retimed (the active primary track by default) and, for alass, which reference it is aligned against (the secondary subtitle track by default). Both are **optional external tools** that must be installed separately and available on your `PATH` (or configured via the path options below).
- [`alass`](https://github.com/kaegi/alass) - fast, audio-independent sync using a secondary subtitle as reference
- [`alass`](https://github.com/kaegi/alass) - fast, audio-independent sync using another subtitle as reference; it can also take the local video file as reference (alass extracts the audio itself)
- [`ffsubsync`](https://github.com/smacke/ffsubsync) - audio-based sync using the video file as reference
```json
+6 -3
View File
@@ -161,10 +161,13 @@ If your subtitle file is out of sync with the audio, SubMiner can resynchronize
1. Open the subsync modal from the overlay.
2. Select the sync engine (alass or ffsubsync).
3. For alass, select a reference subtitle track from the video.
4. SubMiner runs the sync and reloads the corrected subtitle.
3. For alass, pick the **reference** - the subtitle with correct timing. This defaults to the secondary subtitle track. The loaded video file can also be used as the reference (alass extracts the audio itself), but it is never the default.
4. Pick the **out-of-sync subtitle** - the track that gets retimed. This defaults to the active primary subtitle track and applies to both engines.
5. SubMiner runs the sync and reloads the corrected subtitle into the slot the out-of-sync track came from: retiming the secondary track keeps it secondary and leaves the primary track selected.
For remote streams, including Jellyfin playback, the modal only offers alass. Jellyfin subtitle URLs are cached as temporary subtitle files so alass can read them, but the video stream is not downloaded. ffsubsync needs direct access to the local media file and is unavailable for stream URLs.
The reference and the out-of-sync subtitle must be different tracks; the reference list hides whichever track is selected as the target.
For remote streams, including Jellyfin playback, the modal only offers alass with a subtitle reference. Jellyfin subtitle URLs are cached as temporary subtitle files so alass can read them, but the video stream is not downloaded. ffsubsync and the video-file reference need direct access to the local media file and are unavailable for stream URLs.
When you mine a sentence card from the stats dashboard, SubMiner can also use `alass` automatically to align a local English sidecar against the matching local Japanese sidecar before filling the card translation field. The source subtitle files are not modified; SubMiner writes a temporary retimed copy and reuses it while the stats server is running.
+1 -1
View File
@@ -227,7 +227,7 @@ Install ffsubsync or configure the path:
If subtitle sync fails (the error message is prefixed with the engine name):
- Ensure the reference subtitle track exists in the video (alass requires a source track).
- Ensure a reference is selected (alass needs either a second subtitle track or the local video file, and it cannot be the same track that is being retimed).
- Check that `ffmpeg` is available (used to extract the internal subtitle track).
- Try running the sync tool manually to see detailed error output.
- ffsubsync requires local files and cannot handle remote media streams (e.g., streaming URLs).
+1 -1
View File
@@ -149,7 +149,7 @@ Once Jellyfin is configured, the tray menu includes `Jellyfin Discovery` for sta
- `--log-level` controls logger verbosity.
- `--dev` and `--debug` are app/dev-mode switches; they are not log-level aliases.
- `--background` defaults to quieter logging (`warn`) unless `--log-level` is set.
- `--background` starts at the default quieter logging level (`warn`), then follows `logging.level` after config loads. An explicit `--log-level` remains the override.
- `--background` launched from a terminal detaches and returns the prompt; stop it with tray Quit or `SubMiner.AppImage --stop` (`SubMiner.exe --stop` on Windows).
- Linux desktop launcher starts SubMiner with `--background` by default (via electron-builder `linux.executableArgs`).
- On Hyprland and other Wayland compositors, the tray icon appears only when your panel provides a StatusNotifier/AppIndicator tray host.
+3 -1
View File
@@ -64,7 +64,7 @@ Use the basic subtitle websocket when you only need the current subtitle line as
- **Client auth:** none
- **Reconnects:** client-managed
When a client connects, SubMiner immediately sends the latest subtitle payload if one is available. After that, it pushes a new message each time the current subtitle changes.
When a client connects, SubMiner immediately sends the latest subtitle payload if one is available. After that, it pushes a new message each time the current subtitle changes. Annotation-only upgrades do not repeat the same line on this basic stream.
#### Message shape
@@ -96,6 +96,8 @@ Use the annotation websocket for custom clients that want the same structured to
In practice, if you are building a new client, prefer `annotationWebsocket` unless you specifically need compatibility with an existing `websocket` consumer.
On a tokenization cache miss, this stream first sends the cue as plain text with an empty `tokens` array, then sends the annotated replacement when tokenization finishes. Treat each message as the complete current state, replacing the previous payload.
#### Message shape
```json
+28 -9
View File
@@ -3,7 +3,7 @@
# Subtitle Overlay Priming
Status: active
Last verified: 2026-06-14
Last verified: 2026-08-04
Owner: Kyle Yasuda
Read when: debugging subtitle state or blank Linux/X11 overlay windows when the visible overlay is shown or recreated
@@ -47,18 +47,37 @@ subtitles do not draw.
`emitSubtitle(payload)` and `refreshCurrentSubtitle(text)`, then prime secondary subtitles.
6. Tokenization cache hit: call `consumeCachedSubtitle(text)`, `onSubtitleChange(text)`, and
`emitSubtitle(cachedPayload)`, then prime secondary subtitles.
7. Cache miss: call `refreshCurrentSubtitle(text)` and let normal tokenization emit the final
payload.
7. Cache miss: call `refreshCurrentSubtitle(text)`. Normal processing emits a plain payload
synchronously, then replaces it with the tokenized payload when ready.
In `src/main.ts`, both `onSubtitleChange` and `refreshCurrentSubtitle` pause
`subtitlePrefetchService`, notify it with `onSeek(lastObservedTimePos)`, and then call the matching
`subtitleProcessingController` method. This gives the visible overlay priority over background
prefetch work and re-centers prefetch around the live playback time.
Both `onSubtitleChange` and `refreshCurrentSubtitle` pause `subtitlePrefetchService` and then call
the matching `subtitleProcessingController` method, giving the visible overlay priority over
background prefetch work. Prefetch is not re-centered here: restarting the run per line
(`onSeek`) discarded the in-flight tokenization every time the subtitle changed, so only real
seeks restart it (see `onTimePosUpdate` in `src/main.ts`).
The pause is released by the emit that carries the tokenized payload. Both controller methods
return whether an emit is expected, and the caller resumes immediately when it is not — otherwise
a repeated subtitle (which schedules no work) would leave prefetching idle for the rest of the
cue.
## Live Cue Delivery
- A tokenization cache miss emits the plain cue synchronously. Tokenization remains serialized so
live work does not contend for Yomitan state.
- If a newer cue arrives while an older line is still tokenizing, the newer plain cue or empty
clear payload is emitted immediately. The older tokenization result is dropped before it can
replace the current cue.
- The current cue upgrades in place when its tokens and annotations are ready. This can reflow text
or character images, but cue visibility does not wait for that work.
## Emitted State
- `emitSubtitle(payload)` maps to `emitSubtitlePayload(payload)`, which sends the normal
annotated subtitle payload to overlay windows and subtitle websocket listeners.
- `emitSubtitle(payload)` maps to `emitSubtitlePayload(payload)`. Overlay windows and annotation
websocket listeners receive both the immediate plain cue and its later annotation upgrade.
- The basic subtitle websocket receives the immediate plain cue only. Because its serialized
payload discards annotations, the later upgrade would be an identical duplicate and is skipped
when text and cue timing match.
- Secondary priming reads mpv `secondary-sub-text`, stores it in
`mpvClient.currentSecondarySubText`, and broadcasts `secondary-subtitle:set` to overlay windows.
- If secondary `requestProperty` fails, the primary flow stays complete and only a debug line is
+5 -3
View File
@@ -84,8 +84,9 @@
"overrides": {
"@xmldom/xmldom": "0.8.13",
"app-builder-lib": "26.15.3",
"brace-expansion": "5.0.8",
"brace-expansion": "5.0.9",
"electron-builder-squirrel-windows": "26.15.3",
"fast-uri": "3.1.5",
"form-data": "4.0.6",
"ip-address": "10.2.0",
"js-yaml": "4.3.0",
@@ -93,7 +94,8 @@
"minimatch": "10.2.5",
"picomatch": "4.0.4",
"tar": "7.5.21",
"tmp": "0.2.7"
"tmp": "0.2.7",
"undici": "7.29.0"
},
"keywords": [
"anki",
@@ -125,7 +127,7 @@
"@types/ws": "^8.18.1",
"electron": "42.6.0",
"electron-builder": "26.15.3",
"undici": "7.28.0",
"undici": "7.29.0",
"esbuild": "^0.25.12",
"eslint": "^10.8.0",
"prettier": "^3.8.1",
-32
View File
@@ -1,32 +0,0 @@
## Highlights
### Added
- Kiku/Lapis Word Card Type Setting
- A new setting (Mining/Anki > Kiku/Lapis Features > "Word Card Type") lets you choose which card-type flag gets marked on Kiku/Lapis word cards, including a click-card option that SubMiner couldn't set before.
- Handy if you only want click cards flagged instead of the default word-and-sentence marking.
- Choosing a card type now clears any other flags automatically, so a note can't end up marked as two types at once.
### Fixed
- Yomitan Popup on macOS
- Fixed the popup going unresponsive after mining a card — clicks outside it no longer leak through to mpv, and the overlay no longer flickers hidden and shown.
- Scrolling over the popup now scrolls its definitions instead of seeking the video.
- YouTube Playlist Links
- Opening a video from a playlist URL (like a Watch Later link with `list=`/`index=`) no longer times out while loading subtitles, metadata, or playback info.
## What's Changed
- feat(anki): add configurable word card type for Kiku/Lapis by @ksyasuda in #175
- fix(overlay): keep Yomitan popup interactive on macOS/Windows by @ksyasuda in #177
- fix(youtube): prevent playlist URLs from stalling yt-dlp probes by @ksyasuda in #180
## Installation
See the README and docs/installation guide for full setup steps.
## Assets
- Linux: `SubMiner.AppImage`
- macOS: `SubMiner-*.dmg` and `SubMiner-*.zip`
- Windows: `SubMiner-*.exe` and `SubMiner-*-win.zip`
- Optional extras: `subminer-assets.tar.gz` and the `subminer` launcher
Note: the `subminer` wrapper script uses Bun (`#!/usr/bin/env bun`), so `bun` must be installed and on `PATH`.
+5 -1
View File
@@ -1,5 +1,9 @@
export { Texthooker } from './texthooker';
export { hasMpvWebsocketPlugin, SubtitleWebSocket } from './subtitle-ws';
export {
hasMpvWebsocketPlugin,
isSubtitleAnnotationUpgrade,
SubtitleWebSocket,
} from './subtitle-ws';
export { registerGlobalShortcuts } from './shortcut';
export { createIpcDepsRuntime, registerIpcHandlers } from './ipc';
export { shortcutMatchesInputForLocalFallback } from './shortcut-fallback';
+2 -2
View File
@@ -205,7 +205,7 @@ test('runStartupBootstrapRuntime skips lifecycle when generate-config flow handl
assert.deepEqual(calls, ['setLog:warn:cli', 'forceX11', 'enforceWayland']);
});
test('runStartupBootstrapRuntime enables quiet background mode by default', () => {
test('runStartupBootstrapRuntime lets config govern background log level by default', () => {
const calls: string[] = [];
const args = makeArgs({ background: true });
@@ -222,7 +222,7 @@ test('runStartupBootstrapRuntime enables quiet background mode by default', () =
});
assert.equal(result.backgroundMode, true);
assert.deepEqual(calls, ['setLog:warn:cli', 'forceX11', 'enforceWayland', 'startLifecycle']);
assert.deepEqual(calls, ['forceX11', 'enforceWayland', 'startLifecycle']);
});
test('runStartupBootstrapRuntime enables quiet update mode by default', () => {
+1 -1
View File
@@ -45,7 +45,7 @@ export function runStartupBootstrapRuntime(
if (initialArgs.logLevel) {
deps.setLogLevel(initialArgs.logLevel, 'cli');
} else if (initialArgs.background || initialArgs.update) {
} else if (initialArgs.update) {
deps.setLogLevel('warn', 'cli');
}
+397 -22
View File
@@ -8,6 +8,7 @@ import {
runSubsyncManual,
triggerSubsyncFromConfig,
} from './subsync';
import type { SubsyncManualPayload } from '../../types';
function makeDeps(
overrides: Partial<TriggerSubsyncFromConfigDeps> = {},
@@ -76,7 +77,7 @@ test('triggerSubsyncFromConfig opens manual picker', async () => {
await triggerSubsyncFromConfig(
makeDeps({
openManualPicker: (payload) => {
payloadTrackCount = payload.sourceTracks.length;
payloadTrackCount = payload.subtitleTracks.length;
ffsubsyncAvailable = payload.ffsubsyncAvailable;
},
showMpvOsd: (text) => {
@@ -88,9 +89,9 @@ test('triggerSubsyncFromConfig opens manual picker', async () => {
}),
);
assert.equal(payloadTrackCount, 1);
assert.equal(payloadTrackCount, 2);
assert.equal(ffsubsyncAvailable, true);
assert.ok(osd.includes('Subsync: choose engine and source'));
assert.ok(osd.includes('Subsync: choose engine and subtitles'));
assert.equal(inProgressState, false);
});
@@ -140,7 +141,7 @@ test('triggerSubsyncFromConfig does not run automatic sync', async () => {
await triggerSubsyncFromConfig(
makeDeps({
openManualPicker: (payload) => {
payloadTrackCount = payload.sourceTracks.length;
payloadTrackCount = payload.subtitleTracks.length;
},
showMpvOsd: (text) => {
osd.push(text);
@@ -152,9 +153,9 @@ test('triggerSubsyncFromConfig does not run automatic sync', async () => {
}),
);
assert.equal(payloadTrackCount, 1);
assert.equal(payloadTrackCount, 2);
assert.equal(spinnerRan, false);
assert.deepEqual(osd, ['Subsync: choose engine and source']);
assert.deepEqual(osd, ['Subsync: choose engine and subtitles']);
});
test('triggerSubsyncFromConfig dedupes repeated subtitle source tracks', async () => {
@@ -195,12 +196,71 @@ test('triggerSubsyncFromConfig dedupes repeated subtitle source tracks', async (
},
}),
openManualPicker: (payload) => {
payloadTrackCount = payload.sourceTracks.length;
payloadTrackCount = payload.subtitleTracks.length;
},
}),
);
assert.equal(payloadTrackCount, 1);
assert.equal(payloadTrackCount, 2);
});
test('triggerSubsyncFromConfig keeps both active tracks when they share a file', async () => {
let payload: SubsyncManualPayload | null = null;
await triggerSubsyncFromConfig(
makeDeps({
getMpvClient: () => ({
connected: true,
currentAudioStreamIndex: null,
send: () => {},
requestProperty: async (name: string) => {
if (name === 'path') return '/tmp/video.mkv';
if (name === 'sid') return 1;
if (name === 'secondary-sid') return 2;
if (name === 'track-list') {
// mpv appends a duplicate entry when the same file is re-added, so
// the primary and secondary slots can point at one path.
return [
{
id: 1,
type: 'sub',
selected: true,
external: true,
'external-filename': '/tmp/ref.srt',
},
{
id: 2,
type: 'sub',
selected: true,
external: true,
'external-filename': '/tmp/ref.srt',
},
{
id: 3,
type: 'sub',
selected: false,
external: true,
'external-filename': '/tmp/ref.srt',
},
];
}
return null;
},
}),
openManualPicker: (nextPayload) => {
payload = nextPayload;
},
}),
);
assert.ok(payload);
const resolved = payload as SubsyncManualPayload;
assert.deepEqual(
resolved.subtitleTracks.map((track) => track.id),
[1, 2],
);
assert.equal(resolved.defaultReferenceTrackId, 2);
assert.equal(resolved.defaultTargetTrackId, 1);
});
test('triggerSubsyncFromConfig reports failures to OSD', async () => {
@@ -217,15 +277,157 @@ test('triggerSubsyncFromConfig reports failures to OSD', async () => {
assert.ok(osd.some((line) => line.startsWith('Subsync failed: MPV not connected')));
});
test('runSubsyncManual requires a source track for alass', async () => {
const result = await runSubsyncManual({ engine: 'alass', sourceTrackId: null }, makeDeps());
test('runSubsyncManual requires a reference track for alass', async () => {
const result = await runSubsyncManual({ engine: 'alass', referenceTrackId: null }, makeDeps());
assert.deepEqual(result, {
ok: false,
message: 'Select a subtitle source track for alass',
message: 'Select a reference subtitle track for alass',
});
});
test('runSubsyncManual rejects alass when reference and target are the same track', async () => {
const result = await runSubsyncManual(
{ engine: 'alass', referenceTrackId: 2, targetTrackId: 2 },
makeDeps(),
);
assert.deepEqual(result, {
ok: false,
message: 'Reference and out-of-sync subtitles must be different tracks',
});
});
test('runSubsyncManual rejects an unknown target track', async () => {
const result = await runSubsyncManual(
{ engine: 'alass', referenceTrackId: 2, targetTrackId: 99 },
makeDeps(),
);
assert.deepEqual(result, {
ok: false,
message: 'Select the out-of-sync subtitle track to retime',
});
});
test('runSubsyncManual rejects the video reference for remote media', async () => {
const result = await runSubsyncManual(
{ engine: 'alass', referenceMode: 'video' },
makeDeps({
getMpvClient: () => ({
connected: true,
currentAudioStreamIndex: null,
send: () => {},
requestProperty: async (name: string) => {
if (name === 'path') return 'https://jellyfin.example/Videos/movie/stream.mkv';
if (name === 'sid') return 1;
if (name === 'secondary-sid') return null;
if (name === 'track-list') {
return [{ id: 1, type: 'sub', selected: true, lang: 'jpn' }];
}
return null;
},
}),
}),
);
assert.equal(result.ok, false);
assert.match(result.message, /cannot use a stream URL as reference/);
});
test('openSubsyncManualPicker defaults the reference to the secondary subtitle track', async () => {
let payload: SubsyncManualPayload | null = null;
await triggerSubsyncFromConfig(
makeDeps({
getMpvClient: () => ({
connected: true,
currentAudioStreamIndex: null,
send: () => {},
requestProperty: async (name: string) => {
if (name === 'path') return '/tmp/video.mkv';
if (name === 'sid') return 1;
if (name === 'secondary-sid') return 3;
if (name === 'track-list') {
return [
{ id: 1, type: 'sub', selected: true, lang: 'jpn' },
{
id: 2,
type: 'sub',
selected: false,
external: true,
lang: 'eng',
'external-filename': '/tmp/other.srt',
},
{
id: 3,
type: 'sub',
selected: true,
external: true,
lang: 'eng',
'external-filename': '/tmp/secondary.srt',
},
];
}
return null;
},
}),
openManualPicker: (nextPayload) => {
payload = nextPayload;
},
}),
);
assert.ok(payload);
const resolved = payload as SubsyncManualPayload;
assert.deepEqual(
resolved.subtitleTracks.map((track) => track.id),
[1, 2, 3],
);
assert.equal(resolved.defaultReferenceTrackId, 3);
assert.equal(resolved.defaultTargetTrackId, 1);
assert.equal(resolved.videoReferenceAvailable, true);
});
test('openSubsyncManualPicker never defaults to a reference missing from the track list', async () => {
let payload: SubsyncManualPayload | null = null;
await triggerSubsyncFromConfig(
makeDeps({
getMpvClient: () => ({
connected: true,
currentAudioStreamIndex: null,
send: () => {},
requestProperty: async (name: string) => {
if (name === 'path') return '/tmp/video.mkv';
if (name === 'sid') return 1;
if (name === 'secondary-sid') return 2;
if (name === 'track-list') {
return [
{ id: 1, type: 'sub', selected: true, lang: 'jpn' },
// Secondary track with no usable file path: filtered out of the picker.
{ id: 2, type: 'sub', selected: true, external: true, 'external-filename': '' },
{ id: 3, type: 'sub', selected: false, lang: 'eng' },
];
}
return null;
},
}),
openManualPicker: (nextPayload) => {
payload = nextPayload;
},
}),
);
assert.ok(payload);
const resolved = payload as SubsyncManualPayload;
assert.deepEqual(
resolved.subtitleTracks.map((track) => track.id),
[1, 3],
);
assert.equal(resolved.defaultReferenceTrackId, 3);
});
test('triggerSubsyncFromConfig does not validate sync tool paths before manual selection', async () => {
const osd: string[] = [];
const inProgress: boolean[] = [];
@@ -242,7 +444,7 @@ test('triggerSubsyncFromConfig does not validate sync tool paths before manual s
inProgress.push(value);
},
openManualPicker: (payload) => {
payloadTrackCount = payload.sourceTracks.length;
payloadTrackCount = payload.subtitleTracks.length;
},
showMpvOsd: (text) => {
osd.push(text);
@@ -251,8 +453,8 @@ test('triggerSubsyncFromConfig does not validate sync tool paths before manual s
);
assert.deepEqual(inProgress, [false]);
assert.equal(payloadTrackCount, 1);
assert.deepEqual(osd, ['Subsync: choose engine and source']);
assert.equal(payloadTrackCount, 2);
assert.deepEqual(osd, ['Subsync: choose engine and subtitles']);
});
function writeExecutableScript(filePath: string, content: string): void {
@@ -333,7 +535,7 @@ test('runSubsyncManual constructs ffsubsync command and returns success', async
}),
});
const result = await runSubsyncManual({ engine: 'ffsubsync', sourceTrackId: null }, deps);
const result = await runSubsyncManual({ engine: 'ffsubsync' }, deps);
assert.equal(result.ok, true);
assert.equal(result.message, 'Subtitle synchronized with ffsubsync');
@@ -346,7 +548,7 @@ test('runSubsyncManual constructs ffsubsync command and returns success', async
const ffOutputFlagIndex = ffArgs.indexOf('-o');
assert.equal(ffOutputFlagIndex >= 0, true);
assert.equal(ffArgs[ffOutputFlagIndex + 1], toShellPath(primaryPath));
assert.equal(sentCommands[0]?.[0], 'sub_add');
assert.equal(sentCommands[0]?.[0], 'sub-add');
assert.deepEqual(sentCommands[1], ['set_property', 'sub-delay', 0]);
});
@@ -399,7 +601,7 @@ test('runSubsyncManual writes deterministic _retimed filename when replace is fa
}),
});
const result = await runSubsyncManual({ engine: 'ffsubsync', sourceTrackId: null }, deps);
const result = await runSubsyncManual({ engine: 'ffsubsync' }, deps);
assert.equal(result.ok, true);
const ffArgs = fs.readFileSync(ffsubsyncLogPath, 'utf8').trim().split('\n');
@@ -453,7 +655,7 @@ test('runSubsyncManual reports ffsubsync command failures with details', async (
}),
});
const result = await runSubsyncManual({ engine: 'ffsubsync', sourceTrackId: null }, deps);
const result = await runSubsyncManual({ engine: 'ffsubsync' }, deps);
assert.equal(result.ok, false);
assert.equal(result.message.startsWith('ffsubsync synchronization failed'), true);
@@ -518,7 +720,7 @@ test('runSubsyncManual constructs alass command and returns failure on non-zero
}),
});
const result = await runSubsyncManual({ engine: 'alass', sourceTrackId: 2 }, deps);
const result = await runSubsyncManual({ engine: 'alass', referenceTrackId: 2 }, deps);
assert.equal(result.ok, false);
assert.equal(typeof result.message, 'string');
@@ -528,6 +730,179 @@ test('runSubsyncManual constructs alass command and returns failure on non-zero
assert.equal(alassArgs[1], toShellPath(primaryPath));
});
function makeAlassSelectionDeps(tmpDir: string): {
deps: TriggerSubsyncFromConfigDeps;
alassLogPath: string;
videoPath: string;
primaryPath: string;
sourcePath: string;
sentCommands: Array<Array<string | number>>;
} {
const alassLogPath = path.join(tmpDir, 'alass-args.log');
const alassPath = path.join(tmpDir, 'alass.sh');
const ffmpegPath = path.join(tmpDir, 'ffmpeg.sh');
const ffsubsyncPath = path.join(tmpDir, 'ffsubsync.sh');
const videoPath = path.join(tmpDir, 'video.mkv');
const primaryPath = path.join(tmpDir, 'primary.srt');
const sourcePath = path.join(tmpDir, 'source.srt');
fs.writeFileSync(videoPath, 'video');
fs.writeFileSync(primaryPath, 'sub');
fs.writeFileSync(sourcePath, 'sub2');
writeExecutableScript(ffmpegPath, '#!/bin/sh\nexit 0\n');
writeExecutableScript(ffsubsyncPath, '#!/bin/sh\nexit 0\n');
writeExecutableScript(
alassPath,
`#!/bin/sh\n: > "${toShellPath(alassLogPath)}"\nfor arg in "$@"; do printf '%s\\n' "$arg" >> "${toShellPath(alassLogPath)}"; done\n: > "$3"\nexit 0\n`,
);
const trackList: Array<Record<string, unknown>> = [
{ id: 1, type: 'sub', selected: true, external: true, 'external-filename': primaryPath },
{ id: 2, type: 'sub', selected: true, external: true, 'external-filename': sourcePath },
];
const sentCommands: Array<Array<string | number>> = [];
const deps = makeDeps({
getMpvClient: () => ({
connected: true,
currentAudioStreamIndex: null,
send: (payload) => {
sentCommands.push(payload.command);
if (payload.command[0] === 'sub-add' || payload.command[0] === 'sub_add') {
trackList.push({
id: trackList.length + 1,
type: 'sub',
selected: false,
external: true,
'external-filename': payload.command[1],
});
}
},
requestProperty: async (name: string) => {
if (name === 'path') return videoPath;
if (name === 'sid') return 1;
if (name === 'secondary-sid') return 2;
if (name === 'track-list') return trackList;
return null;
},
}),
getResolvedConfig: () => ({
alassPath,
ffsubsyncPath,
ffmpegPath,
replace: false,
}),
});
return { deps, alassLogPath, videoPath, primaryPath, sourcePath, sentCommands };
}
test('runSubsyncManual uses the video file as alass reference when requested', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'subsync-alass-video-ref-'));
const { deps, alassLogPath, videoPath, primaryPath } = makeAlassSelectionDeps(tmpDir);
const result = await runSubsyncManual({ engine: 'alass', referenceMode: 'video' }, deps);
assert.equal(result.ok, true);
const alassArgs = fs.readFileSync(alassLogPath, 'utf8').trim().split('\n');
assert.equal(alassArgs[0], toShellPath(videoPath));
assert.equal(alassArgs[1], toShellPath(primaryPath));
});
test('runSubsyncManual retimes the selected target track instead of the primary', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'subsync-alass-target-'));
const { deps, alassLogPath, primaryPath, sourcePath, sentCommands } =
makeAlassSelectionDeps(tmpDir);
const result = await runSubsyncManual(
{ engine: 'alass', referenceTrackId: 1, targetTrackId: 2 },
deps,
);
assert.equal(result.ok, true);
const alassArgs = fs.readFileSync(alassLogPath, 'utf8').trim().split('\n');
assert.equal(alassArgs[0], toShellPath(primaryPath));
assert.equal(alassArgs[1], toShellPath(sourcePath));
assert.equal(sentCommands[0]?.[0], 'sub-add');
assert.equal(sentCommands[0]?.[2], 'auto');
});
test('runSubsyncManual keeps a retimed secondary track in the secondary slot', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'subsync-alass-secondary-slot-'));
const alassPath = path.join(tmpDir, 'alass.sh');
const ffmpegPath = path.join(tmpDir, 'ffmpeg.sh');
const ffsubsyncPath = path.join(tmpDir, 'ffsubsync.sh');
const videoPath = path.join(tmpDir, 'video.mkv');
const primaryPath = path.join(tmpDir, 'ja.srt');
const secondaryPath = path.join(tmpDir, 'en.srt');
const retimedPath = path.join(tmpDir, 'en_retimed.srt');
fs.writeFileSync(videoPath, 'video');
fs.writeFileSync(primaryPath, 'ja');
fs.writeFileSync(secondaryPath, 'en');
writeExecutableScript(ffmpegPath, '#!/bin/sh\nexit 0\n');
writeExecutableScript(ffsubsyncPath, '#!/bin/sh\nexit 0\n');
writeExecutableScript(alassPath, '#!/bin/sh\n: > "$3"\nexit 0\n');
const trackList: Array<Record<string, unknown>> = [
{ id: 1, type: 'sub', selected: true, external: true, 'external-filename': primaryPath },
{ id: 2, type: 'sub', selected: true, external: true, 'external-filename': secondaryPath },
];
const sentCommands: Array<Array<string | number>> = [];
const deps = makeDeps({
getMpvClient: () => ({
connected: true,
currentAudioStreamIndex: null,
send: (payload) => {
sentCommands.push(payload.command);
if (payload.command[0] === 'sub-add') {
trackList.push({
id: 3,
type: 'sub',
selected: false,
external: true,
'external-filename': payload.command[1],
});
}
},
requestProperty: async (name: string) => {
if (name === 'path') return videoPath;
if (name === 'sid') return 1;
if (name === 'secondary-sid') return 2;
if (name === 'track-list') return trackList;
return null;
},
}),
getResolvedConfig: () => ({
alassPath,
ffsubsyncPath,
ffmpegPath,
replace: false,
}),
});
const result = await runSubsyncManual(
{ engine: 'alass', referenceTrackId: 1, targetTrackId: 2 },
deps,
);
assert.equal(result.ok, true);
assert.deepEqual(sentCommands[0], ['sub-add', retimedPath, 'auto']);
assert.deepEqual(sentCommands[1], ['set_property', 'secondary-sub-delay', 0]);
assert.deepEqual(sentCommands[2], ['set_property', 'secondary-sid', 3]);
assert.equal(
sentCommands.some((command) => command[1] === 'sub-delay'),
false,
);
assert.equal(
sentCommands.some((command) => command[1] === 'sid'),
false,
);
assert.equal(
sentCommands.some((command) => command[1] === 'sid'),
false,
);
});
test('runSubsyncManual keeps internal alass source file alive until sync finishes', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'subsync-alass-internal-source-'));
const alassPath = path.join(tmpDir, 'alass.sh');
@@ -589,11 +964,11 @@ test('runSubsyncManual keeps internal alass source file alive until sync finishe
}),
});
const result = await runSubsyncManual({ engine: 'alass', sourceTrackId: 2 }, deps);
const result = await runSubsyncManual({ engine: 'alass', referenceTrackId: 2 }, deps);
assert.equal(result.ok, true);
assert.equal(result.message, 'Subtitle synchronized with alass');
assert.equal(sentCommands[0]?.[0], 'sub_add');
assert.equal(sentCommands[0]?.[0], 'sub-add');
assert.deepEqual(sentCommands[1], ['set_property', 'sub-delay', 0]);
});
@@ -645,7 +1020,7 @@ test('runSubsyncManual resolves string sid values from mpv stream properties', a
}),
});
const result = await runSubsyncManual({ engine: 'ffsubsync', sourceTrackId: null }, deps);
const result = await runSubsyncManual({ engine: 'ffsubsync' }, deps);
assert.equal(result.ok, true);
assert.equal(result.message, 'Subtitle synchronized with ffsubsync');
+195 -34
View File
@@ -21,6 +21,11 @@ interface FileExtractionResult {
temporary: boolean;
}
type SubtitleSlot = 'primary' | 'secondary';
const SYNCED_TRACK_LOOKUP_ATTEMPTS = 5;
const SYNCED_TRACK_LOOKUP_RETRY_MS = 100;
function summarizeCommandFailure(command: string, result: CommandResult): string {
const parts = [
`code=${result.code ?? 'n/a'}`,
@@ -90,16 +95,28 @@ function getSourceTrackIdentity(track: MpvTrack): string {
return 'unknown';
}
function dedupeSourceTracks(tracks: MpvTrack[]): MpvTrack[] {
const deduped = new Map<string, MpvTrack>();
function isPinned(track: MpvTrack, pinnedIds: Set<number>): boolean {
return typeof track.id === 'number' && pinnedIds.has(track.id);
}
// Pinned tracks (the active primary/secondary) always survive, even when two of
// them point at the same file; only unpinned duplicates are collapsed.
function dedupeSubtitleTracks(tracks: MpvTrack[], pinnedIds: Set<number>): MpvTrack[] {
const pinnedIdentities = new Set(
tracks.filter((track) => isPinned(track, pinnedIds)).map(getSourceTrackIdentity),
);
const winners = new Map<string, MpvTrack>();
for (const track of tracks) {
if (isPinned(track, pinnedIds)) continue;
const identity = getSourceTrackIdentity(track);
const existing = deduped.get(identity);
if (pinnedIdentities.has(identity)) continue;
const existing = winners.get(identity);
if (!existing || (track.selected && !existing.selected)) {
deduped.set(identity, track);
winners.set(identity, track);
}
}
return [...deduped.values()];
const kept = new Set(winners.values());
return tracks.filter((track) => isPinned(track, pinnedIds) || kept.has(track));
}
export interface TriggerSubsyncFromConfigDeps extends SubsyncCoreDeps {
@@ -142,20 +159,21 @@ async function gatherSubsyncContext(client: MpvClientLike): Promise<SubsyncConte
}
const secondaryTrack = subtitleTracks.find((track) => track.id === secondarySid) ?? null;
const sourceTracks = subtitleTracks
.filter((track) => track.id !== sid)
.filter((track) => {
const usableTracks = subtitleTracks.filter((track) => {
if (typeof track.id !== 'number') return false;
if (!track.external) return true;
const filename = track['external-filename'];
return typeof filename === 'string' && filename.length > 0;
});
const uniqueSourceTracks = dedupeSourceTracks(sourceTracks);
return {
videoPath,
primaryTrack,
secondaryTrack,
sourceTracks: uniqueSourceTracks,
subtitleTracks: dedupeSubtitleTracks(
usableTracks,
new Set([sid, secondarySid].filter((id): id is number => typeof id === 'number')),
),
audioStreamIndex: client.currentAudioStreamIndex,
};
}
@@ -271,41 +289,104 @@ async function runFfsubsyncSync(
return runCommand(ffsubsyncPath, args);
}
function loadSyncedSubtitle(client: MpvClientLike, pathToLoad: string): void {
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
// mpv may echo the path back with different separators, and Windows paths are
// case-insensitive, so compare normalized forms instead of raw strings.
function normalizeSubtitlePathForCompare(value: string): string {
const normalized = value.replace(/\\/g, '/');
return process.platform === 'win32' ? normalized.toLowerCase() : normalized;
}
async function findAddedSubtitleTrackId(
client: MpvClientLike,
pathToLoad: string,
): Promise<number | null> {
const wanted = normalizeSubtitlePathForCompare(pathToLoad);
// sub-add is queued, so the track may not appear in the first track-list reply.
for (let attempt = 0; attempt < SYNCED_TRACK_LOOKUP_ATTEMPTS; attempt += 1) {
let tracks: MpvTrack[] = [];
try {
const trackListRaw = await client.requestProperty('track-list');
tracks = Array.isArray(trackListRaw) ? normalizeTrackIds(trackListRaw as MpvTrack[]) : [];
} catch {
return null;
}
// Re-adding a file mpv already knows appends a duplicate entry; the newest
// one holds the retimed content, so prefer the last match.
const matches = tracks.filter((track) => {
if (track.type !== 'sub') return false;
const filename = track['external-filename'];
return typeof filename === 'string' && normalizeSubtitlePathForCompare(filename) === wanted;
});
const added = matches[matches.length - 1];
if (added && typeof added.id === 'number') {
return added.id;
}
if (attempt < SYNCED_TRACK_LOOKUP_ATTEMPTS - 1) {
await delay(SYNCED_TRACK_LOOKUP_RETRY_MS);
}
}
return null;
}
async function loadSyncedSubtitle(
client: MpvClientLike,
pathToLoad: string,
slot: SubtitleSlot,
): Promise<void> {
if (!client.connected) {
throw new Error('MPV disconnected while loading subtitle');
}
client.send({ command: ['sub_add', pathToLoad] });
if (slot === 'secondary') {
// Keep the primary track untouched: load without selecting, then point
// secondary-sid at the freshly added track.
client.send({ command: ['sub-add', pathToLoad, 'auto'] });
client.send({ command: ['set_property', 'secondary-sub-delay', 0] });
const addedTrackId = await findAddedSubtitleTrackId(client, pathToLoad);
if (addedTrackId === null) {
throw new Error('Synchronized subtitle did not appear in the mpv track list');
}
client.send({ command: ['set_property', 'secondary-sid', addedTrackId] });
return;
}
client.send({ command: ['sub-add', pathToLoad] });
client.send({ command: ['set_property', 'sub-delay', 0] });
}
async function subsyncToReference(
engine: 'alass' | 'ffsubsync',
referenceFilePath: string,
targetTrack: MpvTrack,
context: SubsyncContext,
resolved: SubsyncResolvedConfig,
client: MpvClientLike,
slot: SubtitleSlot,
): Promise<SubsyncResult> {
const ffmpegPath = ensureExecutablePath(resolved.ffmpegPath, 'ffmpeg');
const primaryExtraction = await extractSubtitleTrackToFile(
const targetExtraction = await extractSubtitleTrackToFile(
ffmpegPath,
context.videoPath,
context.primaryTrack,
targetTrack,
);
const replacePrimary = resolved.replace !== false && !primaryExtraction.temporary;
const outputPath = buildRetimedPath(primaryExtraction.path, replacePrimary);
const replaceTarget = resolved.replace !== false && !targetExtraction.temporary;
const outputPath = buildRetimedPath(targetExtraction.path, replaceTarget);
try {
let result: CommandResult;
if (engine === 'alass') {
const alassPath = ensureExecutablePath(resolved.alassPath, 'alass');
result = await runAlassSync(alassPath, referenceFilePath, primaryExtraction.path, outputPath);
result = await runAlassSync(alassPath, referenceFilePath, targetExtraction.path, outputPath);
} else {
const ffsubsyncPath = ensureExecutablePath(resolved.ffsubsyncPath, 'ffsubsync');
result = await runFfsubsyncSync(
ffsubsyncPath,
context.videoPath,
primaryExtraction.path,
targetExtraction.path,
outputPath,
context.audioStreamIndex,
);
@@ -319,13 +400,13 @@ async function subsyncToReference(
};
}
loadSyncedSubtitle(client, outputPath);
await loadSyncedSubtitle(client, outputPath, slot);
return {
ok: true,
message: `Subtitle synchronized with ${engine}`,
};
} finally {
cleanupTemporaryFile(primaryExtraction);
cleanupTemporaryFile(targetExtraction);
}
}
@@ -337,6 +418,25 @@ function validateFfsubsyncReference(videoPath: string): void {
}
}
function resolveTargetTrack(
request: SubsyncManualRunRequest,
context: SubsyncContext,
): MpvTrack | null {
if (request.targetTrackId === undefined || request.targetTrackId === null) {
return context.primaryTrack;
}
return getTrackById(context.subtitleTracks, request.targetTrackId);
}
// Retiming the secondary track must not steal the primary slot: the synced file
// goes back where the out-of-sync one was.
function resolveTargetSlot(targetTrack: MpvTrack, context: SubsyncContext): SubtitleSlot {
if (typeof targetTrack.id !== 'number') return 'primary';
if (targetTrack.id === context.primaryTrack.id) return 'primary';
if (context.secondaryTrack && targetTrack.id === context.secondaryTrack.id) return 'secondary';
return 'primary';
}
export async function runSubsyncManual(
request: SubsyncManualRunRequest,
deps: SubsyncCoreDeps,
@@ -345,6 +445,12 @@ export async function runSubsyncManual(
const context = await gatherSubsyncContext(client);
const resolved = deps.getResolvedConfig();
const targetTrack = resolveTargetTrack(request, context);
if (!targetTrack) {
return { ok: false, message: 'Select the out-of-sync subtitle track to retime' };
}
const targetSlot = resolveTargetSlot(targetTrack, context);
if (request.engine === 'ffsubsync') {
try {
validateFfsubsyncReference(context.videoPath);
@@ -354,22 +460,64 @@ export async function runSubsyncManual(
message: `ffsubsync synchronization failed: ${(error as Error).message}`,
};
}
return subsyncToReference('ffsubsync', context.videoPath, context, resolved, client);
return subsyncToReference(
'ffsubsync',
context.videoPath,
targetTrack,
context,
resolved,
client,
targetSlot,
);
}
const sourceTrack = getTrackById(context.sourceTracks, request.sourceTrackId ?? null);
if (!sourceTrack) {
return { ok: false, message: 'Select a subtitle source track for alass' };
if (request.referenceMode === 'video') {
if (isRemoteMediaPath(context.videoPath)) {
return {
ok: false,
message:
'alass cannot use a stream URL as reference. Pick a reference subtitle track instead.',
};
}
return subsyncToReference(
'alass',
context.videoPath,
targetTrack,
context,
resolved,
client,
targetSlot,
);
}
const referenceTrack = getTrackById(context.subtitleTracks, request.referenceTrackId ?? null);
if (!referenceTrack) {
return { ok: false, message: 'Select a reference subtitle track for alass' };
}
if (referenceTrack.id === targetTrack.id) {
return { ok: false, message: 'Reference and out-of-sync subtitles must be different tracks' };
}
const ffmpegPath = ensureExecutablePath(resolved.ffmpegPath, 'ffmpeg');
let sourceExtraction: FileExtractionResult | null = null;
let referenceExtraction: FileExtractionResult | null = null;
try {
sourceExtraction = await extractSubtitleTrackToFile(ffmpegPath, context.videoPath, sourceTrack);
return await subsyncToReference('alass', sourceExtraction.path, context, resolved, client);
referenceExtraction = await extractSubtitleTrackToFile(
ffmpegPath,
context.videoPath,
referenceTrack,
);
return await subsyncToReference(
'alass',
referenceExtraction.path,
targetTrack,
context,
resolved,
client,
targetSlot,
);
} finally {
if (sourceExtraction) {
cleanupTemporaryFile(sourceExtraction);
if (referenceExtraction) {
cleanupTemporaryFile(referenceExtraction);
}
}
}
@@ -377,14 +525,27 @@ export async function runSubsyncManual(
export async function openSubsyncManualPicker(deps: TriggerSubsyncFromConfigDeps): Promise<void> {
const client = getMpvClientForSubsync(deps);
const context = await gatherSubsyncContext(client);
const payload: SubsyncManualPayload = {
ffsubsyncAvailable: !isRemoteMediaPath(context.videoPath),
sourceTracks: context.sourceTracks
const subtitleTracks = context.subtitleTracks
.filter((track) => typeof track.id === 'number')
.map((track) => ({
id: track.id as number,
label: formatTrackLabel(track),
})),
}));
const primaryTrackId =
typeof context.primaryTrack.id === 'number' ? context.primaryTrack.id : null;
const secondaryTrackId =
typeof context.secondaryTrack?.id === 'number' ? context.secondaryTrack.id : null;
const payload: SubsyncManualPayload = {
subtitleTracks,
// The secondary track can be filtered or deduped out of the emitted list,
// so only default to it when the picker actually offers it.
defaultReferenceTrackId:
subtitleTracks.find((track) => track.id === secondaryTrackId)?.id ??
subtitleTracks.find((track) => track.id !== primaryTrackId)?.id ??
null,
defaultTargetTrackId: primaryTrackId,
videoReferenceAvailable: !isRemoteMediaPath(context.videoPath),
ffsubsyncAvailable: !isRemoteMediaPath(context.videoPath),
};
deps.openManualPicker(payload);
}
@@ -397,7 +558,7 @@ export async function triggerSubsyncFromConfig(deps: TriggerSubsyncFromConfigDep
try {
await openSubsyncManualPicker(deps);
deps.showMpvOsd('Subsync: choose engine and source');
deps.showMpvOsd('Subsync: choose engine and subtitles');
} catch (error) {
deps.showMpvOsd(`Subsync failed: ${(error as Error).message}`);
} finally {
+17 -18
View File
@@ -74,7 +74,6 @@ test('prefetch service tokenizes priority window cues and caches them', async ()
preCacheTokenization: (text, data) => {
cached.set(text, data);
},
isCacheFull: () => false,
priorityWindowSize: 3,
});
@@ -91,32 +90,38 @@ test('prefetch service tokenizes priority window cues and caches them', async ()
assert.ok(cached.has('line-2'));
});
test('prefetch service stops when cache is full', async () => {
test('prefetch service warms every cue even when the cache evicts along the way', async () => {
const cues = makeCues(20);
let tokenizeCalls = 0;
let cacheSize = 0;
const tokenized: string[] = [];
// Stand-in for the LRU: only the last 5 entries survive, so later cues evict earlier ones.
const cache = new Set<string>();
const service = createSubtitlePrefetchService({
cues,
tokenizeSubtitle: async (text) => {
tokenizeCalls += 1;
tokenized.push(text);
return { text, tokens: [] };
},
preCacheTokenization: () => {
cacheSize += 1;
preCacheTokenization: (text) => {
cache.add(text);
while (cache.size > 5) {
const oldest = cache.values().next().value;
if (oldest === undefined) break;
cache.delete(oldest);
}
},
isCacheFull: () => cacheSize >= 5,
hasCachedTokenization: (text) => cache.has(text),
priorityWindowSize: 3,
});
service.start(0);
for (let i = 0; i < 30; i += 1) {
for (let i = 0; i < 60; i += 1) {
await flushMicrotasks();
}
service.stop();
// Should have stopped at 5 (cache full), not tokenized all 20
assert.ok(tokenizeCalls <= 6, `Expected <= 6 tokenize calls, got ${tokenizeCalls}`);
assert.equal(tokenized.length, 20, `Expected all 20 cues warmed, got ${tokenized.length}`);
assert.equal(new Set(tokenized).size, 20, 'Each cue is tokenized at most once per run');
});
test('prefetch service can be stopped mid-flight', async () => {
@@ -130,7 +135,6 @@ test('prefetch service can be stopped mid-flight', async () => {
return { text, tokens: [] };
},
preCacheTokenization: () => {},
isCacheFull: () => false,
priorityWindowSize: 3,
});
@@ -159,7 +163,6 @@ test('prefetch service onSeek re-prioritizes from new position', async () => {
preCacheTokenization: (text) => {
cachedTexts.push(text);
},
isCacheFull: () => false,
priorityWindowSize: 3,
});
@@ -183,7 +186,7 @@ test('prefetch service onSeek re-prioritizes from new position', async () => {
assert.ok(hasPostSeekCue, 'Should have cached cues after seek position');
});
test('prefetch service still warms the priority window when cache is full', async () => {
test('prefetch service warms the priority window ahead of the rest of the file', async () => {
const cues = makeCues(20);
const cachedTexts: string[] = [];
@@ -193,7 +196,6 @@ test('prefetch service still warms the priority window when cache is full', asyn
preCacheTokenization: (text) => {
cachedTexts.push(text);
},
isCacheFull: () => true,
priorityWindowSize: 3,
});
@@ -217,7 +219,6 @@ test('prefetch service pause/resume halts and continues tokenization', async ()
return { text, tokens: [] };
},
preCacheTokenization: () => {},
isCacheFull: () => false,
priorityWindowSize: 3,
});
@@ -255,7 +256,6 @@ test('prefetch service skips cues already present in tokenization cache', async
},
preCacheTokenization: () => {},
hasCachedTokenization: (text) => text === 'line-0' || text === 'line-1',
isCacheFull: () => false,
priorityWindowSize: 3,
});
@@ -285,7 +285,6 @@ test('prefetch service deduplicates repeated cue text within a run', async () =>
return { text, tokens: [] };
},
preCacheTokenization: () => {},
isCacheFull: () => false,
priorityWindowSize: 3,
});
+5 -7
View File
@@ -7,7 +7,6 @@ export interface SubtitlePrefetchServiceDeps {
tokenizeSubtitle: (text: string) => Promise<SubtitleData | null>;
preCacheTokenization: (text: string, data: SubtitleData) => void;
hasCachedTokenization?: (text: string) => boolean;
isCacheFull: () => boolean;
priorityWindowSize?: number;
}
@@ -57,11 +56,14 @@ export function createSubtitlePrefetchService(
let paused = false;
let currentRunId = 0;
// A run is a single bounded pass over one file's cues, deduped by `warmedKeys` and by
// `hasCachedTokenization`, so the worst case is one tokenization per cue. The cache is
// an LRU and bounds its own memory, so a full cache is not a reason to stop warming;
// stopping there used to leave the tail of longer media permanently uncached.
async function tokenizeCueList(
cuesToProcess: SubtitleCue[],
runId: number,
warmedKeys: Set<string>,
options: { allowWhenCacheFull?: boolean } = {},
): Promise<void> {
for (const cue of cuesToProcess) {
if (stopped || runId !== currentRunId) {
@@ -77,10 +79,6 @@ export function createSubtitlePrefetchService(
return;
}
if (!options.allowWhenCacheFull && deps.isCacheFull()) {
return;
}
const cacheKey = normalizeSubtitleCacheKey(cue.text);
if (!cacheKey || warmedKeys.has(cacheKey) || deps.hasCachedTokenization?.(cue.text)) {
if (cacheKey) {
@@ -110,7 +108,7 @@ export function createSubtitlePrefetchService(
// Phase 1: Priority window
const priorityCues = computePriorityWindow(cues, currentTimeSeconds, windowSize);
await tokenizeCueList(priorityCues, runId, warmedKeys, { allowWhenCacheFull: true });
await tokenizeCueList(priorityCues, runId, warmedKeys);
if (stopped || runId !== currentRunId) {
return;
@@ -0,0 +1,159 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import type { SubtitleData } from '../../types';
import { createSubtitleProcessingController } from './subtitle-processing-controller';
function flushMicrotasks(): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, 0));
}
test('new subtitle emits plain immediately without parallel tokenization or a stale overwrite', async () => {
const emitted: SubtitleData[] = [];
const resolvers = new Map<string, (value: SubtitleData | null) => void>();
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) =>
await new Promise<SubtitleData | null>((resolve) => {
resolvers.set(text, resolve);
}),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('first');
controller.onSubtitleChange('second');
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: 'second', tokens: null },
]);
assert.equal(resolvers.has('second'), false);
const resolveFirst = resolvers.get('first');
assert.ok(resolveFirst);
resolveFirst({ text: 'first', tokens: [] });
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: 'second', tokens: null },
]);
assert.equal(resolvers.has('second'), true);
const resolveSecond = resolvers.get('second');
assert.ok(resolveSecond);
resolveSecond({ text: 'second', tokens: [] });
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: 'second', tokens: null },
{ text: 'second', tokens: [] },
]);
});
test('subtitle clears immediately while previous tokenization remains pending', async () => {
const emitted: SubtitleData[] = [];
let resolveTokenization: ((value: SubtitleData | null) => void) | undefined;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async () =>
await new Promise<SubtitleData | null>((resolve) => {
resolveTokenization = resolve;
}),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('first');
controller.onSubtitleChange('');
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: '', tokens: null },
]);
assert.ok(resolveTokenization);
resolveTokenization({ text: 'first', tokens: [] });
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: '', tokens: null },
]);
});
test('returning to an uncached completed line emits it while another line is pending', async () => {
const emitted: SubtitleData[] = [];
let resolvePending: ((value: SubtitleData | null) => void) | undefined;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => {
if (text === 'A') {
return { text, tokens: [] };
}
return await new Promise<SubtitleData | null>((resolve) => {
resolvePending = resolve;
});
},
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('A');
await flushMicrotasks();
controller.invalidateTokenizationCache();
controller.onSubtitleChange('B');
controller.onSubtitleChange('A');
assert.deepEqual(emitted.at(-1), { text: 'A', tokens: null });
assert.ok(resolvePending);
resolvePending({ text: 'B', tokens: [] });
});
test('ABA subtitle changes reuse the matching first tokenization only after A is current again', async () => {
const emitted: SubtitleData[] = [];
const tokenizeCalls: string[] = [];
const resolvers: Array<(value: SubtitleData | null) => void> = [];
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => {
tokenizeCalls.push(text);
return await new Promise<SubtitleData | null>((resolve) => {
resolvers.push(resolve);
});
},
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('A');
controller.onSubtitleChange('B');
controller.onSubtitleChange('A');
const resolveFirst = resolvers[0];
assert.ok(resolveFirst);
resolveFirst({ text: 'A', tokens: [{ value: 1 } as never] });
await flushMicrotasks();
assert.deepEqual(tokenizeCalls, ['A']);
assert.deepEqual(emitted, [
{ text: 'A', tokens: null },
{ text: 'B', tokens: null },
{ text: 'A', tokens: null },
{ text: 'A', tokens: [{ value: 1 } as never] },
]);
});
test('cached next subtitle does not downgrade to plain while processing is busy', async () => {
const emitted: SubtitleData[] = [];
let resolveTokenization: ((value: SubtitleData | null) => void) | undefined;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) =>
await new Promise<SubtitleData | null>((resolve) => {
resolveTokenization = () => resolve({ text, tokens: [] });
}),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.preCacheTokenization('cached', { text: 'cached', tokens: [] });
controller.onSubtitleChange('pending');
controller.onSubtitleChange('cached');
assert.deepEqual(emitted, [{ text: 'pending', tokens: null }]);
assert.ok(resolveTokenization);
resolveTokenization({ text: 'pending', tokens: [] });
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: 'pending', tokens: null },
{ text: 'cached', tokens: [] },
]);
});
@@ -7,18 +7,138 @@ function flushMicrotasks(): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, 0));
}
test('subtitle processing emits tokenized payload when tokenization succeeds', async () => {
test('subtitle processing emits plain payload immediately on cache miss, then tokenized payload', async () => {
const emitted: SubtitleData[] = [];
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('字幕');
assert.deepEqual(emitted, [{ text: '字幕', tokens: null }]);
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: '字幕', tokens: null },
{ text: '字幕', tokens: [] },
]);
});
test('cache invalidation during pending tokenization does not re-emit the plain payload', async () => {
const emitted: SubtitleData[] = [];
const resolvers: Array<(value: SubtitleData | null) => void> = [];
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) =>
await new Promise<SubtitleData | null>((resolve) => {
resolvers.push(() => resolve({ text, tokens: [{ value: resolvers.length } as never] }));
}),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('行');
assert.deepEqual(emitted, [{ text: '行', tokens: null }]);
controller.invalidateTokenizationCache();
resolvers[0]?.({ text: '行', tokens: [] });
await flushMicrotasks();
// Retry for the new generation is now pending; still no duplicate plain emit.
assert.deepEqual(emitted, [{ text: '行', tokens: null }]);
resolvers[1]?.({ text: '行', tokens: [] });
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: '行', tokens: null },
{ text: '行', tokens: [{ value: 2 } as never] },
]);
});
test('failed refresh does not downgrade an already emitted tokenized subtitle', async () => {
const emitted: SubtitleData[] = [];
let tokenizeCalls = 0;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => {
tokenizeCalls += 1;
if (tokenizeCalls > 1) {
throw new Error('tokenizer gone');
}
return { text, tokens: [] };
},
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('行');
await flushMicrotasks();
controller.invalidateTokenizationCache();
controller.refreshCurrentSubtitle();
await flushMicrotasks();
assert.equal(tokenizeCalls, 2);
assert.deepEqual(emitted, [
{ text: '行', tokens: null },
{ text: '行', tokens: [] },
]);
});
test('null-tokenization refresh does not downgrade an already emitted tokenized subtitle', async () => {
const emitted: SubtitleData[] = [];
let tokenizeCalls = 0;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => {
tokenizeCalls += 1;
return tokenizeCalls > 1 ? null : { text, tokens: [] };
},
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('行');
await flushMicrotasks();
controller.invalidateTokenizationCache();
controller.refreshCurrentSubtitle();
await flushMicrotasks();
assert.equal(tokenizeCalls, 2);
assert.deepEqual(emitted, [
{ text: '行', tokens: null },
{ text: '行', tokens: [] },
]);
});
test('subtitle processing does not emit plain payload for cached lines', async () => {
const emitted: SubtitleData[] = [];
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.preCacheTokenization('字幕', { text: '字幕', tokens: [] });
controller.onSubtitleChange('字幕');
await flushMicrotasks();
assert.deepEqual(emitted, [{ text: '字幕', tokens: [] }]);
});
test('subtitle processing shows plain line while tokenization is still pending', async () => {
const emitted: SubtitleData[] = [];
let resolveTokenization: ((value: SubtitleData | null) => void) | undefined;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) =>
await new Promise<SubtitleData | null>((resolve) => {
resolveTokenization = () => resolve({ text, tokens: [] });
}),
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('遅い行');
await flushMicrotasks();
assert.deepEqual(emitted, [{ text: '遅い行', tokens: null }]);
assert.ok(resolveTokenization);
resolveTokenization({ text: '遅い行', tokens: [] });
await flushMicrotasks();
assert.deepEqual(emitted, [
{ text: '遅い行', tokens: null },
{ text: '遅い行', tokens: [] },
]);
});
test('subtitle processing drops stale tokenization and delivers latest subtitle only once', async () => {
const emitted: SubtitleData[] = [];
let firstResolve: ((value: SubtitleData | null) => void) | undefined;
@@ -41,7 +161,11 @@ test('subtitle processing drops stale tokenization and delivers latest subtitle
await flushMicrotasks();
await flushMicrotasks();
assert.deepEqual(emitted, [{ text: 'second', tokens: [] }]);
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: 'second', tokens: null },
{ text: 'second', tokens: [] },
]);
});
test('subtitle processing skips duplicate subtitle emission', async () => {
@@ -60,7 +184,10 @@ test('subtitle processing skips duplicate subtitle emission', async () => {
controller.onSubtitleChange('same');
await flushMicrotasks();
assert.equal(emitted.length, 1);
assert.deepEqual(emitted, [
{ text: 'same', tokens: null },
{ text: 'same', tokens: [] },
]);
assert.equal(tokenizeCalls, 1);
});
@@ -84,7 +211,9 @@ test('subtitle processing reuses cached tokenization for repeated subtitle text'
assert.equal(tokenizeCalls, 2);
assert.deepEqual(emitted, [
{ text: 'first', tokens: null },
{ text: 'first', tokens: [] },
{ text: 'second', tokens: null },
{ text: 'second', tokens: [] },
{ text: 'first', tokens: [] },
]);
@@ -100,7 +229,48 @@ test('subtitle processing falls back to plain subtitle when tokenization returns
controller.onSubtitleChange('fallback');
await flushMicrotasks();
assert.deepEqual(
emitted,
[{ text: 'fallback', tokens: null }],
'plain payload should not be re-emitted when tokenization yields nothing new',
);
});
test('null tokenization is not cached and a later cue retries tokenization', async () => {
const emitted: SubtitleData[] = [];
const callsByText = new Map<string, number>();
let failNext = true;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => {
callsByText.set(text, (callsByText.get(text) ?? 0) + 1);
if (text === 'fallback' && failNext) {
failNext = false;
return null;
}
return { text, tokens: [] };
},
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('fallback');
await flushMicrotasks();
assert.equal(callsByText.get('fallback'), 1);
assert.equal(
controller.hasCachedSubtitle('fallback'),
false,
'plain fallback must not be cached when tokenization yields nothing',
);
assert.deepEqual(emitted, [{ text: 'fallback', tokens: null }]);
controller.onSubtitleChange('other');
await flushMicrotasks();
controller.onSubtitleChange('fallback');
await flushMicrotasks();
assert.equal(callsByText.get('fallback'), 2, 'later cue should retry tokenization');
assert.equal(controller.hasCachedSubtitle('fallback'), true);
assert.deepEqual(emitted.at(-1), { text: 'fallback', tokens: [] });
});
test('subtitle processing ignores duplicate current subtitle refresh without cache invalidation', async () => {
@@ -120,7 +290,10 @@ test('subtitle processing ignores duplicate current subtitle refresh without cac
await flushMicrotasks();
assert.equal(tokenizeCalls, 1);
assert.deepEqual(emitted, [{ text: 'same', tokens: [] }]);
assert.deepEqual(emitted, [
{ text: 'same', tokens: null },
{ text: 'same', tokens: [] },
]);
});
test('subtitle processing coalesces refresh requests while current subtitle is processing', async () => {
@@ -146,7 +319,10 @@ test('subtitle processing coalesces refresh requests while current subtitle is p
await flushMicrotasks();
assert.equal(tokenizeCalls, 1);
assert.deepEqual(emitted, [{ text: 'same', tokens: [] }]);
assert.deepEqual(emitted, [
{ text: 'same', tokens: null },
{ text: 'same', tokens: [] },
]);
});
test('subtitle processing refresh re-tokenizes after cache invalidation', async () => {
@@ -168,6 +344,7 @@ test('subtitle processing refresh re-tokenizes after cache invalidation', async
assert.equal(tokenizeCalls, 2);
assert.deepEqual(emitted, [
{ text: 'same', tokens: null },
{ text: 'same', tokens: [{ value: 1 } as never] },
{ text: 'same', tokens: [{ value: 2 } as never] },
]);
@@ -183,7 +360,10 @@ test('subtitle processing refresh can use explicit text override', async () => {
controller.refreshCurrentSubtitle('initial');
await flushMicrotasks();
assert.deepEqual(emitted, [{ text: 'initial', tokens: [] }]);
assert.deepEqual(emitted, [
{ text: 'initial', tokens: null },
{ text: 'initial', tokens: [] },
]);
});
test('subtitle processing cache invalidation only affects future subtitle events', async () => {
@@ -205,10 +385,10 @@ test('subtitle processing cache invalidation only affects future subtitle events
await flushMicrotasks();
assert.equal(callsByText.get('same'), 1);
assert.equal(emitted.length, 3);
assert.equal(emitted.length, 5);
controller.invalidateTokenizationCache();
assert.equal(emitted.length, 3);
assert.equal(emitted.length, 5);
controller.onSubtitleChange('different');
await flushMicrotasks();
@@ -308,25 +488,119 @@ test('hasCachedSubtitle checks prefetched entries without consuming them', async
assert.equal(controller.hasCachedSubtitle('猫\nです'), false);
});
test('isCacheFull returns false when cache is below limit', () => {
test('cache keeps every entry while below the limit', () => {
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: null }),
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: () => {},
cacheLimit: 8,
});
assert.equal(controller.isCacheFull(), false);
for (let i = 0; i < 8; i += 1) {
controller.preCacheTokenization(`line-${i}`, { text: `line-${i}`, tokens: [] });
}
assert.deepEqual(
Array.from({ length: 8 }, (_, i) => controller.hasCachedSubtitle(`line-${i}`)),
Array.from({ length: 8 }, () => true),
);
});
test('isCacheFull returns true when cache reaches limit', async () => {
test('cache evicts least recently used entries once the limit is reached', () => {
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: () => {},
cacheLimit: 3,
});
for (const line of ['a', 'b', 'c']) {
controller.preCacheTokenization(line, { text: line, tokens: [] });
}
// Touching 'a' makes 'b' the eviction candidate.
controller.consumeCachedSubtitle('a');
controller.preCacheTokenization('d', { text: 'd', tokens: [] });
assert.equal(controller.hasCachedSubtitle('b'), false);
assert.deepEqual(
['a', 'c', 'd'].map((line) => controller.hasCachedSubtitle(line)),
[true, true, true],
);
});
test('default cache limit covers a full-length title without evicting', () => {
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: () => {},
});
// Fill cache to the 256 limit
for (let i = 0; i < 256; i += 1) {
for (let i = 0; i < 2000; i += 1) {
controller.preCacheTokenization(`line-${i}`, { text: `line-${i}`, tokens: [] });
}
assert.equal(controller.isCacheFull(), true);
assert.equal(controller.hasCachedSubtitle('line-0'), true);
assert.equal(controller.hasCachedSubtitle('line-1999'), true);
});
test('onSubtitleChange reports whether processing was scheduled', async () => {
const emitted: SubtitleData[] = [];
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: (payload) => emitted.push(payload),
});
// New text schedules work, so an emit (and anything gated on it) will follow.
assert.equal(controller.onSubtitleChange('字幕'), true);
await flushMicrotasks();
// A repeat emits nothing, so callers must not wait on an emit that is never
// coming (subtitle prefetching would stay paused for the rest of the cue).
const emittedCount = emitted.length;
assert.equal(controller.onSubtitleChange('字幕'), false);
await flushMicrotasks();
assert.equal(emitted.length, emittedCount);
});
test('refreshCurrentSubtitle reports the empty-text emit that an in-flight run will deliver', async () => {
const emitted: SubtitleData[] = [];
let resolveFirst: ((value: SubtitleData | null) => void) | undefined;
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => {
if (text === '字幕') {
return await new Promise<SubtitleData | null>((resolve) => {
resolveFirst = resolve;
});
}
return { text, tokens: [] };
},
emitSubtitle: (payload) => emitted.push(payload),
});
controller.onSubtitleChange('字幕');
await flushMicrotasks();
// Clearing the subtitle while tokenization is in flight: the running loop
// picks the empty text up and emits it, so callers gated on that emit (the
// prefetch pause) must be told one is coming.
assert.equal(controller.refreshCurrentSubtitle(''), true);
resolveFirst?.({ text: '字幕', tokens: [] });
await flushMicrotasks();
await flushMicrotasks();
// '字幕' is the provisional plain emit the in-flight run already made before
// the refresh; '' is the emit the refresh promised.
assert.deepEqual(
emitted.map((payload) => payload.text),
['字幕', ''],
);
});
test('refreshCurrentSubtitle reports no emit for empty text when nothing is running', async () => {
const emitted: SubtitleData[] = [];
const controller = createSubtitleProcessingController({
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
emitSubtitle: (payload) => emitted.push(payload),
});
assert.equal(controller.refreshCurrentSubtitle(''), false);
await flushMicrotasks();
assert.deepEqual(emitted, []);
});
@@ -5,16 +5,30 @@ export interface SubtitleProcessingControllerDeps {
emitSubtitle: (payload: SubtitleData) => void;
logDebug?: (message: string) => void;
now?: () => number;
cacheLimit?: number;
}
/**
* Pure memory bound on the LRU, not a coverage limit: prefetching runs to the end of a
* file regardless of cache pressure. Sized to hold a feature-length title (a 24-minute
* episode runs 300-400 lines, a 2-hour film ~2000) plus room for lines that repeat across
* episodes of a series, so openings and endings stay warm between titles.
*/
export const DEFAULT_SUBTITLE_TOKENIZATION_CACHE_LIMIT = 2500;
export interface SubtitleProcessingController {
onSubtitleChange: (text: string) => void;
refreshCurrentSubtitle: (textOverride?: string) => void;
/**
* Returns whether the text was new and processing was scheduled. A false
* return means nothing will be emitted for this event, which callers that
* gate work on the emit (such as pausing subtitle prefetching) need to know.
*/
onSubtitleChange: (text: string) => boolean;
/** Same contract as onSubtitleChange: whether an emit is expected. */
refreshCurrentSubtitle: (textOverride?: string) => boolean;
invalidateTokenizationCache: () => void;
preCacheTokenization: (text: string, data: SubtitleData) => void;
consumeCachedSubtitle: (text: string) => SubtitleData | null;
hasCachedSubtitle: (text: string) => boolean;
isCacheFull: () => boolean;
}
export function normalizeSubtitleCacheKey(text: string): string {
@@ -24,9 +38,15 @@ export function normalizeSubtitleCacheKey(text: string): string {
export function createSubtitleProcessingController(
deps: SubtitleProcessingControllerDeps,
): SubtitleProcessingController {
const SUBTITLE_TOKENIZATION_CACHE_LIMIT = 256;
const SUBTITLE_TOKENIZATION_CACHE_LIMIT =
deps.cacheLimit && deps.cacheLimit > 0
? deps.cacheLimit
: DEFAULT_SUBTITLE_TOKENIZATION_CACHE_LIMIT;
let latestText = '';
let lastEmittedText = '';
// Tracks the latest provisional plain emit across rapid changes and loop retries
// so the same line is never shown plain twice.
let lastPlainEmittedText: string | null = null;
let cacheGeneration = 0;
let lastEmittedGeneration = 0;
let processing = false;
@@ -70,9 +90,12 @@ export function createSubtitleProcessingController(
const startedAtMs = now();
if (!text.trim()) {
if (lastPlainEmittedText !== text) {
deps.emitSubtitle({ text, tokens: null });
}
lastEmittedText = text;
lastEmittedGeneration = generation;
lastPlainEmittedText = null;
break;
}
@@ -82,11 +105,25 @@ export function createSubtitleProcessingController(
if (cachedTokenized) {
output = cachedTokenized;
} else {
// Cache miss: show the plain line on time; the tokenized payload
// upgrades it once ready. Skipped on refreshes of an already
// emitted line so downstream consumers never see a downgrade.
if (text !== lastEmittedText && text !== lastPlainEmittedText) {
deps.emitSubtitle({ text, tokens: null });
lastPlainEmittedText = text;
}
const tokenized = await deps.tokenizeSubtitle(text);
// A null result is a transient tokenizer failure, not a verdict on
// the line: caching the plain fallback would pin it untokenized for
// every later occurrence.
if (tokenized) {
output = tokenized;
// A result computed before an invalidation must not repopulate the
// fresh cache, or the retry below would serve the stale entry.
if (generation === cacheGeneration) {
setCachedTokenization(text, tokenized);
}
}
setCachedTokenization(text, output);
}
} catch (error) {
deps.logDebug?.(`Subtitle tokenization failed: ${(error as Error).message}`);
@@ -107,9 +144,16 @@ export function createSubtitleProcessingController(
continue;
}
// An untokenized result adds nothing when this line was already shown,
// either provisionally or as an earlier full emit (failed refresh) —
// emitting it would duplicate or downgrade what is on screen.
const plainAlreadyShown = lastPlainEmittedText === text || lastEmittedText === text;
if (!(output.tokens === null && output.text === text && plainAlreadyShown)) {
deps.emitSubtitle(output);
}
lastEmittedText = text;
lastEmittedGeneration = generation;
lastPlainEmittedText = null;
deps.logDebug?.(
`Subtitle tokenization delivered; elapsed=${now() - startedAtMs}ms, staleDrops=${staleDropCount}`,
);
@@ -133,25 +177,38 @@ export function createSubtitleProcessingController(
return {
onSubtitleChange: (text: string) => {
if (text === latestText) {
return;
// A run already in flight for this text will still emit for it.
return processing;
}
latestText = text;
if (
processing &&
text !== lastPlainEmittedText &&
!tokenizationCache.has(normalizeSubtitleCacheKey(text))
) {
deps.emitSubtitle({ text, tokens: null });
lastPlainEmittedText = text;
}
processLatest();
return true;
},
refreshCurrentSubtitle: (textOverride?: string) => {
if (typeof textOverride === 'string') {
latestText = textOverride;
}
if (!latestText.trim()) {
return;
// A run in flight will pick this up and emit the empty subtitle, so
// the caller is still waiting on an emit.
return processing;
}
if (
processing ||
(latestText === lastEmittedText && cacheGeneration === lastEmittedGeneration)
) {
return;
if (processing) {
return true;
}
if (latestText === lastEmittedText && cacheGeneration === lastEmittedGeneration) {
return false;
}
processLatest();
return true;
},
invalidateTokenizationCache: () => {
tokenizationCache.clear();
@@ -169,13 +226,11 @@ export function createSubtitleProcessingController(
latestText = text;
lastEmittedText = text;
lastEmittedGeneration = cacheGeneration;
lastPlainEmittedText = null;
return cached;
},
hasCachedSubtitle: (text: string) => {
return tokenizationCache.has(normalizeSubtitleCacheKey(text));
},
isCacheFull: () => {
return tokenizationCache.size >= SUBTITLE_TOKENIZATION_CACHE_LIMIT;
},
};
}
+35
View File
@@ -1,6 +1,7 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
isSubtitleAnnotationUpgrade,
serializeInitialSubtitleWebsocketMessage,
serializeSubtitleMarkup,
serializeSubtitleWebsocketMessage,
@@ -13,6 +14,40 @@ const frequencyOptions = {
mode: 'banded' as const,
};
test('annotation upgrade requires matching text and cue timing', () => {
const current: SubtitleData = {
text: '字幕',
tokens: null,
startTime: 10,
endTime: 12,
};
assert.equal(
isSubtitleAnnotationUpgrade(current, {
...current,
tokens: [],
}),
true,
);
assert.equal(
isSubtitleAnnotationUpgrade(current, {
...current,
tokens: [],
startTime: 11,
}),
false,
);
assert.equal(
isSubtitleAnnotationUpgrade(current, {
...current,
text: '次の字幕',
tokens: [],
}),
false,
);
assert.equal(isSubtitleAnnotationUpgrade(current, current), false);
});
test('serializeSubtitleMarkup escapes plain text and preserves line breaks', () => {
const payload: SubtitleData = {
text: 'a < b\nx & y',
+14
View File
@@ -20,6 +20,20 @@ export type SubtitleWebsocketFrequencyOptions = {
export type SubtitleWebsocketPayloadMode = 'plain' | 'annotated';
export function isSubtitleAnnotationUpgrade(
current: SubtitleData | null,
next: SubtitleData,
): boolean {
return (
current !== null &&
current.tokens === null &&
next.tokens !== null &&
current.text === next.text &&
current.startTime === next.startTime &&
current.endTime === next.endTime
);
}
type SubtitleWebsocketMessageOptions = {
payloadMode?: SubtitleWebsocketPayloadMode;
};
+2 -34
View File
@@ -2934,44 +2934,12 @@ test('tokenizeSubtitle preserves Yomitan compound token when MeCab components ar
return [];
}
if (script.includes('parseText')) {
return [
{
source: 'scanning-parser',
index: 0,
content: [
[
{
text: '取り組んで',
surface: '取り組んで',
reading: 'とりくんで',
headwords: [[{ term: '取り組む' }]],
},
],
[
{
text: 'もらいます',
reading: 'もらいます',
headwords: [[{ term: 'もらう' }]],
},
],
],
},
];
}
return [
{
surface: '取り',
reading: 'とり',
headword: '取る',
headword: '取り組む',
startPos: 0,
endPos: 2,
},
{
surface: '組んで',
reading: 'くんで',
headword: '組む',
startPos: 2,
endPos: 5,
},
{
+50 -2
View File
@@ -70,6 +70,7 @@ export interface TokenizerServiceDeps {
getNameMatchImagesEnabled?: () => boolean;
getCharacterNameImage?: (term: string) => CharacterNameImage | null;
getCurrentCharacterDictionaryMediaId?: () => number | null;
getCharacterNameCandidates?: () => { key: string; forms: string[] } | null;
getFrequencyDictionaryEnabled?: () => boolean;
getFrequencyDictionaryMatchMode?: () => FrequencyDictionaryMatchMode;
getFrequencyRank?: FrequencyDictionaryLookup;
@@ -106,6 +107,7 @@ export interface TokenizerDepsRuntimeOptions {
getNameMatchImagesEnabled?: () => boolean;
getCharacterNameImage?: (term: string) => CharacterNameImage | null;
getCurrentCharacterDictionaryMediaId?: () => number | null;
getCharacterNameCandidates?: () => { key: string; forms: string[] } | null;
getFrequencyDictionaryEnabled?: () => boolean;
getFrequencyDictionaryMatchMode?: () => FrequencyDictionaryMatchMode;
getFrequencyRank?: FrequencyDictionaryLookup;
@@ -266,6 +268,7 @@ export function createTokenizerDepsRuntime(
getNameMatchImagesEnabled: options.getNameMatchImagesEnabled,
getCharacterNameImage: options.getCharacterNameImage,
getCurrentCharacterDictionaryMediaId: options.getCurrentCharacterDictionaryMediaId,
getCharacterNameCandidates: options.getCharacterNameCandidates,
getFrequencyDictionaryEnabled: options.getFrequencyDictionaryEnabled,
getFrequencyDictionaryMatchMode: options.getFrequencyDictionaryMatchMode ?? (() => 'headword'),
getFrequencyRank: options.getFrequencyRank,
@@ -716,15 +719,30 @@ function getAnnotationOptions(deps: TokenizerServiceDeps): TokenizerAnnotationOp
};
}
// Per-line stage durations for the pipeline debug log; every field is filled in
// by the stage that awaits the corresponding work.
interface TokenizationStageTimings {
scanMs?: number;
mecabMs?: number;
frequencyMs?: number;
annotateMs?: number;
}
async function parseWithYomitanInternalParser(
text: string,
deps: TokenizerServiceDeps,
options: TokenizerAnnotationOptions,
stageTimings?: TokenizationStageTimings,
): Promise<MergedToken[] | null> {
const scanStartedAtMs = Date.now();
const selectedTokens = await requestYomitanScanTokens(text, deps, logger, {
includeNameMatchMetadata: options.nameMatchEnabled,
currentCharacterDictionaryMediaId: deps.getCurrentCharacterDictionaryMediaId?.() ?? null,
nameCandidates: deps.getCharacterNameCandidates?.() ?? null,
});
if (stageTimings) {
stageTimings.scanMs = Date.now() - scanStartedAtMs;
}
if (!selectedTokens || selectedTokens.length === 0) {
return null;
}
@@ -757,6 +775,7 @@ async function parseWithYomitanInternalParser(
const frequencyRankPromise: Promise<YomitanFrequencyIndex> = options.frequencyEnabled
? (async () => {
const frequencyStartedAtMs = Date.now();
const frequencyMatchMode = options.frequencyMatchMode;
const termReadingList = buildYomitanFrequencyTermReadingList(
normalizedSelectedTokens,
@@ -767,12 +786,17 @@ async function parseWithYomitanInternalParser(
deps,
logger,
);
return buildYomitanFrequencyIndex(yomitanFrequencies);
const frequencyIndex = buildYomitanFrequencyIndex(yomitanFrequencies);
if (stageTimings) {
stageTimings.frequencyMs = Date.now() - frequencyStartedAtMs;
}
return frequencyIndex;
})()
: Promise.resolve({ byPair: new Map(), byTerm: new Map() });
const mecabEnrichmentPromise: Promise<MergedToken[]> = needsMecabPosEnrichment(options)
? (async () => {
const mecabStartedAtMs = Date.now();
try {
const mecabTokens = await deps.tokenizeWithMecab(text);
const enrichTokensWithMecab = deps.enrichTokensWithMecab ?? enrichTokensWithMecabAsync;
@@ -786,6 +810,10 @@ async function parseWithYomitanInternalParser(
`textLength=${text.length}`,
);
return normalizedSelectedTokens;
} finally {
if (stageTimings) {
stageTimings.mecabMs = Date.now() - mecabStartedAtMs;
}
}
})()
: Promise.resolve(normalizedSelectedTokens);
@@ -876,15 +904,35 @@ export async function tokenizeSubtitle(
const annotationOptions = getAnnotationOptions(deps);
annotationOptions.sourceText = tokenizeText;
const yomitanTokens = await parseWithYomitanInternalParser(tokenizeText, deps, annotationOptions);
const stageTimings: TokenizationStageTimings = {};
const startedAtMs = Date.now();
const logStageTimings = (tokenCount: number): void => {
logger.debug(
`Subtitle tokenization stages; textLength=${tokenizeText.length}, tokenCount=${tokenCount}, ` +
`scanMs=${stageTimings.scanMs ?? '-'}, mecabMs=${stageTimings.mecabMs ?? '-'}, ` +
`frequencyMs=${stageTimings.frequencyMs ?? '-'}, annotateMs=${stageTimings.annotateMs ?? '-'}, ` +
`totalMs=${Date.now() - startedAtMs}`,
);
};
const yomitanTokens = await parseWithYomitanInternalParser(
tokenizeText,
deps,
annotationOptions,
stageTimings,
);
if (yomitanTokens && yomitanTokens.length > 0) {
const annotateStartedAtMs = Date.now();
const annotatedTokens = await applyAnnotationStage(yomitanTokens, deps, annotationOptions);
stageTimings.annotateMs = Date.now() - annotateStartedAtMs;
const renderedTokens = applyCharacterNameImages(annotatedTokens, deps, annotationOptions);
logStageTimings(renderedTokens.length);
return {
text: displayText,
tokens: renderedTokens.length > 0 ? renderedTokens : null,
};
}
logStageTimings(0);
return { text: displayText, tokens: null };
}
@@ -366,8 +366,11 @@ export function createReplayMessageStore(messages: GoldenRecordedMessage[]): Rep
};
}
async function runInjectedScriptInVm(script: string, store: ReplayMessageStore): Promise<unknown> {
return await vm.runInNewContext(script, {
// One persistent context per fixture, matching the real parser window: the
// scan runtime installs itself once into globalThis and later per-line call
// scripts reuse it.
function createInjectedScriptVm(store: ReplayMessageStore): (script: string) => Promise<unknown> {
const context = vm.createContext({
chrome: {
runtime: {
lastError: null,
@@ -393,6 +396,7 @@ async function runInjectedScriptInVm(script: string, store: ReplayMessageStore):
Set,
String,
});
return async (script: string) => await vm.runInContext(script, context);
}
export function createReplayTokenizerDeps(fixture: GoldenFixture): TokenizerServiceDeps {
@@ -400,13 +404,14 @@ export function createReplayTokenizerDeps(fixture: GoldenFixture): TokenizerServ
const scriptResults = new Map(
fixture.recording.scripts.map((entry) => [entry.sha256, entry] as const),
);
const runInjectedScriptInVm = createInjectedScriptVm(store);
const parserWindow = {
isDestroyed: () => false,
webContents: {
executeJavaScript: async (script: string) => {
try {
return await runInjectedScriptInVm(script, store);
return await runInjectedScriptInVm(script);
} catch (vmError) {
const recorded = scriptResults.get(hashInjectedScript(script));
if (recorded) {
File diff suppressed because it is too large Load Diff
@@ -3,6 +3,14 @@ import * as fs from 'fs';
import * as http from 'http';
import * as path from 'path';
import { selectYomitanParseTokens } from './parser-selection-stage';
import {
buildYomitanScanCallScript,
buildYomitanScanNameCandidatesScript,
CHARACTER_DICTIONARY_TITLE_PREFIX,
YOMITAN_SCAN_RUNTIME_INSTALL_SCRIPT,
YOMITAN_SCAN_RUNTIME_MISSING_SENTINEL,
type YomitanFrequencyMode,
} from './yomitan-scan-runtime-script';
interface LoggerLike {
error: (message: string, ...args: unknown[]) => void;
@@ -22,8 +30,6 @@ interface YomitanParserRuntimeDeps {
createYomitanExtensionWindow?: (pageName: string) => Promise<BrowserWindow | null>;
}
type YomitanFrequencyMode = 'occurrence-based' | 'rank-based';
export interface YomitanDictionaryInfo {
title: string;
revision?: string | number;
@@ -74,13 +80,19 @@ export interface YomitanAddNoteResult {
}
const DEFAULT_YOMITAN_SCAN_LENGTH = 40;
const CHARACTER_DICTIONARY_TITLE_PREFIX = 'SubMiner Character Dictionary';
const yomitanProfileMetadataByWindow = new WeakMap<BrowserWindow, YomitanProfileMetadata>();
const yomitanProfileDiagnosticsLoggedByWindow = new WeakSet<BrowserWindow>();
const yomitanFrequencyCacheByWindow = new WeakMap<
BrowserWindow,
Map<string, YomitanTermFrequency[]>
>();
// Epoch passed with every scan request; the in-window termsFind cache clears
// itself when the epoch changes (dictionary imports, settings changes).
const yomitanScanCacheEpochByWindow = new WeakMap<BrowserWindow, number>();
function getYomitanScanCacheEpoch(window: BrowserWindow): number {
return yomitanScanCacheEpochByWindow.get(window) ?? 0;
}
function isObject(value: unknown): value is Record<string, unknown> {
return Boolean(value && typeof value === 'object');
@@ -99,6 +111,7 @@ function isScanTokenArray(value: unknown): value is YomitanScanToken[] {
typeof entry.startPos === 'number' &&
typeof entry.endPos === 'number' &&
(entry.isNameMatch === undefined || typeof entry.isNameMatch === 'boolean') &&
(entry.isUnparsedRun === undefined || typeof entry.isUnparsedRun === 'boolean') &&
(entry.frequencyRank === undefined || typeof entry.frequencyRank === 'number') &&
(entry.wordClasses === undefined ||
(Array.isArray(entry.wordClasses) &&
@@ -107,13 +120,9 @@ function isScanTokenArray(value: unknown): value is YomitanScanToken[] {
);
}
function scanTokenSpanKey(token: YomitanScanToken): string {
return `${token.startPos}:${token.endPos}:${token.surface}`;
}
// Maps a parse-selected token to the scanner-token shape carried out of the
// parser runtime. Shared by both selectYomitanParseTokens fallback paths so the
// projected fields stay in sync as the shape changes.
// parser runtime, used by the parseText fallback path when the in-window
// scanner is unavailable.
function toYomitanScanToken(token: {
surface: string;
reading: string;
@@ -132,66 +141,6 @@ function toYomitanScanToken(token: {
};
}
// parseText segmentation is authoritative (it emits filler chunks for text the
// termsFind scanner skips), but only the termsFind scanner carries annotation
// metadata (isNameMatch, frequencyRank, headwordReading, wordClasses). Graft
// scanner tokens onto the parseText segmentation per matching span so one
// unmatched chunk degrades only itself instead of dropping the whole line's
// metadata.
//
// Exception: character-name tokens. The greedy name scan can re-segment text
// around a name (e.g. とヨータ → と + ヨータ instead of とヨー + タ), so
// parseText segmentation cannot be authoritative there. Each name span is
// expanded until it aligns with token boundaries in both segmentations, then
// the parse tokens inside are replaced with the scanner tokens.
function mergeScannerTokensIntoParseTokens(
parseScanTokens: YomitanScanToken[],
scannerTokens: YomitanScanToken[],
): YomitanScanToken[] {
const scannerTokensBySpan = new Map<string, YomitanScanToken>();
for (const token of scannerTokens) {
scannerTokensBySpan.set(scanTokenSpanKey(token), token);
}
const graftedTokens = parseScanTokens.map(
(token) => scannerTokensBySpan.get(scanTokenSpanKey(token)) ?? token,
);
const nameTokens = scannerTokens.filter((token) => token.isNameMatch === true);
if (nameTokens.length === 0) {
return graftedTokens;
}
const regions = nameTokens.map((token) => ({ start: token.startPos, end: token.endPos }));
const allTokens = [...parseScanTokens, ...scannerTokens];
let expanded = true;
while (expanded) {
expanded = false;
for (const region of regions) {
for (const token of allTokens) {
const overlaps = token.startPos < region.end && token.endPos > region.start;
const extendsBeyond = token.startPos < region.start || token.endPos > region.end;
if (overlaps && extendsBeyond) {
region.start = Math.min(region.start, token.startPos);
region.end = Math.max(region.end, token.endPos);
expanded = true;
}
}
}
}
const isInsideNameRegion = (token: YomitanScanToken): boolean =>
regions.some((region) => token.startPos >= region.start && token.endPos <= region.end);
const merged = graftedTokens.filter((token) => !isInsideNameRegion(token));
for (const token of scannerTokens) {
if (isInsideNameRegion(token)) {
merged.push(token);
}
}
merged.sort((a, b) => a.startPos - b.startPos || a.endPos - b.endPos);
return merged;
}
function makeTermReadingCacheKey(term: string, reading: string | null): string {
return `${term}\u0000${reading ?? ''}`;
}
@@ -208,6 +157,7 @@ function getWindowFrequencyCache(window: BrowserWindow): Map<string, YomitanTerm
function clearWindowCaches(window: BrowserWindow): void {
yomitanProfileMetadataByWindow.delete(window);
yomitanFrequencyCacheByWindow.delete(window);
yomitanScanCacheEpochByWindow.set(window, getYomitanScanCacheEpoch(window) + 1);
}
export function clearYomitanParserCachesForWindow(window: BrowserWindow): void {
clearWindowCaches(window);
@@ -704,6 +654,10 @@ async function ensureYomitanParserWindow(
if (readyPromise) {
await readyPromise;
}
// Eagerly install the scan runtime so the first subtitle line does not
// pay the install round trip; failures fall back to the per-request
// install-and-retry path.
await installYomitanScanRuntime(parserWindow).catch(() => {});
return true;
} catch (err) {
@@ -877,668 +831,42 @@ async function serveDictionaryZipOnce<T>(
}
}
const YOMITAN_SCANNING_HELPERS = String.raw`
const HIRAGANA_CONVERSION_RANGE = [0x3041, 0x3096];
const KATAKANA_CONVERSION_RANGE = [0x30a1, 0x30f6];
const KANA_PROLONGED_SOUND_MARK_CODE_POINT = 0x30fc;
const KATAKANA_SMALL_KA_CODE_POINT = 0x30f5;
const KATAKANA_SMALL_KE_CODE_POINT = 0x30f6;
const KANA_RANGES = [[0x3040, 0x309f], [0x30a0, 0x30ff]];
const JAPANESE_RANGES = [[0x3040, 0x30ff], [0x3400, 0x9fff]];
function isCodePointInRange(codePoint, range) { return codePoint >= range[0] && codePoint <= range[1]; }
function isCodePointInRanges(codePoint, ranges) { return ranges.some((range) => isCodePointInRange(codePoint, range)); }
function isCodePointKana(codePoint) { return isCodePointInRanges(codePoint, KANA_RANGES); }
function isCodePointJapanese(codePoint) { return isCodePointInRanges(codePoint, JAPANESE_RANGES); }
function createFuriganaSegment(text, reading) { return {text, reading}; }
function getSegmentReadingContribution(segment) {
if (typeof segment.reading === "string" && segment.reading.length > 0) { return segment.reading; }
const segmentText = typeof segment.text === "string" ? segment.text : "";
const isKanaOnly = segmentText.length > 0 && [...segmentText].every((char) => isCodePointKana(char.codePointAt(0)));
return isKanaOnly ? segmentText : "";
}
function getProlongedHiragana(previousCharacter) {
switch (previousCharacter) {
case "あ": case "か": case "が": case "さ": case "ざ": case "た": case "だ": case "な": case "は": case "ば": case "ぱ": case "ま": case "や": case "ら": case "わ": case "ぁ": case "ゃ": case "ゎ": return "あ";
case "い": case "き": case "ぎ": case "し": case "じ": case "ち": case "ぢ": case "に": case "ひ": case "び": case "ぴ": case "み": case "り": case "ぃ": return "い";
case "う": case "く": case "ぐ": case "す": case "ず": case "つ": case "づ": case "ぬ": case "ふ": case "ぶ": case "ぷ": case "む": case "ゆ": case "る": case "ぅ": case "ゅ": return "う";
case "え": case "け": case "げ": case "せ": case "ぜ": case "て": case "で": case "ね": case "へ": case "べ": case "ぺ": case "め": case "れ": case "ぇ": return "え";
case "お": case "こ": case "ご": case "そ": case "ぞ": case "と": case "ど": case "の": case "ほ": case "ぼ": case "ぽ": case "も": case "よ": case "ろ": case "を": case "ぉ": case "ょ": return "う";
default: return null;
}
}
function getFuriganaKanaSegments(text, reading) {
const newSegments = [];
let start = 0;
let state = (reading[0] === text[0]);
for (let i = 1; i < text.length; ++i) {
const newState = (reading[i] === text[i]);
if (state === newState) { continue; }
newSegments.push(createFuriganaSegment(text.substring(start, i), state ? '' : reading.substring(start, i)));
state = newState;
start = i;
}
newSegments.push(createFuriganaSegment(text.substring(start), state ? '' : reading.substring(start)));
return newSegments;
}
function convertKatakanaToHiragana(text, keepProlongedSoundMarks = false) {
let result = '';
const offset = (HIRAGANA_CONVERSION_RANGE[0] - KATAKANA_CONVERSION_RANGE[0]);
for (let char of text) {
const codePoint = char.codePointAt(0);
switch (codePoint) {
case KATAKANA_SMALL_KA_CODE_POINT:
case KATAKANA_SMALL_KE_CODE_POINT:
break;
case KANA_PROLONGED_SOUND_MARK_CODE_POINT:
if (!keepProlongedSoundMarks && result.length > 0) {
const char2 = getProlongedHiragana(result[result.length - 1]);
if (char2 !== null) { char = char2; }
}
break;
default:
if (isCodePointInRange(codePoint, KATAKANA_CONVERSION_RANGE)) {
char = String.fromCodePoint(codePoint + offset);
}
break;
}
result += char;
}
return result;
}
function segmentizeFurigana(reading, readingNormalized, groups, groupsStart) {
const groupCount = groups.length - groupsStart;
if (groupCount <= 0) { return reading.length === 0 ? [] : null; }
const group = groups[groupsStart];
const {isKana, text} = group;
if (isKana) {
if (group.textNormalized !== null && readingNormalized.startsWith(group.textNormalized)) {
const segments = segmentizeFurigana(reading.substring(text.length), readingNormalized.substring(text.length), groups, groupsStart + 1);
if (segments !== null) {
if (reading.startsWith(text)) { segments.unshift(createFuriganaSegment(text, '')); }
else { segments.unshift(...getFuriganaKanaSegments(text, reading)); }
return segments;
}
}
return null;
}
let result = null;
for (let i = reading.length; i >= text.length; --i) {
const segments = segmentizeFurigana(reading.substring(i), readingNormalized.substring(i), groups, groupsStart + 1);
if (segments !== null) {
if (result !== null) { return null; }
segments.unshift(createFuriganaSegment(text, reading.substring(0, i)));
result = segments;
}
if (groupCount === 1) { break; }
}
return result;
}
function distributeFurigana(term, reading) {
if (reading === term) { return [createFuriganaSegment(term, '')]; }
const groups = [];
let groupPre = null;
let isKanaPre = null;
for (const c of term) {
const isKana = isCodePointKana(c.codePointAt(0));
if (isKana === isKanaPre) { groupPre.text += c; }
else {
groupPre = {isKana, text: c, textNormalized: null};
groups.push(groupPre);
isKanaPre = isKana;
}
}
for (const group of groups) {
if (group.isKana) { group.textNormalized = convertKatakanaToHiragana(group.text); }
}
const segments = segmentizeFurigana(reading, convertKatakanaToHiragana(reading), groups, 0);
return segments !== null ? segments : [createFuriganaSegment(term, reading)];
}
function getStemLength(text1, text2) {
const minLength = Math.min(text1.length, text2.length);
if (minLength === 0) { return 0; }
let i = 0;
while (true) {
const char1 = text1.codePointAt(i);
const char2 = text2.codePointAt(i);
if (char1 !== char2) { break; }
const charLength = String.fromCodePoint(char1).length;
i += charLength;
if (i >= minLength) {
if (i > minLength) { i -= charLength; }
break;
}
}
return i;
}
function distributeFuriganaInflected(term, reading, source) {
const termNormalized = convertKatakanaToHiragana(term);
const readingNormalized = convertKatakanaToHiragana(reading);
const sourceNormalized = convertKatakanaToHiragana(source);
let mainText = term;
let stemLength = getStemLength(termNormalized, sourceNormalized);
const readingStemLength = getStemLength(readingNormalized, sourceNormalized);
if (readingStemLength > 0 && readingStemLength >= stemLength) {
mainText = reading;
stemLength = readingStemLength;
reading = source.substring(0, stemLength) + reading.substring(stemLength);
}
const segments = [];
if (stemLength > 0) {
mainText = source.substring(0, stemLength) + mainText.substring(stemLength);
const segments2 = distributeFurigana(mainText, reading);
let consumed = 0;
for (const segment of segments2) {
const start = consumed;
consumed += segment.text.length;
if (consumed < stemLength) { segments.push(segment); }
else if (consumed === stemLength) { segments.push(segment); break; }
else {
if (start < stemLength) { segments.push(createFuriganaSegment(mainText.substring(start, stemLength), '')); }
break;
}
}
}
if (stemLength < source.length) {
const remainder = source.substring(stemLength);
const last = segments[segments.length - 1];
if (last && last.reading.length === 0) { last.text += remainder; }
else { segments.push(createFuriganaSegment(remainder, '')); }
}
return segments;
}
function parsePositiveFrequencyNumber(value) {
if (typeof value === 'number' && Number.isFinite(value) && value > 0) {
return Math.max(1, Math.floor(value));
}
if (typeof value === 'string') {
const numericMatch = value.trim().match(/[+-]?(\d+(\.\d*)?|\.\d+)([eE][+-]?\d+)?/)?.[0];
if (!numericMatch) { return null; }
const parsed = Number.parseFloat(numericMatch);
if (!Number.isFinite(parsed) || parsed <= 0) { return null; }
return Math.max(1, Math.floor(parsed));
}
if (Array.isArray(value)) {
for (const item of value) {
const parsed = parsePositiveFrequencyNumber(item);
if (parsed !== null) { return parsed; }
}
}
return null;
}
function parseDisplayFrequencyNumber(value) {
if (typeof value === 'string') {
const leadingDigits = value.trim().match(/^\d+/)?.[0];
if (!leadingDigits) { return null; }
const parsed = Number.parseInt(leadingDigits, 10);
return Number.isFinite(parsed) && parsed > 0 ? parsed : null;
}
return parsePositiveFrequencyNumber(value);
}
function getFrequencyDictionaryName(frequency) {
const candidates = [
frequency?.dictionary,
frequency?.dictionaryName,
frequency?.name,
frequency?.title,
frequency?.dictionaryTitle,
frequency?.dictionaryAlias
];
for (const candidate of candidates) {
if (typeof candidate === 'string' && candidate.trim().length > 0) {
return candidate.trim();
}
}
return null;
}
function getBestFrequencyRank(dictionaryEntry, headwordIndex, dictionaryPriorityByName, dictionaryFrequencyModeByName) {
let best = null;
const headwordCount = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords.length : 0;
for (const frequency of dictionaryEntry?.frequencies || []) {
if (!frequency || typeof frequency !== 'object') { continue; }
const frequencyHeadwordIndex = frequency.headwordIndex;
if (typeof frequencyHeadwordIndex === 'number') {
if (frequencyHeadwordIndex !== headwordIndex) { continue; }
} else if (headwordCount > 1) {
continue;
}
const dictionary = getFrequencyDictionaryName(frequency);
if (!dictionary) { continue; }
if (dictionaryFrequencyModeByName[dictionary] === 'occurrence-based') { continue; }
const rank =
parseDisplayFrequencyNumber(frequency.displayValue) ??
parsePositiveFrequencyNumber(frequency.frequency);
if (rank === null) { continue; }
const priorityRaw = dictionaryPriorityByName[dictionary];
const fallbackPriority =
typeof frequency.dictionaryIndex === 'number' && Number.isFinite(frequency.dictionaryIndex)
? Math.max(0, Math.floor(frequency.dictionaryIndex))
: Number.MAX_SAFE_INTEGER;
const priority =
typeof priorityRaw === 'number' && Number.isFinite(priorityRaw)
? Math.max(0, Math.floor(priorityRaw))
: fallbackPriority;
if (best === null || priority < best.priority || (priority === best.priority && rank < best.rank)) {
best = { priority, rank };
}
}
return best?.rank ?? null;
}
function hasExactSource(headword, token, requirePrimary) {
for (const src of headword.sources || []) {
if (src.originalText !== token) { continue; }
if (requirePrimary && !src.isPrimary) { continue; }
if (src.matchType !== 'exact') { continue; }
return true;
}
return false;
}
function collectExactHeadwordMatches(dictionaryEntries, token, requirePrimary) {
const matches = [];
for (const dictionaryEntry of dictionaryEntries || []) {
const headwords = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords : [];
for (let headwordIndex = 0; headwordIndex < headwords.length; headwordIndex += 1) {
const headword = headwords[headwordIndex];
if (!hasExactSource(headword, token, requirePrimary)) { continue; }
matches.push({ dictionaryEntry, headword, headwordIndex });
}
}
return matches;
}
function sameHeadword(match, preferredMatch) {
if (!match || !preferredMatch) {
return false;
}
if (match.headword?.term !== preferredMatch.headword?.term) {
return false;
}
const matchReading = typeof match.headword?.reading === 'string' ? match.headword.reading : '';
const preferredReading =
typeof preferredMatch.headword?.reading === 'string' ? preferredMatch.headword.reading : '';
if (!matchReading || !preferredReading) {
return true;
}
return matchReading === preferredReading;
}
function getBestFrequencyRankForMatches(matches, dictionaryPriorityByName, dictionaryFrequencyModeByName) {
let best = null;
for (const match of matches) {
const rank = getBestFrequencyRank(
match.dictionaryEntry,
match.headwordIndex,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
);
if (rank === null) { continue; }
if (best === null || rank < best) {
best = rank;
}
}
return best;
}
function normalizeWordClasses(headword) {
if (!Array.isArray(headword?.wordClasses)) { return undefined; }
const classes = headword.wordClasses.filter((wordClass) => typeof wordClass === "string" && wordClass.trim().length > 0);
return classes.length > 0 ? classes : undefined;
}
function appendDictionaryNames(target, value) {
if (!value || typeof value !== 'object') {
return;
}
const candidates = [
value.dictionary,
value.dictionaryName,
value.name,
value.title,
value.dictionaryTitle,
value.dictionaryAlias
];
for (const candidate of candidates) {
if (typeof candidate === 'string' && candidate.trim().length > 0) {
target.push(candidate.trim());
}
}
}
function getDictionaryEntryNames(entry) {
const names = [];
appendDictionaryNames(names, entry);
for (const definition of entry?.definitions || []) {
appendDictionaryNames(names, definition);
}
for (const frequency of entry?.frequencies || []) {
appendDictionaryNames(names, frequency);
}
for (const pronunciation of entry?.pronunciations || []) {
appendDictionaryNames(names, pronunciation);
}
return names;
}
function isNameDictionaryEntry(entry) {
if (!includeNameMatchMetadata || !entry || typeof entry !== 'object') {
return false;
}
return getDictionaryEntryNames(entry).some((name) => name.startsWith(${JSON.stringify(CHARACTER_DICTIONARY_TITLE_PREFIX)}));
}
function parseSubMinerMediaIdFromString(value) {
const imageMatch = value.match(/\bimg\/m(\d+)-/i);
if (imageMatch) {
const parsed = Number.parseInt(imageMatch[1], 10);
if (Number.isSafeInteger(parsed) && parsed > 0) { return parsed; }
}
const titleMatch = value.match(/${CHARACTER_DICTIONARY_TITLE_PREFIX}[^\d]*(?:AniList\s*)?(\d+)/i);
if (titleMatch) {
const parsed = Number.parseInt(titleMatch[1], 10);
if (Number.isSafeInteger(parsed) && parsed > 0) { return parsed; }
}
return null;
}
function parseSubMinerMediaIdCandidate(value) {
if (typeof value === 'number' && Number.isSafeInteger(value) && value > 0) {
return value;
}
if (typeof value === 'string' && /^\d+$/.test(value.trim())) {
const parsed = Number.parseInt(value.trim(), 10);
if (Number.isSafeInteger(parsed) && parsed > 0) { return parsed; }
}
return null;
}
function collectSubMinerMediaIds(value, target) {
if (typeof value === 'string') {
const parsed = parseSubMinerMediaIdFromString(value);
if (parsed !== null) { target.add(parsed); }
return;
}
if (!value || typeof value !== 'object') {
return;
}
if (Array.isArray(value)) {
for (const item of value) { collectSubMinerMediaIds(item, target); }
return;
}
const mediaIdCandidates = [
value.subminerMediaId,
value.subMinerMediaId,
value.characterDictionaryMediaId,
value.data?.subminerMediaId,
value.data?.subMinerMediaId,
value.data?.characterDictionaryMediaId
];
for (const candidate of mediaIdCandidates) {
const parsed = parseSubMinerMediaIdCandidate(candidate);
if (parsed !== null) { target.add(parsed); }
}
for (const child of Object.values(value)) {
collectSubMinerMediaIds(child, target);
}
}
function getSubMinerMediaIds(entry) {
const mediaIds = new Set();
collectSubMinerMediaIds(entry, mediaIds);
return mediaIds;
}
function isCurrentMediaNameDictionaryEntry(entry) {
if (!isNameDictionaryEntry(entry)) {
return false;
}
if (currentCharacterDictionaryMediaId === null) {
return true;
}
const mediaIds = getSubMinerMediaIds(entry);
return mediaIds.size === 0 || mediaIds.has(currentCharacterDictionaryMediaId);
}
function findLongestNameMatch(dictionaryEntries, textWindow) {
let best = null;
for (const dictionaryEntry of dictionaryEntries || []) {
if (!isCurrentMediaNameDictionaryEntry(dictionaryEntry)) { continue; }
const headwords = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords : [];
for (let headwordIndex = 0; headwordIndex < headwords.length; headwordIndex += 1) {
const headword = headwords[headwordIndex];
for (const src of headword?.sources || []) {
if (src.matchType !== 'exact' || src.isPrimary !== true) { continue; }
const originalText = typeof src.originalText === 'string' ? src.originalText : '';
if (!originalText || !textWindow.startsWith(originalText)) { continue; }
if (best === null || originalText.length > best.sourceLength) {
best = { dictionaryEntry, headword, headwordIndex, sourceLength: originalText.length };
}
}
}
}
return best;
}
function findLongestGenericMatchLength(dictionaryEntries, textWindow) {
let best = 0;
for (const dictionaryEntry of dictionaryEntries || []) {
if (isNameDictionaryEntry(dictionaryEntry)) { continue; }
const headwords = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords : [];
for (const headword of headwords) {
for (const src of headword?.sources || []) {
if (src.matchType !== 'exact' || src.isPrimary !== true) { continue; }
const originalText = typeof src.originalText === 'string' ? src.originalText : '';
if (!originalText || !textWindow.startsWith(originalText)) { continue; }
if (originalText.length > best) { best = originalText.length; }
}
}
}
return best;
}
function getPreferredHeadword(dictionaryEntries, token, dictionaryPriorityByName, dictionaryFrequencyModeByName) {
const currentMediaDictionaryEntries =
currentCharacterDictionaryMediaId === null
? (dictionaryEntries || [])
: (dictionaryEntries || []).filter((entry) => {
if (!isNameDictionaryEntry(entry)) { return true; }
return isCurrentMediaNameDictionaryEntry(entry);
});
const exactPrimaryMatches = collectExactHeadwordMatches(currentMediaDictionaryEntries, token, true);
let matchedNameDictionary = false;
if (includeNameMatchMetadata) {
for (const dictionaryEntry of currentMediaDictionaryEntries || []) {
if (!isCurrentMediaNameDictionaryEntry(dictionaryEntry)) { continue; }
for (const match of exactPrimaryMatches) {
if (match.dictionaryEntry !== dictionaryEntry) { continue; }
matchedNameDictionary = true;
break;
}
if (matchedNameDictionary) { break; }
}
}
const preferredMatch = exactPrimaryMatches[0];
if (preferredMatch) {
const exactFrequencyMatches = collectExactHeadwordMatches(currentMediaDictionaryEntries, token, false)
.filter((match) => sameHeadword(match, preferredMatch));
return {
term: preferredMatch.headword.term,
reading: preferredMatch.headword.reading,
wordClasses: normalizeWordClasses(preferredMatch.headword),
isNameMatch:
matchedNameDictionary || isCurrentMediaNameDictionaryEntry(preferredMatch.dictionaryEntry),
frequencyRank: getBestFrequencyRankForMatches(
exactFrequencyMatches.length > 0 ? exactFrequencyMatches : exactPrimaryMatches,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
)
};
}
return null;
}
`;
async function installYomitanScanRuntime(parserWindow: BrowserWindow): Promise<void> {
await parserWindow.webContents.executeJavaScript(YOMITAN_SCAN_RUNTIME_INSTALL_SCRIPT, true);
// A fresh runtime has no candidate list; force the next scan to reinstall it.
yomitanScanNameCandidateKeyByWindow.delete(parserWindow);
}
function buildYomitanScanningScript(
text: string,
profileIndex: number,
scanLength: number,
includeNameMatchMetadata: boolean,
greedyNameScanEnabled: boolean,
currentCharacterDictionaryMediaId: number | null,
dictionaryPriorityByName: Record<string, number>,
dictionaryFrequencyModeByName: Partial<Record<string, YomitanFrequencyMode>>,
): string {
return `
(async () => {
const invoke = (action, params) =>
new Promise((resolve, reject) => {
chrome.runtime.sendMessage({ action, params }, (response) => {
if (chrome.runtime.lastError) {
reject(new Error(chrome.runtime.lastError.message));
// Key of the character-name candidate list currently installed in each parser
// window, so an unchanged list costs nothing per line.
const yomitanScanNameCandidateKeyByWindow = new WeakMap<BrowserWindow, string>();
async function ensureYomitanScanNameCandidates(
parserWindow: BrowserWindow,
nameCandidates: { key: string; forms: string[] } | null,
logger: LoggerLike,
): Promise<void> {
const installedKey = yomitanScanNameCandidateKeyByWindow.get(parserWindow);
const nextKey = nameCandidates?.key ?? '';
if (installedKey === nextKey) {
return;
}
if (!response || typeof response !== "object") {
reject(new Error("Invalid response from Yomitan backend"));
return;
}
if (response.error) {
reject(new Error(response.error.message || "Yomitan backend error"));
return;
}
resolve(response.result);
});
});
${YOMITAN_SCANNING_HELPERS}
const includeNameMatchMetadata = ${includeNameMatchMetadata ? 'true' : 'false'};
const greedyNameScanEnabled = ${greedyNameScanEnabled ? 'true' : 'false'};
const currentCharacterDictionaryMediaId = ${
currentCharacterDictionaryMediaId !== null
? String(currentCharacterDictionaryMediaId)
: 'null'
};
const dictionaryPriorityByName = ${JSON.stringify(dictionaryPriorityByName)};
const dictionaryFrequencyModeByName = ${JSON.stringify(dictionaryFrequencyModeByName)};
const text = ${JSON.stringify(text)};
const details = {matchType: "exact", deinflect: true};
const tokens = [];
const termsFindCache = new Map();
async function termsFindAt(position, windowLength) {
const cacheKey = position + ":" + windowLength;
const cached = termsFindCache.get(cacheKey);
if (cached) { return cached; }
const substring = text.substring(position, position + windowLength);
const result = await invoke("termsFind", { text: substring, details, optionsContext: { index: ${profileIndex} } });
termsFindCache.set(cacheKey, result);
return result;
}
function buildScanToken(position, source, preferredHeadword) {
const reading = typeof preferredHeadword.reading === "string" ? preferredHeadword.reading : "";
const segments = distributeFuriganaInflected(preferredHeadword.term, reading, source);
const tokenPayload = {
surface: segments.map((segment) => segment.text).join("") || source,
reading: segments.map(getSegmentReadingContribution).join(""),
headword: preferredHeadword.term,
headwordReading: reading || undefined,
startPos: position,
endPos: position + source.length,
isNameMatch: includeNameMatchMetadata && preferredHeadword.isNameMatch === true,
frequencyRank:
typeof preferredHeadword.frequencyRank === "number" && Number.isFinite(preferredHeadword.frequencyRank)
? Math.max(1, Math.floor(preferredHeadword.frequencyRank))
: undefined,
};
if (Array.isArray(preferredHeadword.wordClasses) && preferredHeadword.wordClasses.length > 0) {
tokenPayload.wordClasses = preferredHeadword.wordClasses;
}
return tokenPayload;
}
async function findTokenAt(position, windowLength) {
const codePoint = text.codePointAt(position);
const character = String.fromCodePoint(codePoint);
const result = await termsFindAt(position, windowLength);
const dictionaryEntries = Array.isArray(result?.dictionaryEntries) ? result.dictionaryEntries : [];
const originalTextLength = typeof result?.originalTextLength === "number" ? result.originalTextLength : 0;
if (dictionaryEntries.length === 0 || originalTextLength <= 0 || (originalTextLength === character.length && !isCodePointJapanese(codePoint))) {
return { token: null, matchedLength: 0 };
}
const source = text.substring(position, position + originalTextLength);
const preferredHeadword = getPreferredHeadword(
dictionaryEntries,
source,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
try {
await parserWindow.webContents.executeJavaScript(
buildYomitanScanNameCandidatesScript(nameCandidates),
true,
);
if (!preferredHeadword || typeof preferredHeadword.term !== "string") {
return { token: null, matchedLength: originalTextLength };
yomitanScanNameCandidateKeyByWindow.set(parserWindow, nextKey);
} catch (err) {
// The scan falls back to checking every position when the list is absent,
// so a failed install costs speed, never a missed name.
logger.warn?.(
'Failed to install Yomitan character-name scan candidates:',
(err as Error).message,
);
yomitanScanNameCandidateKeyByWindow.delete(parserWindow);
}
return { token: buildScanToken(position, source, preferredHeadword), matchedLength: originalTextLength };
}
// Greedy name pre-pass: character-name matches claim their spans before
// the left-to-right walk, so a longer generic match starting earlier
// (e.g. とヨー → 渡洋) cannot swallow the start of a name (ヨータ).
const nameTokens = [];
if (greedyNameScanEnabled) {
let namePos = 0;
while (namePos < text.length) {
const codePoint = text.codePointAt(namePos);
if (!isCodePointJapanese(codePoint)) {
namePos += String.fromCodePoint(codePoint).length;
continue;
}
const result = await termsFindAt(namePos, ${scanLength});
const dictionaryEntries = Array.isArray(result?.dictionaryEntries) ? result.dictionaryEntries : [];
const textWindow = text.substring(namePos, namePos + ${scanLength});
const nameMatch = findLongestNameMatch(dictionaryEntries, textWindow);
// A name only claims its span when no strictly longer generic word
// starts at the same position (a character named 空 must not split
// 空気). Ties go to the name. Generic matches that start earlier and
// overlap the name are still blocked by the reservation.
if (
!nameMatch ||
findLongestGenericMatchLength(dictionaryEntries, textWindow) > nameMatch.sourceLength
) {
namePos += String.fromCodePoint(codePoint).length;
continue;
}
const source = text.substring(namePos, namePos + nameMatch.sourceLength);
nameTokens.push(buildScanToken(namePos, source, {
term: nameMatch.headword.term,
reading: nameMatch.headword.reading,
wordClasses: normalizeWordClasses(nameMatch.headword),
isNameMatch: true,
frequencyRank: getBestFrequencyRank(
nameMatch.dictionaryEntry,
nameMatch.headwordIndex,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
)
}));
namePos += nameMatch.sourceLength;
}
}
let i = 0;
let nameIndex = 0;
while (i < text.length) {
while (nameIndex < nameTokens.length && nameTokens[nameIndex].startPos < i) { nameIndex += 1; }
const nextNameToken = nameIndex < nameTokens.length ? nameTokens[nameIndex] : null;
if (nextNameToken && nextNameToken.startPos === i) {
tokens.push(nextNameToken);
i = nextNameToken.endPos;
nameIndex += 1;
continue;
}
// Cap the window at the next reserved name span so a generic match
// cannot consume into it.
const windowLength = nextNameToken ? Math.min(${scanLength}, nextNameToken.startPos - i) : ${scanLength};
let attempt = await findTokenAt(i, windowLength);
// Yomitan text normalization can consume characters (whitespace,
// punctuation) beyond the matched term, leaving no headword whose
// source equals the consumed text. Retry with shorter windows so a
// valid prefix term (e.g. a character name before a paren) still
// tokenizes instead of the position being skipped.
let retryLength = Math.min(attempt.matchedLength, windowLength) - 1;
while (!attempt.token && retryLength >= 1) {
const retry = await findTokenAt(i, retryLength);
if (retry.token) {
attempt = retry;
break;
}
retryLength = Math.min(retryLength - 1, retry.matchedLength - 1);
}
if (attempt.token) {
tokens.push(attempt.token);
i += attempt.matchedLength;
continue;
}
i += String.fromCodePoint(text.codePointAt(i)).length;
}
return tokens;
})();
`;
}
export async function requestYomitanParseResults(
@@ -1635,6 +963,20 @@ export async function requestYomitanParseResults(
}
}
// parseText fallback for when the in-window scanner cannot run (script eval
// failure, unexpected payload). The scanner walk is the primary tokenizer and
// emits its own filler runs, so this extra full parse only happens on errors.
async function requestYomitanParseFallbackTokens(
text: string,
deps: YomitanParserRuntimeDeps,
logger: LoggerLike,
): Promise<YomitanScanToken[] | null> {
const parseResults = await requestYomitanParseResults(text, deps, logger);
const selectedTokens = selectYomitanParseTokens(parseResults, () => false, 'headword');
const parseScanTokens = selectedTokens?.map(toYomitanScanToken) ?? null;
return parseScanTokens && parseScanTokens.length > 0 ? parseScanTokens : null;
}
export async function requestYomitanScanTokens(
text: string,
deps: YomitanParserRuntimeDeps,
@@ -1642,6 +984,7 @@ export async function requestYomitanScanTokens(
options?: {
includeNameMatchMetadata?: boolean;
currentCharacterDictionaryMediaId?: number | null;
nameCandidates?: { key: string; forms: string[] } | null;
},
): Promise<YomitanScanToken[] | null> {
const yomitanExt = deps.getYomitanExt();
@@ -1655,10 +998,6 @@ export async function requestYomitanScanTokens(
return null;
}
const parseResults = await requestYomitanParseResults(text, deps, logger);
const selectedParseTokens = selectYomitanParseTokens(parseResults, () => false, 'headword');
const parseScanTokens = selectedParseTokens?.map(toYomitanScanToken) ?? null;
const metadata = await requestYomitanProfileMetadata(parserWindow, logger);
const profileIndex = metadata?.profileIndex ?? 0;
const scanLength = metadata?.scanLength ?? DEFAULT_YOMITAN_SCAN_LENGTH;
@@ -1669,44 +1008,50 @@ export async function requestYomitanScanTokens(
name.startsWith(CHARACTER_DICTIONARY_TITLE_PREFIX),
);
try {
const rawResult = await parserWindow.webContents.executeJavaScript(
buildYomitanScanningScript(
// Candidate name forms let the in-page pre-pass skip positions where no
// character name can start. Installed only when it changes (per media), so
// the per-line call stays a single tiny script.
const nameCandidates = greedyNameScanEnabled ? (options?.nameCandidates ?? null) : null;
await ensureYomitanScanNameCandidates(parserWindow, nameCandidates, logger);
const callScript = buildYomitanScanCallScript({
text,
profileIndex,
scanLength,
includeNameMatchMetadata,
greedyNameScanEnabled,
currentCharacterDictionaryMediaId:
typeof options?.currentCharacterDictionaryMediaId === 'number' &&
Number.isFinite(options.currentCharacterDictionaryMediaId) &&
options.currentCharacterDictionaryMediaId > 0
? Math.floor(options.currentCharacterDictionaryMediaId)
: null,
metadata?.dictionaryPriorityByName ?? {},
metadata?.dictionaryFrequencyModeByName ?? {},
),
true,
);
dictionaryPriorityByName: metadata?.dictionaryPriorityByName ?? {},
dictionaryFrequencyModeByName: metadata?.dictionaryFrequencyModeByName ?? {},
cacheEpoch: getYomitanScanCacheEpoch(parserWindow),
nameCandidateKey: nameCandidates?.key ?? null,
});
try {
let rawResult = await parserWindow.webContents.executeJavaScript(callScript, true);
if (rawResult === YOMITAN_SCAN_RUNTIME_MISSING_SENTINEL) {
// First request for this window, or the page reloaded and dropped the
// installed runtime: install and retry once. The candidate list lives in
// the same page state, so it has to be reinstalled alongside it.
await installYomitanScanRuntime(parserWindow);
await ensureYomitanScanNameCandidates(parserWindow, nameCandidates, logger);
rawResult = await parserWindow.webContents.executeJavaScript(callScript, true);
}
if (isScanTokenArray(rawResult)) {
if (parseScanTokens && parseScanTokens.length > 0) {
return mergeScannerTokensIntoParseTokens(parseScanTokens, rawResult);
// Filler-only results carry no dictionary match; keep the historical
// contract of returning null so callers fall back to raw text.
return rawResult.some((token) => token.isUnparsedRun !== true) ? rawResult : null;
}
return rawResult;
}
if (Array.isArray(rawResult)) {
const selectedTokens = selectYomitanParseTokens(rawResult, () => false, 'headword');
return selectedTokens?.map(toYomitanScanToken) ?? null;
}
if (parseScanTokens && parseScanTokens.length > 0) {
return parseScanTokens;
}
return null;
logger.error('Yomitan scanner returned an unexpected payload; using parseText fallback.');
return await requestYomitanParseFallbackTokens(text, deps, logger);
} catch (err) {
if (parseScanTokens && parseScanTokens.length > 0) {
return parseScanTokens;
}
logger.error('Yomitan scanner request failed:', (err as Error).message);
return null;
return await requestYomitanParseFallbackTokens(text, deps, logger);
}
}
@@ -0,0 +1,908 @@
// In-page Yomitan scan runtime: the helper bundle and scan walk that get
// installed once per parser window as globalThis.__subminerYomitanScan, plus
// the tiny per-line call script. Kept separate from the host runtime module so
// the injected-script text (which is data, not executed here) does not
// dominate that file.
export type YomitanFrequencyMode = 'occurrence-based' | 'rank-based';
export const CHARACTER_DICTIONARY_TITLE_PREFIX = 'SubMiner Character Dictionary';
const YOMITAN_SCANNING_HELPERS = String.raw`
const HIRAGANA_CONVERSION_RANGE = [0x3041, 0x3096];
const KATAKANA_CONVERSION_RANGE = [0x30a1, 0x30f6];
const KANA_PROLONGED_SOUND_MARK_CODE_POINT = 0x30fc;
const KATAKANA_SMALL_KA_CODE_POINT = 0x30f5;
const KATAKANA_SMALL_KE_CODE_POINT = 0x30f6;
const KANA_RANGES = [[0x3040, 0x309f], [0x30a0, 0x30ff]];
const JAPANESE_RANGES = [[0x3040, 0x30ff], [0x3400, 0x9fff]];
function isCodePointInRange(codePoint, range) { return codePoint >= range[0] && codePoint <= range[1]; }
function isCodePointInRanges(codePoint, ranges) { return ranges.some((range) => isCodePointInRange(codePoint, range)); }
function isCodePointKana(codePoint) { return isCodePointInRanges(codePoint, KANA_RANGES); }
function isCodePointJapanese(codePoint) { return isCodePointInRanges(codePoint, JAPANESE_RANGES); }
function createFuriganaSegment(text, reading) { return {text, reading}; }
function getSegmentReadingContribution(segment) {
if (typeof segment.reading === "string" && segment.reading.length > 0) { return segment.reading; }
const segmentText = typeof segment.text === "string" ? segment.text : "";
const isKanaOnly = segmentText.length > 0 && [...segmentText].every((char) => isCodePointKana(char.codePointAt(0)));
return isKanaOnly ? segmentText : "";
}
function getProlongedHiragana(previousCharacter) {
switch (previousCharacter) {
case "あ": case "か": case "が": case "さ": case "ざ": case "た": case "だ": case "な": case "は": case "ば": case "ぱ": case "ま": case "や": case "ら": case "わ": case "ぁ": case "ゃ": case "ゎ": return "あ";
case "い": case "き": case "ぎ": case "し": case "じ": case "ち": case "ぢ": case "に": case "ひ": case "び": case "ぴ": case "み": case "り": case "ぃ": return "い";
case "う": case "く": case "ぐ": case "す": case "ず": case "つ": case "づ": case "ぬ": case "ふ": case "ぶ": case "ぷ": case "む": case "ゆ": case "る": case "ぅ": case "ゅ": return "う";
case "え": case "け": case "げ": case "せ": case "ぜ": case "て": case "で": case "ね": case "へ": case "べ": case "ぺ": case "め": case "れ": case "ぇ": return "え";
case "お": case "こ": case "ご": case "そ": case "ぞ": case "と": case "ど": case "の": case "ほ": case "ぼ": case "ぽ": case "も": case "よ": case "ろ": case "を": case "ぉ": case "ょ": return "う";
default: return null;
}
}
function getFuriganaKanaSegments(text, reading) {
const newSegments = [];
let start = 0;
let state = (reading[0] === text[0]);
for (let i = 1; i < text.length; ++i) {
const newState = (reading[i] === text[i]);
if (state === newState) { continue; }
newSegments.push(createFuriganaSegment(text.substring(start, i), state ? '' : reading.substring(start, i)));
state = newState;
start = i;
}
newSegments.push(createFuriganaSegment(text.substring(start), state ? '' : reading.substring(start)));
return newSegments;
}
function convertKatakanaToHiragana(text, keepProlongedSoundMarks = false) {
let result = '';
const offset = (HIRAGANA_CONVERSION_RANGE[0] - KATAKANA_CONVERSION_RANGE[0]);
for (let char of text) {
const codePoint = char.codePointAt(0);
switch (codePoint) {
case KATAKANA_SMALL_KA_CODE_POINT:
case KATAKANA_SMALL_KE_CODE_POINT:
break;
case KANA_PROLONGED_SOUND_MARK_CODE_POINT:
if (!keepProlongedSoundMarks && result.length > 0) {
const char2 = getProlongedHiragana(result[result.length - 1]);
if (char2 !== null) { char = char2; }
}
break;
default:
if (isCodePointInRange(codePoint, KATAKANA_CONVERSION_RANGE)) {
char = String.fromCodePoint(codePoint + offset);
}
break;
}
result += char;
}
return result;
}
function segmentizeFurigana(reading, readingNormalized, groups, groupsStart) {
const groupCount = groups.length - groupsStart;
if (groupCount <= 0) { return reading.length === 0 ? [] : null; }
const group = groups[groupsStart];
const {isKana, text} = group;
if (isKana) {
if (group.textNormalized !== null && readingNormalized.startsWith(group.textNormalized)) {
const segments = segmentizeFurigana(reading.substring(text.length), readingNormalized.substring(text.length), groups, groupsStart + 1);
if (segments !== null) {
if (reading.startsWith(text)) { segments.unshift(createFuriganaSegment(text, '')); }
else { segments.unshift(...getFuriganaKanaSegments(text, reading)); }
return segments;
}
}
return null;
}
let result = null;
for (let i = reading.length; i >= text.length; --i) {
const segments = segmentizeFurigana(reading.substring(i), readingNormalized.substring(i), groups, groupsStart + 1);
if (segments !== null) {
if (result !== null) { return null; }
segments.unshift(createFuriganaSegment(text, reading.substring(0, i)));
result = segments;
}
if (groupCount === 1) { break; }
}
return result;
}
function distributeFurigana(term, reading) {
if (reading === term) { return [createFuriganaSegment(term, '')]; }
const groups = [];
let groupPre = null;
let isKanaPre = null;
for (const c of term) {
const isKana = isCodePointKana(c.codePointAt(0));
if (isKana === isKanaPre) { groupPre.text += c; }
else {
groupPre = {isKana, text: c, textNormalized: null};
groups.push(groupPre);
isKanaPre = isKana;
}
}
for (const group of groups) {
if (group.isKana) { group.textNormalized = convertKatakanaToHiragana(group.text); }
}
const segments = segmentizeFurigana(reading, convertKatakanaToHiragana(reading), groups, 0);
return segments !== null ? segments : [createFuriganaSegment(term, reading)];
}
function getStemLength(text1, text2) {
const minLength = Math.min(text1.length, text2.length);
if (minLength === 0) { return 0; }
let i = 0;
while (true) {
const char1 = text1.codePointAt(i);
const char2 = text2.codePointAt(i);
if (char1 !== char2) { break; }
const charLength = String.fromCodePoint(char1).length;
i += charLength;
if (i >= minLength) {
if (i > minLength) { i -= charLength; }
break;
}
}
return i;
}
function distributeFuriganaInflected(term, reading, source) {
const termNormalized = convertKatakanaToHiragana(term);
const readingNormalized = convertKatakanaToHiragana(reading);
const sourceNormalized = convertKatakanaToHiragana(source);
let mainText = term;
let stemLength = getStemLength(termNormalized, sourceNormalized);
const readingStemLength = getStemLength(readingNormalized, sourceNormalized);
if (readingStemLength > 0 && readingStemLength >= stemLength) {
mainText = reading;
stemLength = readingStemLength;
reading = source.substring(0, stemLength) + reading.substring(stemLength);
}
const segments = [];
if (stemLength > 0) {
mainText = source.substring(0, stemLength) + mainText.substring(stemLength);
const segments2 = distributeFurigana(mainText, reading);
let consumed = 0;
for (const segment of segments2) {
const start = consumed;
consumed += segment.text.length;
if (consumed < stemLength) { segments.push(segment); }
else if (consumed === stemLength) { segments.push(segment); break; }
else {
if (start < stemLength) { segments.push(createFuriganaSegment(mainText.substring(start, stemLength), '')); }
break;
}
}
}
if (stemLength < source.length) {
const remainder = source.substring(stemLength);
const last = segments[segments.length - 1];
if (last && last.reading.length === 0) { last.text += remainder; }
else { segments.push(createFuriganaSegment(remainder, '')); }
}
return segments;
}
function parsePositiveFrequencyNumber(value) {
if (typeof value === 'number' && Number.isFinite(value) && value > 0) {
return Math.max(1, Math.floor(value));
}
if (typeof value === 'string') {
const numericMatch = value.trim().match(/[+-]?(\d+(\.\d*)?|\.\d+)([eE][+-]?\d+)?/)?.[0];
if (!numericMatch) { return null; }
const parsed = Number.parseFloat(numericMatch);
if (!Number.isFinite(parsed) || parsed <= 0) { return null; }
return Math.max(1, Math.floor(parsed));
}
if (Array.isArray(value)) {
for (const item of value) {
const parsed = parsePositiveFrequencyNumber(item);
if (parsed !== null) { return parsed; }
}
}
return null;
}
function parseDisplayFrequencyNumber(value) {
if (typeof value === 'string') {
const leadingDigits = value.trim().match(/^\d+/)?.[0];
if (!leadingDigits) { return null; }
const parsed = Number.parseInt(leadingDigits, 10);
return Number.isFinite(parsed) && parsed > 0 ? parsed : null;
}
return parsePositiveFrequencyNumber(value);
}
function getFrequencyDictionaryName(frequency) {
const candidates = [
frequency?.dictionary,
frequency?.dictionaryName,
frequency?.name,
frequency?.title,
frequency?.dictionaryTitle,
frequency?.dictionaryAlias
];
for (const candidate of candidates) {
if (typeof candidate === 'string' && candidate.trim().length > 0) {
return candidate.trim();
}
}
return null;
}
function getBestFrequencyRank(dictionaryEntry, headwordIndex, dictionaryPriorityByName, dictionaryFrequencyModeByName) {
let best = null;
const headwordCount = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords.length : 0;
for (const frequency of dictionaryEntry?.frequencies || []) {
if (!frequency || typeof frequency !== 'object') { continue; }
const frequencyHeadwordIndex = frequency.headwordIndex;
if (typeof frequencyHeadwordIndex === 'number') {
if (frequencyHeadwordIndex !== headwordIndex) { continue; }
} else if (headwordCount > 1) {
continue;
}
const dictionary = getFrequencyDictionaryName(frequency);
if (!dictionary) { continue; }
if (dictionaryFrequencyModeByName[dictionary] === 'occurrence-based') { continue; }
const rank =
parseDisplayFrequencyNumber(frequency.displayValue) ??
parsePositiveFrequencyNumber(frequency.frequency);
if (rank === null) { continue; }
const priorityRaw = dictionaryPriorityByName[dictionary];
const fallbackPriority =
typeof frequency.dictionaryIndex === 'number' && Number.isFinite(frequency.dictionaryIndex)
? Math.max(0, Math.floor(frequency.dictionaryIndex))
: Number.MAX_SAFE_INTEGER;
const priority =
typeof priorityRaw === 'number' && Number.isFinite(priorityRaw)
? Math.max(0, Math.floor(priorityRaw))
: fallbackPriority;
if (best === null || priority < best.priority || (priority === best.priority && rank < best.rank)) {
best = { priority, rank };
}
}
return best?.rank ?? null;
}
function hasExactSource(headword, token, requirePrimary) {
for (const src of headword.sources || []) {
if (src.originalText !== token) { continue; }
if (requirePrimary && !src.isPrimary) { continue; }
if (src.matchType !== 'exact') { continue; }
return true;
}
return false;
}
function collectExactHeadwordMatches(dictionaryEntries, token, requirePrimary) {
const matches = [];
for (const dictionaryEntry of dictionaryEntries || []) {
const headwords = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords : [];
for (let headwordIndex = 0; headwordIndex < headwords.length; headwordIndex += 1) {
const headword = headwords[headwordIndex];
if (!hasExactSource(headword, token, requirePrimary)) { continue; }
matches.push({ dictionaryEntry, headword, headwordIndex });
}
}
return matches;
}
function sameHeadword(match, preferredMatch) {
if (!match || !preferredMatch) {
return false;
}
if (match.headword?.term !== preferredMatch.headword?.term) {
return false;
}
const matchReading = typeof match.headword?.reading === 'string' ? match.headword.reading : '';
const preferredReading =
typeof preferredMatch.headword?.reading === 'string' ? preferredMatch.headword.reading : '';
if (!matchReading || !preferredReading) {
return true;
}
return matchReading === preferredReading;
}
function getBestFrequencyRankForMatches(matches, dictionaryPriorityByName, dictionaryFrequencyModeByName) {
let best = null;
for (const match of matches) {
const rank = getBestFrequencyRank(
match.dictionaryEntry,
match.headwordIndex,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
);
if (rank === null) { continue; }
if (best === null || rank < best) {
best = rank;
}
}
return best;
}
function normalizeWordClasses(headword) {
if (!Array.isArray(headword?.wordClasses)) { return undefined; }
const classes = headword.wordClasses.filter((wordClass) => typeof wordClass === "string" && wordClass.trim().length > 0);
return classes.length > 0 ? classes : undefined;
}
function appendDictionaryNames(target, value) {
if (!value || typeof value !== 'object') {
return;
}
const candidates = [
value.dictionary,
value.dictionaryName,
value.name,
value.title,
value.dictionaryTitle,
value.dictionaryAlias
];
for (const candidate of candidates) {
if (typeof candidate === 'string' && candidate.trim().length > 0) {
target.push(candidate.trim());
}
}
}
function getDictionaryEntryNames(entry) {
const names = [];
appendDictionaryNames(names, entry);
for (const definition of entry?.definitions || []) {
appendDictionaryNames(names, definition);
}
for (const frequency of entry?.frequencies || []) {
appendDictionaryNames(names, frequency);
}
for (const pronunciation of entry?.pronunciations || []) {
appendDictionaryNames(names, pronunciation);
}
return names;
}
function isNameDictionaryEntry(entry) {
if (!includeNameMatchMetadata || !entry || typeof entry !== 'object') {
return false;
}
return getDictionaryEntryNames(entry).some((name) => name.startsWith(${JSON.stringify(CHARACTER_DICTIONARY_TITLE_PREFIX)}));
}
function parseSubMinerMediaIdFromString(value) {
const imageMatch = value.match(/\bimg\/m(\d+)-/i);
if (imageMatch) {
const parsed = Number.parseInt(imageMatch[1], 10);
if (Number.isSafeInteger(parsed) && parsed > 0) { return parsed; }
}
const titleMatch = value.match(/${CHARACTER_DICTIONARY_TITLE_PREFIX}[^\d]*(?:AniList\s*)?(\d+)/i);
if (titleMatch) {
const parsed = Number.parseInt(titleMatch[1], 10);
if (Number.isSafeInteger(parsed) && parsed > 0) { return parsed; }
}
return null;
}
function parseSubMinerMediaIdCandidate(value) {
if (typeof value === 'number' && Number.isSafeInteger(value) && value > 0) {
return value;
}
if (typeof value === 'string' && /^\d+$/.test(value.trim())) {
const parsed = Number.parseInt(value.trim(), 10);
if (Number.isSafeInteger(parsed) && parsed > 0) { return parsed; }
}
return null;
}
function collectSubMinerMediaIds(value, target) {
if (typeof value === 'string') {
const parsed = parseSubMinerMediaIdFromString(value);
if (parsed !== null) { target.add(parsed); }
return;
}
if (!value || typeof value !== 'object') {
return;
}
if (Array.isArray(value)) {
for (const item of value) { collectSubMinerMediaIds(item, target); }
return;
}
const mediaIdCandidates = [
value.subminerMediaId,
value.subMinerMediaId,
value.characterDictionaryMediaId,
value.data?.subminerMediaId,
value.data?.subMinerMediaId,
value.data?.characterDictionaryMediaId
];
for (const candidate of mediaIdCandidates) {
const parsed = parseSubMinerMediaIdCandidate(candidate);
if (parsed !== null) { target.add(parsed); }
}
for (const child of Object.values(value)) {
collectSubMinerMediaIds(child, target);
}
}
function getSubMinerMediaIds(entry) {
const mediaIds = new Set();
collectSubMinerMediaIds(entry, mediaIds);
return mediaIds;
}
function isCurrentMediaNameDictionaryEntry(entry) {
if (!isNameDictionaryEntry(entry)) {
return false;
}
if (currentCharacterDictionaryMediaId === null) {
return true;
}
const mediaIds = getSubMinerMediaIds(entry);
return mediaIds.size === 0 || mediaIds.has(currentCharacterDictionaryMediaId);
}
function findLongestNameMatch(dictionaryEntries, textWindow) {
let best = null;
for (const dictionaryEntry of dictionaryEntries || []) {
if (!isCurrentMediaNameDictionaryEntry(dictionaryEntry)) { continue; }
const headwords = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords : [];
for (let headwordIndex = 0; headwordIndex < headwords.length; headwordIndex += 1) {
const headword = headwords[headwordIndex];
for (const src of headword?.sources || []) {
if (src.matchType !== 'exact' || src.isPrimary !== true) { continue; }
const originalText = typeof src.originalText === 'string' ? src.originalText : '';
if (!originalText || !textWindow.startsWith(originalText)) { continue; }
if (best === null || originalText.length > best.sourceLength) {
best = { dictionaryEntry, headword, headwordIndex, sourceLength: originalText.length };
}
}
}
}
return best;
}
function findLongestGenericMatchLength(dictionaryEntries, textWindow) {
let best = 0;
for (const dictionaryEntry of dictionaryEntries || []) {
if (isNameDictionaryEntry(dictionaryEntry)) { continue; }
const headwords = Array.isArray(dictionaryEntry?.headwords) ? dictionaryEntry.headwords : [];
for (const headword of headwords) {
for (const src of headword?.sources || []) {
if (src.matchType !== 'exact' || src.isPrimary !== true) { continue; }
const originalText = typeof src.originalText === 'string' ? src.originalText : '';
if (!originalText || !textWindow.startsWith(originalText)) { continue; }
if (originalText.length > best) { best = originalText.length; }
}
}
}
return best;
}
function getPreferredHeadword(dictionaryEntries, token, dictionaryPriorityByName, dictionaryFrequencyModeByName) {
const currentMediaDictionaryEntries =
currentCharacterDictionaryMediaId === null
? (dictionaryEntries || [])
: (dictionaryEntries || []).filter((entry) => {
if (!isNameDictionaryEntry(entry)) { return true; }
return isCurrentMediaNameDictionaryEntry(entry);
});
const exactPrimaryMatches = collectExactHeadwordMatches(currentMediaDictionaryEntries, token, true);
let matchedNameDictionary = false;
if (includeNameMatchMetadata) {
for (const dictionaryEntry of currentMediaDictionaryEntries || []) {
if (!isCurrentMediaNameDictionaryEntry(dictionaryEntry)) { continue; }
for (const match of exactPrimaryMatches) {
if (match.dictionaryEntry !== dictionaryEntry) { continue; }
matchedNameDictionary = true;
break;
}
if (matchedNameDictionary) { break; }
}
}
const preferredMatch = exactPrimaryMatches[0];
if (preferredMatch) {
const exactFrequencyMatches = collectExactHeadwordMatches(currentMediaDictionaryEntries, token, false)
.filter((match) => sameHeadword(match, preferredMatch));
return {
term: preferredMatch.headword.term,
reading: preferredMatch.headword.reading,
wordClasses: normalizeWordClasses(preferredMatch.headword),
isNameMatch:
matchedNameDictionary || isCurrentMediaNameDictionaryEntry(preferredMatch.dictionaryEntry),
frequencyRank: getBestFrequencyRankForMatches(
exactFrequencyMatches.length > 0 ? exactFrequencyMatches : exactPrimaryMatches,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
)
};
}
return null;
}
`;
// Bump whenever the install script below changes so already-loaded parser
// windows re-install the new scan runtime instead of running the stale one.
export const YOMITAN_SCAN_RUNTIME_VERSION = 4;
export const YOMITAN_SCAN_RUNTIME_MISSING_SENTINEL = '__subminer-yomitan-scan-runtime-missing__';
export interface YomitanScanRequestParams {
text: string;
profileIndex: number;
scanLength: number;
includeNameMatchMetadata: boolean;
greedyNameScanEnabled: boolean;
currentCharacterDictionaryMediaId: number | null;
dictionaryPriorityByName: Record<string, number>;
dictionaryFrequencyModeByName: Partial<Record<string, YomitanFrequencyMode>>;
cacheEpoch: number;
/**
* Key of the character-name candidate list installed for the current media,
* or null to scan every Japanese position (see the pre-pass prefilter).
*/
nameCandidateKey: string | null;
}
// Installed once per parser window (and re-installed after in-page reloads):
// keeps V8 from re-parsing the helper bundle on every subtitle line, and hosts
// the cross-line termsFind cache. Each subtitle line then only evaluates a tiny
// call into globalThis.__subminerYomitanScan.
export const YOMITAN_SCAN_RUNTIME_INSTALL_SCRIPT = String.raw`
(() => {
if (globalThis.__subminerYomitanScanVersion === ${YOMITAN_SCAN_RUNTIME_VERSION}) {
return true;
}
const invoke = (action, params) =>
new Promise((resolve, reject) => {
chrome.runtime.sendMessage({ action, params }, (response) => {
if (chrome.runtime.lastError) {
reject(new Error(chrome.runtime.lastError.message));
return;
}
if (!response || typeof response !== "object") {
reject(new Error("Invalid response from Yomitan backend"));
return;
}
if (response.error) {
reject(new Error(response.error.message || "Yomitan backend error"));
return;
}
resolve(response.result);
});
});
// Cross-line termsFind LRU keyed by profile + substring: subtitle lines
// repeat particles and inflections constantly, so most lookups hit here.
// Entries hold in-flight promises so concurrent identical lookups dedupe.
const termsFindCache = new Map();
const TERMS_FIND_CACHE_LIMIT = 2000;
let termsFindCacheEpoch = -1;
const MAX_SHRINKING_WINDOW_RETRY_LOOKUPS = 4;
// Character-name candidate forms for the current media, installed
// separately from the per-line scan call so the per-line script stays tiny.
// Stored raw here; the normalized lookup index is built inside the scan,
// where the kana-normalization helper is in scope, and reused by key.
let rawNameCandidates = null;
let nameCandidateIndex = null;
globalThis.__subminerYomitanScanSetNameCandidates = (key, forms) => {
if (!key || !Array.isArray(forms) || forms.length === 0) {
rawNameCandidates = null;
nameCandidateIndex = null;
return false;
}
rawNameCandidates = { key, forms };
nameCandidateIndex = null;
return true;
};
globalThis.__subminerYomitanScanVersion = ${YOMITAN_SCAN_RUNTIME_VERSION};
globalThis.__subminerYomitanScan = async (scanParams) => {
const {
text,
profileIndex,
scanLength,
includeNameMatchMetadata,
greedyNameScanEnabled,
currentCharacterDictionaryMediaId,
dictionaryPriorityByName,
dictionaryFrequencyModeByName,
cacheEpoch,
nameCandidateKey
} = scanParams;
if (cacheEpoch !== termsFindCacheEpoch) {
termsFindCache.clear();
termsFindCacheEpoch = cacheEpoch;
}
${YOMITAN_SCANNING_HELPERS}
const CAPTION_OPENING_BRACKETS = new Set(["(", "", "[", "", "{", "", "「", "『", "【", "〈", "《", "≪", "", "<"]);
function shouldEmitUnparsedRunAsToken(runText) {
if (!/[\p{L}\p{N}]/u.test(runText)) { return false; }
const firstChar = Array.from(runText.trim())[0];
return firstChar !== undefined && !CAPTION_OPENING_BRACKETS.has(firstChar);
}
function isLookupWorthyCodePoint(codePoint) {
if (isCodePointJapanese(codePoint)) { return true; }
return /[\p{L}\p{N}]/u.test(String.fromCodePoint(codePoint));
}
function isKanaOnlyRunText(runText) {
const chars = Array.from(runText);
return chars.length > 0 && chars.every((char) => isCodePointKana(char.codePointAt(0)));
}
const details = {matchType: "exact", deinflect: true};
const tokens = [];
async function termsFindAt(position, windowLength) {
const substring = text.substring(position, position + windowLength);
const cacheKey = profileIndex + "" + substring;
const cached = termsFindCache.get(cacheKey);
if (cached !== undefined) {
termsFindCache.delete(cacheKey);
termsFindCache.set(cacheKey, cached);
return await cached;
}
const pending = invoke("termsFind", { text: substring, details, optionsContext: { index: profileIndex } });
termsFindCache.set(cacheKey, pending);
while (termsFindCache.size > TERMS_FIND_CACHE_LIMIT) {
const oldestKey = termsFindCache.keys().next().value;
if (oldestKey === undefined) { break; }
termsFindCache.delete(oldestKey);
}
try {
return await pending;
} catch (error) {
termsFindCache.delete(cacheKey);
throw error;
}
}
// Text the walk skips accumulates into unparsed runs, mirroring the
// filler chunks the parseText segmentation used to provide: runs stay
// hoverable (flagged isUnparsedRun) unless they are punctuation-only or
// caption-style asides, and kana continuations of a longer headword
// extend the previous token instead.
function flushUnparsedRun(runStart, runEnd) {
if (runStart === null || runEnd <= runStart) { return; }
const runText = text.substring(runStart, runEnd);
const previousToken = tokens[tokens.length - 1];
if (
previousToken &&
previousToken.endPos === runStart &&
isKanaOnlyRunText(runText) &&
typeof previousToken.headword === "string" &&
previousToken.headword.length > previousToken.surface.length &&
previousToken.headword.startsWith(previousToken.surface + runText)
) {
previousToken.surface += runText;
// The run is kana-only, so its reading is itself: append it or the
// reading stops covering the surface, which disables the known-word
// reading fallback (isCompleteReadingForSurface) downstream.
previousToken.reading += runText;
// The run is kana-only, so its reading is itself: append it or the
// reading stops covering the surface, which disables the known-word
// reading fallback (isCompleteReadingForSurface) downstream.
previousToken.endPos = runEnd;
return;
}
if (!shouldEmitUnparsedRunAsToken(runText)) { return; }
tokens.push({
surface: runText,
reading: "",
headword: runText,
startPos: runStart,
endPos: runEnd,
isUnparsedRun: true
});
}
function buildScanToken(position, source, preferredHeadword) {
const reading = typeof preferredHeadword.reading === "string" ? preferredHeadword.reading : "";
const segments = distributeFuriganaInflected(preferredHeadword.term, reading, source);
const tokenPayload = {
surface: segments.map((segment) => segment.text).join("") || source,
reading: segments.map(getSegmentReadingContribution).join(""),
headword: preferredHeadword.term,
headwordReading: reading || undefined,
startPos: position,
endPos: position + source.length,
isNameMatch: includeNameMatchMetadata && preferredHeadword.isNameMatch === true,
frequencyRank:
typeof preferredHeadword.frequencyRank === "number" && Number.isFinite(preferredHeadword.frequencyRank)
? Math.max(1, Math.floor(preferredHeadword.frequencyRank))
: undefined,
};
if (Array.isArray(preferredHeadword.wordClasses) && preferredHeadword.wordClasses.length > 0) {
tokenPayload.wordClasses = preferredHeadword.wordClasses;
}
return tokenPayload;
}
async function findTokenAt(position, windowLength) {
const codePoint = text.codePointAt(position);
const character = String.fromCodePoint(codePoint);
const result = await termsFindAt(position, windowLength);
const dictionaryEntries = Array.isArray(result?.dictionaryEntries) ? result.dictionaryEntries : [];
const originalTextLength = typeof result?.originalTextLength === "number" ? result.originalTextLength : 0;
if (dictionaryEntries.length === 0 || originalTextLength <= 0 || (originalTextLength === character.length && !isCodePointJapanese(codePoint))) {
return { token: null, matchedLength: 0 };
}
const source = text.substring(position, position + originalTextLength);
const preferredHeadword = getPreferredHeadword(
dictionaryEntries,
source,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
);
if (!preferredHeadword || typeof preferredHeadword.term !== "string") {
return { token: null, matchedLength: originalTextLength };
}
return { token: buildScanToken(position, source, preferredHeadword), matchedLength: originalTextLength };
}
// Halfwidth katakana survives kana normalization unchanged, so a name
// written that way would not prefix-match a candidate form. Those
// positions bypass the prefilter rather than risk a missed name.
function isHalfwidthKatakanaCodePoint(codePoint) {
return codePoint >= 0xff66 && codePoint <= 0xff9f;
}
// Build (once per candidate list) a first-character bucket index of the
// normalized name forms, so the pre-pass can reject a position with a
// single map hit instead of a backend round trip.
if (rawNameCandidates && nameCandidateIndex?.key !== rawNameCandidates.key) {
const byFirstChar = new Map();
for (const form of rawNameCandidates.forms) {
const normalized = typeof form === "string" ? convertKatakanaToHiragana(form.trim()) : "";
if (!normalized) { continue; }
const bucket = byFirstChar.get(normalized[0]);
if (bucket) { bucket.push(normalized); } else { byFirstChar.set(normalized[0], [normalized]); }
}
nameCandidateIndex = byFirstChar.size > 0 ? { key: rawNameCandidates.key, byFirstChar } : null;
} else if (!rawNameCandidates) {
nameCandidateIndex = null;
}
// Only meaningful when the installed list matches the media this scan is
// for; otherwise fall back to scanning every position.
const activeNameCandidateIndex =
nameCandidateKey !== null && nameCandidateIndex?.key === nameCandidateKey
? nameCandidateIndex
: null;
const normalizedText = activeNameCandidateIndex ? convertKatakanaToHiragana(text) : "";
// Yomitan collapses emphatic sequences before matching (すっっごーーい →
// すごい), so a stretched name still resolves to its entry. Skipping these
// characters keeps such spellings candidates; the filter only ever grows
// the probe set, so a false positive costs one lookup, never a name.
const EMPHATIC_SKIP_CHARS = new Set(["ぁ", "ぃ", "ぅ", "ぇ", "ぉ", "っ", "ゃ", "ゅ", "ょ", "ー"]);
function matchesCandidateFormAt(form, position) {
let textIndex = position;
for (let formIndex = 0; formIndex < form.length; formIndex += 1) {
while (
textIndex < normalizedText.length &&
normalizedText[textIndex] !== form[formIndex] &&
EMPHATIC_SKIP_CHARS.has(normalizedText[textIndex])
) {
textIndex += 1;
}
if (normalizedText[textIndex] !== form[formIndex]) { return false; }
textIndex += 1;
}
return true;
}
function couldNameStartAt(position, codePoint) {
if (!activeNameCandidateIndex) { return true; }
if (isHalfwidthKatakanaCodePoint(codePoint)) { return true; }
const bucket = activeNameCandidateIndex.byFirstChar.get(normalizedText[position]);
if (!bucket) { return false; }
for (const form of bucket) {
if (matchesCandidateFormAt(form, position)) { return true; }
}
return false;
}
// Greedy name pre-pass: character-name matches claim their spans before
// the left-to-right walk, so a longer generic match starting earlier
// (e.g. とヨー → 渡洋) cannot swallow the start of a name (ヨータ).
const nameTokens = [];
if (greedyNameScanEnabled) {
let namePos = 0;
while (namePos < text.length) {
const codePoint = text.codePointAt(namePos);
if (!isCodePointJapanese(codePoint) || !couldNameStartAt(namePos, codePoint)) {
namePos += String.fromCodePoint(codePoint).length;
continue;
}
const result = await termsFindAt(namePos, scanLength);
const dictionaryEntries = Array.isArray(result?.dictionaryEntries) ? result.dictionaryEntries : [];
const textWindow = text.substring(namePos, namePos + scanLength);
const nameMatch = findLongestNameMatch(dictionaryEntries, textWindow);
// A name only claims its span when no strictly longer generic word
// starts at the same position (a character named 空 must not split
// 空気). Ties go to the name. Generic matches that start earlier and
// overlap the name are still blocked by the reservation.
if (
!nameMatch ||
findLongestGenericMatchLength(dictionaryEntries, textWindow) > nameMatch.sourceLength
) {
namePos += String.fromCodePoint(codePoint).length;
continue;
}
const source = text.substring(namePos, namePos + nameMatch.sourceLength);
nameTokens.push(buildScanToken(namePos, source, {
term: nameMatch.headword.term,
reading: nameMatch.headword.reading,
wordClasses: normalizeWordClasses(nameMatch.headword),
isNameMatch: true,
frequencyRank: getBestFrequencyRank(
nameMatch.dictionaryEntry,
nameMatch.headwordIndex,
dictionaryPriorityByName,
dictionaryFrequencyModeByName
)
}));
namePos += nameMatch.sourceLength;
}
}
let i = 0;
let nameIndex = 0;
let unparsedRunStart = null;
while (i < text.length) {
while (nameIndex < nameTokens.length && nameTokens[nameIndex].startPos < i) { nameIndex += 1; }
const nextNameToken = nameIndex < nameTokens.length ? nameTokens[nameIndex] : null;
if (nextNameToken && nextNameToken.startPos === i) {
flushUnparsedRun(unparsedRunStart, i);
unparsedRunStart = null;
tokens.push(nextNameToken);
i = nextNameToken.endPos;
nameIndex += 1;
continue;
}
const codePoint = text.codePointAt(i);
// Punctuation and whitespace can never start a token: skip the backend
// round trip entirely. Latin letters and digits stay lookup-worthy
// (terms like Tシャツ start on an ASCII letter).
if (!isLookupWorthyCodePoint(codePoint)) {
if (unparsedRunStart === null) { unparsedRunStart = i; }
i += String.fromCodePoint(codePoint).length;
continue;
}
// Cap the window at the next reserved name span so a generic match
// cannot consume into it.
const windowLength = nextNameToken ? Math.min(scanLength, nextNameToken.startPos - i) : scanLength;
let attempt = await findTokenAt(i, windowLength);
// Yomitan text normalization can consume characters (whitespace,
// punctuation) beyond the matched term, leaving no headword whose
// source equals the consumed text. Retry with shorter windows so a
// valid prefix term (e.g. a character name before a paren) still
// tokenizes instead of the position being skipped. The ladder is
// capped: without a cap it degrades to O(scanLength) lookups at a
// single position.
let retryLength = Math.min(attempt.matchedLength, windowLength) - 1;
let retryLookupsRemaining = MAX_SHRINKING_WINDOW_RETRY_LOOKUPS;
while (!attempt.token && retryLength >= 1 && retryLookupsRemaining > 0) {
retryLookupsRemaining -= 1;
const retry = await findTokenAt(i, retryLength);
if (retry.token) {
attempt = retry;
break;
}
retryLength = Math.min(retryLength - 1, retry.matchedLength - 1);
}
if (attempt.token) {
flushUnparsedRun(unparsedRunStart, i);
unparsedRunStart = null;
tokens.push(attempt.token);
i += attempt.matchedLength;
continue;
}
if (unparsedRunStart === null) { unparsedRunStart = i; }
i += String.fromCodePoint(text.codePointAt(i)).length;
}
flushUnparsedRun(unparsedRunStart, text.length);
return tokens;
};
return true;
})();
`;
// Installs (or clears) the character-name candidate forms for the current
// media. Runs only when the list changes, not per line. Passing null restores
// the exhaustive every-position pre-pass.
export function buildYomitanScanNameCandidatesScript(
nameCandidates: { key: string; forms: string[] } | null,
): string {
if (!nameCandidates) {
return `
(() => {
if (typeof globalThis.__subminerYomitanScanSetNameCandidates !== "function") {
return false;
}
return globalThis.__subminerYomitanScanSetNameCandidates(null, null);
})();
`;
}
return `
(() => {
if (typeof globalThis.__subminerYomitanScanSetNameCandidates !== "function") {
return false;
}
return globalThis.__subminerYomitanScanSetNameCandidates(
${JSON.stringify(nameCandidates.key)},
${JSON.stringify(nameCandidates.forms)}
);
})();
`;
}
export function buildYomitanScanCallScript(params: YomitanScanRequestParams): string {
return `
(async () => {
if (typeof globalThis.__subminerYomitanScan !== "function") {
return ${JSON.stringify(YOMITAN_SCAN_RUNTIME_MISSING_SENTINEL)};
}
return await globalThis.__subminerYomitanScan(${JSON.stringify(params)});
})();
`;
}
+44 -9
View File
@@ -295,6 +295,7 @@ import {
importYomitanDictionaryFromZip,
initializeOverlayAnkiIntegration as initializeOverlayAnkiIntegrationCore,
initializeOverlayRuntime as initializeOverlayRuntimeCore,
isSubtitleAnnotationUpgrade,
isOverlayWindowContentReady,
jellyfinTicksToSecondsRuntime,
listJellyfinItemsRuntime,
@@ -486,6 +487,7 @@ import { createOverlayVisibilityRuntimeService } from './main/overlay-visibility
import { createDiscordPresenceRuntime } from './main/runtime/discord-presence-runtime';
import { createCharacterDictionaryRuntimeService } from './main/character-dictionary-runtime';
import { createCharacterDictionaryImageLookup } from './main/character-dictionary-runtime/image-lookup';
import { createCharacterNameCandidateLookup } from './main/character-dictionary-runtime/name-candidates';
import {
createCharacterDictionaryAutoSyncRuntimeService,
getCharacterDictionaryManagerSnapshot,
@@ -1815,8 +1817,10 @@ function withCurrentSubtitleTiming(payload: SubtitleData): SubtitleData {
endTime: appState.mpvClient?.currentSubEnd ?? null,
};
}
function emitSubtitlePayload(payload: SubtitleData): void {
function emitSubtitlePayload(payload: SubtitleData, options?: { resumePrefetch?: boolean }): void {
const timedPayload = withCurrentSubtitleTiming(payload);
const currentSubtitleData = appState.currentSubtitleData;
const isAnnotationUpgrade = isSubtitleAnnotationUpgrade(currentSubtitleData, timedPayload);
const frequencyDictionary = configService.getConfig().subtitleStyle.frequencyDictionary;
const frequencyOptions = {
enabled: frequencyDictionary.enabled,
@@ -1825,10 +1829,17 @@ function emitSubtitlePayload(payload: SubtitleData): void {
};
appState.currentSubtitleData = timedPayload;
overlayManager.broadcastToOverlayWindows('subtitle:set', timedPayload);
if (!isAnnotationUpgrade) {
subtitleWsService.broadcast(timedPayload, frequencyOptions);
}
annotationSubtitleWsService.broadcast(timedPayload, frequencyOptions);
autoplayReadyGate.maybeSignalPluginAutoplayReady(timedPayload, { forceWhilePaused: true });
// resumePrefetch: false marks a provisional pre-tokenization emit; prefetch
// stays paused until the tokenized payload for the line lands so it does not
// compete with the on-screen line for the single Yomitan parser window.
if (options?.resumePrefetch !== false) {
subtitlePrefetchService?.resume();
}
}
function getCurrentAutoplaySubtitlePayload(): SubtitleData | null {
const payload = appState.currentSubtitleData;
@@ -1921,7 +1932,7 @@ const autoplaySubtitlePrimingRuntime = createAutoplaySubtitlePrimingRuntime({
appState.activeParsedSubtitleMediaPath = mediaPath;
},
subtitleProcessingController,
emitSubtitlePayload: (payload) => emitSubtitlePayload(payload),
emitSubtitlePayload: (payload, options) => emitSubtitlePayload(payload, options),
getSubtitlePrefetchService: () => subtitlePrefetchService,
getLastObservedTimePos: () => lastObservedTimePos,
getVisibleOverlayVisible: () => overlayManager.getVisibleOverlayVisible(),
@@ -1955,7 +1966,6 @@ const subtitlePrefetchInitController = createSubtitlePrefetchInitController({
subtitleProcessingController.preCacheTokenization(text, data);
},
hasCachedTokenization: (text) => subtitleProcessingController.hasCachedSubtitle(text),
isCacheFull: () => subtitleProcessingController.isCacheFull(),
logInfo: (message) => logger.info(message),
logWarn: (message) => logger.warn(message),
onParsedSubtitleCuesChanged: (cues, sourceKey) => {
@@ -1982,15 +1992,22 @@ const resolveActiveSubtitleSidebarSourceHandler = createResolveActiveSubtitleSid
getFfmpegPath: () => configService.getConfig().subsync.ffmpeg_path.trim() || 'ffmpeg',
extractInternalSubtitleTrack: (ffmpegPath, videoPath, track) =>
extractInternalSubtitleTrackToTempFile(ffmpegPath, videoPath, track),
logDebug: (message) => logger.debug(message),
});
const refreshSubtitlePrefetchFromActiveTrackHandler =
createRefreshSubtitlePrefetchFromActiveTrackHandler({
getMpvClient: () => appState.mpvClient,
getLastObservedTimePos: () => lastObservedTimePos,
shouldKeepExistingCuesOnMissingSource: (videoPath) => isYoutubeMediaPath(videoPath),
// Remote media has no extractable on-disk track to fall back to, so a transient
// resolve miss (sid briefly 'no', a cycle onto an embedded stream track) would
// otherwise drop a working cue list for the rest of the episode.
shouldKeepExistingCuesOnMissingSource: (videoPath) =>
isYoutubeMediaPath(videoPath) || isRemoteMediaPath(videoPath),
subtitlePrefetchInitController,
resolveActiveSubtitleSidebarSource: (input) => resolveActiveSubtitleSidebarSourceHandler(input),
logDebug: (message) => logger.debug(message),
logWarn: (message) => logger.warn(message),
});
const subtitlePrefetchRuntime = {
@@ -2521,6 +2538,10 @@ const characterDictionaryAutoSyncRuntime = createCharacterDictionaryAutoSyncRunt
},
{
hasParserWindow: () => Boolean(appState.yomitanParserWindow),
invalidateCharacterDictionaryLookups: () => {
characterDictionaryImageLookup.invalidate();
characterNameCandidateLookup.invalidate();
},
clearParserCaches: () => {
if (appState.yomitanParserWindow) {
clearYomitanParserCachesForWindow(appState.yomitanParserWindow);
@@ -2546,6 +2567,13 @@ const characterDictionaryImageLookup = createCharacterDictionaryImageLookup({
getCurrentMediaId: () => characterDictionaryAutoSyncRuntime.getCurrentMediaId(),
});
// Lets the Yomitan scan runtime skip name lookups at positions where no
// character name can start; absent candidates just mean the exhaustive scan.
const characterNameCandidateLookup = createCharacterNameCandidateLookup({
userDataPath: USER_DATA_PATH,
getCurrentMediaId: () => characterDictionaryAutoSyncRuntime.getCurrentMediaId(),
});
const overlayVisibilityRuntime = createOverlayVisibilityRuntimeService(
createBuildOverlayVisibilityRuntimeMainDepsHandler({
getMainWindow: () => overlayManager.getMainWindow(),
@@ -2994,6 +3022,8 @@ const {
streamIndex,
delaySeconds,
}),
initSubtitlePrefetch: (sourcePath) =>
subtitlePrefetchRuntime.refreshSubtitleSidebarFromSource(sourcePath),
logDebug: (message, error) => {
logger.debug(message, error);
},
@@ -4359,13 +4389,20 @@ const {
emitSubtitlePayload(payload);
},
onSubtitleChange: (text) => {
// Pause only; restarting the prefetch run here would discard in-flight
// tokenization work on every line. Real seeks restart via onTimePosUpdate.
subtitlePrefetchService?.pause();
subtitlePrefetchService?.onSeek(lastObservedTimePos);
subtitleProcessingController.onSubtitleChange(text);
if (!subtitleProcessingController.onSubtitleChange(text)) {
// Repeat of the current text: nothing will be tokenized, so no emit is
// coming to release the pause. Resume now instead of idling prefetch
// for the rest of the cue.
subtitlePrefetchService?.resume();
}
},
refreshDiscordPresence: () => {
discordPresenceRuntime.publishDiscordPresence();
},
logSubtitleProcessingDebug: (message: string) => logger.debug(message),
ensureImmersionTrackerInitialized: () => {
ensureImmersionTrackerStarted();
},
@@ -4604,6 +4641,7 @@ const {
getCharacterNameImage: (term) => characterDictionaryImageLookup.get(term),
getCurrentCharacterDictionaryMediaId: () =>
characterDictionaryAutoSyncRuntime.getCurrentMediaId(),
getCharacterNameCandidates: () => characterNameCandidateLookup.get(),
getFrequencyDictionaryEnabled: () =>
getRuntimeBooleanOption(
'subtitle.annotation.frequency',
@@ -5658,7 +5696,6 @@ const { registerIpcRuntimeHandlers } = composeIpcRuntimeHandlers({
if (result.ok && result.rebuildRequired) {
try {
await characterDictionaryAutoSyncRuntime.runSyncNow();
characterDictionaryImageLookup.invalidate();
} catch (error) {
logger.warn('Failed to rebuild character dictionary after manager override:', error);
}
@@ -5689,7 +5726,6 @@ const { registerIpcRuntimeHandlers } = composeIpcRuntimeHandlers({
if (result.ok && result.rebuildRequired) {
try {
await characterDictionaryAutoSyncRuntime.runSyncNow();
characterDictionaryImageLookup.invalidate();
} catch (error) {
logger.warn('Failed to rebuild character dictionary after manager removal:', error);
}
@@ -5706,7 +5742,6 @@ const { registerIpcRuntimeHandlers } = composeIpcRuntimeHandlers({
if (result.ok && result.rebuildRequired) {
try {
await characterDictionaryAutoSyncRuntime.runSyncNow();
characterDictionaryImageLookup.invalidate();
} catch (error) {
logger.warn('Failed to rebuild character dictionary after manager reorder:', error);
}
@@ -1,7 +1,7 @@
export const ANILIST_GRAPHQL_URL = 'https://graphql.anilist.co';
export const ANILIST_REQUEST_DELAY_MS = 2000;
export const CHARACTER_IMAGE_DOWNLOAD_DELAY_MS = 250;
export const CHARACTER_DICTIONARY_FORMAT_VERSION = 19;
export const CHARACTER_DICTIONARY_FORMAT_VERSION = 20;
export const CHARACTER_DICTIONARY_MERGED_TITLE = 'SubMiner Character Dictionary';
export const HONORIFIC_SUFFIXES = [
@@ -0,0 +1,163 @@
import assert from 'node:assert/strict';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import test from 'node:test';
import { CHARACTER_DICTIONARY_FORMAT_VERSION } from './constants';
import { createCharacterNameCandidateLookup } from './name-candidates';
function writeSnapshot(outputDir: string, mediaId: number, entries: Array<[string, string]>): void {
const snapshotsDir = path.join(outputDir, 'snapshots');
fs.mkdirSync(snapshotsDir, { recursive: true });
fs.writeFileSync(
path.join(snapshotsDir, `anilist-${mediaId}.json`),
JSON.stringify({
formatVersion: CHARACTER_DICTIONARY_FORMAT_VERSION,
mediaId,
mediaTitle: `title-${mediaId}`,
entryCount: entries.length,
updatedAt: 1,
termEntries: entries.map(([term, reading]) => [
term,
reading,
'name main',
'',
100,
[],
0,
'',
]),
images: [],
}),
);
}
function withTempDir<T>(run: (dir: string) => T): T {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-name-candidates-'));
try {
return run(dir);
} finally {
fs.rmSync(dir, { recursive: true, force: true });
}
}
test('collects terms and readings for the current media', () => {
withTempDir((dir) => {
writeSnapshot(dir, 1, [
['ミナト', 'みなと'],
['湊', 'みなと'],
]);
writeSnapshot(dir, 2, [['カズマ', 'かずま']]);
const lookup = createCharacterNameCandidateLookup({
outputDir: dir,
getCurrentMediaId: () => 1,
});
const candidates = lookup.get();
assert.ok(candidates);
assert.deepEqual([...candidates.forms].sort(), ['みなと', 'ミナト', '湊'].sort());
// Deduplicated: both entries share the みなと reading.
assert.equal(candidates.forms.length, 3);
});
});
test('returns null without a media scope so the scanner stays exhaustive', () => {
withTempDir((dir) => {
writeSnapshot(dir, 1, [['ミナト', 'みなと']]);
const lookup = createCharacterNameCandidateLookup({
outputDir: dir,
getCurrentMediaId: () => null,
});
assert.equal(lookup.get(), null);
});
});
test('returns null for a media with no cached snapshot', () => {
withTempDir((dir) => {
writeSnapshot(dir, 1, [['ミナト', 'みなと']]);
const lookup = createCharacterNameCandidateLookup({
outputDir: dir,
getCurrentMediaId: () => 999,
});
assert.equal(lookup.get(), null);
});
});
test('key changes when the snapshot content changes', () => {
withTempDir((dir) => {
writeSnapshot(dir, 1, [['ミナト', 'みなと']]);
const lookup = createCharacterNameCandidateLookup({
outputDir: dir,
getCurrentMediaId: () => 1,
});
const first = lookup.get();
writeSnapshot(dir, 1, [
['ミナト', 'みなと'],
['アクア', 'あくあ'],
]);
lookup.invalidate();
const second = lookup.get();
assert.ok(first && second);
assert.notEqual(first.key, second.key);
assert.equal(second.forms.length, 4);
});
});
// The lookup runs once per subtitle line, so it must not stat the snapshot
// directory every call. Asserted behaviorally: an unannounced on-disk change is
// invisible until the recheck interval elapses, which can only be true if the
// filesystem is not consulted per lookup.
test('does not re-read the snapshot directory on every lookup', () => {
withTempDir((dir) => {
writeSnapshot(dir, 1, [['ミナト', 'みなと']]);
let nowMs = 1_000_000;
const lookup = createCharacterNameCandidateLookup({
outputDir: dir,
getCurrentMediaId: () => 1,
now: () => nowMs,
});
assert.equal(lookup.get()?.forms.length, 2);
writeSnapshot(dir, 1, [
['ミナト', 'みなと'],
['アクア', 'あくあ'],
]);
nowMs += 1000;
assert.equal(lookup.get()?.forms.length, 2, 'expected the cached list within the interval');
nowMs += 10_000;
assert.equal(lookup.get()?.forms.length, 4, 'expected a refresh past the interval');
});
});
test('invalidate picks up a snapshot change immediately', () => {
withTempDir((dir) => {
writeSnapshot(dir, 1, [['ミナト', 'みなと']]);
let nowMs = 1_000_000;
const lookup = createCharacterNameCandidateLookup({
outputDir: dir,
getCurrentMediaId: () => 1,
now: () => nowMs,
});
assert.equal(lookup.get()?.forms.length, 2);
writeSnapshot(dir, 1, [
['ミナト', 'みなと'],
['アクア', 'あくあ'],
]);
nowMs += 1;
lookup.invalidate();
assert.equal(lookup.get()?.forms.length, 4);
});
});
@@ -0,0 +1,142 @@
import * as fs from 'fs';
import * as path from 'path';
import { readCachedSnapshots } from './cache';
import type { CharacterDictionarySnapshot } from './types';
// Candidate name forms for the greedy name pre-pass in the Yomitan scan
// runtime. The scanner otherwise has to ask the backend at every Japanese
// position, because a character name can start mid-token; knowing which forms
// exist lets it look up only where a name can actually begin.
//
// A form is any string Yomitan could match a character entry by: the term and
// its reading. Both come from the dictionary SubMiner generated, so the pair is
// the complete matchable set for an entry. Callers treat a missing list as
// "scan every position", so a stale or absent snapshot costs speed, never a
// missed name.
function getSnapshotsDir(outputDir: string): string {
return path.join(outputDir, 'snapshots');
}
function collectSnapshotNameForms(snapshot: CharacterDictionarySnapshot): string[] {
const forms = new Set<string>();
for (const entry of snapshot.termEntries) {
const term = typeof entry[0] === 'string' ? entry[0].trim() : '';
if (term) {
forms.add(term);
}
const reading = typeof entry[1] === 'string' ? entry[1].trim() : '';
if (reading) {
forms.add(reading);
}
}
return [...forms];
}
function getSnapshotDirectorySignature(outputDir: string): string {
let entries: fs.Dirent[] = [];
try {
entries = fs.readdirSync(getSnapshotsDir(outputDir), { withFileTypes: true });
} catch {
return '';
}
const parts: string[] = [];
for (const entry of entries) {
if (!entry.isFile() || !/^anilist-\d+\.json$/.test(entry.name)) {
continue;
}
try {
const stat = fs.statSync(path.join(getSnapshotsDir(outputDir), entry.name));
parts.push(`${entry.name}:${stat.mtimeMs}:${stat.size}`);
} catch {
// Ignore files that disappear during a refresh; the next lookup rebuilds.
}
}
return parts.sort().join('|');
}
export interface CharacterNameCandidateSet {
/** Identifies this exact form list, so the scan runtime can cache it. */
key: string;
forms: string[];
}
// This lookup is consulted once per subtitle line, so it must not stat the
// snapshot directory every time. Dictionary writes are rare and always call
// invalidate(), which forces the next lookup to re-read; the interval only
// bounds staleness from changes made behind our back.
const SNAPSHOT_SIGNATURE_RECHECK_INTERVAL_MS = 5000;
export function createCharacterNameCandidateLookup(deps: {
userDataPath?: string;
outputDir?: string;
getCurrentMediaId?: () => number | null | undefined;
now?: () => number;
}): {
get: (mediaId?: number | null) => CharacterNameCandidateSet | null;
invalidate: () => void;
} {
const outputDir =
deps.outputDir ??
(deps.userDataPath ? path.join(deps.userDataPath, 'character-dictionaries') : '');
const now = deps.now ?? (() => Date.now());
let signature: string | null = null;
let lastSignatureCheckAtMs = 0;
let formsByMediaId = new Map<number, string[]>();
function refreshIfNeeded(): void {
if (!outputDir) {
formsByMediaId = new Map<number, string[]>();
signature = '';
return;
}
const nowMs = now();
if (
signature !== null &&
nowMs - lastSignatureCheckAtMs < SNAPSHOT_SIGNATURE_RECHECK_INTERVAL_MS
) {
return;
}
lastSignatureCheckAtMs = nowMs;
const nextSignature = getSnapshotDirectorySignature(outputDir);
if (nextSignature === signature) {
return;
}
signature = nextSignature;
formsByMediaId = new Map<number, string[]>();
for (const snapshot of readCachedSnapshots(outputDir)) {
const forms = collectSnapshotNameForms(snapshot);
if (forms.length > 0) {
formsByMediaId.set(snapshot.mediaId, forms);
}
}
}
return {
get(mediaId?: number | null): CharacterNameCandidateSet | null {
refreshIfNeeded();
const rawMediaId = mediaId ?? deps.getCurrentMediaId?.() ?? null;
const normalizedMediaId =
typeof rawMediaId === 'number' && Number.isFinite(rawMediaId) && rawMediaId > 0
? Math.floor(rawMediaId)
: null;
// Without a media scope the pre-pass would need every character of every
// cached title, which is both slow to match and pointless: report no
// candidates so the scanner keeps its exhaustive behavior.
if (normalizedMediaId === null) {
return null;
}
const forms = formsByMediaId.get(normalizedMediaId);
if (!forms || forms.length === 0) {
return null;
}
return { key: `${signature ?? ''}:${normalizedMediaId}`, forms };
},
invalidate(): void {
signature = null;
lastSignatureCheckAtMs = 0;
},
};
}
@@ -36,3 +36,38 @@ test('buildNameTerms adds surname honorifics from Japanese localized aliases', (
assert.ok(terms.includes('馬渕さん'));
assert.ok(!terms.includes('송치'));
});
test('buildNameTerms drops the disambiguator letter of a mob character name', () => {
const terms = buildNameTerms(
characterRecord({
firstNameHint: '',
lastNameHint: '',
fullName: 'Joshi A',
nativeName: '女子A',
}),
);
// ア would match every あ〜 in the subtitles; the letter is a disambiguator
// (Girl A / Girl B), not a name.
assert.ok(!terms.includes('ア'));
assert.ok(!terms.includes('アさん'));
assert.ok(terms.includes('女子A'));
assert.ok(terms.includes('ジョシア'));
});
test('buildNameTerms keeps a single-kanji name part', () => {
// The name is an alias, not the native name, so the parts come from the
// space split rather than from the native-name split.
const terms = buildNameTerms(
characterRecord({
firstNameHint: 'Sora',
lastNameHint: 'Yamada',
fullName: 'Sora Yamada',
nativeName: '',
alternativeNames: ['山田 空'],
}),
);
assert.ok(terms.includes('山田'));
assert.ok(terms.includes('空'));
});
@@ -42,6 +42,15 @@ export function expandRawNameVariants(rawName: string): string[] {
return [...variants];
}
// AniList disambiguates unnamed mob characters with a trailing letter (女子A /
// "Joshi A"), and a single letter romanizes into a single-kana alias (A → ア)
// that collides with interjections (あ〜 matching ア). A one-character form is
// only a real lookup target when it is kanji, so every other single-character
// form is dropped before it can become a term.
function isUsableNameTerm(name: string): boolean {
return [...name].length > 1 || containsKanji(name);
}
export function isJapaneseNameSplitCandidate(name: string): boolean {
const compact = name.replace(/[\s\u3000・・·•]/g, '');
return (
@@ -97,8 +106,11 @@ export function buildNameTerms(
const split = name.split(/[\s\u3000]+/).filter((part) => part.trim().length > 0);
if (split.length === 2) {
target.add(split[0]!);
target.add(split[1]!);
for (const part of split) {
if (isUsableNameTerm(part)) {
target.add(part);
}
}
}
const splitByMiddleDot = name
@@ -107,9 +119,11 @@ export function buildNameTerms(
.filter((part) => part.length > 0);
if (splitByMiddleDot.length >= 2) {
for (const part of splitByMiddleDot) {
if (isUsableNameTerm(part)) {
target.add(part);
}
}
}
if (target === base) {
addJapaneseNameParts(character, name, base, resolvedSplits);
@@ -136,6 +150,7 @@ export function buildNameTerms(
const withHonorifics = new Set<string>();
for (const entry of base) {
if (!isUsableNameTerm(entry)) continue;
withHonorifics.add(entry);
for (const suffix of HONORIFIC_SUFFIXES) {
withHonorifics.add(`${entry}${suffix.term}`);
+64 -8
View File
@@ -176,6 +176,29 @@ test('subtitle sidebar media path tag is assigned after prefetch succeeds', () =
);
});
test('remote media keeps parsed cues when the active subtitle source cannot be resolved', () => {
const source = readMainSource();
const actionBlock = source.match(
/createRefreshSubtitlePrefetchFromActiveTrackHandler\(\{(?<body>[\s\S]*?)\n \}\);/,
)?.groups?.body;
assert.ok(actionBlock);
assert.match(actionBlock, /isYoutubeMediaPath\(videoPath\) \|\| isRemoteMediaPath\(videoPath\)/);
});
test('jellyfin subtitle preload seeds the tokenization prefetch directly', () => {
const source = readMainSource();
const actionBlock = source.match(
/preloadJellyfinExternalSubtitlesMainDeps:\s*\{(?<body>[\s\S]*?)\n \},/,
)?.groups?.body;
assert.ok(actionBlock);
assert.match(
actionBlock,
/initSubtitlePrefetch: \(sourcePath\) =>\s*subtitlePrefetchRuntime\.refreshSubtitleSidebarFromSource\(sourcePath\),/,
);
});
test('update overlay notification action triggers install flow', () => {
const source = readMainSource();
const runtimeSource = readSource('src/main/runtime/overlay-notifications-runtime.ts');
@@ -200,7 +223,7 @@ test('update overlay notification action triggers install flow', () => {
assert.match(runtimeSource, /fallbackClient\.openNoteInBrowser\(noteId\)/);
});
test('subtitle change re-prioritizes prefetch around live playback before tokenizing current line', () => {
test('subtitle change pauses prefetch without restarting its run before tokenizing current line', () => {
const source = readMainSource();
const actionBlock = source.match(
/onSubtitleChange:\s*\(text\)\s*=>\s*\{(?<body>[\s\S]*?)\n \},\n refreshDiscordPresence:/,
@@ -208,15 +231,19 @@ test('subtitle change re-prioritizes prefetch around live playback before tokeni
assert.ok(actionBlock);
assert.match(actionBlock, /subtitlePrefetchService\?\.pause\(\);/);
assert.match(actionBlock, /subtitlePrefetchService\?\.onSeek\(lastObservedTimePos\);/);
assert.match(actionBlock, /subtitleProcessingController\.onSubtitleChange\(text\);/);
// Restarting the run per line (onSeek) discards in-flight prefetch work;
// only real seeks restart via onTimePosUpdate.
assert.doesNotMatch(actionBlock, /subtitlePrefetchService\?\.onSeek\(/);
assert.match(actionBlock, /subtitleProcessingController\.onSubtitleChange\(text\)/);
assert.ok(
actionBlock.indexOf('subtitlePrefetchService?.pause();') <
actionBlock.indexOf('subtitlePrefetchService?.onSeek(lastObservedTimePos);'),
actionBlock.indexOf('subtitleProcessingController.onSubtitleChange(text)'),
);
assert.ok(
actionBlock.indexOf('subtitlePrefetchService?.onSeek(lastObservedTimePos);') <
actionBlock.indexOf('subtitleProcessingController.onSubtitleChange(text);'),
// A repeated subtitle emits nothing, so the pause has to be released here or
// prefetching idles until the next distinct line.
assert.match(
actionBlock,
/if \(!subtitleProcessingController\.onSubtitleChange\(text\)\) \{[\s\S]*?subtitlePrefetchService\?\.resume\(\);/,
);
});
@@ -570,7 +597,7 @@ test('YouTube media cache lifecycle routes through configured status notificatio
test('subtitle broadcasts share one frequency options snapshot per emitted payload', () => {
const source = readMainSource();
const emitBlock = source.match(
/function emitSubtitlePayload\(payload: SubtitleData\): void \{(?<body>[\s\S]*?)\n\}/,
/function emitSubtitlePayload\([\s\S]*?\): void \{(?<body>[\s\S]*?)\n\}/,
)?.groups?.body;
const frequencyOptionsSnapshot = emitBlock?.match(
/const frequencyDictionary = configService\.getConfig\(\)\.subtitleStyle\.frequencyDictionary;(?<body>[\s\S]*?)\n \};/,
@@ -590,6 +617,35 @@ test('subtitle broadcasts share one frequency options snapshot per emitted paylo
);
});
test('annotation upgrades skip the duplicate basic websocket event', () => {
const source = readMainSource();
const emitBlock = source.match(
/function emitSubtitlePayload\([\s\S]*?\): void \{(?<body>[\s\S]*?)\n\}/,
)?.groups?.body;
assert.ok(emitBlock);
assert.match(
emitBlock,
/const isAnnotationUpgrade = isSubtitleAnnotationUpgrade\(currentSubtitleData, timedPayload\);/,
);
assert.match(
emitBlock,
/if \(!isAnnotationUpgrade\) \{\s+subtitleWsService\.broadcast\(timedPayload, frequencyOptions\);\s+\}/,
);
assert.equal(
(emitBlock.match(/overlayManager\.broadcastToOverlayWindows\('subtitle:set'/g) ?? []).length,
1,
);
assert.equal(
(
emitBlock.match(
/annotationSubtitleWsService\.broadcast\(timedPayload, frequencyOptions\)/g,
) ?? []
).length,
1,
);
});
test('websocket frequency options callbacks each read one configuration snapshot', () => {
const source = readMainSource();
const subtitleBlock = source.match(
+14 -2
View File
@@ -326,7 +326,13 @@ test('handleOverlayModalClosed hides modal window only after all pending modals
});
runtime.sendToActiveOverlayWindow(
'subsync:open-manual',
{ ffsubsyncAvailable: true, sourceTracks: [] },
{
ffsubsyncAvailable: true,
videoReferenceAvailable: true,
subtitleTracks: [],
defaultReferenceTrackId: null,
defaultTargetTrackId: null,
},
{
restoreOnModalClose: 'subsync',
},
@@ -560,7 +566,13 @@ test('modal runtime notifies callers when modal input state becomes active/inact
});
runtime.sendToActiveOverlayWindow(
'subsync:open-manual',
{ ffsubsyncAvailable: true, sourceTracks: [] },
{
ffsubsyncAvailable: true,
videoReferenceAvailable: true,
subtitleTracks: [],
defaultReferenceTrackId: null,
defaultTargetTrackId: null,
},
{
restoreOnModalClose: 'subsync',
},
@@ -1,5 +1,7 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { createSubtitleProcessingController } from '../../core/services/subtitle-processing-controller';
import type { SubtitleData } from '../../types';
import {
createAutoplaySubtitlePrimingRuntime,
setMpvCurrentSecondarySubText,
@@ -42,8 +44,8 @@ test('scheduleSubtitlePrefetchRefresh logs refresh failures from timer callback'
setActiveParsedSubtitleMediaPath: () => {},
subtitleProcessingController: {
consumeCachedSubtitle: () => null,
onSubtitleChange: () => {},
refreshCurrentSubtitle: () => {},
onSubtitleChange: () => true,
refreshCurrentSubtitle: () => true,
},
emitSubtitlePayload: () => {},
getSubtitlePrefetchService: () => null,
@@ -93,13 +95,24 @@ test('primeCurrentSubtitleForAutoplay refreshes active subtitle cues when mpv su
setActiveParsedSubtitleMediaPath: () => {},
subtitleProcessingController: {
consumeCachedSubtitle: () => null,
onSubtitleChange: (text) => calls.push(`change:${text}`),
refreshCurrentSubtitle: (text) => calls.push(`refresh:${text ?? ''}`),
onSubtitleChange: (text) => {
calls.push(`change:${text}`);
return true;
},
emitSubtitlePayload: (payload) => calls.push(`emit:${payload.text}`),
refreshCurrentSubtitle: (text) => {
calls.push(`refresh:${text ?? ''}`);
return true;
},
},
emitSubtitlePayload: (payload, options) =>
calls.push(`emit:${payload.text}:resume=${options?.resumePrefetch !== false}`),
getSubtitlePrefetchService: () => ({
pause: () => calls.push('prefetch:pause'),
onSeek: (timePos) => calls.push(`prefetch:seek:${timePos}`),
pause: () => {
calls.push('prefetch:pause');
},
resume: () => {
calls.push('prefetch:resume');
},
}),
getLastObservedTimePos: () => 12,
getVisibleOverlayVisible: () => true,
@@ -120,8 +133,10 @@ test('primeCurrentSubtitleForAutoplay refreshes active subtitle cues when mpv su
'request:time-pos',
'set:起動字幕',
'prefetch:pause',
'emit:起動字幕',
'change:起動字幕',
'emit:起動字幕:resume=false',
// Uncached priming refreshes rather than announcing a change, so an
// invalidated-but-unchanged line is still re-tokenized.
'refresh:起動字幕',
]);
});
@@ -151,13 +166,24 @@ test('primeCurrentSubtitleForAutoplay emits raw first paint on cache miss before
setActiveParsedSubtitleMediaPath: () => {},
subtitleProcessingController: {
consumeCachedSubtitle: () => null,
onSubtitleChange: (text) => calls.push(`change:${text}`),
refreshCurrentSubtitle: (text) => calls.push(`refresh:${text ?? ''}`),
onSubtitleChange: (text) => {
calls.push(`change:${text}`);
return true;
},
emitSubtitlePayload: (payload) => calls.push(`emit:${payload.text}`),
refreshCurrentSubtitle: (text) => {
calls.push(`refresh:${text ?? ''}`);
return true;
},
},
emitSubtitlePayload: (payload, options) =>
calls.push(`emit:${payload.text}:resume=${options?.resumePrefetch !== false}`),
getSubtitlePrefetchService: () => ({
pause: () => calls.push('prefetch:pause'),
onSeek: (timePos) => calls.push(`prefetch:seek:${timePos}`),
pause: () => {
calls.push('prefetch:pause');
},
resume: () => {
calls.push('prefetch:resume');
},
}),
getLastObservedTimePos: () => 12,
getVisibleOverlayVisible: () => true,
@@ -175,7 +201,138 @@ test('primeCurrentSubtitleForAutoplay emits raw first paint on cache miss before
'request:sub-text',
'set:起動字幕',
'prefetch:pause',
'emit:起動字幕',
'change:起動字幕',
'emit:起動字幕:resume=false',
// Uncached priming refreshes rather than announcing a change, so an
// invalidated-but-unchanged line is still re-tokenized.
'refresh:起動字幕',
]);
});
// Driven by the real processing controller rather than a stub: the failure this
// covers is a disagreement between the priming path and the controller's own
// staleness rules, which a hand-written stub cannot reproduce.
function createPrimingRuntimeWithRealController(options: {
text: string;
calls: string[];
onTokenize: () => void;
cacheLimit?: number;
}) {
const { text, calls } = options;
let currentSubText = '';
let currentSubtitleData: SubtitleData | null = null;
const mediaPath = '/media/video.mkv';
const subtitleProcessingController = createSubtitleProcessingController({
tokenizeSubtitle: async (subtitleText) => {
options.onTokenize();
return { text: subtitleText, tokens: [] };
},
emitSubtitle: (payload) => {
currentSubtitleData = payload;
calls.push(`emit:${payload.text}:tokens=${payload.tokens === null ? 'none' : 'yes'}`);
},
...(options.cacheLimit === undefined ? {} : { cacheLimit: options.cacheLimit }),
});
const runtime = createAutoplaySubtitlePrimingRuntime({
getCurrentMediaPath: () => mediaPath,
getMpvClient: () => ({
connected: true,
currentVideoPath: mediaPath,
requestProperty: async (name) => (name === 'sub-text' ? text : null),
}),
setCurrentSubText: (value) => {
currentSubText = value;
},
getCurrentSubText: () => currentSubText,
getCurrentSubtitleData: () => currentSubtitleData,
getActiveParsedSubtitleCues: () => [],
setActiveParsedSubtitleMediaPath: () => {},
subtitleProcessingController,
emitSubtitlePayload: (payload, emitOptions) => {
if (emitOptions?.resumePrefetch === false) {
calls.push(`emit-raw:${payload.text}`);
return;
}
calls.push(`emit-direct:${payload.text}`);
},
getSubtitlePrefetchService: () => ({
pause: () => calls.push('prefetch:pause'),
resume: () => calls.push('prefetch:resume'),
}),
getLastObservedTimePos: () => 12,
getVisibleOverlayVisible: () => true,
emitSecondarySubtitle: () => {},
initSubtitlePrefetch: async () => {},
refreshSubtitlePrefetchFromActiveTrack: async () => {},
logDebug: () => {},
});
return { runtime, subtitleProcessingController, mediaPath };
}
test('primeCurrentSubtitleForAutoplay re-tokenizes text whose cached annotation was invalidated', async () => {
const calls: string[] = [];
let tokenizations = 0;
const text = '起動字幕';
const { runtime, subtitleProcessingController, mediaPath } =
createPrimingRuntimeWithRealController({
text,
calls,
onTokenize: () => {
tokenizations += 1;
},
});
// The line was already tokenized and cached during normal playback.
subtitleProcessingController.onSubtitleChange(text);
await new Promise((resolve) => setTimeout(resolve, 0));
const tokenizationsBeforeInvalidation = tokenizations;
// Mining a card drops every cached tokenization.
subtitleProcessingController.invalidateTokenizationCache();
calls.length = 0;
await runtime.primeCurrentSubtitleForAutoplay(mediaPath);
await new Promise((resolve) => setTimeout(resolve, 0));
// The cache miss must schedule fresh work, or the line stays unannotated for
// as long as it is on screen.
assert.equal(
tokenizations,
tokenizationsBeforeInvalidation + 1,
'expected the invalidated subtitle to be tokenized again',
);
assert.ok(
calls.includes(`emit:${text}:tokens=yes`),
`expected an annotated emit, saw ${JSON.stringify(calls)}`,
);
});
test('primeCurrentSubtitleForAutoplay releases the prefetch pause when nothing is scheduled', async () => {
const calls: string[] = [];
const text = '起動字幕';
const { runtime, subtitleProcessingController, mediaPath } =
createPrimingRuntimeWithRealController({
text,
calls,
onTokenize: () => {},
cacheLimit: 1,
});
// Emitted at the current cache generation, then evicted from the one-entry
// cache: priming misses the cache but the controller has nothing to redo, so
// no emit is coming and the pause must be released here.
subtitleProcessingController.onSubtitleChange(text);
await new Promise((resolve) => setTimeout(resolve, 0));
subtitleProcessingController.preCacheTokenization('別の字幕', {
text: '別の字幕',
tokens: [],
});
calls.length = 0;
await runtime.primeCurrentSubtitleForAutoplay(mediaPath);
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, ['prefetch:pause', `emit-raw:${text}`, 'prefetch:resume']);
});
@@ -17,7 +17,7 @@ type AutoplaySubtitlePrimingMpvClient = {
type AutoplaySubtitlePrimingPrefetchService = {
pause: () => void;
onSeek: (timePos: number) => void;
resume: () => void;
};
export interface AutoplaySubtitlePrimingRuntimeDeps {
@@ -30,10 +30,11 @@ export interface AutoplaySubtitlePrimingRuntimeDeps {
setActiveParsedSubtitleMediaPath: (mediaPath: string | null) => void;
subtitleProcessingController: {
consumeCachedSubtitle: (text: string) => SubtitleData | null;
onSubtitleChange: (text: string) => void;
refreshCurrentSubtitle: (text: string) => void;
// Both report whether an emit is expected; see pausePrefetchUntilEmit.
onSubtitleChange: (text: string) => boolean;
refreshCurrentSubtitle: (text: string) => boolean;
};
emitSubtitlePayload: (payload: SubtitleData) => void;
emitSubtitlePayload: (payload: SubtitleData, options?: { resumePrefetch?: boolean }) => void;
getSubtitlePrefetchService: () => AutoplaySubtitlePrimingPrefetchService | null;
getLastObservedTimePos: () => number;
getVisibleOverlayVisible: () => boolean;
@@ -64,6 +65,18 @@ export function setMpvCurrentSecondarySubText(
export function createAutoplaySubtitlePrimingRuntime(deps: AutoplaySubtitlePrimingRuntimeDeps) {
const { subtitleProcessingController, emitSubtitlePayload } = deps;
// Prefetching is paused so the on-screen line gets the parser to itself, and
// the resume rides on the tokenized emit. When the controller reports that no
// emit is coming (repeat text with nothing scheduled), release it here or
// prefetching idles until some later line happens to complete.
function pausePrefetchUntilEmit(scheduleTokenization: () => boolean): void {
const prefetch = deps.getSubtitlePrefetchService();
prefetch?.pause();
if (!scheduleTokenization()) {
prefetch?.resume();
}
}
let subtitlePrefetchRefreshTimer: ReturnType<typeof setTimeout> | null = null;
let autoplaySubtitlePrimedMediaPath: string | null = null;
let visibleOverlaySubtitleRefreshAfterFirstPaintTimer: ReturnType<typeof setTimeout> | null =
@@ -104,12 +117,23 @@ export function createAutoplaySubtitlePrimingRuntime(deps: AutoplaySubtitlePrimi
const cachedPayload = subtitleProcessingController.consumeCachedSubtitle(text);
if (cachedPayload) {
subtitleProcessingController.onSubtitleChange(text);
// This emit resumes prefetching, so no pause is left outstanding.
emitSubtitlePayload(cachedPayload);
return true;
}
emitSubtitlePayload({ text, tokens: null });
subtitleProcessingController.onSubtitleChange(text);
// Provisional raw emit: keep prefetch paused until the tokenized payload
// for this line is delivered by the processing controller.
emitSubtitlePayload({ text, tokens: null }, { resumePrefetch: false });
// refreshCurrentSubtitle, not onSubtitleChange: the cache miss above can be
// an invalidation (mining a card) on text the controller still holds, and
// onSubtitleChange treats unchanged text as nothing to do, which would
// leave this line permanently unannotated. refreshCurrentSubtitle also
// re-tokenizes for a new cache generation.
if (!subtitleProcessingController.refreshCurrentSubtitle(text)) {
// Nothing scheduled, so no emit is coming to release the pause.
deps.getSubtitlePrefetchService()?.resume();
}
return true;
}
@@ -153,14 +177,10 @@ export function createAutoplaySubtitlePrimingRuntime(deps: AutoplaySubtitlePrimi
getCurrentSubtitleData: () => deps.getCurrentSubtitleData(),
consumeCachedSubtitle: (text) => subtitleProcessingController.consumeCachedSubtitle(text),
onSubtitleChange: (text) => {
deps.getSubtitlePrefetchService()?.pause();
deps.getSubtitlePrefetchService()?.onSeek(deps.getLastObservedTimePos());
subtitleProcessingController.onSubtitleChange(text);
pausePrefetchUntilEmit(() => subtitleProcessingController.onSubtitleChange(text));
},
refreshCurrentSubtitle: (text) => {
deps.getSubtitlePrefetchService()?.pause();
deps.getSubtitlePrefetchService()?.onSeek(deps.getLastObservedTimePos());
subtitleProcessingController.refreshCurrentSubtitle(text);
pausePrefetchUntilEmit(() => subtitleProcessingController.refreshCurrentSubtitle(text));
},
deferUncachedRefresh: true,
emitSubtitle: (payload) => emitSubtitlePayload(payload),
@@ -204,9 +224,7 @@ export function createAutoplaySubtitlePrimingRuntime(deps: AutoplaySubtitlePrimi
if (!text.trim()) {
return;
}
deps.getSubtitlePrefetchService()?.pause();
deps.getSubtitlePrefetchService()?.onSeek(deps.getLastObservedTimePos());
subtitleProcessingController.refreshCurrentSubtitle(text);
pausePrefetchUntilEmit(() => subtitleProcessingController.refreshCurrentSubtitle(text));
}, VISIBLE_OVERLAY_SUBTITLE_REFRESH_AFTER_FIRST_PAINT_DELAY_MS);
visibleOverlaySubtitleRefreshAfterFirstPaintTimer.unref?.();
}
@@ -53,3 +53,52 @@ test('character dictionary sync completion refreshes subtitle state when diction
'log:[dictionary:auto-sync] refreshed current subtitle after sync (AniList 1, changed=yes, title=Frieren)',
]);
});
test('character dictionary sync completion drops cached dictionary reads before refreshing', () => {
const calls: string[] = [];
handleCharacterDictionaryAutoSyncComplete(
{
mediaId: 1,
mediaTitle: 'Frieren',
changed: true,
},
{
hasParserWindow: () => true,
invalidateCharacterDictionaryLookups: () => calls.push('invalidate-dictionary-lookups'),
clearParserCaches: () => calls.push('clear-parser'),
invalidateTokenizationCache: () => calls.push('invalidate'),
refreshSubtitlePrefetch: () => calls.push('prefetch'),
refreshCurrentSubtitle: () => calls.push('refresh-subtitle'),
logInfo: () => {},
},
);
// Must run before the refreshes, or they re-tokenize against the character
// names and images from the previous dictionary build.
assert.equal(calls[0], 'invalidate-dictionary-lookups');
assert.ok(calls.indexOf('invalidate-dictionary-lookups') < calls.indexOf('refresh-subtitle'));
});
test('character dictionary sync completion leaves cached dictionary reads alone when unchanged', () => {
const calls: string[] = [];
handleCharacterDictionaryAutoSyncComplete(
{
mediaId: 1,
mediaTitle: 'Frieren',
changed: false,
},
{
hasParserWindow: () => true,
invalidateCharacterDictionaryLookups: () => calls.push('invalidate-dictionary-lookups'),
clearParserCaches: () => calls.push('clear-parser'),
invalidateTokenizationCache: () => calls.push('invalidate'),
refreshSubtitlePrefetch: () => calls.push('prefetch'),
refreshCurrentSubtitle: () => calls.push('refresh-subtitle'),
logInfo: () => {},
},
);
assert.deepEqual(calls, []);
});
@@ -7,6 +7,12 @@ export function handleCharacterDictionaryAutoSyncComplete(
deps: {
hasParserWindow: () => boolean;
clearParserCaches: () => void;
/**
* Drops cached reads of the generated dictionary (character images, and the
* name candidates the scanner uses to skip lookups). Runs before the
* refreshes below so they re-tokenize against the new dictionary content.
*/
invalidateCharacterDictionaryLookups?: () => void;
invalidateTokenizationCache: () => void;
refreshSubtitlePrefetch: () => void;
refreshCurrentSubtitle: () => void;
@@ -14,6 +20,7 @@ export function handleCharacterDictionaryAutoSyncComplete(
},
): void {
if (completion.changed) {
deps.invalidateCharacterDictionaryLookups?.();
if (deps.hasParserWindow()) {
deps.clearParserCaches();
}
@@ -4,7 +4,7 @@ import { composeIpcRuntimeHandlers } from './ipc-runtime-composer';
test('composeIpcRuntimeHandlers returns callable IPC handlers and registration bridge', async () => {
let registered = false;
let receivedSourceTrackId: number | null | undefined;
let receivedReferenceTrackId: number | null | undefined;
const composed = composeIpcRuntimeHandlers({
mpvCommandMainDeps: {
@@ -25,7 +25,7 @@ test('composeIpcRuntimeHandlers returns callable IPC handlers and registration b
},
handleMpvCommandFromIpcRuntime: () => {},
runSubsyncManualFromIpc: async (request) => {
receivedSourceTrackId = request.sourceTrackId;
receivedReferenceTrackId = request.referenceTrackId;
return {
ok: true,
message: 'ok',
@@ -124,10 +124,10 @@ test('composeIpcRuntimeHandlers returns callable IPC handlers and registration b
const result = await composed.runSubsyncManualFromIpc({
engine: 'alass',
sourceTrackId: 7,
referenceTrackId: 7,
});
assert.deepEqual(result, { ok: true, message: 'ok' });
assert.equal(receivedSourceTrackId, 7);
assert.equal(receivedReferenceTrackId, 7);
composed.registerIpcRuntimeHandlers();
assert.equal(registered, true);
@@ -225,24 +225,35 @@ export function composeMpvRuntimeHandlers<
}
return tokenizationWarmupInFlight;
};
const tokenizeSubtitle = async (text: string): Promise<TTokenizedSubtitle> => {
if (!tokenizationWarmupCompleted) void startTokenizationWarmups();
await ensureTokenizationPrerequisites();
// Built once and reused for every tokenization: per-call rebuilds create
// fresh closures, which defeats identity-keyed caches downstream (the JLPT
// lookup cache keys on the getJlptLevel function, and the mecab availability
// WeakSet keys on the runtime deps instance).
let cachedTokenizerRuntimeDeps: TTokenizerRuntimeDeps | null = null;
const getTokenizerRuntimeDeps = (): TTokenizerRuntimeDeps => {
if (cachedTokenizerRuntimeDeps) {
return cachedTokenizerRuntimeDeps;
}
const tokenizerMainDeps = buildTokenizerDepsHandler();
if (shouldWarmupAnnotationDictionaries()) {
const onTokenizationReady = tokenizerMainDeps.onTokenizationReady;
const baseOnTokenizationReady = tokenizerMainDeps.onTokenizationReady;
tokenizerMainDeps.onTokenizationReady = (tokenizedText: string): void => {
if (!shouldWarmupAnnotationDictionaries()) {
baseOnTokenizationReady?.(tokenizedText);
return;
}
markTokenizationPlaybackReady();
onTokenizationReady?.(tokenizedText);
baseOnTokenizationReady?.(tokenizedText);
if (!tokenizationWarmupCompleted) {
void prewarmSubtitleDictionaries({ showLoadingOsd: true }).catch(() => {});
}
};
}
return options.tokenizer.tokenizeSubtitle(
text,
options.tokenizer.createTokenizerRuntimeDeps(tokenizerMainDeps),
);
cachedTokenizerRuntimeDeps = options.tokenizer.createTokenizerRuntimeDeps(tokenizerMainDeps);
return cachedTokenizerRuntimeDeps;
};
const tokenizeSubtitle = async (text: string): Promise<TTokenizedSubtitle> => {
if (!tokenizationWarmupCompleted) void startTokenizationWarmups();
await ensureTokenizationPrerequisites();
return options.tokenizer.tokenizeSubtitle(text, getTokenizerRuntimeDeps());
};
const launchBackgroundWarmupTask = createLaunchBackgroundWarmupTaskFromStartup(
@@ -11,7 +11,7 @@ import { createOverlayNotificationDelivery } from './overlay-notification-delive
test('notifyConfiguredStatus routes both to overlay and system without osd', () => {
const calls: string[] = [];
notifyConfiguredStatus('Subsync: choose engine and source', {
notifyConfiguredStatus('Subsync: choose engine and subtitles', {
getNotificationType: () => 'both',
showOsd: (message) => {
calls.push(`osd:${message}`);
@@ -25,8 +25,8 @@ test('notifyConfiguredStatus routes both to overlay and system without osd', ()
});
assert.deepEqual(calls, [
'overlay::SubMiner:Subsync: choose engine and source:info:auto',
'desktop:SubMiner:Subsync: choose engine and source',
'overlay::SubMiner:Subsync: choose engine and subtitles:info:auto',
'desktop:SubMiner:Subsync: choose engine and subtitles',
]);
});
@@ -28,6 +28,9 @@ export function createBuildPreloadJellyfinExternalSubtitlesMainDepsHandler(
? (itemId, streamIndex, delaySeconds) =>
deps.saveSubtitleDelay!(itemId, streamIndex, delaySeconds)
: undefined,
initSubtitlePrefetch: deps.initSubtitlePrefetch
? (sourcePath) => deps.initSubtitlePrefetch!(sourcePath)
: undefined,
logDebug: (message: string, error: unknown) => deps.logDebug(message, error),
});
}
@@ -40,6 +40,9 @@ function makeDeps(overrides: {
>[0]['setActiveSubtitleDelayKey'];
loadSubtitleSourceText?: (source: string) => Promise<string>;
saveSubtitleDelay?: (itemId: string, streamIndex: number, delaySeconds: number) => void;
initSubtitlePrefetch?: Parameters<
typeof createPreloadJellyfinExternalSubtitlesHandler
>[0]['initSubtitlePrefetch'];
logDebug?: Parameters<typeof createPreloadJellyfinExternalSubtitlesHandler>[0]['logDebug'];
}) {
return {
@@ -58,6 +61,7 @@ function makeDeps(overrides: {
setActiveSubtitleDelayKey: overrides.setActiveSubtitleDelayKey,
loadSubtitleSourceText: overrides.loadSubtitleSourceText,
saveSubtitleDelay: overrides.saveSubtitleDelay,
initSubtitlePrefetch: overrides.initSubtitlePrefetch,
logDebug: overrides.logDebug ?? (() => {}),
};
}
@@ -134,6 +138,92 @@ test('preload jellyfin subtitles caches external tracks locally and chooses japa
]);
});
test('preload jellyfin subtitles starts prefetch for the selected japanese track', async () => {
const prefetched: string[] = [];
const preload = createPreloadJellyfinExternalSubtitlesHandler(
makeDeps({
listJellyfinSubtitleTracks: async () => [
{ index: 0, language: 'jpn', title: 'Japanese', deliveryUrl: 'https://sub/a.srt' },
{ index: 1, language: 'eng', title: 'English', deliveryUrl: 'https://sub/b.srt' },
],
getMpvClient: () => ({
requestProperty: async () => [
{
type: 'sub',
id: 5,
lang: 'jpn',
title: 'Japanese',
external: true,
'external-filename': '/tmp/subminer-jellyfin-subtitles/0.srt',
},
{
type: 'sub',
id: 6,
lang: 'eng',
title: 'English',
external: true,
'external-filename': '/tmp/subminer-jellyfin-subtitles/1.srt',
},
],
}),
cacheSubtitleTrack: async (track) => ({
path: `/tmp/subminer-jellyfin-subtitles/${track.index}.srt`,
cleanupDir: '/tmp/subminer-jellyfin-subtitles',
}),
initSubtitlePrefetch: (sourcePath) => {
prefetched.push(sourcePath);
},
}),
);
await preload({ session, clientInfo, itemId: 'item-1' });
await new Promise((resolve) => setImmediate(resolve));
assert.deepEqual(prefetched, ['/tmp/subminer-jellyfin-subtitles/0.srt']);
});
test('preload jellyfin subtitles survives prefetch start failures', async () => {
const logs: string[] = [];
const commands: Array<Array<string | number>> = [];
const preload = createPreloadJellyfinExternalSubtitlesHandler(
makeDeps({
listJellyfinSubtitleTracks: async () => [
{ index: 0, language: 'jpn', title: 'Japanese', deliveryUrl: 'https://sub/a.srt' },
],
getMpvClient: () => ({
requestProperty: async () => [
{
type: 'sub',
id: 5,
lang: 'jpn',
title: 'Japanese',
external: true,
'external-filename': '/tmp/subminer-jellyfin-subtitles/0.srt',
},
],
}),
sendMpvCommand: (command) => commands.push(command),
cacheSubtitleTrack: async (track) => ({
path: `/tmp/subminer-jellyfin-subtitles/${track.index}.srt`,
cleanupDir: '/tmp/subminer-jellyfin-subtitles',
}),
initSubtitlePrefetch: async () => {
throw new Error('parse failed');
},
logDebug: (message) => logs.push(message),
}),
);
await preload({ session, clientInfo, itemId: 'item-1' });
await new Promise((resolve) => setImmediate(resolve));
assert.deepEqual(logs, ['Failed to start subtitle prefetch for Jellyfin subtitle']);
assert.ok(
commands.some((command) => command[0] === 'set_property' && command[1] === 'sid'),
'subtitle selection still happens when prefetch start fails',
);
});
test('preload jellyfin subtitles stages tracks without temporary subtitle selection', async () => {
const commands: Array<Array<string | number>> = [];
const preload = createPreloadJellyfinExternalSubtitlesHandler(
@@ -320,6 +320,7 @@ export function createPreloadJellyfinExternalSubtitlesHandler(deps: {
setActiveSubtitleDelayKey?: (key: JellyfinSubtitleDelayKey | null) => void;
loadSubtitleSourceText?: (source: string) => Promise<string>;
saveSubtitleDelay?: (itemId: string, streamIndex: number, delaySeconds: number) => boolean | void;
initSubtitlePrefetch?: (sourcePath: string) => void | Promise<void>;
logDebug: (message: string, error: unknown) => void;
}): PreloadJellyfinExternalSubtitlesHandler {
const activeCacheDirs = new Set<string>();
@@ -329,6 +330,18 @@ export function createPreloadJellyfinExternalSubtitlesHandler(deps: {
deps.sendMpvCommand(['set_property', 'sub-delay', 0]);
}
// mpv's sid property-change is the only thing that normally starts prefetching, so a
// coalesced or missed event leaves the whole episode uncached. The downloaded path is
// known here, so seed the pipeline directly instead of waiting on the observer.
function startSubtitlePrefetchForCachedTrack(sourcePath: string): void {
if (!deps.initSubtitlePrefetch) return;
void Promise.resolve()
.then(() => deps.initSubtitlePrefetch!(sourcePath))
.catch((error) => {
deps.logDebug('Failed to start subtitle prefetch for Jellyfin subtitle', error);
});
}
function cleanupActiveCache(): void {
const dirs = [...activeCacheDirs];
if (dirs.length === 0) return;
@@ -438,6 +451,7 @@ export function createPreloadJellyfinExternalSubtitlesHandler(deps: {
}
}
deps.sendMpvCommand(['set_property', 'sid', japanesePrimaryId]);
startSubtitlePrefetchForCachedTrack(selectedCachedTrack.path);
} else {
deps.setActiveSubtitleDelayKey?.(null);
resetManagedSubtitleDelay();
@@ -70,6 +70,24 @@ test('subtitle change handler broadcasts cached annotated payload immediately wh
]);
});
test('subtitle change handler logs debug when a cached payload is emitted immediately', () => {
const debugs: string[] = [];
const handler = createHandleMpvSubtitleChangeHandler({
setCurrentSubText: () => {},
getImmediateSubtitlePayload: (text) => (text ? { text, tokens: [] } : null),
broadcastSubtitle: () => {},
onSubtitleChange: () => {},
refreshDiscordPresence: () => {},
logDebug: (message) => debugs.push(message),
});
handler({ text: 'キャッシュ済みの行' });
handler({ text: '' });
assert.equal(debugs.length, 1);
assert.match(debugs[0]!, /cached subtitle/);
});
test('subtitle change handler emits cached annotation after forwarding the subtitle change', () => {
const calls: string[] = [];
const handler = createHandleMpvSubtitleChangeHandler({
@@ -20,11 +20,15 @@ export function createHandleMpvSubtitleChangeHandler(deps: {
broadcastSubtitle: (payload: SubtitleData) => void;
onSubtitleChange: (text: string) => void;
refreshDiscordPresence: () => void;
logDebug?: (message: string) => void;
}) {
return ({ text }: { text: string }): void => {
deps.setCurrentSubText(text);
const immediatePayload = deps.getImmediateSubtitlePayload?.(text) ?? null;
if (immediatePayload) {
deps.logDebug?.(
`[subtitle-processing] emitted cached subtitle immediately (${text.length} chars)`,
);
deps.onSubtitleChange(text);
(deps.emitImmediateSubtitle ?? deps.broadcastSubtitle)(immediatePayload);
} else {
@@ -47,6 +47,7 @@ export function createBindMpvMainEventHandlersHandler(deps: {
emitImmediateSubtitle?: (payload: SubtitleData) => void;
broadcastSubtitle: (payload: SubtitleData) => void;
onSubtitleChange: (text: string) => void;
logSubtitleProcessingDebug?: (message: string) => void;
refreshDiscordPresence: () => void;
setCurrentSubAssText: (text: string) => void;
@@ -123,6 +124,9 @@ export function createBindMpvMainEventHandlersHandler(deps: {
: undefined,
broadcastSubtitle: (payload) => deps.broadcastSubtitle(payload),
onSubtitleChange: (text) => deps.onSubtitleChange(text),
logDebug: deps.logSubtitleProcessingDebug
? (message) => deps.logSubtitleProcessingDebug?.(message)
: undefined,
refreshDiscordPresence: () => deps.refreshDiscordPresence(),
});
const handleMpvSubtitleAssChange = createHandleMpvSubtitleAssChangeHandler({
@@ -54,6 +54,7 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
getImmediateSubtitlePayload?: (text: string) => SubtitleData | null;
emitImmediateSubtitle?: (payload: SubtitleData) => void;
onSubtitleChange: (text: string) => void;
logSubtitleProcessingDebug?: (message: string) => void;
onSubtitleTrackChange?: (sid: number | null) => void;
onSubtitleTrackListChange?: (trackList: unknown[] | null) => void;
updateCurrentMediaPath: (path: string) => void;
@@ -155,6 +156,9 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
broadcastSubtitle: (payload: SubtitleData) =>
deps.broadcastToOverlayWindows('subtitle:set', payload),
onSubtitleChange: (text: string) => deps.onSubtitleChange(text),
logSubtitleProcessingDebug: deps.logSubtitleProcessingDebug
? (message: string) => deps.logSubtitleProcessingDebug!(message)
: undefined,
onSubtitleTrackChange: deps.onSubtitleTrackChange
? (sid: number | null) => deps.onSubtitleTrackChange!(sid)
: undefined,
+7 -1
View File
@@ -5,7 +5,13 @@ import type { SubsyncManualPayload } from '../../types';
const payload: SubsyncManualPayload = {
ffsubsyncAvailable: true,
sourceTracks: [{ id: 2, label: 'External #2 - eng' }],
videoReferenceAvailable: true,
subtitleTracks: [
{ id: 1, label: 'Internal #1 - jpn (active)' },
{ id: 2, label: 'External #2 - eng' },
],
defaultReferenceTrackId: 2,
defaultTargetTrackId: 1,
};
test('subsync manual open prefers dedicated modal window on first attempt', async () => {
@@ -54,7 +54,6 @@ test('latest subtitle prefetch init wins over stale async loads', async () => {
}),
tokenizeSubtitle: async () => null,
preCacheTokenization: () => {},
isCacheFull: () => false,
logInfo: () => {},
logWarn: () => {},
});
@@ -99,7 +98,6 @@ test('cancelPendingInit prevents an in-flight load from attaching a stale servic
}),
tokenizeSubtitle: async () => null,
preCacheTokenization: () => {},
isCacheFull: () => false,
logInfo: () => {},
logWarn: () => {},
});
@@ -137,7 +135,6 @@ test('subtitle prefetch init publishes parsed cues and clears them on cancel', a
}),
tokenizeSubtitle: async () => null,
preCacheTokenization: () => {},
isCacheFull: () => false,
logInfo: () => {},
logWarn: () => {},
onParsedSubtitleCuesChanged: (cues) => {
@@ -181,7 +178,6 @@ test('subtitle prefetch init publishes the provided stable source key instead of
}),
tokenizeSubtitle: async () => null,
preCacheTokenization: () => {},
isCacheFull: () => false,
logInfo: () => {},
logWarn: () => {},
onParsedSubtitleCuesChanged: (_cues, source) => {
@@ -222,7 +218,6 @@ test('subtitle prefetch init clears parsed cues when initialization fails', asyn
}),
tokenizeSubtitle: async () => null,
preCacheTokenization: () => {},
isCacheFull: () => false,
logInfo: () => {},
logWarn: () => {},
onParsedSubtitleCuesChanged: (cues) => {
@@ -234,3 +229,25 @@ test('subtitle prefetch init clears parsed cues when initialization fails', asyn
assert.deepEqual(cueUpdates, [null]);
});
test('subtitle prefetch init logs a warning when the source parses to zero cues', async () => {
const warnings: string[] = [];
const controller = createSubtitlePrefetchInitController({
getCurrentService: () => null,
setCurrentService: () => {},
loadSubtitleSourceText: async () => 'not really subtitles',
parseSubtitleCues: (): SubtitleCue[] => [],
createSubtitlePrefetchService: () => {
throw new Error('should not create a service without cues');
},
tokenizeSubtitle: async () => null,
preCacheTokenization: () => {},
logInfo: () => {},
logWarn: (message) => warnings.push(message),
});
await controller.initSubtitlePrefetch('/tmp/broken.ass', 0);
assert.equal(warnings.length, 1);
assert.match(warnings[0]!, /\[subtitle-prefetch\].*0 cues.*\/tmp\/broken\.ass/);
});
+3 -2
View File
@@ -14,7 +14,6 @@ export interface SubtitlePrefetchInitControllerDeps {
tokenizeSubtitle: (text: string) => Promise<SubtitleData | null>;
preCacheTokenization: (text: string, data: SubtitleData) => void;
hasCachedTokenization?: (text: string) => boolean;
isCacheFull: () => boolean;
logInfo: (message: string) => void;
logWarn: (message: string) => void;
onParsedSubtitleCuesChanged?: (cues: SubtitleCue[] | null, sourceKey: string | null) => void;
@@ -59,6 +58,9 @@ export function createSubtitlePrefetchInitController(
const cues = deps.parseSubtitleCues(content, sourcePath);
if (revision !== initRevision || cues.length === 0) {
if (revision === initRevision) {
deps.logWarn(
`[subtitle-prefetch] parsed 0 cues from ${sourcePath}; prefetch disabled for this source`,
);
deps.onParsedSubtitleCuesChanged?.(null, null);
}
return;
@@ -69,7 +71,6 @@ export function createSubtitlePrefetchInitController(
tokenizeSubtitle: (text) => deps.tokenizeSubtitle(text),
preCacheTokenization: (text, data) => deps.preCacheTokenization(text, data),
hasCachedTokenization: (text) => deps.hasCachedTokenization?.(text) ?? false,
isCacheFull: () => deps.isCacheFull(),
});
if (revision !== initRevision) {
@@ -130,3 +130,121 @@ test('subtitle prefetch runtime does not extract internal subtitle tracks from r
assert.equal(resolved, null);
assert.equal(extracted, false);
});
test('subtitle prefetch refresh logs a warning when source resolution throws', async () => {
const warnings: string[] = [];
const refresh = createRefreshSubtitlePrefetchFromActiveTrackHandler({
getMpvClient: () => ({
connected: true,
requestProperty: async (name) => (name === 'path' ? '/media/video.mkv' : null),
}),
getLastObservedTimePos: () => 0,
subtitlePrefetchInitController: {
cancelPendingInit: () => {},
initSubtitlePrefetch: async () => {},
},
resolveActiveSubtitleSidebarSource: async () => {
throw new Error('ffmpeg ENOENT');
},
logWarn: (message) => warnings.push(message),
});
await refresh();
assert.equal(warnings.length, 1);
assert.match(warnings[0]!, /\[subtitle-prefetch\].*ffmpeg ENOENT/);
});
test('subtitle prefetch refresh logs debug when mpv client is not connected', async () => {
const debugs: string[] = [];
const refresh = createRefreshSubtitlePrefetchFromActiveTrackHandler({
getMpvClient: () => null,
getLastObservedTimePos: () => 0,
subtitlePrefetchInitController: {
cancelPendingInit: () => {},
initSubtitlePrefetch: async () => {},
},
resolveActiveSubtitleSidebarSource: async () => null,
logDebug: (message) => debugs.push(message),
});
await refresh();
assert.equal(debugs.length, 1);
assert.match(debugs[0]!, /\[subtitle-prefetch\].*not connected/);
});
test('subtitle prefetch refresh logs debug when no subtitle source resolves', async () => {
const debugs: string[] = [];
const cancels: number[] = [];
const refresh = createRefreshSubtitlePrefetchFromActiveTrackHandler({
getMpvClient: () => ({
connected: true,
requestProperty: async (name) => (name === 'path' ? '/media/video.mkv' : null),
}),
getLastObservedTimePos: () => 0,
subtitlePrefetchInitController: {
cancelPendingInit: () => {
cancels.push(1);
},
initSubtitlePrefetch: async () => {},
},
resolveActiveSubtitleSidebarSource: async () => null,
logDebug: (message) => debugs.push(message),
});
await refresh();
assert.deepEqual(cancels, [1]);
assert.equal(debugs.length, 1);
assert.match(debugs[0]!, /\[subtitle-prefetch\].*no active subtitle source/);
});
test('subtitle source resolver logs debug when internal track extraction is unavailable', async () => {
const debugs: string[] = [];
const resolveSource = createResolveActiveSubtitleSidebarSourceHandler({
getFfmpegPath: () => 'ffmpeg',
extractInternalSubtitleTrack: async () => null,
logDebug: (message) => debugs.push(message),
});
const resolved = await resolveSource({
currentExternalFilenameRaw: null,
currentTrackRaw: {
type: 'sub',
id: 3,
'ff-index': 7,
codec: 'hdmv_pgs_subtitle',
},
trackListRaw: [],
sidRaw: 3,
videoPath: '/media/video.mkv',
});
assert.equal(resolved, null);
assert.equal(debugs.length, 1);
assert.match(debugs[0]!, /\[subtitle-prefetch\].*extraction.*hdmv_pgs_subtitle/);
});
test('subtitle source resolver logs debug when no active subtitle track is selected', async () => {
const debugs: string[] = [];
const resolveSource = createResolveActiveSubtitleSidebarSourceHandler({
getFfmpegPath: () => 'ffmpeg',
extractInternalSubtitleTrack: async () => {
throw new Error('should not extract without a track');
},
logDebug: (message) => debugs.push(message),
});
const resolved = await resolveSource({
currentExternalFilenameRaw: null,
currentTrackRaw: null,
trackListRaw: [],
sidRaw: null,
videoPath: '/media/video.mkv',
});
assert.equal(resolved, null);
assert.equal(debugs.length, 1);
assert.match(debugs[0]!, /\[subtitle-prefetch\].*no active subtitle track/);
});
+22 -2
View File
@@ -86,6 +86,7 @@ export function createResolveActiveSubtitleSidebarSourceHandler(deps: {
videoPath: string,
track: MpvSubtitleTrackLike,
) => Promise<{ path: string; cleanup: () => Promise<void> } | null>;
logDebug?: (message: string) => void;
}) {
return async (input: {
currentExternalFilenameRaw: unknown;
@@ -104,6 +105,7 @@ export function createResolveActiveSubtitleSidebarSourceHandler(deps: {
const track = getActiveSubtitleTrack(input.currentTrackRaw, input.trackListRaw, input.sidRaw);
if (!track) {
deps.logDebug?.('[subtitle-prefetch] no active subtitle track selected yet');
return null;
}
@@ -114,6 +116,7 @@ export function createResolveActiveSubtitleSidebarSourceHandler(deps: {
}
if (isRemoteMediaPath(input.videoPath)) {
deps.logDebug?.('[subtitle-prefetch] skipping internal subtitle extraction for remote media');
return null;
}
@@ -123,6 +126,9 @@ export function createResolveActiveSubtitleSidebarSourceHandler(deps: {
track,
);
if (!extracted) {
deps.logDebug?.(
`[subtitle-prefetch] internal subtitle extraction unavailable (codec=${String(track.codec ?? 'unknown')}, ff-index=${String(track['ff-index'] ?? 'unknown')})`,
);
return null;
}
@@ -144,10 +150,13 @@ export function createRefreshSubtitlePrefetchFromActiveTrackHandler(deps: {
resolveActiveSubtitleSidebarSource: (
input: Parameters<ReturnType<typeof createResolveActiveSubtitleSidebarSourceHandler>>[0],
) => Promise<ActiveSubtitleSidebarSource | null>;
logDebug?: (message: string) => void;
logWarn?: (message: string) => void;
}) {
return async (): Promise<void> => {
const client = deps.getMpvClient();
if (!client?.connected) {
deps.logDebug?.('[subtitle-prefetch] skipped refresh: mpv client not connected');
return;
}
@@ -162,6 +171,7 @@ export function createRefreshSubtitlePrefetchFromActiveTrackHandler(deps: {
]);
const videoPath = typeof videoPathRaw === 'string' ? videoPathRaw : '';
if (!videoPath) {
deps.logDebug?.('[subtitle-prefetch] skipped refresh: no media path');
deps.subtitlePrefetchInitController.cancelPendingInit();
return;
}
@@ -175,8 +185,14 @@ export function createRefreshSubtitlePrefetchFromActiveTrackHandler(deps: {
});
if (!resolvedSource) {
if (deps.shouldKeepExistingCuesOnMissingSource?.(videoPath) === true) {
deps.logDebug?.(
'[subtitle-prefetch] no active subtitle source resolved; keeping existing cues',
);
return;
}
deps.logDebug?.(
'[subtitle-prefetch] no active subtitle source resolved; cancelling prefetch',
);
deps.subtitlePrefetchInitController.cancelPendingInit();
return;
}
@@ -190,8 +206,12 @@ export function createRefreshSubtitlePrefetchFromActiveTrackHandler(deps: {
} finally {
await resolvedSource.cleanup?.();
}
} catch {
// Skip refresh when the track query fails.
} catch (error) {
deps.logWarn?.(
`[subtitle-prefetch] failed to refresh from active track: ${
error instanceof Error ? error.message : String(error)
}`,
);
}
};
}
@@ -9,6 +9,9 @@ type TokenizerMainDeps = TokenizerDepsRuntimeOptions & {
getCurrentCharacterDictionaryMediaId?: NonNullable<
TokenizerDepsRuntimeOptions['getCurrentCharacterDictionaryMediaId']
>;
getCharacterNameCandidates?: NonNullable<
TokenizerDepsRuntimeOptions['getCharacterNameCandidates']
>;
getFrequencyDictionaryEnabled: NonNullable<
TokenizerDepsRuntimeOptions['getFrequencyDictionaryEnabled']
>;
@@ -84,6 +87,11 @@ export function createBuildTokenizerDepsMainHandler(deps: TokenizerMainDeps) {
getCurrentCharacterDictionaryMediaId: () => deps.getCurrentCharacterDictionaryMediaId!(),
}
: {}),
...(deps.getCharacterNameCandidates
? {
getCharacterNameCandidates: () => deps.getCharacterNameCandidates!(),
}
: {}),
getFrequencyDictionaryEnabled: () => deps.getFrequencyDictionaryEnabled(),
getFrequencyDictionaryMatchMode: () => deps.getFrequencyDictionaryMatchMode(),
getFrequencyRank: (text: string) => deps.getFrequencyRank(text),
+7 -3
View File
@@ -359,9 +359,13 @@
ffsubsync
</label>
</div>
<label id="subsyncSourceLabel" class="subsync-field">
<span>Source Subtitle (for alass)</span>
<select id="subsyncSourceSelect"></select>
<label id="subsyncReferenceLabel" class="subsync-field">
<span>Reference (correct timing, for alass)</span>
<select id="subsyncReferenceSelect"></select>
</label>
<label id="subsyncTargetLabel" class="subsync-field">
<span>Out-of-sync Subtitle (gets retimed)</span>
<select id="subsyncTargetSelect"></select>
</label>
</div>
<div id="subsyncStatus" class="runtime-options-status"></div>
+183 -32
View File
@@ -2,6 +2,7 @@ import test from 'node:test';
import assert from 'node:assert/strict';
import { createSubsyncModal } from './subsync.js';
import type { SubsyncManualPayload, SubsyncManualRunRequest } from '../../types';
type Listener = () => void;
@@ -52,18 +53,54 @@ function createDeferred<T>() {
return { promise, resolve };
}
function createTestHarness(runSubsyncManual: () => Promise<{ ok: boolean; message: string }>) {
function createSelectStub() {
const options: Array<{ value: string; textContent: string }> = [];
const events = createEventTarget();
let innerHTML = '';
let value = '';
return {
options,
disabled: false,
addEventListener: events.addEventListener,
dispatch: events.dispatch,
get innerHTML(): string {
return innerHTML;
},
set innerHTML(next: string) {
innerHTML = next;
if (next === '') {
options.length = 0;
value = '';
}
},
get value(): string {
return value;
},
set value(next: string) {
value = next;
},
appendChild(option: { value: string; textContent: string }) {
options.push(option);
if (!value) value = option.value;
return option;
},
};
}
function createTestHarness(
runSubsyncManual: (request: SubsyncManualRunRequest) => Promise<{ ok: boolean; message: string }>,
) {
const overlayClassList = createClassList();
const modalClassList = createClassList();
const statusClassList = createClassList();
const sourceLabelClassList = createClassList();
const referenceLabelClassList = createClassList();
const targetLabelClassList = createClassList();
const runButtonEvents = createEventTarget();
const closeButtonEvents = createEventTarget();
const engineAlassEvents = createEventTarget();
const engineFfsubsyncEvents = createEventTarget();
const sourceOptions: Array<{ value: string; textContent: string }> = [];
const runButton = {
disabled: false,
addEventListener: runButtonEvents.addEventListener,
@@ -77,6 +114,7 @@ function createTestHarness(runSubsyncManual: () => Promise<{ ok: boolean; messag
const subsyncEngineAlass = {
checked: false,
disabled: false,
addEventListener: engineAlassEvents.addEventListener,
dispatch: engineAlassEvents.dispatch,
};
@@ -88,18 +126,8 @@ function createTestHarness(runSubsyncManual: () => Promise<{ ok: boolean; messag
dispatch: engineFfsubsyncEvents.dispatch,
};
const sourceSelect = {
innerHTML: '',
value: '',
disabled: false,
appendChild: (option: { value: string; textContent: string }) => {
sourceOptions.push(option);
if (!sourceSelect.value) {
sourceSelect.value = option.value;
}
return option;
},
};
const referenceSelect = createSelectStub();
const targetSelect = createSelectStub();
let notifyClosedCalls = 0;
let notifyOpenedCalls = 0;
@@ -139,8 +167,10 @@ function createTestHarness(runSubsyncManual: () => Promise<{ ok: boolean; messag
subsyncCloseButton: closeButton,
subsyncEngineAlass,
subsyncEngineFfsubsync,
subsyncSourceLabel: { classList: sourceLabelClassList },
subsyncSourceSelect: sourceSelect,
subsyncReferenceLabel: { classList: referenceLabelClassList },
subsyncReferenceSelect: referenceSelect,
subsyncTargetLabel: { classList: targetLabelClassList },
subsyncTargetSelect: targetSelect,
subsyncRunButton: runButton,
subsyncStatus: {
textContent: '',
@@ -149,7 +179,7 @@ function createTestHarness(runSubsyncManual: () => Promise<{ ok: boolean; messag
},
state: {
subsyncModalOpen: false,
subsyncSourceTracks: [],
subsyncSubtitleTracks: [],
subsyncSubmitting: false,
isOverSubtitle: false,
},
@@ -166,6 +196,9 @@ function createTestHarness(runSubsyncManual: () => Promise<{ ok: boolean; messag
ctx,
modal,
runButton,
referenceSelect,
targetSelect,
referenceLabelClassList,
statusClassList,
getNotifyClosedCalls: () => notifyClosedCalls,
getNotifyOpenedCalls: () => notifyOpenedCalls,
@@ -187,16 +220,28 @@ async function flushMicrotasks(): Promise<void> {
await Promise.resolve();
}
const BASE_PAYLOAD: SubsyncManualPayload = {
subtitleTracks: [
{ id: 1, label: 'External #1 - jpn (active)' },
{ id: 2, label: 'External #2 - eng' },
],
defaultReferenceTrackId: 2,
defaultTargetTrackId: 1,
videoReferenceAvailable: true,
ffsubsyncAvailable: true,
};
function payloadWith(overrides: Partial<SubsyncManualPayload>): SubsyncManualPayload {
return { ...BASE_PAYLOAD, ...overrides };
}
test('manual subsync failure closes during run, then reopens modal with error', async () => {
const deferred = createDeferred<{ ok: boolean; message: string }>();
const harness = createTestHarness(async () => deferred.promise);
try {
harness.modal.wireDomEvents();
harness.modal.openSubsyncModal({
sourceTracks: [{ id: 2, label: 'External #2 - eng' }],
ffsubsyncAvailable: true,
});
harness.modal.openSubsyncModal(BASE_PAYLOAD);
harness.runButton.dispatch('click');
await Promise.resolve();
@@ -219,7 +264,8 @@ test('manual subsync failure closes during run, then reopens modal with error',
assert.equal(harness.statusClassList.contains('error'), true);
assert.equal(harness.ctx.dom.subsyncRunButton.disabled, false);
assert.equal(harness.ctx.dom.subsyncEngineAlass.checked, true);
assert.equal(harness.ctx.dom.subsyncSourceSelect.value, '2');
assert.equal(harness.referenceSelect.value, '2');
assert.equal(harness.targetSelect.value, '1');
assert.equal(harness.getNotifyClosedCalls(), 1);
assert.equal(harness.getNotifyOpenedCalls(), 1);
} finally {
@@ -231,15 +277,17 @@ test('subsync modal disables ffsubsync when payload marks it unavailable', () =>
const harness = createTestHarness(async () => ({ ok: true, message: 'ok' }));
try {
harness.modal.openSubsyncModal({
sourceTracks: [{ id: 2, label: 'External #2 - eng' }],
ffsubsyncAvailable: false,
});
harness.modal.openSubsyncModal(
payloadWith({ ffsubsyncAvailable: false, videoReferenceAvailable: false }),
);
assert.equal(harness.ctx.dom.subsyncEngineAlass.checked, true);
assert.equal(harness.ctx.dom.subsyncEngineFfsubsync.checked, false);
assert.equal(harness.ctx.dom.subsyncEngineFfsubsync.disabled, true);
assert.equal(harness.ctx.dom.subsyncStatus.textContent, 'Choose alass source, then run.');
assert.equal(
harness.ctx.dom.subsyncStatus.textContent,
'Choose the alass reference and out-of-sync subtitle, then run.',
);
} finally {
harness.restoreGlobals();
}
@@ -253,10 +301,14 @@ test('subsync modal ignores enter submission when no sync engine is available',
});
try {
harness.modal.openSubsyncModal({
sourceTracks: [],
harness.modal.openSubsyncModal(
payloadWith({
subtitleTracks: [{ id: 1, label: 'External #1 - jpn (active)' }],
defaultReferenceTrackId: null,
videoReferenceAvailable: false,
ffsubsyncAvailable: false,
});
}),
);
harness.modal.handleSubsyncKeydown({
key: 'Enter',
@@ -270,3 +322,102 @@ test('subsync modal ignores enter submission when no sync engine is available',
harness.restoreGlobals();
}
});
test('subsync modal defaults reference to the secondary track and target to the primary', async () => {
let request: SubsyncManualRunRequest | null = null;
const harness = createTestHarness(async (nextRequest) => {
request = nextRequest;
return { ok: true, message: 'ok' };
});
try {
harness.modal.wireDomEvents();
harness.modal.openSubsyncModal(BASE_PAYLOAD);
assert.equal(harness.referenceSelect.value, '2');
assert.equal(harness.targetSelect.value, '1');
harness.runButton.dispatch('click');
await flushMicrotasks();
assert.deepEqual(request, {
engine: 'alass',
targetTrackId: 1,
referenceMode: 'track',
referenceTrackId: 2,
});
} finally {
harness.restoreGlobals();
}
});
test('subsync modal offers the video file as an alass reference and excludes the target track', () => {
const harness = createTestHarness(async () => ({ ok: true, message: 'ok' }));
try {
harness.modal.wireDomEvents();
harness.modal.openSubsyncModal(BASE_PAYLOAD);
assert.deepEqual(
harness.referenceSelect.options.map((option) => option.value),
['2', 'video'],
);
harness.targetSelect.value = '2';
harness.targetSelect.dispatch('change');
assert.deepEqual(
harness.referenceSelect.options.map((option) => option.value),
['1', 'video'],
);
assert.equal(harness.referenceSelect.value, '1');
} finally {
harness.restoreGlobals();
}
});
test('subsync modal sends the video reference mode when the video file is selected', async () => {
let request: SubsyncManualRunRequest | null = null;
const harness = createTestHarness(async (nextRequest) => {
request = nextRequest;
return { ok: true, message: 'ok' };
});
try {
harness.modal.wireDomEvents();
harness.modal.openSubsyncModal(BASE_PAYLOAD);
harness.referenceSelect.value = 'video';
harness.runButton.dispatch('click');
await flushMicrotasks();
assert.deepEqual(request, {
engine: 'alass',
targetTrackId: 1,
referenceMode: 'video',
referenceTrackId: null,
});
} finally {
harness.restoreGlobals();
}
});
test('subsync modal hides the reference picker for ffsubsync but keeps the target picker', () => {
const harness = createTestHarness(async () => ({ ok: true, message: 'ok' }));
try {
harness.modal.wireDomEvents();
harness.modal.openSubsyncModal(BASE_PAYLOAD);
assert.equal(harness.referenceLabelClassList.contains('hidden'), false);
harness.ctx.dom.subsyncEngineAlass.checked = false;
harness.ctx.dom.subsyncEngineFfsubsync.checked = true;
harness.ctx.dom.subsyncEngineFfsubsync.dispatch('change');
assert.equal(harness.referenceLabelClassList.contains('hidden'), true);
assert.equal(harness.targetSelect.value, '1');
} finally {
harness.restoreGlobals();
}
});
+151 -67
View File
@@ -1,6 +1,14 @@
import type { SubsyncManualPayload } from '../../types';
import type { SubsyncManualPayload, SubsyncManualRunRequest } from '../../types';
import type { ModalStateReader, RendererContext } from '../context';
const VIDEO_REFERENCE_VALUE = 'video';
interface SubsyncSelection {
engine: 'alass' | 'ffsubsync';
referenceValue: string;
targetTrackId: number | null;
}
export function createSubsyncModal(
ctx: RendererContext,
options: {
@@ -8,27 +16,103 @@ export function createSubsyncModal(
syncSettingsModalSubtitleSuppression: () => void;
},
) {
let ffsubsyncAvailable = true;
let currentPayload: SubsyncManualPayload | null = null;
function setSubsyncStatus(message: string, isError = false): void {
ctx.dom.subsyncStatus.textContent = message;
ctx.dom.subsyncStatus.classList.toggle('error', isError);
}
function updateSubsyncSourceVisibility(): void {
const useAlass = ctx.dom.subsyncEngineAlass.checked;
ctx.dom.subsyncSourceLabel.classList.toggle('hidden', !useAlass);
function hasAlassReference(): boolean {
if (!currentPayload) return false;
return currentPayload.videoReferenceAvailable || currentPayload.subtitleTracks.length > 1;
}
function renderSubsyncSourceTracks(): void {
ctx.dom.subsyncSourceSelect.innerHTML = '';
for (const track of ctx.state.subsyncSourceTracks) {
const option = document.createElement('option');
option.value = String(track.id);
option.textContent = track.label;
ctx.dom.subsyncSourceSelect.appendChild(option);
function updateSubsyncFieldVisibility(): void {
const useAlass = ctx.dom.subsyncEngineAlass.checked;
ctx.dom.subsyncReferenceLabel.classList.toggle('hidden', !useAlass);
ctx.dom.subsyncTargetLabel.classList.toggle(
'hidden',
ctx.state.subsyncSubtitleTracks.length === 0,
);
}
ctx.dom.subsyncSourceSelect.disabled = ctx.state.subsyncSourceTracks.length === 0;
function appendOption(select: HTMLSelectElement, value: string, label: string): void {
const option = document.createElement('option');
option.value = value;
option.textContent = label;
select.appendChild(option);
}
function getSelectedTargetTrackId(): number | null {
const raw = Number.parseInt(ctx.dom.subsyncTargetSelect.value, 10);
return Number.isFinite(raw) ? raw : null;
}
function renderTargetTracks(preferredTrackId: number | null): void {
const select = ctx.dom.subsyncTargetSelect;
select.innerHTML = '';
select.value = '';
for (const track of ctx.state.subsyncSubtitleTracks) {
appendOption(select, String(track.id), track.label);
}
select.disabled = ctx.state.subsyncSubtitleTracks.length === 0;
const preferred = ctx.state.subsyncSubtitleTracks.find(
(track) => track.id === preferredTrackId,
);
const fallback = ctx.state.subsyncSubtitleTracks[0];
const selected = preferred ?? fallback;
if (selected) {
select.value = String(selected.id);
}
}
function renderReferenceTracks(preferredValue: string | null): void {
const select = ctx.dom.subsyncReferenceSelect;
const targetTrackId = getSelectedTargetTrackId();
const values: string[] = [];
select.innerHTML = '';
select.value = '';
for (const track of ctx.state.subsyncSubtitleTracks) {
if (track.id === targetTrackId) continue;
appendOption(select, String(track.id), track.label);
values.push(String(track.id));
}
if (currentPayload?.videoReferenceAvailable) {
appendOption(select, VIDEO_REFERENCE_VALUE, 'Video file (audio reference)');
values.push(VIDEO_REFERENCE_VALUE);
}
select.disabled = values.length === 0;
const preferred = preferredValue && values.includes(preferredValue) ? preferredValue : null;
const defaultTrackValue =
currentPayload?.defaultReferenceTrackId !== null &&
currentPayload?.defaultReferenceTrackId !== undefined
? String(currentPayload.defaultReferenceTrackId)
: null;
const fallback =
defaultTrackValue && values.includes(defaultTrackValue) ? defaultTrackValue : values[0];
const selected = preferred ?? fallback;
if (selected) {
select.value = selected;
}
}
function describeSubsyncState(): string {
if (!currentPayload) return '';
const alassReady = hasAlassReference();
if (alassReady && currentPayload.ffsubsyncAvailable) {
return 'Choose engine, reference and out-of-sync subtitle, then run.';
}
if (alassReady) {
return 'Choose the alass reference and out-of-sync subtitle, then run.';
}
if (currentPayload.ffsubsyncAvailable) {
return 'No reference available for alass. Use ffsubsync.';
}
return 'No sync engine available for current media.';
}
function closeSubsyncModal(): void {
@@ -46,30 +130,27 @@ export function createSubsyncModal(
}
}
function openSubsyncModal(payload: SubsyncManualPayload): void {
function openSubsyncModal(payload: SubsyncManualPayload, selection?: SubsyncSelection): void {
ctx.state.subsyncSubmitting = false;
ctx.state.subsyncSourceTracks = payload.sourceTracks;
ffsubsyncAvailable = payload.ffsubsyncAvailable;
ctx.state.subsyncSubtitleTracks = payload.subtitleTracks;
currentPayload = payload;
const hasSources = ctx.state.subsyncSourceTracks.length > 0;
ctx.dom.subsyncEngineAlass.checked = hasSources;
ctx.dom.subsyncEngineFfsubsync.checked = !hasSources && ffsubsyncAvailable;
ctx.dom.subsyncEngineFfsubsync.disabled = !ffsubsyncAvailable;
ctx.dom.subsyncRunButton.disabled = !hasSources && !ffsubsyncAvailable;
const alassReady = hasAlassReference();
const useAlass = selection ? selection.engine === 'alass' && alassReady : alassReady;
ctx.dom.subsyncEngineAlass.checked = useAlass;
ctx.dom.subsyncEngineFfsubsync.checked = !useAlass && payload.ffsubsyncAvailable;
ctx.dom.subsyncEngineAlass.disabled = !alassReady;
ctx.dom.subsyncEngineFfsubsync.disabled = !payload.ffsubsyncAvailable;
ctx.dom.subsyncRunButton.disabled = !alassReady && !payload.ffsubsyncAvailable;
renderSubsyncSourceTracks();
updateSubsyncSourceVisibility();
setSubsyncStatus(
!ffsubsyncAvailable && hasSources
? 'Choose alass source, then run.'
: !ffsubsyncAvailable
? 'No source subtitles available for alass.'
: hasSources
? 'Choose engine and source, then run.'
: 'No source subtitles available for alass. Use ffsubsync.',
false,
renderTargetTracks(
selection
? (selection.targetTrackId ?? payload.defaultTargetTrackId)
: payload.defaultTargetTrackId,
);
renderReferenceTracks(selection?.referenceValue ?? null);
updateSubsyncFieldVisibility();
setSubsyncStatus(describeSubsyncState(), false);
ctx.state.subsyncModalOpen = true;
options.syncSettingsModalSubtitleSuppression();
@@ -80,25 +161,11 @@ export function createSubsyncModal(
}
function reopenSubsyncModalWithError(
sourceTracks: SubsyncManualPayload['sourceTracks'],
engine: 'alass' | 'ffsubsync',
sourceTrackId: number | null,
payload: SubsyncManualPayload,
selection: SubsyncSelection,
message: string,
): void {
openSubsyncModal({ sourceTracks, ffsubsyncAvailable });
if (engine === 'alass' && sourceTracks.length > 0) {
ctx.dom.subsyncEngineAlass.checked = true;
ctx.dom.subsyncEngineFfsubsync.checked = false;
if (Number.isFinite(sourceTrackId)) {
ctx.dom.subsyncSourceSelect.value = String(sourceTrackId);
}
} else if (ffsubsyncAvailable) {
ctx.dom.subsyncEngineAlass.checked = false;
ctx.dom.subsyncEngineFfsubsync.checked = true;
}
updateSubsyncSourceVisibility();
openSubsyncModal(payload, selection);
setSubsyncStatus(message, true);
window.electronAPI.notifyOverlayModalOpened('subsync');
}
@@ -106,6 +173,7 @@ export function createSubsyncModal(
async function runSubsyncManualFromModal(): Promise<void> {
if (ctx.state.subsyncSubmitting) return;
if (ctx.dom.subsyncRunButton.disabled) return;
if (!currentPayload) return;
const useAlass = ctx.dom.subsyncEngineAlass.checked;
const useFfsubsync = ctx.dom.subsyncEngineFfsubsync.checked;
@@ -115,33 +183,46 @@ export function createSubsyncModal(
}
const engine = useAlass ? 'alass' : 'ffsubsync';
const sourceTrackId =
engine === 'alass' && ctx.dom.subsyncSourceSelect.value
? Number.parseInt(ctx.dom.subsyncSourceSelect.value, 10)
: null;
const referenceValue = ctx.dom.subsyncReferenceSelect.value;
const targetTrackId = getSelectedTargetTrackId();
if (engine === 'alass' && !Number.isFinite(sourceTrackId)) {
setSubsyncStatus('Select a source subtitle track for alass.', true);
if (targetTrackId === null) {
setSubsyncStatus('Select the out-of-sync subtitle track to retime.', true);
return;
}
if (engine === 'alass' && !referenceValue) {
setSubsyncStatus('Select a reference for alass.', true);
return;
}
const sourceTracksSnapshot = ctx.state.subsyncSourceTracks.map((track) => ({ ...track }));
const useVideoReference = referenceValue === VIDEO_REFERENCE_VALUE;
const request: SubsyncManualRunRequest = {
engine,
targetTrackId,
};
if (engine === 'alass') {
request.referenceMode = useVideoReference ? 'video' : 'track';
request.referenceTrackId = useVideoReference ? null : Number.parseInt(referenceValue, 10);
}
const payloadSnapshot: SubsyncManualPayload = {
...currentPayload,
subtitleTracks: currentPayload.subtitleTracks.map((track) => ({ ...track })),
};
const selection: SubsyncSelection = { engine, referenceValue, targetTrackId };
ctx.state.subsyncSubmitting = true;
ctx.dom.subsyncRunButton.disabled = true;
closeSubsyncModal();
try {
const result = await window.electronAPI.runSubsyncManual({
engine,
sourceTrackId,
});
const result = await window.electronAPI.runSubsyncManual(request);
if (result.ok) return;
reopenSubsyncModalWithError(sourceTracksSnapshot, engine, sourceTrackId, result.message);
reopenSubsyncModalWithError(payloadSnapshot, selection, result.message);
} catch (error) {
reopenSubsyncModalWithError(
sourceTracksSnapshot,
engine,
sourceTrackId,
payloadSnapshot,
selection,
`Subsync failed: ${(error as Error).message}`,
);
} finally {
@@ -171,10 +252,13 @@ export function createSubsyncModal(
closeSubsyncModal();
});
ctx.dom.subsyncEngineAlass.addEventListener('change', () => {
updateSubsyncSourceVisibility();
updateSubsyncFieldVisibility();
});
ctx.dom.subsyncEngineFfsubsync.addEventListener('change', () => {
updateSubsyncSourceVisibility();
updateSubsyncFieldVisibility();
});
ctx.dom.subsyncTargetSelect.addEventListener('change', () => {
renderReferenceTracks(ctx.dom.subsyncReferenceSelect.value || null);
});
ctx.dom.subsyncRunButton.addEventListener('click', () => {
void runSubsyncManualFromModal();
+3 -3
View File
@@ -18,7 +18,7 @@ import type {
SubtitlePosition,
SubtitleSidebarSnapshotConfig,
SubtitleCue,
SubsyncSourceTrack,
SubsyncSubtitleTrack,
YoutubePickerOpenPayload,
} from '../types';
@@ -85,7 +85,7 @@ export type RendererState = {
characterDictionaryStatus: string;
subsyncModalOpen: boolean;
subsyncSourceTracks: SubsyncSourceTrack[];
subsyncSubtitleTracks: SubsyncSubtitleTrack[];
subsyncSubmitting: boolean;
controllerSelectModalOpen: boolean;
@@ -213,7 +213,7 @@ export function createRendererState(): RendererState {
characterDictionaryStatus: '',
subsyncModalOpen: false,
subsyncSourceTracks: [],
subsyncSubtitleTracks: [],
subsyncSubmitting: false,
controllerSelectModalOpen: false,
+8 -4
View File
@@ -90,8 +90,10 @@ export type RendererDom = {
subsyncCloseButton: HTMLButtonElement;
subsyncEngineAlass: HTMLInputElement;
subsyncEngineFfsubsync: HTMLInputElement;
subsyncSourceLabel: HTMLLabelElement;
subsyncSourceSelect: HTMLSelectElement;
subsyncReferenceLabel: HTMLLabelElement;
subsyncReferenceSelect: HTMLSelectElement;
subsyncTargetLabel: HTMLLabelElement;
subsyncTargetSelect: HTMLSelectElement;
subsyncRunButton: HTMLButtonElement;
subsyncStatus: HTMLDivElement;
@@ -255,8 +257,10 @@ export function resolveRendererDom(): RendererDom {
subsyncCloseButton: getRequiredElement<HTMLButtonElement>('subsyncClose'),
subsyncEngineAlass: getRequiredElement<HTMLInputElement>('subsyncEngineAlass'),
subsyncEngineFfsubsync: getRequiredElement<HTMLInputElement>('subsyncEngineFfsubsync'),
subsyncSourceLabel: getRequiredElement<HTMLLabelElement>('subsyncSourceLabel'),
subsyncSourceSelect: getRequiredElement<HTMLSelectElement>('subsyncSourceSelect'),
subsyncReferenceLabel: getRequiredElement<HTMLLabelElement>('subsyncReferenceLabel'),
subsyncReferenceSelect: getRequiredElement<HTMLSelectElement>('subsyncReferenceSelect'),
subsyncTargetLabel: getRequiredElement<HTMLLabelElement>('subsyncTargetLabel'),
subsyncTargetSelect: getRequiredElement<HTMLSelectElement>('subsyncTargetSelect'),
subsyncRunButton: getRequiredElement<HTMLButtonElement>('subsyncRun'),
subsyncStatus: getRequiredElement<HTMLDivElement>('subsyncStatus'),
+16 -3
View File
@@ -295,14 +295,27 @@ export function parseControllerConfigUpdate(value: unknown): ControllerConfigUpd
export function parseSubsyncManualRunRequest(value: unknown): SubsyncManualRunRequest | null {
if (!isObject(value)) return null;
const { engine, sourceTrackId } = value;
const { engine, referenceMode, referenceTrackId, targetTrackId } = value;
if (engine !== 'alass' && engine !== 'ffsubsync') return null;
if (sourceTrackId !== undefined && sourceTrackId !== null && !isInteger(sourceTrackId)) {
if (referenceMode !== undefined && referenceMode !== 'track' && referenceMode !== 'video') {
return null;
}
const parseOptionalTrackId = (raw: unknown): number | null | undefined | false => {
if (raw === undefined) return undefined;
if (raw === null) return null;
return isInteger(raw) ? raw : false;
};
const parsedReferenceTrackId = parseOptionalTrackId(referenceTrackId);
const parsedTargetTrackId = parseOptionalTrackId(targetTrackId);
if (parsedReferenceTrackId === false || parsedTargetTrackId === false) return null;
return {
engine,
sourceTrackId: sourceTrackId === undefined ? undefined : (sourceTrackId as number | null),
referenceMode,
referenceTrackId: parsedReferenceTrackId,
targetTrackId: parsedTargetTrackId,
};
}
+2 -1
View File
@@ -33,7 +33,8 @@ export interface SubsyncContext {
videoPath: string;
primaryTrack: MpvTrack;
secondaryTrack: MpvTrack | null;
sourceTracks: MpvTrack[];
/** Every usable subtitle track, including the primary one. */
subtitleTracks: MpvTrack[];
audioStreamIndex: number | null;
}
+12 -3
View File
@@ -78,19 +78,28 @@ export interface MpvClient {
send(command: { command: unknown[]; request_id?: number }): boolean;
}
export interface SubsyncSourceTrack {
export interface SubsyncSubtitleTrack {
id: number;
label: string;
}
export interface SubsyncManualPayload {
sourceTracks: SubsyncSourceTrack[];
subtitleTracks: SubsyncSubtitleTrack[];
defaultReferenceTrackId: number | null;
defaultTargetTrackId: number | null;
videoReferenceAvailable: boolean;
ffsubsyncAvailable: boolean;
}
export type SubsyncReferenceMode = 'track' | 'video';
export interface SubsyncManualRunRequest {
engine: 'alass' | 'ffsubsync';
sourceTrackId?: number | null;
/** alass reference source: another subtitle track, or the loaded media file itself. */
referenceMode?: SubsyncReferenceMode;
referenceTrackId?: number | null;
/** Subtitle track to retime. Defaults to the active primary track. */
targetTrackId?: number | null;
}
export interface SubsyncResult {